Reported Outages

10pm CT: This is now resolved.

Due to a disk failure, we need to restart the management node running CHI@UC.

We will be bringing down our authentication system for about an hour at 9 AM CT on December 5th for scheduled maintenance. If there are any conflicts with your deadlines, please contact us immediately.

UPDATE: Services are now working as expected. Please contact the help desk if you encounter any issues.

Requests to CHI@UC are not working. Running experiments should be unaffected.  

Due to an issue with SSL certificate generation, CHI@TACC was unavailable between 10:00am and 10:50am Central Time. Running instances were not affected.

Nodes at UC are currently failing to provision due to an issue communicating with the switches’ control plane, staff are investigating.

Resolved: The failed switch has been replaced, and nodes are once again available.

Due to hardware issues with a swtich, The Skylake compute nodes for CHI@TACC are currently unavailable. Staff are working to restore connectivity for these nodes.

The TACC site and our authentication system will be down for scheduled maintenance for an hour on September 26th. 

This has been resolved: Users receiving "No active allocations" in error can fix it by logging out and back in.

UPDATE: All services should now be working as expected

---

Due to a networking issue at TACC, Chameleon’s portal, CHI@TACC, KVM@TACC, and CHI@EDGE are currently unavailable. 
This affects site access as well as already running resources. Site networking staff are investigating, but there is no ETA
for resolution at this time.
 

The outage is resolved, and the object store is available again.

A rebalancing operation in the underlying storage cluster caused higher than normal I/O, and pushed some of the storage nodes into an unstable state. The node configuration has been tuned to prevent this instability from recurring.