Reported Outages

Update July 6th: Reservations for floating IPs at CHI@TACC should work as normal now. 

Update: July 5th

Reservations for Floating IPs at CHI@TACC are still failing, and we are working on a fix.

This only affects reservation for a floating IP. You can work-around this issue by not reserving IP addresses, and using ad-hoc IPs instead. You can find instructions for this method here: https://chameleoncloud.readthedocs.io/en/latest/getting-started/index.html#associating-an-ip-address

RESOLVED: Maintenance is complete.

The ticketing system for the Chameleon help desk will be down from 12:00 noon (CDT) on Friday July 1 2022 until 8:00 AM on Tuesday July 5th for maintenance. For assistance during this time, please send an email to users@lists.chameleoncloud.org.

We discovered an issue preventing DHCP from working when provisioning nodes, this has now been resolved.
If you saw error messages like:

Exceeded maximum number of retries. Failed to provision instance <uuid>: Timeout reached while waiting for callback for node <uuid>

or that the instance simply failed to start after a long while at TACC during this time, please try again, as this may have been the cause.

Between 10AM and 11 AM Central Time, there will be brief interruptions of the out-of-band network for certain racks at UC.

Users may notice failures to power instances on/off, or to deploy new instances. Running instances are unaffected.

Scheduled TLS Certificate Maintenance

Update: 06/15/2022:

Provisioning of new instances at UC is now functional with all the Chameleon supported images available. 

Unfortunately, we were not able to restore all of the images. We are still in the process of restoring some of them and will be contacting users whose images have not been possible to restore and working with them through available options. If you don’t find the image you are looking for, please reach out to help desk.

RESOLVED: The upgrade is complete! All services are operating normally.

On June 21st, Chameleon services at TACC will be unavailable to permit an OpenStack version upgrade. During this time, the CHI@TACC dashboard and API will be unavailable. Network access to running instances will also be interrupted, but instances will continue to run and will become available again after the upgrade. KVM@TACC will not be impacted.

UPDATE: The Network Maintenance scheduled for Sunday, June 26th, 2022 has been suspended. 
 

Network maintenance will be carried out between 10:00 AM and 4:00 PM (CST) on Sunday, June 26th. Access to all TACC systems will be unavailable during this time, including CHI@TACC, KVM@TACC, and the Chameleon Portal. Instances will continue to run, but users will have no access to TACC services and systems until the upgrade is complete.

Please submit any questions you may have via the TACC User Portal.

Update: 4:36 PM. NU has restored network connectivity, and the site is back up.

The network providing access to CHI@NU has gone down, preventing access to the site. The web UI and all instances are inaccessible.

Site staff are investigating, and we'll update here with a timeline for resolution.

Resolved: The system is now operating normally. Thank you for your patience.

CHI@TACC is currently experiencing an outage in provisioning baremetal nodes. This does not affect currently running instances, but prevents the launch of new ones. The team is working to resolve the issue.