CHI@NCAR Unplanned Outage

Resolved Posted by Bodhi Rubinstein on October 16, 2026
Outage start Friday, October 16, 2026 2:35 p.m.
Expected end Monday, October 19, 2026 8:20 p.m.

RESOLVED: All compute_zen5 and tharm nodes at CHI@NCAR are back in production. We are waiting for a replacement SSD for our switch, but have moved the primary links for the affected nodes to a different switch. The secondary links will be unavailable until we receive the replacement hardware. All existing instances on the affected nodes have been migrated and should be accessible. The impacted leases have been extended to account for the unplanned downtime. Please let us know if you experience any issues or have any questions at help@chameleoncloud.org. Thank you for your patience!


Update: We have identified the issue as a failed SSD drive on our network switch. We are reaching out to our vendor for support and will have an update on Monday, Oct 19th. The compute_zen5_grado and tharm nodes are unavailable, but we have frozen the instances to prevent any deletion and are working to extend the current leases. Thank you for your patience!

compute_zen5 and tharm nodes at CHI@NCAR are currently impacted by a network switch failure.

Users are unable to launch instances and may not be able to access existing ones.

We are working to restore service, and will provide updates as the situation evolves.