Reported Outages

CHI@UC Uplink networking

Resolved Posted by Michael Sherman on May 29, 2024
Outage start Wednesday, May 29, 2024 10:28 a.m.
Expected end Wednesday, May 29, 2024 11:28 a.m.

CHI@UC had a brief interruption in connectivity, preventing access to chi.uc.chameleoncloud.org. This issue manifested as a failure for the UC control-plane servers to contact the chameleon authentication server, triggered by what was planned to be unrelated work elsewhere in the network.

Upon discovering the issue, the other network changes were backed out, and service has been restored.

CHI@TACC Outage May 25-27, 2024

Resolved Posted by Cody Hammock on May 28, 2024
Outage start Saturday, May 25, 2024 8:30 a.m.
Expected end Tuesday, May 28, 2024 10 a.m.

CHI@TACC was unavailable from May 25 through the morning of May 28, 2024. This effected the API services and web interface, but did not impact running instances.

We have corrected the problem, and service is restored.

Upcoming Maintenance window for Chameleon Auth server

Resolved Posted by Mark Powers on May 28, 2024
Outage start Tuesday, June 04, 2024 9:30 a.m.
Expected end Tuesday, June 04, 2024 10:10 a.m.

UPDATE: All services are updated and should be working as expected


On the morning of Tuesday, June 4th, there will be a brief outage affecting login to all Chameleon sites and services at 9:30 AM central time.

We expect a 5-10 minute outage while we apply updates to the service that handles federated login.
This won't affect any running workloads or nodes, but you may need to refresh your browser once the outage ends.

CHI@TACC Certificate Expiry

Resolved Posted by Cody Hammock on May 21, 2024
Outage start Monday, May 20, 2024 7:45 p.m.
Expected end Tuesday, May 21, 2024 9:40 a.m.

CHI@TACC was unavailble between 7:15PM CDT Monday May 20, 2024 and 9:40AM CDT Tuesday May 21, 2024 due to an automation failure issuing an updated SSL certificate. The issue has been corrected and we do not expect further disruption.

CHI@Edge outage

Resolved Posted by Mark Powers on May 16, 2024
Outage start Thursday, May 16, 2024 5 p.m.
Expected end Friday, May 17, 2024 5 p.m.

The outage on CHI@Edge is now resolved, thank you for your patience.


CHI@Edge is experiencing issues launching and interactiving with containers. Operators are looking into this issue.

 

Planned Help Desk Outage

Resolved Posted by Mark Powers on April 24, 2024
Outage start Monday, April 29, 2024 9 a.m.
Expected end Monday, April 29, 2024 1:14 p.m.

UPDATE: The ticketing system is now back online, and operating as normal.


The ticketing system for the Chameleon help desk will be down from 9:00 AM (CDT) on Monday, April 29 until 5:00 PM for maintenance. For assistance during this time, please send an email to users@lists.chameleoncloud.org.

April 4 - Jupyterhub Outage

Resolved Posted by Mark Powers on April 04, 2024
Outage start Thursday, April 04, 2024 10 a.m.
Expected end Thursday, April 04, 2024 12:14 p.m.

The networking issue has now been resolved


Chameleon's JupyterHub is current experiencing an unexpected outage. Operators are investigating.

April 16th: Brief maintenance affecting CHI@UC network uplink

Resolved Posted by Michael Sherman on April 03, 2024
Outage start Tuesday, April 16, 2024 9:30 a.m.
Expected end Tuesday, April 16, 2024 10 a.m.

The morning of April 16th, from 9:30-10:00am, we are scheduling a maintenance window which will affect the CHI@UC network uplink.
This is in order to debug some issues affecting the routers "upstream" from CHI@UC.

For the duration, you may encounter failures to access the CHI@UC website or API, or access to public IPs.

Partial Authentication Outage

Resolved Posted by Mark Powers on March 06, 2024
Outage start Wednesday, March 06, 2024 1:55 p.m.
Expected end Thursday, March 07, 2024 1:36 p.m.

UPDATE: The instability should be fixed, and logging into services should work as normal.

----

Some users are reporting issues authenticating to CHI@UC, CHI@TACC, and CHI@Edge dashboards. We are working on resolving this issue.

KVM@TACC System maintenance

Resolved Posted by Cody Hammock on March 06, 2024
Outage start Wednesday, March 27, 2024 8 a.m.
Expected end Wednesday, March 27, 2024 10:33 a.m.

Resolved: Work is complete and KVM@TACC is available.

On Wednesday, March 27 2024 KVM@TACC will be unavailable between 8:00am CDT and 4:00pm CDT to perform necessary system maintenance. Access to the KVM@TACC web interface and API will be unavaiable, and network access to instances will be interrupted. Instances will continue to run, even though they will not be reachable.