Reported Outages

Auth servers down

Resolved Posted by Michael Sherman on April 30, 2025
Outage start Wednesday, April 30, 2025 5:18 p.m.
Expected end Wednesday, April 30, 2025 5:30 p.m.

Update: 5:30pm: services appear to be reachable again, we saw a downtime of ~ 10 minutes. Our aplogies for the interruption.

We're waiting for more information about whether to excpect any more interruptions.

KVM@TACC CLI / Jupyter Authentication Issues

Resolved Posted by Michael Sherman on April 24, 2025
Outage start Thursday, April 24, 2025 12 p.m.
Expected end Friday, April 25, 2025 9:27 a.m.

9:00 AM 04/25/25: A fix has been deployed and we belive the issue to be resolved,  but please reach out if you observe further issues.


Currently, authenticating from jupyter to KVM@TACC, or using "CLI" auth with federated ID credentials to KVM@TACC is failing.

KVM@TACC Maintnenance April 22, 2025

Resolved Posted by Cody Hammock on April 21, 2025
Outage start Tuesday, April 22, 2025 9 a.m.
Expected end Tuesday, April 29, 2025 5:19 p.m.

Update: All work associated with this outage is complete.

CHI@Edge scheduled maintenance 2025/04/29

Resolved Posted by Michael Sherman on April 01, 2025
Outage start Tuesday, April 29, 2025 9 a.m.
Expected end Tuesday, April 29, 2025 9 a.m.

On April 29th, CHI@Edge will be inaccessible as we migrate the control-plane to more robust hosting and upgrade several dependent services. During the maintenance window, the dashboard, running containers, and device enrollement will be unavailable.

KVM Disruptive Maintenance April 10, 2025

Resolved Posted by Cody Hammock on March 27, 2025
Outage start Thursday, April 10, 2025 9 a.m.
Expected end Thursday, April 10, 2025 4:23 p.m.

Resolved

To prepare for upgrades to KVM@TACC, we are scheduling maintenance starting at 9:00 AM CDT on Thursday April 10th.
During this time, Staff will Stop All Running Instances, perform upgrades, then restart those instances.
This work will ensure that VMs are compatible with following upgrades, preventing further downtime.

CHI@Edge lease issues

Resolved Posted by Michael Sherman on March 17, 2025
Outage start Monday, March 17, 2025 11:35 a.m.
Expected end Monday, March 17, 2025 6 p.m.

We're observing issues with leases for some devices in CHI@edge, and have identified some configuration inconistency on the backend that may be contributing. While debugging, there was a brief outage in the reservation api service, which manifested as a failure to load the "leases" dialog.

Thing seem to be back, but we're still investigating.

upstream networking issue for CHI@UC

Resolved Posted by Michael Sherman on March 04, 2025
Outage start Tuesday, March 04, 2025 7:05 p.m.
Expected end Wednesday, March 05, 2025 12 p.m.

11AM March 5th:

This outage is now resolved, CHI@UC appears to be working as normal.

We don't yet have a root cause from our provider, but are working with them to prevent a repeat.

Jupyter Planned Outage

Resolved Posted by Mark Powers on February 19, 2025
Outage start Monday, March 03, 2025 9 a.m.
Expected end Tuesday, March 04, 2025 4:48 p.m.

Resolved: Jupyter service should be working as normal.


Update: We are waiting on DNS updates to finish the updating of our Jupyter infrastructure, and so the outage is ongoing. Thank you for your patience.


We will be updating our Jupyter infrastructure on the morning of Monday, March 3, which will take down JupyterHub and running JupyterLab instances.

Jupyterhub severs launch errors

Resolved Posted by Mark Powers on February 11, 2025
Outage start Tuesday, February 11, 2025 3 p.m.
Expected end Tuesday, February 11, 2025 4:30 p.m.

UPDATE: Jupyterhub service is back to normal

---

Users are experiencing errors when launching Jupyter environments on Chameleon's Jupyterhub. We are working on a resolution.

TACC Network Maintenance Wednesday 19 February 2025

Resolved Posted by Cody Hammock on February 10, 2025
Outage start Wednesday, February 19, 2025 6 a.m.
Expected end Wednesday, February 19, 2025 6:47 a.m.

Resolved: Work has been completed.

TACC network infrastructure will not be available from 6:00AM to 7:00AM (CDT) on Wednesday, February 19 2025. Network maintenance will be performed during this time. This will impact access to the Chameleon Portal, JupyterHub, CHI@TACC, KVM@TACC, and CHI@Edge. Experiments and VMs will continue to run, but will not have connectivity outside of TACC.