Reported Outages

CHI@IIT down

Resolved Posted by Michael Sherman on August 15, 2023
Outage start Friday, July 14, 2023 6:13 p.m.
Expected end Wednesday, August 16, 2023 6:13 p.m.

CHI@IIT is currently offline.

We suspect a hardware failure in the controller node, and have escalated to site staff.

CHI@TACC Object Store Unavailable

Resolved Posted by Cody Hammock on August 15, 2023
Outage start Tuesday, August 15, 2023 9:35 a.m.
Expected end Tuesday, August 15, 2023 11:43 a.m.

Resolved: The Object Store is once again available.

The Object Store for CHI@TACC is currently unavailable. Staff is working to restore access.

TACC Network maintenance 20 August 2023

Resolved Posted by Cody Hammock on August 04, 2023
Outage start Sunday, August 20, 2023 8 a.m.
Expected end Sunday, August 20, 2023 11:45 a.m.

Resolved: TACC network maintnance is concluded as of 11:45 AM (CDT)

TACC network infrastructure will not be available from 8 AM to 2:00 PM (CDT) on Sunday, 20 August 2023. Network maintenance will be performed during this time.

CHI@TACC, KVM@TACC, and the Chameleon Portal will be affected. Baremetal instances and VMs will continue to run, but will be unreachable.

CHI@UC Datacenter outage July 28th-31st

Resolved Posted by Michael Sherman on July 17, 2023
Outage start Friday, July 28, 2023 12 p.m.
Expected end Monday, July 31, 2023 4:54 p.m.

CHI@UC is back online, and we're observing normal usage patterns so far.

Please reach out if you're encountering issues.

CHI@TACC Liqid nodes

Resolved Posted by Cody Hammock on July 05, 2023
Outage start Wednesday, July 05, 2023 11:13 a.m.
Expected end Wednesday, October 18, 2023 11:01 a.m.

RESOLVED: The faulty hardware has been identified and replaced. The Liqid subsystem is once again available to use.

The Liqid composeable hardware subsystem for CHI@TACC is experiencing intermittent issues with connecting the PCI devices with the connected hosts.

CHI@UC Network maintenance July 5th

Resolved Posted by Michael Sherman on June 22, 2023
Outage start Wednesday, July 05, 2023 9 a.m.
Expected end Wednesday, July 05, 2023 11:30 a.m.

The maintenance is proceeding well, but will likely take an extra 30 minutes or so, finishing at 11:30 rather than 11AM.

Afterwards, all instances will be acceissable as normal.

CHI@UC Issues mounting shared filesystem

Resolved Posted by Michael Sherman on June 12, 2023
Outage start Monday, June 12, 2023 6 a.m.
Expected end Tuesday, June 13, 2023 12 a.m.

We're currently observing an issue where some projects fail to mount the filesystem at CHI@UC, receiving an error message about an "RBAC Quota". We are currently investigating.

This is purely an access issue with new "storage leases", existing storage leases/mounts should be unaffected.

CHI@IIT down

Resolved Posted by Michael Sherman on June 12, 2023
Outage start Saturday, June 03, 2023 6 p.m.
Expected end Tuesday, June 13, 2023 6 p.m.

CHI@IIT is currently down, we've had a persistent hardware issue with the controller node at this side.

Site staff are currently investigating.

CHI@NU down

Resolved Posted by Michael Sherman on May 30, 2023
Outage start Tuesday, May 30, 2023 2 a.m.
Expected end Wednesday, June 07, 2023 11:36 a.m.

CHI@NU was down due to a hardware failure in the controller node. This has now been replaced, and the site is back online.

Datacenter Maintenance affecting CHI@UC May 7-12!

Resolved Posted by Michael Sherman on April 19, 2023
Outage start Sunday, May 07, 2023 5 p.m.
Expected end Friday, May 12, 2023 5:55 p.m.

05/12/23: CHI@UC is back online. All systems look stable, but we'll be keeping an eye on it over the weekend to be sure. 

Pleas let us know if you encounter issues!