Reported Outages

Outage @ UC leases

Resolved Posted by Jacob Colleran on August 23, 2019
Outage start Friday, August 23, 2019 6:30 a.m.
Expected end Friday, August 23, 2019 9:15 a.m.

UC experienced an outage on the Lease and Reservation system starting at 6:30am CST, but usage was restored by 9:15am.

System Planned Maintenance on CHI@UC

Resolved Posted by John Roberts on August 19, 2019
Outage start Friday, September 06, 2019 4 p.m.
Expected end Tuesday, September 10, 2019 10 a.m.

There is an upcoming data center outage in the room hosting the CHI@UC resources set for Friday, September 6, 2019 at 4PM CST through Monday, September 9, inclusive. This work requires that the cooling for the data center will be offline during this time while the building management brings another cooling system online, thus CHI@UC will be unavailable for this entire time. We don't currently have an estimate on the exact time CHI@UC will be available again, but we will post an all clear message once we are back online.

Network down for CHI@TACC instances 2019-08-09

Resolved Posted by Francois Halbach on August 09, 2019
Outage start Friday, August 09, 2019 8:15 a.m.
Expected end Friday, August 09, 2019 9:08 a.m.

Networking at TACC has experienced an outage since 8:15am CST, but usage has been restored.  CHI@UC was not affected.

Networking for CHI@TACC is down

Resolved Posted by Alexander Barnes on August 07, 2019
Outage start Wednesday, August 07, 2019 11 a.m.
Expected end Wednesday, August 07, 2019 1 p.m.

Networking outage for CHI@TACC.

Network Outage CHI@TACC

Resolved Posted by Alexander Barnes on July 25, 2019
Outage start Thursday, July 25, 2019 11:45 a.m.
Expected end Thursday, July 25, 2019 4 p.m.

Networking at TACC outage

Network outage at UC starting 11:30am

Resolved Posted by Jason Anderson on June 14, 2019
Outage start Friday, June 14, 2019 11:30 a.m.
Expected end Saturday, June 15, 2019 4:30 p.m.

Update (06/15 4:30pm CT): The storage node has been successfully restored and things are now back online. There may be some flaky behavior over the next bit as the system converges back to a steady state. Thanks again for your patience; please reach out if you are still experiencing issues over the rest of the weekend.

Network maintenance at UC June 17 8am-11am CT

Resolved Posted by Jason Anderson on June 11, 2019
Outage start Friday, June 14, 2019 8 a.m.
Expected end Friday, June 14, 2019 11 a.m.

Update (06/17 2:00pm): we completed the maintenance successfully at 10:10am, but were affected by an unrelated power maintenance elsewhere in the data center. Those issues have also been resolved now and this maintenance can be considered complete. Hoping for smoother sailing through the rest of June.

Short planned network maintenance window at UC Friday morning

Resolved Posted by Jason Anderson on May 23, 2019
Outage start Friday, May 24, 2019 9 a.m.
Expected end Friday, May 24, 2019 9:15 a.m.

We are testing a failure case on some of our networking hardware on Friday morning at 9am Central Time. The Bare Metal site and Jupyter environments will be down for about 15 minutes as a result. We expect the disruption to be minimal.

Update: the maintenance completed successfully. Happy Friday.

Network outage at UC

Resolved Posted by Jason Anderson on May 16, 2019
Outage start Thursday, May 16, 2019 12:11 p.m.
Expected end Thursday, May 16, 2019 1 p.m.

We are experiencing a network outage at the UC site. This affects all experiments running on the bare metal site and also the Jupyter Notebook environment. We are working on identifying the root cause.

Update: this was resolved at 13:00 CT. Apologies for the disturbance!

Snapshot service disruption at baremetal sites

Resolved Posted by Jason Anderson on May 02, 2019
Outage start Wednesday, May 01, 2019 8 a.m.
Expected end Thursday, May 02, 2019 1 p.m.

We are experiencing an issue with snapshotting baremetal instances at both the TACC and UC site since yesterday morning. We are looking in to resolving the issue right now and hope to have updates soon.