Network outage at UC starting 11:30am

Resolved Posted by Jason Anderson on June 14, 2019
Outage start Friday, June 14, 2019 11:30 a.m.
Expected end Saturday, June 15, 2019 4:30 p.m.

Update (06/15 4:30pm CT): The storage node has been successfully restored and things are now back online. There may be some flaky behavior over the next bit as the system converges back to a steady state. Thanks again for your patience; please reach out if you are still experiencing issues over the rest of the weekend.

Update (5:10pm CT): We've got most stuff back up at the UC site, but have to restore one of our storage nodes that didn't come back up cleanly. This will unfortunately take some time to move all the relevant bits around. The site will remain offline until this process completes, which we expect to take until tomorrow given the amount of data on this node. Apologies again for the inconvenience.

Update (2:00pm CT): A brief power outage at our datacenter affected several of our network switches. We are still working to restore the system to a working configuration.

We are currently experiencing a network interruption at the UC site at Argonne. As a result, CHI@UC and the Jupyter Notebook environment are currently down. We are working on diagnosing the problem and will update with a fix estimate shortly.