storage system interruption at UC

Resolved Posted by Michael Sherman on June 13, 2022
Outage start Monday, June 13, 2022 3:08 p.m.
Expected end Wednesday, June 15, 2022 2:13 p.m.

Update: 06/15/2022:

Provisioning of new instances at UC is now functional with all the Chameleon supported images available. 

Unfortunately, we were not able to restore all of the images. We are still in the process of restoring some of them and will be contacting users whose images have not been possible to restore and working with them through available options. If you don’t find the image you are looking for, please reach out to help desk.


We've discovered data-loss affecting the glance images stored at CHI@UC. We've frozen writes to the storage cluster to improve the odds of data recovery, and are investigating options.

Once we have identified a safe route to resume operations, we will restore the Chameleon maintained images to allow normal testbed usage. In the meantime, provisioning of new nodes will fail with the message "Failed to download image, 0 bytes found".