Provisioning network failure at CHI@UC

Resolved Posted by Michael Sherman on May 02, 2022
Outage start Friday, April 29, 2022 11:20 a.m.
Expected end Monday, May 02, 2022 1:17 p.m.

Update 05/03/22: This issue is now resolved. It was caused by a combination of two factors: misconfiguration of the DHCP behavior for out-of-band interfaces, and a failure causing an out of band switch to power off.

All affected nodes should be reservable again. If you have an instance that has become inaccessable, please get in touch with us via the helpdesk.

Update 05/02/22:  Connectivity has been restored for nodes nc01-nc35. Nodes nc36-nc64 are still having issues.

05/02/22:
An issue affecting the out-of-band switches at CHI@UC is currently preventing provisioning of new instances on P2 nodes at UC. This is includes compute_skylake and gpu_rtx6000 nodes, but all nodes prefixed with "P3-" are unaffected. Existing instances cannot have their power states changed, or be rebuilt, but are otherwise unaffected. We will update here with more information.