Reported Outages

CHI@NU currently down

Resolved Posted by Michael Sherman on February 17, 2022
Outage start Thursday, February 17, 2022 4:50 p.m.
Expected end Friday, February 18, 2022 5:50 p.m.

This outage has been resolved, and CHI@NU is fully operational.


The CHI@NU site is currently inaccessible due to unexpected issues during a service upgrade. Any running nodes should be unaffected, but are currently inaccessible, along with the Horizon WebUI and API. Other sites are unaffected.

If this outage interrupts your work, feel free to use resources at another site, and please let us know via the helpdesk if you have a use-case that requires the CHI@NU site.

CHI@TACC, KVM@TACC, and CHI@EDGE networking outage

Resolved Posted by Francois Halbach on February 11, 2022
Outage start Thursday, February 10, 2022 4:14 p.m.
Expected end Thursday, February 10, 2022 5:38 p.m.

Outage Start: 2022-02-10 16:14

Outage End: 2022-02-10 17:38

Update: this outage has been resolved.


Due to a networking issue at TACC, CHI@TACC, KVM@TACC, and CHI@EDGE are currently unavailable. 

This affects site access as well as already running resources.

Site networking staff are investigating, but there is no ETA for resolution at this time.

CHI@UC Networking outage

Resolved Posted by Michael Sherman on February 03, 2022
Outage start Thursday, February 03, 2022 11:27 a.m.
Expected end Friday, February 04, 2022 3:10 p.m.

Update - 3:09 PM CST - The outage should now be resolved.


Due to an upstream hardware failure, L2 stitching connectivity, and the creation of new instances is failing at CHI@UC.

Existing instances not using stitching or the SharedWAN network are not affected.

Site networking staff are investingating, but there is no ETA for resoluton at this time.

CHI@UC network maintenance

Resolved Posted by Michael Sherman on January 28, 2022
Outage start Tuesday, February 01, 2022 9 a.m.
Expected end Tuesday, February 01, 2022 2 p.m.

Update: This maintenance has been reschedule for Feb 25th. Phase2 nodes (all nodes starting with names nc...) will be mostly unaffected.


On Tuesday, Feb 1st, there will be a short interruption the management network for CHI@UC due to network upgrades.

During this time, it will not be possible to create new instances, or change the power state of existing ones. Running instances will not be interrupted.

Chameleon at TACC experiencing networking disruptions (January 10th 2022)

Resolved Posted by Francois Halbach on January 11, 2022
Outage start Monday, January 10, 2022 2:14 p.m.
Expected end Monday, January 10, 2022 6:32 p.m.
Resolved: The networking outage at TACC has been resolved. Access to Chameleon resources at TACC is now restored.
 
 
Dear Chameleon users,
 
The networking outage at TACC has been resolved. Access to Chameleon resources at TACC is now restored.
 
Kind regards,
 
François Halbach

TACC Network Maintenance 28 November 2021

Resolved Posted by Cody Hammock on October 25, 2021
Outage start Sunday, November 28, 2021 10 a.m.
Expected end Sunday, November 28, 2021 4 p.m.

Resolved: The network maintenance at TACC was completed at 1:19 PM (CST) on Sunday, November 28

Network maintenance will be carried out between 10:00 AM and 4:00 PM (CST) on Sunday, November 28. Access to all TACC systems (CHI@TACC, KVM@TACC, and Chameleon Portal) will be unavailable during this time. Instances will continue to run, but users will have no access to TACC-based services and systems until the upgrade is complete.

Power Maintenance at CHI@UC November 19th - 22nd

Resolved Posted by Michael Sherman on October 25, 2021
Outage start Friday, November 19, 2021 6 p.m.
Expected end Monday, November 22, 2021 2:10 p.m.

Update: The outage is now over. Jupyterhub, Trovi, and CHI@UC are now available.

Important Note: Instances created prior to the outage are available, but must be powered back on manually. You can do so via the web ui, or other CLI / API as you prefer. They were shut down prior to the outage to avoid any data-loss.


There will be a total outage for the CHI@UC site 6pm, Friday November 19th - 12pm, Monday November 22nd, due to maintenance on the power system in the data center.

Oct 24 login system outage

Resolved Posted by Jason Anderson on October 24, 2021
Outage start Sunday, October 24, 2021 7 a.m.
Expected end Sunday, October 24, 2021 2:05 p.m.

Due to a certificate expiration, the authentication system for the Chameleon sites is effectively online, preventing web and CLI-based logins. This affects all Chameleon sites, the user portal, and the Jupyter environment.

We are working with the team providing the certificate service and will update when it is back online.

Jupyter.chameleoncloud.org not accessible

Resolved Posted by Michael Sherman on October 20, 2021
Outage start Wednesday, October 20, 2021 3 p.m.
Expected end Wednesday, October 20, 2021 3:42 p.m.

Update 3:35pm: Access has been restored. Please let us know if you continue to see issues.


The chameleon Jupyterhub is currently not accessible, as of ~2:40 PM today.

We believe the cause to be related to a network configuration change, and are investigating.

We'll update here with an ETA or resolution.

 

Ticketing system e-mail outage

Resolved Posted by Francois Halbach on October 15, 2021
Outage start Wednesday, October 13, 2021 12 a.m.
Expected end Friday, October 15, 2021 6 p.m.

The helpdesk email help@chameleoncloud.org is currently non-operational. New tickets and ticket replies submitted by e-mail are not being delivered.

New tickets and replies can still be sent via the 'My Tickets' portal page:
https://chameleoncloud.org/user/help/

It is unclear when the ticketing system will be restored, we apologize for the inconvenience.