9th June 2021

Peering Peering Outage

4:01PM: An official email from Megaport has been received acknowledging the broadcast storm today. We have been advised that "An official report outlining the root cause, restorative actions and any mitigation actions will be provided within the next 3-5 business days."

3:25PM: Resolving incident as cause has been identified and Megaport IX has been placed back into production.

2:12 PM: This has been confirmed to be a broadcast storm of 600Gbit of traffic across the Megaport IX NSW peering links. The graph can be viewed at https://www.megaport.com/services/ix-statistics/

2:01 PM: We have identified the peering issue to be with Megaport IX in NSW. A broadcast storm occurred across the domestic peering. This link connects to Microsoft, Office 365, and Google among other cloud services. Other ISP's observed this issue and had to mitigate the service to stabilise routing.

1:49 PM: It appears other ISPs and networks observed an outage also related to domestic peering. No additional details but Infinite is not the only ISP affected.

1:26 PM: We have identified one of our peering links that connects to major cloud providers has gone offline and we are investigating the cause. In the interim we have turned off peering to this link and routed traffic to a secondary link.

1:25PM: Outage finished

1:05 PM: Outage begun