Infinite Networks Status

Today, it was identified that the Data Centre Provider had moved the wrong cross connect yesterday at 1030hrs AEDT causing the original outage. While testing for light on our cross connect, third-party technicians from our Dark Fibre provider and Data Centre restored the connection, which wasn't part of our planned procedure. The Dark Fibre had been disabled as a precaution, but during testing the service was reconnected. This unplanned action triggered an outage and caused one of the switches in the core switch stack at our other Canberra Data Centre to partially crash.

It took some time to work out what the issue but once we had realised what had happened one of technicians quickly drove to the affected site and identified errors on the switch. The affected switch was rebooted, which restored some services, but parts of our core network between Canberra and Sydney remained unresponsive. We had to reroute traffic to regain connectivity, and the process took longer than expected due to limited access to core equipment.

We sincerely apologise for the disruption. We’re currently investigating why our network redundancy didn’t perform as planned and will be running after-hours tests over the coming weeks to prevent a recurrence. While some of the network worked as expected, others parts didn’t respond as documented, which contributed to the downtime and delayed our field updates.

To address this, we’re scheduling an emergency upgrade and full reboot of the entire switch stack at 05:00 AEDT on 6th March 2025. We’ll keep you posted on this status page with official updates. This outage will remain on watching status until the upgrade and reboot of the switch stack is successful.

We currently have techs testing the cross connect at one of the Data Centre's to ensure light is passing to assist with diagnosis for Dark Fibre Providers. We are awaiting the Dark Fibre Provider response from testing yesterday.

If you have any services down please email the helpdesk to let us know, call volumes are high at the moment due to the outage, if you must call please leave a message and we will call you back. Most services should be online and we can apply work arounds or check if the service has an issue.

After investigating issue the Dark Fibre Provider we have not been able to resolve the issue. We have implemented a work around to restore connectivity for affected services. We will continue to work with the Dark Fibre Provider to regain connectivity. Any connectivity restoration will be done in a controlled manner. We will continue to provide updates as they come to hand.

We are still working with the Dark Fibre Provider, they are at the data centre and can not find the issue. We are meeting them onsite to resolve, we are begining Disaster Recovery procedures for shared hosting servers in the mean time to get our clients services back online.

The Dark Fibre Provider has called back asking us to test as it should now be restored but we are still not seeing any light on the circuit. They are investigating their end, we are monitoring the light levels at both ends and we are awaiting a call from the tech for further testing.

We have heard from the Dark Fibre Provider, they have found damage fibre cores in their rack at TransACT House. They have another tech enroute to the data centre to change the cores over with an ETA of 1 hour. We do not have a ETR as yet but we would hope this would be resolved in the next 2 hours.

Dark Fibre Provider's tech are now onsite at Data Centre conducting tests to determine what the issue is.

All phones systems are now back up and operational, please contact us if your phones are still down. We will continue to restore services by moving workloads to appropriate data centres. No response from Dark Fibre Provider or ETR.

We have had an update from the Dark Fibre provider and say that they have observed or reported any issues but they are organising a tech to inspect the dark fibre.

Our phones are currently down due to a failure in automatic failover, we are restoring this as a priority. Helpdesk and Email functions are working as expected.

We have heard from the Dark Fibre provider and they are investigating the issue.

Most services have automatically failed over but other services require manual intervention and we are begining to restore these.

We have observed an outage on our dark fibre between NextDC C1 and TransACT House data centres, we can see reduced light levels on our interfaces. One core is reporting -40db for total loss and the other is -27db for a partial loss indicating a fibre cut.