Network Outage (In Progress)
  • Priority - Medium
  • Affecting Other - Cle Elum, WA
  • Updates
    12/28 1:50 PM - Atlas Networks technicians have detected and are investigating a network outage in Cle Elum, WA.  Details will be published as more information becomes available.

    12/28 2:13 PM - The issue appears to be a regional fiber outage upstream of Atlas, isolating our Cle Elum POP from the network.  The carrier that provides long-haul transport has been advised and is escalating.

    12/28 2:20 PM - Onsite technicians have verified that all equipment at our Seattle facility and Cle Elum POP is powered up and functioning properly, which confirms our earlier belief that the issue is a long-haul fiber cut.

    12/28 2:54 PM - The long haul carrier has confirmed the outage is on their end, and is dispatching technicians to investigate.  Atlas personnel are calling their NOC every 15 minutes to check for updates.

    12/28 3:30 PM - The long haul carrier has not isolated the fault yet, but confirms multiple fibers are down in Cle Elum.

    12/28 4:25 PM - The long haul carrier has isolated the Fiber fault to the general area. No ETA yet.

    12/28 5:09 PM -  Services have been restored by the carrier onto a secondary fiber path with less available capacity.  The primary path is still under repair, and traffic will be shifted back to this path once repairs are completed.

    12/30 10:13 AM - Services remain on the alternate path.  Repairs to the primary path are in progress.

    01/02 9:00 PM - Services remain on the alternate path.  Full capacity is available, and there is no service impact at this time - however this notice will remain open until a permanent repair is made.

    01/08 9:26 AM - Atlas has been notified by the cable operator that the original route's repairs are underway, and that a switchover event is scheduled for 1/28/2020 between the hours of 1:00AM and 6:00AM, with an anticipated impact duration of 15 minutes.

    Cause: Bridge construction involving a hole boring machine in Seattle resulted in the cut of a major telecommunications cable.  This cable carries a significant quantity of regional circuits to Seattle datacenter facilities.  The impact of this cable cut was felt widely across multiple states by many providers.
    Current Status: 
    Services are online via an alternate route.  Due to the underwater nature of the cut and the difficulty of accessing the specific site, local repairs have taken longer than typical, and are now underway.  A cutover to the original path is scheduled for 1/28/2020 between the hours of 1:00AM and 6:00AM, with an anticipated impact duration of 15 minutes.
    Mitigation: We are in contact with the carrier's executive team, and are working with them to develop a more diverse fiber path to Cle Elum.

  • Date - 2019/12/28 13:50
  • Last Updated - 2020/01/08 09:29