Report (RFO) Date of Outage: 07/03/2018
Services Affected: Inbound for some clients.
Root Causes and Remediation: We will be assessing this issue to determine the best course of action to allow extra redundancy for these services.
[07-03-2018 11:00] All services are now operational at this data center.
[07-03-2018 10:20] Building power remains on backup systems. There is still no ETTR at this time. Additional updates will be provided as we receive more information.
[07-03-2018 10:00] Partial restoration of services at this data center.
[07-03-2018 9:20] The building remains on backup systems. There is still no ETTR at this time. Additional updates will be provided as we receive more information.
[07-03-2018 9:15] One of our nodes was having issues booting. This was resolved and all systems starting coming back online.
[07-03-2018 08:00] Both A-side and B-side power are running on backup systems at 400 76th St SW, Byron Center, MI as commercial power remains down. A-side was restored after minor repairs on the ATS switch after it failed to transfer to the backup systems. There is no ETTR at this time. We will provide updates as soon as they are available.
[07-03-2018 07:40] Power was restored to equipment but could not access remotely.
[07-03-2018 06:50] 123Net is part of a wide area commercial power outage. Power remains down and technicians are actively isolating further onsite. There is no ETTR at this time.
[07-03-2018 05:30] 123Net datacenter (BYTPMIBJ) at Byron Center 76th St. is currently on generator power. We have a technician on site investigating the cause. We will provide updates as soon as they are available.
Report By: Jeremy Berens - Hosted Infrastructure Coordinator Date: 07/03/2018