Time: Sept. 6, 2018, 10:40 p.m.

Status: Resolved

Update: Although Microsoft Azure current status (as of Thursday, Sept. 6 @ 3:30pm pacific time) shows two services still impacted from Tuesday outage, we believe all of our users should be fully back online with their GoFormz services. If you are still experiencing any errors or issues please reach out to us at [email protected] and we will do everything we can to help.

Time: Sept. 5, 2018, 3:46 p.m.

Status: Monitoring

Update: While some GoFormz customers have reported trouble-free services, others will continue to be impacted as Azure services continue to come back online. -- Here is the latest from the Microsoft Azure engineering team -- Multiple Services - Applying Extended Mitigation UPDATE AS OF 15:00 UTC: Engineers have restored storage availability for the majority of impacted services, and customers should be continuing to see improvements to service availability. CUSTOMER IMPACT: Starting at 09:29 UTC on 04 Sep 2018, customers with resources in South Central US may experience difficulties connecting to resources hosted in this region. A complete list of impacted services can be found below. PRELIMINARY ROOT CAUSE: A severe weather event, including lightning strikes, occurred near one of the South Central US datacenters. This resulted in a power voltage increase that impacted cooling systems. Automated datacenter procedures to ensure data and hardware integrity went into effect and critical hardware entered a structured power down process. ENGINEERING STATUS: Engineers have restored access to storage resources for the majority of services, and most customers should be seeing signs of recovery. Engineers are continuing to work on any residual storage impact to fully mitigate this issue. We are also working with the impacted service teams to validate their service health in order to update this status page. NEXT UPDATE: The next update will be provided by 17:00 UTC 05 Sep 2018 or as events warrant. https://azure.microsoft.com/en-us/status/

Time: Sept. 5, 2018, 3:40 a.m.

Status: Monitoring

Update: While some GoFormz customers have reported trouble-free services, others will continue to be impacted as Azure services continue to come back online. Microsoft Azure engineers continue to focus at this time on recovery to impacted Storage scale units in South Central US. Mitigation efforts continue for storage-dependent services as well. -- Here is the latest from Microsoft on the Azure service outage -- Multiple Services - Applying mitigation with some recovery CUSTOMER IMPACT: There are currently three identified impact workstreams. Starting at 09:29 UTC on 04 Sep 2018: 1) Customers with resources in South Central US may experience difficulties connecting to resources hosted in this region. A complete list of impacted services can be found below. 2) Customers using some non-regional services may experience intermittent failures in multiple regions. 3) Customers may encounter errors when provisioning new subscriptions. PRELIMINARY ROOT CAUSE: A severe weather event, including lightning strikes, occurred near one of the South Central US datacenters. This resulted in a power voltage increase that impacted cooling systems. Automated datacenter procedures to ensure data and hardware integrity went into effect and critical hardware entered a structured power down process. ENGINEERING STATUS: Engineers continue to implement the necessary mitigation steps according to the outlined mitigation workflow: 1) Restore power to the South Central US datacenter (COMPLETED) 2) Recover software load balancers for Azure Storage scale units in South Central US (COMPLETED) 3) Recover impacted Azure Storage scale units in South Central US (In Progress) 4) Recover the remaining Storage-dependent services in South Central US (In Progress) NEXT UPDATE: The next update will be provided by 05:00 UTC 05 Sep 2018 or as events warrant.

Time: Sept. 4, 2018, 9:22 p.m.

Status: Monitoring

Update: Microsoft Azure is still reporting issues with multiple services that are impacting Azure Active Directory across all locations, including but not limited to South Central US. -- LATEST UPDATE FROM MICROSOFT AZURE -- Warning Multiple Services - Applying mitigation with some recovery CUSTOMER IMPACT: There are currently two identified impact workstreams: 1) Customers with resources in South Central US may experience difficulties connecting to resources hosted in this region. A complete list of impacted services can be found below. 2) Customers using non-regional services, such as Azure Active Directory, may experience intermittent authentication failures in any region. PRELIMINARY ROOT CAUSE: 1) A severe weather event, including lightning strikes, occurred near one of the South Central US datacenters. This resulted in a power voltage increase that impacted cooling systems. Automated datacenter procedures to ensure data and hardware integrity went into effect and critical hardware entered a structured power down process. 2) As a result, non-regional services, such as Azure Active Directory, encountered an operational threshold for processing requests through the South Central US datacenter. Initial attempts to fail over into other datacenters resulted in temporary traffic congestion for those regions. ENGINEERING STATUS: Engineers have successfully restored power to the datacenter. Additionally, engineers have recovered a majority of the impacted network devices. While some services are starting to see signs of recovery, mitigation efforts are still ongoing.

Time: Sept. 4, 2018, 4:26 p.m.

Status: Monitoring

Update: Microsoft Azure is still reporting issues with multiple services that are impacting Azure Active Directory and Resource Manager across all locations, including but not limited to South Central US. We are monitoring the issue and will update accordingly. https://azure.microsoft.com/en-us/status/

Time: Sept. 4, 2018, 3:52 p.m.

Status: Identified

Update: GoFormz sync and other network services impacted by MS Azure outage. Starting at 09:29 UTC on 04 Sep 2018, Microsoft Azure reported technical issues with their South Central US location. Also, some services may also be experiencing intermittent authentication issues due to downstream Azure Active Directory impact, and engineers are separately working on mitigation options for this also.

Time: Sept. 6, 2018, 10:40 p.m.

Status: Resolved

Update: Although Microsoft Azure current status (as of Thursday, Sept. 6 @ 3:30pm pacific time) shows two services still impacted from Tuesday outage, we believe all of our users should be fully back online with their GoFormz services. If you are still experiencing any errors or issues please reach out to us at [email protected] and we will do everything we can to help.

Time: Sept. 5, 2018, 3:46 p.m.

Status: Monitoring

Update: While some GoFormz customers have reported trouble-free services, others will continue to be impacted as Azure services continue to come back online. -- Here is the latest from the Microsoft Azure engineering team -- Multiple Services - Applying Extended Mitigation UPDATE AS OF 15:00 UTC: Engineers have restored storage availability for the majority of impacted services, and customers should be continuing to see improvements to service availability. CUSTOMER IMPACT: Starting at 09:29 UTC on 04 Sep 2018, customers with resources in South Central US may experience difficulties connecting to resources hosted in this region. A complete list of impacted services can be found below. PRELIMINARY ROOT CAUSE: A severe weather event, including lightning strikes, occurred near one of the South Central US datacenters. This resulted in a power voltage increase that impacted cooling systems. Automated datacenter procedures to ensure data and hardware integrity went into effect and critical hardware entered a structured power down process. ENGINEERING STATUS: Engineers have restored access to storage resources for the majority of services, and most customers should be seeing signs of recovery. Engineers are continuing to work on any residual storage impact to fully mitigate this issue. We are also working with the impacted service teams to validate their service health in order to update this status page. NEXT UPDATE: The next update will be provided by 17:00 UTC 05 Sep 2018 or as events warrant. https://azure.microsoft.com/en-us/status/

Time: Sept. 5, 2018, 3:40 a.m.

Status: Monitoring

Update: While some GoFormz customers have reported trouble-free services, others will continue to be impacted as Azure services continue to come back online. Microsoft Azure engineers continue to focus at this time on recovery to impacted Storage scale units in South Central US. Mitigation efforts continue for storage-dependent services as well. -- Here is the latest from Microsoft on the Azure service outage -- Multiple Services - Applying mitigation with some recovery CUSTOMER IMPACT: There are currently three identified impact workstreams. Starting at 09:29 UTC on 04 Sep 2018: 1) Customers with resources in South Central US may experience difficulties connecting to resources hosted in this region. A complete list of impacted services can be found below. 2) Customers using some non-regional services may experience intermittent failures in multiple regions. 3) Customers may encounter errors when provisioning new subscriptions. PRELIMINARY ROOT CAUSE: A severe weather event, including lightning strikes, occurred near one of the South Central US datacenters. This resulted in a power voltage increase that impacted cooling systems. Automated datacenter procedures to ensure data and hardware integrity went into effect and critical hardware entered a structured power down process. ENGINEERING STATUS: Engineers continue to implement the necessary mitigation steps according to the outlined mitigation workflow: 1) Restore power to the South Central US datacenter (COMPLETED) 2) Recover software load balancers for Azure Storage scale units in South Central US (COMPLETED) 3) Recover impacted Azure Storage scale units in South Central US (In Progress) 4) Recover the remaining Storage-dependent services in South Central US (In Progress) NEXT UPDATE: The next update will be provided by 05:00 UTC 05 Sep 2018 or as events warrant.

Time: Sept. 4, 2018, 9:22 p.m.

Status: Monitoring

Update: Microsoft Azure is still reporting issues with multiple services that are impacting Azure Active Directory across all locations, including but not limited to South Central US. -- LATEST UPDATE FROM MICROSOFT AZURE -- Warning Multiple Services - Applying mitigation with some recovery CUSTOMER IMPACT: There are currently two identified impact workstreams: 1) Customers with resources in South Central US may experience difficulties connecting to resources hosted in this region. A complete list of impacted services can be found below. 2) Customers using non-regional services, such as Azure Active Directory, may experience intermittent authentication failures in any region. PRELIMINARY ROOT CAUSE: 1) A severe weather event, including lightning strikes, occurred near one of the South Central US datacenters. This resulted in a power voltage increase that impacted cooling systems. Automated datacenter procedures to ensure data and hardware integrity went into effect and critical hardware entered a structured power down process. 2) As a result, non-regional services, such as Azure Active Directory, encountered an operational threshold for processing requests through the South Central US datacenter. Initial attempts to fail over into other datacenters resulted in temporary traffic congestion for those regions. ENGINEERING STATUS: Engineers have successfully restored power to the datacenter. Additionally, engineers have recovered a majority of the impacted network devices. While some services are starting to see signs of recovery, mitigation efforts are still ongoing.

Time: Sept. 4, 2018, 4:26 p.m.

Status: Monitoring

Update: Microsoft Azure is still reporting issues with multiple services that are impacting Azure Active Directory and Resource Manager across all locations, including but not limited to South Central US. We are monitoring the issue and will update accordingly. https://azure.microsoft.com/en-us/status/

Time: Sept. 4, 2018, 3:52 p.m.

Status: Identified

Update: GoFormz sync and other network services impacted by MS Azure outage. Starting at 09:29 UTC on 04 Sep 2018, Microsoft Azure reported technical issues with their South Central US location. Also, some services may also be experiencing intermittent authentication issues due to downstream Azure Active Directory impact, and engineers are separately working on mitigation options for this also.

Time: June 13, 2018, 7:41 p.m.

Status: Resolved

Update: Microsoft Azure is reporting recovery of services and GoFormz is confirming that our service issues related to this outage have been resolved.

Time: June 13, 2018, 5:20 p.m.

Status: Identified

Update: Due to an outage of our cloud provider Microsoft Azure, we are experiencing a partial outage of services that also include our API and some mobile functions such as sync.

Time: June 13, 2018, 7:41 p.m.

Status: Resolved

Update: Microsoft Azure is reporting recovery of services and GoFormz is confirming that our service issues related to this outage have been resolved.

Time: June 13, 2018, 5:20 p.m.

Status: Identified

Update: Due to an outage of our cloud provider Microsoft Azure, we are experiencing a partial outage of services that also include our API and some mobile functions such as sync.