Time: July 9, 2024, 8:47 p.m.

Status: Postmortem

Update: Incident: An off-peak routine update of our service handling new page loads caused us to deploy code that depended on an artifact that wasn't yet available. The net effect was that all traffic attempting to do a fresh page load was broken for 48 minutes from June 28 5:52 pm PT to June 28 6:39 pm PT. Impact: Asana was unavailable to new page loads, but existing sessions continued to work as normal. Moving forward: This incident involved interaction across teams and systems. Our postmortem of this event identified operational and tooling changes to reduce the likelihood of incidents -- and decrease the time to resolution. We have an ongoing [5 Whys](https://wavelength.asana.com/workstyle-ask-5-whys-to-get-to-the-root-of-any-problem/) analysis for this incident.

Time: July 3, 2024, 2:22 a.m.

Status: Resolved

Update: We've resolved the issue where some users were unable to load the Asana application. We're investigating the root cause. We apologize for the inconvenience.

Time: July 9, 2024, 8:47 p.m.

Status: Postmortem

Update: Incident: An off-peak routine update of our service handling new page loads caused us to deploy code that depended on an artifact that wasn't yet available. The net effect was that all traffic attempting to do a fresh page load was broken for 48 minutes from June 28 5:52 pm PT to June 28 6:39 pm PT. Impact: Asana was unavailable to new page loads, but existing sessions continued to work as normal. Moving forward: This incident involved interaction across teams and systems. Our postmortem of this event identified operational and tooling changes to reduce the likelihood of incidents -- and decrease the time to resolution. We have an ongoing [5 Whys](https://wavelength.asana.com/workstyle-ask-5-whys-to-get-to-the-root-of-any-problem/) analysis for this incident.

Time: July 3, 2024, 2:22 a.m.

Status: Resolved

Update: We've resolved the issue where some users were unable to load the Asana application. We're investigating the root cause. We apologize for the inconvenience.

Time: July 3, 2024, 6:18 p.m.

Status: Postmortem

Update: **Incident**: On June 28, 2024 between 0334  and 0356 UTC, network maintenance was performed to remap internal endpoints, with unforecasted negative effects on communication between internal services. This caused a large amount of requests to Asana during the maintenance period to either drop or time out. **Impact**: App usability was degraded for US-based users, resulting in load times of up to a minute. Users of the mobile app in our US, European, Australian, and Japanese data centers were also affected to a lesser extent by elevated error rates. **Moving forward**: We've identified process gaps during internal retrospective. Moving forward, we'll perform internal endpoint remappings more incrementally to avoid app disruption, and will apply a more thorough approval process for similar maintenance procedures. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.

Time: July 3, 2024, 6:17 p.m.

Status: Resolved

Update: Incident: On June 28, 2024 between 0334  and 0356 UTC, network maintenance was performed to remap internal endpoints, with unforecasted negative effects on communication between internal services. This caused a large amount of requests to Asana during the maintenance period to either drop or time out. Impact: App usability was degraded for US-based users, resulting in load times of up to a minute. Users of the mobile app in our US, European, Australian, and Japanese data centers were also affected to a lesser extent by elevated error rates. Moving forward: We've identified process gaps during internal retrospective. Moving forward, we'll perform internal endpoint remappings more incrementally to avoid app disruption, and will apply a more thorough approval process for similar maintenance procedures. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.

Time: July 3, 2024, 6:18 p.m.

Status: Postmortem

Update: **Incident**: On June 28, 2024 between 0334  and 0356 UTC, network maintenance was performed to remap internal endpoints, with unforecasted negative effects on communication between internal services. This caused a large amount of requests to Asana during the maintenance period to either drop or time out. **Impact**: App usability was degraded for US-based users, resulting in load times of up to a minute. Users of the mobile app in our US, European, Australian, and Japanese data centers were also affected to a lesser extent by elevated error rates. **Moving forward**: We've identified process gaps during internal retrospective. Moving forward, we'll perform internal endpoint remappings more incrementally to avoid app disruption, and will apply a more thorough approval process for similar maintenance procedures. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.

Time: July 3, 2024, 6:17 p.m.

Status: Resolved

Update: Incident: On June 28, 2024 between 0334  and 0356 UTC, network maintenance was performed to remap internal endpoints, with unforecasted negative effects on communication between internal services. This caused a large amount of requests to Asana during the maintenance period to either drop or time out. Impact: App usability was degraded for US-based users, resulting in load times of up to a minute. Users of the mobile app in our US, European, Australian, and Japanese data centers were also affected to a lesser extent by elevated error rates. Moving forward: We've identified process gaps during internal retrospective. Moving forward, we'll perform internal endpoint remappings more incrementally to avoid app disruption, and will apply a more thorough approval process for similar maintenance procedures. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.

Time: June 19, 2024, 3:45 p.m.

Status: Postmortem

Update: Incident: Asana uses several load balancers to route incoming requests to the correct backend service. As our volume of traffic increased over time, these load balancers consumed more resources to handle the additional requests. High traffic pushed computing resource utilization past a critical threshold leading to queueing of slow requests and eventual failures. This incident was made worse by certain error reporting mechanisms sending additional requests through the same load balancing infrastructure. ‌ Impact: Between 15:08 and 16:50 UTC on June 17, 2024, requests to load the Asana application, use of the Asana mobile applications, and calls to our API were delayed or failed at an elevated rate. Users in our European, Australian, and Japanese data centers may have experienced additional crashes while using Asana. ‌ Moving forward: In the short term, we have increased the capacity for our load balancing infrastructure to handle our current traffic as well as anticipated growth, and improved the monitoring to alert us before failure. In the longer term, we have efforts underway to replace this component entirely with routing and load balancing infrastructure that automatically scales in response to increases in traffic. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.

Time: June 17, 2024, 7:49 p.m.

Status: Resolved

Update: This incident has been resolved.

Time: June 17, 2024, 4:52 p.m.

Status: Monitoring

Update: A fix has been implemented and we are monitoring the results.

Time: June 17, 2024, 4:44 p.m.

Status: Investigating

Update: We're currently experiencing some difficulties; as a result, a majority of our users are unable to access Asana. Our Development Team is currently working to resolve this issue as soon as possible. Sincere apologies for the inconvenience caused, please keep an eye on this page for the latest updates.