Time: Jan. 4, 2024, 4:21 a.m.

Status: Resolved

Update: Between 19:03 UTC and 20:39 UTC, we experienced degraded performance accessing Confluence pages for some customers in the EU and US West region. The issue has been resolved and the service is operating normally.

Time: Jan. 3, 2024, 8:39 p.m.

Status: Monitoring

Update: We have identified the root cause of the intermittent copy page degradation and have mitigated the problem. We are now monitoring closely.

Time: Jan. 3, 2024, 7:03 p.m.

Status: Investigating

Update: We are investigating cases of degraded performance for Confluence Cloud customers in EU and US West regions. We will provide more details within the 2 hours.

Time: Dec. 21, 2023, 4:21 a.m.

Status: Resolved

Update: Between 00:00 UTC and 00:20 UTC, we experienced degraded performance accessing Confluence pages for some customers in the Asia Pacific region. The issue has been resolved and the service is operating normally.

Time: Dec. 21, 2023, 1:51 a.m.

Status: Monitoring

Update: We have identified the root cause of the increased errors and have mitigated the problem. We are now monitoring this closely.

Time: Dec. 21, 2023, 1:45 a.m.

Status: Investigating

Update: We are investigating cases of degraded performance for some Confluence Cloud customers between 00:00 and 00:20 UTC affecting the view/edit experience.. We will provide more details within the next hour.

Time: Dec. 21, 2023, 4:21 a.m.

Status: Resolved

Update: Between 00:00 UTC and 00:20 UTC, we experienced degraded performance accessing Confluence pages for some customers in the Asia Pacific region. The issue has been resolved and the service is operating normally.

Time: Dec. 21, 2023, 1:51 a.m.

Status: Monitoring

Update: We have identified the root cause of the increased errors and have mitigated the problem. We are now monitoring this closely.

Time: Dec. 21, 2023, 1:45 a.m.

Status: Investigating

Update: We are investigating cases of degraded performance for some Confluence Cloud customers between 00:00 and 00:20 UTC affecting the view/edit experience.. We will provide more details within the next hour.

Time: Dec. 28, 2023, 3:18 a.m.

Status: Postmortem

Update: ### **SUMMARY** On Dec 18, 2023, between 12:29 p.m. and 3:35 p.m. UTC, Dec 18, 2023, Atlassian's cloud customers using Atlas, Bitbucket Cloud, Compass, Confluence Cloud, Jira Service Management, Jira Software, Jira Work Management, Jira Product Discovery products were unable to search for users or use the "@mention" functionality. Customers' user search results failed or were delayed as Atlassian's service returning user search results was degraded in several regions. The incident originated from a computationally intensive operation that was triggered multiple times in rapid succession, resulting in degraded performance of Atlassian's user search service across several regions. Notably, customers in the EU west region were most affected. The incident was detected within 2 minutes by automated monitoring, and our team promptly took action by recovering unhealthy systems and scaling up the service's infrastructure temporarily. The resolution process concluded in 3 hours and 06 minutes. ### **IMPACT** The overall impact was between Dec 18, 2023, between 12:29 p.m. UTC and Dec 18, 2023, 3:35 p.m. UTC. The Incident caused service disruption to cloud customers worldwide. Customers experienced delayed or failed user searches when using the following Atlassian cloud products: * Atlas * Bitbucket Cloud * Compass * Confluence Cloud * Jira Service Management * Jira Software * Jira Work Management * Jira Product Discovery ### **ROOT CAUSE** The incident stemmed from Atlassian's user search service receiving commands to process multiple computationally intensive operations in rapid succession. These operations were directed at the same customer data set, and therefore overloaded resources within a clustered database system, leading to memory exhaustion and subsequent unresponsiveness to user search requests.  ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** To prevent a recurrence of such incidents, we are implementing the following measures: * Implement a mechanism to queue computationally intensive operations in order to avoid overloading the resources within the systems and process them without impact on customer experience. * Fine-tune our clustered database settings to mitigate the impact of resource exhaustion on the overall system.  We apologize to customers whose services were affected during this incident; we are taking immediate steps to improve the service’s resiliency. Thanks, Atlassian Customer Support

Time: Dec. 28, 2023, 3:18 a.m.

Status: Postmortem

Update: ### **SUMMARY** On Dec 18, 2023, between 12:29 p.m. and 3:35 p.m. UTC, Dec 18, 2023, Atlassian's cloud customers using Atlas, Bitbucket Cloud, Compass, Confluence Cloud, Jira Service Management, Jira Software, Jira Work Management, Jira Product Discovery products were unable to search for users or use the "@mention" functionality. Customers' user search results failed or were delayed as Atlassian's service returning user search results was degraded in several regions. The incident originated from a computationally intensive operation that was triggered multiple times in rapid succession, resulting in degraded performance of Atlassian's user search service across several regions. Notably, customers in the EU west region were most affected. The incident was detected within 2 minutes by automated monitoring, and our team promptly took action by recovering unhealthy systems and scaling up the service's infrastructure temporarily. The resolution process concluded in 3 hours and 06 minutes. ### **IMPACT** The overall impact was between Dec 18, 2023, between 12:29 p.m. UTC and Dec 18, 2023, 3:35 p.m. UTC. The Incident caused service disruption to cloud customers worldwide. Customers experienced delayed or failed user searches when using the following Atlassian cloud products: * Atlas * Bitbucket Cloud * Compass * Confluence Cloud * Jira Service Management * Jira Software * Jira Work Management * Jira Product Discovery ### **ROOT CAUSE** The incident stemmed from Atlassian's user search service receiving commands to process multiple computationally intensive operations in rapid succession. These operations were directed at the same customer data set, and therefore overloaded resources within a clustered database system, leading to memory exhaustion and subsequent unresponsiveness to user search requests.  ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** To prevent a recurrence of such incidents, we are implementing the following measures: * Implement a mechanism to queue computationally intensive operations in order to avoid overloading the resources within the systems and process them without impact on customer experience. * Fine-tune our clustered database settings to mitigate the impact of resource exhaustion on the overall system.  We apologize to customers whose services were affected during this incident; we are taking immediate steps to improve the service’s resiliency. Thanks, Atlassian Customer Support

Time: Dec. 19, 2023, 3:28 a.m.

Status: Resolved

Update: It has been resolved. Atlassian's cross product user search is working.

Time: Dec. 18, 2023, 4:23 p.m.

Status: Investigating

Update: Atlassian's cross product user search service is currently healthy. Searches for users within Atlassian products are working as expected. We are in the process of investigating the root cause of this incident.

Time: Dec. 18, 2023, 4:23 p.m.

Status: Investigating

Update: Atlassian's cross product user search service is currently healthy. Searches for users within Atlassian products are working as expected. We are in the process of investigating the root cause of this incident.

Time: Dec. 18, 2023, 4:06 p.m.

Status: Investigating

Update: Atlassian's cross product user search service is recovering. Searches for users within Atlassian products are returning to normal.

Time: Dec. 18, 2023, 4:06 p.m.

Status: Investigating

Update: Atlassian's cross product user search service is recovering. Searches for users within Atlassian products are returning to normal.

Time: Dec. 18, 2023, 4:02 p.m.

Status: Investigating

Update: Atlassian's cross product user search service is recovering. Searches for users within Atlassian products are returning to normal.

Time: Dec. 18, 2023, 4:02 p.m.

Status: Investigating

Update: Atlassian's cross product user search service is recovering. Searches for users within Atlassian products are returning to normal.

Time: Dec. 18, 2023, 4:01 p.m.

Status: Investigating

Update: Atlassian's cross product user search service is recovering. Searches for users within Atlassian products are returning to normal.

Time: Dec. 18, 2023, 4:01 p.m.

Status: Investigating

Update: Atlassian's cross product user search service is recovering. Searches for users within Atlassian products are returning to normal.

Time: Dec. 18, 2023, 3:04 p.m.

Status: Investigating

Update: We are investigating reports of intermittent errors for <SOME/ALL> Atlassian, Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Align, Jira Product Discovery, Atlas, and Compass Cloud customers. We will provide more details once we identify the root cause.

Time: Dec. 18, 2023, 3:04 p.m.

Status: Investigating

Update: We are investigating reports of intermittent errors for <SOME/ALL> Atlassian, Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Align, Jira Product Discovery, Atlas, and Compass Cloud customers. We will provide more details once we identify the root cause.

Time: Dec. 4, 2023, 5:15 p.m.

Status: Resolved

Update: The systems are stable after the fix and monitoring for a specified duration

Time: Dec. 4, 2023, 4:59 p.m.

Status: Monitoring

Update: The issue was identified and a fix implemented. We are monitoring currently.

Time: Dec. 4, 2023, 4:54 p.m.

Status: Identified

Update: We are currently investigating an incident that result in outbound connections from Atlassian cloud in us-east-1 intermittently timing out. This affects Jira, Trello, Confluence, Ecosystem products. The features affected for these products are those that require opening a connection from Atlassian Cloud to public endpoints on the Internet

Time: Dec. 4, 2023, 4:43 p.m.

Status: Identified

Update: Including Atlassian Developer

Time: Dec. 4, 2023, 4:33 p.m.

Status: Identified

Update: We are currently investigating an incident that result in connection time outs on service egress proxy. This affects Jira, JSM, Confluence, BitBucket, Trello, Ecosystem products. The features affected for these products are those that require a connection to service egress.