Last checked: 4 minutes ago
Get notified about any outages, downtime or incidents for Jira and 1800+ other cloud vendors. Monitor 10 companies, for free.
Outage and incident data over the last 30 days for Jira.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage. It's completely free and takes less than 2 minutes!
Sign Up NowOutlogger tracks the status of these components for Xero:
Component | Status |
---|---|
Administration | Active |
Authentication and User Management | Active |
Automation for Jira | Active |
Create and edit | Active |
Marketplace | Active |
Mobile | Active |
Notifications | Active |
Purchasing & Licensing | Active |
Search | Active |
Signup | Active |
Viewing content | Active |
View the latest incidents for Jira and check for official updates:
Description: ### Summary On August 19, 2024, between 6:00 am and 9:58 am UTC, Jira customers located in the EU West 1 region experienced intermittent failures when loading boards and backlogs inside team-managed projects. These failures were triggered by a bug introduced to a backend service that increased load which then triggered downstream services to reject some requests due to rate-limiting. The incident was detected within five minutes by automated monitoring systems and mitigated by a rollback of the faulty service which put Atlassian systems into a known good state. The total time to resolution was approximately four hours. ### Impact All customers in the EU West 1 region experienced elevated error rates when trying to access the Jira team-managed boards and backlogs on Monday, August 19, between 6:00 am UTC and 10:00 am UTC. Customers may have noticed the board and backlog views failing to load due to 429 and 500 error responses. However, they may have been able to eventually view the page after multiple retries. ### Root Cause The issue was caused by a change to a service backing the team-managed experiences. Specifically, a caching layer was accidentally removed which caused a large increase in the number of requests being sent to a downstream service. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We know that outages impact your productivity. While we have a number of testing and preventative processes in place, this specific issue wasn’t identified because the change was related to the introduction of a feature flag that was not picked up by our automated continuous deployment suites and manual test scripts. New feature changes are usually behind feature flags and rolled out progressively to customers to allow for increased safety when making new changes. However, in this case, the bug that caused this incident came as a result of an unintended change in behaviour when introducing this flag into our code base. We are prioritizing the following improvement actions to avoid repeating this type of incident: * Improving the test coverage in our services to enforce caching within the service. * Improving the scaling configurations of our services to allow them to handle large increases in load and make them more resilient to spikes in traffic. * Increasing the coverage of rate limiting within our services. We apologize to customers whose services were impacted during this incident; we are taking immediate steps to improve the platform’s performance and availability. Thanks, Atlassian Customer Support
Status: Postmortem
Impact: Major | Started At: Aug. 19, 2024, 7:27 a.m.
Description: Between 15 Aug 2024 - 10:30 UTC to 15 Aug 2024 - 22:40 UTC, editing of Jira Workflow Schemes were affected in Jira, Jira Service Management, Jira work management. The issue has been resolved and the service is operating normally. If you face any challenges please reach out to us via support ticket.
Status: Resolved
Impact: Minor | Started At: Aug. 15, 2024, 2:45 p.m.
Description: ### Summary From `12/Aug/24 10:39 UTC` to `13/Aug/24 14:38 UTC` some customers experienced degraded performance on Issue View in Jira. This was caused by a data processing compatibility problem between a cache, the underlying database, and the new deployment. Due to a slow increase in failure rate and a small initial surface area of impact, the problem didn’t immediately trigger our continuous error monitoring and alerting. Once we identified the issue it was resolving itself through self-healing mechanisms in the infrastructure. However, in a few outlier cases, we had to intervene with tenant specific cache recalculations. All but 6 tenants were fully remediated by `12/Aug/24 21:30 UTC`. The issue occurred on the read layer of our architecture so while customer experience was degraded, there was no data loss. ### **IMPACT** About 1% of instances in were impacted over the lifetime of the incident. Users on those impacted instances would have experienced degradation when loading Issue View in a specific scenario. This was when a Multi Select Custom Field is enabled on an Issue and where that Custom Field also had a Default Value set. ### **ROOT CAUSE** We introduced a change in our code which caused processing of Custom Fields in specific configurations to fail. This prevented Issue View from loading issues for projects with the above specific configuration applied. This problem occurred because of different representations of the data in the database, in the code base, and in the cache in the production environment. These multiple representations caused an exception when translating the data from one representation to the next. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** The problem largely self healed as the cache expired and was refreshed with compatible data; however, we chose to force cache re-computation for affected tenants in order to expedite this process. We chose not to roll back the deployment at that point as that would have created the reverse compatibility issue with the already healed tenants. Instead, we focussed on forward fixing with a hotfix and accelerating remediation for still affected tenants. For a small very number of tenants, forced re-computation did not immediately rectify and we had to roll forward a code hotfix to remediate. We are prioritizing the following improvement actions to avoid repeating this type of incident: * The already deployed hot fix to stop this particular problem recurring. * A series of tests for this class of issue in our read layer. * A review of monitoring to detect these fine grained problems before they cause more customer impact. We apologize for any disruption this issue may have caused and are taking steps to help ensure it does not occur again. Thanks, Atlassian Customer Support
Status: Postmortem
Impact: Minor | Started At: Aug. 12, 2024, 3:51 p.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Minor | Started At: Aug. 2, 2024, 2:24 p.m.
Description: Between 28 July 2024, 23:00 UTC, and 29 July 2024, 10:22 UTC, some customers experienced performance degradation issues for Jira and JSM. The root cause was a problem with the propagation of configuration in our system during the migration of one of the database instances hosting your Jira cloud site. This caused the Jira application not to correctly balance load against database nodes within the database cluster, leading to CPU saturation of the database. We have deployed a fix to mitigate the issue and have verified that the services have recovered. The conditions that cause the bug have been addressed and we're actively working on a permanent fix. The issue has been resolved and the service is operating normally.
Status: Resolved
Impact: None | Started At: July 29, 2024, 8:27 a.m.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage or down time. Join for free - no credit card required.