Is there an Molecule outage?
Molecule status: Systems Active
Last checked: 6 minutes ago
Get notified about any outages, downtime or incidents for Molecule and 1800+ other cloud vendors. Monitor 10 companies, for free.
Last checked: 6 minutes ago
Get notified about any outages, downtime or incidents for Molecule and 1800+ other cloud vendors. Monitor 10 companies, for free.
Outage and incident data over the last 30 days for Molecule.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage. It's completely free and takes less than 2 minutes!
Sign Up NowOutlogger tracks the status of these components for Xero:
| Component | Status |
|---|---|
| Europe | Active |
| Amazon Web Services - Databases | Active |
| Amazon Web Services - DNS | Active |
| Amazon Web Services - EC2 | Active |
| Amazon Web Services - EKS | Active |
| Amazon Web Services - File Storage | Active |
| Auth0 User Authentication | Active |
| Molecule Europe | Active |
| US | Active |
| Amazon Web Services - Databases | Active |
| Amazon Web Services - DNS | Active |
| Amazon Web Services - EC2 | Active |
| Amazon Web Services - EKS | Active |
| Auth0 User Authentication | Active |
| Molecule | Active |
View the latest incidents for Molecule and check for official updates:
Description: This incident has been resolved.
Status: Resolved
Impact: Maintenance | Started At: May 15, 2023, 8:55 p.m.
Description: The underlying Auth0 incident has been resolved, without needing to roll out the backup login system.
Status: Resolved
Impact: Minor | Started At: March 13, 2023, 4:09 p.m.
Description: The underlying Auth0 incident has been resolved, without needing to roll out the backup login system.
Status: Resolved
Impact: Minor | Started At: March 13, 2023, 4:09 p.m.
Description: Last Friday, Molecule experienced a handful of 1- to 4-minute service interruptions between 5:55 PM and 6:59 PM Central Time. We apologize for the interruption, and would like to share the findings from our internal investigation. ## What Happened At 5:55 PM, the Molecule app was unavailable for 4 minutes, because our primary database was suffering excessive load. This happened 3 more times \(with up to 4-minute interruptions each\) over the next hour. ## Why? The database load was primarily due to a mis-timed backup job running during a high-load time for Molecule. The backup is designed to run during the US night, but due to a time zone configuration change \(i.e., US Central to UTC\) it was running during the US day. Complicating this was an unusual set of customer activity querying a large amount of old data \(which is more resource-intensive to query\). ## Preventing This Type of Issue We have, of course, fixed the backup time zone issue. Other items we are working on as well: * Addressing the issue of high-load queries for old data * Ensuring that all of our scheduled jobs are set to the same time zone, to prevent confusion Thank you for your patience, and again--we’re sorry for the interruption.
Status: Postmortem
Impact: Major | Started At: Dec. 3, 2022, 12:46 a.m.
Description: Last Friday, Molecule experienced a handful of 1- to 4-minute service interruptions between 5:55 PM and 6:59 PM Central Time. We apologize for the interruption, and would like to share the findings from our internal investigation. ## What Happened At 5:55 PM, the Molecule app was unavailable for 4 minutes, because our primary database was suffering excessive load. This happened 3 more times \(with up to 4-minute interruptions each\) over the next hour. ## Why? The database load was primarily due to a mis-timed backup job running during a high-load time for Molecule. The backup is designed to run during the US night, but due to a time zone configuration change \(i.e., US Central to UTC\) it was running during the US day. Complicating this was an unusual set of customer activity querying a large amount of old data \(which is more resource-intensive to query\). ## Preventing This Type of Issue We have, of course, fixed the backup time zone issue. Other items we are working on as well: * Addressing the issue of high-load queries for old data * Ensuring that all of our scheduled jobs are set to the same time zone, to prevent confusion Thank you for your patience, and again--we’re sorry for the interruption.
Status: Postmortem
Impact: Major | Started At: Dec. 3, 2022, 12:46 a.m.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage or down time. Join for free - no credit card required.
