Is there an Ultimate AI outage?
Ultimate AI status: Systems Active
Last checked: 53 seconds ago
Get notified about any outages, downtime or incidents for Ultimate AI and 1800+ other cloud vendors. Monitor 10 companies, for free.
Last checked: 53 seconds ago
Get notified about any outages, downtime or incidents for Ultimate AI and 1800+ other cloud vendors. Monitor 10 companies, for free.
Outage and incident data over the last 30 days for Ultimate AI.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage. It's completely free and takes less than 2 minutes!
Sign Up NowOutlogger tracks the status of these components for Xero:
| Component | Status |
|---|---|
| Analytics | Active |
| Backend Integrations | Active |
| Core Services | Active |
| Dashboard | Active |
| Public API | Active |
| Web Chat Widget | Active |
| Chat integrations | Active |
| Freshchat | Active |
| Freshdesk Automation | Active |
| Giosg Automation | Active |
| Intercom Automation | Active |
| LiveChat.com CRM Integration | Active |
| Salesforce | Active |
| Sunshine | Active |
| Zendesk Chat | Active |
| Zendesk Support Automation | Active |
| Thirdparty Providers | Active |
| LiveChat Agent apps | Active |
| LiveChat API | Active |
| LiveChat Chat widget | Active |
| LiveChat Integrations | Active |
| LiveChat Subprocessors' service | Active |
| Salesforce EU37 | Active |
| Salesforce EU40 | Active |
| Salesforce UM7 | Active |
| Zendesk Sunshine Conversations Sunshine Conversations Core API | Active |
| Zendesk Sunshine Conversations Web SDK | Active |
View the latest incidents for Ultimate AI and check for official updates:
Description: **Time of Incident** Detection: 06:00 UTC, May 23rd Resolution: 09:15 UTC, May 23rd **Impact** Total Failed Requests: 1809 **Incident Details** At approximately 06:00 UTC, we observed that some customers were encountering errors when attempting to use our Public API service. Initial diagnostics indicated that the issue was not universal, as we continued to receive successful traffic alongside the errors. Upon further investigation, we identified that the failed requests were being routed through our US ingress. The root cause was traced back to a deployment made the previous day \(May 22nd\) at around 15:15 UTC. This deployment involved our US version of the services, which was not yet intended to handle traffic. However, due to an error in our DNS configuration, both our EU and US domains were inadvertently sharing the same DNS records, leading to traffic being routed to both. As a result of this misconfiguration, all incoming requests to the US service failed, but it is important to note that no customer data was processed or stored by the US service. **Resolution** Upon identifying the cause, at around 09:15 UTC, we deployed a fix to prevent traffic from being routed through the US ingress. This successfully resolved the problem, and our Public API service resumed normal operations. **Preventive Measures** To prevent a recurrence of this issue, we have taken the following immediate actions: We have shut down our US ingress entirely, this ensures that no traffic will be routed to the US ingress until we are confident the issue is fully resolved. Before bringing the US ingress back online, we will conduct a thorough review and reconfiguration of our DNS settings to ensure that the EU and US domains are properly isolated and do not share traffic inadvertently.
Status: Postmortem
Impact: Major | Started At: May 23, 2024, 9:25 a.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Major | Started At: May 10, 2024, 5:17 p.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Major | Started At: May 7, 2024, 5:07 p.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Minor | Started At: May 7, 2024, 9:55 a.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Critical | Started At: May 5, 2024, 9:18 a.m.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage or down time. Join for free - no credit card required.
