Last checked: 5 minutes ago
Get notified about any outages, downtime or incidents for Ultimate AI and 1800+ other cloud vendors. Monitor 10 companies, for free.
Outage and incident data over the last 30 days for Ultimate AI.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage. It's completely free and takes less than 2 minutes!
Sign Up NowOutlogger tracks the status of these components for Xero:
Component | Status |
---|---|
Analytics | Active |
Backend Integrations | Active |
Core Services | Active |
Dashboard | Active |
Public API | Active |
Web Chat Widget | Active |
Chat integrations | Active |
Freshchat | Active |
Freshdesk Automation | Active |
Giosg Automation | Active |
Intercom Automation | Active |
LiveChat.com CRM Integration | Active |
Salesforce | Active |
Sunshine | Active |
Zendesk Chat | Active |
Zendesk Support Automation | Active |
Thirdparty Providers | Active |
LiveChat Agent apps | Active |
LiveChat API | Active |
LiveChat Chat widget | Active |
LiveChat Integrations | Active |
LiveChat Subprocessors' service | Active |
Salesforce EU37 | Active |
Salesforce EU40 | Active |
Salesforce UM7 | Active |
Zendesk Sunshine Conversations Sunshine Conversations Core API | Active |
Zendesk Sunshine Conversations Web SDK | Active |
View the latest incidents for Ultimate AI and check for official updates:
Description: This incident has been resolved by LiveChat
Status: Resolved
Impact: Critical | Started At: April 19, 2023, 11:16 a.m.
Description: All systems are operational again and we will keep monitoring the situation.
Status: Resolved
Impact: Major | Started At: March 20, 2023, 3:42 a.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Minor | Started At: Feb. 20, 2023, 2:56 p.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Minor | Started At: Feb. 15, 2023, 11:17 a.m.
Description: A misconfiguration in [ArgoCD](https://argoproj.github.io/cd/), our deployment git-ops tool, resulted in issues with [shared resources](https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#fail-the-sync-if-a-shared-resource-is-found) between two projects \(A and B\). By default, Argo CD applies all manifests found in the git path defined in the application, regardless of whether the resources have already been applied by another application. Due to this issue, both projects claimed ownership of the resources \(in this case, the namespaces in our Kubernetes clusters that host all our applications\). In an effort to resolve the issue, a decision was made with to remove Project B. Post incident, we later learned that it would have been more efficient to fail the sync as described [here](https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#fail-the-sync-if-a-shared-resource-is-found). However, since Project B also claimed ownership of the namespace resources across all our clusters and environments, a [cascading deletion](https://kubernetes.io/docs/concepts/architecture/garbage-collection/#cascading-deletion) event took place once we removed project B, resulting in the removal of all applications and [orphan objects ](https://kubernetes.io/docs/concepts/architecture/garbage-collection/#orphaned-dependents)across our environments. This caused significant downtime across our platform. To resolve the matter, we made sure ArgoCD slowly brought the applications back up in batches, to not overload our clusters. We have already created follow-up tasks to put safeguards in place to avoid this in the future. If you have further questions please contact us through [[email protected]](mailto:[email protected])
Status: Postmortem
Impact: None | Started At: Jan. 30, 2023, 12:45 p.m.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage or down time. Join for free - no credit card required.