Company Logo

Is there an Ultimate AI outage?

Ultimate AI status: Systems Active

Last checked: 5 minutes ago

Get notified about any outages, downtime or incidents for Ultimate AI and 1800+ other cloud vendors. Monitor 10 companies, for free.

Subscribe for updates

Ultimate AI outages and incidents

Outage and incident data over the last 30 days for Ultimate AI.

There have been 3 outages or incidents for Ultimate AI in the last 30 days.

Severity Breakdown:

Tired of searching for status updates?

Join OutLogger to be notified when any of your vendors or the components you use experience an outage. It's completely free and takes less than 2 minutes!

Sign Up Now

Components and Services Monitored for Ultimate AI

Outlogger tracks the status of these components for Xero:

Analytics Active
Backend Integrations Active
Core Services Active
Dashboard Active
Public API Active
Web Chat Widget Active
Freshchat Active
Freshdesk Automation Active
Giosg Automation Active
Intercom Automation Active
LiveChat.com CRM Integration Active
Salesforce Active
Sunshine Active
Zendesk Chat Active
Zendesk Support Automation Active
LiveChat Agent apps Active
LiveChat API Active
LiveChat Chat widget Active
LiveChat Integrations Active
LiveChat Subprocessors' service Active
Salesforce EU37 Active
Salesforce EU40 Active
Salesforce UM7 Active
Zendesk Sunshine Conversations Sunshine Conversations Core API Active
Zendesk Sunshine Conversations Web SDK Active
Component Status
Analytics Active
Backend Integrations Active
Core Services Active
Dashboard Active
Public API Active
Web Chat Widget Active
Active
Freshchat Active
Freshdesk Automation Active
Giosg Automation Active
Intercom Automation Active
LiveChat.com CRM Integration Active
Salesforce Active
Sunshine Active
Zendesk Chat Active
Zendesk Support Automation Active
Active
LiveChat Agent apps Active
LiveChat API Active
LiveChat Chat widget Active
LiveChat Integrations Active
LiveChat Subprocessors' service Active
Salesforce EU37 Active
Salesforce EU40 Active
Salesforce UM7 Active
Zendesk Sunshine Conversations Sunshine Conversations Core API Active
Zendesk Sunshine Conversations Web SDK Active

Latest Ultimate AI outages and incidents.

View the latest incidents for Ultimate AI and check for official updates:

Updates:

  • Time: Nov. 22, 2024, 11:34 a.m.
    Status: Resolved
    Update: This incident has been resolved.
  • Time: Nov. 22, 2024, 10:42 a.m.
    Status: Monitoring
    Update: All services should be back to normal. We continue to monitor.
  • Time: Nov. 22, 2024, 10:29 a.m.
    Status: Investigating
    Update: AI Agents are operational again, we have identified a partial outage still only in the Salesforce integration.
  • Time: Nov. 22, 2024, 10:28 a.m.
    Status: Investigating
    Update: AI Agents are operational again. Some users still experience issues when logging into the dashboard - we are investigating.
  • Time: Nov. 22, 2024, 10:09 a.m.
    Status: Monitoring
    Update: We are continuing to monitor for any further issues.
  • Time: Nov. 22, 2024, 10:09 a.m.
    Status: Monitoring
    Update: We have rolled back latest deployment, services are operational again. We continue to monitor.
  • Time: Nov. 22, 2024, 10:08 a.m.
    Status: Monitoring
    Update: Users experience issues logging into dashboard. We noticed that ai agents encounter issues responding to messages.

Updates:

  • Time: Nov. 18, 2024, 2:29 p.m.
    Status: Resolved
    Update: This incident has been resolved.
  • Time: Nov. 18, 2024, 2:27 p.m.
    Status: Monitoring
    Update: The update has been under monitoring and no other issues have been identified
  • Time: Nov. 18, 2024, 2:25 p.m.
    Status: Monitoring
    Update: We are continuing to monitor for any further issues.
  • Time: Nov. 15, 2024, 2 p.m.
    Status: Monitoring
    Update: After deploying an improvement aimed at multi-touch ticket automation to enable support for Zendesk API, we forced an update on the JSON body of the ultimate trigger - Impacting some customers. Feature has been already rolled back - we are monitoring the impact.

Updates:

  • Time: Nov. 11, 2024, 9:17 p.m.
    Status: Resolved
    Update: We have not observed any errors, and all of our services are functioning normally.
  • Time: Nov. 11, 2024, 6:21 p.m.
    Status: Monitoring
    Update: We are observing that the overall performance is a lot better already, most of the requests are successful and with the expected latency. We'll still monitor the situation while the cluster continues escalating.
  • Time: Nov. 11, 2024, 4:50 p.m.
    Status: Monitoring
    Update: The infrastructure is currently escalating and we expect this to take a couple of hours more to complete. We'll be monitoring how it affects the performance, for now we continue to see degraded performance for some bots.
  • Time: Nov. 11, 2024, 3:39 p.m.
    Status: Identified
    Update: We are seeing occasional 503 errors from our provider, as a first measure we increased our service tier and we are trying to connect with their support team.
  • Time: Nov. 11, 2024, 3:21 p.m.
    Status: Investigating
    Update: We are currently investigating an issue with uGPT replies which seem to be failing in some scenarios.

Updates:

  • Time: Nov. 22, 2024, 11:36 a.m.
    Status: Postmortem
    Update: RCA that we got from Azure regarding to incident: > **Azure OpenAI – 408 and 503 errors while using GPT-4o model** > > **What happened?** > Between 05:08 UTC on 15 October 2024 and 15:37 UTC on 17 October 2024, a platform issue resulted in an impact to the Azure OpenAI service. Customers experienced high latency and 5XX/408 errors when making requests to GPT-4o, version 2024-08-06.**What went wrong and why?** > The system is generally designed for resiliency, ensuring it recovers after a crash and continues to serve traffic. Upon investigating this incident, we identified two issues. Firstly, large JSON schema inputs caused crashes due to specific customer usage patterns. Secondly, slow memory leaks led to low available memory, causing system component failures. This prevented the service from restarting normally after crashes, resulting in a deadlock that required a full node reboot to recover. Notably, the two issues are interrelated - the first leads to failures with crashes, and the second worsens due to crashes. However, any failure causing a crash can exacerbate the latter, not just those crashes caused by the former.**How did we respond?** > After identifying a usage pattern causing a code regression in the service infrastructure, we deployed a hotfix following our Safe Deployment Practices to resolve the issue. * 05:08 UTC on 15 October 2024 – Customer impact began, trace number of errors and latency. * 06:27 UTC on 15 October 2024 – Service monitoring threshold exceeded, and internal alerts fired for an increase of errors. * 07:29 UTC on 15 October 2024 – The team starts the investigation and attempts to use the automated solution to mitigate the problem. * 08:08 UTC on 15 October 2024 – Engineers remain engaged and determined impact to a small subset of customers. * 12:46 UTC on 15 October 2024 – We concluded that the automated solution was ineffective in assisting with mitigation efforts. * 13:30 UTC on 15 October 2024 – Additional impact detected, and initial communications sent to targeted impacted subscriptions. * 14:45 UTC on 15 October 2024 – We identified the usage pattern that triggered the bug and initiated throttling measures to mitigate its impact. * 14:45 UTC on 15 October 2024 – Availability and latency stabilizing for most customers. * 15:44 UTC on 15 October 2024 – A hotfix has been developed and validated for a permanent solution, following safe deployment practices. * 19:00 UTC on 15 October 2024 – Additional improvement in availability and latency for a subset of customers. * 20:21 UTC on 15 October 2024 – The deployment of the hotfix in the Sweden Central region has been successfully completed. * 23:28 UTC on 15 October 2024 – A temporary solution was communicated, advising customers to switch to model version 2024-05-13 as a workaround. * 00:00 UTC on 16 October 2024 – Mitigation measures were being maintained for customer workloads. The deployment of the hotfix was ongoing. * 06:03 UTC on 16 October 2024 – During the hotfix deployment across the remaining impacted regions, we initiated manual mitigation that required rebooting some nodes. * 10:20 UTC on 16 October 2024 – We have also started the automated mitigation, which has begun the reboot process for any affected nodes.  * 14:22 UTC on 16 October 2024 – All nodes have recovered from auto reboot mitigation. * 15:37 UTC on 17 October 2024 – All regions have been deployed with the hotfix. The issue has been mitigated, and customer subscriptions that were throttled have returned to normal. Service is restored, and customer impact is resolved. >   > **How are we making incidents like this less likely or less impactful?** * We are working to resolve the memory leak issue in the service. \(Estimated completion: December 2024\) * We are optimizing hotfix deployment to reduce the time from 1.5 days. \(Estimated completion: February 2025\)
  • Time: Oct. 18, 2024, 7:45 a.m.
    Status: Resolved
    Update: This incident has been resolved.
  • Time: Oct. 18, 2024, 7:44 a.m.
    Status: Identified
    Update: Azure has notified us the issue related to latency has been fixed and our internal team has confirmed that no errors related to GPT-4o has been observed the last 12 hours.
  • Time: Oct. 17, 2024, 2:38 p.m.
    Status: Identified
    Update: We are continuing to work on a fix for this issue.
  • Time: Oct. 17, 2024, 2:29 p.m.
    Status: Identified
    Update: As we are actively monitoring our services we observe some degraded performance in our models again. We will continue to follow up with Microsoft, and once we have an update, we will share it here.

Updates:

  • Time: Oct. 17, 2024, 8:28 a.m.
    Status: Resolved
    Update: We have not observed any errors, and all of our services are functioning normally.
  • Time: Oct. 16, 2024, 12:41 p.m.
    Status: Monitoring
    Update: We are continuing to monitor for any further issues.
  • Time: Oct. 16, 2024, 8:22 a.m.
    Status: Monitoring
    Update: We are actively monitoring our services as we observe improved performance in our models. We will continue to follow up with Microsoft, and once we have an update, we will share it here.
  • Time: Oct. 15, 2024, 6:13 p.m.
    Status: Monitoring
    Update: Our services have been stable for the last hours and performance has increased over time. We are still waiting for confirmation from Azure team on a permanent solution.
  • Time: Oct. 15, 2024, 4:43 p.m.
    Status: Monitoring
    Update: We are Monitoring the our services as we observe better performance on our Models.
  • Time: Oct. 15, 2024, 3:43 p.m.
    Status: Monitoring
    Update: We are Monitoring the our services as we observe better performance on our Models.
  • Time: Oct. 15, 2024, 3:01 p.m.
    Status: Monitoring
    Update: We are Monitoring the our services as we observe better performance on our Models.
  • Time: Oct. 15, 2024, 2:21 p.m.
    Status: Monitoring
    Update: We are Monitoring the our services as we observe better performance on our Models.
  • Time: Oct. 15, 2024, 1:58 p.m.
    Status: Identified
    Update: We are still waiting a permanent solution from Azure.
  • Time: Oct. 15, 2024, 1:26 p.m.
    Status: Identified
    Update: We are still waiting a permanent solution from Azure.
  • Time: Oct. 15, 2024, 12:55 p.m.
    Status: Identified
    Update: We are still waiting a permanent solution from Azure.
  • Time: Oct. 15, 2024, 12:24 p.m.
    Status: Identified
    Update: We are still waiting a permanent solution from Azure.
  • Time: Oct. 15, 2024, 11:37 a.m.
    Status: Identified
    Update: We are still waiting a permanent solution from Azure.
  • Time: Oct. 15, 2024, 10:52 a.m.
    Status: Identified
    Update: We are still waiting a permanent solution from Azure.
  • Time: Oct. 15, 2024, 10:23 a.m.
    Status: Identified
    Update: We have found a temporary solution for most of our issues while we are waiting for a more permanent solution from Azure.
  • Time: Oct. 15, 2024, 9:54 a.m.
    Status: Identified
    Update: Deploying alternative models has helped solve some of the problems but we are still continuing to see issues that we are working on.
  • Time: Oct. 15, 2024, 9:26 a.m.
    Status: Identified
    Update: The issue affects specific LLM models and we are testing deploying alternative models.
  • Time: Oct. 15, 2024, 8:58 a.m.
    Status: Identified
    Update: We have identified the root cause of the issue as Azure Platform and are continuing to investigate solutions.
  • Time: Oct. 15, 2024, 8:27 a.m.
    Status: Identified
    Update: We are continuing to investigate possible solutions to the issue.
  • Time: Oct. 15, 2024, 7:56 a.m.
    Status: Identified
    Update: We are having issues with our LLM provider.
  • Time: Oct. 15, 2024, 7:29 a.m.
    Status: Investigating
    Update: We are currently investigating the issue.

Check the status of similar companies and alternatives to Ultimate AI

UiPath
UiPath

Systems Active

Scale AI
Scale AI

Systems Active

Notion
Notion

Systems Active

Brandwatch
Brandwatch

Systems Active

Harness
Harness

Systems Active

Olive AI
Olive AI

Systems Active

Sisense
Sisense

Systems Active

HeyJobs
HeyJobs

Systems Active

Joveo
Joveo

Systems Active

Seamless AI
Seamless AI

Systems Active

hireEZ
hireEZ

Systems Active

Frequently Asked Questions - Ultimate AI

Is there a Ultimate AI outage?
The current status of Ultimate AI is: Systems Active
Where can I find the official status page of Ultimate AI?
The official status page for Ultimate AI is here
How can I get notified if Ultimate AI is down or experiencing an outage?
To get notified of any status changes to Ultimate AI, simply sign up to OutLogger's free monitoring service. OutLogger checks the official status of Ultimate AI every few minutes and will notify you of any changes. You can veiw the status of all your cloud vendors in one dashboard. Sign up here
What does Ultimate AI do?
Efficiently manage customer service with AI-powered automation platform for conversational and generative support.

Start monitoring now!

Join OutLogger to be notified when any of your vendors or the components you use experience an outage or down time. Join for free - no credit card required.

Get Started for Free