Is there an iAdvize (HA) outage?
iAdvize (HA) status: Systems Active
Last checked: 7 minutes ago
Get notified about any outages, downtime or incidents for iAdvize (HA) and 1800+ other cloud vendors. Monitor 10 companies, for free.
Last checked: 7 minutes ago
Get notified about any outages, downtime or incidents for iAdvize (HA) and 1800+ other cloud vendors. Monitor 10 companies, for free.
Outage and incident data over the last 30 days for iAdvize (HA).
Join OutLogger to be notified when any of your vendors or the components you use experience an outage. It's completely free and takes less than 2 minutes!
Sign Up NowOutlogger tracks the status of these components for Xero:
| Component | Status |
|---|---|
| Mobile App | Active |
| Administration | Active |
| Channel settings | Active |
| Distribution settings | Active |
| Engagement settings | Active |
| Login | Active |
| Sending email | Active |
| Statistics | Active |
| Users management | Active |
| APIs | Active |
| GraphQL API | Active |
| Rest API | Active |
| Webhook | Active |
| Bot | Active |
| Bot service (except IA features) | Active |
| Copilot Agent | Active |
| Copilot Shopper | Active |
| Conversation Panel | Active |
| Canned answer | Active |
| Conversation closure | Active |
| Conversation transfert | Active |
| Conversation views | Active |
| Login | Active |
| Message exchange | Active |
| Mirroring / Cobrowsing | Active |
| Onsite Channels | Active |
| Call | Active |
| Chat | Active |
| Mobile SDK | Active |
| Video | Active |
| Third Party Channels | Active |
| Active | |
| Facebook Messenger | Active |
| SMS | Active |
| Active | |
| X | Active |
| Visitor’s interface | Active |
| Engagement Notification | Active |
| iAdvize Messenger | Active |
View the latest incidents for iAdvize (HA) and check for official updates:
Description: This incident has been resolved.
Status: Resolved
Impact: Minor | Started At: Oct. 3, 2023, 5:32 p.m.
Description: **1 - What happened?** Over the past few days, we've encountered several instabilities in the processing of conversations by iAdvize bots. Instead of unfolding the expected scenario, the bots failed to process visitors' messages. As a result, the bots were displaying error messages or no response at all, resulting in a severe degradation of the user experience on your websites. These instabilities occurred : * August 11th : 9:57 > 11:37 CEST * August 15th : 4:45 > 7:54 CEST **2 - What caused the outage?** We are currently working on a major revamp of the technical core of iAdvize bots. The aim of this redesign is to improve the speed of execution of bots' scenarios, and make them easier to maintain especially during future technical updates. This work includes the implementation of a new service dedicated to the reception of new conversations by bots. These conversations are then distributed to a second service, which executes the scenario defined in iAdvize administration.Instabilities occurred on this new service due bot messages parsing. Unknown format messages were pushed to the system.This resulted in a delay in the processing of new bot conversations.We have identified that on 2 occasions. * On August 11th, the delay was small, and only a third of the bots managed by iAdvize were affected. * On August 15th, despite the patches applied after previous instability, we experienced a new problem. The accumulated delay was enough to interrupt all bots conversation processing. **3 - What was the fix?** * On August 11th, we mitigated the problem by manually identifying and restarting the instances of the bot conversation reception service that were causing problems. * On August 15th, our new probes detected a new delay. We also restarted faulty instances. However, these actions didn't work as expected. A new problem with bots message parsing was discovered. We had to urgently develop a hotfix to unblock a message type that was not recognized by the system. **4 - How will iAdvize prevent this issue in the future?** * \(In progress\) \(Tech\) Technical audit of the new bot service in order to identify any new potential failure points * \(Done\) \(Tech\) Improve our probes to detect potential delays in the processing of new conversations by bots * \(Done\) \(Tech\) Improve bot service reliability by adding safeguards to prevent bots from blocking the processing of new conversations in the case of unsupported message types
Status: Postmortem
Impact: Major | Started At: Aug. 15, 2023, 5:16 a.m.
Description: ## **Incident** We have encountered instabilities in the loading of the discussion panel and mobile app for agents. Instead of loading the expected interface on desktop, a blank screen was displayed to the agents.Regarding mobile app, an error was displayed as soon as agent IDs were entered.These regressions prevented agents from processing the incoming flow of conversations. These instabilities occurred : - August 11th : 9:35 > 10:42 CEST ## **Reasons** We are currently working on some major innovations that will be live in a few weeks' time.The first pieces of this in-depth work are starting to be delivered in production. Although inactive for our end users, they impact the source code already in production.The incident came from one of these supposedly transparent deliveries. It introduced a new environment variable required to start the discussion panel and mobile app. On our build and test environments, no problems were detected as this variable is instantiated automatically on the discussion panel and mobile app login.On production servers, due to the fact that all new pieces are not online yet, the variable has not been instantiated as expected. It generated an internal error. ## **Resolution** We solved the problem by taking several successive actions : - Our probes and automated non-regression tests detected the incident within minutes of deployment to production. We therefore proceeded to a rollback of the release. This action did not restore the solution, as a database schema update accompanied the release.- We had to urgently develop a hotfix to bypass the database schema update. ## **Actions for the future** - \(Done\) \(Tech\) Add new Unit tests on build and test environments in order to check newly added environment variable integrity - \(Done\) \(Tech\) Remove dependencies between in progress developments but not visible yet and initial discussion panel/mobile apps loading
Status: Postmortem
Impact: Major | Started At: Aug. 11, 2023, 8:04 a.m.
Description: This morning between 10am CEST and 11:30 CEST, we're seeing errors when bots are taking over conversations. In some cases, visitors could get an unavailability message after initiating a conversation with a bot. The problem was not systematic (1/3 of bot conversations affected) and was identified and resolved by our teams. As of 11:30 CEST, the service is now back to normal.
Status: Resolved
Impact: None | Started At: Aug. 11, 2023, 8 a.m.
Description: After several days of monitoring the platform, we've noticed no more errors. An incident is still open on Meta's status, but the deployed hotfix is working. We therefore decide to close this incident.
Status: Resolved
Impact: Minor | Started At: Aug. 4, 2023, 12:31 p.m.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage or down time. Join for free - no credit card required.
