Is there an ServiceChannel outage?
ServiceChannel status: Systems Active
Last checked: 8 minutes ago
Get notified about any outages, downtime or incidents for ServiceChannel and 1800+ other cloud vendors. Monitor 10 companies, for free.
Last checked: 8 minutes ago
Get notified about any outages, downtime or incidents for ServiceChannel and 1800+ other cloud vendors. Monitor 10 companies, for free.
Outage and incident data over the last 30 days for ServiceChannel.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage. It's completely free and takes less than 2 minutes!
Sign Up NowOutlogger tracks the status of these components for Xero:
| Component | Status |
|---|---|
| WorkForce | Active |
| Analytics | Active |
| Analytics Dashboard | Active |
| Analytics Download | Active |
| Data Direct | Active |
| API | Active |
| API Response | Active |
| Authentication | Active |
| Budget Insights | Active |
| SendXML | Active |
| SFTP | Active |
| Universal Connector | Active |
| Mobile Applications | Active |
| SC Mobile | Active |
| SC Provider | Active |
| Provider Automation | Active |
| Fixxbook | Active |
| Invoice Manager | Active |
| IVR | Active |
| Login | Active |
| Proposal Manager | Active |
| Work Order Manager | Active |
| Service Automation | Active |
| Asset Manager | Active |
| Compliance Manager | Active |
| Dashboard | Active |
| Inventory Manager | Active |
| Invoice Manager | Active |
| Locations List | Active |
| Login | Active |
| Maps | Active |
| Project Tracker | Active |
| Proposal Manager | Active |
| Supply Manager | Active |
| Weather | Active |
| Work Order Manager | Active |
| Service Center | Active |
| Email - servicechannel.com | Active |
| Email - servicechannel.net | Active |
| Phone - Inbound | Active |
| Phone - Outbound | Active |
| Third Party Components | Active |
| Avalara Tax Calculation Service | Active |
| Rackspace - Inbound Email | Active |
| Twilio REST API | Active |
| Zendesk | Active |
View the latest incidents for ServiceChannel and check for official updates:
Description: Security Notification On December 10th a repeat incident of malicious code was discovered in ServiceChannel’s informational website https://servicechannel.com. After investigating this issue, it was determined that the code was left dormant. ServiceChannel’s cybersecurity team immediately responded to this threat and believes the dormant code was eliminated before it could distribute malware to visitors to the website. The ServiceChannel website is completely segregated from the ServiceChannel software platform and we believe there was no impact to any customer or service provider data or to ServiceChannel systems. There is no action required of you at this point. We are just sharing this in the spirit of full transparency. ServiceChannel’s cybersecurity team constantly monitors all our systems to ensure the security of our customers, service providers and their data. Best, ServiceChannel’s cybersecurity team
Status: Resolved
Impact: None | Started At: Dec. 10, 2022, 11 p.m.
Description: Security Notification On December 6th malicious code was discovered in ServiceChannel’s informational website https://servicechannel.com. After investigating this issue, it was determined that the code was left dormant. ServiceChannel’s cybersecurity team immediately responded to this threat and believes the dormant code was eliminated before it could distribute malware to visitors to the website. The ServiceChannel website is completely segregated from the ServiceChannel software platform and we believe there was no impact to any customer or service provider data or to ServiceChannel systems. There is no action required of you at this point. We are just sharing this in the spirit of full transparency. ServiceChannel’s cybersecurity team constantly monitors all our systems to ensure the security of our customers, service providers and their data. Best, ServiceChannel’s cybersecurity team
Status: Resolved
Impact: None | Started At: Dec. 6, 2022, 10:35 p.m.
Description: **Emergency Release Resulting in Service Outage - Incident Report** **Date of Incident:** 11/18/2022 **Time/Date Incident Started:** 11/18/2022, 12:38 pm EDT **Time/Date Stability Restored:** 11/18/2022, 2:58 pm EDT **Time/Date Incident Resolved:** 11/18/2022, 3:36 pm EDT **Users Impacted:** Some users **Frequency:** Intermittent **Impact:** Major **Incident description:** The DevOps CICD team executed an emergency production code deployment for the Subscriber website code to remediate an issue reported by one client. The system exhibited periods of degraded performance for 3 hours during the emergency release. **Root Cause Analysis:** After the emergency code release, SRE team determined there were unhealthy nodes present against one of the ServiceClick web application pools. The SRE team responded by increasing node/instance capacity and restarted existing nodes with a degraded status. DevOps team later identified there was a miscommunication between the Release team and the SRE team during release hand-off wasn't properly stated. This resulted in additional delay to remediate the remaining unhealthy nodes. **Actions Taken:** 1. Release team performed an emergency code release 2. SRE team restarted some degraded app service nodes 3. A Zoom bridge was started with the release team and the SRE team 4. SRE team manually scaled up capacity on one of the ServiceClick web application pools **Mitigation Measures:** 1. Review the criteria for approving an emergency code release. 2. Update the emergency release process documentation, to including the following points: 1. If an emergency release is happening in the service time range \(USA: 9 AM – 2 PM EDT, EU: 9 AM – 2 PM GMT\+1\) then the release team and the SRE team will start a bridge \(Zoom or Microsoft Teams call\) immediately when the release begins. 2. Review the deployment strategy for high-traffic web applications and create new methods to ensure zero downtime deployments at any time of day.
Status: Postmortem
Impact: Major | Started At: Nov. 18, 2022, 7:54 p.m.
Description: Thanks for your patience. We consider this issue to be resolved.
Status: Resolved
Impact: Critical | Started At: Nov. 17, 2022, 1:44 p.m.
Description: **Primary Database Memory Dump Crash** - **Incident Report** **Date of Incident:** 10/20/2022 **Time/Date Incident Started:** 10/20/2022, 10:59am EDT **Time/Date Stability Restored:** 10/20/2022, 11:06am EDT **Time/Date Incident Resolved:** 10/20/2022, 12:08pm EDT **Users Impacted:** All **Frequency:** Continuous **Impact:** Critical **Incident description:** An issue with the underlying hardware used by the Virtual Machine \(VM\) hosting the primary database replica caused a transient memory error in the database service. The process that writes out the affected memory space to a diagnostic file caused the system to become unresponsive and the primary database to be briefly unavailable, causing systemwide downtime of the ServiceChannel platform. **Root Cause Analysis:** A general hardware issue on one of our infrastructure partner’s hypervisors resulted in a transient memory error in the VM running our primary database service. Upon detecting a malfunctioning process, the failure triggered the self-healing process. As a part of the recovery process, the impacted service generated a memory dump of crucial information about the state of the service and content of impacted memory and log files. Normally this process is seamless, however in this event it resulted in a temporary freeze of the database service, thereby causing systemwide downtime. The database returned to normal operation as soon as the memory dump completed, returning the platform to service. **Actions Taken:** 1. The Database Administration \(DBA\) team investigated alerts triggered by an unresponsive SQL Server service on the primary replica. 2. The DBA team verified that the database service had recovered from a system freeze which occurred while a diagnostic memory dump took place. 3. The DBA team confirmed that full functionality returned to database services and the synchronization process between the impacted primary and the secondary database replicas. 4. The Site Reliability Engineering \(SRE\) team confirmed that the ServiceChannel platform returned to normal operation. **Mitigation Measures:** 1. Work with our cloud infrastructure partner to develop additional details pertaining to the nature of this failure to implement a robust strategy to survive unavoidable system instability such as this. 2. Enhance the current high availability remediation and disaster recovery mechanism within SQL Server for additional operational resiliency.
Status: Postmortem
Impact: Major | Started At: Oct. 20, 2022, 3:06 p.m.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage or down time. Join for free - no credit card required.
