Is there an UserVoice outage?
UserVoice status: Systems Active
Last checked: 14 seconds ago
Get notified about any outages, downtime or incidents for UserVoice and 1800+ other cloud vendors. Monitor 10 companies, for free.
Last checked: 14 seconds ago
Get notified about any outages, downtime or incidents for UserVoice and 1800+ other cloud vendors. Monitor 10 companies, for free.
Outage and incident data over the last 30 days for UserVoice.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage. It's completely free and takes less than 2 minutes!
Sign Up NowOutlogger tracks the status of these components for Xero:
| Component | Status |
|---|---|
| Admin Console | Active |
| Email (delivery) | Active |
| Email (incoming) | Active |
| Helpdesk API | Active |
| Search | Active |
| User Analytics | Active |
| UserVoice API | Active |
| Web Portal (subdomain) | Active |
| Widgets | Active |
View the latest incidents for UserVoice and check for official updates:
Description: On November 13th between 13:38 and 14:14 PT, UserVoice experienced a networking infrastructure issue that caused a sitewide outage and system unavailability. **Business Impact** * During the outage end users and admins would have been unable to load or interact with UserVoice sites, widgets or the API. * Email would have been delayed, but no emails were lost. **Root Cause** In the process of cleaning up unused resources in the UserVoice infrastructure an old kubernetes cluster was removed from production. The automated cleanup of this cluster unintentionally removed a networking firewall rule that allowed our active application cluster to communicate with our backend infrastructure. Initial debugging was incorrectly focused around in-cluster symptoms and we did not immediately determine proper cause of the issue. Manual restoration of a proper firewall rule allowed the service to be fully restored. **What we are Doing to Prevent This** * Proper failover firewall rules are now being controlled via our infrastructure-as-code system preventing automated cleanup of old rules. * Infrastructure cleanup tasks will be scheduled during maintenance windows going forward. We didn’t meet our own or your expectations for using UserVoice with this outage. We do apologize for the pain points this caused for you and your team. If you have any questions or concerns, please reach out and let me know. Claire Talbott Support Manager [email protected]
Status: Postmortem
Impact: Critical | Started At: Nov. 13, 2018, 9:44 p.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Major | Started At: Nov. 8, 2018, 3:51 a.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Major | Started At: Nov. 8, 2018, 3:51 a.m.
Description: On Friday 11/2/18 from 5AM to 5:13AM PDT, UserVoice experienced downtime. ## Business Impact During the time of the incidents end users and admins would have seen 500 errors. They wouldn’t have been able to load the admin console, use the API, interact with ideas on the front end or use the widget or Contributor Sidebar. Email would have been delayed, but no emails were lost. ## Root Cause We saw an issue similar to last Friday’s incident where one of the servers in Uservoice's database cluster experienced an application stall event. This caused a pause in database writes. Our engineering team manually removed the affected node to allow the cluster to resume operation. ## What we are Doing to Prevent This Our team has been focused, since last week, on finding the root issue that is caused one of our database clusters to stall. This work is still in progress. Once the root issue is identified, we will be implementing a fix and updating this report with the information discovered. In the meantime, we have put increased alerting in place so that should the issue repeat, we will identify it immediately. We understand UserVoice being down is an interruption to you and your team, and impacts your workflows! We take this downtime seriously, and are all hands on deck to get this issue fully addressed, so we can prevent it happening again. If you have any questions or feedback for us about the incident please don’t hesitate to contact me at [[email protected]](mailto:[email protected]). Claire Talbott Support Manager
Status: Postmortem
Impact: Major | Started At: Nov. 2, 2018, 1:03 p.m.
Description: On October 26th between 10:20 and 10:35 PDT UserVoice experienced an infrastructure issue that caused intermittent system unavailability. ## Business Impact During the outage end-users and admins may have been unable to load or interact with UserVoice sites or widgets. ## Root Cause One of the servers in Uservoice's database cluster experienced an application stall event. Due to misconfiguration of our cluster, this caused a pause in database writes. Our engineers were required to manually remove the affected node to allow the cluster to resume writing. ## What we are Doing to Prevent This We have updated our database cluster configuration to more aggressively monitor and remove cluster members that are non-performant. This caused an interruption for your team in UserVoice and your end users trying to view and submit feedback. We are sorry for the pain point this caused. We have already put improvements in place to prevent this type of issue from happening again. If you have any questions, don’t hesitate to reach out at [[email protected]](mailto:[email protected]). Claire Talbott Support Manager
Status: Postmortem
Impact: Major | Started At: Oct. 26, 2018, 5:30 p.m.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage or down time. Join for free - no credit card required.
