Last checked: 7 minutes ago
Get notified about any outages, downtime or incidents for RebelMouse and 1800+ other cloud vendors. Monitor 10 companies, for free.
Outage and incident data over the last 30 days for RebelMouse.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage. It's completely free and takes less than 2 minutes!
Sign Up NowOutlogger tracks the status of these components for Xero:
Component | Status |
---|---|
AWS ec2-us-east-1 | Active |
AWS elb-us-east-1 | Active |
AWS RDS | Active |
AWS route53 | Active |
AWS s3-us-standard | Active |
AWS ses-us-east-1 | Active |
Braintree API | Active |
Braintree PayPal Processing | Active |
CDN | Active |
Celery | Active |
Content Delivery API | Active |
Discovery | Active |
EKS Cluster | Active |
Active | |
Fastly Amsterdam (AMS) | Active |
Fastly Hong Kong (HKG) | Active |
Fastly London (LHR) | Active |
Fastly Los Angeles (LAX) | Active |
Fastly New York (JFK) | Active |
Fastly Sydney (SYD) | Active |
Full Platform | Active |
Google Apps Analytics | Active |
Logged In Users | Active |
Media | Active |
Mongo Cluster | Active |
Pharos | Active |
RabbitMQ | Active |
Redis Cluster | Active |
Sentry Dashboard | Active |
Stats | Active |
Talaria | Active |
Active | |
WFE | Active |
View the latest incidents for RebelMouse and check for official updates:
Description: This incident has been resolved.
Status: Resolved
Impact: Minor | Started At: Oct. 11, 2023, 1:11 p.m.
Description: This incident has been resolved.
Status: Resolved
Impact: Minor | Started At: Oct. 11, 2023, 1:11 p.m.
Description: **Summary:** Today, our MongoDB cluster experienced an overload incident, resulting in system instability. We took immediate action to resolve the issue, and after a thorough investigation, we identified the root cause of the problem. It was discovered that the issue stemmed from a bug in the routing configuration feature. This bug caused an abnormal number of requests to the database, leading to its overload. **Incident Resolution:** We've acted swiftly to address the issue. Our team successfully deployed a fix, and as a result, system load returned to normal levels. **Preventive Measures:** Our QA team has been promptly informed of the incident, and they are now tasked with creating comprehensive tests to cover this scenario. This proactive approach will help us prevent similar incidents from occurring in the future.
Status: Postmortem
Impact: Major | Started At: Oct. 11, 2023, 10:44 a.m.
Description: **Summary:** Today, our MongoDB cluster experienced an overload incident, resulting in system instability. We took immediate action to resolve the issue, and after a thorough investigation, we identified the root cause of the problem. It was discovered that the issue stemmed from a bug in the routing configuration feature. This bug caused an abnormal number of requests to the database, leading to its overload. **Incident Resolution:** We've acted swiftly to address the issue. Our team successfully deployed a fix, and as a result, system load returned to normal levels. **Preventive Measures:** Our QA team has been promptly informed of the incident, and they are now tasked with creating comprehensive tests to cover this scenario. This proactive approach will help us prevent similar incidents from occurring in the future.
Status: Postmortem
Impact: Major | Started At: Oct. 11, 2023, 10:44 a.m.
Description: **Incident During Application Deployment** During a regular application deployment, our team encountered two critical issues that affected the functionality of our application. 1. During the deployment, we introduced a new field to the post model. After the deployment, we observed that some posts were not loading, necessitating an immediate update of the post cache version to accommodate the changes in the post model. To address this issue, we promptly updated the post cache version to align with the modified model, restoring the functionality of the affected posts. We are reviewing as a team to make sure that no deploy with new fields or changes to data storage are done without further understanding how to avoid incidents. 2. The time to recover was slower than it should have been because of a network connection error occurred during the deployment process, preventing the successful completion of code deployment to one of our clusters. This situation left us unable to restart the Celery processes, even when all other clusters were in a ready state. As a result, we had to restart the deployment and wait for its completion, causing an unexpected delay in the deployment process. **Immediate Actions Taken:** For the issue with the post model field addition, we acted swiftly by updating the post cache version to ensure compatibility with the modified model. **Mitigation and Preventive Measures:** Based on the incident analysis we've already integrated a new test which won't allow deployment to happen in case the post model was changed without post cache modification. "The network issue occurred randomly and, regrettably, coincided with the ongoing deployment process."
Status: Postmortem
Impact: Minor | Started At: Sept. 21, 2023, 11:08 a.m.
Join OutLogger to be notified when any of your vendors or the components you use experience an outage or down time. Join for free - no credit card required.