Time: Nov. 8, 2023, 4:18 p.m.

Status: Postmortem

Update: On October 29th 2023, between 07:10 and 07:30 UTC, a small percentage of Pantheon sites started reporting downtime per internal monitoring. Upon further investigation it was determined that a single malicious site was receiving a massive amount of requests, which in turn overloaded certain internal services. This prevented other traffic from being served. The site was immediately blocked at the edge and put into maintenance mode due to its abusive/bad actor nature. We are actively working to reduce our exposure to this sort of incident in the future.

Time: Oct. 29, 2023, 10 a.m.

Status: Resolved

Update: The issue has been successfully resolved and there should be no further disruptions. If you come across any unexpected issues, please don't hesitate to get in touch with our support team for immediate assistance. We appreciate your for your patience and continued support during this incident.

Time: Oct. 29, 2023, 9:30 a.m.

Status: Monitoring

Update: A fix has been implemented and we are actively monitoring the situation as we continue to observe a full recovery in progress.

Time: Oct. 29, 2023, 8:47 a.m.

Status: Identified

Update: The issue has been identified and a fix is being implemented.

Time: Oct. 29, 2023, 8:10 a.m.

Status: Investigating

Update: Our investigation into this issue is ongoing, and we are dedicated to resolving it at the earliest opportunity. Rest assured, we will provide updates on this page as soon as new information becomes available. Should you encounter urgent matters or require immediate assistance, please don't hesitate to reach out to us at [email protected]. We are here to address your urgent concerns.

Time: Oct. 29, 2023, 7:33 a.m.

Status: Investigating

Update: We are investigating an infrastructure failure that is affecting customer sites. The current symptoms identified are 503 errors leading to site unavailability. For urgent issues, please contact [email protected]

Time: Oct. 26, 2023, 10:47 p.m.

Status: Resolved

Update: The incident has been resolved, and our system is now stable. We apologize for any inconvenience caused and are actively working on a long-term solution to prevent similar incidents in the future. Your satisfaction is our top priority, and we appreciate your understanding and patience as we continue to improve our services.

Time: Oct. 26, 2023, 9:25 p.m.

Status: Monitoring

Update: The system remains stable. We have also identified the root cause and are working on a solution to prevent this type of failure from recurring. The issue will be addressed once the fix is implemented.

Time: Oct. 26, 2023, 8:21 p.m.

Status: Monitoring

Update: The Dashboard is currently stable, but we are actively working on a fix to prevent this issue from happening again. Should you experience any persistent issues, please don’t hesitate to reach out to our support team for assistance.

Time: Oct. 26, 2023, 7:20 p.m.

Status: Monitoring

Update: The Dashboard functions are currently stable and we are actively working to apply a fix to prevent this issue from recurring. Should you experience any persistent issues, please don't hesitate to reach out to our support team for assistance.

Time: Oct. 26, 2023, 6:06 p.m.

Status: Monitoring

Update: The Dashboard is now functioning as expected. We apologize for any disruption and will continue monitoring for any further issues. If you encounter any persisting issues, please do not hesitate to contact our support team for assistance.

Time: Oct. 26, 2023, 5:42 p.m.

Status: Investigating

Update: We've noticed a decrease in error rates and have received confirmation that the dashboard is now functioning as expected. We will continue to monitor the situation for any potential further issues. If you are still running into issues, please don’t hesitate to reach out to our support.

Time: Oct. 26, 2023, 5:22 p.m.

Status: Investigating

Update: We've observed increased error rates affecting the Dashboard, which might result in slow page loading or login failures issues. If you encounter urgent concerns, kindly reach out to our support team by emailing [email protected]

Time: Oct. 25, 2023, 9:58 p.m.

Status: Resolved

Update: The earlier disruption is now resolved, with site traffic restored to its normal state. We appreciate your patience throughout. Should you have any concerns, please initiate a chat or open a ticket via the dashboard.

Time: Oct. 25, 2023, 9:18 p.m.

Status: Monitoring

Update: We've addressed the disruptions in Pantheon's infrastructure and are now monitoring for stability. Should you have any concerns, please open a support chat or ticket and we will be happy to assist.

Time: Oct. 25, 2023, 8:39 p.m.

Status: Identified

Update: We believe we've identified the root cause of the ongoing disruptions in Pantheon's infrastructure. Our engineers have taken measures to counteract this, but we are continuing our investigation to ensure we fully understand the scope and nature of the issue.

Time: Oct. 25, 2023, 8:28 p.m.

Status: Investigating

Update: We're observing intermittent issues characterized by the messages: "Error 503 Maximum threads for service reached" and "Page Could Not Be Loaded". The behavior seems unpredictable with sites going up and down sporadically. Please be assured, our team is diligently working to identify and resolve the underlying cause. At the moment we are pursuing a platform-wide resolution and do not expect to need to work on individual sites. We appreciate your patience and understanding as we work to resolve this matter. We do not have an ETA for a fix as this is still under investigation.

Time: Oct. 25, 2023, 8:15 p.m.

Status: Investigating

Update: We’ve noticed multiple site outages and are actively working to address and prevent further occurrences. For immediate assistance, please initiate a chat or contact our support team at [email protected].

Time: Oct. 23, 2023, 7:30 p.m.

Status: Resolved

Update: This incident has been resolved.

Time: Oct. 23, 2023, 2:50 p.m.

Status: Monitoring

Update: A fix has been implemented and we are monitoring the results.

Time: Oct. 23, 2023, 2:36 p.m.

Status: Identified

Update: We have identified an issue relating to billing functions in the dashboard. Affected functions may include site plan or billing changes. We will issue further updates on this status page as we work with our providers to resolve this issue.

Time: Oct. 24, 2023, 7:43 p.m.

Status: Resolved

Update: Pantheon's engineering teams have conducted thorough audits and have taken all necessary precautions to mitigate the associated risk at the platform level.

Time: Oct. 11, 2023, 6:59 p.m.

Status: Investigating

Update: Pantheon is aware of the recent vulnerabilities reported in curl. As a precautionary measure, we are continuing to work proactively to mitigate impact on Pantheon customers and infrastructure. Details released on October 11th demonstrate that the vulnerability is only possible in a very limited set of conditions where an application is using curl with a SOCKS5 proxy with remote host resolving enabled. This is not found within any WordPress core or Drupal core code. It could hypothetically be introduced by third party or custom code, though our assessment is that this is an unlikely scenario. We are committed to keeping our customers informed and will provide updates from our investigation on our status page. As we learn more, we will promptly communicate any actions that need to be taken to address the vulnerability.