Time: Jan. 26, 2022, 10:51 a.m.

Status: Resolved

Update: Our team has resolved the incident and services have returned to their normal operation. <b>Start time:</b> 09:43 UTC <b>End time:</b> 10:22 UTC <b>What was impacted?</b> • Incident Sharing - Users may have experienced latencies when sharing BigPanda Incidents to external systems <b>What was not impacted?</b> All other BigPanda functionality, including data ingest and processing, were not impacted, and Incident updates to users' consoles continued to arrive in a timely manner. We apologize for any inconvenience that this may have caused. Should you continue to experience any issues, please reach out to us via our in-app chat or at [email protected].

Time: Jan. 26, 2022, 10:27 a.m.

Status: Monitoring

Update: Our team has implemented a fix for the issue, and impact to Incident Sharing has been mitigated. <b>What's impacted?</b> • Incident Sharing <b>What’s not impacted?</b> All other BigPanda functionality, including data ingest and processing, are impacted at this time. Incident updates to users' consoles should be arriving in a timely manner. We will continue to monitor for any change in operational capacity.

Time: Jan. 26, 2022, 10:19 a.m.

Status: Investigating

Update: Some users may be experiencing issues with Incident Sharing from BigPanda. Our teams are engaged and investigating. <b>What's impacted?</b> • Incident Sharing <b>What’s not impacted?</b> All other BigPanda functionality, including data ingest and processing, are impacted at this time. Incident updates to users' consoles should be arriving in a timely manner. We are working to mitigate impacts to our customers as quickly as possible. Additional updates will be shared every 30 minutes or less as more details become available. We apologize for any inconvenience that this may be causing.

Time: Jan. 9, 2022, 7:11 p.m.

Status: Resolved

Update: Our team has resolved the incident and services have returned to their normal operation. <b>Start time:</b> 4:43 PM UTC <b>End time:</b> 6:36 PM UTC <b>What was impacted?</b> • APIs • BigPanda Console • Pipeline We apologize for any inconvenience that this may have caused. Should you continue to experience any issues, please reach out to us via our in-app chat or at [email protected].

Time: Jan. 9, 2022, 6:47 p.m.

Status: Monitoring

Update: Our team has resolved the issue with our Enrichment Service. We are monitoring the Pipeline and will provide another update within the next 20 minutes.

Time: Jan. 9, 2022, 6:34 p.m.

Status: Identified

Update: We are currently experiencing an issue with our Enrichment v1 Service. Some customers may be experiencing Pipeline Latencies. Our Teams are investigating the issue, please expect further updates from us shortly. <b>What's impacted?</b> • Enrichment v1 Service • Pipeline

Time: Jan. 9, 2022, 5:36 p.m.

Status: Monitoring

Update: Our team has implemented a fix for the issue. We will continue to monitor for any change in operational capacity. Please expect another update within the next 30 minutes. <b>What's impacted?</b> • BigPanda Console • APIs

Time: Jan. 9, 2022, 5:12 p.m.

Status: Identified

Update: Our team has identified the root cause and are working on a resolution. <b>What's impacted?</b> • BigPanda Console We will provide another update shortly

Time: Jan. 9, 2022, 4:43 p.m.

Status: Investigating

Update: Some users may be experiencing issues with our API. Our teams are engaged and investigating. What's impacted? BigPanda Console We are working to mitigate impacts to our customers as quickly as possible. Additional updates will be shared every 30 minutes or less as more details become available. We apologize for any inconvenience that this may be causing.

Time: Dec. 19, 2021, 4:38 p.m.

Status: Resolved

Update: The issues with the Inbound Alerts API have been mitigated. Users may have seen an increased rate of 404 response code from several out of the box integrations. This was due to a change within the recent Maintenance Window. The team has subsequently rolled back the change, returning BigPanda back to service. Should you continue to experience any issues, please reach out to us via our in-app chat or at [email protected].

Time: Dec. 19, 2021, 3:45 p.m.

Status: Monitoring

Update: The issues with the Inbound Alerts API have been mitigated. Users may have seen an increased rate of 404 response code from several out of the box integrations. This was due to a change within the recent Maintenance Window. The team has subsequently rolled back the change, returning BigPanda back to service. We continue to monitor the situation for any change in status. Should you continue to experience any issues, please reach out to us via our in-app chat or at [email protected].

Time: Dec. 19, 2021, 3:39 p.m.

Status: Investigating

Update: We are currently experiencing issues with the Inbound BigPanda API and are investigating the matter to identify the cause of the problem. Additional updates will be shared as more details become available. We apologize for any inconvenience that this may be causing.

Time: Dec. 15, 2021, 4:53 p.m.

Status: Resolved

Update: Our service provider has fixed the root cause of today's Incident, and all impacted services in the US-WEST-1 and US-WEST-2 regions have returned to normal operating levels. The following BigPanda functionality should now be operating within normal operational levels as well: • Incident Sharing • BigPanda Console We apologize for any inconvenience that this may have caused. Should you continue to experience any issues, please reach out to us via our in-app chat or at [email protected].

Time: Dec. 15, 2021, 4:24 p.m.

Status: Monitoring

Update: Our service provider has reported a return to service in both impacted regions of US-WEST-1 and US-WEST-2, and we are seeing signs of recovery as well. We will continue to monitor systems for any change in operational capacity. Should you continue to experience any issues, please reach out to us via our in-app chat or at [email protected].

Time: Dec. 15, 2021, 4:22 p.m.

Status: Monitoring

Update: Our service provider has reported a return to service in the primary impacted region US-WEST-1, and we are seeing signs of recovery as well. We will continue to monitor systems for any change in operational capacity. Should you continue to experience any issues, please reach out to us via our in-app chat or at [email protected].

Time: Dec. 15, 2021, 4:17 p.m.

Status: Identified

Update: Our service provider is reporting reports of connectivity issues in the US-WEST-1 and US-WEST-2 regions. Users may see intermittent connectivity issues to the BigPanda console and delays or failures with incident sharing. We continue to explore possible mitigation options.

Time: Dec. 15, 2021, 3:41 p.m.

Status: Investigating

Update: We are currently investigating possible issues with the BigPanda console. This status will be updated as more information is known.

Time: Dec. 12, 2021, 6:37 p.m.

Status: Resolved

Update: <big><b>UPDATE:</b></big> On January 20, 2022, BigPanda completed the upgrade to Apache Log4j version 2.17.1 to address the Medium Risk finding detailed in CVE-2021-44832 on December 28, 2021. The upgrade was completed in all BigPanda environments, and no customer data was impacted by this vulnerability. We will continue to monitor Log4j security announcements and respond if additional vulnerabilities are identified. <b>BigPanda’s Log4j Vulnerability Remediation History:</b> <i>• (CVE-2021-44228) Mitigate with Config Change completed 12-Dec-2021 • (CVE-2021-44228) Upgrade to v2.15 completed 14-Dec-2021 • (CVE-2021-45046) Upgrade to v2.16 completed 16-Dec-2021 • (CVE-2021-45105) Upgrade to v2.17 completed 21-Dec-2021 • (CVE-2021-44832) Upgrade to v2.17.1 completed 23-Jan-2022</i> <hr/>On December 20, 2021, BigPanda assessed the risk associated Apache Log4j vulnerability CVE-2021-45105 and determined the upgrade to version 2.17.0 could be completed safely in all BigPanda environments. BigPanda successfully completed the upgrade to Apache Log4j version 2.17.0 on December 21, 2021, to mitigate all known vulnerabilities in Log4j v2. We will continue to monitor for additional Log4j vulnerabilities and patches. <hr/>On December 12, 2021, BigPanda mitigated the initial risk from CVE-2021-44228 on by configuring the Log4j2 properties to prohibit the capability that the vulnerability depends upon. On December 14, 2021, BigPanda performed upgrades to v2.15. When v2.15 was found to be incomplete, BigPanda remediated CVE-2021-45046 on December 16, 2021, with final upgrades to v2.16. No customer data was impacted. <hr/>On Friday, December 10, 2021, the world became aware of a remote code execution vulnerability affecting Apache Log4j2. The zero-day, critical-severity exploit is also known as the “Log4Shell” vulnerability, <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228" target="_blank">CVE-2021-44228</a>. It has an assigned CVSS severity score of 10. Apache Log4j2 is a Java-based logging framework widely used in commercial and open-source software products. The Log4Shell vulnerability exploit can be used by attackers to execute code on backend servers that log unescaped user input, putting these machines at risk by potentially installing malware, accessing customer data, and worse. As soon as BigPanda learned of this vulnerability, the R&D, Professional Services, Integrations, Security and Engineering teams promptly evaluated the assessed threat and risk to BigPanda and our customers, covering four pillars of our service: • BigPanda web app, core APIs, and infrastructure • Out-of-the-box (OOTB) Integrations • Custom Solutions & Integrations • Sub-processors The team analyzed code to identify any components affected, the risk exposure, whether the affected service was public facing or internal, the owner, and the plan for remediation. BigPanda’s web app & core APIs had three domain areas affected. Two of the areas were internal facing only and could not be accessed directly from the public internet. The third area identified is only called by the front-end web app and requires a customer access token or the API call fails. All vulnerabilities were successfully mitigated on December 12, 2021, by updating all mapped services and platform components, and configuring the log4j2 properties to prohibit the capability that the vulnerability depends upon. OOTB Integrations and Custom Solutions & Integrations were analyzed for the affected Log4j2 library for BigPanda Hosted and BigPanda OOTB on-premise Agent and SNMP Daemon. It has been determined we are not dependent on the Apache Log4j2 for logging in those solutions. Analysis identified Log4j2 as one plug-in used by an OOTB integration, but it does not use a CVE affected version. In short, the vulnerability does not affect BigPanda’s OOTB Integrations and Custom Solutions & Integrations. BigPanda’s security team is reaching out to all sub-processors to identify which sub-processors have been affected by the vulnerability, if there is any risk to BigPanda data, and confirmation the vulnerability has been remediated. With our thorough analysis and response, we are confident that BigPanda's core platform and integrations are fully patched against the Log4Shell vulnerability (<a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228" target="_blank">CVE-2021-44228</a>). Customers do not need to take any action.