Time: June 6, 2023, 7:20 p.m.

Status: Postmortem

Update: ### Floify Service Interruption Jun 5, 2023 **\(All times Mountain Time\)** **Issue Summary** Between July 5, 11:12 AM and July 6, 10:31 AM, certain users encountered intermittent issues with receiving emails. The root cause was related to an unplanned update by SendGrid, our third party email delivery provider. SendGrid stopped support for certain versions of TLS \(Transport Layer Security\) without informing their customers.‌ **Timeline Jun 5, 2023** * 11:38 AM Certain emails encountered sending issues. * 5:57 PM Floify Engineering team were alerted of email issues * 6:30 PM Floify  Engineering Ops team reached out to SendGrid support to triage the issue. A P2 severity ticket was created with SendGrid. * 10:15 PM We were able to determine the root cause of the failures. **Jun 6, 2023** * 6:00 AM Floify Engineering team identified a solution * 8:30 AM - 9:30 AM team expedited development and testing for the fix * 10:00 AM - 10:30 AM The fix was deployed and verified **Root Cause** SendGrid, our email delivery provider stopped supporting TLS v1.0 and v1.1 \(Transport Layer Security\) without informing their customers. Certain emails sent from Floify through SendGrid were erroring out. **Resolution and recovery** At 5:57 PM on June 5,  Floify Engineering was alerted by Floify Support about intermittent issues with emails By 10:15 PM on June 5, Floify Engineering had identified the root cause of the problem. Between 6:00 AM and 10.30 AM on June 6, A fix was developed, tested and deployed to Production **Corrective and Preventative Measures** The following are actions we are taking to address the underlying causes of the issue and to help prevent recurrence and improve response times: * Implement an update to fix the email deliverability issues. \(Completed\) * Work with the SendGrid Support Team to ensure that Floify is notified before they make any breaking changes. \(In progress\) Floify is committed to continually adapting and improving our technology and process to prevent service interruptions. We appreciate your patience and apologize for the impact of this issue. Sincerely, The Floify Team

Time: June 6, 2023, 5:37 p.m.

Status: Resolved

Update: Emails are being delivered properly now. This issue has been resolved.

Time: June 6, 2023, 4:44 p.m.

Status: Monitoring

Update: A fix has been implemented and we are monitoring the results.

Time: June 6, 2023, 4:18 p.m.

Status: Identified

Update: We will be performing an application refresh shortly. This refresh does not require application downtime, however, it could cause an unexpected latency while using the application. This page will be updated once the refresh is completed.

Time: June 6, 2023, 3:48 p.m.

Status: Identified

Update: We have identified the root cause of this issue : a change introduced by our email integration partner (SendGrid) yesterday. A fix to the issue is being tested right now and we are looking to deploy it to Production as soon as possible.

Time: June 6, 2023, 12:58 p.m.

Status: Identified

Update: We've identified certain users have encountered intermittent issues with receiving emails. Emails may not be sent successfully in certain instances. While we work to resolve this issue, when selecting to receive the MFA code, the user should select to receive the code via SMS. We've identified the root cause and are working to deploy a fix as soon as possible.

Time: June 1, 2023, 6:35 p.m.

Status: Postmortem

Update: ### Floify Service Interruption June 1, 2023 This incident report details the Floify partial outage that occurred on June 1, 2023.  **Issue Summary** **All times in this report are MST** From 4:30 am to 8:35 am, Floify experienced a partial system outage which prevented some users from completing the setup process for Multi-Factor Authentication \(MFA\). This outage impacted certain customers who had MFA enabled in their Company settings. Users were able to re-start MFA setup and complete it. The root cause of this outage was found to be certain MFA backup codes were not saving to the database. **Timeline \(all times in Mountain Time\)** * 4:30 AM - Database errors began to populate for a small percentage of users with MFA enabled. * 8:08 AM - Floify Engineers began to review the logs. * 8:19 AM - Floify Engineering triaged the issue and found the specific errors in the logs for when users were trying to save the backup codes for MFA. This resulted in a small percentage of users being unable to complete the MFA setup process. * 8:25 AM - Floify Engineering determined an application restart was required in order to clear the block. * 8:35 AM - The refresh was completed. * 8:50 AM - Floify Engineering team reviewed the logs and determined no further errors to be found. The block was cleared. **Root Cause** It was determined the MFA backup codes were failing when certain users tried to complete the MFA process due to database timeout errors requiring performance optimization adjustments. **Resolution and Recovery** A rolling restart of the Floify app was performed. Unaffected users had little to no interruption using Floify. **Corrective and Preventative Measures** The following are actions we are taking to address the underlying causes of the issue and to help prevent recurrence and improve response times: * Increased monitoring and alerts at the database layer * Database optimization changes \(under development\) Floify is committed to continually adapting and improving our technology and process to prevent service interruptions. We appreciate your patience and again apologize for the impact of this outage. We thank you for your business and continued support. ‌ Sincerely, The Floify Team

Time: June 1, 2023, 6:34 p.m.

Status: Resolved

Update: This incident has been resolved.

Time: June 1, 2023, 3 p.m.

Status: Monitoring

Update: The restart has been completed and the application is now accessible. We are actively monitoring the situation and will provide a further update once the issue has been fully resolved.

Time: June 1, 2023, 2:35 p.m.

Status: Investigating

Update: We've identified system logging errors and will need to restart the application. We are investigating this issue and will post an update when available. During this time, the application will be unavailable.

Time: May 30, 2023, 9:36 p.m.

Status: Resolved

Update: This incident has been resolved.

Time: May 30, 2023, 6:16 p.m.

Status: Identified

Update: Font Awesome, a service hosting certain icons in Floify, is experiencing an outage. As a result, some icons in Floify are taking longer than expected to load. Font Awesome is working on resolving this issue and we will update our status page once it has been resolved.

Time: May 26, 2023, 7:02 p.m.

Status: Resolved

Update: This incident has been resolved.

Time: May 26, 2023, 6:47 p.m.

Status: Monitoring

Update: We will be performing a unscheduled maintenance that does not require the application to go down, however, it may cause latency while using the application. This page will be updated once the maintenance is completed.

Time: May 26, 2023, 6:46 p.m.

Status: Monitoring

Update: We will be performing a unscheduled maintenance that does not require the application to go down, however, it may cause latency while using the application. This page will be updated once the maintenance is completed.

Time: April 18, 2023, 4:12 p.m.

Status: Resolved

Update: This incident has been resolved.

Time: April 13, 2023, 4:08 p.m.

Status: Monitoring

Update: We have been monitoring this issue and no known incidents have occurred since 2:43pm ET on Monday, April 10th. We will be releasing a bug fix to resolve the issue on Friday, April 14th at 3:00am ET. This incident will be updated to Resolved once we have confirmed it is working as expected.

Time: April 10, 2023, 6:43 p.m.

Status: Monitoring

Update: We've identified the issue and will be monitoring disclosures until we can implement a code change to fully resolve the issue after business hours today. We will be doing an app refresh after 7pm MT today. For impacted disclosures, our team is manually processing these and will reach out directly if any action is needed from you. This status will be updated to Resolved once the code change is implemented tonight.

Time: April 10, 2023, 6:39 p.m.

Status: Investigating

Update: We've identified the issue and will be monitoring disclosures until we can implement a code change to fully resolve the issue after business hours today. We will be doing an app refresh after 7pm MT today. For impacted disclosures, our team is manually processing these and will reach out directly if any action is needed from you. This status will be updated to Resolved once the code change is implemented tonight.

Time: April 10, 2023, 6:24 p.m.

Status: Investigating

Update: We are currently investigating an issue with all disclosures showing as expired. We will provide an update once the issue has been identified and a resolution plan has been established.