-
Time: Oct. 21, 2023, 5:17 p.m.
Status: Resolved
Update: This incident has been resolved.
-
Time: Oct. 21, 2023, 3:07 p.m.
Status: Monitoring
Update: The backlog of data has been processed and delays should have returned to normal levels. We are currently monitoring to ensure all projects have fully recovered.
-
Time: Oct. 21, 2023, 12:39 a.m.
Status: Monitoring
Update: Our Engineering team remains at work continuing to process the backlog of data. Our current estimation for processing the remaining backlog is Sunday, October 22 by end of day PT time.
We greatly appreciate your patience as we continue to work diligently on resolving this issue. We'll keep you updated here as we make progress.
-
Time: Oct. 20, 2023, 4:57 a.m.
Status: Monitoring
Update: Our Engineering team on call continues to process the backlog of data. At this time, projects with IDs below 2500000 may be intermittently seeing accurate data from Oct 10th through the 17th. There is no data loss and events will show in reports as expected when the backlog finishes processing.
We will continue to update as we get more information, thank you for your patience.
-
Time: Oct. 19, 2023, 3:42 p.m.
Status: Monitoring
Update: At this time, the Engineering team remains at work addressing the backlog with unifying data from merged IDs. Our estimation is that merging times will be back to normal by Friday the 20th by end of day PT time. For events already merged, but not showing, per the previous update, this is limited to a subset of events from Oct 10th through the 17th. Those will show in reports normally as the backlog finishes processing on Friday. Events after the 17th are not affected.
We greatly appreciate your patience as we continue to work diligently on resolving this issue. We'll keep you updated here as we make progress.
-
Time: Oct. 19, 2023, 1:37 a.m.
Status: Monitoring
Update: The Engineering team continues to process the backlog of data, and our estimate to process the backlog continues to stay on track. We are currently estimating ~40 hours to return to normal.
We discovered is that some data from October 10-17th is not appearing in reports for some projects. There is no data loss and events will re-appear in your project as expected when the incident is resolved.
Thank you for your patience while we work on resolving this.
-
Time: Oct. 18, 2023, 3:26 p.m.
Status: Monitoring
Update: We'd like to provide a brief update on the estimated resolution time for the current data processing backlog. Our current estimate is that it will take approximately 48 hours to process the backlog and return to normal.
We appreciate your patience during this time, and we're committed to getting things back on track as quickly as possible. Rest assured, we're diligently working on resolving this matter.
-
Time: Oct. 18, 2023, 12:40 p.m.
Status: Monitoring
Update: Our Engineering team on call has been working tirelessly to address the backlog of pending merge operations, and we appreciate your patience as we continue to make progress on this front. While we cannot provide an exact timeline, we estimate that it may take another 24 hours to fully clear the queue and fully resolve the issue for all projects but we will provide a more accurate resolution time in the next update. We understand that this situation has affected data reporting, which may have led to a higher drop-off in funnels for the last week and the potential for some events to be missing from your reports. We want to reassure you that no data has been lost, and we are actively working on processing all data to ensure the integrity and completeness of your reports.
Our team is committed to resolving this matter as swiftly as possible. We will provide further updates as we make progress and expect to give you a final resolution notice once the incident is fully resolved. We sincerely apologize for any inconvenience this may have caused and appreciate your understanding as we work to resolve this issue.
-
Time: Oct. 18, 2023, 12:29 a.m.
Status: Monitoring
Update: We are continuing to monitor for any further issues.
-
Time: Oct. 17, 2023, 4:33 p.m.
Status: Monitoring
Update: We are continuing to monitor for any further issues.
-
Time: Oct. 17, 2023, 7:41 a.m.
Status: Monitoring
Update: A fix has been deployed to resolve this issue. However, it is important to note that the fix is expected to take a minimum of 24 hours to complete due to the current backlog of merge operations. We are committed to reducing this time as much as possible while ensuring a thorough resolution. The Engineering team on call is actively monitoring the situation round the clock to ensure the fix progresses smoothly. We will continue to provide updates as we make progress and expect to give you a final resolution notice once the incident is fully resolved.
-
Time: Oct. 16, 2023, 10:40 p.m.
Status: Monitoring
Update: A fix has been deployed, which will take 24+ hours to resolve the delayed merges. We are monitoring the results while the fix is in progress.
-
Time: Oct. 16, 2023, 8:44 p.m.
Status: Identified
Update: The issue has been identified and a fix is being implemented.
-
Time: Oct. 16, 2023, 6:44 p.m.
Status: Investigating
Update: A subset of projects is currently experiencing a delay with merging profiles and event data since October 10th. We appreciate your patience while our engineers work to restore real-time functionality. If you have any questions, please contact support.
