Time: Sept. 6, 2024, 1:15 a.m.

Status: Resolved

Update: This incident has been resolved.

Time: Sept. 5, 2024, 7:44 p.m.

Status: Monitoring

Update: A fix has been implemented and we are monitoring the results.

Time: Sept. 5, 2024, 6:56 p.m.

Status: Identified

Update: We are seeing instances of Syncs failing with a message indicating "Parquet conversion failed." The problematic system has been identified and we are working to mitigate and resolve the issue as soon as possible.

Time: Aug. 23, 2024, 7:15 p.m.

Status: Resolved

Update: This incident has been resolved.

Time: Aug. 23, 2024, 7:04 p.m.

Status: Monitoring

Update: Sync success rates have returned to expected levels. There is some delay in sync execution as we work through the backlog during the outage period. We are continuing to monitor

Time: Aug. 23, 2024, 6:48 p.m.

Status: Identified

Update: We have identified the likely root cause and are deploying a fix at this time

Time: Aug. 23, 2024, 6:42 p.m.

Status: Identified

Update: We have determined that the bug does not affect all syncs, nor does it cause records to be sent multiple times - this was a red herring. A high percentage of syncs are still failing, but there is not a correctness issue, so we are reenabling the sync engine globally so that syncs that are not affected by the issue can return to operation.

Time: Aug. 23, 2024, 5:58 p.m.

Status: Identified

Update: We have identified an issue which could cause records to be synced multiple times to destinations. Since this can affect the correctness of syncs for certain destinations, we have paused all syncs proactively until the issue can be diagnosed and resolved. We are continuing to investigate and are in the act of rolling back to a previous version of the sync engine.

Time: Aug. 23, 2024, 5:41 p.m.

Status: Identified

Update: Production sync execution has been paused while we investigate an issue

Time: Aug. 16, 2024, 6:36 p.m.

Status: Resolved

Update: This incident has been resolved.

Time: Aug. 16, 2024, 5:33 p.m.

Status: Monitoring

Update: We have fixed the issue. All Linkedin syncs affected by the issue should succeed the next time they run.

Time: Aug. 16, 2024, 4:57 p.m.

Status: Identified

Update: Linkedin Syncs are failing as the API version used by Census has been deprecated. The issue has been identified, and we are bumping the API version to unblock syncs.

Time: July 31, 2024, 3:23 a.m.

Status: Resolved

Update: This incident has been resolved.

Time: July 31, 2024, 2:19 a.m.

Status: Monitoring

Update: Mitigation has been deployed to aid in reducing failures from our upstream provider. We have seen a significant reduction in errors and are now monitoring the system.

Time: July 31, 2024, 12:47 a.m.

Status: Investigating

Update: We are currently seeing increased error rates in our us region.

Time: July 18, 2024, 10:49 p.m.

Status: Resolved

Update: This change has been rolled back to reduce noisy alerts for existing customers. Any alerts that were opened yesterday due to this change have been closed. Recovery emails will not be sent for the alerts opened yesterday. Failed record alerts will be triggered with the old logic moving forward. ------------Original Update-------------------------- To improve accuracy, we recently updated the calculation that powers the Failed Records % Sync Alert. This one-time correction could result in new alerts to those syncs where the correction applies. The new logic calculates the percent of failed records based on the number of changed records in a sync run, rather than the total number of records in the source.