Time: Dec. 11, 2017, 10:59 p.m.

Status: Resolved

Update: At this time all Zoey stores should be accessible. We will be scheduling a maintenance window in the near future to apply a fix to prevent this error from occurring again. Thank you for your patience and understanding as we continue to make the Zoey infrastructure more resilient.

Time: Dec. 11, 2017, 10:43 p.m.

Status: Monitoring

Update: Sites are now coming online. We expect all Zoey Stores to be accessible within the next ten to fifteen minutes.

Time: Dec. 11, 2017, 10:39 p.m.

Status: Identified

Update: We are in the process of restoring connectivity, we expect sites to start being accessible again within the next few minutes.

Time: Dec. 11, 2017, 10:17 p.m.

Status: Investigating

Update: We are investigating reports of customers being unable to access certain stores

Time: Oct. 24, 2017, 8:01 p.m.

Status: Resolved

Update: All Zoey Stores should now be operational. If you are still experiencing issues please open a support ticket. We apologize for this issue.

Time: Oct. 24, 2017, 7:41 p.m.

Status: Monitoring

Update: A fix is being rolled out to all stores, we expect this to be completed within the next 20 minutes.

Time: Oct. 24, 2017, 7:32 p.m.

Status: Identified

Update: We have identified the issue causing this error and are now fixing it. We will update you shortly.

Time: Oct. 24, 2017, 7:32 p.m.

Status: Investigating

Update: We are aware of customers receiving error messages accessing their stores and are investigating the issue now.

Time: Oct. 18, 2017, 9:35 p.m.

Status: Resolved

Update: Amazon Web Services has resolved their problem and Zoey service should now function normally. If you experience any issues please open a ticket.

Time: Oct. 18, 2017, 9:34 p.m.

Status: Monitoring

Update: At this time we continue to monitor the Amazon Web Services Status Page but are seeing that some customers are able to login again.

Time: Oct. 18, 2017, 9:10 p.m.

Status: Investigating

Update: Amazon Web Services is currently experiencing an issue which is preventing users from accessing the Zoey Admin panel. This should not present any issues with the front end of your stores. We will provide more information as soon as Amazon makes it available. You can find out more at https://status.aws.amazon.com/

Time: June 27, 2017, 10:05 p.m.

Status: Resolved

Update: We have successfully upgraded the storage array and all impacted stores are back online. This work is part of our overall work in upgrading our infrastructure. Merchants will notice dramatic improvements in speed and stability as well as major changes in the near future that will make your Zoey store more resilient than ever! We thank you for your patience during this work.

Time: June 27, 2017, 9:49 p.m.

Status: Identified

Update: We are currently upgrading the storage array tied to the a computing node within Zoey. We expect this operation to be completed by 18:00 EST. Thank you for your patience.

Time: March 31, 2017, 12:28 a.m.

Status: Resolved

Update: We are going to mark this incident resolved as all stores are back online and transactional for the past couple of hours. The root cause of this issue was due a bug in the storage array. Under very rare circumstances, if an unexpected controller restart or power failure occurred, an internal save might not have completed correctly. This issue results in a state requiring support intervention and discarding cached data. Due to the loss of this cached data we had to rebuilt several key pieces of the infrastructure that were impacted by the loss of the cached data. Due to the architecture of Zoey, it is possible that up to 1 second of data may have been lost although highly unlikely. The most probable manifestation of this issue would be if in that 1 second window a customer was placing an order, received approval from the payment gateway and Zoey could not complete the transaction due to the outage. If you notice any such issues please open a ticket with Support but the chances of this are extremely low given the 1 second window it could occur in. This window would have been at approximately 7:10am Eastern Daylight Time. We are now in the process of applying updated software to resolve this issue and will also verify the remainder of our infrastructure for this potential bug. If you notice or see any other issues please open a ticket with Support. Thank you for your patience during this time.

Time: March 30, 2017, 10:35 p.m.

Status: Monitoring

Update: All stores should now be back online and operational with no data loss. We apologize for this problem and thank you for your patience. We are continuing our final testing/cleanup procedures and as a result there may be a brief additional window of downtime (less than 20 minutes) impact as we finish restoration. We will post an additional update when we have completed this work.

Time: March 30, 2017, 9:22 p.m.

Status: Identified

Update: At this time all but one computing node has been restored. All other stores should be up and running. We are commencing final testing/cleanup procedures and as a result there may be a brief additional window of downtime (less than 20 minutes) impact as we finish restoration. We will post an additional update when we have completed this work.

Time: March 30, 2017, 8:20 p.m.

Status: Identified

Update: We are now in the final stages of bringing the remaining stores up. We expect to fully resolve this issue within the next few hours barring any unforeseen circumstances.

Time: March 30, 2017, 6:33 p.m.

Status: Identified

Update: Our recovery efforts appear to be successful on one server and we are currently checking to ensure the recovery was complete. We will then be expanding this process to all remaining servers to restore all remaining stores.

Time: March 30, 2017, 5:58 p.m.

Status: Identified

Update: We are in the final stages of testing a recovery method to bring sites back online without data loss. We expect these tests to be finished shortly. If they are successful we will bring back additional stores shortly thereafter. A root cause of the issue has also been identified and we will begin steps to mitigate this from happening in the future shortly after service is restored. Thank you for your patience during this process.

Time: March 30, 2017, 5:18 p.m.

Status: Identified

Update: We continue to work on a recovery plan for the affected sites. We will post more information as soon as possible.

Time: March 30, 2017, 3:45 p.m.

Status: Identified

Update: We have been successful at identifying the issue and are working on bringing all sites back online as quickly as possible.

Time: March 30, 2017, 3:15 p.m.

Status: Investigating

Update: We are still working directly with our storage provider to assess and resolve this issue.

Time: March 30, 2017, 2:17 p.m.

Status: Investigating

Update: We are currently in direct contact with our storage provider in assessing the issue and coming up with a resolution plan. Please continue to monitor this page for more information.

Time: March 30, 2017, 1:31 p.m.

Status: Investigating

Update: We are still investigating the root cause of this storage connectivity issue.

Time: March 30, 2017, 12:38 p.m.

Status: Investigating

Update: We are currently investigating a storage connectivity issue affecting a number of our stores. Updates will be posted shortly.