Service disruption - EU
Incident Report for Interact
Postmortem

At approximately 3:40pm BST an update was applied to an internal monitoring system used to record and monitor usage of the interact application. An issue with this deployment process inadvertently resulted in an authentication issue when recording this information. This had the unexpected side effect of causing pages within Interact to automatically refresh every page. This caused a significant increase in load and resultant degradation of performance

As part of standard procedures engineers were notified. To recover the service as soon as possible engineers restored to a previous version of the monitoring system. The Interact service became fully available at 4:10pm BST.

We are reviewing processes for the deployment of this system along with making modifications to the application code in order to ensure that future updates for this system will not have the same impact.

Posted 5 months ago. Jul 27, 2018 - 06:57 UTC

Resolved
This issue has been marked has resolved. We will update with a postmortem soon.
Posted 5 months ago. Jul 26, 2018 - 15:39 UTC
Monitoring
We are now monitoring the infrastructure to ensure all affected sites are functional and working as expected.
Posted 5 months ago. Jul 26, 2018 - 15:05 UTC
Update
We are continuing to work on a fix for this issue.
Posted 5 months ago. Jul 26, 2018 - 14:51 UTC
Identified
We have identified an issue within a control component which caused the issue and have reverted the update
Posted 5 months ago. Jul 26, 2018 - 14:46 UTC
Investigating
We are currently investigating issues within the EU of higher than normal latency which is causing slowness across all EU customers.
Posted 5 months ago. Jul 26, 2018 - 14:34 UTC
This incident affected: Europe Public Cloud 1 and Europe Public Cloud 2.