EU Cloud - Service Disruption
Incident Report for Interact
We've confirmed that the service is healthy and that the issue is now resolved. The infrastructure that handles file storage/search index became degraded, causing availability issues. This negatively impacted our web application servers which were overloaded with connections due to increased timeouts. Interact's automatic redundancy system compensated by bringing online more web servers whilst our engineers investigated. Interact initiated a hardware refresh across the application infrastructure at 18.30pm this evening. Interact's APIs, iOS and Andriod apps were unaffected by this incident.
Posted 10 months ago. Feb 07, 2017 - 20:14 UTC
Rolling refresh has been completed. Service has been restored.
Posted 10 months ago. Feb 07, 2017 - 18:35 UTC
Rolling refresh of hardware has started. will update once complete.
Posted 10 months ago. Feb 07, 2017 - 18:30 UTC
Engineers are commencing a rolling refresh of hardware in conjunction with our hosting provider (Amazon Web Services). We expect partial disruption to the service to last no longer than 8 minutes @ 18.30pm GMT.
Posted 10 months ago. Feb 07, 2017 - 18:19 UTC
Service is recovering for a few remaining customers using custom URLs (e.g. Engineers are currently monitoring the final customers.
Posted 10 months ago. Feb 07, 2017 - 16:25 UTC
We observing increased failure rates between our application servers and file storage environments. This is impacting service availability for some users. We have increased service capacity for our application servers to mitigate. We are advising all customers to clear cache if they are receiving a server unavailable message.
Posted 10 months ago. Feb 07, 2017 - 14:49 UTC
Engineers have identified an issue affecting some of the servers in use in the EU cloud. We have increased capacity on the relevant pods to mitigate this issue and we will continue to monitor the issue.
Posted 10 months ago. Feb 07, 2017 - 14:20 UTC
Engineers are investigating issues affecting connectivity for a subset of customers on the EU public cloud. Updates to follow shortly.
Posted 10 months ago. Feb 07, 2017 - 14:05 UTC