We've confirmed that the service is healthy and that the issue is now resolved. The infrastructure that handles file storage/search index became degraded, causing availability issues. This negatively impacted our web application servers which were overloaded with connections due to increased timeouts. Interact's automatic redundancy system compensated by bringing online more web servers whilst our engineers investigated. Interact initiated a hardware refresh across the application infrastructure at 18.30pm this evening. Interact's APIs, iOS and Andriod apps were unaffected by this incident.
Posted Feb 07, 2017 - 20:14 UTC
Update
Rolling refresh has been completed. Service has been restored.
Posted Feb 07, 2017 - 18:35 UTC
Update
Rolling refresh of hardware has started. will update once complete.
Posted Feb 07, 2017 - 18:30 UTC
Update
Engineers are commencing a rolling refresh of hardware in conjunction with our hosting provider (Amazon Web Services). We expect partial disruption to the service to last no longer than 8 minutes @ 18.30pm GMT.
Posted Feb 07, 2017 - 18:19 UTC
Monitoring
Service is recovering for a few remaining customers using custom URLs (e.g. intranet.companyname.com). Engineers are currently monitoring the final customers.
Posted Feb 07, 2017 - 16:25 UTC
Update
We observing increased failure rates between our application servers and file storage environments. This is impacting service availability for some users. We have increased service capacity for our application servers to mitigate. We are advising all customers to clear cache if they are receiving a server unavailable message.
Posted Feb 07, 2017 - 14:49 UTC
Identified
Engineers have identified an issue affecting some of the servers in use in the EU cloud. We have increased capacity on the relevant pods to mitigate this issue and we will continue to monitor the issue.
Posted Feb 07, 2017 - 14:20 UTC
Investigating
Engineers are investigating issues affecting connectivity for a subset of customers on the EU public cloud. Updates to follow shortly.