On 27 April 2020 at 11:25 GMT, our Cloud team started investigating a potential issue with our back-end service (called Tachyon) that handles such functions as creating indexed content and sending emails. Indexing jobs were not running properly and appeared to be queuing up. At 12:05 GMT, we started receiving client incident reports via our Technical Support team saying that pages were not loading properly, widgets did not have content and videos were unable to be loaded. Through a combination of restarting backend services, resetting dependent services and resetting the queued index jobs, the EU system was returned to a normal state around 13:50 GMT.
Posted Apr 27, 2020 - 13:55 UTC
Resolved
This incident has been resolved.
On 27 April 2020 at 11:25 GMT, our Cloud team started investigating a potential issue with our back-end service that handles such functions as creating indexed content and sending emails. Indexing jobs were not running properly and appeared to be queuing up. At 12:05 GMT, we started receiving client incident reports via our Technical Support team saying that pages were not loading properly, widgets did not have content and videos were unable to be loaded. Through a combination of restarting backend services, resetting dependent services and resetting the queued index jobs, the EU system was returned to a normal state around 13:50 GMT.
Posted Apr 27, 2020 - 13:47 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Apr 27, 2020 - 12:56 UTC
Identified
The issue has been identified and we are in the process of correcting it.
Posted Apr 27, 2020 - 12:54 UTC
Investigating
We are currently investigating an issue with certain pages not loading. widget content not loading and videos not uploading.