At approximately 10:40am EDT / 7:40am PDT an update was applied to an internal monitoring system used to record and monitor usage of the interact application. An issue with this deployment process inadvertently resulted in an authentication issue when recording this information. This had the unexpected side effect of causing pages within Interact to automatically refresh every page. This caused a significant increase in load and resultant degradation of performance
As part of standard procedures engineers were notified. To recover the service as soon as possible engineers restored to a previous version of the monitoring system. The Interact service became fully available at 11:10am EDT / 8:10am PDT.
We are reviewing processes for the deployment of this system along with making modifications to the application code in order to ensure that future updates for this system will not have the same impact.