Web outage and processing pipeline delay

Resolved·Partial outage

The delay in the processing pipeline has resolved.

Fri, May 29, 2020, 06:14 PM

(5 years ago)

Affected components

May 29, 2020, 04:27 PM

06:14 PM

Updates

Resolved

The delay in the processing pipeline has resolved.

Fri, May 29, 2020, 06:14 PM

Monitoring

A fix has been implemented. The Processing Pipeline is working through backlog of events received during the outage; we now expect that backlog to be cleared about 2 hours, but prioritizing processing recent events.

Fri, May 29, 2020, 05:07 PM(1 hour earlier)

Identified

We have identified the issue. The Web App is operational, the Processing Pipeline is stopped while we implement a fix. Currently the real time notifications are delayed by 45 minutes.

Fri, May 29, 2020, 04:42 PM(24 minutes earlier)

Investigating

We are experiencing web outage and delay in the processing pipeline. We are investigating the issue.

Fri, May 29, 2020, 04:27 PM(15 minutes earlier)