Processing pipeline and web outage

Incident Report for Rollbar

Resolved

We'll definitely keep a closer than usual eye on things for the rest of the day, but at this point the network issues have been resolved. The web app and processing pipeline are functioning normally.

Posted May 02, 2018 - 14:25 PDT

Monitoring

A fix has been implemented and we are monitoring the results.

Posted May 02, 2018 - 14:21 PDT

Update

Our networking issues have been resolved. The web UI should be fully operational and the processing pipeline is recovering, processing data received and cached during the outage.

Posted May 02, 2018 - 14:20 PDT

Identified

We're experiencing degraded performance because of networking issues on the Google Cloud Platform. We're taking steps to mitigate the issue and are tracking the upstream issue at https://status.cloud.google.com/incident/cloud-networking/18007

Posted May 02, 2018 - 14:09 PDT

Investigating

We're currently investigating issues with the processing pipeline and web UI

Posted May 02, 2018 - 14:00 PDT

This incident affected: Web App (rollbar.com), API Tier (api.rollbar.com) and Processing pipeline (Core Processing Pipeline).