All Systems Operational
Web App (rollbar.com)   Operational
API Tier (api.rollbar.com)   ? Operational
Processing Pipeline   ? Operational
rollbar.min.js   Operational
Mailgun SMTP   ? Operational
Mailgun Outbound Delivery   ? Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
System Metrics Month Week Day
api.rollbar.com uptime ?
Fetching
Processing latency (Default) ?
Fetching
Processing latency (JavaScript Source Maps) ?
Fetching
Processing latency (iOS Symbolication) ?
Fetching
Past Incidents
Oct 18, 2017

No incidents reported today.

Oct 17, 2017

No incidents reported.

Oct 16, 2017
One of the shards of our processing pipeline was stalled for about 35 minutes, causing processing and notifications to be paused for about 5% of items. We've cleared the stall and events are flowing through. The pipeline has caught up and is fully operational.

Additionally, we've found and corrected the monitoring misconfiguration that caused us to not notice this sooner.
Oct 16, 18:29 PDT
The web tier and processing pipeline experienced a ~3-minute outage from 3:44:40pm to 3:47:00pm PDT where the pipeline was stalled and the web tier was mostly inaccessible. The system recovered on its own and is now functioning normally.
Oct 16, 15:50 PDT
Oct 15, 2017

No incidents reported.

Oct 14, 2017

No incidents reported.

Oct 13, 2017
Resolved - This incident has been resolved.
Oct 13, 14:22 PDT
Investigating - We are investigating an issue which is causing high response times in the web tier.
Oct 13, 11:20 PDT
Resolved - We have identified and deployed a fix for the performance degradation. The issue was caused by a recent DB maintenance change. We have reverted the change and will investigate workarounds later today.
Oct 13, 06:11 PDT
Monitoring - We are investigating periodic degradation of system performance which leads to brief spikes in processing delays and slow web tier response times.
Oct 13, 05:21 PDT
Oct 12, 2017
Resolved - Maintenance is complete and all systems are functioning normally.
Oct 12, 23:09 PDT
Monitoring - We believe maintenance is complete. The processing pipeline experienced a ~1 minute delay. We'll monitor the system and resolve this incident once we've verified everything is operating normally.
Oct 12, 22:22 PDT
Update - We're beginning our DB maintenance. You'll see a maintenance page at rollbar.com and processing will be delayed briefly.
Oct 12, 22:16 PDT
Investigating - We'll be performing DB maintenance tonight at approximately 10 PM PDT. There may be a period or two lasting a few minutes where the web interface is in read-only mode and the processing pipeline is delayed briefly. We'll provide additional updates as we begin the maintenance and throughout the process.
Oct 12, 21:11 PDT
At 12:07 AM PDT this morning a server in our cache cluster failed. This resulted in elevated error rates for clients connecting to our API and a 2 minute stall of the processing pipeline. The API had recovered by 12:09 AM, but there were some residual delays in source map processing until around 12:20 AM when the cache cluster fully recovered.
Oct 12, 00:42 PDT
Oct 11, 2017

No incidents reported.

Oct 10, 2017

No incidents reported.

Oct 9, 2017

No incidents reported.

Oct 8, 2017

No incidents reported.

Oct 7, 2017

No incidents reported.

Oct 6, 2017

No incidents reported.

Oct 5, 2017

No incidents reported.

Oct 4, 2017

No incidents reported.