All Systems Operational

About This Site

Status updates for Buildkite’s services and components. You can also follow @buildkitestatus on Twitter for updates.

Uptime over the past 90 days. View historical uptime.
Web Operational
90 days ago
99.97 % uptime
Today
Agent API Operational
90 days ago
99.62 % uptime
Today
REST API Operational
90 days ago
99.99 % uptime
Today
Job Queue Operational
90 days ago
99.68 % uptime
Today
SCM Integrations Operational
90 days ago
100.0 % uptime
Today
Hosted Agents ? Operational
90 days ago
99.85 % uptime
Today
Notifications Operational
90 days ago
99.99 % uptime
Today
GitHub Commit Status Notifications Operational
Email Notifications Operational
Slack Notifications Operational
Webhook Notifications Operational
90 days ago
99.99 % uptime
Today
SCM Providers ? Operational
GitHub Operational
GitHub API Requests Operational
GitHub Webhooks Operational
Atlassian Bitbucket SSH Operational
Atlassian Bitbucket Website and API Operational
Atlassian Bitbucket Git via HTTPS Operational
Third Party Services ? Operational
AWS ec2-us-east-1 Operational
AWS elasticache-us-east-1 Operational
AWS elb-us-east-1 Operational
AWS rds-us-east-1 Operational
PagerDuty Notification Delivery Operational
Test Engine Operational
90 days ago
100.0 % uptime
Today
Web ? Operational
90 days ago
100.0 % uptime
Today
Ingestion ? Operational
90 days ago
100.0 % uptime
Today
REST API Operational
90 days ago
100.0 % uptime
Today
Docs ? Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.

No incidents or maintenance related to this downtime.

System Metrics Month Week Day
Web Response Time ?
Fetching
Agent API Response Time ?
Fetching
REST API Response Time ?
Fetching
Agent Job Dispatch
Fetching
Past Incidents
Jan 8, 2025

No incidents reported today.

Jan 7, 2025
Elevated response time in the agent API
Resolved - Incident is resolved as we no longer see elevated latency issues after applying the necessary mitigations.
Jan 7, 20:38 UTC
Monitoring - We are observing latency is returning to normal levels after the mitigation and we are actively monitoring it.
Jan 7, 20:12 UTC
Identified - We applied some mitigations and are observing improvements in the elevated latency but continuing to investigate this further.
Jan 7, 19:54 UTC
Update - We are seeing elevated latency with Agent API and team is investigating the issue.
Jan 7, 19:18 UTC
Investigating - We've spotted that something has gone wrong. We're currently investigating the issue, and will provide an update soon.
Jan 7, 18:49 UTC
Degraded performance
Resolved - We have completed our mitigation efforts, and have seen a full restoration of service for all users. Our monitoring shows that all customers are now operational and processing normally.
Jan 7, 07:33 UTC
Monitoring - The fix has been rolled out and all customers should now see recovery. We will continue to monitor.
Jan 7, 07:20 UTC
Update - The majority of customers are now operational and processing normally. Remaining customers experiencing issues are having targeted mitigations applied.
Jan 7, 06:10 UTC
Update - The majority of customers are now operational and processing normally. Remaining customers experiencing issues are having targeted mitigations applied.
Jan 7, 04:02 UTC
Identified - We continue to see the majority of customers see improvements as jobs are picked up and ran. We are implementing a further mitigation for the remaining impacted customers.
Jan 7, 02:48 UTC
Update - We continue to see the majority of customers see improvements as jobs are picked up and ran. We are investigating means to expand these mitigations to all customers.
Jan 7, 01:55 UTC
Update - We are continuing to see a restoration of services for the majority of our customers.
Jan 7, 00:44 UTC
Update - We’re seeing a partial restoration of services for majority of our customers.
Jan 7, 00:08 UTC
Update - We are still experiencing significant performance degradation to a database cluster. We are performing targeted load shedding to help restore service to broader customer base, before bringing the specific customers online.
Jan 6, 23:48 UTC
Update - We are still experiencing significant database degradation due to load. We are investigating multiple paths to try and resolve the issue.
Jan 6, 23:12 UTC
Update - We are currently experiencing significant database degradation and are continuing to investigate the issue.
Jan 6, 22:12 UTC
Investigating - The fix rolled out fixed the notification latency but we have run into another issue during this mitigation which the team is actively investigating.
Jan 6, 21:42 UTC
Monitoring - We've identified the cause of delayed notification delivery, a fix is in place and notification latency is recovering
Jan 6, 20:53 UTC
Identified - We identified the possible root cause of the issue and are actively working on mitigating the issue
Jan 6, 19:46 UTC
Update - We are currently experiencing degraded performance due to a recurrence of recent database performance issues. Our engineering team is actively investigating and working on mitigating the impact
Jan 6, 19:04 UTC
Update - We are continuing to investigate this issue
Jan 6, 18:11 UTC
Investigating - We are currently investigating this issue.
Jan 6, 17:55 UTC
Jan 6, 2025
API performance degredation
Resolved - This incident has been resolved.
Jan 6, 03:39 UTC
Monitoring - Load has returned to normal levels after mitigations were put in place. We will continue to monitor the situation for any further impact.
Jan 6, 00:37 UTC
Identified - We are seeing degraded performance across web and API due to a re-occurance of recent database performance issues. We are actively mitigating the problem.
Jan 6, 00:23 UTC
Jan 5, 2025
Increased error rate in Agent API
Resolved - This incident has been resolved.
Jan 5, 18:04 UTC
Monitoring - We've identified the root cause of the degraded performance on one of our database clusters. System performance has returned to normal. We continue to monitor for any changes.
Jan 5, 17:56 UTC
Investigating - We are currently investigating this issue.
Jan 5, 17:03 UTC
Update - We continue to investigate the root cause of performance issues on one of our database clusters.
Jan 5, 16:50 UTC
Update - We are experiencing further issues with degraded performance on the Agent API.
Jan 5, 16:21 UTC
Monitoring - We continue to monitor the performance of the Agent API.
Jan 5, 15:40 UTC
Update - Agent API has returned to normal performance. We continue to investigate the root cause.
Jan 5, 15:37 UTC
Identified - We've identified that the increased error rate is isolated to a single database shard with reduced impact. We continue to investigate the root cause.
Jan 5, 15:31 UTC
Update - We continue to investigate the root cause of the increased error rate and latency in the Agent API.
Jan 5, 14:40 UTC
Update - Errors rate has increased. We continue to investigate the root cause.
Jan 5, 14:13 UTC
Update - Service status has returned to normal. We continue to investigate the root cause.
Jan 5, 14:10 UTC
Investigating - We're experiencing an increased error rate in the Agent API and are investigating the cause and impact.
Jan 5, 13:58 UTC
Jan 4, 2025

No incidents reported.

Jan 3, 2025

No incidents reported.

Jan 2, 2025

No incidents reported.

Jan 1, 2025

No incidents reported.

Dec 31, 2024

No incidents reported.

Dec 30, 2024

No incidents reported.

Dec 29, 2024

No incidents reported.

Dec 28, 2024

No incidents reported.

Dec 27, 2024

No incidents reported.

Dec 26, 2024

No incidents reported.

Dec 25, 2024

No incidents reported.

← Incident History Powered by Atlassian Statuspage