Resolved -
Mitigation strategies worked as expected and now agent registration is back to normal. All systems are stable now.
Aug 26, 03:13 UTC
Monitoring -
Recovery is underway; we'll keep monitoring until recovery is complete.
Aug 26, 02:43 UTC
Update -
We have rolled out a few changes to fix the underlying issues and are seeing signs of recovery.
We'll keep monitoring until all systems are fully recovered and communicate once we are confident everything is stable again.
Aug 26, 02:36 UTC
Update -
We identified issues affecting certain customers when the buildkite-agent attempts to register with Buildkite. When agents are not able to register, builds may start to back up.
We've identified a potential root case and are attempting changes to fix it.
Aug 26, 02:02 UTC
Investigating -
We're seeing increased latency and error rates for a subset of our customers. We're currently investigating and will provide status updates as they become available.
Aug 26, 01:42 UTC