Delay in notification dispatch

Incident Report for Buildkite

Resolved

This incident has been resolved.
Posted Oct 17, 2023 - 03:56 UTC

Update

New notifications are expected to be delivered normally. Any notifications dispatched during the outage are expected to be delivered but may be delayed.
Posted Oct 17, 2023 - 03:55 UTC

Monitoring

The error has been fixed and notifications are now being processed. We will continue monitoring.
Posted Oct 17, 2023 - 03:49 UTC

Identified

We have identified an error which is causing notification dispatch to be delayed. This includes notifications such as GitHub Commit Status updates, Slack Notifications, Webhooks and email. We are working to resolve the bug that is causing the error. We will provide a status update in 10 minutes.
Posted Oct 17, 2023 - 03:40 UTC
This incident affected: Notifications (GitHub Commit Status Notifications, Email Notifications, Slack Notifications, Webhook Notifications).