Resolved
All systems appear to be operating normally following the deployed fix.
Monitoring
All systems appear to be operating normally following the deployed fix. We're continuing to closely monitor notification delivery to confirm full stability before closing this incident.
Monitoring
The fix has been deployed. The root cause (error handling and deduplication in the notification pipeline) has been resolved. The notification service has been re-enabled and we're closely monitoring its behavior to confirm full stability.
Monitoring
The fix has been deployed. The root cause (error handling and deduplication in the notification pipeline) has been resolved. The notification service has been re-enabled and we're closely monitoring its behavior to confirm full stability.
Identified
We've identified the root cause of the issue: a technical problem in our notification pipeline caused repeated notifications to be sent for a single event to a limited number of users. No data was compromised, this incident only affects the volume of notifications sent.
Investigating
We've identified an issue with our notification service causing repeated notifications to be sent for a single event to some members. The notification service has been suspended as a precaution while we investigate.