Issue with On-Premise API Message Delivery
- Resolved · UTC
The issue is resolved:
Issue description and impact:
On 17.01.2024 we received reports from some customers that message and status webhooks were being delayed.Cause of the issue:
We identified an issue with an internal component (callbacker) that was having issues with the latest deployment resulting in the delays.Mitigation Steps / Summary:
We reverted the deployment until we are able to fix underlying issues.Prevention of recurrence:
We have completed an internal review with the relevant teams and work is underway to further strengthen our callback service. - Monitoring · UTC
We are recovering from an issue that affected On-Premise API message delivery from ~6:20am to ~12:20pm UTC on Jan 17th. The situation has stabilized and we are continuing to monitor.
- Investigating · UTC
We are receiving reports of delays in messaging for On-Premise setups. Our engineering team are currently investigating it.
