Intercom

Problems with our APIs in all regions.
Affected components
Updates

Write-up published

Read it here

Resolved

This issue should now be resolved and all services are restored to normal functionality.

Please don’t hesitate to reach out to team@intercom.io if you have any further questions or if you’re still seeing any unexpected behaviour.

Mon, Mar 31, 2025, 01:52 PM

Monitoring

We've seen a reoccurance of these errors between 1330 and 1339 UTC+1 - error rates have returned to normal, but we continue to monitor.

Mon, Mar 31, 2025, 12:44 PM(1 hour earlier)

Resolved

This issue is resolved and all services are working as normal.

Please don’t hesitate to reach out to team@intercom.io if you have any further questions or if you’re still seeing any unexpected behaviour

Mon, Mar 31, 2025, 11:33 AM(1 hour earlier)

Monitoring

Intercom's REST API in the USA hosting region went down due to a bad deployment referencing newly built caching infrastructure that was not fully accessible to all fleets.

The web fleets serving the messenger, REST APIs and inbox in Europe also went down due to the same underlying cause, and processing capacity was exhausted as a result, causing a full outage of the endpoints there.

The Australia region endpoints had the same problem, but the problem only caused higher latency than usual.

The deployments were automatically rolled back.

The downtime on the EU endpoints was between 10:46-10:59 UTC

The downtime on the USA REST API endpoint was between 10:46-10:59 UTC, with some residual errors until 11:03 UTC

Mon, Mar 31, 2025, 11:15 AM(17 minutes earlier)

Investigating

We're seeing issues with our APIs in all regions. Our team is aware and investigating the issue. We’ll update you here as soon as we have more information.

Mon, Mar 31, 2025, 11:00 AM(15 minutes earlier)