Partial outage of
Incident Report for Matrix
At 13:10 UTC a new deployment of Synapse was rolled out to We were alerted to increased failure rates and a drop in database throughput at 14:20 UTC. During investigation we identified that users started to see failing requests from 13:40 UTC onwards.

At 14:30 UTC we decided to roll back the deployment. Error rates improved starting at 14:35 UTC, and have returned to normal now that the rollback has completed (at 14:58 UTC). We are following up by investigating the root cause.
Posted Nov 29, 2022 - 13:00 UTC