Microsoft blames massive Microsoft 365 outage on maintenance bug
Microsoft 365 outage caused by bug in automated network maintenance system.
A bug in Microsoft's automated network maintenance system mistakenly removed IP routes from numerous devices, leading to a widespread outage affecting Microsoft 365 and Azure services. The issue, which began on July 23rd, disrupted services like SharePoint, Excel, and Teams, with Microsoft eventually reverting the maintenance change to restore functionality.
AI summary of the linked source. Not reviewed by an editor. How this works
Run Microsoft?
Get an email when a new story names Microsoft, usually within the hour.
Free. Your list stays private and never appears in a subject line. One click stops it. How Vendor Watch worksPrivacy
The lesson from this incident
Full lessonA flaw in Microsoft's automated network maintenance system incorrectly removed IP routes from a large number of network devices, causing a cascading failure across Microsoft 365 and Azure services. This incident highlights the critical risk of insufficient validation and testing in automated infrastructure management pipelines — a single unchecked change propagated broadly before it could be caught. The blast radius was magnified by the automated nature of the system, which applied the erroneous change at scale faster than human review could intervene. This matters because enterprise organizations depend on cloud platforms like Microsoft 365 for mission-critical operations, meaning even short outages translate to significant productivity losses, financial impact, and erosion of customer trust.
What helps against this
How we list vendors
- Log management and security data pipelines
Security logs are so expensive to collect and keep that teams drop the ones they need when an incident happens.
- Incident response retainer
Experts are on call before the worst day, not searched for during it.