Faulty CrowdStrike update takes out Windows machines worldwide

Houndreds of housands and possibly millions of Windows computers and servers worldwide have been made inoperable by a faulty update of Crowdstrike Falcon Sensors, and the outage affected transport, broadcast, financial, retail and other organizations in Europe, Australia, the US and elsewhere.

Sky News is off the air in the UK; Delta, United and American Airlines have paused flights around the world.

What happened?

What initially seemed like it might be a Microsoft problem is now confirmed to have been created by Crowdstrike, i.e., its endpoint security agent.

The malfunctioning update throws Windows hosts into a blue-screen-of-death (BSOD) loop that – as advised by Crowdstrike – can be interrupted by:

1. Booting Windows into Safe Mode or the Windows Recovery Environment 2. Navigating to the C:\Windows\System32\drivers\CrowdStrike directory 3. Locating the file matching “C-00000291*.sys” and deleting it, then 4. Booting the host normally.

Unfortunately, in many cases this will have to be a manual intervention that has to be performed via a local admin account, and it will take a while at companies with huge fleets of Windows PC workstations to restore them – and on a Friday, too. IT/support teams will have to work through the weekend, it seems.

Crowdstrike is surely analyzing the “bad” update to see what happened, and security researchers are trying to do the same.

While the cause of the outage is likely to end up being a simple coding error (and not deliberate sabotage due to unnoticed supply-chain compromise), it affected one component of the CIA triad (availability), making this effectively an information security issue.