Beginning at 04:09 UTC, CrowdStrike published an update to Falcon sensors that caused Microsoft Windows hosts to experience blue screen crashes (“blue screen of death”) and reboots, preventing them from receiving the latest update from Crowdstrike that fixes the issue. Hosts that are unable to receive the latest update require manual intervention to remediate.
Sevco is helping our customers triage the outages caused by the CrowdStrike update. This morning, Sevco reached out to all affected customers with the following queries in the Sevco platform to track the Crowdstrike update causing MSFT outages.
- Devices that are potentially experiencing the issue.
- This query identifies any device running Windows, running Crowdstrike, and was online three days ago but is not online after Crowdstrike pushed the fix at 05:27 UTC.
- Devices not at risk.
- This query identifies any device not running Windows, not running Crowdstrike, or on one of the two OS releases known to be unaffected – Windows 7 and Windows 2008 R2.
- Devices that are resolved.
- This query identifies any device that may have been affected but are currently communicating with Crowdstrike or other systems.
- Users that are potentially experiencing the issue.
- This query identifies users associated with devices running Windows, running Crowdstrike, and was online 3 days ago but is not online after Crowdstrike pushed the fix.
We have also released a custom dashboard in the Sevco platform customers can use to track the queries above and their progress towards resolution.
The dashboard tracks:
- The percentage of devices that may have been impacted by this issue (all Windows devices with the exception of Windows 7 and Windows Server 2008 R2).
- The percentage of the potentially impacted devices that appear to be resolved. This includes devices that are operating normally by actively communicating with systems tracked in Sevco after CrowdStrike corrected its content file.
- Actionable modules for quick and easy one-click access to the list of devices and users that may currently be experiencing the CrowdStrike Falcon Content Update issue
We are continuing to monitor and respond. Sevco’s Customer Success team is committed to helping affected customers address the outage. If you need any further support in managing this incident, please reach out to our Customer Success team at support@sevco.io.