Customers who were impacted by a flaw discovered in a single Windows host content update are currently receiving assistance from CrowdStrike. Linux and Mac hosts are unaffected. There was no cyberattack here.
According to Crowdstrike “After the problem was discovered, it was isolated, and a solution was implemented. For the most recent information, we are directing clients to the support site. We will keep our blog updated with comprehensive and frequent public information.“
Details Provided By Crowstrike
- Among the symptoms are hosts reporting a Falcon sensor-related bugcheck\blue screen error.
- The affected Windows hosts do not need to take any action because the problematic channel file has been reversed.
- Additionally, Windows hosts that go online after 05:27 UTC will not be affected.
- This problem does not affect hosts running Linux or Mac OS X.
- The reversed (good) version is the channel file “C-00000291.sys” with a timestamp of 0527 UTC or later. The faulty version of the channel file is “C-00000291.sys,” with a timestamp of 0409 UTC.
- Note: It is common for the CrowdStrike directory to contain numerous “C-00000291*.sys files.” The active content will be the file in the folder with a timestamp of 0527 UTC or later.
Information about a temporary fix for systems impacted by a recent CrowdStrike Falcon Sensor update has been made public by CrowdStrike. Blue Screen of Death (BSOD) errors have been reported by users of suffering systems.
To get around this problem, impacted system administrators and users can do the following:
- Use the Windows Recovery Environment or Safe Mode to boot Windows.
- Go to the CrowdStrike directory located at C:\Windows\System32\drivers.
- Find and remove the file that matches the pattern “C-00000291*.sys.”
- Perform a standard host boot.
In order to work around this issue, users and administrators of affected systems in public clouds and comparable environments (including virtual ones) have two options: they can either roll back to a snapshot taken prior to 0409 UTC or take the following actions:
- Disconnect the virtual server that is affected by the operating system disc volume.
- As a safeguard against accidental modifications, take a snapshot or backup of the disc volume before continuing.
- Mount or attach the volume to a fresh virtual server.
- Go into the directory %WINDIR%\System32\drivers\CrowdStrike.
- Find the file that ends with “C-00000291*.sys” and remove it.
- Take the volume out of the newly created virtual server.
- Attach the corrected volume back to the virtual server that was affected.
Making It Easier of non technical users
- To see a menu, allow the machine to boot up and crash three times.
- Select Troubleshoot.
- Select Advanced Settings.
- Select Command Prompt.
- You must enter your BitLocker Recovery Key if your system is BitLocker secured.
- If BitLocker is controlled by Intune, this can be accessed under “devices” at https://myaccount.microsoft.com. Verify that the device’s hostname and the key ID match.
- If not, request your BitLocker Recovery Key from your local IT administrator.
- Enter the following commands in the command prompt window, then press the Enter key.
- Warning: The X:\ drive is where the Command Prompt opens. Please remember to type these commands exactly in order to switch to C:. Some customers report needing to change the BIOS settings if the C: drive is not accessible.
- The path is C:\Windows\System32\drivers.
- del C-00000291* and exit
- Select “Continue to Windows”
Azure’s serial instructions for entering safe mode
- Login to Azure console –> Go to Virtual Machines –> Select the VM
- Upper left on console –> Click : “Connect” –> Click –> Connect –> Click “More ways to Connect” –> Click : “Serial Console”
- Once SAC has loaded, type in ‘cmd’ and press enter.
- type in ‘cmd’ command
- type in : ch -si 1
- Press any key (space bar). Enter Administrator credentials
- Type the following:
- bcdedit /set {current} safeboot minimal
- bcdedit /set {current} safeboot network
- Restart VM
Optional: How to confirm the boot state? Run command:
- wmic COMPUTERSYSTEM GET BootupState
Will I be affected since I still have the file on my system?
Channel file “C-00000291*.sys” with timestamp of 0527 UTC or later is the reverted (good) version.
Channel file “C-00000291*.sys” with timestamp of 0409 UTC is the problematic version.
So the file existing on a system does not mean the workaround needs to be applied.
Is it necessary to remove CrowdStrike?
Remove CrowdStrike from your PC if it has booted up and is (back) online.
We briefly enabled our customers to uninstall CrowdStrike by turning off Uninstall Protection in the early hours of the incident. Since CrowdStrike is no longer releasing the update that is causing the problems, it has been enabled again. As a remedy is now available, uninstalling CrowdStrike is not advised as it could endanger your systems.