Yesterday evening, at 6pm BST, Microsoft released an ‘Emergency’ Security Update MS08-67, for Windows-based Operating Systems. The update plugs a hole in Windows that could allow a Virus/Worm to automatically infect a Windows PC without any user intervention.
I thought I’d document what actions I took, in case it helps out anyone in the future. I’d also be interested to hear how you handled the situation, particularly if you did something I missed, or if you think I could have done things better!
History repeating…
Although I remember the impact of Sasser and MyDoom, I’ve never been in the trenches when such a critical update has been launched for Windows.
No-one likes working late at night, but I didn’t fancy the chances that a 0-day exploit may be released and in the wild before we can patch our mission critical servers; so as soon as I found out, I started working on a plan.
The Plan
The plan was relatively simple: Get the update to as many PCs as possible, as soon as possible; with an emphasis on any Servers that provide business-critical services.
Simple enough, but what next?
WSUS
About a month back we setup an internal WSUS server to centralise Windows Updates – quite handy for this type of scenario! The main thing here is to ensure that WSUS has the updates downloaded and approved, ready for deployment. Fortunately it had, as it performs a sync every evening, and automatically approves Critical Updates.
Group Policy
To ensure PCs get the update as fast as possible, we needed to open up GPMC and re-configure all existing Group Policy Objects (GPOs) that address Windows Update configuration.
The Windows Updates settings are under Computer Configuration > Administrative Templates > Windows Components > Windows Update.
Note that, if you don’t have WSUS, you can still make the changes outlined below in order to minimise Time-to-Patch. If you haven’t set “Specify intranet Microsoft update service location”, PCs will automatically ask Microsoft’s update servers on the internet.
What we’re looking to do is:
– Set all PCs to download and schedule updates. This is abnormal for us as we allow our Engineers to dictate when to install updates as it can interfere with Software development and testing.
– Make sure each PC checks for updates with our WSUS server every hour, as opposed to every 22 hours.
– Set PCs to install the updates at 11am. This gives time for people to turn on their PCs, for the PCs to update their Group Policy settings and pick up the new settings, and then to check in with the WSUS server for the new update.
– If the PC missed the 11am deadline (e.g. it wasn’t on) it’ll check whether or not it has updates, and then install the updates after 30 minutes.
Informing End-users
A notification email was crafted to all employees, informing them of the severity of the update, what was being done, and what actions they should take. I’ll include a copy of the email I sent out at the end of the post
Protecting the business
Last night, we couldn’t wait for WSUS to “offer” the update to our servers so I grabbed the Update and manually installed it on each business-critical server, rebooting them promptly.
This morning
That was last night out of the way. This morning and this afternoon I’ve been checking WSUS’s reports to see which PCs have the update installed. As of 1pm, at least 90% of PCs had installed and rebooted. I’ll be chasing the rest later 😉
The notification
As promised, here’s the Email notification sent out to employees:
Hi all,Microsoft has just released a very serious critical security update for Windows operating systems.
To see how this affects you, please see below.
Cambridge Employees
Tomorrow we will be rolling out an essential security update to all Domain-connected Windows PCs. This update is mandatory. If you press Control+Alt+Delete to log in, you are on the domain. If you do not press Ctrl+Alt+Del to log in you should follow the advice for Non-Cambridge employees below.
Although we will be trying our best to force this update out. It’s advisable that if you see the “Yellow shield” in your Task Bar, you should click it and install all updates reboot as soon as possible.
Not doing so poses a serious risk to DisplayLink’s networks.
Non-Cambridge Employees
If you are not based in Cambridge, you should visit Windows Update as soon as possible and install all updates, specifically this one.
DisplayLink Servers
Servers in the UK will have the update installed and be rebooted as soon as possible to ensure we’re protected.
Further information
Further information on this Critical update can be found on Microsoft’s KB article.
Thanks go to Dave Hill for spotting this one on The Register!
Cheers,
Phil Wiffen
IT Engineer
How did you handle it?
As I said earlier, I’d also be interested to hear how you handled the situation, particularly if you did something I missed, or if you think I could have done things better! Let me know in the comments 🙂
