Security: Why it pays to be proactive

Following on from my post yesterday about reacting to critical updates

It seems that no more than a day after Microsoft released a Critical Security update, someone’s released a Trojan into the wild that exploits the vulnerability.

Given the “exploit potential”, this one sounds relatively tame. I suspect it’ll only be a matter of time before the exploit code is perfected and turned into a much more potent animal.

Putting a few hours in on Thursday night, has potentially saved us exponentially more hours in data and service recovery, as well as general IT support. It definitely pays to be proactive!

Reacting to Super-Critical Updates (MS08-67)

Yesterday evening, at 6pm BST, Microsoft released an ‘Emergency’ Security Update MS08-67, for Windows-based Operating Systems. The update plugs a hole in Windows that could allow a Virus/Worm to automatically infect a Windows PC without any user intervention.

I thought I’d document what actions I took, in case it helps out anyone in the future. I’d also be interested to hear how you handled the situation, particularly if you did something I missed, or if you think I could have done things better!

History repeating…

Although I remember the impact of Sasser and MyDoom, I’ve never been in the trenches when such a critical update has been launched for Windows.

No-one likes working late at night, but I didn’t fancy the chances that a 0-day exploit may be released and in the wild before we can patch our mission critical servers; so as soon as I found out, I started working on a plan.

The Plan

The plan was relatively simple: Get the update to as many PCs as possible, as soon as possible; with an emphasis on any Servers that provide business-critical services.

Simple enough, but what next?

WSUS

About a month back we setup an internal WSUS server to centralise Windows Updates – quite handy for this type of scenario! The main thing here is to ensure that WSUS has the updates downloaded and approved, ready for deployment. Fortunately it had, as it performs a sync every evening, and automatically approves Critical Updates.

Group Policy

To ensure PCs get the update as fast as possible, we needed to open up GPMC and re-configure all existing Group Policy Objects (GPOs) that address Windows Update configuration.

The Windows Updates settings are under Computer Configuration > Administrative Templates > Windows Components > Windows Update.

Note that, if you don’t have WSUS, you can still make the changes outlined below in order to minimise Time-to-Patch. If you haven’t set “Specify intranet Microsoft update service location”, PCs will automatically ask Microsoft’s update servers on the internet.

What we’re looking to do is:

– Set all PCs to download and schedule updates. This is abnormal for us as we allow our Engineers to dictate when to install updates as it can interfere with Software development and testing.

– Make sure each PC checks for updates with our WSUS server every hour, as opposed to every 22 hours.

– Set PCs to install the updates at 11am. This gives time for people to turn on their PCs, for the PCs to update their Group Policy settings and pick up the new settings, and then to check in with the WSUS server for the new update.

– If the PC missed the 11am deadline (e.g. it wasn’t on) it’ll check whether or not it has updates, and then install the updates after 30 minutes.

Informing End-users

A notification email was crafted to all employees, informing them of the severity of the update, what was being done, and what actions they should take. I’ll include a copy of the email I sent out at the end of the post

Protecting the business

Last night, we couldn’t wait for WSUS to “offer” the update to our servers so I grabbed the Update and manually installed it on each business-critical server, rebooting them promptly.

This morning

That was last night out of the way. This morning and this afternoon I’ve been checking WSUS’s reports to see which PCs have the update installed. As of 1pm, at least 90% of PCs had installed and rebooted. I’ll be chasing the rest later 😉

The notification

As promised, here’s the Email notification sent out to employees:

 
Hi all,

Microsoft has just released a very serious critical security update for Windows operating systems.

To see how this affects you, please see below.

Cambridge Employees

Tomorrow we will be rolling out an essential security update to all Domain-connected Windows PCs. This update is mandatory. If you press Control+Alt+Delete to log in, you are on the domain. If you do not press Ctrl+Alt+Del to log in you should follow the advice for Non-Cambridge employees below.

Although we will be trying our best to force this update out. It’s advisable that if you see the “Yellow shield” in your Task Bar, you should click it and install all updates reboot as soon as possible.

Not doing so poses a serious risk to DisplayLink’s networks.

Non-Cambridge Employees 

If you are not based in Cambridge, you should visit Windows Update as soon as possible and install all updates, specifically this one.

DisplayLink Servers

Servers in the UK will have the update installed and be rebooted as soon as possible to ensure we’re protected.

Further information

Further information on this Critical update can be found on Microsoft’s KB article.

Thanks go to Dave Hill for spotting this one on The Register!

Cheers,
Phil Wiffen
IT Engineer

 

How did you handle it?

As I said earlier, I’d also be interested to hear how you handled the situation, particularly if you did something I missed, or if you think I could have done things better! Let me know in the comments 🙂

Signal vs. Noise

It’s a title that may need a little explanation.

Lately, work and life have been a bit crazy and I’ve had very little time to blog anything of merit. I began to experiment with twitter to post short updates on things. then as time got more constrained, my twittering increased and blogging floundered.  At this point, I figured it’d be a good idea to integrate my twitterings directly into the blog feed to prevent things going stale.

I was wrong.

Last week, I started to get the feeling that the integration wasn’t working. The blog didn’t feel right; something more akin to a life status feed rather than a blog :/

Then Danny sent me some constructive feedback this morning. He’s right: the tweets just create too much noise; and quickly, any value that the blog had evaporates.

So, the integrated “in-feed” twitter updates are gone. They’re still in my sidebar, but you’ll no longer see them as part of the RSS feed or in the normal blog bits.

I’m hoping now that the ratio of Signal vs Noise can be restored.

Cheers,
Phil

Twitter Updates for 2008-10-12

  • isinteg fixed a bunch of errors. I tried a defrag with eseutil afterwards but it failed with error -1014. Seems to be a Pagefile issue… #
  • Here’s a site with an alleged solution: http://tinyurl.com/3q5l3b. Gonna try resizing the page file, but the downtime isn’t cool 🙁 #

An Open Source Alternative to Microsoft Project

My colleague Dave just pointed me to an open-source alternative to MS Project:

OpenProj is a free, open source desktop alternative to Microsoft Project. The OpenProj solution is ideal for desktop project management and is available on Linux, Unix, Mac or Windows. OpenProj is a complete desktop replacement of Microsoft Project and even opens existing native Project files. OpenProj shares the most advanced scheduling engine in the industry with Project-ON-Demand. The OpenProj solution has Gantt Charts, Network Diagrams (PERT Charts), WBS and RBS charts, Earned Value costing and more. You can get more detailed information on OpenProj or download now!

Sounds promising. If it really is a viable alternative, it’ll save us a good £350ish per volume licence! Now, to find a few willing Project Managers to try it out… 😉