Why Patch Management is still the Best First Line of Defense
April 13, 2012
Vulnerabilities are on the rise - especially from third party, non-Microsoft applications, which have four times more vulnerabilities than Microsoft applications. And cybercriminals have taken notice, exploiting these vulnerabilities at a faster rate than ever before. Today more than 2 million malware signatures are identified each month and traditional anti-virus defenses simply can’t keep up. Even the major anti-virus vendors have concluded that stand-alone anti-virus no longer provides an effective defense and that additional layers of security technology are needed to address the rising volume and sophistication of threats.
Sam: Hello everyone? And welcome to this Lumension webcast titled, "Why Patch Management is still your Best First Line of Defense." My name is Sam Erdheim and I'll be your moderator for today's event. Vulnerabilities are on the rise especially from third party non-Microsoft applications, which according to Secunia's latest report have four times more vulnerability than Microsoft applications, and cyber criminals have taken notice exploiting these vulnerabilities at a faster rate than ever before. To date, more than two million malware signatures are identified each month and traditional antivirus defenses can't keep up. Even the major antivirus vendors have concluded that standalone antivirus no longer provides an effective defense and that additional layers of security technology are needed to address the rise in volume and sophistication of threats that are taking advantage of these vulnerabilities.
So in this discussion with us today, is our keynote speaker, Paul Henry. Paul is one of the world's foremost global information and security and computer forensic experts with more than 20 years experience managing security initiatives for Global 2000 enterprises, and government organizations worldwide. Welcome, Paul.
Paul: Hey, good to be here, Sam.
Sam: In today's webcast, Paul will examine why you can't forget about older vulnerabilities, how to reduce exposure from both operating system and third party application vulnerabilities beyond just Microsoft. The sophistication of...increased sophistication of attacks to challenges with reliance upon free patching tools and native updaters, and why patch management should be considered as the core of an effective defense in depth endpoint security approach. We'll conclude with a question and answer session with Paul, but before we dive into that, I'd like to open up with a polling question for the audience, "Do you rely solely upon Microsoft WSUS for flaw remediation? Yes or No?" Please make your...submit your answers then we will get the results in a little bit. Also throughout this presentation, if you have any questions, you can please submit them via the questions tab at the top of the panel, and we will get to them in our Q&A section.
Thanks so much, and, Paul, at this point I will turn it back over to you.
Paul: Sounds great Sam, thank you very much. We're gonna start out essentially covering, you know, the simple fact of the matter that vulnerabilities are about much more than simply Microsoft. You know, they haven't cornered the market on vulnerabilities. You know, there's a couple of disturbing trends that we're seeing out there, the graph that's shown here it happens to be one of them. The conversion rate of sound flaws, you know, essentially being converted into workable exploits has been increasing. If you look at the chart you'll note that it's now reached 61% in January 2011. Again, what this boils down to is these are known flaws, you know, published flaws in software being converted into repeatable, reliable vulnerabilities and exploits. That's quite frightening by and of itself that we're seeing that change come into play. You know, as the number of vulnerabilities are increasing, we're seeing the bad guys being able to take those vulnerabilities and convert them into reliable exploits quicker and faster and more reliably, we'll say, than ever before.
Now, looking at the bigger picture here, you know, patched vulnerabilities still remain a prime exploit vector. If you look at any of the toolkits that are being used by the script kiddie level guys out there in the wild, the vast majority still rely on exploits that are up to and in some cases more than a year old. You know, in my view, there is simply no excuse for it. You know, there's a graph here we're showing on this slide from M86 Security that shows their top 15 exploits, you know, these would typically be exploits used within kits, and you'll note that some of them date back as far back as 2006. I know in my own research I typically find that some of the more popular vulnerabilities used with an exploit toolkits are up to a year or more older. You know, like I said, there's really no good excuse for it. Take a look at some of the more pervasive issues we've had over the last couple of years. Everybody remembers Conficker, you know, the bottom line with Conficker is that we had the patch that would have completely eliminated the issue yet the patch wasn't deployed, and look at what happened with Conficker.
Now, again, just to reemphasize the whole point of all of this, it really is not simply a Microsoft issue. We're not patching those third party applications that we're adding to our systems, we're not patching those browser add-ons that we're using for our Web 2.0 applications, etc. And it's really coming home to roost. You know, bottom line is social networking apps were detected in 95% of organizations today, 78% of Web 2.0 apps support file transfers, makes it trivial to move data in or out of your environment. Now, when I'm saying data and that could very well be malware. Two-thirds of applications associated with these have known vulnerabilities themselves and 28% of these applications were known to effectively be used to propagate malware. Not a very pretty picture being painted out there. You know, the bottom line is cyber criminals are targeting these social apps today, greatest opportunity for them is the amount of trust that end-users are putting into these social apps that imply trust, you know, and once they can replicate the malware with amazing speed and devastating impact within these social environments, all bets are off.
Now, we have a rather blatant trend here and, again, it all ties back to popularity. As we've seen the popularity of Web Apps increase, we naturally are seeing a distinct trend in the number of web app vulnerabilities. You know, I've always said myself that, you know, tracking vulnerabilities in software has very little to do with the quality of the software, it really has more to do with the popularity of that software. I would have to say that probably the best example of that would be Apple. You know, we've all heard of the increasing risk associated with Apple. It's not because they're writing less secure software, it's simply because their software has become more popular so it's a better target for the bad guys. Clearly as shown in this graph, we saw a huge uptake beginning at the end of 2004 which marked the, we'll say the birth of the popularity of web applications. And today, in 2010, we have a significant run rate in new vulnerabilities appearing in our web applications.
Again, just to drill the point home. Looking at another graph, this is specifically from another survey, the Verizon 2010 Data Breach Investigative Report. The vast majority of the exploits investigated were found to have involved web applications. If you look at the specific numbers, the attack pathway by percentage of breaches associated with the hacks is at 54% for web apps. And the bigger number, and was originally surprising to me, the percent of records lost, 92% were tied to web applications. Clearly, web apps had become the vector of choice for the bad guys in grabbing our data in the current environment.
Now, we need to absolutely talk a bit here about the sophistication level of attacks that we're seeing within the environment. You know, I really love this slide. You really had to have taken a little bit of time and read a little bit about the Stuxnet issue itself that we've seen here in the last couple of months, because it really was a great example of a well-written exploit. I mean, the target vector here was quite interesting. They had hosted a security conference, and at that security conference they had given away USB sticks that had malware embedded on the USB stick. You know, they did that knowing that surely some of the operators that they invited to this conference would plug those USB sticks into PCs when they returned back to work. And sure enough, they did. Now, once the USB sticks were plugged into the PCs it was a simple matter for the malware to begin actively looking for sharers that would be tied into this data network itself. They very quickly propagated the malware into the actual PCs used for the man-machine interface to this data network. Once they have the software on the PCs that were the man-machine interfaces, they were then able to propagate into the actual data systems themselves and literally build a rootkit into these data systems. So we have multiple vectors, we have multiple exploits all tied together into one very powerful piece of malware. Now, the scariest part about all of this is the simple fact of the matter that the malware associated with Stuxnet can now easily be found downloadable on the public internet. Now, yes, it is reverse engineered code it's not C-source code, it would actually be, you know, a bit of machine language code. But, again, it does make it a little bit easier for a bad guy to potentially plug the modules that were used from the Stuxnet exploit into their own exploits, as well as simply learning the methodologies for propagating malware and perhaps, you know, taking a cue from that learning and building it into their malware by and of itself.
So, again, you know, we seem to be getting much better at writing malware today as evidenced with Stuxnet. And they, again, are taking advantage of multiple vectors and are combining multiple exploits to get the job done and get into our environment.
Now, Abusing Unintended Consequences. This slide really has some special meaning for me today as a great example. You know, here we sit with another brand new Adobe vulnerability out there in the wild in flash, that's plugged into Word documents. I mean, who would have thought that we would have Adobe Flash issues with Word documents? We, in fact, do have them and it's actively being exploited today. Now, this particular slide shows the old Microsoft Jet exploit being embedded within a Word document. This goes back in time a bit but it does show that we can abuse unintended consequences by taking a vulnerability and plugging it into some form of a document or spreadsheet or a PowerPoint file, to help get it past that gateway. It does work, it works quite well, and again, we're seeing it today with a new Adobe Flash exploit currently being, you know, actively exploited in the wild where the flash has been embedded into a Word document. You know, just a few weeks ago, we were hearing about the same issue with a malicious Flash component embedded within an Excel spreadsheet. In fact, if you read the news that's what got the bad guys into RSA in the first place.
You know, it all boils down to where we're getting much better tools available out there for the bad guys. You know, this is Point Click hacking, if you do a little digging out there and take a look at the number of vulnerabilities that are bundled into these packages that are available out there for any 13-year-old with a cable modem connection to purchase on a public internet, you know, CrimePack is a very popular exploit plug. If you look over on the left side of the slide, it lists the specific applications that are included with crime pack. We also have Point Click malware design today. You know, we have high tech we have a number of unique Packer's out there. Packers that include the ability to evade anti-virus, packers that have the ability to detect when in fact software is running on top of a virtual environment causing that software to literally do nothing of any consequence because it might be being looked at by a researcher if it, in fact, discovers that it's being run on top of something like VMware. You know, most security researchers today do use products such as VMware, so it's a great heads up for the bad guy at protecting their software, if they can, in fact, detect it and block the perhaps generation of a signature by keeping the software benign, if in fact the text is being run on top of VMware.
Again, Point and Click malware design today, they can pack it up such that every version that gets pushed out onto the public internet is new and unique, no signature can possibly match it, and they have a significant number of, you know, security bypass capabilities built into it. Quite pervasive and getting very, very popular out there. Again, Point and Click malware design has become the norm on the public internet today. And, of course, you know, why buy an exploit kit when you can really just rent it? We're reaching a bit of a crossroads here on the net today. You know, we have all of our, you know, quality vendors out there talking about software as a service, while at the same time we have the bad guys talking about hacking as a service. You know, yeah, you could go out and buy that malware toolkit for $200 and, you know, do damage worth millions. But, hey, why bother when you can simply rent it? You know, you can rent a botnet today for well under a grand that could, you know, take complete control of an entire network, or at bare minimum infect them and paralyze them for days or at least douse them with damages that are simply uncountable. Look for continued growth in software, in fact, as…or I should say hacking as a service, on the public internet.
That being said, I'd like to jump back over to Sam here at the end of this slide and we'll have him read the poll. But let me go through the common denominator here very quickly. You know, why do hackers use old vulnerabilities to exploit systems? Simply put, because it works. I mean, again, in the Verizon breach, 90% of the exploits used for entry had patches available for six months or longer. There really is no excuse here, folks. The same study went on to point out that 50% of systems had 10 or more vulnerabilities for which patches are currently available for. Bottom line is we're not getting the job done with patching and it's really troubling for me. I mean, patching is low hanging fruit. There is no better return for your dollar in network security, in my opinion, than patching. The best way to mitigate the risk of a vulnerability is to patch it. End of story. That being said, I'm gonna turn it back over to Sam for a brief minute here to share the results from the poll.
Sam: Thanks, Paul. We'll share out the results right now. And the question was, "Do you rely solely upon WSUS for flaw remediation?" Twenty-two percent of you said, yes, 78% percent said, No. Paul, any thoughts or surprises there?
Paul: Well, I am in fact surprised in these, you know, just basic impromptu surveys I've done at conferences. I would have thought the number would have been lower for those who are using WSUS. You know, the bottom line there is WSUS is wonderful, it's a great free tool but it's only patching a fraction of what we need to be concerned with, it's not getting, you know, up to third party products. You know, it's surprising but in my view, it's a wonderful surprise. I'm glad that people are in fact getting something accomplished beyond just using WSUS.
Sam: Okay, thanks...
Sam: Yeah, you can keep going, please.
Paul: It sounds good. Sorry about that, Sam. We're gonna look now at some of the challenges, you know, within patch management by and of itself. Okay, bottom line here is that, you know, your environment is always gonna have all sorts of risk added every day and in different ways. We're constantly adding new applications, patching applications, etc., etc., across the environment so the environment is constantly in flux. You know, your software and your OS life cycle by and of itself assumes that new bugs are gonna be there. You know, you're gonna have design flaws that are gonna be discovered as technology is adopted and deployed. You know, on average, 15 new vulnerabilities are released per day and that's a very conservative number. And over 90% of vulnerabilities today could be exploited remotely which is very, very troubling by and of itself. You know, software vulnerabilities are a fact of life and they're gonna grow daily. Understanding these risks is really critical in, you know, defining your ability to address risk effectively.
You know, network and endpoint recesses today are really taxed for bandwidth. You know, we have increased usage of, you know, remote storage and processing time directly affects the bottom line. You know, IT organizations have less personal resources today to manage endpoint operations and security, to begin with. You know, it's also a problem with a lack of visibility and coordination between functional areas of IT organizations and the security impact and the ability to effectively manage organizational compliance and IT risk in today's expanding environment.
You know, bottom line here is that the old approaches clearly haven't worked. You know, we have disparate products we have processes which are simply expensive and require high management overhead. You know, today without centralized management and centralized reporting across your distributed systems, your platforms, and your applications, you simply can't achieve the operational efficiency and the cost savings that's really required in today's economy.
So, again, your best line of defense, to me, it's simply indisputable. You know, patching your client-side apps is your number one priority. The problem of unpatched client-side vulnerabilities is one of the two most pressing priorities organizations need to address to mitigate, you know, cyber risk. Most organizations take at least twice as long to patch third party application vulnerabilities than they do to patch operating system vulnerabilities. You know, this is at a time and place in the life cycle here where the bad guys take dramatically less time to literally create a Diff [SP} of them soft patch and generate reliable, repeatable exploit code. Again, it takes literally, today…you know, we've seen it in the wild, less than 24 hours to 48 hours, that's the window we're working with, to create a Diff of a Microsoft patch and generate repeatable exploit code. The window for getting those patches pushed out is shrinking, literally, on a daily basis. We have gotta do a better job of getting our patches pushed out faster and wider than ever before.
You know, looking at some best practices for patching in general, of course, you know, you got to identify all of those assets, you've gotta monitor external sources for vulnerabilities, you know, the list serves out there that feed the information up to us, and, of course, Microsoft. I personally don't believe anyone does a better job of providing us with information regarding vulnerabilities than Microsoft. And, of course, you know, scanning all your IT assets on a regular basis, looking for, you know, unpatched vulnerabilities and configuration changes.
Now, on the prioritization side of it, you know, you really have to maintain a complete inventory of your IT assets, it has to be accurate, you know, built essentially into a database. This allows you to prioritize the order of remediation as a function of risk. Of course, tight compliance and business value is a major component of it.
On the remediation side of it, you know, you've got to basically be able to stage and pass to your deployment, internally, you've got to be able to, you know, be able to handle automation, and where necessary be able to do it manually. You got to be able to train those admins and your end-users in vulnerability best management practices. And, of course, once you've remediated you start all over again. You scan to verify that in fact you've gotten it done right, and in fact the mitigation that you put in place has been effective, you, of course, would need to handle your reporting for audit and compliance. And, again, it's a continuous process it's not something that can simply be put on the shelf.
Now, Lumension and their endpoint security suite for patch and remediation does a great job of handling this. You know, again, looking at the bullets called out here, it handles your discovery, can fully automate it, scanning those machines to get a handle on what specifically is out there tied to the net both managed and unmanaged, handles the assessment, performs a deep analysis, looking through the OS, the applications, the specific security configuration and performs those timely vulnerability assessments. It allows you to actually tie values to the assets which can really help you in prioritizing your most critical security risks. Provides for automatic remediation, deploys your patches across an entire network defined by policy, supports multiple OSs and applications, and, more importantly, it handles both online and offline machines, and last but not least, it handles all of that critical reporting that management seems to thrive on from within a single management console.
Looking at streamlining it overall, again, the Lumension Endpoint Management Suite does a great job of doing it, I really like the single dashboard view, you know, it provides for management, definitely reduces complexity, helps reduce that total cost ownership, provide great visibility and control of your complete environment including your endpoints, provides that operational efficiency, and, of course, helps elevate that security compliance posture.
Now, we started at the very beginning of this noting that, you know, patch management is a core component for risk remediation. You know, the bottom line here is, and I wanna make this very clear, you know, patch management is incredibly important. It's not the Holy Grail, but it is an absolute core component of defense in depth for securing any environment. Again, in my view, some of the best investment you can make in risk mitigation today, resides within patch management. Again, the best way to mitigate the risk of a vulnerability, in my view, is to in fact apply the vendor's patch and eliminate the risk of that vulnerability. You know, of course we still have to face the issue of Day Zero Code and all of that fun stuff out there, but, again, you can eliminate the vast majority of the risk of known vulnerabilities by simply applying the vendor's patch for that vulnerability and literally eliminating the risk completely. That being said, I turn it over to Sam and we jump into our Q&A.
Sam: Thanks, Paul. Again, if anybody has questions please submit them via the questions tab at the top of the interface and we will get to as many as we can. Paul, I know we had Patch Tuesday, yesterday, so I thought that it might be good from a timing perspective to give a quick overview of that and what we're seeing coming out of yesterday.
Paul: Well, you know, it was a huge Patch Tuesday, yet again. You know, a few people joke around with the numbers being some form of a, you know, April Fool's Day joke, but it was no joke. You know, roughly 64 vulnerabilities were patched in what was rolled out by Microsoft. Interestingly enough, about 30 or so of those were tied to older code, you know, some Windows 32K, the access related vulnerabilities. And that was more of Microsoft giving credit where credit is due. A security researcher had found roughly 30 or so vulnerabilities in that old code base, and Microsoft had boiled it down to really three specific vulnerabilities, but in fairness to the researcher, published all 30 or so. I believe the exact number was 32. So I say good on Microsoft for giving credit due where credit is due. It did inflate their number, made it look a little bit worse than what it actually was but they did the right thing in giving that researcher credit and publishing all 32 as separate individual vulnerabilities, when in fact it tied back to just three.
The other interesting thing and good thing that we saw this Patch Tuesday, is we finally got a patch for that MHTML issue. You know, that vulnerability required significant testing by Microsoft because it ties to just so much that touches the internet today, so it did delay the release of that patch. We also did get the patch we've been waiting for, you know, for the SMB issue, glad to see that one as well. From a priority perspective again, you know, we've seen exploit code out there for MHTML and the SMB issue, so get that done right away. Follow that up with the other nine or so critical patches, and then follow those up with the important patches and you're good to go for this Patch Tuesday.
Sam: Thanks, Paul. Question in from the audience. "What is generally considered an acceptable goal of patch completion for a large enterprise? Is 100% an attainable goal?"
Paul: Well, I personally believe that it is. Again, if you've done the proper assessment, you know what's sitting out there on the wire, being able to get that nailed down and patched is not something that's unattainable, it, in fact, could be.
Sam: Okay. What about patching endpoints that are offline?
Paul: Again, that's one of the strengths of a good flaw remediation program. Having the ability to patch offline today is a critical thing. You know, that's built into products such as Lumension and something that I highly endorse.
Sam: Okay. Obviously, it's another question. "Obviously, patching itself carries the risk that it may cause outage if untested. Have there been any advancements in the reliability of patches to not crash systems? Obviously, testing increases the window of opportunity for an exploit to occur."
Paul: You know, I have not seen as many failed patches as we have historically. You know, and sure enough I've said that and we'll have one next month. But, you know, the frequency has gone way down on that. I know that most of flaw remediation vendors that offer patching solutions do provide some testing themselves, I do know that in most enterprise environments, they all tend to have some form of lab environment where they can test these patches. And, again, it is a necessary step to test it, you don't wanna throw a patch out there that's gonna break things. But, at the same time, fellows, we gotta make sure that we get these patches rolled out as quickly as possible to help reduce that threat window.
Sam: Okay. Another question, going back to the offline patching question is, how does it work?
Paul: You know, we could end up spending a day discussing that. What I'd like to do would be to follow up with a... We've got a white paper that we've put together on or offline patching, I'd like to submit that if you've got the email address of the person that submitted the question and we'll follow up with that.
Sam: Okay. Let's see. Another question is, "We currently have nothing in place to deploy patches and have in the past in had service fail to restart after patching? We had no way to test patching, so where do we start?"
Paul: Start by calling your local reseller for your flaw remediation product, because in general, they can handle all of that for you. Again, in an enterprise environment typically you are going to install a server that's gonna be used as the repository for the patches themselves. That same server can be used to host the software that's going to handle scanning, etc., within the environment to get a good handle on what's plugged in as well as to understand the current status of patches within the environment. So, again, you know, you're starting from scratch, your best bet is to talk to a reseller that has the knowledge of flaw remediation to help you with a proper rollout so that you don't hit any roadblocks.
Sam: Okay. When discussing... So you talked about the defense in depth, Paul. One of the...and there was a graph that had patch sort of in the middle there. Could you expand on that a little bit in terms of, you know, if we're saying patch is the best first thing you can do, what are some of the additional things and how do they work together?
Paul: Well, of course, you know, the patching for me is a great first step. Again, the best way to eliminate the risk of a vulnerability being exploited within your environment is to patch the underlying vulnerability and fully mitigate the risk. You know, you, of course, have to follow that up. You know, patching by and of itself is not the Holy Grail, you still have the issue of Day Zero exploits. So, of course, you know, we're gonna do our normal AV trick at the gateway using that as a screening filter for stuff that's coming down the pipe. But personally, I like also to use whitelisting on the desktop just as a stopgap to catch anything that might be Day Zero related that a patch had not been released for, and then, you know, I'll typically I'll use AV to help clean up beyond what might be discovered by whitelisting.
Sam: Excellent. Just trying to read through the questions here. You talked about free patch tools. We asked the question about WSUS, I know there are lots of other native updaters available. You know, why should someone go...you sort of touched on it but if you could dig in a little bit deeper, why should an organization look at a solution such as what Lumension has, compared to, you know, some of the things that are, "Free out there."
Paul: Well, you know, the free stuff is great but unfortunately most of it is fairly vertical. You know, it silos things for you, you know, you're gonna find an updater for Adobe, you'll find an updater, will say, for Oracle, or an updater from Microsoft, etc., etc. But it all relies on you coordinating everything, which can become a huge, huge issue. I personally find it best to look for a flaw remediation solution that actually covers the applications that I run in my environment and, you know, that's really a primary factor for my decision-making process.
Sam: Excellent. With that, I would actually like to close up the Q&A. Any questions that we weren't able to get to, we will follow up with you via email, personally we'll have a product manager follow up with you in the next....by next week at the latest, with answers to your question. Before we close out, just a few additional resources that Lumension has made available. You can learn more about Lumension patch remediation by going to the link as shown on the screen here. It's a brief overview of the solution and how it fits into the endpoint management and security suite that Lumension provides. There's also a vulnerability scanner tool that will help identify all the vulnerabilities on your network. You just download that and run it and it'll provide vital information to you in terms of your patch remediation process. And then also a couple third party research reports, one from Forrester. Their Wave report on vulnerability management vendors, and Tolly which is a third party consulting group, they did a TCO comparison on a few patch products.
And with that, I would like to remind everyone that we will have a recording of this webcast made available shortly on, lumension.com. We will also make sure to send out a copy of the slide deck to each of you who have attended. Thank you so much, Paul, for your time. Thank you, everybody else for joining us today. Again, if you have any questions or more…or would like more information, on Lumension or any of Lumension solutions, you can contact us by gonna, www.lumension.com Checking out our blog where Paul, the frequent blogger, at blog.lumension.com, and he has some great information there on every Patch Tuesday. You can call us at 1888-725-7828. International callers please review the appropriate number on the, 'Contact Us' page on Lumension's website. You can also email us at, [email protected]
Thanks again everyone, for attending, and this now concludes our webcast.