Bridging the Patching Gap in Microsoft System Center

June 25, 2015

System Center Configuration Manager is the solution of choice for managing today's enterprise environments. But what’s an administrator to do when a security audit includes vulnerabilities in non-Microsoft applications and server platforms? Introducing the only enterprise-class 3rd party patching solutions designed exclusively for System Center environments that require operationally efficient and reliable security and compliance patching solutions. Join us for a discussion of practical methods to improve and automate the patch process for your most targeted 3rd party desktop applications and non-Windows server platforms, reduce the time you spend building updates, and improve audit readiness in your Windows environment.


Melissa: Hello everyone and welcome to today's presentation, Practical Patch Compliance and Using System Center to Reduce Your Audit Pain. My name is Melissa Russell I'm the Marketing Programs Manager for Lumension, and I'd like to welcome you to our presentation today. Just a few housekeeping things before we jump right on in. We are recording today's session so if you have to jump off early or you wanna make sure you're able to view it later we will make that available. And we will also make the slides available too. If you do have any questions, please go ahead and put them in the Q&A box and we will address them as time allows towards the end of the presentation. So, at this time I'd like to go ahead and turn it over to our presenter, Russ Ernst who is our Director of Product Management. Russ, go ahead.
Russ: Thank you, Melissa. As Melissa just mentioned, my name is Russ Ernst, I'm the Director of Product Management here at Lumension. Responsible for our patch and remediation solutions including all the patch content that we produce and supply it through our numerous patch channels including our plugins into Microsoft System Center. So today we'll be talking about bridging the patching gap and using practical patch compliance, to reduce audit pain and to patch points through Microsoft System Center. So really today I wanted to kind of to talk through a few items, first off, I wanna kind to set the stage around the security landscape and talk about three things to consider as we consider our overall endpoint defense-in-depth. Focusing our internal factors that such as risk tolerance, as well as external factors, the threat environment, and the attack framework. Then also talk about some back to basics techniques that we should use to think about our overall patch practices, talk about our patch best practices and then finally end up with little bit of a wrap-up.
So first off let's go through our security landscape. And the first question you want to ask yourself is, you really need to understand your security, your organization's level of risk tolerance. And in determining that level of enforcement that your organization needs, will best prepare you for a successful implementation of your overall patch remediation solution. From our experience, organizations typically fall into one of three different categories they are permissive, moderate or stringent. And what we mean by that is from a permissive standpoint, these organizations are looking to do a very little in terms of overall enforcement. Primary goal here is to usually allow for auditing and reporting of user activity or the need to address some very specific issues such as limiting access to USB-connected removable storage devices. 
The next category's what we like call moderate, and what we find is actually in most organizations fall into this category. These organizations typically have some written security policies and want to enforce that policy without relying on voluntary user compliance. So, this goal is to prevent any specifically unauthorized usage and allow for some flexibility in some permitted usage cases to resolve in maximum organization productivity. So, these organizations typically have some level of external audit or compliance needs that they must address. 
The third bucket is what we like to call this stringent category. So, these organizations typically deal in highly confidential information, typically very closely monitored either from within the organization or by an external authority. This is where we typically see a Federal offices fall into this category, and this is an area where we typically see device usage that's restricted to encrypted devices and every file transferred to or from this device could be retained or reviewed. So again, determining which category your organization fits into really provides that baseline perspective when determining how to configure your security policies.
Keep in mind here, the overall goal of the bad guys and we're talking about what level of security we want to have in place is the endpoint. The endpoint is the target, it doesn't really matter what that delivery method is for these security vulnerabilities whether it's by phishing, or by drive-by, or by an especially configured Word document, whatever happens, to be. The bad guys are really trying to get at the endpoint. So, our goal when we talk about this is to change the timeline by disrupting the attack methodology. This threat framework or the kill chain that we're gonna outline here a little bit is to make it basically put some roadblocks in the way of the bad guys so organizations have much faster time to react to these types of attacks. So, let's take to look at what we can do on the endpoint to disrupt that timeframe by first walking through what we mean by this timeframe.
So, when we talk about this attack and response timeframe what we typically see is an overall timespan of events, this is taken from the Verizon data breach investigations report. They did a very good job of talking about what this overall kill chain happens to be when you talk about the time it takes for bad guys to compromise a source to the time it takes bad guys to start to exfiltrate some data. Then the time it takes the organizations to actually discover that there's been a breach, and then the time it takes the organization to contain that breach. And when we look at that overall kill chain, we were seeing that what wasn't reported by Verizon is that 84% of those compromises take seconds to hours. So, if you've got a dedicated bad guy who's looking to target a specific organization, you're talking you know, again seconds to hours that they're going to start, that they're gonna have a compromise in that organization. 
Sixty-nine percent of exfiltrations take seconds to hours, so once there has been a breach, you'll think you're looking here at almost immediately when cybercriminals start to remove data from that breached organization. On the flip side of that, you actually see that the organization's taking days to months to even recognize that there's been a breach. So, although the exfiltration starts in hours if days to months 85% of these companies take days to months to actually be discovered. 
And then finally 77% of containments take days to months. That's a huge shift in terms of how fast the bad guys are acting to, how slow the good guys, the organizations themselves are to respond. So, what we're talking about here, when we talk about patch, and patch best practices, and a value of patch best practices is really the most highly used remediation technique to block the holes for these bad guys is to keep your systems fully patched. So, at the end of the day patch disrupts the overall kill chain.
So, with that let's go talk about a kind of a back to basics approach. When we look at the overall number of annual reported vulnerabilities since MITRE, the MITRE database is starting to collect these. We actually saw a large increase in the number of total vulnerabilities reported starting back in a mid-2000s then we see things starting to level off about through the 2000s into the early 2010s. But as of last year, there was a huge resurgence in the total number of vulnerabilities reported, tapping out at an all-time high of nearly 8,000 vulnerabilities. And these vulnerabilities are still on track for the first half of this year to stay on that increase number throughout 2015 as well. So again, these are the total number of vulnerabilities this isn't just Microsoft, this is Microsoft, this is across Linux, and UNIX platforms, and MAC, as well as all third-party applications being reported back through the MITRE database. So, vulnerabilities are on upswing.
And to pair this, we're gonna drill in a little bit here too, if we look at just what the security bulletins that were released by Microsoft over the last few years. If you look at this 2012 trend line that's the... or I'm sorry the 2013 line was way high versus the number of bulletins released by Microsoft in 2014 that's that blue trend line below. Microsoft was actually on track to be at a three-year low over the last three years except in November of 2014 when they released an all-time high of about 15 bulletins that pushed them over the 2012 line.  
But what's interesting here is that in 2013 the number of vulnerabilities accounted for by Microsoft of the total number of vulnerabilities out there was about 10% and actually what we saw in 2014 is that, that overall vulnerability shared by Microsoft dropped to just over 6%. And what that means to me is, although the total number of vulnerabilities increased in 2014 we saw that on the last line to almost 8,000, Microsoft's share on that and actually decreased. So, what that says to me is that the importance of patching not just your out of the box Microsoft updates has been dramatically increased because the bad guys are really targeting the non-windows platforms and pervasive third-party applications on the typical desktop. This is identifying the gap right there, right? When we talk about bridging the patching gap, it's really defined by this, these last two slides. When we talk about how you know, out of the box with System Center especially, yeah, that provides all of your Windows and your Microsoft application updates but that's not really where the bad guys are going these days. 
So finally, this is the summary of kind of the back to basics approach. When we look at the SANS Institute really what it comes down to in knowing your environment to provide these best practices from a back to basics approach, know your devices. Make sure you've got some discovery inside your environment to know what's even connecting within your network. Know your software make sure you have thorough inventory collection going on in your environment. Make sure you've got secure configurations and make sure you've got continuous vulnerability assessment remediation, this is where we're gonna spend them majority of our time today is talking about these patch best practices really around that number four item continuous vulnerability assessment and remediation, remediation being the patch. And then finally malware defenses, that's your typical AV. So, these are the most basic security controls that you can have in your environment and if you've got these ticked off in your...if you're a systems administrator and you've got these ticked off and you're gonna be looked at as a hero from your security team.
So, let's see how these actually translate into patch best practices. So, we have these actually laid out into four generic categories what we like to call "Laying the Groundwork," "Before Patch Tuesday," "On Patch Tuesday," and "After Patch Tuesday." Now before I even get too far, I expect to see a message or a question pop in here and say, "Well what about Windows 10? Does Patch Tuesday go away?" And we know a lot of those things are gonna be changing here once Windows 10 is released at the end of next month. But realistically there will always be an Update Tuesday depending on which tier or which channel or update channel you're subscribed to. At least through the end of support for Windows 7 which is all the way out to 2020, Window 7 updates will still be released on a Patch Tuesday schedule. 
So, Patch Tuesday absolutely is still important if for no on that reason than to set a monthly cadence for your patch best practices. Realistically would it be great if you had your patches rolled out every two weeks? That would be fantastic, or even on an automated fashion, we can talk about that in a little bit here too. But if you're even getting to a monthly cadence marked by the second Tuesday of the month, you're gonna be better off than a lot of organizations out there. So, we still like to call this our groundwork for the patch best practices, laying the groundwork before Patch Tuesday, and on Patch Tuesday, then after Patch Tuesday.
So, as we talked about in laying the ground work, first thing you want to do is just discover your assets. I mean, identify all the hardware and software on your network and categorize them by platform, application department, whatever works in your environment. Start to do some maintenance, agent maintenance on those endpoints, make sure that all the assets in the network have been fully installed with an automated patching solution. Install any new patch management agents or your Windows System Center client make sure that's fully functioning on those end points. Make sure that Windows update is configured to receive those updates. So. this is all around again, in laying the groundwork this talks to those first two back to basics from the SANS Institute. Know your environment, know what's on those devices.
It's also important to start to classify your value and risk. Determine which of those systems are the most critical to protect based on the assets that are housed or the function that they provide, to define this level of risk by criticality of the system and help and prone it is to attack. Remember, the highest priority machines are gonna be where the data is located. This is all around understanding your environment. And this needs to be done in cooperation between IT and the business, right? IT understands the network, IT understands the assets that are in the network, but they really need to be working with the business to understand where is that business-critical information stored so that they can have the highest priority or the highest security levels on those machines. 
Now, that doesn't mean you're gonna be targeting those machines to be patched first, but you wanna make sure that those machines are patched in an expedient fashion after the patches are fully tested.
Also, while laying the groundwork, make sure that you have staff training. So, this is training your applicable staff on the vulnerability monitoring and the remediation techniques. Make sure that they understand the value of keeping those systems fully patched. Use some Lumension learning resources, we encourage you to go out to other learning portals or to start your own internal learning portal to understand the value of keeping those machines patched. It's gonna be a lot easier to explain or reboot in the middle of the day to your workforce when your users actually understand the value of the patches that are being applied than it is they just get a reboot notification or worst case those machines just reboot without notification in the middle of the day. So, this is all about educating your internal users, educating your workforce, and educating your executive team on the importance of a patch management process.
Next, you wanna start scheduling your resources. Make sure you've got your allocated IT resources for Patch Tuesday, integrating your additional patch release schedules from third-party applications, making sure those are being monitored from Adobe, Apple, Java, etc. Keep in mind that most of, like we talked about before, most of those vulnerabilities are now being opened up against third-party applications and non-Microsoft platforms. So, understanding do you have any Mac desktop in your environment? How much of your data center is non-Windows? Start reserve some downtime for those data center servers. Make sure you understand within your environment that those data centers typically have maintenance windows. So, start to schedule those updates between those maintenance windows. 
Confirm that your reporting's up-to-date, this is a key piece to this overall patch management best practice is that, making sure that you have buy-in from the organization on what it is to have a successful patch management process. Whether that's being 95% patched within 30 days or whether it's 75% patched within two weeks, or it's 100% patched in one-quarter. That up to you to define, again it goes back to whether or not your organization is more open or more stringent or more moderate. And then finally, keeping those endpoints up to a certain baseline, so deploy any missing updates, or your missing service packs, or missing prerequisites to get you to that baseline.
And then finally as you're starting to finish laying the groundwork, study this information and security briefings that we find from the different supported vendors. Now again, this is an area where we know that it's gonna be changing, it already has changed from Microsoft's perspective when we know that Microsoft used to be very good about providing some information about what was going to be released the Thursday before Patch Tuesday, Microsoft has now ceased doing that advance notification. But they still provide, Microsoft still provides a lot of very in-depth detail about the content that has been released on Patch Tuesday itself as part of that Security Response Center. 
So, it's still a very good idea to make sure you're up-to-date on what all this information is. This is typically the last thing that happens before Patch Tuesday so this getting into that before Patch Tuesday bucket. Start to understand what are these that you see in the hopper, we know that a lot of, from our experience as well, on the week of Patch Tuesday, you can expect to see updates from Adobe. We've already seen nine updates this year alone just on Adobe Flash Player. So, you want to make sure you have some time budgeted to get the Flash Player plugin patched in your environment. You can expect to see a quarterly update from Java JRE, our goal has actually been pretty consistent about getting the Java JRE update done on a quarterly basis. 
And then you can expect to see at least a monthly update for your browsers whether that be, of course, Internet Explorer being the most targeted Microsoft application month in and month out with an average of 20 vulnerabilities, at least against Internet Explorer every month. But also, you're gonna look at you know, your Chrome or your Firefox browsers also being updated basically on a monthly schedule. So, start to understand what the security impact is to your environment on that monthly basis.
So, you've seen now you start to lay that groundwork, you started to do some before Patch Tuesday work and on Patch Tuesday start to understand what has been released. This is where we start to do that prioritization. Start to understand the threat level itself, when that information, that vendor information's been gathered in the last step, start to use some patch impacts, you know? Understand if it's a critical security patch that's probably gonna be at the highest level. Start to understand those threat levels. It's really important to know if there are any known active exploits. Again, from all the major vendors Microsoft, Adobe, Oracle are very good in their released notifications in saying which of those applications actually have or which of those vulnerabilities actually have, known active exploits. And if you've got a known active exploit in a vulnerability for a pervasive application that update, that patch has got to be at the top of your list because again, that's that getting back to that seconds to hours for compromise as soon as the bad guys see that in one of their attack kits it's basically an open door to get to your data in your environment. 
So, this is where you really need to understand what the threat level, these risk of compromise, the known active exploits, and again in the information a lot of times the vendors will also provide the consequences of compromise. Meaning, understand how easy it is to exploit and then understanding if that machine actually houses your critical data it gets back to why we went back and did that prioritization exercise at beginning of understanding you know, the level of importance of those machines in your environment. If that machine is applicable to that update, again you better make sure that's at the top to your priority list to get out. So, this is all about prioritization.
The next step we see here is all around deployment. And yeah, I've got you know, a simple easy button deployment button here but deployment is probably one of the most complicated aspects of this entire process. You gotta think about your change control, make sure you're following any internal planning or approval processes for your patchment deployment process. Make sure you're doing some staged testing you know, this is incredibly vital because although we know in Microsoft actually as part of this whole changeover to the patch process for Windows 10, has been very open about their overall testing process. Each update, each bulletin released by Microsoft goes through internal testing on tens of thousands of machines. Then it goes to their Windows Insiders Program which goes to hundreds of thousands of machines then it goes out to essentially a consumer network of millions of machines. 
So, each of these updates has, for even though we hear of one or two of these updates causing some blue screens, it's been through pretty thorough testing. However, in all that testing none of those environments is gonna exactly mimic your own internal environment. So, it's incredibly vital to have this staged testing if you don't have the ability to have an actual lab in your environment to do the testing which most organizations don't have that luxury you have to identify a certain number of machines that mimic that those critical machines. Have the same applications running, have the same kind of load on those machines and deploy your patches to those production machines first. 
Make sure you've got some scheduled deployments. Make sure you've got a staged rollout, the staged rollout and this is incredibly vital too. So many different patch solutions, you can actually start to automate this patch process where if after a couple of days if no errors are reported from those test machines auto deploy out to another tier of machines internally. And then auto deploy out to a third tier of machines internally. Start to identify some non-critical departments that can be fully patched first before they go out to your critical departments. These are all vitally important before you actually start to patch your set, your business-critical machines. 
Make sure you maintain a deployment history, this is actually incredibly important when you start to talk about any kind of understanding of forensics if you have any machines that have gone down. You need to understand, you have some common machines that are experiencing the same issues. Is the same type of application running on those machines? So, maintaining an accurate [audio skips] is vital. And then finally, start to review the success and failure results. Start to understand, are you getting any customer or your end-users, are they complaining about the patch load? Are they complaining about the reboots in the middle of the day? Are they complaining about the patch schedule? These are all vital key points to make sure you've got a successful patch deployment process.
So, the next stage here is, this is all about continuous improvement. So, we've gone through and we've done our laying the groundwork, we've done the before Patch Tuesday, we've done our on Patch Tuesday, and now we start to get up to this after Patch Tuesday. So, after Patch Tuesday is when you're starting to get to all of the reporting and the continuous improvement processes this is all about you know, your checks and balances. Review the patch effectiveness. How many machines came back as not patched and why? It is just because you had end-users delaying the installation of the patches? Calculate that time to deploy, this is where you can start to understand about how you're actually filling those gaps and then what's provided out of the box just from Microsoft. Is how much of your end machines are still not patched for some critical vulnerabilities especially in your third-party applications? 
Start to monitor for compliance, this comes back to that reporting where you have an understanding from your executive team on the effectiveness of your patch process. How effective is your patch process? Are you at that 95% patch within 30 days? Are you above that? That's great, you can start to look like a hero. And then start to talk about where you want to improve those metrics. Maybe you want to set that goal at 90% first and you're coming in at 88% of what have you. Make sure you're continuously improving this overall patch management process.
And I can't emphasize enough the importance of these most prevalent and targeted third-party applications. When it comes down to it, you really need to keep your eye on these third-party applications that are resident on the endpoint. You may not even realize that like VLC media player for example, if you've got users that are allowed to install software on their machines and they want to watch cat videos in the middle of the day and they're using VLC media player to watch their cat videos, you know that's a targeted application. There are seven vulnerabilities opened up against it last year so, make sure that these third-party applications are fully patched. If we actually look at the most prevalent applications on a typical desktop, 27 of those top 50 had at least one vulnerability opened up against them. 
When we actually look at which of those applications are third-party applications versus native Microsoft applications, only a third of them or 34% of the top 50 are third-party applications, but those third-party applications accounted for three quarters of the total number of vulnerabilities. And if you actually look at that, those applications you know, it's the usual suspects, it's the browsers. Although we know that Internet Explorer's highly targeted, we also know that Firefox, and Chrome, and Java, and Adobe Reader, and iTunes, and Flash Player, these are on nearly every desktop machine that are in your environment. Whether it's Windows or Mac, right? Because a lot of those Mac desktops are gonna have reader or iTunes, or Flash Player installed on them also. So, make sure you really have a keen understanding in those third-party applications in your environment.
So how can Lumension help you? Especially if you're running System Center, especially if you're running System Center in your environment, Lumension has a number of solutions to help you out here. So, the overall Lumension for System Center Solution, we basically have coverage from the data center to the desktop and we have different products to handle either the data center application or the data center servers or the desktop applications. The data center servers are really focused on the non-Windows OS's, so we have a plugin into System Center to in extension into System Center that's integrated to patch or Linux and UNIX as well as your Mac desktops all through this solution. And to be able to provide a patch compliance report through this single SCCM pane of glass. 
On the desktop side, this is on the Windows desktop side, we have an automated plugin to patch third-party applications so it support for the broadest range of third-party applications out there. And we fully leverage the System Center infrastructure to follow all of System Center's scalability. And any update that's provided by Lumension will automatically show up in your existing System Center workflow in your System Center reports without the need for any additional endpoint component.
As we start to drill into what we mean by the broadest range of third-party content it's beyond the typical readers, and Flash Players, and iTunes, and Java, and the browsers, but we start to get into some of those utilities as well. I mean, how many of your users actually have 7-Zip, or maybe VNC, or WinZip, or Wireshark on those endpoints? Probably quite a few actually. So, we want to make sure we provide those updates for those applications as well.
You also want to be sure you're using Enterprise class content. We know that System Center provides the System Center Updates Publisher which is actually a great custom content utility for creating your own custom content to distribute in your environment. But if you actually look at some of the catalogs that are provided by vendors such as Adobe provides a catalog to patch just reader and Flash Player. You want to look at the type of content that's being distributed and what is the value of that free content. So, if we actually look at a side-by-side with a patch that Lumension created side-by-side with a native Adobe patch that's provided through their free catalog, make sure you look at some of the extended content descriptions, make sure you've got the supersedence information in there, make sure you've got the CVE IDs. 
But really when it comes down to it when the rubber hits the road, make sure you've got some Enterprise class fingerprint technology. And if you look at the updates that are provided through the free Adobe catalog, their assessments are based on only off of a single registry key. We know how fragile that can be, the registry can be mucked up with either by the end user or it can left in a corrupted state with a failed install or uninstall. So, the Lumension fingerprints actually go in and look at the specific file that's in place that's being called out by the update itself to make sure that it's there for the actual assessment to be accurate.
So finally, we want to look at the, what we call our Patent-pending Patch-Smart technology. So, this is some technology that we created in-house to make sure you've got a greater first-time patch installation success. And this technology's all around normalizing the installation process across all of our supported third-party applications. When we created this catalog years ago we found that the number of third-party vendors out there are very inconsistent in the way that they create their installers. So, we're providing some normalcy in that installation process to turn off the auto updates, to remove any bloat where that might be riding alongside the Java updates for example. To provide just to the update that you need to resolve the vulnerabilities in the environment.
So, in conclusion, I know we've talked about a lot of things today in talking about bridging the patching gap and providing some patch best practices here, but really kind of boils down into three basic steps. First and foremost, make sure you get some alignment in your organization. Know patch process is gonna be successful unless you have buy-in at all stages of the organization from the executive team all the way down to the end user. Make sure it's well documented. 
Educate management, educate your end users. Step two educate. Make sure there's awareness of the importance of patch processes in your organization. And then finally, implement the patch process. Rinse and repeat, make sure you've got something, a metric to achieve, continually monitor that metric.
So, we're all done with our talk today on our bridging the patching gap and then walking through some patch management best practices to hopefully reduce your audit pain. If you'd like some more information, we do provide a free LPM DeskTop Evaluation, this is an evaluation of Lumension's patch manager desktop plugin into Microsoft System Center. It's a fully functional product with all of our Adobe content. And once you are a subscribed user you'll get access to the full catalog with the free evaluation and to all of our Adobe content. 
So, you can do your own comparison with the free catalog that's out there. We also have a free Patch Scanner tool as well, if you wanna go out to our website. Download this cloud-based utility, it allows you to do an assessment on your endpoint, completely free of charge on all of the updates that we monitor so that can give you a very quick score actually on how patched your environment is up to 50 machines. So, for more information about that, I definitely recommend you try out those resources and those links will be active in the slides that'll be made available. So, at this point I'm gonna draw this to a close, do we have any questions that we want to cover?
Melissa: I haven't seen any questions come in yet and we have run a few minutes over our scheduled time, but if you do have questions please go ahead and submit them now. If you we see something come in here in the next minute we'll go ahead and address it, otherwise, if you don't have any questions we thank you again for joining us. There will be again, a follow-up email sent to you with all the information. And we greatly appreciate you taking the time out of your day to join us. So, with that, I'll just give it a moment and see if anything comes in and we'll take it from there.
Russ: All right. Well, follow upI'm not seeing anything come in. So, with that, let's go ahead and draw this to a close. Thanks for taking the time to talk or to listen in to this webinar today. Again, if you do have any other questions feel free to submit them or respond back to the notification that goes out at the end to this webinar. And I believe Melissa might have some more details about that.
Melissa: Yes, so we will be senfollow-up email with a link to the recording. And we'll be making available the slides and if you wouldn't mind giving us a rating on the event I'd greatly appreciate it. And, oh there is a question.
Russ: Actually, there is one question that just popped in, it says, "This product will install Linux OS patches via SCCM." Well, I talked about two different products. There was the Patch Manager Data Center and that is our plugin for Linux and UNIX as well as Mac OS's within System Center. We also have the third-party plugin for what we call Patch Manager Desktop. And our Patch Manager Data Center Solution will install Linux OS patches via SCCM, that's correct.
Melissa: Thank you, Russ. Just give it just another quick moment to see if there's any other questions. And I'm not seeing any, so again everyone thank you so much for joining us today. And thank you, Russ, for the presentation and we hope to see you again on the next webinar.