Main Nav

Message from dsarazen@umassp.edu

Hi All,

 

Quick Question: If Windows were to release a critical patch for a server today, how long should it take to install the patch before you’d consider it TOO long?

 

Thanks, 

 

:: Daniel Sarazen, CISSP, CISA

:: Senior Information Technology Auditor
:: University Internal Audit
:: University of Massachusetts President's Office

:: 774-455-7558

:: 781-724-3377 Cell
:: 774-455-7550 Fax
:: Dsarazen@umassp.edu


University of Massachusetts : 333 South St. : Suite 450 : Shrewsbury, MA 01545 : www.massachusetts.edu

 

Confidentiality Note:  This email is intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information.  If you are not the intended recipient(s), any dissemination, use, distribution or copying is strictly prohibited.

 

AttachmentSize
image001.gif1.84 KB

Comments

 

That depends on the patch and our exposure to it. Every patch Tuesday, we review all patches with our desktop and server groups and develop a game plan for the patches as a whole, and identify any particular patches that require special attention.

 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Brian Basgen

Director of Client Services (Acting)

& Information Security Officer

Pima Community College

Office: 520-206-4873

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

On 03/23/2012 02:04 PM, Sarazen, Daniel wrote: > Hi All, > > Quick Question: If Windows were to release a critical patch for a server today, how long should it take to install the > patch before you’d consider it TOO long? > > Thanks, > > Description: http://media.umassp.edu/pix/mail/umass.gif > > > > :: *Daniel Sarazen*, CISSP, CISA > > :: Senior Information Technology Auditor > :: University Internal Audit > :: University of Massachusetts President's Office > 2 1/2 hours? Is this a trick question? I'm assuming worst case (old underpowered hardware with not enough RAM and a fully loaded link because everyone else is trying to download the patch and install it at the same time). Sure, it *ought* to take no more than 1/2 hour (at the very most) but this is Microsoft you're talking about after all. ~c -- Charlie Derr Director of Instructional Technology Bard College at Simon's Rock
Message from dsarazen@umassp.edu

Sorry. I worded that poorly. I wouldn't expect that staff is sitting around waiting for critical patches to install. What I would expect is that there's some defined standard, i.e.: "Critical and Important Patches must be reviewed and installed within 1 week." When I was in public accounting we used to ding them if it took more than 2 days, but I'm not certain that's a reasonable standard for higher education. Thanks,
The correct answer is ... It's TOOOO long if the compromised is out in the wild It was estimated that the RDP compromise would take about a month to appear in the wild, they were off by about 2 weeks ... so, if you played the odds and waited, you are getting screwed right now :-) I like to say ASAP is the right answer, every day that you wait, you are taking a chance that the bad guys will win - you don't want to break your servers, but aren't they broken if they get hacked? My 2 cents Joel --On Friday, March 23, 2012 2:04 PM -0400 "Sarazen, Daniel" wrote: > Hi All, > > Quick Question: If Windows were to release a critical patch for a server today, how long should it take to install the patch before you'd consider it TOO > long? > > Thanks, > > [cid:image001.gif@01CD08FD.E6C2DA10] > > :: Daniel Sarazen, CISSP, CISA > :: Senior Information Technology Auditor > :: University Internal Audit > :: University of Massachusetts President's Office > > :: 774-455-7558 > :: 781-724-3377 Cell > :: 774-455-7550 Fax > :: Dsarazen@umassp.edu > > University of Massachusetts : 333 South St. : Suite 450 : Shrewsbury, MA 01545 : www.massachusetts.edu > > > Confidentiality Note: This email is intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. > If you are not the intended recipient(s), any dissemination, use, distribution or copying is strictly prohibited. > Joel Rosenblatt, Director Network & Computer Security Columbia Information Security Office (CISO) Columbia University, 612 W 115th Street, NY, NY 10025 / 212 854 3033 http://www.columbia.edu/~joel Public PGP key http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x90BD740BCC7326C3
Message from r-safian@northwestern.edu

No more than 48 hours…less time if there is an active exploit and no reasonable means to mitigate the attack.

 

I think you're looking for a general rule of thumb when it really depends on your environment. This is why auditors with checklists who don't know how to weigh the mitigating factors and overall risk only cause more headaches. Consider factors like: a) The nature of the vulnerability, is it denial of service or remote code execution for the server OSes in question Sometimes patches are code execution for 2003 that are just DoS for 2008 R2 b) What's the scope of people who could exploit the vuln? If the exploit requires credentials and only a handful of people can get to the server thanks to network acls or firewalls, then the risk is lowered. If it's remotely exploitable, doesn't require user credentials or any user interaction and the service is exposed to the Internet, then the risk is much higher and so the patch time should be now. c) Have previous patches to the server caused issues for third party software that either runs on or depends on the server? We've had issues with patches breaking third party software so we're more likely to test first before rolling out a patch for those specific servers. We also tend to isolate those servers so that the scope of people who could attack them is lowered, see b). And testing may include working with other departments who depend on the third party software and can take a while. d) What business processes depend on the server and what mitigations are available for the vuln? Sometimes adding more network isolation, making a small configuration change or increasing monitoring may be a better compensating control then rebooting a server outside of it's defined maintenance window and disrupting the business processes that need the server. Ted Pham Information Security Office Carnegie Mellon University ________________________________________
Too many variables to give a time but ... As many of us know, and any student who's done well in CCDC, patching is not the only way to secure a system. It's all about reducing risk. For MS12-020 you could have disabled RDP on your systems, or used a firewall to block RDP connections from all but your management systems. As far as Critical goes, that depends on your environment. Just because Microsoft rates it as critical doesn't mean it is for us and just because they don't rate it as critical doesn't mean it isn't for us. Also, all critical patches aren't of the same level of risk. All of that being said, I don't know that we have a defined standard but it's likely not as fast as it should be. Ben
Message from dsarazen@umassp.edu

Ted, Thanks for the feedback. If YOU were the auditor, and the department had no standards, procedures or documentation, how would you review the patch history? It sounds like the best way is to go over the patches that weren't installed and ask them what their rational was. Could you think of a better way to test them? I'd love to tell my audit director that the server admins use "The Force", but I don't think he'd consider that an adequate control. Thanks again
Message from valdis.kletnieks@vt.edu

On Fri, 23 Mar 2012 14:52:33 -0400, "Sarazen, Daniel" said: > Thanks for the feedback. If YOU were the auditor, and the department had no > standards, procedures or documentation, how would you review the patch history? Wrong question. You want to review why there's no standards, procedures, or docs. :)
Message from dsarazen@umassp.edu

I'm WAY past that ;)

Sent from my Android phone using TouchDown (www.nitrodesk.com)

If the department has no standards, procedures or documentation, there is no need to review further. Auditors require controls. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Brian Basgen Director of Client Services (Acting) & Information Security Officer Pima Community College Office: 520-206-4873 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It really depends on the circumstances. In general, though, I rely on a very structured monthly patching process for Windows servers that patches the servers soon after patch Tuesday. In that process, I have one of my administrative assistance check reports from our central patching server and post the results in our work order system. I then review the results so there are two sets of eyes checking the patching process and the process is documented. I close the work order or, if I am not here, my person who is responsible for our Windows patching server. Barron Barron Hulver Director of Networking, Operations, and Systems Center for Information Technology Oberlin College 148 West College Street Oberlin, OH 44074 440-775-8798 Barron.J.Hulver@oberlin.edu http://www2.oberlin.edu/staff/bhulver/
> It's TOOOO long if the compromised is out in the wild That is, I believe, the definition of a "0-day": The "patch window" between discovery of a vulnerability and appearance of an exploit "in the wild" is of zero length, usually because the vulnerability was identified (by vendor or researchers) only by reverse-engineering some new exploit.... ... I believe it is the task of auditors to verify two assertions: 1. The institution's policies/procedures meet its business needs. where "meet the requirements of thus-and-such standard" is often accepted as a minimum set of requirements, necessary but not necessarily sufficient to satisfy this assertion, and 2. Actual operations conform to those policies/procedures. So to pass an audit is going to require at least (a) a written standard/policy, and (b) an "audit trail" logging actions performed with time and date. If you don't have both of those, you shouldn't pass an audit. Ever. Once you have them, then it's time to look closer at #1 and at the actual needs of the business -- whether regulatory compliance is mandated by legislation, what degree of risk is acceptable, and so forth -- stuff where Senior Management makes some choices, even if the choice is only whether to remain in business or not.... David Gillett, CISSP CCNP
So, if I wait a month or two before installing patches normally and the exploit comes out before the end of my normally (very lax) install cycle, I can call it a 0-day ... I guess that would work if your boss is really clueless :-) My 2 cents Joel --On Monday, March 26, 2012 1:06 PM -0700 David Gillett wrote: >> It's TOOOO long if the compromised is out in the wild > > That is, I believe, the definition of a "0-day": The "patch window" > between discovery of a vulnerability and appearance of an exploit "in the > wild" is of zero length, usually because the vulnerability was identified > (by vendor or researchers) only by reverse-engineering some new exploit.... > > ... > > I believe it is the task of auditors to verify two assertions: > > 1. The institution's policies/procedures meet its business needs. > > where "meet the requirements of thus-and-such standard" is often accepted > as a minimum set of requirements, necessary but not necessarily sufficient > to satisfy this assertion, and > > 2. Actual operations conform to those policies/procedures. > > So to pass an audit is going to require at least (a) a written > standard/policy, and (b) an "audit trail" logging actions performed with > time and date. If you don't have both of those, you shouldn't pass an > audit. Ever. > > Once you have them, then it's time to look closer at #1 and at the actual > needs of the business -- whether regulatory compliance is mandated by > legislation, what degree of risk is acceptable, and so forth -- stuff where > Senior Management makes some choices, even if the choice is only whether to > remain in business or not.... > > > David Gillett, CISSP CCNP > Joel Rosenblatt, Director Network & Computer Security Columbia Information Security Office (CISO) Columbia University, 612 W 115th Street, NY, NY 10025 / 212 854 3033 http://www.columbia.edu/~joel Public PGP key http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x90BD740BCC7326C3
I guess the question is, do you want to be the guy who was responsible because a vulnerability, that was known for several weeks or even months .. and is 0 day (ie Microsoft or Apple learned about it because it was actively being exploited), infected your systems or allowed for a breach because you waited unnecessarily? Remember, attackers these days are not college kids, they're organized crime. -Brian
Message from valdis.kletnieks@vt.edu

On Tue, 27 Mar 2012 16:07:25 -0000, Brian Helman said: > I guess the question is, do you want to be the guy who was responsible because a vulnerability, Or you can be the guy who installed the patch that broke Payroll... :) It's often not as cut-n-dried as one would hope. Which is worse, getting pwned or not getting paid?
If you can't do your quality assurance in two weeks, you have bigger problems... not to mention using Windows for payroll ;)
Message from dsarazen@umassp.edu

Hi All, I really didn't mean to start a holy war over patching, but I've found the responses very helpful in coming to a reasonable conclusion for the testing I'm trying to complete. What I'd say, as an auditor, to anyone who'd listen, is that first you need to have some sort of formal process that addresses patching. If your formal process says that you wait a week after patches come out to make sure they don't break payroll, then I'd look to see if your patches were applied about a week later. By that same token, if you don't have a formal process and your servers haven't been patch in months...well, for an auditor, that's shooting fish in a barrel. I've also learned to request a report showing what applications the server is running, so that I can compare the available critical patches to see if the patch needed to be applied. However, if I see that you've installed the patch for Window's Media player on your server, you'd probably get dinged for having THAT app (or Google Toolbar, or iTunes) on the server in the first place. Again, it's a complicated and I was looking for a rule of thumb and, as has been pointed out to me, a good one doesn't really exist. Thanks for all the feedback Dan PS: I've given my notice and will be leaving UMass (But staying in higher-education) next week. Thanks for everybody's help these past four years. I'm sure I'll still be asking these annoying questions on this list serve with my new post.
Close
Close


Annual Conference
September 29–October 2
Register Now!

Events for all Levels and Interests

Whether you're looking for a conference to attend face-to-face to connect with peers, or for an online event for team professional development, see what's upcoming.

Close

Digital Badges
Member recognition effort
Earn yours >

Career Center


Leadership and Management Programs

EDUCAUSE Institute
Project Management

 

 

Jump Start Your Career Growth

Explore EDUCAUSE professional development opportunities that match your career aspirations and desired level of time investment through our interactive online guide.

 

Close
EDUCAUSE organizes its efforts around three IT Focus Areas

 

 

Join These Programs If Your Focus Is

Close

Get on the Higher Ed IT Map

Employees of EDUCAUSE member institutions and organizations are invited to create individual profiles.
 

 

Close

2014 Strategic Priorities

  • Building the Profession
  • IT as a Game Changer
  • Foundations


Learn More >

Uncommon Thinking for the Common Good™

EDUCAUSE is the foremost community of higher education IT leaders and professionals.