Process-to-port Mapping

During first responder activities, be they simple troubleshooting or incident response related, one of the important pieces of information that can really provide some insight into the behavior and status of a system is process-to-port mapping.

Why is this information important? Well, it really depends on what you're looking for, but if your initial indicator or notification of a potential security incident is "unusual traffic" originating from a system, then you'd want to know what process was generating that traffic...right? After all, there are a couple of truths you need to keep in mind when troubleshooting or performing IR activities. One is that network traffic never spontaneously emanates from a system without a cause...there's always some process involved. Another is that nothing happens on a system without some code executing as part of a thread/process.

Okay, moving on...

With NT, getting process-to-port mapping information wasn't easy. You had to use a tool like fport to get the information. The same was true for Windows 2000, and it was really cool that as of XP, netstat.exe had the '-o' switch added so that you could get the network connections with the associated PID all on one line (this is great for scripting! =) )

Many folks may not be aware that there was an update to Windows 2000 that added the '-o' capability to netstat.exe on that platform. If you've got Windows 2000 systems, this is a great update to add, so that (as an admin) you have the capability to get the information you need.

Another tool you can use to get this information during IR activities is tcpvcon. I prefer this tool because it can be included in a batch file (reducing overhead), and the output can be sent to .csv format, which is great for parsing with Perl scripts.

Analysis Tip: One of the things you might run across during an engagement is working with admins who are network-centric...or being one of those network-centric admins. For example, you may get some indication of an issue from traffic captures, or IDS/IPS/firewall notifications or logs. In such cases, you very often have a couple of pieces of information available to you, one of them being the source IP of the traffic, which you would likely use to locate the system from which the traffic originated (assuming that it wasn't spoofed). But guess what? You will very likely also have the source port for the traffic...and this will information (if you don't ignore it) may assist you in identifying the process from which the network traffic had been generated. Something as simple as "netstat -ano | find " may be all you need to do to find the process that generated the traffic.

How is this important? Well, one more than one occasion, incident responders such as myself have encountered those who've said, "...we saw some unusual traffic, found the system, and scanned it with [insert AV product name], and found a virus." In such cases, there may be no correlation whatsoever between the traffic and whatever the AV product found (could've been a DLL or a Registry key...) - this is "incident response by assumption/speculation", which is about as bad as "security through obscurity".

Cutaway does Windows IR

The Security Ripcord site has a great new article on Windows IR, and using system resources to get the data you need.

I know that some folks are going to have an issue with using system resources (ie, DLLs, etc) when performing any kind of IR, but to be honest, I honestly believe (based on experience) that if more folks had stopped using that as a roadblock for doing any kind of response at all, there would likely be fewer instances of reported breaches, and breaches may have been less severe.

All incident response starts with the same basic elements and questions as any other system troubleshooting. The problem seems to start when admins and responders simply have no idea what it is they need to be looking at, or for. Don's article does a great job of bringing that to light, as well as providing a means of acquiring the necessary data. Not only does Don explain what is accessed and why, he also provides caveats about the artifacts left on the system as a result of the admin or responder's interactions with the system. This is very important, as anytime you access a live system, you're going to leave artifacts of one kind or another...being able to distinguish between your actions and the user's actions may be very important. Scripts such as what Don provided are self-documenting, in that all you have to do is ensure that you keep track of when (as in "what time") you ran the script, and then include a copy of the script along with your case notes.

A great big thanks and Semper Fi to Don for providing this article and script! It's information like this that's going to break down the barriers of inaction and provide for better response to all sorts of issues, large and small.

When First Responders Attack!!

It still happens...an "event" or "incident" occurs within an organization, and the initial response from the folks on-site (most often, the organization's IT staff) obliterates some or all of the "evidence" of the event. Consultants are called to determine "how they got in", "how far they got" into the infrastructure, and "what data was taken", and as such, are unable to completely answer those questions (if at all) due to what happened in the first hours (or in some cases, days) after the incident was discovered.

Check out Ignorance wrecking evidence, from AdelaideNow in Australia. It's an excellent read from the perspective of law enforcement, but a good deal of what's said applies across the board.

One of the things that consultants see very often is a disparity between what first responders say they did during an initial interview, and what the analyst sees during an examination. Very often, the consultant is told that the first responders took the system offline, but didn't do anything else. However, analysis of the image shows that installing and running anti-virus and -spyware tools, deleting files, and even restoring files from backup all happened. A great deal of this can be seen once the approximate timeline of the incident is determined...and very often, you'll see an administrator login, install or delete/remove stuff, etc., and then say that they didn't do anything.

Why would this matter? Let's take a look...

Many analysts still rely on traditional examination techniques, focusing on file MAC times, etc. So an admin logs into a system and runs an AV or anti-spyware scan (or both...or just 'pokes around'...something that happens a LOT more than I care to think about...), so now all of the file access times on the system have been modified, and perhaps some files have been deleted. Anyone remember this article on anti-forensics that appeared in CIO Magazine? Why worry about that stuff, when there is more activity of this nature occurring due to either the operating system itself, or due to regular, day-to-day IT network ops?

So what's the solution? Education and training, starting with senior management. They have to make it important. After all, they're the ones that tell IT that systems have to stay up, right? If senior management were really aware of how many times (and how easily) their organization got punked or p0wned by some 15 yr old kid, then maybe they'd put some serious thought and effort into protecting their organization, their IT assets, and more importantly, the (re: YOUR) data that they store and process.

IR Immediate Actions

As you may remember, in December 2007 I presented at the HTCIA conference in Hong Kong, the first HTCIA conference for this chapter. It was a great conference, and a great opportunity to meet a lot of folks in this field (see the Speaker's page).

While there, I presented on Registry Analysis and Windows Memory Analysis, both of which seemed to be well received. One question came up during discussions of Windows Memory Analysis...during the presentation, I talk about different tools and techniques that are available for extracting the contents of RAM, and also talk about the difference between tools I can use as a private individual and those I can use as a consultant. At one point, someone asked me which tools and techniques I use most often as a consultant...and my response was simply "none of them". The reason for this is that in most all of the responses I've dealt with, the system (or systems) have already been turned off, rebooted, or sufficiently imposed upon by others (AV scans, running multiple tools, etc.) that even if I did, as a consultant, have an option available for collecting the contents of RAM available to me, doing so would be of little benefit to either my analysis, or any information I could provide to the customer.

Part of the issue is that as a consultant, I am extremely limited in the tools I can use to collect the contents of physical memory from a system, not so much due to the availability of a particular tool, but more so due to the license agreement associated with that tool.

However, a bigger issue is that the immediate actions performed by most first responders on-site are to take systems offline, shutdown them down, or "clean" and reboot them prior to any calls for outside assistance being made. While this may be pertinent for business preservation and continuity, it most often has a detrimental impact on any follow-on analysis that may take place.

If there is data leakage due to an intrusion (or this is suspected, or this is just a question that needs to be answered...), then the immediate reaction is (apparently) to shut the system down. This may be pertinent, particularly if there is no incident response plan in place that lets people know what they need to do, and time is required to notify and get approval for follow-on activities (such as calling consultants). This reaction appears to be fairly ingrained, and I'm not suggesting that we change it by saying DO NOT shut systems down. What I am going to suggest is that we modify those immediate actions such that pertinent information is collected from systems before they are shut down.

Wait...what? It looks like what I am saying is, go ahead and shut the systems down, or reboot them, or "clean" them...and I am. This is going to happen anyway. There's nothing I can say or do, either on this blog or as a paid consultant, to change that. In order to change that behavior, there has to be an incident response plan (CSIRP) in place that tells people what to do, and when someone shuts a system off inappropriately or incorrectly, that issue needs to be addressed. The issue is that some kind of overwhelming stimulus needs to be put in place to change this behavior, and until this happens, nothing anyone can say or do is going to change that. So...it's gonna happen anyway, and instead of changing it, let's go ahead and go with it...but throw something else in there along the way.

What can we do? Well, one thing is to put tools on systems (for a list of tools, check out my book) when they are set up, or at least at some point prior to an incident occurring. If you can't put tools and a simple batch file on systems (for whatever reason...the most common of which seems to be, "...we don't know how..."), then issue each admin/responder a CD with tools, and a thumb drive. Then make it a policy within the organization that a batch file (could be a DOS batch file, or WMI script, etc.) be run and the output saved to a thumb drive, shared drive, etc., prior to the system being taken offline.

Vista IR

I recently started doing some testing of IR tools on Vista, using Vista Ultimate (32-bit) installed into a VMWare Workstation 6.0 virtual machine.

Part of my testing involved running some tools on Vista to see how they worked, and another part involved mounting the *.vmdk file for my Vista VM using the latest versions of VDK and VDKWin.

IR Tools
I started off by downloading (via IE7) a couple of tools...specifically Autorunsc.exe and Tcpvcon.exe. Both seemed to work quite well, the only real hiccup being the GUI EULA dialog that pops up if you run the tools without the "/accepteula" switch (either way, the tools create a Registry key...be sure you understand and document this as part of your IR methodology if you're using these tools). An interesting part of the tcpvcon output was the amount of IPv6 stuff visible.

My next steps are to test additional tools, as well as the use of WMI-based tools.

Extracting Files from a Vista VM
Mounting the Vista VM as a read-only file system/drive letter on my XP system went off without a hitch. I was already in the process of updating the VDK drivers and VDKWin GUI files, and on a whim I pointed the mount utility at the Vista VM *.vmdk file. I was pleasantly surprised to see the VM mounted as J:\. As expected, some of the directories (specifically, System Volume Information) could not be accessed...this is due to ACLs on those objects. However, I had fairly unrestricted access to the rest of the file system.

A friend mentioned to me recently that the offsets for the Last Run time and runcount in Vista Prefetch files is different from those of XP. I extracted a Prefetch file from the Vista VM and opened it in UltraEdit to look for the offset for the last run time. I found what appeared to be a FILETIME object at offset 0x80, and modified my existing code to extract those 8 bytes. The result matched up quite nicely:

C:\Perl>vista_pref.pl d:\hacking\vista_autorunsc.exe-7bca361f.pf
Last Run = Mon Aug 20 22:50:43 2007 (UTC)

I also tried running some of the Registry parsing tools (using the Parse::Win32Registry module by James McFarlane) against files extracted from the Vista VM. I started with a Perl script that would parse the contents of the UserAssist key - here's an extract of the results:

C:\Perl\reg>pnu.pl d:\hacking\vista_ntuser.dat
LastWrite time = Mon Aug 20 22:53:02 2007 (UTC)
Mon Aug 20 22:53:02 2007 (UTC)
UEME_RUNPATH
UEME_RUNPIDL
UEME_RUNPIDL:%csidl2%\Accessories\Command Prompt.lnk
UEME_RUNPATH:C:\Windows\System32\cmd.exe
Tue Aug 14 18:47:55 2007 (UTC)
UEME_RUNPATH:C:\Windows\system32\control.exe
Wed Jul 11 20:37:27 2007 (UTC)
UEME_RUNPATH:C:\Windows\system32\Wuauclt.exe

That seems to work quite nicely! It looks like I won't have any trouble accessing the raw Registry files using James' module, at least not on 32-bit versions of Vista, so that's good news!

There's still more testing and analysis to do, but this is a good start!

IR "Best Practices"

I know it's been a while since I've posted, but work has been really keeping me busy...that's an excuse that we all use, but that's my story and I'm stickin' with it!

So, I've been talking to a number of different folks recently, having discussions during my travels to and fro about incident response and computer forensics. Many times, the issue of "best practices" has come up and that got me thinking...with no specific standards body governing computer forensics or incident response, who decides what "best practices" are? Is it FIRST? After all, they have "IR" in their name, and it does stand for "incident response". Is it the ACPO Guidelines that specify "best practices"?

Well, the easy answer (for me, anyway) is that "it depends". Funny, haha, I know. But in a sense, it does. "Best practices" depend a great deal on the political and cultural makeup of your organization; I've seen places where the incident response team has NO access to the systems, and must request data and information from the network operations staff, which has their own daily tasks and requirements.

But all that aside, we can focus on the technical details of responding to Windows systems. When we say "best practices", we can specify the information we need to get in order to properly assess the situation/incident, how to go about getting it (retrieve it locally, remotely, via live acquisition, via post-mortem exam, etc.), and in what order to retrieve that data (the ACPO Guidelines specify dumping the contents of physical memory last, rather than first...), etc.

For example, we know that during live response, anything we do is going to leave an artifact. So, why not specify what data you're going to collect and the method by which you're going to collect that data? Then determine the artifacts that are left through testing, and document your procedure and artifacts (hint: things like batch files and scripts can constitute "documentation", particularly when burned to CD), and implement that procedure via some means (ie, batch file, script, etc.).

One of the questions that invariably comes up during discussions of live response and "best practices" is, can data collected via live response be used as evidence in court? I would suggest that the answer is, "why not?" After all, an assault victim can be treated by EMTs, operated on by a surgeon, and cared for in a hospital, and the police can still collect evidence and locate and prosecute the perpetrator, right? It's all about documentation, folks.

Thoughts on Live Acquisition

Recently, I've been doing some thinking about issues surrounding live acquisitions - specifically, acquiring an image from a system that is running, as opposed to either booting the system to an alternate OS, or removing the hard drive and hooking it up to a write-blocker.

There are several methods for performing a live acquisition, but most involve running an agent or application of some kind on the system itself. You can do this using ProDiscover (install or run the PDServer agent from a CD or thumb drive), FTK Imager (run from a CD or thumb drive, and write the image files to an external drive or an already-mapped share), or with good ol' dd and netcat running from a CD. Regardless of how you choose to do this, there is an additional process running on the system...so think Heisenberg's Uncertainty Principle, but in the digital realm.

So in acquiring an image from a live system, we need to introduce a process into a system of already running processes. While our intention is to take care and disturb the 'scene' as little as possible, it is simply a fact that we're going to leave some artifacts of our activities on the system...memory is consumed by our process, as are buffers, perhaps other processes have pages written out to the pagefile, etc. However, we address this issue with thorough documentation of our procedures and methodologies.

Now, in his book Computer Evidence: Collection & Preservation, Chris Brown describes the result of a live acquisition as a "smear", as from a temporal perspective, that's what we've got. Remember, it takes time to acquire an image from a hard drive, and if the system is still live and running when you acquire that image, then there is the distinct possibility that some sectors may change after you acquire them. Rather than having a sharp, distinct snapshot in time as when you acquire an image from a hard drive that has been removed from a system, you get a "smudge" or a "smear" instead. Also, as Greg Kelly pointed out in another forum recently, some of what we would normally consider stagnant data (Registry, files, etc.) can actually be considered volatile, and subject to change during the course of our acquisition...think log files, Event Log entries, the pagefile, etc.

Now, would it be possible to minimize this effect, by limiting what's running on the system during the acquisition? I believe that the answer to this is "yes". To do this, we'd need to take a couple of things into consideration. We'd should first ask ourselves, is this necessary? Hhhhmmm...if you're imaging the system over the network, and that system is still connected to the network, what is it doing on the network while you're acquiring your image? Is it serving up web pages, processing email, etc?

Before we continue, remember, we're talking about a live acquisition here, getting an image of the hard drive, NOT collecting the contents of memory.

Okay, that being said...if you're imaging over the network, is the system still providing services (shares via the Server service, etc.) that may have a significant effect on what you walk away with? We have to consider this in the face of what effect our actions will have on the system itself when we, say, disable a service. First, we have to see what processes are running, and to do that, we need to load some software and run another process (unless you're using the ProDiscover PDServer, as the agent provides this functionality, as well). Then we have to weigh the benefits of disabling the process or service against the "costs" of the effect that our actions have on the contents of the hard drive. Is that process really processing anything? We know that it may have file handles open, etc. But is there enough of an effect on the contents of the drive that it would make a significant difference within the time it takes to acquire the image? Also, if I disable that process/service, what happens? Any log files may be closed, perhaps the operating system itself will write an entry into the System or Application Event Log, etc.

Some of the processes or services I might consider shutting down include:

• AntiVirus products - these things reach out on their own and update themselves, or run scans automatically
  
• Task Scheduler - check to see if there are any jobs scheduled to run...this can get in the way of your acquisition (also, see the "Notes from the Underground..." sidebar on pp 215-216 of my book for an interesting tidbit on hiding scheduled tasks)
• Windows Firewall - depending upon how it's configured (we'd want to check, of course) there may be some significant issues. I included a sample pfirewall.log file on the DVD with my book...I'd turned up logging on the firewall and hit it with nmap. ;-)
• Exchange/IIS - if the system is still connected to the network do you want it processing email and web pages during the acquisition? Think the same thing for the FTP and SMTP services that sometimes get installed along with IIS.

This isn't meant to be a complete list, of course, but instead it's meant to start some thinking about what we have to consider before shutting a service down, or disabling a process.

So, before doing any of this, we need to put some thought into it. What, if anything, am I going to disable or shut down? Do I even need to? In some cases, live acquistions I've done have been of systems that had already been taken off of the network (not by me) and shut down, and then rebooted so they could be acquired (acquisition via write-blocker was not possible). With no network connections, I'm not overly concerned about major changes to the system during the acquisition process...the NTP service may log a complaint with the Event Log, but little else may happen. However, this isn't always the case...we're seeing a greater need for live response, both in acquiring the contents of memory, as well as performing live acquisitions of major systems that cannot be brought down or offline. If this is the case, document it. On your acquisition worksheet or in your case notes, clearly state "these services could not be halted or disabled due to...".

This is just a first blush...getting my thoughts down, thinking through things, determining if there is even a strong enough need for something like this...perhaps a matrix of services and processes, and when its a good idea to shut them down, how to do so, etc. Is this necessary?

Also, because I know the question is going to come up...addressing this same issue in the face of acquiring the contents of physical memory is an entirely separate post. Stay tuned!

Great news for IR and live response!

There was some great news recently for IR and live response!

Over the past couple of years, when discussing the viability and usefulness of live response, particularly as a source of evidence to be used in court, I have very often heard folks say, "I won't perform live response until there is case law to support its use."

Is it just me, or does sound like a chicken-or-the-egg thing? How can something be accepted in court if you're not willing to do the work and bring it into court in the first place? After all, look at everything that is used in court as evidence now, but at one time wasn't...fingerprints, DNA, even computer forensic evidence.

I was reading the TaoSecurity blog post regarding the Heckencamp case, and came across something interesting...that the court accepted a sysadmin's actions of logging into Heckencamp's computer to definitively determine that it was, in fact, the system being used to attack a mail server.

The Wired story mentions things like "counter-crack" and "counter-hacking", and I shudder at the use of both terms. The court's ruling includes a lot of discussion about expectation of privacy, but also includes such things as the fact that the sysadmin wasn't acting as an agent of law enforcement, but instead was acting to preserve the integrity of the mail server that was under attack. Basically, from what I can see in the opinion, the sysadmin confirmed that the system was used to attack the mail server by examining "network logs" and "after approximately 15 minutes of looking only in the temporary directory, without deleting, modifying, or destroying any files, Savoy [the sysadmin] logged off of the computer."

Okay, if anyone believes that nothing was modified in 15 minutes...well, that's a discussion for another time. After all, in order to access "network logs", a file would have had to have been accessed, modifying the last access time of the file...logging into the system itself would have modified logs, the contents of memory, etc...but I digress.

The Wired article ends in a monologue about vigilantism and student privacy, but that's not what I'm seeing here or interested in at all. Sure, the sysadmin used a username and password from a previous portion of his "investigation" to access Heckencamp's system, and the ethics of this can be argued until the cows come home. However, what I'm seeing is that live response may be starting to gain acceptance in court. If a sysadmin can log into a system and muck about for 15 minutes, why can't someone with a detailed process access a live system, collect necessary evidence as part of a thoroughly documented methodology, and then use that evidence in court?

Teaching IR/CF

My recent post on mounting a dd image got me thinking that there are plenty of freeware tools available for performing incident response as well as computer forensic analysis. Given this, how cool would it be to teach high school kids computer forensic techniques? Or, if you added something like this to the curriculum, you could easily add another course or two to a community college or undergraduate degree program.

Let's say that all you have avaible is a couple of systems. You can easily set up a "lab" with these systems, and then with minimal cost, add some external drives for storing images. There's plenty of free software available for acquiring and analyzing images, even if you're restricted to a particular platform.

I think that one of the benefits of this is that at the end of the program you'd have folks who had experience with different situations, different tools, and actually had to think through their approach to collection and analysis, not simply clicked a button.

Why current IR models don't work, part deux

As a follow-up to my previous post, I thought of another way to (hopefully) shed some light on this issue.

Sometimes, we (forensics weenies such as myself) like to illustrate technical points with real/analog world analogies. As some of you may know, one of my favorites is the stabbing victim you find in an alleyway, and call 911...the EMTs (incident responders) arrive, triage the victim, stabilize and move him, etc. The cops are ultimately able to locate the perpetrator of the crime, and he can be convicted, even given the fact that the victim was moved, etc. Using the traditional, purist CF model, a doctor would have to kill the victim on the scene and perform an autopsy in order to catch the bad guy.

Let's take a look at a similar analogy for IR, using real-world crime as an example. Say that an office building is a network infrastructure, and within that building is are file cabinets that contains sensitive data. The office building has doors, windows, elevators, roof access, etc., just like a normal office building...these are akin to access points to the network infrastructure.

Let's say that during the night, someone breaks into the building and attempts to steal some of the sensitive data kept there. What happens in the real/analog world? Usually, if there were no alarms, then someone notices something when they get into work the next day (or you hope that they do, anyway) and alerts security, who calls the local police (ie, first responders). Access to the area is immediately restricted, and an "incident response" plan of some kind kicks in...eventually a report is produced, and the perpetrators may be caught. If the folks who owned the building or occupied the office spaces actually made it difficult for someone to break in (by locking doors, using additional levels of access control, etc.) or monitored the area (video cameras, etc.) then there would be more evidence available for the police to review, and it would be more likely that the bad guy would be caught.

So, what does this have to do with anything? Well, here are some of what usually happens in today's digital realm, mapped to the real world:

1. Many buildings have no restrictions to access...doors (front, loading dock, etc.) are all open and unmonitored. First and second floor windows are open for convenience, to make it easier for employees (users) to come and go (yes, I've actually seen employees climb out first floor windows to avoid being seen leaving by their boss). There is usually a back door that's propped open. Access via adjacent buildings and even the roof is very often unfettered.

2. Bad guys get into the building at all hours, even during regular working hours. They come in through the front door, loading dock entrance, etc. They aren't challenged or even noticed. They lock/unlock doors, clog toilets, and generally make a nuisance of themselves. Many times, they are there for several months, taking files, sitting at employee's workstations, etc., and no one seems to notice or even respond.

3. When someone does notice that a bad guy is or has been there, the usual approach is that one of the employees does not alert anyone, and tries to investigate the situation themselves. As they have no training in this sort of thing (or did receive training, but have not used it, or been required to keep that training current), their investigation misses a lot of very basic stuff, like footprints, fingerprints, etc., and a lot of obvious stuff (contents of file cabinets and storage closets thrown all over the room, etc.). This may go on for weeks or even months, and when someone finally does call the police, there's been no record of who did what, what was found and when, and there may even have been broken doors and windows replaced.

Does any of this make any sense in the real world? Probably not. So why do we see this so often in the digital realm?

So, I then have to ask, is there any reason why IT staffs cannot be trained in first response, providing a tier 1 response capability? Basically, tier 1 IR is akin to advanced troubleshooting. Corporations would pay a nominal fee to have experts come on-site and train their IT staff in basic IR procedures, raising the level of their awareness and expanding their skill sets. The IT staff would realize the benefit of things like network diagrams, troubleshooting and IR procedures, etc (which, by the way, are required by most regulatory bodies, including FISMA and Visa PCI). They would then be able to handle first level/tier 1 response. Then, if additional assistance is required, reach out to those experts for tier 2 and/or 3 response, but only when needed. The flip side is that corporations end up paying much, much more because the first responders are called for every apparent "incident" that occurs; they resolve the situation, provide a report, and go back to wait for the next call...but no knowledge transfer occurs and the "victim" is no better off than they were before the "incident" occurred.

Another benefit of having that on-site, functional, hands-on training and producing things like network diagrams is that the IT staff gets to "know" their network. By working through scenarios ahead of time, network administrators learn what the important pieces of network- and host-based information are that they are interested in when investigating an incident. Sometimes just asking the question of "what systems store or process sensitive data?" during training evolutions is much more beneficial than asking that same question after someone has p0wned the box and carted that data off.

Current IR models don't work because we don't spend enough time looking at what works in the real world and mapping that same sort of mechanism into the digital realm.

Why current IR models don't work

A while ago, a buddy of mine was on travel, leaving his family at home. He called home from the airport whilst awaiting a flight and got some interesting news. He'd winterized their house several months earlier, which included shutting off the hose bibs from inside the house and purging the water from the pipes so they didn't freeze and burst. Well, about 36 hours prior to his call, there had been a need to run water from the faucet on the back of the house, so his wife had turned the hose bib on. However, being unfamiliar with all of the pipes and everything running through the closet, she'd also accidently turned off the gas to the house, which caused all of the pilot lights (on the gas range, in the gas fireplaces, and for the water heater) to go out. Listening to this, my friend started asking questions of his wife, many of which were answered "I don't know".

Listening to him tell this story, it became clear what his concerns were, but I wanted to hear it from him. While his wife was saying, "honey, you're going to come home from a long flight with several stops, layovers, and delays and not have any hot water to shower...sorry", he was hearing that there were possibly places in the house where gas could be seeping into the house. Thankfully, the gas used in homes smells, so it can be detected, but if the source is in an enclosed room...you see where I'm going with this.

I thought about this for a bit while I was running the other day, and it occurred to me that this almost exactly mirrors current incident response models. I know, I know...bear with me here. Consider the wife in this story to be the CEO (hey, don't we all???), and my friend his a CISO or security admin. The kids and pets are the staff. In this case, we have an "incident"...the gas coming into the house was shut off long enough to cause the pilot lights to go out. The CEO, based on her sphere of perception, understands the issue to be one of inconvenience...lack of hot water means no hot shower. She feels that they can wait until my buddy gets home to deal with it. However, my friend sees an even bigger "threat"...not only to property (ie, his house) but more importantly, to the health and safety of his family (corporate officers failing to accurately identify critical assets).

In response to this, my friend decided to embark on a training and educational approach to changing the "corporate culture" of his family to be even more cognizant of the issues and potential impacts of such incidents. In part, this is where the story takes a turn and ends differently from what happens in business (corporate, government, etc.) organizations today.

So how does all this apply to these business organizations? Look around...or look here. It's quite a long list, but see any duplicates, either in what reportedly occurred, or in the name of the organization?

So, how do we fix this, you ask? Glad you asked!! Apparently, security overall (and not just IR) is something that is not part of the "corporate culture" of many organizations. Responding to incidents often times gets me nothing but blank stares when I ask questions about the status of systems, where systems are "located" in relation to each other, etc. That's nothing unusual.

Speaking of "corporate culture", when I was in the Marine Corps, we had a corporate culture, one that was easy to remember - "every Marine a rifleman." Basically, what that meant was that every single Marine, officer or enlisted, must be qualified in care, feeding, and effective and deadly use of the M-16 rifle. Every Marine received training in its use, and had to go through annual requalification. The same was true for officers...up through the rank of Captain, every officer had to qualify annually with the M-16 as well as their own TO weapon, the M-9. But this wasn't all we did...this was part of the many things we did, but it was part of our corporate culture. I believe this served us well in many incidents, particular in Iraq, where the "front lines" were often right in front of you, and it didn't matter if you were an infantry Marine or a cook or a Motor T driver.

My point is that we can talk about IR all day long, but things won't change unless someone with the ability to change the corporate culture does so. Everyone, particularly the IT staff, needs to be more security conscious, and be more familiar with the assets their protecting. What are the critical assets of the company, as defined by the CEO? How does an IT admin's job of maintaining servers and systems relate to accomlishing the mission of protecting those assets?

Also, consider this...in the military, everyone's trained in "immediate actions". If an M-16 jams, every Marine knows the immediate actions to perform to get that weapon back into service. Marines on patrol know that if caught in a near ambush, then the response is to attack the source of the ambush, immediately and with maximum violence. Consider this...why can't the IT staff be trained in "immediate actions", as well? If unusual traffic is seen on the network or anomolous behavior appears on a system, there are things that the IT staff can do immediately to identify the issue. There are other things that they can do in order to quickly address the situation.

It all starts with a top-down approach to the corporate culture. From there, IT staffs need to receive training...functional, hands-on training that applies to the systems they work with every day. Going away to Linux-centric training for someone in an all- or predominantly-Windows shop is a waste of time and money. There needs to be core, central set of knowledge and training that every IT staff member receives and is responsible for knowing...in the Marines, every officer, be they a pilot, a communicator, or a supply officer, goes through the same training at The Basic School. From there, certain members of the IT staff should receive specific training based on their areas of responsibility, be they routers, firewalls, servers, applications, etc. They need to understand these areas and systems inside-out, in much the same way a Marine can field strip, clean, and reassemble the M-16.

So, if you're reading all this and you're not someone in the IR business, you're probably thinking, "oh, he's just saying this so that he can get our money." No, I'm saying this so that you don't loose my data...my SSN, my credit card number, etc. Or my wife's. Or, several years from now, my daughter's or my niece's and nephew's. We see it in the media all the time, with these high profile "hacks" involving someone breaking in and stealing data, or sensitive personal information being lost. Or critical applications and systems being taken over, compromised, and p0wned. These incidents are no longer the result of joy riders on the Information Superhighway...that ended a while ago. There is a profit motive behind this...these crimes are being committed because there is money involved. There needs to be a shift in the corporate culture such that this data is better protected, and when something does happen, the response is something more than paralysis. What would you do if you had a fire in your home? Would you evacuate your family, and if the fire were small and isolated enough, would you attempt to put it out? Would you call the fire department? Dumb questions, I know...but when incidents are occurring today, many corporate cultures make it okay to ignore the situation and simply go back to sleep. Worse, the situation is recognized as something unusual, but everyone is paralyzed, and the fire department gets called only after the house has burned down.

Thoughts?

Thoughts on Incident Response/Management

Others have recently posted their thoughts on incidents and incident response (1, 2), and since this is an IR blog, of sorts, I thought I'd throw my hat into the ring, as well.

I don't really have a list of thoughts on IR, per se...its more a case of one thought from which all others spring. Basically, that is that organizations really need to start taking a proactive approach to Incident Response and Incident Management.

Incidents are going to happen. We can take that as a given. The days of joy-riding on the Information Superhighway are about over...there's simply too strong a profit motive these days. Why get a copy of BO2K onto a system and open and close the user's "cup holder", when you can quietly collect the contents of Protected Storage, capture keystrokes as the user logs into his online banking site, etc.

So what do you do? Well, first off, senior management needs to take this "security thing" seriously and treat it like a business process, not an ad hoc thing that you scramble to implement when you need it, and then drop it (cut the budget) when you don't. Walk around headquarters one day and look at the finance department...or marketing...or Ops. HR. All of these are business processes that you need to run a successful company. But how did you know that? Did someone tell you? Or did you suddenly see the need? Either way, people have been saying for years that you need to have security, and if you don't recognize the need simply by reading the paper everyday...

At some point, someone's going to throw up their hands, throw a huge budget at security, hire a bunch of (the wrong) people, and then proclaim security a failure. It doesn't work! Sound familiar? Is that how you would run your Finance Department? How about Payroll?

This security thing is really simple. Start by looking at your assets...what are you trying to protect? Is it sensitive personal data (a la CA SB1386)? Are we talking about critical systems, applications, or data...or some combination?

Once you've identified the assets, look at the threats. But keep in mind that this is no longer seige warfare, where you watch in wonder while barbarians pound at your gates...or your firewall. We're well into what the United States Marine Corps refers to as "manuever warfare". We're facing a battle with no defined front lines; ingress points include web browsers, rogue WAPs, etc. Have an assessment done, and if the result is "pay us a ton of money to fix it", then you may need to ask for your money back! In all infrastructures, there are things that can be done that don't cost money, but do have an inherent cost...things like cultural and political changes, changes in how you do things (ie, communal Admin/root accounts, etc.).

Train your people and hold them accountable. You're not going to hire an IR staff and keep them in a room with "break glass in case of emergency" written on the door. However, a great deal can be accomplished by training your current staff in how to manage and respond to incidents. Designate some personnel as incident responders, with a suitable manager, and get them some training. Even if they are familiar with handling some incidents, the best training is functional and hands-on. Have regular refresher training in the form of incident response exercises...maybe even coordinate that with an unannounced pen test.

In fact, here's a great way to go about the whole thing. Get your folks the training they need to be tier 1 incident responders. This means that there are come tasks that they'll be able to handle, like EMTs. Then, seek out a professional response firm, and keep them on retainer so that they're available should you call, and then can be on-scene and paid on a per-incident basis. This kind of approach has is a win-win, particularly if the professional responders are the ones who are training and working with your team...they get to know and trust each other, and there's a great deal of knowledge transfer that goes on.

Where this kind of thing breaks down is when the responders are held accountable for something, and senior management isn't. Security (in general) and incident response are business processes, folks, and need to be thought of and treated that way.

BTW, one of the links in the first line of this post is to a fairly new blog: ForensicIR. Welcome aboard!

Getting service information during IR

During his BlackHat DC presentation last week, Kevin Mandia said that the persistence method used by many malware authors seems to have shifted to Windows Services. During the presentation, he mentioned using psservice.exe from MS/SysInternals to get information about the services on a system, and said that psservice.exe doesn't show the executable image used by the service, and that you'd have to get that information from the Registry.

Well, maybe not. Kevin's a really bright guy, and very busy. There are ways to get the executable image path...using WMI for example. Writing a quick Perl script (and then compiling using Perl2Exe so that it can be used easily with the FRUC/FSP), one can get the following:

Name : wltrysvc
Display : Dell Wireless WLAN Tray Service
Start : LocalSystem
Desc : Provides automatic configuration for the 802.11 adapter using the Broadcom supplicant.
PID : 716
Path : C:\WINDOWS\System32\WLTRYSVC.EXE C:\WINDOWS\System32\bcmwltry.exe
Mode : Auto
State : Running
Status : OK
Type : Own Process

Pretty cool, eh? Path the executable image, PID, start mode and state. Of course, CSV output is easier to parse...and yes, this program does come included on the DVD accompanying my book.

Fundamentals

MS recently posted their Fundamental Computer Investigation Guide for Windows. You can download the document or read the entire doc online.

I haven't heard too many opinions yet about this document, good or bad. It may be because most folks aren't aware of it. I will say right up front that I was involved in the editing process, so I'll have a slightly different view. I've seen this document in various forms, and read each of those iterations at least twice.

The paper is intended for IT professionals in the United States who need a general understanding of computer investigations, including many of the procedures that can be used in such investigations and protocols for reporting incidents. Okay, that's a great way to start when writing a fundamentals paper. The paper seems to be directed at folks who either don't normally respond to incidents, or haven't done so before and are now in a position where they are required to do so.

The paper consists of five chapters and an appendix. I'm not going to go through each one, but rather just mention some highlights. I do think that the paper is worth the time to read it, as anyone who reads it is going to get something out of it, positive or negative.

Chapter 1: Assess the Situation: This chapter provides some good advice and things to think about when an incident occurs; I would suggest, however, that most of those things (obtain authorization, review laws, etc.) be done before an incident occurs...that way, attention can be focused on responding to the incident. The Conduct a Thorough Assessment section is fairly comprehensive, and as you read it, keep in mind that no one will be able to list everything you need to know for every incident. Not only do incidents vary, but the same worm on two different networks will be handled differently, because each infrastructure is different, both from a technical/network perspective and a human/socio-political perspective. I would say that there's enough there to get you started, and going forward, it's better to know or not know than to make things up; many an investigation has gone down the wrong path because someone made assumptions based on too little information.

One thing I would like to see changed about the paper is how data about an incident is referred to. In the Prepare for Evidence Acquisition section, the first sentence says in part, "To prepare for the Acquire the Data phase..." So which is it...evidence or data? Given that many states are considering PI laws, it is important to differentiate between the two. Also, consistency of terminology is just a good idea.

On a positive note, the first chapter ends by referring to documentation. Documentation is very important during an investigation. Let's say that you're sweeping across servers to identify a piece of malware...if you don't document your process (what's being checked and why), you're going to have folks who go to a server and don't know what they're supposed to do. Also, you need to document which systems have been scanned and by whom, so that you don't spend a great deal of time rescanning the same server over and over again. Remember, if you didn't document it, it didn't happen.

Chapter 2: Acquire the Data: This chapter glosses over data acquisition as part of incident response. There is a reference to the Tools section in the Appendix, but the tools listed are exclusively either SysInternals tools or native commands on the system. Don't get me wrong...there are some extremely useful tools from SysInternals, but what's missing is a description of what information needs to be collected, and the best tools for doing so. For example, when responding to an incident, the first things I want to know are:

1. The list of active processes, with the full path to the executable image and the command line to launch each one. I'd also like to know the user context of each process.
2. Network connections.
3. Process-to-port mappings, so that I know which network connection is owned or used by which process.
4. Who is logged into the system (NetBIOS, mapped shares, Terminal Services, etc.)
5. Running services.

This is just the basics...there's more info that I won't get into here (hey, I need something to blog about later, right?). It's not that the tools mentioned won't provide that information...some do. It's that the paper doesn't mention what's important to know...it just tells the reader that there are some tools they can use.

Chapter 3: Analyze the Data tells the reader what they should do with the data they've collected. In the section on analyzing host data, you'll see the following statement:

...you might use the Microsoft Windows® Sysinternals Strings tool to search the files located in the \Windows\Prefetch folder. This folder contains information such as when and where applications were launched.

Don't get me wrong...I agree with both sentences. I just don't think that the two of them should be next to each other. Why? Well, the information maintained in a .pf file regarding when the application was launched is a 64-bit FILETIME object...strings.exe won't find that for you. While using strings or BinText from FoundStone can be very useful when looking for ASCII, Unicode, or Resource strings within a file, neither will help you locate the FILETIME object.

There's more to it than that, though. When responding to an incident, how do you identify a "suspicious" file or process? Within your infrastructure, what constitutes suspicious? Some might think that running three or even four remote desktop applications is suspicous, while others may think that multiple copies of svchost.exe running on a system is suspicious.

At this point, I think that I've provided enough comments regarding what I saw to be good in the paper, as well as what, IMHO, could be improved of changed. I think that the paper is a good start, but don't expect to sit down and read it, and then be able to conduct an investigation. I still think that MS would be better off with a different structure to documents such as these, directing different versions to different audiences. For example, a high level document for incident managers, one that's a bit more technical for incident team leaders (so that they can evaluate the performance of the team members), and then separate documents for data acquisition and analysis for host- and network-based live acquisition, as well as acquiring an image.

Thoughts?

What's New So Far in 2007

Less than a week into 2007, and there's already a bunch of new stuff out that is really useful and really cool.

First off, there's a new blog on the streets...uh, web. Check out Mark McKinnon's CFED-TTF blog. One post up (at the time of this writing) and he already has some good info on the drivetable.txt file in XP System Restore Points. Great start so far!

Next, I was doing my monthly check on the E-Evidence site, and I found some really good stuff. No, wait...I mean REALLY good stuff! Raphael Bousquet has an interesting presentation on forensic triage. I know that its a product pitch, but the information on the idea of doing a triage is interesting, and something that should be, at the very least, discussed.

Golden Richard has an interesting PPT available that discusses next-gen digital forensics, to include topics such as live forensics. In the PPT, Prof. Richard points out that evidence (and he uses the term "evidence") exists in places other than those thought of in the "traditional" sense of forensics. He also talks about all the work that needs to be done...and while I agree, the question is, who will do that work? We do have a plethora (like that? No, I didn't get a thesaurus for Christmas) of students now that many universities and even community colleges here in the US have started offering courses and degrees in digital and computer forensics, but how long will it be before the big-brain ideas become something useable by investigators and examiners?

There are other presentations and papers available in this month's "What's New", but IMHO, the best paper availabe in this collection of links is Jessica Reust's paper on AIM trace evidence. What struck me most about her paper is that by the time I finished it, I actually had something useful, something I could use in an examination.

Now, on the flip side of all this, we should take it upon ourselves, as a community, to identify those things that we need, and either create them or put them on the table. What am I talking about? If you see a need or have a question, get it out there. Let someone know. Maybe someone out there already has the information you need, or is working on it.

Thoughts? Ideas? Comments?

New Year's Resolutions

I read today that there are some technical bloggers that have resolved to not make any New Years resolutions. Uh...okay...but isn't saying that you're not going to do that, in essence, a resolution? Hey, I'm just sayin'...

To kick 2007 off, I'm going to resolve to think big thoughts about IR and CF this year. Seriously. There has to be more to IR and forensic analysis than just what we're seeing. Think about it. There's got to be ton of evidence in Registry, right? After all, no one goes there. What about in RAM? And I know that there are a lot of questions out there, as I see some of them again and again. Questions like:

• How do I show files were copied to/from a system?
• How do I show that a CD/DVD was created on a system, and by whom?
• How do I show that a user account was changed from a User to an Administrator, and when?
  

What I would ask of all of you for 2007 is to build the knowledge base of the forensic community. Remember, it takes a village...I know, I can't believe I said that either, but hey, it makes my point. I've heard that a lot of folks don't post or comment or ask questions online because they don't want to look stupid. Okay, so post under someone else's name so they look stupid. Or post anonymously. Whatever works. The point is that there're a bunch of us out there working in this area, and every now and then a "hey, what about..." or "hey, what if..." or "hey, look what I found..." would really go a long way toward adding to all of our knowledge.

I've also been told that LEOs don't like to post questions because opposing counsel might see it and hold that against them in court. Well, if that's the case, then couldn't opposing counsel pretty much get any testimony thrown out because at one point, before all of your training and reading, you didn't know anything? I mean, doesn't it make sense that you'd ask, get an answer, and verify it, and let that be the case, rather than go into court with less of a case, all because you didn't want to ask a question?

One last thing...please resolve that in 2007, when posting questions, you'll include the OS and version. Seriously. I know some of you think that when someone responds to your post with "what OS/version?", you've been "chastized" or p0wned...whatever. Get over it. Most times, the answer will be different, depending on whether we're talking Windows 2000, XPSP2, or Vista, and I don't have the time to write or read an if...then...else encyclopedic answer.

A little programming fun, etc.

I was working on writing my book yesterday afternoon (for those of you who live in my area and know how nice it was outside, I know...shame on me for being inside), and was working on a section that has to do with the Windows Recycle Bin. Most forensic analysts are likely familiar with the Recycle Bin and how it works, as well as with Keith Jones's excellent description of the Recycle Bin records (1, 2), so this is more than likely nothing new.

For the book, I wanted to throw together a little script that can be used to parse the INFO2 file, in order to illustrate some concepts that I am trying to convey in section. To my surprise, it was easier than I thought...I got something put together that works amazingly well - my only stumbling block at this point is how to present the data in such a way as to be useful to widest range of folks.

This just shows how useful a little bit of programming skill can be. Something I've been thinking about adding to my list of ProDiscover ProScripts is a script that will go through the Recycler directory and look for files that are out of place...not only files that are in the Recycler directory itself, but those that are in the user's deleted files but not listed in the INFO2 file.

Way Too Funny
Okay, now this is just way too funny...I was doing a search on Amazon to see if my next book is up for early order yet (my publisher said that it would be soon), and I ran across my master's thesis! If you're suffering from insomnia, it's well worth the price to grab a copy of this thesis...I used video teleconferencing traffic to verify the fractal nature of ethernet traffic, so that real statistical models could be used (rather than assumed) for constructing buffers on switches at the edge of ATM clouds.

Anyway, this was really "back in the day" (re: 1996). I was using Java (which, with the help of O'Reilly, I had taught myself) to write an SNMP application to poll Cisco routers for traffic data. I then used MatLab to perform statistical analysis tests of the data to determine its "burstiness" and verify its fractal nature. The video conferencing came in because it allowed me to generate data...I could sit in front of one camera, across the room from the other camera, and have measurable traffic generated across the network. At the time, Notepad was my Java IDE...this was before IBM's Java IDE came out (I saw my first copy of the IBM Java IDE at the end of July, 1997).

Something new from ISC
There were a couple of items I found interesting on the SANS Internet Storm Center handler's diary recently.

One was this item about malware that uses a commercial tool to detect the existence of a virtual machine. Very interesting.

Another is this writeup on the FreeVideo Player Trojan, submitted by Brian Eckman (the writeup also appears on SecGeeks and First.org, among other locations). Brian's writeup is thorough and very comprehensive...I like seeing this sort of thing posted, as it's a nice break to see a submission like this. Brian put a lot of work into the writeup, and looked at a variety of areas on the system that were affected by this malware.

The one question I have (and would have for anyone) is this statement that was made:

Further analysis shows that this appears to inject itself into running processes.

Given the rest of the writeup, there really isn't anything that specifically states how Brian determined this. I'm sure we can come up with a variety of assumptions as to how he did it, or how we would go about doing it, but I'd like to hear from the author. I'm going to submit that as a question to him, and I'll let you know what I find out.

How do you do that voodoo that you do?

After reading through the SecuriTeam documents, and then reading this SANS ISC post, I have to admit, incident response can be extremely complex. This is particularly true if the bad guys catch you with your pants down (figuratively, or hey, even literally). From the SecuriTeam write-ups, we get an insight into how systems are targetted...via Google searches (with books such as Google Hacking from Johnny Long, there's really no big mystery to this) and even freely available vulnerability scanning tools.

In the SecuriTeam documents, the decision to conduct a live response is documented, as a "full forensic analysis" would take too long...it was evidently determined that the attackers hadn't simply defaced a web page, but were actively on the system and involved in creating mayhem. This isn't unusual...many organizations decide that affected systems cannot be taken down. However, with the Team Evil incident, there didn't seem to be any volatile data collected or analyzed. It appears that because a web page was defaced and some directories created, that the investigation and the IR team's reactions focused solely on the web application(s).

One thing I took away from the second SecuriTeam document was that there were...what? Eight attacks?

The post from the Internet Storm Center doesn't seem to make things any easier, does it? After all, not only do multiple downloaders dump a "zoo of malware" (i.e., keylogger, BHOs, even disables the Security Center) on the system, but it also reports the infected system's location so it can be tracked via Google Maps. While this is an interesting new capability, the real issue IMHO is the stuff that's dumped on the system. How do you track it all? If you really don't know a great deal about your systems (hosts and networks) and the applications that you're running, and you haven't taken any real steps to protect those systems, I'd have to say that the ISC recommendation to reinstall is the only option.

If you really think about it, though, maybe this is the intention. If you've been looking at some of the various conference presentations over the past couple of years, there have been some anti-forensics and "how to use the analyst's training against them" presentations. One way to look at these attacks is that the attacker is just being greedy. Another is to think that the intention of dumping multiple programs (keyloggers, BHOs, etc.) on the system is that IF the sysadmin detects any one of them, they'll reinstall the system...and it's likely that the system can be easily reinfected.

So, on the face of things, we've got a denial of service attack...compromise and infect a critical-use system, and it's so important that the sysadmin takes it offline for a reinstall. This also lends itself to persistence...the reinstalled system may have the same or additional vulnerabilities, so the attacker can reinfect the system once it's back up.

Of course, I can't help but think that this could also be a distraction, a little bit of misdirection...get the IT staff looking one direction while the attacker is siphoning data off of another system.

So I guess I have to concede the point that reinstallation is the thing to do. If you (a) don't really know your infrastructure (hosts or network) that well, (b) have no idea where critical (processing or storing sensitive personal information) applications are, (c) haven't really taken any defense-in-depth measures, (d) have no clue what to do, and (e) don't have an IR plan that is supported and endorsed by senior management, I guess there really isn't any other option.

Thoughts?

Trends in Digital Forensics, and news

I ran across a Dr. Dobbs article of the same name as the title of this post...very interesting. The subtitle is Are "live" investigations are the trend we are heading towards?

An interesting quote from the article:
Thus the new trend in digital forensics is to to use the corporate network to immediately respond to incidents.

Hhhhmmm...this sounds pretty definitive.

My thoughts on the article are two-fold. First, I have to ask...is this, in fact, the trend (or at least a coming trend that we're seeing more of)? Are IT and IR teams using tools like those mentioned in the article (Encase, Wetstone's LiveWire - I have to wonder why the author doesn't mention ProDiscover) to perform incident response via the network? If so, how effective are these efforts?

Overall, the author discusses "live investigations" (which is cool, because my next book covers that, in part) but I have to wonder how much this is being done, and how effective it is.

Now for the "news"...there's a new CyberSpeak podcast out, I just downloaded it and still have to listen to it. I took a look at the show notes (which have moved) and saw that Jesse Kornblum is again being interviewed. Very cool. One of the news items I picked up from the show notes was about a guy in the UK who took over young girls' computers and extorted them into sending him dirty pictures of themselves. The scary thing about the article isn't things like this:

...used some of the most advanced computer programmes seen by police to hack into their PCs...

One of the youngsters said his level of expertise and his power over her PC reminded her of the cult science fiction film Matrix.

Well, okay...I take it back...maybe those excerpts do represent some scary things about the article..."scary" in the sense that an email-borne Trojan of some kind is equated to level of technology seen in the Matrix. Or maybe it's the fact that according to the article, these kids actually fell prey to this guy and sent the pictures, rather than notifying their parents.

Okay, I'm off to listen to the show...

ComputerWorld: Undisclosed Flaws Undermine IT Defenses

I ran across this article today. All I can say is...sad.

Undisclosed flaws may undermine IT defenses...but that presupposes that the organization has some kind of "IT defenses" in place. Take the first sentence of the article:

Attacks targeting software vulnerabilities that haven’t been publicly disclosed pose a silent and growing problem for corporate IT.

Silent? The vulnerabilities themselves may be "silent", as they're just sitting there. But I have to ask...if someone attempts to exploit the vulnerability, is that silent, and if so, why?

One of my favorite examples of defense (the military analogies just don't work today, as there are simply more and more folks in the IT realm who don't have any military experience) is from the first Mission Impossible movie. Cruise's character, Ethan Hawke, has made his way back to a safehouse after the catastrophic failure of a mission. As he approaches the top of the stairs, he removes his jacket, unscrews the lightbulb from the socket, crushes the bulb inside his jacket, and scatters the shards on the floor as he walks backward to the door of his room. How is this defensive? If someone approaches the room, the hallway is dark and they end up stepping on the shards (which they can't see) and making noise...announcing their presence.

The problem is that there are great number of organizations out there that have very little in the way of protection, defense mechanisms, etc. Defense in depth includes not only putting a firewall in place, but patching/configuring systems, monitoring, etc. Many organizations have some of these in place, to varying degrees, and the ones that have more/most of them generally do not appear in the press.

This also highlights another issue near and dear to my heart...incident response. The current "state of IR" within most organizations is abysmal. Incidents get ignored, or the default reaction is to wipe the "victim" system and reinstall the operating system and data. This issue, as well as that of preparedness, can only be adequately addressed when senior management understands that IT and IT security are business processes, rather than necessary expenses of doing business. IT and in particular IT security/infosec can be business enablers, rather than drains to the bottom line, IF they are recognized as such by senior management. If organizations dedicated resources to IT and infosec the way they did to payroll, billing and collections, HR, etc., there would actually be IT defenses in place.

An interesting quote from a Gartner analyst is that much of the confusion about what constitutes a zero-day threat stems from the manner in which some security vendors have used the term when pitching their products. Remember, folks, it's all market-speak. Keep that in mind when you read/listen to "security vendors".

Another quote from the same analyst: But the reality is that most organizations “aren’t experiencing pain” from less-than-zero-day attacks. I firmly believe that this is perhaps partly true, due to the fact that many incidents simply go undetected. If an attack or incident is limited in scope and doesn't do anything to get the IT staff's attention