It Bears Repeating

There are just somethings that you can't say often enough. For example, tell the ones you love that you love them. Also, tell them that when they suspect an incident, do NOT shut the system off!

Think of Brad Pitt and Ed Norton (no, not that way!!) in Fight Club, but paraphrase..."the first rule of incident response is...don't panic. The second rule of incident response is...DO NOT PANIC!!"

Okay, let's go back to the beginning...well, maybe not that far. Let's go back to an InformationWeek article published in Aug, 2006, in which Kevin Mandia was quoted...here's one interesting quote:

One of the worst things users can do if they think their systems have been compromised by a hacker is to shut off their PCs, because doing so prevents an investigator from analyzing the contents of the machine's RAM, which often contains useful forensic evidence, Mandia said.

So, what do you think? Is he right? Don't say, "well, yeah, he must be...he's Kevin Mandia!" Think about it. While you're doing that, let me give you a scenario...you're an admin, and you detect anomalous traffic on your network. First, you see entries in firewall or IDS logs. You may even turn on a sniffer to capture network traffic information. So let's say that at that point, you determine that a specific system is connecting to a server on the Internet on an extremely high port, and the traffic appears to be IRC traffic. What do you do? More importantly, what do you want to know?

The usual scenario continues like this...the system is taken offline and shutdown, and stored in an office someplace. An outside third party may be contacted to provide assistance. At this point, once the incident is reported to management, the questions become...what is our risk or exposure? Was sensitive information taken? Were other machines affected? If so, how many?

This is where panic ensues, because not only do most organizations not know the answers to the questions, but they don't know how to get the answers themselves, either. Here's another quote from the article:

...Mandia said rumors of a kernel level rootkits always arise within the company that's being analyzed.

You'll see this in the public lists a lot..."I don't know what this issue is, and I really haven't done any sort of investigation...it's just easier to assume that it's a rootkit, because at the very least, that says the attacker is a lot smarter than me." Just the term rootkit implies a certain sexiness to the incident, doesn't it? After all, it implies that you've got something someone else wants, and an incredibly sophisticated attacker, a cyberninja, is coming after you. In some cases, it ends up being just a rumor.

The point of all this is that many times, the questions that management has about an incident cannot be answered, at least not definitively, if the first step is to shut down the system. At the very least, if you have detected anomalous traffic on your network, and traced it back to a specific system, rather than shutting the system off, take that additional step to collect process and process-to-port mapping info (for a list of tools, see chapter 5 of my first book) from the system. That way, you've closed the loop...you've not only tied the traffic to a system, but you've also tied it to a specific process on the system.

Of course, if you're going to go that far, you might was well use something like the Forensic Server Project to gather even more information.