Why current IR models don't work, part deux

As a follow-up to my previous post, I thought of another way to (hopefully) shed some light on this issue.

Sometimes, we (forensics weenies such as myself) like to illustrate technical points with real/analog world analogies. As some of you may know, one of my favorites is the stabbing victim you find in an alleyway, and call 911...the EMTs (incident responders) arrive, triage the victim, stabilize and move him, etc. The cops are ultimately able to locate the perpetrator of the crime, and he can be convicted, even given the fact that the victim was moved, etc. Using the traditional, purist CF model, a doctor would have to kill the victim on the scene and perform an autopsy in order to catch the bad guy.

Let's take a look at a similar analogy for IR, using real-world crime as an example. Say that an office building is a network infrastructure, and within that building is are file cabinets that contains sensitive data. The office building has doors, windows, elevators, roof access, etc., just like a normal office building...these are akin to access points to the network infrastructure.

Let's say that during the night, someone breaks into the building and attempts to steal some of the sensitive data kept there. What happens in the real/analog world? Usually, if there were no alarms, then someone notices something when they get into work the next day (or you hope that they do, anyway) and alerts security, who calls the local police (ie, first responders). Access to the area is immediately restricted, and an "incident response" plan of some kind kicks in...eventually a report is produced, and the perpetrators may be caught. If the folks who owned the building or occupied the office spaces actually made it difficult for someone to break in (by locking doors, using additional levels of access control, etc.) or monitored the area (video cameras, etc.) then there would be more evidence available for the police to review, and it would be more likely that the bad guy would be caught.

So, what does this have to do with anything? Well, here are some of what usually happens in today's digital realm, mapped to the real world:

1. Many buildings have no restrictions to access...doors (front, loading dock, etc.) are all open and unmonitored. First and second floor windows are open for convenience, to make it easier for employees (users) to come and go (yes, I've actually seen employees climb out first floor windows to avoid being seen leaving by their boss). There is usually a back door that's propped open. Access via adjacent buildings and even the roof is very often unfettered.

2. Bad guys get into the building at all hours, even during regular working hours. They come in through the front door, loading dock entrance, etc. They aren't challenged or even noticed. They lock/unlock doors, clog toilets, and generally make a nuisance of themselves. Many times, they are there for several months, taking files, sitting at employee's workstations, etc., and no one seems to notice or even respond.

3. When someone does notice that a bad guy is or has been there, the usual approach is that one of the employees does not alert anyone, and tries to investigate the situation themselves. As they have no training in this sort of thing (or did receive training, but have not used it, or been required to keep that training current), their investigation misses a lot of very basic stuff, like footprints, fingerprints, etc., and a lot of obvious stuff (contents of file cabinets and storage closets thrown all over the room, etc.). This may go on for weeks or even months, and when someone finally does call the police, there's been no record of who did what, what was found and when, and there may even have been broken doors and windows replaced.

Does any of this make any sense in the real world? Probably not. So why do we see this so often in the digital realm?

So, I then have to ask, is there any reason why IT staffs cannot be trained in first response, providing a tier 1 response capability? Basically, tier 1 IR is akin to advanced troubleshooting. Corporations would pay a nominal fee to have experts come on-site and train their IT staff in basic IR procedures, raising the level of their awareness and expanding their skill sets. The IT staff would realize the benefit of things like network diagrams, troubleshooting and IR procedures, etc (which, by the way, are required by most regulatory bodies, including FISMA and Visa PCI). They would then be able to handle first level/tier 1 response. Then, if additional assistance is required, reach out to those experts for tier 2 and/or 3 response, but only when needed. The flip side is that corporations end up paying much, much more because the first responders are called for every apparent "incident" that occurs; they resolve the situation, provide a report, and go back to wait for the next call...but no knowledge transfer occurs and the "victim" is no better off than they were before the "incident" occurred.

Another benefit of having that on-site, functional, hands-on training and producing things like network diagrams is that the IT staff gets to "know" their network. By working through scenarios ahead of time, network administrators learn what the important pieces of network- and host-based information are that they are interested in when investigating an incident. Sometimes just asking the question of "what systems store or process sensitive data?" during training evolutions is much more beneficial than asking that same question after someone has p0wned the box and carted that data off.

Current IR models don't work because we don't spend enough time looking at what works in the real world and mapping that same sort of mechanism into the digital realm.

Why current IR models don't work

A while ago, a buddy of mine was on travel, leaving his family at home. He called home from the airport whilst awaiting a flight and got some interesting news. He'd winterized their house several months earlier, which included shutting off the hose bibs from inside the house and purging the water from the pipes so they didn't freeze and burst. Well, about 36 hours prior to his call, there had been a need to run water from the faucet on the back of the house, so his wife had turned the hose bib on. However, being unfamiliar with all of the pipes and everything running through the closet, she'd also accidently turned off the gas to the house, which caused all of the pilot lights (on the gas range, in the gas fireplaces, and for the water heater) to go out. Listening to this, my friend started asking questions of his wife, many of which were answered "I don't know".

Listening to him tell this story, it became clear what his concerns were, but I wanted to hear it from him. While his wife was saying, "honey, you're going to come home from a long flight with several stops, layovers, and delays and not have any hot water to shower...sorry", he was hearing that there were possibly places in the house where gas could be seeping into the house. Thankfully, the gas used in homes smells, so it can be detected, but if the source is in an enclosed room...you see where I'm going with this.

I thought about this for a bit while I was running the other day, and it occurred to me that this almost exactly mirrors current incident response models. I know, I know...bear with me here. Consider the wife in this story to be the CEO (hey, don't we all???), and my friend his a CISO or security admin. The kids and pets are the staff. In this case, we have an "incident"...the gas coming into the house was shut off long enough to cause the pilot lights to go out. The CEO, based on her sphere of perception, understands the issue to be one of inconvenience...lack of hot water means no hot shower. She feels that they can wait until my buddy gets home to deal with it. However, my friend sees an even bigger "threat"...not only to property (ie, his house) but more importantly, to the health and safety of his family (corporate officers failing to accurately identify critical assets).

In response to this, my friend decided to embark on a training and educational approach to changing the "corporate culture" of his family to be even more cognizant of the issues and potential impacts of such incidents. In part, this is where the story takes a turn and ends differently from what happens in business (corporate, government, etc.) organizations today.

So how does all this apply to these business organizations? Look around...or look here. It's quite a long list, but see any duplicates, either in what reportedly occurred, or in the name of the organization?

So, how do we fix this, you ask? Glad you asked!! Apparently, security overall (and not just IR) is something that is not part of the "corporate culture" of many organizations. Responding to incidents often times gets me nothing but blank stares when I ask questions about the status of systems, where systems are "located" in relation to each other, etc. That's nothing unusual.

Speaking of "corporate culture", when I was in the Marine Corps, we had a corporate culture, one that was easy to remember - "every Marine a rifleman." Basically, what that meant was that every single Marine, officer or enlisted, must be qualified in care, feeding, and effective and deadly use of the M-16 rifle. Every Marine received training in its use, and had to go through annual requalification. The same was true for officers...up through the rank of Captain, every officer had to qualify annually with the M-16 as well as their own TO weapon, the M-9. But this wasn't all we did...this was part of the many things we did, but it was part of our corporate culture. I believe this served us well in many incidents, particular in Iraq, where the "front lines" were often right in front of you, and it didn't matter if you were an infantry Marine or a cook or a Motor T driver.

My point is that we can talk about IR all day long, but things won't change unless someone with the ability to change the corporate culture does so. Everyone, particularly the IT staff, needs to be more security conscious, and be more familiar with the assets their protecting. What are the critical assets of the company, as defined by the CEO? How does an IT admin's job of maintaining servers and systems relate to accomlishing the mission of protecting those assets?

Also, consider this...in the military, everyone's trained in "immediate actions". If an M-16 jams, every Marine knows the immediate actions to perform to get that weapon back into service. Marines on patrol know that if caught in a near ambush, then the response is to attack the source of the ambush, immediately and with maximum violence. Consider this...why can't the IT staff be trained in "immediate actions", as well? If unusual traffic is seen on the network or anomolous behavior appears on a system, there are things that the IT staff can do immediately to identify the issue. There are other things that they can do in order to quickly address the situation.

It all starts with a top-down approach to the corporate culture. From there, IT staffs need to receive training...functional, hands-on training that applies to the systems they work with every day. Going away to Linux-centric training for someone in an all- or predominantly-Windows shop is a waste of time and money. There needs to be core, central set of knowledge and training that every IT staff member receives and is responsible for knowing...in the Marines, every officer, be they a pilot, a communicator, or a supply officer, goes through the same training at The Basic School. From there, certain members of the IT staff should receive specific training based on their areas of responsibility, be they routers, firewalls, servers, applications, etc. They need to understand these areas and systems inside-out, in much the same way a Marine can field strip, clean, and reassemble the M-16.

So, if you're reading all this and you're not someone in the IR business, you're probably thinking, "oh, he's just saying this so that he can get our money." No, I'm saying this so that you don't loose my data...my SSN, my credit card number, etc. Or my wife's. Or, several years from now, my daughter's or my niece's and nephew's. We see it in the media all the time, with these high profile "hacks" involving someone breaking in and stealing data, or sensitive personal information being lost. Or critical applications and systems being taken over, compromised, and p0wned. These incidents are no longer the result of joy riders on the Information Superhighway...that ended a while ago. There is a profit motive behind this...these crimes are being committed because there is money involved. There needs to be a shift in the corporate culture such that this data is better protected, and when something does happen, the response is something more than paralysis. What would you do if you had a fire in your home? Would you evacuate your family, and if the fire were small and isolated enough, would you attempt to put it out? Would you call the fire department? Dumb questions, I know...but when incidents are occurring today, many corporate cultures make it okay to ignore the situation and simply go back to sleep. Worse, the situation is recognized as something unusual, but everyone is paralyzed, and the fire department gets called only after the house has burned down.

Thoughts?