Thursday, December 17, 2009

When a tool is just a tool, pt II

Okay, this is part II for this post, because I posted an awesome rant to a thread in one of the forums, and I wanted to include that here, as well, because it kind of applied...and it's my blog, I can do what I want. ;-)

The thread can be found here, and the post I'm referring to is on the third page, in response to someone mentioning, "...don't forget that if you wind up working civil or criminal cases your tools are going to get challenged in court. Open source tools are more vulnerable to attack by the opposition than commercial tools that are standardized."

My rant, if you want to call it that, had to do with what I see as a gross misconception with respect to court cases; specifically, some commercial tools are used primarily because the analysts themselves are familiar with them, and perhaps as a result, the players in the court system have also become familiar with them. That is to say that some commercial tools are recognized within the court system, and therefore, not a great deal of additional explanation is required.

As such, it isn't the tools that are challenged in's the analyst and their processes.

Also, I think another huge issue that doesn't appear to be considered when folks are making statements such as the one I quoted above is that an analyst just doesn't decide one day to walk into court, take the stand, and testify. It just doesn't happen that way.

Instead, the attorney you're working with or for (prosecution or defense) is gonna want to know your answers before he asks you questions on the stand, and the fact that you're testifying and what you're testifying about are going to be part of the discovery the other side is going to have a chance to cross-examine you. As such, if there's anything that would lead the attorney you're working with to suspect that you being cross-examined would sink the case, they're not going to put you on the stand. The same is true if he or she simply doesn't feel that the results of your analysis are pertinent to their case.

With me so far? I guess what I'm saying here is that there's a heck of a lot that goes on before an analyst ever gets to the point of approaching the stand in a court of law.

Now, can we agree that an acquired image is nothing more than a stream of bits, 1s and 0s, in a file on a disk? If we can agree to that, and if the integrity of that data, that stream of bits, can be verified and validated, then why does it matter what tool I use to extract data? What does it matter if an analyst determines that an illicit image is in the image using some commercial tool's Gallery View, or by mounting the image read-only with ImDisk and viewing the image file through Windows Explorer? Regardless of the tool used, the image was there, and that doesn't change. The same is true with other card numbers, other sensitive data, etc. One tool doesn't necessarily magically make it visible where some other free and/or open source tool wouldn't be able to extract the same data.

Now, don't get me wrong...I'm not against using commercial tools. I've used them myself (and I'm seeking therapy...just kidding) when the need has arisen. But the fact of the matter is that commercial forensic applications are just like any other tool, with their own inherent strengths and weaknesses. In some cases, I've found that processes using open-source and free tools, such as timeline creation tools, have allowed me to structure data for analysis in ways not possible through the use of commercial tools. In other cases, I've found short-comings in using commercial tools, just as I've found short-comings in using open-source and free tools. That doesn't mean that commercial tools shouldn't be just means that all tools should be considered just for what they

What should matter most is the process used and documentation created by the analyst. If you thoroughly document what you've done, then why shouldn't you be able to testify about it on the stand, regardless of the tools used? I know a few analysts who've documented their work such that someone else (i.e., LE) could validate their findings via commercial tools (because that's what the LE analyst was most comfortable with) and then testify about the "findings".

So, what do you think? Are open-source tools "more vulnerable to attack"? Why does it matter if I extracted a Registry hive file from an image, and then extracted the LastWrite time from a specific key using a Perl script? Or a hex editor? Or if someone else did the same thing, but through EnCase or FTK? The fact of the matter is that if you go to that location on the disk or within the hive file, extract 64-bits, everyone who does so should arrive at the same answer...right?

Or should I just curl up in the fetal position in the corner of my office, and rock myself to sleep, chanting, "I'm a pretty girl" over and over again?


Unknown said...

I think the answer to your last question is: It depends on how much you like chanting (though that may not be the best mantra for you). :)

But, seriously, thanks for posting this. There are those of us who work IR/Forensics within organizations and have never had call to interact with the legal system. As such, I think there's a lot of FUD surrounding going to court. I've seen it, and felt it, a number of times. It's useful, in that regard, to hear voices of experience describe the process. Especially in a good rant.

Thanks again.

Jimmy_Weg said...

I'd be surprised if a court even heard the names of the tools you used in an exam. While I mention some of my tools in my exam report, the only tool I mention on the stand is my imaging/hashing tool. I've been on the stand a "few" times and the opposition has never questioned a specific tool. They have, however, questioned my validation.

Your comments on the forum were on the money with respect to everything coming down to the examiner's creds. I suggest that open source tools actually are challenged less, because fewer folks know how they work. Can someone cite one that is flawed? I'll bet somone can cite a commercial tool, or at least an aspect of one.

I use all kinds of tools, though I like my (certain ones) commercial tools better, in the sense of an overall approach to an exam. I like RegRipper and RipXP. When I first used them, I checked them against Registry Viewer, which is a pretty good commercial tool. What it comes down to is this: if I qualify as being a knowledgable witness, I'm going to survice any tool challenge because I use tools that consistently produce accurate results.

As far as the basis for the "rant" is concerned, I can defend the remark that spawned your response by saying that it is generally easier to say that one million people rely on EnCase, and I don't know how many people use the SleuthKit. For the record, I use neither, and both may be superb tools. Numbers give comfort to the court, be that right or wrong. If I may be a little redundant, it really doesn't matter what you use, as long as you know that the results are accurate and can explain why you know that they are accurate.

H. Carvey said...

Guys, thanks for the comments...and Jimmy, thanks for that validation.