Programming and DFIR

I was browsing through an online list recently and I came across an older post that I'd written, that had to do with tools.  In it, I'd made the statement, "Tweaked my browser history parser to add other available data to the events, giving me additional context."  This brought to mind just how valuable even the smallest modicum of programming skill can be to an analyst.

This statement takes understanding data structures a step further because we're not simply recognizing that, say, a particular data structure contains a time stamp.  In this case, we're modifying code to meet the needs of a specific task.  However, simply understanding basic programming principles can be a very valuable skill for DFIR work, in general, as the foundational concepts behind programming teach us a lot about scoping, and programming in practice allows us to move into task automation and eventually code customization.

Note
David Cowen has been doing very well on his own blog-a-day-for-a-year challenge, and recently posted a blog regarding some DFIR analyst milestones that he outlined. In this post, David mentions that milestone 11 includes "basic programming".  This could include batch file programming, which is still alive and well, and extremely valuable...just ask Corey Harrell.  Corey's done some great things, such as automating exploiting VSCs, through batch files.

Scoping
My programming background goes back to the early '80s, programming BASIC on the Timex-Sinclair 1000 and Apple IIe.  In high school, I learned some basic Pascal on the TRS-80, and then in college, moved on to BASIC on the same platform.  Then in graduate school, I picked up some C (one course), some M68K

assembly, and a LOT of Java and MatLab, to the point that I used both in my thesis.  This may seem like a lot, but none of it was really very extensive.  For example, when I was programming BASIC in college, my programs included one that displayed the Punisher skull on the screen and played the "Peter Gunn theme" in the background, and another one interfaced with a temperature sensor to display fluctuations on the screen.  In graduate school, the C programming course required as part of the MSEE curriculum really didn't have us to much more than open, write to or read from, and then close a file.  Some of the MatLab stuff was a bit more extensive, as we used it in linear algebra, digital signal processing and neural network courses.  But we weren't doing DFIR work, nor anything close to it.

The result of this is not that I became an expert programmer...rather, take a look that something David had said in a recent blog post, specifically that an understanding of programming helps you put your goals into perspective and reduce the scope of the problem you are trying to solve.  This is the single most valuable aspect of programming experience...being able to look at the goals of a case, and break them down into compartmentalized, achievable tasks.  Far too many times, I have seen analysts simply overwhelmed by goals such as, "Find all bad stuff", and even when going back to the customer to get clarification as to what the goals of the case should be, they still are unable to compartmentalize the tasks necessary to complete the examination.

Task Automation
There's a lot that we do that is repetitive...not just in a single case, but if you really sit down and think about the things you do during a typical exam, I'm sure that you'll come across tasks that you perform over and over again.  One of the questions I've heard at conferences, as well as while conducting training courses, is, "How do I fully exploit VSCs?"  My response to that is usually, "what do you want to do?"  If your goal is to run all the tools that you ran against the base portion of the image against the available VSCs, then you should consider taking a look at what Corey did early in 2012...as far as I can see, and from my experience, batch scripting such as this is still one of the most effective means of automating tasks such as this, and there is a LOT of information and sample code freely available on the Interwebs for automating an almost infinite number of tasks.

If batch scripting doesn't provide the necessary flexibility, there are scripting languages (Python, Perl) that might be more suitable, and there are a number of folks in the DFIR community with varying levels of experience using these languages...so don't be afraid to reach out for assistance.

Code Customization
There's a good deal of open source code out there that allows us to do the things we do.  In other cases, a tool that we use may not be open source, but we do have open source code that allows us to manipulate the output of the tool into a format that is more useful, and more easily incorporated into our analysis process.  Going back to the intro paragraph to this post, sometimes we may need to tweak some code, even if it's to simply change one small portion of the output from a decimal to hex when displaying a number.  Understanding some basic coding lets us not only be able to see what a tool is doing, but it also allows us to adjust that code when necessary.

Being able to customize code as needed also means that we can complete our analysis tasks in a much more thorough and timely manner.  After all, for "heavy lifting", or highly repetitive tasks, why not let the computer do most of the work?  Computers are really good at doing the same thing, over and over again, really fast...so why not take advantage of that?

Summary
While there is no requirement within the DFIR community (at large) to be able to write code, programming principles can go a long way toward developing our individual skills, as well as developing each of us into better analysts.  My advice to you is:

Don't be overwhelmed when you see code...try opening the code in a text viewer and just reading it.  Sure, you may not understand Perl or C or Python, but most times, you don't need to understand the actual code to figure out what it's doing.

Don't be afraid to reach out for help and ask a question.  Have a question about some code?  Reach out to the author.  Many times, folks crowdsource their questions, reaching to the "community" as a whole, and that may work for some.  However, I've had much better success by reaching directly to the coder...I can usually find their contact info in the headers of the code they wrote.  Who better to answer a question about some code than the person who wrote it?

Don't be afraid to ask for assistance in writing or modifying code.  From the very beginning (circa 2008), I've had a standing offer to modify RegRipper plugins or create custom plugins...all you gotta do is ask (provide a concise description of what's needed, and perhaps some sample data...).  That's it.  I've found that in most cases, getting an update/modification is as simple as asking.

Make the effort to learn some basic coding, even if it's batch scripting.  Program flow control structures are pretty consistent...a for loop is a for loop.  Just understanding programming can be so much more valuable than simply allowing you to write a program.

Contributing to the Community

So, here we go with my first post of 2012...

Not long ago, I posted some thoughts on how analysts can contribute to the DFIR community.  What I wanted to do was offer suggestions to those within the community who had read Rob's post and had maybe thought that writing a batch script or a full-on program was a pretty daunting endeavor, let alone standing up an entire project such as log2timeline.  Recently, there have been some interesting exchanges on Twitter, and I think that this is a good time for folks to consider how they might make a contribution to the DFIR community in 2012.

I think that one of the biggest misconceptions within the DFIR community is how one person can make a contribution.  I think that a lot of analysts get themselves into a nice, comfortable little place called "complacency" through paralysis.  What I mean by that is that too many analysts will convince themselves that they can't contribute to the community because they don't know how to program.  Some analysts seem to look around, see how some others contribute, and say to themselves, "I can't contribute to the community because I don't know how to program."  I think that this applies to other ways of contributing besides just programming, and I think this is just an excuse, and a pretty bad one at that.

At the first Open Source Digital Forensics Conference (put on by Basis Tech and Brian Carrier...thanks, Brian!) in 2010, toward the end of the conference a member of the audience asked, "Why can't I dump and parse memory from Windows 7 systems??"  To that, a prominent member of the community asked, "What have you contributed?"...the idea being that one can't...and shouldn't...simply sit back and expect everything to come to them for free without putting forth something.  But the response didn't center around someone's ability to code in Python...rather, it was about other aspects of how someone could go about making contributions.  Had the person asking the question offered up extra hardware to support development efforts, offered to write up documentation, or just said, "Thank you"?

Not everyone (me, in particular) expects all DFIR analysts to be able to write code.  There are a lot of really good analysts out there who don't go beyond simple scripts and regexes, and others who don't code at all.  Corey Harrell has made some pretty fantastic contributions, simply by having written a number of batch scripts, in essence tying together other tools.

There are a number of ways that anyone can make a contribution to the community, and most do not require the ability to write code.  Some of the ways you can do that in 2012 include, but are not limited to, the following:

Case Studies - One of the things that is definitely true about the DFIR community is that folks really love hearing how others have done things.  Many of us encounter the same or similar cases, or have those "one-offs" that don't get seen very often, and we all enjoy hearing about novel approaches to solving problems.  Admit it...just to be in this community, you have to have a little nerd in you, and there's a part of you that likes to hear how someone else may have overcome an obstacle that they encountered.  Keeping that in mind...that you like to hear those "war stories"...consider sharing yours with others.

If you can't program, but you are able to download and use a tool (commercial, open source), then a great way to make a contribution is to comment on how you used the tool.  Was it useful/sufficient/accurate?  Was it easy to use? 

Ask a question - Very often, this is a huge contribution!  Asking a question very often has the effect of sharing your perspective on things with others, and seeing different perspectives can very often be extremely beneficial.  For example, I like to dig into the Registry, but many times I don't really know what it is that other analysts find useful, or what would be most valuable to their case work.  If someone asks a question about the Registry (specific keys/values, how to locate something, etc.), that gives others a perspective of how they look at things, how they approach problems, and how they go about solving them...and many times, just this perspective can help someone else with an issue that they're working on.

If you download a free, open source tool and you're having trouble using it, start by asking the author for pointers or assistance.  Maybe there's something  wrong with how you're using it...maybe you're missing a switch.  Or maybe you're running an MBR parsing tool against a .vmem file (hey, I can't make this stuff up).  Asking the author your question gives them insight into who's using the tool, how it's being used, and maybe how to improve it...and it's far better (and much more appreciated) than going to a public forum and stating, "...this tool don't work."  

Here's a great example of how you can ask a question...the example is specific to the DFIROnline meetups, but it demonstrates how a number of folks can come together to provide different perspectives when addressing issues and answering questions.

Review a blog post, book or paper - Don't code, and can't share case studies?  Don't feel as if you can maintain a blog?  No problem.  How about this...have you read Corey's blog?  Did you find something interesting in one of his posts?  Did you think what he posted was cool?  Did you tell him about it?  Did you comment on it, and share your thoughts?  If you can't do so directly to the comment section of his blog, have you considered sending him an email?

If you decide to review a book, consider doing something just a little bit more than repeating the table of contents.  While authors appreciate knowing that someone picked up their book, they appreciate it even more knowing (a) if the material was useful, and (b) how it was useful.  Again, sharing your perspective can be very valuable.

What Not To Do
In 2012, consider what you can do, and consider not spending time worrying about (or stating) what you can't do.  "I can't program" isn't a contribution to the community.

If you feel strongly enough to download a tool that someone wrote, take the time to thank them.  Okay, you may not have time to do any in-depth testing of the tool, and you may have downloaded it just to have it for future use, but you had the interest and took the time to download the tool.  Now, this doesn't mean that you have to reach out and thank someone every time you actually run the tool, but just having the courtesy to thank someone for their efforts can go a long way toward the development of that tool, or others.

I think that most times, folks look at what others do to contribute within the community and think to themselves, "I can't program, I don't have the time to write books, and I don't like public speaking, so I can't contribute."...and to be honest, nothing could be further from the truth.  No one is expected to make contributions all the time...hey, we're people and have lives.  But there's really no reason why, if you're capable of doing the work that we do in this field, that at some point in the space of a year, you can't make some contribution of some kind, no matter how small.

Clicking "Like", "+1", or re-tweeting a post isn't a contribution, as it doesn't add anything to whatever it is you're commenting on.  If you like something enough to click a button, take a moment to say what you like about it, or how it was useful or valuable to you.  The same holds true for book reviews...if you're going to review a book, reiterating the table of contents isn't a review; however, describing what you found valuable (or not) and how it was valuable to you is what most of us look for in a review, right?  "The car has cup holders" isn't so much a review of a vehicle as "the car has three cup holders, none of which the driver can easily reach."

What if you're one of those folks who is bound by a corporate policy or something else that prevents you from contributing?   What does this policy prevent you from doing?  You can't talk about casework?  That's fine...in a lot of cases, many of us are thankful that you're the one dealing with the details of the specific case and not us. 

Final Thoughts
No one of us is as smart as all of us, and the best way to get smarter within this community is to engage with each other, share perspectives and thoughts, and then build from there.

Sometimes, the biggest contribution you can make is to simply thank someone for their contribution and efforts.  Seriously.  This means a lot.  Think about it...if you did something, no matter how small, wouldn't you appreciate it if someone said, "thanks"?

As an example of mandating that members contribute, check out the NoVA Hackers blog...they follow the AHA model of participation...which, in short, says that if you want to remain a member, you have to participate.  The AHA page lists what you must do in order to remain a member of their group, and remember, membership is voluntary, so one must accept these conditions upon becoming part of the group.

Here's looking forward to a great 2012, everyone...

Addendum:
Erika posted on this topic, as did Ken...both are excellent posts that take the conversation regarding contributing to the community several steps further.  More than anything, I think that it's valuable to hear from others in this regard, in particular those within the community who might say, "...I want to contribute, but I don't think I have anything of value to share."  I've said it before and I'll say it again...sometimes, the best question to ask is "why?".  When I was on the IBM ERS team, we brought Don Weber on board, and besides just being a great guy, he'd ask me "why?" during engagements, and that got me to re-think (and in many cases, justify) my base assumptions with respect to next steps on the engagement.  That isn't to say that it changed what I was going to do as the engagement lead, but it did open up discussion so that Don could understand what I was thinking.  It also afforded me the opportunity to get Don's input, which was invaluable.  Sometimes, the most information can come from questions such as, "why did you do it this way?" or "how did you go about accomplishing this?"

An additional thought or two that might help...choose your circles and choose your medium.  If you don't feel comfortable posting to an open list, find some other medium.  One way to ask the questions you may have would be to send them directly to someone you trust, and either ask them to post them as a proxy, or just see if they know the answer.

What happens sometimes is that someone will ask a question, and the response will be terse or concise, or include a link to LMGTFY, or just be, "...which OS?" These happen very often when little initial thought, effort or research is put into the question, and are often viewed by the recipient as a "slam".  I think that what folks really don't realize is that you can't convey tone in 140 characters or less, so many times it's assumed because it can't be implied.  If a medium like Twitter (limit of 140 char) leaves you thinking to yourself, "...hey, I asked a question, but that guy's response being mean to me...", then maybe that isn't the right medium for what you're trying to accomplish. 

Not every medium is suitable for everyone in this industry.  For example, the Win4n6 Yahoo group currently has 830 members listed in the forum, and only about a dozen or so "regular" contributors.  As I approve every membership application (solely as an effort to keep bots out), I see all of those who post during the application process that their reason for joining is to "contribute" and "take part" in discussions...and we never hear from them again.  So maybe this medium isn't something that works for them.

Final thought...this time around, anyway...if you don't have the time to put into a question, maybe it's not a good time to ask it.  I'm not saying that you shouldn't ask your question, I'm just suggesting that if you don't have the time to do some research on your own, or don't have the time to let folks know that you're looking at a Windows 7 system and not Windows XP (if you don't know why that matters, please feel free to ask...), maybe now is not a good time to ask the question.  Maybe it's better to hold off until you have more time to do a thorough job, rather than just throwing it out to "the collective".  Just something to think about...Ken referred to this when he mentioned "...ask stronger questions about forensic topics."  Excellent point, Ken.

Even More Stuff

DFIROnline
Last Thu, we had (at one point) 32 attendees to the #DFIROnline online meetup, and my impression is that overall, it went pretty well.  Mike took the time to post his impressions, as well.

I think it would be very helpful to hear from others who attended and find out what they liked or didn't like about this format.  What works, what doesn't, what would folks like to see?  I know that with the NoVA Forensics Meetups, most (albeit not all) of the comments about content that I received were from out of town folks, and included, "...set up a meetup in my town...".  Well, Mike's brought that to you...in fact, you can battend from anywhere.  Mike's survey results indicated that case studies and malware analysis are things that folks are interested in, and that's a great start.

Also, I've been thinking...what do folks think about moving the NoVA Forensics Meetups to DFIROnline?

For those interested, I posted my slides (in PDF format) to the Win4n6 Yahoo Group Files section.

A a great big, huge, Foster's thanks to Mike for setting this up. 

Cool Stuff
If you do timeline analysis, David Nides has posted a great little log2timeline cheat sheet over on the SANS Forensics blog.  David made this cheat sheet available at the recent SANS360 event as a single laminated sheet...if you weren't able to make it and didn't get one, download the PDF and print out your own.  The content of the cheat sheet goes right along with Rob's SANS360 presentation, which you can watch here (actually, it's the entire set of presentations).

A huge thanks to David for putting this together and making it available.  This is another great example of how someone can contribute to the community, without having to be able to stand up in front of people, or write code. 

Jump Lists
I recently received a question about Windows 7 Jump Lists, and dusted off some of the code I wrote last summer for parsing Jump Lists.  Yes, it's in Perl...but the way I wrote it was to use just core Perl functions (i.e., no esoteric, deprecated, or OS-specific modules) so that it is platform-independent, as well as much easier to install and run.  Also, I wrote it as Perl modules, so I have additional flexibility in output formats...in short, I can have a script spit out text in a table format, CSV, or even TLN format.

If you haven't yet, check out Mark Woan's JumpLister...it's at version 1.0.5, and does a great job of parsing not only the LNK streams, but also the DestList stream (partial structure of which was first publicly documented here).  It also maps the AppId to an application name...a list of which can be found here, and here. 

Another use I've found for this code is Windows 8 forensics.  I've had a VirtualBox VM of Windows 8 Dev Build running, but recently set up a laptop (wiped XP off of it forever) to dual boot Win7 & 8, so that I could look at some of the various artifacts available, such as wireless networks within the Registry, the use of a Windows Live account to log into Win8, and the Jump Lists...yep, Win8 uses Jump Lists and at this point, they appear to be consistent in format with the Win7 Jump Lists.

Speaking Engagements
My upcoming speaking engagements include the DoD CyberCrime Conference (the conference even has a Facebook page), where I'll be presenting on Timeline Analysis.  I've also submitted to the CfP for the SANS Forensic Summit this next summer (topic: Windows 7 Forensic Analysis), so we'll see how that goes.