Forensic Scanner

With the manuscript for WFA 3/e submitted, I have time now to focus on other projects (while I wait for the proofs to review), including the next step for or next generation of RegRipper, which is something I call the "forensic scanner"...for now, anyway, until I come up with a really cool name for it.  All new projects need a cool name, right?

As I work on developing this project, I wanted to share some of the thoughts behind it, in part to see if they make sense, but also to show the direction of the project.

Why?
Why have a "forensic scanner" at all?  That's a great question.  Other areas of information security have scanners...when I started my infosec career after leaving the military, I used ISS's Internet Scanner.  Consider Nessus, which was the inspiration behind the design for RegRipper.  And there are others...the idea being that once you've discovered some artifact or "check", you can automate that check for future use, without having to memorize the specifics.  After all, by creating a "plugin" for the check, you're documenting it.  Another strength of something like this is that one analyst can create a check, documenting the plugin, and provide it to others, sharing that information so that those other analysts don't have to have the same experiences.  This way, a team can focus on analysis and benefit from the analysis performed by others.

Look at it this way...do you want to do the same checks that you always do for malware, manually?  Let's say that a customer believes that a system is infected with Zeus/ZBot...do you want to manually check for sdra64.exe every time?  What about the persistence mechanism?  What if the persistence mechanism is the same, but the file name has changed?  What if you could automate your entire malware detection checklist?  Most forensic analysts are familiar with the detection rates of AV products, and sometimes it's a matter of looking not for the malware itself, but rather the effects that the malware had on it's ecosystem...what if you could automate that?

Purpose
The purpose of the forensic scanner is not to replace anything that's already out there, but instead to augment what's currently available.  For instance, the Digital Forensics Framework (DFF) was recently updated to version 1.2 and includes some great features, none of which the forensic scanner includes (i.e., search, etc.).  The forensic scanner is a targeted tool with a specific purpose, and not a general analysis framework.  Instead, much like other scanners (Nessus, ISS's Internet Scanner, etc.), the forensic scanner is intended to fill a gap; using frameworks and applications (ProDiscover, etc.), analysts will find artifacts and indicators of compromise, and then document them as plugins as a means of automation.  Then whenever the scanner is run against an acquired image, checks for those artifacts, as well as processing and even inclusion of references, are run automatically.  This is intended to quickly allow the analyst to analyze, by running checks that have already been discovered.  Learn it once, document it, run it every time.

Scanner Attributes
Here are some of the forensic scanner attributes that I've come up with:

Flexibility - From the beginning, I wanted the scanner to be flexible, so I designed it to be run against a mounted volume.  You're probably  wondering, "how is this flexible?"  Well, how can you mount a volume, particularly in read-only mode?  You can convert a raw/dd image to a .vhd file (using vhdtool.exe, or the VirtualBox convertfromraw command), and mount that .vhd file read-only via the Disk Management tool.  You can use FTK Imager, ImDisk, or another image mounting tool.  You can also connect to a remote system via F-Response and run the scanner.  You can mount an image by converting it to a .vmdk file, and mount is as an independent, non-persistent hard drive.  Using either the .vhd or .vmdk methods, you can also mount VSCs as volumes and scan those; as with RegRipper, a CLI engine for the scanner can be included in a batch file to automate the scans.

Even though I'm writing the plugins that I'm using for Windows, there's nothing that really restricts me to that  platform.  The scanner is written in Perl, and can be run from Linux or MacOS X (the GUI would require the appropriate modules, of course...) and run against pretty much any mounted volume.

Force Multiplier - One of the things I really like about RegRipper is the ability to write my own plugins.  So, let's say I find something of interest...I write a plugin for it.  I can (and do) include appropriate references (i.e., to malware write-ups, MS KB articles, etc.) in the comments of the plugin, or even have those spit out in the output.  I can even add an explanation to the plugin itself, in the comments, describing the reasoning behind the plugin, why it was written, and how to use or interpret the output.  That plugin then persists, along with the documentation.  This plugin can then be shared amongst analysts, increasing their capability while reducing their need to experience the same analysis I did to zero.  So, let's say I find something that I'd never seen before and it took me 10 hrs of dedicated analysis to find it.  If there are 5 other analysts on my team (and we're all of approximately equal skill levels), and I share that plugin with all of them, then I've just added to their capability and saved the team 50 hrs of dedicated work.

This section could also be referred to as Preservation of Corporate Knowledge or Competitive Advantage, depending on who you are.  For example, both LE and private industry consultants benefit from retaining corporate knowledge; also, LE would greatly benefit from any plugins shared by private industry.

Knowledge Retention
Within the private sector, the information security industry can be fluid.  Analysts have changes in their lives, or develop new skills (or want to do so) and move on.  Having a means of documenting and retaining their experiences within the environment can be valuable; have a means of incorporating that knowledge directly into the field can be critical.  It's one thing for an analyst to talk about something they found, or write a white paper...it's something else entirely to have a forensic analyst write a dozen or so plugins throughout their tenure and have those available for use, by all of the other analysts, well after he or she has left.

LE experiences something similiar; many times, an analyst receives training, works on some cases, and is then off to do other LE things.  And often, their wealth of knowledge leaves with them.  With a framework such as the forensic scanner, not only is an individual analyst's knowledge retained, but it can be provided other analysts, even ones that haven't been hired or completely trained.

Competitive Advantage is usually associated with private industry consulting firms, but I'm sure that you can see how this would apply.  Any analyst who finds something and documents it through a plugin can then share that plugin with others...100% capability for 0 experience; the time-to-market for the capability is pretty much as long as it takes to open an email and extract the plugins from an attached archive.  Ideally, you'd want to have an "armorer", like a lab tech or analyst who either gets information from other analysts and writes and tests the plugins, or receives the plugins and tests them before rolling them out.  The approved plugins can be placed in an archive and emailed to analysts, or you can use some form of distribution mechanism that each analyst initiates.

Self-Documenting - The forensic scanner has an interesting feature that I'm carrying over from RegRipper - when running, it produces an activity log, collecting information about the scanned volume and tracking the plugins that were run.  So, not only will your output make it clear what the results of the scan were, but the activity log can tell you exactly which versions of the plugins had been run; if there's a plugin that wasn't run, or an updated version of a plugin comes out, you can simply re-run the scan. 

This information can also be used from a debugging standpoint.  If something didn't work as planned, why was that?  How can the process be improved?

Common Format - One of the things that we're all familiar with how there are a number of tools out there that parse information for us, but these tools all have different output formats, and it can be a very manual process to work through Prefetch files, Jump Lists, etc., and have to manually convert all of that information into a common output format.  Even if we get several tools from the same site and author, and we can format the output in .csv or .xml, we still have to run the tools, and manage the output.  Using the scanner, the plugins will handle the output format.  I can write one plugin, and have .csv output...then modify the output in another version of the plugin to .tln output, and include each plugin in the appropriate scanner profile.

Plugins
When scanning a mounted volume, the engine exports a number of variables that you can use to tailor your plugins; however, as the target is a mounted volume, there is no proprietary API that you need to learn.  Want to get a list of files in a directory?  Use the standard opendir(), readdir(), and closedir() function that ship with Perl.  What this means is that learning to write plugins is as easy as learning to program in Perl, and if you don't know (or want to learn) how to program in Perl, that's okay...find someone who does and buy them a beer.

The plugins can also be flexible, ranging from the broad to the narrowly-focused.  An example of a broad plugin might be one that scans the Windows\Temp (or the user's Temp) folder for PE files.  I know how tedious something like that can be...particularly with a reprovisioned system that has a dozen or more user accounts on it...but how would you like to have a report of all of the .tmp files in all of the user's Temp folders that are actually PE files?

A plugin that's a bit more tactical might be one that looks for a specific file, such as ntshrui.dll in the C:\Windows directory.  The "strategic" variant of that plugin might be one to list all of the DLLs in the Windows directory.

However, plugins don't have to be just Perl; using Perl functions, you can also create plugins to run external commands.  For example, you can use strings and find to parse through the pagefile, and retain the output.  Or you can run sigcheck.  Using Perl functions that allow you to launch external commands, you can automate running (and processing/parsing the output of) external commands against the mounted volume.

Deploying the Scanner
I alluded to some of the deployment scenarios for the scanner earlier in this post, but I'll reiterate some of them here because I think they're important.

When I was on the IBM response team (and the ISS team before that), each responder had two MacBooks that we had in our jump kits, as well as Mac Server in our office; lots of horsepower, with a reduced form factor and weight (over the comparable Dell Latitudes).  I opted to primarily run Windows, as I wanted to be as familiar with the most predominant platform that we encountered.  Our team was also geographically dispersed.  So how would something like the scanner be deployed in such an environment?

Now, if we had a central intake point, such as a lab were images were received and processed (image file system verified, documented, and a working copy made to a storage facility) by a lab tech, the scanner could be deployed to and maintained by the lab tech.  Once an image was processed, the working copy could be scanned, and the analyst could VPN into the lab, fire up the appropriate analysis VM, and review the output report from the scanner.

What's coming?
Recently on Twitter, Ken Johnson (@Patories) point out an artifact that he'd found on the Windows 8 dev build, and likely associated with IE 10...a key named "TypedURLsTime".  The data for each of the listed values is a FILETIME object...when the time comes that Win8 is seen on the desktop, this will likely be a very useful artifact to be included in a plugin. 

So, let me ask you this...who's going to remember that when Windows 8 actually hits the streets?  I'm running the dev build of Windows 8 in a VirtualBox VM, as a .vhd file; anyone doing so can easily mount the .vhd (read-only) on their Windows 7 system, write a plugin for the artifact, and there it is...documented.