Sniper Forensics, Memory Analysis, and Malware Detection

I conducted some analysis recently where I used timeline analysis, Volatility, and the Sniper Forensics concepts shared by Chris Pogue to develop a thorough set of findings in relatively short order.

I was analyzing an image acquired from a system thought to have been infected with Poison Ivy.  All I had to go on were IPS alerts of network traffic originating from this system on certain dates...but I had no way to determine how "close" the clocks were for the system and the monitoring device.

I started by creating a timeline, and looking for the use of publicly documented persistence mechanisms employed by the malware in question (Windows service, StubPath values within "Installed Components" Registry keys, etc.), as well as looking for the use of NTFS alternate data streams.  All of these turned up...nothing.  A quick review of the timeline showed me that McAfee AV was installed on the system, so I mounted the image via FTK Imager and scanned it with ClamWin 0.98 and MS Security Essentials (both of which were updated immediately prior to the scan).  MSSE got a hit on a file in the system32 folder, which later turned out to be the initial installer.  I searched the timeline, as well as the Software and System Registry hives for the file name, and got no hits beyond just the existence of that file within the file system.

The system was a laptop system and had a hibernation file, which was last modified the day that the system itself had been shut down.  I exported the hibernation file for analysis, and then downloaded the Win32 standalone version of Volatility version 2.3.1, and used that to convert the hibernation file to raw format, then identify the profile (via the imageinfo plugin) and get a process list (via the pstree plugin).

I got in touch with Jamie Levy (@gleeda), who pointed me to this post, which was extremely helpful.  Based on the output of the malfind plugin, I was able to identify a possible process of interest, and I searched the timeline for the executable file name, only to find nothing.  However, I did have a process ID to correlate to the output of the pstree plugin, and I could see that the process in question was a child of services.exe, which told me not only where to look, but also that the installation process required at least Administrator-level privileges.  Correlating information between the RegRipper samparse.pl and profilelist.pl plugins, I was able to see which user profiles (both local and domain) on the system were members of the appropriate group.  This also provided me with the start time for both processes, which correlated to the system being started.

A fellow team member had suggested running strings across the memory dump, and then searching the output for specific items; doing so, I found what appeared to be the configuration information used by Poison Ivy (which was later confirmed via malware RE performed by another team member).  As a refinement to that approach, I ran the Volatility vaddump plugin, targeting the specific process, and dumped all of the output files to a folder.  I then ran strings across all of the output folders, and found a similar list of items as when I'd run strings across the entire memory dump.

I validated the configuration item findings via the Volatility handles plugin, looking for mutants within the specific process.  I also dumped the process executable from the memory dump via the procmemdump plugin; the first time I did it, MSSE lit up and quarantined the file, so I disabled MSSE and re-ran the plugin (I set my command prompts with QuickEdit and Insert modes, so re-running the command line as as simple as hitting the up-arrow once, and then hitting Enter...).  I was able to ship the dumped PE file and the initial wrapper I found to a specialist for analysis while I continued my analysis.

I then ran the svcscan plugin and searched the output for the executable file name, and found the Windows service settings, to include the service name.  I also ran the dlllist plugin to see if there were any DLLs that I might be interested in extracting from the image.

I also ran the netscan plugin, and reviewing the output gave no indication of network connections associated with the process in question.  Had there been a connection established associated with the process of interest, I would've run Jamaal's ethscan module, which he talked about (however briefly) at the recent OMFW.

I had downloaded the Poison Ivy plugin, which according to the headers, is specific to one version of the PIvy server.   The plugin ran for quite a while, and because I had what I needed, I simply halted it before it completed.

Timing
Something important to point out here is that the analysis that I've described here did not take days or weeks to complete; rather, the research and memory analysis was all completed in just a few hours on one evening.  It could have gone a bit quicker, but until recently, I haven't had an opportunity to do a lot of this type of analysis.  However, incorporating a "Sniper Forensics" thought process into my analysis, and in turn making the best use of available tools and processes, allowed me develop findings in an efficient and thorough manner.

In short, the process (with Volatility plugins) looks like this:
- convert, imageinfo, pstree
- malfind - get PID
- run vaddump against the PID, then run strings against the output
- run handles against the PID
- run procmemdump against the PID (after disabling AV)
- run dlllist against the PID
- run netscan; if connections are found associated with the PID, run ethscan

At this point, I had a good deal of information that I could add to my timeline in order to provide additional context, granularity, and relative confidence in the data I was viewing.  File system metadata can be easily modified, but Windows Event Log records indicating service installation and service start can be correlated with process start times, assisting you in overcoming challenges imposed by the use of anti-forensics techniques.

It's also important to point out that Registry, timeline, and memory analysis are all tools, and while they're all useful, each of them in isolation is nowhere near as powerful as using them all together.  Each technique can provide insight that can either augment or be used as pivot points for another.

Resources
A couple of interesting items have popped up recently regarding memory acquisition and analysis, and I thought it would be a good idea to provide them here:

• Jamie also pointed me to this post, which, while it wasn't directly helpful (the image I was working with was from a Win7SP1 system), it does show the thought process and work flow in solving the challenge
• From MoonSols, some new features in DumpIt
• Mark Baggett's ISC Diary posts on winpmem and its uses: here, and here
• Page_brute - check out how you can incorporate the pagefile into your analysis, until Volatility v3 hits the streets

Tools, Malware, and more conference follow-up

Tools
Arsenal Recon - Image Mounter utility: the web site says that the tool is "currently geared towards digital forensics and incident response developers, but we have released a sample GUI and command line mount/management utility known as "MountTool" as a proof of concept."

The web site goes on to say, "We have heard that others in the DFIR community are already building better GUIs and extending the functionality of Arsenal Image Mounter..."; this is great to hear, and I hope that this is something that becomes public.  Having a proof of concept tool is great to test things out, but seeing something that extends the capabilities of the utility is even better, and I think, far more useful.

Willi Ballenthin's EVXTract: Here is Willi's presentation from the SANS Summit in Prague, on 6 Oct 2013.  In the sixth slide, Willi mentions the 6 steps to carving EVT records from unallocated space or other unstructured data.  Somewhere in there, perhaps step 4a, should be "given the record size, read that number of bytes into the buffer, and the last 4 bytes should be the same as the first four bytes".  This is the "sanity check" I've used as an initial step for validating EVT records.

Anyway, what Willi's presentation and materials really demonstrate for me is the move to carving for records rather than full files.  Yes, others have been doing this for a while, in particular the MagnetForensics folks with their IEF product, but that's not an open source tool.  Understanding data structures and going after the individual records can be far more fruitful than going after full files.

Mari has written up an excellent post detailing how to retrieve deleted data from a SQLite database; she's even provided a Python-based parser.  This is yet another aspect of data retrieval that is based on a detailed knowledge and understanding of data structures, similar to what was discussed here (IE index.dat deleted records), and here (MFT resident file residue).

Lance's post regarding the tools he turns to during examinations, broken down by tiers.  His list is very useful in that it illustrates a thought process that others can look at for comparison.  I can't say that I have a similar breakdown, which probably has more to do with the types of analysis I tend to get engaged in, and the prior information that each of us may have available.

Of course, how a tool is used can also be very valuable information.  For example, from Lance's post, as

well as what I saw at OSDFC, folks use RegRipper, but apparently not the way I use it.  It seems that the predominant means of using RegRipper is to just run all the plugins against the available hives.  That may be good for some folks, but like Corey Harrell, I tend to take something of a more targeted approach.  Often during my own examinations, I don't need everything...in fact, getting everything can often be an expenditure of time and effort that I can't afford, as I have to sort through a lot of data that isn't particularly useful or beneficial.  While in some instances it is useful to take a broader approach, there are times and circumstances when it's far more beneficial to take something of a targeted approach, if for no other reason to do a quick check and get something of value quickly, leaving the everything for later.  At times, I've found it very beneficial to use a Registry viewer (WRR is pretty good, although it doesn't handle "big data" values) for viewing and searches, then run specific RegRipper plugins, and then have a plugin open in an editor for tweaking.  That's kind of how I went about this work involving shell items.

Based on recent conferences, I'm going to have to look at adding TSK Autopsy  to one of my own tiers.

Malware Response and Analysis
Claus has a new post up regarding his Anti-Malware Response Go Kit; in this case, Claus discusses more about his requirements and process, whereas some of his previous posts have been admittedly valuable lists of tools.  The one correction I would offer up, however, is in the section where he briefly discusses RegRipper: "Plugins are developed by the community..." - actually, not so much, really.  Yes, there are a number of plugins developed by other folks, for which I am very thankful, and offer a huge thanks to them for doing so. Other plugins were developed or extended because someone (Corey Harrell) saw the need and reached out to me, but for the most part, the majority of the development of plugins have not been something that's sought after by the community at large.  I have updated a couple of plugins (appcompatcache.pl) to support Windows 8/8.1 thanks to the suggestion and submission of additional test data by Eric Z.  Also, I've written a number of malware-specific, "tactical" plugins based on something I've seen, usually within just a couple of minutes of reading the material...but again, these are not something that have been requested or sought by the community.

From the Handler's Diary blog, Analyzing Malicious Processes: very cool use case for Volatility.  Not only does the post illustrate how to use Volatility, but it also clearly illustrates what an artifact (in this case, of lateral movement) "looks like", which is something that we often don't see addressed.  Even though artifacts are mentioned, many times, there's very little information about what to look for on disk or in memory with respect to those artifacts.

Also from the Handler's Diary - jackcr's DFIR challenge summary; Jamie Levy (@gleeda) referred to this challenge in her "Every Step You Take: Profiling the System" talk at OMFW2013.

From Lenny Zeltser, a list of XOR obfuscation examination tools.

Win32/Trooti is an example of malware that "parks" on a vacant SvcHost value.  [TrendMicro, M86 Security]  It's interesting how the different write-ups address the particular value on which the malware "parks"...some tell us that it happens, others share the results of their particular analysis.  More than anything else, I think that this information can help us in reviewing and if necessary, updating our own malware detection techniques.

Conference Followup
There's another write up available regarding recent conferences, in this case, the same ones I was able to attend.  As such, the write-up provides something of a different perspective.  You can find the OMFW & OSDFC Recap post over at the HiddenIllusion blog site.  This post is more of a blow-by-blow of the conferences, and it does give an interesting perspective on the conferences.  A couple of things I wanted to comment on (and this doesn't mean that I disagree...):

From the general notes: "I was told that my tweets and recap post of last years activities was helpful to those who couldn't attend..." - this is much more of a truism than most really recognize, I think.  Not everyone can attend conferences, but very often we're able to learn more about which conferences or events would be most beneficial to us by what others share about their experiences; what was good, what wasn't so good, what there should be more or less of, etc.  This sort of feedback can not only be valuable to folks who are planning out their conference attendance, but it can also be extremely valuable to conference organizers and presenters, as well.  For example, in the OSDFC section, there was this quote:

On the disappointing side - I did feel like I was seeing a noticeable amount of people doing the same things as others have already done.

That's extremely valuable to know.  Yes, you're right...there was content that would make folks think, "wow, I've seen this stuff before", and there could be any number of reasons for that.  Sometimes, I think folks pursue something because they're seeing it for the first time, and they don't do a comprehensive literature search prior to kicking off their research.  Sometimes, this is good, because it's done for educational purposes, and the researcher may find something new.  Remember though...this year, the conference program was crowd-sourced, so it wasn't Brian and the BasisTech staff who decided which presentations would be given.

Under the "Memoirs of a Hidsight[sic] Hero" section, the author says, "Don't try and write a book about Mac rootkits..."; it wasn't so much that Cem was out to disprove the authors because they were writing a book.  Rather, my take was that Cem heard the claims of near-complete anonymity and thought, "that can't be right", and ended up disproving the claims.  Maybe the take away from this one would be, "...if you're going to write a book on Mac rootkits, get Cem as a co-author or tech editor."  ;-)

With respect to the MantaRay presentation: "...maybe useful for others but doesn't fit into my process flow." This is exactly right...depending upon the types of issues you face, you may not need to run every tool every time.  However, one of the useful things about tools like MantaRay and it's predecessor, TapeWorm, is that very often, they're configurable.  That is, you can trim them down or add to them in order to meet your needs.  The guys who developed MantaRay have provided the tools for use by others, which is great, particularly for folks with similar use-cases, or those new to the issue at hand.

Conferences

I recently had the opportunity (and honor) of attending the Open Memory Forensics Workshop (OMFW) and the Open Source Digital Forensics Conference (#OSDFCon), held in Chantilly, VA.  I've attended both conferences in the past, had a great time, and this time around was no different.

OMFW
I've always enjoyed the format that Aaron has used for the OMFW, going back to the very first one.  That first time, there was a short presentation followed by a panel, and back and forth, with breaks.  It was fast-moving, the important stuff was shared, and if you wanted more information, there was usually a web site that you could visit in order to download the tools, etc.

This time around, there was greater focus on things like upcoming updates to Volatility, and the creation of the Volatility Foundation.  Also, a presentation by George M. Garner, Jr., was added, so there were more speakers, more variety in topics discussed, and a faster pace, all of which worked out well.

The presentations that I really got the most out of were those that were more akin to use cases.

Sean and Steven did a great job showing how they'd used various Volatility plugins and techniques to get ahead of the bad guys during an engagement, by moving faster than the bad guys could react and getting inside their OODA loop.

Cem's presentation was pretty fascinating, in that it all seemed to have started with a claim by someone that they could hide via a rootkit on Mac OSX systems.  Cem's very unassuming, and disproved the claim pretty conclusively, apparently derailing a book (or at least a chapter of the book) in the process!

Jamie's presentation involved leveraging CybOX with Volatility, and was very interesting, as well as well-received.

There was more build-up and hype to Jamaal's presentation than there was actual presentation!  ;-)  But that doesn't take anything at all from what Jamaal talked about...he'd developed a plugin called ethscan that will scan a memory dump (Windows, Linux, Mac) and produce a pcap.  Jamaal pointed out quite correctly that many times when responding to an incident, you won't have access to a pcap file from the incident; however, it's possible that you can pull the information you need out of the memory buffer from the system(s) involved.

What's really great about OMFW is that not only does Aaron get some of the big names that are really working hard (thanks to them!) to push the envelope in this area of study to present, but there are also a lot of great talks in a very short time period.  I'll admit that I wasn't really interested in what goes into the framework itself (that's more for the developers), but there were presentations on Android and Linux memory analysis; there's something for everyone.  You may not be interested in one presentation, but wait a few minutes...someone will talk about a plugin or a process, and you'll be glued to what they're saying.

Swag this year was a cool coffee mug and Volatility stickers.

Here's a wrap-up from last year's conference.  You can keep up on new developments in Volatility, as well as the Volatility training schedule, at the Volatility Labs blog.

OSDFCon
I've attended this conference before, and just as in the past, there is a lot of great information shared, with something for everyone.  Personally, I'm more interested in the talks that present how a practitioner used open source tools to accomplish something, solve a problem, or overcome a challenge.  I'm not so much interested in academic presentations, nor so much in talks that talk about open source tools that folks have developed.  As in the past, I'd suggest yet again that there be multiple tracks for this conference...one for academics and developers, and another for practitioners, by practitioners.

As part of full disclosure, I did not attend any of the training or tutorials, and I could not attend all of the presentations.

You can see the program of talks here.

Thoughts and Take Aways
Visualization in DFIR is a sticky point...in some ways, it may be a solution without a problem.  Okay, so the recommendation is, "don't use pie charts"...got it.  But how does one use visualization techniques to perform analysis, when malware and intrusions follow the Least Frequency of Occurrence?  How can a histogram show an analyst when the bad guy or the malware compromised a system when activities such as normal user activity, software and system updates, etc., are the overwhelming available activity?  Maybe there is a way to take a bite out of this, but I'm not sure that academics can really start to address this until there is a crossover into the practitioner's end of the pool.  I only mention this because it's a recurring thought that I have each time I attend this conference.

As Simson pointed out, much of the current visualization occurs after the analyst has completed their examination and is preparing a report, either for a customer or for presentation in court.  Maybe that's just the nature of the beast.

Swag this year was a plastic coffee cup for the car with the TSK logo, TSK stickers, and a DVD of Autopsy.

Resources
Link to Kristinn's stuff

Thanks
We should all give a great, big Thank You to everyone involved in making both of these conferences possible.  It takes a lot of work to organize a conference...I can only imagine that it's right up there with herding cats down a beach...and providing a forum to bring folks together.  So, to the organizers and presenters, to everyone who worked so hard on making these conferences possible, to those who sat at tables to provide clues to the clueless ("...where's the bathroom?")...thank you.

What else
There is another thing that I really like about DFIR-related conferences; interacting with other DFIR folks that I don't get to see very often, and even those who are not directly involved with what we do on a day-to-day basis.  Unfortunately, it seems that few folks who attend these conferences want to engage and talk about DFIR topics, but now and again I find someone who does.

In this case, a good friend of mine wanted to discuss "...is what we do a 'science' or an 'art'?" at lunch.  And when I say "discuss", I don't mean stand around and listen to others, I mean actively engaging in discussion.  That's what a small group of us...there were only four of us at the table...did during lunch on Tuesday.  Many times, finding DFIR folks at DFIR conferences that want to actively engage in discussion and sharing of DFIR topics...new malware autostart/persistence mechanisms seen, new/novel uses of tools, etc...is hard to do.  I've been to conferences before where, for whatever reason, you just can't find anyone to discuss anything related to DFIR, or to what they do.  In this instance, that wasn't the case, and some fascinating discussion ensued.