Saturday, March 04, 2006

Windows Physical Memory Analysis

I'll be writing more on this particular subject as time goes on, but I wanted to post something right away. I've noticed over the past couple of weeks that Andreas Schuster has been doing some work with debuggers, etc., to document some of the structures found in physical memory for various versions of Windows, from Win2000 SP4 through Vista (he's noted that most of the structures change between versions and even between Service Packs). Well, I took a look the other day and noticed that he'd posted something called PTFinder, which as it just so happens is a Perl script that parses a dump of physical memory (this version works for dumps of physical memory from Win2000 SP4 systems).

So, I took a look at what he was doing, and exchanged some emails, and expressed my desire to assist with what he's doing. As it turns out, the DFRWS 2005 Memory Challenge provides a great set of test files (2, actually) for testing any tools you're writing to parse a memory dump generated using dd.exe.

I started working on something of my own, using the exercise as a learning process so that I can not only be smarter on this stuff, but to also assist Andreas with what he's doing. I've got the output of the Memory Challenge submissions to check my work against, and so far, things are working pretty well. Here's an excerpt of the output from my version of the script:

Possible EPROCESS block located at offset 0x3e35ae0
Process Name : Explorer.Exe
PID : 820
Parent PID : 800
Exit Status : 259
Create Time : Sun Jun 5 00:33:53 2005
Exit Called : 0

Possible EPROCESS block located at offset 0x40b4660
Process Name : PcfMgr.exe
PID : 1048
Parent PID : 820
Exit Status : 259
Create Time : Sun Jun 5 00:34:01 2005
Exit Called : 0

Possible EPROCESS block located at offset 0x414dd60
Process Name : dd.exe
PID : 284
Parent PID : 1112
Exit Status : 259
Create Time : Sun Jun 5 14:53:42 2005
Exit Called : 0

Pretty cool, eh? One thing to point out is that processes (specifically, EPROCESS blocks) are maintained in a doubly-linked list. One way to walk through the process blocks is to find one and follow the links...but you can miss things this way. Andreas' approach, and the one I've chosen to follow, is to start at the beginning of the dump file and start looking for process blocks.

There's a lot of work that still needs to be done. Last week, a LEO I know attending the Southeast CyberCrime Summit sent me a message from his Blackberry, asking me if I'd come up with a way to run a tool to dump the contents of physical memory to a thumb drive or some other removable storage platform. In the past, I've talked to folks (particularly LEOs) about why they collect the contents of physical memory, and most have told me that they do so in order to run 'strings' against it to see if they can find leads (not evidence) such as passwords, IP or email addresses, IM screennames, etc. The DFRWS 2005 Memory Challenge didn't result in any publicly available tools but research has continued (like I said, I'll be posting more on this later, and recognizing others who've done research along these lines, particularly Mariusz Burdach and his presentation at BlackHat Federal 2006).

Addendum 7 Mar 2006: Like I said, I wanted to post some more information on this subject, describing what has already gone on, and then hopefully where we need to go with this research.

Last year, I became aware of a paper by Mariusz Burdach that described how to analyze the contents of physical memory that was collected using dd/dd.exe. Being interested primarily in how to do this on Windows, I was a little disappointed that the paper focused on Linux. This sort of thing has been discussed in other areas, the most formal being the DFRWS 2005 Memory Challenge. There has been discussion of this subject in other forums, such as Rootkit.com and Windows Forensic Analysis (Yahoo! Group).

The results of the DFRWS Memory Challenge included two winning responses, one by Chris Betz, and the other by George M. Garner, Jr. and Robert-Jan Mora. Both of these approaches are very involved, and produced some interesting results. The down-side of this work is that neither of the tools mentioned in both winning submissions is publicly available. Andreas Schuster released ptfinder.pl, a Perl script that parses through a dump of Windows physical memory searching for the different structures. Andreas recently posted on the difference between tools like ptfinder.pl and the "list-walkers" produced by Betz and Garner/Mora.

Addendum 9 Mar: Yesterday I became aware of some work by Joe Stewart over at LURHQ.com...specifically, the TRUMAN Project, described as the "reusable unknown malware analysis net". Part of the project is a Perl script called pmodump.pl
, "a Perl-based tool to reconstruct the virtual memory space of a process from a PhysicalMemory dump". The description goes on to say that "with this tool it is possible to circumvent most packers to perform strings analysis on the dumped malware". Very cool, and a great idea.

Pmodump.pl operates a bit differently from the other methods presented so far. The script locates potential Page Directory blocks and then does a translation between "logical" or virtual addresses and physical offsets within the dump file, using the default value for the pointer to the Process Environment Block (PEB) for pre-WinXP systems. The output of the script is very comprehensive, including such things as PE headers for executables loaded into memory (check out the File::ReadPE module to see how to parse this info), module lists, etc.

What this shows is that there have been several different, disparate approaches to this issue, different authors taking different approaches, as their needs and skill sets have been different. This looks like a great opportunity to bring all of these efforts together into a single project.

Another thing I'd like to see is information regarding other kernel structures that can be found within memory. I've got the MS Debugging Tools installed on my system, so all I'm really missing at this point is the correct names of the structures. Does anyone have any pointers?

Addendum 11 Mar: More great reading posted...Andreas has posted a blog entry on translating virtual addresses located in memory dumps to physical offsets within the dump file. Great stuff!

Addendum 14 Mar: I got a call late last night, just before CSI:Miami started (note to self: unplug phone when any version of CSI starts...)...evidently this blog entry had been mentioned in the 11 Mar Cyberspeak podcast! If you look at the main page, you'll see that many a famous name in the community has been interviewed by these guys...I haven't made it a habit of listening to these podcasts, but that's about to change. For the record, though, guys, it's "windowsir", not "windows", "s", "i", "r".

11 comments:

Anonymous said...

Don't forget pmodump.pl by Joe Stewart from LURHQ. It is part of the TRUMAN project for creating sandnets and studying malware in a controlled network without using virtual machines. It is a very cool project. I just saw him do a demo during a SANS conference. Excellent tools. If you aren't familiar with Joe's work, he is one of the best malware analysts in the industry.

http://www.lurhq.com/truman/

Anonymous said...

Also interessting stuff is on trapkit.de
Process Dumper allows you to make a dump of a running process and Memory Parser can be used to analyse process dumps made with pd.

http://www.trapkit.de/research/itd06/

H. Carvey said...

You're right...very interesting. Of course, it would be much more helpful if this information, particularly memory parser, were open source.

Thanks for the info...it's greatly appreciated.

Anonymous said...

Those interested in volatile memory forensics may also want to keep an eye on:

www.4tphi.net/fatkit

Anonymous said...

You may also find these useful:

Sandra Ring and Eric Cole, Volatile Memory Computer Forensics to Detect Kernel-Level Compromise

http://web.archive.org/web/20040405032635/http://www.sytexif.com/whitepaper.htm

Anonymous said...

Not to mention this thread on Dailydave:

[Dailydave] Memory, Elephantine

http://lists.immunitysec.com/pipermail/dailydave/2006-March/thread.html

H. Carvey said...

Aaron,

Thanks for the links. I've seen the first entry in the DailyDave link...Dave's description of the MandiaNT tool is...uhm...a bit off.

One thing I did get from the thread was a Windows tool for dumping the descriptor tables from the CPU:
http://www.cybertech.net/~sh0ksh0k/projects/DumpCPU

I think that this would be useful in helping you determine things from a dd dump of RAM such as the OS, etc.

The paper by Cole and Ring doesn't seem to be available anywhere.

Anonymous said...

I eventually found the Cole/Ring paper but I had to buy it from Springer. As for the DD, I'm glad you found something useful. I thought it might be useful to people working in the area so I wanted to post a link to the thread to forensics@SF, but they rejected my email ;).
(It just makes me wonder what else is getting rejected)

What things are you trying to determine from RAM?

H. Carvey said...

Aaron,

"Another thing I'd like to see is information regarding other kernel structures that can be found within memory."

Thanks for the info.

Anonymous said...

"Another thing I'd like to see is information regarding other kernel structures that can be found within memory."

Just pipe the PDB file through TypeInfoDump by Oleg Starodumov. http://www.debuginfo.com/tools/typeinfodump.html

It doesn't recursively list nested structures, so I recommend to get the structure's name from the listing and let kd or windbg do the pretty printing.

Of course there are still a lot of interesting structures not listed in the PDBs. Consider to disassemble the kernel if you're really interested in those (or ask the usual suspects in the reverse engineer communities).

H. Carvey said...

Andreas,

Great site, thanks.

*Who* are the "usual suspects"?