Shellbag Analysis, Revisited...Some Testing

I blogged previously on the topic of Shellbag Analysis, but I've found that in presenting on the topic and talking to others, there may be some misunderstanding of how these Registry artifacts may be helpful to an analyst.  With Jamie's recent post on the Shellbags plugin for Volatility, I thought it would be a good idea to revisit this information, as sometimes repeated exposure is the best way to start developing an understanding of something.  In addition, I wanted to do some testing in order to determine the nature of some of the metadata associated with shellbags.

In her post, Jamie states that the term "Shellbags" is commonly used within the community to indicate artifacts of user window preferences specific to Windows Explorer.  MS KB 813711 indicates that the artifacts are created when a user repositions or resizes an Explorer windows.

ShellItem Metadata
As Jamie illustrates in her blog post, many of the structures that make up the SHELLITEMS (within the Shellbags) contain embedded time stamps, in DOSDate format.  However, there's still some question as to what those values mean (even though the available documentation refers to them as MAC times for the resource in question) and how an analyst may make use of them during an examination.

Having some time available recently due to inclement weather, I thought I would conduct a couple of very simple tests in to begin to address these questions.

Testing Methodology
On a Windows 7 system, I performed a number of consecutive, atomic actions and recorded the system time (visible via the system clock) for when each action was performed.  The following table lists the actions I took, and the time (in local time format) at which each action occurred.

Action	Time
Create a dir: mkdir d:\shellbag	12:54pm
Create a file in the dir: echo "..."  > d:\shellbag\test.txt	1:03pm
Create another dir: mkdir d:\shellbag\test	1:08pm
Create a file in the new dir: echo "..." > d:\shellbag\test\test.txt	1:16pm
Delete a file: del d:\shellbag\test.txt	1:24pm
Open D:\shellbag\test via Explorer, reposition/resize the window	1:29pm
Close the Explorer window opened in the previous step	1:38pm

The purpose of having some time pass between actions is so that they can be clearly differentiated in a timeline.

Once these steps were completed, I restarted the system, and once it came back up, I extracted the USRCLASS.DAT hive from the relevant user account into the D:\shellbag directory for analysis (at 1:42pm).  I purposely chose this directory in order to determine how actions external to the shellbags artifacts affect the overall data seen.

 Results
The following table lists the output from the shellbags.pl RegRipper plugin for the directories in question (all times are in UTC format):

Directory	MRU Time	Modified	Accessed	Created
Desktop\My Computer\D:\shellbag	2012-10-29 17:29:25	2012-10-29 17:24:26	2012-10-29 17:24:26	2012-10-29 16:55:00
Desktop\My Computer\D:\shellbag\test	2012-10-29 17:29:29	2012-10-29 17:16:20	2012-10-29 17:16:20	2012-10-29 17:08:18

Let's walk through these results.  First, I should remind you that that MRU Time is populated from Registry key LastWrite times (FILETIME format, granularity of 100 ns) while the MAC times are embedded within the various shell items (used to reconstruct the paths) in DOSDate time format (granularity of 2 seconds).

First, we can see that the Created dates for both folders correspond approximately to when the folders were actually created.  We can also see that the same thing is true for the Modified dates.  Going back to the live system and typing "dir /tw d:\shell*" shows me that the last modification time for the directory is 1:42pm (local time), which corresponds to changes made to that directory after the USRCLASS.DAT hive file was extracted.

Next, we see that MRU Time values correspond approximately to when the D:\shellbag\test folder was opened and then resized/repositioned via the Explorer shell, and not to when the Explorer window was actually closed.

Based on this limited test, it would appear that the DOSDate time stamps embedded in the shell items for the folders correspond to the MAC times of that folder, within the file system, at the time that the shell items were created.  In order to test this, I deleted the d:\shellbag\test\test.txt file at 2:14pm, local time, and then extracted a copy of the USRCLASS.DAT and parsed it the same way I had before...and saw no changes in the Modified times listed in the previous table.

In order to test this just a bit further, I opened Windows Explorer, navigated to the D:\shellbag folder, and repositioned/resized the window at 2:21pm (local time), waited 2 minutes, and closed the window.  I extracted and parsed the USRCLASS.DAT hive again, and this time, the MRU Time for the D:\shellbag folder had changed to 18:21:48 (UTC format).  Interestingly, that was the only time that had changed...the Modified time for the D:\shellbag\test folder remained the same, even though I had deleted the test.txt file from that directory at 2:14pm local time ("dir /tw d:\shellbag\te*" shows me that the last written time for that folder is, indeed, 2:14pm).

Summary
Further testing is clearly required; however, it would appear that based on this initial test, we can draw the following conclusions with respect to the shellbag artifacts on Windows 7:

1.  The embedded DOSDate time stamps appear to correspond to the MAC times of the resource/folder at the time that the shell item was created.  If the particular resource/folder was no longer present within the active file system, an analyst could use the Created date for that resource in a timeline.

2.  Further testing needs to be performed in order to determine the relative value of the Modified date, particularly given that events external to the Windows Explorer shell (i.e., creating/deleting files and subfolders after the shell items have been created) may have limited effect on the embedded dates.

3.  The MRU Time appears to correspond to when the folder was resized or repositioned.  Analysts should keep in mind that (a) there are a number of ways to access a folder that do not require the user to reposition or resize the window, and (b) the MRU Time is a Registry key LastWrite time that only applies to one folder within the key...the Most Recently Used folder, or the one listed first in the MRUListEx value.

I hope that folks find this information useful.  I also hope that others out there will look at this information, validate it through their own testing, and even use it as a starting point for their own research.

Links

Being socked in by the weather, I thought it would be a good time to throw a couple of things out there...

Mounting an Image
Folks...in order to test or make use of the Forensic Scanner, you first need to have an image.  If you don't have an image available, you can download sample images from a number of locations online.  Or you can image your own system, or you can use virtual machine files (FTK Imager will mount a .vmdk file with no issues).  However, the Forensic Scanner was not intended to be run against your local, live system.

Once you have an image to work with, you need to mount it as a volume in order to run the Forensic Scanner against it.  If you have a raw/dd image, a .vmdk or .vhd file, or a .E0x file, FTK Imager will allow you to mount any of these in read-only format.

If you have a raw/dd format image file, you can use vhdtool to add a footer to the file, and then use the Disk Manager to attach the VHD file read-only.  If you use this method, or if you mount your image file as VMWare virtual machine, you will also be able to list and mount available VSCs from within the image, and you can run the Scanner against each of those.

If you have any version of F-Response, you can mount a remote system as a volume, and run the Forensic Scanner against it.  Don't take my word for it...see what Matt, the founder of F-Response, says about that!

If you have issues with accessing the contents of the mounted image...Ken Johnson recently tried to access a mounted image of a Windows 8 system from a Windows 7 analysis system...you may run into issues with permissions.  After all, you're not accessing the  image as a logical volume...so, you might try mounting the image as "File System/Read-Only", rather than the default "Block Device/Read-Only", or you may want to run the Scanner using something like RunAsSystem in order to elevate your privileges.

If your circumstances require it, you can even use FTK Imager (FTK Imager Lite v3.x is now available and supports image mounting) to access an acquired image, and then use the export function to export copies of all of the folders and files from the image to a folder on your analysis system, or on a USB external drive, and then run the scanner against that target.

Okay, but what about stuff other than Windows as your target?  Say that you have an iDevice (or an image acquired from one...)...the Forensic Scanner can be updated (it's not part of the current download, folks) to work with these images, courtesy of HFSExplorer.  Caveat: I haven't tested this yet, but from the very beginning, the Forensic Scanner was designed to be extensible in this manner.

Again, if you opt to run the Forensic Scanner against your local drive (by typing "C:\Windows\system32" into the tool), that's fine.  However, I can tell you it's not going to work, so please don't email me telling me that it didn't work.  ;-)

Forensic Scanner Links
Forensic Scanner Links - links where the Forensic Scanner is mentioned:
F-Response Blog: F-Response and the ASI Forensic Scanner
Grand Stream Dreams: Piles o' Linkage
SANS Forensics Blog: MiniFlame, Open Source Forensics Edition

Apparently, Kiran Vangaveti likes to post stuff that other people write...oh, well, I guess that imitation really is the sincerest form of flattery!  ;-)

Observables
The good folks over at RSA have had some interesting posts of late to their "Speaking of Security" blog, and the most recent one by Branden Williams is no exception.  In the post, Branden mentions "observables", as well as Locard's Exchange Principle...but what isn't explicitly stated is the power of correlating various events in order to develop situational awareness and context, something that we can do with timeline analysis.

An example of this might be a failed login attempt or a file modification.  In and of themselves, these individual events tell us something, but very little.  If we compile a timeline using the data sources that we have available, we can begin to see much more with regards to that individual event, and we go from, "...well, it might/could be..." to "...this is what happened."

SANS Forensic Summit 2013
The next SANS #DFIR Summit is scheduled for July 2013 (in Austin, TX) and the call for speakers is now open.

Prefetch Analysis
Adam posted recently regarding Prefetch file names and UNC paths, and that reminded me of my previous posts regarding Prefetch Analysis.  The code I currently use for parsing Prefetch files includes parsing of paths that include "temp" anywhere in the path (via grep()), and provides those paths separately at the end of the output (if found).  Parsing of UNC paths (any path that begins with two back slashes, or begins with "\Device") can also be included in that code.  The idea is to let the computer extract and present those items that might be of particular interest, so that the analyst doesn't have to dig through multiple lines of code.

DOSDate Time Stamps in Shell Items

Since this past spring, the term "shellbags" has been heard more and more often.  Searching for "shellbag analysis" via Google reveals a number of very informative links.  I'm going to gloss over the specifics of these links, but my doing so in no way minimizes any of the research, analysis and documentation by those who have contributed to the understanding of these Windows artifacts.

What I want to get to directly is the underlying data structures associated with the shellbags artifacts, specifically, the shell items and shell item ID lists, structures that Joachim Metz and others such as Kevin Moore have worked to identify and define.  Again, mentioning the contributions made by these two individuals is in no way intended to take away from work performed by others in this area.

Shell items and shell item ID lists are used in a number of artifacts and data structures on Windows systems.  Perhaps one of the most well known of those structures is the Windows shortcut/LNK files; you can see from the MS specification regarding the file format where the shell items exist within the structure.  A number of Registry keys also use these data structures, including (but not limited to) Shellbags, ComDlg32, and MenuOrder.

 Several of the data structures that make up the shell item ID lists include embedded data, to include time stamps.  In many cases (albeit not all), these embedded time stamps are DOSDate format, which is a 32-bit time stamp with a granularity of two seconds.

Now, since a lot of the analysis that we do is often based heavily upon not simply that an event or action occurred, but when it occurred, often having additional sources of time stamped data can be extremely valuable to an analyst.  However, there is much more to timeline analysis than simply having a time stamp from an artifact...the analyst must understand the context of that artifact with respect to the time stamp in question.

The question I would like to pose to the community is...what is the value of the embedded DOSDate time stamps within the shell items?

Let's first consider shellbags.  The keys that store these artifacts are mentioned in MS KB 813711, so we have an idea of how these artifacts are created.  In short, it appears that the shellbags artifacts are created (or modified) when a user accesses a folder via the Windows Explorer shell, and then repositions or resizes the window that appears on the desktop.  So let's say that I open Windows Explorer, navigate to the "C:\Windows\Temp" directory, and resize the window.  I would then expect to find indications of the path in the shellbags artifacts.  At this point, we would expect that the time stamps embedded within the shellbags artifacts (and keep in mind, more testing is required in order to verify this...) refer to the MAC times from the "Windows" and "Temp" folders at the time that the artifact was created.

If we can agree on this, even for the moment, can we then also agree that other activities outside of those that create or modify the shellbags artifacts will also act upon the MAC times for those folders?  For example, adding or deleting files or subfolders, or any other action that causes those folders to be modified will cause the last modified ("M") date to be...well...modified.

On Vista systems and above, the updating of last access times for file system objects has been disabled by default.  Even if it weren't, other actions and events not associated with the shellbags artifacts (AV scans, user activity, normal system activity, etc.) would also cause these times to be modified.

The same thing could be said for the ComDlg32 artifacts.  On Vista systems and above, several of the subkeys beneath the ComDlg32 key in the user's NTUSER.DAT hive contain values that are consistent with shell items, and are parsed in a similar manner.  The data structures that describe the files and folders in these shell items contain embedded DOSDate time stamps, but as with the shellbags, these artifacts can be affected by other actions and events that occur outside of the scope of the ComDlg32 key.

Given this, I would like to reiterate my question: what is the value of the "M" and "A" DOSDate time stamps embedded within shell item data structures?  The "C" time is defined as the creation date, and even with a 2 second granularity, I can see how this time stamp can be of value, particularly if (a) the described resource no longer exists on the system, or (b) the described resource is on remote or removable storage media.  However, I would think that adding the "M" and "A" times for a resource to a timeline could potentially add considerable noise and confusion, particularly if the nature and context of the information is not completely understood.  In fact, simply having so many artifacts that are not easily understood can have a significant detrimental impact on analysis. 

What are your thoughts? 

Motivations behind the Forensic Scanner

At the recent OSDFC, I presented on and announced the release of the Forensic Scanner.

Since then, I've been asked to talk about things like the motivations behind the development of the Scanner, and what need I was trying to fill by creating it.  At the conference, shortly after my presentation, it was suggested to me that I should talk about what's different about the Scanner, how it's different from the other available frameworks, such as Autopsy and DFF.

One of the reasons I wrote the Scanner (and this applies to RegRipper, as well) is that over the years of performing analysis, I've found that as I've maintained a checklist of things to "look for" during an exam, that checklist has grown, become a spreadsheet, and continued to grow...but no matter how well-organized the spreadsheet is, it's still a spreadsheet and doesn't help me perform those checks faster.  If I have preconditions (something else that needs to be checked first) listed in the spreadsheet for a specific check, I have to go back into the image, do the initial check, and then see if the conditions have been met to perform the check I have listed in that row in the spreadsheet.

One good example of this is the ACMru Registry key in Windows XP.  This key can exist within the user's NTUSER.DAT hive, and illustrates what the user searched for on the system.  With Windows Vista, indications of the user's searches were moved out of the Registry, and with Windows 7, they were added back into the Registry, into the WordWheelQuery key.  So, if you wanted to see what a user had searched for, you had to first check the version of the Windows operating system (the "precondition"), and then refer to the checklist and follow the appropriate path to the Registry key for that particular platform.

All of this can be hard to remember over time, and this is true not only for someone who performs a good deal of research into the Windows Registry.  Imagine what this would be like for someone who doesn't spend a great deal of time analyzing Windows systems in general.  The fact is that even with an extensive checklist or spreadsheet, this still requires a good deal of time and effort to maintain, as well as to engage.  After all, you can't just lay a spreadsheet over your acquired image and have all the answers pop up...yet, anyway.  An analyst has to actually sit down with the checklist and the acquired image and do something.

The plugins used by the Forensic Scanner, like those used by RegRipper, can be as simple as you want them to be.  For example, with RegRipper, you may not want to look for specific values in the the Run key, because malware names and paths can change.  As such, you may want to dump all of the values in order to present them to the analyst.  You can also perform a modicum of analysis by grepping for terms such as "temp", to find any executables being launched from a temp directory (i.e., "Local Settings\Temp", or "Temporary Internet Files", etc.).  The plugins used by the Forensic Scanner can do something very similar.  For example, remember the ntshrui.dll issue?  You can write a plugin that specifically checks for a file with that name in the Windows directory, or you can write a plugin that dumps a list of all of the DLLs in the Windows directory, and either apply a whitelist ("these are the known good files"), or a blacklist.

Note: The above paragraph is intended as an example, and does not indicate that current RegRipper plugins will work with the Forensic Scanner.  They won't.  A small amount of work is necessary to get the current RegRipper plugins to work with the Forensic Scanner.

Here's another good example...ever seen these blog posts?  Do they make sense?  How would you check for this?  Well, I'd start with a plugin that parses the shell item ID lists of all of the LNK files on the user Desktop, and compares those elements to the ASCII in the LinkInfo block, if there is one.  You could then extend this to the Recents folder, and because it's all automated, it will be done the same way, every time, no matter how many user profiles are on the system.

A final example...Corey Harrell reached to me a bit ago, asking me to create a plugin for a couple of Registry keys (discussed here, in an MS KB article).  I did some looking around and found this valuable blog post on the topic of those keys.  The Order values beneath the subkeys in question contain shell item ID lists, much like the values found in the ShellBags, as well as ComDlg32 entries.  The linked blog post explains that the Order values often contain references to items the user explicitly selected, even after the application or resource has been deleted.  As such, why not include this in every scan you run...who knows when something of critical value to your examination is going to pop-up?  Most of us are aware that the one time we don't check these Registry keys will be the time that critical information is found to reside within the keys.  So...write the plugin once, and run it with every scan, automatically; if you don't need it, ignore it.  However, if you do need the information, it's right there.

One of the motivations for this sort of automation is to bridge a critical gap and get more organizations to perform root cause analysis, so that they can develop the necessary intelligence they need to properly defend themselves.  Right now, many organizations will react to an apparent compromise or infection by wiping the system in question, and trying to address all of the vulnerabilities that are out there..that is, if they do anything at all.  However, if they performed a root cause analysis, and determined the initial vector of compromise or infection, they could provide an intelligence-driven defense, reducing and optimizing their resources. One of the reasons for this sort of reaction may be performing a root cause analysis takes too long...the Forensic Scanner does not obviate analysis; rather, it is intended to get you there faster.

I understand that the idea of the Forensic Scanner is outside what most would consider "normal" thinking within the community.  This is also true for RegRipper, as well as Timeline Analysis.  The idea is not to obviate the need for analysis...rather, it's to get analysts to the actual analysis faster.  After all, tools like RegRipper and now the Forensic Scanner will extract in seconds what currently takes an analyst hours or even days to do.

NOTE: In order to run the Scanner, you'll want to either disable UAC on your Windows 7 analysis system, or right-click on the program icon and choose "Run as administrator". 

Finally, it appears that the Forensic Scanner is already making it's rounds at least one conference.  Also, the Forensic Scanner was designed to work with F-Response, and Matt Shannon has a blog post indicating that F-Response + Forensic Scanner = Awesome!

Forensic Scanner

I've posted regarding my thoughts on a Forensic Scanner before, and just today, I gave a short presentation at the #OSDFC conference on the subject.

Rather than describing the scanner, this time around, I released it.  The archive contains the Perl source code, a 'compiled' executable that you can run on Windows without installing Perl, a directory of plugins, and PDF document describing how to use the tool.  I've also started populating the wiki with information about the tool, and how it's used.

Some of the feedback I received on the presentation was pretty positive.  I did chat with Simson Garfinkel for a few minutes after the presentation, and he had some interesting thoughts to share.  The second was on making the Forensic Scanner thread-safe, so it can be multi-threaded and use multiple cores.  That might make it on to the "To-Do" list, but it was Simson's first thought that I wanted to address.  He suggested that at a conference were the theme seemed to revolve around analysis frameworks, I should point out the differences between the other frameworks and what I was presenting on, so I wanted to take a moment to do that.

Brian presented on Autopsy 3.0, and along with another presentation in the morning, discussed some of the features of the framework.  There was the discussion of pipelines, and having modules to perform specific functions, etc.  It's an excellent framework that has the capability of performing functions such as parsing Registry hives (utilizing RegRipper), carving files from unallocated space, etc.  For more details, please see the web site.  

I should note that there are other open source frameworks available, as well, such as DFF.

The Forensic Scanner is...different.  Neither better, nor worse...because it addresses a different problem.  For example, you wouldn't use the Forensic Scanner to run a keyword search or carve unallocated space.  The scanner is intended for quickly automating repetitive tasks of data collection, with some ability to either point the analyst in a particular direction, or perform a modicum of analysis along with the data presentation (depending upon how much effort you want to put into writing the plugins).  So, rather than providing an open framework which an analyst can use to perform various analysis functions, the Scanner allows the analyst to perform discrete, repetitive tasks.

The idea behind the scanner is this...there're things we do all the time when we first initiate our analysis.  One is to collect simple information from the system...it's a Windows system, which version of Windows is it, it's time zone settings, is it 32- or 64-bit, etc.  We collect this information because it can significantly impact our analysis.  However, keeping track of all of these things can be difficult.  For example, if you're looking at an image acquired from a Windows system and don't see Prefetch files, what's your first thought?  Do you check the version of Windows you're examining?  Do you check the Registry values that apply to and control the system's prefetching capabilities?  I've talked with examiners who's first thought is that the user must have deleted the Prefetch files...but how do you know?

Rather than maintaining extensive checklists of all of these artifacts, why not simply write a plugin to collect what data it is that you want to collect, and possibly add a modicum of analysis into that plugin?  One analyst writes the plugin, shares it, and anyone with that plugin will have access to the functionality without having to have had the same experiences as the analyst.  You share it with all of your analysts, and they all have the capability at their fingertips.  Most analysts recognize the value of the Prefetch files, but some may not work with Windows systems all of the time, and may not stay up on the "latest and greatest" in analysis techniques that can be applied to those files.  So, let's say that instead of dumping all of the module paths embedded in Prefetch files, you add some logic to search for .exe files, .dat files, and any file that includes "temp" in the path, and display that information?  Or, why not create whitelists of modules over time, and have the plugin show you all modules not in that whitelist?

Something that I and others have found useful is that, instead of forcing the analyst to use "profiles", as with the current version of RegRipper, the Forensic Scanner runs plugins automatically based on OS type, class, and then organizes the plugin results by category.  What this means is that for the system class of plugins, all of the plugins that pertain to "Program Execution" will be grouped together; this holds true for other plugin categories, as well.  This way, you don't have to go searching around for the information in which you're interested.

As I stated more than once in the presentation, the Scanner is not intended to replace analysis; rather, it's intended to get you to the point of performing analysis much sooner.  For example, I illustrated a plugin that parses a user's IE index.dat file.  Under normal circumstances when performing analysis, you'd have to determine which version of Windows you were examining, determine the path to the particular index.dat file that you're interested in, and then extract it and parse it.  The plugin is capable of doing all of that...in the test case, all of the plugins I ran against a mounted volume completed in under 2 seconds...that's scans of the system, as well as both of the selected user profiles.

So...please feel free to try the Scanner.  If you have any questions, you know where to reach me.  Just know that this is a work-in-progress, with room for growth.

Addendum
Matt Presser identified an issue that the Scanner has with identifying user profiles that contain a dot.  I fixed the issue and will be releasing an update once I make a couple of minor updates to other parts of the code.