Saturday, March 16, 2013

BinMode: IE Index.dat

I've posted on the need for analysts to understand data structures before, and I recently conducted some analysis that I think illustrates this point extremely well.

The purpose of this blog post is to illustrate how certain artifacts can be used to detect the presence of malware on a system.  While a tool for doing so is described, this is not a blog post about parsing histories for all of the browsers a user may or could have used, in part because the artifacts examined do not pertain to other browsers.

Not long ago, I tweeted that I'd written a plugin for the Forensic Scanner that gets statistics from IE (version 5 - 9) index.dat browser history files for all user profiles on the system.  Almost immediately, someone tweeted asking, "what if the user isn't using IE?"  That's a good question, but it misses the point of the analysis technique and of having the plugin in the first place.

I was analyzing a system recently that had been infected with ZeroAccess (see the Sophos report), and one of the things I was aware of the malware was capable of doing was click-fraud.  In my analysis, I saw that the malware used an autostart persistence mechanism that was outside the scope of the user timeline illustrated the artifacts being created.  Knowing that much of the malware that communicates off-system will use the WinInet API functions to do so, I began looking at the index.dat files for the various user profiles available on the system.  What I found was that the NetworkService account had much more significant "browser history" than the 'normal' user account on the system.

That's exactly right...there's no typo.  The NetworkService account.  How could that be?  That's not something you see very often, is it?  I mean, how does someone sit down at the keyboard and log into the account, and launch IE?  The answer is...they don't.  What happens is that when code using the WinInet API is run at privileges other than those of a user, the artifacts are created in another account profile.  For example, back when Windows XP was more prevalent in my analysis lab, I would see systems on which the Default User profile had a populated index.dat file.  I've seen the same thing with the LocalService account; this may depend upon which process the malware is injected into, and where that process falls in the svchost.exe hierarchy.

So my point is that for malware detection, checking all user accounts for statistics regarding their index.dat files might be a good idea.  Once you understand the data structures in question - that is, the headers of the index.dat file, which, thanks to Joachim Metz, are well documented - this becomes a trivial task.

I started by writing a simple script that would parse the contents of the header of the index.dat file and tell me a little bit about what I could expect to see.  Based on the format specification for the file, I was interested in things like the offset to the HASH table, as well as the directories beneath the "Temporary Internet Files\Content.IE5" folder and the number of cache files in each folder.  This information is stored in the headers of the files, and is very easy to parse out and display.  I got the script working and it proved to be very useful.  However, I know that there's a process to using the script...I have to determine which user profiles are available, determine the version of Windows being examined, and based on those two pieces of information, type in the appropriate path to the index.dat file in question.  By hand.  Seriously?

So, I created a system class plugin for the Forensic Scanner to do all of this for me.  Automatically.  System class plugins are run against the entire system, whereas user class plugins are run against each user profile selected by the analyst.  Based on the specific artifacts that I'm looking for, a system class plugin is exactly what I need.

What follows is an excerpt of the output from the plugin.  First, the Administrator account profile:

g:\Documents and Settings\Administrator\Local Settings\Temporary Internet Files\Content.IE5\index.dat
File size              : 163840
Hash Table Offset      : 0x5000
Number of blocks       : 1152
Number of alloc. blocks: 1089

Dir: LPVS8JVQ  Files: 90
Dir: IR1PLUTE  Files: 89
Dir: 5K3JMTA3  Files: 88
Dir: O3VB95DY  Files: 89

As you can see, the Administrator account has some browser history associated with it.  The hash table is located at offset 0x5000 within the index.dat file, and there are four subdirectories, each containing a number of cache files.

g:\Documents and Settings\Default User\Local Settings\Temporary Internet Files\Content.IE5\index.dat
File size              : 32768
Hash Table Offset      : 0x0
Number of blocks       : 128
Number of alloc. blocks: 32

Dir: O8WMK2SC  Files: 0
Dir: UAAUTN4C  Files: 0
Dir: Q323MBYB  Files: 0
Dir: 4VRPMD81  Files: 0

Okay, so this is what an empty index.dat looks like; the Default User profile has no IE history associated with it...there is no hash table, and the subdirectories don't contain any files.

g:\Documents and Settings\NetworkService\Local Settings\Temporary Internet Files\Content.IE5\index.dat
File size              : 9437184
Hash Table Offset      : 0x5000
Number of blocks       : 73600
Number of alloc. blocks: 58784

Dir: EIQLTWH3  Files: 384
Dir: KFSPU8SK  Files: 384
Dir: ABR75H1M  Files: 384
Dir: 5QI7EWW7  Files: 384
Dir: TTYX5IX2  Files: 384
Dir: 5GDCH3XG  Files: 383
Dir: KZZXKXWH  Files: 383
Dir: LA2SZ5HL  Files: 383
Dir: TLRE11UL  Files: 383
Dir: IYT9OTGD  Files: 383
Dir: 58I9EM25  Files: 383


The NetworkService account was the one that set off alarms!  As you can see, this profile has a significant history!  Parsing the actual index.dat file and displaying the entries in a micro-timeline (just the URL records) illustrated a significant amount of activity in a relatively short amount of time, with more URLs being requested per second than most users are capable of typing in or clicking via the browser.

In this case, I used the command line version of the Forensic Scanner to run a single plugin.  I had mounted the image file via FTK Imager, and it appeared on my system as the G:\ volume.  I then typed the following command:

C:\Perl\scanner> -s g:\windows\system32 -p ie_stats

That's all it took.  Again, this is NOT a comprehensive analysis...this is a quick check to see if I could expect any potential issues.  Another way to run this would be to select the "malware" artifact category...the plugin is included in that category.  Also, the purpose of running this plugin is NOT to find indications of browser activity, for all browsers, and for all users.  Instead, the purpose of this plugin is to check for something very does so automatically, accurately, and very, very quickly.  This entire exercise took only a couple of minutes, most of which was spent mounting the image file...once that was done, the plugin ran very quickly, and provided me with the information I needed.  I don't want to list all of the URL and REDR records from the NetworkService profile index.dat because just from what we see above (not all of the directories are listed) there are around 4000 or so files in the cache subdirectories.

Again, the purpose of this blog post is to illustrate an analysis technique.  In the past, what I've done is to use ProDiscover to populate the "Internet History View" from within the image, and look for indications of service accounts with URL records in their index.dat.  However, the methodology used by ProDiscover is more searches the entire file system, and parses all of the records out of each index.dat file that it finds.  In this case, that's much more than what I'm looking for.

This analysis technique can be combined with other tools into a more comprehensive process, as described in this blog post.  Parsing application prefetch (*.pf) files and finding indications of wininet.dll as one of the loaded modules might be something to correlate with this analysis technique.

Rob Hensing's post on the Default User with an IE browser history
ForensicsWiki page: IE History File Format
Extremely relevant post from Hogfly (2007)


Greg Kelley said...

Nice idea on the analysis. Typically, I use NetAnalysis for browser history extraction and parsing. What I really like that tool for is putting all of the history it extracts in a database. Then I can run my own analysis to determine:

1. How much history I have for various accounts on a system

2. Do I have history during the relevant time frame

3. Other statistical analysis slicing and dicing the history by record type, user, domain, date, etc.

Looking for oddities and other data that doesn't seem to match what one would expect to see.

Keydet89 said...


Thanks for the comment.

This approach is a technique I've been using for a number of years now, originally through ProDiscover. However, I wanted to find a way to implement the check, without have to run a tool like NetAnalysis or ProDiscover to do a full collection. Why run a full scan that can take considerable time when all I have to do is run a quick check that takes less than a second or two to run?

The important points here are to:

(a) Understand the nature of what we're looking at; that is, is there any reason why the LocalService or NetworkService account should have a web history? If this isn't something that the analyst understands to look for and what it means, it doesn't matter which tool they use.

(b) understand the underlying data structures. The tools provide a layer of abstraction over the data, and analysts need to understand the nature of the data itself, beyond what the tools present. For example, in the format specification for URL records in the index.dat file, Joachim Metz identifies a total of 5 time stamps...for those that are populated, what creates/modifies those entries, in what format are they maintained, and how can they be used in analysis?

Greg Kelley said...

Yes, that does appear to be a much faster method of analysis and as we all know clients want results yesterday.

Keydet89 said...


I use this more to determine what I need to analyze (ie, "Sniper Forensics") and not so much to actually perform analysis.

Also, this really doesn't have so much to do with meeting the customer's requirements to get them answers yesterday, because I do that even if the customer is okay with having results by next week. My overall approach is to quickly target (via automation) those things that I can get done soonest, which allows me to perform more focused, deeper analysis, and gets me to that point quicker.

kurt gillis said...

Hey Harlan

Is it just index.dat files you examine to look at potential malware infections? I was infeted by zeroaccess a while ago and noticed that my seach urls would start collecting in a /TEMP/ folder similar to the old one. Why does malware like this change index.dat locations? Does the remote hacker do this manually, or is it just a by-product of the infection?

Also, that plug in you mentioned that was able to collect all users index.dat files together, would that also work on index.dat files from deleted user accounts, perhaps if there is still some peice of it left deep in the harddrive?

Harlan Carvey said...


Is it just index.dat files you examine to look at potential malware infections?

No, of course not.

...would that also work on index.dat files from deleted user accounts...

That's sort of a moot point, given the nature of the application itself.


kurt gillis said...

Is the plug in called I'm not really familiar with gibhub, cant see it listed with the other plugins


Harlan Carvey said...

I don't own the Forensic Scanner, and haven't touched it in three years. Sorry. The company who owns it let me go.

annie093 said...

Hey, awesome blog!

I have an old machine running XP and I checked the index.dat files on the system and I have quite a bit of browsing history in the NetworkService account too!

I checked the temporary internet files and discovered many of the files may have been from my own browsing sessions. Was this still malware related?

I was wondering, did you happen to notice whether or not this infection created a new internet explorer file structure in the temp folder?

For example,

The normal location for history for internet explorer (in xp) is;

C:\Documents and Settings\ Local Settings\ History\ History.ie5\index.dat

But I have another in the temp location where it should not be; notice the temp directory.

C:\Documents and Settings\ Local Settings\Temp\ History\ History.ie5\index.dat

There are these duplicate files in the temp folder for cookies and temporary internet files as well. Did you happen to notice if this happened?