Timeline analysis

I've blogged about different forms of analysis in the past, and thought I'd take a look at timeline analysis. I had an opportunity to dig around in EnCase v5.0 recently and noticed a nice graphical timeline visualization tool. Very cool. This would have been helpful in a recent situation, had the issue been with filetimes, and not Registry key LastWrite times.

I know that the TimeLine view is not a new feature for EnCase...I simply haven't had to get really deeply involved in using EnCase in a while. However, lsevt.pl and regp.pl. These scripts parse the Windows Event Log and Registry files, respectively. Lsevt.pl contains instructions within the comments for the code for formatting the output in semi-colon-delimited format, suitable for opening in Excel. Let's say we did that, and then sorted everything on the column with the "Time Generated" field from the event records. Then we could do easy (albiet non-graphical) analysis of events that occurred at a certain time.

The regp.pl script doesn't do this specifically, but minor modifications to the file will print out only the key name and LastWrite times, in semi-colon- or comma-delimited format. Again, open the file in Excel, sort on the LastWrite time column, etc.

I think that it would be interesting to have a tool that does all of that, don't you? One that presents a timeline in a spreadsheet or graphical format, but incorporates not only file/directory MAC times, but Registry key LastWrite times, Event Log event record "time generated" times, etc. Being able to trace back to a specific date and time that, say, a file was created on the system, you might see the progression of files installed, Registry keys created, etc., as well as any preceeding events, such a failed login attempts to an Admin-level account, etc.

Addendum 17 Sept: The more I think about it, the more I find that I'm not really sure what I mean when I say "timeline analysis". I received an email from someone that pointed out limitations in Excel (i.e., the number of lines it could handle) and recommending that I look at gnuplot. That's good information, but how useful is it, really? Think about it. A colorful histogram might be nice, but what does it tell me?

When I've had to use timeline analysis of some sort, I've had a date in mind...usually from a complaint or incident report. In some cases, I've noticed "interesting" events that occurred around the same time, such as the LastWrite time on the Registry key for a service called "rdriv.sys". At that point, what I'd like to be able to do is (a) get a snapshot of everything else that occurred around that time...file changes, other Registry keys, events from the Event Log, etc....within a pretty immediate time frame (within seconds), (b) get another snapshot, but with a bit wider scope (hours, maybe less than a day), and (c) "interesting" events that occurred following the initial event.

In my mind, I'm not entirely sure that this is something that is suitable, particularly during the initial phase of the investigation, to be displayed in a candlestick or even a histogram plot. In some cases, I think it would be way too messy. In others, I'm not sure that sorting on groupings of activities or concentrations of events would be necessarily informative, either...you'd see events like reboots.

Don't get me wrong, though...I do think that perhaps something like gnuplot would be useful in the presentation phase of the investigation. During the investigation, a plot of the frequency of certain types of events, such as failed login attempts, network logins, or of the types of queries to appear in IIS logs, would be useful, I think.

With the glut of files on a Windows system, one would need some method for winnowing the wheat from the chaff. Any sort of plot you do based on the last accessed time of a file, looking for the "bad things" will likely be hidden behind the noise of all of the "normal things" that go on on a system.

So...at this point, my thoughts are along these lines...some sort of database will need to be used in order to facilitate searching...I'm thinking mySql. At this point, I'm not entirely sure what kind of table structure would be most useful, but I do have an idea of the data that would need to go into the database. From files, you'd want things like full path, MAC times, hashes, file version information (useful in narrowing down searches to all non-MS files), file signature status (i.e., pass or fail), any alternate data streams, etc. I think I'd want to populate the database with information from Event Logs, and other logs on the system...schedlogu.txt, IIS logs, setupapi.log, etc. Of course, I'd also want to include Registry key LastWrite times, as well.

Once all this information was in the database, I'd start with some sort of default search queries that allow the investigator to target specific dates and look for anything "interesting" within certain windows of time, around the target event. I don't see simply having a dot showing a file change, or a number representing how many files changed, as useful. For me, I'd have to see which files changed, by name and location, as well as Registry keys that may have changed just prior to or immediately following those file changes/additions. I think that used in combination with the file hashes and versioning information, something like this might be very useful to investigators, and help in narrowing down the time it takes to find "interesting" events.

Addendum 19 Sept: I received the following email over the weekend: "I've long desired a consolidated timeline view of "what happened on this box and when". The filesystem MAC times tell a big part of the story, and now with your perl scripts, I can add two more important pieces of events and registry key writes. In a single view, I can see "DCOM service unexpectedly restarted, file backdoor.exe was created, and the CurrentVersion\Run key was last written". That is powerful!"

Combining MAC times with Event Logs, Registry key LastWrite times, and other log resources, along with explanations (ie, what the Event Log entries mean) would be a pretty valuable source of information, wouldn't it?