Mark Russinovich over on SysInternals.com posted a blog entry yesterday about how he located a rootkit that was installed after he purchased a CD.
One has to raise the "HUH??!?!" flag (those are the three letters I chose, but I'm sure that there are other blog posts out there on this subject that will be brought to you by the letters "W", "T", and "F"...) on this one.
The SANS Internet Storm Center picked it up, as well. So did F-Secure, and they have their own technical description here.
So...imagine if you will...not only are you looking over your shoulder with regards to the bad guys and malcode, but now you have to be careful of the legitimate guys, as well. Sony is legitimate, right? What happens if a worm comes out that doesn't have its own rootkit payload (there was a worm for AIM recently that did), but instead looks for the presence of a commercial DRM rootkit, and hides itself that way? What will that worm be called? "W32.SonyPiggyback"?
Face it folks...do a search on SARC for "rootkit" and see how many entries you find over last year. Ouch!
Addendum 3 Nov: I generally try to stay on-track in this space, my blog...keeping on topic with regards to forensic analysis of Windows systems. However, I have to say that Wired really has it wrong. Specifically, "But in our view, the hacker and virus threat is something of a red herring...". Red herring? How so? The article goes on to say, "Sony may even have committed a crime under the U.S. Computer Fraud and Abuse Act...".
Wow. The EULA doesn't specifically state that its installing a rootkit, but given the knee-jerk reactions of most folks, is anyone really surprised? After all, it was just last February that Microsoft hit the stage of the RSA Conference, full of sound and fury over rootkits, but signifying nothing. So they bring something that's been around for a while to the forefront of attention...and because Microsoft said it, the media jumps all over it. And guess what? There are even courses you can take in writing rootkits. So...someone in Sony is trying to figure out how to protect their "property", and look what they stumble across.
Should Sony be faulted for what they did...perhaps. Should they be faulted more so than other companies (a company is a business, remember...a real-world construct designed to make money) who've taken "extreme" measures that end up making self-righteous folks indignant?
I'm not making a call on that. What I am concerned about is the number of folks out there who've installed this stuff, and suddenly there are a whole bunch of other files on the system with the same prefix (in this case, "$sys$"). I'm concerned about the first worm that does a check for the Sony stuff...why download your own stealth technology when there's already a perfectly good one on the system.
Okay, enough of that. Brian Krebs picked up on this and blogged...if you have a few minutes, scroll through some of the responses. Maybe you'll feel moved to leave a comment...
Tuesday, November 01, 2005
Wednesday, October 26, 2005
Peeking inside Word documents
I was chatting with someone yesterday who asked me where I found the files I used to test the Word Metadata Dumper, and I simply said, "Google."
Yep, that's right...I just Googled for Word documents from the .mil domain, as well as the .gov domain. Wanna know how to search Google for all sorts of other goodies? Check out Johnny Long's web site, or grab a copy of his book on "Google hacking".
Want to get a little up-close-and-personal with someone else, maybe even someone else you don't know? Remember the "Extreme File Sharing" post from Security Fix? I'd tried it and found some of the very same things...files left behind by malware with keylogger capability, etc.
Yep, that's right...I just Googled for Word documents from the .mil domain, as well as the .gov domain. Wanna know how to search Google for all sorts of other goodies? Check out Johnny Long's web site, or grab a copy of his book on "Google hacking".
Want to get a little up-close-and-personal with someone else, maybe even someone else you don't know? Remember the "Extreme File Sharing" post from Security Fix? I'd tried it and found some of the very same things...files left behind by malware with keylogger capability, etc.
Tuesday, October 25, 2005
Sleuthkit on Windows
Hey, guess what I did last night!! I installed Sleuthkit and Autopsy on Windows XP!
For you Linux and *BSD gurus who felt a nauseating disturbance in the Force...it wasn't that burrito you had...it was me!
Okay, okay...it wasn't just me...I had help.
When I'm working with things on my home systems and looking at forensic analysis, I like to use ProDiscover to grab a dd image of a VMWare session, my thumb drive, etc. As it turns out, I had a 5GB image of an XP Pro system, so I copied that over to the evidence locker and fired up Autopsy. I didn't run completely through many of the things that I could have done, because it would take some time to do so...but as far as the things I did try, they worked great.
Don't have an image of your own to play with? Well, the instructions for installing Sleuthkit and Autopsy on Windows also has instructions for how to image a floppy drive...so, you can entertain and amaze your friends by recovering deleted files! Or, you can go to the Digitial Forensics Tool Testing site and grab an image or two to work with.
My hat's off to Brian Carrier, for having created these tools.
For you Linux and *BSD gurus who felt a nauseating disturbance in the Force...it wasn't that burrito you had...it was me!
Okay, okay...it wasn't just me...I had help.
When I'm working with things on my home systems and looking at forensic analysis, I like to use ProDiscover to grab a dd image of a VMWare session, my thumb drive, etc. As it turns out, I had a 5GB image of an XP Pro system, so I copied that over to the evidence locker and fired up Autopsy. I didn't run completely through many of the things that I could have done, because it would take some time to do so...but as far as the things I did try, they worked great.
Don't have an image of your own to play with? Well, the instructions for installing Sleuthkit and Autopsy on Windows also has instructions for how to image a floppy drive...so, you can entertain and amaze your friends by recovering deleted files! Or, you can go to the Digitial Forensics Tool Testing site and grab an image or two to work with.
My hat's off to Brian Carrier, for having created these tools.
Monday, October 24, 2005
Perl for Forensics
Perl is freely available, and Perl scripts are essentially open source. Perl is used by forensics products such as The Sleuthkit and ProDiscover. Perl is used in the Metasploit framework. Perl is great for automating repetitive tasks, parsing files...and it's free. Perl runs on Windows, most Unices, the Mac, and a plethora of other platforms.
O'Reilly has a ton of books on Perl...from how to program to how to use Perl for a variety of tasks.
So my question is, how useful would a book on using Perl for forensics be to you? Say, a reference tome that discusses:
O'Reilly has a ton of books on Perl...from how to program to how to use Perl for a variety of tasks.
So my question is, how useful would a book on using Perl for forensics be to you? Say, a reference tome that discusses:
- Collecting live/volatile data using Perl
- Correlating data from multiple sources using Perl
- Analyzing data, or presenting data for analysis
- Analyzing file formats (retrieving metadata, etc.)
Obviously, a book like this should include copies of all code used or mentioned in the book. As ProDiscover uses Perl as it's scripting language, a book such as this should also include a variety of "ProScripts". The book should also include not only the files analyzed in the book, but additional example files that the reader can explore and practice on...perhaps even an image of drive to examine.
Is this a book you'd be interested in? If so, what would you like to see? What topics do you think should be covered? How would you envision such a book, particularly as something that you'd pick up off of a shelf at a bookstore and decide to purchase? What do you see as the market for such a book?
Saturday, October 22, 2005
VMWare Playa
As a user of VMWare, I received an email the other day that mentioned a new, free product called the VMWare Player. This is a free product that allows you to play a single VMWare virtual machine on Windows or Linux (rpm and tar versions available for Linux). Very cool. Want to share tools, and other stuff that you may not have been able to share before? Want to try out Windows or Linux, but didn't want to shell out the almost $200 for VMWare Workstation?
VMWare also provides some pre-built virtual machines for you to download. What good is a player if you don't have something to play? One is a browser appliance, which you can use for safer web surfing.
This was also picked up by TaoSecurity, along with some comments from readers of that blog, and a link to a chart showing differences in functionality between the Player and other VMWare products.
VMWare also provides some pre-built virtual machines for you to download. What good is a player if you don't have something to play? One is a browser appliance, which you can use for safer web surfing.
This was also picked up by TaoSecurity, along with some comments from readers of that blog, and a link to a chart showing differences in functionality between the Player and other VMWare products.
Word metadata code posted
All,
I've posted the code that I mentioned in my previous post on Word metadata. This code produces the output seen in the blog entry.
The code is commented, including how to obtain the necessary modules if you're using ActiveState Perl. The PPM commands look like this:
ppm install OLE-Storage
ppm install Startup
ppm install Unicode-Map
Give the code a shot, and let me know what you think. As I said in my earlier post, I'm working on producing a standalone EXE via Perl2Exe, for Windows users.
I've posted the code that I mentioned in my previous post on Word metadata. This code produces the output seen in the blog entry.
The code is commented, including how to obtain the necessary modules if you're using ActiveState Perl. The PPM commands look like this:
ppm install OLE-Storage
ppm install Startup
ppm install Unicode-Map
Give the code a shot, and let me know what you think. As I said in my earlier post, I'm working on producing a standalone EXE via Perl2Exe, for Windows users.
Sunday, October 16, 2005
Yet, even more on Word metadata
While awaiting information on the binary format of shortcut (LNK) files, I decided to try to learn more about structured storage and metadata in Word documents. The best example I've seen of that describes some of the metadata in Word documents is available at the Computerbytesman site, and addresses an issue that Tony Blair's government had a while back. While I was researching my book, Richard Smith was kind enough to share his code for retrieving the last 10 author's for within the Word document with me. Since that time, I've thought about taking another look at the sort of metadata that one can retrieve from within a Word document.
I included a Perl script for retrieving Word metadata with my book. The code is on the CD that accompanies the book, in the code directory for chapter 3. The script is called "meta.pl" and uses the Win32::OLE module to create an instance of Word, and use the API to retrieve metadata. Well, as I've seen with the work that I did on reading Event Log files, the API doesn't always get everything. Also, I've been looking for something a little more platform-independant.
Thanks to Richard Smith, I dug into the OLE::Storage module a bit, and found exactly what I was looking for. First, a quick caveat...the POD for this module, as well as some of the supporting modules, is a bit out of date. However, by using some of the accompanying examples (such as ldat, written by Martin Schwartz, copyright '96-'97) and simply trying some things out, I was able to figure things out. So the script uses that module, and a couple of others...but only after it opens the file in binary mode to retrieve other information from the file.
Okay, on to the output. I started with the Blair document from the Computerbytesman site, and got the same information (I didn't include the VBA Macro information, though). I downloaded a couple of arbitrary Word documents from the Web, via Google, and found some interesting info:
--------------------
Statistics
--------------------
File = d:\cd\wd\04_007.doc
Size = 322560 bytes
Magic = 0xa5ec (Word 8.0)
Version = 193
LangID = English (US)
Document has picture(s).
Document was created on Windows.
Magic Created : MS Word 97
Magic Revised : MS Word 97
--------------------
Last Author(s) Info
--------------------
1 : Susan and Shawn Sutherland :
2 : Susan and Shawn Sutherland :
3 : Susan and Shawn Sutherland :
4 : Susan and Shawn Sutherland :
5 : picketb :
6 : padilld :
7 : ONR :
8 : John T. McCain :
9 : horvats :
10 : arbaizd :
--------------------
Summary Information
--------------------
Title : I
Subject :
Authress : PICKETB
LastAuth : arbaizd
RevNum : 2
AppName : Microsoft Word 10.0
Created : 08.12.2003, 16:11:00
Last Saved : 08.12.2003, 16:11:00
Last Printed : 08.12.2003, 16:11:00
--------------------
Document Summary Information
--------------------
Organization : Office of Naval Research
Pretty cool, eh? Again, I found this document on the web. From my previous post, I asked some folks to send me documents written on the Mac platform, and I received a couple. Here's what the output looks like:
--------------------
Statistics
--------------------
File = d:\cd\wd\ex1.doc
Size = 21504 bytes
Magic = 0xa5ec (Word 8.0)
Version = 193
LangID = English (US)
Document was created on a Mac.
File was last saved on a Mac.
Magic Created : Word 98 Mac
Magic Revised : Word 98 Mac
--------------------
Last Author(s) Info
--------------------
1 :
--------------------
Summary Information
--------------------
Title : The quick brown fox jumps over the lazy dog
Subject :
Authress : name
LastAuth : name
RevNum : 1
AppName : Microsoft Word 10.1
Created : 12.10.2005, 02:51:00
Last Saved : 12.10.2005, 02:58:00
Last Printed :
--------------------
Document Summary Information
--------------------
Organization :
Okay, I made a couple of obvious changes, but the point is that there is information within the binary contents of the file information block (FIB) that tells you the platform that a document was created on...for example, if it was created on a Mac, or on a Windows platform. Pretty cool, eh?
So...what do you think? I'll be posting the script soon, along with a couple of other scripts...for example, I'm going to include one that I used for troubleshooting, which simply writes all of the structured storage streams to files on the system. After all, MS describes structured storage as "a file system within a file", so wouldn't you like to see the contents of each of those files? I'm not entirely sure of the usefulness of this with regards to forensic analysis, but someone might find it useful.
An offshoot of all this involves the MergeStreams application (here's something I found at UTulsa) that I've used in some of my presentations. This application allows you to merge an Excel spreadsheet into a Word document, resulting in a much larger, but otherwise unchanged Word doc. However, if you change the resulting file's extension to ".xls", and double-click on it, you'll see the entire, unmodified contents of the spreadsheet. This is due to the streams being merged, and handled by the appropriate application (no, this is not steganography!!). Whenever I've presented on this, I've been asked how this sort of thing can be detected, and up until now, the only solutions I've been able to come up with have include the use of 'strings' and 'find'. With this module, however, you can dump the names of the streams from an OLE document, and if you see a stream named "Workbook" inside a Word document, you can be pretty sure that you've got an embedded document. This is a more accurate method than using 'strings'.
I'll be releasing the scripts soon...there are a couple of things I need to clean up, and I'm having a small issue with the compiled EXE version of the main script (above) that I'm trying to clear up.
I included a Perl script for retrieving Word metadata with my book. The code is on the CD that accompanies the book, in the code directory for chapter 3. The script is called "meta.pl" and uses the Win32::OLE module to create an instance of Word, and use the API to retrieve metadata. Well, as I've seen with the work that I did on reading Event Log files, the API doesn't always get everything. Also, I've been looking for something a little more platform-independant.
Thanks to Richard Smith, I dug into the OLE::Storage module a bit, and found exactly what I was looking for. First, a quick caveat...the POD for this module, as well as some of the supporting modules, is a bit out of date. However, by using some of the accompanying examples (such as ldat, written by Martin Schwartz, copyright '96-'97) and simply trying some things out, I was able to figure things out. So the script uses that module, and a couple of others...but only after it opens the file in binary mode to retrieve other information from the file.
Okay, on to the output. I started with the Blair document from the Computerbytesman site, and got the same information (I didn't include the VBA Macro information, though). I downloaded a couple of arbitrary Word documents from the Web, via Google, and found some interesting info:
--------------------
Statistics
--------------------
File = d:\cd\wd\04_007.doc
Size = 322560 bytes
Magic = 0xa5ec (Word 8.0)
Version = 193
LangID = English (US)
Document has picture(s).
Document was created on Windows.
Magic Created : MS Word 97
Magic Revised : MS Word 97
--------------------
Last Author(s) Info
--------------------
1 : Susan and Shawn Sutherland :
2 : Susan and Shawn Sutherland :
3 : Susan and Shawn Sutherland :
4 : Susan and Shawn Sutherland :
5 : picketb :
6 : padilld :
7 : ONR :
8 : John T. McCain :
9 : horvats :
10 : arbaizd :
--------------------
Summary Information
--------------------
Title : I
Subject :
Authress : PICKETB
LastAuth : arbaizd
RevNum : 2
AppName : Microsoft Word 10.0
Created : 08.12.2003, 16:11:00
Last Saved : 08.12.2003, 16:11:00
Last Printed : 08.12.2003, 16:11:00
--------------------
Document Summary Information
--------------------
Organization : Office of Naval Research
Pretty cool, eh? Again, I found this document on the web. From my previous post, I asked some folks to send me documents written on the Mac platform, and I received a couple. Here's what the output looks like:
--------------------
Statistics
--------------------
File = d:\cd\wd\ex1.doc
Size = 21504 bytes
Magic = 0xa5ec (Word 8.0)
Version = 193
LangID = English (US)
Document was created on a Mac.
File was last saved on a Mac.
Magic Created : Word 98 Mac
Magic Revised : Word 98 Mac
--------------------
Last Author(s) Info
--------------------
1 :
--------------------
Summary Information
--------------------
Title : The quick brown fox jumps over the lazy dog
Subject :
Authress : name
LastAuth : name
RevNum : 1
AppName : Microsoft Word 10.1
Created : 12.10.2005, 02:51:00
Last Saved : 12.10.2005, 02:58:00
Last Printed :
--------------------
Document Summary Information
--------------------
Organization :
Okay, I made a couple of obvious changes, but the point is that there is information within the binary contents of the file information block (FIB) that tells you the platform that a document was created on...for example, if it was created on a Mac, or on a Windows platform. Pretty cool, eh?
So...what do you think? I'll be posting the script soon, along with a couple of other scripts...for example, I'm going to include one that I used for troubleshooting, which simply writes all of the structured storage streams to files on the system. After all, MS describes structured storage as "a file system within a file", so wouldn't you like to see the contents of each of those files? I'm not entirely sure of the usefulness of this with regards to forensic analysis, but someone might find it useful.
An offshoot of all this involves the MergeStreams application (here's something I found at UTulsa) that I've used in some of my presentations. This application allows you to merge an Excel spreadsheet into a Word document, resulting in a much larger, but otherwise unchanged Word doc. However, if you change the resulting file's extension to ".xls", and double-click on it, you'll see the entire, unmodified contents of the spreadsheet. This is due to the streams being merged, and handled by the appropriate application (no, this is not steganography!!). Whenever I've presented on this, I've been asked how this sort of thing can be detected, and up until now, the only solutions I've been able to come up with have include the use of 'strings' and 'find'. With this module, however, you can dump the names of the streams from an OLE document, and if you see a stream named "Workbook" inside a Word document, you can be pretty sure that you've got an embedded document. This is a more accurate method than using 'strings'.
I'll be releasing the scripts soon...there are a couple of things I need to clean up, and I'm having a small issue with the compiled EXE version of the main script (above) that I'm trying to clear up.
Thursday, October 13, 2005
Recent rootkit news
I hope you weren't expecting things to stand still...
This past Monday, F-Secure had an entry in their blog about a custom version of Hacker Defender. In this case, "custom" means "private commercial", meaning that someone paid for a specific version of the rootkit. And don't think for an instant that this is the only one...there are other rootkit authors who do the very same thing.
According to the F-Secure blog entry, the version of the rootkit has anti-detection mechanisms. Specifically, it detects modern rootkit detectors via their binary signature, and if it does find one of the detectors, it can modify itself or the detector. F-Secure says that the most recent version of their BlackLight product can detect this rootkit.
This brings up something I saw over on the Incidents.org blog. Handler Patrick Nolan posted an entry about rootkits that run even in safe mode. Yes, that's right...when you try to boot your computer in safe mode (here's the description of Safe Mode for Windows 2000) so that certain Registry keys aren't parsed, such as autostart locations, the rootkit will still launch. Check out this description from Symantec (btw...take a look at everything that last bit of malware does...).
The Registry key you're interested in is:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SafeBoot
On a side note, Autoruns has been updated to v8.22, and includes new functionality. I've run it on my system, and it doesn't seem to check the SafeBoot key mentioned above. However, when running your cases and parsing the Registry files from an image, be sure to add this one this one to your list of keys to check. Remember, though...on an image, the correct path would be "ControlSet00x", rather than "CurrentControlSet".
Addendum 14 Oct: I caught an interesting item on the Spire Security ViewPoint blog this morning...there's a link to a VNUNet article (ZDNet's version) that mentions three guys in the Netherlands who got busted with a 'botnet army of 100K nodes/zombies. The bots were evidently W32.ToxBot, which Symantec states has "0-49" infections. In all fairness, though, Symantec's definition of "number of infections" is "Measures the number of computers known to be infected". This leads me over to a post on the TaoSecurity blog about digital security, and the differences between the real, "analog" world and the indicators of engineering failures, and those in the digital world. I can't imagine that all 100K of the zombies infected with W32.ToxBot were simply home user systems. It's entirely possible that many of them were academic and corporate systems...and in the case of the corporate systems, someone should have realized that something was going on.
I've dealt with incidents in the past in which admin machines were infected. When I was the security admin at a financial services company, I had a script that would pull down the most recent IIS 4.0 web server logs from a system that we had (and that I'd locked down, in part by removing all but the necessary script mappings) and parse out the other-than-ordinary entries. Over the course of a couple of days, I noticed Nimda scans from the same IP address. So, I did a lookup of the IP space to see who owned it, and in the end, I got lucky. The infected system was owned by the administrator, who was also the technical contact for the domain. I talked to him via the phone...he stated that he didn't realize that he'd had a web server on his system, and didn't know that his system was infected with Nimda (had been for several days), but once he started receiving calls (mine wasn't the first), he really had no idea what do to about it.
Okay...back to our little 'bot. Take a look at the Symantec write-up for the 'bot, in particular these items:
This past Monday, F-Secure had an entry in their blog about a custom version of Hacker Defender. In this case, "custom" means "private commercial", meaning that someone paid for a specific version of the rootkit. And don't think for an instant that this is the only one...there are other rootkit authors who do the very same thing.
According to the F-Secure blog entry, the version of the rootkit has anti-detection mechanisms. Specifically, it detects modern rootkit detectors via their binary signature, and if it does find one of the detectors, it can modify itself or the detector. F-Secure says that the most recent version of their BlackLight product can detect this rootkit.
This brings up something I saw over on the Incidents.org blog. Handler Patrick Nolan posted an entry about rootkits that run even in safe mode. Yes, that's right...when you try to boot your computer in safe mode (here's the description of Safe Mode for Windows 2000) so that certain Registry keys aren't parsed, such as autostart locations, the rootkit will still launch. Check out this description from Symantec (btw...take a look at everything that last bit of malware does...).
The Registry key you're interested in is:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SafeBoot
On a side note, Autoruns has been updated to v8.22, and includes new functionality. I've run it on my system, and it doesn't seem to check the SafeBoot key mentioned above. However, when running your cases and parsing the Registry files from an image, be sure to add this one this one to your list of keys to check. Remember, though...on an image, the correct path would be "ControlSet00x", rather than "CurrentControlSet".
Addendum 14 Oct: I caught an interesting item on the Spire Security ViewPoint blog this morning...there's a link to a VNUNet article (ZDNet's version) that mentions three guys in the Netherlands who got busted with a 'botnet army of 100K nodes/zombies. The bots were evidently W32.ToxBot, which Symantec states has "0-49" infections. In all fairness, though, Symantec's definition of "number of infections" is "Measures the number of computers known to be infected". This leads me over to a post on the TaoSecurity blog about digital security, and the differences between the real, "analog" world and the indicators of engineering failures, and those in the digital world. I can't imagine that all 100K of the zombies infected with W32.ToxBot were simply home user systems. It's entirely possible that many of them were academic and corporate systems...and in the case of the corporate systems, someone should have realized that something was going on.
I've dealt with incidents in the past in which admin machines were infected. When I was the security admin at a financial services company, I had a script that would pull down the most recent IIS 4.0 web server logs from a system that we had (and that I'd locked down, in part by removing all but the necessary script mappings) and parse out the other-than-ordinary entries. Over the course of a couple of days, I noticed Nimda scans from the same IP address. So, I did a lookup of the IP space to see who owned it, and in the end, I got lucky. The infected system was owned by the administrator, who was also the technical contact for the domain. I talked to him via the phone...he stated that he didn't realize that he'd had a web server on his system, and didn't know that his system was infected with Nimda (had been for several days), but once he started receiving calls (mine wasn't the first), he really had no idea what do to about it.
Okay...back to our little 'bot. Take a look at the Symantec write-up for the 'bot, in particular these items:
- Installs as a service, and oddly enough, it actually writes a description for the service, as well
- Besides the Registry keys for the service, it adds entries under "Control\SafeBoot\Minimal" and "Control\SafeBoot\Network" so that it is initiated even if the system is booted to Safe Mode
- It looks for the "HKLM\Software\VMware" key, and doesn't run if it finds it (Note: this same technique was used in SotM 32)
Nothing in the write-up indicates the use of rootkit capabilities, but from the capabilities this bot does have...wow. How much harder would it have been for normal admins to detect it if it did have rootkit capabilities (ie, the use of rdriv.sys, for example)?
CA's write up on the ToxBot family
Addendum 21 Oct: VNUNet posted an article 2 days ago, announcing that rootkit creators have gone professional, selling custom versions of their software. While "creators" is plural, there is only one such rootkit announced in the article. This was /.'d, as well. Contrary to what the author of the article would have you believe, this is NOT new.
Tuesday, October 11, 2005
More on Word Metadata
This past summer, I gave a couple of presentations, one that covered file metadata. I got to thinking...I've parsed Event Log files and Registry files in binary format...why not do the same with Word documents and see what else is in there besides what the MS API is telling me? After all, a particular value that references "hidden" data may be set to 0 (or 0x0000), but the actual data itself may still be there.
Remember the issue with Blair's gov't? When I found this, I tried using the MS API (via OLE) to retrieve the metadata concerning the last 10 authors of the file, and I simply could not get it to work. However, Richard Smith had no trouble doing so.
I started looking around and found the MS Word 97 binary file format (BFF) (here in HTML). I haven't had any trouble parsing the file information block, but what I am having a bit of trouble doing is locating the beginning of the table stream. Many of the values I'm interested in are listed as "offset in the table stream", indicating (to me, anyway) that the offset is from the beginning of the table stream.
If anyone has any information on this, I'd greatly appreciate some help with this.
Also, for testing/verification purposes, I was wondering if anyone out there with a Mac would do me a favor and create a couple of simple Word documents on that platform, zip them up, and send them to me. Some of the metadata within the Word document tells you whether the file was created or revised on a Mac. When you send the files, if you could specify the platform and version numbers (of the os and the application), I'd appreciate it. Thanks!
Remember the issue with Blair's gov't? When I found this, I tried using the MS API (via OLE) to retrieve the metadata concerning the last 10 authors of the file, and I simply could not get it to work. However, Richard Smith had no trouble doing so.
I started looking around and found the MS Word 97 binary file format (BFF) (here in HTML). I haven't had any trouble parsing the file information block, but what I am having a bit of trouble doing is locating the beginning of the table stream. Many of the values I'm interested in are listed as "offset in the table stream", indicating (to me, anyway) that the offset is from the beginning of the table stream.
If anyone has any information on this, I'd greatly appreciate some help with this.
Also, for testing/verification purposes, I was wondering if anyone out there with a Mac would do me a favor and create a couple of simple Word documents on that platform, zip them up, and send them to me. Some of the metadata within the Word document tells you whether the file was created or revised on a Mac. When you send the files, if you could specify the platform and version numbers (of the os and the application), I'd appreciate it. Thanks!
Monday, October 10, 2005
Perl Programming on Windows
Let's see a show of hands for everyone out there who uses programs Perl on Windows systems. Okay, thank you...please put your hands down. Now, how many of you use Perl to manage Windows systems? Okay, thank you very much.
Now...how many of you want to use Perl to manage your systems?
Whether you're an experienced Perl programmer and not familiar with Windows, or you're a Windows admin and don't know much about Perl, let me the first to tell you...Perl is a very powerful tool that you can learn to use, and use to harness your infrastructure.
Books like Learning Perl and Learning Perl on Win32 Systems will get you started. Even Advanced Perl Programming and Perl for System Administration can help. Dave Roth's books and web site can help. But to really get into the guts of what you can do, you need to (in the words of Nike), just do it.
At it's simplest, Perl can be used to automate tasks. Using Perl, you can create a Scheduled Task that reports certain information and has it waiting and available when the sysadmin comes in in the morning. Throw in a little error checking, and you will have reports on why some things may not have completed successfully...like systems being turned off, services not being available, etc. What would you like to do? Run nmap? Not a problem. Run it against your systems first thing in the morning, or over lunch, and have the output written to a file on your system. Once that's done, use Nmap::Parser to sort through the data and create reports. Great for sysadmins, pen testers, and security analysts running vulnerability assessments.
Perl can be used to implement WMI, and collect information from managed systems. Many of the tools I have available on my web site implement WMI. Using WMI, you can scan remote systems for processes, services, and even perform software inventory scanning from a continent away. Or how about reaching out across the country to locate rogue WAPs via managed Windows XP systems?
Perl is a very powerful tool that can harnessed to automate a wide variety of tasks performed by sysadmins, as well as security analysts. Data collection and parsing, as well as some modicum of analysis, can all be easily automated. Some of the things I use Perl for include:
If this is something you're interested in, drop me a line, post a comment, etc.
Now...how many of you want to use Perl to manage your systems?
Whether you're an experienced Perl programmer and not familiar with Windows, or you're a Windows admin and don't know much about Perl, let me the first to tell you...Perl is a very powerful tool that you can learn to use, and use to harness your infrastructure.
Books like Learning Perl and Learning Perl on Win32 Systems will get you started. Even Advanced Perl Programming and Perl for System Administration can help. Dave Roth's books and web site can help. But to really get into the guts of what you can do, you need to (in the words of Nike), just do it.
At it's simplest, Perl can be used to automate tasks. Using Perl, you can create a Scheduled Task that reports certain information and has it waiting and available when the sysadmin comes in in the morning. Throw in a little error checking, and you will have reports on why some things may not have completed successfully...like systems being turned off, services not being available, etc. What would you like to do? Run nmap? Not a problem. Run it against your systems first thing in the morning, or over lunch, and have the output written to a file on your system. Once that's done, use Nmap::Parser to sort through the data and create reports. Great for sysadmins, pen testers, and security analysts running vulnerability assessments.
Perl can be used to implement WMI, and collect information from managed systems. Many of the tools I have available on my web site implement WMI. Using WMI, you can scan remote systems for processes, services, and even perform software inventory scanning from a continent away. Or how about reaching out across the country to locate rogue WAPs via managed Windows XP systems?
Perl is a very powerful tool that can harnessed to automate a wide variety of tasks performed by sysadmins, as well as security analysts. Data collection and parsing, as well as some modicum of analysis, can all be easily automated. Some of the things I use Perl for include:
- Retrieve data from deep within the local system, or from remote systems
- Parse binary files, based on structure documentation, knowing what each DWORD means, etc. (ie, PE header analysis, Event Log and Registry parsing, etc.)
- Retrieve metadata from files (ie, Word/Excel docs, JPGs, PDF files, Prefetch files, etc.)
- Querying service information
- Data correlation across multiple sources (ie, Registry, files, etc.)
- Automation of information discovery in ProDiscover IR
If this is something you're interested in, drop me a line, post a comment, etc.
Tuesday, October 04, 2005
Book Report
I haven't blogged in a while, and I came across something worth blogging about. While I don't have the actual numbers in front of me, I've received word from my publisher that my book has only shipped 3500 copies domestically since it was published in July, 2004. From the numbers I received in April of this year, 3055 of those copies were in the first couple of months.
So what does this mean? I have no idea at this point, other than it doesn't seem to be enough justify another book. That's right...given all the material I've produced in the 15 or so months since the first book was published, I've already started putting another book together - an advanced version of the first one, with more technical, detailed information.
The last bit I got from the publisher is that it's up to me to find out what you, the readers, want in another book in order to get the final, published product to move off of the shelves. From what you've told me so far, it just about amounts to incident response war stories, case studies, and maybe even challenges you can work through. All that I can do...but again, it really doesn't sound promising.
I guess I need to start looking around for another avenue for publication. Fortunately, I got one good pointer at lunch today that I need to follow up on...
Addendum 5 Oct: I thought maybe I should give a brief description of what I was looking to provide in the next book. I wasn't planning for my next effort to be a second edition of the first...rather, my thought was to use the first as a stepping stone and launch off into a more advanced effort. I'd like to go more deeply into actual forensic analysis, with the focus being on analysis. Too many times, I've read papers and books that talk about analysis, and for the most part will only go so far as to say "run this tool, and if you see this in the output, something may be amiss..." I'd like to address data correlation and analysis, across the board...use multiple sources of information (i.e., file system, Registry, Event Log, etc.) to build out as complete a view of the issue as possible. I think that the best way to do that is to present the information, and then present examples via live case studies. This book would be interspersed with "war stories", case studies, and examples. I'd also like to include challenges, and exercises for the reader to work. This one would cover both live and post-mortem analysis.
If you've followed this blog, you're familiar with some of what's going to appear in the book...the tools I've released, things I've mentioned here (with more detailed research and analysis) will all be part of the book.
What do you think of something like this? Is this a pipe dream, or is it something you'd like to have on your reference shelf?
So what does this mean? I have no idea at this point, other than it doesn't seem to be enough justify another book. That's right...given all the material I've produced in the 15 or so months since the first book was published, I've already started putting another book together - an advanced version of the first one, with more technical, detailed information.
The last bit I got from the publisher is that it's up to me to find out what you, the readers, want in another book in order to get the final, published product to move off of the shelves. From what you've told me so far, it just about amounts to incident response war stories, case studies, and maybe even challenges you can work through. All that I can do...but again, it really doesn't sound promising.
I guess I need to start looking around for another avenue for publication. Fortunately, I got one good pointer at lunch today that I need to follow up on...
Addendum 5 Oct: I thought maybe I should give a brief description of what I was looking to provide in the next book. I wasn't planning for my next effort to be a second edition of the first...rather, my thought was to use the first as a stepping stone and launch off into a more advanced effort. I'd like to go more deeply into actual forensic analysis, with the focus being on analysis. Too many times, I've read papers and books that talk about analysis, and for the most part will only go so far as to say "run this tool, and if you see this in the output, something may be amiss..." I'd like to address data correlation and analysis, across the board...use multiple sources of information (i.e., file system, Registry, Event Log, etc.) to build out as complete a view of the issue as possible. I think that the best way to do that is to present the information, and then present examples via live case studies. This book would be interspersed with "war stories", case studies, and examples. I'd also like to include challenges, and exercises for the reader to work. This one would cover both live and post-mortem analysis.
If you've followed this blog, you're familiar with some of what's going to appear in the book...the tools I've released, things I've mentioned here (with more detailed research and analysis) will all be part of the book.
What do you think of something like this? Is this a pipe dream, or is it something you'd like to have on your reference shelf?
Wednesday, September 28, 2005
Remote Forensics System
I was grabbing some tools from the Foundstone tonight, for a presentation I'm working on, when I ran across something called the "Remote Forensics System". That sounded very interesting, so I Googled it and found a PDF at the Foundstone site that describes the RFS.
I read through the document and found a lot of very good information. In fact, I was envious...to be a grad student again, and have the time to do this kind of work [heavy sigh]. At least part of the system runs on the JRE, and they've incorporated triggers in to the overall system, so that if something pops up on the IDS, for example, data will be automatically retrieved from systems. Cool!
I'll need to read through the document again to get some idea of how the data collected is actually analyzed. However, it does sound like a great idea! My hat's off to Chris Prosise, listed in the Acknowledgements as the advisor.
Imagine my surprise when I found the Forensic Server Project mentioned in section 5 of the document! Wow! Someone actually went by the site and took a look at what I'd done. Unfortunately, the bibliography
I read through the document and found a lot of very good information. In fact, I was envious...to be a grad student again, and have the time to do this kind of work [heavy sigh]. At least part of the system runs on the JRE, and they've incorporated triggers in to the overall system, so that if something pops up on the IDS, for example, data will be automatically retrieved from systems. Cool!
I'll need to read through the document again to get some idea of how the data collected is actually analyzed. However, it does sound like a great idea! My hat's off to Chris Prosise, listed in the Acknowledgements as the advisor.
Imagine my surprise when I found the Forensic Server Project mentioned in section 5 of the document! Wow! Someone actually went by the site and took a look at what I'd done. Unfortunately, the bibliography
System Clock
Recently, a post in a public forum appeared, asking about how to verify the accuracy of the system clock. That got me to thinking...how do you verify the accuracy of the system clock, given nothing more than an image of a Windows system?
I decided to start some of my own research, and to post in other lists to see what others were doing. I received several responses, but most have been along the lines of:
So I set about performing my own experiments to see what happened. First, I ran InControl5 to baseline my system. When the baseline was complete, I double-clicked the time display on the far right of my Task Bar, and that opened the "Date and Time Properties" window, which is essentially the timedate.cpl Control Panel applet. I modified the system time, clicked "OK", then ran the second half of the InControl5 process.
Viewing the results, I ran across a couple of interesting things. First off, the following UserAssist key had been updated.
HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{75048700-EF1F-11D0-9888-006097DEACF9}/Count
I ran my "uassist.pl" Perl script to translate the values under this key, and found "timedate.cpl". Unfortunately, this key doesn't maintain its values along with an MRU list, so we don't know which was the last value, and therefore, the LastWrite time of the key is of little value to us.
When I actually opened the Control Panel and double-clicked the "Date and Time" applet, another entry occurred in the above UserAssist key...specifically, "UEME_RUNCPL:"C:\WINDOWS\system32\timedate.cpl",Date and Time".
Since I'm on an XP system, I thought I'd take a look in the Prefetch directory and see if there were any tidbits lurking around. Well, the thought didn't simply occur to me...the output of InControl5 told me that the file named "Rundll32.exe-randomstuff.pf" had been changed in some way. Knowing that rundll32.exe is a legit MS app, and that it's used to run things like Control Panel applets, I opened the .pf file in BinText and found Unicode strings that referenced timedate.cpl and w32time.dll, amongst other things.
All this is fine, in that it gives us clues, but I don't think that it's all that definitive. Another place to look, however, is in the Event Log. Specifically, within the Security Event Log there may be an event ID 520 (Category is "System Event") that states that the system time was changed, and includes the username, previous time, and new time. Very helpful!
Barring that (pretty definitive, isn't it??), there may be discrepancies in the actual times associated with the event records themselves. For example, if an event record with a higher number (more recent event) has a generated or written time that's before a previous event, then you may be on to something.
So far, many of the responses I've seen have said something along the lines of "look at your watch when you're standing in front of the system, before you unplug it and image it"...but, like I said, answers like that sort of miss the point of the question.
So...thoughts?
I decided to start some of my own research, and to post in other lists to see what others were doing. I received several responses, but most have been along the lines of:
- Completely missing the part about an "image"
- Using email headers
So I set about performing my own experiments to see what happened. First, I ran InControl5 to baseline my system. When the baseline was complete, I double-clicked the time display on the far right of my Task Bar, and that opened the "Date and Time Properties" window, which is essentially the timedate.cpl Control Panel applet. I modified the system time, clicked "OK", then ran the second half of the InControl5 process.
Viewing the results, I ran across a couple of interesting things. First off, the following UserAssist key had been updated.
HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{75048700-EF1F-11D0-9888-006097DEACF9}/Count
I ran my "uassist.pl" Perl script to translate the values under this key, and found "timedate.cpl". Unfortunately, this key doesn't maintain its values along with an MRU list, so we don't know which was the last value, and therefore, the LastWrite time of the key is of little value to us.
When I actually opened the Control Panel and double-clicked the "Date and Time" applet, another entry occurred in the above UserAssist key...specifically, "UEME_RUNCPL:"C:\WINDOWS\system32\timedate.cpl",Date and Time".
Since I'm on an XP system, I thought I'd take a look in the Prefetch directory and see if there were any tidbits lurking around. Well, the thought didn't simply occur to me...the output of InControl5 told me that the file named "Rundll32.exe-randomstuff.pf" had been changed in some way. Knowing that rundll32.exe is a legit MS app, and that it's used to run things like Control Panel applets, I opened the .pf file in BinText and found Unicode strings that referenced timedate.cpl and w32time.dll, amongst other things.
All this is fine, in that it gives us clues, but I don't think that it's all that definitive. Another place to look, however, is in the Event Log. Specifically, within the Security Event Log there may be an event ID 520 (Category is "System Event") that states that the system time was changed, and includes the username, previous time, and new time. Very helpful!
Barring that (pretty definitive, isn't it??), there may be discrepancies in the actual times associated with the event records themselves. For example, if an event record with a higher number (more recent event) has a generated or written time that's before a previous event, then you may be on to something.
So far, many of the responses I've seen have said something along the lines of "look at your watch when you're standing in front of the system, before you unplug it and image it"...but, like I said, answers like that sort of miss the point of the question.
So...thoughts?
Tuesday, September 27, 2005
Projects
Every now and again, I poke my head up for a breath of fresh air and a look around...and I wonder if others face the same issues and challenges I do. For example, knowing where to look during forensic analysis for information relevant to the case at hand.
So, my question to all of you out there is this...what issues do you face? What things do you see, need more information/documentation about? What are the things during a case (or just after you've completed one) that leave you wondering? What are those things that would make great research projects?
Sometimes, those things you are wondering about may already have been solved, addressed, or encountered by someone else.
Please feel free to post a comment here, or email me directly...which ever works. If you email me, I might post your idea, but I won't use your name without your consent.
Addendum 28 Sept: Well, I've received a single email so far in response to this post, and the ideas are (a) case studies, and (b) challenges. I'll see what I can do about posting both, but I'm sure that it would be extremely beneficial to hear from others.
Besides the usual suspects that provide forensic challenges (ie, HoneyNet, DFRWS), there are others available. Try TigerTools (the page has links to three different challenges; Feb, March, and July). I'm sure there are others...
So, my question to all of you out there is this...what issues do you face? What things do you see, need more information/documentation about? What are the things during a case (or just after you've completed one) that leave you wondering? What are those things that would make great research projects?
Sometimes, those things you are wondering about may already have been solved, addressed, or encountered by someone else.
Please feel free to post a comment here, or email me directly...which ever works. If you email me, I might post your idea, but I won't use your name without your consent.
Addendum 28 Sept: Well, I've received a single email so far in response to this post, and the ideas are (a) case studies, and (b) challenges. I'll see what I can do about posting both, but I'm sure that it would be extremely beneficial to hear from others.
Besides the usual suspects that provide forensic challenges (ie, HoneyNet, DFRWS), there are others available. Try TigerTools (the page has links to three different challenges; Feb, March, and July). I'm sure there are others...
Monday, September 26, 2005
Some way cool visualization stuff
F-Secure has a way cool visualization presentation on the Bagle worm...check it out. Scroll down to the Fri, 23 Sept entry entitled, "A different look at Bagle". Very cool.
I know that there are visualization tools available for social network analysis. Raytheon's SilentRunner (who owns it now??) uses n-gram analysis to build context and create a basis for it's mapping, and is very interesting. I wonder if the above malware visualization will eventually include details of the actual functions themselves...
I know that there are visualization tools available for social network analysis. Raytheon's SilentRunner (who owns it now??) uses n-gram analysis to build context and create a basis for it's mapping, and is very interesting. I wonder if the above malware visualization will eventually include details of the actual functions themselves...
Friday, September 23, 2005
Creating a timeline analysis tool
I'm not a DBA, and I don't play one on TV. So when it comes to writing a timeline analysis tool, I'm not going to be able to do it myself. But here's what I propose...
I'm most comfortable on Windows platforms, and I know folks that prefer Linux, so that's cool. I know that there are people out there who are familiar with databases, and others that are good graphics programmers. What I'd like to see about doing is opening a project on SourceForge, and see if we can't get a decent start on developing a solution. Here's how I envision it going...and please keep in mind that this isn't the be-all-and-end-all, just my impressions:
As far as the graphics programming goes, I'm not really sure where to go with that one. Would it be better to go with some of the stuff that's already out there, or create something new with Java or some other cross-platform solution?
So...who's interested? If I get enough interest, I'll go ahead and see about creating a project on SourceForge. Like I said, I can provide various means of extracting the data. For example, most of my Perl scripts that parse raw binary files from Windows can be made cross-platform, and easily modified to dump the information into a database. I can also write the necessary scripts for ProDiscover. Providing solutions to pull data from a source and populate the database is something that will be on-going, I'm sure.
Now, there's no doubt in my mind that this sort of project will take a while...if it were easy, everyone would be doing it. I don't expect it to be done overnight. However, I would like to see it get done, b/c I do think that it would be extremely useful to a lot of people.
So...thoughts?
Addendum: I just did a search on SourceForge for "timeline" and there are several projects listed, but most don't have anything more than a simple page...no project files, nothing. There are some projects with files, but I'm not entirely sure that they'd be suitable.
However, there is something promising...check out the Timeline view of GenealogyJ. My concern would be the volume of data, but it does look like a good start.
Also, my hope is that this will be compartmentalized...meaning that the graphics component won't be database-dependant. That way, there can be several different interfaces for presenting the data, and they can develop over time.
I'm most comfortable on Windows platforms, and I know folks that prefer Linux, so that's cool. I know that there are people out there who are familiar with databases, and others that are good graphics programmers. What I'd like to see about doing is opening a project on SourceForge, and see if we can't get a decent start on developing a solution. Here's how I envision it going...and please keep in mind that this isn't the be-all-and-end-all, just my impressions:
- Identify data sources (we've already gotten started with this, but we don't have to be restricted to Windows systems)
- Identify data sets (data source/log normalization, etc.)
- Identify a database format, using mySql (table definitions, etc.)
- Identify and create a graphic component for presenting the data sets
As far as the graphics programming goes, I'm not really sure where to go with that one. Would it be better to go with some of the stuff that's already out there, or create something new with Java or some other cross-platform solution?
So...who's interested? If I get enough interest, I'll go ahead and see about creating a project on SourceForge. Like I said, I can provide various means of extracting the data. For example, most of my Perl scripts that parse raw binary files from Windows can be made cross-platform, and easily modified to dump the information into a database. I can also write the necessary scripts for ProDiscover. Providing solutions to pull data from a source and populate the database is something that will be on-going, I'm sure.
Now, there's no doubt in my mind that this sort of project will take a while...if it were easy, everyone would be doing it. I don't expect it to be done overnight. However, I would like to see it get done, b/c I do think that it would be extremely useful to a lot of people.
So...thoughts?
Addendum: I just did a search on SourceForge for "timeline" and there are several projects listed, but most don't have anything more than a simple page...no project files, nothing. There are some projects with files, but I'm not entirely sure that they'd be suitable.
However, there is something promising...check out the Timeline view of GenealogyJ. My concern would be the volume of data, but it does look like a good start.
Also, my hope is that this will be compartmentalized...meaning that the graphics component won't be database-dependant. That way, there can be several different interfaces for presenting the data, and they can develop over time.
ISC Rootkit Discovery
This post appeared on the Incidents.org (ISC) blog two days ago, and is a very interesting read. The handler, Tom Liston, who works with Ed Skoudis over at IntelGuardians, writes the post in a humorous, Ian Fleming-esque style.
Take a look at the section marked "A view to a kill". Here, Tom mentions a couple of .sys files that seem to be a rootkit. I've run across this before...specifically a file named "rdriv.sys".
Tom's write-up in the final section of the post that describes what actions the malware takes on a system is very interesting, and an excellent read. The one big thing I took away from all this is that the good guys really need to get off their butts and start sharing information like this...tools, techniques, what's been found, etc. This needs to really start happening, because the bad guys are obviously doing it...and doing it much better than the good guys. It's pretty clear that the bad guys are moving away from the old days of "hacking" and writing malware as pranks, and this sort of activity is now driven, at least in part, by economics and financial gain.
Take a look at the section marked "A view to a kill". Here, Tom mentions a couple of .sys files that seem to be a rootkit. I've run across this before...specifically a file named "rdriv.sys".
Tom's write-up in the final section of the post that describes what actions the malware takes on a system is very interesting, and an excellent read. The one big thing I took away from all this is that the good guys really need to get off their butts and start sharing information like this...tools, techniques, what's been found, etc. This needs to really start happening, because the bad guys are obviously doing it...and doing it much better than the good guys. It's pretty clear that the bad guys are moving away from the old days of "hacking" and writing malware as pranks, and this sort of activity is now driven, at least in part, by economics and financial gain.
Thursday, September 22, 2005
Visualization
I've started to see that this issue of "timeline analysis" really isn't one of getting data as much as it is one of visualization. Graphically representing data in some manner for presention it to the audience, can be very powerful. It's been said that a "picture is worth a thousand words", and in many cases, this is true. So, the question becomes, how does one best present a timeline of activity on a system?
For starters, let's simply consider any system. One would hope that any solution would provide for mulitple systems, with the Windows host-based data sources having been covered in previous posts. We could incorporate firewall logs, logs from other systems, syslog, etc., all under the same case heading. There would need to be some sort of normalization process, of course, before the data was incorporated into a database. Along these lines, I'd met with the MountainWave folks many moons ago, in a previous life (...once, in a galaxy far, far away...), before they were purchased by Symantec. Their CyberWolf product was pretty cool, and performed normalization of logs, in a manner similar to what I'm referring to here.
Okay...so once you've populated your database, what next? Ah, yes...queries.
For presenting your data, there are many freeware visualization toolkits available, such as VTK, OpenDX, and GraphViz...but how useful are these...really? Well, GraphViz may have some potential.
One of the commercial tools I've been told is being used is CaseMap from CaseSoft. From what I've been told, though, getting the data into CaseMap can be almost as much of a manual process as Analyst's Notebook. A caveat, though...I haven't worked a great deal with either of these products, so I don't know if the issue of manually entering data is one of operator error or not.
This is all still kind of up in the air...how do you present the data? I think that culling information from a database and presenting a scalable view is still a viable option. The analyst can choose a date and time, and the tool will provide a zoomable view of the data, much in the same way as when you do a search on
For starters, let's simply consider any system. One would hope that any solution would provide for mulitple systems, with the Windows host-based data sources having been covered in previous posts. We could incorporate firewall logs, logs from other systems, syslog, etc., all under the same case heading. There would need to be some sort of normalization process, of course, before the data was incorporated into a database. Along these lines, I'd met with the MountainWave folks many moons ago, in a previous life (...once, in a galaxy far, far away...), before they were purchased by Symantec. Their CyberWolf product was pretty cool, and performed normalization of logs, in a manner similar to what I'm referring to here.
Okay...so once you've populated your database, what next? Ah, yes...queries.
For presenting your data, there are many freeware visualization toolkits available, such as VTK, OpenDX, and GraphViz...but how useful are these...really? Well, GraphViz may have some potential.
One of the commercial tools I've been told is being used is CaseMap from CaseSoft. From what I've been told, though, getting the data into CaseMap can be almost as much of a manual process as Analyst's Notebook. A caveat, though...I haven't worked a great deal with either of these products, so I don't know if the issue of manually entering data is one of operator error or not.
This is all still kind of up in the air...how do you present the data? I think that culling information from a database and presenting a scalable view is still a viable option. The analyst can choose a date and time, and the tool will provide a zoomable view of the data, much in the same way as when you do a search on
Tuesday, September 20, 2005
Cross-platform scripts
I've heard back from one or two people who've run the lsevt.pl, lsreg.pl, and regp.pl scripts I posted a bit ago. For the most part, I've heard pretty positive comments...things have worked well for most folks. The scripts seem to work just fine so far, regardless of the operating system they're run on...as long as it's running on an x86 processor. Yep, you guessed it...endianness is an issue.
However, one astute user was running the scripts on a G5 (PPC processor) and let me know that if you change the arguments of the unpack() function, the scripts work just fine, regardless of which microprocessor they're run on. The change comes in replacing all of the "S" (short, WORD, 2 bytes) and "L" (long, DWORD, 4 bytes) with "v" and "V", respectively. So, take the regp.pl script for example...in the _getNodeType() subroutine, you'll see:
return unpack("S",$record);
Change that to:
return unpack("v", $record);
In the readNkRecord() subroutine, you'll find:
my (@recs) = unpack("SSL3LLLLLLLLLL4LSS",$record);
Change that to:
my (@recs) = unpack("vvV3VVVVVVVVVV4Vvv",$record);
I won't be making these changes to the scripts myself...at least not right away. However, I am working on another book, so I will include those changes in the scripts before I add them to the CD.
However, one astute user was running the scripts on a G5 (PPC processor) and let me know that if you change the arguments of the unpack() function, the scripts work just fine, regardless of which microprocessor they're run on. The change comes in replacing all of the "S" (short, WORD, 2 bytes) and "L" (long, DWORD, 4 bytes) with "v" and "V", respectively. So, take the regp.pl script for example...in the _getNodeType() subroutine, you'll see:
return unpack("S",$record);
Change that to:
return unpack("v", $record);
In the readNkRecord() subroutine, you'll find:
my (@recs) = unpack("SSL3LLLLLLLLLL4LSS",$record);
Change that to:
my (@recs) = unpack("vvV3VVVVVVVVVV4Vvv",$record);
I won't be making these changes to the scripts myself...at least not right away. However, I am working on another book, so I will include those changes in the scripts before I add them to the CD.
Issues with timeline analysis
I've been doing some searches regarding timeline analysis, delving deeper into this. I still don't think that scatter plots and histograms are the way to report on and present this sort of information...there is just too much information that can and needs to be presented, and too many possible ways that it can be viewed.
In my Googling, I'm finding a good deal of references to "timeline analysis", as well as to the term "reconstruction". A lot of this is being offered as a service. There are also products available that can assist you in your timeline development and analysis, but many seem to be limited strictly to file MAC times. Given the various sources for timeline analysis that are available on a Windows system, relying simply on the file MAC times is not doing anyone any good.
So, I think that we're doing pretty well when considering sources of information, and now the question is, how do we present this information so that it's understandable? Is there an "access view" that looks at last access times of files, where a "modified view" would look at last modification times of files, as well as Registry key LastWrite times? What about things like times maintained in the document properties of OLE docs (ie, Word docs, Excel spreadsheets, etc.)? At what point is there too much information? How does one winnow out the valid, normal, usual stuff to get to the interesting stuff?
There's definitely some room for thought and development here. At this point, my thoughts are that there there'd be some sort of database involved, along with perhaps a Java interface for issuing queries and displaying the information. I haven't written Java GUIs in while, but perhaps a zoomable, scalable depiction of a timeline would be a good start...the investigator could pick a time of interest, and the initial queries would display a view of the timeline of activity, plus and minus a certain amount of time (determined by the investigator). Perhaps an interface into the database that shows which information is available, and lets the investigator select items to add and subtract from the view would be helpful. Add to that color coding of event records from the Event Log, and some other graphical representations of severity of events, and we may have something useful.
I mentioned Java above so that the whole thing would be cross-platform, but I can easily see where HTML or XML would work as well. With a mySql database, and the necessary filters to parse out any sort of information that is available to the investigator and get it into the database, I think we may have a pretty serious tool on our hands.
Thoughts? I know for my own part, I still have a lot of thinking to do, regarding such things as anomoly detection and anti-forensics. However, I think that this may be best handled by discussion within the community.
Addendum 21 Sept: As I think about this more and more, and even go so far as to draw out diagrams on a sheet of paper, trying to conceptualize what a "timeline" should look like, I'm even more convinced that a scatter plot isn't the way to go. Why? Well, a horizontal line (representing a time scale) with a bunch of little dots is meaningless...it has no context. Even if you gave different sources separate icons, and even color-coded them, without more information, it can be useless. Let's say you have a cluster of events around a particular time...Registry key LastWrite times, file last access times, and event records. Well, this could be a boot event. But you don't know that until you dig into things and take a look at the event records from the Event Logs.
Somehow I think that a scatter plot in which each of the dots has some identifying information would be just too much...the graph would be far too busy.
Something that may be of value in this effort is something like fe3d. Yes, I know that it's intended to provide visualization for nmap scans, but I think that something like this, with modifications, would be of value. Take a look at some of the screenshots and try to imagine how to map timeline information to this sort of representation. Of course, there are other freeware visualization tools out there...something else may be easier to use.
I will say this...one of the things that caught my eye with fe3d is the different nodes. Given information dumped into a database, one could include sources from other systems, such as Event Log records from a file server, or data from an IDS, or even firewall logs or syslog data...and have that timeline information represented on a different node, but correlated at the same time scale as the system(s) you're investigating.
In my Googling, I'm finding a good deal of references to "timeline analysis", as well as to the term "reconstruction". A lot of this is being offered as a service. There are also products available that can assist you in your timeline development and analysis, but many seem to be limited strictly to file MAC times. Given the various sources for timeline analysis that are available on a Windows system, relying simply on the file MAC times is not doing anyone any good.
So, I think that we're doing pretty well when considering sources of information, and now the question is, how do we present this information so that it's understandable? Is there an "access view" that looks at last access times of files, where a "modified view" would look at last modification times of files, as well as Registry key LastWrite times? What about things like times maintained in the document properties of OLE docs (ie, Word docs, Excel spreadsheets, etc.)? At what point is there too much information? How does one winnow out the valid, normal, usual stuff to get to the interesting stuff?
There's definitely some room for thought and development here. At this point, my thoughts are that there there'd be some sort of database involved, along with perhaps a Java interface for issuing queries and displaying the information. I haven't written Java GUIs in while, but perhaps a zoomable, scalable depiction of a timeline would be a good start...the investigator could pick a time of interest, and the initial queries would display a view of the timeline of activity, plus and minus a certain amount of time (determined by the investigator). Perhaps an interface into the database that shows which information is available, and lets the investigator select items to add and subtract from the view would be helpful. Add to that color coding of event records from the Event Log, and some other graphical representations of severity of events, and we may have something useful.
I mentioned Java above so that the whole thing would be cross-platform, but I can easily see where HTML or XML would work as well. With a mySql database, and the necessary filters to parse out any sort of information that is available to the investigator and get it into the database, I think we may have a pretty serious tool on our hands.
Thoughts? I know for my own part, I still have a lot of thinking to do, regarding such things as anomoly detection and anti-forensics. However, I think that this may be best handled by discussion within the community.
Addendum 21 Sept: As I think about this more and more, and even go so far as to draw out diagrams on a sheet of paper, trying to conceptualize what a "timeline" should look like, I'm even more convinced that a scatter plot isn't the way to go. Why? Well, a horizontal line (representing a time scale) with a bunch of little dots is meaningless...it has no context. Even if you gave different sources separate icons, and even color-coded them, without more information, it can be useless. Let's say you have a cluster of events around a particular time...Registry key LastWrite times, file last access times, and event records. Well, this could be a boot event. But you don't know that until you dig into things and take a look at the event records from the Event Logs.
Somehow I think that a scatter plot in which each of the dots has some identifying information would be just too much...the graph would be far too busy.
Something that may be of value in this effort is something like fe3d. Yes, I know that it's intended to provide visualization for nmap scans, but I think that something like this, with modifications, would be of value. Take a look at some of the screenshots and try to imagine how to map timeline information to this sort of representation. Of course, there are other freeware visualization tools out there...something else may be easier to use.
I will say this...one of the things that caught my eye with fe3d is the different nodes. Given information dumped into a database, one could include sources from other systems, such as Event Log records from a file server, or data from an IDS, or even firewall logs or syslog data...and have that timeline information represented on a different node, but correlated at the same time scale as the system(s) you're investigating.
Subscribe to:
Posts (Atom)