The subject thread I've been following and contributing to has proved to be interesting, to say the least, and the most interesting aspects have little to do with the subject...
First off, I guess I really shouldn't be surprised how many respondants go almost immediately off-topic, without bothering to change the subject line. Bad etiquette, guys, but I guess that's to be expected.
I want to take a second or two so summarize some of the most popular responses I've seen in watching this thread, and comment on them...
There's a lot of talk about "protecting against currently unknown exploits"...and I have to ask, if it's unknown, how do you protect against it and how do you know A/V will help you? Really. One example came up of a new exploit in which the bad guy took control of the system and dropped an already-known rootkit (ie, HackerDefender, etc.) on the system...in such a case, A/V would work. Yes, and you'd be very lucky, b/c the bad guy was very stupid. If the bad guy gained such access, why would he not bother to disable or even uninstall the A/V first? Or roll back the .dat file? Or why use something that's old? If he's using a zero-day exploit, why would he then use an old rootkit?
There have been mentions of Code Red, SQL Spida, and SQL Slammer. To me, it's odd that someone would use those examples and say, "I've seen A/V come to the rescue with these viruses", when the infections could have been easily prevented in the first place with configuration settings. With Code Red, all the IIS admin had to do was disable the .ida/.idq script mappings...something not many folks used anyway. With SQL Spida, all the SQL admin had to do was put a password on the 'sa' account. Need I say it? Duh!
Another biggy is that A/V protects against the things that have been missed, the things that haven't been done. Good point, but I would suggest that perhaps the security process itself needs to revisited. If something wasn't done, why was that? Was it part of the process and the setting wasn't verified before putting the server into production? Or had it been changed? Yet again, A/V software is justified...but as a band-aid approach.
There have been respondants who are in positions where the web servers are administered by multiple folks, and perhaps even developers have admin access to the servers, for adding updates. This falls right into the same category as malware that gains SYSTEM level access...A/V isn't going to help, b/c at that point, it's GAME OVER, guys! Someone with that level of access can simply disable your A/V software.
So...am I being arrogant? I don't think so. I've managed IIS 4.0/5.0 web servers and watched the logs fill up with failed Code Red/Nimda attempts, etc. I've had NT 4.0 boxes (my own) connected to a DSL hookup with no firewall, and no A/V software...and the only time I've ever gotten a virus, worm, or rootkit on my system is when I put it there myself.
Am I saying that A/V software shouldn't be used? Not at all...I think like every tool, it has it's place. However, I do think that if a web server admin is doing their job, then it's not necessary to put A/V software on a web server. That was the original question I responded to.
Thursday, July 21, 2005
Training
I recently had someone contact me, someone who had read my book and wanted to see about attending my course. Well, one way to do that would be to sign up at MegaMind.org (note: the course actually covers much more than just Window2000). Another way to do it is to have me come on-site and teach it at your facility.
This person suggested that I contact Amazon for a list of names of people who are interested in the training, but to my knowledge, Amazon doesn't maintain anything like that.
So I thought I would pose the question via the blog and see what kind of response I got...so, here it is...
If you're interested in incident response training specific to Windows systems, what sort of program do you prefer? Would you prefer traveling to another location to take the course, or having me come to your facility and teach it? How about the material? The course is specific to incident detection, verification, and resolution on live systems...but I have tailored it to the specific needs of the certain organizations (i.e., I've extended the 2-day course to 5 days, added or removed material as needed, etc.).
How about forensic analysis of Windows systems? As I've already started working on another book along those lines, I can easily see the necessary material for a detailed, hands-on course coming together.
So...what are your thoughts, preferences, comments?
This person suggested that I contact Amazon for a list of names of people who are interested in the training, but to my knowledge, Amazon doesn't maintain anything like that.
So I thought I would pose the question via the blog and see what kind of response I got...so, here it is...
If you're interested in incident response training specific to Windows systems, what sort of program do you prefer? Would you prefer traveling to another location to take the course, or having me come to your facility and teach it? How about the material? The course is specific to incident detection, verification, and resolution on live systems...but I have tailored it to the specific needs of the certain organizations (i.e., I've extended the 2-day course to 5 days, added or removed material as needed, etc.).
How about forensic analysis of Windows systems? As I've already started working on another book along those lines, I can easily see the necessary material for a detailed, hands-on course coming together.
So...what are your thoughts, preferences, comments?
Tuesday, July 19, 2005
A/V software on web servers
There's an interesting thread over on the SecurityFocus Focus-MS list, regarding the installation of anti-virus software on IIS 6.x web servers. I attempted to ignore it, but this morning felt the need to interject.
In a nutshell, my point has simply been that if a web server is just that...a web server...then for the most part, it's serving up content via port 80 (and possibly port 443). If the system is properly configured, why then is anti-virus software needed? If it's just a web server, and your LAN/infrastructure is properly configured and administered, what is the purpose of adding yet another software package to a system, that's going to generate yet another set of logs that will be ignored?
One post mentioned the fact that you can't possibly know all of the threats you'll face. Well, that's true...if you've got your head stuck in the sand. Remote attacks come down to two basic types...those that exploit poor configurations (ie, weak passwords, running unnecessary services, etc.), and those that exploit improper bounds checking in the software itself (ie, buffer overflows). Understanding this makes it easier to reduce the attack surface by greatly restricting the avenues available to an attacker. In fact, it is possible to reduce the attack surface to the point where you still have your necessary functionality, but pretty much the only way to compromise the system is to be the administrator sitting at the console...and at that point, no amount of anti-virus software is going to do you any good.
Another post mentioned that the web server is the public interface to the rest of the world, and anything that gets on the LAN can make it over to the web server, and you may possibly end up with an embarrassing situation (ie, defacement, malware proliferation, etc.). This may be the case...if you've got your web server installed on the same LAN segment as your employee desktops. However, the most likely avenue of attack at that point would be via NetBEUI/file sharing, and it that's enabled on your web server, then it's no longer just a web server, is it?
Take a look at some of the stuff that's hit the Internet. Remember Code Red? Well, some folks at MS set up an IIS 4.0 web server, and it was not susceptible to Code Red a full year before Code Red was launched. In fact, if everyone had simply disabled the proper script mapping the day before Code Red came out, they would not have been vulnerable.
If you're considering putting anti-virus software on a web server, I'd suggest that you look at the root cause as to why you want to do that. Perhaps a better investment would be in training of your staff (ie, support staff, as well as management), or in another product all together.
Am I saying that web servers should never have A/V installed? No, not at all...what I am saying is that before doing so, you should take a hard look at the reasons why you're doing so. Sometimes just setting some ACLs and removing unnecessary services will go a lot further than installing another software package that needs to be maintained.
As a side note, it seems to me that a lot of folks out there are of the mind, "if you're running Windows, you must have anti-virus installed." To me, this seems to be a very uneducated and misinformed position. I do not use A/V software on any of my personal systems, and have never been infected when I haven't done so intentionally. At work, there are no log entries from the corporate A/V software to indicate an infection of any kind on my workstation. Sure, if your kids are using a home system, you'd want to have anti-virus software installed, but that applies regardless of the platform.
Addendum: I was checking out Bruce Schneier's blog this morning and noticed something that rang true with this blog entry. Specifically, in one of his entries about turning off cell phones in tunnels, Bruce says, "This is as idiotic as it gets. It's a perfect example of what I call "movie plot security": imagining a particular scenario rather than focusing on the broad threats." Wow. Bruce gets lauded as a security expert when he says things like that (and I happen to think he's right). However, when I say something like, "There's no point in installing A/V software on a web server", which is pretty much along the same lines as what Bruce said, only mapped to the digital world, I end up getting emails that are better not quoted in public.
Most known threats can be protected against without the use of A/V software on web servers. New threats won't be caught because they're new, and not yet subject to scrutiny by the A/V community.
Oh, and one other thing...more than one person sent me email telling me that they "saw" A/V software protect systems from being infected with the SQLSpida worm. All I can say is, thanks guys, but you made my point for me.
In a nutshell, my point has simply been that if a web server is just that...a web server...then for the most part, it's serving up content via port 80 (and possibly port 443). If the system is properly configured, why then is anti-virus software needed? If it's just a web server, and your LAN/infrastructure is properly configured and administered, what is the purpose of adding yet another software package to a system, that's going to generate yet another set of logs that will be ignored?
One post mentioned the fact that you can't possibly know all of the threats you'll face. Well, that's true...if you've got your head stuck in the sand. Remote attacks come down to two basic types...those that exploit poor configurations (ie, weak passwords, running unnecessary services, etc.), and those that exploit improper bounds checking in the software itself (ie, buffer overflows). Understanding this makes it easier to reduce the attack surface by greatly restricting the avenues available to an attacker. In fact, it is possible to reduce the attack surface to the point where you still have your necessary functionality, but pretty much the only way to compromise the system is to be the administrator sitting at the console...and at that point, no amount of anti-virus software is going to do you any good.
Another post mentioned that the web server is the public interface to the rest of the world, and anything that gets on the LAN can make it over to the web server, and you may possibly end up with an embarrassing situation (ie, defacement, malware proliferation, etc.). This may be the case...if you've got your web server installed on the same LAN segment as your employee desktops. However, the most likely avenue of attack at that point would be via NetBEUI/file sharing, and it that's enabled on your web server, then it's no longer just a web server, is it?
Take a look at some of the stuff that's hit the Internet. Remember Code Red? Well, some folks at MS set up an IIS 4.0 web server, and it was not susceptible to Code Red a full year before Code Red was launched. In fact, if everyone had simply disabled the proper script mapping the day before Code Red came out, they would not have been vulnerable.
If you're considering putting anti-virus software on a web server, I'd suggest that you look at the root cause as to why you want to do that. Perhaps a better investment would be in training of your staff (ie, support staff, as well as management), or in another product all together.
Am I saying that web servers should never have A/V installed? No, not at all...what I am saying is that before doing so, you should take a hard look at the reasons why you're doing so. Sometimes just setting some ACLs and removing unnecessary services will go a lot further than installing another software package that needs to be maintained.
As a side note, it seems to me that a lot of folks out there are of the mind, "if you're running Windows, you must have anti-virus installed." To me, this seems to be a very uneducated and misinformed position. I do not use A/V software on any of my personal systems, and have never been infected when I haven't done so intentionally. At work, there are no log entries from the corporate A/V software to indicate an infection of any kind on my workstation. Sure, if your kids are using a home system, you'd want to have anti-virus software installed, but that applies regardless of the platform.
Addendum: I was checking out Bruce Schneier's blog this morning and noticed something that rang true with this blog entry. Specifically, in one of his entries about turning off cell phones in tunnels, Bruce says, "This is as idiotic as it gets. It's a perfect example of what I call "movie plot security": imagining a particular scenario rather than focusing on the broad threats." Wow. Bruce gets lauded as a security expert when he says things like that (and I happen to think he's right). However, when I say something like, "There's no point in installing A/V software on a web server", which is pretty much along the same lines as what Bruce said, only mapped to the digital world, I end up getting emails that are better not quoted in public.
Most known threats can be protected against without the use of A/V software on web servers. New threats won't be caught because they're new, and not yet subject to scrutiny by the A/V community.
Oh, and one other thing...more than one person sent me email telling me that they "saw" A/V software protect systems from being infected with the SQLSpida worm. All I can say is, thanks guys, but you made my point for me.
Monday, July 18, 2005
Bots writing Registry entries
As I've purused some of the anti-virus sites of late, I've noticed a trend that malware...specifically, bots...are writing two particular Registry entries:
[HKCUHKLM]\System\CurrentControlSet\Control\Lsa
and
[HKCUHKLM]\Software\Microsoft\OLE
I'm seeing this with several bots...W32.Bropia, W32.MyTob, etc. Some A/V sites point out that these are variations of SD-Bot, which wrote to the keys, as well...but why? A/V companies do a great job of saying which keys get created or modified, but it's tough to figure out *why*.
What's the purpose for writing to these keys? Does it have something to do with the LSASS vulnerality in MS04-011? Is this another autostart location?
[HKCUHKLM]\System\CurrentControlSet\Control\Lsa
and
[HKCUHKLM]\Software\Microsoft\OLE
I'm seeing this with several bots...W32.Bropia, W32.MyTob, etc. Some A/V sites point out that these are variations of SD-Bot, which wrote to the keys, as well...but why? A/V companies do a great job of saying which keys get created or modified, but it's tough to figure out *why*.
What's the purpose for writing to these keys? Does it have something to do with the LSASS vulnerality in MS04-011? Is this another autostart location?
Wednesday, July 13, 2005
Prefetch file metadata
I exchanged emails with the anonymous poster from my previous metadata entry, and got an interesting perspective. Specifically, not enough of the folks actually performing forensic analysis of Windows XP systems are aware of the Prefetch directory and what it contains.
This reminds me of a very brief exchange I had w/ one of the virus writers from the group 29A a while back. Specifically, Benny and Ratter had written some viruses that took advantage of NTFS alternate data streams, and I asked them where they saw things going. The response I got back stated, in brief, that it was a deadend b/c everyone knows about ADSs.
Hhhhmmm...so why is it that when I talk about them at conferences, attendees sit up and say things like, "Okay...go back a sec..."??
My point is that just b/c some of us know something, we have to realize that not everyone does. Just b/c someone is, say, a forensic analyst for local, state, or even federal law enforcement, that doesn't mean that they know all of the ins and outs of Windows XP.
Keeping that in mind, .pf files within the Prefetch directory have certain metadata associated with them, specifically, the file contains several Unicode strings (view using strings.exe or BinText from FoundStone), one of which is the path to executable image. So you will see from where the executable was launched.
So...outside of strings and MAC times...what is there? Has anyone ever seen an ADS associated with a .pf file?
This reminds me of a very brief exchange I had w/ one of the virus writers from the group 29A a while back. Specifically, Benny and Ratter had written some viruses that took advantage of NTFS alternate data streams, and I asked them where they saw things going. The response I got back stated, in brief, that it was a deadend b/c everyone knows about ADSs.
Hhhhmmm...so why is it that when I talk about them at conferences, attendees sit up and say things like, "Okay...go back a sec..."??
My point is that just b/c some of us know something, we have to realize that not everyone does. Just b/c someone is, say, a forensic analyst for local, state, or even federal law enforcement, that doesn't mean that they know all of the ins and outs of Windows XP.
Keeping that in mind, .pf files within the Prefetch directory have certain metadata associated with them, specifically, the file contains several Unicode strings (view using strings.exe or BinText from FoundStone), one of which is the path to executable image. So you will see from where the executable was launched.
So...outside of strings and MAC times...what is there? Has anyone ever seen an ADS associated with a .pf file?
Tuesday, July 12, 2005
"Hidden" event records
I can't release the info I have yet, and I'm not trying to tease anyone...honest. However, I can say this...the method I'm using to retrieve event records from an Event Log file is very interesting. In my single-minded simplicity, I've found a way to locate "hidden" event records!
Seriously. I have a test Event Log file from a system, and using the MS API (ie, Event Viewer, psloglist.exe, and the Perl Win32::EventLog module), I can "see" 2363 total event records, running from 11239 to 13601, inclusive. However, using my Perl script, and verifying it by hand, I can also "see" event number 11238, which wasn't over written yet...the header information for the Event Log file simply tells the MS API to start with event ID 11239 (by giving not only the event ID number, but the offset where it's located within the file).
Very interesting stuff.
Seriously. I have a test Event Log file from a system, and using the MS API (ie, Event Viewer, psloglist.exe, and the Perl Win32::EventLog module), I can "see" 2363 total event records, running from 11239 to 13601, inclusive. However, using my Perl script, and verifying it by hand, I can also "see" event number 11238, which wasn't over written yet...the header information for the Event Log file simply tells the MS API to start with event ID 11239 (by giving not only the event ID number, but the offset where it's located within the file).
Very interesting stuff.
Saturday, July 09, 2005
File Metadata
I've been working on my GMU2005 presentation regarding file metadata on Windows systems...basically, showing the types of metadata that are in and associated with various files on Windows boxen.
The stuff I've covered includes Office documents (I even include my MergeStreams demo, b/c it's way cool), PDF documents, Event Log files, and PE file headers. I also cover NTFS Alternate Data Streams and MAC times.
Am I missing anything really obvious here? My goal with this presentation is to tell the audience, "hey, guys and gals...there're all these files on Windows systems, and they're usually there by default, in many environments. There's a lot more information you can pull from them than just the fact that they exist."
I'm just trying to do a sanity check. I went back through my book to see if there's anything I really missed, and I think I've got it. Sometimes, you get to doing this stuff so often, that you stop seeing how important it is for others in the field to know it...you stop seeing the forest for the trees, so to speak.
Oh, and I emailed the guy in charge of the GMU2005 conference, and asked if I could be squeezed in at the last minute with a presentation on the Event Log file format. I specifically asked to get a slot during prime time...not at 5pm on the last day. We'll see how it goes...but I'll be putting the actual presentation together next week and getting it in the approval pipeline. That'll give me 5 presentations at a single conference. Ouch!
And yes, once the presentations have been approved for public release, I'll post them.
The stuff I've covered includes Office documents (I even include my MergeStreams demo, b/c it's way cool), PDF documents, Event Log files, and PE file headers. I also cover NTFS Alternate Data Streams and MAC times.
Am I missing anything really obvious here? My goal with this presentation is to tell the audience, "hey, guys and gals...there're all these files on Windows systems, and they're usually there by default, in many environments. There's a lot more information you can pull from them than just the fact that they exist."
I'm just trying to do a sanity check. I went back through my book to see if there's anything I really missed, and I think I've got it. Sometimes, you get to doing this stuff so often, that you stop seeing how important it is for others in the field to know it...you stop seeing the forest for the trees, so to speak.
Oh, and I emailed the guy in charge of the GMU2005 conference, and asked if I could be squeezed in at the last minute with a presentation on the Event Log file format. I specifically asked to get a slot during prime time...not at 5pm on the last day. We'll see how it goes...but I'll be putting the actual presentation together next week and getting it in the approval pipeline. That'll give me 5 presentations at a single conference. Ouch!
And yes, once the presentations have been approved for public release, I'll post them.
Friday, July 08, 2005
Where, oh, where did my little SSID go...?
Got wireless? Ever go to the Control Panel, to the Network Connections applet, open up the properties for the Wireless Network Connection, click on the Wireless Networks tab, and see a whole bunch of SSIDs listed in the "Preferred Networks" box? You probably know how they got there, and you can easily get rid of them...but have you ever wondered where they're kept on the system?
Ever imaged a Windows XP drive and wondered what wireless networks the suspect connected to?
Well, I've been digging around and I've found it. Open up RegEdit and navigate to the following key:
HKLM\Software\Microsoft\WZCSVC\Parameters\Interfaces
See a subkey that looks like a GUID there? I've got one on my system, you may have more. Well, click on the subkey and look over in the right-hand panel. If you don't see values named "ActiveSettings", "Static#0000", etc., then move on to the next GUID.
If you find one of these values, right-click it and choose "Modify". See the SSID in the binary data?
Now, my laptop is a Dell system and uses BroadCom software. If you don't see the values I mentioned in your Registry, check your client application for your wireless stuff, and let me know what you've got. I've read on the 'Net that Cisco and 3Com client apps keep the SSIDs in the Registry in plain text.
Ever imaged a Windows XP drive and wondered what wireless networks the suspect connected to?
Well, I've been digging around and I've found it. Open up RegEdit and navigate to the following key:
HKLM\Software\Microsoft\WZCSVC\Parameters\Interfaces
See a subkey that looks like a GUID there? I've got one on my system, you may have more. Well, click on the subkey and look over in the right-hand panel. If you don't see values named "ActiveSettings", "Static#0000", etc., then move on to the next GUID.
If you find one of these values, right-click it and choose "Modify". See the SSID in the binary data?
Now, my laptop is a Dell system and uses BroadCom software. If you don't see the values I mentioned in your Registry, check your client application for your wireless stuff, and let me know what you've got. I've read on the 'Net that Cisco and 3Com client apps keep the SSIDs in the Registry in plain text.
Thursday, July 07, 2005
Rootkit Detection, and a prediction (of sorts)
I was over on Rootkit.com again today, reading up on some of the recent entires...if you're at all interested in Windows security, you should really consider signing up. Anyway, I was reading an article by the erstwhile Joanna Rutkowska on crossview-based rootkit detection, and was really fascinated by what I was reading. Her article on Rootkit.com discusses various issues and does a good job of outlining the "war of attrition" as the good guys develop new ways to detect rootkits, and the rootkit authors (some can be good guys, too...) develop new ways to avoid detection.
Joanna makes some interesting comments, in particular:
One may ask a simple question now: why bother to hide files at all? Isn’t the idea of “hide in the crowed” equally stealth? The answer, fortunately, is no...
and
The answer is no, because the current antivirus technology is able to find all (unhidden) executable files and then perform some kind of analysis if the given PE file looks like a potential rootkit/malware installer (for e.g. check if it uses functions like OpenProcess(), OpenSCManager(), ZwSetSystemInformation() and similar). When designing such scanner we need to remember that rootkit executable can comprises of two parts, one being an actual malware loader and the other being a (polymorphic) decoder.
I'm not sure that I entirely agree with her statement, though her reasoning is certainly sound. First off, let me just say that we all come from different places and have different opinions based on different experiences. For example, I have military training in my background, which includes the concept of Maneuver Warfare (as practiced by experts). Given that, and also given that we're seeing more and more attacks in the media that seem to take a more economic or financial focus, my thought is that we're going to see more targetted attacks.
What does this mean? Well, rather than going for mass infections, we'll likely see programs installed on fewer, but targetted machines. Am I saying that this is the death of Internet worms? Not at all...we'll have those around for a long while yet. But what I am saying is that it's very likely that the worms will be test cases...what works, how "noisy" is it, how quickly is something detected and turned over to the anti-virus vendors for analysis and signature creation? With this kind of information, the attacker can target his approach...and all without rootkit technology.
I'll give you an example. In the military, it's commonly known that during inspections, you give the inspector something to find...b/c if you don't, he won't leave until he's gone to some very dark, uncomfortable places with a microscope and a pen light. So, you give him something to find...not too significant, but enough to satisfy him so he'll...well...go away. Well, map that sort of thing over to what we've been seeing since the inception of viruses, and especially since backdoors like Back Orifice were released...when the incident occurs, it's detected b/c it has a significant and often immediately noticeable impact on systems. Well, what if the attacker decided to be really stealthy, and not give the inspector (Administrator, in this case) any cause to even look around in the first place?
Why are rootkits used? To hide the attacker's presence when the administrator or investigator comes looking. So...don't do anything to cause the investigator to look in the first place.
Where are attacks going? Think about maneuver warfare...one of the concepts is to bypass strongpoints. Marines assaulting a beach will bypass a bunker that's facing the beach, and cut it off from the rear, choking off the supply routes that keep the guys in the bunker in beans, bullets, and band-aids. The same holds true with crime...bad guys are going to attack the easy targets first...the unlocked cars and houses, the unescorted children and women, etc. Online, something that looks like it's fairly unattended/unmanaged will be attacked first. Why go after the heavily protected server, where *if* you do get in, you'll create a lot of noise in doing so (in my book, I used the example of Ethan Hawke in Mission Impossible crushing up a light bulb and spreading the shards outside the apartment in the safe house...), and someone's going to come looking.
Attacks are likely going to be targetting less well protected systems, and the attacks are likely going to have less of an impact on the systems over all. The attacks will be more subtle, and the attacker is going to take great pains to stay stealthy and hidden, by not attracting attention to the fact that he's there. Do you need rootkit technology for this? No. It's been widely seen that it doesn't take a lot of effort to remain hidden from most administrators, even if you're hiding in plain sight (no disrespect intended, guys and gals). Adding a program to a system that isn't going to be detected by anti-virus software (all that takes is something new), isn't going to create a lot of noise, and isn't going crash or overwhelm the system is all it takes.
Are you like me, and need examples and specifics? No problem. Anyone remember
Joanna makes some interesting comments, in particular:
One may ask a simple question now: why bother to hide files at all? Isn’t the idea of “hide in the crowed” equally stealth? The answer, fortunately, is no...
and
The answer is no, because the current antivirus technology is able to find all (unhidden) executable files and then perform some kind of analysis if the given PE file looks like a potential rootkit/malware installer (for e.g. check if it uses functions like OpenProcess(), OpenSCManager(), ZwSetSystemInformation() and similar). When designing such scanner we need to remember that rootkit executable can comprises of two parts, one being an actual malware loader and the other being a (polymorphic) decoder.
I'm not sure that I entirely agree with her statement, though her reasoning is certainly sound. First off, let me just say that we all come from different places and have different opinions based on different experiences. For example, I have military training in my background, which includes the concept of Maneuver Warfare (as practiced by experts). Given that, and also given that we're seeing more and more attacks in the media that seem to take a more economic or financial focus, my thought is that we're going to see more targetted attacks.
What does this mean? Well, rather than going for mass infections, we'll likely see programs installed on fewer, but targetted machines. Am I saying that this is the death of Internet worms? Not at all...we'll have those around for a long while yet. But what I am saying is that it's very likely that the worms will be test cases...what works, how "noisy" is it, how quickly is something detected and turned over to the anti-virus vendors for analysis and signature creation? With this kind of information, the attacker can target his approach...and all without rootkit technology.
I'll give you an example. In the military, it's commonly known that during inspections, you give the inspector something to find...b/c if you don't, he won't leave until he's gone to some very dark, uncomfortable places with a microscope and a pen light. So, you give him something to find...not too significant, but enough to satisfy him so he'll...well...go away. Well, map that sort of thing over to what we've been seeing since the inception of viruses, and especially since backdoors like Back Orifice were released...when the incident occurs, it's detected b/c it has a significant and often immediately noticeable impact on systems. Well, what if the attacker decided to be really stealthy, and not give the inspector (Administrator, in this case) any cause to even look around in the first place?
Why are rootkits used? To hide the attacker's presence when the administrator or investigator comes looking. So...don't do anything to cause the investigator to look in the first place.
Where are attacks going? Think about maneuver warfare...one of the concepts is to bypass strongpoints. Marines assaulting a beach will bypass a bunker that's facing the beach, and cut it off from the rear, choking off the supply routes that keep the guys in the bunker in beans, bullets, and band-aids. The same holds true with crime...bad guys are going to attack the easy targets first...the unlocked cars and houses, the unescorted children and women, etc. Online, something that looks like it's fairly unattended/unmanaged will be attacked first. Why go after the heavily protected server, where *if* you do get in, you'll create a lot of noise in doing so (in my book, I used the example of Ethan Hawke in Mission Impossible crushing up a light bulb and spreading the shards outside the apartment in the safe house...), and someone's going to come looking.
Attacks are likely going to be targetting less well protected systems, and the attacks are likely going to have less of an impact on the systems over all. The attacks will be more subtle, and the attacker is going to take great pains to stay stealthy and hidden, by not attracting attention to the fact that he's there. Do you need rootkit technology for this? No. It's been widely seen that it doesn't take a lot of effort to remain hidden from most administrators, even if you're hiding in plain sight (no disrespect intended, guys and gals). Adding a program to a system that isn't going to be detected by anti-virus software (all that takes is something new), isn't going to create a lot of noise, and isn't going crash or overwhelm the system is all it takes.
Are you like me, and need examples and specifics? No problem. Anyone remember
Event Log file format
As a follow-up to my earlier post on the EventLogRecord structure, I wanted to mention that after no small effort, and with some assistance from someone involved with PyFlag, I was able to figure out the format of Event Log files.
Okay, that this point, you're probably thinking...so what? Well, consider this..you're analyzing a Windows system, but you're on Linux. Or you have a corrupted Event Log file, and the Event Viewer (and even psloglist.exe) can't open it. Or you're looking for event records in slack space. In any one of these instances, knowing the structure of the Event Log file would be very helpful. I've drafted a paper, and I need to see about getting it through my employee for public release. If that doesn't work, or it gets stalled, I'll see about releasing the information via another (albiet acceptable) means.
So why am I telling you this, only to say, "I can't release it yet?" Well, my thought was that if anyone has a pressing need to know something about the Event Log file format now (and I mean right now), send me an email and I'll see what I can do to answer your question(s). Otherwise, hang tight and I'll see about getting the information out.
The other reason I'm posting this is to ask about forums/magazines suitable for posting this sort of information. I've been working on articles for the Digital Investigation Journal, but it does take a while for the article to be available to the public. If there's a forum similar to the DIJ, but quicker...let me know.
Addendum: I wanted to add a couple of comments with regards to my effort in this project. First off, since I started this project, a tool called GrokEVT was released. A post hit the SF Forensics list, and I initially caught wind of it there, and since I'd posted asking about this subject, I got a couple of emails from folks pointing it out. There are also some other materials out there, but I can't provide links, b/c right now the links seem to be broken.
Anyway...GrokEVT looks like an excellent tool. It seems to do pretty much everything; extract the event records from the file, search the Registry for message files, then extract the message strings from the file. However, the documentation does state that some of these functions are "unstable". Well...it's a good start.
The one thing that the package doesn't seem to do is explain the format of the Event Log file itself. Yes, I've only looked at a small piece of the puzzle, and no, my solution isn't as comprehensive as GrokEVT or PyFlag. However, the little bit that I've done does provide the forensic analyst with the necessary information to locate event records in slack space, and extract and interpret those records. What I've also done is create a Perl script that uses several functions to retrieve event records from a file. These functions can be used to retrieve records from a corrupted or partially deleted Event Log file.
Knowledge of the Event Log file format is also useful in understanding and detecting anti-forensics techniques involving the Event Log.
My hope is that someone finds this information useful.
Okay, that this point, you're probably thinking...so what? Well, consider this..you're analyzing a Windows system, but you're on Linux. Or you have a corrupted Event Log file, and the Event Viewer (and even psloglist.exe) can't open it. Or you're looking for event records in slack space. In any one of these instances, knowing the structure of the Event Log file would be very helpful. I've drafted a paper, and I need to see about getting it through my employee for public release. If that doesn't work, or it gets stalled, I'll see about releasing the information via another (albiet acceptable) means.
So why am I telling you this, only to say, "I can't release it yet?" Well, my thought was that if anyone has a pressing need to know something about the Event Log file format now (and I mean right now), send me an email and I'll see what I can do to answer your question(s). Otherwise, hang tight and I'll see about getting the information out.
The other reason I'm posting this is to ask about forums/magazines suitable for posting this sort of information. I've been working on articles for the Digital Investigation Journal, but it does take a while for the article to be available to the public. If there's a forum similar to the DIJ, but quicker...let me know.
Addendum: I wanted to add a couple of comments with regards to my effort in this project. First off, since I started this project, a tool called GrokEVT was released. A post hit the SF Forensics list, and I initially caught wind of it there, and since I'd posted asking about this subject, I got a couple of emails from folks pointing it out. There are also some other materials out there, but I can't provide links, b/c right now the links seem to be broken.
Anyway...GrokEVT looks like an excellent tool. It seems to do pretty much everything; extract the event records from the file, search the Registry for message files, then extract the message strings from the file. However, the documentation does state that some of these functions are "unstable". Well...it's a good start.
The one thing that the package doesn't seem to do is explain the format of the Event Log file itself. Yes, I've only looked at a small piece of the puzzle, and no, my solution isn't as comprehensive as GrokEVT or PyFlag. However, the little bit that I've done does provide the forensic analyst with the necessary information to locate event records in slack space, and extract and interpret those records. What I've also done is create a Perl script that uses several functions to retrieve event records from a file. These functions can be used to retrieve records from a corrupted or partially deleted Event Log file.
Knowledge of the Event Log file format is also useful in understanding and detecting anti-forensics techniques involving the Event Log.
My hope is that someone finds this information useful.
System Analysis
Ever notice how you go to a conference or view something online that talks about "analysis", and all you see/hear/read about is data collection?
I was at a conference a couple of weeks ago, and one of the speakers was giving a presentation on "live data analysis". The speaker did a great job talking about collecting data...but that's not analysis. Or is it?
I have to say at this point, I'm confused. I've been thinking for a while that data collection and analysis are two different things. After all, we see it all around us as two separate actions. The men and women on TV shows like "CSI" do collection...and then they perform their analysis.
At the beginning of every month, I like to drop by the E-Evidence.info site to see what wonderful new papers and presentations are posted. I love reading through some of the stuff that appears on the site. At one point, I even went back through the archives from previous months and years. There's always something interesting posted here.
Yesterday, I ran across a paper about checking Windows boxen for signs of compromise. Reading through the paper, I think it's extremely useful to the right audience, but it doesn't say anything about analysis...it's all about running tools. Some of the descriptions of tools and why they are used are pretty bare, to say the least. But like I said, it's a great paper for the right audience.
I then read through a presentation on the "live analysis" of a Linux system. Again...go here, get this tool, run it. While the presentation does present issues such as Tor networks used for anonymity and privacy (if you're interested in that kind of thing, check out VPM), it really doesn't do much to cover "analysis". Even Dana Epp's 2004 presentation includes the word "analysis" in the title, but the presentation itself only glosses over any actual analysis.
My point is that data collection is easy...it's the analysis that's the hard part, and what we need to start focusing on. The tools are there. Techniques and methodologies for collecting data abound. But I'm not saying that we shouldn't keep presenting and writing on data collection...what I am saying is that if the presentation or paper has the word "analysis" in the title, then analysis should be discussed.
Okay, I know what you're going to say..."hey, dude, chill! There are just too many possible things that someone can look for when doing analysis." And I'd agree with you. But I also think that a really cool way to do presentations is to pick something...something you've seen or done, or something that's of interest to your audience (I know, it's really hard to get information on what others are interested in...)...and go over that in detail. I've decided that for my part, I'm going to start doing more to cover the actual analysis...now that you have the data, what is it telling you...in my papers and presentations. In fact, I submitted an abstract for the DoD CyberCrime Conference for a paper/presentation that does exactly that. I don't know if the proposal has been accepted yet or not, but my intention is to walk through the analysis of a corporate case, specifically the theft of proprietary information. It should be interesting...not only in putting the presentation together, but also in the audience's reaction.
I was at a conference a couple of weeks ago, and one of the speakers was giving a presentation on "live data analysis". The speaker did a great job talking about collecting data...but that's not analysis. Or is it?
I have to say at this point, I'm confused. I've been thinking for a while that data collection and analysis are two different things. After all, we see it all around us as two separate actions. The men and women on TV shows like "CSI" do collection...and then they perform their analysis.
At the beginning of every month, I like to drop by the E-Evidence.info site to see what wonderful new papers and presentations are posted. I love reading through some of the stuff that appears on the site. At one point, I even went back through the archives from previous months and years. There's always something interesting posted here.
Yesterday, I ran across a paper about checking Windows boxen for signs of compromise. Reading through the paper, I think it's extremely useful to the right audience, but it doesn't say anything about analysis...it's all about running tools. Some of the descriptions of tools and why they are used are pretty bare, to say the least. But like I said, it's a great paper for the right audience.
I then read through a presentation on the "live analysis" of a Linux system. Again...go here, get this tool, run it. While the presentation does present issues such as Tor networks used for anonymity and privacy (if you're interested in that kind of thing, check out VPM), it really doesn't do much to cover "analysis". Even Dana Epp's 2004 presentation includes the word "analysis" in the title, but the presentation itself only glosses over any actual analysis.
My point is that data collection is easy...it's the analysis that's the hard part, and what we need to start focusing on. The tools are there. Techniques and methodologies for collecting data abound. But I'm not saying that we shouldn't keep presenting and writing on data collection...what I am saying is that if the presentation or paper has the word "analysis" in the title, then analysis should be discussed.
Okay, I know what you're going to say..."hey, dude, chill! There are just too many possible things that someone can look for when doing analysis." And I'd agree with you. But I also think that a really cool way to do presentations is to pick something...something you've seen or done, or something that's of interest to your audience (I know, it's really hard to get information on what others are interested in...)...and go over that in detail. I've decided that for my part, I'm going to start doing more to cover the actual analysis...now that you have the data, what is it telling you...in my papers and presentations. In fact, I submitted an abstract for the DoD CyberCrime Conference for a paper/presentation that does exactly that. I don't know if the proposal has been accepted yet or not, but my intention is to walk through the analysis of a corporate case, specifically the theft of proprietary information. It should be interesting...not only in putting the presentation together, but also in the audience's reaction.
Friday, July 01, 2005
The media, and how they skew "attacks"
I read an article in the Bozeman Daily Chronicle today, about a database system housing personal information about hunters in Montana was compromised.
I have to say, I'm extremely disappointed, not only with the state of IT, but of the popular media.
Reading through the article, it's pretty clear what happened. A system owned and administered by folks in Montana was compromised, and someone tried to turn it into a warez server. In fact, the activity that appeared in the logs could have been completely automated. Many of us have seen this sort of thing before...an automated script scans for FTP servers and tries to log into the "anonymous" account. If it's able to do so, it tries to create a directory, in order to see if it has write access. If the script is successful, it either logs the IP address of the vulnerable system and moves on, or it creates the necessary directory structure and starts uploading files. Of course, this is one of many ways that this kind of activity can occur.
So what's my point? Well, if I were a hunter in Montana, I'd want to know why my personal information was on a database system that was accessible from the Internet, in a manner such that it could be attacked in this way. What service was attacked? The database is Oracle, and though that software has had it's share of vulnerabilities, I don't get the sense (again, my source being the article in question) that the database itself was attacked...but that the system it was running on was attacked through another (possibly unnecessary) service. So...why was it connected to the Internet in such a way as to be accessed by this "attacker", and what (potentially unnecessary) services/daemons were running on it and why?
Speaking of questions, the author of the article had an excellent opportunity to make a mark by asking those tough questions. I believe that legislation is getting us to the point where incidents such as this must be reported. Now what needs to happen is that knowledgeable people need to ask the tough questions...why was the system connected to the Internet in this manner? Who is responsible for the design decision? Who is responsible for the administration of the system? Once these questions start to be asked, maybe the IT folks actually making the decisions will start thinking a bit harder about what they're doing.
So, while we don't...and probably never will...have all of the information about the attack and what actually occurred, articles like this tend to spread FUD amongst the parts of the population that aren't as familiar with security (this includes a lot of IT folks) issues as some of us. I'm not saying that I'm an expert, but I do know enough to recognize FUD like this...
I have to say, I'm extremely disappointed, not only with the state of IT, but of the popular media.
Reading through the article, it's pretty clear what happened. A system owned and administered by folks in Montana was compromised, and someone tried to turn it into a warez server. In fact, the activity that appeared in the logs could have been completely automated. Many of us have seen this sort of thing before...an automated script scans for FTP servers and tries to log into the "anonymous" account. If it's able to do so, it tries to create a directory, in order to see if it has write access. If the script is successful, it either logs the IP address of the vulnerable system and moves on, or it creates the necessary directory structure and starts uploading files. Of course, this is one of many ways that this kind of activity can occur.
So what's my point? Well, if I were a hunter in Montana, I'd want to know why my personal information was on a database system that was accessible from the Internet, in a manner such that it could be attacked in this way. What service was attacked? The database is Oracle, and though that software has had it's share of vulnerabilities, I don't get the sense (again, my source being the article in question) that the database itself was attacked...but that the system it was running on was attacked through another (possibly unnecessary) service. So...why was it connected to the Internet in such a way as to be accessed by this "attacker", and what (potentially unnecessary) services/daemons were running on it and why?
Speaking of questions, the author of the article had an excellent opportunity to make a mark by asking those tough questions. I believe that legislation is getting us to the point where incidents such as this must be reported. Now what needs to happen is that knowledgeable people need to ask the tough questions...why was the system connected to the Internet in this manner? Who is responsible for the design decision? Who is responsible for the administration of the system? Once these questions start to be asked, maybe the IT folks actually making the decisions will start thinking a bit harder about what they're doing.
So, while we don't...and probably never will...have all of the information about the attack and what actually occurred, articles like this tend to spread FUD amongst the parts of the population that aren't as familiar with security (this includes a lot of IT folks) issues as some of us. I'm not saying that I'm an expert, but I do know enough to recognize FUD like this...
Thursday, June 30, 2005
Reading Windows files in binmode()
For the past couple of days, I've been writing Perl scripts to parse binary data on Windows systems...I've been staring at 1s and 0s, hex, parsing strings, etc.
My first exercise was to parse PE headers...and the script works very well for legit PE files. By reviewing the information available on MSDN, and correlating that information with other sources (sorry, guys, but there are some holes in your docs!), I have been able to parse all the way down paste the data directories and into the section headers. Very cool! There's still a lot of work to do to make this script really useful, but developing it has really been very beneficial in understanding PE headers and malware.
Now, I'm working on a Perl script to parse .evt files manually, by opening the file in binmode() and parsing the byte stream. The problem I'm having is that even though the EVENTLOGRECORD structure is well documented at the MSDN site, I have not been able to find any information about the data located between offset 0 of the file, and the offset of the first record (which itself seems variable, depending on log type, operating system, etc.). Byte alignment is important, so I know that the API has some inherent method for locating the various records. However, I'm trying to read in the file, basically, a byte at a time...does anyone have any information about the .evt file header info? I'd like to figure out how to parse and make sense of this data.
While this whole thing seems like a pointless exercise, there is a method to my madness. For example, I can use scripts like this with ProDiscover 4.0 to reduce the time it takes to analyze a system. If you know what it is you're looking for, you can automate the activity, increasing efficiency, reducing mistakes, etc. So let's say I have a ProScript (ProDiscover uses a Perl module called ProScript to implement Perl as its scripting language) that I use to identify can copy files from an image. I can then automate scripts using...you guessed it...Perl in order to help me find only the suspicious things. This is referred to as "data reduction". By using the ProScript API, I can automate adding this information to my reports, as well.
My first exercise was to parse PE headers...and the script works very well for legit PE files. By reviewing the information available on MSDN, and correlating that information with other sources (sorry, guys, but there are some holes in your docs!), I have been able to parse all the way down paste the data directories and into the section headers. Very cool! There's still a lot of work to do to make this script really useful, but developing it has really been very beneficial in understanding PE headers and malware.
Now, I'm working on a Perl script to parse .evt files manually, by opening the file in binmode() and parsing the byte stream. The problem I'm having is that even though the EVENTLOGRECORD structure is well documented at the MSDN site, I have not been able to find any information about the data located between offset 0 of the file, and the offset of the first record (which itself seems variable, depending on log type, operating system, etc.). Byte alignment is important, so I know that the API has some inherent method for locating the various records. However, I'm trying to read in the file, basically, a byte at a time...does anyone have any information about the .evt file header info? I'd like to figure out how to parse and make sense of this data.
While this whole thing seems like a pointless exercise, there is a method to my madness. For example, I can use scripts like this with ProDiscover 4.0 to reduce the time it takes to analyze a system. If you know what it is you're looking for, you can automate the activity, increasing efficiency, reducing mistakes, etc. So let's say I have a ProScript (ProDiscover uses a Perl module called ProScript to implement Perl as its scripting language) that I use to identify can copy files from an image. I can then automate scripts using...you guessed it...Perl in order to help me find only the suspicious things. This is referred to as "data reduction". By using the ProScript API, I can automate adding this information to my reports, as well.
Friday, June 24, 2005
More on memory dump analysis, and some other stuff
First, I'm back from the MISTI "Cracking eFraud" conference. It was great to meet a lot of the folks there, both other presenters, as well as attendees.
I attended Brian Carrier's presentation entitled "Live Forensic Analysis". Brian was kind enough to sign my copy of his book for me. I also need to get a copy of Dan Farmer's book, as well.
While I enjoyed Brian's presentation, I have to say that there wasn't a great deal of "analysis" discussed. Yes, Brian covered tools and techniques, talked about rootkits...but analysis techniques weren't really discussed. However, I don't think that a one-hour presentation was really the venue for that sort of topic. I gave a presentation earlier in the morning on looking in the Registry for specific data, and went over by a few minutes...even though I felt that I was glossing over a lot of things.
Maybe the whole idea of presenting on "analysis" really needs to be a full-day presentation, with hands-on exercises, etc.
Anyway, I wanted to blog on memory dump analysis a bit more this morning. What I'm talking about here is using tools such as dd.exe to grab the contents of physical memory; i.e., RAM. As I blogged earlier, on Windows systems, if you want to grab an "image" of physical memory, you have to generate a crash dump, in which the system halts and the contents of physical memory are written to a file. However, most of the systems we would likely see aren't set up for this...so lots of folks, including LEOs, are using dd.exe to dump the contents of physical memory.
Once you have this file, what do/can you do with it? Well, the first and most obvious thing is to run strings.exe against the file or open it up in BinText. You might also run scripts against the file, looking for specific types of strings, such as email addresses, IP addresses, etc. Such things might be useful.
Another thing you might do is to break the dump down into 4K pages and generate hashes for those pages, and then parse through the file system, doing the same thing for the files. Matches would let you know if the pages were loaded in memory (credit for that one goes to Dan Farmer).
But what about parsing kernel structures? Microsoft has many of the available structures documented in MSDN, so we know what they look like. We know, for instance, that a particular structure is so many bytes long, based on the values in the structure, and we can parse these with computer code, either in C, or in scripting languages such as Perl (using unpack()). Knowing this, we only have one thing left that we need to know...the offsets. The image file itself starts at 0 (or 0x00000000)...we need someway of finding out where the Waldo structure begins in that file, before we can start parsing it.
So...what are your thoughts? Am I off base? Am I close? Do you have have any input at all on how to determine the offsets...other than "ask Microsoft"? Are there any developers out there who can comment on this?
I attended Brian Carrier's presentation entitled "Live Forensic Analysis". Brian was kind enough to sign my copy of his book for me. I also need to get a copy of Dan Farmer's book, as well.
While I enjoyed Brian's presentation, I have to say that there wasn't a great deal of "analysis" discussed. Yes, Brian covered tools and techniques, talked about rootkits...but analysis techniques weren't really discussed. However, I don't think that a one-hour presentation was really the venue for that sort of topic. I gave a presentation earlier in the morning on looking in the Registry for specific data, and went over by a few minutes...even though I felt that I was glossing over a lot of things.
Maybe the whole idea of presenting on "analysis" really needs to be a full-day presentation, with hands-on exercises, etc.
Anyway, I wanted to blog on memory dump analysis a bit more this morning. What I'm talking about here is using tools such as dd.exe to grab the contents of physical memory; i.e., RAM. As I blogged earlier, on Windows systems, if you want to grab an "image" of physical memory, you have to generate a crash dump, in which the system halts and the contents of physical memory are written to a file. However, most of the systems we would likely see aren't set up for this...so lots of folks, including LEOs, are using dd.exe to dump the contents of physical memory.
Once you have this file, what do/can you do with it? Well, the first and most obvious thing is to run strings.exe against the file or open it up in BinText. You might also run scripts against the file, looking for specific types of strings, such as email addresses, IP addresses, etc. Such things might be useful.
Another thing you might do is to break the dump down into 4K pages and generate hashes for those pages, and then parse through the file system, doing the same thing for the files. Matches would let you know if the pages were loaded in memory (credit for that one goes to Dan Farmer).
But what about parsing kernel structures? Microsoft has many of the available structures documented in MSDN, so we know what they look like. We know, for instance, that a particular structure is so many bytes long, based on the values in the structure, and we can parse these with computer code, either in C, or in scripting languages such as Perl (using unpack()). Knowing this, we only have one thing left that we need to know...the offsets. The image file itself starts at 0 (or 0x00000000)...we need someway of finding out where the Waldo structure begins in that file, before we can start parsing it.
So...what are your thoughts? Am I off base? Am I close? Do you have have any input at all on how to determine the offsets...other than "ask Microsoft"? Are there any developers out there who can comment on this?
Saturday, June 18, 2005
Upcoming conferences
Next week, I'll be presenting at the MISTI "Cracking eFraud" conference in Boston. My presentation is on using the Windows Registry as a forensic resource". I'm presenting at (get this!) 0730 on Wed. If you're there, stop by and say hi...I promise that it'll be worth your time.
In August, I'll be giving a couple of presentations at GMU2005. I'll be presenting on document metadata on Windows systems, and on tracking USB devices across Windows systems. It looks like quite a bit of time is scheduled for each of these presentations, and I'm giving them both twice...so I may sneak in MISTI presentation, as well...particularly since the USB data is incorporated in that presentation.
Once again, if you're there, stop by and say hi. At the end of my presentations, I like to give a pop quiz, with copies of my book as prizes.
Also, if you've got or know of a conference coming up where topics like this (or other topics concerning the forensic analysis of Windows systems) would be beneficial to the audience, let me know.
In August, I'll be giving a couple of presentations at GMU2005. I'll be presenting on document metadata on Windows systems, and on tracking USB devices across Windows systems. It looks like quite a bit of time is scheduled for each of these presentations, and I'm giving them both twice...so I may sneak in MISTI presentation, as well...particularly since the USB data is incorporated in that presentation.
Once again, if you're there, stop by and say hi. At the end of my presentations, I like to give a pop quiz, with copies of my book as prizes.
Also, if you've got or know of a conference coming up where topics like this (or other topics concerning the forensic analysis of Windows systems) would be beneficial to the audience, let me know.
Memory collection and analysis follow-up
This topic is by no means closed...in fact, I think that the discussion of dealing with physical memory, memory analysis, etc., for forensics and/or malware analysis is just beginning. Given that I'm one of those folks who has spent the past couple of years knodding my head about using dd.exe to image RAM, simply because I didn't know any better, and given the number of folks I've run into who have done the same thing, it's pretty clear to me that this is an issue that needs to be addressed.
IMHO, the best way to approach this is to provide the knowledge, weigh the pros and cons, and let the user/reader make the decision on how to proceed. For example, sometimes, running tools such as the FSP would be the way to go...correlating and analyzing the results can get you a long way. However, those are user-mode tools and things might be missed...but then again, with the right combination of tools and analysis, you may be able identify those things that are missed, at least...not what the data is, but just the fact that it's missing (think of this as akin to identifying the wind...we can't see it or taste it, but we know it's there based on how it affects the environment).
In other cases, such as malware analysis (and forensics involving specific processes running on the victim system), using the debugger tools to grab the contents of process memory, and then using the same debugger tools to analyze the information retrieved, might be more desireable. You'd get some of the same information about the process as you would with the user-land tools from the FSP (i.e., loaded modules, handles, etc.), but this might be the preferred approach. Putting the tools on a CD and writing the process memory dump file to a thumb drive would be a great way to handle this.
Now, when it comes to a full-out memory dump, so far as I've been able to determine, the only real way to do this is with a crash dump. Crash dumps can be triggered manually, as mentioned in my previous post (via the MS KB article), but doing so requires advanced planning. I'd highly recommend incorporating these changes into malware analysis systems. I'd also recommend that the settings be incorporated into critical systems, as at the very least, analyzing the memory dump would make root cause analysis much easier. One of the methods MS mentions for performing troubleshooting is "send us your crash dump"...Oracle does this as well for their products. For forensic purposes, one has to take this into consideration, but I tend to believe that the benefits may outweigh the risks (i.e., overwriting exculpatory evidence when the full crash dump file is written to disk)...depending upon the situation.
I'd appreciate hearing your thoughts...considering that in the past couple of days, I've completely thrown out the idea of using dd.exe to image physical memory, and thrown out pmdump.exe for getting process memory. Now, I've got to go back and rewrite my already-published articles...
IMHO, the best way to approach this is to provide the knowledge, weigh the pros and cons, and let the user/reader make the decision on how to proceed. For example, sometimes, running tools such as the FSP would be the way to go...correlating and analyzing the results can get you a long way. However, those are user-mode tools and things might be missed...but then again, with the right combination of tools and analysis, you may be able identify those things that are missed, at least...not what the data is, but just the fact that it's missing (think of this as akin to identifying the wind...we can't see it or taste it, but we know it's there based on how it affects the environment).
In other cases, such as malware analysis (and forensics involving specific processes running on the victim system), using the debugger tools to grab the contents of process memory, and then using the same debugger tools to analyze the information retrieved, might be more desireable. You'd get some of the same information about the process as you would with the user-land tools from the FSP (i.e., loaded modules, handles, etc.), but this might be the preferred approach. Putting the tools on a CD and writing the process memory dump file to a thumb drive would be a great way to handle this.
Now, when it comes to a full-out memory dump, so far as I've been able to determine, the only real way to do this is with a crash dump. Crash dumps can be triggered manually, as mentioned in my previous post (via the MS KB article), but doing so requires advanced planning. I'd highly recommend incorporating these changes into malware analysis systems. I'd also recommend that the settings be incorporated into critical systems, as at the very least, analyzing the memory dump would make root cause analysis much easier. One of the methods MS mentions for performing troubleshooting is "send us your crash dump"...Oracle does this as well for their products. For forensic purposes, one has to take this into consideration, but I tend to believe that the benefits may outweigh the risks (i.e., overwriting exculpatory evidence when the full crash dump file is written to disk)...depending upon the situation.
I'd appreciate hearing your thoughts...considering that in the past couple of days, I've completely thrown out the idea of using dd.exe to image physical memory, and thrown out pmdump.exe for getting process memory. Now, I've got to go back and rewrite my already-published articles...
Thursday, June 16, 2005
RAM, memory dumps, and debuggers...oh, my!
One of my recent posts on obtaining the contents of physical memory using userdump.exe attracted the attention of some folks as MS, and I ended up having a long conversation with Robert Hensing on the topic of obtaining and analyzing the contents of (physical) memory.
The long and short of it is this...the tools and techniques you use all depend upon what you want to do. How's that for a direct answer? No, that wasn't Robert's response...he provided much more information than that...I'm just summing it up for you. I'm going to give you a short and sweet explanation, opening this topic up for discussion.
If you want to grab the contents of physical memory, you can do so with dd.exe...keeping in mind that tools like this, as well as LiveKD, don't lock kernel memory prior to performing their dump. Therefore, what you get is a smear, of sorts, as the contents of physical memory are changing all the time, while you're dumping those contents. Also, keep in mind that dd.exe does not produce output that is compatible with MS debuggers.
Now, if you're looking to get the memory used by a process, then the way to go is to download the MS debugging tools, and run those from a thumb drive or CD. The debugging tools have a plethora of switches, but Robert was nice enough to write a VB script that's included in the tools called "adplus.vbs" that will let you easily dump the memory contents of multiple processes. The debugging tools do this by first suspending the process, and after the dump is complete, you can then either resume or kill the process. The output can then be analyzed using the debugging tools.
I have to say that this makes a lot more sense than using just pmdump.exe and strings. Robert even went so far as to point out that this can even be used when hunting for rootkits, as some rootkits will insert code into memory used by all processes...when attempting to perform analysis of the memory dump, the debugger will crash when it attempts to reference memory that doesn't exist or is hidden.
Now, what if you want to capture a snapshot of the system? Well, the only real way to do that is with a crash dump (ie, Blue Screen). And when I say "real", I'm referring to the most forensically sound method of doing this...that being said, being able to do this requires some preparation. I'd highly recommend that if you have critical systems that you're really concerned about, that you put a lot of thought into considering these options. First, take a look at KB254649 for an overview of dump file options on 2000, XP, and 2003, and then KB244139 for a feature that allows a memory dump file to be created from the keyboard. Go through the second KB article thoroughly and consider adding the recommended modifications to critical systems. I'm told that some folks out there have done this.
I'd also consider setting up testing systems with the same options, so that when you're testing malware, you can generate dumps of process memory or even crashdumps as part of your testing and then use the MS debugging tools for your analysis.
Robert added that there are debugger plugins that allow you to view and analyze very useful information, such as TCP/IP connection information, from a full crashdump, as this information is handled by the kernel.
So...the long and short of this is that some recommendations that work on Linux systems aren't exactly advisable on Windows boxen, depending upon what you want to do. Sure, you can use dd.exe to image memory, but the output isn't compatible with the MS debugging tools, and since there don't seem to be any tools available for really parsing and analyzing this information, it's of limited use.
Other useful resources include MS KB articles on debugging.
The long and short of it is this...the tools and techniques you use all depend upon what you want to do. How's that for a direct answer? No, that wasn't Robert's response...he provided much more information than that...I'm just summing it up for you. I'm going to give you a short and sweet explanation, opening this topic up for discussion.
If you want to grab the contents of physical memory, you can do so with dd.exe...keeping in mind that tools like this, as well as LiveKD, don't lock kernel memory prior to performing their dump. Therefore, what you get is a smear, of sorts, as the contents of physical memory are changing all the time, while you're dumping those contents. Also, keep in mind that dd.exe does not produce output that is compatible with MS debuggers.
Now, if you're looking to get the memory used by a process, then the way to go is to download the MS debugging tools, and run those from a thumb drive or CD. The debugging tools have a plethora of switches, but Robert was nice enough to write a VB script that's included in the tools called "adplus.vbs" that will let you easily dump the memory contents of multiple processes. The debugging tools do this by first suspending the process, and after the dump is complete, you can then either resume or kill the process. The output can then be analyzed using the debugging tools.
I have to say that this makes a lot more sense than using just pmdump.exe and strings. Robert even went so far as to point out that this can even be used when hunting for rootkits, as some rootkits will insert code into memory used by all processes...when attempting to perform analysis of the memory dump, the debugger will crash when it attempts to reference memory that doesn't exist or is hidden.
Now, what if you want to capture a snapshot of the system? Well, the only real way to do that is with a crash dump (ie, Blue Screen). And when I say "real", I'm referring to the most forensically sound method of doing this...that being said, being able to do this requires some preparation. I'd highly recommend that if you have critical systems that you're really concerned about, that you put a lot of thought into considering these options. First, take a look at KB254649 for an overview of dump file options on 2000, XP, and 2003, and then KB244139 for a feature that allows a memory dump file to be created from the keyboard. Go through the second KB article thoroughly and consider adding the recommended modifications to critical systems. I'm told that some folks out there have done this.
I'd also consider setting up testing systems with the same options, so that when you're testing malware, you can generate dumps of process memory or even crashdumps as part of your testing and then use the MS debugging tools for your analysis.
Robert added that there are debugger plugins that allow you to view and analyze very useful information, such as TCP/IP connection information, from a full crashdump, as this information is handled by the kernel.
So...the long and short of this is that some recommendations that work on Linux systems aren't exactly advisable on Windows boxen, depending upon what you want to do. Sure, you can use dd.exe to image memory, but the output isn't compatible with the MS debugging tools, and since there don't seem to be any tools available for really parsing and analyzing this information, it's of limited use.
Other useful resources include MS KB articles on debugging.
Wednesday, June 15, 2005
Dumping and analyzing physical memory
Well, my research into dumping and analyzing physical memory is progressing. I can't say that I'm finding a positive answer...all I can say is that the research is going well. ;-)
I got in touch with Joanna Rutkowska over at invisiblethings.org about a presentation she gave in Oct '04, and made reference to dd.exe and memory dumps (i.e., crashdumps) created by Windows tools are not compatible. This has been confirmed via other sources.
MS has a tool called userdump.exe (1, 2) that you can use to collect process memory, but it requires that you run a setup program that installs a kernel-mode driver, so it has to be done ahead of time.
An alternative to this kind of crashdump analysis and debugging is LiveKD.
I got in touch with Joanna Rutkowska over at invisiblethings.org about a presentation she gave in Oct '04, and made reference to dd.exe and memory dumps (i.e., crashdumps) created by Windows tools are not compatible. This has been confirmed via other sources.
MS has a tool called userdump.exe (1, 2) that you can use to collect process memory, but it requires that you run a setup program that installs a kernel-mode driver, so it has to be done ahead of time.
An alternative to this kind of crashdump analysis and debugging is LiveKD.
Memory Dump Analysis
One of the things I'm seeing, or should I say, have been seeing for a while, is a move away from the purist approach to forensics, in that actual practitioners are moving away from the thinking that the process starts by shutting off power to the system. I've corresponded with folks using the FSP, and other similar toolkits, be they homebrew, or the WFT.
Besides collecting volatile data, one of the things that's talked about is imaging memory, or collecting a memory dump. For the most part, this has been talked about, but when I've asked the question (as I did at HTCIA2004), "what would you do with it?", most people respond with a blank stare. Then someone way, way in the back (or it sounds like they're way, way in the back...) says, "run strings on it." Okay, that's fine...but what then? How do you associate anything that you find in memory with the case you're working on?
Well, as a start, Mariusz Burdac, over at seccure.net, has released a white paper entitled, "Digital forensics of the physical memory". Now, some of the grammar may throw you off a bit, but keep in mind that this is a start in the right direction. The table of contents of the paper are pretty impressive, addressing some of the issues faced in performing analysis of memory dumps. Over all, I think that the paper is a really good contribution, and a definite step in the right direction. The collection and analysis of information (re: evidence) from live systems is going to become even more important as time goes on.
However, one thing that really threw me was the fact that the author started the paper off by using the FU Rootkit and SQL Slammer worm as examples to justify performing memory dump analysis. The issue I have with this is that the author then goes into analyzing memory from Linux systems...the FU Rootkit and SQL Slammer worm affect Windows systems. The author makes no mention of Windows systems other than to say that the analysis of memory dumps "can be done".
In order to collect the contents of physical memory from a Windows system, you should look into the Forensic Acquisition Utilities from George Garner. The web site includes examples of how to use dd.exe to obtain an image of PhysicalMemory.
One needs to keep in mind, however, that while the system is running, pages are swapped out of memory and into the pagefile (ie, pagefile.sys). Therefore, if you were able to identify the area of memory used by a process, it would be a fragment of the total amount of memory (RAM + pagefile) used by the process. If you're interested in a particular process, I'd recommend using pmdump, instead.
MS does have some documentation regarding using debugger tools to analyze memory dumps. According to MS documentation:
A complete memory dump file contains the entire contents of physical memory when the Stop error occurred. The file size is equal to the amount of physical memory installed plus 1 MB. When a Stop error occurs, the operating system saves a complete memory dump file to a file named systemroot\Memory.dmp and creates a small memory dump file in the systemroot\Minidump folder.
So, maybe that gets us on our way a bit. This is used to analyze Stop errors, but perhaps it can also be used to analyze memory dumps, as well.
It looks like more research is required, as well as some testing. As I'm digging into this, I'd appreciate hearing from folks with regards to what they did, what worked, what didn't work, etc.
Besides collecting volatile data, one of the things that's talked about is imaging memory, or collecting a memory dump. For the most part, this has been talked about, but when I've asked the question (as I did at HTCIA2004), "what would you do with it?", most people respond with a blank stare. Then someone way, way in the back (or it sounds like they're way, way in the back...) says, "run strings on it." Okay, that's fine...but what then? How do you associate anything that you find in memory with the case you're working on?
Well, as a start, Mariusz Burdac, over at seccure.net, has released a white paper entitled, "Digital forensics of the physical memory". Now, some of the grammar may throw you off a bit, but keep in mind that this is a start in the right direction. The table of contents of the paper are pretty impressive, addressing some of the issues faced in performing analysis of memory dumps. Over all, I think that the paper is a really good contribution, and a definite step in the right direction. The collection and analysis of information (re: evidence) from live systems is going to become even more important as time goes on.
However, one thing that really threw me was the fact that the author started the paper off by using the FU Rootkit and SQL Slammer worm as examples to justify performing memory dump analysis. The issue I have with this is that the author then goes into analyzing memory from Linux systems...the FU Rootkit and SQL Slammer worm affect Windows systems. The author makes no mention of Windows systems other than to say that the analysis of memory dumps "can be done".
In order to collect the contents of physical memory from a Windows system, you should look into the Forensic Acquisition Utilities from George Garner. The web site includes examples of how to use dd.exe to obtain an image of PhysicalMemory.
One needs to keep in mind, however, that while the system is running, pages are swapped out of memory and into the pagefile (ie, pagefile.sys). Therefore, if you were able to identify the area of memory used by a process, it would be a fragment of the total amount of memory (RAM + pagefile) used by the process. If you're interested in a particular process, I'd recommend using pmdump, instead.
MS does have some documentation regarding using debugger tools to analyze memory dumps. According to MS documentation:
A complete memory dump file contains the entire contents of physical memory when the Stop error occurred. The file size is equal to the amount of physical memory installed plus 1 MB. When a Stop error occurs, the operating system saves a complete memory dump file to a file named systemroot\Memory.dmp and creates a small memory dump file in the systemroot\Minidump folder.
So, maybe that gets us on our way a bit. This is used to analyze Stop errors, but perhaps it can also be used to analyze memory dumps, as well.
It looks like more research is required, as well as some testing. As I'm digging into this, I'd appreciate hearing from folks with regards to what they did, what worked, what didn't work, etc.
Sunday, June 12, 2005
Shooting oneself in the foot...
I haven't found any really good malware analysis postings lately, but higB (*secureme blog) came to my rescue and posted about a recent, and personal, incident.
In a nutshell, he infected himself with a Trojan, and then went about figuring out what it did. Reading through it, I see that he did a lot of things right.
One of his comments in particular seemed interesting to me: "system.exe looked normal to me." I'm sure this is the case a lot of times, to a lot of admins. I'm on XP Home right now, and don't see "system.exe", though I do see "System" and "System Idle Process" (via Task Manager). Even using tlist.exe, I don't see anything called "system.exe".
Take a look at his post...what would you have done differently? What things would you have done that higB didn't do? What do you think of his tools and techniques for analyzing the file?
In a nutshell, he infected himself with a Trojan, and then went about figuring out what it did. Reading through it, I see that he did a lot of things right.
One of his comments in particular seemed interesting to me: "system.exe looked normal to me." I'm sure this is the case a lot of times, to a lot of admins. I'm on XP Home right now, and don't see "system.exe", though I do see "System" and "System Idle Process" (via Task Manager). Even using tlist.exe, I don't see anything called "system.exe".
Take a look at his post...what would you have done differently? What things would you have done that higB didn't do? What do you think of his tools and techniques for analyzing the file?
Subscribe to:
Posts (Atom)