The Windows Incident Response Blog is dedicated to the myriad information surrounding and inherent to the topics of IR and digital analysis of Windows systems. This blog provides information in support of my books; "Windows Forensic Analysis" (1st thru 4th editions), "Windows Registry Forensics", as well as the book I co-authored with Cory Altheide, "Digital Forensics with Open Source Tools".
Wednesday, February 23, 2005
The Original Debuggers
Well, here's your answer...
Tuesday, February 22, 2005
SysInternals gets in on the game
I ran the tool on my system and saw a lot of "hidden from Windows API" messages. I need to read through the information about the tool to see what that's all about, but this seems like a step in the right direction. At the very least, it's another tool for the arsenal that the bad guys are going to have to figure out a way around.
More thoughts on GhostBuster, etc.
However, there are several caveats that need to be mentioned and understood.
First, rootkits are available. Without question. 'Nuff said on that.
Second, GhostBuster is NOT available. To get Windows PE, you need to pay money. There is no ISO image, nothing. BartPE is available, but it is limited, as well (ie, see #4).
Third, GhostBuster must be used locally. This may not be an option for those organizations with remote sites with no admin (or for those with admins, but such things are beyond their skill sets).
So, what are your options. I know what some of you are going to say, and yes, Knoppix and it's variants are freely available, but the first phase of the scan isn't an immediate option. A way around this might be using something like Helix (thanks for all you work and effort, Dru!)...put the CD in a Windows system, run a batch file with the appropriate commands, and then boot to the Linux distro and continue from there...you will most likely have to have some sort of script (Perl!!) to modify the output of your "ls" command so that it can more easily be diff'ed against the 'dir /s /ah c:\*' command you used in your batch file.
Yet there may be another option. Rootkits work well when viewing the Registry and/or file system locally, but how about remotely? I found that AFX Rootkit 2003 hid Registry keys locally, but when the Registry was viewed remotely, the "hidden" keys were visible. The same thing applies to the file system in cases I've looked at. So, in situations where you can't reboot a system, or where you can't go to the machine and insert a CD, you might consider using psexec.exe from SysInternals.
To use this tool, make an Administrator connection to the remote system (via net.exe) and use psexec to run the local copy of reg.exe on that system (reg.exe ships with XP and above). Then, use reg.exe on your system to run a remote query for the same key, and compare the two outputs.
The first command will look something like this:
C:\>psexec \\computername reg.exe query HKLM\Software\Microsoft\Windows\CurrentVersion\Run
The second command will look something like this:
C:\>reg query \computername\HKLM\Software\Microsoft\Windows\CurrentVersion\Run
Note: The above syntax is taken directly from running 'reg query /?' on an XP system.
Of course, you'll want to redirect the output of each command to a file and then run diff on the two files. Or you may want to wrap the entire thing in Perl so that only the differences are observed.
Now, the same thing applies to the file system...use psexec.exe to run the 'dir' commands locally, then map the root of the C:\ drive, run the commands again against the mapped drive, and diff the output of the files.
This isn't guaranteed to work with all rootkits because to be honest, I haven't seen all possible rootkits. However, this is the same behaviour-based approach I talked about in my book.
More stuff on rootkits from Microsoft
It's interesting that one of the publications refers to a "Strider Ghostbuster Enterprise Scanner", but there's nothing available beyond a paper to download. If you're wondering what I would expect...well, how about instructions from MS regarding how to build bootable Windows CD (it would have to be from them, so as not to violate any licenses), and a downloadable archive of the actual tools themselves that you can add to the ISO image to create your own tool? I'm sure many of us could figure this out on our own, but the folks at Microsoft have already done so...surely they can provide the implementation that they're addressing in their papers.
Microsoft also discussed rootkits at RSA. I have to say that some of the verbage smacks of FUD...particularly where the term "newer" is used. Who are they trying to kid? "Newer" in what sense? While I agree that rootkit authors are smart, I don't necessarily think that it takes a great deal of effort to go undetected on a Windows system. After all, BO was fairly unobtrusive until someone connected to it and started opening and closing the CD tray and generally being annoying. Also, consider that spyware goes pretty much unnoticed until there's enough of it on a system to adversely affect the user experience.
My point is that this is not really that new at all. The chicken little "sky is falling" thing doesn't work that well anymore, guys.
Some of the topics I've been tossing around include:
- Log analysis - This includes Event Logs, primarily, but will also include IIS, PortReporter, etc.
- Registry Mining - Mining for "gold" in the Registry; this is extremely useful for both "live" analysis (admins can do so remotely with simple tools, or during incident response), as well as during forensic analysis of imaged drives.
- Malware Analysis for the Administrator - A detailed look at how to go about figuring out what that suspicious file does, with walk-throughs, caveats, gotchas, etc.
In this book, I would include more detailed walk-throughs, more case studies, more code and examples, etc.
How does this sound? What other topics could/should I address (keeping in mind that this is Windows-specific)? What are some of the topics of interest, the kinds of things that keep you awake at night, that scare the bejebbers out of you? Drop me a comment here, or email me...
Friday, February 18, 2005
Incident Scenario #1, follow up
First off, just as a reminder, this incident scenario was based on a real-world issue, something I dealt with. I have no doubt that others have had to deal with similar incidents.
With regards to your comments, Brandon hit the nail pretty squarely on the head. Remember, this laptop was only used for dial-up access to the Internet...others may have read a bit too much into things and blown the scenario out of proportion.
One things many folks don't seem to remember is that, particularly with Windows 2000, dialing into the Internet still leaves you exposed. I didn't know for sure because the appropriate logging wasn't enabled, but I suspect that a null session was used to enumerate user names, and from there, it was pretty trivial to guess the password...a simple script could be used to enumerate through the biggies (ie, blank, "password", the username, etc.).
Having gone through the system, there really wasn't any malware involved. There wasn't too much in the way of spyware on the system, because the user (a) used a browser other than IE, and (b) pretty much only stayed to a small handful of specific sites.
I fully agree with many of the comments, but not necessarily the order. For example, what's the point of installing patches, and then wiping the drive?
If the system is a multi-user system (several users use the same system), then one way option for addressing this is to install VMWare. What you do is install the host operating system, then put XP or 2K on as a guest os. Now, I'm not sure of the steps required (or if there are any) to really lock things down so that the user is forced to use the VMWare session, but once you've got the guest os patched and configured, create a snapshot. Let the other users (ie, your kids) do their surfing, and if the configuration gets out of hand...revert to the snapshot (after making sure that any important data or documents are saved, of course).
Thanks for the input, folks...keep it coming!
Wednesday, February 16, 2005
Interesting Research at MS
While that approach is interesting, how about something like WIPFW, or perhaps just snort? If we're talking about protecting systems, why not install snort with some application specific rules during your build process, and then simply add the necessary rules when a vulnerability is discovered? That way, you wouldn't be installing anything new on a machine, simply adding a rule or two and restarting the process. Since things are already running fine, you won't have to worry about something new taking your system down.
The other advantage to using snort is that it's also useful for protecting home-grown apps created by in-house development teams.
Shields would be effective if the framework were such that the service could be installed, and the shields added to that framework as they become available, and without rebooting the system. Also, providing a scalable, enterprise management framework would be nice, too, so that you can track managed systems, what shields (and versions) have been installed on what machines, what shields are available for download, etc. MS has been pretty notorious for creating network operating systems (beginning with NT) but not providing a simple, viable enterprise management framework; case in point, the Event Log. 'nuff said!
The thing is, I can't seem to find a download location for anything other than the papers for this, and the same is true for the AskStrider tool mentioned in one of the papers from the previous blog entry. This seems to be pretty standard for MS...look at WOLF, mentioned in Robert Hensing's blog..."we've got a useful tool that's been developed, we have screencaptures of the tool in action, and we're going to talk about it and how useful and good it is...but we're not going to release it." Ugh!
Rootkit detection, the MS way
The technical paper lays out an interesting approach taken to detect rootkits. The authors (Wang, Vo, Roussev, Verbowski, and Johnson) take the fundamental approach that in order to be persistent, the rootkit files must exist on disk. So, what they do is insert their Ghostbuster CD into a system, and the CD runs a 'dir /s /a' command against the entire file system, saving the output of the command to a file on the hard drive. Then, the CD boots to a WinPE-based installation of Windows and walks the file system again. Once done, a clean version of Windiff.exe is used to compare the two files.
This is a very interesting development...not so much that it's a "new approach", because I don't think it is...more so because this is coming out of Microsoft. Take a look at the Strider page at Microsoft...I found the GateKeeper paper to be quite interesting.
While I think that there are a lot of positive things in the Ghostbuster paper, I also think that there are some caveats that need to be pointed out. Again, let me say that I think that the Ghostbuster approach is a good one, easy for just about anyone to use. After all, what could be easier than inserting a CD and walking away to get a cup of coffee, and than coming back a few minutes later? Yes, I know it's not entirely scalable, but still. However, the first thing I thought after reading the paper (and admittedly, when I went back to Bruce's blog entry, I found the same thoughts there, too) was that, as good as this approach is, it only works for those rootkits that try to hide files. Oddly enough, some of the most successfully hidden files do not require rootkits or any API hooking techniques at all...simply change the name of the file.
Here's an example...a while ago, I did an analysis of an IRCbot that I dubbed "russiantopz", because the bot seemed to have come from Russia, and the name was found in one of the scripts. When the bot installed itself, it ran two processes...the first was simply mIRC32.exe with some scripts, and the second was a program called hidewndw.exe, which is a simple little program that modified the settings for any window to make it invisible. Hidewndw.exe was launched immediately after mIRC32.exe and effectively hid it from view. Now, admittedly, both filenames had been changed...mIRC32.exe was 'statistics.exe', and hidewndw.exe became 'TeamScan32.exe'. So, while the window for the IRC client wasn't visible on the desktop, there was a process visible in the Task Manager called 'statistics.exe'. When I received initial notification of this bot, I was told that the administrator had checked Task Manager, and there was NO (an emphatic "no") evidence of anything suspicious.
Simply changing the filename is an extremely effective way of hiding something in plain site on a system. Forget hiding executable code in .png files (I didn't make this one up, someone actually said this on the comments to Bruce's blog entry)...that's not necessary. Too often it seems, Windows admins seem to be more interested in discussing all of the possible ways of hiding things, without really getting down to finding them. I mean, think about it...if you know something about computer technology (and something about Windows), you'll know that while there are hundreds (or thousands) of files on a Windows system, it's still a finite number. And of all the entries in the Registry, there are only a limited number of auto-start locations.
On a side note, ProDiscover from TechPathways is a forensics toolkit that includes a similar functionality in it's Incident Response version. Basically, it walks the file system using 'dir /s /a', and then walks the actual MFT.
One more thing...I took a pretty good look at the Ghostbuster paper and while I was impressed with the overall concept coming out of MS, I was taken aback by a couple of simple things. In a nutshell, I took a look at the rootkits they'd tested their methodology against. Two in particular jumped out...HackerDefender 1.0 and AFX Rootkit 2003. First off, as I've blogged about before, RKDetect already detects the presence of HackerDefender, without having to reboot the machine. In fact, some modifications (I'd use Perl myself) to the script would allow it to run against a set of machines, or an entire domain. I also presented a rootkit detection methodology in my book (along with a sample script) that can detect some rootkits, such as this. Further, I did an actual analysis of AFX Rootkit 2003 (haven't worked closely with the 2004 version yet) and found that it was pretty trivial to detect.
Tuesday, February 15, 2005
Thoughts on the use of WMI and WDM in a managed environment
So let's say that you've got several wireless laptops running Windows XP located in different locations of your building. You use WAPs (of course) with specific SSIDs and security settings. However, you don't always have the time to perform your own warwalks to look for rogue APs, or to see if companies in adjoining offices have set up insecure APs.
Using WMI, wouldn't it be possible to run queries on your managed XP systems and get the necessary information, rather than walking around? In fact, these systems wouldn't even have to be dedicated...you could run the query against any system with a wireless adaptor (and I know I'm not taking chipsets into consideration at this point). Here's the scenario...you know where certain systems are located, for the most part...if the CEO, CFO, or Corporate Counsel is in, you know where his/her office is. There are other folks who are in on a regular basis, and are located in other areas...CIO, CTO, IT Manager, folks in finance, etc. You may even be able to throw the receptionist into the mix, depending upon your infrastructure.
So you run the queries against these systems...being known systems, you know the system names, as well as the names of the wireless interface(s) in the system. So, your script can connect to each one, enumerate all SSIDs, as well as information about each (signal strength, etc.) one. This way, you should be able to detect new APs, and potentially even triangulate their location.
This would also work equally well if the managed systems were in a different city...
Thoughts? Am I out of my freakin' mind here or what?
Incident Scenario #1
A friend brings you a Windows2000 laptop that no longer boots normally. You insert a Windows2000 distribution CD, and run a repair installation. When you're finally able to access to the system, you call your friend and ask for a logon...you opt not to mess around with the ntpasswd utility on this system.
The first red flag you have is that your friend's username and password are the same. Hhhmmm...Very Bad. Before hanging up, you ask your friend how he accesses the Internet, and he admits to still using dial-up (major procrastinator). He does offer up, however, that while most of the time he'll disconnect his system while he's not actually working (ie, surfing), many times he'll take a break...sometimes for half an hour or more...before coming back to the system. Hanging up and shaking your head, you log into the system. You don't see anything unusual on the desktop, so you open the User Manager. The first thing you notice is that besides the Administrator account and your friend's account, there are several others (not just the Guest account)...one called 'god', one called 'gawd', and another called 'godd'. Oddly enough, all visible accounts (with the exception of the Guest account) are part of the Administrators group.
Opening the Service Control Manager, you don't see the Internet Information Server (IIS) installed. From everything you can see, this system is a normal Windows2000 distribution, installed directly from CD. The system does seem to be up-to-date with regards to Service Packs and hotfixes, but you notice that there is no personal firewall installed, nor is there an anti-virus package installed.
At this point, consider these questions:
1. How might this system have been compromised?
2. What is a likely scenario that would explain how these accounts got on the system? How would you go about proving this?
3. What are some other things you might look for? What tools would you use?
4. How would you go about securing the system?
If you have questions and comments, please feel free to contact me. Responses to this scenario/questions should be placed in the Comments...
Incidents Question, part 2
Incidents Question, part 1
Thursday, February 10, 2005
From the world of, "Is this really such a good idea?"
So I'm over on O'ReillyNet.com today, and ran across the NTFS Performance Hacks on the Windows Dev Center. Tip #8, Disable Last Access Time, freaked me out a little bit. Well...okay...a lot.
You're probably asking yourself, "what's the big deal?" Well, if you want to increase the performance of your box, sure, make the recommended change. Will home users see a significant performance increase? Not likely. How about most users? Depending on their activities...again, not likely.
So let me ask you this...let's say that you have some data worth protecting. Now, would you rather get to it faster, or would you be okay with giving up a small amount of access time (network latency would probably consume more time), or would you like to have some evidence if something happened to that data?
I know what most of you are going to say...and you're right...there're other things you could do, and cases rarely hinge on a single piece of evidence. In most cases, there are multiple, supporting pieces of evidence. However, would you be willing to give that up? Particularly when you're looking at a live incident similar to what Robert Hensing discussed...how many times do you see "last_access_time", for directories or files, in that blog entry?
So...is that really such a good idea?
Wednesday, February 09, 2005
I did some looking around, specifically at the classes WMI uses to interface with Windows Driver Model (WDM) and found the MSNdis_CurrentPacketFilter class. Nice! The class provides three properties...Active (which is a boolean...basically, "yes" or "no"), the InstanceName of the class (a string), and NdisCurrentPacketFilter (a 32-bit integer). So, you can get the name of the interface, and it's current packet filter settings.
To see these classes on your Windows system (I'm on XP), go to the 'Run' box in the Start menu and type "wbemtest". When the WMI Tester window appears, click on "Connect", and type "root\wmi" into the first box, and click on "Connect". This takes you back to the WMI Tester window...from there, click on "Enum Classes". Click the "Recursive" radio button, and then "OK". Scroll down in the Query Result dialog window until you find "MSNdis_CurrentPacketFilter". Double click this to see the properties of the class.
So, what do you do with this? Well, to see what's encoded in the OID_GEN_CURRENT_PACKET_FILTER, go here or here. We know that if the value for NDIS_PACKET_TYPE_PROMISCUOUS is set, that the NIC is in promiscuous mode. Since this value is 0x0020, we know that basically, the 6th bit of the 32-bit packet filter is set. Neat, eh?
So, using this information, we can write a nice little Perl script that implements WMI, and retrieves the current packet filter for all active interfaces. I did so (I added a little code to report only unique interface names, and skip over the "WAN Miniport" blah blah blah), and simply had the script check to see if the returned current packet filter was greater than 36. I did some initial testing with this and Ethereal, and it seemed to work just fine.
So what does all this mean? Well, in the big scheme of things...not a whole lot. Yeah, I like to see how things work and drill down as far as I'm able. And yes, there are other tools that will let you do this, such as promsicdetect.exe and promqry.exe...but sometimes it's just fun to see what you can do.
Want to see some really cool examples of the use of WMI and WDM? Check out Beetle's (of Shmoo fame) presentation materials on Wireless Weapons of Mass Destruction for Windows. He's got some VBScripts included in the package that, at some point, I'd like to convert to Perl and play around with.
Tuesday, February 08, 2005
Tools of the Trade
So, when you're doing incident response, some of the first information you're looking for is volatile information, such as running processes, network connections, etc. So, here are some tools:
- Tlist.exe (part of the debugging tools) - run it successively with the '-c', '-t', and '-s' switches
- cmdline.exe from DiamondCS - getting just the name of a process is next to useless...what you really want to get is the path to the executable image and the command line used to launch it
- Listdlls.exe from SysInternals - get the modules (ie, DLLs) loaded by each process...may show you some DLLs that have hooked other processes
- handle.exe from SysInternals - get all of the handles accessed by a process; may indicate suspicious activity, or at least show you which process has a file open
- If you're running XP SP2, use 'netstat -bnao'; if you're using the FRU within an environment that's not homogenous with regards to XP SP2, you can use Perl to write a wrapper around it to do some checking for you.
- PortQry v2.0 from Microsoft - while this tool can be used remotely, it can also be used locally (ie, 'portqry -l -v')
- Openports.exe from DiamondCS - use 'openports -fport' to get output in fport-style format; doesn't require an admin account to run, the way fport.exe does
- Tcpvcon.exe from SysInternals - use 'tcpvcon -a -n -c' (see the SysInternals page)
- PSFile.exe from the SysInternals PSToolkit - lists all files on the local system that are open as a result of remote connections
- Psloggedon.exe from SysInternals -see who's logged on locally and remotely (via Windows logon)
- LogonSessions.exe from SysInternals - lists currently active logon sessions, and with the '-p' switch, processes used by each session.
- NetUser from SomarSoft.com - utility to show all users who have ever logged on to the local system
- Sigcheck.exe from SysInternals - determines if images are signed, and dumps version information (ie, 'sigcheck -i -q -u -v c:\windows\system32')
- Psloglist.exe from SysInternals - dump the contents of the Event Log (ie, 'psloglist -s -t ; appsyssec')
- PSInfo.exe from SysInternals - get system information (ie, 'psinfo -h -s -d -c')
- GPlist.exe from NTSecurity.nu - get info on the applied Group Policies
- Promiscdetect.exe from NTSecurity.nu - see if a NIC is in promiscuous mode (also, promqry.exe from MS)
- PStoreView.exe from NTSecurity.nu - get information from the Protected Storage service, such as passwords, etc. - this may develop leads more than it will evidence
- Autorunsc.exe from SysInternals - dump the contents of autostart locations within the Registry and file system (ie, 'autorunsc.exe -a -c -d -e -m -s -w')
- Rifuiti.exe from FoundStone.com - dump the contents of the Recycle Bin; this utility will require a wrapper to get the SID, then pass the correct path information to the tool.
So, how's that for a start?
What tools would you recommend?
Forensic Server Project
The FSP is a client-server based approach to information collection during incident response. I'm specifying "information collection", b/c I've explicitly separated the collection and analysis phases. The FSP, and the client component First Responder Utility (FRU), work together to perform collection. In a nutshell, the admin/first responder downloads the utilities (both are standalone .EXEs, with their Perl source code included), and places the FSP server component on a system of her choosing. She then configures the fruc.ini file (per the documentation) and copies fruc.ini, p2x584.dll, and fruc.exe (along with all of the necessary supporting tools) to a CD or USB thumb drive. The fruc.ini also allows the admin/first responder to specify Registry keys and values to query.
When responding to an incident, the first responder place the media containing the FRU in the system, and launches the FRU with the necessary command line switches (if applicable). The FRU will automatically connect to the FSP server, and place all information it collects on that system. The server component of the FSP handles storage and checksumming of the data that its sent, as well as logging of all activity.
Again, this is simply the data collection phase...analysis is another thing all together. I provided some useful Perl scripts with my book for performing data correlation, and I would like to expand the analysis suite. However, nothing prevents the user from doing the same...using any programming language(s) (ie, Perl, VBScript, Python, Ruby, etc.), and any other utilities (mySql, Excel, etc), you can perform data reduction, analysis, and presentation.
Now, my request to the readers...actually, this is a couple of requests:
1. Are you using the FRU/FSP? If so, what are your comments? What's good or bad about it, how could it be improved, etc.?
2. If you are using the FRU, what does your fruc.ini file look like? Got a link to it someplace where you'd care to share? If you don't feel like sharing it in this forum, is there another forum that you'd be more comfortable with?
3. What kind of analysis are you doing? Do you have any analysis tools? What specific things are you looking for, or at?
One final thought...Robert states in his blog that the WOLF tool is not available publicly. I can definitely understand the reasons why. If you're interested in a tool like that, that will write it's data to a drive (mapped, USB-connected, etc.), let me know. I've been toying with the idea of writing a version of the FRU that does that.
Microsoft Sniffer Detector
According to the KB article, promqry uses WMI to query systems for the necessary information. The article doesn't state which class and property(ies) are used, and I haven't seen anything in the Win32_NetworkAdapterConfiguration class that could be used.
I wrote a similar tool for detecting sniffers, but it worked by looking for the WinPcap driver on a system. This is a freely available driver that's used by tools like Ethereal, L0phtcrack, and other sniffers on Windows systems. This detects the presence of the driver, and determines whether its in use or not...thereby detecting systems that *could be used* for sniffing.
Looks like there's another good tool to add to your arsenal, either for system compliance monitoring or incident response. In fact, it looks as if you can download the command line version of the tool and incorporate that into the FSP, adding it to your fruc.ini file, along with other tools, such as portqry v2.0, etc.
So, give a big shout-out to both Robert and Tim...thanks, guys!
Monday, February 07, 2005
Blog topics follow-up
First off, there really weren't many responses at all...less than half a dozen. Of those, the predominance seemed to be along the lines of "talk about how you've used tools to investigate incidents". Hhhmmm...that's all well and good, and I'll try to do more of that sort of thing, either using real incidents or by using more contrived examples.
Two of the emails I received asked specifically about open-source tools. This was interesting, as I've been considering talking more about the Forensic Server Project and it's uses. I'm going to be making posts on the FSP and accompanying First Responder Utility (or "FRU") in the future. I'm also working on a standalone version of the FRU that incorporates much of the functionality of the FSP itself, but writes to an arbitrary storage location, rather than a server.
As I've stated before, though, data collection is easy...it's the analysis that's the key to an investigation. As such, I am leaning more towards posting collected data for the reader to examine, and providing my analysis at a later date, rather than just posting a "here's what happened". I did this in my book, providing the output of the FSP for the reader to comb through.
So, keep your eyes peeled for upcoming posts, and as always, comments and emails are welcome.
Tuesday, February 01, 2005
Post a comment, or drop me an email at "keydet89 at yahoo dot com".