Thoughts on Tool Features, pt II

My previous post on this topic addressed an apparent dichotomy (admittedly, based on a limited aperture) of thought between vendors and users when it comes to features being added to commercial forensic suites. This was the result of a road I'd started down a while back, trying to see if there was any value proposition to RegRipper at all (turns out, no, there isn't), and was simply the most recent pit stop along that road.

From my previous post, there seem to be two thoughts on the matter, or maybe it's more correct to say that only two basic positions or perspectives were shared. One is from vendors, some who rely on users to express the need for features; as such, vendors are relying on users to drive their own investigations, requesting or expressing the need for features as they arise. I'm sure vendors have ways of prioritizing those requests/needs, based on limited/available resources.

The other perspective is from users of those forensic tools; the views expressed are that if a vendor finds or 'sees' a cool feature, it should simply be added to the tool or framework, regardless of whether anyone actually wants it or not. To me, this seems to be users relying on vendors to drive investigations.

While I tend to agree more with the vendor perspective, as a user of forensic tools (albeit not commercial products), it seems that I have a different perspective from most users. There have been a few times in my career where I've had to deal with the issue of tool features; some examples follow:

CCN Searches
Circa 2009-ish (give or take), Chris Pogue and I were working a PCI forensic investigation, and found that while Discover and JCB cards had reportedly been processed by the merchant, none were appearing in our searches. As our team had recently grown, we had settled on EnCase (then owned by Guidance Software) as the tool used for all of the PCI-specific searches (CCNs, hashes, file names, etc.); this tool was commonly understood, and we wanted accuracy and consistency above all else.

We began digging into the issue, even going to the brands and getting test data. We kept reducing the scope of our testing, even to the point of, "here's a file with 3 Discover CCNs in it and nothing else...find all Discover CCNs", and each time, got no hits on either Discover or JCB card numbers. We determined that the IsValidCreditCard() built-in function, which was a closed-source "black box" for us, did not consider either brand of card number valid. We needed this capability now, and we were getting nowhere with the vendor, so we reached out to someone who was known for their EnScripting ability and asked for help. We ultimately ended up with a new function, one that included 7 Regexs, that we used to overload the built-in function. Borrowing a trick I learned from one my professors during my undergrad studies, Chris and I wrote up a quick email to everyone, stating, "copy this file to this location within the EnCase installation, etc.", and got everyone up and running at a consistent level pretty quickly. Yes, the process for searching for CCNs was a tad slower, but it was more accurate now, it was consistent across the team, and Chris and I could easily help troubleshoot any issues folks had in running the new function. After all, the only thing that could really go wrong at that point was that the file we sent was copied to the wrong location.

This all boiled down to the fact that we recognized that the tool we were using did not have the functionality we needed, even though, across the industry, everyone assumed that it did. We knew what we needed, we knew we needed it immediately, and we knew that we needed to ensure accuracy and consistency across the team. To do so, we sought help where we felt we needed it, and were more than willing to accept, "yeah, okay, we missed that..." along the way, in order to get to our goal. 

Carving Records
In 2012, I attended the DC3 conference in Atlanta, and after spending several days there and returning home, I ran into a fellow attendee at the baggage claim for our flight. We knew of each other within the community, and I had no idea we'd been on the same flight. As we were waiting, we got to chatting, and they mentioned that they'd been "looking at" a system for about 3 months, and were stuck. I wanted to know more, and they said they were stuck trying to figure out how to recover cleared Event Logs. As soon as they said that, I said that I'd send them a tool I'd written as soon as I got home...which I did. It was late when I got home, so I kissed my wife, turned my computer on, and sent them the tool along with instructions, as promised.

In this case, someone working for the government had a situation, an investigation, where they were stalled "looking at" the image...for 3 months. Now, I have no doubt that that didn't amount to 8 hrs a day, 5 days a week, etc., but that it was more on-and-off over the span of 3 months. However, during that entire time, I had a tool that I'd developed that would have been perfect for the situation, one that I'd not only developed but used to great effect. In fact, the tool was originally written because an Event Log on an XP system had 2 more records within the logical .evt file than were reported by the API, which I'd been able to recover by basically writing a carver. As such, what would work on a logical .evt file would work equally well on unallocated space extracted from an image via blkls.

RegRipper
Something I really like about RegRipper that wasn't something I specifically thought of during the original design is how easily it can be updated (the same applies to Events Ripper). For example, not long ago, I was looking into nssm, the "non-sucking service manager", and ran across the usage page. This page describes a number of Registry values and keys that are used by nssm, as well as by tools like it, such as MS's svrany.exe (which nssm replaces). Both provide persistence and privilege escalation (Admin to LOCALSYSTEM) capabilities, and I knew that I'd never remember to specifically look for some of these keys and values...so I wrote a RegRipperPro plugin to do it for me; the output of a test is illustrated in the figure below.

Fig: Appenvironment.pl Plugin Output

So, as you're reading this, you're probably thinking to yourself, "...but I can't write my own plugin." I get it. Some folks don't bother thinking, "I can't...", and just do it, but I get it..Perl isn't in common usage any longer, and programming simply intimidates some folks. That's cool...it's fine. 

Because all you need to do is ask for help, and maybe provide some sample data to test against. I've consistently turned working plugins around in about an hour, when folks have asked (which hasn't been very often).

The plugin I wrote (output in the figure above) took me just a couple of hours in the evening...I had to stop working to make dinner, take the dog out, etc., but it was pretty straightforward to complete. Now, I don't have to specifically remember these values (and the included key) when conducting an investigation; it's trivial to run all available plugins against various hives, so even a year from now, if I never revisit the issue that led to the plugin again, I'll still have the "corporate knowledge" and the query will still be part of my process.

So, I have something of a different perspective regarding tool features and who drives an investigation. My perspective has always been that as the analyst/investigator, it was my job to drive the investigation. Now, that doesn't prevent me from asking for assistance, or seeking someone else's perspective, but ultimately, the tools I use and the functionality I need are up to me, as an investigator, and I'm not going to structure my investigation around the functionality provided by a tool (or not, as the case may be).

Thoughts on Tool Features

Not long ago, some exchanges and conversations led me to do something I'd never done before...post a poll on LinkedIn. These conversations had to do with whether or not analysts and practitioners within the industry felt there was adequate value proposition to incorporate RegRipper in to a commercial forensic suite. No conditions, no specific goals or outcomes, just "is this something that makes sense?" As you can see from the poll, the responses (the few that there are) are pretty much 4:1 in favor of the idea.

I posted the poll because when I asked a vendor for their thoughts, the response was, "...none of our users have asked for it." Another vendor responded with:

...we need to focus on two things - what customers tell us they want (preferably things that are both powerful and unique), and what helps us in significant ways in our own casework.

There have been times we go pretty far out on a limb in terms of functionality we think people will want, and no one gives a shit. 

From a user perspective, some of the feedback from the poll, as well as from other conversations and exchanges, indicates that some users feel that vendors should take charge of providing "needed" functionality without being asked. 

This really seems like two diametrically opposed views on the subject, with the vendor side saying, "we rely on our users to tell us their needs", and the users saying, "we rely on our vendors to guide our investigations."

In 2020, I presented at OSDFCon on effectively using RegRipper. On pg 3 of the linked PDF, in the second slide, there are several thoughts I had regarding possible useful updates to RegRipper, including adding MITRE ATT&CK mapping, Analysis Tips, and reference URLs to the plugin output. I did not receive any feedback to this presentation, either during or following the presentation itself. No, "hey, this is a great idea!", and no, "OMG, this is the dumbest thing I've ever heard." However, I felt that these were things that I would find useful in the tool, and since other analysts didn't seem to be interested, I set about creating my own internal repo of RegRipper v4.0, or "Pro". This has become the tool I use, update, and continue to develop. In fact, earlier this year, I started in the first steps of creating plugins with JSON output, starting with the appcompatcache.pl plugin.

Back around 2007, I started developing what became RegRipper because none of the tools I had access to at the time, either commercial or free, provided the capability I needed for the work I was doing and the process I was developing. I opted to create a tool in the model of Nessus (i.e., plugins) so that I could easily update the tool without completely rewriting it. I developed the tool so that I could run individual plugins, or "profiles", which are designated groups of plugins that can be run in support of a playbook. I later added the ability to run all available plugins as it seemed to be how most folks were wanting to use the tool anyway.

The idea of "profiles", while I thought it would be an incredible capability, never caught on. You could run "profiles", or analysis playbooks based on file access, USB device usage, etc. I know that there are commercial tools out there that have these capabilities, but what RegRipper provided me was the ability not only to pick and choose, but to also update the tool with new plugins, new capabilities and functionality, with minimal turn-around time. I've had a few instances over the years where folks have reached out and asked for assistance, provided test data, and I've been able to turn around a working plugin in under an hour.

This is what I wanted for the community...for folks using the tool to find something they hadn't seen before, something new, and add to the tool. Unfortunately, that really never caught on, and to some extent, now I know why.

The question becomes, where do you stand? Do you think that vendors providing commercial forensic suites should drive investigations based on the features they provide, or should analysts and investigators be the ones who drive their investigations, and look for the right tool, or correct too usage?

The Next Step: VHD Files and Metadata

Keeping with the theme from my previous blog post of building on what others have done and written about, and of assembling the pieces that are already available to build on a foundation built by others, I found something interesting that seems like a great "next step".

Anyone who's followed this blog for any length of time knows that I'm a huge fan of metadata, and that anytime a threat actor sends you metadata from their endpoint or environment, it's basically free money. For example, my posts on LNK metadata extraction and what can be derived from analysis of this data go back quite a ways. Another great example of the use of LNK metadata to further investigations comes from Mandiant, in this Cozy Bear blog post from Nov, 2018 (see fig. 5 and 6).

Tony Lambert recently shared a blog post where he dug deep into getting intel from VHD file metadata, and then digging in even further to get intel from metadata within the files found within the VHD file, in this case, an LNK file. In his blog post, Troy used EXIFTOOL to extract metadata from both the VHD file and the LNK file.

I used my own toolset and as with EXIFTOOL, found that the LNK file had no available "machine ID" or NetBIOS system name for the host on which it was built; this is one of the elements of an LNK file, based on the documented structure format. However, this may not be unusual or unexpected, because the time stamps embedded in the shell item ID list elements were zeroed out, as illustrated in figure 1.

Fig. 1: Shell Item ID List Time Stamps

If you took a close look at the Mandiant blog post mentioned above, you'd see that the time stamps from the shell item ID list elements were what the analysts used to develop intrustion/threat intel, based on how the time stamps changed between the 2016 and 2018 campaigns. Zero'ing these values out is not difficult (I've done it before), and may actually be scripted, or simply added to the LNK builder.

While the lack of time stamps doesn't give us anything to pivot on or track, there is something else embedded within the available PropertyStoreDataBlock within the LNK file; specifically, a SID, as illustrated in figure 2.

Fig. 2: SID embedded in LNK 

Using this SID, a Yara rule can be used to perform a VirusTotal retro-hunt for similar LNK files, or to scan across local malware repositories. For example, from this LNK file, we can see that it was built on a system using a built-in Administrator account (RID: 500). 

From an intrusion intel perspective, it's pretty clear that if a threat actor sends files to a target, they're very likely sharing a wealth of information about their development environment. When it comes to LNK files, even those embedded in disk image (ISO, VHD) files, a great deal of information about the development environment may be shared (as described by the folks at JPCERT in 2016).

Issues With VHD Files

Two of the issues with image files (ISO, IMG, VHD[X]) is that users can automatically mount them via a double-click, and that ISO files in particular did not not propagate mark-of-the-web (MotW), a security "setting" available on files downloaded from the Internet. The MotW issue with ISO files in particular was addressed in a Nov 2022 security update. If users have no legitimate business reason to automatically mount disk image files via double-clicking them, Huntress has a blog post that addresses disabling this capability, while still permitting programmatic access to the files, such as mounting a VHD or VHDX file via the Disk Manager. Disabling the ability for users to double-click and automatically mount these files as accessible file systems has been available for some time but the Huntress blog makes it a simple matter of copying and pasting the PowerShell code to change the setting.

Shoutz to Tony, not only for his work, but for sharing it with us all! We're all the better for it when we take the time to write about our research or experiences, sharing them with others.

Additional Resources
VHDX metadata table entry 

The Next Step

A lot of times, we'll run across something or read something really very profound and valuable, something that opens our eyes and makes us go, "oh, wow", and impacts us enough that it changes the way we do things. I can say that of a number of blogs posts and articles, by various authors, going back quite a few years. And then later, being able to add additional information to the original findings, information that may not have been recognized or available at the time, can really aid in our investigations.

A while back, Krz shared this blog post, detailing how default settings for Scheduled Tasks include the "stop if the computer switches to battery power" setting by default, and the impact that setting can have on a forensic investigation. For example, PCI forensic investigations require the analyst to specifically address the "window of compromise", and malware that persists via a Scheduled Task will be impacted by whether or not the system in question was running on battery power or not. Krz's previous blog post addressed using the SRUM database to determine the battery charge level, and in that post, Krz linked to a tool he'd written to extract and display that data.

I ran across something recently that I wanted to use to build on Krz's excellent work; from the Microsoft-Windows-UniversalTelemetryClient%4Operational.evtx Event Log, we see a UniversalTelemetryClient/60 event record that lets us know if the system was on battery power or not, as illustrated in figure 1:

Fig 1: UniversalTelemetryClient/60 event, on batter power 

In this particular case, I took my laptop from my office to another room, so I could attach it to the TV via an HDMI cable and watch our church service remotely. When service was complete, and I reconnected the laptop to the power cord in my office, I saw the UniversalTelemetryClient/60 record illustrated in figure 2.

Fig. 2: UniversalTelemetryClient/60 event, off batter power

From the same Windows Event Log file, UniversalTelemetryClient/55 records will let us know if the system had access to the Internet or not, further aiding us in our investigation. This information (system is on or off battery, on or off the Internet) can be extremely valuable during an investigation, particularly when incorporated into a timeline. If a system has malware that persists via a Scheduled Task using default settings, and the system if on battery power when the task was scheduled to run, then it will not run. It may also be helpful to understand when the system did and did not have an Internet connection.

Events Ripper Update

Something I really, really like about tools like RegRipper and Events Ripper is that when I see something in the data during an investigation, I can explore whether it makes sense to pull that out and make it visible to the investigator, and if so, figure out how it makes sense to do so. Because of how these tools are set up, the turn around for something new is pretty quick, and the retention (and sharing) of corporate knowledge is pretty freakin' cool!

This time, I updated the logins.pl plugin, so that it collects and displays some additional data points beyond the 'current' version; specifically, usernames and SIDs for type 3 and type 10 logins, as well as source system names (if not "-" in the event record), correlated to source IP address, for type 3 and type 10 logins.

During a recent incident we were digging into a bit further, we were seeing some odd source system names, ones we'd seen previously, on other incidents for other customers. Now, this isn't stating definitively that it's the same source system, of course, but it is an interesting data point worth tracking...so why not extract that information and make it easy for an analyst to see. We were also seeing repeated type 3 logins for a source system named "nmap", and I wanted to see the logins and source IP addresses that may have had the same source system name. Interestingly, we also saw some additional anomalies, but the point is that this information is easy to see now.

With the usernames, in one instance, we were seeing repeated type 3 logins for the "Guest" account; including the SID alongside the username for the type 3 (and type 10) logins provides insight into possibly compromised credentials. In the case we were looking at, the RID for the account was 501, which tells us that someone had enabled the Guest account, something that serves as an interesting data point for the incident.

I'd love to provide some sample output but most of the CTF data I have access to doesn't include a great deal of terribly interesting data points. 

Hiding In The Windows Event Log

In May 2022, Kaspersky published a write-up on a newly-discovered campaign where malware authors wrote shellcode to the Windows Event Log. This was pretty interesting, and just about 4 months later, Tim Fowler published this blog post over at BlackHillsInfoSec, digging into this a bit deeper and offering several variations of the technique up to red teamers.

Now, I found this technique interesting, not because it's not really something I'd seen before, but because of how Windows Event Logs, and just "Event Logs" prior to Vista, have been used by DFIR analysts. Back in the days of WinXP and Windows 2000/2003, there were The Big Three...Security, System, and Application Event Logs. With the advent of Vista, and then Windows 7, the numbers of Windows Event Logs available to analysts exploded; on my Windows 10 system, a 'dir' of the winevt\logs folder reveals 400 files with the ".evtx" extension. However, not all logs are populated, or even enabled. 

However, this doesn't mean that these logs are used during analysis; in fact, much like the Registry, the Windows Event Logs are largely misunderstood by a great many analysts, to the point where I've seen log collection processes that are still restricted to just the Security, System, and Application Event Logs. Further, there seems to be a great deal of Windows forensic analysis training that persists in identifying Windows Event Log records solely by their event ID, even when it's been stated and shown that event IDs are not unique. For example, we often refer to "event ID 4624" when identifying successful login events, but when the event source is "EventSystem", that event ID has an entirely different meaning and significance. And there's nothing the prevents someone from creating an application that writes it's logs to a current or it's own Windows Event Log, using the same event ID. In just the past year, I've seen several tools used by threat actors that create Windows Event Log records, two of which use event ID 0 (zero) for everything, literally every record written, regardless of the message, is event ID 0.

In short, using a Windows Event Log file as a persistent repository is a great idea because responders and analysts aren't likely to look there, nor consider it as a source. I found the use of the "Key Management Service" Event Log pretty interesting, because while it's enabled on the systems I have access to, it's not populated on any of them. 

So, I went ahead and tried a variation of one of Tim's commands, as illustrated in figure 1.

Fig. 1 - Powershell command 

The resulting WEVT record can be seen in figure 2.

Fig. 2 - Resulting Windows Event Log record

This is the first record written to that WEVT file on this system, and as you'd expect, the file last modification time reflects that. This illustrates why this particular Windows Event Log file serves as a pretty decent persistent repository. You could change the log file used, but you'd have to find one that either is extremely low volume, or enable one that is similarly low volume. A Windows Event Log that regularly has records written to it does not serve as a suitable persistence mechanism, unless you're able to increase the size of the file,

Tim goes on in his article to extend the technique beyond what Kaspersky discovered, and what this really demonstrates is that there's a great deal that can be done with a few simple (native) tools, some knowledge, and some imagination. And, what makes it "interesting" is that it relies on a data source not often leveraged or exploited by analysts.

Tools like Chainsaw and Events Ripper would not be effective for detecting the use of this technique, particularly if the Windows Event Log used for this technique was not included in the collection process. An Events Ripper plugin that listed all source/ID pairs and their frequency might provide a pivot point for the analyst, but a timeline of system activity would certainly show any suspicious records, again, as long as the impacted WEVTX log is included in the collection process.

This StackOverflow question resulted in several ways to create Windows Event Log records using native methods such as eventcreate.exe, Powershell, etc.  Note that using eventcreate.exe is restricted to just the Application Event Log, but the availability of anything written to this (or another) Windows Event Log can be adjusted by the file size and retention settings.