My recent post regarding LNK files got me thinking about other uses of LNK files. That previous post really illustrated how some analysts are following Jesse Kornblum's adage of "using every part of the buffalo", in that they made use of everything (or as close to it as they could) they had available in order to develop a #threatintel picture.
This got me to thinking...taking a step back, how else can LNK files be used?
DFIR Analysis
Some DFIR analysts are aware of the fact that when analyzing Windows systems, you're going to find Windows shortcut/LNK files in a number of locations. For example, there're the user's desktop, the user's Recent folder, etc. In addition, Automatic JumpList files are OLE structured storage format files, and all but one of the streams follow the LNK file format. So, the file format is widely used on Windows systems, and the location of the file or stream provides some useful context that can also be applied to the content.
LNK files contain a good bit of metadata, as they contain shell items (as do other artifacts, such as shellbags), which are blobs of binary data that describe various objects on Windows systems. In the case of folder objects, specifically, one of the elements found to be embedded within the metadata is the MFT reference number for that folder object. This reference number is comprised of the record number (i.e., location within the MFT), as well as the sequence number (in short, "...this is the nth time this record has been used.")
Okay...so what?
Well, when it comes to artifacts of use, or more specifically, file access, the LNK files created as a result of user activity can remain on the system long after the files and applications with which they're associated have been removed or deleted. Say that you're investigating access to a particular file (by name or folder path) and your goal is to illustrate that the user had knowledge of that file. If the user accessed the file by double-clicking on it, an LNK file will have been created, and that file will persist well beyond the deletion of the target file itself. These artifacts will also persist beyond the removal of the associated application, as well.
This can be useful to us because it gives us something of a historical view of the file system. For example, let's say that the per the MFT, record number 2357, with sequence number 5, points to a folder on the user's desktop called "Personal Stuff". Now, suppose we find a LNK file that points to the fact that the user opened a file that was located in a folder named "Hacker Tools", and the MFT reference number extracted from the LNK file is 2357/4, or 2357/3. This provides us with a view of what the file system used to look like, giving us something a historical "smear" of the file system.
Also, these LNK files can provide information regarding external devices, as well. The basic shell items are created in the user's shellbags when they use Windows Explorer to navigate folders on an external device, such as a USB thumb drive, USB-connected smartphone, etc. Then if they open files, shell items are created to populate LNK files pointing to those files.
Adversary Persistence
Adversaries have been observed persisting beyond password updates by modifying the iconfile name attribute of an LNK file to point to a resource that they (the adversary) control. What happens if the LNK file is at the root of a directory is that when a user/admin browsers to the folder, Windows parses the file and attempts to load the icon from the remote resource, using the user's credentials to authenticate, first via SMB, and then via WebDAV.
Other Ways To Use LNK Files
"Using" LNK files can apply to the adversary, as well as to an analyst (DFIR, intel, etc.). There's this write-up on Kovter, describing how the adversary uses/used LNK files, and there's this BitOfHex blog post describing how to derive intel from LNK files. There's also hexacorn's older blog post regarding the use of hotkeys and LNK files, and this USCERT alert that describes the use of the adversary persistence technique described above (not 'new' or an 'other' use, but placed here as an illustration).
Saturday, November 02, 2019
Monday, October 28, 2019
Return of the LNK Files...
I wanted to put something scary together in time for Halloween; I was gonna go with a mullet wig, a la Joe Dirt, or maybe pass out cards with truly scary cards for those of us who are adulting, full time, such as, "...your septic field just rose and flowed down the yard into your porch...", or "...your teenager just go their license and want to drive home...". You know, truly scary stuff. However, from a #DFIR perspective (and maybe even a little bit of #threatintel) this just seemed a bit more fun and appropriate.
A recent Tweet thread from Nick Carr regarding the use of LNK files in developing threat intel caught my attention. In that thread, Nick mentions learning "about LNK analysis as a DFIR and threat intel tool", and that's something I wholeheartedly agree with. A great deal of value can be derived from LNK files...I've described them as "free money" in the past, due to the fact that an adversary sending an LNK file to a target is essentially giving away information about their infrastructure, particularly when data from LNK files is mapped across multiple campaigns.
Links from Nick's Tweet:
2017 - Fin7
2018 - Cozy Bear
Remember what I said about "multiple campaigns"? If you take a look at the second blog post linked above, you'll see the statement:
The 2018 and 2016 LNK files are similar in structure and code, and contain significant metadata overlap, including the MAC address of the system on which the LNK was created.
I'd had an opportunity to dig into the Cozy Bear LNK files a bit myself, as illustrated here, and described here (i.e., in an additional post regarding LNK file toolmarks).
Not long after Nick's tweet, I saw this one from ZwSetInformation, and was able to get a copy of the zipped archive, extract the LNK file and run it through my parser. Not only did I see the individual bits of information exposed by the parser, but looking across the entirety of the information showed me the toolmarks and provided an indication as to how the LNK file had been crafted.
A recent Tweet thread from Nick Carr regarding the use of LNK files in developing threat intel caught my attention. In that thread, Nick mentions learning "about LNK analysis as a DFIR and threat intel tool", and that's something I wholeheartedly agree with. A great deal of value can be derived from LNK files...I've described them as "free money" in the past, due to the fact that an adversary sending an LNK file to a target is essentially giving away information about their infrastructure, particularly when data from LNK files is mapped across multiple campaigns.
Links from Nick's Tweet:
2017 - Fin7
2018 - Cozy Bear
Remember what I said about "multiple campaigns"? If you take a look at the second blog post linked above, you'll see the statement:
The 2018 and 2016 LNK files are similar in structure and code, and contain significant metadata overlap, including the MAC address of the system on which the LNK was created.
I'd had an opportunity to dig into the Cozy Bear LNK files a bit myself, as illustrated here, and described here (i.e., in an additional post regarding LNK file toolmarks).
Not long after Nick's tweet, I saw this one from ZwSetInformation, and was able to get a copy of the zipped archive, extract the LNK file and run it through my parser. Not only did I see the individual bits of information exposed by the parser, but looking across the entirety of the information showed me the toolmarks and provided an indication as to how the LNK file had been crafted.
Monday, October 21, 2019
Registry Analysis
Something I've observed over the years is that analysis of the Windows Registry is still a largely misunderstood, misinterpreted, and under-appreciated aspect of analysis of Windows systems.
What is "Registry analysis"? Registry analysis is the observation and interpretation of data or metadata from the Windows Registry, in the context of other data/metadata, also from the Registry or other sources. The correct interpretation of this data can add an unprecedented level of granularity to the context surrounding various events observed during, for example, timeline analysis.
"Registry analysis" is not the parsing and display of individual artifacts from within the Registry; that's "parsing and display". It's also not keyword searching of the Registry; that's "keyword searching". Keyword searches or searches for indicators do not constitute "analysis". I do understand, however, that not all cases require Registry analysis. There are more than a few types of cases out there where keyword searching is all that is necessary, and I get it. Through engaging with other analysts and following up on conversations, I've seen where keyword searches have more than sufficed for a number of types of cases. But that's not the "analysis" we're talking about here.
That is not to say that keyword or indicator searches cannot be used as pivot points into more fully-fledged analysis. Very often, these searches can be an excellent entry point into analysis, allowing the investigator to winnow through vast amounts of data to determine what is initially important.
Analysis techniques where I have been able to incorporate data and metadata from the Windows Registry into an overall analysis process, have allowed me to get a better understanding of inherent activity derived or extracted from a system. This has occurred to the point, in some cases, of changing the direction of my investigation. Further, I'll be the first to admit that I haven't seen everything there is to see; in fact, within just the past weeks, I've observed activity that I had never seen before, and was able to develop an understanding of what activity led to the observed artifacts. This is not something for which keyword searching would have sufficed, and was only achieved by bringing together multiple data sources in a manner that allowed me to discern context.
Why is this important? Well, for one, Windows keeps changing. Yes, yes, I know, we've heard this before...Windows Vista was a big leap forward (with respect to DFIR artifacts) over XP/2003, and Windows 7 a similar "leap". There were some pretty interesting changes in the short-lived Windows 8/8.1 world, but we've arguably seen a (dare I say, "significant"?) spike in changes just between versions of Windows 10 since that version has been out. The point is that you can't often go for a week or two before something new with respect to the Windows operating system (and specifically the Registry) is discovered or observed. An analysis process accounts for these changes occurring. For example, one of the approaches I use to winnow down data is to create mini-timelines and overlays; in short, using very specific data sources to create a mini-timeline in order to get a view of the data without all the inherent "noise" of the operating system. I'll create a timeline from a user's Registry hives, incorporating not just time-based data extracted from the hives (i.e., shellbags, UserAssist data, etc.), but also the overall metadata from those hives. This way, I can 'see' a user's activities over time, without having to wade through massive amounts of "noise", such as Windows Updates. And, I'll not only be able to develop context, but I'll also see new things, as well.
Speaking of which, let's take a look at the great work Jason Hale has done, as an example. Some of the things we've (I use the universal "we") seen over the years have been Registry key LastWrite times associated with USB devices being universally 'stomped' by updates, as well as Registry hive "backups" no longer being written (by default) to the RegBack folder.
Jason recently pointed out that key LastWrite times associated with MRUs within the shellbags artifacts have been 'stomped' by an update, similar to what was observed with USBStor keys. What impact would this have had on your analysis had this happened 6 months, or 2 years ago? How would this have impacted your findings in a case?
Why is understanding the use and function of the Registry important when it comes to analyzing Windows systems?
Two important aspects of the Windows Registry that I've discussed during many of my speaking engagements have been:
The Windows Registry contains a great deal of configuration information about the system that, if correctly understood and correctly interpreted, can have a significant impact on your analysis.
The Registry contains a lot of configuration settings for the operating system, many set by default, right "out of the box". There are others that can be changed along the way that have an impact on what users of the systems can see and do, as well as what attackers can do. For example, there's a setting that tells the Windows operating system to store credentials in memory, in plain text. Attackers can set this value to "1", wait a week, and come back and dump the credentials from memory, and not have to spend time cracking the passwords. There are settings that tell the Windows shell to not show all user accounts on the Welcome screen, as well as settings that control the functionality of Terminal Services, etc. There are even settings within the Registry that control what users can and cannot access, and how their account functions (i.e., deleted files bypass the Recycle Bin, etc.). Understanding these settings, as well as knowing how to determine when (or if) they changed, can have a significant impact on an investigation.
There are also a number of "autostart" locations within the Registry, keys or values that tell the operating system to start applications with no other interaction from the user beyond booting the system, or logging in. Consider this Threat Research blog post regarding newly-discovered malware, for example. Not only does it include what is reportedly a new persistence mechanism, but it also includes a figure illustrating how code used by the malware is encoded and placed in the Registry for later use. Knowing this, encountering an infected system and incorporating this into our analysis will allow us to discover new information about how the malware operated or was used.
As a side-note, does anyone have an NTUSER.DAT file from a user profile infected with the malware identified in the Juniper Threat Research blog post? I'm curious if the binaryImage32_* values are detected by the RegRipper sizes.pl plugin. Thanks in advance to anyone who can check, or share such a hive file.
The Windows Registry records and maintains a great deal of user activity that, if correctly understood and interpreted, can illustrate "humanness"; that is, provide strong indications of specific, purposeful human activity, as opposed to the automatic effects of the operating system, or of malware.
There are a number of user actions...opening or saving files, launching applications, etc...that are recorded within the Windows Registry, as part of tracking user activity in order to improve the "user experience". The idea is to make Windows more "user friendly", in part by making the more frequently used functionality more easily accessible to the user. The key here is that actions specifically taken by the user can be interpreted as "humanness", as the data recorded is the direct result of a person interacting with the Windows shell, applications, etc. This can provide information to inform an investigation, particularly when knowing when someone was sitting at the keyboard is important, or when discerning whether a user or some automated function accessed data is a critical aspect of an investigation.
Thankz/Shout-Outz
Thankz and shout-outz are due to a number of folks within the #DFIR community who've contributed to the topic through research, creating tools, etc. In no specific order:
Mari DeGrazia
Maxim Suhanov
Jason Hale
Eric Zimmerman
farmerK
I apologize profusely for any names I may have missed, as it was not intentional.
What is "Registry analysis"? Registry analysis is the observation and interpretation of data or metadata from the Windows Registry, in the context of other data/metadata, also from the Registry or other sources. The correct interpretation of this data can add an unprecedented level of granularity to the context surrounding various events observed during, for example, timeline analysis.
"Registry analysis" is not the parsing and display of individual artifacts from within the Registry; that's "parsing and display". It's also not keyword searching of the Registry; that's "keyword searching". Keyword searches or searches for indicators do not constitute "analysis". I do understand, however, that not all cases require Registry analysis. There are more than a few types of cases out there where keyword searching is all that is necessary, and I get it. Through engaging with other analysts and following up on conversations, I've seen where keyword searches have more than sufficed for a number of types of cases. But that's not the "analysis" we're talking about here.
That is not to say that keyword or indicator searches cannot be used as pivot points into more fully-fledged analysis. Very often, these searches can be an excellent entry point into analysis, allowing the investigator to winnow through vast amounts of data to determine what is initially important.
Analysis techniques where I have been able to incorporate data and metadata from the Windows Registry into an overall analysis process, have allowed me to get a better understanding of inherent activity derived or extracted from a system. This has occurred to the point, in some cases, of changing the direction of my investigation. Further, I'll be the first to admit that I haven't seen everything there is to see; in fact, within just the past weeks, I've observed activity that I had never seen before, and was able to develop an understanding of what activity led to the observed artifacts. This is not something for which keyword searching would have sufficed, and was only achieved by bringing together multiple data sources in a manner that allowed me to discern context.
Why is this important? Well, for one, Windows keeps changing. Yes, yes, I know, we've heard this before...Windows Vista was a big leap forward (with respect to DFIR artifacts) over XP/2003, and Windows 7 a similar "leap". There were some pretty interesting changes in the short-lived Windows 8/8.1 world, but we've arguably seen a (dare I say, "significant"?) spike in changes just between versions of Windows 10 since that version has been out. The point is that you can't often go for a week or two before something new with respect to the Windows operating system (and specifically the Registry) is discovered or observed. An analysis process accounts for these changes occurring. For example, one of the approaches I use to winnow down data is to create mini-timelines and overlays; in short, using very specific data sources to create a mini-timeline in order to get a view of the data without all the inherent "noise" of the operating system. I'll create a timeline from a user's Registry hives, incorporating not just time-based data extracted from the hives (i.e., shellbags, UserAssist data, etc.), but also the overall metadata from those hives. This way, I can 'see' a user's activities over time, without having to wade through massive amounts of "noise", such as Windows Updates. And, I'll not only be able to develop context, but I'll also see new things, as well.
Speaking of which, let's take a look at the great work Jason Hale has done, as an example. Some of the things we've (I use the universal "we") seen over the years have been Registry key LastWrite times associated with USB devices being universally 'stomped' by updates, as well as Registry hive "backups" no longer being written (by default) to the RegBack folder.
Jason recently pointed out that key LastWrite times associated with MRUs within the shellbags artifacts have been 'stomped' by an update, similar to what was observed with USBStor keys. What impact would this have had on your analysis had this happened 6 months, or 2 years ago? How would this have impacted your findings in a case?
Why is understanding the use and function of the Registry important when it comes to analyzing Windows systems?
Two important aspects of the Windows Registry that I've discussed during many of my speaking engagements have been:
The Windows Registry contains a great deal of configuration information about the system that, if correctly understood and correctly interpreted, can have a significant impact on your analysis.
The Registry contains a lot of configuration settings for the operating system, many set by default, right "out of the box". There are others that can be changed along the way that have an impact on what users of the systems can see and do, as well as what attackers can do. For example, there's a setting that tells the Windows operating system to store credentials in memory, in plain text. Attackers can set this value to "1", wait a week, and come back and dump the credentials from memory, and not have to spend time cracking the passwords. There are settings that tell the Windows shell to not show all user accounts on the Welcome screen, as well as settings that control the functionality of Terminal Services, etc. There are even settings within the Registry that control what users can and cannot access, and how their account functions (i.e., deleted files bypass the Recycle Bin, etc.). Understanding these settings, as well as knowing how to determine when (or if) they changed, can have a significant impact on an investigation.
There are also a number of "autostart" locations within the Registry, keys or values that tell the operating system to start applications with no other interaction from the user beyond booting the system, or logging in. Consider this Threat Research blog post regarding newly-discovered malware, for example. Not only does it include what is reportedly a new persistence mechanism, but it also includes a figure illustrating how code used by the malware is encoded and placed in the Registry for later use. Knowing this, encountering an infected system and incorporating this into our analysis will allow us to discover new information about how the malware operated or was used.
As a side-note, does anyone have an NTUSER.DAT file from a user profile infected with the malware identified in the Juniper Threat Research blog post? I'm curious if the binaryImage32_* values are detected by the RegRipper sizes.pl plugin. Thanks in advance to anyone who can check, or share such a hive file.
The Windows Registry records and maintains a great deal of user activity that, if correctly understood and interpreted, can illustrate "humanness"; that is, provide strong indications of specific, purposeful human activity, as opposed to the automatic effects of the operating system, or of malware.
There are a number of user actions...opening or saving files, launching applications, etc...that are recorded within the Windows Registry, as part of tracking user activity in order to improve the "user experience". The idea is to make Windows more "user friendly", in part by making the more frequently used functionality more easily accessible to the user. The key here is that actions specifically taken by the user can be interpreted as "humanness", as the data recorded is the direct result of a person interacting with the Windows shell, applications, etc. This can provide information to inform an investigation, particularly when knowing when someone was sitting at the keyboard is important, or when discerning whether a user or some automated function accessed data is a critical aspect of an investigation.
Thankz/Shout-Outz
Thankz and shout-outz are due to a number of folks within the #DFIR community who've contributed to the topic through research, creating tools, etc. In no specific order:
Mari DeGrazia
Maxim Suhanov
Jason Hale
Eric Zimmerman
farmerK
I apologize profusely for any names I may have missed, as it was not intentional.
Friday, September 20, 2019
A Brief History of DFIR Time, pt II
Continuing on from my previous post...
Once I left active duty, one of my first jobs was in an information security role, and I was doing a fair bit of "war dialing", which was fun. The main programs we used at the time were THC Scan and Tone Loc, but in a pinch, you could use the Windows dialer to connect to a number, and listen to the laptop speaker to determine what was on the other end. In most instances, you'd get someone saying, "hello?", but in the few instances where a computer or fax would pick up, we'd note that and move on. The laptops available at the time had built-in modems, as well as PCMCIA slots if you wanted to insert a network (ethernet or, yes, token ring) card. Our laptop bags had phone and ethernet cables, but most companies wouldn't spring for a token ring card because (a) they were expensive and (b) not many customers had token ring networks. Until you went on-site and someone said, "uh...oh, yeah...we use token ring." Of course, no one would say anything during the initial call unless you specifically asked, so that question went on script.
At that time, "computer forensics" was really not well known outside of very small circles, so there weren't much in the way of "go bags" or kits. The few folks "doing" computer forensics at the time (that I was aware of) were largely former AF OSI enlisted folks, sequestered behind heavy duty doors with special locks. When you did get a peek inside their wizardy world, the biggest component was a custom-built tower system with extra bays, and everything running on Linux.
At one point early in my career, I worked for a company that was trying to get into the security business, and while I was waiting for a contract, I taught myself Perl because that's a skill that the network engineering folks were looking for in candidates at the time, and I wanted to help out. I never did get a chance to do any really extensive work for them, and I later moved on to a different company, this time performing more extensive (than just war dialing) vulnerability assessments. My boss at the time told me that I would need to run the commercial scanner (ISS's Internet Scanner) for "about 2 to 3 yrs" before I would really understand what it was doing; within 6 months of getting the job, I was writing a tool to replace the commercial product, due to the number of false "hits" we were getting, many due to misinterpretation of the data returned from the query.
For example, there was the AutoAdminLogon value in the Registry; if the commercial tool found the value name, it responded with "AutoAdminLogon value set", even if the data for the value was "0". Further, it never checked for the DefaultUserName or DefaultPassword values. In one instance, the commercial tool determined that 22 systems within a customer's infrastructure had the value 'set', while the customer knew that it was only 1 system for which the value was actually set, and the system would automatically log into the Administrator account upon system boot; the other 21 had the value name, but the data was "0", and there was no username or password in the Registry. They had already found those 21 systems and disabled the functionality through the UI; had we provided the findings from the commercial tool in our report, we would have been remiss, and the customer would have been correct in questioning the rest of the report.
Looking back, I realize that what I'd written was a "threat hunting" tool. We didn't have any software at the time that could perform EDR functions; "visibility" consisted of either sitting at the console and opening Task Manager, or having the admin send a screen capture of Task Manager. However, this tool was accessing systems and getting all sorts of data from it, including things like active modems, running services, applications, etc. So while the tool wasn't able to monitor processes and network connections over time, it was able get point-in-time data, as well as look at what had occurred in the past on the system. This was the '98-'99 time frame, and as such, a bit before terms like "adversary" and "APT" started to appear in our everyday usage.
I haven't always been in a consulting role. At one point in my career, I took a position as a computer security engineer in an FTE role. During that time, I responded to a couple of internal incidents, and wrote another "threat hunting" tool, albeit on a very limited scale. The tool would access the Windows domain and get a list of all currently running systems; from there, it would access each system and collect the contents of several persistence locations. When I first ran the tool, I'd get a LOT of data back, but over time and with no small amount of investigation, I developed a whitelist of authorized entries. So, after a couple of weeks, I could launch the tool when I headed to a meeting or to lunch, and come back to a list of entries about half a page long. This allowed me to see some issues, sometimes before they became really big issues, as well as identify trends across the infrastructure. For example, there was one system that, because of it's location and the fact that it was used by overnight staff, kept popping up with "issues", no matter how many times I cleaned it.
Right around the '99-'00 time frame, I started attending and, more importantly, speaking at security conferences, including BlackHat and DefCon. I had a great deal of experience with public speaking (I had been an instructor while I was on active duty, and classroom audiences were generally 250+ students...) but I was still a little nervous about presenting, because I had this image in my mind of what the audience was going to be like; today, we call this "imposter syndrome". When I was an instructor in the military, I was a more experienced officer (albeit not by much...just a few years), speaking on my profession (i.e., communications). I was teaching new officers the basic skills they needed to learn, using the equipment I was very familiar with. It was an entirely different environment, and here I was speaking to a roomful of people who, I figured, had a great deal more experience in the industry than I did. And in some cases, that was a correct assumption, albeit not in all cases. What I found when presenting on a solution I'd found to a problem I had was that there were others who had a similar problem, but had not yet arrived at a solution. This realization really changed things for me, it really impacted my perspective, and it subsequently led to me submitting more and more.
For all of you out there who are thinking about submitting a presentation, and that thought scares you to death, ask yourself why that is. More than likely, it's because you're thinking that you'll be judged. And you're right, that's what people do. However, the next time you're attending a speaking event, sit in the back of the room and just watch what everyone else is doing. There will be lots of things they're doing...but one of them won't be paying attention, not for many folks, anyway. Case in point, just a little over a month and a half ago, I gave a presentation, and at the beginning of the presentation, I stated...twice...when copies of the slides would be available. The first question at the end of the presentation was...well, get three guesses, and the first two don't count.
My point is that a great deal of the anxiety that you feel when thinking about submitting to or just speaking at a conference is pretty normal, but it's also largely self-inflicted. Don't let it paralyze you; instead, use it to fuel your development. Use that energy to check the details of your presentation one more time, to rehearse one more time, to seek out feedback on the content one more time.
And don't be afraid of people asking questions, because that fear will prevent you from actually listening to the question. Remember, for all intents and purposes, you are the expert on the topic, and you're presenting your view, based on your perspective. Yes, there are going to be other perspectives; don't be so overwhelmed by the fear of a question that you don't actually listen to the question. I've been asked, "...did you look at...", as well as the more pointed, "...why didn't you consider...", and by listening to the question, I was able to get beyond that "imposter syndrome" anxiety and actually address the question.
One question I received back in the early 2000's was at an HTCIA International conference in Fairfax, VA...I was presenting on Registry analysis and someone in the audience, with a laptop in front of them, asked me, "what happens when you do X?" I had a sudden flash of inspiration, and I turned the question around...I asked the person asking the question to try what he'd suggested, and tell us all what he found. No, what he'd asked wasn't something I'd considered, but it did seem like a good idea...so rather than going back and forth on the specifics, I thought it would be a great idea to have them try it, in hopes of getting others to see that rather than going to someone else for answers, there are a great number of things we can try on our own, and discover for ourselves.
In 2005, Cory Altheide and I wrote the first published paper on tracking USB devices across Windows systems. It's fascinating to look back and see not only how far we've come with this topic, particularly given how much the Windows operating system has changed over that time, but to also see how many times the paper is referenced. In most cases, the articles that reference our work are peer-reviewed articles, ones for which a literature search is a requirement. Even so, it's pretty cool to see how many times that article is referenced. Yes, there are a lot of those in the industry (as with any other industry) who "do" research without first performing a literature search, but that search is a pretty hard-and-fast requirement for academic, peer-reviewed papers, and it is pretty fascinating to see the number of references to our paper.
As digital forensics and incident response grew into something around which a service could be built and sold to customers, we started to develop "go kits", and there were lots of discussions and arguments on the Internet regarding what went into those kits. Prior to the advent of enterprise-wide response capabilities (i.e., deploying an EDR monitoring tool, etc.), I had a Pelican case that weighed 65 lbs (I know because I had to check it in every time I flew out...), and contained two MacBook laptops, running Windows XP, two sets of hardware write-blockers, a wide assortment of cables, as well as hard copies of documentation. I also had a laptop in my backpack with backup copies of all documentation, as well as hard copy of all pertinent phone numbers; if EVERYTHING failed, including my cell phone (notice I didn't say "smart phone", because we didn't have those at the time) battery, I still needed to be able to contact my boss, the customer, etc. If I lost everything else, I could still get to a store, purchase a new laptop, put the tools I used on it (from a CD...remember those?), and get to work.
With the enterprise reach of EDR tools that we have at our disposal, there's a shift in how the DFIR industry reacts and provides services, but we still have a lot our original or age-old issues, due to the fact that as the industry has progressed, we've never really dealt with those issues. Things like documentation and sharing of information or threat intel, specificity of language, correct data interpretation, not interpreting artifacts in isolation from other artifacts, etc. These are things that we need to improve upon, as an industry.
Even so, it's been pretty fascinating to me to see how, in some cases, DFIR work has really progressed, particularly with respect to enterprise-wide response. There's quick/timely deployment of visibility (i.e., EDR) from a remote location, data is collected and analyzed, and then answers are provided, very often before the next available flight to the location departs. It's a brave new world out there regarding what can be done to respond to incidents.
Once I left active duty, one of my first jobs was in an information security role, and I was doing a fair bit of "war dialing", which was fun. The main programs we used at the time were THC Scan and Tone Loc, but in a pinch, you could use the Windows dialer to connect to a number, and listen to the laptop speaker to determine what was on the other end. In most instances, you'd get someone saying, "hello?", but in the few instances where a computer or fax would pick up, we'd note that and move on. The laptops available at the time had built-in modems, as well as PCMCIA slots if you wanted to insert a network (ethernet or, yes, token ring) card. Our laptop bags had phone and ethernet cables, but most companies wouldn't spring for a token ring card because (a) they were expensive and (b) not many customers had token ring networks. Until you went on-site and someone said, "uh...oh, yeah...we use token ring." Of course, no one would say anything during the initial call unless you specifically asked, so that question went on script.
At that time, "computer forensics" was really not well known outside of very small circles, so there weren't much in the way of "go bags" or kits. The few folks "doing" computer forensics at the time (that I was aware of) were largely former AF OSI enlisted folks, sequestered behind heavy duty doors with special locks. When you did get a peek inside their wizardy world, the biggest component was a custom-built tower system with extra bays, and everything running on Linux.
At one point early in my career, I worked for a company that was trying to get into the security business, and while I was waiting for a contract, I taught myself Perl because that's a skill that the network engineering folks were looking for in candidates at the time, and I wanted to help out. I never did get a chance to do any really extensive work for them, and I later moved on to a different company, this time performing more extensive (than just war dialing) vulnerability assessments. My boss at the time told me that I would need to run the commercial scanner (ISS's Internet Scanner) for "about 2 to 3 yrs" before I would really understand what it was doing; within 6 months of getting the job, I was writing a tool to replace the commercial product, due to the number of false "hits" we were getting, many due to misinterpretation of the data returned from the query.
For example, there was the AutoAdminLogon value in the Registry; if the commercial tool found the value name, it responded with "AutoAdminLogon value set", even if the data for the value was "0". Further, it never checked for the DefaultUserName or DefaultPassword values. In one instance, the commercial tool determined that 22 systems within a customer's infrastructure had the value 'set', while the customer knew that it was only 1 system for which the value was actually set, and the system would automatically log into the Administrator account upon system boot; the other 21 had the value name, but the data was "0", and there was no username or password in the Registry. They had already found those 21 systems and disabled the functionality through the UI; had we provided the findings from the commercial tool in our report, we would have been remiss, and the customer would have been correct in questioning the rest of the report.
Looking back, I realize that what I'd written was a "threat hunting" tool. We didn't have any software at the time that could perform EDR functions; "visibility" consisted of either sitting at the console and opening Task Manager, or having the admin send a screen capture of Task Manager. However, this tool was accessing systems and getting all sorts of data from it, including things like active modems, running services, applications, etc. So while the tool wasn't able to monitor processes and network connections over time, it was able get point-in-time data, as well as look at what had occurred in the past on the system. This was the '98-'99 time frame, and as such, a bit before terms like "adversary" and "APT" started to appear in our everyday usage.
I haven't always been in a consulting role. At one point in my career, I took a position as a computer security engineer in an FTE role. During that time, I responded to a couple of internal incidents, and wrote another "threat hunting" tool, albeit on a very limited scale. The tool would access the Windows domain and get a list of all currently running systems; from there, it would access each system and collect the contents of several persistence locations. When I first ran the tool, I'd get a LOT of data back, but over time and with no small amount of investigation, I developed a whitelist of authorized entries. So, after a couple of weeks, I could launch the tool when I headed to a meeting or to lunch, and come back to a list of entries about half a page long. This allowed me to see some issues, sometimes before they became really big issues, as well as identify trends across the infrastructure. For example, there was one system that, because of it's location and the fact that it was used by overnight staff, kept popping up with "issues", no matter how many times I cleaned it.
Right around the '99-'00 time frame, I started attending and, more importantly, speaking at security conferences, including BlackHat and DefCon. I had a great deal of experience with public speaking (I had been an instructor while I was on active duty, and classroom audiences were generally 250+ students...) but I was still a little nervous about presenting, because I had this image in my mind of what the audience was going to be like; today, we call this "imposter syndrome". When I was an instructor in the military, I was a more experienced officer (albeit not by much...just a few years), speaking on my profession (i.e., communications). I was teaching new officers the basic skills they needed to learn, using the equipment I was very familiar with. It was an entirely different environment, and here I was speaking to a roomful of people who, I figured, had a great deal more experience in the industry than I did. And in some cases, that was a correct assumption, albeit not in all cases. What I found when presenting on a solution I'd found to a problem I had was that there were others who had a similar problem, but had not yet arrived at a solution. This realization really changed things for me, it really impacted my perspective, and it subsequently led to me submitting more and more.
For all of you out there who are thinking about submitting a presentation, and that thought scares you to death, ask yourself why that is. More than likely, it's because you're thinking that you'll be judged. And you're right, that's what people do. However, the next time you're attending a speaking event, sit in the back of the room and just watch what everyone else is doing. There will be lots of things they're doing...but one of them won't be paying attention, not for many folks, anyway. Case in point, just a little over a month and a half ago, I gave a presentation, and at the beginning of the presentation, I stated...twice...when copies of the slides would be available. The first question at the end of the presentation was...well, get three guesses, and the first two don't count.
My point is that a great deal of the anxiety that you feel when thinking about submitting to or just speaking at a conference is pretty normal, but it's also largely self-inflicted. Don't let it paralyze you; instead, use it to fuel your development. Use that energy to check the details of your presentation one more time, to rehearse one more time, to seek out feedback on the content one more time.
And don't be afraid of people asking questions, because that fear will prevent you from actually listening to the question. Remember, for all intents and purposes, you are the expert on the topic, and you're presenting your view, based on your perspective. Yes, there are going to be other perspectives; don't be so overwhelmed by the fear of a question that you don't actually listen to the question. I've been asked, "...did you look at...", as well as the more pointed, "...why didn't you consider...", and by listening to the question, I was able to get beyond that "imposter syndrome" anxiety and actually address the question.
One question I received back in the early 2000's was at an HTCIA International conference in Fairfax, VA...I was presenting on Registry analysis and someone in the audience, with a laptop in front of them, asked me, "what happens when you do X?" I had a sudden flash of inspiration, and I turned the question around...I asked the person asking the question to try what he'd suggested, and tell us all what he found. No, what he'd asked wasn't something I'd considered, but it did seem like a good idea...so rather than going back and forth on the specifics, I thought it would be a great idea to have them try it, in hopes of getting others to see that rather than going to someone else for answers, there are a great number of things we can try on our own, and discover for ourselves.
In 2005, Cory Altheide and I wrote the first published paper on tracking USB devices across Windows systems. It's fascinating to look back and see not only how far we've come with this topic, particularly given how much the Windows operating system has changed over that time, but to also see how many times the paper is referenced. In most cases, the articles that reference our work are peer-reviewed articles, ones for which a literature search is a requirement. Even so, it's pretty cool to see how many times that article is referenced. Yes, there are a lot of those in the industry (as with any other industry) who "do" research without first performing a literature search, but that search is a pretty hard-and-fast requirement for academic, peer-reviewed papers, and it is pretty fascinating to see the number of references to our paper.
As digital forensics and incident response grew into something around which a service could be built and sold to customers, we started to develop "go kits", and there were lots of discussions and arguments on the Internet regarding what went into those kits. Prior to the advent of enterprise-wide response capabilities (i.e., deploying an EDR monitoring tool, etc.), I had a Pelican case that weighed 65 lbs (I know because I had to check it in every time I flew out...), and contained two MacBook laptops, running Windows XP, two sets of hardware write-blockers, a wide assortment of cables, as well as hard copies of documentation. I also had a laptop in my backpack with backup copies of all documentation, as well as hard copy of all pertinent phone numbers; if EVERYTHING failed, including my cell phone (notice I didn't say "smart phone", because we didn't have those at the time) battery, I still needed to be able to contact my boss, the customer, etc. If I lost everything else, I could still get to a store, purchase a new laptop, put the tools I used on it (from a CD...remember those?), and get to work.
With the enterprise reach of EDR tools that we have at our disposal, there's a shift in how the DFIR industry reacts and provides services, but we still have a lot our original or age-old issues, due to the fact that as the industry has progressed, we've never really dealt with those issues. Things like documentation and sharing of information or threat intel, specificity of language, correct data interpretation, not interpreting artifacts in isolation from other artifacts, etc. These are things that we need to improve upon, as an industry.
Even so, it's been pretty fascinating to me to see how, in some cases, DFIR work has really progressed, particularly with respect to enterprise-wide response. There's quick/timely deployment of visibility (i.e., EDR) from a remote location, data is collected and analyzed, and then answers are provided, very often before the next available flight to the location departs. It's a brave new world out there regarding what can be done to respond to incidents.
Monday, September 09, 2019
The Ransomware Economy
There's no doubt about it...cybercrime, and especially ransomware, is an entire economy in and of itself.
Don't believe me?
Read through this ProPublica article, not just once, but a couple of times. And take notes. Then go back and read the notes. Here's what I got from the article:
Not long ago, a fellow responder shared that many of the ransomware cases he works include an element of data exfiltration. A recent 60Minutes segment on ransomware includes a similar statement; if you watch until 9:50 in the segment, you'll see mention of the bad guy further extorting an organization by threatening to leak their "internal data".
Let's look at some of the reporting on ransomware, such as this The Conversation article. At one point in the article, we see the statement:
Ransomware usually spreads via phishing emails or links...
Perhaps "usually", yes, but not always. The 60Minutes segment mentioned the Samsam ransomware; during the first half of 2016, these guys were seen using the publicly available JexBoss exploit to gain access to organizations through JBoss CMS servers. At that time, the average time between initial access to the organization and deploying the ransomware was 4 months. In 2017, in some cases, they switched to Terminal Services servers, gaining access via easily-guessed passwords. Yes, some ransomware (some Ryuk incidents, for example) incidents begin with a phishing email, and then branch off into deploying remote access tools, internal reconnaissance, possibly privilege escalation, networking mapping, and finally, deploying the ransomware.
Another quote from the article:
Offenders will do their homework before launching an attack, in order to create the most severe disruption they possibly can.
Yes, they will. But what does this mean? This means a couple of things; first, they decide who to target, and when. Employees within companies have targets against which they're judged; sales reps, for example, usually hit crunch time at the end of a quarter. So, what the bad guys will do is send something to a sales rep that looks legit, and it's something that they need to open. Yes, they're targeting individuals.
What does this look like, you ask? While not related to ransomware, but take a look at the Mia Ash story, and you'll see what targeting looks like. Going after sales reps, or the finance department, legal counsel...all of these are targets within an organization, and very often the "lure" looks attractive enough to obviate phishing awareness training. However, this is only the beginning. In the Mia Ash story, the adversary developed a relationship with their targets, to the point where, when it came time to send a weaponized document for the target to open, the target had no doubt in their mind regarding the fact that they were dealing with "Mia".
Something that isn't stated in the media is that, for some ransomware cases, once an adversary gains initial access to an infrastructure, there are a number of actions that must take place in order for them to have such an impact as to make paying the ransom the obvious choice going forward. They need to observe and orient to where they are, collect information about the infrastructure, make decisions (that's the easy part, they're often quite practiced at this), and then act. This is Col Boyd's OODA loop. In some cases, this can take weeks, and in others, months. Unfortunately, one of the things missing from public reporting of ransomware incidents, in addition to the observed initial access method, is the time that the adversary is on target before deploying ransomware. It's not an easy task to go into a completely new infrastructure and find those files and systems that, if unavailable, would bring the organization to a halt.
With visibility, these actions can be detected, and responded to in a timely manner. When I say, "responded to", I mean determining the initial infection vector and following a containment and eradication plan early in the adversary's process. Let's say that you detect a new account being created on a system, because you have the visibility to do so...which user account was used to create the new one? How did that user account gain access to the system on which the command was run? Follow the tracks back to the starting point, and determine how the adversary got on the system, and then search your infrastructure for other, similar artifacts.
It all starts with visibility. Don't address ransomware by trying to figure out if you should restore systems from backup or pay the ransom; instead, catch the adversary early in their process and stop them before they encrypt their first file.
Don't believe me?
Read through this ProPublica article, not just once, but a couple of times. And take notes. Then go back and read the notes. Here's what I got from the article:
- Organizations are looking to insurance policies to defray the costs of incidents. Rather than investing in prevention, detection, and response, they're accepting (to some degree) that these incidents are going to happen, and seeking to establish a means to minimize their financial risk. Hence, insurance policies.
- A ransomware incident occurs, and the policy kicks in. Depending upon how the policy was set up, and what it covers, the deductible may be much less than the ransom. Financial risk minimized.
- Insurance providers are more interested in getting ransoms paid quickly; getting the encryption keys and recovering files minimizes down time, and therefore any additional costs incurred as a result of services not being available. So, insurance providers want the ransom paid, in order to minimize their financial exposure.
- There's also an entire economy that's popped up around ransom payment brokers, organizations that act as intermediaries between victim organizations, insurance providers, and the bad guys.
Not long ago, a fellow responder shared that many of the ransomware cases he works include an element of data exfiltration. A recent 60Minutes segment on ransomware includes a similar statement; if you watch until 9:50 in the segment, you'll see mention of the bad guy further extorting an organization by threatening to leak their "internal data".
Let's look at some of the reporting on ransomware, such as this The Conversation article. At one point in the article, we see the statement:
Ransomware usually spreads via phishing emails or links...
Perhaps "usually", yes, but not always. The 60Minutes segment mentioned the Samsam ransomware; during the first half of 2016, these guys were seen using the publicly available JexBoss exploit to gain access to organizations through JBoss CMS servers. At that time, the average time between initial access to the organization and deploying the ransomware was 4 months. In 2017, in some cases, they switched to Terminal Services servers, gaining access via easily-guessed passwords. Yes, some ransomware (some Ryuk incidents, for example) incidents begin with a phishing email, and then branch off into deploying remote access tools, internal reconnaissance, possibly privilege escalation, networking mapping, and finally, deploying the ransomware.
Another quote from the article:
Offenders will do their homework before launching an attack, in order to create the most severe disruption they possibly can.
Yes, they will. But what does this mean? This means a couple of things; first, they decide who to target, and when. Employees within companies have targets against which they're judged; sales reps, for example, usually hit crunch time at the end of a quarter. So, what the bad guys will do is send something to a sales rep that looks legit, and it's something that they need to open. Yes, they're targeting individuals.
What does this look like, you ask? While not related to ransomware, but take a look at the Mia Ash story, and you'll see what targeting looks like. Going after sales reps, or the finance department, legal counsel...all of these are targets within an organization, and very often the "lure" looks attractive enough to obviate phishing awareness training. However, this is only the beginning. In the Mia Ash story, the adversary developed a relationship with their targets, to the point where, when it came time to send a weaponized document for the target to open, the target had no doubt in their mind regarding the fact that they were dealing with "Mia".
Something that isn't stated in the media is that, for some ransomware cases, once an adversary gains initial access to an infrastructure, there are a number of actions that must take place in order for them to have such an impact as to make paying the ransom the obvious choice going forward. They need to observe and orient to where they are, collect information about the infrastructure, make decisions (that's the easy part, they're often quite practiced at this), and then act. This is Col Boyd's OODA loop. In some cases, this can take weeks, and in others, months. Unfortunately, one of the things missing from public reporting of ransomware incidents, in addition to the observed initial access method, is the time that the adversary is on target before deploying ransomware. It's not an easy task to go into a completely new infrastructure and find those files and systems that, if unavailable, would bring the organization to a halt.
With visibility, these actions can be detected, and responded to in a timely manner. When I say, "responded to", I mean determining the initial infection vector and following a containment and eradication plan early in the adversary's process. Let's say that you detect a new account being created on a system, because you have the visibility to do so...which user account was used to create the new one? How did that user account gain access to the system on which the command was run? Follow the tracks back to the starting point, and determine how the adversary got on the system, and then search your infrastructure for other, similar artifacts.
It all starts with visibility. Don't address ransomware by trying to figure out if you should restore systems from backup or pay the ransom; instead, catch the adversary early in their process and stop them before they encrypt their first file.
A Brief History of DFIR Time, pt I
Whether we like it or not, we're all time travelers. We're all moving through time, caught in the flow. In the western world, we're moving left-to-right, going along with the flow of time, from point A to point B.
Sometimes it's interesting to look back at where we've been, what we've been witness to, and to reflect on and appreciate it. Here's an abridged version of my take...
As a kid, my parents purchased a Timex-Sinclair 1000 computer. I started out by following instructions for writing programs and saving them to a cassette tape...or trying to, as the case may be. This wasn't the most reliable means (although it was the only one) for saving programs, and sometimes things would get corrupted, and I'd have to start all over. As I learned a little bit of coding, I'd try different things...I'd start with the basic (no pun intended) recipe, and then make small modifications to see what happened.
In the early '80s, I was programming BASIC on the Apple IIe during a summer course. Later, my parents purchased an Epson QX-10, which my father used for word processing. During my senior year in high school, I took AP Computer Science, which involved programming PASCAL on the TRS-80 systems at the school. My folks found a copy of Turbo PASCAL, which meant I could easily compile my programs at home in minutes, rather than trying to schedule time to get access to one of the TRS-80 systems at school, and get in before lunch, because compilation took over half an hour for some programs.
When I went to college (circa '85) we had a BASIC programming course, and we were still using the TRS-80 systems. There were some mainframe systems in the physics building, and while I didn't get a real introduction to networking, some of did have fun sending messages to each other using the "wall" command.
After I got commissioned and went on active duty, I really didn't have a great deal of contact with computers. In the Marine Corps at that time, Communications was a separate MOS from Data Processing, and as such, officers (and enlisted) for the MOSs attended separate schools. For officers, both school houses were located on Quantico, at the time. After training, I found that there was a great deal of cross-training in the fleet; quite often, CommOs were sent to data processing courses by their units. The Marine Corps later combined the MOSs, along with the school houses and the curricula.
In the mid-'90s, I had the opportunity to attend graduate school, and I really got much more involved with computers. I showed up with a 486DX desktop system that I used at home, and one of the first things I did was add a hard drive. At the time, that meant putting it in the right location on the ribbon cable, and setting the correct jumpers on the drive chassis. I later saved up and purchased additional RAM, going from 4MB to 16MB. Yes, with an "M". I also began going beyond Windows for Workgroups 3.11, and expanding into OS/2 2.1, and then later, OS/2 3.0 Warp. At the time, I was using a SLIP/PPP script to dial into a local ISP, and then connecting remotely to the school systems.
Interestingly enough, I found someone in my local community who was running a BBS based on Amiga systems, and got a look at his setup. That was a big deal at the time, because the town I lived in was close to a LATA border, meaning that while I could dial a number that was physically located about 10 miles south of me for no extra charge, the closest AOL POP was two miles north, and therefore, a long distance charge. Eventually an airline pilot who lived in the local community set up an ISP, and I used that to access the Internet.
At school, I was working on SparcStations, using the Netscape browser. I was learning about UseNet, SunOS, *nix-based systems, etc., none of which had anything to do with the curriculum. I was the student rep to the sysadmin council when SATAN was released. During the course of my "studies", I learned a little bit of C and C++ coding, a lot of MatLab, and a good bit of Java, at a time before Java was 1.0 GA status. I played around with a bunch of different things with Java...I wrote programs to query fingerd, wrote an email spoofing program, and I wrote some code that connected the chargen port to the echo port...that was fun!
When I first started at graduate school, I didn't know it at the time, but I spent about 4 months walking by Gary Kildall's office every day on my way out of the building. His office was next to one of the main doors that led out to the quad, where I'd go sit to each lunch. I never met Gary, nor took one of his courses, and again, it wasn't until much later that I found out who he was, and the role he played (or depending on your perspective, didn't play...) in the history of computing. In one of my courses, I learned about the Hamming distance, and later took a seminar from Dr. Hamming himself.
As part of my master's thesis, I set up a lab; it consisted of two Cisco 2514 routers that I cross-connected, and from which I ran two small networks. One was 10BaseT, the other 10Base2, and both had one Windows NT 3.51 server and three Windows 95 workstations. The entire set up was connected to the campus backbone via a 10Base5 "vampire tap". To collect data for my thesis, I wrote an SNMP polling application in Java, and processed the data using various statistical techniques in MatLab.
While I was in graduate school, one of my favorite courses was a new class in neural networks. Part of the reason I liked it was due to how it was structured; the first half of the course was some instruction and small projects to get out feet wet, but the projects were small enough to allow us to stretch a bit, as well. In many of the courses available at the time, the labs were such that it took most, if not all of the week to get them done, so there was very little learning beyond just finishing the minimum requirements for the lab. In this course (and a few others), a different approach was taken, one that allowed the students to engage, experiment, and learn. The second half of the course was a project, which was really cool to work on. As it turned out, several of the students used that course as the basis for their master's thesis...one wrote a program that could discern 'dirty' images of six consecutive Cyrillic characters (i.e., something you'd seen in a satellite photo of Red Square, for example.) Another student created a neural network to assist with sonar identification.
So, how does all this matter? Well, 24+ years later, I can discern what's behind the terms "ML" and "AI" that we see with respect to cyber security products. ;-)
My time in grad school was also when I started brushing up against "information security" in the world of computers. During a C programming course, I finished my assigned labs and wanted to learn a bit more, so I downloaded a file called 'crack.c' to see what it did. All I ever did was open it in an editor, but the senior sysadmin for the department got upset. She even told me that I had "violated security policies". When I asked her to see the policies, knowing that I had never signed such a policy, I learned that there really was no written "policy". That was to change more than a year later when a new Admiral took over the school, but at the time, there was no written security policy that any students read or signed.
After I graduated, I spent 8 months processing out of the military, and during that time was assigned to the Marine detachment at the Defense Language Institute (DLI). While there, one of the things I did was get the detachment's computer systems connected to the DLI campus area network (CAN), which was token ring. Also during that time, the Commandant of the Marine Corps (Gen. Krulak) had stated that Marines were authorized to play "Marine DOOM"; the setup at the detachment was six Gateway systems connected via 10Base2, running IPX. I was able to use what I had learned just down the street (literally) to help get the "network" up and running.
Sometimes it's interesting to look back at where we've been, what we've been witness to, and to reflect on and appreciate it. Here's an abridged version of my take...
As a kid, my parents purchased a Timex-Sinclair 1000 computer. I started out by following instructions for writing programs and saving them to a cassette tape...or trying to, as the case may be. This wasn't the most reliable means (although it was the only one) for saving programs, and sometimes things would get corrupted, and I'd have to start all over. As I learned a little bit of coding, I'd try different things...I'd start with the basic (no pun intended) recipe, and then make small modifications to see what happened.
In the early '80s, I was programming BASIC on the Apple IIe during a summer course. Later, my parents purchased an Epson QX-10, which my father used for word processing. During my senior year in high school, I took AP Computer Science, which involved programming PASCAL on the TRS-80 systems at the school. My folks found a copy of Turbo PASCAL, which meant I could easily compile my programs at home in minutes, rather than trying to schedule time to get access to one of the TRS-80 systems at school, and get in before lunch, because compilation took over half an hour for some programs.
When I went to college (circa '85) we had a BASIC programming course, and we were still using the TRS-80 systems. There were some mainframe systems in the physics building, and while I didn't get a real introduction to networking, some of did have fun sending messages to each other using the "wall" command.
After I got commissioned and went on active duty, I really didn't have a great deal of contact with computers. In the Marine Corps at that time, Communications was a separate MOS from Data Processing, and as such, officers (and enlisted) for the MOSs attended separate schools. For officers, both school houses were located on Quantico, at the time. After training, I found that there was a great deal of cross-training in the fleet; quite often, CommOs were sent to data processing courses by their units. The Marine Corps later combined the MOSs, along with the school houses and the curricula.
In the mid-'90s, I had the opportunity to attend graduate school, and I really got much more involved with computers. I showed up with a 486DX desktop system that I used at home, and one of the first things I did was add a hard drive. At the time, that meant putting it in the right location on the ribbon cable, and setting the correct jumpers on the drive chassis. I later saved up and purchased additional RAM, going from 4MB to 16MB. Yes, with an "M". I also began going beyond Windows for Workgroups 3.11, and expanding into OS/2 2.1, and then later, OS/2 3.0 Warp. At the time, I was using a SLIP/PPP script to dial into a local ISP, and then connecting remotely to the school systems.
Interestingly enough, I found someone in my local community who was running a BBS based on Amiga systems, and got a look at his setup. That was a big deal at the time, because the town I lived in was close to a LATA border, meaning that while I could dial a number that was physically located about 10 miles south of me for no extra charge, the closest AOL POP was two miles north, and therefore, a long distance charge. Eventually an airline pilot who lived in the local community set up an ISP, and I used that to access the Internet.
At school, I was working on SparcStations, using the Netscape browser. I was learning about UseNet, SunOS, *nix-based systems, etc., none of which had anything to do with the curriculum. I was the student rep to the sysadmin council when SATAN was released. During the course of my "studies", I learned a little bit of C and C++ coding, a lot of MatLab, and a good bit of Java, at a time before Java was 1.0 GA status. I played around with a bunch of different things with Java...I wrote programs to query fingerd, wrote an email spoofing program, and I wrote some code that connected the chargen port to the echo port...that was fun!
When I first started at graduate school, I didn't know it at the time, but I spent about 4 months walking by Gary Kildall's office every day on my way out of the building. His office was next to one of the main doors that led out to the quad, where I'd go sit to each lunch. I never met Gary, nor took one of his courses, and again, it wasn't until much later that I found out who he was, and the role he played (or depending on your perspective, didn't play...) in the history of computing. In one of my courses, I learned about the Hamming distance, and later took a seminar from Dr. Hamming himself.
As part of my master's thesis, I set up a lab; it consisted of two Cisco 2514 routers that I cross-connected, and from which I ran two small networks. One was 10BaseT, the other 10Base2, and both had one Windows NT 3.51 server and three Windows 95 workstations. The entire set up was connected to the campus backbone via a 10Base5 "vampire tap". To collect data for my thesis, I wrote an SNMP polling application in Java, and processed the data using various statistical techniques in MatLab.
While I was in graduate school, one of my favorite courses was a new class in neural networks. Part of the reason I liked it was due to how it was structured; the first half of the course was some instruction and small projects to get out feet wet, but the projects were small enough to allow us to stretch a bit, as well. In many of the courses available at the time, the labs were such that it took most, if not all of the week to get them done, so there was very little learning beyond just finishing the minimum requirements for the lab. In this course (and a few others), a different approach was taken, one that allowed the students to engage, experiment, and learn. The second half of the course was a project, which was really cool to work on. As it turned out, several of the students used that course as the basis for their master's thesis...one wrote a program that could discern 'dirty' images of six consecutive Cyrillic characters (i.e., something you'd seen in a satellite photo of Red Square, for example.) Another student created a neural network to assist with sonar identification.
So, how does all this matter? Well, 24+ years later, I can discern what's behind the terms "ML" and "AI" that we see with respect to cyber security products. ;-)
My time in grad school was also when I started brushing up against "information security" in the world of computers. During a C programming course, I finished my assigned labs and wanted to learn a bit more, so I downloaded a file called 'crack.c' to see what it did. All I ever did was open it in an editor, but the senior sysadmin for the department got upset. She even told me that I had "violated security policies". When I asked her to see the policies, knowing that I had never signed such a policy, I learned that there really was no written "policy". That was to change more than a year later when a new Admiral took over the school, but at the time, there was no written security policy that any students read or signed.
After I graduated, I spent 8 months processing out of the military, and during that time was assigned to the Marine detachment at the Defense Language Institute (DLI). While there, one of the things I did was get the detachment's computer systems connected to the DLI campus area network (CAN), which was token ring. Also during that time, the Commandant of the Marine Corps (Gen. Krulak) had stated that Marines were authorized to play "Marine DOOM"; the setup at the detachment was six Gateway systems connected via 10Base2, running IPX. I was able to use what I had learned just down the street (literally) to help get the "network" up and running.
Wednesday, August 28, 2019
DFIR Open Mic Night
So, here's a thought...
At a well-attended DFIR conference, there should be a DFIR open mic comedy night. In the evening, after the event is done for the day, use the venue for something a little light-hearted. For example, OSDFCon has had a mixer at the end of one of the days, where there've been finger foods and adult beverages, and there's a bar right there in the lobby. Since the venue already has chairs and a mic, why not use them?
So, Chatham House rules apply, as well as:
I think a lot of folks would enjoy something like this...it's a great way to engage, and provide some folks who maybe didn't get their talk accepted a chance to get up on stage and see how they do with public speaking.
After all, there are a LOT of weird or funny things that go on during an IR engagement. Some may not be funny at the time, but with a little embellishment ("don't let the facts get in the way of a good story...") some time later, they're freakin' hilarious! So why not share these with everyone else?
For example, I did an IR years ago with a global organization, one that had multiple tools available. We'd seen Mimikatz being run in the environment; in fact, the SOC (which was located in another country) had alerted the headquarters organization to this finding. A manager in charge of one the other tools was running searches for "mimikatz" across the infrastructure; unfortunately, one of the detections being used was, "any command line that includes "mimikatz"". This was intended to catch things like "Invoke-Mimikatz", but it caught everything else. The first couple of times this happened, the SOC would send an alert, and the local SOC manager (a guy about 6' 7", 275 lbs, bodybuilder type) would go nuts, telling (well, not "telling", per se...) us that the bad guy was back, while the veins in his head and neck were popping out.
So, not funny at the time, but something we can laugh about now...
Thoughts? Go? No go? Is this something you'd participate in, or just want to watch? Or watch until maybe you got your nerve up (liquid courage) and then got up on stage?
At a well-attended DFIR conference, there should be a DFIR open mic comedy night. In the evening, after the event is done for the day, use the venue for something a little light-hearted. For example, OSDFCon has had a mixer at the end of one of the days, where there've been finger foods and adult beverages, and there's a bar right there in the lobby. Since the venue already has chairs and a mic, why not use them?
So, Chatham House rules apply, as well as:
- No sales pitches
- No shaming anyone, any company or organization, product, etc.
- Use no names, unless you're sending praise/shout-outs
- Be cool - a little light profanity isn't an issue, but don't be vulgar or disgusting
I think a lot of folks would enjoy something like this...it's a great way to engage, and provide some folks who maybe didn't get their talk accepted a chance to get up on stage and see how they do with public speaking.
After all, there are a LOT of weird or funny things that go on during an IR engagement. Some may not be funny at the time, but with a little embellishment ("don't let the facts get in the way of a good story...") some time later, they're freakin' hilarious! So why not share these with everyone else?
For example, I did an IR years ago with a global organization, one that had multiple tools available. We'd seen Mimikatz being run in the environment; in fact, the SOC (which was located in another country) had alerted the headquarters organization to this finding. A manager in charge of one the other tools was running searches for "mimikatz" across the infrastructure; unfortunately, one of the detections being used was, "any command line that includes "mimikatz"". This was intended to catch things like "Invoke-Mimikatz", but it caught everything else. The first couple of times this happened, the SOC would send an alert, and the local SOC manager (a guy about 6' 7", 275 lbs, bodybuilder type) would go nuts, telling (well, not "telling", per se...) us that the bad guy was back, while the veins in his head and neck were popping out.
So, not funny at the time, but something we can laugh about now...
Thoughts? Go? No go? Is this something you'd participate in, or just want to watch? Or watch until maybe you got your nerve up (liquid courage) and then got up on stage?
Friday, August 16, 2019
Program Execution...Or Not
Over the years, different means have been used to discuss the DFIR analysis process, and one of those has been artifact categories. This is where categories are created and artifacts placed in the various columns, as they relate to those categories. One such example is the SANS IR poster, which provides a great visual reminder for folks looking to employ this approach. Honestly, it is a good way to approach analysis...looking at even a single system image, artifacts have grown over the years as the versions of Windows have progressed from XP to Win7, through to Win10, and as such, it benefits a large portion of the community to have a repeatable approach to analysis.
However, when using approaches such as this, we need to keep in mind the context of the category titles. Yes, the artifacts of "program execution" provide an indication of applications that were launched on a system, but what does that mean?
At this point, I can guess what you're thinking...wait, what?? And believe me, I get it. "Program execution" means exactly that...that a program was executed. End of story. But wait...is that what it really means?
Consider this for a second...someone unfamiliar with a program or application might "open" it on their first try, to see what it does. Command line tools, for example, often contain information about their usage, which we can see by either typing the name of the program at the command prompt (no arguments), or by adding "/?" or "-h" ("--help" for Linuxphiles). This causes the program to run, but not in the sense that the functionality of the program is actually used.
As an example, I opened a command prompt, and changed directories to the C:\Windows\Prefetch folder. Most analysts who've been in the industry for some time are familiar with the application prefetch files, often referred to as simply "Prefetch". Specifically, these are files are widely known as an artifact of "program execution".
I first typed "dir ftp*.pf" to see if there were any Prefetch files that appeared to point to the use of ftp.exe, and got the expected result: File Not Found. Next, I typed "ftp /?" at the prompt, which displayed the usage syntax of the application.
I then retyped (actually, I hit the up arrow twice...) the 'dir' command, and this time, I found that there was a file named FTP.EXE-7BA637EA.pf, which was 2,685 bytes in size.
So, what happened? I ran the program, but only to the point where I could read the usage syntax. I didn't actually use the program to transfer files or exfil data in any way. However, the artifacts of program execution were populated.
Now, the same thing applies to GUI applications, maybe even more so. You can launch a GUI application, look around at the interface, maybe click a few of the options to see what functionality is available, and then close the UI without ever having employed the functionality provided by the application.
Case in point...consider this analysis of the DefCon 2018 CTF file server image. Other publicly available write-ups addressed the question of interest (which application was used to delete forensic artifacts?) with various findings. One was the result of the itempos.pl RegRipper plugin; not an artifact normally associated with program execution, but rather that the application was resident on the desktop. The two other write-ups went with the UserAssist artifacts, widely associated with program execution; however, there was no verification that the application was actually used to, as stated in the CTF question, delete forensic artifacts. As such, the GUI application could have been launched, closed, and then something else could have been used to take the specified actions. In fact, the actions in question were never verified.
As such, something to consider going forward is, when artifacts of program execution are found, what do they really mean?
Finally, a question...there is a way to make use of the FTP protocol on Windows workstations (XP, 7, 8, 10) that does not leave the 'normal' artifacts of program execution (i.e., Prefetch file, UserAssist entry) that does not involve disabling any default functionality. What is it, and how would you determine/verify it?
Addendum, 18 Aug: So far, there's only been one attempt to answer the final question. I know that there's more out there...check the comments to see the answer, but there's at least one more, and maybe even more than one!
However, when using approaches such as this, we need to keep in mind the context of the category titles. Yes, the artifacts of "program execution" provide an indication of applications that were launched on a system, but what does that mean?
At this point, I can guess what you're thinking...wait, what?? And believe me, I get it. "Program execution" means exactly that...that a program was executed. End of story. But wait...is that what it really means?
Consider this for a second...someone unfamiliar with a program or application might "open" it on their first try, to see what it does. Command line tools, for example, often contain information about their usage, which we can see by either typing the name of the program at the command prompt (no arguments), or by adding "/?" or "-h" ("--help" for Linuxphiles). This causes the program to run, but not in the sense that the functionality of the program is actually used.
As an example, I opened a command prompt, and changed directories to the C:\Windows\Prefetch folder. Most analysts who've been in the industry for some time are familiar with the application prefetch files, often referred to as simply "Prefetch". Specifically, these are files are widely known as an artifact of "program execution".
I first typed "dir ftp*.pf" to see if there were any Prefetch files that appeared to point to the use of ftp.exe, and got the expected result: File Not Found. Next, I typed "ftp /?" at the prompt, which displayed the usage syntax of the application.
I then retyped (actually, I hit the up arrow twice...) the 'dir' command, and this time, I found that there was a file named FTP.EXE-7BA637EA.pf, which was 2,685 bytes in size.
So, what happened? I ran the program, but only to the point where I could read the usage syntax. I didn't actually use the program to transfer files or exfil data in any way. However, the artifacts of program execution were populated.
Now, the same thing applies to GUI applications, maybe even more so. You can launch a GUI application, look around at the interface, maybe click a few of the options to see what functionality is available, and then close the UI without ever having employed the functionality provided by the application.
Case in point...consider this analysis of the DefCon 2018 CTF file server image. Other publicly available write-ups addressed the question of interest (which application was used to delete forensic artifacts?) with various findings. One was the result of the itempos.pl RegRipper plugin; not an artifact normally associated with program execution, but rather that the application was resident on the desktop. The two other write-ups went with the UserAssist artifacts, widely associated with program execution; however, there was no verification that the application was actually used to, as stated in the CTF question, delete forensic artifacts. As such, the GUI application could have been launched, closed, and then something else could have been used to take the specified actions. In fact, the actions in question were never verified.
As such, something to consider going forward is, when artifacts of program execution are found, what do they really mean?
Finally, a question...there is a way to make use of the FTP protocol on Windows workstations (XP, 7, 8, 10) that does not leave the 'normal' artifacts of program execution (i.e., Prefetch file, UserAssist entry) that does not involve disabling any default functionality. What is it, and how would you determine/verify it?
Addendum, 18 Aug: So far, there's only been one attempt to answer the final question. I know that there's more out there...check the comments to see the answer, but there's at least one more, and maybe even more than one!
Chasing the DFIR Cure, pt II
Following my first post on this topic, an interesting comment was shared that I thought would really benefit the discussion, as well as benefit from a further look.
To paraphrase, the comment was along the lines of,"...how do you justify the additional cost of a second (or third) look when the results are coming out the same?"
In my experience, that hasn't been an issue.
First, when someone has decided that additional eyes on the data is a justified step to take, the value is already understood, and the cost-benefit analysis has already been done. Take the pro bono case I mentioned in my previous post; in that case, the value was understood prior to the attorney contacting me, and the decision was made after contact and initial discussions/scoping to do the work pro bono.
Second, while I have been aware of and party to "second looks", in my experience, the results are rarely the same as the "first look". The comment above assumes that the results would be the same, but I simply haven't found that to be the case.
I did some pro bono work (different case from the one previously mentioned) involving someone leaving a firm, and "salting the earth" on their way out. What I mean by this is that someone submitted their letter of resignation, and after they left the building, it was found that their system was infected with ransomware. In this case, the system they were assigned was "communal", in that there was a critical application installed on the system, with just one license, so the CEO needed to have access to it at all times. My analysis was actually the third look, and I was able to demonstrate that the evening prior to the "incident", someone had logged in at 9pm and browsed the web for about 6 minutes. During that time, the system became infected, and a persistence mechanism was established, which led to the ransomware launching when the user (not the CEO) logged in the next morning and wrote out their letter of resignation. The case was thrown out, and a counter suit for libel went forward, based on my report.
This is just one example, but the findings were different enough from the law enforcement officer's report and another consultant's report that the outcome was significantly impacted, and the direction of the case altered.
To paraphrase, the comment was along the lines of,"...how do you justify the additional cost of a second (or third) look when the results are coming out the same?"
In my experience, that hasn't been an issue.
First, when someone has decided that additional eyes on the data is a justified step to take, the value is already understood, and the cost-benefit analysis has already been done. Take the pro bono case I mentioned in my previous post; in that case, the value was understood prior to the attorney contacting me, and the decision was made after contact and initial discussions/scoping to do the work pro bono.
Second, while I have been aware of and party to "second looks", in my experience, the results are rarely the same as the "first look". The comment above assumes that the results would be the same, but I simply haven't found that to be the case.
I did some pro bono work (different case from the one previously mentioned) involving someone leaving a firm, and "salting the earth" on their way out. What I mean by this is that someone submitted their letter of resignation, and after they left the building, it was found that their system was infected with ransomware. In this case, the system they were assigned was "communal", in that there was a critical application installed on the system, with just one license, so the CEO needed to have access to it at all times. My analysis was actually the third look, and I was able to demonstrate that the evening prior to the "incident", someone had logged in at 9pm and browsed the web for about 6 minutes. During that time, the system became infected, and a persistence mechanism was established, which led to the ransomware launching when the user (not the CEO) logged in the next morning and wrote out their letter of resignation. The case was thrown out, and a counter suit for libel went forward, based on my report.
This is just one example, but the findings were different enough from the law enforcement officer's report and another consultant's report that the outcome was significantly impacted, and the direction of the case altered.
Tuesday, August 13, 2019
Chasing the DFIR Cure
I've wondered for some time now, how do "customers" (recipients of DFIR services) determine the quality and accuracy of the rendered product? Throughout my time in the industry, I've known some customers to "seek a second opinion", but is this something that's really pervasive?
I was watching a new medical mystery show recently called "Chasing the Cure". This show is all about folks with severe, debilitating illnesses that have gone un- or mis-diagnosed for extended periods of time. This subject is near and dear to me because about 25 years ago, I went through a similar situation, although the duration of my issue was a few months, rather than years. In one of the show segments, a woman was suffering from a debilitating ailment that had gone misdiagnosed for several years, and she was able to receive a confident diagnosis on the show (i.e., PCOS). What I found interesting was that the doctors who made the diagnosis (on the show) were looking at the results of tests that had been ordered by previous doctors, meaning that several medical professionals had the same data in front of them, and in the case, the woman had been told by previous doctors that did not have the condition for which she was finally diagnosed. So, this was not just a matter of two (or more) professionals having the same data and arriving at different conclusions, as much as it was about a professional in the field specifically discounting a diagnosis. The result was that the woman and her family suffered for several more years, where she could have received treatment much earlier. However, not being satisfied with the answer she was given led her to continue seeking a diagnosis.
That got me to thinking...how do those who contract for DFIR services know if the analysis and findings they're receiving are correct and accurate? Sure, if the findings don't go their way, they can seek a second (or third) opinion, but how do to they know that what they receive is correct?
Several years ago, I was contacted by an attorney who had a case that involved a person being in a specific location based on computer evidence. In short, the case had to do with someone claiming that they were at a convenience store, and this attorney's case would be held up if that person had been behind the computer keyboard. The attorney had first 'contracted' with a part-time IT sysadmin who serviced their office to conduct analysis of the computer data, and the sysadmin had reportedly found evidence that the person had, in fact, been behind the keyboard. The attorney asked me to confirm the finding, which I was able to do. However,
The question, "...are these findings correct?" ever asked? Does it matter? I believe it does, and I also believe that there are a number of circumstances where it may behoove a customer/recipient to seek a second opinion:
PCI Investigations - the "window of compromise" is a variable in the "potentially how many credit card numbers were compromised" equation. This then leads to corrective or punitive actions, such as fines, and as such, is significantly impacted by the findings from the investigation. For example, misinterpreting the time stamp associated with the AppCompatCache data can extend the "window of compromise" from weeks to years, and severely impact the merchant, who receives a much greater fine.
Compliance - this goes back to things like PCI investigations, as during the course of analysis, the analyst may find that the merchant was not compliant with the PCI DSS (or standards set by another regulatory body) at the time of the breach.
Cyber Insurance - the results of an investigation can significantly impact the results of a claim; issues with data collection and interpretation may lead to findings that indicate considerable gaps in "due diligence", and claims may not be paid.
HR - findings as a result of a DFIR investigation in support of HR can significantly impact the employee, or the company. Misinterpretations of the data may lead to an employee being unjustly accused or dismissed; I worked a pro bono case to this effect several years ago.
Ransomware - something we see reported quite often in the media is that "...there was no evidence of data exfiltration found...", and that's a good thing which fits the desired narrative. But is it correct? Were the DFIR analysts aware of the artifact locations within Windows systems that might provide a different view of that answer? After all, the actors behind Samas and Ryuk ransomware deployments have been observed spending months (yes, I did spell that correctly...) within an infrastructure before deploying the ransomware, so...yeah...
I'm not suggesting that the industry is rampant with errors in data collection and interpretation, not at all. There are a lot of great analysts out there doing a lot of great work, and providing accurate results and findings to their customers. However, like any industry, these things do happen, and like other situations, when they happen they can have a serious impact. We also have to look at the fact that operating systems are getting more sophisticated all the time, and applications are flourishing and getting more numerous. This is all to say that things are much more complex than they were 20 years ago, and with the number of people coming into the DFIR field, how do we keep up on keeping everyone at a common level of knowledge?
Another aspect of the industry that I've seen change over time is the use of collection, parsing, and pre-processing frameworks. When I started out in the industry, even if a DFIR analyst collected a dozen or more images, they analyzed those images themselves. Over time, as there's been a move to cover and address the enterprise, there's been a subsequent increase in the amount of available data. As such, in a move establish a level of consistency, a lot of DFIR teams have developed means for the enterprise-level collection and pre-processing of data. All of this can add an additional layer of abstraction between the data and the analyst.
I'm also fully aware that over time, we learn things. I was talking to Brett Shavers recently, and he brought up the scenario of going back and looking at previous cases. Like Brett, when I've done this, I've marveled at how far I've come since that case; what are some of the things I've learned since then that I could apply to the case if I were to address the issue today?
I would think, then, that without some compelling reason, most who purchase DFIR services accept the findings they receive as correct and accurate. In this age of legal and regulatory requirements that both impact and depend on the results of DFIR analysis, the correct and accurate collection and interpretation of digital data is paramount, and there are a number of cases where the "customer" may benefit from a second, or even a third opinion. After all, we do this with medical issues, don't we?
To that point, that 'compelling reason' would likely consist of findings that are markedly contrary and contradictory to the desired narrative. There is likely a 'threshold' that some may accept; for example, consider the PCI example above...there are likely merchants who receive information about those findings and are able to absorb whatever judgement is levied by the bank or the PCI Council. However, there are also those merchants for whom the judgement is what I've referred to as a "cratering fine"; that is, once the fine is levied, the business (a small mom-and-pop restaurant, for example) ceases to exist. I've seen this happen. In such cases, given what's at stake, it may behoove the merchant to seek a second opinion.
I was watching a new medical mystery show recently called "Chasing the Cure". This show is all about folks with severe, debilitating illnesses that have gone un- or mis-diagnosed for extended periods of time. This subject is near and dear to me because about 25 years ago, I went through a similar situation, although the duration of my issue was a few months, rather than years. In one of the show segments, a woman was suffering from a debilitating ailment that had gone misdiagnosed for several years, and she was able to receive a confident diagnosis on the show (i.e., PCOS). What I found interesting was that the doctors who made the diagnosis (on the show) were looking at the results of tests that had been ordered by previous doctors, meaning that several medical professionals had the same data in front of them, and in the case, the woman had been told by previous doctors that did not have the condition for which she was finally diagnosed. So, this was not just a matter of two (or more) professionals having the same data and arriving at different conclusions, as much as it was about a professional in the field specifically discounting a diagnosis. The result was that the woman and her family suffered for several more years, where she could have received treatment much earlier. However, not being satisfied with the answer she was given led her to continue seeking a diagnosis.
That got me to thinking...how do those who contract for DFIR services know if the analysis and findings they're receiving are correct and accurate? Sure, if the findings don't go their way, they can seek a second (or third) opinion, but how do to they know that what they receive is correct?
Several years ago, I was contacted by an attorney who had a case that involved a person being in a specific location based on computer evidence. In short, the case had to do with someone claiming that they were at a convenience store, and this attorney's case would be held up if that person had been behind the computer keyboard. The attorney had first 'contracted' with a part-time IT sysadmin who serviced their office to conduct analysis of the computer data, and the sysadmin had reportedly found evidence that the person had, in fact, been behind the keyboard. The attorney asked me to confirm the finding, which I was able to do. However,
The question, "...are these findings correct?" ever asked? Does it matter? I believe it does, and I also believe that there are a number of circumstances where it may behoove a customer/recipient to seek a second opinion:
PCI Investigations - the "window of compromise" is a variable in the "potentially how many credit card numbers were compromised" equation. This then leads to corrective or punitive actions, such as fines, and as such, is significantly impacted by the findings from the investigation. For example, misinterpreting the time stamp associated with the AppCompatCache data can extend the "window of compromise" from weeks to years, and severely impact the merchant, who receives a much greater fine.
Compliance - this goes back to things like PCI investigations, as during the course of analysis, the analyst may find that the merchant was not compliant with the PCI DSS (or standards set by another regulatory body) at the time of the breach.
Cyber Insurance - the results of an investigation can significantly impact the results of a claim; issues with data collection and interpretation may lead to findings that indicate considerable gaps in "due diligence", and claims may not be paid.
HR - findings as a result of a DFIR investigation in support of HR can significantly impact the employee, or the company. Misinterpretations of the data may lead to an employee being unjustly accused or dismissed; I worked a pro bono case to this effect several years ago.
Ransomware - something we see reported quite often in the media is that "...there was no evidence of data exfiltration found...", and that's a good thing which fits the desired narrative. But is it correct? Were the DFIR analysts aware of the artifact locations within Windows systems that might provide a different view of that answer? After all, the actors behind Samas and Ryuk ransomware deployments have been observed spending months (yes, I did spell that correctly...) within an infrastructure before deploying the ransomware, so...yeah...
I'm not suggesting that the industry is rampant with errors in data collection and interpretation, not at all. There are a lot of great analysts out there doing a lot of great work, and providing accurate results and findings to their customers. However, like any industry, these things do happen, and like other situations, when they happen they can have a serious impact. We also have to look at the fact that operating systems are getting more sophisticated all the time, and applications are flourishing and getting more numerous. This is all to say that things are much more complex than they were 20 years ago, and with the number of people coming into the DFIR field, how do we keep up on keeping everyone at a common level of knowledge?
Another aspect of the industry that I've seen change over time is the use of collection, parsing, and pre-processing frameworks. When I started out in the industry, even if a DFIR analyst collected a dozen or more images, they analyzed those images themselves. Over time, as there's been a move to cover and address the enterprise, there's been a subsequent increase in the amount of available data. As such, in a move establish a level of consistency, a lot of DFIR teams have developed means for the enterprise-level collection and pre-processing of data. All of this can add an additional layer of abstraction between the data and the analyst.
I'm also fully aware that over time, we learn things. I was talking to Brett Shavers recently, and he brought up the scenario of going back and looking at previous cases. Like Brett, when I've done this, I've marveled at how far I've come since that case; what are some of the things I've learned since then that I could apply to the case if I were to address the issue today?
I would think, then, that without some compelling reason, most who purchase DFIR services accept the findings they receive as correct and accurate. In this age of legal and regulatory requirements that both impact and depend on the results of DFIR analysis, the correct and accurate collection and interpretation of digital data is paramount, and there are a number of cases where the "customer" may benefit from a second, or even a third opinion. After all, we do this with medical issues, don't we?
To that point, that 'compelling reason' would likely consist of findings that are markedly contrary and contradictory to the desired narrative. There is likely a 'threshold' that some may accept; for example, consider the PCI example above...there are likely merchants who receive information about those findings and are able to absorb whatever judgement is levied by the bank or the PCI Council. However, there are also those merchants for whom the judgement is what I've referred to as a "cratering fine"; that is, once the fine is levied, the business (a small mom-and-pop restaurant, for example) ceases to exist. I've seen this happen. In such cases, given what's at stake, it may behoove the merchant to seek a second opinion.
Thursday, July 04, 2019
IWS Review
A while back, I sent a signed copy of IWS to someone who was willing to write a review, and they've gone about trying to post their review with some difficulty, albeit not for a lack of trying on their part. I greatly appreciate the effort, not just in reading and engaging with the book, and writing a review, but also trying to get it posted and shared! So much so, in fact, that I offered to host the review here.
While the book hasn't been out for a full year yet, I'm still just as nervous as I was the first day, regarding how the book will be viewed by the community. This book is a pretty radical departure from my previous books, as well as from any other DFIR book I could find. As such, with something this new (I had pretty much the same feelings regarding Windows Registry Forensics...), I greatly appreciate hearing how folks found the book, as well as if they were able to get anything from it...did it provide value?
Thanks, Dmitri, for taking the time read the book, and for putting in the additional effort to write down and share your thoughts!
So...a review, by Dmitri...
Investigating Windows Systems - Review
I've read several books by Harlan, and I've never been disappointed. I love his direct way of writing. IWS is thinner and smaller than his other books but no less important, on the contrary.
Harlan writes that IWS is not for beginners, I still see myself as a beginner and should contradict Harlan here, also IWS is a book that is important, or may be, for any beginner, although some pieces in the book are not so easy with an effort of the reader and a search on the Internet everything becomes understandable.
The book is well organized. It teaches you from the beginning that a good analysis plan is important. It teaches you to focus between 'nice to know' and 'need to know'.
The book is divided into several cases (finding malware, user activity, web server compromise). Harlan explains to you how he would deal with these cases himself, and then teaches you how to make a self-reflection. What did you learn from your case, and how would you tackle it next time?
The book is not about the analysis of images themselves, nor about which tools you should use, but about how you should do the analysis, what plan you make. He teaches you to make the difference between a targeted approach and an automated approach.
In the last part, Harlan will teach you how to set up a testing environment, and convince you that testing changes in the file system yourself by deleting files, installing programs, …, is often more instructive than just asking for help on the net.
I really enjoyed the book.
Dimitri Deryckere
1st Inspector
Computer Crime Unit – Pz Regio Tielt - Belgium
While the book hasn't been out for a full year yet, I'm still just as nervous as I was the first day, regarding how the book will be viewed by the community. This book is a pretty radical departure from my previous books, as well as from any other DFIR book I could find. As such, with something this new (I had pretty much the same feelings regarding Windows Registry Forensics...), I greatly appreciate hearing how folks found the book, as well as if they were able to get anything from it...did it provide value?
Thanks, Dmitri, for taking the time read the book, and for putting in the additional effort to write down and share your thoughts!
So...a review, by Dmitri...
Investigating Windows Systems - Review
I've read several books by Harlan, and I've never been disappointed. I love his direct way of writing. IWS is thinner and smaller than his other books but no less important, on the contrary.
Harlan writes that IWS is not for beginners, I still see myself as a beginner and should contradict Harlan here, also IWS is a book that is important, or may be, for any beginner, although some pieces in the book are not so easy with an effort of the reader and a search on the Internet everything becomes understandable.
The book is well organized. It teaches you from the beginning that a good analysis plan is important. It teaches you to focus between 'nice to know' and 'need to know'.
The book is divided into several cases (finding malware, user activity, web server compromise). Harlan explains to you how he would deal with these cases himself, and then teaches you how to make a self-reflection. What did you learn from your case, and how would you tackle it next time?
The book is not about the analysis of images themselves, nor about which tools you should use, but about how you should do the analysis, what plan you make. He teaches you to make the difference between a targeted approach and an automated approach.
In the last part, Harlan will teach you how to set up a testing environment, and convince you that testing changes in the file system yourself by deleting files, installing programs, …, is often more instructive than just asking for help on the net.
I really enjoyed the book.
Dimitri Deryckere
1st Inspector
Computer Crime Unit – Pz Regio Tielt - Belgium
Tuesday, June 04, 2019
What's New
A New Blogger
I ran across a fascinating blog post during my Sunday morning reading, one that was interesting in a number of ways. First off, this is apparently Charity's first blog post...so, congrats! Thank you for stepping up and sharing your insight. It is refreshing and oh-so-important for those newer to the field to speak up and share their perspective, as those who've been around for a while may find this perspective valuable; I know I do.
I don't know Charity, we've never met. I simply ran across a tweet announcing her first blog post, and thought, cool...I'll take a look. I like to see what folks are bringing to the table, in particular due to the fact that when I was in college and 'coming up' in my career, there were no resources such as college programs for infosec or "the cybers". It's been fascinating to watch the evolution of the field over time, and to see folks like Charity bring a new perspective to the field.
I took martial arts training for a bit while I was in junior high school, and then again during my first deployment to Okinawa. In both instances, sparring was involved...when I was studying shorin ryu, a good bit of the sparring came during testing before the grandmaster. One of the take-aways for me from that sparring was that while there are rules for such things, they only apply to those who follow them. Admittedly, I did learn this truism while playing soccer and wrestling in high school, but I think it really became a bit more obvious while I was sparring. For example, we didn't wear head gear during the sparring, because one of the rules was that the head was not a target. I seem to remember that this was a test of your understanding and use of the technique, as well as self-control, under controlled circumstances. During my training sessions prior to testing, everyone adhered to that rule...likely because we were all from the same unit. However, during the testing, there were athletes from different locations on the island, and as such, you never knew how they trained. During one sparring session, the first shot my opponent took was directly to my head.
The point is that as an IR consultant, you may find yourself sparring not only with the threat actor with whom you are engaged, but also with the culture of the folks you are assisting. Agreed upon rules, such as not forcing the password change until everyone is ready, only works if everyone who agrees to them actually adheres to and follows them.
To one of Charity's points, during IR engagements, I also got to see how those I was working with reacted to stress, and very often, my first goal was to bring down that stress level so that we could move and respond in more of a coordinated fashion, and less based on adrenaline. During one IR engagement, I arrived on-site at 10:30pm on a Friday night, knowing that the team had already been working 20+ hr days for the last 10 days or so. My first action was to get everyone to go home; it took some time, but we got the last person out the door by 11:30pm. Even the following day, within the first few hours, some of the folks at the site were having trouble typing commands, and would click the wrong buttons in a UI. Too many mistakes were being made, making simple actions take far too long to accomplish. Having everyone get some rest, and then focusing their efforts once we were back in the fight got us to the point where we were making some headway, and once we were able to build up some forward momentum, the stress levels started to come down.
Also, it makes me wonder if Charity and Lesley or Richard have had a chance to meet yet.
Roll-Ups
Roll-ups are a great way to get an overview of what happened during a given week or month, and to maybe see something that you might have missed. ThisWeekIn4n6 is a great resource for just this purpose, but like other such roll-ups and consolidations, the descriptions of various articles or events can sometimes fall short of the mark.
Okay, before we go any further, I am not knocking the effort that goes into producing a roll-up like this every week, along with a monthly overview. Not at all. It's a lot of work and greatly appreciated, as I know I've seen things in the weekly roll-up that I would not have seen otherwise. As such, I do not expect a full-blown review of each of the resources listed; however, I will say that I'm no different from anyone else, in that I may not look at something that contains some real solid information, based on the description.
One example is Ali's recent blog post titled Creating a Hidden Prefetch File to Bypass Normal Forensic Analysis. Ali addresses what happens on a system when an executable is launched from within an ADS, and the effect that may have on "normal" DF analysis. I was interested in the article because (a) Ali is a friend, and (b) I've been interested in ADSs since about 1998. As such, I was enticed to read the full content of the article, and I found value in it.
In the roll-up description, there is the statement, "This prefetch file was not detected by forensic tools". Should it have been? Most tools, particularly the commercial ones, present data on which the examiner then needs to perform analysis; however, is it incumbent upon a tool to 'detect' this sort of thing?
As another example, a co-worker and I had a blog post published on our corporate web site recently; the roll-up description simply states that we "looked at increased TrickBot activity from GRIM SPIDER.". Really, it is SO much more than that! The purpose of the article that Eric and I worked on (Eric did the lion's share of the work) was to stand on the shoulders of the CS Intel team and illustrate what was seen "on the ground" across a number of IR engagements. Yes, the article started out mentioning the described increase in activity, in order to tie it to the previous blog post from our Intel team. However, it then extended that information by incorporating what was observed through incident response, given the aperture and collection bias of the IR team. Not only was this deep dive discussed in the article, but we wanted to make it easier to digest by presenting a table of observables (table 1 in the article) by mapping MITRE ATT&CK tactics and techniques to the actual data from the engagement, to illustrate what the technique "looks like".
My point is simply this...sometimes the description of an article isn't inline with the content, and sometimes it isn't even enticing. However, there is a lot of great content out there, content that shouldn't be passed over because of a synopsis that perhaps doesn't generate interest.
LNK Files
Huntress Labs recently posted an article regarding an LNK file they'd seen, and the deep dive that they took into the LNK file and the PowerShell that it launched. While the article includes 'deep dive' in the title, there is no mention of LNK file metadata. Now, this may be due to the fact that the LNK file was created on, rather than sent to, the target system, but there isn't enough detail within the article for a reader to know.
The technique described, while very interesting, is not new. I mentioned Felix's post regarding booby-trapped LNK files. It is fascinating to see other examples of how this technique is used, and it would be very interesting to see some of the other TTPs surrounding this particular use of the technique, for context. Also, it would be very interesting to see what this 'looks like' with respect to EDR tooling...I suspect that the parent process would be the Windows Explorer shell, shown launching PowerShell, with a child process of mshta.exe; however, there doesn't seem to be any information regarding where the file was found with in the file system, how it is launched, etc.
I think that this was a great dive into the technique used, and I enjoy reading technical analyses such as this, as they show the lengths that an adversary will go to in order to avoid detection, as well as buy themselves some time as these things are unraveled. However, in this particular case, there's a good bit that isn't said; for example, if the LNK file was created on the system where it was found, then one can assume that there was a bit of a 'noisy' process to create it. After all, this isn't something that you can create directly via the API. Yes, you can get it started via the API, but additional tooling is required to append data to the end of the LNK file AND include a command in the LNK file to read that data; this isn't 'normal'. Yes, it can be accomplished via scripting, but that script would have had to have been copied over and executed, TTPs which can be detected. If the LNK file was created on another system, such as the bad guy's system, then there is likely metadata in the file that can tell us about that system, as well as toolmarks that can tell us something about how the LNK file was created.
The folks at Huntress Labs did a great job of unraveling the LNK file, but for whatever reason, a great deal was left unsaid. I, for one, would be very interested in knowing more about the file, as there is undoubtedly valuable information that can be used to develop threat intelligence.
Speaking of unraveling LNK files, the folks at JPCERT/CC recently blogged about weaponized LNK files being used against targets. This one is interesting due to the fact that targeted users receive an email that contains a link to the weaponized LNK file, which is hosted in a cloud service. As such, the LNK file is not an attachment to the email, nor is it in a zipped archive that is attached to the email; it's downloaded to the target system and executed after the user clicks a link.
Unfortunately, the article only gives an overview of the LNK file; there are some aspects of this issue that bear closer examination. I understand that some things may have been presented from a high-level view due to the sensitivity of the case being worked, translating to English, etc. However, based on prior experience, there may be go deal of intelligence that can be gleaned from the samples.
I did a search for several (albeit not all) of the hashes listed, and could not find samples. I think that there may be something interesting there, if we're able to look at the full metadata of the LNK files, especially if it's plotted across time (when the phishing emails were sent) or campaigns. I'd like to see more, in part because I think that there's more to see...
I ran across a fascinating blog post during my Sunday morning reading, one that was interesting in a number of ways. First off, this is apparently Charity's first blog post...so, congrats! Thank you for stepping up and sharing your insight. It is refreshing and oh-so-important for those newer to the field to speak up and share their perspective, as those who've been around for a while may find this perspective valuable; I know I do.
I don't know Charity, we've never met. I simply ran across a tweet announcing her first blog post, and thought, cool...I'll take a look. I like to see what folks are bringing to the table, in particular due to the fact that when I was in college and 'coming up' in my career, there were no resources such as college programs for infosec or "the cybers". It's been fascinating to watch the evolution of the field over time, and to see folks like Charity bring a new perspective to the field.
I took martial arts training for a bit while I was in junior high school, and then again during my first deployment to Okinawa. In both instances, sparring was involved...when I was studying shorin ryu, a good bit of the sparring came during testing before the grandmaster. One of the take-aways for me from that sparring was that while there are rules for such things, they only apply to those who follow them. Admittedly, I did learn this truism while playing soccer and wrestling in high school, but I think it really became a bit more obvious while I was sparring. For example, we didn't wear head gear during the sparring, because one of the rules was that the head was not a target. I seem to remember that this was a test of your understanding and use of the technique, as well as self-control, under controlled circumstances. During my training sessions prior to testing, everyone adhered to that rule...likely because we were all from the same unit. However, during the testing, there were athletes from different locations on the island, and as such, you never knew how they trained. During one sparring session, the first shot my opponent took was directly to my head.
The point is that as an IR consultant, you may find yourself sparring not only with the threat actor with whom you are engaged, but also with the culture of the folks you are assisting. Agreed upon rules, such as not forcing the password change until everyone is ready, only works if everyone who agrees to them actually adheres to and follows them.
To one of Charity's points, during IR engagements, I also got to see how those I was working with reacted to stress, and very often, my first goal was to bring down that stress level so that we could move and respond in more of a coordinated fashion, and less based on adrenaline. During one IR engagement, I arrived on-site at 10:30pm on a Friday night, knowing that the team had already been working 20+ hr days for the last 10 days or so. My first action was to get everyone to go home; it took some time, but we got the last person out the door by 11:30pm. Even the following day, within the first few hours, some of the folks at the site were having trouble typing commands, and would click the wrong buttons in a UI. Too many mistakes were being made, making simple actions take far too long to accomplish. Having everyone get some rest, and then focusing their efforts once we were back in the fight got us to the point where we were making some headway, and once we were able to build up some forward momentum, the stress levels started to come down.
Also, it makes me wonder if Charity and Lesley or Richard have had a chance to meet yet.
Roll-Ups
Roll-ups are a great way to get an overview of what happened during a given week or month, and to maybe see something that you might have missed. ThisWeekIn4n6 is a great resource for just this purpose, but like other such roll-ups and consolidations, the descriptions of various articles or events can sometimes fall short of the mark.
Okay, before we go any further, I am not knocking the effort that goes into producing a roll-up like this every week, along with a monthly overview. Not at all. It's a lot of work and greatly appreciated, as I know I've seen things in the weekly roll-up that I would not have seen otherwise. As such, I do not expect a full-blown review of each of the resources listed; however, I will say that I'm no different from anyone else, in that I may not look at something that contains some real solid information, based on the description.
One example is Ali's recent blog post titled Creating a Hidden Prefetch File to Bypass Normal Forensic Analysis. Ali addresses what happens on a system when an executable is launched from within an ADS, and the effect that may have on "normal" DF analysis. I was interested in the article because (a) Ali is a friend, and (b) I've been interested in ADSs since about 1998. As such, I was enticed to read the full content of the article, and I found value in it.
In the roll-up description, there is the statement, "This prefetch file was not detected by forensic tools". Should it have been? Most tools, particularly the commercial ones, present data on which the examiner then needs to perform analysis; however, is it incumbent upon a tool to 'detect' this sort of thing?
As another example, a co-worker and I had a blog post published on our corporate web site recently; the roll-up description simply states that we "looked at increased TrickBot activity from GRIM SPIDER.". Really, it is SO much more than that! The purpose of the article that Eric and I worked on (Eric did the lion's share of the work) was to stand on the shoulders of the CS Intel team and illustrate what was seen "on the ground" across a number of IR engagements. Yes, the article started out mentioning the described increase in activity, in order to tie it to the previous blog post from our Intel team. However, it then extended that information by incorporating what was observed through incident response, given the aperture and collection bias of the IR team. Not only was this deep dive discussed in the article, but we wanted to make it easier to digest by presenting a table of observables (table 1 in the article) by mapping MITRE ATT&CK tactics and techniques to the actual data from the engagement, to illustrate what the technique "looks like".
My point is simply this...sometimes the description of an article isn't inline with the content, and sometimes it isn't even enticing. However, there is a lot of great content out there, content that shouldn't be passed over because of a synopsis that perhaps doesn't generate interest.
LNK Files
Huntress Labs recently posted an article regarding an LNK file they'd seen, and the deep dive that they took into the LNK file and the PowerShell that it launched. While the article includes 'deep dive' in the title, there is no mention of LNK file metadata. Now, this may be due to the fact that the LNK file was created on, rather than sent to, the target system, but there isn't enough detail within the article for a reader to know.
The technique described, while very interesting, is not new. I mentioned Felix's post regarding booby-trapped LNK files. It is fascinating to see other examples of how this technique is used, and it would be very interesting to see some of the other TTPs surrounding this particular use of the technique, for context. Also, it would be very interesting to see what this 'looks like' with respect to EDR tooling...I suspect that the parent process would be the Windows Explorer shell, shown launching PowerShell, with a child process of mshta.exe; however, there doesn't seem to be any information regarding where the file was found with in the file system, how it is launched, etc.
I think that this was a great dive into the technique used, and I enjoy reading technical analyses such as this, as they show the lengths that an adversary will go to in order to avoid detection, as well as buy themselves some time as these things are unraveled. However, in this particular case, there's a good bit that isn't said; for example, if the LNK file was created on the system where it was found, then one can assume that there was a bit of a 'noisy' process to create it. After all, this isn't something that you can create directly via the API. Yes, you can get it started via the API, but additional tooling is required to append data to the end of the LNK file AND include a command in the LNK file to read that data; this isn't 'normal'. Yes, it can be accomplished via scripting, but that script would have had to have been copied over and executed, TTPs which can be detected. If the LNK file was created on another system, such as the bad guy's system, then there is likely metadata in the file that can tell us about that system, as well as toolmarks that can tell us something about how the LNK file was created.
The folks at Huntress Labs did a great job of unraveling the LNK file, but for whatever reason, a great deal was left unsaid. I, for one, would be very interested in knowing more about the file, as there is undoubtedly valuable information that can be used to develop threat intelligence.
Speaking of unraveling LNK files, the folks at JPCERT/CC recently blogged about weaponized LNK files being used against targets. This one is interesting due to the fact that targeted users receive an email that contains a link to the weaponized LNK file, which is hosted in a cloud service. As such, the LNK file is not an attachment to the email, nor is it in a zipped archive that is attached to the email; it's downloaded to the target system and executed after the user clicks a link.
Unfortunately, the article only gives an overview of the LNK file; there are some aspects of this issue that bear closer examination. I understand that some things may have been presented from a high-level view due to the sensitivity of the case being worked, translating to English, etc. However, based on prior experience, there may be go deal of intelligence that can be gleaned from the samples.
I did a search for several (albeit not all) of the hashes listed, and could not find samples. I think that there may be something interesting there, if we're able to look at the full metadata of the LNK files, especially if it's plotted across time (when the phishing emails were sent) or campaigns. I'd like to see more, in part because I think that there's more to see...
Wednesday, May 29, 2019
Troubleshooting and Deep Knowledge
What do we do when a tool that we're using doesn't return what we expect to see?
How often does this happen? How often do we run a tool in the course of our DFIR investigation process and for some reason, we don't see something in the output of the tool that we thought we would see? How often do we look for something, only to find that it's not there? More importantly, what do we do about it?
I can't speak for anyone else, but I tend to want to figure out why, particularly if the data I'm trying to parse is important to my overall analysis, based on my goals. If it's just a curiosity, I'll leave it until later, but if the data itself is paramount to the analysis, I generally tend to want to know why I'm seeing (or not, as the case may be) what I'm seeing.
Several years ago, while I was QSA certified and working on PCI cases (sssshhhh...don't let that little secret out...), we ran across a case where we knew that two specific brands of credit cards were being used at the site, but for some reason, our scans were coming up empty for those specific brands. We were getting lots of search results for the major card brands, but nothing for these two brands that we expected to see. We started peeling back the layers of the process, and go to the point where we were running scans of just straight text files that contained test data (several of the brands provided a small number of "valid" but unassigned credit card numbers), but we were still coming up empty on the brands. As we investigated further, we narrowed the issue down to a built-in API function, and determined that it was not, in fact, returning "valid" credit cards numbers. Rather, it was, just not all of them; it did not consider these two brands as 'valid' for some reason. We got some programming assistance and built our own replacement function, one that was a bit slower but more complete.
The moral of the story is that we looked at the components involved...the data, the tool, and the examiner...and worked methodically to figure out where the issue lay. Was it an assumption on our part, that these two brands were actually being used at the site? How did we know? Was it an assumption, or was it information that came from a specific, identifiable source? What about the data? Was there an issue with the data? Or was it the tool?
As it turned out, it was an issue of deep knowledge. We knew what the data should look like, and we had an understanding of where the data should reside. As a bit of a follow-on to my post on Deep Knowledge in DFIR work, I wanted to take a few moments to discuss an issue I ran into with the DefCon 2018 CTF File Server image, and what I did to address it.
I was reviewing some of the data, and I wanted to check and see if I could see a reference to the counter-forensics tool in the AppCompatCache data; as such, I ran the RegRipper appcompatcache.pl plugin, and saw the following:
Launching appcompatcache v.20190112
appcompatcache v.20190112
(System) Parse files from System hive AppCompatCache
Signature: 0x0
That was it...nothing else. No listing of file paths and names, no time stamps...nada.
Okay, so...now what? Do I throw up my hands and assume that the tool doesn't work?
No, not at all. I start troubleshooting the issue.
As I mentioned, there are three components to this process...the data, the tool, and the operator. Any one of the three could have an "issue". It's entirely possible that the tool does not function properly. For example, when looking at a Registry hive via a viewer, I like to use the MiTeC Windows Registry Recovery (WRR) tool, even though I know that the tool has a deficiency. Specifically, for value data over a certain size, a different data type is used to store that data (similar to non-resident files within the MFT using a run list), and WRR does not handle those data types. The AppCompatCache data is exactly that data type. However, I could navigate through the Registry structure to the key in question, even though I could not directly view the raw data value via WRR.
First step...check the tool. Is the plugin working correctly? It works correctly against the System hive from the HR server, but that's a different version of Windows. I don't have a System hive available that matches the Windows version from the file server image, but for the most part, the plugins seems to be working correctly.
RegRipper plugins are Perl scripts, which means that they're essentially text files. Opening the appcompatcache.pl plugin in Notepad++, we see that lines 96 and 97 are commented out (that is, the lines start with "#"):
# ::rptMsg("Length of data: ".length($app_data));
# probe($app_data);
This is essentially some troubleshooting code; when it's not commented out, it will tell me the length of the data, and then print a hex dump of the data (via the probe() function).
Assuming that the issue may have been with the plugin, I removed those "#" symbols, saved the file, and re-ran the plugin in hopes that I'd get some more information. For example, if the plugin wasn't parsing the data correctly, I'd see a line telling me the length of the data, and then a hex dump of the data itself. However, when I ran the plugin, I saw simply "Length of data:" and nothing else; no length value, no hex dump.
Next step...check the data itself. I opened the hive in WRR, and went to the Control\Session Manager key to check the AppCompatibility subkey (the path is listed in the RegRipper plugin), and...nothing. I couldn't find the AppCompatibility subkey. It didn't seem to exist.
I ran the RegRipper del.pl plugin against the hive file, redirecting the output to a file:
rip.pl -r f:\defcon2\files\system -p del > f:\defcon2\files\system_del.txt
From the output:
------------- Deleted Data ------------
598220 10 00 00 00 6f 6d 70 61 ....ompa
598230 74 43 61 63 68 65 00 00 tCache..
Okay, so let's try another tool. Using yarp-print.py, I ran the following command line:
yarp-print.py --deleted e:\defcon2\files\system > e:\defcon2\files\system_del_yarp.txt
Opening the output file, there were quite a few recovered keys and values. I ran a search for "ompat", as well as one for "cache", and found nothing related to the AppCompatCache value.
Okay, so now we're pretty sure that the reason the RegRipper plugin didn't return what we expected is because the key and value question don't exist within the hive file. This is pretty unusual, so the next question would be, "why?"
I checked the LastWrite time of the ControlSet001\Control\Session Manager key via WRR, and then pivoted into the timeline of system activity. Events near that time are shown below:
Wed Aug 8 03:51:12 2018 Z
FILE - M... [37092] C:\ProgramData\Microsoft\Windows Defender\Support\MPLog-02112017-053110.log
Wed Aug 8 03:51:03 2018 Z
REG - M... HKLM/Software/Microsoft/MpSigStub
FILE - M... [5732] C:\Windows\Temp\MpSigStub.log
FILE - MA.. [4096] C:\ProgramData\Microsoft\Windows Defender\Definition Updates\{8DCEFA0C-BEA3-48DC-B17D-E363C91F2F5D}\$I30
REG - M... HKLM/Software/Microsoft/Windows/CurrentVersion/WindowsUpdate/Auto Update/Results/Install
FILE - MA.. [48] C:\Windows\Temp\2A62F43A-0724-4D0C-8B60-BC284D249D64-Sigs\
FILE - MA.. [48] C:\ProgramData\Microsoft\Windows Defender\Definition Updates\{8DCEFA0C-BEA3-48DC-B17D-E363C91F2F5D}\
Wed Aug 8 03:51:02 2018 Z
REG - M... HKLM/Software/Microsoft/Windows Defender/Signature Updates
FILE - M... [5699701] C:\ProgramData\Microsoft\Windows Defender\Scans\mpcache-008087D650ED729E08CB8F27E1DE2E1889585057.bin
FILE - MA.B [1999848] C:\ProgramData\Microsoft\Windows Defender\Scans\mpcache-008087D650ED729E08CB8F27E1DE2E1889585057.bin.83
REG - M... HKLM/System/ControlSet001/Control/Session Manager
Interesting. It looks as if Windows Defender was being updated or a scan was being run at about that time. Checking the support log from 03:51:12, we see the following at the end of the file:
Checking the timezone settings for the system, we see that the ActiveTimeBias is 7 hrs, and that a scan was started and completed shortly after the Registry key in question was last modified.
What about the other ControlSet? Pivoting within the timeline to the point where the Control\Session Manager key within ControlSet002 was modified, we see the following:
Tue Aug 7 20:25:33 2018 Z
FILE - MA.. [655360] C:\Windows\System32\$I30
FILE - MA.. [56] C:\Program Files (x86)\
FILE - MA.. [56] C:\Program Files\
FILE - MA.. [56] C:\Program Files (x86)\$TXF_DATA
FILE - MA.. [56] C:\Program Files\$TXF_DATA
FILE - .A.. [20] C:\Windows\AppCompat\Programs\RecentFileCache.bcf
EVTX WIN-M5327EF98B9 - NetBT/4321;,WIN-M5327EF98B9:0,74.118.139.11,74.118.139.201
REG - M... HKLM/Software/Wow6432Node/Microsoft/Windows/CurrentVersion/explorer
FILE - MA.. [520] C:\Users\mpowers\AppData\Local\Microsoft\Windows Mail\Backup\new\
FILE - MA.. [56] C:\Windows\System32\
FILE - MA.. [4096] C:\Windows\SysWOW64\config\systemprofile\AppData\LocalLow\Microsoft\CryptnetUrlCache\Content\$I30
FILE - MA.. [4096] C:\Program Files\$I30
FILE - MA.. [256] C:\Windows\System32\sysprep\Panther\IE\
REG - M... HKLM/System/ControlSet002/Control/Session Manager
Looking nearby within the timeline, we see that there are a number of modifications to both the file system and Registry hives going on leading up to, as well as following that time.
Remember my previous blog post regarding the deletion of forensic artifacts? It appeared that the PrivaZer application was launched at approximately 20:21:51 Z on 7 Aug, and completed its work approximately 10 minutes later; the LastWrite time for the Session Manager key seen above is right in the middle of that activity. As such, it would appear that the reason why there is no AppCompatCache data available is due to the deletion of forensic artifacts by the PrivaZer application. The later LastWrite time for the key found within ControlSet001 may be due to other factors, such as activity that occurred after the key of interest was deleted.
Summary
When engaged in analysis, there are number of components at play, in particular the data, the tool being used, and the analyst. Any one of these, or any combination thereof, could result in an "issue". For example, in this case, an examiner could have easily come to the determination that the tool (i.e., the RegRipper plugin) was the issue. I can't remember ever seeing a System hive file that did not have an AppCompatCache value. As such, why wouldn't it be an issue with the tool?
It turns out that all of the components for running this issue down were right there. We had an open source tool, one which we could literally open in Notepad and read what it does, that would start us down the troubleshooting road. We had other tools available; without WRR, I could have just as easily imported the hive into RegEdit, and from there, seen that the AppCompatibility subkey didn't exist.
How often does this happen? How often do we run a tool in the course of our DFIR investigation process and for some reason, we don't see something in the output of the tool that we thought we would see? How often do we look for something, only to find that it's not there? More importantly, what do we do about it?
I can't speak for anyone else, but I tend to want to figure out why, particularly if the data I'm trying to parse is important to my overall analysis, based on my goals. If it's just a curiosity, I'll leave it until later, but if the data itself is paramount to the analysis, I generally tend to want to know why I'm seeing (or not, as the case may be) what I'm seeing.
Several years ago, while I was QSA certified and working on PCI cases (sssshhhh...don't let that little secret out...), we ran across a case where we knew that two specific brands of credit cards were being used at the site, but for some reason, our scans were coming up empty for those specific brands. We were getting lots of search results for the major card brands, but nothing for these two brands that we expected to see. We started peeling back the layers of the process, and go to the point where we were running scans of just straight text files that contained test data (several of the brands provided a small number of "valid" but unassigned credit card numbers), but we were still coming up empty on the brands. As we investigated further, we narrowed the issue down to a built-in API function, and determined that it was not, in fact, returning "valid" credit cards numbers. Rather, it was, just not all of them; it did not consider these two brands as 'valid' for some reason. We got some programming assistance and built our own replacement function, one that was a bit slower but more complete.
The moral of the story is that we looked at the components involved...the data, the tool, and the examiner...and worked methodically to figure out where the issue lay. Was it an assumption on our part, that these two brands were actually being used at the site? How did we know? Was it an assumption, or was it information that came from a specific, identifiable source? What about the data? Was there an issue with the data? Or was it the tool?
As it turned out, it was an issue of deep knowledge. We knew what the data should look like, and we had an understanding of where the data should reside. As a bit of a follow-on to my post on Deep Knowledge in DFIR work, I wanted to take a few moments to discuss an issue I ran into with the DefCon 2018 CTF File Server image, and what I did to address it.
I was reviewing some of the data, and I wanted to check and see if I could see a reference to the counter-forensics tool in the AppCompatCache data; as such, I ran the RegRipper appcompatcache.pl plugin, and saw the following:
Launching appcompatcache v.20190112
appcompatcache v.20190112
(System) Parse files from System hive AppCompatCache
Signature: 0x0
That was it...nothing else. No listing of file paths and names, no time stamps...nada.
Okay, so...now what? Do I throw up my hands and assume that the tool doesn't work?
No, not at all. I start troubleshooting the issue.
As I mentioned, there are three components to this process...the data, the tool, and the operator. Any one of the three could have an "issue". It's entirely possible that the tool does not function properly. For example, when looking at a Registry hive via a viewer, I like to use the MiTeC Windows Registry Recovery (WRR) tool, even though I know that the tool has a deficiency. Specifically, for value data over a certain size, a different data type is used to store that data (similar to non-resident files within the MFT using a run list), and WRR does not handle those data types. The AppCompatCache data is exactly that data type. However, I could navigate through the Registry structure to the key in question, even though I could not directly view the raw data value via WRR.
First step...check the tool. Is the plugin working correctly? It works correctly against the System hive from the HR server, but that's a different version of Windows. I don't have a System hive available that matches the Windows version from the file server image, but for the most part, the plugins seems to be working correctly.
RegRipper plugins are Perl scripts, which means that they're essentially text files. Opening the appcompatcache.pl plugin in Notepad++, we see that lines 96 and 97 are commented out (that is, the lines start with "#"):
# ::rptMsg("Length of data: ".length($app_data));
# probe($app_data);
This is essentially some troubleshooting code; when it's not commented out, it will tell me the length of the data, and then print a hex dump of the data (via the probe() function).
Assuming that the issue may have been with the plugin, I removed those "#" symbols, saved the file, and re-ran the plugin in hopes that I'd get some more information. For example, if the plugin wasn't parsing the data correctly, I'd see a line telling me the length of the data, and then a hex dump of the data itself. However, when I ran the plugin, I saw simply "Length of data:" and nothing else; no length value, no hex dump.
Next step...check the data itself. I opened the hive in WRR, and went to the Control\Session Manager key to check the AppCompatibility subkey (the path is listed in the RegRipper plugin), and...nothing. I couldn't find the AppCompatibility subkey. It didn't seem to exist.
I ran the RegRipper del.pl plugin against the hive file, redirecting the output to a file:
rip.pl -r f:\defcon2\files\system -p del > f:\defcon2\files\system_del.txt
From the output:
------------- Deleted Data ------------
598220 10 00 00 00 6f 6d 70 61 ....ompa
598230 74 43 61 63 68 65 00 00 tCache..
Okay, so let's try another tool. Using yarp-print.py, I ran the following command line:
yarp-print.py --deleted e:\defcon2\files\system > e:\defcon2\files\system_del_yarp.txt
Opening the output file, there were quite a few recovered keys and values. I ran a search for "ompat", as well as one for "cache", and found nothing related to the AppCompatCache value.
Okay, so now we're pretty sure that the reason the RegRipper plugin didn't return what we expected is because the key and value question don't exist within the hive file. This is pretty unusual, so the next question would be, "why?"
I checked the LastWrite time of the ControlSet001\Control\Session Manager key via WRR, and then pivoted into the timeline of system activity. Events near that time are shown below:
Wed Aug 8 03:51:12 2018 Z
FILE - M... [37092] C:\ProgramData\Microsoft\Windows Defender\Support\MPLog-02112017-053110.log
Wed Aug 8 03:51:03 2018 Z
REG - M... HKLM/Software/Microsoft/MpSigStub
FILE - M... [5732] C:\Windows\Temp\MpSigStub.log
FILE - MA.. [4096] C:\ProgramData\Microsoft\Windows Defender\Definition Updates\{8DCEFA0C-BEA3-48DC-B17D-E363C91F2F5D}\$I30
REG - M... HKLM/Software/Microsoft/Windows/CurrentVersion/WindowsUpdate/Auto Update/Results/Install
FILE - MA.. [48] C:\Windows\Temp\2A62F43A-0724-4D0C-8B60-BC284D249D64-Sigs\
FILE - MA.. [48] C:\ProgramData\Microsoft\Windows Defender\Definition Updates\{8DCEFA0C-BEA3-48DC-B17D-E363C91F2F5D}\
Wed Aug 8 03:51:02 2018 Z
REG - M... HKLM/Software/Microsoft/Windows Defender/Signature Updates
FILE - M... [5699701] C:\ProgramData\Microsoft\Windows Defender\Scans\mpcache-008087D650ED729E08CB8F27E1DE2E1889585057.bin
FILE - MA.B [1999848] C:\ProgramData\Microsoft\Windows Defender\Scans\mpcache-008087D650ED729E08CB8F27E1DE2E1889585057.bin.83
REG - M... HKLM/System/ControlSet001/Control/Session Manager
Interesting. It looks as if Windows Defender was being updated or a scan was being run at about that time. Checking the support log from 03:51:12, we see the following at the end of the file:
[Tue Aug 07 2018 20:51:03] Process scan started.
[Tue Aug 07 2018 20:51:04] Process scan completed.
Checking the timezone settings for the system, we see that the ActiveTimeBias is 7 hrs, and that a scan was started and completed shortly after the Registry key in question was last modified.
What about the other ControlSet? Pivoting within the timeline to the point where the Control\Session Manager key within ControlSet002 was modified, we see the following:
Tue Aug 7 20:25:33 2018 Z
FILE - MA.. [655360] C:\Windows\System32\$I30
FILE - MA.. [56] C:\Program Files (x86)\
FILE - MA.. [56] C:\Program Files\
FILE - MA.. [56] C:\Program Files (x86)\$TXF_DATA
FILE - MA.. [56] C:\Program Files\$TXF_DATA
FILE - .A.. [20] C:\Windows\AppCompat\Programs\RecentFileCache.bcf
EVTX WIN-M5327EF98B9 - NetBT/4321;,WIN-M5327EF98B9:0,74.118.139.11,74.118.139.201
REG - M... HKLM/Software/Wow6432Node/Microsoft/Windows/CurrentVersion/explorer
FILE - MA.. [520] C:\Users\mpowers\AppData\Local\Microsoft\Windows Mail\Backup\new\
FILE - MA.. [56] C:\Windows\System32\
FILE - MA.. [4096] C:\Windows\SysWOW64\config\systemprofile\AppData\LocalLow\Microsoft\CryptnetUrlCache\Content\$I30
FILE - MA.. [4096] C:\Program Files\$I30
FILE - MA.. [256] C:\Windows\System32\sysprep\Panther\IE\
REG - M... HKLM/System/ControlSet002/Control/Session Manager
Looking nearby within the timeline, we see that there are a number of modifications to both the file system and Registry hives going on leading up to, as well as following that time.
Remember my previous blog post regarding the deletion of forensic artifacts? It appeared that the PrivaZer application was launched at approximately 20:21:51 Z on 7 Aug, and completed its work approximately 10 minutes later; the LastWrite time for the Session Manager key seen above is right in the middle of that activity. As such, it would appear that the reason why there is no AppCompatCache data available is due to the deletion of forensic artifacts by the PrivaZer application. The later LastWrite time for the key found within ControlSet001 may be due to other factors, such as activity that occurred after the key of interest was deleted.
Summary
When engaged in analysis, there are number of components at play, in particular the data, the tool being used, and the analyst. Any one of these, or any combination thereof, could result in an "issue". For example, in this case, an examiner could have easily come to the determination that the tool (i.e., the RegRipper plugin) was the issue. I can't remember ever seeing a System hive file that did not have an AppCompatCache value. As such, why wouldn't it be an issue with the tool?
It turns out that all of the components for running this issue down were right there. We had an open source tool, one which we could literally open in Notepad and read what it does, that would start us down the troubleshooting road. We had other tools available; without WRR, I could have just as easily imported the hive into RegEdit, and from there, seen that the AppCompatibility subkey didn't exist.
Subscribe to:
Posts (Atom)