Sunday, April 05, 2020

Going Beyond

As an industry and community, we need to go beyond...go beyond looking at single artifacts to indicate or justify "evidence", and we need to go beyond having those lists of single artifacts provided to us.  Lists, such as the SANS DFIR poster of artifacts, are a good place to start, but they are not intended to be the end-all.  And we need to go beyond our own analysis, in isolation, and discuss and share what we see with others.

Here's a good example...in this recent blog post, the author points to Prefetch artifacts as evidence of file execution.  Prefetch artifacts are a great source of information, but (a) they don't tell the full story, and (b) they aren't the only artifact that illustrates "file execution".  They're one of many.  While it's a good idea to start with one artifact, we need to build on that one artifact and create (and pursue) artifact constellations.

This post, and numerous others, tend to look at artifacts in isolation, and not as part of an overall artifact constellation.  Subsequently, attempts at analysis fall apart (or simply fall short) when that one artifact, the one we discussed in isolation, is not present.  Consider Prefetch files...yes, they are great sources of information, but they are not the only source of information, and they are not present by default on Windows servers. 

And, no, I do not think that one blog post speaks for the entire community...not at all.  Last year, I took the opportunity to explore the images provided as part of the DefCon 2018 CTF.  I examined two of the images, but it was the analysis of the file server image that I found most interesting.  Rather than attempting to answer all of the questions in the CTF (CTF questions generally are not a good representation of real world engagements), I focused on one or two questions in particular.  In the case of the file server, there was a question regarding the use of an anti-forensics tool.  If you read my write-up, you'll see that I also reviewed three other publicly available write-ups...two relied on a UserAssist entry to answer the question, and the third relied on a Registry value that provided information about the contents of a user's desktop.  However, none of them (and again, these are just the public write-ups that I could find quickly) actually determined if the anti-forensics tool had been used, if the functionality in question had been deployed.

Wait...what?  What I'm saying is that one write up had answered the question based on what was on the user's desktop, and the two others had based their findings on UserAssist entries (i.e., that the user had double-clicked on an icon or program on their desktop).  However, neither had determined if anything had actually been deleted. I say this because there was also evidence that another anti-forensics tool (CCleaner) had been of interest to the user, as well. 

My point is that when we look at artifacts in isolation from each other, we only see part of the picture, and often a very small part.  If we only look at indications of what was on the user's desktop, that doesn't tell us if the application was ever launched.  If we look at artifacts of program execution (UserAssist, Prefetch, etc.), those artifacts, in and of themselves, will not tell us what the user did once the application was launched; it won't tell us what functionality the user employed, if any.

Here's another way to look at it.  Let's say the user has CCleaner (a GUI tool) on their desktop.  Looking at just UserAssist or Prefetch...or, how about UserAssist and Prefetch...artifacts, what is the difference between the user launching CCleaner and deleting stuff, and launching CCleaner, waiting and then closing it?

None.  There is no difference. Which is why we need to go beyond just the initial, easy artifacts, and instead look at artifact clusters or constellations, as much as possible, to provide a clear(er) picture of behavior.  This is due to the nature of what we, as examiners, are looking at today.  None of the incidents we're looking at...targeted threats/APTs, ransomware/crimeware, violations of acceptable use policies, insider threats, etc...are based on single events or records. 

Consider ransomware...for the most part, these events were looked at, more often than not, as, "files were encrypted". End of story. But the reality is that in many cases, going back years, ransomware incidents involved much more than just encrypting files.  Threat actors were embedded within environments for weeks or months before ever encrypting a file, and during that time they were collecting information and modifying the infrastructure to meet their needs.  I say "were", but "still are" applies equally well.  And we've seen an evolution of this "business model" over the past few months, in that we know that data was exfil'd during the time the actor was embedded within the infrastructure, not due to our analysis, but because the threat actor releases it publicly, in order to "encourage" victims to pay. A great deal of activity needs to occur for all of this to happen...settings need to be modified, tools need to be run, data needs to be pulled back to the threat actor's environment, etc.  And because these actions occur over time, we cannot simply look to one, or a few artifacts in isolation, in order to see the full picture (or as full a picture as possible).

Dr. Ali Hadi recently authored a pair of interesting blog posts on the topic of USB devices (here, and here).  In these posts, Dr. Hadi essentially addresses the question of, how do we go about performing out usual analysis when some of the artifacts in our constellation are absent? 

Something I found fascinating about Dr. Hadi's approach is that he's essentially provided a playbook for USB device analysis.  While he went back and forth between two different tools, both of his blog posts provide sufficient information to develop that playbook in either tool.  For example, while Dr Hadi incorporated the use of Registry Explorer, all of the artifacts (as well as others) can also be derived via RegRipper plugins.  As such, you can create RegRipper profiles of those plugins, and then run the automatically against the data you've collected, automating the extraction of the necessary data.  Doing so means that while some things may be missing, others may not, and analysts will be able to develop a more complete picture of activity, and subsequently, more accurate findings.  And automation will reduce the time it takes to collect this information, making analysis more efficient, more accurate, and more consistent across time, analysts, etc.

Okay, so what?  Well, again...we have to stop thinking in isolation.  In this case, it's not about just looking at artifact constellations, but it's also about sharing what we see and learn with other analysts.  What one analyst learns, even the fact that a particular technique is still in use, is valuable to other analysts, as it can be used to significantly decrease their analysis time, while at the same time increasing accuracy, efficiency, and consistency. 

Let's think bigger picture...are we (DFIR analysts) the only ones involved?  In today's business environment, that's highly unlikely.  Most things of value to a DFIR analyst, when examined from a different angle, will also be valuable to a SOC analyst, or an EDR/EPP detection engineer.  Here's an example...earlier this year, I read that a variant of Ryuk had been analyzed and found to contain code for deploying Wake-on-LAN packets in order to increase the number of systems it could reach, and encrypt. As a result, I wrote a detection rule to alert when such packets were found originating from a system; the point is that something found by malware reverse engineers could be effectively employed by SOC analysts, which in turn would result in more effective response from DFIR analysts.

We need to go beyond.  It's not about looking at artifacts in isolation, and it's not about conducting our own analysis in isolation.  The bad guys don't do it...after all, we track them as groups.  So why not pursue all aspects of DFIR as a 'group'; why not look at groups of artifacts (constellations), and share our analysis and findings not just with other DFIR analysts, but other members of our group (malware RE, threat intel analyst, SOC analyst, etc.), as well?

Wednesday, March 11, 2020

Ransomware

Hardly a week (sometimes a day??) passes without some mention of ransomware, and another organization or municipality (or three) feeling the impact of a ransomware attack. In fact, just recently, the City of Durham, NC, was hit with a Ryuk ransomware infection, which by some reports, impacted their 911 capability.  At the end of February, the BBC reported that two organizations impacted by a ransomware infection had been down for three weeks.

CrowdStrike recently released their 2020 Global Threat Report includes a great deal of information regarding ransomware, as viewed through CS's lens. The report includes more than a few pages of what CS had seen over the previous year, with some thoughts as to what they expect to see going forward.

In addition, just last week, Microsoft published an interesting blog post regarding human-operated ransomware attacks (with a very telling graphic available here).  All of the events at which I spoke in 2019 focused on this very topic, that many of the ransomware attacks weren't about something that AV products would detect and prevent. The general perception of these attacks seemed to be predominantly, "oh, if I have AV or NGAV, I'm good..."; well, no.  Because these are human-operated attacks, the human operator is able to modify the infrastructure to meet their needs.  For example, pull plain text credentials from memory, in order to escalate privileges and extend their reach to other systems.  The better part of this activity is missed by AV, because sometimes, malware isn't required to perform the "attack".  Instead, attackers simply use the native MS tools provided within the operating system distribution, something referred to as "living off the land". 

Further, as discussed in the CrowdStrike GTR, ransomware actors are increasingly modifying the infrastructure's they've targeted by disabling security products, enabling WinRM (sometimes through GPOs), and just making things easier for themselves.  These changes often go unnoticed by the system owners but do serve as precursors to the actor deploying ransomware.  This means that if these infrastructure modifications are detected, and there's a response plan in place, the overall impact of the ransomware being deployed can be obviated.

Impact
Something that is rarely discussed at length, or in an inclusive manner, is the impact of a ransomware attack, and why some organizations choose to pay the ransom.  Sure, we're all generally aware of what happens...files are encrypted, everyone's caught by surprise, and suddenly things need to happen that no one was prepared to do.  In some cases, such as with hospitals, diagnostics and patient record keeping gets reduced to by-hand processes, and the same is often true when a municipality's 911 services are taken down by ransomware.

I recently read this ZDNet article (similarly discussed in this NakedSecurity article), which discusses how 11 cases against six criminals were dismissed because the data was lost as a result of a ransomware attack.  The article also provides a list of other similar issues (police depts experiencing ransomware attacks), going back to 2017.

Tuesday, March 10, 2020

Revisiting Program Execution

As I prepare a presentation for a government agency, I've been thinking quite a bit about the idea of "program execution".  I've actually blogged on this topic before, and I thought that maybe now was a good time to revisit the topic.

What does that mean?  Well, generally speaking, it is accepted within the community that there are artifacts on systems that indicate that an application was executed.  One popular example is Prefetch files; the existence of an application prefetch file indicates that an application was run. There are other artifacts, as well, including UserAssist subkeys, etc.  Many of these artifacts tell us that an application was launched, or that a user launched an application (and when), but there's also a good bit that these artifacts don't tell us.

Here's an example...I was working a PCI case a number of years ago (and that "number" could be pretty big, like double digits...), and for those working those cases at the time, one of the things that Visa required of examiners was to populate a dashboard in the reports that included things such as "window of compromise", which is sort of what we refer to as 'dwell time'.  In this case, the actor gained access to a system and loaded their malware on it, launched it, and left...and within a relatively short period of time, AV ran a scheduled scan, found the malware, and quarantined it.  Through the analysis, we saw that it was another six weeks before the bad guy came back, found that the malware wasn't running, put a new version of the malware on the system and launched it.

Now, our finding was pretty huge, particularly for PCI cases.  A lot of organizations are aware of the number of credit cards they process on a regular basis, and there's always a bump around the holidays.  In this case, the malware was first placed on the system just prior to Thanksgiving, and wasn't refreshed until after the Christmas holiday.  Visa used the number of transactions and the "window of compromise" values to help them determine fines; as such, being able to demonstrate that the malware was not running on the system for a specific time period really had a significant impact on that finding, and subsequently, the fine.

My point is, just because we know that an application was launched, do we then definitively know that it was run, or in the case a GUI application, what functionality was employed?  The answer is no.  I've seen a number of cases, during timeline analysis, where something was launched, only to have the system immediately respond with Application Error and Windows Error Reporting events in the Windows Event Log.

My previous blog post on this topic linked to another article I'd written, one that addressed analysis of one of the images from the DefCon 2018 CTF challenge.  One of the questions I'd looked at specifically was question regarding the use of an anti-forensics application.  In the three publicly available write-ups I'd reviewed, one had answered based on an item on the user's desktop, and the other two responded based on program execution artifacts.  However, none of them had seen that attempts had been made to launch another anti-forensics application, and similarly, nor confirmed that actual anti-forensics had taken place, that anything had been deleted or modified.

The issue is that specifically with respect to GUI-based applications, the "program execution" artifacts will illustrate that the application was launched, but other efforts are required to determine what functionality was actually employed.  After all, there aren't that many applications that record what options the user selected, nor which buttons they pushed.  As such, "program execution" artifacts alone provide little qualitative difference between the user launching the application and letting is sit dormant on the screen, and the user actually using the application's functionality to do something.  In fact, additional analysis steps are required; in the case of the DefCon 2018 CTF image, determining if the observed "program execution" artifacts truly resulted in the use of the applications' functionality.


Thursday, February 20, 2020

RegRipper Update

Based on a Twitter thread from 19 Feb 2020, during which Phill Moore made the request, I updated RegRipper to check for "dirty" hives, and provided a warning that RegRipper does NOT automatically process Registry transaction logs.  This can be an important component of your investigation, and so per Phill's request, I updated RegRipper (both the UI and rip.pl/.exe) to provide the warning, as well as check to see if the hive is 'dirty'.

If you decide that you need to process the transaction logs, there are a couple of options available.  One is using yarp (from Maxim Suhanov) along with registryFlush.py (per this blog post).  Written in Python, this provides a cross-platform approach, if you need that flexibility.  Another method is to use Eric Zimmerman's rla.exe (part of Registry Explorer/RECmd) tool, which is Windows-based.

So, to re-iterate, RegRipper 2.8 does NOT automatically process transaction logs.  I haven't developed the code to do so, and there are a number of variables to doing so. For example, the current RegRipper repo contains plugins either intended just for XP systems (i.e., acmru.pl), and also contains plugins that can process data from hives from XP through Win10 (i.e., appcompatcache.pl, shimcache.pl).  Transaction logs from older systems (XP, etc.) follow a different format that more modern (Win8.1+) systems, and as such, would require additional code to address those files.

Also, don't forget about the update made to RegRipper on 4 Jan 2020, fixing the issue that began as of 1 Jan 2020 where Registry key LastWrite times were incorrectly reported as "0".  The updated code was added to one of the core module files (i.e., Base.pm), which is available in the RegRipper Github repo. 

These updates have been "compiled" into the Windows executables for the RegRipper tools (rr.exe, rip.exe), but if you're looking to install RegRipper on Linux, be sure to read the RegRipper readme file and update the Perl modules files according.

Monday, February 17, 2020

RID Hijacking

I read a fascinating blog post recently that described something called RID hijacking, which can be used as a method for maintaining elevated privileges on a system.  The PenTestLabs article not only outlines how to perform RID hijacking manually, but also using MetaSploit, Empire, a Powershell module, and then using the module via POSHC2.  In short, there are no shortage of ways to go about performing RID hijacking from an offensive perspective.

But how would this look from a blue team perspective?  I made a minor tweak to the output of one of the RegRipper plugins (the data was already available) so that the output appears as follows (run against a SAM hive from a Win2000 system):

Username          : Guest [501]
Full Name          :
User Comment    : Built-in account for guest access to the computer/domain
Account Type      :
Account Created : Fri Sep 27 03:32:48 2002 Z
Name                  :
Last Login Date   : Never
Pwd Reset Date   : Never
Pwd Fail Date      : Sat Aug 25 09:21:50 2012 Z
Login Count         : 0
Embedded RID    : 501
  --> Password does not expire
  --> Account Disabled
  --> Password not required
  --> Normal user account

I added the emphasis on the "Embedded RID" entry.  Again, the data itself was available in the plugin, all I did was add the line to display the RID from within the F value data (as described in the PenTestLab article).  I also added a bit of logic to compare the embedded value with the RID value from the Username field (in the brackets) and if they aren't same, print out a warning message.

I can't say that I've ever intentionally looked for this sort of thing during an investigation, as there was nothing that pointed to something like RID hijacking having occurred.  For example, there were no indications of no suspicious actions involving the Guest account, and subsequently the Guest account being used to log into the system and perform actions requiring Admin privileges. 

I have seen threat actors establish multiple means for returning to systems.  I've seen threat actors hit an RDP server, enable Sticky Keys, and then create an account, add it to the Administrators group (as well as the Remote Users group) and then add that account to the "SpecialAccounts\UserList" key path so that the icon for the account does not appear on the Welcome screen. 

Given that this capability is available, and not only via popular offensive security toolsets, I'd recommend checking for RID hijacking during investigations.  Automating the checking means that it's done every time, regardless of who runs the process.

Saturday, February 15, 2020

Using Intrusion Intelligence

In his book, "Call Sign Chaos", Jim Mattis made the statement that 'your own personal experiences are not enough to sustain you."  This statement was made in the context of reading for professional development, and it applies to much more than just warfighting.  It also applies equally well in the DFIR field, including DFIR analysis, threat hunting, and developing intrusion intelligence.  This is due to the fact that our own experiences are not enough to sustain us, neither as individual analysts nor as teams.  None of us want to go to an auto mechanic or a neurosurgeon who stopped their education the moment they received their diploma; as such, it's incumbent upon each of us to further our education and professional development through whatever means works for us.  Sometimes, that's as simple as reading...really reading...what someone else has written, and then incorporating what we learn into our own analysis process or methodology.

I recently read this excellent article from the Elastic team regarding their insights into the Gamaredon Group. I found that the article included a good bit of detail, not just in how the Elastic team goes about running down an incident, but also detail with respect to findings written in a manner that can be incorporated into threat hunting and DFIR analysis methodologies with little effort.

I've written and spoken often on the topic of threat actors modifying the target environment to meet their needs.  This has been about much more than just modifying the Windows firewall to allow an application to access the Internet; threat actors have been observed making what appear to be subtle changes to target environments, but when those changes are fully understood, they're actually very significant.  From enabling systems to store credentials in memory in plain text to disabling security tools, threat actors have been observed making significant modifications to the target environment.

In figure 12 in the Elastic article, we see where, via the macro, the actors modified the infrastructure to permit the execution of untrusted macros, as well as disabled warnings. However, this is nothing new.  Mari discussed the VBAWarnings Registry value in Feb, 2016, and the value was mentioned on TechNet as far back as 2012. 

As stated in the Elastic article, These small changes can end up having larger implications, and defenders can look for them as symptoms of more serious security issues. This is absolutely the case.  The article goes on to identify additional entities that have been observed modifying target environments in a similar manner.

Okay...so what? 

Well, there are a number of ways you can make use of this information...and this isn't all of the information from the article.  There are a total of 25 figures in the article, and the information discussed above is from three of those figures, two of which are EQL queries.  Depending upon the EDR solution you're using, you can monitor for modifications to the Registry, and specifically for that key path, and those values.  If you're in a position where you're not able to employ an EDR solution, or if you're limited to just DFIR analysis or retro-active threat hunting, you can check that the values exist, and if so, their data, in order to see if there is or was an issue.  If you do find the value data set in accordance with the article, the key LastWrite time may provide a suitable pivot point for analysis.

Thursday, February 13, 2020

Update: Prefetch + Stealth ADS Analysis

Not long ago, I took at look at an image that Dr. Ali Hadi had put together to demonstrate an aspect of digital analysis to his students.  Dr. Hadi's blog post describes how the use of the ADSs, particularly when launching programs from ADSs, bypasses "normal" analysis methodologies which can tend to focus on one, or just a few, artifacts.  I completely understand and agree with the point that Dr. Hadi made, and wanted to demonstrate the value of analysis that incorporates a corpus of artifacts, or 'artifact clusters'.  As we saw in my previous post, there were a number of areas (file system metadata, BAM key, AppCompatCache data, SRUM data) where red flags related to the specific activity could be found, all of which were separate from the artifacts on which Dr. Hadi's article focused.

I decided to take a further look at data sources on the system to see if there were other artifacts that would serve as pivot points in analysis.  For example, I found that the AmCache.hve file contained the following entry:

c:\users\ieuser\desktop\creepy\welcome.txt:putty.exe 
LastWrite: Sun May 26 08:41:35 2019
Hash: 2662d1bd840184ec61ddf920840ce28774078134

Interestingly, the hash maintained in the AmCache entry is for putty.exe, rather than welcome.txt, and was detected by 47 engines on VT.  I say "interestingly" because in some cases where hashes have been generated, they've been for the carrier file, not the ADS.

From the user's ActivitiesCache.db file, specifically the Activity table, I saw this entry in the ContentInfo column (the Executable column listed notepad.exe):

C:\Users\IEUser\Desktop\creepy\welcome.txt (file:Unmapped GUID: //C:/Users/IEUser/Desktop/creepy/welome.txt?VolumeId={20B25A2A-0000-0000-0000-100000000000}&ObjectId={0282E6B5-7F90-11E9-A75B-000C29C3F036}&KnownFolderId=ThisPCDesktopFolder&KnownFolderLength=23)

I added the bold text for a rather obvious misspelling that jumped out; however, there's nothing in the entry that specifically stands out as being associated with the ADSs.

I also took some other parsing steps that were not fruitful.  For example, I parsed out all of the unallocated space from the NTUSER.DAT, and also merged the transaction logs into the hive, and re-ran several RegRipper plugins.  Like I said, neither were fruitful, in this case.

I'm not sharing this because I disagree with Dr. Hadi's thesis...in fact, I completely agree with him.  Too often, we may find ourselves focusing on just one artifact, and as Dr. Hadi pointed out, we can get caught off-guard by some variation of that artifact with which we weren't familiar.  I've shared these articles, and the artifacts in them, in order to illustrate the value of using multiple data sources, and being able to find pivot points in your analysis.

Tuesday, January 28, 2020

"Hidden" Prefetch File Analysis and Alternate Data Sources

One of the things I like to do is engage in DFIR analysis of CTF and challenge images, just to see what pops out and what new things I can learn.  I did just that recently with Dr. Ali Hadi's challenge #4, described as "launching attacks from alternate data streams".  Dr. Hadi also has a corresponding blog post where he digs a bit deeper into how this approach can "bypass normal forensic analysis".  Actually, that last part of the title, about bypassing "normal forensic analysis", is what really got me interested in exploring the topic a bit further.  Let's take a look...

I've been aware of and fascinated by NTFS alternate data streams (or "ADSs") for quite sometime; not just that they exist, but also how they can be and have been used.  ADSs have always been pretty cool, and something of a novelty to me; while I know what they are and look for them during analysis, I don't often find them used for malicious purposes during DFIR engagements.

Dr. Hadi's blog post focuses primarily on the analysis of a single artifact, the application Prefetch files, which won't be available by default on Windows server systems. Also, you're not likely to find these files on workstation versions of Windows running on SSD drives.  In the blog post, he uses several tools for parsing and deriving data from Prefetch files, but again the focus is on those files as the data source, when it comes to finding the use of ADSs.

While I don't want to think that is constitutes "normal forensic analysis", that may be the case.  I like to use multiple data sources, in particular the MFT, Registry, and Windows Event Log when looking at data, and I prefer to correlate the data sources in a timeline format. 

One of the first things I noticed when parsing the MFT is that the existence of the ADS jumps right out, as shown below:

Sun May 26 08:36:19 2019 Z
  MFT_SI          - ...B [16] C:\Users\IEUser\Desktop\creepy\LPT1.txt  
  MFT_FN         - MACB [16] C:\Users\IEUser\Desktop\creepy\LPT1.txt  
  MFT_ADS       - ...B [809984] C:\Users\IEUser\Desktop\creepy\LPT1.txt:putty.exe

Now, an ADS shares the time stamps of its "parent" file, and does not have time stamps of its own.  It does, however, have its own size value, which is good.  So, while the parser I used clearly identifies the ADS, the time listed may not correlate to the actual creation date of the ADS itself.  In fact, the creation time ("...B") of the ADS may more closely correspond to the metadata modification ("..C.") time from the $STANDARD_INFORMATION attribute, depending upon the actual circumstances.

In this case, we have another data source available, the "bam" key from the System hive, as shown below:

Sun May 26 08:41:41 2019 Z
  MFT_SI        - MA.B [0] C:\Windows\Prefetch\WELCOME.TXT  
  BAM            - \Device\HarddiskVolume1\Users\IEUser\Desktop\creepy\
              welcome.txt:putty.exe (S-1-5-21-321011808-3761883066-353627080-1000)
  MFT_ADS    - MA.B [6462] C:\Windows\Prefetch\WELCOME.TXT:PUTTY.EXE-A6BB0639.pf
  MFT_FN      - MACB [0] C:\Windows\Prefetch\WELCOME.TXT  

In addition, there were remnants found in other data sources, as well.  For example, parsing the SRUDB.DAT database file, the following entries were located in the Application Resource Usage tab of the resulting XLSX file:

\Device\HarddiskVolume1\Users\IEUser\Desktop\creepy\LPT1.txt:putty.exe
\Device\HarddiskVolume1\Users\IEUser\Desktop\creepy\COM1.txt:revshell.exe

Further, the following entry was found in the Network Usage tab of the same XLSX file:

\device\harddiskvolume1\users\ieuser\desktop\creepy\com1.txt:revshell.exe

Finally, the following entries were located in the AppCompatCache data within the System Registry hive:

\\?\C:\Users\IEUser\Desktop\creepy\welcome2.txt:revshell.exe  Sun May 26 08:33:33 2019 Z
\\?\C:\Users\IEUser\Desktop\creepy\COM1.txt:revshell.exe  Sun May 26 08:37:34 2019 Z
\\?\C:\Users\IEUser\Desktop\creepy\LPT1.txt:putty.exe  Sun May 26 08:37:19 2019 Z
\\?\C:\Users\IEUser\Desktop\creepy\welcome.txt:putty.exe  Sun May 26 08:33:19 2019 Z

As you can see, there is no shortage of artifacts related to ADSs on a Windows system.  The key take-away here for analysis is to not restrict your analysis to just a single artifact; instead, make a holistic approach your new "normal forensic analysis" process.

I want to thank Dr. Hadi for not only providing the image file, but for also putting together the blog post describing his findings with respect to the use of ADSs, and the application Prefetch file artifacts.

Monday, January 27, 2020

Developing and Using Lessons Learned

From the very beginning of my time in DFIR, and even before then, I saw the value in having some form of "lessons learned" process. 

By "even before then", I mean that while I was on active duty in the military, we had "lessons learned" events following major exercises that proved to be very valuable.  After exercises such as Valiant Blitz '90 and Team Spirit '91, units involved (MACS-4, MATCS-18, MASS-2, LAAD-2, etc.) got together a did a "hot washup", which was a formalized lessons learned process.  I was a communications officer, providing communications services to the "operators", as well as connecting them to other units.  This sort of process is immensely valuable to military units, particularly given that I and others in my unit (my Gunnery Sgt, my CO, etc.) would likely not be with our unit for the next major event, due to rotations.  This meant that lessons learned were preserved for others for future use.

Within DFIR work, things are very different.  We often do not have the luxury of conducting a "lessons learned" process for most of our engagements, and because it's not part of our culture, when we do have the opportunity to do so, we don't do it well.  Further, there needs to be much more to a process such as this than just knowledge sharing via the oral tradition (i.e., having a 'brown bag' lunch with someone giving a presentation), as this sort of passing along of 'corporate knowledge' is extremely transient.  What this means is that if a lesson is indeed learned, a presentation may be provided, but two additional questions need to be addressed.  First, are analysts held accountable for using the new knowledge, and second, how is this new information preserved such that it persists beyond the employment lifetime of the author?

I can't really address the first question, as I'm not a manager.  I have my own thoughts as to how this is accomplished; specifically, managers have to remain knowledgeable themselves, and then engage with the analysts (even if through customer updates and reports) in a purposeful, intentional manner.  Part of this includes purposeful, intentional reviews of the analysts' work (via customer updates, reports, etc.), but given that the business model used in most IR consulting organizations is one in which utilization is the primary metric, managers do not often have the time to engage in such a manner.

As to the second question, however, there are a number of ways that the corporate knowledge can be preserved, one being something as simple as a wiki.  A corporate or team-based wiki can be a central repository, and it's particularly valuable if it's maintained, updated, curated and searchable.

Another way of preserving the corporate knowledge well beyond the lifetime of any individual analyst or manager is to bake findings back into the overall process, such as via the collection or pre-processing mechanism.

For example, several years ago, I was listening to a "lessons learned" call when the analyst speaking glossed over something seemingly unimportant (to them), and I got curious about it.  As such, I accessed the data and found something extremely valuable...in this particular instance, the adversary had launched their remote access Trojan (RAT) via a Scheduled Task, rather than launching the RAT installer. Immediately thereafter, once the RAT was running, the adversary deleted the scheduled task.  This was on a Windows 7 system, so deleting the scheduled task meant that both the XML and XP-style .job files were deleted, but the RAT was still running in memory.  Approximately 72 hrs after the RAT was launched, an event record with event ID 709 was generated in the Task Scheduler Event Log.  This event indicated that an error had occurred while attempting to update the backwards-compatible .job file...in this case, because that file no longer existed.  Additional research indicated that this was known functionality within Windows...specifically, that the Task Scheduler service would attempt to update the job information after...you guessed it...72 hours.

It turned out that this was an extremely high fidelity indicator of the adversary's behavior.  Rather than launching the RAT installer via the Scheduled Task (something that generally runs pretty quickly and is complete in a matter of seconds), the actor launched the RAT itself, something that was intended to be "long lived".  There was speculation at that time as to whether the threat actor knew about the 72 hr time frame or not, and how 'sophisticated' they were.  I simply reminded myself that this had not been self-detected by the customer and we were analyzing data months after the intrusion occurred.

As cool as the event record was, it was highly unlikely that I'd remember to look for something like that on an engagement 6, 8, or 12 months from then, so I added the indicator to my pre-processing methodology, and from that point on, that record was automatically tagged every time I parsed that Windows Event Log file.  My pre-processing methodology includes documentation as to the nature and context of the event record, so I don't have to memorize anything.  This also means that by sharing the methodology with others, every analyst can benefit from the experience of one, without having to have engaged in the analysis themselves (i.e., knowledge sharing).

Threat Hunting Process
Threat hunting is often described as pursuing a hypothesis, or finding a thread and pulling it.  Even if nothing is found, something can be learned.

So how do you do this?  Well, if you're in an FTE (full-time employment) internal position, start with understanding something about your infrastructure.  For example, I know of one analyst who knew that their organization did NOT use native Windows tools (i.e., net.exe) to manage user accounts; as such, they found a thread (i.e., use of net.exe to create and manager user accounts), used that as the basis for hunting across telemetry collected from their environment...and found a bad guy.  Start with, "this is how we do it", and "this is how the bad guy might do it", and pull that thread.  Lather, rinse, repeat.

A great place to get these threads to pull is from social media, particularly sources (Twitter, LinkedIn) that point to write-ups shared by others.  Very often, we might develop a hypothesis or "thread to pull" from something we find online, and from there, our hunt may turn up a system or two (or three) of interest.  For example, not long I saw on Twitter that the Ryuk ransomware had new functionality that allowed it to send a "wake-on-LAN" packet to turn other systems on, so it could then be used to infect them.  If your threat hunting capability allows you to detect UDP datagrams being sent to port 7, why not conduct a hunt to see if this has happened within your environment?

What threat hunting then leads us to is the systems which require closer attention, via either full image acquisition, or triage data collection. For example, during targeted threat response for 150,000 endpoints, we were able to reduce this "data ocean" to just the 8 systems the threat actor had engaged with, and then using triage data collection, reduced what we had to look at to a "data cup". 

Developing Lessons Learned
In a manner similar to the threat hunting process, intrusion intelligence can be developed from DFIR engagements.  I say "can be" because in my experience, this is something rarely done.  When I started in DFIR, many of those I worked with came from either military or LE backgrounds, and one would think that a "lessons learned" session after an engagement would be just something that just happened, but as it turned out, it didn't.  Not doing so became part of the culture.

What happens if you do a "lessons learned" session after engagements?  At the very least, you get things like this, this, and eventually this.  The common factor amongst all three of those examples is that they include data that is immediately usable to someone who reads them; they offer actionable intelligence that someone who did not experience one of those engagements can now take full advantage. 

Data Sources
If you do DFIR work, the best data source  for producing intrusion intelligence is your own DFIR data.  If you have access to EDR telemetry, as well, then the data is even richer, because you can correlate the actions of the threat actor directly to the artifacts on systems.

If you're in an FTE position internal to a company, this is a great way to develop very detailed insight into internal operations within your own organization.  If you're a consultant, this (DFIR + EDR) is THE best way to produce intrusion intelligence, hands down.

Another data source is to monitor social media (as well as specific blogs and other resources) and determine what you can use within your own environment.  Use open source threat intel to develop your own hunting leads (for EDR and/or DFIR data), and then bake what you learn from those hunts back into your own process.

A good example of this is this MenaSec blog post; there's a good explanation of what was observed, as well as some great indicators (file names) provided that are generated by the operating system as a result of the activity of interest.  As such, these indicators are not as easy to change, unlike IP addresses or file hashes.

Another good example is the SecureList description of Project TajMahal.  One of the aspects of this write-up that caught my eye was the module that facilitates data theft by enabling the KeepPrintedJobs flag for printers on the system; this allows the adversary to collect data that is normally deleted as soon as the print job has been completed.  This provides a great opportunity for high fidelity detections in both EDR and DFIR pre-processing.

Using Lessons Learned
Now that we have these items of interest, these indicators, what do we do with them?  Well, that really depends on what you have available.

For example, are you using an EDR tool that can detect Registry operations (key creations, writes to values, etc.)?  If so, create an EDR detection or filter that will tell you when the Attributes value for printers on a system has been modified.  If you're doing DFIR work, add the appropriate check to your pre-processing methodology.  It's pretty trivial to write a RegRipper plugin that will not only collect the Attributes value from each printer, but also check to see if the KeepPrintedJobs flag is set.

Lessons learned items can be implemented via a number of means, including (but not limited to):
- Yara Rules
- RegRipper plugins
- WEVTX eventmap,txt entry (here's a good one to add...)
- EDR detection rules
- snort rules
- Etc.

I call this as"baking" what we've learned back into our processes and methodologies.  A great example of implementing something like this occurred several years ago when Nuix added two extensions to their Workstation product, one for Yara, and one for RegRipper.  This allows anyone who adds these extensions (and the open source tools) to their analysis platform can now take full advantage of the collective experience of hundreds of analysts and investigators, without ever having to engaged in the same hundreds or thousands of investigations.  An analyst who'd downloaded this Yara rule, for example, could now use it to scan images for all of the tools listed without ever having seen a single one of them.  The same is true for web shells, PHP backdoors, etc.; acquired images can be automatically scanned for these items without the analyst having to have ever seen one.

To some extent, this is more of using the lessons learned by others to extend our analysis, but anything you learn yourself from this process could then be added right back into the environment.  If someone wanted, they could also add an extension for processing Windows Event Log files and tagging specific events (by source and ID, or other criteria)

Extending Your Reach
Not everything we know needs to be the result of our own personal experiences.  Jim Mattis stated in his book that "our own personal experiences are not enough to sustain us", and the same applies to DFIR.  No one of us is as smart as all of us working together.  If someone finds something and makes it public, it would behoove those of us impacted (even potentially so) by the finding to implement that finding in a manner that works for our environment and tooling.

Clearly, this goes back to the Data Sources section of this blog post, but that simply echos the fact that this is an iterative process.  As you find or learn new things, you add them to collection and pre-processing methodology.  As these new findings are tagged in subsequent engagements, analysis will raise new findings "near" our tagged items, allowing us to then extend our collection and pre-processing methodology further.

The same methodology can be applied to EDR, as well.  Let's say you create a detection that alerts you when printer Attributes values are modified, and you receive such an alert.  A review of EDR metadata "near" that alert may not give you the visibility you need, so you implement your forensic collection process and run the collected data through your pre-processing toolset.

Saturday, January 04, 2020

First RegRipper 2020 Update

'cogphn' recently reached to me via the RegRipper GitHub repo to let me know that they'd found an issue with a plugin, and this was followed by a similar issue posted by William Schaefer.  It seems that as soon as the clocks rolled over to 2020, the function within the Parse::Win32Registry module that gets key LastWrite times and translates them from FILETIME to Unix epoch format broke, and started returning 0.  I fixed the function, and recompiled the executable files for rip.exe and the RegRipper GUI (rr.exe), and uploaded copies of the files to GitHub.

If you're just using the Windows executables, all you should need to do at this point is simply download the updated EXE files and you'll be on your way.  I've done some local testing and things seem to be back on track.  If you do find that there are continued issues, please let me know.

If you're running RegRipper as its Perl variant, I've also uploaded the 'fixed' Base.pm file, and included instructions in the readme file regarding updating your local copy of the file.

Addenum, 10 Jan
This issue also impacted regtime.exe, so I went ahead and recompiled the EXE and uploaded it to the Github repo.  The source code (Perl script) did not require any modifications, but if you are running the Perl script, be sure to get the new Base.pm file and replace the one in your Parse::Win32Registry install, if you haven't already.