Threat Hunting, IRL

While I worked for one company, I did a lot of public speaking on the value of threat hunting. During these events, I met a lot of folks who were interested to learn what "threat hunting" was, and how it could be of value to them.

I live in a very rural area, on just shy of 19 acres. One neighbor has 15 acres up front and another 20 in the back, and he adjoins a large property with just a trailer. My neighbor on the other side has 19 acres of...just 19 acres. We have animals, as well as more than a few visitors, which makes for a great analogy for threat hunting.

Within the borders of my property, we have three horses and a mini-donkey, and we have different paddocks and fields for them. We can restrict them to certain areas, or allow them to roam freely. We do this at different times of the year, depending upon weather, availability of hay, etc. For example, in the spring, when the grass is coming in really well, we don't want the horses on it too soon or for too long, because they can colic (which is a bad thing). And we may want to cut the grass (do maintenance), so we'll restrict the horses from that area.

I understand the normal comings and goings of the horses, because I have full visibility. I can not only see most of the areas (albeit not all) from the house, but I get out and walk around the property. I am familiar with the normal habits of the horses, and understand how they respond to various "events". I also know when something is amiss, simply by watching the horses. This is my "infrastructure".

Like most horse owners, we provide them with salt and mineral licks, in the form of 40 lb blocks. We make this available to them year-round, replacing blocks as they get diminished. Even so, we've also notices that the horses will scratch at certain spots on the ground, and then spend a good bit of time happily licking the ground. Knowing this, we try to keep up on "pasture maintenance"; we pick up the poop, or drag the field, so that the horses don't get worms. We also know what the spots look like, and that they're different from where the horses like to roll. Where they scrape and lick, the ground is bare, and there are usually rounded marks where their hoof initially contacts the ground, before they drag it across the ground to break up the earth. Where they roll, there is usually still some semblance of grass left, and there's also hair left. In addition, the marks from their hoofs, where the horses circle before laying down and then when they get back up, are different from where they scrape the ground. All in all, this is normal, expected "user behavior". 

Walking the dog around the property this year, I noticed something I haven't seen before...there are bare spots on the trails where the leaves and grass have been scraped away and the ground exposed. This seems very similar to what I've seen the horses do, however, there are differences. First, these spots are in areas that the horses do not usually frequent unless we're riding them. Second, instead of rounded hoof marks, there are distinct two-toed marks in the ground. Third, these exposed areas are usually much smaller than what I've seen associated with the horses. From what I've observed, these are deer doing the same thing as the horses, and if the only IOC someone shared with me was "bare spot on the ground", I would assume that it was most likely from the horses (i.e., normal user activity). However, if I look beyond the IOC, and look at the specifics of the activity (i.e., TTPs), I'd then be able to clearly differentiate between "normal user activity" and "potentially unwanted/malicious activity". 

My neighbors recently shared "threat intel"...they'd seen a bear on their property. Now, they'd done more than just stated, "hey, we saw a bear!" That's right...they did more than just share an IOC. In fact, they took a picture, indicated the time of day, and the picture gave an indication of where the bear was located on their property. So we had other things to consider than just "a bear"...size, color, type, direction of movement, etc. As a result, I'm now on the look-out for these TTPs, as well as others known to be associated with this particular threat; scat, claw marks on trees, etc. We have persimmon trees in the area, and while I have seen scat from various animals that contains persimmon seeds, I have yet to see bear scat. But I am aware of the "threat", and looking for clear indications of that threat.

My neighbors did more than just share an IOC, they shared clear TTPs, and enough of it such that I could search for indications of that threat within my infrastructure.

We've lived on this property for more than 4 1/2 yrs, and it's only been in the past year that I've found two turkey eggs. In both cases, they've been broken, and based on the condition of the eggs, I can't tell if the chick hatched, or if the egg was a meal for a raccoon. Regardless, it does tell me that there are very likely turkeys in the area. While we are knowledgeable and understand the nature of turkeys and the "risk" they bring to the "infrastructure", the horses have a completely different view. To the horses, a turkey scurrying through the underbrush may as well be a velociraptor released from it's cage, right out of Jurrasic Park. While sitting in my office, a turkey is not a threat to me; however, while I am on horseback, a turkey could be a "threat", in that spooking the horse I am riding might have a severely negative impact on my day. I have another threat that I'm aware of, and because I have detailed visibility of my environment, and because I understand the nature of the threat, I understand the risk.

For me, it's interesting to take a step back and look at how my IRL life parallels my work life. Or, maybe my IRL life is being viewed through the lens of my work life. Either way, I thought that there were some interesting parallels.

Tips for DFIR Analysts, pt. V

Over the years, I've seen DFIR referred to in terms of special operations forces. I've seen incident response teams referred to as "Cyber SEALs", as well as via various other terms. However, when you really look at, incident response is much more akin to the US Army Special Forces, aka "Green Berets"; you have to "parachute in" to a foreign environment, and quickly develop a response capability making use of the customer's staff ("the natives"), all of whom live in a "foreign culture". As such, IR is less about "direct action" and "hostage rescue", and more about "foreign internal defense".

Analysis occurs when an analyst applies their knowledge and experience to data, and is usually predicated by a parsing phase. We can learn a great deal about the analyst's level of knowledge and experience by what data they collect, how they approach data NOT addressed during the initial collection phase, how they go about parsing the data, and then how they go about presenting their findings.

Analysts need to understand that there is NO SUCH THING as an attack that leaves no traces. You may hear pen testers and fellow DFIR analysts saying this, often with great authority, but the simple fact is that this is NOT THE CASE. EVER. I heard this quite often when I was a member of the ISS X-Force ERS team; at one point, the vulnerability discovery team told me that they'd discovered a vulnerability to Excel that left NO TRACES of execution on the endpoint. Oddly enough, they stopped talking to me all together when I asked if the user had to open...wait for it...an Excel file. Locard's Exchange Principle tells us that there will be some traces of the activity. Some artifacts in the constellation may be more transient than others, but any "attack" requires the execution of code (processing of instructions through the CPU) on the system, even those that include the use of "native tools" or "LOLBins". 

Take what you learn from one engagement, and bake it back into your process. Automate it, so that the process is documented (or self-documenting) and repeatable.

Over on Twitter, Chris Sanders has a fascinating thread "discussing" the ethics around the release of OSTs. He has some pretty important points, one being that cybersecurity does not have any licensing body, and as such, no specific code of ethics or conduct to which one must adhere. Section 17 of the thread mentions releasing OSTs being a "good career move"; do we see that on the "blue" side? I'd suggest, "no". Either way, this is a really good read, and definitely something to consider...particularly the source of "data" in the thread. Chris later followed up with this article, titled, "A Socratic Outline for Discussing the OST Release Debate".

Tools
Not long ago, I ran across this LinkedIn post on SCCM Software Metering; this can potentially provide insight into program execution on host (where other artifacts, such as AmCache and ShimCache, do not explicitly illustrate program execution). Remember, this should be combined with other validating artifacts, such as Prefetch files, EDR telemetry (if possible), etc., and not considered in isolation. Microsoft has a document available that addresses software metering, and FireEye has some great info available on the topic, and David Pany has a tool available for parsing this information from the OBJECTS.DATA file.

Inversecos had a great Tweet thread on RDP artifacts not long ago, and in that thread, linked to an artifact parsing tool, BMC-Tools. Looking around a bit, I found another one, RdpCacheStitcher. Both (or either) can provide valuable insight into putting together "the story" and validating activity.

Endpoint Tools
I wanted to pull together a compilation of tools I'd been collecting, and just put them out there for inspection. I haven't had the opportunity to really use these tools, but in putting them together into the below rather loose list, I'm hoping that folks will see and use them...

DFIR Orc/Zircolite - LinkedIn message
DFIR Orc - Forensic artifact collection tool
Zircolite - Standalone SIGMA-based tool for EVTX
Chainsaw - tool for rapidly searching Windows Event Logs
Aurora from Nextron Systems (presentation) - SIGMA-based EDR agent
PCAP Parser (DaisyWoman) - HTTP/S queries & responses, VT scans 
*This parser is great for using after you use bulk_extractor to retrieve a PCAP file from memory, or other unstructured data
Kaspersky Labs parse_evtx.exe binary

Images/Labs
If you want to practice using any of the above tools, or you want to practice using other tools and techniques, and (like me), you're somewhat limited at to what you have access, here are some resources you might consider exploring...

Digital Forensics Lab & Shared Cyber Forensic Intelligence Repository
CyberDefenders Labs - a fair number of labs available
Cado Security REvil Ransomware Attack Image/Data
Ali Hadi's DataSets - lots of good data sets available 
MUS2019 DFIR CTF (via David Cowen) 

2018 DefCon CTF (here, here)
DigitalCorpora Narcos Scenario (write-up)
CalPoly 2019 DF Downloads
DFIRMadness Stolen Szechuan Sauce (Twitter) (Answers)

Unofficial DefCon 2019 CTF (write-up, write-up)

Stark4N6 BloomCon CTF

Tips for DFIR Analysts, pt IV

Context is king, it makes all the difference. You may see something run in EDR telemetry, or in logs, but the context of when it ran in relation to other activities is often critical. Did it occur immediately following a system reboot or a user login? Does it occur repeatedly? Does it occur on other systems? Did it occur in rapid succession with other commands, indicating that perhaps it was scripted? The how and when of the context then leads to attribution.

Andy Piazza brings the same thoughts to CTI in his article, "CTI is Better Served with Context".

Automation can be a wonderful thing, if you use it, and use it to your advantage. The bad guys do it all the time. Automation means you don't have to remember steps (because you will forget), and it drives consistency and efficiency. Even at the micro-level, at the level of the individual analyst's desktop, automation means that data sources can be parsed, enriched, decorated and presented to the analyst, getting them to analysis faster. Admit it...parsing all of the data sources you have access to, the way you're doing it now, is terribly inefficient and error-prone. Imagine if you could use the system, the CPU, to do that for you, and have pivot points identified when you access the data, rather than having to discover them for yourself. Those pivot points could be based on your own individual experience, but what if it were based on the sum total experience of all analysts on the team, including analysts who were on the team previously, but are no longer available?

Understand and use the terminology of your industry. All industries have their own common terminology because if facilitates communications, as well as identifying outsiders. Doctors, lawyers, Marines all have terms and phrases that mean something very specific to the group or "tribe". Learn what those are for your industry or community, understand them and use them.  

Not all things are what we think they are...this is why we need to validate what we think are "findings", but are really assumptions. Take Windows upgrades vs. updates...an analyst may believe that a lack of Windows Event Logs is the result of an upgrade when they (a) do not have Windows Event Log records that extend back to the time in question, and (b) see that there was a great deal of file system and Registry activity associated with an update, "near" that time. Most often the lack of Windows Event Logs is assumed to be the threat actor "covering" their tracks, and this assumption will often persist despite the lack of evidence pointing to this activity (i.e., batch files or EDR telemetry illustrating the necessary command, specific event IDs, etc.). The next step, in the face of specific event IDs "missing", is to assume that the update caused the "data loss", without realizing the implication of that statement...that a Windows update would lead to data loss. How often do we see these assumptions, when the real reason for a dearth of Windows Event Logs covering a specific time frame is simply the passage of time?