There's been a lot of chatter over the use of AI in various fields, and because it's my professional focus, I'm most interested in how it's used in cybersecurity. Now, that doesn't mean that I'm not aware of how it's used...or more appropriately, misused...in other fields, as well. For example, how it's been misused in the legal field has been around for more than 2 years now, and just last year, we saw the term "AI slop" be adopted in the software dev/cybersecurity field.
Something we also saw in 2025 was the release of the Anthropic report regarding how AI was used by threat actors, in a cyber espionage campaign. The report is 14 pages long, with the title page, table of contents, and a 2-pg Executive Summary; the contents of the report itself starts on pg 6.
The "TL;DR" of the report, if you need it, is that nation-state threat actors used Claude to target 30 organizations, and "...to execute 80-90% of tactical operations independently at physically impossible request rates."
That's right...they used AI to run up to an estimated 90% of their attack chain autonomously.
So, what does this mean? It means that "low and slow" was out the window, and that the attack chains were automated a "physically impossible request rates".
That's it. Everything was faster. Reading through the report, it becomes clear that tools and techniques employed were akin to those commonly observed in human-operated attacks, but the OODA loop was much smaller, much tighter, and iterated through much faster than humanly possible. On the defender's side, this means that artifacts were generated (and hopefully, alerts fired) much closer together than what would've been observed earlier in the year.
In response to the report, Matthew shared his thoughts, in which he shared the following:
That report, fascinating as it was, also left some cybersecurity pros with a bad taste in their mouth. Specifically: They wanted Anthropic to share some real threat intelligence. Get into the IOCs. The specific prompts. The real nitty-gritty details that defenders can use, that are a staple of the threat-sharing genre.
I have to say, and I did say in the comments, that I'm not sure that based on my aperture, I agree with Matthew. For example, I'm not sure what value the prompts used by the threat actors would have for defenders. While it's clear that using AI increases the velocity and volume of attacks, there was nothing obvious shared in the Anthropic report that says any of the tools or techniques used by the AI was any different from what a human would do, only that it was faster. Anthropic has clearly addressed issues that they've found as a result of their investigation, and the prompts would be of little value to defenders, and at this point, maybe provide indications (for the offensive side) of what worked, at one point in time, with Claude.
Okay, to level set...we're at a place where AI has been used in offensive cybersecurity, but let's be clear as to how it's been used, rather than freaking out about...we don't know what. According to Anthropic's report, their product was used to attack 30 or so organizations, with up to (estimated) 90% of the tactical operations handled autonomously by Claude, making the overall attacks much faster, moving away from the "low and slow" that some of us are used to seeing. What any individual, targeted organization likely saw was probably fast and loud.
And before you go thinking that using AI to autonomously run attacks is "all that", Figure 1 illustrates an excerpt from the report's Executive Summary.
 |
| Fig. 1: Report excerpt |
Now, whether you call these "hallucinations" or coding errors, it does illustrate issues with running with whatever AI says, as others have found.
Finally, consider this...threat actors used Anthropic's Claude, which apparently is known to log prompts. In the Executive Summary of the report, it states, "...while we only have visibility into Claude usage...". I'll just leave that right there for you to consider, in terms of operational security.
On the Blue Side
So what does, or would, use of AI on the blue side, during DF, SOC, or CTI work look like?
On 15 Jan 2026, I saw this LinkedIn post from Rob Lee (of SANS fame) that announced the release of "Protocol SIFT", described as "Claude Code integrated with SIFT Workstation, using MCP". Figure 2 illustrates an excerpt of the LinkedIn post that describes how this all works together.
 |
| Fig. 2: LinkedIn post excerpt |
Are users really going to use "find evil" as a prompt? I hope not, but I can also see that Rob is likely being a bit "light" in this instance. My concern would be that, if you're unable to articulate the goals of your examination, then how are you going to get Claude to do it for you?
Recently, Chiara Gallese shared this post on LinkedIn, in which she described an MIT lecturer who used ChatGPT in an attempt to understand the Riemann Hypothesis, a math problem that hasn't been solved in 160 years. I think what stood out to me most of all in Chiara's post was the following two statements:
And in the end, he did not understand the problem.
He just understood ChatGPT’s simplifications of his own confusion.
If we don't understand that problem, how do we know if AI (Claude, ChatGPT, whatever) is doing a better job, or just getting us to failure faster?
Something SOC and DF/IR analysts have moved to over the years is triage collections, acquiring minimal data from endpoints in order to answer investigation goals in as timely a manner as possible. After all, why acquire a full image (which can take hours) when the data you need (during a breach investigation) is often limited to a few megabytes. Could you use something like Protocol SIFT to conduct analysis? Sure, but what happens if the AI hallucinates something not supported by the data, such as an endpoint being configured to not record successful logins (saw just such an endpoint today)? What happens if the AI hallucinates that source of the login activity, and then performs attribution (because you asked it to) based on that hallucination? How would you know that the findings provided are incorrect, if you don't understand the goals of the investigation and the data retrieved to support that investigation?
About half a dozen years ago, I was supporting a DF/IR team that was addressing a fair number of ransomware incidents when Microsoft published their first iteration of the human-operated ransomware attacks blog post. In that blog post, where the Doppelpaymer ransomware is discussed, the authors mention that the ransomware operators may have relied on WMI persistence mechanisms, so I asked our team if they'd seen any. The response was "no".
However, while the front-end data collection did include the WMI repository, the middleware used to parse the data did not include a parser for the WMI repository, so the question became, how can you say "no" if the WMI repository wasn't parsed?
This may sound like a one-off, but I saw the same mentality years before, and I've seen it since then, as well. Given everything else we're facing in 2026, I'm going to state, emphatically, that using AI on the blue side for no other reason than that the bad guys are using it is not a good reason, and that if you can't clearly articulate your analysis goals, then AI should not be used at all.
Look at it this way...
No comments:
Post a Comment