I've posted regarding my thoughts on a Forensic Scanner before, and just today, I gave a short presentation at the #OSDFC conference on the subject.
Rather than describing the scanner, this time around, I released it. The archive contains the Perl source code, a 'compiled' executable that you can run on Windows without installing Perl, a directory of plugins, and PDF document describing how to use the tool. I've also started populating the wiki with information about the tool, and how it's used.
Some of the feedback I received on the presentation was pretty positive. I did chat with Simson Garfinkel for a few minutes after the presentation, and he had some interesting thoughts to share. The second was on making the Forensic Scanner thread-safe, so it can be multi-threaded and use multiple cores. That might make it on to the "To-Do" list, but it was Simson's first thought that I wanted to address. He suggested that at a conference were the theme seemed to revolve around analysis frameworks, I should point out the differences between the other frameworks and what I was presenting on, so I wanted to take a moment to do that.
Brian presented on Autopsy 3.0, and along with another presentation in the morning, discussed some of the features of the framework. There was the discussion of pipelines, and having modules to perform specific functions, etc. It's an excellent framework that has the capability of performing functions such as parsing Registry hives (utilizing RegRipper), carving files from unallocated space, etc. For more details, please see the web site.
I should note that there are other open source frameworks available, as well, such as DFF.
The Forensic Scanner is...different. Neither better, nor worse...because it addresses a different problem. For example, you wouldn't use the Forensic Scanner to run a keyword search or carve unallocated space. The scanner is intended for quickly automating repetitive tasks of data collection, with some ability to either point the analyst in a particular direction, or perform a modicum of analysis along with the data presentation (depending upon how much effort you want to put into writing the plugins). So, rather than providing an open framework which an analyst can use to perform various analysis functions, the Scanner allows the analyst to perform discrete, repetitive tasks.
The idea behind the scanner is this...there're things we do all the time when we first initiate our analysis. One is to collect simple information from the system...it's a Windows system, which version of Windows is it, it's time zone settings, is it 32- or 64-bit, etc. We collect this information because it can significantly impact our analysis. However, keeping track of all of these things can be difficult. For example, if you're looking at an image acquired from a Windows system and don't see Prefetch files, what's your first thought? Do you check the version of Windows you're examining? Do you check the Registry values that apply to and control the system's prefetching capabilities? I've talked with examiners who's first thought is that the user must have deleted the Prefetch files...but how do you know?
Rather than maintaining extensive checklists of all of these artifacts, why not simply write a plugin to collect what data it is that you want to collect, and possibly add a modicum of analysis into that plugin? One analyst writes the plugin, shares it, and anyone with that plugin will have access to the functionality without having to have had the same experiences as the analyst. You share it with all of your analysts, and they all have the capability at their fingertips. Most analysts recognize the value of the Prefetch files, but some may not work with Windows systems all of the time, and may not stay up on the "latest and greatest" in analysis techniques that can be applied to those files. So, let's say that instead of dumping all of the module paths embedded in Prefetch files, you add some logic to search for .exe files, .dat files, and any file that includes "temp" in the path, and display that information? Or, why not create whitelists of modules over time, and have the plugin show you all modules not in that whitelist?
Something that I and others have found useful is that, instead of forcing the analyst to use "profiles", as with the current version of RegRipper, the Forensic Scanner runs plugins automatically based on OS type, class, and then organizes the plugin results by category. What this means is that for the system class of plugins, all of the plugins that pertain to "Program Execution" will be grouped together; this holds true for other plugin categories, as well. This way, you don't have to go searching around for the information in which you're interested.
As I stated more than once in the presentation, the Scanner is not intended to replace analysis; rather, it's intended to get you to the point of performing analysis much sooner. For example, I illustrated a plugin that parses a user's IE index.dat file. Under normal circumstances when performing analysis, you'd have to determine which version of Windows you were examining, determine the path to the particular index.dat file that you're interested in, and then extract it and parse it. The plugin is capable of doing all of that...in the test case, all of the plugins I ran against a mounted volume completed in under 2 seconds...that's scans of the system, as well as both of the selected user profiles.
So...please feel free to try the Scanner. If you have any questions, you know where to reach me. Just know that this is a work-in-progress, with room for growth.
Addendum
Matt Presser identified an issue that the Scanner has with identifying user profiles that contain a dot. I fixed the issue and will be releasing an update once I make a couple of minor updates to other parts of the code.
Showing posts with label scanner. Show all posts
Showing posts with label scanner. Show all posts
Wednesday, October 03, 2012
Friday, November 04, 2011
DF Analysis Lifecycle
In an effort to spur some interest within the DFIR community (and specifically with the NoVA Forensics Meetup group) in engaging and sharing information, I thought it would be a good idea to point out "forensic challenges" or exercises that are available online, as well as to perhaps setup and conduct some exercises of our (the meetup group) own.
As I was thinking about how to do this, one thing occurred to me...whenever I've done something like this as part of a training exercise or engagement, many times the first things folks say is that they don't know how to get started. When I've conducted training exercises, they've usually been for mixed audiences..."mixed" in the sense that the attendees often aren't all just DF analysts/investigators; some do DF work part-time, some do variations of DF work (such as "online forensics") and others are SOC monitors and may not really do DF analysis.
As such, what I wanted to do was lay out the way I approach analysis engagements, and make that process available for others to read and comment on; I thought that would be a good way to get started on some of the analysis exercises that we can engage in going forward. I've included some additional resources (by no means is this a complete list) at the end of this blog post.
Getting Started
The most common scenario I've faced is receiving either a hard drive or an image for analysis. In many cases, it's been more than one, but if you know how to conduct the analysis of one image, then scaling it to multiple images isn't all that difficult. Also, acquiring an image is either one of those things that you can gloss over in a short blog post, or you have to write an entire blog post (or series of posts) on how to do it...so let's just start our examination based on the fact that we received an image.
Documentation
Documentation is the key to any analysis. It's also the hardest thing to get technical folks to do. For whatever reason, getting technical folks to document what they're doing is like herding cats down a beach. If you don't believe me...try it. Why it's so hard is up for discussion...but the fact of the matter is that proper documentation is an incredibly useful tool, and when you do it, you'll find that it will actually allow you to do more of the cool, sexy analysis stuff that folks like to do.
Document all the things!
Most often when we talk about documentation during analysis, we're referring to case notes, and as such, we need to document pretty much everything (please excuse the gratuitous meme) about the case that we're working on. This includes when we start, what we start with, the tools and processes/procedures we use, our findings, etc.
One of the documentation pitfalls that a lot of folks run into is that they start their case notes on a "piece of paper", and by the end of the engagement, those notes never quite make it into an electronic document. It's best to get used to (and start out) documenting your analysis in electronic format, particularly so your notes can be stored and shared. One means of doing so is to use Forensic CaseNotes from QCC. You can modify the available tabs to meet your needs. However, you can just as easily document what you're doing in MS Word; you can add bold and italics to the document to indicate headers, and you can even add images and tables (or embed Visio diagrams) to the document, if you need to.
The reasons why we document what we do are (1) you may get "hit by a bus" and another analyst may need to pick up your work, and (2) you may need to revisit your analysis (you may be asked questions about it) 6 months or a year later. I know, I know...these examples are used all the time and I know folks are tired of hearing them...but guess what? We use these examples because they actually happen. No, I don't know of an analyst who was actually "hit by a bus", but I do know of several instances where an analyst was on vacation, in surgery, or had left the organization, and the analysis had to be turned over to someone else. I also know of several instances where a year or more after the report was delivered to the customer, questions were posed...this can happen when you're engaged by LE and the defense has a question, or when you're engaged by an organization, and their compliance and regulatory bodies have additional questions. We often don't think much about these scenarios, but when they do occur, we very often finding ourselves wishing we'd kept better notes.
So, one of the questions I hear is, "...to what standard should I keep case notes?" Well, consider the two above scenarios, and keep your case notes such that (1) they can be turned over to someone else or (2) you can come back a year later and clearly see what you did. I mean, honestly...it really isn't that hard. For example, I start my case notes with basic case information...customer point of contact (PoC), exhibits/items I received, and most importantly, the goals of my exam. I put the goals right there in front of me, and have them listed clearly and concisely in their own section so that I can always see them, and refer back to them. When I document my analysis, I do so by including the tool or process that I used, and I include the version of the tool I used. I've found this to be critical, as tools tend to get updated. Look at EnCase, ProDiscover, or Mark Woan's JumpLister. If you used a specific version of a tool, and a year later that tool had been updated (perhaps even several times), then you'd at least have an explanation as to why you saw the data that you did.
Case notes should be clear and concise, and not include the complete output from every tool that you use or run. You can, however, include pertinent excerpts from tool output, particularly if that output leads your examination in a particular direction. By contrast, dumping the entire output of a tool into your case notes and including a note that "only the 3 of the last 4 lines in the output are important" is far from clear or concise. I would consider including information about why something is important or significant to your examination, and I've even gone so far as to include references, such as links to Microsoft KnowledgeBase articles, particularly if those references support my reasoning and conclusions.
If you keep your case notes in a clear and concise manner, then the report almost writes itself.
Now, I will say that I have heard arguments against keeping case notes; in particular, that they're discoverable. Some folks have said that because case notes are discoverable, the defense could get ahold of them and make the examiner's life difficult, at best. And yet, for all of these comments, no one has ever elaborated on this beyond the "maybe" and the "possibly". To this day, I do not understand why an analyst, as a matter of course, would NOT keep case notes, outside of being explicitly instructed to do so (i.e., to not keep case notes) by whomever you're working for.
Checklists
Often, we use tools and scripts in our analysis process in order to add some level of automation, particularly when the tasks are repetitive. A way to expand that is to use checklists, particularly for involved sets of tasks. I use a malware detection checklist that I put together based on a good deal of work that I'd done, and I pull out a copy of that checklist whenever I have an exam that involves attempting to locate malware within an acquired image. The checklist serves as documentation...in my case notes, I refer to the checklist, and I keep a completed copy of the checklist in the case directory along with my case notes. The checklist allows me to keep track of the steps, as well as the tools (and versions) I used, any significant findings, as well as any notes or justification I may have for not completing a step. For example, I won't run a scan for NTFS ADSs if the file system of the image is FAT.
The great thing about using a checklist is that it's a living document...as I learn and find new things, I can add them to the checklist. It also allows me to complete the analysis steps more thoroughly and completely, and in a timely manner. This, in turn, leaves me more time for things like conducting deep(er) analysis. Checklists and procedures can also be codified into a forensic scanner, allowing the "low hanging fruit" and artifacts that you've previously found to searched for quickly, thereby allowing you to focus on further analysis. If the scanner is designed to keep a log of it's activity, then you've got a good deal of documentation right there.
Remember that when using a checklist or just conducting your analysis, no findings can be just as important as an interesting finding. Let's say that you have a checklist that includes 10 steps, and of those, only 1 step finds anything interesting. Let's say you follow all 10 (again, purely arbitrary number, used only as an example) steps of your malware detection checklist, and only the ADS detection step finds anything of interest, but it turns out to be nothing. If you choose to not document the steps that had no significant findings, what does that tell another analyst who picks up your case, or what does it tell the customer who reads your report? Not much. In fact, it sounds like all you did was run a scan for ADSs...and the customer is paying how much for that report? Doing this makes whomever reads your report think that you weren't very thorough, when you were, in fact, extremely thorough.
One final note about checklists and procedures...they're a good place to start, but they're by no means the be-all-end-all. They're tools...use them as such. Procedures and checklists often mean the difference between conducting "Registry analysis" and getting it knocked out, and billing a customer for 16 hrs of "Registry analysis", with no discernible findings or results. If you run through your checklist and find something odd or interesting (for example, no findings), use that as a launching point from which to continue your exam.
Start From The End
This is advice that I've given to a number of folks, and I often get a look like I just sprouted a third eye in the middle of my forehead. What do you mean, "start at the end"? Well, this goes back to the military "backwards planning" concept...determine where you need to be at the end of the engagement (clear, concise report delivered to a happy customer), and plan backwards based on where you are now (sitting at your desk with a drive image to analyze). In other words, rather than sitting down with a blank page, start with a report template (you know you're going to have to deliver a report...) and work from there.
Very often when I have managed engagements, I would start filling in the report template while the analyst (or analysts) was getting organized, or even while they were still on-site. I'll get the executive summary knocked out, putting the background and goals (the exact same goals that the analyst has in their case notes) into the report, and replicating that information into the body of the report. That leaves the analyst to add the exhibits (what was analyzed) and findings information into the report, without having to worry about all of the other "stuff", and allows them to focus on the cool part of the engagement...the analysis. Using a report template (and using the same one every time), they know what needs to be included where, and how to go about writing their findings (i.e., clear and concise). As mentioned previously, the analysis steps and findings are often taken directly from the case notes.
What's the plan, Stan?
Having an analysis plan to start with can often be key to your analysis. Have you ever seen someone start their analysis by loading the image into an analysis application and start indexing the entire image? This activity can take a great deal of time, and we've all seen even commercial applications crash during this process. If you're going to index an entire image, why are you doing so? In order to conduct keyword searches? Okay...what's your list of keywords?
My point is to think critically about what you're doing, and how you're going to go about doing it. Are you indexing an entire image because doing so is pertinent to your analysis, or "because that's what we've always done"? If it's pertinent, that's great...but consider either extracting data from the image or making an additional working copy of the image before kicking off the indexing process. That way, you can be doing other analysis during the indexing process. Also, don't waste time doing stuff that you don't need to be doing.
Report Writing
No one likes to write reports. However, if we don't write reports, how do we get paid? How do we communicate our findings to others, such as the customer, or the prosecutor, or to anyone else? Writing reports should not be viewed as a necessary evil, but instead as a required skill set.
When writing your report, as with your case notes, be clear and concise. There's no need to be flowery and verbose in your language. Remember, you're writing a report that takes a bunch of technical information and very often needs to translate that into something a non-technical person needs to understand in order to make a business or legal decision. It's not only harder to make up new verbiage for different sections of your report, it also makes the finished product harder to read and understand.
When walking through the analysis or findings portion of the report (leading up to my conclusions), I've found that it's best to use the same cadence and structure in my writing. It not only makes it easier to write, but it also makes it easier to read. For example, if I'm analyzing an image in order to locate suspected malware, in each section, I'll list what I did ("ran AV scan"), which tools I used ("AV scanner blah, version X"), and what I found ("no significant/pertinent findings", or "Troj/Win32.Blah found"). I've found that when trying to convey technical information to a non-technical audience, using the same cadence and structure over and over often leaves the reader remembering the aspects of the report that you want them to remember. In particular, you want to convey that you did a thorough job in your analysis. In contrast, having each section worded in a significantly different manner not only makes it harder for me to write (I have to make new stuff up for each section), but the customer just ends up confused, and remembering only those things that were different.
Be professional in your reporting. You don't have to be verbose and use $5 words; in fact, doing so can often lead to confusion because you've used a big word incorrectly. Have someone review your report, and for goodness sake, run spell check before you send it in for review! If you run spell check and see a bunch of words underlined with red squiggly lines, or phrases underlined with green squiggly lines, address them. Get the report in for review early enough for someone to take a good look at it, and don't leave it to the last minute. Finally, if there's something that needs to be addressed in the report, don't tell your reviewer, "fine, if you don't like it, fix it yourself." Constructive criticism is useful and helps us all get better at what we do, but the petulant "whatever...fix it yourself" attitude doesn't go over well.
The report structure is simple...start with an executive summary (ExSumm). This is exactly as described...it's a summary for executives. It's not a place for you to show off how many really cool big words you know. Make it simple and clear...provide some background info on the incident, the goals of the analysis (as decided upon with the customer) and your conclusions. Remember your audience...someone non-technical needs a clear and concise one-pager (no more than 2) with the information that they can use to make critical business decisions. Were they compromised? Yes or no? There's no need to pontificate on how easily they had been compromised...just be clear about it. "A successful SQL injection attack led to the exposure of 10K records."
The body of the report should include background on the incident (with a bit more detail than the ExSumm), followed by the exhibits (what was analyzed), and the goals of the analysis. From there, provide information on the analysis you conducted, your findings, and your conclusions. The goals and conclusions from the body of the report should be identical...literally, copy-and-paste...from the ExSumm.
Finally, many reports include some modicum of recommendations...sometimes this is appropriate, other times it isn't. For example, if you're looking at 1 or 10 images, does that really give you an overall view into the infrastructure as a whole? Just because MRT isn't up-to-date on 5 systems, does that mean that the organization needs to develop and implement a patch management infrastructure? How do you know that they haven't already? This is the part of the report that is usually up for discussion, as to whether or not it's included.
Summary
So, my intention with this post has been to illustrate an engagement lifecycle, and to give an overview of what an engagement can look like, cradle-to-grave. This has by no means been intended to be THE way of doing things...rather, this is a way of conducting an engagement that has been useful to me, and I've found to be successful.
Resources
Chris Pogue's "Sniper Forensics: One Shot, One Kill" presentation from DefCon18
Chris Pogue's "Sniper Forensics v.3" from the most recent SecTor (scroll down)
TrustWave SpiderLabs "Sniper Forensics" blog posts (five posts in the series)
Girl, Unallocated On Writing
UnChained Forensics Lessons Learned
Brad Garnett's tips on Report Writing (SANS)
Computer Forensics Processing Checklist
Useful Analysis Tidbits
Corey's blog posts on exploit artifacts
As I was thinking about how to do this, one thing occurred to me...whenever I've done something like this as part of a training exercise or engagement, many times the first things folks say is that they don't know how to get started. When I've conducted training exercises, they've usually been for mixed audiences..."mixed" in the sense that the attendees often aren't all just DF analysts/investigators; some do DF work part-time, some do variations of DF work (such as "online forensics") and others are SOC monitors and may not really do DF analysis.
As such, what I wanted to do was lay out the way I approach analysis engagements, and make that process available for others to read and comment on; I thought that would be a good way to get started on some of the analysis exercises that we can engage in going forward. I've included some additional resources (by no means is this a complete list) at the end of this blog post.
Getting Started
The most common scenario I've faced is receiving either a hard drive or an image for analysis. In many cases, it's been more than one, but if you know how to conduct the analysis of one image, then scaling it to multiple images isn't all that difficult. Also, acquiring an image is either one of those things that you can gloss over in a short blog post, or you have to write an entire blog post (or series of posts) on how to do it...so let's just start our examination based on the fact that we received an image.
Documentation
Documentation is the key to any analysis. It's also the hardest thing to get technical folks to do. For whatever reason, getting technical folks to document what they're doing is like herding cats down a beach. If you don't believe me...try it. Why it's so hard is up for discussion...but the fact of the matter is that proper documentation is an incredibly useful tool, and when you do it, you'll find that it will actually allow you to do more of the cool, sexy analysis stuff that folks like to do.
Most often when we talk about documentation during analysis, we're referring to case notes, and as such, we need to document pretty much everything (please excuse the gratuitous meme) about the case that we're working on. This includes when we start, what we start with, the tools and processes/procedures we use, our findings, etc.
One of the documentation pitfalls that a lot of folks run into is that they start their case notes on a "piece of paper", and by the end of the engagement, those notes never quite make it into an electronic document. It's best to get used to (and start out) documenting your analysis in electronic format, particularly so your notes can be stored and shared. One means of doing so is to use Forensic CaseNotes from QCC. You can modify the available tabs to meet your needs. However, you can just as easily document what you're doing in MS Word; you can add bold and italics to the document to indicate headers, and you can even add images and tables (or embed Visio diagrams) to the document, if you need to.
The reasons why we document what we do are (1) you may get "hit by a bus" and another analyst may need to pick up your work, and (2) you may need to revisit your analysis (you may be asked questions about it) 6 months or a year later. I know, I know...these examples are used all the time and I know folks are tired of hearing them...but guess what? We use these examples because they actually happen. No, I don't know of an analyst who was actually "hit by a bus", but I do know of several instances where an analyst was on vacation, in surgery, or had left the organization, and the analysis had to be turned over to someone else. I also know of several instances where a year or more after the report was delivered to the customer, questions were posed...this can happen when you're engaged by LE and the defense has a question, or when you're engaged by an organization, and their compliance and regulatory bodies have additional questions. We often don't think much about these scenarios, but when they do occur, we very often finding ourselves wishing we'd kept better notes.
So, one of the questions I hear is, "...to what standard should I keep case notes?" Well, consider the two above scenarios, and keep your case notes such that (1) they can be turned over to someone else or (2) you can come back a year later and clearly see what you did. I mean, honestly...it really isn't that hard. For example, I start my case notes with basic case information...customer point of contact (PoC), exhibits/items I received, and most importantly, the goals of my exam. I put the goals right there in front of me, and have them listed clearly and concisely in their own section so that I can always see them, and refer back to them. When I document my analysis, I do so by including the tool or process that I used, and I include the version of the tool I used. I've found this to be critical, as tools tend to get updated. Look at EnCase, ProDiscover, or Mark Woan's JumpLister. If you used a specific version of a tool, and a year later that tool had been updated (perhaps even several times), then you'd at least have an explanation as to why you saw the data that you did.
Case notes should be clear and concise, and not include the complete output from every tool that you use or run. You can, however, include pertinent excerpts from tool output, particularly if that output leads your examination in a particular direction. By contrast, dumping the entire output of a tool into your case notes and including a note that "only the 3 of the last 4 lines in the output are important" is far from clear or concise. I would consider including information about why something is important or significant to your examination, and I've even gone so far as to include references, such as links to Microsoft KnowledgeBase articles, particularly if those references support my reasoning and conclusions.
If you keep your case notes in a clear and concise manner, then the report almost writes itself.
Now, I will say that I have heard arguments against keeping case notes; in particular, that they're discoverable. Some folks have said that because case notes are discoverable, the defense could get ahold of them and make the examiner's life difficult, at best. And yet, for all of these comments, no one has ever elaborated on this beyond the "maybe" and the "possibly". To this day, I do not understand why an analyst, as a matter of course, would NOT keep case notes, outside of being explicitly instructed to do so (i.e., to not keep case notes) by whomever you're working for.
Checklists
Often, we use tools and scripts in our analysis process in order to add some level of automation, particularly when the tasks are repetitive. A way to expand that is to use checklists, particularly for involved sets of tasks. I use a malware detection checklist that I put together based on a good deal of work that I'd done, and I pull out a copy of that checklist whenever I have an exam that involves attempting to locate malware within an acquired image. The checklist serves as documentation...in my case notes, I refer to the checklist, and I keep a completed copy of the checklist in the case directory along with my case notes. The checklist allows me to keep track of the steps, as well as the tools (and versions) I used, any significant findings, as well as any notes or justification I may have for not completing a step. For example, I won't run a scan for NTFS ADSs if the file system of the image is FAT.
The great thing about using a checklist is that it's a living document...as I learn and find new things, I can add them to the checklist. It also allows me to complete the analysis steps more thoroughly and completely, and in a timely manner. This, in turn, leaves me more time for things like conducting deep(er) analysis. Checklists and procedures can also be codified into a forensic scanner, allowing the "low hanging fruit" and artifacts that you've previously found to searched for quickly, thereby allowing you to focus on further analysis. If the scanner is designed to keep a log of it's activity, then you've got a good deal of documentation right there.
Remember that when using a checklist or just conducting your analysis, no findings can be just as important as an interesting finding. Let's say that you have a checklist that includes 10 steps, and of those, only 1 step finds anything interesting. Let's say you follow all 10 (again, purely arbitrary number, used only as an example) steps of your malware detection checklist, and only the ADS detection step finds anything of interest, but it turns out to be nothing. If you choose to not document the steps that had no significant findings, what does that tell another analyst who picks up your case, or what does it tell the customer who reads your report? Not much. In fact, it sounds like all you did was run a scan for ADSs...and the customer is paying how much for that report? Doing this makes whomever reads your report think that you weren't very thorough, when you were, in fact, extremely thorough.
One final note about checklists and procedures...they're a good place to start, but they're by no means the be-all-end-all. They're tools...use them as such. Procedures and checklists often mean the difference between conducting "Registry analysis" and getting it knocked out, and billing a customer for 16 hrs of "Registry analysis", with no discernible findings or results. If you run through your checklist and find something odd or interesting (for example, no findings), use that as a launching point from which to continue your exam.
Start From The End
This is advice that I've given to a number of folks, and I often get a look like I just sprouted a third eye in the middle of my forehead. What do you mean, "start at the end"? Well, this goes back to the military "backwards planning" concept...determine where you need to be at the end of the engagement (clear, concise report delivered to a happy customer), and plan backwards based on where you are now (sitting at your desk with a drive image to analyze). In other words, rather than sitting down with a blank page, start with a report template (you know you're going to have to deliver a report...) and work from there.
Very often when I have managed engagements, I would start filling in the report template while the analyst (or analysts) was getting organized, or even while they were still on-site. I'll get the executive summary knocked out, putting the background and goals (the exact same goals that the analyst has in their case notes) into the report, and replicating that information into the body of the report. That leaves the analyst to add the exhibits (what was analyzed) and findings information into the report, without having to worry about all of the other "stuff", and allows them to focus on the cool part of the engagement...the analysis. Using a report template (and using the same one every time), they know what needs to be included where, and how to go about writing their findings (i.e., clear and concise). As mentioned previously, the analysis steps and findings are often taken directly from the case notes.
What's the plan, Stan?
Having an analysis plan to start with can often be key to your analysis. Have you ever seen someone start their analysis by loading the image into an analysis application and start indexing the entire image? This activity can take a great deal of time, and we've all seen even commercial applications crash during this process. If you're going to index an entire image, why are you doing so? In order to conduct keyword searches? Okay...what's your list of keywords?
My point is to think critically about what you're doing, and how you're going to go about doing it. Are you indexing an entire image because doing so is pertinent to your analysis, or "because that's what we've always done"? If it's pertinent, that's great...but consider either extracting data from the image or making an additional working copy of the image before kicking off the indexing process. That way, you can be doing other analysis during the indexing process. Also, don't waste time doing stuff that you don't need to be doing.
Report Writing
No one likes to write reports. However, if we don't write reports, how do we get paid? How do we communicate our findings to others, such as the customer, or the prosecutor, or to anyone else? Writing reports should not be viewed as a necessary evil, but instead as a required skill set.
When writing your report, as with your case notes, be clear and concise. There's no need to be flowery and verbose in your language. Remember, you're writing a report that takes a bunch of technical information and very often needs to translate that into something a non-technical person needs to understand in order to make a business or legal decision. It's not only harder to make up new verbiage for different sections of your report, it also makes the finished product harder to read and understand.
When walking through the analysis or findings portion of the report (leading up to my conclusions), I've found that it's best to use the same cadence and structure in my writing. It not only makes it easier to write, but it also makes it easier to read. For example, if I'm analyzing an image in order to locate suspected malware, in each section, I'll list what I did ("ran AV scan"), which tools I used ("AV scanner blah, version X"), and what I found ("no significant/pertinent findings", or "Troj/Win32.Blah found"). I've found that when trying to convey technical information to a non-technical audience, using the same cadence and structure over and over often leaves the reader remembering the aspects of the report that you want them to remember. In particular, you want to convey that you did a thorough job in your analysis. In contrast, having each section worded in a significantly different manner not only makes it harder for me to write (I have to make new stuff up for each section), but the customer just ends up confused, and remembering only those things that were different.
Be professional in your reporting. You don't have to be verbose and use $5 words; in fact, doing so can often lead to confusion because you've used a big word incorrectly. Have someone review your report, and for goodness sake, run spell check before you send it in for review! If you run spell check and see a bunch of words underlined with red squiggly lines, or phrases underlined with green squiggly lines, address them. Get the report in for review early enough for someone to take a good look at it, and don't leave it to the last minute. Finally, if there's something that needs to be addressed in the report, don't tell your reviewer, "fine, if you don't like it, fix it yourself." Constructive criticism is useful and helps us all get better at what we do, but the petulant "whatever...fix it yourself" attitude doesn't go over well.
The report structure is simple...start with an executive summary (ExSumm). This is exactly as described...it's a summary for executives. It's not a place for you to show off how many really cool big words you know. Make it simple and clear...provide some background info on the incident, the goals of the analysis (as decided upon with the customer) and your conclusions. Remember your audience...someone non-technical needs a clear and concise one-pager (no more than 2) with the information that they can use to make critical business decisions. Were they compromised? Yes or no? There's no need to pontificate on how easily they had been compromised...just be clear about it. "A successful SQL injection attack led to the exposure of 10K records."
The body of the report should include background on the incident (with a bit more detail than the ExSumm), followed by the exhibits (what was analyzed), and the goals of the analysis. From there, provide information on the analysis you conducted, your findings, and your conclusions. The goals and conclusions from the body of the report should be identical...literally, copy-and-paste...from the ExSumm.
Finally, many reports include some modicum of recommendations...sometimes this is appropriate, other times it isn't. For example, if you're looking at 1 or 10 images, does that really give you an overall view into the infrastructure as a whole? Just because MRT isn't up-to-date on 5 systems, does that mean that the organization needs to develop and implement a patch management infrastructure? How do you know that they haven't already? This is the part of the report that is usually up for discussion, as to whether or not it's included.
Summary
So, my intention with this post has been to illustrate an engagement lifecycle, and to give an overview of what an engagement can look like, cradle-to-grave. This has by no means been intended to be THE way of doing things...rather, this is a way of conducting an engagement that has been useful to me, and I've found to be successful.
Resources
Chris Pogue's "Sniper Forensics: One Shot, One Kill" presentation from DefCon18
Chris Pogue's "Sniper Forensics v.3" from the most recent SecTor (scroll down)
TrustWave SpiderLabs "Sniper Forensics" blog posts (five posts in the series)
Girl, Unallocated On Writing
UnChained Forensics Lessons Learned
Brad Garnett's tips on Report Writing (SANS)
Computer Forensics Processing Checklist
Useful Analysis Tidbits
Corey's blog posts on exploit artifacts
Wednesday, October 05, 2011
Forensic Scanner
With the manuscript for WFA 3/e submitted, I have time now to focus on other projects (while I wait for the proofs to review), including the next step for or next generation of RegRipper, which is something I call the "forensic scanner"...for now, anyway, until I come up with a really cool name for it. All new projects need a cool name, right?
As I work on developing this project, I wanted to share some of the thoughts behind it, in part to see if they make sense, but also to show the direction of the project.
Why?
Why have a "forensic scanner" at all? That's a great question. Other areas of information security have scanners...when I started my infosec career after leaving the military, I used ISS's Internet Scanner. Consider Nessus, which was the inspiration behind the design for RegRipper. And there are others...the idea being that once you've discovered some artifact or "check", you can automate that check for future use, without having to memorize the specifics. After all, by creating a "plugin" for the check, you're documenting it. Another strength of something like this is that one analyst can create a check, documenting the plugin, and provide it to others, sharing that information so that those other analysts don't have to have the same experiences. This way, a team can focus on analysis and benefit from the analysis performed by others.
Look at it this way...do you want to do the same checks that you always do for malware, manually? Let's say that a customer believes that a system is infected with Zeus/ZBot...do you want to manually check for sdra64.exe every time? What about the persistence mechanism? What if the persistence mechanism is the same, but the file name has changed? What if you could automate your entire malware detection checklist? Most forensic analysts are familiar with the detection rates of AV products, and sometimes it's a matter of looking not for the malware itself, but rather the effects that the malware had on it's ecosystem...what if you could automate that?
Purpose
The purpose of the forensic scanner is not to replace anything that's already out there, but instead to augment what's currently available. For instance, the Digital Forensics Framework (DFF) was recently updated to version 1.2 and includes some great features, none of which the forensic scanner includes (i.e., search, etc.). The forensic scanner is a targeted tool with a specific purpose, and not a general analysis framework. Instead, much like other scanners (Nessus, ISS's Internet Scanner, etc.), the forensic scanner is intended to fill a gap; using frameworks and applications (ProDiscover, etc.), analysts will find artifacts and indicators of compromise, and then document them as plugins as a means of automation. Then whenever the scanner is run against an acquired image, checks for those artifacts, as well as processing and even inclusion of references, are run automatically. This is intended to quickly allow the analyst to analyze, by running checks that have already been discovered. Learn it once, document it, run it every time.
Scanner Attributes
Here are some of the forensic scanner attributes that I've come up with:
Flexibility - From the beginning, I wanted the scanner to be flexible, so I designed it to be run against a mounted volume. You're probably wondering, "how is this flexible?" Well, how can you mount a volume, particularly in read-only mode? You can convert a raw/dd image to a .vhd file (using vhdtool.exe, or the VirtualBox convertfromraw command), and mount that .vhd file read-only via the Disk Management tool. You can use FTK Imager, ImDisk, or another image mounting tool. You can also connect to a remote system via F-Response and run the scanner. You can mount an image by converting it to a .vmdk file, and mount is as an independent, non-persistent hard drive. Using either the .vhd or .vmdk methods, you can also mount VSCs as volumes and scan those; as with RegRipper, a CLI engine for the scanner can be included in a batch file to automate the scans.
Even though I'm writing the plugins that I'm using for Windows, there's nothing that really restricts me to that platform. The scanner is written in Perl, and can be run from Linux or MacOS X (the GUI would require the appropriate modules, of course...) and run against pretty much any mounted volume.
Force Multiplier - One of the things I really like about RegRipper is the ability to write my own plugins. So, let's say I find something of interest...I write a plugin for it. I can (and do) include appropriate references (i.e., to malware write-ups, MS KB articles, etc.) in the comments of the plugin, or even have those spit out in the output. I can even add an explanation to the plugin itself, in the comments, describing the reasoning behind the plugin, why it was written, and how to use or interpret the output. That plugin then persists, along with the documentation. This plugin can then be shared amongst analysts, increasing their capability while reducing their need to experience the same analysis I did to zero. So, let's say I find something that I'd never seen before and it took me 10 hrs of dedicated analysis to find it. If there are 5 other analysts on my team (and we're all of approximately equal skill levels), and I share that plugin with all of them, then I've just added to their capability and saved the team 50 hrs of dedicated work.
This section could also be referred to as Preservation of Corporate Knowledge or Competitive Advantage, depending on who you are. For example, both LE and private industry consultants benefit from retaining corporate knowledge; also, LE would greatly benefit from any plugins shared by private industry.
Knowledge Retention
Within the private sector, the information security industry can be fluid. Analysts have changes in their lives, or develop new skills (or want to do so) and move on. Having a means of documenting and retaining their experiences within the environment can be valuable; have a means of incorporating that knowledge directly into the field can be critical. It's one thing for an analyst to talk about something they found, or write a white paper...it's something else entirely to have a forensic analyst write a dozen or so plugins throughout their tenure and have those available for use, by all of the other analysts, well after he or she has left.
LE experiences something similiar; many times, an analyst receives training, works on some cases, and is then off to do other LE things. And often, their wealth of knowledge leaves with them. With a framework such as the forensic scanner, not only is an individual analyst's knowledge retained, but it can be provided other analysts, even ones that haven't been hired or completely trained.
Competitive Advantage is usually associated with private industry consulting firms, but I'm sure that you can see how this would apply. Any analyst who finds something and documents it through a plugin can then share that plugin with others...100% capability for 0 experience; the time-to-market for the capability is pretty much as long as it takes to open an email and extract the plugins from an attached archive. Ideally, you'd want to have an "armorer", like a lab tech or analyst who either gets information from other analysts and writes and tests the plugins, or receives the plugins and tests them before rolling them out. The approved plugins can be placed in an archive and emailed to analysts, or you can use some form of distribution mechanism that each analyst initiates.
Self-Documenting - The forensic scanner has an interesting feature that I'm carrying over from RegRipper - when running, it produces an activity log, collecting information about the scanned volume and tracking the plugins that were run. So, not only will your output make it clear what the results of the scan were, but the activity log can tell you exactly which versions of the plugins had been run; if there's a plugin that wasn't run, or an updated version of a plugin comes out, you can simply re-run the scan.
This information can also be used from a debugging standpoint. If something didn't work as planned, why was that? How can the process be improved?
Common Format - One of the things that we're all familiar with how there are a number of tools out there that parse information for us, but these tools all have different output formats, and it can be a very manual process to work through Prefetch files, Jump Lists, etc., and have to manually convert all of that information into a common output format. Even if we get several tools from the same site and author, and we can format the output in .csv or .xml, we still have to run the tools, and manage the output. Using the scanner, the plugins will handle the output format. I can write one plugin, and have .csv output...then modify the output in another version of the plugin to .tln output, and include each plugin in the appropriate scanner profile.
Plugins
When scanning a mounted volume, the engine exports a number of variables that you can use to tailor your plugins; however, as the target is a mounted volume, there is no proprietary API that you need to learn. Want to get a list of files in a directory? Use the standard opendir(), readdir(), and closedir() function that ship with Perl. What this means is that learning to write plugins is as easy as learning to program in Perl, and if you don't know (or want to learn) how to program in Perl, that's okay...find someone who does and buy them a beer.
The plugins can also be flexible, ranging from the broad to the narrowly-focused. An example of a broad plugin might be one that scans the Windows\Temp (or the user's Temp) folder for PE files. I know how tedious something like that can be...particularly with a reprovisioned system that has a dozen or more user accounts on it...but how would you like to have a report of all of the .tmp files in all of the user's Temp folders that are actually PE files?
A plugin that's a bit more tactical might be one that looks for a specific file, such as ntshrui.dll in the C:\Windows directory. The "strategic" variant of that plugin might be one to list all of the DLLs in the Windows directory.
However, plugins don't have to be just Perl; using Perl functions, you can also create plugins to run external commands. For example, you can use strings and find to parse through the pagefile, and retain the output. Or you can run sigcheck. Using Perl functions that allow you to launch external commands, you can automate running (and processing/parsing the output of) external commands against the mounted volume.
Deploying the Scanner
I alluded to some of the deployment scenarios for the scanner earlier in this post, but I'll reiterate some of them here because I think they're important.
When I was on the IBM response team (and the ISS team before that), each responder had two MacBooks that we had in our jump kits, as well as Mac Server in our office; lots of horsepower, with a reduced form factor and weight (over the comparable Dell Latitudes). I opted to primarily run Windows, as I wanted to be as familiar with the most predominant platform that we encountered. Our team was also geographically dispersed. So how would something like the scanner be deployed in such an environment?
Now, if we had a central intake point, such as a lab were images were received and processed (image file system verified, documented, and a working copy made to a storage facility) by a lab tech, the scanner could be deployed to and maintained by the lab tech. Once an image was processed, the working copy could be scanned, and the analyst could VPN into the lab, fire up the appropriate analysis VM, and review the output report from the scanner.
What's coming?
Recently on Twitter, Ken Johnson (@Patories) point out an artifact that he'd found on the Windows 8 dev build, and likely associated with IE 10...a key named "TypedURLsTime". The data for each of the listed values is a FILETIME object...when the time comes that Win8 is seen on the desktop, this will likely be a very useful artifact to be included in a plugin.
So, let me ask you this...who's going to remember that when Windows 8 actually hits the streets? I'm running the dev build of Windows 8 in a VirtualBox VM, as a .vhd file; anyone doing so can easily mount the .vhd (read-only) on their Windows 7 system, write a plugin for the artifact, and there it is...documented.
As I work on developing this project, I wanted to share some of the thoughts behind it, in part to see if they make sense, but also to show the direction of the project.
Why?
Why have a "forensic scanner" at all? That's a great question. Other areas of information security have scanners...when I started my infosec career after leaving the military, I used ISS's Internet Scanner. Consider Nessus, which was the inspiration behind the design for RegRipper. And there are others...the idea being that once you've discovered some artifact or "check", you can automate that check for future use, without having to memorize the specifics. After all, by creating a "plugin" for the check, you're documenting it. Another strength of something like this is that one analyst can create a check, documenting the plugin, and provide it to others, sharing that information so that those other analysts don't have to have the same experiences. This way, a team can focus on analysis and benefit from the analysis performed by others.
Look at it this way...do you want to do the same checks that you always do for malware, manually? Let's say that a customer believes that a system is infected with Zeus/ZBot...do you want to manually check for sdra64.exe every time? What about the persistence mechanism? What if the persistence mechanism is the same, but the file name has changed? What if you could automate your entire malware detection checklist? Most forensic analysts are familiar with the detection rates of AV products, and sometimes it's a matter of looking not for the malware itself, but rather the effects that the malware had on it's ecosystem...what if you could automate that?
Purpose
The purpose of the forensic scanner is not to replace anything that's already out there, but instead to augment what's currently available. For instance, the Digital Forensics Framework (DFF) was recently updated to version 1.2 and includes some great features, none of which the forensic scanner includes (i.e., search, etc.). The forensic scanner is a targeted tool with a specific purpose, and not a general analysis framework. Instead, much like other scanners (Nessus, ISS's Internet Scanner, etc.), the forensic scanner is intended to fill a gap; using frameworks and applications (ProDiscover, etc.), analysts will find artifacts and indicators of compromise, and then document them as plugins as a means of automation. Then whenever the scanner is run against an acquired image, checks for those artifacts, as well as processing and even inclusion of references, are run automatically. This is intended to quickly allow the analyst to analyze, by running checks that have already been discovered. Learn it once, document it, run it every time.
Scanner Attributes
Here are some of the forensic scanner attributes that I've come up with:
Flexibility - From the beginning, I wanted the scanner to be flexible, so I designed it to be run against a mounted volume. You're probably wondering, "how is this flexible?" Well, how can you mount a volume, particularly in read-only mode? You can convert a raw/dd image to a .vhd file (using vhdtool.exe, or the VirtualBox convertfromraw command), and mount that .vhd file read-only via the Disk Management tool. You can use FTK Imager, ImDisk, or another image mounting tool. You can also connect to a remote system via F-Response and run the scanner. You can mount an image by converting it to a .vmdk file, and mount is as an independent, non-persistent hard drive. Using either the .vhd or .vmdk methods, you can also mount VSCs as volumes and scan those; as with RegRipper, a CLI engine for the scanner can be included in a batch file to automate the scans.
Even though I'm writing the plugins that I'm using for Windows, there's nothing that really restricts me to that platform. The scanner is written in Perl, and can be run from Linux or MacOS X (the GUI would require the appropriate modules, of course...) and run against pretty much any mounted volume.
Force Multiplier - One of the things I really like about RegRipper is the ability to write my own plugins. So, let's say I find something of interest...I write a plugin for it. I can (and do) include appropriate references (i.e., to malware write-ups, MS KB articles, etc.) in the comments of the plugin, or even have those spit out in the output. I can even add an explanation to the plugin itself, in the comments, describing the reasoning behind the plugin, why it was written, and how to use or interpret the output. That plugin then persists, along with the documentation. This plugin can then be shared amongst analysts, increasing their capability while reducing their need to experience the same analysis I did to zero. So, let's say I find something that I'd never seen before and it took me 10 hrs of dedicated analysis to find it. If there are 5 other analysts on my team (and we're all of approximately equal skill levels), and I share that plugin with all of them, then I've just added to their capability and saved the team 50 hrs of dedicated work.
This section could also be referred to as Preservation of Corporate Knowledge or Competitive Advantage, depending on who you are. For example, both LE and private industry consultants benefit from retaining corporate knowledge; also, LE would greatly benefit from any plugins shared by private industry.
Knowledge Retention
Within the private sector, the information security industry can be fluid. Analysts have changes in their lives, or develop new skills (or want to do so) and move on. Having a means of documenting and retaining their experiences within the environment can be valuable; have a means of incorporating that knowledge directly into the field can be critical. It's one thing for an analyst to talk about something they found, or write a white paper...it's something else entirely to have a forensic analyst write a dozen or so plugins throughout their tenure and have those available for use, by all of the other analysts, well after he or she has left.
LE experiences something similiar; many times, an analyst receives training, works on some cases, and is then off to do other LE things. And often, their wealth of knowledge leaves with them. With a framework such as the forensic scanner, not only is an individual analyst's knowledge retained, but it can be provided other analysts, even ones that haven't been hired or completely trained.
Competitive Advantage is usually associated with private industry consulting firms, but I'm sure that you can see how this would apply. Any analyst who finds something and documents it through a plugin can then share that plugin with others...100% capability for 0 experience; the time-to-market for the capability is pretty much as long as it takes to open an email and extract the plugins from an attached archive. Ideally, you'd want to have an "armorer", like a lab tech or analyst who either gets information from other analysts and writes and tests the plugins, or receives the plugins and tests them before rolling them out. The approved plugins can be placed in an archive and emailed to analysts, or you can use some form of distribution mechanism that each analyst initiates.
Self-Documenting - The forensic scanner has an interesting feature that I'm carrying over from RegRipper - when running, it produces an activity log, collecting information about the scanned volume and tracking the plugins that were run. So, not only will your output make it clear what the results of the scan were, but the activity log can tell you exactly which versions of the plugins had been run; if there's a plugin that wasn't run, or an updated version of a plugin comes out, you can simply re-run the scan.
This information can also be used from a debugging standpoint. If something didn't work as planned, why was that? How can the process be improved?
Common Format - One of the things that we're all familiar with how there are a number of tools out there that parse information for us, but these tools all have different output formats, and it can be a very manual process to work through Prefetch files, Jump Lists, etc., and have to manually convert all of that information into a common output format. Even if we get several tools from the same site and author, and we can format the output in .csv or .xml, we still have to run the tools, and manage the output. Using the scanner, the plugins will handle the output format. I can write one plugin, and have .csv output...then modify the output in another version of the plugin to .tln output, and include each plugin in the appropriate scanner profile.
Plugins
When scanning a mounted volume, the engine exports a number of variables that you can use to tailor your plugins; however, as the target is a mounted volume, there is no proprietary API that you need to learn. Want to get a list of files in a directory? Use the standard opendir(), readdir(), and closedir() function that ship with Perl. What this means is that learning to write plugins is as easy as learning to program in Perl, and if you don't know (or want to learn) how to program in Perl, that's okay...find someone who does and buy them a beer.
The plugins can also be flexible, ranging from the broad to the narrowly-focused. An example of a broad plugin might be one that scans the Windows\Temp (or the user's Temp) folder for PE files. I know how tedious something like that can be...particularly with a reprovisioned system that has a dozen or more user accounts on it...but how would you like to have a report of all of the .tmp files in all of the user's Temp folders that are actually PE files?
A plugin that's a bit more tactical might be one that looks for a specific file, such as ntshrui.dll in the C:\Windows directory. The "strategic" variant of that plugin might be one to list all of the DLLs in the Windows directory.
However, plugins don't have to be just Perl; using Perl functions, you can also create plugins to run external commands. For example, you can use strings and find to parse through the pagefile, and retain the output. Or you can run sigcheck. Using Perl functions that allow you to launch external commands, you can automate running (and processing/parsing the output of) external commands against the mounted volume.
Deploying the Scanner
I alluded to some of the deployment scenarios for the scanner earlier in this post, but I'll reiterate some of them here because I think they're important.
When I was on the IBM response team (and the ISS team before that), each responder had two MacBooks that we had in our jump kits, as well as Mac Server in our office; lots of horsepower, with a reduced form factor and weight (over the comparable Dell Latitudes). I opted to primarily run Windows, as I wanted to be as familiar with the most predominant platform that we encountered. Our team was also geographically dispersed. So how would something like the scanner be deployed in such an environment?
Now, if we had a central intake point, such as a lab were images were received and processed (image file system verified, documented, and a working copy made to a storage facility) by a lab tech, the scanner could be deployed to and maintained by the lab tech. Once an image was processed, the working copy could be scanned, and the analyst could VPN into the lab, fire up the appropriate analysis VM, and review the output report from the scanner.
What's coming?
Recently on Twitter, Ken Johnson (@Patories) point out an artifact that he'd found on the Windows 8 dev build, and likely associated with IE 10...a key named "TypedURLsTime". The data for each of the listed values is a FILETIME object...when the time comes that Win8 is seen on the desktop, this will likely be a very useful artifact to be included in a plugin.
So, let me ask you this...who's going to remember that when Windows 8 actually hits the streets? I'm running the dev build of Windows 8 in a VirtualBox VM, as a .vhd file; anyone doing so can easily mount the .vhd (read-only) on their Windows 7 system, write a plugin for the artifact, and there it is...documented.
Friday, April 22, 2011
Extending RegRipper (aka, "Forensic Scanner")
I'll be presenting on "Extending RegRipper" at Brian Carrier's Open Source Digital Forensics Conference on 14 June, along with Cory Altheide, and I wanted to provide a bit of background with regards to what my presentation will cover...
In '98-'99, I was working for Trident Data Systems, Inc., (TDS) conducting vulnerability assessments for organizations. One of the things we did as part of this work was run ISS’s Internet Scanner (now owned by IBM) against the infrastructure; either a full, broad-brush scan or just very specific segments, depending upon the needs and wants of the organization. I became very interested in how the scanner worked, and began to note differences in how the scanner would report its findings based on the level of access we had to the systems within the infrastructure. Something else I noticed was that many of the checks that were scanned were a result of the ISS X-Force vulnerability discovery team. In short, a couple of very smart folks would discover a vulnerability, add a means of scanning for that vulnerability via the Internet Scanner framework, and roll it out to thousands of customers. Within fairly short order, this check can be rolled out to hundreds or thousands of analysts, none of whom have any prior knowledge of the vulnerability, nor have had to invest the time to investigate it. This became even more clear as I started to create an open-source (albeit proprietary) scanner to replace the use of Internet Scanner, due in large part to significant issues with inaccurate checks, and the need to adapt the output. I could create a check to be run, and give it to an analyst going on-site, and they wouldn't need to have any prior knowledge of the issue, nor would they have to invest time in discovery and analysis, but they could run the check and easily review and understand the results.
Other aspects of information security also benefit from the use of scanners. Penetration testing and web application assessments benefit from scanners that include frameworks for providing new and updated checks to be run, and many of the analysts running the scanners have no prior knowledge of the checks that are being run. Nessus (from Tenable) is a very good example of this sort of scanner; the plugins run by the scanner are text-based, providing instructions for the scanner. These plugins are easy to open and read, and provide a great deal of information regarding how the checks are constructed and run.
Given all of the benefits derived from scanners in other disciplines within information security, it just stands to reason that digital forensic analysis would also benefit from a similar framework.
The forensic scanner is not intended to replace the analyst; rather, it is intended as a framework for documenting and retaining the institutional knowledge of all analysts on the team, and remove the tedium of looking for that "low-hanging fruit" that likely exists in most, if not all, exams.
A number of commercially available forensic analysis applications (EnCase, ProDiscover) have scripting languages and scanner-like functionality; however, in most cases, this functionality is based on proprietary APIs, and in some cases, scripting languages (ProDiscover uses Perl as it's scripting language, but the API for accessing the data is unique to the application).
A scanner framework is not meant to replace the use of commercial forensic analysis applications; rather, the scanner framework would augment and enhance the use of those applications, by providing an easy and efficient means for educating new analysts, as well as "sweeping up" the "low-hanging fruit", leaving the deeper analysis for the more experienced analysts.
This scanner framework would be based on easily available tools and techniques. For example, the scanner would be designed to access acquired images mounted read-only via the operating system (Linux mount command) or via freely available applications (Windows - FTK Imager v3.0, ImDisk, vhd/vmdk, etc.); that way, the scanner can make use of currently available APIs (via Perl, Python, etc.) in order to access data within the acquired image, and do so in a "forensically sound manner" (i.e., not making any changes to the original data).
The scanner is not intended to run in isolation; rather, it is intended to be used with other tools (here, here) as part of an overall process. The purpose of the scanner is to provide a means for retention, efficient deployment, and proliferation of institutional digital forensic knowledge.
Benefits
Some benefits of a forensic scanner framework such as this include, but are not limited to, the following:
1. Knowledge Retention - None of us knows everything, and we all see new things during examinations. When an analyst sees or discovers something new, a plugin can be written or updated. Once this is done, that knowledge exists, regardless of the state of the analyst (she goes on vacation, leaves for another position, etc.). Enforcing best practice documentation of the plugin ensures that as much knowledge as possible is retained along with the application, providing an excellent educational tool, as well as a ready means for adapting or improving the plugin.
2. Establish a career progression - When new folks are brought aboard a team, they have to start somewhere. In most cases, particularly with consulting organizations, skilled/experienced analysts are hired, but as the industry develops, this won't always be the case. The forensic scanner provides an ancillary framework for developing "home grown" expertise where inexperienced analysts are hired. Starting the new analysts off in a lab environment and having them begin learning the necessary procedures by acquiring and verifying media puts them in an excellent position to run the scanner. For example, the analyst either goes on-site and conducts acquisition, or acquires media sent to the lab, and prepares the necessary documentation. Then, they mount the acquired image and run the scanner, providing the more experienced analyst with the path to the acquired image and the report.
This framework also provides an objective means for personnel assessment; managers can easily track the plugins that are improved or developed by various analysts.
3. Teamwork - In many environments, development of plugins likely will not occur in a vacuum or in isolation. Plugins need to be reviewed, and can be improved based on the experience of other analysts. For example, let's say an analyst runs across a Zeus infection and decides to write a plugin for the artifacts. When the plugin is reviewed, another analyst mentions that Zeus will load differently based on the permissions of the user upon infection. The plugin can them be documented and modified to include additional conditions.
New plugins can be introduced and discussed during team meetings or through virtual conferences and collaboration, but regardless of the method, it introduces a very important aspect of forensic analysis...peer review.
4. Ease of modification - One size does not fit all. There are times when analysts will not be working with full images, but instead will only have access to selected files from systems. A properly constructed framework will provide the means necessary for accessing and scanning these limited data sets, as well. Also, reporting of the scanner can be modified according to the needs of the analyst, or organization.
5. Flexibility - A scanner framework is not limited to just acquired images. For example, F-Response provides a means of access to live, remote systems in a manner that is similar to an acquired image (i.e., much of the same API can be used, as with RegRipper), so the framework used to access images can also be used against systems accessed via F-Response. As the images themselves would be mounted read-only in order to be scanned, Volume Shadow Copies could also be mounted and scanned using the same scanner and same plugins.
Another means of flexibility comes about through the use of "idle" resources. What I mean by that is that many times, analysts working on-site or actively engaged in analysis may be extremely busy, so running the scanner and providing the output to another, off-site analyst who is not actively engaged frees up the on-site team and provides answers/solutions in a timely and efficient manner. Or, data can be provided and the off-site analyst can write a plugin based on that data, and that plugin can be run against all other systems/images. In these instances, entire images do not have to be sent to the off-site analyst, as this takes considerable time and can expose sensitive data. Instead, only very specific data is sent, making for a much smaller data set (KB as opposed to GB).
In '98-'99, I was working for Trident Data Systems, Inc., (TDS) conducting vulnerability assessments for organizations. One of the things we did as part of this work was run ISS’s Internet Scanner (now owned by IBM) against the infrastructure; either a full, broad-brush scan or just very specific segments, depending upon the needs and wants of the organization. I became very interested in how the scanner worked, and began to note differences in how the scanner would report its findings based on the level of access we had to the systems within the infrastructure. Something else I noticed was that many of the checks that were scanned were a result of the ISS X-Force vulnerability discovery team. In short, a couple of very smart folks would discover a vulnerability, add a means of scanning for that vulnerability via the Internet Scanner framework, and roll it out to thousands of customers. Within fairly short order, this check can be rolled out to hundreds or thousands of analysts, none of whom have any prior knowledge of the vulnerability, nor have had to invest the time to investigate it. This became even more clear as I started to create an open-source (albeit proprietary) scanner to replace the use of Internet Scanner, due in large part to significant issues with inaccurate checks, and the need to adapt the output. I could create a check to be run, and give it to an analyst going on-site, and they wouldn't need to have any prior knowledge of the issue, nor would they have to invest time in discovery and analysis, but they could run the check and easily review and understand the results.
Other aspects of information security also benefit from the use of scanners. Penetration testing and web application assessments benefit from scanners that include frameworks for providing new and updated checks to be run, and many of the analysts running the scanners have no prior knowledge of the checks that are being run. Nessus (from Tenable) is a very good example of this sort of scanner; the plugins run by the scanner are text-based, providing instructions for the scanner. These plugins are easy to open and read, and provide a great deal of information regarding how the checks are constructed and run.
The forensic scanner is not intended to replace the analyst; rather, it is intended as a framework for documenting and retaining the institutional knowledge of all analysts on the team, and remove the tedium of looking for that "low-hanging fruit" that likely exists in most, if not all, exams.
A number of commercially available forensic analysis applications (EnCase, ProDiscover) have scripting languages and scanner-like functionality; however, in most cases, this functionality is based on proprietary APIs, and in some cases, scripting languages (ProDiscover uses Perl as it's scripting language, but the API for accessing the data is unique to the application).
A scanner framework is not meant to replace the use of commercial forensic analysis applications; rather, the scanner framework would augment and enhance the use of those applications, by providing an easy and efficient means for educating new analysts, as well as "sweeping up" the "low-hanging fruit", leaving the deeper analysis for the more experienced analysts.
This scanner framework would be based on easily available tools and techniques. For example, the scanner would be designed to access acquired images mounted read-only via the operating system (Linux mount command) or via freely available applications (Windows - FTK Imager v3.0, ImDisk, vhd/vmdk, etc.); that way, the scanner can make use of currently available APIs (via Perl, Python, etc.) in order to access data within the acquired image, and do so in a "forensically sound manner" (i.e., not making any changes to the original data).
The scanner is not intended to run in isolation; rather, it is intended to be used with other tools (here, here) as part of an overall process. The purpose of the scanner is to provide a means for retention, efficient deployment, and proliferation of institutional digital forensic knowledge.
Benefits
Some benefits of a forensic scanner framework such as this include, but are not limited to, the following:
1. Knowledge Retention - None of us knows everything, and we all see new things during examinations. When an analyst sees or discovers something new, a plugin can be written or updated. Once this is done, that knowledge exists, regardless of the state of the analyst (she goes on vacation, leaves for another position, etc.). Enforcing best practice documentation of the plugin ensures that as much knowledge as possible is retained along with the application, providing an excellent educational tool, as well as a ready means for adapting or improving the plugin.
2. Establish a career progression - When new folks are brought aboard a team, they have to start somewhere. In most cases, particularly with consulting organizations, skilled/experienced analysts are hired, but as the industry develops, this won't always be the case. The forensic scanner provides an ancillary framework for developing "home grown" expertise where inexperienced analysts are hired. Starting the new analysts off in a lab environment and having them begin learning the necessary procedures by acquiring and verifying media puts them in an excellent position to run the scanner. For example, the analyst either goes on-site and conducts acquisition, or acquires media sent to the lab, and prepares the necessary documentation. Then, they mount the acquired image and run the scanner, providing the more experienced analyst with the path to the acquired image and the report.
This framework also provides an objective means for personnel assessment; managers can easily track the plugins that are improved or developed by various analysts.
3. Teamwork - In many environments, development of plugins likely will not occur in a vacuum or in isolation. Plugins need to be reviewed, and can be improved based on the experience of other analysts. For example, let's say an analyst runs across a Zeus infection and decides to write a plugin for the artifacts. When the plugin is reviewed, another analyst mentions that Zeus will load differently based on the permissions of the user upon infection. The plugin can them be documented and modified to include additional conditions.
New plugins can be introduced and discussed during team meetings or through virtual conferences and collaboration, but regardless of the method, it introduces a very important aspect of forensic analysis...peer review.
4. Ease of modification - One size does not fit all. There are times when analysts will not be working with full images, but instead will only have access to selected files from systems. A properly constructed framework will provide the means necessary for accessing and scanning these limited data sets, as well. Also, reporting of the scanner can be modified according to the needs of the analyst, or organization.
5. Flexibility - A scanner framework is not limited to just acquired images. For example, F-Response provides a means of access to live, remote systems in a manner that is similar to an acquired image (i.e., much of the same API can be used, as with RegRipper), so the framework used to access images can also be used against systems accessed via F-Response. As the images themselves would be mounted read-only in order to be scanned, Volume Shadow Copies could also be mounted and scanned using the same scanner and same plugins.Another means of flexibility comes about through the use of "idle" resources. What I mean by that is that many times, analysts working on-site or actively engaged in analysis may be extremely busy, so running the scanner and providing the output to another, off-site analyst who is not actively engaged frees up the on-site team and provides answers/solutions in a timely and efficient manner. Or, data can be provided and the off-site analyst can write a plugin based on that data, and that plugin can be run against all other systems/images. In these instances, entire images do not have to be sent to the off-site analyst, as this takes considerable time and can expose sensitive data. Instead, only very specific data is sent, making for a much smaller data set (KB as opposed to GB).
Subscribe to:
Posts (Atom)