Threat hunting includes the policies, methods, and techniques used to scour networks for hidden threats that have evaded traditional defenses and detection systems. Generally, threat hunting is a proactive approach to cyber defense. Threat hunting might be sparked by emerging threats, newly identified vulnerabilities, or a hunter’s hypothesis.
Cyber threat intelligence (CTI) can help shape a hunt by detailing emerging threats or new tactics, techniques, and procedures (TTP) used by known threat actors. Hunters create hypotheses and hunt plans based on how they think actors might exploit vulnerabilities via the methods shown in MITRE’s ATT&CK Framework.
In addition, threat hunters often help incident response efforts by determining the scope of an intrusion, finding where threat actors are hiding, and containing threats. This is a more reactive use of threat hunting, but the same approaches apply.
No matter how a hunt starts, threat hunters focus on the actions and behaviors of the threat actor and not the hard indicators of compromise like file hashes, internet protocol (IP) addresses, and domain names. The “Pyramid of Pain” shows the level of effort required by an attacker to change or obfuscate attack artifacts. Focusing on TTPs increases the likelihood of finding advanced actors that are skilled in evading cyber defenses.
Today’s advanced persistent threat (APT) actors often execute long-term campaigns to compromise target networks. They seek to gain and maintain a hidden presence. APT actors constantly change their malicious TTPs to skillfully defeat reactive, rule-based cybersecurity defenses. For instance, they use modern polymorphic and obfuscated malware, dynamic infrastructure, fileless malware, operating system hijacking, and supply chain attack techniques to evade defenses or operate below the alert threshold of traditional EDR and security information and event management (SIEM) platforms.
Threat hunting is considered so important that in 2021 the White House released Executive Order 14028 directing all Federal Civilian Executive Branch (FCEB) agencies to develop and deploy the necessary capabilities to support threat hunting activities. Also, in the 2021 National Defense Authorization Act, Congress provided the Cybersecurity and Infrastructure Security Agency (CISA) with authority to conduct threat hunting in all FCEB agencies “with or without advance notice to or authorization from agencies.”
To combat the rise in these kinds of adversaries, Booz Allen has developed an advanced threat hunting (ATH) methodology—a proactive, cyclical process for identifying anomalous or malicious behavior in environments that have yet to deploy traditional defenses like EDR and SIEM. Advanced threat hunting also lets organizations enhance their security capabilities within all their environments. This includes environments that have yet to deploy an EDR, as well as environments that don’t easily support EDR deployment—for instance, operational technology (OT)/supervisory control and data acquisition (SCADA) systems, medical device environments, or isolated cyber/physical systems.
In practice, threat hunting applies analytics against one or more datasets to prove or disprove a hypothesis. The quality of a hunt largely depends on the quality of both the analytics and the data available. Incomplete or incorrect data reduces confidence in the findings of threat hunting efforts. Organizations need a strong understanding of their cybersecurity data to qualify and quantify threat hunting activities.
Data coverage is often expressed as the percentage of existing endpoints from which an organization is collecting security logs. This can only be calculated if the organization is fully aware of all the endpoints in the environment. Such awareness generally requires high-quality hardware asset management (HWAM) and software asset management (SWAM).
Incomplete data coverage would not allow a hunter to declare that a particular threat doesn’t exist on the network but only that it wasn’t detected on the devices where data was collected. In some cases, this could only include 70% – 80% of assets.
We sometimes assess coverage by the source types and fields being collected against the MITRE ATT&CK Framework. By evaluating logs against the various frameworks, we can create a visual representation of our coverage via the ATT&CK Navigator.
In addition to maximizing data coverage, organizations need to ensure the quality of the coverage is high. In other words, when organizations think about data quality, they need to ask whether they are collecting the right data. Often, cybersecurity tools are deployed with minimal changes to the default vendor configuration. And that means even if some tuning has been done for the specific environment there’s a good chance the capabilities of the tools are not being fully used. Taking the time to further adjust the settings on these tools consistent with leading cybersecurity practices will likely lead to better outcomes for threat hunting.
Data quantity is often a concern of organizations for several reasons. There is such a thing as too much or too little data when it comes to security logging and storage. While many security professionals may say, “I want as much data as I can get,” that is often not the right approach and can create issues with data retention, ingest, and storage costs, and effective investigations. Collecting too little data, on the other hand, has the obvious drawback of limiting what anomalous or malicious behaviors can be detected.
What’s more, for FCEB agencies, limiting retention is not an option if they are to stay compliant with OMB (Office of Management and Budget) Memo M-21-31, which specifies security logging and retention requirements. Also, implementing zero trust as required across the federal government involves the creation of many new logs, given the strong emphasis on continuous monitoring.
Process logging generates a large amount of data—and for threat hunters, it is one of the most effective log sources for detection and investigation. Whatever insight an organization has about activity on its network is often only as good as the data it is collecting and storing.
MITRE ATT&CK supplies a taxonomy for the many kinds of actions threat actors might take while carrying out a cyberattack. The framework also includes various matrices that apply to enterprise networks, cloud and containerized environments, mobile devices, and industrial control systems (ICS). These matrices specify the various tactics and techniques that threat actors are known to have used. For each action specified, MITRE gives a description, examples of how it has been used, and mitigation recommendations.
MITRE also shows which data sources and components are needed to detect potentially malicious behaviors. Here you can find recommended data sources, techniques that each source can help you detect, and examples. In addition, CISA has released a free tool called Decider to help cybersecurity professionals map threat actor behavior to the MITRE ATT&CK framework.
Collecting and storing all available data sources is typically impractical. The sheer volume could easily overwhelm ingest, storage, and budget availability. Now, for those organizations with the freedom to make decisions about data collection, how do they choose what to collect?
One approach involves pinpointing and prioritizing protection for high-value assets (HVA). These are the “crown jewels,” whose loss, degradation, or compromise would cause the most significant harm to critical business functions or, in some cases, reputation. HVAs are likely to be unique for each organization. For instance, HVAs might be servers that store trade secrets or sensitive client data.
Another approach is to prioritize the data sources that would have the broadest impact on an organization’s ability to detect threats. MITRE’s list of data sources is already mapped to attack techniques. And great work by Roberto Rodriguez, a.k.a. Cyb3rWard0g, produced analysis showing that command execution and process data sources detect the most ATT&CK techniques. These sources have MITRE designation DS0017 and DS009, respectively.
EDRs are only one of several ways to collect data on malicious activity. In fact, there are a lot of misconceptions about EDR and SIEM solutions for threat hunting.
The truth is that organizations need little more than the data source that supplies identified data and an affordable way to aggregate, store, and analyze that data.
There are multiple alternatives to EDRs that are free and open source or reasonably affordable. Below are a few of these options.
Microsoft has included the Windows Event Log in the operating system since Windows Vista and Windows Server 2008. According to Microsoft, event logging “provides a standard, centralized way for applications (and the operating system) to record important software and hardware events. The event logging service records events from various sources and stores them in a single collection called an event log.”
Another Microsoft solution for building customized log creation is Event Tracing for Windows (ETW), “an efficient kernel-level tracing facility that lets you log kernel or application-defined events to a log file.” While ETW is less applicable for most organizations, it can allow those with custom applications deployed to gain greater visibility into their behavior and potentially identify attempts to exploit or otherwise manipulate an application.
Another free option to enhance security event visibility is Windows System Monitor (Sysmon), “a Windows system service and device driver that, once installed on a system, remains resident across system reboots to monitor and log system activity to the Windows event log.” Sysmon greatly expands the types of events that are logged, making network defenders and threat hunters more effective than if they were to just use Windows Event Log. What’s more, in late 2021, Microsoft released the Linux version of Sysmon to extend an organization’s logging capabilities to at least 11 different Linux distributions.
More organizations are migrating to the cloud or expanding their networks into the cloud’s available resources. Unfortunately, not all organizations similarly expand their detection and defensive capabilities. In most cases, the necessary logs are already being generated.
Threat hunting is highly dependent on available data—but it's also largely independent of the mechanism used to generate and collect the data. By applying a range of different techniques, organizations can achieve effective, proactive threat hunting with the tools available in a particular environment.
Threat hunting is crucial for well-rounded cybersecurity. There are multiple options for generating and collecting the most effective data available to support threat hunting. Some of these options are free or significantly less expensive than traditional EDRs and come with entire communities that provide open-source guidance and resources for effectively using the solution of choice. Remember, the absence of a mature “traditional” EDR solution is not a barrier to entry into the development of a threat hunting capability.
This article is for informational purposes only; its content may be based on employees’ independent research and does not represent the position or opinion of Booz Allen. Furthermore, Booz Allen disclaims all warranties in the article's content, does not recommend/endorse any third-party products referenced therein, and any reliance and use of the article is at the reader’s sole discretion and risk. Booz Allen maintains corporate partnerships with AWS, Google Cloud, and Microsoft.
Are you ready to take a disruptive approach to safeguard your business? DarkLabs offers a portfolio of solutions built by industry experts with decades of experience countering nation-state-level cyber threats, offering actionable insights, force-multiplying research and training, and leveraging rock-solid defenses to secure your organization against even the most advanced adversaries.
Unrivaled Expertise: Our team comprises top experts with unparalleled experience in combating cyber threats at the highest level. We understand the evolving cyber battlespace and deliver the deepest understanding of your adversaries.
Customized Integration: DarkLabs solutions integrate effortlessly into any security mission, empowering your organization with the tools to defend against threats effectively.
Multidomain Research: Access our tradecraft, tooling, and operational capabilities to accelerate your cybersecurity R&D efforts. Collaborate with our teams and harness the power of AI hyper-computing infrastructure.
