Threat Hunting on a Budget
Joshua Murchie
Dalton Ireland
Joseph Pisano
Slides available live at bsidesdfw.murchie.me
Disclaimer
The views and opinions expressed in this presentation are our own. They do not represent the views of the United States Air Force, Department of Defense or any other government entity. No official endorsement of the products/software shown in this presentation is implied.
Who We Are
- Prior aircraft avionics maintainers turned infosec professionals
- Work at the Air Force Computer Emergency Response Team (AFCERT)
- We homelab, drink third wave coffee and rank flavors of Whiteclaw
- Not necessarily in that order
Who We Are
- Joshua Murchie
- Forensics and Malware Analyst
- @josh_murchie, in/jmurchie, blog.murchie.me
- Dalton Ireland
- Lead Threat Hunter
- Joseph Pisano
- Senior Threat Hunter
- @ihighjynx, in/jpisano
Overview
- What Is Threat Hunting?
- How to Hunt
- Why on a Budget?
- Data Sets for Threat Hunting
- Sysmon/Winlogbeat and ELK
- OSQuery and Kolide
- Hunt Results for Improving Detection Capabilities
What Is Threat Hunting?
“The process of proactively and iteratively searching through networks to detect and isolate advanced threats that evade existing security solutions”1
“The proactive, analyst-driven process to search for attacker tactics, techniques and procedures (TTP) within an environment.”2
“Consists of searching iteratively through networks to detect indicators of compromise (IOCs) and threats such as Advanced Persistent Threats (APTs) evading your existing security system.”3
What Is Threat Hunting?
- Three separate definitions, striking similarities
- Process driven
- Focuses on attacker TTPs
- Tailored to the things you cannot already detect
- Why hunt for what you are already alerting on?
- Threat hunting != fancy incident response
- Hunts may result in initiating incident response actions
- Scoping an incident is not a “threat hunt”
Why Threat Hunting?
- Targets the upper half of the Pyramid of Pain5
- Imposes cost on the adversary
- Top 10 things hackers hate, #6 will shock you!
- Imposes cost on the adversary
- Reduces overall organizational risk
- Improves security maturity
Pyramid of Pain
Why on a Budget?
- Management may be skeptical of a fledgling threat hunt program
- Security costs $$$
- Why spend money you don’t need to?
- Open source/free != less quality
How to Hunt
4
How to Hunt
- Create Hypothesis
- Should ideally be informed by threat intelligence or other realistic threat to organization
- “An attacker has spear phished a domain admin and gained elevated permissions to the network”
- Investigate
- Work to prove or disprove the hypothesis
- PCAP analysis, reviewing host logs anywhere the admin has logged in, etc
- Attempting to build a profile of how the adversary operates
- Uncover New Patterns/TTPs
- Discovered the admin was phished and the adversary utilized a unique method of persistence
- Build new hypothesis off of your newfound TTP
- Discovered the admin was phished and the adversary utilized a unique method of persistence
- Enrich Analytics
- Can the organization signaturize a detection of the phishing email or persistence?
- Start the cycle again with newly created hypothesis
Data Sets for Threat Hunting
- Host logs
- Native Windows logging and Sysmon
- Anti-virus/EDR logs/telemetry
- Network logs
- PCAP indexing
- Email gateway/firewall/proxy logs
Data Sets for Threat Hunting
- Live host queries/data
- “What hosts have C:\Windows\Temp\evil.exe?”
- Data will make or break a threat hunt
- How do you hunt what you can’t see?
- Don’t underestimate aggregation/correlation
Implementation
- ELK
- Elastic, Logstash, Kibana
- Provides log aggregation to survey trends and/or anomalies on the network
- Historical record of what is “normal” on your network
- Beats (Winlogbeat)
- Ships Windows Events to ELK
- Sysmon
- Greater fidelity of Windows logging
- Elastic, Logstash, Kibana
Implementation
- OSQuery
- Agent running on host which can perform queries against the host
- Utilizes SQL-like syntax to gather information
- Kolide Fleet
- Provides a GUI for crafting queries
- Can be used to scope a hunt or find artifacts on a host
Scenario 1
APT1337, FEISTY FELINE, has been observed targeting businesses in your industry. Reporting indicates they may be using malware which uses a high amount of CPU processing power.
Create Hypothesis
APT1337 is using their new malware, which exhibits signs of high processor utilization, on our network and has gone undetected by our security stack.
Investigate
Perform a Kolide query for the top 10 processes running on each host.
Investigate
Review results looking for outliers.
Investigate
Utilize our host logs in an attempt to obtain a better understanding of what regsvr32.exe is doing.
Investigate
What is this odd process running out of Temp calling regsvr32? It may serve as an interesting pivot point.
Investigate
Sysmon provides an MD5 hash of the executable for a quick check of what it may be.
Investigate
Well this isn’t good.
Uncover New Patterns/TTPs
- Adversary is naming their malware after keywords related to the company
- Reverse engineer the malware to identify potential unique characterstics
- New hypothesis:
- APT1337 has compromised the network utilizing malware named after the company/related keywords and using an unusual coding technique
Enrich Analytics
- Downloading executables named after the company or with relevant keywords from external sources is likely non-typical
- Implement signatures to detect or block these types of downloads at the boundary
- Host-based signatures to detect the presence of the APT-specific coding technique
Scenario 2
APT101, AGELESS AARDVARK, previously gained access to your organization. Your infallible threat feed, Twitter.com, says they may be reusing old TTPs and dumping credentials and using the ADMIN$ share to move laterally and execute files.
Create Hypothesis
- APT101 has gained access to our network and is using a pass-the-ticket attack via dumped credentials to spread malicious code and utilizing ADMIN$ shares to move laterally.
Investigate
Since our threat intelligence specifies executing files from the ADMIN$ share we can query ELK for anything to do with that share.
Investigate
Reviewing the logs we see notepad.exe opening a text file stored in ADMIN$ of the Domain Controller.
Investigate
Pivoting off this information we can search for events surrounding the execution of notepad.exe. We see that jpisano.adm, who is an admin as confirmed by the logon event code 4672, is the one who performed the action.
Investigate
There is no reason at face value for jpisano.adm to do this. Maybe it wasn’t him? We know APT101 likes pass-the-ticket attacks which requires dumping credentials. Mimikatz is a popular tool for credential harvesting so we can use this as a pivot.
Investigate
A simple query in ELK for Mimikatz shows our hunch was correct and have evidence of Mimikatz execution.
Investigate
We can confirm our ELK logs with a Kolide query that the mimikatz binary is indeed sitting on the host.
Uncover New Patterns/TTPs
- Why was Mimikatz able to run on the host, shouldn’t antivirus have blocked it?
- Was antivirus disabled? If so, that could make for a new TTP/hypothesis
- New hypothesis:
- APT101 utilizes a capability to disable antivirus before dropping their tools and progressing an attack
Enrich Analytics
- Did the threat group utilize a new tool to disable the antivirus?
- If we can find that tool, which we did not previously alert on, we can craft a new antivirus signature
- If we cannot find the tool does our antivirus generate logs when it is disabled?
- We can alert on antivirus disabled alerts to enable more rapid detection
Summary
- Threat hunting does not have to be reserved for an organization with a hefty budget
- Processes and methodology carry over regardless of tools available
- Log aggregation/correlation enables pivoting off of key indicators more efficiently
- Hunting feeds the betterment of our other analytics/alerting
Special Thanks
- Ashley Pearson, Threat Hunter @ AFCERT
- @onfvp, in/ashley-onfvp
- Provided research and feedback on this presentation
- Sam Kimmons, Red Team Operator @ AFCERT
- @Valcan_K, in/kimmons
- Provided recommendations for offensive actions to hunt for
- Recon InfoSec’s OpenSOC Network Defense Range
- @recon_infosec
- Provided valuable “real world” use cases for tools mentioned here