Understanding Threat Hunting

Organisations use Security Information and Event Management (SIEM) software which comprises security monitoring and log management tools to detect suspicious activities on their network. Intrusion Detection System (IDS) and Intrusion Prevention System (IPS) are two essential parts of security monitoring, and they both entail the use of signatures to identify patterns that may indicate a compromise. Signatures are regular expressions (RegEx) matching for known attacks and are platform-specific. Therefore, if an IDS detects activities that match a signature, it will trigger an alert. An IPS does the same thing but goes a step further to prevent suspicious activity from taking place.

The above process is an excellent strategy to prevent attackers that aim to infect hosts on the network with known malware indiscriminately. In recent times, however, attackers' motives have shifted from such gains, and their techniques no longer depend on existing attacks.

Most modern adversaries engage in targeted attacks and are often motivated by money. A targeted attack is an attack specifically tailored to a company's IT infrastructure. It is often conducted as a campaign, a series of organised and strategic actions aimed at exploiting a company's security vulnerabilities and circumventing its security measures to achieve a compromise. Before the attack, threat actors would research the target environment. Then, they would use the knowledge gained to deliver customised malware through an infrastructure to exploit the environment's unique vulnerabilities.

Since targeted attacks use tailored malware and infrastructure, there exist no signature for them. Therefore, they go undetected by both the IDS as well as their IPS counterparts.

According to CrowdStrike Cyber Front Lines Report 2020, the average dwell-time; that is, the time adversaries spend in an enterprise network before being detected is 2-months. This ample time allows the threat actors to settle comfortably in the environment and carry out a holistic and sophisticated attack.

During one of the most sophisticated attacks in recent times, the SolarWinds attack, attackers spent approximately ten months in the organisation's systems before being detected. According to CHANNELe2e, the SolarWinds hackers deployed SUNBURST in February 2020 by changing the source code of SolarWinds Orion IT Monitoring and Management Software and in December 2020, SUNBURST was discovered by a third party, FireEye. This attack shows that relying solely on IDS and/or IPS and log reviews is insufficient to protect systems against modern attacks. Another existing method used to catch adversaries in a system is Threat Intelligence Feeds. However, like signatures, they too are only available after an attack has taken place. Therefore, there's a need for a proactive approach to threat discovery; one that does not rely solely on pattern matching of signatures, log reviews and reports on malicious events that had already taken place.

What is Threat Hunting?

Threat Hunting is the proactive and repetitive assessment of all systems in an enterprise network for abnormal behaviour or symptoms of compromise. It is done by searching through networks to identify and contain targeted threats that circumvent existing security measures.

Where to Carry Out a Threat Hunt

Compromise manifests differently across operating systems. However, the network behaviour of any given malicious activity is the same irrespective of the platform. Modern adversaries use command and control (CnC) channels over a network to control malware in a compromised host. Therefore, it is ideal to start a hunt by investigating the network to find CnC activities.

How Command and Control Channel Works

Enterprise firewalls are often configured to block traffic from the internet, making it difficult for threat actors to communicate directly with systems on the network. Therefore, when adversaries compromise a host on a system, they include software that instructs the host to connect to a command and control server at a regular interval or within a time range, requesting instructions. The CnC server acts as a proxy or an intermediary communication channel between the attacker and the compromised host. The CnC server will send any instructions queued up by the attacker to the compromised host. If there are no instructions in its queue, the channel closes and connects again after an interval of time. This process whereby an infected host contacts a CnC server to request instructions is known as a beacon or calling home or heartbeat.

Attackers can also leverage CnC over DNS. This happens when an adversary compromises a host on a network and includes software that instructs the compromised host to do unique queries within a specific domain. This process ensures that the malicious host sends queries to the external DNS server, a CnC. These queries are a way of asking the CnC DNS for instructions.

To find CnC over DNS, investigate queries resolved to a particular domain but with nothing connecting to that domain. Also, look for unique applications that might be running on the network by investigating unique user agent strings.

How to Carry Out Threat Hunting by Targeting CnC

Step 1: Look for Connection Persistency

Connection persistency is any repetitive communication between two IP addresses happening at a regular interval or within a time range. We look for persistent connections because it's synonymous with a beacon activity.

Beacon is general, the process whereby a host contacts a server on a network, requesting instructions. Beacon, as it relates to DNS, is the repetitive connection establishment between an internal IP and a Fully Qualified Domain Name FQDN.

You can search for connection persistency on an enterprise network by:

• Monitoring the internal interface of the firewall and observing the traffic to and from the internet.

• Monitoring packets captured and analysing them over a large block of time. This is because beaconing activities of modern attacks are seldom at regular intervals. Threat actors are known to introduce jitter, the process of randomising heartbeat interval by a certain percentage. Therefore monitoring packet captures over a large block of time (minimum of 12-24hours) will give you more data to work with and, in turn, more accuracy.

While every beacon activity is worth investigating, not all beacon is malicious. For instance, Network Time Protocol (NTP) is a beacon activity between an NTP client and an NTP server. The time-request exchange occurs at a specific interval of time depending on the operating system involved.

Step 2: Is There a Business Need for the Persistent Connection?

To further the hunt, find out if there's a business need for the persistent connection. To do this, you investigate the applications involved in the communication.

Step 3: Investigate Further for Malicious Activities

Once a persistent connection is observed, and there exist no business needs for it, investigate the connection further by inspection for other suspicious activities such as abnormal protocol behaviour, unexpected protocol usage on known ports, high volume data movement to a specific unknown or threat Intel IP, unique client signature and self-signed digital certificate. Reputation checking should also be carried out on the IPs associated with this connection.

Step 4: Use Threat Score System

Threat Score System is the process of assigning a numeric value to suspicious network activities in order of severity. Each of the post persistent connection investigations should be scored based on threat severity. This would make for easy disposition or grouping.

Step 5: Disposition

Based on the overall score on each suspicious activity, categorise the network communication as either whitelist or compromised. Whitelist for activities with low scores and Compromised for those with high scores.

Step 6: Incident Response Mode or Data Extraction

For all network activities that are compromised, move to Incidence Response mode. For all activities that are whitelisted, extract the data from the set being investigated.

Step 7: Analyse Your End Points

All end-user devices such as mobile devices, laptops, and desktops should be examined for symptoms of compromise or involvement in any malicious activity.

Shortcomings of hunting by targeting CnC

• Attackers motivated by gains: As earlier stated, most modern attackers are motivated by money. However, should attackers be motivated by other gains, such as access to information, control of resources, or destruction of the target, they might not necessarily use a CnC. Therefore, targeting a CnC may prove ineffective.

Side Note

There are many beacon detection software and AI automation tools for threat hunting out there. However, this article advocates for human involvement in threat hunting because most software tools for beacon detection make use of K-means clustering, which effectively finds a repetitive pattern in a large dataset but cannot detect jitters. Also, AI has been known to go wrong by producing otherwise unintended results or missing details. An excellent example of such a result is Microsoft Chatbox, an AI software that simulates teen conversation. The software was meant to get smarter the more you chat with it, but it became a racial bot in less than 24 hours.

Conclusion

Cyber-attacks have become targeted and highly sophisticated. As a result, there's a need for a proactive method for detecting and containing threats as they arrive on the system; and this can be achieved through threat hunting.

[This article is a summary of the knowledge I gained from Active Countermeasures hands-on training on Threat Hunting, delivered by Chris Brenton on August 28th, and also, my independent research. In my next blog post, I'll do a walkthrough of the threat hunting labs.]

References

Brenton C. Network Threat Hunting, (2021), Active Countermeasure, activecountermeasures.com/wp-content/upload..

CrowdStrike, Cyber Front Lines Report: Incident Response and Proactive Services From 2020 and Insights That Matter for 2021, CrowdStrike (2020), crowdstrike.com/services/cyber-front-lines

Panettieri J., Solarwinds Orion Security Breach: Cyberattack Timeline and Hacking Incident Details (2021), CHANNELe2e, channele2e.com/technology/security/solarwin..

Vincent J., Twitter taught Microsoft's AI Chatbot to be a Racist Asshole in Less Than a Day, (2016) The Verge, theverge.com/2016/3/24/11297050/tay-microso..