Target Audience

The target audience for this blog post is individuals who have a basic understanding of cybersecurity concepts and terminology and looking to expand their knowledge on adversary emulation. This post delves into the details of adversary emulation with the Caldera framework exploring the benefits it offers. By catering to a beginner to intermediate audience, the blog post aims to strike a balance between providing fundamental information for newcomers and offering valuable insights and techniques that can benefit individuals who are already familiar with the basics of cybersecurity.

What is Adversary Emulation

Adversary emulation is a methodology used to simulate the Tactics, Techniques, and Procedures (TTPs) used by known Advanced Persistent Threats (APTs), with the goal of identifying vulnerabilities in an organization’s security defenses. By emulating real-world attacks and incident response techniques, such as exploitation of vulnerabilities and lateral movement within a network, cybersecurity teams can gain a better understanding of their security posture and identify areas for improvement.

The Need for Adversary Emulation

Adversary emulation can help organizations test their security defenses against real-world threats. Some of the benefits the emulation offers are:

• Identifying vulnerabilities: Adversary emulation assists organizations in identifying vulnerabilities, weaknesses or misconfigurations in their security defenses that might not have been detected through conventional security testing. This information can enhance the existing detection mechanisms by creating new alerts and rules that are triggered when similar activities are detected. The emulation results can also work as a guide in prioritizing mitigation and patching activities.
• Improving security controls: By identifying weaknesses in their security defenses, organizations can make informed decisions about how to improve their security controls. This can include implementing new security technologies, updating security policies, or providing additional security awareness training to employees.
• Measuring security effectiveness: Adversary emulation enables organizations to assess the effectiveness of their security defenses within a controlled environment. Through analyzing the emulation results, organizations can have a clearer understanding of how well their incidence response plan operates in real-world scenarios.If any gaps or inefficiencies are identified, the plan can be refined based on the new data.
• Staying ahead of emerging threats: Adversary emulation exercises can help organizations stay ahead of emerging threats by testing their security defenses against new and evolving attack techniques. This can help organizations prepare for future threats and ensure that their security defenses are effective in protecting against them.

Emulation VS Simulation

Emulation involves creating a replica of a specific system or environment, such as an operating system, network, or application. It provides a more realistic testing environment, which can help identify vulnerabilities and test the effectiveness of security controls in a more accurate and reliable way. However, creating an emulation environment can be time-consuming and resource-intensive, and it may not always be feasible to replicate every aspect of a real-world environment.

Simulation, on the other hand, involves creating a hypothetical scenario that models a real-world attack. It is often quicker and easier to set up, and can be used to test response plans and procedures without the need for a complex emulation environment. However, simulations may not always provide a completely accurate representation of a real-world attack scenario, and the results may be less reliable than those obtained through emulation.

The Caldera Framework

MITRE’s Caldera project is an open-source platform that allows organizations to automatically emulate the tactics, techniques, and procedures (TTPs) used by real-world APTs. The platform is designed to be modular, which means that it can be customized to fit the specific needs of an organization. More information can be found in the official documentation and on GitHub. Red team operators can benefit from this by manually executing TTPs and blue team operators can run automated incident response actions. Caldera is also highly extensible, meaning that it can be integrated with other security tools to provide a comprehensive view of an organization’s security defenses. Moreover, it is built on the MITRE ATT&CK framework which is where the platform draws all the Tactics, Techniques and Procedures (TTPs) from.

Most common use cases of this framework include but not limited to:

• Autonomous Red Team Engagements: This case is used to emulate the TTPs of known adversary profiles to discover gaps across an infrastructure, test the defenses currently in place and train operators on detection different threats.
• Manual Red Team Engagements: This case allows red team operators to replace or extend the attack capabilities of a scenario, giving them more freedom and control over the current emulation.
• Autonomous Incident Response: This case is used by blue team operators to perform automated incident response actions to aid them in identifying TTPs and threats that other security solutions may not detect and/or prevent.

Caldera consists of two main components:

• The core system, which is the framework’s code, including an asynchronous command-and-control (C2) server with a REST API and a web interface.
• Plugins which are separate repositories that expand the core framework capabilities and provide additional functionality. Examples include agents, GUI interfaces, collections of TTPs and more.

In Figure 1 below, we are greeted with when we login either as user red or blue and some basic terminology.

1. Agents: An agent is another name for Remote Access Trojan (RAT). These programs written in any language, execute an adversary’s instructions on compromised systems (victims). Often, an agent will communicate back to the adversary’s server through an internet protocol, such as HTTP, UDP or DNS. Agents also beacon into the C2 on a regular basis, asking the adversary if there are new instructions. If a beacon misses a regularly scheduled interval, there is a chance the agent itself has been discovered and compromised.
2. Abilities: An ability is a specific set of instructions to be run on a compromised host by an agent immediately after sending the first beacon in.
3. Adversaries: Adversary profiles are groups of abilities, representing the tactics, techniques, and procedures (TTPs) of known real-world APT groups. Adversary profiles are used when running an operation to determine which abilities will be executed.
4. Operations: An operation is an attack scenario which uses the TTPs of pre-configured adversary profiles. An operation can be run automatically where the agents and the C2 server run without the operator’s interference and can only run tasks in the adversary profile. On the other hand, there is the manual mode where the operator approves every command before it is tasked to an agent and executed. Additionally in manual mode the operator can add extra TTPs. In order to run an operation at least one agent must be active.
5. Plugins: They provide additional functionality over the usage of the framework.

Configuring an Agent

When we select “agents” from the figure 1 menu above, we are greeted with the figure 2 page.

If we select the “Configuration” button, a new window opens where we can configure different options for all the agents created afterwards.

• Beacon Timer(s) = This fields sets the minimum and maximum amount of seconds the agent will take to beacon back home.
• Watchdog Timer(s) = This field sets the number of seconds an agent has to wait, if the server is unreachable, before it is killed.
• Untrusted Timer(s) = This field sets the number of seconds the server has to wait before marking a missing or unresponsive agent as untrusted. Furthermore, operations will not generate new links or send new instructions to untrusted agents.
• Implant Name = This field sets the name for the newly created agents.
• Bootstrap Abilities = This is a list of abilities to be run when a new agent beacons back to the server. By default, it runs a command which clears the command history.
• Deadman Abilities = This is a list of abilities to be run immediately before an agent is killed.

To deploy an agent, we can press the “Deploy an Agent” button and we are greeted with this page. For this example, the agent Sandcat will be used.

By deploying the agent we refer to the process of installing and setting up the agent on the target system to enable it to perform specific actions or functions such as: monitoring, management, data collection, exploitation, reconnaissance and many more.

In figure 4, we can select the agent we want to deploy.

Next, in figure 5, we have to select the operating systems the agent will be deployed on.

In this example, the Linux operating system has been chosen and Caldera provides us with some options and some pre-built commands. These commands can be copied and run directly to the victim’s terminal for the agent to be deployed. There are different variations for the deployment of the selected agent such as:

• It can be deployed as a red or blue agent.
• It can be downloaded with a random name and start as a background process.
• It can be deployed as a peer-to-peer (P2P) agent with known peers included in the compiled agent.

Moreover, the settings that can be modified are:

• app.contact.http = This field is where the URL of the server’s address can be specified.
• agents.implant_name = This field represents the name of the agent binary.
• agent.extensions = This field takes a list of agent extensions to compile with the binary.

After an agent has been deployed it will be shown in the agent’s window, as illustrated in Figure 7.

If an agent is selected, a new window opens that shows some settings that can be modified along with some information about the system the agent is installed on and a kill switch, as shown in figure 8.

• Contact = This field specifies the protocol in which the agent will communicate with the server.
• Sleeper Timer = This is the same as the Beacon Timer(s).

Configuring an Adversary Profile

Caldera comes with pre-defined profiles to choose from, loaded with known TTPs. There is also the option to create a new profile with mixed TTPs, providing an operator more flexibility over the operation. An adversary profile can be created and configured in the “adversaries” window as shown below in figure 9.

After the “New profile” button is pressed, a name and a description for the new adversary profile will be asked.

A new ability can be added to the newly created profile by pressing the “add Ability” button.

Then a new window will open where the specific ability can be chosen and configured, as depicted in figure 11.

Here an already existing ability can be added by searching for it in the search bar or a new one can be configured by choosing a specific Tactic, Technique and Ability as shown above, along with all the details shown in the “Ability Details” section.

This newly created ability can be added to the TTPs of an already existing adversary profile by pressing the “Add Adversary” button. A new window will open to choose the appropriate profile.

Finally, by pressing the “Save Profile” button the new profile is created and can be added to an operation.

Configuring an Operation

An operation can be created and configured in the “operations” window.

After that a new window will open with all the modifiable settings.

• Operation Name = Specifies the name of the operation.
• Adversary = Specifies a specific adversary profile to emulate along with the pre-configured TTPs associated with this profile.
• Fact Source = In this field a fact source can be attached to the current operation. This means that the operations will start with some knowledge of the facts which can be used to fill in different variable inside some abilities. A fact is identifiable information about the target machine that can be used by some abilities, such as usernames, passwords, hostname etc.
• Group = Specifies the collection of agents to run against
• Planner = Specifies which logic library to use for the current operation. A planner is a Python module which contains logic that allows a running operation to make decisions about which abilities to use and in what order. The default planner is the “Atomic” which sends a single ability command to each agent in a group at a time. The order in which the commands are sent is the same as in the adversary’s profile.
• Obfuscators = This field specifies which obfuscator to use to encode each command before they are sent to the agents. The available options are:
  • Base64 = Encodes the commands in base64
  • Base64jumble = Encodes the commands in base64 and then adds characters
  • Base64noPadding = Encodes the commands in base64 and then removes padding
  • Caesar cipher = Obfuscates the commands with the Caesar cipher algorithm
  • Plain text = No obfuscation
  • Steganography = Obfuscates the commands with image-based steganography
• Autonomous = Specifies if the operations will run autonomously or manually. In manual mode the operator will have to approve each command.
• Parser = Parsers are Python modules that are used to extract facts from command outputs. For instance, some reconnaissance commands can output file paths, usernames, passwords, shares etc. these facts can then be fed back into future abilities. Parsers can also be used to create facts with relationships between them, such as username and password facts.
• Auto-close = This option automatically terminates the operation when there are no further actions left. Alternatively, it keeps the operation open until the operator terminates it manually.
• Run state = This option pauses the operation on start or runs immediately
• Jitter = Specifies the minimum and maximum number of seconds the agents will check in with the server while they are part of an active operations.
• Visibility = This option specifies how visible should the operation be to the defense. Abilities with higher visibility than the operation’s will be skipped.

After the “start” button is pressed the operation will start and the results will be shown on the screen whether each task fails or succeeds. There is also the option to view each command and its result, as illustrated in figure 16.

This was a red team operation, but in order to see the full picture some security solutions should also be running on the target systems to examine what was prevented and what went undetected.

Configure Automated Incident Response Plan

To form an incident response plan the “blue” user must be logged in.

The blue team’s main menu is a little different than the red team’s one. The main change is the “response plugin” which is a counterpart of the threat emulation plugins. At the time of writing this blog post, it contains 37 abilities and 4 defender profiles that focus on detection and response actions.

In the “Defenders” tab a new custom defender profile can be created and configured with the same way as the adversaries profile.

The profiles included in this plugin are:

• Incident Responder
• Elastic Hunter
• Query Sysmon
• Task Hunter

All available abilities for each defender profile can be viewed in the “abilities” section, after the specific profile has been chosen from the “response” tab, as shown in figure 17.

Defender abilities are classified by four different tactics:

• Setup: These abilities prepare information to be used by other abilities
• Detect: These abilities focus on finding suspicious behavior by continuously monitoring the ingested information and run as long as the operation is active.
• Response: These abilities act autonomously once suspicious is detected. Such actions include, killing a process, modifying firewall rules, deleting of a file and so on.
• Hunt: These abilities focus on searching for Indicators of Compromise (IOCs) via logs or file hashes.

Blue team operations are configured the same way as the red team operations. The main difference in the procedure is that the agent must be deployed as blue instead of red, in the “adversary” option a defender profile must be selected and in the Fact source section the “response” option must be selected.

The result structure is the same as the red team operation. The commands and their output are shown and whether they were successful or not.

Conclusion

In conclusion, leveraging the Caldera framework for adversary emulation presents a robust and proactive approach to enhancing cybersecurity defenses. Through the simulation of real-world attack scenarios, organizations can acquire invaluable insights into potential vulnerabilities and subsequently strengthen their incident response capabilities. The flexibility, modularity, and extensibility of Caldera establish it as an ideal tool for executing sophisticated emulation exercises.

By harnessing adversary emulation in conjunction with the Caldera framework, cybersecurity experts are equipped with the means to proactively safeguard their organizations against potential threats.