About The Project
CVE-2021-3156 is a 10-year-old sudo vulnerability that allows for privilege escalation in Linux environments. If you’re responsible for a Linux server, this definitely caught your attention due to the severity. Some rough PoCs wound up Github and also on exploit-db recently. Besides patching through upstream providers supplied pathches[0,1], how would you hunt for this in your environment? This leads me to leveraging auditd in the previously blogged about red team range. Let’s take a look at how to create and search for malicious activity with auditd.
Auditd - Linux Logging on Steroids
Auditd is the default auditing daemon for modern RHEL systems and other Linux systems. From file access to specific users running specific commands, Auditd can log it all! Even down to specific syscalls made by processes, auditd is an incredibly powerful logging utility. However, setting up the rules can be difficult, to begin with, if you’re new to the syntax. Red Hat’s documentation is a great starting point.
The auditd package comes with a handful of utilities that interact with the auditd daemon.
Three we’ll talk about in this blog post is as follows:
-
auditctl: add, delete, etc… rules for auditd. -
auditctl -l: will list active rules -
ausearch: Search the auditd logs.- executing
ausearch -k $KEY_NAMEwill identify rules under $KEY_NAME
- executing
-
aureport: Get summary/statistic reporting from auditd.
The next section will focus on adding rules manually, but be aware you can add
several rules to /etc/audit/rules.d/audit.rules and auditd will read from
said file.
Defining Rules - Watching Files
Let’s say we want to watch anytime someone reads or writes to /etc/passwd,
the command below adds a watch (-w) to the file /etc/passwd with
permissions of read write (-p rw) to be logged under the name passwd-rw-check
(passwd-rw-check)
auditctl -w /etc/passwd -p rw -k passwd-rw-check
Now running passwd and changing (or not changing) your passwd will result
in us reading from /etc/passwd and causing this rule to trigger. Searching for this rule via ausearch -k passwd-rw-check we can see the time it occurred, the user that executed it, the pid, ppid, current working directory, and even some selinux related content.
All of the content in the image above came from just running passwd. With that in mind, imagine the difficulties with effectively scaling these logs into some sort of SIEM to do hunting. Tuning your rules on a test system and kind of “purple teaming” yourself by executing what you’re looking for is a great way to understand what a utilitiy does and also limit false possitives in production.
Defining Rules - Checking for Executables
Let’s consider a handful of commands typically used in post-exploitation on Linux systems. In my experience, most droppers are small written in Bash or Python and simply triage the system and then bring a second-stage payload onto the system dependent on the underlying architecture or Linux distribution. From there the rest of the execution chain kicks of or additional processes are killed, etc…
To download second stage payloads wget, curl, and netcat can be leveraged. The three lines below add rules to always watch for when the exes exit if the x64 (-F arch=b64) executable defined (-F cmd=) runs a specific
syscall labeled by -S.
auditctl -a always,exit -F arch=b64 -F exe=/usr/bin/wget -S execve -k www-cmd
auditctl -a always,exit -F arch=b64 -F exe=/usr/bin/curl -S execve -k www-cmd
auditctl -a always,exit -F arch=b64 -F exe=/usr/bin/nc -S execve -k www-cmd
The syscall we’re focusing on is execve.
It is common place to see execve or another member of the execve familiy be called as this performs execution of a particular binary from a syscall perspective. You could also focus on read, as each binary listed below will be reading from some system library. This can be seen by executing an strace -e read wget.
One large caveat to the syscall rule with the hardcoded path is that files that are moved or copied by the initial stager will not be caught. A file watch command would be great for the scenario of files being copied
and then executed under a new name. This is analogous to adversaries on Windows machines renaming “powershell.exe” and bypassing allowlists that are configured within the environment.
Defining Rules - Adding Parameters
Expanding on the previous two examples, it is also possible to add parameters around additional paramaters (man audit) to only log for certain user accoutns. For example, adding -F uid!=0 would indicate our rule wants
non-root users to be logged. However if UID was 0 (I.E root) then it would not be logged.
Hunting for CVE-2021-3156
A more in-depth write up on the vulnerability can be found here. The jist is that there’s a heap buffer overflow in sudo since 2011. The PoCs on
exploit-db [0,1]
rely on a race condition to succeed. Both PoCs execute sudoedit well over three thousand times by default.
Considering that sudoedit is going to be invoked several times within a small time period with strangely large parameters, we have a unique signature to inspect.
So to begin our hunting, lets create a rule that look for the execution of sudo by a non-root user.
auditctl -a always,exit -F arch=b64 -S execve -F exec=/usr/bin/sudo -F uid!=0 -F key=sudo-cmd
auditctl -a always,exit -F arch=b32 -S execve -F exec=/usr/bin/sudo -F uid!=0 -F key=sudo-cmd
Now, all execution of sudo will be logged, but where does sudoedit come into play?
When executing ausearch, we can provide an argument of the comm-name (-c) which according to the man pages is “the executable’s name from the task structure”. The comm-name parammeter is how to filter for sudoedit as it’s actually a symlink to sudo CentOS 8 systems.
After executing the previously discussed PoCs from exploit-db, and searching for our sudo-cmd rule, you should see an enermous amount of logs returning. That’s a big enough number to raise an eyebrow. To narrow it down further, the --start argument can be added with a date and time set within quotes (ex: --start 02/07/2021 '12:00:00') and you can do the same with an end time with --end.
Searching and removing the pipe to wc -l you will notice that the proctitle seems a little bizzare, and you would be correct!
The proctitle is hexadecimal encoded. However, you can decode it by appending -i within your ausearch command.
Man that long string of characters seems pretty odd. Maybe it’s worth a deeper look!
- Anyone seeing this in a SIEM.
At this point, we can gather other context around a particular event and take a
deeper look with the other piexes of information that auditd gives us.For
example, the current working directory (cwd) is /home/jroot/malware, let’s
go pull artifacts from that directory. We see that the syscall was successful, and
have the time that it executed. Auditd can add a great amount of information to
a linux investigation if the rules are tuned ahead of time. You also can
always get a quick snapshot of your environment via aureport
Beyond The Blog
This content can be extended into Auditbeat and integrate your logs into ELK or something similar. Plenty of awesome well structrured auditbeat rules exist on Github, and can give you a head start in your rule creation. If you frequently perform red team engagements in a Linux environment, consider leveraging auditctl to understand what is logged and how you may circumvent it. Thank you for reading!