Welcome back to part 2 of our IoT Malware Emulation series. If you are new here, check out part 1 where we describe the setup to emulate malware.

Just as a reminder, we have a Docker container and are using our script “Qiliot,” which is based on Qiling, to emulate the IoT wiper malware AcidRain. AcidRain searches for some storage devices and primarily targets both character-based and block-based devices. We manually created these devices as files.

In this episode, we will discuss the technical aspects. We fronted some issues, which we will describe along with our solutions. Let’s start with an issue we teased in part 1, but first, a small addition.

Define Analysis Cases

As we mentioned before, AcidRain overwrites two different storage devices, which will be discussed later. We want to mention that we defined four emulation cases. The goal is to run every execution path. Each emulation execution path is defined by which storage device we have and whether AcidRain has root privileges. Root privileges don’t change the functionality in AcidRain; they just alter the sequence of its tasks.

        # Set attributes for AcidRain cases.
        match emucase:
            case 0:
                self.root = True
                self.mtd_type = 4
            case 1: 
                self.root = True
                self.mtd_type = 0
            case 2:
                self.root = False
                self.mtd_type = 4
            case 3:
                self.root = False
                self.mtd_type = 0

To select a defined emulation case in our environment, we will set a parameter when running Qiliot:

parser.add_argument('-c', '--emucase', required=True, type=int, metavar="EMULATION CASE", default="INFO", choices=[0, 1, 2, 3])

Bugs and Issues

Right at the beginning, when we emulated AcidRain, we ran into an error with getdents64 which occurs in the Qiling framework.

open("/", 0x10080, 00) = 3
0x004018f4: fstat(fd = 0x3, buf_ptr = 0x7ff3cb38) = 0x0
0x00403418: fcntl(fd = 0x3, cmd = 0x2, arg = 0x1) = 0x0
brk: increasing program break from 0x488000 to 0x48a000
Write dir entries: bytearray(b’\x00\x00\x00\x00\x00\x06R\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x15\x04.\x00’)

[... Write dir entries logs ..]

Write dir entries: bytearray(b’\x00\x00\x00\x00\x00\x06O<\x00\x00\x00\x00\x00\x00\x00\x00\x00\x1e\x08bram_1.bin\x00’)
getdents64(0, /* 37 entries */, 0x1000) = 1046
0x0040365c: getdents64(fd = 0x3, dirp = 0x7ff3bc00, count = 0x1000) = 0x416
Traceback (most recent call last):

[...Traceback log ...]

qiling.exception.QlErrorCoreHook: _hook_intr_cb : not handled

The problem is, AcidRain is compiled and runs under the MIPS architecture. MIPS is part of the RISC architecture family, where data alignment is crucial. Whenever data access happens, it needs to be naturally aligned (e.g., a 16-bit access must be on a 2-byte boundary). Qiling had not considered this case. That’s why we needed to fix it and create a pull request in the Qiling project (by the way: it is still open).

Note: Until the pull request is merged, we have a workaround that includes patch files rolled out by the Dockerfile.

From here, the fun part starts: Hooking syscalls … - Not yet!

The Daemon Process

After the bug fix, we can see that Qiling emulates AcidRain from beginning to end (without the malicious part because of “not implemented syscalls”, which I will get to shortly). In the terminal, we can see all the syscalls that AcidRain uses. However, in the log file, we only see the beginning, meaning a significant amount of logs are missing.

Long story short: The problem is that AcidRain creates a new child process with the fork syscall and terminates the parent process, hiding itself as a daemon process. It invokes the setsid syscall to initiate a new session. Since a terminal can only host one session, this action classifies the process as a daemon and appoints it as the group leader. As a result, AcidRain tries to function as a daemon, remaining undetected and avoiding termination via the terminal.

However, by terminating the parent process with the exit syscall, Qiling halts the process with ql.os.stop, thereby concluding the emulation and triggering the summary output. In contrast, for child processes, Qiling employs os._exit(0), leading to an immediate termination without a proper halt in emulation or a summary output. This scenario is not indicative of a flaw in Qiling but rather highlights a workaround that fails to fulfill the specific requirements of this analysis. Therefore, modifications have been made to ensure child processes are halted in the same manner as the parent processes.

The hook looks like this:

    def hook_fork(self, ql:Qiling, *args, **kwargs):
        '''
        Hook fork to set ql.os.child_processes to False for every child to prevent qiling to do os.exit()
        
        Args:
            ql (Qiling): The Qiling emulator instance
            *args, **kwards: Additional and keyword argumentd
        '''
        # Prevent qiling doing os.exit()        
        ql.os.child_processes = False

And now, before the ql.run() command which starts the emulation, we add a hook to Qiling that is always triggered after fork has been called:

ql.os.set_syscall("fork", self.hook_fork, QL_INTERCEPT.EXIT)

Qiling checks whether the current process is a child by using the attribute child_processes. To achieve comprehensive logging, it is practical to patch this attribute to false after a fork syscall, ensuring that each child process is appropriately stopped. If the analyzed process forks, the analysis also forks to track all paths of execution. It is worth noting that this customization does not affect the functionality of AcidRain’s emulation.

The modifications ensure that it now starts and exits properly.

And now … Hooking the syscalls.

Hooking Overwrite

Qiling emulates a variety of syscalls, including the ioctl syscall. This syscall is used to perform operations on storage devices (or various other devices). These operations are defined as commands within the ioctl syscall. Qiling has not implemented all commands, only a few. So the goal is to hook the ioctl syscall and implement the commands that AcidRain needs.

First of all, AcidRain uses the following ioctl syscalls for overwriting storage devices:

While Qiling has implemented the ioctl syscall, it does not support these specific commands. Therefore, we are creating an enum of the necessary ioctl commands containing both the name and the hex value like we can find here:

class IoctlCommands(Enum):
    BLKGETSIZE64 = 0x40041272
    MEMGETINFO = 0x40204d01
    MEMUNLOCK = 0x80084d06
    MEMERASE = 0x80084d02
    MEMWRITEOOB = 0xc00c4d03
    MEMLOCK = 0x80084d05

Custom Hook

We defined different emulation cases. To set the storage device (mtd_type), we need to create a custom hook for the ioctl syscall to expand the parameter list. Our custom call hook is assembled with partial and then attached to the ioctl syscall with the intercept type CALL:

custom_hook = partial(self.hook_ioctl, mtd_type=self.mtd_type)
custom_hook.__name__ = "hook_ioctl"
ql.os.set_syscall('ioctl', custom_hook, QL_INTERCEPT.CALL)

Hook for Block-Based Devices

AcidRain searches for storage devices by generating device names. If it finds a match, it starts its overwriting function. As mentioned before, this function differentiates between block-based and character-based storage devices. We will begin with the block-based devices and later build up the hook for character-based devices.

So, let’s begin. If a name matches, AcidRain uses the ioctl command BLKGETSIZE64 to determine the block size and then overwrites it with decrementing data. The data is overwritten in 0x40000 byte blocks, with synchronization every 1024 loops using fsync.

[ ... ]
0x004019d4: getuid() = 0x0
open("/dev/sda", 0x1, 00) = 3
0x00401b28: open(filename = 0x7ff3cd98, flags = 0x1, mode = 0x0) = 0x3
0x00401a44: ioctl(fd = 0x0, cmd = 0x40041272, arg = 0x7ff3cd58) = -0x1 (EPERM)
lseek(3, 0x0, 0) = 0
0x00401aa4: lseek(fd = 0x0, offset = 0x0, origin = 0x0) = 0x0
0x00401d34: write(fd = 0x3, buf = 0x447004, count = 0x40000) = 0x40000
0x00401d34: write(fd = 0x3, buf = 0x447004, count = 0x40000) = 0x40000
0x00401d34: write(fd = 0x3, buf = 0x447004, count = 0x40000) = 0x40000
0x00401d34: write(fd = 0x3, buf = 0x447004, count = 0x40000) = 0x40000
[ ... ]

In our environment, /dev/sda is not a real block device, so ioctl syscalls cannot be effectively executed on it. In cases where the ioctl command fails, AcidRain sets the value to 2^64 - 1 (0xFFFFFFFFFFFFFFFF), indicating the size of data intended for writing. This would result in AcidRain calling the write system call approximately 70 trillion times. Consequently, the fsync function would be invoked approximately 69 billion times, either as a routine operation or if the write process fails due to reaching the end of the file. The emulation of the ioctl syscall below returns a result value of 0x0, signaling successful execution, and writes the size to the memory pointed to by the ioctl parameter.

def hook_ioctl(self, ql: Qiling , fd:int, request, pointer, mtd_type, *args, **kwargs):
    '''
    Hook ioctl to simulate request commands with MTD_NANDFLASH storage devices.
    Return successful operation and write needed values of structs onto the stack.
    '''

    if request == IoctlCommands.BLKGETSIZE64:
        # Write size for block devices onto stack. Value of 270008320 is 1026 x 0x40000
        # to trigger in AcidRain the fsync after every 1024 x 0x40000 write syscall
        ql.mem.write(pointer , ql.pack64(270008320))
        return 0x0
    return -0x1

Like with the fork hook, we need to add the hook to the emulation. But here, we will use the intercept type CALL as shown below::

ql.os.set_syscall('ioctl', custom_hook, QL_INTERCEPT.CALL)

AcidRain synchronizes the data with the fsync syscall after every 1024 write operations, so in the emulation, the size is set to 1026 times 0x40000, equaling 0x10080000 (270008320). This setting triggers the synchronization process once and executes it. After 1026 iterations of the write system call, a return value of -1 is observed, indicating the completion of the overwrite process. AcidRain then synchronizes the new data and closes the file before searching for the next device.

How we implemented the ioctl hook in Qiliot is described and implemented in our GitHub project Qiliot.

Okay, got all that? Great - that was just the warm-up. Now, let’s extend the hook for character-based devices.

Hook for Character-Based Devices

AcidRain has a special case for those with names starting with /dev/mtd and generates device names from 0 to 99. AcidRain starts as follows:

The fstat function provides the status of the current file in the form of a stat structure, which includes the mode_t field describing how the file is protected. Various flags can be set in this field, as detailed in the Linux Man Page for fstat. For AcidRain, the initial flag in the mode that describes the file type is critical.

AcidRain performs a bitwise AND operation between the mode value and the file mask type value 0xf000, expecting the result 0x2000, which represents a character device.

If the file is indeed a character-based device, AcidRain uses the ioctl syscall with the MEMGETINFO command to retrieve further data. The data will be written into the MtdInfoUser struct, which contains information such as the size of the storage device, the sizes of the areas for writing and erasing, and the type of MTD device which we mentioned in the beginning where we defined the emulation cases.

class MtdInfoUser:
    '''
    Initialize struct for mtd_type. Is utilzed for ioctl syscall with MEMGETINFO.
    Needed for MTD type = 4 (NANDFLASH)
    '''
    def __init__(self, type:int):
        self.type = type # Type 0 or Type 4
        self.flags = 0
        self.size = 100  # 1 MB in local_d4l
        self.erasesize = 50 # 0x1000 in local_d0
        self.writesize = 256
        self.oobsize = 64

To handle the MEMGETINFO write the required information into memory, you can extend the hook_ioctl function as follows:

        if request == IoctlCommands.BLKGETSIZE64:
            # Write size for block devices onto stack. Value of 270008320 is 1026 x 0x40000
            # to trigger in AcidRain the fsync after every 1024 x 0x40000 write syscall
            ql.mem.write(pointer , ql.pack64(270008320))
            return 0x0
        elif request == IoctlCommands.MEMGETINFO:
            # Handle MEMGETINFO and write onto stack the mtd_info_user structure with values.
            mtd_info = MtdInfoUser(type=mtd_type)
            ql.mem.write(pointer, ql.pack8(mtd_info.type))
            ql.mem.write(pointer + 4, ql.pack32(mtd_info.flags))
            ql.mem.write(pointer + 8, ql.pack32(mtd_info.size))
            ql.mem.write(pointer + 12, ql.pack32(mtd_info.erasesize))
            ql.mem.write(pointer + 16, ql.pack32(mtd_info.writesize))
            ql.mem.write(pointer + 20, ql.pack32(mtd_info.oobsize))
            return 0x0

These hooks ensure that AcidRain can retrieve the necessary information from the character-based MTD devices and proceed with its intended overwrite operations.

AcidRain then unlocks the MTD device to progressively erase it based on the erasure size from the MEMGETINFO struct. AcidRain distinguishes between MTD devices of type MTD_NANDFLASH (0x4) and others. We have hardcoded all this information and written it to the current memory pointer using our ioctl hook, so that AcidRain can function properly.

For all other ioctl commands, we will expand the ioctl hook. The next step is to unlock the storage device. AcidRain uses the MEMUNLOCK command, and it needs the EraseInfoUser struct:

class EraseInfoUser:
    '''
    Initialize struct for erase_info_user. Is utilzed for ioctl syscall with MEMUNLOCK and MEMERASE request.
    Needed for MTD type = 4 (NANDFLASH)
    '''
    def __init__(self, start, length):
        self.start = start
        self.length = length

This defines the block where it starts and the length, which AcidRain can delete. MEMUNLOCK and MEMERASE use this struct. MEMUNLOCK is used to open the area to erase, and MEMERASE contains the information of the next block to be deleted. So we will expand our hook with the following commands:

        elif request == IoctlCommands.MEMUNLOCK:
            # Handle MEMUNLOCK to unlock a device and write onto stack the erase_info_user struct
            # with informations where acidrain starts to erase the flash and the length
            erase_info_user = EraseInfoUser(20, 40)
            ql.mem.write(pointer, ql.pack32(erase_info_user.start))
            ql.mem.write(pointer, ql.pack32(erase_info_user.length))
            return 0x0
        elif request == IoctlCommands.MEMERASE:
            # Handle MEMERASE and write new section onto stack which acidrain trys to deletes next
            erase_info_user = EraseInfoUser(80, 250)
            ql.mem.write(pointer, ql.pack32(erase_info_user.start))
            ql.mem.write(pointer, ql.pack32(erase_info_user.length))
            return 0x0

In the case of other MTD types, AcidRain utilizes the MEMWRITEOOB command to overwrite the storage device. To lock the device, it uses MEMLOCK. Here, we need to expand our ioctl hook again but only with the return code 0x0, and pretend that the operation is successful. Feel free to implement additional functionality, but for the purpose of emulating and executing AcidRain, it is not necessary:

        elif request == IoctlCommands.MEMWRITEOOB:
            return 0x0
        elif request == IoctlCommands.MEMLOCK:
            return 0x0

After these operations, AcidRain uses implemented syscalls to ensure that the data modifications are fully written to the storage device and that the device is in a consistent state post-deletion.

open("/dev/mtd0", 0x2, 00) = 3
0x00401b28: open(filename = 0x7ff3cd98, flags = 0x2, mode = 0x0) = 0x3
Received interrupt: 0x11
0x004018f4: fstat(fd = 0x3, buf_ptr = 0x7ff3cbd8) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMGETINFO, pointer = 0x7ff3cca4) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMUNLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMERASE, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMUNLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMERASE, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMUNLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMERASE, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMUNLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMERASE, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMWRITEOOB pointer = 0x7ff3cc98) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMUNLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMERASE, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMWRITEOOB, pointer = 0x7ff3cc98) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMUNLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMERASE, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMWRITEOOB, pointer = 0x7ff3cc98) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMUNLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMERASE, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMWRITEOOB, pointer = 0x7ff3cc98) = 0x0
fsync(3) = -1
0x00401974: fsync(fd = 0x3) = -0x1 (EPERM)
0x00401aa4: lseek(fd = 0x3, offset = 0x0, origin = 0x0) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMLOCK, pointer = 0x7ff3cc90) = 0x0
0x00401a44: ioctl(fd = 0x3, request = MEMLOCK, pointer = 0x7ff3cc90) = 0x0
fsync(3) = -1
0x00401974: fsync(fd = 0x3) = -0x1 (EPERM)
close(3) = 0

Hooking Reboot

After the overwrite functionality, AcidRain attempts to reboot the device in a redundant and excessive manner.

First, it uses the reboot syscall with different magic numbers specifying how the system should reboot. This syscall is called four times. Secondly, it utilizes the fork syscall also four times to continue executing Linux reboot binaries and to exit the child processes.

For both the reboot and execve syscalls, we hook them and simulate successful execution. We do not actually want to reboot the device nor execute the reboot process. Generally, we aim to emulate a new process which the malware is invoking, but in this case, the reboot binary would typically be emulated by the Qiling framework. We choose to avoid this as it would include all related information in the logs, which is not desired in this scenario. The implemented hooks look like this:

    def hook_reboot(self, ql:Qiling, magic:int, magic2: int , cmd:int, *arg):
        return 0x0

    def hook_execve(self, ql: Qiling, filename, argv, envp, **kwargs):
        ql.log.info(f"Simulates execve: {ql.os.utils.read_cstring(filename)}")
        return 0x0

These hooks are integrated as usual with the following commands:

        ql.os.set_syscall('execve', self.hook_execve, QL_INTERCEPT.CALL)
        ql.os.set_syscall('reboot', self.hook_reboot, QL_INTERCEPT.CALL)

With the implementation of the hooks and fixes, AcidRain was successfully emulated. According to the log messages, it appears that AcidRain executed its primary functionalities.

Note: The logging mechanism was patched to yield direct and readable output. For instance, in Debug mode, the written data is displayed, which can clutter the log; hence, only the length of the written data is shown. You can find the patches in our GitHub project of Qiliot.

Now we want to see, I mean really see, if this works, and we love nothing more than numbers. For this, Qiling offers a function to create coverage files. How we utilize this functionality, what the results look like, and whether it’s as simple as it sounds will be covered in part 3.

Spoiler alert: No, it didn’t work out of the box. :D