class TargetMemory {
  virtual uint64_t mmap(uint64_t hint, size_t len) = 0;
  virtual bool mprotect(uint64_t addr, size_t len, int prot) = 0;
  virtual void write(uint64_t addr,  const void *buf, size_t len) = 0;
  virtual void read(void *dst, uint64_t addr, size_t len) = 0;
};

inline void read(void *dst, uint64_t addr, size_t len) override {
  return memcpy(dst , (void *)addr , len);
}

class TritonVM(pyqbdl.TargetMemory):
    def __init__(self, ctx: TritonContext):
        super().__init__()
        self.ctx = ctx

    def read(self, ptr, size):
        return self.ctx.getConcreteMemoryAreaValue(ptr, size)

Symbols resolution

The three executable formats ELF, PE and, Mach-O supported by QBDL use different mechanisms to bind imported symbols. This section introduces these differences and how they are managed in QBDL's loaders.

$ readelf -r ./elf.bin

Relocation section '.rela.plt' at offset 0x348 contains 3 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
0000006008f8  000100000007 R_X86_64_JUMP_SLO 0000000000000000 printf@GLIBC_2.2.5 + 0

const uintptr_t addr_target = base_address_ + reloc.address();

// [...]

case RELOC_x86_64::R_X86_64_GLOB_DAT: {
    const uintptr_t addr = engine_->symlink(*this, reloc.symbol());
    engine_->mem().write_ptr(binarch, addr_target, addr + reloc.addend());
    break;
  }

for (const BindingInfo &info : binary.dyld_info().bindings()) {

  // [...]

  const auto &sym = info.symbol();
  const uint64_t ptrRVA = get_rva(binary, info.address());
  const uint64_t ptrAddr = base_address_ + ptrRVA;
  const uint64_t symAddr = engine_->symlink(*this, sym);

  engine_->mem().write_ptr(binarch, ptrAddr, symAddr);
}

for (const Import &imp : binary.imports()) {
  for (const ImportEntry &entry : imp.entries()) {
    const uint64_t iatAddr = entry.iat_address();

    LIEF::Symbol sym{entry.name()};
    const uintptr_t symAddr = engine_->symlink(*this, sym);
    engine_->mem().write_ptr(binarch, base_address_ + iatAddr, symAddr);
  }
}

What is NVIDIA NGX?

NVIDIA NGX is an SDK provided by NVIDIA that uses machine learning techniques to provide image and video enhancement algorithms, like upscaling, slow motion or image reconstruction. It is also the framework used by DLSS, an upscaling technology tailored for video games.

In order to download the SDK, you need to register to the NVIDIA Developer program [1]. At the time of this writing, it is free, and access is granted directly. Unfortunately, only a Windows version is provided.

That being said, NVIDIA announced in 2020 [2] that the NGX library was shipped to Linux starting from the 450.51 driver. Let's understand what is really shipped and what is missing, by first understanding how all of this works.

NGX architecture

First of all, NGX is only made of userland components, no specific driver interface is involved. It relies on the Direct3D, Vulkan and/or CUDA APIs (depending on the component).

The first main component is the NGX shared library, which is:

• nvngx.dll on Windows.
• libnvidia-ngx.so on Linux (can be found on Debian systems in /usr/lib/x86_64-linux-gnu/libnvidia-ngx.so).

These components expose the final user API, that is for instance used in the sample provided in the NGX SDK archive. Then, each NGX "feature" is implemented in a dedicated shared library:

• Image Super Resolution (isr): nvngx_dlisr.dll.
• In-Painting (inpainting): nvngx_dlinpainting.dll.
• Video Super Resolution (vsr): nvngx_dlvsr.dll.
• Slow Motion (slowmo): nvngx_dlslomo.dll.

The Linux NGX implementation (libnvidia-ngx.so) looks for libnvidia-ngx-dl{feature}.so files (e.g. libnvidia-ngx-dlisr.so). These ELF shared libraries are not publicly provided by NVIDIA.

The last component of the NGX framework is the updater. It is implemented within the ngx-updater binary under Linux, and is transparently launched by the libnvidia-ngx.so component. According to the documentation [3], its role is to download updated machine learning models for each feature. We didn't try to reverse engineer that part.

So the missing bits with the current NGX Linux version are the implementation of the actual features (the libnvidia-ngx-dl*.so files). In the next sections, we will use QBDL to implement them based on their Windows counterparts.

Challenges

We started with the NGX feature that looked like the easiest one: Image Super Resolution (nvngx_dlisr.dll). Indeed, in terms of data, it looks like it takes an image, upscales it, and regenerates an image. No video processing in the middle.

After a quick look at nvngx_dlisr.dll libraries, we can see that:

• It imports functions from the cuDNN [8] library. Good news for us, it also exists natively for Linux.
• It imports functions from the Windows API (kernel32.dll, user32.dll & so on). We will have to implement some of them, or at least emulate the behavior we need.
• The Visual C runtime library (aka MSVCRT) is statically linked.
• The NVIDIA CUDA runtime is also statically linked.

So we have a few challenges ahead, and others we discovered while making this happen:

• Signature verification on shared libraries.
• Setting up a valid Windows environment for MSVCRT to work (or, more exactly, how not to do it).
• Making the CUDA runtime work.

Let's describe these in the following sections.

Bypassing signature verification

The libnvidia-ngx.so main library verifies a signature within the nvngx_dl*.so files, which should be in a .nvsig section. We didn't take the time to reverse engineer the format of this signature. Indeed, even if we knew how to sign, chances are we wouldn't have a valid private key to sign our wrapping binary. So, whatever happens, we either need to patch libnvidia-glx.so to change a potential public key/root of trust that is used to verify the signature, or to just bypass that signature. We chose the second option.

This is basically only modifying a flag passed to the function at address 0x6EF00, which is responsible for loading these .so plugins. This is done by patching the code in memory.

The static library problem

The NGX runtime libraries are linked with the static Visual C RT library. This means that all the necessary C and C++ library implementations are shipped within the shared libraries. While this is theoretically fine, these implementations rely on many components of the Windows OS (e.g. the PEB). Even if it is possible to setup a PEB-like structure under Linux [4] - as Wine does -, it is not something QBDL provides at this point.

The same kind of issue arises with the CUDA runtime: it is also statically linked. The CUDA runtime makes calls to the CUDA userspace driver, but some structures are not defined and would require further reverse engineering. wine-staging contains such a wrapper [6], but using them involved many CUDA-related failures in our case. It would be better to directly call the Linux CUDA runtime libraries.

One idea to fix these problems is to identify the entry points of these static libraries within the NGX's DLL, and hook them. This is far from ideal, as it binds our QBDL wrapper to a specific version of the NGX libraries (e.g. with this version-specific list of offsets), but it made the whole thing work in the end.

IDA provides a signature mechanism named FLIRT [5]. It first creates a signature database from a static library with symbols, and then searches for these signatures in our DLLs. How to do this is out of the scope of this blog post, but this article is a good reference.

Putting everything together

The final wrapper can be downloaded here: https://github.com/quarkslab/nvidia-ngx-wrapper. Please follow the README for build & usage instructions.

In the end, we generate a libnvidia-ngx-dlisr.so shared library that uses QBDL to load nvngx_dlisr.dll, and then uses the Linux implementation of cuDNN & the CUDA runtime. This is all called through the NGX Linux interface implementation (in libnvidia-ngx.so). One last detail we didn't mention yet is that we need to adapt the interface of an NGX C++ virtual interface (that is used to basically describe options) from the MSVC C++ ABI to the Linux one [7]. This is necessary because the order of the functions in the vtable is not the same between the two ABIs. Having tools that automate that kind of wrapping would greatly help (an idea for yet another Clang-based project?).

In the end, here is an example of the wrapper in action that uses NGX to upscale an image:

With the original and upscaled image: