If you have ever looked at fuzzing in any depth you will quickly realize it’s not as trivial as it first appears.

There are many different types of fuzzers, but here we are focused on network fuzzers.  These fuzzers are of particular interest as they are most suited to fuzzing telecoms products/protocols, where the application and source code are generally not available.  There are very few fuzzer options when the input to these applications is via network sockets instead of the more traditional command line interface.

In this blog we will cover some basic background of fuzzing, before diving into the specifics of trying to fuzz telecoms 5G protocols using both proprietary and open source fuzzers.  We will aim to assess these fuzzers for their suitability to fuzz 5G network protocols.  We will end this post with a comparison of the fuzzers, some of the findings, and a general conclusion regarding the fuzzing of 5G telecoms protocols.

The focus of this research is on the use of the different fuzzers with regard to 5G telecoms protocols and not the specific vulnerabilities that were found, although one example vulnerability found is cited within.

Background

So, what is fuzzing?  Fuzzing is simply an automated process of sending invalid or random inputs to a program/system under test in an attempt to cause a crash or malfunction.

Fuzzing is not a new technology, however it is becoming more prominent in today’s software development life cycle. It is often used to find vulnerabilities in software that might otherwise be missed by normal unit/system tests. 

While the high-level concept of fuzzing is easy to grasp, the actual implementation of a good fuzzer is significantly more challenging.

The renewed interest in fuzzing has come about due to the increasing complexity of software and the need to effectively test and secure it.  While positive testing is generally more obvious (i.e. testing the software does what it was designed to do), negative testing in normally forgotten about (testing the software handles unexpected input without crashing or malfunctioning).

Traditional fuzzers tend to focus on fuzzing a piece of software and generate inputs via the command line or input files.  Some of the popular continuous integration frameworks (GitLab) are now starting to include fuzzing as part of the continuous integration build pipeline.

Fuzzing network protocols is a little different however, and requires sending input via network ports.  There are typically multiple network protocols involved in any communication and these protocols are layered on top of each other.  Some protocols are stateless, and others have state which adds to the complexity.  Due to the nature of network protocols the System Under Test (SUT) could be either local (on the same physical/virtual machine) or on a remote physical/virtual machine.  These differences add to the challenge of fuzzing a SUT using network protocols.

The next difficulty specific to fuzzing 5G protocols is getting access to a 5G Core or component.  There are two open-source solutions Free5GC and Open5GS which were examined for our testing.  Open5GS was chosen due to it being more stable and easier to install than Free5GC.  Neither of these solutions are commercial grade but they do give a reasonable test target to fuzz for free.

We also need some network fuzzers…

Meet the Fuzzers

One in-house, and two open-source network protocol fuzzers were used for testing: Fuzzowski 5GC, Frizzer², and AFLNet³.  They all have completely different approaches to fuzzing from the generation of the test cases to the feedback on progress made.

Fuzzowski¹ was chosen as it had previously been used to fuzz network printer protocols and was developed by NCC Group from the open-source project BooFuzz⁴.  Fuzzowski is open source, however the modified version used here for fuzzing 5G is not currently an open-source project.

Frizzer is a black box fuzzer that uses dynamic binary instrumentation to give coverage feedback to guide the fuzzer.  No source code or recompilation of the SUT is needed.

AFLNet was used as a comparison due to the reputation of AFL being a well-used fuzzer with proven results.  AFLNet builds on top of AFL to perform network fuzzing and track state changes using the network responses.

Fuzzowski 5GC

Fuzzowski 5GC is a template based mutational/generational fuzzer.  This simply means that the format of the input is specified, and the values defined within the format are mutated using selected algorithms depending upon the data type.

Fuzzowski 5GC can fuzz sequences of messages however it has no concept of state.  As Fuzzowski 5GC is aware of the data structure being fuzzed, it can fix checksums and length fields which helps avoid early parsing errors that would prevent testing of deeper parts of the protocol stack.

Fuzzowski 5GC is a black box fuzzer meaning it has no knowledge of the SUT other than the network API.

Frizzer

Frizzer is a black box guided mutational based fuzzer.  It uses example input and randomly mutates it using Radamsa⁷.  It uses Frida⁶ to dynamically instrument the SUT to provide code coverage feedback.

AFLNet

AFLNet is a guided mutational based fuzzer.  It uses example input and randomly mutates part of the input based on different mutation algorithms.  It has no knowledge of the input data format and uses state feedback from the network messaging to guide the fuzzing process.

AFLNet is a grey box fuzzer as it uses source code instrumentation to generate the code coverage feedback.

The Test Environment

For testing the fuzzers an ubuntu environment running Open5GS⁵ was used.

Open5GS is an open-source implementation of a 5G Core and EPC written using the C language.  Open5GS was chosen to emulate the 5G core as it is freely available and actively being maintained.  Due to it not being a commercial product and written in C, it makes an ideal target for fuzzing as it is unlikely to be as thoroughly tested.  It is also more likely to be focused primarily on functionality, rather than security.

As the protocol specifications are the same for both open source and commercial products, the network message formats should be representative of a real 5G Core network. We chose to limit the scope of testing to look at the NGAP, GTPU, PFCP, and DIAMETER protocols, which we will explain briefly below.

All the fuzzers were tested against the AMF component of the Open5GS software to fuzz the more complex NGAP protocol. The theory being that as one of the more complex 5G protocols, it has a higher probability of bugs – however, it is also more difficult to fuzz as a result of its complexity.

Fuzzowski 5GC was also tested against other 5G components to fuzz the GTPU, PFCP and DIAMETER protocols.

Finding vulnerabilities in these protocols and their implementation can lead to various types of attack scenarios such as denial of service, privilege escalation, remote code execution, user information disclosure and capture.

GTPU

GPRS Tunneling User Data Protocol (GTPU) is effectively a relatively simple IP based tunnelling protocol, which can have many tunnels between each set of end points.  It is used to tunnel user plane data between different network nodes.  In our testing this is the N3 interface between gNB and UPF.

PFCP

Packet Forwarding Control Protocol (PFCP) facilitates the establishment, modification, and deletion of Sx sessions within the user plane function.  PFCP rules which are passed from the control plane function to the user plane function include things like Packet Detection Rule (PDR), QoS Enforcement Rule (QER), Buffering Action Rule (BAR) etc.  In our testing this is the N4 interface between the UPF and SMF.

DIAMETER

DIAMETER is an authentication, authorization and accounting protocol and is an application layer protocol.  The base protocol can be extended by adding new commands and/or attributes.  In our testing this is the S6a interface between the MME and HSS.  The DIAMETER S6a interface allows for mobile device related location information and subscriber management information between MME and HSS.

NGAP

Next Generation Application Protocol (NGAP) supports both UE and non-UE associated services.  It includes operations such as configuration updates, UE context transfer, PDU session resource management and also support for mobility procedures.  In our testing this is the N2 interface between the AMF and gNB.

Fuzzing Results

Fuzzowski 5GC found several issues with GTPU, PFCP and DIAMETER but failed to find anything for the NGAP protocol.

Frizzer and AFLNet were only run against a subset of the 5G protocols and found some issues which at time of writing are under further investigation and, as appropriate, coordinated disclosure.

The types of crashes that can be observed in these targets could cause loss of service for subscribers of the network, preventing them from connecting to the network (denial of service), or potentially other security implications if stack/heap corruptions can be exploited to execute code or gain privileged access.

The following is an example crash caused while fuzzing the GTPU/PFCP protocols using Fuzzowski 5GC. This bug has now been patched as of October 6^th 2021 (fix committed to main branch of Open5GS and released in version 2.3.4).

In the next section, we’ll discuss this bug in more depth, but also share the associated Technical Advisory and CVE details below:

• Technical Advisory – Open5GS Stack Buffer Overflow During PFCP Session Establishment on UPF
• CVE-2021-41794

PFCP Bug (CVE-2021-41794)

This shows a stack corruption caused by the function ‘ogs_fqdn_parse’ writing beyond the end of the ‘dnn’ character buffer which is defined on the stack as 100 characters (OGS_MAX_DNN_LEN = 100). If ‘message->pdi.network_instance.data’ contains the value ‘internet’ for example, it causes stack corruption to occur.

There are a few issues with the function ‘ogs_fqdn_parse’. The first is the calculation of the variable ‘len’ which is being set to the value of the first character in the ‘src’ parameter. In this example it is the lower case letter ‘i’ which equates to the numerical value 105. The length of the ‘src’ parameter is 8, however this is not checked until it’s too late. The memcpy reads past the end of the ‘src’ parameter and also writes beyond the end of the ‘dst’ parameter (which is actually the variable ‘dnn’ on the stack).

As the ‘src’ parameter is ultimately coming from the PFCP Session Establishment Request it could be manipulated to contain any value and the length controlled by setting the first byte.

Comparative Performance of the Selected Fuzzers

Fuzzowski 5G

Fuzzowski 5GC proved to be good at finding bugs in state less protocols, however various modifications were made to Fuzzowski 5GC in an attempt to fuzz the NGAP protocol.

The biggest issue with using Fuzzowski 5GC is that it needs the structure of the messages to be defined.  Creating message definitions, sequences, and functionality to handle message sequences is a slow and manual process.  The messages are generally created from WireShark captures and therefore tend not to cover all parts of the protocol specification (e.g., optional elements).

Frizzer

Frizzer was easy to setup as the source code was available for the SUT.  If there had been no source code, some reverse engineering would be required to find the function/address of the network processing section of the application.

The SCTP protocol was added to Frizzer so that it could connect to the AMF.  It was also modified to keep the SCTP connection open between tests as the AMF would fail otherwise.

As frizzer uses Frida⁶ to dynamically instrument the binary there is no need for special compilation of the application, as required with AFLNet.

Frizzer has the same issues as AFLNet regarding checksums, although it may be possible to use Frida⁶ to dynamically change the execution path of the SUT and force checksum calculations to pass.

Frizzer testing was limited as without fixing a bug in the AMF the testing kept stopping after a few hundred test cases.

AFLNet

AFLNet required a reasonable amount of work to setup.  The program defaults were not suitable for our testing purposes, so they were overridden on the command line.  The SCTP protocol was added to the connection types to prevent the SCTP protocol from being fuzzed.

For AFLNet to function the SUT needs to be compiled with instrumentation.  To compile the AMF application, the build process for Open5GS was modified to instrument the code.  Due to the AMF application being initialized for every test by AFLNet, the process of fuzzing was very slow compared to the other fuzzers.

Because of the slow speed of fuzzing, minimal time and effort was spent to enable fuzzing of a single NGAP message.  The mutational nature of AFLNet means it would not be very effective at dealing with length and checksum parameters, making it difficult for it to explore the deeper parts of the protocol.

What We Learnt by Fuzzing 5G Protocols

This research shows that fuzzing 5G telecoms protocols is not as straightforward as downloading your favorite open source fuzzer and hitting go!  Sure, they might find a few bugs in the simple stateless protocols, but they fail to find those deeper, harder to reach issues. Fuzzing 5G protocols introduces specific challenges, such as the need for binary instrumentation of commercial 5G components for which source code is unavailable.

A good starting point for any telecoms fuzzer would be to create input from the ASN1 definitions of the protocols. This would make it easier to create test cases for specific versions of protocols, and give better coverage of the protocol compared to manually defining the input.  It would also be quicker and a lot less error prone to produce the test cases.  This approach would require writing an ASN1 parser which could generate suitable output for use with the fuzzer (a reasonable challenge in itself).

It is unlikely that source code would be available when testing a commercial 5G component.  For this reason, binary instrumentation would greatly help in guiding a fuzzer.  It is possible to use tools like Frida⁶ to instrument the SUT to give coverage feedback similar to AFL.

Monitoring for crashes is more challenging as the SUT may be on a remote server.  A monitoring application would need to run on the same server as the SUT to feed status information back to the fuzzer.  As Frida⁶ runs in the target process it could be used for monitoring as well as providing other feedback.

Another issue encountered is unexpected messages.  Some 5G protocols (e.g., NGAP) repeatedly send requests at timed intervals if an invalid response is received.  This causes problems for fuzzers like Fuzzowski 5GC which has a predefined message sequence.  Issues such as this render Fuzzowski 5GC less effective when testing a real system (these messages were disabled during our testing for the open-source products).

There are very few companies offering network fuzzers for 5G protocols and it is easy to see why.  Some fuzzers are more costly and require long testing cycles with complex configuration.  All this extra time and complexity, eats into any development cycle especially if deadlines are tight.  Outsourcing security testing particularly if a business is not resourced to conduct assessments to certain levels of accreditation, is key to easing this burden.

The Catalogue of General Security Assurance Requirements (3GPP TS 33.117 version 16.5.0 Release 16) contains a section on Robustness and Fuzz Testing with more and more operators, regulators and accreditation bodies requiring thorough fuzzing and testing of 5G components in the coming years.  To satisfy these requirements, it is clear a combination of fuzzing tools and techniques are required.

Fuzzowski 5G Modifications

A lot of work has been put into improving Fuzzowski to create our proprietary version for 5G protocols, Fuzzowski 5GC.  The following is a high-level list of functionality/fixes that have been added to the publicly available Fuzzowski¹:

• Global/Local JSON configuration files
• Variables for requests
• Groups to fuzz sections of a request with a set of specific values
• Setup/Teardown for each testcase (used in GTPU fuzzer to setup GTP tunnel)
• SCTP connections
• HTML documentation
• Render option to output byte stream for a request to help with debug
• Help option to display all available options
• Added receive strategy
• Added more test cases to validate functionality
• Protocols added: GTPU, PFCP, DIAMETER, SIP, NGAP/NAS
• Lots of bug fixes and code improvements

Glossary

Term	Description
SUT	System Under Test
AMF	Access and Mobility Management Function
UPF	User Plane Function
Fuzzer	Software that generates invalid input for testing applications
Coverage Guided	Source code is instrumented to give source code coverage metrics to help guide the fuzzer to generate data that uncovers new execution paths within the source code.
Generational Template Fuzzing	Uses a predefined template to specify the structure of the data and then generates invalid data using an algorithm.
MME	Mobility Management Entity
HSS	Home Subscriber Server
gNB	New Radio Node B
ASN1	Abstract Syntax Notation One (ASN.1)

References

[1] GitHub – nccgroup/fuzzowski: the Network Protocol Fuzzer that we will want to use.

[2] GitHub – demantz/frizzer: Frida-based general purpose fuzzer

[3] GitHub – aflnet/aflnet: AFLNet: A Greybox Fuzzer for Network Protocols

[4] GitHub – jtpereyda/boofuzz: A fork and successor of the Sulley Fuzzing Framework

[5] GitHub – open5gs/open5gs: Open5GS is an Open Source implementation for 5G Core and EPC

[6] Frida – A world-class dynamic instrumentation framework

[7] GitLab – Aki Helin / radamsa

How To Work with Us on Commercial Telecommunications Security Testing

NCC Group has performed cybersecurity audits of telecommunications equipment for both small and large enterprises. We have experts in the telecommunications field and work with world-wide operators and vendors on securing their networks. NCC Group regularly undertake assessments of 3G/4G/5G networks as well as providing detailed threat assessments for clients. We have the consultant base who can look at the security threats in detail of your extended enterprise equipment, a mobile messaging platform or perhaps looking in detail at a vendor’s hardware. We work closely with all vendors and have extensive knowledge of each of the major vendor’s equipment.

NCC Group is at the forefront of 5G security working with network equipment manufacturers and operators alike. We have the skills and capability to secure your mobile network and provide unique insights into vulnerabilities and exploit vectors used by various attackers. Most recently, we placed first in the 5G Cyber Security Hack 2021 Ericsson challenge in Finland.

NCC Group can offer proactive advice, security assurance, incident response services and consultancy services to help meet your security needs.

If you are an existing customer, please contact your account manager, otherwise please get in touch with our sales team.