Back in 2021, I stumbled upon a proof of concept describing an arbitrary file read vulnerability in the Ivanti Avalanche mobile device management tool. As I was not aware of this product, I decided to take a quick look at the vendor’s website to learn more:

“Avalanche Enterprise Mobile Device Management manages some of the most demanding, high-profile supply chain mobility solutions in the world. So, we understand the pressure you're under to maximize worker (and device) uptime.”

“Manage all mobile devices in one place. Smartphones, barcode scanners, wearables and more—configure, deploy, update, and maintain them all in one system.”

“30,000 organizations. Over 10 million devices.”

As the product specification seemed to be both intriguing and encouraging, I decided to give it a shot and look for some vulnerabilities.

I was able to quickly identify a chain of three vulnerabilities in the Ivanti Avalanche Web Application:

- ZDI-21-1298 (CVE-2021-42124): Session takeover vulnerability, which requires user interaction.
- ZDI-21-1300 (CVE-2021-42126): privilege escalation vulnerability, which allows an attacker to gain administrative privileges.
- ZDI-21-1299 (CVE-2021-42125) – Remote code execution vulnerability, which can be exploited from the level of an administrator account.

Even though this chain is powerful, its first part heavily depends on factors that are not within the attacker’s control. We can do better, right?

The Inspiration

Later, I identified several interesting facts concerning Ivanti Avalanche:

- It contains multiple XStream 1.4.12 jar packages in its directories.
- It implements multiple ObjectGraph classes, which wrap the XStream serializer.
- Some of those classes define an allowlist of classes permitted for deserialization, while others lack such an allowlist.

This led me to suspect that there are multiple untrusted deserialization issues in the product. However, I was not yet able to determine how to reach those deserialization routines. Fortunately, I decided to pursue the matter further, and it turned out to be a long, exciting journey. It allowed me to not only exploit the XStream deserialization issues but to significantly increase the attack surface and find many more 0-days.

This blog post describes the first part of my Ivanti Avalanche research, where I identified the Ivanti Avalanche custom network protocol and abused it to perform several remote code executions. Moreover, it describes a cool race condition vulnerability that leads to an authentication bypass.

Identifying Ivanti Avalanche Services

To begin, let’s have a look at the Avalanche services:

We find we are dealing with the Tomcat Web Application plus other services. “Wavelink Information Router” seems to be the most interesting of them, due to the following description: “Coordinates communication between Wavelink processes…”.

At this stage, it all seemed complicated. Because of this, I used the following methods to gain some more insight:
• Network traffic analysis
• Static analysis
• Log file analysis

After a while, I was able to see a bigger picture. It seemed that the Avalanche Web Application is not able to perform tasks by itself, not even a login operation. However, it can freely use different services to perform tasks. The Information Router, also known as the InfoRail service, stands in between, and is responsible for the distribution of messages between the services. The InfoRail service listens on TCP port 0.0.0.0:7225 by default.

As can be seen in the above diagram, the communication flow is straightforward. The Web Backend creates a message and sends it to the InfoRail service, which forwards it to the appropriate destination service. The destination service handles the message and returns the response to the Web Backend, once more via the InfoRail service. In the default installation, all these services are running on the same host. However, it is possible to place some of these services on remote machines.

On the other hand, it is not easy to get more detailed information about the messages. Network traffic analysis was not of much use, as the messages seem to be obfuscated or encrypted. In order to learn something more about them, I had to conduct a detailed code analysis.

The following section presents an overview of the InfoRail protocol and all the basics needed to:
• Recreate the protocol and messages
• Understand its basics
• Understand the payload delivery mechanism

InfoRail Protocol Basics

A typical InfoRail message consists of three main parts:
• Preamble
• Header
• Optional XML payload

The following picture presents a message structure:

The InfoRail Preamble

The preamble always has a length of 24 bytes. It consists of the following parts:
• Bytes 1-4: Length of the whole message
• Bytes 5-8: Length of the header
• Bytes 9-12: Length of the payload
• Bytes 13-16: Length of the uncompressed payload (payload can be optionally compressed)
• Bytes 17-20: Message ID (typically an incremented number, starting from 1)
• Byte 21: Protocol version (typically 0x10)
• Bytes 22-23: Reserved • Byte 24: Encryption flag (payload and header can be optionally encrypted) – 0 or 1

The InfoRail Header

The header of the typical message consists of multiple keys together with their corresponding values. Those keys and values are included in the header in the following way:
• 3 null bytes
• 0x02 byte
• 3 null bytes
• 1 byte which provides the length of a key (e.g. 0x08)
• 3 null bytes
• 1 byte which provides the length of a value (e.g. 0x06)
• Key + value

Here’s an example key and value:

         \x00\x00\x00\x02\x00\x00\x00\x08\x00\x00\x00\x06h.msgcat999999

As was mentioned before, a header can contain multiple keys. The most important ones that appear in most or all messages are:
• h.msgcat - in typical requests, it is equal to 10 (request)
• h.msgsubcat - information about the request type, thus it is crucial for the payload processing
• h.msgtag - usually stores the JSESSIONID cookie, although no method that verifies this cookie was spotted, thus this value can be random.
• h.distlist - specifies one or more services to which the message will be forwarded by InfoRail

The header must be padded with null bytes to a multiple of 16 bytes, due to the optional encryption that can be applied.

The Payload

As was mentioned before, the majority of messages contain an XML payload. An XStream serializer is used for the payload serialization and deserialization operations.

Different serialized objects can be transferred depending on the target service and the message subcategory. However, in most cases we are dealing with the RequestPayload object . The following snippet presents an example of the XML that is sent during an authentication operation. In this case, Avalanche Web Backend sends this XML to the Enterprise Server service, which then verifies user credentials.

RequestPayload.xml

As you can see, this message contains a serialized UserCredentials object, which consists of the loginName, encrypted password, domain, and clientIpAddress. You can also see that the RequestPayload contains the web cookie (sessionId). However, this cookie is never verified, so it can be set to any MD5 hash. As the payload can be optionally encrypted (and compressed), it must be padded to a multiple of 16 bytes.

Typically, every service implements its own ObjectGraph class, which defines the instance of the XStream serializer and configures it. Here’s a fragment of an example implementation:

ObjectGraphPart.java

When the service receives the XML payload, it deserializes it with the ObjectGraph.fromXML method.

Encryption

As was mentioned before, the header and the payload of the message can be encrypted. Furthermore, the payload can be compressed. InfoRail does not use the SSL/TLS protocol for encryption. Instead, it manually applies an encryption algorithm using a hardcoded encryption key.

As both the encryption algorithm and the encryption key can be extracted from the source code, the potential attacker can perform both the decryption and encryption operations without any obstacles.

Distribution to Services

The InfoRail service needs to know where to forward the received message. This information is stored in the h.distlist key of the message header. Values that can be used are stored in the IrConstants class. Variables that start with IR_TOPIC define the services where the message can be forwarded. The following code snippet presents several hardcoded variables:

DistributionVariables.java

If the sender wants the message to be forwarded to the license server, the h.distlist value must be set to 255.3.2.8.

Message Processing and Subcategories

This is probably the most important part of the protocol description. As was mentioned before, the message category is included in the message header under the h.msgsubcat key. As seen in the last line of the code snippet below, services protect themselves against the handling of an unknown message and, if the provided message subcategory is not implemented, the message is dropped.

Message processing may be implemented in slightly different ways depending on the service. Here, we are going to take a brief look at the Avalanche Notification Server. The following snippet represents the processMessage method found in the MessageDispatcher class:

processMessage.java

At [1], the message subcategory is retrieved from the header. Then, it is passed to the MessageProcessorVector.getProcessor method to obtain a reference to the appropriate message handler

At [2], the switch-case statement begins.

If the category is equal to 10 (request), the processInfoRailRequest method is invoked at [3]. Its arguments contain the handler that was retrieved at step [1]. We can have a quick look at this method here:

processInfoRailRequest.java

Basically, if the handler is not null, the code will invoke the handler.ProcessMessage method.

Finally, we must investigate how handlers are retrieved. The most important part is as follows:

MessageProcessorVector.java

At [1], a HashMap<Integer, IMessageProcessor> is defined.

At [2] and subsequent lines, the code invokes the setProcessor method. It accepts the message subcategory integer and the corresponding object that implements IMessageProcessor, such as AnsCredentialsHandler.

At [3], the setProcessor method is defined. It inserts the subcategory and appropriate object into the HashMap defined at [1].

At [4], getProcessor retrieves the handler based on the provided subcategory.

To summarize, there is a HashMap that stores the message subcategories and their corresponding handler objects. If we send a message to the Avalanche Notification Server with the subtype equal to 3706, the AnsTestHandler.processMessage method will be invoked.

Let’s have a look at one example of a message handler, AnsTestHandler:.

AnsTestHandler.java

At [1], the SUBCATEGORY variable is defined. It is common for the message processors to define such a variable.

At [2], the processMessage method is defined.

At [3], it retrieves the XML payload from the message.

At [4], it deserializes the payload with the ObjectGraph.fromXML method. It then casts it to AnsTestPayload, although the casting occurs after the deserialization. According to that, if the ObjectGraph does not implement any additional protections (such as an allowlist), we should be able to do harm here.

Success! We have identified the message processing routine. Moreover, we were able to quickly map the subcategory to the corresponding fragment of code that will handle our message. It seems that right now, we should be able to craft our own message.

Is There Any Authentication?

At this point, it seemed that I had everything necessary to send my own message. However, when I sent one, I saw no reaction from the targeted service. I looked at the InfoRail service log file and spotted these interesting lines:

It seems that I had missed an important part: authentication. Having all the details regarding the protocol and encryption, I was able to quickly identify a registration message in the network traffic. It is one of the rare examples where the payload is not stored in the XML form. The following screenshot presents a fragment of a registration message:

Several important things can be spotted here:
         -- reg.appident – specifies the name of the service that tries to register.
         -- reg.uname / reg.puname – specifies something that looks like a username.
         -- reg.cred / reg.pcred – specifies something that looks like a hashed password.

After a good deal of code analysis, I determined the following:
         -- Uname and puname are partially random.
         -- Cred and pcred values are MD5 hashes, based on the following values:
                  -- Username (.anonymous.0.digits).
                  -- Appropriate fragment of the key, which is hardcoded in the source code.

Once more, the only secret that is needed is visible in the source code. The attacker can retrieve the key and construct his own, valid registration message.

Finally, we can properly register with the InfoRail service and send our own messages to any of the Avalanche services.

Just Give Me Those Pwns Please – Code Execution on Four Services

At this stage, one can verify the services that do not implement a allowlist in the ObjectGraph class. I identified 5 of them:

- Data Repository Service (ZDI-CAN-15169).
- StatServer Service (ZDI-CAN-15130).
- Notification Server Service (ZDI-CAN-15448).
- Certificate Management Server Service (ZDI-CAN-15449).
- Web File Server Service (ZDI-CAN-15330).

We have five XStream deserialization endpoints that will deserialize anything we provide. One can immediately begin to consider ways to exploit this deserialization. For starters, XStream is very transparent about its security. Their security page (available here) presents multiple gadgets based on classes available in the Java runtime. Sadly, no suitable gadget worked for the first four services that I tried to exploit, as the required classes were not loaded.

Going further, one can have a look at Moritz Bechler’s “Java Unmarshaller Security” paper:

“XStream tries to permit as many object graphs as possible – the default converters are pretty much Java Serialization on steroids. Except for the call to the first non-serializable parent constructor, it seems that everything that can be achieved by Java Serialization can be with XStream – including proxy construction.”

 Proxy construction does not seem to be a thing in the newer versions of XStream. However, we should still be able to exploit it with ysoserial gadgets. Ysoserial gadgets for XStream were not available back then, so I created several myself. They can be found in this GitHub repository.

 Using my ysoserial XStream gadgets, I was successful in getting remode code execution in four Avalanche services. Here is a summary of the services I was able to exploit, and the required gadgets:

- StatServer: Exploited using AspectJWeaver and CommonsBeanutils1
- Data Repository: Exploited using C3P0 and CommonsBeanutils1
- Certificate Management Server: Exploited using CommonsBeanutils1
- Avalanche Notification Server: Exploited using CommonsBeanutils1

For no particular reason, let’s focus on the StatServer. To begin, we must find the topic and subcategory of some message that will deserialize the included XML payload. According to that, the InfoRail protocol message headers should contain the following keys and values:
• h.distlist = 255.3.2.12
• h.msgsubcat = 3502 (GetMuCellTowerData message)

In this example, we will make use of the AspectJWeaver gadget, which allows us to upload files. Below is the AspectJWeaver gadget for the XStream:

AspectJWeaver.xml

Several things can be highlighted in this gadget:
• The iConstant tag contains the Base64 file content.
• The folder tag contains a path to the upload directory. I am targeting the Avalanche Web Application root directory here.
• The key tag specifies the name of the file.

Having all the data needed, we can start the exploitation process:

The following screenshot presents the uploaded web shell and the execution of the whoami command.

Success! To sum up this part, an attacker who can send messages to Avalanche services can abuse the XStream deserialization in 4 different services.

I also got remote code execution on a fifth serviceHowever, exploiting this one was way trickier.

Exploiting JNDI Lookups Before It Was Cool

Java Naming and Directory Interface (JNDI) Lookups have a long history, and a lot of researchers were familiar with this vector long before the Log4Shell vulnerability. One of the proofs for that is CVE-2021-39146 – an XStream deserialization gadget that triggers a lookup. It has occurred to be the only XStream gadget that worked against the Web File Server service, for which I was not able to craft a valid ysoserial gadget.

Still, we are dealing with a fresh Java version. Thus, we are not able to abuse remote class loading. Moreover, the attacker is not able to abuse the LDAP deserialization vector (described here [PDF]). Using the JNDI injection, we can deliver the serialized payload, which will be then deserialized by our target. However, we are not aware of any deserialization gadgets that could be abused in the Web File Server.  If there were any gadgets, we would not need the JNDI lookup in the first place. Luckily, there are several interesting JAR packages included in the Web File Server Classpath.

As you can see, the Web File Server loads several Tomcat JAR packages. You might perhaps also be familiar with the great technique discovered by Michael Stepankin, which abuses unsafe reflection in Tomcat BeanFactory to execute arbitrary commands through JNDI Lookup. The original description can be found here.

In summary, we can execute the following attack:

1. Set up malicious LDAP server, which will serve the malicious BeanFactory. We will use the Rogue JNDI tool.
2. Register with the InfoRail service.
3. Send the message containing the CVE-2021-39146 gadget, targeting the Web File Server pointing to the server defined in step 1.
4. The Web File Server makes an LDAP lookup and retrieves data from the malicious server.
5. Remote code execution.

The following screenshot presents the setup of the LDAP server:

The next snippet presents the CVE-2021-39146 gadget, which was used for this Proof of Concept:

jndiGadget.xml

If everything goes well and the lookup was performed successfully, Rogue JNDI should show the following message and the code should be executed on the targeted system.

To summarize, we were able to abuse the custom Ivanti Avalanche protocol and XML message deserialization mechanisms to exploit five different services and get remote code execution with SYSTEM privileges. I have found more deserialization vulnerabilities in Ivanti Avalanche, but I am leaving them for the second part of the blog post. Right now, I would like to stay with the protocol and inter-services communication.

Abusing a Race Condition in the Communication and Authentication Messages

As detailed above, the various Avalanche services communicate with each other with the help of InfoRail. When a service provides a response, the response is again forwarded via the InfoRail service. The idea for this research was: is it possible for an attacker to spoof a response? If so, it may be possible to abuse Ivanti Avalanche behavior and perform potentially malicious actions.

I focused on the authentication operation, which is presented in the following scheme (read from top to bottom):

When a user authenticates through the Avalanche Web Application login panel, the backend transfers the authentication message to the Enterprise Server. This service verifies credentials and sends back an appropriate response. If the provided credentials were correct, the user gets authenticated.

During this research, I learned two important things:
• The attacker can register as any service.
• The authentication message is distributed amongst every instance of the registered Enterprise Server.

According to that, the attacker can register himself as the Enterprise Server and intercept the incoming authentication messages. However, this behavior does not have much of an immediate consequence, as the transferred password is hashed and encrypted.

The next question was whether it is possible to deliver the attacker’s own response to the Avalanche Web and whether it will be accepted or not. It turns out that yes, it is possible! If you would like to deliver your own response, you must properly set two values in the message header:
• Origin – The topic (ID) of the Avalanche Web backend that sent the message.
• MsgId – The message ID of the original authentication message.

Both values are relatively easy to get, and thus the attacker can provide his own response to the message. It will be accepted by the service, which is waiting for the response. An example attack scenario is presented in the following figure (read from top to bottom ).

The attack scenario works as follows:
         -- The attacker tries to log into the Web Application with the wrong credentials.
         -- Web Application sends the authentication message.
         --The InfoRail service sends the message to both Enterprise Servers: the legitimate one and the malicious one.
         --Race time:
                  --The legitimate server responds with the “wrong credentials” message.
                  --The malicious server responds with the “credentials OK” message.          --If the attacker’s server was first to deliver the message, it is forwarded to the Avalanche Web Application.
         --The attacker gets authenticated.

Please note that the “Login message” targeting the attacker’s server is not present here on purpose (although it is in reality transmitted to the attacker). I wanted to highlight the fact, that this issue can be exploited without a possibility to read messages. In such a case, the attacker has to brute-force the already mentioned message ID value. It complicates the whole attack, but the exploitation is still possible.

To summarize this part, the attacker can set up his own malicious Enterprise Server and abuse a race condition to deliver his own authentication response to the Web Application. There are two more things to investigate: what does the response message look like and are we able to win the race?

Authentication Handling

The following snippet presents an example response to a login message. Please note that those messages are way bigger during legitimate usage. However, for the Proof of Concept, I have minimized them and stored only those parts that are needed for exploitation.

The response message consists of several important parts:
• It contains a responseObject tag, which is a serialized User object.
• It contains a responseCode tag, which will be important.

At some point during authentication, the Web Backend invokes the UserService.doLogin method:

doLogin.java

At [1], the UserCredentials object is instantiated. Then, its members are set.

At [2], the authenticate method is called, and the object initialized at [1] is passed as an argument:

authenticate.java

At [1], the UserLogin object is initialized.

At [2], the UserCredentials object is serialized.

At [3], the message is being sent to the Enterprise Server, and the Web Backend waits for the response.

At [4], the responseCode included in the response is verified. We want it to be equal to 0.

At [5], the userLogin.authenticated is set to True.

At [6], the userLogin.currentUser is set to the object that was included in the responseObject.

At [7], the method returns the userLogin object.

Basically, the response should have a responseCode equal to 0. It should also include a properly serialized User object in the responseObject tag.

Finally, we analyze the fragment of UserBean.loginInner method that was responsible for the invocation of the doLogin function.

loginInner.java

At [1], the doLogin method is invoked. It retrieves the object of UserLogin type (see the returns of previous code snippets).

At [2], it sets this.currentUser to the userLogin.currentUser (the one obtained from the response).

At [3], it sets various other settings. These will not be relevant for us.

One very important thing to note: the Web Backend does not compare the username provided in the login panel and the username retrieved by the Enterprise Server. Accordingly, the attacker can:
• Trigger the authentication with the username “poc”.
• Win the race and provide the user “amcadmin” in the response.
• The attacker will then be authenticated as the “amcadmin”.

To summarize, it seems that there are no obstacles for the attacker and his response should be handled by the Web Backend without any problems. Next, we turn our focus to winning the race.

Winning the Race

In a default installation, the Enterprise Server and InfoRail services are located on the same host as the Web Backend. This makes the exploitation of the race condition harder, as the legitimate communication is handled by the local interface, which is much faster than communication through an external network interface.

Still, the attacker has some advantages. For example, he does not have to generate dynamic responses, as the response payload can be hardcoded in the exploit. The following table presents a general overview of actions that must be performed by both the attacker and the legitimate Enterprise Server.

The remote attacker needs to perform far fewer operations and can prepare his response more rapidly. It makes the race winnable. One may ask – what if the attacker’s position in the network too disadvantageous and he still has no chance to win the race? Even then, there is an effective solution.

The unauthenticated attacker could modify the Avalanche System Settings. This is due to a separate vulnerability that allows to bypassing domain authentication (ZDI-CAN-15919). The remote attacker could enable the LDAP-based authentication and set any address for the LDAP server configuration. In such a scenario, the legitimate Enterprise Server will first try to access this “rogue” authentication server. This will give the attacker an additional second or two (or even more if played properly). In this way, an attacker can gain more time to win the race, should this be necessary

Exploitation

Having all the pieces, the race condition vulnerability can be exploited. The following video shows an example of the attacker providing wrong credentials via the panel but was still successfully authenticating as the “amcadmin” user.

Conclusion

When you don’t have ideas for further research, make sure that you have discovered the complete attack surface of your recent targets. When you find a spot that was not already studied by others, you might find a path to discovering a whole new set of exciting vulnerabilities.

Keep an eye out for the second part of this blog, where I will share some details about my favorite vulnerabilities in the Ivanti Avalanche services. It will cover one more deserialization vulnerability, cool chains leading to the remote code execution, and a pretty rare information disclosure vulnerability.

Until then, you can follow me @chudypb  and the team @thezdi on Twitter for the latest in exploit techniques and security patches.