We’d like to introduce you to the upcoming features for the next release of Sidekick! If you want to see some of the new features in action, check out the demonstration we gave during a recent Binary Ninja live stream. This post is the first in a series that will explain how it works and showcase its capabilities. To start the series, we want to first highlight what is unique about our approach and why Sidekick is more than just a thin wrapper around a Large Language Model (LLM).

Unless you’ve been living completely off the grid, you already know that there’s been a lot of integration of LLMs into software tools, both in general and specifically with reverse engineering tools. A common approach is using LLMs to improve decompilation (e.g, naming functions and variables) or to apply static prompts over batches of code. While these techniques are indeed valuable, they lack the flexibility and user-driven control needed to truly accelerate complex reverse engineering tasks. Enter Sidekick’s Orchestration Architecture, which introduces a unique approach that accelerates analysis tasks and puts the user firmly in the driver’s seat.

Sidekick’s Orchestration Architecture

At the core of our approach are two key layers:

1. Collaborative Script Creation Layer: This layer is responsible for translating a user’s task description into an orchestrated system. It combines the functions of script writing and prompt engineering to create a cohesive analysis workflow, all through natural language interaction with an AI assistant.
2. AI-Assisted Analysis Layer: This is where the actual analysis takes place, seamlessly integrating both traditional Python code execution and LLM-powered analysis through the innovative LLMOperator concept.

Sidekick’s Orchestration Architecture: Bridging Collaborative Script Creation and AI-Assisted Analysis in Binary Ninja.

The LLMOperator is a crucial innovation that allows us to seamlessly integrate LLM capabilities into the analysis process. It enables the decomposition and distribution of complex tasks across Python code and LLM models, achieving a level of sophistication that distinguishes Sidekick from simpler integrations.

With the combination of script creation and LLMOperators, complex processes for answering non-trivial analysis questions can be constructed, while maintaining simplicity and accessibility for the user through natural language dialog with an AI assistant. The result is a powerful framework that balances advanced capabilities with user-friendly interaction and control.

Translating Natural Language into Orchestrated Systems

The Collaborative Script Creation Layer of Sidekick’s Orchestration Architecture acts as a sophisticated “compiler” for natural language analysis task descriptions. This layer transforms user inputs into working, repeatable analysis scripts, bridging the gap between human intent and machine execution.

Key aspects of this translation process include:

1. Intent Recognition: The system interprets the user’s natural language description, identifying key analysis goals and requirements.
2. Script Generation: Based on the recognized intent, the system constructs a Python script that orchestrates the analysis. This script combines calls to the Binary Ninja API with strategically placed LLMOperators.
3. Iterative Refinement: Through an AI-mediated dialogue, users can review, modify, and expand upon the generated script, ensuring it precisely meets their analysis needs.
4. Abstraction of Complexity: While the resulting script may involve intricate logic and sophisticated LLM interactions, the user interface remains at the natural language level, maintaining accessibility.

This approach is particularly powerful when dealing with complex investigative questions about malware or vulnerable code. It can construct the sophisticated machinery necessary to answer these questions effectively, all while keeping the user experience straightforward and intuitive.

By “compiling” natural language into orchestrated systems, Sidekick enables reverse engineers to leverage advanced AI capabilities without requiring expertise in LLM programming or prompt engineering.

The LLMOperator: Seamlessly Deploying LLMs in AI-Assisted Analysis

Central to our Orchestration Architecture is the concept of the LLMOperator. This innovative feature allows for the seamless integration of LLMs into the AI-Assisted Analysis Layer, extending analysis capabilities beyond the Binary Ninja API. Here’s how it works:

1. As a Python class, the LLMOperator constructor takes a text description of the operation to be performed.
2. It examines the surrounding Python code to determine the input type, to contextualize the operation description, and to devise an appropriate output schema.
3. When called, it injects Binary Ninja API objects along with the relevant contextual information into the LLM prompt, calls the LLM, and returns the structured results.

The LLMOperator concept brings about a fundamental change in how analysis scripts can be written. It’s not just about running Python code anymore; it’s about liberally leveraging the power of LLMs at execution time, allowing for more sophisticated and adaptive analyses.

Crucially, users have direct control over the LLMOperator specifications. You can modify the prompts sent to the LLMs in the AI-Assisted Analysis Layer and choose which models are used. This fine-grained control ensures we don’t sacrifice user control for convenience.

By integrating LLMOperators into the analysis process, complex analytical tasks can be decomposed into a series of LLM interactions, all orchestrated within a Python script that users can understand and modify. This approach combines the power of AI with the flexibility and transparency that reverse engineers need for their work.

Python Scripts as User-Controlled Orchestrators with Flexible LLM Deployment

In the Collaborative Analysis Console, generated Python scripts act as powerful orchestrators, tying together various LLMOperators and traditional Python code into a cohesive analysis workflow. This approach offers several key advantages:

• Transparency: Users can see and understand the exact steps being taken in their analysis.
• Controlled Automation: The script provides a clear structure for automated analysis tasks, guided by user input.
• Customization: Users can modify both the script and the underlying LLMOperator specifications to fit their specific needs.
• Repeatability: Analyses can be easily repeated or adapted for similar tasks.

Beyond these benefits, Sidekick’s architecture optimizes LLM deployment within these scripts, bringing additional advantages:

• Specialization: Each LLMOperator can select the best model for its task from a collection of user-supplied LLMs. This allows for more accurate and efficient results compared to using a single, general-purpose model.
• Scalability: Users have the flexibility to run LLMs on local hardware or with bring-your-own-API-key proprietary models. This means that Sidekick can scale to meet the needs of different users, from individual researchers to large organizations with significant computational resources.
• Privacy: Using local LLMs for sensitive binary data, Sidekick addresses a critical concern in reverse engineering. Users can keep their data on-premises or in their preferred secure environments, reducing the risk of exposure.
• Future-proofing: As new LLMs become available, they can be easily incorporated into the AI-Assisted Analysis Layer. This ensures that users can always leverage the most advanced AI capabilities.

By framing our generated scripts as customizable “agent systems” under user control, and coupling this with flexible LLM deployment, we achieve a lot of power and adaptability.

Script Creation in the Collaborative Analysis Console

Sidekick’s Collaborative Analysis Console is where the power of our Orchestration Architecture comes to life. The console offers an iterative and AI-mediated approach to creating analysis scripts that leverage both the Binary Ninja API and LLMs for sophisticated reverse engineering tasks.

Sidekick’s Collaborative Analysis Console in action: Generating a script to find string decryption functions within Binary Ninja, showcasing the integration of AI-assisted analysis with traditional reverse engineering tools.

Interactive Script Generation

When you begin a new analysis task, you start by describing your objective in natural language. For example, you might say, “Find all functions that use AES encryption and identify their key sizes.” The Collaborative Script Creation Layer then translates this request into an initial Python script, leveraging appropriate LLMOperators and Binary Ninja API calls.

But the process doesn’t stop there. The Collaborative Analysis Console displays this generated script and allows you to review, run, modify, and refine it. This is where the true collaboration begins.

Let’s say you review the initial script and want to make some changes. Instead of manually editing the Python code, you can simply chat with the AI assistant:

“The script looks good, but can we also check for functions that might be using homebrew encryption? And let’s add some comments to explain the key size identification process.”

The assistant processes your request and updates the script accordingly, adding new LLMOperators to identify potential homebrew encryption and inserting explanatory comments. This back-and-forth can continue as many times as needed, allowing you to shape the analysis script to your exact situation.

Bridging Expertise Gaps

This collaborative approach to script creation bridges the gap between reverse engineering expertise and programming skills. It allows reverse engineers to leverage advanced analysis techniques without needing to be proficient in Python or familiar with the intricacies of the Binary Ninja API or being experts at prompt engineering or injecting essential context for LLMs. At the same time, it provides transparency and control, enabling users to understand and customize the analysis process to their specific needs.

By combining natural language interaction with AI-assisted script generation, the next version of Sidekick will let reverse engineers create sophisticated, tailored analysis workflows more efficiently than ever before.

Conclusion

In this post, we’ve introduced you to Sidekick’s innovative approach to AI-Augmented Reverse Engineering. From our unique Orchestration Architecture to the powerful LLMOperator concept and the collaborative script creation process, Sidekick represents a significant step forward in leveraging AI to enhance reverse engineering workflows.

We encourage you to check out the demonstration video linked at the beginning of this post to see these concepts in action. In future posts, we’ll dive deeper into how to use specific features of Sidekick and explore the inner workings of key components like the LLMOperator.

If you want to be the first to explore how Sidekick is pushing the boundaries of what’s possible in reverse engineering with AI assistance, sign up today.