Intro

When analyzing the security of mobile applications, it’s important to verify that all data is stored securely (See OWASP MASVS-STORAGE-1). A recent engagement involved a Flutter app that uses the Isar/Hive framework to store data. The engagement was unfortunately blackbox, so we did not have access to any of the source code. This especially makes the assessment more difficult, as Flutter is pretty difficult to decompile, and tools like Doldrums or reFlutter only work for very specific (and old) versions. Frida can be used (see e.g. Intercepting Flutter traffic)

The files we extracted from the app were encrypted and we needed to figure out what kind of data was stored. For example, storing the password of the user (even if it’s encrypted) would be an issue, as the password can for example be extracted using a device backup.

In order to figure out how the data is encrypted, we needed to analyze the Hive framework and find some way to extract that data in cleartext. Hive is a “Lightweight and blazing fast key-value database written in pure Dart.” which means we can’t easily monitor what is stored inside of the databases using Frida. There also isn’t a publicly available Hive viewer that we could find, and there’s a probably good reason for that, as we will see.

The goal of this blogpost is to obtain the content of an encrypted Hive without having access to the source code. This means we will:

• Create a Flutter test app to get some useful Hives
• Understand the internals of the Hive framework
• Create a generic Hive reader that works on encrypted Hives containing custom objects
• Obtain the password of the encrypted Hive
• (Bonus) Recover deleted items

Let’s start!

Isar / Hive

Hive is a key-value framework built on top of Isar, which is a no-sql library for Flutter applications. It is possible to store all the simple Dart types, but also more complex types like a List or Map, or even custom objects via custom TypeAdapters. The project is currently in a transition phase to v4 so the focus is on v2.2.3, which is what the target application was most likely using.

While Hive is the name of the framework, what we are actually interested in are boxes. Boxes are the actual files that are stored on the system and each box contains one or more data frames. A data frame simply holds one key-value pair.

Each box is either plaintext or encrypted. The encryption is based on AES-256, which means you need a 256-bit key to open an encrypted box. The storage of this key is not the responsibility of Hive, and the documentation suggests to store your key using the flutter_secure_storage plugin. This is interesting, as the flutter_secure_storage plugin does use the system credential storage of the device to store data, so we can potentially intercept the key when it is being retrieved using Frida.

Keys are not encrypted!
One very important thing to realize is that an encrypted box is not actually fully encrypted. For each key-value pair that is stored, only the value is stored encrypted, while the key is stored in plaintext. This is mentioned in the documentation, but it’s easy to miss it. Now this is generally not a big deal, except of course if sensitive data is being used as the key (e.g. a user ID).

Creating a small test app

Let’s create a small Flutter application that uses Hive and saves some data into a box. The code for this was mostly generated by poking Chat-GPT so that we could spend our time reverse-engineering (and fighting XCode). For simplicity’s sake, I’m deploying to macOS so that the boxes are stored directly on the system and we can easily analyze them.

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  final dir = await getApplicationDocumentsDirectory();
  Hive.init(dir.path);
  await createBox();
  runApp(MaterialApp(home:MyApp()));
}

void createBox() async {
  // Storing examples of each supported datatype
  var box = await Hive.openBox('basicBox');
  box.put('myInt', 123); // int
  box.put('myDouble', 123.456); // double
  box.put(0x22, true); // bool
  box.put('myString', 'Hello Hive'); // String
  box.put('myBytes', Uint8List.fromList([68, 97, 114, 116])); // List<int>
  box.put('myList', [1, 2, 3]); // List<dynamic>
  box.put('myMap', {'name': 'Hive', 'isCool': true}); // Map<dynamic, dynamic>
  box.put('myDateTime', DateTime.now()); // DateTime
}

And our dependencies:

dependencies:
  flutter:
    sdk: flutter

  hive: ^2.2.3
  hive_flutter: ^1.1.0
  isar_flutter_libs: ^3.1.0+1
  path_provider: ^2.1.2
  file_picker: ^6.1.1
  path: ^1.9.0

dev_dependencies:
  flutter_test:
    sdk: flutter

  hive_generator: ^1.1.0 
  build_runner: ^2.0.1

When we run the application, it creates a new .hive file, which is in fact a box containing all of our key-value pairs:

Hive internals

A box contains multiple frames, and each frame is responsible for indicating how long it is. There is no global index of the frame offsets, which means that we can’t jump directly to a specific frame and we have to parse all the frames one by one until we have parsed all frames. Each frame consists of a key (either a string or an index) and a value, which can be any default or custom type:

If the key is a String (frames 1 and 2), it has type 0x01, followed by the length and then the actual ASCII value. If the key is an int (frame 3), the encoding is slightly different. The type is 0x00 and the key is encoded as a uInt32. The key will be an int if you specify an int as the key (e.g. myBox.put(0x22, true)) or if you use the autoIncrement feature (myBox.add("test")).

If we run the application a second time, Hive will open the box from the filesystem (based on the name of the box) and load all the current values. When the put instructions are executed again, Hive doesn’t overwrite the frame belonging to the given key (as that would require the entire file to be shifted based on the new lengths, a very intensive operation), but rather it appends a new frame with the new value. As a result, running the code twice will double the size of the box. When the box is read, all frames are parsed sequentially and all key-value pairs simply overwrite any previously loaded information.

For example, let’s take the following box:

var emptyBox = await Hive.openBox("emptyBox");
emptyBox.put("mySecret", "Don't tell anyone");
emptyBox.delete("mySecret");

The created box still contains the secret if you look at the binary content:

The delete frame is simply a frame with a key and no value.

Custom types

In addition to storing normal Dart types in a box, it is possible to store custom types as long as Hive knows how to serialize/deserialize them. Let’s look at a quick example with a custom Bee class:

import 'package:hive/hive.dart';

part 'BeeModel.g.dart';

@HiveType(typeId: 1)
class Bee extends HiveObject{
  @HiveField(0)
  final String name;
  @HiveField(1)
  final int age;
  Bee({
    required this.name, 
    required this.age,
  });
}

The Bee class extends HiveObject and defines two string properties. It’s not technically necessary to extend the HiveObject class, but it makes things easier. We’ve also added annotations so that we can use the hive_generator package to automatically generate a serializer by running dart run build_runner build:

This will generate a new class called BeeModel.g.dart which takes care of serializing/deserializing:

// GENERATED CODE - DO NOT MODIFY BY HAND

part of 'BeeModel.dart';

// **************************************************************************
// TypeAdapterGenerator
// **************************************************************************

class BeeAdapter extends TypeAdapter<Bee> {
  @override
  final int typeId = 1;

  @override
  Bee read(BinaryReader reader) {
    final numOfFields = reader.readByte();
    final fields = <int, dynamic>{
      for (int i = 0; i < numOfFields; i++) reader.readByte(): reader.read(),
    };
    return Bee(
      name: fields[0] as String,
      age: fields[1] as int,
    );
  }

  @override
  void write(BinaryWriter writer, Bee obj) {
    writer
      ..writeByte(2)
      ..writeByte(0)
      ..write(obj.name)
      ..writeByte(1)
      ..write(obj.age);
  }

  @override
  int get hashCode => typeId.hashCode;

  @override
  bool operator ==(Object other) =>
      identical(this, other) ||
      other is BeeAdapter &&
          runtimeType == other.runtimeType &&
          typeId == other.typeId;
}

We can see that the serialization format is pretty straightforward: it first writes the number of fields (2), followed by an index-value pair which correspond to the HiveField annotations. The write() function is the function that is used during a normal put() operation, so the fields will have the same structure as seen earlier.

Finally, to be able to use this new type, the Adapter needs to be registered:

  Hive.registerAdapter(BeeAdapter());
  var beeBox = await Hive.openBox("beeBox");
  beeBox.put("myBee", Bee(name: "Barry", age: 1));

After running this code, the beeBox is generated, containing one frame:

Decoding unknown types

Let’s now assume that we have access to a box with unknown types. We can still load it, as long as we can figure out a suitable deserializer. It’s not a far stretch to assume that the developer has used the automatically generated adapter, so let’s focus on that. If they haven’t, you’ll have to dive into Ghidra and start disassembling the Hive deserialization, or make some educated guesses based on the hexdump.

We can take the BeeAdapter as a starting point, but rather than creating Bee objects, let’s create a generic List object in which we can store all the deserialized values. Luckily a List can contain any type of data in Dart, so we don’t have to worry about the actual types of the different fields. Additionally, we want to make the typeId dynamic since we want to register all of the possible custom typeIds.

The following GenericAdapter does exactly that:

import 'package:hive/hive.dart';

class GenericAdapter extends TypeAdapter<List> {
  @override
  final int typeId;

  GenericAdapter(this.typeId);

  @override
  List read(BinaryReader reader) {
    final numOfFields = reader.readByte();
    var list = List<dynamic>.filled(numOfFields, null, growable: true);

    for (var i = 0; i < numOfFields; i++) {
      list[reader.readByte()] = reader.read();
    }
    return list;
  }

  @override
  int get hashCode => typeId.hashCode;
  
  @override
  void write(BinaryWriter writer, List obj) {
    // No write needed
  }
}

We can then register this GenericAdapter for all the available custom typeIds (0 > 223) and read the beeBox we created earlier without needing the Bee or BeeAdapter class:

for(var i = 0; i<223; i++)
{
   Hive.registerAdapter(GenericAdapter(i));
}
var beeBox = await Hive.openBox("beeBox");
List myBee = beeBox.get("myBee");
print(myBee.toString()); // prints [Barry, 1]

Encrypted hives

As mentioned earlier, it’s possible to encrypt boxes. Let’s see how this changes the internals of the box:

final encryptionKey = Hive.generateSecureKey();
final encryptedBox = await Hive.openBox('encryptedBox', 
                          encryptionCipher: HiveAesCipher(encryptionKey));
encryptedBox.put("myString", "Hello World");
encryptedBox.close();

The code above generates the box below:

As explained earlier, the key is not encrypted, but the value is. The encryption covers all the bytes between the KEY and the CRC code. There is no special format for indicating an encrypted value, but Hive knows that it needs to decrypt the data due to the encryptionCipher parameter while opening the box. When the frames are read, the value is decrypted and parsed according to the normal deserialization logic. This means that we can use our GenericAdapter for encrypted boxes too, as long as we have the password.

Obtaining the password

Potentially the most tricky part, as this can be very easy, or very difficult. In general, there are a few different options:

1. Intercept the password when it is loaded from storage
2. Intercept the password when the box is opened
3. Extract the password from storage

The first option is only possible if the password is actually stored somewhere (rather than being hardcoded). In the official Hive documentation, the developer recommends to use the flutter_secure_storage plugin, which will use either the KeyStore (Android) or KeyChain (iOS).

On Android, we can hook the Java code to intercept the password when it is loaded from the encrypted shared preferences. For example, there is the FlutterSecureStorage.read function which returns the value for a given key. By default, flutter optimizes the application in release mode, which means we can’t directly hook into FlutterSecureStorage.read because the class and method name will be stripped. It takes a little bit of effort to find the correct method, but the hook is straightforward:

Java.perform(() => {
    // Replace with correct class and method
    let a = Java.use("c0.a");
    a["l"].implementation = function (str) {
        console.log(`a.l is called: str=${str}`);
        let result = this["l"](str);
        console.log(`a.l result=${result}`);
        return result;
    };
});

Running this with Frida will print the base64 encoded password:

On iOS, the flutter_secure_storage plugin has moved to Swift, so intercepting the call is not straightforward. We do know, however, that the flutter_secure_storage plugin uses the KeyChain, and it does so without any additional encryption. This means we can obtain the password by dumping the keychain with objection‘s ios dump keychain command:

In case these options don’t work, you’ll probably want to dive into Ghidra and start reverse-engineering the app.

Recovering deleted items

We now have the password and a generic parser, so we can extract the items from the Hive. Unfortunately, if we use the normal API, we will only see the latest version of each item, or nothing at all in case there are delete frames. We could modify the Hive source code to notify us whenever a Frame is loaded (and there is actually some unreachable debugging code available that does just that), but it would be nicer to have a solution that doesn’t require a custom version of the library.

The way that Hive makes sure that only the latest version of an item is available is by adding each frame to a dictionary based on the frame’s key. Newer frames automatically overwrite older frames, so only the final value is kept. To make sure values don’t get overwritten, let’s just make sure that each frame key is unique by changing it if the key has already been used. Similarly, if we rename delete frames, they will not overwrite the old value either.

When we rename the key of a frame, we need to update the size of the frame and update the CRC32 checksum at the end so that Hive can still load the modified box. The following code copies a given box to a temporary location and updates all the frames to have unique names. It uses the Crc32 class which was copied from the source of the Hive framework so that we can be sure the logic is consistent:

Future<File> recoverHive(originalFile, HiveAesCipher? cipher) async {
  var filePath = await copyFileToTemp(originalFile);
  var file = File(filePath);
  var bytes = await file.readAsBytes();
  int offset = 0;
  var allFrames = BytesBuilder();
  var keyNames = <String, int>{};
  var keyInts = [];

  while (offset < bytes.length) {
    var frameLength = ByteData.sublistView(bytes, offset, offset + 4)
                              .getUint32(0, Endian.little);
    var keyOffset = offset + 4; // Skip frame length
    var endOffset = offset + frameLength;
    if (bytes.length > keyOffset + 2) {

      Uint8List newKey;
      int frameResize;
      int keyLength;
      if(bytes[keyOffset] == 0x01){
        // Key is String
        keyLength = bytes[keyOffset + 1];
        var keyBytes = bytes.sublist(keyOffset + 2, keyOffset + 2 + keyLength);
        var keyName = String.fromCharCodes(keyBytes);

         if (keyNames.containsKey(keyName)) {
            keyNames[keyName] = keyNames[keyName]! + 1;
            keyName = "${keyName}_${keyNames[keyName]}";
          } else {
            keyNames[keyName] = 1;
          }
          var modifiedKeyBytes = Uint8List.fromList(keyName.codeUnits);
          var modifiedKeyLength = modifiedKeyBytes.length;

          // get bytes for TYPE + LENGTH + VALUE
          var bb = BytesBuilder();
          bb.addByte(0x01);
          bb.addByte(modifiedKeyLength);
          bb.add(modifiedKeyBytes);
          newKey = bb.toBytes();
          frameResize = modifiedKeyLength - keyLength;
          keyLength += 2; // add the length of the type
      }
      else{
        // Key is int
        keyLength = 5; // type + uint32
        var keyIndexOffset = keyOffset + 0x01;
        var keyInt = ByteData.sublistView(bytes, keyIndexOffset, keyIndexOffset + 4)
                              .getUint32(0, Endian.little);

        while(keyInts.contains(keyInt)){
          keyInt += 1;
        }
        keyInts.add(keyInt);

        var index = ByteData(4)..setUint32(0, keyInt, Endian.little);
        
        // get bytes for TYPE + index
        var bb = BytesBuilder();
        bb.addByte(0x00);
        bb.add(index.buffer.asUint8List());
        newKey = bb.toBytes();
        frameResize = 0;
      }

      // If there is no value, it's a delete frame, so we don't add it again
      if(frameLength == keyLength + 8){ // 4 bytes CRC, 4 bytes frame length
        offset = endOffset;
        print("Dropping delete frame for " + newKey.toString());
        continue;
      }
      
      // Calculate new length of frame
      frameLength += frameResize;

      // Create a new frame bytes builder
      var frameBytes = BytesBuilder();
      
      // Prepare the frame length in ByteData and add it to the frame
      var frameLengthData = ByteData(4)..setUint32(0, frameLength, Endian.little);
      frameBytes.add(frameLengthData.buffer.asUint8List());

      // Add the new key
      frameBytes.add(newKey);
      
      // Add the rest of the frame after the original key. Don't include the CRC
      frameBytes.add(bytes.sublist(keyOffset + keyLength, endOffset-4));

      // Compute CRC using Hive's Crc32 class
      var newCrc = Crc32.compute(
        frameBytes.toBytes(),
        offset: 0,
        length: frameLength - 4,
        crc: cipher?.calculateKeyCrc() ?? 0,
      );

      // Write Crc code
      var newCrcBytes = Uint8List(4)..buffer.asByteData()
                                            .setUint32(0, newCrc, Endian.little);
      frameBytes.add(newCrcBytes);

      // Update the overall frames with the modified frame
      allFrames.add(frameBytes.toBytes());
    }

    offset = endOffset; // Move to the next frame
  }

  var reconstructedBytes = allFrames.takeBytes();

  try {
    await file.writeAsBytes(reconstructedBytes);
    print('Bytes successfully written to temporary file: ${file.path}');
  } catch (e) {
    print('Failed to write bytes to temporary file: $e');
  }
  return file;
}
Future<String> copyFileToTemp(String sourcePath) async {
  var sourceFile = File(sourcePath);
  // Generate a random subfolder name
  var rng = Random();
  var tempSubfolderName = "temp_${rng.nextInt(10000)}"; // Random subfolder name
  var tempDir = Directory.systemTemp.createTempSync(tempSubfolderName);
  
  // Create a File instance for the destination file in the new subfolder
  var tempFile = File('${tempDir.path}/${sourceFile.uri.pathSegments.last}');

  try {
    await sourceFile.copy(tempFile.path);
    print('File copied successfully to temporary directory: ${tempFile.path}');
  } catch (e) {
    print('Failed to copy file to temporary directory: $e');
  }
  return tempFile.path;
}

Putting it all together

Now that we can recover deleted items, read encrypted vaults and view custom objects, let’s put it all together. The target vault is created as follows:

final ultimateBox = await Hive.openBox('ultimateBox', 
                                    encryptionCipher: HiveAesCipher(hiveKey));
ultimateBox.add(123);
ultimateBox.add(456);
ultimateBox.deleteAt(1);
ultimateBox.put("myString", "Hello World");
ultimateBox.put("anotherString", "String2");
ultimateBox.add("Something");
ultimateBox.delete("myString");
ultimateBox.add(Bee(age: 12, name: "Barry"));
ultimateBox.put("test", 99999);
ultimateBox.put("anotherString", 200);

Reading is straightforward, we only need to specify the key and register the Generic adapter:

// Register the GenericAdapter for all available typeIds
for(var i = 0; i<223; i++)
{
  Hive.registerAdapter(GenericAdapter(i));
}
// Decode password and open box
var passwordBytes = base64.decode(password);
var encryptionCipher = HiveAesCipher(passwordBytes);
box = await Hive.openBox<dynamic>(boxName, path: directory, 
                                  encryptionCipher: encryptionCipher);

Finally, we can create a small UI around this functionality so that we can easily view all of the frames. In the screenshot below, we can see that the none of the data is deleted, and old values (anotherString > String 2) are still visible. The source code for this app can be found here.

Conclusion

It’s always faster to use an available library than create a solution yourself, but for security-critical applications it’s very important to fully understand the libraries you’re using. As we saw above, the Hive framework:

• Keeps old values in the box until it is compacted
• Only encrypts values, not keys

In this case, the documentation is clear on both facts, so it’s not really a security vulnerability. However, developers should be aware of the correct way to use the Hive framework in case any type of sensitive information is stored.

Finally, the fact that we don’t have access to the source code doesn’t stop us from identifying weaknesses, it just takes more time to reverse engineer the application/frameworks and develop custom tooling.

Jeroen Beckers

Jeroen Beckers is a mobile security expert working in the NVISO Software Security Assessment team. He is a SANS instructor and SANS lead author of the SEC575 course. Jeroen is also a co-author of OWASP Mobile Security Testing Guide (MSTG) and the OWASP Mobile Application Security Verification Standard (MASVS). He loves to both program and reverse engineer stuff.