realm_flutter_vector_db 1.0.2 copy "realm_flutter_vector_db: ^1.0.2" to clipboard
realm_flutter_vector_db: ^1.0.2 copied to clipboard

Realm Flutter SDK with built-in HNSW vector search - a mobile database alternative to SQLite with AI/ML vector similarity search capabilities for semantic search, RAG, and recommendation systems.

Warning

We announced the deprecation of Atlas Device Sync + Realm SDKs in September 2024. For more information please see:

For a version of realm-dart without sync features, install version 20 or see the community branch.

License Realm Dart CI Coverage Status

Realm is a mobile database that runs directly inside phones, tablets or wearables. This repository holds the source code for the Realm SDK for Flutter™ and Dart™.

Features #

  • Mobile-first: Realm is the first database built from the ground up to run directly inside phones, tablets, and wearables.
  • Simple: Realm's object-oriented data model is simple to learn, doesn't need an ORM, and the API lets you write less code to get apps up & running in minutes.
  • Modern: Realm supports latest Dart and Flutter versions and is built with sound null-safety.
  • Fast: Realm is faster than even raw SQLite on common operations while maintaining an extremely rich feature set.
  • Vector Search (HNSW): Built-in support for high-performance vector similarity search using Hierarchical Navigable Small World (HNSW) algorithm. Perfect for AI/ML applications, semantic search, recommendation systems, and RAG (Retrieval-Augmented Generation) patterns.
  • MongoDB Atlas Device Sync: Makes it simple to keep data in sync across users, devices, and your backend in real-time. Get started for free with a template application and create the cloud backend.

Getting Started #

  • Import Realm in a dart file app.dart

    import 'package:realm/realm.dart';  // import realm package
    
    part 'app.realm.dart'; // declare a part file.
    
    @RealmModel() // define a data model class named `_Car`.
    class _Car {
      late String make;
    
      late String model;
    
      int? kilometers = 500;
    }
    
  • Generate RealmObject class Car from data model class _Car.

    dart run realm generate
    
  • Open a Realm and add some objects.

    var config = Configuration.local([Car.schema]);
    var realm = Realm(config);
    
    var car = Car("Tesla", "Model Y", kilometers: 5);
    realm.write(() {
      realm.add(car);
    });
    
  • Query objects in Realm.

    var cars = realm.all<Car>();
    Car myCar = cars[0];
    print("My car is ${myCar.make} model ${myCar.model}");
    
    cars = realm.all<Car>().query("make == 'Tesla'");
    
  • Get stream of result changes for a query.

    final cars = realm.all<Car>().query(r'make == $0', ['Tesla']);
    cars.changes.listen((changes) {
      print('Inserted indexes: ${changes.inserted}');
      print('Deleted indexes: ${changes.deleted}');
      print('Modified indexes: ${changes.modified}');
    });
    realm.write(() => realm.add(Car('VW', 'Polo', kilometers: 22000)));
    

Vector Search with HNSW #

Realm now supports high-performance vector similarity search using the Hierarchical Navigable Small World (HNSW) algorithm. This enables AI/ML applications including semantic search, recommendation systems, image similarity, and RAG (Retrieval-Augmented Generation) patterns.

  • Define a model with vector embeddings:

    import 'package:realm/realm.dart';
    
    part 'app.realm.dart';
    
    @RealmModel()
    class _Document {
      @PrimaryKey()
      late String id;
    
      late String title;
      late String content;
      late List<double> embedding;  // Vector embeddings
    }
    
  • Generate the RealmObject class:

    dart run realm generate
    
  • Create a vector index and perform similarity search:

    var config = Configuration.local([Document.schema]);
    var realm = Realm(config);
    
    // Create HNSW vector index
    realm.write(() {
      realm.createVectorIndex<Document>(
        'embedding',
        metric: VectorDistanceMetric.cosine,  // or euclidean, dotProduct
        m: 16,                                // connections per layer (default: 16)
        efConstruction: 200,                  // build quality (default: 200)
      );
    });
    
    // Add documents with embeddings
    realm.write(() {
      realm.add(Document(
        '1',
        'AI Technology',
        'Machine learning and neural networks',
        embedding: [0.95, 0.85, 0.05, 0.10, 0.02, 0.08],
      ));
      realm.add(Document(
        '2',
        'Nature Guide',
        'Forest ecosystems and wildlife',
        embedding: [0.08, 0.12, 0.95, 0.88, 0.02, 0.05],
      ));
    });
    
    // K-Nearest Neighbors (KNN) search
    final queryVector = [0.9, 0.8, 0.1, 0.1, 0.05, 0.05];
    final results = realm.vectorSearchKnn<Document>(
      'embedding',
      queryVector: queryVector,
      k: 5,  // Return top 5 similar documents
    );
    
    for (var result in results) {
      print('${result.object.title}: distance=${result.distance}');
    }
    
    // Radius search (all documents within distance threshold)
    final radiusResults = realm.vectorSearchRadius<Document>(
      'embedding',
      queryVector: queryVector,
      maxDistance: 0.5,
    );
    

Vector Search Features #

  • Distance Metrics:

    • VectorDistanceMetric.cosine - Cosine similarity (recommended for normalized vectors)
    • VectorDistanceMetric.euclidean - Euclidean distance
    • VectorDistanceMetric.dotProduct - Dot product similarity
  • Search Types:

    • KNN Search: Find K nearest neighbors (vectorSearchKnn)
    • Radius Search: Find all vectors within distance threshold (vectorSearchRadius)
  • Index Management:

    • createVectorIndex() - Create HNSW index on vector property
    • removeVectorIndex() - Remove index (preserves data)
    • hasVectorIndex() - Check if index exists
    • getVectorIndexStats() - Get index statistics (numVectors, maxLayer)
  • Tuning Parameters:

    • m (default: 16) - Number of bi-directional links per node. Higher values = better recall, more memory
    • efConstruction (default: 200) - Build-time quality parameter. Higher values = better index quality, slower indexing

Production Migration Pattern #

When changing vector dimensions (e.g., 4D → 6D), use this safe migration pattern:

realm.write(() {
  // 1. Remove existing index (data is preserved!)
  if (realm.hasVectorIndex<Document>('embedding')) {
    realm.removeVectorIndex<Document>('embedding');
  }

  // 2. Transform embeddings
  for (final doc in realm.all<Document>()) {
    final oldValues = List<double>.from(doc.embedding);  // Create defensive copy
    final newValues = [...oldValues, 0.0, 0.0];          // Add new dimensions
    doc.embedding.clear();
    doc.embedding.addAll(newValues);
  }

  // 3. Create new index with updated dimensions
  realm.createVectorIndex<Document>(
    'embedding',
    metric: VectorDistanceMetric.cosine,
    m: 16,
    efConstruction: 200,
  );
});

Key points:

  • removeVectorIndex() does NOT delete your data
  • Always create a defensive copy with List<double>.from() before modifying
  • This pattern avoids data loss unlike shouldDeleteIfMigrationNeeded: true

Performance Benchmarks #

Benchmark results with 100 queries (1024-dimensional embeddings):

Metric Performance
Bulk Insert 0.90ms per record
Index Creation 125ms (m=16, efConstruction=200)
KNN Search (Cold Start) 2,016μs
KNN Search (Warm) ~102μs (9,766 searches/sec)
Radius Search 104-959μs
Filtered Search 162-629μs
Memory Overhead ~100% (index size ≈ data size)

Distance Metrics Comparison (all perform similarly):

  • Cosine: 190ms index creation, 155μs search
  • Euclidean: 183ms index creation, 152μs search
  • Dot Product: 178ms index creation, 157μs search

Parameter Tuning Impact:

  • m=8, efConstruction=100: 118μs search
  • m=16, efConstruction=200: 112μs search
  • m=32, efConstruction=400: 104μs search (fastest)

Higher HNSW parameters yield better search performance at the cost of slightly larger index size and longer index creation time.

Use Cases #

  • Semantic Search: Find documents by meaning, not just keywords
  • Recommendation Systems: Suggest similar items based on embeddings
  • Image Similarity: Find visually similar images using vision model embeddings
  • RAG Applications: Retrieve relevant context for AI chatbots and assistants
  • Duplicate Detection: Find near-duplicate content
  • Clustering & Classification: Group similar items together

For a complete example with 26 comprehensive tests, see example/lib/main.dart. Performance benchmarks are available in the test suite.

Samples #

For complete samples check the Realm Flutter and Dart Samples.

Documentation #

For API documentation go to

Use realm package for Flutter and realm_dart package for Dart applications.

For complete documentation of the SDKs, go to the Realm SDK documentation.

If you are using the Realm SDK for the first time, refer to the Quick Start documentation.

To learn more about using Realm with Atlas App Services and Device Sync, refer to the following Realm SDK documentation:

Realm Flutter SDK #

Realm Flutter package is published to realm.

Environment setup for Realm Flutter #

  • Realm Flutter supports the platforms iOS, Android, Windows, MacOS and Linux.
  • Flutter 3.10.2 or newer.
  • For Flutter Desktop environment setup, see Desktop support for Flutter.
  • Cocoapods v1.11 or newer.
  • CMake 3.21 or newer.

Usage #

The full contents of catalog.dart is listed after the usage

  • Add realm package to a Flutter application.

    flutter pub add realm
    
  • For running Flutter widget and unit tests run the following command to install the required native binaries.

    dart run realm install
    
  • Import Realm in a dart file (ex. catalog.dart).

    import 'package:realm/realm.dart';
    
  • Declare a part file catalog.realm.dart in the begining of the catalog.dart dart file after all imports.

    import 'dart:io';
    
    part 'catalog.realm.dart';
    
  • Create a data model class.

    It should start with an underscore _Item and be annotated with @RealmModel()

    @RealmModel()
    class _Item {
        @PrimaryKey()
        late int id;
    
        late String name;
    
        int price = 42;
    }
    
  • Generate RealmObject class Item from data model class _Item.

    On Flutter use dart run realm to run realm package commands

    dart run realm generate
    

    A new file catalog.realm.dart will be created next to the catalog.dart.

    *The generated file should be committed to source control

  • Use the RealmObject class Item with Realm.

    // Create a Configuration object
    var config = Configuration.local([Item.schema]);
    
    // Opean a Realm
    var realm = Realm(config);
    
    var myItem = Item(0, 'Pen', price: 4);
    
    // Open a write transaction
    realm.write(() {
        realm.add(myItem);
        var item = realm.add(Item(1, 'Pencil')..price = 20);
    });
    
    // Objects `myItem` and `item` are now managed and persisted in the realm
    
    // Read object properties from realm
    print(myItem.name);
    print(myItem.price);
    
    // Update object properties
    realm.write(() {
        myItem.price = 20;
        myItem.name = "Special Pencil";
    });
    
    // Get objects from the realm
    
    // Get all objects of type
    var items = realm.all<Item>();
    
    // Get object by index
    var item = items[1];
    
    // Get object by primary key
    var itemByKey = realm.find<Item>(0);
    
    // Filter and sort object
    var objects = realm.query<Item>("name == 'Special Pencil'");
    var name = 'Pen';
    objects = realm.query<Item>(r'name == $0', [name]);
    
    // Close the realm
    realm.close();
    

Full contents of catalog.dart #

import 'package:realm/realm.dart';

part 'catalog.realm.dart';

@RealmModel()
class _Item {
    @PrimaryKey()
    late int id;

    late String name;

    int price = 42;
}

// Create a Configuration object
var config = Configuration.local([Item.schema]);

// Open a Realm
var realm = Realm(config);

var myItem = Item(0, 'Pen', price: 4);

// Open a write transaction
realm.write(() {
    realm.add(myItem);
    var item = realm.add(Item(1, 'Pencil')..price = 20);
});

// Objects `myItem` and `item` are now managed and persisted in the realm

// Read object properties from realm
print(myItem.name);
print(myItem.price);

// Update object properties
realm.write(() {
    myItem.price = 20;
    myItem.name = "Special Pencil";
});

// Get objects from the realm

// Get all objects of type
var items = realm.all<Item>();

// Get object by index
var item = items[1];

// Get object by primary key
var itemByKey = realm.find<Item>(0);

// Filter and sort object
var objects = realm.query<Item>("name == 'Special Pencil'");
var name = 'Pen';
objects = realm.query<Item>(r'name == $0', [name]);

// Close the realm
realm.close();

Realm Dart Standalone SDK #

Realm Dart package is published to realm_dart.

Environment setup for Realm Dart #

  • Realm Dart supports the platforms Windows, Mac and Linux.
  • Dart SDK 3.0.2 or newer.

Usage #

  • Add realm_dart package to a Dart application.

    dart pub add realm_dart
    
  • Install the realm_dart package into the application. This downloads and copies the required native binaries to the app directory.

    dart run realm_dart install
    
  • Import realm_dart in a dart file (ex. catalog.dart).

    import 'package:realm_dart/realm.dart';
    
  • To generate RealmObject classes with realm_dart use this command.

    On Dart use dart run realm_dart to run realm_dart package commands

    dart run realm_dart generate
    

    A new file catalog.realm.dart will be created next to the catalog.dart.

    *The generated file should be committed to source control

  • The usage of the Realm Dart SDK is the same like the Realm Flutter above.

Sync data with Realm Flutter and Dart using Device Sync #

This section is about how to use the Realm with Device Sync and how to connect to Atlas App Services.

I. Set up Atlas App Services #

  1. Create an account on cloud.mongodb.com. Follow the instructions: Register a new Atlas Account.
  2. Create a new App following the instructions here: Create an App with Atlas App Services UI.
  3. Read Authentication Providers to see how to configure the appropriate authentication provider type.
  4. Go to the Device Sync menu and Enable Flexible Sync.
  5. Find and Copy the App ID of your new application.

II. Use Device Sync with the Realm #

  1. Initialize the App Services App client and authenticate a user.

    String appId = "<Atlas App ID>";
    final appConfig = AppConfiguration(appId);
    final app = App(appConfig);
    final user = await app.logIn(Credentials.anonymous());
    
  2. Open a synced realm.

    final config = Configuration.flexibleSync(user, [Task.schema]);
    final realm = Realm(config);
    
  3. Add a sync subscription and write data.

    Only data matching the query in the subscription will be synced to the server and only data matching the subscription will be downloaded to the local device realm file.

    realm.subscriptions.update((mutableSubscriptions) {
    mutableSubscriptions.add(realm.query<Task>(r'status == $0 AND progressMinutes == $1', ["completed", 100]));
    });
    await realm.subscriptions.waitForSynchronization();
    realm.write(() {
      realm.add(Task(ObjectId(), "Send an email", "completed", 4));
      realm.add(Task(ObjectId(), "Create a meeting", "completed", 100));
      realm.add(Task(ObjectId(), "Call the manager", "init", 2));
    });
    realm.close();
    

To learn more about how to sync data with Realm using Device Sync, refer to the Quick Start with Sync documentation.

Building the source #

See CONTRIBUTING.md for instructions about building the source.

Code of Conduct #

This project adheres to the MongoDB Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].

License #

Realm Flutter and Dart SDKs and Realm Core are published under the Apache License 2.0.

The "Dart" name and logo and the "Flutter" name and logo are trademarks owned by Google.
1
likes
0
points
481
downloads

Publisher

unverified uploader

Weekly Downloads

Realm Flutter SDK with built-in HNSW vector search - a mobile database alternative to SQLite with AI/ML vector similarity search capabilities for semantic search, RAG, and recommendation systems.

Repository (GitHub)
View/report issues

License

unknown (license)

Dependencies

flutter, realm_dart_vector_db

More

Packages that depend on realm_flutter_vector_db

Packages that implement realm_flutter_vector_db