realm_flutter_vector_db 1.0.2
realm_flutter_vector_db: ^1.0.2 copied to clipboard
Realm Flutter SDK with built-in HNSW vector search - a mobile database alternative to SQLite with AI/ML vector similarity search capabilities for semantic search, RAG, and recommendation systems.
Warning
We announced the deprecation of Atlas Device Sync + Realm SDKs in September 2024. For more information please see:
For a version of realm-dart without sync features, install version 20 or see the community branch.
Realm is a mobile database that runs directly inside phones, tablets or wearables. This repository holds the source code for the Realm SDK for Flutter™ and Dart™.
Features #
- Mobile-first: Realm is the first database built from the ground up to run directly inside phones, tablets, and wearables.
- Simple: Realm's object-oriented data model is simple to learn, doesn't need an ORM, and the API lets you write less code to get apps up & running in minutes.
- Modern: Realm supports latest Dart and Flutter versions and is built with sound null-safety.
- Fast: Realm is faster than even raw SQLite on common operations while maintaining an extremely rich feature set.
- Vector Search (HNSW): Built-in support for high-performance vector similarity search using Hierarchical Navigable Small World (HNSW) algorithm. Perfect for AI/ML applications, semantic search, recommendation systems, and RAG (Retrieval-Augmented Generation) patterns.
- MongoDB Atlas Device Sync: Makes it simple to keep data in sync across users, devices, and your backend in real-time. Get started for free with a template application and create the cloud backend.
Getting Started #
-
Import Realm in a dart file
app.dartimport 'package:realm/realm.dart'; // import realm package part 'app.realm.dart'; // declare a part file. @RealmModel() // define a data model class named `_Car`. class _Car { late String make; late String model; int? kilometers = 500; } -
Generate RealmObject class
Carfrom data model class_Car.dart run realm generate -
Open a Realm and add some objects.
var config = Configuration.local([Car.schema]); var realm = Realm(config); var car = Car("Tesla", "Model Y", kilometers: 5); realm.write(() { realm.add(car); }); -
Query objects in Realm.
var cars = realm.all<Car>(); Car myCar = cars[0]; print("My car is ${myCar.make} model ${myCar.model}"); cars = realm.all<Car>().query("make == 'Tesla'"); -
Get stream of result changes for a query.
final cars = realm.all<Car>().query(r'make == $0', ['Tesla']); cars.changes.listen((changes) { print('Inserted indexes: ${changes.inserted}'); print('Deleted indexes: ${changes.deleted}'); print('Modified indexes: ${changes.modified}'); }); realm.write(() => realm.add(Car('VW', 'Polo', kilometers: 22000)));
Vector Search with HNSW #
Realm now supports high-performance vector similarity search using the Hierarchical Navigable Small World (HNSW) algorithm. This enables AI/ML applications including semantic search, recommendation systems, image similarity, and RAG (Retrieval-Augmented Generation) patterns.
Quick Start with Vector Search #
-
Define a model with vector embeddings:
import 'package:realm/realm.dart'; part 'app.realm.dart'; @RealmModel() class _Document { @PrimaryKey() late String id; late String title; late String content; late List<double> embedding; // Vector embeddings } -
Generate the RealmObject class:
dart run realm generate -
Create a vector index and perform similarity search:
var config = Configuration.local([Document.schema]); var realm = Realm(config); // Create HNSW vector index realm.write(() { realm.createVectorIndex<Document>( 'embedding', metric: VectorDistanceMetric.cosine, // or euclidean, dotProduct m: 16, // connections per layer (default: 16) efConstruction: 200, // build quality (default: 200) ); }); // Add documents with embeddings realm.write(() { realm.add(Document( '1', 'AI Technology', 'Machine learning and neural networks', embedding: [0.95, 0.85, 0.05, 0.10, 0.02, 0.08], )); realm.add(Document( '2', 'Nature Guide', 'Forest ecosystems and wildlife', embedding: [0.08, 0.12, 0.95, 0.88, 0.02, 0.05], )); }); // K-Nearest Neighbors (KNN) search final queryVector = [0.9, 0.8, 0.1, 0.1, 0.05, 0.05]; final results = realm.vectorSearchKnn<Document>( 'embedding', queryVector: queryVector, k: 5, // Return top 5 similar documents ); for (var result in results) { print('${result.object.title}: distance=${result.distance}'); } // Radius search (all documents within distance threshold) final radiusResults = realm.vectorSearchRadius<Document>( 'embedding', queryVector: queryVector, maxDistance: 0.5, );
Vector Search Features #
-
Distance Metrics:
VectorDistanceMetric.cosine- Cosine similarity (recommended for normalized vectors)VectorDistanceMetric.euclidean- Euclidean distanceVectorDistanceMetric.dotProduct- Dot product similarity
-
Search Types:
- KNN Search: Find K nearest neighbors (
vectorSearchKnn) - Radius Search: Find all vectors within distance threshold (
vectorSearchRadius)
- KNN Search: Find K nearest neighbors (
-
Index Management:
createVectorIndex()- Create HNSW index on vector propertyremoveVectorIndex()- Remove index (preserves data)hasVectorIndex()- Check if index existsgetVectorIndexStats()- Get index statistics (numVectors, maxLayer)
-
Tuning Parameters:
m(default: 16) - Number of bi-directional links per node. Higher values = better recall, more memoryefConstruction(default: 200) - Build-time quality parameter. Higher values = better index quality, slower indexing
Production Migration Pattern #
When changing vector dimensions (e.g., 4D → 6D), use this safe migration pattern:
realm.write(() {
// 1. Remove existing index (data is preserved!)
if (realm.hasVectorIndex<Document>('embedding')) {
realm.removeVectorIndex<Document>('embedding');
}
// 2. Transform embeddings
for (final doc in realm.all<Document>()) {
final oldValues = List<double>.from(doc.embedding); // Create defensive copy
final newValues = [...oldValues, 0.0, 0.0]; // Add new dimensions
doc.embedding.clear();
doc.embedding.addAll(newValues);
}
// 3. Create new index with updated dimensions
realm.createVectorIndex<Document>(
'embedding',
metric: VectorDistanceMetric.cosine,
m: 16,
efConstruction: 200,
);
});
Key points:
removeVectorIndex()does NOT delete your data- Always create a defensive copy with
List<double>.from()before modifying - This pattern avoids data loss unlike
shouldDeleteIfMigrationNeeded: true
Performance Benchmarks #
Benchmark results with 100 queries (1024-dimensional embeddings):
| Metric | Performance |
|---|---|
| Bulk Insert | 0.90ms per record |
| Index Creation | 125ms (m=16, efConstruction=200) |
| KNN Search (Cold Start) | 2,016μs |
| KNN Search (Warm) | ~102μs (9,766 searches/sec) |
| Radius Search | 104-959μs |
| Filtered Search | 162-629μs |
| Memory Overhead | ~100% (index size ≈ data size) |
Distance Metrics Comparison (all perform similarly):
- Cosine: 190ms index creation, 155μs search
- Euclidean: 183ms index creation, 152μs search
- Dot Product: 178ms index creation, 157μs search
Parameter Tuning Impact:
- m=8, efConstruction=100: 118μs search
- m=16, efConstruction=200: 112μs search
- m=32, efConstruction=400: 104μs search (fastest)
Higher HNSW parameters yield better search performance at the cost of slightly larger index size and longer index creation time.
Use Cases #
- Semantic Search: Find documents by meaning, not just keywords
- Recommendation Systems: Suggest similar items based on embeddings
- Image Similarity: Find visually similar images using vision model embeddings
- RAG Applications: Retrieve relevant context for AI chatbots and assistants
- Duplicate Detection: Find near-duplicate content
- Clustering & Classification: Group similar items together
For a complete example with 26 comprehensive tests, see example/lib/main.dart. Performance benchmarks are available in the test suite.
Samples #
For complete samples check the Realm Flutter and Dart Samples.
Documentation #
For API documentation go to
Use realm package for Flutter and realm_dart package for Dart applications.
For complete documentation of the SDKs, go to the Realm SDK documentation.
If you are using the Realm SDK for the first time, refer to the Quick Start documentation.
To learn more about using Realm with Atlas App Services and Device Sync, refer to the following Realm SDK documentation:
Realm Flutter SDK #
Realm Flutter package is published to realm.
Environment setup for Realm Flutter #
- Realm Flutter supports the platforms iOS, Android, Windows, MacOS and Linux.
- Flutter 3.10.2 or newer.
- For Flutter Desktop environment setup, see Desktop support for Flutter.
- Cocoapods v1.11 or newer.
- CMake 3.21 or newer.
Usage #
The full contents of catalog.dart is listed after the usage
-
Add
realmpackage to a Flutter application.flutter pub add realm -
For running Flutter widget and unit tests run the following command to install the required native binaries.
dart run realm install -
Import Realm in a dart file (ex.
catalog.dart).import 'package:realm/realm.dart'; -
Declare a part file
catalog.realm.dartin the begining of thecatalog.dartdart file after all imports.import 'dart:io'; part 'catalog.realm.dart'; -
Create a data model class.
It should start with an underscore
_Itemand be annotated with@RealmModel()@RealmModel() class _Item { @PrimaryKey() late int id; late String name; int price = 42; } -
Generate RealmObject class
Itemfrom data model class_Item.On Flutter use
dart run realmto runrealmpackage commandsdart run realm generateA new file
catalog.realm.dartwill be created next to thecatalog.dart.*The generated file should be committed to source control
-
Use the RealmObject class
Itemwith Realm.// Create a Configuration object var config = Configuration.local([Item.schema]); // Opean a Realm var realm = Realm(config); var myItem = Item(0, 'Pen', price: 4); // Open a write transaction realm.write(() { realm.add(myItem); var item = realm.add(Item(1, 'Pencil')..price = 20); }); // Objects `myItem` and `item` are now managed and persisted in the realm // Read object properties from realm print(myItem.name); print(myItem.price); // Update object properties realm.write(() { myItem.price = 20; myItem.name = "Special Pencil"; }); // Get objects from the realm // Get all objects of type var items = realm.all<Item>(); // Get object by index var item = items[1]; // Get object by primary key var itemByKey = realm.find<Item>(0); // Filter and sort object var objects = realm.query<Item>("name == 'Special Pencil'"); var name = 'Pen'; objects = realm.query<Item>(r'name == $0', [name]); // Close the realm realm.close();
Full contents of catalog.dart #
import 'package:realm/realm.dart';
part 'catalog.realm.dart';
@RealmModel()
class _Item {
@PrimaryKey()
late int id;
late String name;
int price = 42;
}
// Create a Configuration object
var config = Configuration.local([Item.schema]);
// Open a Realm
var realm = Realm(config);
var myItem = Item(0, 'Pen', price: 4);
// Open a write transaction
realm.write(() {
realm.add(myItem);
var item = realm.add(Item(1, 'Pencil')..price = 20);
});
// Objects `myItem` and `item` are now managed and persisted in the realm
// Read object properties from realm
print(myItem.name);
print(myItem.price);
// Update object properties
realm.write(() {
myItem.price = 20;
myItem.name = "Special Pencil";
});
// Get objects from the realm
// Get all objects of type
var items = realm.all<Item>();
// Get object by index
var item = items[1];
// Get object by primary key
var itemByKey = realm.find<Item>(0);
// Filter and sort object
var objects = realm.query<Item>("name == 'Special Pencil'");
var name = 'Pen';
objects = realm.query<Item>(r'name == $0', [name]);
// Close the realm
realm.close();
Realm Dart Standalone SDK #
Realm Dart package is published to realm_dart.
Environment setup for Realm Dart #
- Realm Dart supports the platforms Windows, Mac and Linux.
- Dart SDK 3.0.2 or newer.
Usage #
-
Add
realm_dartpackage to a Dart application.dart pub add realm_dart -
Install the
realm_dartpackage into the application. This downloads and copies the required native binaries to the app directory.dart run realm_dart install -
Import realm_dart in a dart file (ex.
catalog.dart).import 'package:realm_dart/realm.dart'; -
To generate RealmObject classes with realm_dart use this command.
On Dart use
dart run realm_dartto runrealm_dartpackage commandsdart run realm_dart generateA new file
catalog.realm.dartwill be created next to thecatalog.dart.*The generated file should be committed to source control
-
The usage of the Realm Dart SDK is the same like the Realm Flutter above.
Sync data with Realm Flutter and Dart using Device Sync #
This section is about how to use the Realm with Device Sync and how to connect to Atlas App Services.
I. Set up Atlas App Services #
- Create an account on cloud.mongodb.com. Follow the instructions: Register a new Atlas Account.
- Create a new App following the instructions here: Create an App with Atlas App Services UI.
- Read Authentication Providers to see how to configure the appropriate authentication provider type.
- Go to the Device Sync menu and Enable Flexible Sync.
- Find and Copy the App ID of your new application.
II. Use Device Sync with the Realm #
-
Initialize the App Services
Appclient and authenticate a user.String appId = "<Atlas App ID>"; final appConfig = AppConfiguration(appId); final app = App(appConfig); final user = await app.logIn(Credentials.anonymous()); -
Open a synced realm.
final config = Configuration.flexibleSync(user, [Task.schema]); final realm = Realm(config); -
Add a sync subscription and write data.
Only data matching the query in the subscription will be synced to the server and only data matching the subscription will be downloaded to the local device realm file.
realm.subscriptions.update((mutableSubscriptions) { mutableSubscriptions.add(realm.query<Task>(r'status == $0 AND progressMinutes == $1', ["completed", 100])); }); await realm.subscriptions.waitForSynchronization(); realm.write(() { realm.add(Task(ObjectId(), "Send an email", "completed", 4)); realm.add(Task(ObjectId(), "Create a meeting", "completed", 100)); realm.add(Task(ObjectId(), "Call the manager", "init", 2)); }); realm.close();
To learn more about how to sync data with Realm using Device Sync, refer to the Quick Start with Sync documentation.
Building the source #
See CONTRIBUTING.md for instructions about building the source.
Code of Conduct #
This project adheres to the MongoDB Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].
License #
Realm Flutter and Dart SDKs and Realm Core are published under the Apache License 2.0.