executorch_dart #

Pure-Dart on-device ML inference with PyTorch ExecuTorch. No Flutter SDK required — runs in any Dart program, including servers and command-line tools, on Android, iOS, macOS, Linux, and Windows.

pub.dev | Example (CLI)

Building a Flutter app instead? Use executorch_flutter — it wraps this package and adds Flutter asset-bundle loading plus Web support.

Table of Contents #

• Overview
• Installation
• Quick Start
• Platform Support
• API Reference
• Build Configuration
• Example
• License

Overview #

executorch_dart loads and runs ExecuTorch .pte models directly from Dart via dart:ffi and native assets — no platform channel, no Flutter engine, no widget tree. It's the pure-Dart core that executorch_flutter builds on; use it directly whenever your code doesn't need Flutter, such as a CLI tool or a Dart server.

• Vision inference: load a model, run forward(), get tensors back.
• On-device LLM (experimental): streaming text generation with Gemma 4 — see ExecuTorchLLM below.
• Backends: XNNPACK (all platforms), CoreML + Metal + MLX (Apple), Vulkan (opt-in).

Installation #

Requirements: Dart SDK 3.10+ with native assets support. dart run, dart test, and dart build all invoke this package's build hook automatically — there's nothing to compile by hand.

dependencies:
  executorch_dart: ^0.7.1

Quick Start #

1. Load a Model #

import 'package:executorch_dart/executorch_dart.dart';

final model = await ExecuTorchModel.load('/path/to/model.pte');

2. Run Inference #

final inputTensor = TensorData(
  shape: [1, 3, 224, 224],
  dataType: TensorType.float32,
  data: yourImageBytes,
);

final outputs = await model.forward([inputTensor]);

for (var output in outputs) {
  print('Shape: ${output.shape}, Type: ${output.dataType}');
}

3. Clean Up #

await model.dispose();

Model Loading Options #

// From bytes
final bytes = await File('/path/to/model.pte').readAsBytes();
final model = await ExecuTorchModel.loadFromBytes(bytes);

Flutter apps should load from the asset bundle with loadModelFromAsset from package:executorch_flutter/executorch_flutter.dart instead — this package has no dependency on dart:ui or rootBundle.

Platform Support #

*Opt-in. MLX is the Apple-Silicon GPU runtime used by the LLM path (macOS 14+, arm64).

There's no Web target here — dart:ffi doesn't run in a browser. Use executorch_flutter for Web; it ships a WebAssembly build behind the same API.

API Reference #

ExecuTorchModel #

The entire model API is four instance members plus two static loaders:

// Load (static factories)
static Future<ExecuTorchModel> load(String filePath)
static Future<ExecuTorchModel> loadFromBytes(Uint8List modelBytes)

// Inference
Future<List<TensorData>> forward(List<TensorData> inputs)

// Lifecycle
Future<void> dispose()
bool get isDisposed
String get modelId

No options, no timeouts, no singleton manager — you own the model's lifecycle.

TensorData #

final tensor = TensorData(
  shape: [1, 3, 224, 224],       // Dimensions
  dataType: TensorType.float32,  // See dtype table below
  data: Uint8List(...),          // Raw bytes, little-endian
  name: 'input_0',               // Optional
);

Supported dtypes — all 13 map 1:1 to ExecuTorch's native types:

Building data by hand is error-prone, so use ExecutorchManager to encode numeric lists — it handles float16/bfloat16 conversion (round-to-nearest-even) and endianness:

final tensor = ExecutorchManager.instance.createTensorData(
  shape: [1, 4],
  dataType: TensorType.float16,
  data: [1.0, 2.5, -3.25, 0.5],
);

BackendQuery #

Query available backends at runtime:

if (BackendQuery.isAvailable(Backend.coreml)) {
  model = await ExecuTorchModel.load('/path/model_coreml.pte');
} else {
  model = await ExecuTorchModel.load('/path/model_xnnpack.pte');
}

final backends = BackendQuery.available;
print('Available: ${backends.map((b) => b.displayName).join(", ")}');

Tokenizer #

Text to token ids and back, with no model attached. This is what encoder models need — embeddings, classification, retrieval — because they take token ids as an input tensor and produce a vector in a single forward(), with no generation loop to borrow a tokenizer from.

final tokenizer = await Tokenizer.load('/path/to/tokenizer.json');

final ids = tokenizer.encode('some text');
final input = TensorData(
  shape: [1, ids.length],
  dataType: TensorType.int64,
  data: Int64List.fromList(ids).buffer.asUint8List(),
);
final outputs = await model.forward([input]);

print(tokenizer.decode(ids));   // back to text
tokenizer.dispose();

Supported formats: HuggingFace tokenizer.json built on BPE (GPT-2, Llama, Gemma, Qwen, Mistral), SentencePiece .model, TikToken, and the llama2.c binary format.

Not supported: WordPiece / BERT-family tokenizers. The reader derives from a BPE base and implements no BertNormalizer, which rules out BERT, DistilBERT, MiniLM and most sentence-transformers models. Passing one raises an error naming the specific reason rather than a generic parse failure.

Not available on Web — this is dart:ffi throughout. For generative models use ExecuTorchLLM below, which owns its tokenizer internally.

ExecuTorchLLM (experimental) #

On-device generative text — Google Gemma 4 E2B — with token-by-token streaming, separate from the tensor API (load / generate Stream<String> / stop / reset / dispose). See executorch_flutter's LLM guide for model export and setup — the API is identical from pure Dart; just import package:executorch_dart/executorch_dart.dart instead of the Flutter package.

Exception Hierarchy #

ExecuTorchException (base)
├── ExecuTorchModelException      // Model loading/lifecycle
├── ExecuTorchInferenceException  // Inference execution
├── ExecuTorchValidationException // Tensor validation
├── ExecuTorchMemoryException     // Memory/resources
├── ExecuTorchIOException         // File I/O
└── ExecuTorchPlatformException   // Platform communication

Build Configuration #

The native library is downloaded (or compiled) by a native-assets build hook. Configure it under hooks: user_defines: executorch_dart: in your application's pubspec.yaml — the package that will actually run, not a library you merely depend on:

hooks:
  user_defines:
    executorch_dart:
      debug: false                  # Debug logging + debug binaries
      build_mode: "prebuilt"        # "prebuilt", "local", or "source"
      # prebuilt_version: "1.4.0.6" # Optional: pin a specific native release
      llm: false                    # Opt in to the LLM runner (ExecuTorchLLM)
      # For source mode: build from local ExecuTorch checkout
      # build_mode: "source"
      # executorch_source: "/path/to/executorch"
      # For local mode: point at pre-compiled libraries
      # local_lib_dir: "/path/to/compiled/libs"
      backends:
        - xnnpack
        - coreml
        - metal

All seven keys #

Default backends by platform #

See executorch_flutter's README for the environment-variable overrides and troubleshooting — it's the same build hook underneath.

Example #

A minimal CLI example lives in example/:

cd example
dart run bin/infer.dart /path/to/model.pte

Or compile it to a self-contained bundle with the native library included:

dart build cli --target bin/infer.dart
./build/cli/*/bundle/bin/infer /path/to/model.pte

License #

MIT License - see LICENSE.

Built with love for the Dart and PyTorch communities.