Liquid AI LEAP Flutter Plugin #

A Flutter plugin for on-device AI inference using Liquid AI's LEAP SDK. Run Liquid Foundation Models (LFM) directly on iOS and Android devices with no cloud dependencies.

Why this plugin? #

This supports GGUF models! Compared to LiteRT or ONNX GenAI, this give access to a much larger range of models.

Features #

• 🚀 On-device inference - Run AI models locally without internet
• 💬 Streaming responses - Real-time token-by-token output
• 🖼️ Multimodal support - Text, images, and audio inputs
• 🔧 Function calling - Let models call your app's functions
• 📝 Constrained generation - JSON schema validation for structured output
• 📦 Automatic model management - Download, cache, and manage models

Supported Models #

The plugin supports models from LiquidAI but also any GGUF model!

Liquid AI Models #

LFM2- Multimodal Models from liquid AI LEAP.

• Leap Library: LFM2-1.2B
• Leap Bundles: Hugging Face: LiquidAI/LeapBundles
• Liquid AI Models: Hugging Face: LiquidAI/models

GGUF Models #

More about GGUF.

• Find GGUF models on Hugging Face.

Note: The SDK now recommends using GGUF models instead of ExecuTorch bundles.

Tested models

These models have been successfully tested for on-device inference using the plugin.

Requirements #

iOS #

• iOS 15.0+
• Xcode 15.0+
• Swift 5.9+
• Physical device recommended (3GB+ RAM)

Un-tested: if you have, let me know.

Android #

• API 31+ (Android 12)
• arm64-v8a ABI
• Physical device recommended (3GB+ RAM)

Tested on a Pixel 8a

Installation #

Add this to your package's pubspec.yaml:

dependencies:
  liquid_ai_leap: ^0.1.0

Then run:

flutter pub get

Quick Start #

1. Initialize and Load a Model #

import 'package:liquid_ai_leap/liquid_ai_leap.dart';

// Create plugin instance
final leap = LiquidAiLeap();

// Load a model (downloads if not cached)
final modelRunner = await leap.loadModel(
  model: 'LFM2-1.2B',
  quantization: 'Q5_K_M',
  onProgress: (progress, bytesPerSecond) {
    print('Downloading: ${(progress * 100).toStringAsFixed(1)}%');
  },
);

Note: The loadModel will search Leap models library using model and quantization parameters. During testing this was found to be somewhat unrialiable. If you are having issues with the auto-download like me, use the downloadModel method to download the model first, then use the loadModel method to load the model. downloadModel uses the url parameter to download the model (.bundle or .gguf) from Hugging Face for example.

2. Create a Conversation #

// Create a conversation with optional system prompt
final conversation = modelRunner.createConversation(
  systemPrompt: 'You are a helpful assistant.',
);

3. Generate Responses #

// Send a message and stream the response
final message = ChatMessage.user('What is the capital of France?');

await for (final response in conversation.generateResponse(message: message)) {
  switch (response) {
    case ChunkResponse(:final text):
      // Print streamed text
      stdout.write(text);
    case CompleteResponse(:final message, :final stats):
      // Generation complete
      print('\n\nTokens: ${stats?.totalTokens}');
      print('Speed: ${stats?.tokensPerSecond.toStringAsFixed(1)} tok/s');
  }
}

Advanced Usage #

Generation Options #

Control generation behavior with GenerationOptions:

await for (final response in conversation.generateResponse(
  message: ChatMessage.user('Write a haiku about Flutter'),
  options: GenerationOptions(
    temperature: 0.7,      // Creativity (0.0-2.0)
    topP: 0.9,             // Nucleus sampling
    maxTokens: 100,        // Maximum tokens to generate
    repetitionPenalty: 1.1, // Reduce repetition
  ),
)) {
  // Handle response
}

Function Calling #

Register functions for the model to call:

// Define a function
final weatherFunction = LeapFunction(
  name: 'get_weather',
  description: 'Get the current weather for a location',
  parameters: [
    LeapFunctionParameter(
      name: 'city',
      type: LeapFunctionParameterType.string(),
      description: 'The city name',
    ),
    LeapFunctionParameter(
      name: 'units',
      type: LeapFunctionParameterType.string(
        enumValues: ['celsius', 'fahrenheit'],
      ),
      description: 'Temperature units',
      optional: true,
    ),
  ],
);

// Register the function
conversation.registerFunction(weatherFunction);

// Handle function calls in responses
await for (final response in conversation.generateResponse(
  message: ChatMessage.user('What is the weather in Paris?'),
)) {
  switch (response) {
    case FunctionCallResponse(:final calls):
      for (final call in calls) {
        final result = await handleFunctionCall(call);
        // Provide result back to model
      }
    case ChunkResponse(:final text):
      stdout.write(text);
  }
}

Constrained JSON Generation #

Force the model to output valid JSON matching a schema:

final response = await conversation.generateResponse(
  message: ChatMessage.user('Generate a user profile'),
  options: GenerationOptions(
    jsonSchemaConstraint: '''
    {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "age": { "type": "integer" },
        "email": { "type": "string" }
      },
      "required": ["name", "email"]
    }
    ''',
  ),
);

Image Input (Vision Models) #

import 'dart:io';
import 'dart:typed_data';

// Load image bytes
final imageBytes = await File('photo.jpg').readAsBytes();

// Create message with image
final message = ChatMessage(
  role: ChatMessageRole.user,
  content: [
    ChatMessageContent.text('What do you see in this image?'),
    ChatMessageContent.image(imageBytes),
  ],
);

await for (final response in conversation.generateResponse(message: message)) {
  // Handle response
}

Audio Input (Audio Models) #

// Create message with audio
final message = ChatMessage(
  role: ChatMessageRole.user,
  content: [
    ChatMessageContent.text('Transcribe this audio:'),
    ChatMessageContent.audio(wavBytes),
  ],
);

Model Management #

// Check if a model is cached
final isCached = await leap.isModelCached(
  model: 'LFM2-1.2B',
  quantization: 'Q5_K_M',
);

// Download without loading
final manifest = await leap.downloadModel(
  model: 'LFM2-1.2B',
  quantization: 'Q5_K_M',
  url: 'https://huggingface.co/LiquidAI/LeapBundles/resolve/main/LFM2-1_2B_8da4w.bundle?download=true',
  onProgress: (progress, bytesPerSecond) {
    print('Downloading: ${(progress * 100).toStringAsFixed(1)}%');
  },
);

// Delete a cached model
await leap.deleteModel(
  model: 'LFM2-1.2B',
  quantization: 'Q5_K_M',
);

Cleanup #

// Unload model to free memory
await modelRunner.unload();

Error Handling #

import 'package:liquid_ai_leap/liquid_ai_leap.dart';

try {
  final runner = await leap.loadModel(...);
} on LeapNetworkException catch (e) {
  print('Network error: ${e.message}');
} on LeapModelNotFoundException catch (e) {
  print('Model not found: ${e.model}');
} on LeapInsufficientMemoryException catch (e) {
  print('Not enough memory to load model');
} on LeapException catch (e) {
  print('LEAP error: ${e.message}');
}

Platform Setup #

iOS #

The plugin uses Swift Package Manager to fetch the LEAP SDK. No additional setup required.

If you need to customize the build, update ios/liquid_ai_leap.podspec.

Android #

The plugin uses Maven to fetch the LEAP SDK. You may need to configure authentication for the Maven repository.

Add to your app's android/build.gradle:

allprojects {
    repositories {
        maven {
            url 'https://maven.pkg.github.com/Liquid4All/leap-android'
            credentials {
                username = System.getenv('GITHUB_USERNAME')
                password = System.getenv('GITHUB_TOKEN')
            }
        }
    }
}

Scripts #

Dependency Management #

# Sync dependencies to pinned versions
./scripts/dependencies.sh sync

# Check for and upgrade to latest SDK versions
./scripts/dependencies.sh upgrade

Publishing #

# Publish a bug fix (0.1.0 -> 0.1.1)
./scripts/publish.sh fix

# Publish a new feature (0.1.0 -> 0.2.0)
./scripts/publish.sh minor

# Publish a breaking change (0.1.0 -> 1.0.0)
./scripts/publish.sh major

API Reference #

See the API documentation for detailed information.

Contributing #

Contributions are welcome!

License #

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments #

• Liquid AI for the LEAP SDK