Initial Release #

• Core Features

  • FFI-based Dart bindings for llama.cpp
  • Low-level LlamaModel API for direct text generation control
  • Support for loading GGUF model files
  • Automatic memory management with proper cleanup
  • Real-time streaming support with token-by-token generation
  • Configurable stop sequences for controlling generation boundaries

• API Features

  • LlamaModel - Main class for model initialization and text generation
  • GenerationRequest - Configurable generation parameters
  • GenerationResponse - Detailed generation results with token counts
  • Streaming and non-streaming generation modes
  • Temperature, top-p, top-k, and repeat penalty sampling controls
  • Random seed support for reproducible generation
  • Token counting and generation time tracking

• Stop Sequence Support

  • Configurable stop sequences in GenerationRequest
  • Proper handling of stop sequences split across multiple tokens
  • Automatic trimming of stop sequences from output
  • Works correctly in both streaming and non-streaming modes

• Memory Management

  • Fixed double-free error in sampler disposal
  • Proper lifecycle management for all native resources
  • RAII pattern with reliable dispose() methods

• Examples

  • example/completion.dart - Simple text completion with streaming support
  • example/gemma_chat.dart - Full Gemma chat implementation with proper formatting

• Developer Experience

  • Automated FFI binding generation with ffigen
  • Comprehensive build scripts for llama.cpp compilation
  • Model download script for testing (Gemma 3 1B)
  • Unit and integration tests
  • Code quality enforcement with very_good_analysis

• Platform Support

  • macOS (ARM64 and x86_64) with Metal acceleration
  • Linux (x86_64)
  • Windows support planned for future release