flutter_litert library

LiteRT (formerly TensorFlow Lite) for Flutter

Classes

AsyncLock
A lightweight sequential async lock that serializes tasks through a future chain, preventing concurrent access to shared mutable state that is not thread-safe (e.g. a CompiledModel slot or interpreter input/output buffers).
BoundingBox
An axis-aligned or rotated bounding box defined by four corner points.
ByteConversionUtils
Utilities for converting between Dart objects and raw tensor byte buffers.
CameraFrame
A camera frame packaged for off-thread colour conversion and inference.
CameraFrameDecodePlan
Backend-neutral instructions for decoding a CameraFrame.
CompiledModel
LiteRT Next CompiledModel inference API.
CompiledModelPool
A round-robin pool of CompiledModel instances for the LiteRT Next engine.
CoreMlDelegate
CoreMl Delegate (no-op on web)
CoreMlDelegateOptions
CoreMlDelegate Options (no-op on web)
CoverFitTransform
A precomputed cover-fit mapping from source-image pixel coordinates to viewport coordinates, with optional horizontal mirroring for a front-camera preview.
Delegate
Web implementation of Delegate.
Detection
A single object detection result from a detection model.
FlexDelegate
FlexDelegate (no-op on web)
FpsCounter
A simple 1-second rolling FPS counter for camera-preview apps.
FrameThrottle
A single-slot gate for camera-frame processing that drops frames arriving while a previous frame is still being processed.
GpuDelegate
Metal Delegate for iOS/macOS (no-op on web)
GpuDelegateOptions
Metal Delegate options (no-op on web)
GpuDelegateOptionsV2
GPU delegate options for Android (no-op on web)
GpuDelegateV2
GPU delegate for Android (no-op on web)
Interpreter
Web implementation of Interpreter.
InterpreterOptions
Web implementation of InterpreterOptions.
IsolateInterpreter
Web implementation of IsolateInterpreter.
LetterboxParams
Parameters for aspect-preserving resize with centered padding.
LiteRtInterpreter
LiteRT.js-backed interpreter. Construct via fromBytes.
OneEuroFilter
One Euro filter for low-latency smoothing of a noisy 1D signal sampled at an irregular rate.
PackedImageLayout
A tightly packed source-buffer layout for image backend reconstruction.
PackedYuv
A contiguous YUV buffer produced by packYuv420, ready to hand to a native colour-conversion routine.
PerformanceConfig
Configuration for interpreter hardware acceleration and threading.
Point
A point with x, y, and optional z coordinates.
QuantizationParams
Quantization parameters for a tensor.
RoundRobinPool<T>
A generic round-robin pool with per-slot future-chain serialization locks.
SignatureRunner
Web platform stub for SignatureRunner.
SSDAnchorOptions
SSD anchor configuration options for SSD-style detection models.
Tensor
Web implementation of Tensor.
TransposeConvBiasOp
Loads and provides access to the Convolution2DTransposeBias custom op.
XNNPackDelegate
XNNPack Delegate (no-op on web)
XNNPackDelegateOptions
XNNPackDelegate Options (no-op on web)

Enums

Accelerator
Hardware accelerator requested for LiteRT Next compilation.
CameraFrameConversion
The colour conversion a CameraFrame's bytes need before being used as a 3-channel BGR image. Detector packages map this to an opencv COLOR_* code at the point of decode, inside their existing detection isolate.
CameraFrameDecodeOrder
Operation ordering for a backend-specific CameraFrame decoder.
CameraFrameRotation
Optional rotation applied after colour conversion. Detector packages map this to an opencv ROTATE_* code.
IsolateInterpreterState
State of an IsolateInterpreter lifecycle.
PackedImageFormat
Backend-neutral packed image formats used by camera frame decode plans.
PerformanceMode
Hardware acceleration mode for LiteRT inference.
Precision
GPU precision mode for LiteRT Next compilation.
TensorBufferMode
Tensor buffer allocation mode for CompiledModel I/O.
TensorType
Data type of a LiteRT tensor.
YuvLayout
Memory layout of a packed YUV buffer produced by packYuv420.

Mixins

LandmarkMixin
Mixin providing normalized coordinate utilities for landmarks with x/y pixel coordinates.
WebGpuFallback
Mixin that adds transparent WebGPU-to-WASM runtime fallback to a web detector class.

Extensions

ListShape on List

Constants

decodeFailurePrefix → const String
Wire-prefix marking an isolate error as an undecodable-image failure, so the main side can map it to a FormatException. Paired with throwDecodeFailure (isolate side) and rethrowOrFormatException (main side).
kInt32ByteSize → const int
Size of int32 in bytes (avoids dart:ffi dependency for web compatibility).

Properties

version → String
no setter

Functions

aggregateActiveAccelerator(Iterable<String?> backends) → String?
Aggregates the per-runner web backends of a multi-model detector into the single accelerator it should report.
allocTensorShape(List<int> shape) → Object
Allocates a nested list structure matching the given tensor shape.
argSortDesc(List<double> a) → List<int>
Returns indices that sort a in descending order.
barQuarterTurns(DeviceOrientation orientation) → int
Quarter-turns (clockwise) to rotate a top-bar widget so it reads upright when the device is in landscape. Use with RotatedBox(quarterTurns: ...).
bgrBytesToRgbFloat32({required Uint8List bytes, required int totalPixels, Float32List? buffer}) → Float32List
Converts BGR bytes to a flat Float32List with 0.0..1.0 normalization.
bgrBytesToSignedFloat32({required Uint8List bytes, required int totalPixels, Float32List? buffer}) → Float32List
Converts BGR bytes to a flat Float32List with -1.0..1.0 normalization.
cameraFrameFromRpcMessage(Map message, Uint8List bytes) → CameraFrame
Reconstructs a CameraFrame from an isolate-request message built by cameraFrameRpcFields and the already-materialized bytes.
cameraFrameRpcFields(CameraFrame frame, [Map<String, dynamic> extra = const {}]) → Map<String, dynamic>
Builds the isolate-request field map for a CameraFrame payload, merged with any extra per-op fields (e.g. maxDim, mode, outputFormat).
clamp01(double v) → double
Clamps v to the range 0.0..1.0. Returns 0.0 for NaN inputs.
clip(double v, double lo, double hi) → double
Clamps v to the range lo..hi.
compiledFloatCount(int byteSize, {String label = 'tensor'}) → int
Helpers for deriving tensor geometry from a CompiledModel.
compiledModelFromBufferAuto(Uint8List bytes, {Set<Accelerator> accelerators = const {Accelerator.gpu, Accelerator.cpu}, Precision precision = Precision.fp16, TensorBufferMode tensorBufferMode = TensorBufferMode.managed, bool forceCpu = false, void onGpuFallback(Object error)?}) → CompiledModel
Builds a CompiledModel from bytes, choosing between the GPU-with-CPU fallback path and an explicit accelerator set.
compiledOutputFloatCounts(CompiledModel model, {String label = 'model'}) → List<int>
Float counts for every output tensor of model, in order.
compiledSquareInputSide(CompiledModel model, {int channels = 3, String label = 'model'}) → int
The square input side (e.g. 192 or 224) of a single-input model, derived from its input byte size.
computeLetterboxParams({required int srcWidth, required int srcHeight, required int targetWidth, required int targetHeight, bool roundDimensions = true}) → LetterboxParams
Computes letterbox parameters for resizing srcWidthxsrcHeight to fit within targetWidthxtargetHeight while preserving aspect ratio.
concat0(List<List<List<double>>> parts) → List<List<double>>
Concatenates a list of 2D matrices along axis 0.
configureLiteRtLoader({String? moduleUrl, String? wasmUrl, bool? autoLoad}) → void
Configure the URLs / behavior of the LiteRT.js web auto-loader.
configureLiteRtWebLoader({String? moduleUrl, String? wasmUrl, bool? autoLoad}) → void
Configure the LiteRT.js web auto-loader.
coverFitScaleOffset(int sourceW, int sourceH, double viewW, double viewH) → ({double offsetX, double offsetY, double scale})
Cover-fit scale + offset for rendering a source region of size (sourceW, sourceH) into a viewport of size (viewW, viewH).
createNHWCTensor4D(int height, int width) → List<List<List<List<double>>>>
Creates a pre-allocated [1][height][width][3] tensor structure.
createOutputBuffers(List<List<int>> shapes) → Map<int, Object>
Creates output buffers matching the given output tensor shapes.
debugResetAcceleratorResolution() → void
Test hook: clears the cached 'auto' decision.
decodeAndSplitOutputs(List outputs) → List<Map<String, dynamic>>
Decodes raw detection model outputs and splits each row into xywh, rest, and C.
decodeBitmap(Uint8List bytes) → Future<ImageBitmap?>
Decodes encoded image bytes (JPEG, PNG, etc.) to an web.ImageBitmap.
decodeDetectionOutputs(List outputs) → List<List<double>>
Decodes raw detection model outputs into a standardized 2D matrix. Handles both [1, numBoxes, 5 + classes] and [1, 5 + classes, numBoxes] formats.
detectionSize({required int width, required int height, required CameraFrameRotation? rotation, required int maxDim}) → Size
Compute the final detection-image size used by overlay painters to map detector coordinates back onto the widget coord space.
drawBoundingBoxOutline({required Canvas canvas, required BoundingBox bbox, required double scaleX, required double scaleY, required double offsetX, required double offsetY, required Paint paint}) → void
Draw the axis-aligned outline of a BoundingBox transformed by a linear scale + offset. Use a stroked Paint for an outline, or a filled one to tint the interior.
drawLandmarkMarker(Canvas canvas, double x, double y, {double glowRadius = 8, double pointRadius = 5, double centerRadius = 2, Paint? glowPaint, Paint? pointPaint, Paint? centerPaint}) → void
Draw a standard "glow + point + center dot" triple-circle landmark marker at (x, y) in canvas coordinates.
drawSkeletonConnections({required Canvas canvas, required List<Offset> scaledPoints, required List<(int, int)> connections, required Paint paint}) → void
Draw straight-line connections between pre-scaled landmark points.
ensure2D(List raw) → List<List<double>>
Ensures a dynamic list is a proper 2D List<List<double>>.
fillNHWC4D(Float32List flat, List<List<List<List<double>>>> cache, int inH, int inW) → void
Fills an NHWC 4D tensor cache from a flat Float32List.
fillNHWC4DFromBgrBytes({required Uint8List bytes, required List<List<List<List<double>>>> tensor, required int width, required int height, double scale = 1.0 / 255.0, double offset = 0.0}) → void
Fills a 4D NHWC tensor in-place from raw BGR bytes with a BGR-to-RGB channel swap.
flattenDynamicTensor(Object? out) → Float32List
Flattens an arbitrarily nested tensor to a flat Float32List.
float16ToFloat32(int value) → double
Converts a float16 (as int) to float32.
float32ToFloat16(double value) → int
Converts a float32 to float16 representation (as int).
generateAnchors(SSDAnchorOptions options) → List<List<double>>
Generates SSD anchors based on the given options.
indexWhereFloatCount(List<int> floatCounts, bool test(int floats)) → int
Index of the first entry in floatCounts for which test returns true, or -1 if none match.
initializeWeb({String? tfJsScriptUrl, List<String>? tfBackendScriptUrls, String? tfliteScriptUrl}) → Future<void>
Initializes the TFLite.js WASM runtime for web.
iouLTRB(double aLeft, double aTop, double aRight, double aBottom, double bLeft, double bTop, double bRight, double bBottom) → double
Exact intersection-over-union of two axis-aligned boxes in left/top/right/ bottom coordinates. Returns 0.0 when the boxes are degenerate or disjoint.
isDefaultGpuCpuAccelerators(Set<Accelerator> accelerators) → bool
Whether accelerators is exactly the default "try GPU, fall back to CPU" set of {Accelerator.gpu, Accelerator.cpu}.
logCompileFallback({required String model, required String requested, required String actual}) → void
Logs when LiteRT.js silently compiled a model on a different backend than requested (its compile-time WebGPU-to-WASM fallback), so the swap is visible in the console instead of only in perf traces.
median(List<double> a) → double
Returns the median of a non-empty list.
nms(List<List<double>> boxes, List<double> scores, {double iouThres = 0.45, int maxDet = 100}) → List<int>
Non-Maximum Suppression over XYXY-format bounding boxes.
normalizeRadians(double angle) → double
Normalizes an angle in radians to the range -pi..pi.
packYuv420({required int width, required int height, required YuvPlane y, required YuvPlane u, YuvPlane? v, Uint8List? into}) → PackedYuv?
Packs a YUV420 camera frame into a single contiguous buffer suitable for native colour conversion (e.g. opencv's cvtColor with a COLOR_YUV2BGR_NV21 / COLOR_YUV2BGR_NV12 / COLOR_YUV2BGR_I420 code).
postProcessDetections({required List outputs, required int inputWidth, required int inputHeight, required double r, required int dw, required int dh, required int imageWidth, required int imageHeight, required double confThres, required double iouThres, required int topkPreNms, required int maxDet, int? filterClassId, bool scoresAreProbabilities = false}) → List<Detection>
Post-processes detection model outputs into Detection results.
postProcessDetectionsFlat(Float32List out, {required int channels, required int anchors, required bool channelMajor, required int inputWidth, required int inputHeight, required double r, required int dw, required int dh, required int imageWidth, required int imageHeight, required double confThres, required double iouThres, required int maxDet, int? filterClassId, bool useFastSingleClass = false, bool scoresAreProbabilities = false}) → List<Detection>
Decodes a YOLOv8 detection head directly from the flat out Float32 buffer.
prepareCameraFrame({required int width, required int height, required List<CameraPlane> planes, CameraFrameRotation? rotation, bool isBgra = true}) → CameraFrame?
Prepare a CameraFrame descriptor from raw camera planes, for use with a detector package's detectFromCameraFrame(...) method.
prepareCameraFrameFromImage(Object cameraImage, {CameraFrameRotation? rotation, bool? isBgra}) → CameraFrame?
Convenience wrapper around prepareCameraFrame that accepts any object duck-typed to package:camera's CameraImage (i.e. exposing width, height, and a planes iterable of objects with bytes, bytesPerRow, and bytesPerPixel getters).
resolveWebAccelerator(String requested) → Future<String>
Resolves a requested LiteRT.js accelerator ('auto' / 'webgpu' / 'wasm') into a concrete backend for this browser.
rethrowOrFormatException(Object error, [Uint8List? source]) → Never
Main-side: if error is a decode-failure signal (a StateError whose message starts with decodeFailurePrefix, as produced by throwDecodeFailure in the isolate and re-wrapped by IsolateRpcClient), throw a FormatException carrying the detail and optional source bytes; otherwise rethrow error unchanged.
rgbaToRgbFloat32(Uint8List rgbaData, Float32List output) → void
Converts RGBA pixel data to a normalized RGB Float32List tensor in-place.
rgbaToSignedRgbFloat32(Uint8List rgbaData, Float32List output) → void
Converts RGBA pixel data to a signed RGB Float32List tensor in-place.
rotationForFrame({required int width, required int height, required int sensorOrientation, required bool isFrontCamera, required DeviceOrientation deviceOrientation}) → CameraFrameRotation?
Compute the rotation needed to present a camera frame upright to an on-device detection model, given the camera's sensor orientation and the device's current physical orientation.
scaleFromLetterbox(List<double> xyxy, double ratio, int dw, int dh) → List<double>
Transforms bounding box coordinates from letterbox space back to original image space.
serveIsolateRpc({required Object mainSendPort, required Object receivePort, required Map<String, IsolateRpcHandler> handlers, String disposeOp = 'dispose', Future<void> onDispose()?}) → void
Web stub: isolate RPC serving requires dart:isolate, which is unavailable on web/WASM (detectors run on the main isolate). Present only so the symbol resolves on the web build; calling it throws.
sigmoid(double x) → double
Sigmoid activation function.
sigmoidClipped(double x, {double limit = 80.0}) → double
Sigmoid with input clipping to prevent overflow.
squareSideFromFloats(int floats, {int channels = 3, String label = 'input'}) → int
The side length S of a square [1, S, S, channels] tensor that holds floats float32 values.
throwDecodeFailure([String detail = 'Image bytes could not be decoded.']) → Never
Isolate-side: signal that image bytes could not be decoded.
transpose2D(List<List<double>> a) → List<List<double>>
Transposes a 2D list (swaps rows and columns).
weightedNms(List<List<double>> boxes, List<double> scores, {double iouThres = 0.45, int maxDet = 100}) → List<({List<double> box, int index, double score})>
Weighted Non-Maximum Suppression over XYXY-format bounding boxes.
xywhToXyxy(List<double> xywh) → List<double>
Converts XYWH bounding box format to XYXY format.
zeroOutputBuffers(Map<int, Object> outputs, List<List<int>> shapes) → void
Zeros all values in pre-allocated output buffers.

Typedefs

CameraPlane = ({Uint8List bytes, int pixelStride, int rowStride})
A single camera frame plane exposed by a camera plugin.
IsolateRpcHandler = Future<Object?> Function(Map message)
Handler for a single RPC operation running inside the isolate.
YuvPlane = ({Uint8List bytes, int pixelStride, int rowStride})
A single YUV plane exposed by a camera plugin, decoupled from any specific Flutter plugin's type (e.g. CameraImage.Plane).

Exceptions / Errors

ByteConversionError
Error thrown when an input value cannot be converted to the expected tensor type.
IsolateRpcExactError
Thrown by an isolate RPC handler to send an EXACT error string to the client, bypassing serveIsolateRpc's default '$e\n$st' stringification.
LiteRtRuntimeError
Thrown by LiteRtInterpreter.runForMultipleInputs when the underlying LiteRT.js call fails at inference time (as opposed to at compile time).