verifyCompiledModel function

BackendVerification verifyCompiledModel(
  1. Uint8List modelBytes,
  2. CompiledModel compiled, {
  3. double tolerance = kDefaultBackendTolerance,
})

Checks whether compiled computes the same thing as a bare-CPU Interpreter built from the same modelBytes.

Motivation: LiteRT Next has shipped defects where CompiledModel returns kLiteRtStatusOk while producing output that is wrong, or never written at all. Neither is detectable from a status code or from timing, so the only reliable check is to compare against a backend already known to be correct. Run this once, at initialization, before trusting a CompiledModel.

The reference deliberately uses PerformanceConfig.disabled rather than XNNPACK or GPU: a delegate can silently decline to run any ops, so the slowest path is the only one that is unambiguously the plain CPU kernels.

This consumes one inference on compiled. That is safe for a healthy model, and a model that fails here should be discarded rather than reused.

Only single-input float32 models are supported; anything else returns a BackendVerification with BackendVerification.skipped set, and BackendVerification.agrees false so an unchecked model is never mistaken for a verified one.

Implementation

BackendVerification verifyCompiledModel(
  Uint8List modelBytes,
  CompiledModel compiled, {
  double tolerance = kDefaultBackendTolerance,
}) {
  BackendVerification skip(String reason) => BackendVerification(
    agrees: false,
    absoluteDeviation: double.nan,
    outputRange: double.nan,
    relativeDeviation: double.nan,
    skippedReason: reason,
  );

  if (compiled.inputCount != 1) {
    return skip('model has ${compiled.inputCount} inputs, expected 1');
  }

  Interpreter? reference;
  Delegate? delegate;
  final List<double> expected;
  final Float32List input;
  try {
    final (options, d) = InterpreterFactory.create(PerformanceConfig.disabled);
    delegate = d;
    reference = Interpreter.fromBuffer(modelBytes, options: options);
    reference.allocateTensors();

    final inputTensors = reference.getInputTensors();
    if (inputTensors.length != 1) {
      return skip('reference has ${inputTensors.length} inputs, expected 1');
    }
    final inputFloats = _elementCount(inputTensors.first.shape);

    // Deterministic, non-degenerate ramp. A constant or all-zero input can
    // mask a backend that ignores its input entirely, and 251 being prime
    // keeps the pattern from aligning with channel or row strides.
    input = Float32List(inputFloats);
    for (var i = 0; i < inputFloats; i++) {
      input[i] = (i % 251) / 251.0;
    }

    final outputTensors = reference.getOutputTensors();
    final outputs = [
      for (final t in outputTensors) Float32List(_elementCount(t.shape)),
    ];
    reference.runForMultipleInputs(
      [input.buffer],
      {for (var i = 0; i < outputs.length; i++) i: outputs[i].buffer},
    );
    expected = [for (final o in outputs) ...o];
  } catch (e) {
    return skip('reference Interpreter failed: $e');
  } finally {
    reference?.close();
    delegate?.delete();
  }

  final List<double> actual;
  try {
    actual = [
      for (final o in compiled.run([input])) ...o,
    ];
  } catch (e) {
    return BackendVerification(
      agrees: false,
      absoluteDeviation: double.infinity,
      outputRange: _range(expected),
      relativeDeviation: double.infinity,
      error: e,
    );
  }

  if (actual.length != expected.length) {
    return BackendVerification(
      agrees: false,
      absoluteDeviation: double.infinity,
      outputRange: _range(expected),
      relativeDeviation: double.infinity,
      skippedReason:
          'output length mismatch: CompiledModel produced ${actual.length} '
          'values, reference produced ${expected.length}',
    );
  }

  var deviation = 0.0;
  for (var i = 0; i < expected.length; i++) {
    final a = actual[i];
    // NaN never compares greater, so a NaN output would otherwise slip past a
    // running maximum and read as perfect agreement.
    if (a.isNaN != expected[i].isNaN ||
        a.isInfinite != expected[i].isInfinite) {
      deviation = double.infinity;
      break;
    }
    final d = (a - expected[i]).abs();
    if (d > deviation) deviation = d;
  }

  final range = _range(expected);
  // A constant reference output has no range to normalise against, so compare
  // against the magnitude of the values themselves instead of dividing by zero.
  final scale = range > 0
      ? range
      : expected.fold<double>(0, (m, v) => v.abs() > m ? v.abs() : m);
  final relative = scale > 0
      ? deviation / scale
      : (deviation == 0 ? 0.0 : double.infinity);

  return BackendVerification(
    agrees: relative <= tolerance,
    absoluteDeviation: deviation,
    outputRange: range,
    relativeDeviation: relative,
  );
}