face_gesture_detector #

A Flutter widget that translates camera frames and MediaPipe facial landmarks into high-level semantic callbacks — exactly as GestureDetector translates pointer events into touch gestures.

Features #

• FaceGestureDetector — Declarative facade with nullable callbacks per gesture family
• RawFaceGestureDetector — Power-user widget for custom recognizer maps
• 10 recognizers — Presence, Quality, Pose, Blink, Smile, Mouth, Brow, HeadTurn, HeadNod, RawFrame
• Temporal logic — Sustained-duration thresholds, cycle detection, state-transition filtering
• FaceGestureDetectorController — Imperative pause / resume / reset
• Configurable — All thresholds and durations via FaceGestureConfiguration
• Native Android — MediaPipe Face Landmarker with GPU delegate, single-slot backpressure, Euler angle extraction

Quick Start #

import 'package:camera/camera.dart';
import 'package:face_gesture_detector/face_gesture_detector.dart';

// Minimal: presence detection only
FaceGestureDetector(
  configuration: FaceGestureConfiguration(),
  cameraController: cameraController,
  onFaceDetected: (details) => print('Face found: ${details.confidence}'),
  onFaceLost: () => print('Face lost'),
  child: CameraPreview(cameraController),
)

// Full: all gesture families
FaceGestureDetector(
  configuration: FaceGestureConfiguration(
    blinkThreshold: 0.35,
    smileThreshold: 0.5,
    headTurnYawThreshold: 25.0,
    sustainedGestureDuration: Duration(milliseconds: 400),
    frameSkipCount: 2,
  ),
  cameraController: cameraController,
  controller: detectorController,
  onFaceDetected: (d) => handleFace(d),
  onFaceLost: () => handleLost(),
  onBlinkDetected: (d) => handleBlink(d),
  onSmileDetected: (d) => handleSmile(d),
  onHeadTurnDetected: (d) => handleTurn(d),
  onHeadNodDetected: (d) => handleNod(d),
  onBrowRaised: (d) => handleBrow(d),
  onMouthOpened: (d) => handleMouth(d),
  onPoseChanged: (d) => handlePose(d),
  onDistanceChanged: (d) => handleDistance(d),
  onQualityChanged: (d) => handleQuality(d),
  onFaceFrame: (frame) => handleRawFrame(frame),
  child: CameraPreview(cameraController),
)

Architecture #

Five-layer stack following Flutter's gesture system analogy:

See doc/architecture.md for full details.

Status #

v0.2.0 — Full end-to-end face gesture detection on Android. Pass a CameraController and receive callbacks.

• ✅ Data model with 52 blendshapes, landmarks, pose angles
• ✅ Platform interface (MethodChannel + EventChannel + processFrame)
• ✅ 10 recognizers with full test coverage
• ✅ Layer 2 widget with frame routing, recognizer diff, controller
• ✅ Layer 1 facade with callback-to-recognizer mapping
• ✅ Native Android — MediaPipe Face Landmarker (LIVE_STREAM + GPU delegate)
• ✅ Camera integration — cameraController parameter wires the full pipeline
• ✅ Example app — live camera preview with real-time gesture detection
• ❌ Native iOS