kwon_mediapipe_landmarker #

A Flutter plugin for real-time face and pose landmark detection using MediaPipe. Designed for interview practice apps, presentation coaching, and any application requiring facial expression, eye contact, and posture analysis.

Features #

• Face Landmarker: 478 facial landmarks with 52 ARKit-compatible blendshapes
• Pose Landmarker: 33 body pose landmarks with visibility and world coordinates
• Real-time Camera Analysis: Optimized native YUV-to-RGB conversion for smooth performance
• Rich Helper Extensions: 14 face analysis methods + 19 pose analysis methods
• Simultaneous Detection: Run face and pose detection together

Performance #

Android (Samsung SM-S938N, Snapdragon 8 Gen 3) #

iOS (iPhone 16, A18 GPU) #

Installation #

Add to your pubspec.yaml:

dependencies:
  kwon_mediapipe_landmarker: ^0.0.1

Android Setup #

1. Set minimum SDK version in android/app/build.gradle:

android {
    defaultConfig {
        minSdkVersion 24
    }
}

2. Add camera permission to AndroidManifest.xml:

<uses-permission android:name="android.permission.CAMERA" />

iOS Setup #

1. Set minimum deployment target in ios/Podfile:

platform :ios, '12.0'

2. Add camera permission to ios/Runner/Info.plist:

<key>NSCameraUsageDescription</key>
<string>Camera access is required for face and pose detection.</string>

Quick Start #

Initialization #

import 'package:kwon_mediapipe_landmarker/kwon_mediapipe_landmarker.dart';

// Initialize with both face and pose detection
await KwonMediapipeLandmarker.initialize(
  face: true,
  pose: true,
  faceOptions: const FaceOptions(
    numFaces: 1,
    minDetectionConfidence: 0.5,
    minTrackingConfidence: 0.5,
    outputBlendshapes: true,
    outputTransformationMatrix: false,
  ),
  poseOptions: const PoseOptions(
    numPoses: 1,
    minDetectionConfidence: 0.5,
    minTrackingConfidence: 0.5,
  ),
);

Single Image Detection #

import 'dart:io';

final imageBytes = await File('path/to/image.jpg').readAsBytes();
final result = await KwonMediapipeLandmarker.detect(imageBytes);

if (result.hasFace) {
  print('Smile score: ${result.face!.smileScore}');
  print('Eye contact: ${result.face!.eyeContactScore}');
}

if (result.hasPose) {
  print('Posture score: ${result.pose!.postureScore}');
}

Camera Stream Detection #

import 'package:camera/camera.dart';

// Start camera stream
controller.startImageStream((CameraImage image) async {
  final result = await KwonMediapipeLandmarker.detectFromCamera(
    planes: image.planes.map((p) => p.bytes).toList(),
    width: image.width,
    height: image.height,
    rotation: controller.description.sensorOrientation,
    format: 'YUV420',
    bytesPerRow: image.planes.map((p) => p.bytesPerRow).toList(),
  );

  // Process result
  if (result.hasFace) {
    setState(() => _faceResult = result.face);
  }
});

Cleanup #

@override
void dispose() {
  KwonMediapipeLandmarker.dispose();
  super.dispose();
}

API Reference #

KwonMediapipeLandmarker #

FaceOptions #

PoseOptions #

LandmarkerResult #

FaceResult #

PoseResult #

Extensions (Helpers) #

FaceResultHelper (14 methods) #

Usage:

if (result.hasFace) {
  final face = result.face!;

  // Check eye contact
  if (face.eyeContactScore > 0.8) {
    print('Good eye contact!');
  }

  // Check smile
  if (face.smileScore > 0.5) {
    print('Nice smile!');
  }

  // Detect tension
  if (face.tensionScore > 0.6) {
    print('You seem tense. Try to relax.');
  }
}

PoseResultHelper (19 methods) #

Usage:

if (result.hasPose) {
  final pose = result.pose!;

  // Check posture
  if (pose.postureScore < 0.6) {
    print('Try to straighten your posture');
  }

  // Detect nervous gestures
  if (pose.isHandNearFace) {
    print('Avoid touching your face');
  }

  // Check positioning
  if (pose.centerPositionScore < 0.7) {
    print('Try to center yourself in the frame');
  }
}

Constants #

FaceLandmarkIndex (Key Points) #

PoseLandmarkIndex (All 33 Points) #

FaceBlendshape (52 Blendshapes) #

Use Cases #

Interview Practice App #

class InterviewFeedback {
  void analyze(LandmarkerResult result) {
    final feedback = <String>[];

    if (result.hasFace) {
      final face = result.face!;

      // Eye contact feedback
      if (face.eyeContactScore < 0.6) {
        feedback.add('Maintain better eye contact with the camera');
      }

      // Expression feedback
      if (face.smileScore < 0.3 && face.tensionScore > 0.4) {
        feedback.add('Try to relax and smile occasionally');
      }
    }

    if (result.hasPose) {
      final pose = result.pose!;

      // Posture feedback
      if (pose.isShoulderTensed) {
        feedback.add('Relax your shoulders');
      }

      if (pose.postureScore < 0.7) {
        feedback.add('Sit up straight');
      }

      // Gesture feedback
      if (pose.isHandNearFace) {
        feedback.add('Avoid touching your face');
      }

      if (pose.isArmsCrossed) {
        feedback.add('Uncross your arms for more open body language');
      }
    }

    return feedback;
  }
}

Presentation Coach #

class PresentationScore {
  double calculate(LandmarkerResult result) {
    double score = 0;
    int factors = 0;

    if (result.hasFace) {
      final face = result.face!;
      score += face.eyeContactScore;
      score += face.naturalExpressionScore;
      score += face.symmetryScore;
      factors += 3;
    }

    if (result.hasPose) {
      final pose = result.pose!;
      score += pose.postureScore;
      score += pose.centerPositionScore;
      score += pose.shoulderSymmetryScore;
      factors += 3;
    }

    return factors > 0 ? score / factors : 0;
  }
}

Real-time Feedback UI #

Widget buildFeedbackOverlay(LandmarkerResult result) {
  return Stack(
    children: [
      // Eye contact indicator
      if (result.hasFace)
        Positioned(
          top: 10,
          left: 10,
          child: Row(
            children: [
              Icon(
                Icons.visibility,
                color: result.face!.eyeContactScore > 0.7
                    ? Colors.green
                    : Colors.orange,
              ),
              SizedBox(width: 8),
              Text('${(result.face!.eyeContactScore * 100).toInt()}%'),
            ],
          ),
        ),

      // Posture indicator
      if (result.hasPose)
        Positioned(
          top: 10,
          right: 10,
          child: Row(
            children: [
              Icon(
                Icons.accessibility_new,
                color: result.pose!.postureScore > 0.7
                    ? Colors.green
                    : Colors.orange,
              ),
              SizedBox(width: 8),
              Text('${(result.pose!.postureScore * 100).toInt()}%'),
            ],
          ),
        ),
    ],
  );
}

Error Handling #

The plugin provides structured error handling through LandmarkerException:

try {
  await KwonMediapipeLandmarker.initialize(face: true);
} on LandmarkerException catch (e) {
  switch (e.error) {
    case LandmarkerError.notInitialized:
      print('Landmarker not initialized');
      break;
    case LandmarkerError.modelLoadFailed:
      print('Failed to load model files');
      break;
    case LandmarkerError.invalidImage:
      print('Invalid image format');
      break;
    case LandmarkerError.detectionFailed:
      print('Detection failed');
      break;
    case LandmarkerError.initializationFailed:
      print('Initialization failed: ${e.message}');
      break;
    default:
      print('Error: ${e.code} - ${e.message}');
  }
}

Troubleshooting #

Common Issues #

1. "NOT_INITIALIZED" error

• Make sure to call KwonMediapipeLandmarker.initialize() before using detect() or detectFromCamera()
• Check that initialization completed successfully

2. "MODEL_LOAD_FAILED" error

• Verify that model files exist in the correct locations:
  • Android: android/src/main/assets/face_landmarker.task, pose_landmarker_lite.task
  • iOS: ios/Assets/face_landmarker.task, pose_landmarker_lite.task

3. "INVALID_IMAGE" error

• Ensure image data is in a supported format (JPEG, PNG)
• For camera frames, verify YUV data planes are correctly passed

4. Low FPS on Android

• This is expected due to MediaPipe running on CPU on Android
• Consider reducing resolution or using face-only mode for better performance

5. Camera permission denied

• Add camera permission to AndroidManifest.xml and Info.plist
• Request permission at runtime using permission_handler package

6. MediaPipe warnings in console

• Warnings like landmark_projection_calculator.cc:186 are internal MediaPipe messages and can be safely ignored

Platform-Specific Notes #

Android:

• Minimum SDK: 24
• Uses native YUV-to-RGB conversion for performance
• Performance varies by device GPU/CPU

iOS:

• Minimum iOS: 12.0
• Uses Core Video for optimal frame processing
• Significantly faster than Android due to GPU acceleration

License #

Copyright 2026 kwon

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.