Nosmai Flutter Plugin #

A Flutter plugin for integrating the Nosmai SDK - Real-time video filtering and beauty effects for iOS applications.

Features #

🎥 Real-time video processing with GPU acceleration
✨ Beauty filters (smoothing, whitening)
👤 Face reshape effects (slim face, eye enlargement)
💄 Makeup filters (lipstick, blusher, plump)
🎨 Artistic filters (toon, emboss, grayscale, etc.)
📱 Camera controls (front/back switching)
🔧 RGB adjustments with individual channel control
📁 Custom filter loading (.nosmai files)
🔍 Face detection integration
📡 Stream-based events for real-time callbacks
🏷️ Metadata-based filter categorization (beauty, effect, filter)

Platform Support #

Platform	Status
iOS	✅ Supported (iOS 11.0+)
Android	🚧 Planned

Requirements #

iOS: 11.0+
Flutter: 3.0.0+
Dart: 2.17.0+
Nosmai SDK: Compatible version

Installation #

Add this to your package's pubspec.yaml file:

dependencies:
  nosmai_flutter:
    path: path/to/nosmai_flutter

Setup #

iOS Setup #

Add camera permissions to your ios/Runner/Info.plist:

<key>NSCameraUsageDescription</key>
<string>This app uses the camera to apply real-time filters and beauty effects.</string>
<key>NSMicrophoneUsageDescription</key>
<string>This app may use the microphone for video recording with filters.</string>

Update your Podfile to include the Nosmai SDK:

# ios/Podfile
target 'Runner' do
  use_frameworks!
  use_modular_headers!
  
  flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))
  
  # The plugin will automatically configure header search paths
  # to find your Nosmai SDK in the parent project
end

Set iOS deployment target to 11.0+ in your ios/Runner.xcodeproj.

Framework Integration #

This plugin is designed to work with your existing Nosmai SDK. The plugin will automatically find the SDK headers at:

../../../src/sdk/include/NosmaiSDK.h
../../../src/sdk/include/NosmaiTypes.h

If your SDK is in a different location, update the header search paths in the ios/nosmai_camera_sdk.podspec file.

Usage #

Basic Setup #

import 'package:nosmai_flutter/nosmai_flutter.dart';

class CameraScreen extends StatefulWidget {
  @override
  _CameraScreenState createState() => _CameraScreenState();
}

class _CameraScreenState extends State<CameraScreen> {
  final NosmaiFlutter _nosmai = NosmaiFlutter.instance;
  bool _isInitialized = false;

  @override
  void initState() {
    super.initState();
    _initializeSDK();
    _setupErrorHandling();
  }

  void _setupErrorHandling() {
    _nosmai.onError.listen((error) {
      print('Nosmai Error: ${error.message}');
    });
  }

  Future<void> _initializeSDK() async {
    // Initialize with your license key
    final success = await _nosmai.initWithLicense('YOUR_LICENSE_KEY');
    
    if (success) {
      // Configure camera
      await _nosmai.configureCamera(
        position: NosmaiCameraPosition.front,
        sessionPreset: 'AVCaptureSessionPresetHigh',
      );
      
      // Enable face detection for beauty filters
      await _nosmai.setFaceDetectionEnabled(true);
      
      // Set up preview view
      await _nosmai.setPreviewView();
      
      setState(() {
        _isInitialized = true;
      });
    }
  }

  Future<void> _startProcessing() async {
    if (_isInitialized) {
      await _nosmai.startProcessing();
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Nosmai Camera')),
      body: Column(
        children: [
          // Your camera preview will be handled natively
          Expanded(
            child: Container(
              color: Colors.black,
              child: Center(
                child: Text(
                  'Camera Preview\n(Native iOS View)',
                  style: TextStyle(color: Colors.white),
                  textAlign: TextAlign.center,
                ),
              ),
            ),
          ),
          
          // Control buttons
          Row(
            mainAxisAlignment: MainAxisAlignment.spaceEvenly,
            children: [
              ElevatedButton(
                onPressed: _startProcessing,
                child: Text('Start'),
              ),
              ElevatedButton(
                onPressed: () => _nosmai.stopProcessing(),
                child: Text('Stop'),
              ),
            ],
          ),
        ],
      ),
    );
  }

  @override
  void dispose() {
    _nosmai.cleanup();
    super.dispose();
  }
}

Applying Filters #

Basic Filters

// Brightness adjustment
await _nosmai.applyFilter(
  filterType: NosmaiFilterType.brightness,
  value: 0.3, // -1.0 to 1.0
);

// Contrast adjustment
await _nosmai.applyFilter(
  filterType: NosmaiFilterType.contrast,
  value: 1.5, // 0.0 to 2.0
);

// Saturation adjustment
await _nosmai.applyFilter(
  filterType: NosmaiFilterType.saturation,
  value: 1.8, // 0.0 to 2.0
);

Beauty Filters

// Skin smoothing
await _nosmai.applyBeautyFilter(
  beautyType: NosmaiBeautyType.smoothing,
  intensity: 0.7, // 0.0 to 1.0
);

// Skin whitening
await _nosmai.applyBeautyFilter(
  beautyType: NosmaiBeautyType.whitening,
  intensity: 0.5, // 0.0 to 1.0
);

Face Reshape

// Face slimming and eye enlargement
await _nosmai.applyFaceReshape(
  slimLevel: 0.3, // 0.0 to 1.0
  eyeLevel: 0.2,  // 0.0 to 1.0
);

RGB Color Adjustments

// Adjust individual color channels
await _nosmai.applyRGBFilter(
  red: 1.2,   // 0.0 to 2.0
  green: 0.8, // 0.0 to 2.0
  blue: 1.1,  // 0.0 to 2.0
);

Makeup Effects

// Lipstick
await _nosmai.applyLipstick(
  intensity: 0.8,
  colorHex: '#FF1493', // Deep pink
);

// Blusher
await _nosmai.applyBlusher(
  intensity: 0.6,
  colorHex: '#FFB6C1', // Light pink
);

// Plump effect
await _nosmai.applyPlump(
  intensity: 0.5,
  cheekLevel: 0.8,
  lipLevel: 1.0,
);

Artistic Filters

// Toon effect
await _nosmai.applyFilter(
  filterType: NosmaiFilterType.toon,
  value: 0.8,
);

// Emboss effect
await _nosmai.applyFilter(
  filterType: NosmaiFilterType.emboss,
  value: 0.7,
);

Camera Controls #

// Switch between front and back camera
final success = await _nosmai.switchCamera();

// Remove all applied filters
await _nosmai.removeAllFilters();

// Load custom filter file
final loaded = await _nosmai.loadNosmaiFilter('/path/to/filter.nosmai');

Event Handling #

// Listen for errors
_nosmai.onError.listen((error) {
  print('Nosmai Error: ${error.message}');
  // Handle error appropriately
});

// Listen for face detection results
_nosmai.onFaceDetection.listen((faces) {
  print('Detected ${faces.length} faces');
  for (var face in faces) {
    print('Face ID: ${face.faceID}');
    print('Bounding box: ${face.boundingBox}');
  }
});

Metadata-Based Filter System #

The SDK now uses metadata-based filter categorization instead of name-based detection:

// Get all filters organized by category
final filtersByCategory = await _nosmai.organizeFiltersByCategory();

// Check if a filter is a beauty filter
final isBeauty = _nosmai.isBeautyFilter(filter);

// Get only beauty filters
final beautyFilters = await _nosmai.getFiltersByCategory(NosmaiFilterCategory.beauty);

// Apply filter based on metadata
for (final filter in filters) {
  if (_nosmai.isBeautyFilter(filter)) {
    // Beauty filters can be stacked
    await _nosmai.applyFilter(filter.path);
  } else {
    // Other filters replace existing effects
    await _nosmai.removeAllEffects();
    await _nosmai.applyFilter(filter.path);
  }
}

See METADATA_FILTER_GUIDE.md for detailed documentation.

API Reference #

NosmaiFlutter #

Main class for interacting with the Nosmai SDK.

Properties

bool isInitialized - Whether the SDK has been initialized
bool isProcessing - Whether video processing is active
Stream<List<NosmaiFaceInfo>> onFaceDetection - Stream of face detection results
Stream<NosmaiError> onError - Stream of error events

Methods

Initialization

Future<bool> initWithLicense(String licenseKey) - Initialize SDK with license
Future<void> configureCamera({required NosmaiCameraPosition position, String? sessionPreset}) - Configure camera
Future<void> setPreviewView() - Set up preview view
Future<void> cleanup() - Clean up resources

Processing Control

Future<void> startProcessing() - Start video processing
Future<void> stopProcessing() - Stop video processing

Filter Application

Future<void> applyFilter({required NosmaiFilterType filterType, required double value}) - Apply basic filter
Future<void> applyBeautyFilter({required NosmaiBeautyType beautyType, required double intensity}) - Apply beauty filter
Future<void> applyFaceReshape({required double slimLevel, required double eyeLevel}) - Apply face reshape
Future<void> applyRGBFilter({required double red, required double green, required double blue}) - Apply RGB filter
Future<void> applyLipstick({required double intensity, String? colorHex}) - Apply lipstick
Future<void> applyBlusher({required double intensity, String? colorHex}) - Apply blusher
Future<void> applyPlump({required double intensity, required double cheekLevel, required double lipLevel}) - Apply plump effect

Filter Management

Future<List<dynamic>> getFilters() - Get all available filters
bool isBeautyFilter(dynamic filter) - Check if a filter is a beauty filter using metadata
Future<List<dynamic>> getFiltersByCategory(NosmaiFilterCategory category) - Get filters by category
Future<Map<NosmaiFilterCategory, List<dynamic>>> organizeFiltersByCategory() - Organize all filters by category

Utility

Future<bool> loadNosmaiFilter(String filePath) - Load custom filter
Future<bool> switchCamera() - Switch camera
Future<void> setFaceDetectionEnabled(bool enable) - Enable/disable face detection
Future<void> removeAllFilters() - Remove all filters

Types #

NosmaiCameraPosition

front - Front-facing camera
back - Back-facing camera

NosmaiFilterType

brightness, contrast, saturation - Basic adjustments
rgb - RGB channel adjustments
toon, emboss - Artistic effects
plump, lut, png, sequence, nosmai - Advanced filters

NosmaiFilterCategory

beauty - Beauty enhancement filters (lipstick, face slimming, etc.)
effect - Creative/artistic effects (glitch, holographic, etc.)
filter - Standard filters (color adjustments, basic effects, etc.)
unknown - Unknown or uncategorized filters

NosmaiBeautyType

smoothing - Skin smoothing
whitening - Skin whitening

NosmaiFaceInfo

int faceID - Unique face identifier
Map<String, double> boundingBox - Face bounding box coordinates

NosmaiError

String code - Error code
String message - Error message
String? details - Additional error details

Example #

The example folder contains a comprehensive demo app showcasing all plugin features:

Real-time filter switching
Interactive parameter adjustment with sliders
Camera controls
Error handling and status monitoring
All filter types demonstration

To run the example:

cd example
flutter pub get
cd ios && pod install
cd .. && flutter run

Integration with Existing Project #

If you're integrating this plugin into an existing project that already uses the Nosmai SDK:

The plugin expects to find your SDK headers at the relative path shown in the podspec
Update the header search paths in ios/nosmai_flutter.podspec if needed
Ensure your license key is valid and matches the one used in your existing app
The plugin will work alongside your existing Nosmai implementation

Troubleshooting #

Common Issues #

SDK initialization fails
- Verify your license key is correct and active
- Check that the Nosmai SDK headers are found at the expected path
- Ensure iOS deployment target is 11.0+
Camera permission denied
- Add camera usage description to Info.plist
- Request permissions before initializing SDK
Build errors on iOS
- Clean build folder: flutter clean
- Update CocoaPods: cd ios && pod update
- Check header search paths in nosmai_camera_sdk.podspec
Framework not found
- Verify the relative path to your Nosmai SDK
- Check that all required frameworks are linked
- Ensure the SDK is built for the correct architecture

Performance Tips #

Initialize SDK once and reuse the instance
Stop processing when not needed to save battery
Use appropriate session presets for your use case
Remove filters when switching between different effect types
Enable face detection only when using beauty/face reshape filters

License #

This plugin is provided under the MIT License. However, the use of the underlying Nosmai SDK is subject to separate licensing terms and conditions.

To use this plugin, you must:

Obtain a valid license for the Nosmai SDK
Comply with all Nosmai SDK licensing terms
Include the Nosmai SDK framework in your application

Support #

For issues related to:

Plugin functionality: Create an issue in this repository
Nosmai SDK: Contact Nosmai support
Flutter integration: Check Flutter documentation

Contributing #

Fork the repository
Create a feature branch
Make your changes
Add tests if applicable
Submit a pull request

Version History #

1.0.0 #

Initial release
iOS platform support
Complete filter API implementation
Example app with comprehensive demos
Real Nosmai SDK integration
Stream-based event handling
Comprehensive documentation

nosmai_camera_sdk 1.0.0 nosmai_camera_sdk: ^1.0.0 copied to clipboard

Metadata