Multi-Model Pipelines

Many real-world AI applications require chaining multiple models together. For example, face landmark detection requires first detecting faces and then analyzing landmarks within each detected face. This guide shows how to build multi-model pipelines with Melange.

Pipeline Pattern

The basic pattern for multi-model pipelines is:

Initialize all models.
Run the first model and postprocess its outputs.
Use the processed outputs as inputs for the next model.
Repeat until the pipeline is complete.

Example: Face Detection + Face Landmark

This pipeline first detects faces in an image, then extracts landmarks from each detected face.

Step 1: Face Detection

// Initialize face detection model
val faceDetectionModel = ZeticMLangeModel(this, PERSONAL_KEY, "face_detection")
val faceDetection = FaceDetectionWrapper()

// Preprocess image
val faceDetectionInputs = faceDetection.preprocess(bitmap)

// Run face detection
faceDetectionModel.run(faceDetectionInputs)
val faceDetectionOutputs = faceDetectionModel.outputBuffers

// Postprocess to get face regions
val faceRegions = faceDetection.postprocess(faceDetectionOutputs)

Step 2: Face Landmark

// Initialize face landmark model
val faceLandmarkModel = ZeticMLangeModel(this, PERSONAL_KEY, "face_landmark")
val faceLandmark = FaceLandmarkWrapper()

// Preprocess with detected face regions
val faceLandmarkInputs = faceLandmark.preprocess(bitmap, faceRegions)

// Run face landmark model
faceLandmarkModel.run(faceLandmarkInputs)
val faceLandmarkOutputs = faceLandmarkModel.outputBuffers

// Postprocess to get landmarks
val landmarks = faceLandmark.postprocess(faceLandmarkOutputs)

Step 1: Face Detection

// Initialize face detection model
let faceDetectionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "face_detection")
let faceDetection = FaceDetectionWrapper()

// Preprocess image
let faceDetectionInputs = faceDetection.preprocess(image)

// Run face detection
try faceDetectionModel.run(faceDetectionInputs)
var faceDetectionOutputs = faceDetectionModel.getOutputDataArray()

// Postprocess to get face regions
let faceRegions = faceDetection.postprocess(&faceDetectionOutputs)

Step 2: Face Landmark

// Initialize face landmark model
let faceLandmarkModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "face_landmark")
let faceLandmark = FaceLandmarkWrapper()

// Preprocess with detected face regions
let faceLandmarkInputs = faceLandmark.preprocess(image, faceRegions)

// Run face landmark model
try faceLandmarkModel.run(faceLandmarkInputs)
var faceLandmarkOutputs = faceLandmarkModel.getOutputDataArray()

// Postprocess to get landmarks
let landmarks = faceLandmark.postprocess(&faceLandmarkOutputs)

Pipeline Design Tips

Initialize models once: Model initialization involves downloading and NPU context creation. Do this once at app startup, not per inference call.
Reuse model instances: The same model instance can be used for multiple run() calls.
Process on background threads: Pipeline execution involves multiple inference calls and should run off the main thread.
Handle pipeline failures: If an earlier stage produces no results (e.g., no faces detected), skip subsequent stages gracefully.

For complete pipeline implementations, see the ZETIC Melange Apps repository, which includes Face Detection, Face Landmark, Face Emotion Recognition, and more.

Next Steps

Custom Preprocessing: Implement input preprocessing
Performance Optimization: Optimize pipeline performance
Tutorials: Complete end-to-end examples

Multi-Model Pipelines

Pipeline Pattern

Example: Face Detection + Face Landmark

Pipeline Design Tips

Next Steps

On this page