# ZETIC Melange (/) Deploy any AI model to iOS and Android with automatic NPU acceleration. No hardware expertise required. Just upload your model, integrate the SDK, and ship. We use the name **Melange** for the product. However, you will see `MLange` in the source code and library names (e.g., `ZeticMLangeModel`). *** Quick Start [#quick-start] Generate Model Key and Personal Key [#generate-model-key-and-personal-key] Go to [Melange Dashboard](https://melange.zetic.ai) to generate your keys.
1\. Upload & Auto-Compilation [#1-upload--auto-compilation] Simply upload your model file to our dashboard. **Melange** automatically analyzes, quantizes, and compiles the graph for heterogeneous NPU targets in the background. * Supporting model format: * Pytorch Exported Program(`.pt2`) * Onnx Model(`.onnx`) Generate Model Key
2\. Get Your Deployment Keys [#2-get-your-deployment-keys] Once optimized, specific keys are provisioned for your project. * **Model Key**: The unique identifier for your hardware-accelerated binary. * **Personal Key**: Your secure credential for on-device authentication. Copy Personal Key
For comprehensive details on key provisioning and security policies, please consult: * [Personal Key: Sign up to Melange Dashboard](/quick-start/prerequisites) * [Model Key: Deploy your model](/model-deployment/dashboard) The Melange Dashboard provides **ready-to-use source code** with your keys already pre-filled. You can simply copy and paste it directly into your project! Integrate SDK & Run Inference [#integrate-sdk--run-inference] Initialize the **ZeticMLangeModel** with your keys to trigger hardware-accelerated execution. ```kotlin val model = ZeticMLangeModel(this, PERSONAL_KEY, "Steve/YOLOv11_comparison") val outputs = model.run(inputs) ``` ```swift let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "Steve/YOLOv11_comparison") let outputs = try model.run(inputs: inputs) ``` ```dart final model = await ZeticMLangeModel.create( personalKey: personalKey, name: 'Steve/YOLOv11_comparison', ); final outputs = model.run(inputs); ``` The demo model key `Steve/YOLOv11_comparison` works without an account. Try it now in the [Quick Start](/quick-start). *** Why Melange? [#why-melange] No manual driver management. Melange handles hardware abstraction across Qualcomm, MediaTek, Samsung, and Apple chipsets. One API for Android and iOS. Write your integration once and deploy everywhere. From model upload to on-device execution. Automatic graph optimization, quantization, and compilation. Ship AI features in hours, not months. No C++, no OpenCL, no Metal shaders. Performance [#performance] > **YOLOv11n on iPhone 16: CPU 102ms โ†’ NPU 1.9ms (54x faster)** Melange automatically selects the optimal hardware accelerator for each device and model, delivering peak performance without manual tuning. See [full benchmark results](/performance/benchmarks). *** Platform Support [#platform-support] | Platform | SDK | NPU Targets | | ----------- | ----------------------- | ------------------------------------------------------------------------------------------- | | **Android** | Kotlin / Java via Maven | Qualcomm HTP/DSP, Google Tensor, MediaTek APU (Enterprise), Samsung Exynos DSP (Enterprise) | | **iOS** | Swift via SPM | Apple Neural Engine (A11+) | Supported Model Formats [#supported-model-formats] | Format | Extension | Status | | ------------------------ | --------- | --------------- | | ONNX | `.onnx` | Fully supported | | PyTorch Exported Program | `.pt2` | Fully supported | *** Get Started [#get-started] Run your first on-device inference with a demo model. No account required. Step-by-step guides for object detection, face detection, speech recognition, and more. Prepare and export your model in ONNX or PyTorch Exported Program format. Upload your model on the Melange Dashboard and get your keys. Integrate the Melange SDK into your Android or iOS app. Full SDK documentation for Android and iOS. Tutorials [#tutorials] Step-by-step guides for common on-device AI use cases: Real-time object detection with YOLOv8/YOLOv11. Detect faces in camera frames using MediaPipe. Classify facial expressions in real-time. On-device speech-to-text with OpenAI Whisper. Classify environmental sounds on-device. Example Applications [#example-applications] Explore complete working examples for Android and iOS: Full source code for YOLOv11, Face Detection, Face Landmark, Face Emotion Recognition, Whisper, and more. *** Need Help? [#need-help] We are developing rapidly and welcome all questions and feedback. Join our developer community for real-time help. Open-source repos, SDKs, and sample apps. Reach us for technical support. # Cache Management (/api-reference/cache-management) This page is a placeholder. Detailed cache management documentation is still being prepared. This page will be expanded to cover managed model cache behavior, `ModelCacheHandlingPolicy`, and `ModelCacheManager`. Current Scope [#current-scope] For now, the main distinction is: * `cacheHandlingPolicy` controls managed model artifacts stored on disk * `kvCacheCleanupPolicy` controls the in-memory LLM conversation KV cache Do not treat them as the same setting. ModelCacheHandlingPolicy [#modelcachehandlingpolicy] The full behavior of overlapping aliases, artifact retention, and cache cleanup policy combinations is still `TBD`. Until this page is expanded, the safest interpretation is: * `REMOVE_OVERLAPPING`: prefer replacing overlapping managed cache entries for the same model selection flow * `KEEP_EXISTING`: prefer leaving existing managed cache entries in place ModelCacheManager [#modelcachemanager] The SDK now includes a managed cache utility object, but detailed usage examples are still `TBD`. Android [#android] Android exposes `ModelCacheManager` for managed cache deletion and pruning operations. Current public operations include: * `removeGeneral(...)` * `removeLlm(...)` * `removeHf(...)` * `removeAll()` * `prune()` iOS [#ios] iOS also exposes `ModelCacheManager` with corresponding managed cache deletion and pruning operations. Current public operations include: * `removeGeneral(...)` * `removeLlm(...)` * `removeHf(...)` * `removeAll()` * `prune()` Planned Expansion [#planned-expansion] This page will later include: * exact `ModelCacheHandlingPolicy` semantics * managed alias and artifact lifecycle * `ModelCacheManager` examples * platform-specific notes for Android and iOS See Also [#see-also] * [ZeticMLangeLLMModel (Android)](/api-reference/android/ZeticMLangeLLMModel) * [ZeticMLangeLLMModel (iOS)](/api-reference/ios/ZeticMLangeLLMModel) * [Enums and Constants (Android)](/api-reference/android/enums-and-constants) * [Enums and Constants (iOS)](/api-reference/ios/enums-and-constants) # Error Codes (/api-reference/error-codes) This page documents error codes you may encounter when using the ZETIC Melange SDK. SDK Runtime Errors [#sdk-runtime-errors] Authentication Errors [#authentication-errors] | Error | Platform | Cause | Solution | | ------------------------ | -------- | --------------------------------- | -------------------------------------------------------- | | Model not found | Android | Invalid model key or personal key | Verify keys on the [Dashboard](https://melange.zetic.ai) | | Failed to download model | iOS | Invalid key or network failure | Check keys and network connectivity | | HTTP 401 / 403 | Both | Authentication failure | Regenerate your personal key | Inference Errors [#inference-errors] | Error | Platform | Cause | Solution | | -------------------- | -------- | ------------------------------------------------------- | ---------------------------------------------- | | Input shape mismatch | Both | Input tensor dimensions do not match model expectations | Check expected shapes on the Dashboard | | UnsatisfiedLinkError | Android | JNI libraries not extracted correctly | Add `useLegacyPackaging true` to Gradle config | Compilation Errors [#compilation-errors] | Error | Context | Cause | Solution | | --------------------- | ------------ | ------------------------------ | --------------------------------------- | | Unsupported operation | Model upload | Model contains unsupported ops | Simplify model or use a different opset | | Conversion failed | Model upload | General compilation failure | Check model format and try `onnxsim` | For detailed troubleshooting steps for each error type, see: * [Common Errors](/troubleshooting/common-errors) * [Android Issues](/troubleshooting/android-issues) * [iOS Issues](/troubleshooting/ios-issues) * [Model Conversion Issues](/troubleshooting/model-conversion-issues) *** Getting Help [#getting-help] If you encounter an error not listed here: * Check the [FAQ](/troubleshooting/faq) * Join the [Discord community](https://discord.gg/q6vW4UscRY) * Email [contact@zetic.ai](mailto:contact@zetic.ai) # Custom Preprocessing (/how-to-guides/custom-preprocessing) Most AI models require input preprocessing before inference: resizing images, normalizing pixel values, tokenizing text, or converting audio samples. This guide covers common preprocessing patterns for ZETIC Melange. Image Preprocessing [#image-preprocessing] Vision models typically expect inputs in a specific format (e.g., `[1, 3, 640, 640]` in NCHW layout with normalized pixel values). ```kotlin import android.graphics.Bitmap fun preprocessImage(bitmap: Bitmap, targetWidth: Int, targetHeight: Int): FloatArray { // Resize the image val resized = Bitmap.createScaledBitmap(bitmap, targetWidth, targetHeight, true) // Convert to float array with normalization (0.0 to 1.0) val pixels = IntArray(targetWidth * targetHeight) resized.getPixels(pixels, 0, targetWidth, 0, 0, targetWidth, targetHeight) val floatArray = FloatArray(3 * targetWidth * targetHeight) for (i in pixels.indices) { val pixel = pixels[i] // NCHW layout: separate R, G, B channels floatArray[i] = ((pixel shr 16) and 0xFF) / 255.0f // R floatArray[i + pixels.size] = ((pixel shr 8) and 0xFF) / 255.0f // G floatArray[i + 2 * pixels.size] = (pixel and 0xFF) / 255.0f // B } return floatArray } ``` ```swift import UIKit import CoreGraphics func preprocessImage(_ image: UIImage, targetSize: CGSize) -> [Float] { // Resize the image UIGraphicsBeginImageContextWithOptions(targetSize, false, 1.0) image.draw(in: CGRect(origin: .zero, size: targetSize)) let resized = UIGraphicsGetImageFromCurrentImageContext()! UIGraphicsEndImageContext() // Convert to float array with normalization guard let cgImage = resized.cgImage else { return [] } let width = Int(targetSize.width) let height = Int(targetSize.height) var pixelData = [UInt8](repeating: 0, count: width * height * 4) let context = CGContext( data: &pixelData, width: width, height: height, bitsPerComponent: 8, bytesPerRow: width * 4, space: CGColorSpaceCreateDeviceRGB(), bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue ) context?.draw(cgImage, in: CGRect(x: 0, y: 0, width: width, height: height)) // Normalize to 0.0-1.0 var floatArray = [Float](repeating: 0, count: 3 * width * height) for i in 0..<(width * height) { floatArray[i] = Float(pixelData[i * 4]) / 255.0 // R floatArray[i + width * height] = Float(pixelData[i * 4 + 1]) / 255.0 // G floatArray[i + 2 * width * height] = Float(pixelData[i * 4 + 2]) / 255.0 // B } return floatArray } ``` ```dart import 'dart:typed_data'; Float32List preprocessImageRgba( Uint8List rgbaPixels, int targetWidth, int targetHeight, ) { final planeSize = targetWidth * targetHeight; final values = Float32List(3 * planeSize); for (var i = 0; i < planeSize; i++) { final rgbaOffset = i * 4; values[i] = rgbaPixels[rgbaOffset] / 255.0; values[i + planeSize] = rgbaPixels[rgbaOffset + 1] / 255.0; values[i + 2 * planeSize] = rgbaPixels[rgbaOffset + 2] / 255.0; } return values; } ``` Different models expect different input formats. Some models use NCHW layout (batch, channels, height, width) while others use NHWC (batch, height, width, channels). Check your model's specification. Audio Preprocessing [#audio-preprocessing] Audio models like Whisper typically expect specific sample rates and formats. ```python # Example: preparing audio input for upload import numpy as np import librosa # Load and resample audio to 16kHz audio, sr = librosa.load("audio.wav", sr=16000) # Convert to numpy and save np_audio = audio.astype(np.float32) np.save("audio_input.npy", np_audio) ``` Common Preprocessing Patterns [#common-preprocessing-patterns] | Model Type | Common Preprocessing | | -------------------- | --------------------------------------------------------- | | Image classification | Resize, normalize (0-1 or ImageNet mean/std), NCHW layout | | Object detection | Resize to fixed size, normalize, add batch dimension | | Audio classification | Resample to target rate, convert to mel spectrogram | | Text / NLP | Tokenize, pad to fixed length, create attention masks | Feature Extractor Modules [#feature-extractor-modules] For common model architectures, Melange provides feature extractor modules that handle preprocessing automatically. See the [ZETIC Melange Apps](https://github.com/zetic-ai/ZETIC_Melange_apps) repository for examples. *** Next Steps [#next-steps] * [Multi-Model Pipelines](/how-to-guides/multi-model-pipeline): Chain models together * [Basic Inference (Android)](/platform-integration/android/basic-inference): Android inference guide * [Basic Inference (iOS)](/platform-integration/ios/basic-inference): iOS inference guide # Inference Mode Selection (/how-to-guides/inference-mode-selection) Melange provides several inference modes for general (non-LLM) models to balance between speed and accuracy based on your application's requirements. Available Modes [#available-modes] Default / Auto (`RUN_AUTO`) [#default--auto-run_auto] Intelligently balances speed and accuracy for optimal performance. This mode automatically selects the fastest configuration while ensuring high-quality results (SNR > 20dB). This is the recommended mode for most use cases. Speed-First (`RUN_SPEED`) [#speed-first-run_speed] Maximizes inference speed with minimum latency. Recommended for real-time applications where response time is the top priority. Accuracy-First (`RUN_ACCURACY`) [#accuracy-first-run_accuracy] Delivers the highest precision based on maximum SNR scores. Best suited for applications where accuracy is more critical than speed. The optimal mode is automatically determined based on: * **Speed metrics**: Inference time (latency in ms) * **Accuracy metrics**: SNR (Signal-to-Noise Ratio in dB) You can override this automatic selection by explicitly specifying a mode. API Usage [#api-usage] ```kotlin // Default: Auto mode // Speed first, but maintains SNR above 20dB val modelDefault = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = ModelMode.RUN_AUTO ) // Speed First Mode val modelFast = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = ModelMode.RUN_SPEED ) // Accuracy First Mode val modelAccurate = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = ModelMode.RUN_ACCURACY ) ``` ```swift // Default: Auto mode // Speed first, but maintains SNR above 20dB let modelDefault = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_AUTO ) // Speed First Mode let modelFast = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_SPEED ) // Accuracy First Mode let modelAccurate = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_ACCURACY ) ``` ```dart // Default: Auto mode // Speed first, but maintains SNR above 20dB final modelDefault = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, modelMode: ModelMode.runAuto, ); // Speed First Mode final modelFast = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, modelMode: ModelMode.runSpeed, ); // Accuracy First Mode final modelAccurate = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, modelMode: ModelMode.runAccuracy, ); ``` Choosing the Right Mode [#choosing-the-right-mode] | Use Case | Recommended Mode | Why | | -------------------------- | ---------------- | --------------------------------- | | Real-time video processing | `RUN_SPEED` | Minimize frame processing latency | | Medical image analysis | `RUN_ACCURACY` | Precision is critical | | General mobile app | `RUN_AUTO` | Best balance for most users | | Prototype / testing | `RUN_AUTO` | Good default behavior | *** Next Steps [#next-steps] * [LLM Inference Modes](/llm-inference/inference-modes): Modes specific to LLM models * [Performance Optimization](/how-to-guides/performance-optimization): Additional tuning tips * [Performance-Adaptive Deployment](/model-deployment/performance-adaptive-deployment): How Melange selects optimal binaries # Multi-Model Pipelines (/how-to-guides/multi-model-pipeline) Many real-world AI applications require chaining multiple models together. For example, face landmark detection requires first detecting faces and then analyzing landmarks within each detected face. This guide shows how to build multi-model pipelines with Melange. Pipeline Pattern [#pipeline-pattern] The basic pattern for multi-model pipelines is: 1. Initialize all models. 2. Run the first model and postprocess its outputs. 3. Use the processed outputs as inputs for the next model. 4. Repeat until the pipeline is complete. Example: Face Detection + Face Landmark [#example-face-detection--face-landmark] This pipeline first detects faces in an image, then extracts landmarks from each detected face. **Step 1: Face Detection** ```kotlin // Initialize face detection model val faceDetectionModel = ZeticMLangeModel(this, PERSONAL_KEY, "face_detection") val faceDetection = FaceDetectionWrapper() // Preprocess image val faceDetectionInputs = faceDetection.preprocess(bitmap) // Run face detection val faceDetectionOutputs = faceDetectionModel.run(faceDetectionInputs) // Postprocess to get face regions val faceRegions = faceDetection.postprocess(faceDetectionOutputs) ``` **Step 2: Face Landmark** ```kotlin // Initialize face landmark model val faceLandmarkModel = ZeticMLangeModel(this, PERSONAL_KEY, "face_landmark") val faceLandmark = FaceLandmarkWrapper() // Preprocess with detected face regions val faceLandmarkInputs = faceLandmark.preprocess(bitmap, faceRegions) // Run face landmark model val faceLandmarkOutputs = faceLandmarkModel.run(faceLandmarkInputs) // Postprocess to get landmarks val landmarks = faceLandmark.postprocess(faceLandmarkOutputs) ``` **Step 1: Face Detection** ```swift // Initialize face detection model let faceDetectionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "face_detection") let faceDetection = FaceDetectionWrapper() // Preprocess image let faceDetectionInputs = faceDetection.preprocess(image) // Run face detection let faceDetectionOutputs = try faceDetectionModel.run(inputs: faceDetectionInputs) // Postprocess to get face regions let faceRegions = faceDetection.postprocess(faceDetectionOutputs) ``` **Step 2: Face Landmark** ```swift // Initialize face landmark model let faceLandmarkModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "face_landmark") let faceLandmark = FaceLandmarkWrapper() // Preprocess with detected face regions let faceLandmarkInputs = faceLandmark.preprocess(image, faceRegions) // Run face landmark model let faceLandmarkOutputs = try faceLandmarkModel.run(inputs: faceLandmarkInputs) // Postprocess to get landmarks let landmarks = faceLandmark.postprocess(faceLandmarkOutputs) ``` **Step 1: Face Detection** ```dart // Initialize face detection model final faceDetectionModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'face_detection', ); final faceDetection = FaceDetectionPipeline(); // Preprocess image final faceDetectionInputs = faceDetection.preprocess(image); // Run face detection final faceDetectionOutputs = faceDetectionModel.run(faceDetectionInputs); // Postprocess to get face regions final faceRegions = faceDetection.postprocess(faceDetectionOutputs); ``` **Step 2: Face Landmark** ```dart // Initialize face landmark model final faceLandmarkModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'face_landmark', ); final faceLandmark = FaceLandmarkPipeline(); // Preprocess with detected face regions final faceLandmarkInputs = faceLandmark.preprocess(image, faceRegions); // Run face landmark model final faceLandmarkOutputs = faceLandmarkModel.run(faceLandmarkInputs); // Postprocess to get landmarks final landmarks = faceLandmark.postprocess(faceLandmarkOutputs); ``` Pipeline Design Tips [#pipeline-design-tips] * **Initialize models once**: Model initialization involves downloading and NPU context creation. Do this once at app startup, not per inference call. * **Reuse model instances**: The same model instance can be used for multiple `run()` calls. * **Process on background threads**: Pipeline execution involves multiple inference calls and should run off the main thread. * **Handle pipeline failures**: If an earlier stage produces no results (e.g., no faces detected), skip subsequent stages gracefully. For complete pipeline implementations, see the [ZETIC Melange Apps](https://github.com/zetic-ai/ZETIC_Melange_apps) repository, which includes Face Detection, Face Landmark, Face Emotion Recognition, and more. *** Next Steps [#next-steps] * [Custom Preprocessing](/how-to-guides/custom-preprocessing): Implement input preprocessing * [Performance Optimization](/how-to-guides/performance-optimization): Optimize pipeline performance * [Tutorials](/tutorials/object-detection-yolo): Complete end-to-end examples # Performance Optimization (/how-to-guides/performance-optimization) This guide covers strategies for maximizing the performance of your on-device AI applications with ZETIC Melange. Inference Mode Selection [#inference-mode-selection] The most impactful optimization is choosing the right inference mode. See [Inference Mode Selection](/how-to-guides/inference-mode-selection) for details. | Mode | Trade-off | | -------------- | ---------------------------------------------- | | `RUN_SPEED` | Fastest inference, may sacrifice some accuracy | | `RUN_AUTO` | Balanced: fast while maintaining SNR > 20dB | | `RUN_ACCURACY` | Highest precision, may be slower | Model Format Selection [#model-format-selection] * **Simplify ONNX models** with `onnxsim` before uploading to reduce redundant operations. ```bash pip install onnxsim onnxsim input_model.onnx output_model.onnx ``` Input Optimization [#input-optimization] * **Use fixed input shapes.** Dynamic shapes prevent NPU compilation. Export with static dimensions. * **Match expected input sizes.** Do not upload inputs larger than necessary. Smaller inputs mean faster inference. * **Use Float32 inputs.** Melange handles quantization internally: provide full-precision inputs. Runtime Best Practices [#runtime-best-practices] Initialize Once, Run Many [#initialize-once-run-many] Model initialization involves downloading and NPU context creation. Do this once and reuse the model instance: ```kotlin // Do this once val model = ZeticMLangeModel(context, PERSONAL_KEY, MODEL_NAME) // Reuse for multiple inferences for (frame in videoFrames) { val outputs = model.run(preprocessFrame(frame)) } ``` Background Threading [#background-threading] Always run inference on a background thread to keep the UI responsive: ```kotlin lifecycleScope.launch(Dispatchers.IO) { val outputs = model.run(inputs) withContext(Dispatchers.Main) { updateUI(outputs) } } ``` ```swift DispatchQueue.global().async { let outputs = try? model.run(inputs: inputs) DispatchQueue.main.async { self.updateUI(outputs) } } ``` ```dart Future runInference() async { final outputs = await Future(() => model.run(inputs)); if (!context.mounted) return; updateUI(outputs); } ``` Minimize Preprocessing Overhead [#minimize-preprocessing-overhead] Preprocessing (image resize, normalization) can become a bottleneck. Profile your preprocessing code alongside inference time. Device Considerations [#device-considerations] * **Physical devices only.** Emulators and simulators do not have NPU hardware. * **Keep firmware updated.** NPU driver updates can improve performance. * **Test on target devices.** Performance varies significantly across chipsets. Melange automatically selects the optimal compiled binary for each device through [Performance-Adaptive Deployment](/model-deployment/performance-adaptive-deployment). Your model is benchmarked on 200+ physical devices to ensure the best possible performance on each hardware configuration. *** Next Steps [#next-steps] * [Inference Mode Selection](/how-to-guides/inference-mode-selection): Choose the right mode * [Device Compatibility](/performance/device-compatibility): Supported NPU chipsets * [Benchmark Methodology](/performance/benchmark-methodology): How performance is measured # Key Concepts (/introduction/key-concepts) This page defines the core terms and concepts you will encounter when working with ZETIC Melange. Model Key [#model-key] A unique identifier for a deployed model on the Melange platform. Model keys follow the format `owner/model-name` (e.g., `google/MediaPipe-Face-Detection`, `OpenAI/whisper-tiny-encoder`). When you upload a model through the dashboard, Melange assigns it a model key that you reference in your mobile application code. Personal Key [#personal-key] An authentication token that identifies your Melange account. You generate a Personal Key from the [Melange Dashboard](https://melange.zetic.ai) and use it when initializing models in your Android or iOS application. The Personal Key controls access to the models associated with your account. Keep your Personal Key secure. Do not commit it to public repositories or embed it in client-side code that can be easily decompiled. Inference Mode [#inference-mode] Melange supports multiple inference modes that balance speed and accuracy for your deployed models. General Model Modes [#general-model-modes] | Mode | Description | Best For | | ----------------------- | -------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | | **RUN\_AUTO** (Default) | Automatically selects the fastest configuration while maintaining high accuracy (SNR > 20dB) | Most use cases | | **RUN\_SPEED** | Maximizes inference speed with minimum latency | Real-time applications where response time is the top priority | | **RUN\_ACCURACY** | Delivers the highest precision based on maximum SNR scores | Applications where accuracy is more critical than speed | LLM Model Modes [#llm-model-modes] LLM models have their own set of inference modes optimized for token generation on different hardware backends. See the [LLM Inference Modes](/llm-inference/inference-modes) documentation for details. Performance-Adaptive Deployment [#performance-adaptive-deployment] Melange does not rely on static rules (e.g., "use GPU if version > X") to determine which model binary to serve. Instead, it performs **on-target performance measurement** by running your model on a farm of **200+ physical devices** covering a wide range of chipsets and OS versions. Based on actual latency, throughput, and stability measurements, Melange selects the optimal model binary for each specific device model. When a user installs your app, the Melange runtime automatically fetches the best-performing binary for their device. For details, see [Performance-Adaptive Deployment](/model-deployment/performance-adaptive-deployment). ZeticMLangeModel [#zeticmlangemodel] The primary SDK class for running general AI models (computer vision, audio, etc.) on-device. It handles model download, NPU initialization, and inference execution. ```kotlin val model = ZeticMLangeModel(context, PERSONAL_KEY, MODEL_NAME) val outputs = model.run(inputs) ``` ```swift let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) let outputs = try model.run(inputs: inputs) ``` ```dart final model = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, ); final outputs = model.run(inputs); ``` ZeticMLangeLLMModel [#zeticmlangellmmodel] A specialized SDK class for running Large Language Models on-device with token streaming. It manages the full LLM lifecycle including KV-cache management, token generation, and multi-backend orchestration. ```kotlin val model = ZeticMLangeLLMModel(context, PERSONAL_KEY, MODEL_NAME) model.run("prompt") while (true) { val result = model.waitForNextToken() if (result.generatedTokens == 0) break print(result.token) } ``` ```swift let model = try await ZeticMLangeLLMModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) try model.run("prompt") while true { let result = model.waitForNextToken() if result.generatedTokens == 0 { break } print(result.token) } ``` ```dart final model = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: modelName, ); model.run('prompt'); while (true) { final result = model.waitForNextToken(); if (result.isFinished) break; print(result.token); } ``` ZeticMLangeHFModel [#zeticmlangehfmodel] A convenience class for loading models directly from [Hugging Face](https://huggingface.co/) using only a repository ID. No personal key or model key is required. The SDK automatically downloads, compiles, and caches the model on first use. ```kotlin val model = ZeticMLangeHFModel(context, "zetic-ai/yolov11n") val outputs = model.run(arrayOf(inputTensor)) ``` ```swift let model = try await ZeticMLangeHFModel("zetic-ai/yolov11n") let outputs = try model.run(inputs: [inputTensor]) ``` ```dart final model = await ZeticMLangeHFModel.create('zetic-ai/yolov11n'); final outputs = model.run([inputTensor]); ``` See [Hugging Face Models](/model-preparation/hugging-face-models) for details. Global Device Benchmark [#global-device-benchmark] Melange maintains a continuously updated benchmark database spanning **200+ physical devices** across Android and iOS. This benchmark farm covers: * **Chipset vendors**: Qualcomm Snapdragon, MediaTek Dimensity, Samsung Exynos, Apple A-series and M-series * **Processing units**: CPU, GPU, and NPU on each device * **Device tiers**: Flagship, mid-range, and budget devices The benchmark data drives the [Performance-Adaptive Deployment](#performance-adaptive-deployment) system, ensuring every end user receives the best-performing model binary for their specific hardware. Model Pipeline [#model-pipeline] Some applications require chaining multiple models together. For example, face landmark detection uses a two-model pipeline: 1. **Face Detection**: Locates faces in the image 2. **Face Landmark**: Extracts landmark points from the detected face region Each model in the pipeline is a separate `ZeticMLangeModel` instance, and you pass the output of one model as input to the next. See the [Face Landmark tutorial](/tutorials/face-landmark) for a complete example. Next Steps [#next-steps] * [Platform Support Matrix](/introduction/platform-support-matrix): Check supported devices and formats * [Quick Start](/quick-start): Deploy your first model * [Tutorials](/tutorials/object-detection-yolo): Step-by-step guides for specific models # Platform Support Matrix (/introduction/platform-support-matrix) This page lists the platforms, chipsets, model formats, and minimum requirements supported by ZETIC Melange. Supported Platforms [#supported-platforms] | Feature | Android | iOS | | -------------------------------------------- | ------- | --- | | General model inference (`ZeticMLangeModel`) | Yes | Yes | | LLM inference (`ZeticMLangeLLMModel`) | Yes | Yes | | NPU acceleration | Yes | Yes | | GPU acceleration | Yes | Yes | | CPU fallback | Yes | Yes | | Performance-Adaptive Deployment | Yes | Yes | Minimum OS Versions [#minimum-os-versions] | Platform | Minimum Version | | -------- | --------------------------- | | Android | API 24 (Android 7.0 Nougat) | | iOS | 15.0 | NPU Support by Chipset [#npu-support-by-chipset] Melange automatically compiles and optimizes models for the following NPU architectures: Qualcomm Snapdragon [#qualcomm-snapdragon] | Chipset | NPU | Status | | ------------------ | ------- | --------- | | Snapdragon 8 Gen 3 | Hexagon | Supported | | Snapdragon 8 Gen 2 | Hexagon | Supported | | Snapdragon 8 Gen 1 | Hexagon | Supported | | Snapdragon 888 | Hexagon | Supported | | Snapdragon 7 Gen 3 | Hexagon | Supported | | Snapdragon 7 Gen 1 | Hexagon | Supported | | Snapdragon 6 Gen 1 | Hexagon | Supported | Google Pixel (Tensor) [#google-pixel-tensor] | Chipset | NPU | Status | | --------- | -------- | --------------- | | Tensor G4 | Edge TPU | Enterprise Only | | Tensor G3 | Edge TPU | Enterprise Only | | Tensor G2 | Edge TPU | Enterprise Only | MediaTek Dimensity [#mediatek-dimensity] | Chipset | NPU | Status | | -------------- | --- | --------------- | | Dimensity 9300 | APU | Enterprise Only | | Dimensity 9200 | APU | Enterprise Only | | Dimensity 8300 | APU | Enterprise Only | | Dimensity 7200 | APU | Enterprise Only | Samsung Exynos [#samsung-exynos] | Chipset | NPU | Status | | ----------- | --- | --------------- | | Exynos 2400 | NPU | Enterprise Only | | Exynos 2200 | NPU | Enterprise Only | Google Pixel (Tensor), MediaTek Dimensity, and Samsung Exynos NPU support are available for enterprise customers only. [Contact us](mailto:contact@zetic.ai) for enterprise pricing and access. Apple Silicon [#apple-silicon] | Chipset | NPU | Status | | --------------- | ------------------- | --------- | | A17 Pro | Apple Neural Engine | Supported | | A16 Bionic | Apple Neural Engine | Supported | | A15 Bionic | Apple Neural Engine | Supported | | M-series (iPad) | Apple Neural Engine | Supported | The chipset support list is continuously expanding. If your target device is not listed here, Melange will still work using GPU or CPU fallback. [Contact us](mailto:contact@zetic.ai) for specific chipset inquiries. Supported Model Formats [#supported-model-formats] | Format | Extension | Status | Notes | | ------------------------ | --------- | ----------- | -------------------------------------------------------------------------------------- | | ONNX | `.onnx` | Recommended | Broadest compatibility, supports models from TensorFlow, PyTorch, and other frameworks | | PyTorch Exported Program | `.pt2` | Recommended | Native PyTorch 2.0+ export format with full graph capture | Input Format [#input-format] All model inputs must be provided as **NumPy arrays** (`.npy` files) during the upload step. These sample inputs are used for: * Defining fixed input shapes for NPU compilation * Running accuracy validation during the benchmark phase * Generating optimized model binaries NPU compilation hard-codes input shapes to maximize throughput. Even if your original model supports dynamic sizes, the accelerated Melange model will accept only the exact shape of the sample input provided during upload. SDK Distribution [#sdk-distribution] | Platform | Distribution Method | Package | | -------- | --------------------------- | --------------------------- | | Android | Maven Central | `com.zeticai.mlange:mlange` | | iOS | Swift Package Manager (SPM) | `ZeticMLange` | Next Steps [#next-steps] * [Supported Model Formats](/model-preparation/supported-formats): Get your model ready for upload * [Quick Start](/quick-start): Deploy your first model * [Changelog](/release-notes/changelog): Latest SDK updates and version history * [Migration Guide](/release-notes/migration-mlange-to-melange): Migrating from MLange to Melange naming # Pricing and Plans (/introduction/pricing-and-plans) ZETIC Melange offers four pricing tiers to match your stage of development: from prototyping to global-scale deployment. Plans Overview [#plans-overview] | | Free | Pro | Pro+ | Enterprise | | -------------------------- | ------------- | ------------------ | --------------- | ---------- | | **Price** | $0/mo | $39/mo | $149/mo | Custom | | **Model Catalog** | Public only | Public + Custom | Extended | Extended | | **Optimized Devices** | Flagship only | Flagship + popular | Global coverage | Custom | | **Bandwidth** | 100 GB/mo | 500 GB/mo | 2 TB/mo | Custom | | **Auto-Scaling** | N/A | $0.09/GB | $0.09/GB | Custom | | **Model Server** | Hugging Face | ZETIC | ZETIC | Custom | | **Successful Conversions** | N/A | 3/mo | 10/mo | Custom | | **Benchmark Report** | Limited | Limited | Full | Full | | **Prompt Generations** | 3 | 10 | Unlimited | Custom | | **A/B Testing** | No | No | No | Yes | | **Support** | Community | Standard | Priority | Dedicated | *** Free: $0/month [#free-0month] Prototype your AI app at no cost. * Public models only * Optimized for 4 flagship devices * Community support * Model delivery from Hugging Face The free tier is designed to let you evaluate Melange end-to-end, from model upload through on-device inference with NPU acceleration. No credit card required. *** Pro: $39/month [#pro-39month] Launch your MVP with custom models and broader device coverage. * Public + Custom models * Optimized for 15 devices * Benchmark reports * Standard bandwidth (500 GB/mo) + Auto-scaling at $0.09/GB * Standard support * 3 successful conversions per month * 10 prompt generations *** Pro+: $149/month [#pro-149month] Scale and optimize for global coverage. * Extended model catalog * Optimized for 50 devices * Advanced benchmark reports * Expanded bandwidth (2 TB/mo) + Auto-scaling at $0.09/GB * Priority support * 10 successful conversions per month * Unlimited prompt generations *** Enterprise: Custom Pricing [#enterprise-custom-pricing] Full market coverage and custom infrastructure. * Bespoke model optimization * Most commercial target devices * Advanced benchmark reports * Custom model delivery and bandwidth * Enterprise features including A/B Testing * Dedicated support Google Pixel (Tensor), MediaTek Dimensity, and Samsung Exynos NPU support are available for enterprise customers only. *** Get Started [#get-started] * **Free**: Sign up at [melange.zetic.ai](https://melange.zetic.ai): no credit card required * **Pro / Pro+**: Upgrade from your [Melange Dashboard](https://melange.zetic.ai) * **Enterprise**: [Contact Sales](mailto:contact@zetic.ai) *** Next Steps [#next-steps] * [What is Melange?](/introduction/what-is-melange): Learn about the platform * [Quick Start](/quick-start): Deploy your first model * [Tutorials](/tutorials/object-detection-yolo): Step-by-step implementation guides # What is Melange? (/introduction/what-is-melange) ZETIC Melange is the **essential software infrastructure** for automated on-device AI deployment. It bridges the gap between high-level AI development and low-level hardware complexity, making NPU utilization accessible to every developer. The Problem [#the-problem] Deploying AI models to mobile devices with NPU acceleration is notoriously difficult: * **Manual NPU optimization**: Each chipset vendor (Qualcomm, MediaTek, Samsung, Apple) requires different SDKs, toolchains, and optimization strategies. * **Cross-platform fragmentation**: Android and iOS have fundamentally different hardware architectures, and even within Android, the NPU landscape is fragmented across hundreds of device models. * **Months of engineering effort**: Getting a model to run efficiently on a single NPU can take weeks. Supporting all target devices multiplies that effort. The Solution: 3-Step Workflow [#the-solution-3-step-workflow] Melange reduces on-device AI deployment to three steps: Upload [#upload] Provide your trained model in a supported format (ONNX or PyTorch Exported Program) along with sample input tensors. Upload via the [web dashboard](https://melange.zetic.ai). Benchmark [#benchmark] Melange automatically compiles your model for multiple NPU targets and runs it on a farm of **200+ physical devices**. The system measures real latency, throughput, and accuracy to determine the optimal binary for every device model. Deploy [#deploy] Integrate the Melange SDK into your Android or iOS application. At runtime, the SDK automatically downloads and executes the best-performing model binary for the end user's specific device. Who is Melange For? [#who-is-melange-for] * **Mobile AI engineers** who need NPU acceleration without vendor-specific SDK expertise * **ML teams** shipping models to production mobile applications * **Product teams** that want on-device AI for privacy, latency, or cost reasons * **Enterprises** deploying AI across a diverse fleet of Android and iOS devices Core Value Propositions [#core-value-propositions] Abstracts the complexity of NPU execution. Delivers hardware-accelerated throughput without managing vendor-specific SDKs (Qualcomm QNN, MediaTek NeuroPilot, Samsung ENN, Apple Core ML). A single pipeline for all edge targets. Handles the complete lifecycle from graph optimization and quantization to on-device runtime execution. Write once, run optimally everywhere. Provides a unified API layer across fragmented mobile architectures including Snapdragon, MediaTek, Exynos, and Apple Neural Engine. Eliminates months of manual tuning. Replaces bespoke hardware integration with an automated compilation workflow that gets your model running on NPUs in hours, not months. Next Steps [#next-steps] Learn the essential terminology. Check supported devices and formats. Deploy your first model in 5 minutes. # Function Calling (/llm-inference/function-calling) Function calling lets an on-device LLM ask your app to run a tool and feed the tool result back into the same generation session. Use it for app-local actions such as search, settings lookup, inventory checks, or deterministic calculations. This page reflects ZeticMLange 1.10.0 on Android and iOS, and `zetic_mlange 1.10.0` on Flutter. Tool Shape [#tool-shape] Each tool has a name, description, and JSON parameter schema. When it emits a tool call, the SDK invokes your registered executor and continues the tool session with the returned content. ```kotlin val weatherTool = LLMToolSpec( name = "get_weather", description = "Get the weather for a city.", parametersJson = """ { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } """.trimIndent(), ) model.registerTool(weatherTool) { call -> LLMToolResult(content = """{"summary":"Sunny, 24C"}""") } ``` ```swift let weatherTool = LLMToolSpec( name: "get_weather", description: "Get the weather for a city.", parametersJson: """ { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } """ ) try model.registerTool(weatherTool) { call in LLMToolResult(content: #"{"summary":"Sunny, 24C"}"#) } ``` ```dart final weatherTool = LLMToolSpec( name: 'get_weather', description: 'Get the weather for a city.', parametersJson: ''' { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } ''', ); model.registerTool(weatherTool, (call) { return const LLMToolResult(content: '{"summary":"Sunny, 24C"}'); }); ``` Run With Tools [#run-with-tools] Register tools before calling `runWithTools(...)`. Use `run(...)` only when no tools are registered. ```kotlin model.functionCallingSystemPrompt = "Use tools only when they are needed. Explain the final answer clearly." model.runWithTools("What is the weather in Seoul?") .collect { token -> append(token) } ``` ```swift model.functionCallingSystemPrompt = "Use tools only when they are needed. Explain the final answer clearly." for try await token in try model.runWithTools("What is the weather in Seoul?") { append(token) } ``` ```dart model.functionCallingSystemPrompt = 'Use tools only when they are needed. Explain the final answer clearly.'; await for (final token in model.runWithTools('What is the weather in Seoul?')) { append(token); } ``` Tool Management [#tool-management] | Platform | Register | Remove | Clear | List | | -------- | ------------------------------ | ----------------------- | -------------- | ----------------------- | | Android | `registerTool(spec, executor)` | `unregisterTool(name)` | `clearTools()` | `registeredTools()` | | iOS | `registerTool(_:executor:)` | `unregisterTool(name:)` | `clearTools()` | `registeredToolSpecs()` | | Flutter | `registerTool(spec, executor)` | `unregisterTool(name)` | `clearTools()` | `registeredTools()` | Keep executors fast and deterministic. Android and iOS tool executors support asynchronous work; Flutter tool executors may return a `Future`. API Reference [#api-reference] * [Android `ZeticMLangeLLMModel`](/api-reference/android/ZeticMLangeLLMModel) * [iOS `ZeticMLangeLLMModel`](/api-reference/ios/ZeticMLangeLLMModel) * [Flutter `ZeticMLangeLLMModel`](/api-reference/flutter/ZeticMLangeLLMModel) # LLM Inference Modes (/llm-inference/inference-modes) `LLMModelMode` controls the automatic selection strategy used by `ZeticMLangeLLMModel`. Available Modes [#available-modes] | Mode | Purpose | | ------------------------------ | ------------------------------------------------------------------------------------------------- | | `RUN_AUTO` / `runAuto` | Default strategy. Lets the SDK select the best available runtime and quantization for the device. | | `RUN_SPEED` / `runSpeed` | Prioritizes lower latency. | | `RUN_ACCURACY` / `runAccuracy` | Prioritizes better accuracy when multiple candidates are available. | API Usage [#api-usage] ```kotlin val modelSpeed = ZeticMLangeLLMModel( context = context, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = LLMModelMode.RUN_SPEED, ) ``` ```swift let modelSpeed = try await ZeticMLangeLLMModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_SPEED ) ``` ```dart final modelSpeed = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: modelName, modelMode: LLMModelMode.runSpeed, ); ``` Next Steps [#next-steps] * [LLM Inference Overview](/llm-inference/overview) * [Streaming Token Generation](/llm-inference/streaming-generation) * [Enums and Constants (Android)](/api-reference/android/enums-and-constants) * [Enums and Constants (iOS)](/api-reference/ios/enums-and-constants) * [Enums and Constants (Flutter)](/api-reference/flutter/enums-and-constants) # LLM Inference Overview (/llm-inference/overview) Examples on this page reflect `ZeticMLange Android 1.10.0`, `ZeticMLange iOS 1.10.0`, and `zetic_mlange 1.10.0`. `ZeticMLangeLLMModel` runs text generation on-device and exposes token streaming through `waitForNextToken()`. The public LLM surface also includes function calling, composition-based RAG, vision-language image response, and KV state persistence on native Android/iOS. Load A Model [#load-a-model] Use a model name from the Melange Dashboard or a supported public Hugging Face model name. ```kotlin val model = ZeticMLangeLLMModel( context = context, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = LLMModelMode.RUN_AUTO, initOption = LLMInitOption(nCtx = 4096), ) ``` ```swift let model = try await ZeticMLangeLLMModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_AUTO, initOption: LLMInitOption(nCtx: 4096) ) ``` ```dart final model = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: modelName, modelMode: LLMModelMode.runAuto, initOption: const LLMInitOption(nCtx: 4096), ); ``` Generate Text [#generate-text] `run(...)` starts generation. Read tokens with `waitForNextToken()` until the stream is finished. ```kotlin model.run("What is on-device AI?") val output = StringBuilder() while (true) { val next = model.waitForNextToken() if (next.isFinal || next.token.isEmpty()) break output.append(next.token) } ``` ```swift try model.run("What is on-device AI?") var output = "" while true { let next = model.waitForNextToken() if next.isFinished { break } output.append(next.token) } ``` ```dart model.run('What is on-device AI?'); final output = StringBuffer(); while (true) { final next = model.waitForNextToken(); if (next.isFinished) break; output.write(next.token); } ``` Selection Controls [#selection-controls] Use `modelMode` to select the automatic strategy. | Option | Purpose | | --------------------- | -------------------------------------------------------- | | `modelMode` | Selects automatic strategy: auto, speed, or accuracy. | | `initOption.nCtx` | Requests the context size. The runtime may normalize it. | | `cacheHandlingPolicy` | Controls downloaded model artifact cleanup on disk. | 1.10.0 Capabilities [#1100-capabilities] Connect model output to app-defined tools. Ground generation with retrieved local context. Ask image + text questions with LFM-VL-capable models. Quick Start Templates [#quick-start-templates] Next Steps [#next-steps] * [Streaming Token Generation](/llm-inference/streaming-generation) * [LLM Inference Modes](/llm-inference/inference-modes) * [Function Calling](/llm-inference/function-calling) * [RAG](/llm-inference/rag) * [Vision-Language Inference](/llm-inference/vision-language) # Retrieval-Augmented Generation (/llm-inference/rag) Retrieval-augmented generation (RAG) lets your app retrieve relevant text chunks and pass them into an on-device LLM as grounded context. This page reflects ZeticMLange 1.10.0. RAG APIs are available on Android, iOS, and Flutter through a composition-based `RagPipeline` shape. Concepts [#concepts] | Concept | Purpose | | ------------------ | ---------------------------------------------------------------------------------- | | Retriever | App-provided component that returns relevant chunks for a query. | | Retrieved chunk | Text plus optional score, source, and metadata. | | RAG pipeline | Combines retrieval, prompt assembly, and token streaming. | | Local RAG pipeline | Flutter, Android, and iOS helper for on-device chunking, embedding, and retrieval. | Android [#android] Android exposes `RagPipeline`, `Retriever`, `RetrievedChunk`, and `LocalRagPipeline`. ```kotlin class MyRetriever : Retriever { override suspend fun retrieve(query: String, topK: Int): List { return listOf( RetrievedChunk( text = "ZeticMLange runs optimized models on-device.", score = 0.94f, source = "docs", ), ) } } val pipeline = RagPipeline( retriever = MyRetriever(), llm = model, profile = profile, ) pipeline.respond("What does ZeticMLange do?").collect { token -> append(token) } ``` iOS [#ios] iOS exposes `RagPipeline`, `Retriever`, `RetrievedChunk`, and `LocalRagPipeline`. ```swift final class MyRetriever: Retriever { func retrieve(query: String, topK: Int) async throws -> [RetrievedChunk] { [ RetrievedChunk( text: "ZeticMLange runs optimized models on-device.", score: 0.94, source: "docs" ) ] } } let pipeline = RagPipeline( retriever: MyRetriever(), llm: model, profile: profile ) for try await token in pipeline.respond(query: "What does ZeticMLange do?") { append(token) } ``` For fully local retrieval: ```swift let localRag = try LocalRagPipeline.create( embedderGgufPath: embedderPath, profile: profile ) try await localRag.indexDocs([ (text: "ZeticMLange runs optimized models on-device.", source: "docs") ]) ``` Flutter [#flutter] Flutter exposes `RagPipeline`, `RagRetriever`, `RetrievedChunk`, and `LocalRagPipeline`. ```dart final retriever = MyRetriever(); final rag = RagPipeline( retriever: retriever, llm: model, profile: const RagProfile.qwen25(), ); await for (final token in rag.respond(query: 'What does ZeticMLange do?')) { append(token); } ``` `MyRetriever` can be any app-owned class that implements `RagRetriever`: ```dart final class MyRetriever implements RagRetriever { @override Future> retrieve(String query, {required int topK}) async { return const [ RetrievedChunk( text: 'ZeticMLange runs optimized models on-device.', score: 0.94, source: 'docs', ), ]; } } ``` For a local retrieval pipeline: ```dart final localRag = await LocalRagPipeline.create( profile: const RagProfile.qwen25(backboneGgufPath: backbonePath), embedderGgufPath: embedderPath, ); await localRag.indexDocs([ const RagDocument( text: 'ZeticMLange runs optimized models on-device.', source: 'docs', ), ]); final rag = RagPipeline( retriever: localRag, llm: model, profile: const RagProfile.qwen25(backboneGgufPath: backbonePath), ); await for (final token in rag.respond(query: 'What does ZeticMLange do?')) { append(token); } ``` API Reference [#api-reference] * [Android `ZeticMLangeLLMModel`](/api-reference/android/ZeticMLangeLLMModel) * [iOS `ZeticMLangeLLMModel`](/api-reference/ios/ZeticMLangeLLMModel) * [Flutter `ZeticMLangeLLMModel`](/api-reference/flutter/ZeticMLangeLLMModel) * [Flutter `RagPipeline`](/api-reference/flutter/RagPipeline) # Streaming Token Generation (/llm-inference/streaming-generation) Melange streams generated tokens incrementally, so you can render output while the model is still decoding. How Streaming Works [#how-streaming-works] 1. Call `run(prompt)` to start the generation context. 2. Call `waitForNextToken()` in a loop to receive tokens one at a time. 3. Stop when generation completes. Basic Streaming [#basic-streaming] ```kotlin val model = ZeticMLangeLLMModel(context, PERSONAL_KEY, MODEL_NAME) model.run(userPrompt) val sb = StringBuilder() while (true) { val result = model.waitForNextToken() if (result.generatedTokens == 0) break if (result.token.isNotEmpty()) sb.append(result.token) } val output = sb.toString() ``` ```swift let model = try await ZeticMLangeLLMModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) try model.run(userPrompt) var buffer = "" while true { let result = model.waitForNextToken() if result.generatedTokens == 0 { break } buffer.append(result.token) } let output = buffer ``` ```dart final model = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: modelName, ); model.run(userPrompt); final buffer = StringBuffer(); while (true) { final result = model.waitForNextToken(); if (result.isFinished) { break; } buffer.write(result.token); } final output = buffer.toString(); ``` Streaming to the UI [#streaming-to-the-ui] For a chat UI, update the screen every time a new token arrives. ```kotlin lifecycleScope.launch(Dispatchers.IO) { val model = ZeticMLangeLLMModel(context, PERSONAL_KEY, MODEL_NAME) model.run(userPrompt) while (true) { val result = model.waitForNextToken() if (result.generatedTokens == 0) break withContext(Dispatchers.Main) { textView.append(result.token) } } } ``` ```swift Task.detached { do { let model = try await ZeticMLangeLLMModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) try model.run(userPrompt) while true { let result = model.waitForNextToken() if result.generatedTokens == 0 { break } await MainActor.run { self.textView.text?.append(result.token) } } } catch { print("LLM error: \(error)") } } ``` ```dart Future streamAnswer() async { final model = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: modelName, ); model.run(userPrompt); while (true) { final result = model.waitForNextToken(); if (result.isFinished) { break; } setState(() { generatedText += result.token; }); await Future.delayed(Duration.zero); } } ``` `waitForNextToken()` is blocking. For high-frequency UI updates or long generations, run token polling from a worker isolate or yield back to the event loop between updates. Conversation Reset [#conversation-reset] If you want a fresh conversation, call `cleanUp()`. ```kotlin model.cleanUp() model.run("Start a new conversation") ``` ```swift try model.cleanUp() try model.run("Start a new conversation") ``` ```dart model.cleanUp(); model.run('Start a new conversation'); ``` Keeping Context Between Turns [#keeping-context-between-turns] Use `LLMInitOption.kvCacheCleanupPolicy` to control what happens when the KV cache fills up. * `CLEAN_UP_ON_FULL`: Clears the conversation context automatically. * `DO_NOT_CLEAN_UP`: Keeps the existing context. You must manually call `cleanUp()` before starting a new conversation. When you use `DO_NOT_CLEAN_UP`, do not call `run()` again for a new conversation until you have called `cleanUp()`. Releasing the Model [#releasing-the-model] When the model instance is no longer needed: ```kotlin model.deinit() ``` ```swift model.forceDeinit() ``` ```dart model.close(); ``` Next Steps [#next-steps] * [LLM Inference Modes](/llm-inference/inference-modes): Speed vs. accuracy configuration * [LLM Inference Overview](/llm-inference/overview): Automatic vs explicit initialization * [ZeticMLangeLLMModel (Android)](/api-reference/android/ZeticMLangeLLMModel): Android API reference * [ZeticMLangeLLMModel (iOS)](/api-reference/ios/ZeticMLangeLLMModel): iOS API reference * [ZeticMLangeLLMModel (Flutter)](/api-reference/flutter/ZeticMLangeLLMModel): Flutter API reference # Supported LLM Models (/llm-inference/supported-models) Melange supports a growing list of large language models for on-device inference. Models are validated weekly as new architectures are added. Available Models [#available-models] | Model | Hugging Face ID | Parameters | | ----------------------------- | ------------------------------- | ---------- | | Google Gemma 3 4B Instruct | `google/gemma-3-4b-it` | 4B | | LiquidAI LFM2.5 1.2B Instruct | `LiquidAI/LFM2.5-1.2B-Instruct` | 1.2B | For the most up-to-date list of supported models, visit the [Melange Dashboard Use Cases](https://melange.zetic.ai/model-library) page. Using Pre-Built Models [#using-pre-built-models] Select a model from the [Melange Dashboard](https://melange.zetic.ai) and use the provided model key directly in your application: ```kotlin val model = ZeticMLangeLLMModel( context = context, personalKey = PERSONAL_KEY, name = "pre-built-model-key", modelMode = LLMModelMode.RUN_AUTO, ) ``` ```dart final model = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: 'pre-built-model-key', modelMode: LLMModelMode.runAuto, ); ``` Using Hugging Face Models [#using-hugging-face-models] You can also use models directly from Hugging Face by providing the repository ID: ```kotlin val model = ZeticMLangeLLMModel( context = context, personalKey = PERSONAL_KEY, name = "google/gemma-3-4b-it", modelMode = LLMModelMode.RUN_AUTO, ) ``` ```dart final model = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: 'google/gemma-3-4b-it', modelMode: LLMModelMode.runAuto, ); ``` Currently supports public repositories with permissive open-source licenses. Private repository authentication is on the roadmap. Model Compatibility Notes [#model-compatibility-notes] * Models must have an architecture supported by the Melange LLM engine. * Very large models (>7B parameters) may require devices with sufficient RAM. * Quantized variants are automatically selected based on your [inference mode](/llm-inference/inference-modes) settings. Requesting New Models [#requesting-new-models] If you need a specific model that is not yet supported: * Contact [contact@zetic.ai](mailto:contact@zetic.ai) with the model name and Hugging Face repository link. * Join the [Discord community](https://discord.gg/q6vW4UscRY) to discuss model requests. *** Next Steps [#next-steps] * [LLM Inference Overview](/llm-inference/overview): Getting started with LLM inference * [Streaming Token Generation](/llm-inference/streaming-generation): Implement token streaming * [LLM Inference Modes](/llm-inference/inference-modes): Configure speed vs. accuracy # Vision-Language Inference (/llm-inference/vision-language) Vision-language inference lets an on-device LLM answer questions about an RGB image plus a text prompt. This page reflects ZeticMLange 1.10.0 on Android and iOS, and `zetic_mlange 1.10.0` on Flutter. It covers image + text LFM-VL usage only; lower-level embedding injection APIs are not part of the public website surface for this release. Image Input [#image-input] Images are passed as RGB bytes with explicit width and height. ```kotlin val image = ZeticMLangeLLMModel.Image( rgb = rgbBytes, width = width, height = height, ) ``` ```swift let image = try ZeticMLangeLLMModel.Image( rgb: rgbBytes, width: width, height: height ) ``` ```dart final image = ZeticMLangeLLMImage( rgb: rgbBytes, width: width, height: height, ); ``` The RGB byte length must be `width * height * 3`. Respond To An Image [#respond-to-an-image] ```kotlin model.respond( systemPrompt = "Answer briefly.", userText = "What is in this image?", image = image, ).collect { token -> append(token) } ``` ```swift for try await token in try model.respond( systemPrompt: "Answer briefly.", userText: "What is in this image?", image: image ) { append(token) } ``` ```dart final response = await model.respond( systemPrompt: 'Answer briefly.', userText: 'What is in this image?', image: image, ); ``` Use an LFM-VL-capable model. Calling image response APIs on a text-only model returns an unsupported-operation error. API Reference [#api-reference] * [Android `ZeticMLangeLLMModel`](/api-reference/android/ZeticMLangeLLMModel) * [iOS `ZeticMLangeLLMModel`](/api-reference/ios/ZeticMLangeLLMModel) * [Flutter `ZeticMLangeLLMModel`](/api-reference/flutter/ZeticMLangeLLMModel) # Build with AI (/melange-cli/agents) The Melange CLI gives AI agents a structured way to discover models, compare device benchmarks, manage repositories, upload models, and produce credential-safe deployment code. It works with Claude Code, Cursor, Codex, or any coding agent. Install the agent skill [#install-the-agent-skill] **One line on macOS or Linux โ€” installs the CLI and the skill together:** ```sh curl -fsSL https://raw.githubusercontent.com/zetic-ai/melange-cli/main/script/install.sh | sh ``` Restart your agent afterward so it can discover the skill. Re-run the same line any time to update both. To choose a version or agents: ```sh curl -fsSL https://raw.githubusercontent.com/zetic-ai/melange-cli/main/script/install.sh \ | sh -s -- --version v1.2.3 --agent "universal claude-code codex" ``` The installer uses [`npx skills`](https://github.com/vercel-labs/skills) when a recent Node is available, and copies the skill directly into the agent directories otherwise. Additional options: `--cli-only` / `MELANGE_SKIP_SKILL`, `--skill-only` / `MELANGE_SKIP_CLI`, `--install-dir` / `MELANGE_INSTALL_DIR`, `--require-signature` / `MELANGE_REQUIRE_SIGNATURE`. Or install the skill separately after a CLI-only install (Homebrew, npm, Go, manual): ```sh npx skills add zetic-ai/melange-cli --skill melange-cli \ --agent universal claude-code --global --yes ``` To use another coding agent, let the installer show its interactive agent selector: ```sh npx skills add zetic-ai/melange-cli --skill melange-cli --global ``` Restart the agent after installation. Update the installed skill after a CLI release: ```sh # if you used the one-liner, just re-run it: curl -fsSL https://raw.githubusercontent.com/zetic-ai/melange-cli/main/script/install.sh | sh # or if you installed the skill separately: npx skills update melange-cli --global ``` Homebrew, npm, Go, and manual installs provide the CLI only โ€” add the skill separately. The `curl | sh` installer installs and updates both together. MCP server alternative [#mcp-server-alternative] `melange mcp` serves the same operations as MCP tools (18 over stdio, 17 over HTTP โ€” `upload_model` is stdio-only). When the MCP server is already connected, prefer its tools over shelling out. See the [MCP server](/melange-cli/mcp) page for transports, per-request auth, and client setup. Every rule below applies unchanged whether data came from CLI or MCP. Whatโ€™s supported [#whats-supported] With the Melange skill or MCP server, your agent can: * Search the public model library and inspect available model versions. * Compare real device benchmarks, targets, and report availability. * Create and manage repositories, imports, uploads, and model versions. * Track a conversion through its phases and report each transition as it lands. * Generate exact deployment guides for Android, iOS, and Flutter. * Check authentication, usage, quotas, and plan-specific availability. The agent only sees data and actions available to your Melange account. Example usages [#example-usages] After authenticating with `melange auth login`, describe the outcome you want. You can copy these prompts as-is or add your own device, platform, and performance constraints. ```text Find a small language model in the Melange public library, compare its real device benchmark results, and give me the Android Kotlin deployment guide in auto mode. Do not import or modify a model. ``` ```text Compare LFM2.5_350M with another small language model available in Melange. Use only benchmark values returned by Melange. Show throughput and peak memory for iPhone 16 and Galaxy S25 where available. Explain missing or plan-limited data, then recommend a model and target. Do not import or modify anything. ``` ```text I want to upload model.pt2 with sample.npy to ACCOUNT/REPO. First verify my authentication, repository context, and upload quota. Show me the dry-run manifest and explain any missing inputs. Upload the model, monitor conversion, and report the final status without retrying implicitly. ``` Best practices [#best-practices] For imports, uploads, downloads, repository changes, or other consequential work, ask the agent to make a plan first. Review the resolved target and side effects before approving execution. 1. **Describe the outcome.** Include your target device, platform, inference mode, and latency or memory constraints when they matter. 2. **Start with read-only discovery.** Ask the agent to inspect existing models and reports before importing or uploading anything. 3. **Require evidence.** Ask for real Melange results, clear labels for unavailable or plan-limited data, and no estimated metrics. Expect the full report in the reply โ€” every quantization or device, like the model page in the dashboard โ€” rather than a summary or a single device's numbers. 4. **Expect conversion to run in the background.** An import or upload returns as soon as the model is registered, then converts and benchmarks server-side. The agent should tell you which phase is running and what remains instead of blocking, and note that the model becomes downloadable once conversion finishes, while benchmarking continues. 5. **Approve changes explicitly.** Keep confirmation enabled for billable, destructive, or external-side-effect actions. 6. **Protect credentials.** Let the CLI handle your personal access token; never paste it into a prompt or request it in generated code. 7. **Keep the skill current.** Update the skill after CLI releases, then restart your agent before starting a new workflow. # Getting Started (/melange-cli/getting-started) The Melange CLI (`melange`) brings model discovery, benchmarks, repository management, model uploads, and deployment guides to your terminal. It uses the same Melange platform as the dashboard and works well for both people and automation. The one-line installer below installs the CLI and the skill together โ€” restart your agent afterward. Or install the skill separately and continue with [Build with AI](/melange-cli/agents): ```sh npx skills add zetic-ai/melange-cli --skill melange-cli \ --agent universal claude-code --global --yes ``` Install [#install] Install the prebuilt CLI on macOS or Linux. Go is not required. ```sh brew install zetic-ai/tap/melange melange version ``` Install the prebuilt CLI on macOS, Linux, or Windows. Go is not required. ```sh npm install -g @zetic-ai/melange-cli melange version ``` One line on macOS or Linux โ€” installs the CLI and the agent skill: ```sh curl -fsSL https://raw.githubusercontent.com/zetic-ai/melange-cli/main/script/install.sh | sh ``` Restart your coding agent afterward so it can discover the skill. Re-run the same line any time to update both. The installer downloads the release binary for your platform, verifies its SHA-256 checksum, and installs the skill for universal agents and Claude Code. When [`cosign`](https://docs.sigstore.dev/cosign/system_config/installation/) is on your PATH it also verifies the release-workflow signature; pass `| sh -s -- --require-signature` to make it mandatory. Installer options โ€” append after `sh -s --` or set the environment variable: | Flag | Environment variable | Effect | | --------------------- | ----------------------------- | ------------------------------------------------------------------------------ | | `--version vX.Y.Z` | `MELANGE_VERSION` | Install a specific release instead of latest | | `--install-dir DIR` | `MELANGE_INSTALL_DIR` | Binary directory; defaults to `/usr/local/bin`, falling back to `~/.local/bin` | | `--cli-only` | `MELANGE_SKIP_SKILL=1` | Skip the agent skill | | `--skill-only` | `MELANGE_SKIP_CLI=1` | Skip the CLI | | `--agent "A B"` | `MELANGE_SKILL_AGENTS` | Agents to install for; defaults to `universal claude-code` | | `--require-signature` | `MELANGE_REQUIRE_SIGNATURE=1` | Fail unless the release signature is verified | ```sh curl -fsSL https://raw.githubusercontent.com/zetic-ai/melange-cli/main/script/install.sh \ | sh -s -- --version v1.2.3 --agent "universal claude-code codex" ``` The skill is installed with [`npx skills`](https://github.com/vercel-labs/skills) when a recent Node is available, and copied directly into the agent skill directories otherwise. With a current Go toolchain: ```sh go install github.com/zetic-ai/melange-cli/cmd/melange@latest melange version ``` Make sure your Go binary directory is on `PATH`. Download the archive for your operating system and architecture from [GitHub Releases](https://github.com/zetic-ai/melange-cli/releases). Each release includes checksums, a Sigstore bundle, and SBOMs. Place the `melange` binary on `PATH`, then verify the installation: ```powershell melange version ``` Authenticate [#authenticate] Log in with browser OAuth (recommended). The CLI stores the credential in your operating system keyring and refreshes it automatically: ```sh melange auth login ``` For CI or a headless environment, create a personal access token in [Melange Dashboard](https://melange.zetic.ai/) under **Settings โ†’ Personal Access Tokens**, then read it from standard input: ```sh melange auth login --with-token < token.txt ``` Or provide a credential through the environment: ```sh export MELANGE_API_KEY="ztp_your_personal_access_token" ``` Check status [#check-status] ```sh melange auth status ``` Explore the public library [#explore-the-public-library] Start with a read-only search. This does not import, upload, or modify a model. ```sh melange library list --search LFM2.5-350M --json melange library view shinilheo/LFM2.5_350M --json ``` Use `--json` or `--jq` when another program consumes the output. Run `melange library list --help` to see the available filters. Next steps [#next-steps] Install the skill and give an agent safe, structured access to Melange. Browse command groups, output formats, environment variables, and exit codes. View the source, generated command reference, and release artifacts. # MCP server (/melange-cli/mcp) `melange mcp` serves the same Melange operations as MCP tools instead of shelling out. 18 tools over stdio (17 over Streamable HTTP โ€” `upload_model` needs the caller's local files) so MCP clients call Melange directly. The stdio server reuses the CLI's credentials (`MELANGE_API_KEY` or `melange auth login`), resolved lazily on the first tool call. The HTTP server is credential-less: every request must carry its own `Authorization: Bearer `, so one deployment serves many callers. Install the CLI first so `melange` is on your `PATH`, then register it with your client. The full tool catalog and per-transport details are in [`llms.txt`](https://github.com/zetic-ai/melange-cli/blob/main/llms.txt). Transports [#transports] | Transport | Where it runs | Credentials | Tools | | ----------------- | ----------------------- | --------------------------------------------- | --------------------------------- | | `stdio` (default) | your machine | your `MELANGE_API_KEY` / `melange auth login` | 18 | | `http` Streamable | remote host / container | per-request `Authorization: Bearer ` | 17 (`upload_model` is stdio-only) | `upload_model` is stdio-only because the server cannot see the caller's filesystem. `request_model_download` never writes files โ€” it only authorizes. Claude Code [#claude-code] ```sh claude mcp add melange -- melange mcp ``` Verify with `claude mcp list` โ€” the entry should show `โœ” Connected`. Claude Desktop [#claude-desktop] Add to `claude_desktop_config.json` (Settings โ†’ Developer โ†’ Edit Config; macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`, Windows: `%APPDATA%\Claude\claude_desktop_config.json`), then restart Claude Desktop: ```json { "mcpServers": { "melange": { "command": "melange", "args": ["mcp"] } } } ``` Cursor [#cursor] Add to `.cursor/mcp.json` in your project (or `~/.cursor/mcp.json` for all projects): ```json { "mcpServers": { "melange": { "command": "melange", "args": ["mcp"] } } } ``` Remote (Streamable HTTP) [#remote-streamable-http] For remote agent clients, serve the Streamable HTTP transport. The server itself holds no credentials: every request must carry its own token as `Authorization: Bearer `, so one deployment serves many callers. The server speaks plain HTTP; terminate TLS in front of it (load balancer, reverse proxy, or ingress). The `https://` client URLs below assume that. ```sh melange mcp --transport http --listen 0.0.0.0:8080 ``` Claude Code: ```sh claude mcp add --transport http melange https://your-host:8080/ \ --header "Authorization: Bearer ztp_your_personal_access_token" ``` Cursor (`.cursor/mcp.json`): ```json { "mcpServers": { "melange": { "url": "https://your-host:8080/", "headers": { "Authorization": "Bearer ztp_your_personal_access_token" } } } } ``` Claude Desktop registers local stdio servers through `claude_desktop_config.json` (above); remote servers are added as custom connectors in claude.ai settings instead. See `melange mcp --help` for HTTP flags. HTTP flags [#http-flags] | Flag | Purpose | | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--listen 0.0.0.0:8080` | address to listen on (use `0.0.0.0:PORT` in a container) | | `--validate-tokens` | verify each Bearer against the API before serving it | | `--allowed-origins https://app.example.com` | browser Origins allowed (empty rejects all) | | `--resource https://mcp.example.com` | canonical resource URL, enables OAuth audience enforcement + RFC 9728 discovery at `/.well-known/oauth-protected-resource` (also `MELANGE_MCP_RESOURCE`) | `GET /healthz` is an unauthenticated liveness probe. `/.well-known/oauth-protected-resource` is public discovery. Every other path requires `Authorization: Bearer `; missing token is `401 Bearer`, wrong audience/scope is `403 insufficient_scope` (OAuth) or an in-band tool error (PAT passthrough). Tool catalog [#tool-catalog] The catalog is typed and versioned with the OpenAPI spec. Every tool declares `readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint` explicitly, and every `outputSchema` is `type: object`. Byte-exact JSON passthrough preserves API response order and characters (`<`, `&` survive). Stdio 18 vs HTTP 17 is pinned by `internal/mcp/testdata/tools_list_*.json` goldens; a new mutating tool cannot ship without joining the `write` scope. When to use CLI vs MCP [#when-to-use-cli-vs-mcp] * **Prefer MCP** when the server is connected: typed arguments, no shell, confirm gates. * **Use CLI** when there is no tool: `melange auth login/status/token/logout`, `melange model download` to disk, bucketed `.pt2` or `--input-manifest` uploads, `melange api` raw, or when over HTTP you need local files (`upload_model`). All agent rules (non-blocking conversion monitoring, pipeline panel, report templates) apply unchanged whether data came from CLI or MCP โ€” see the `## melange mcp` section of `llms.txt` and the `skills/melange-cli/SKILL.md` skill. Exit codes [#exit-codes] `0` clean disconnect (stdio) or completed drain after `SIGINT/SIGTERM` (http), `1` serve failure, `2` usage, `130` interrupted (or http drain cut short by second signal). `STDOUT` carries JSON-RPC frames only; diagnostics go to `STDERR`. Registry [#registry] `server.json` (`ai.zetic/melange`, `0.5.0`) is the MCP registry entry (`stdio` via `npx` `@zetic-ai/melange-cli`). `npm/package.json` carries `mcpName`. # CLI Reference (/melange-cli/reference) `melange` uses human-readable tables in a terminal and structured output for scripts and agents. Run `melange --help` for every flag and example. Command groups [#command-groups] | Command | Purpose | | ----------------- | ------------------------------------------------------------------------- | | `melange auth` | Sign in, inspect authentication, print the resolved token, or sign out | | `melange library` | Search and inspect public library repositories and providers | | `melange repo` | List, inspect, create, edit, and delete repositories | | `melange model` | Upload, import, inspect, monitor, download, and manage repository models | | `melange report` | Read general, LLM, and package benchmark reports | | `melange deploy` | List deployment options and generate exact SDK deployment guides | | `melange usage` | Inspect current usage and plan quotas | | `melange mcp` | Serve the MCP server (stdio or Streamable HTTP) โ€” 18 tools (17 over HTTP) | | `melange api` | Call a public `/v1` endpoint that does not yet have a dedicated command | `melange model download` is billable. Repository deletion and other consequential commands require confirmation. Agents should resolve the complete target and ask the user before passing `--yes`. Structured output [#structured-output] These flags are shared by server-backed commands: | Flag | Behavior | | ---------------------- | ------------------------------------------------------- | | `--json` | Print the complete documented JSON result | | `--jq EXPRESSION` | Filter JSON and print the selected value | | `--template TEMPLATE` | Format JSON with a Go template | | `--paginate` / `--all` | Fetch every page and merge list results where supported | Data is written to standard output. Progress and diagnostics are written to standard error. Prefer `--json` or `--jq` for scripts and agents. ```sh melange repo list --paginate --jq '.results[].full_name' melange usage quotas --json ``` Human output [#human-output] In a terminal, lists print as aligned tables under a ruled header with a row count, and detail commands print aligned `Label: value` blocks. When output is piped or redirected, the same commands print headerless tab-separated values โ€” stable for scripts. `--format` selects that layout explicitly instead of by detection: | Value | Behavior | | ------- | ------------------------------------------------------------- | | `auto` | Table in a terminal, tab-separated values otherwise (default) | | `table` | Force the table, for example when piping into a pager | | `tsv` | Force tab-separated values in a terminal | `--format` also governs value formatting, so a forced table shows relative times and human byte sizes, and forced `tsv` shows RFC 3339 timestamps and raw byte counts. It never adds color to a non-terminal; color follows `NO_COLOR` and `TERM=dumb`. ```sh melange model list -R acme/whisper --format table | less -R ``` Agents should leave `--format` at `auto` and read `--json`. Repository and model targeting [#repository-and-model-targeting] Repositories use `ACCOUNT/REPO`. Every `melange model` command requires an explicit `-R ACCOUNT/REPO`; model keys are not repository names. ```sh melange model list -R zetic/whisper-tiny --json melange model status MODEL_KEY -R zetic/whisper-tiny --json ``` Public Hugging Face IDs and Melange repository names may look similar. Use the repository and model identifiers returned by Melange instead of guessing. Environment [#environment] | Variable | Purpose | | ---------------------- | ---------------------------------------------------------------------------- | | `MELANGE_API_KEY` | Use an access token or PAT from the environment; highest credential priority | | `MELANGE_API_KEY_FILE` | Read a credential from a file and fail if the file is unavailable | | `MELANGE_HOST` | Override the API host; defaults to `https://api.zetic.ai` | | `MELANGE_API_TIMEOUT` | Set the timeout for an ordinary API call, such as `45s` or `2m` | | `MELANGE_DEBUG` | Log sanitized request and response lines to standard error | | `NO_COLOR` | Disable colored output | | `TERM=dumb` | Disable colored output | Credential precedence is: ```text MELANGE_API_KEY > MELANGE_API_KEY_FILE > OAuth (operating system keyring or config, auto-refreshed) > PAT in the operating system keyring > PAT config fallback ``` Run `melange auth status` to see the active API host and credential source without revealing the token. Use `melange help environment` for config and state-file locations. Exit codes [#exit-codes] | Code | Meaning | Recommended action | | ----- | -------------------------------- | ---------------------------------------------------- | | `0` | Success | Continue | | `1` | API, network, or command failure | Inspect the error; it may be transient | | `2` | Invalid arguments or flags | Fix the command; do not retry unchanged | | `4` | Missing or rejected credentials | Authenticate again | | `130` | Interrupted | Treat as cancellation; upload sessions are preserved | Use `--no-input` when automation must never prompt. Full command help [#full-command-help] Browse the generated page for every command and subcommand. View releases, source code, the agent skill, and issue tracker. Useful built-in topics: ```sh melange help environment melange help formatting melange help exit-codes ``` # Web Dashboard (/model-deployment/dashboard) This guide explains how to deploy your models using the Melange Web Dashboard at [melange.zetic.ai](https://melange.zetic.ai). *** Step 1: Sign Up for Free [#step-1-sign-up-for-free] Access the **[Melange Dashboard](https://melange.zetic.ai)** to start accelerating your AI on-device instantly. * **Completely Free to Start**: No credit card required. * **One-Click Login**: Sign up in seconds using your Google or GitHub account. * **Instant Access**: Immediately start creating projects and generating keys. [**Go to Dashboard (melange.zetic.ai)**](https://melange.zetic.ai) Step 2: Create a Repository [#step-2-create-a-repository] Set up a new repository to manage your model versions: 1. Click the **+** button in the top-left corner. 2. Enter your **repository name** (this will be part of your model identifier: `username/repository_name`). 3. Add an optional **description** to document the repository's purpose. 4. Click **Create** to finalize. Step 3: Upload Your Model [#step-3-upload-your-model] Generate Model Key Deploy your model to the repository: 1. Click the **Upload** button in the top-right corner. 2. Provide the **model path** (local file or URL). 3. Specify the **input path(s)** for your model. 4. Click **Upload** to begin the conversion process. Monitor Deployment Status [#monitor-deployment-status] Your model will go through several stages: | State | Description | Can Be Used? | | -------------- | -------------------------------------------- | ----------------- | | **Converting** | Model is being converted to on-device format | No | | **Optimizing** | Available for testing (not fully tuned) | Yes (sub-optimal) | | **Ready** | Fully optimized and production-ready | Yes | Ensure your inputs are in the correct order. Learn more about input ordering in [Supported Formats](/model-preparation/supported-formats). Step 4: Use Your Model [#step-4-use-your-model] Once your model reaches **Ready** status, integrate it into your application: ```kotlin val model = ZeticMLangeModel( context, "PERSONAL_KEY", "username/repository_name" ) ``` ```swift let model = try ZeticMLangeModel( personalKey: "personalKey", name: "username/repository_name" ) ``` ```dart final model = await ZeticMLangeModel.create( personalKey: 'PERSONAL_KEY', name: 'username/repository_name', ); ``` *** Next Steps [#next-steps] * [Understanding Model Keys](/model-deployment/understanding-model-keys): Learn about model identifiers and personal keys * [Android Setup](/platform-integration/android/setup): Integrate the SDK in your Android app * [iOS Setup](/platform-integration/ios/setup): Integrate the SDK in your iOS app # Performance-Adaptive Deployment (/model-deployment/performance-adaptive-deployment) Melange provides the best user experience by benchmarking AI model performance on a pool of real-world devices. It benchmarks different processors from various manufacturers, including CPU, GPU, and NPU. Based on these results, Melange ensures optimal performance on the deployed user's target device, regardless of the device type. Measurement-Based, Not Rule-Based [#measurement-based-not-rule-based] Traditional deployment uses static rules (e.g., "Use GPU if version > X"). This often fails due to driver fragmentation and thermal throttling. **Melange is different.** We establish ground truth by measuring: * **Actual Latency**: Millisecond-precision inference time measured on physical devices. * **Throughput**: Real-world tokens/frames per second capacity. Based on this data, we identify the specific model binary that yields the highest performance for each device model. Global Deployment Assurance [#global-deployment-assurance] By testing against the fragmented landscape of Android and iOS hardware, we guarantee: * **Guaranteed Runtime Compatibility**: Your model is rigorously verified to load and execute correctly on every variation of Android and iOS targets. * **Adaptive Binary Selection**: The runtime dynamically resolves the exact quantized binary that yields maximum throughput for the specific NPU chipset. * **Optimal Deployment Strategy**: Deployment decisions are governed by deterministic benchmark data from our device farm, eliminating theoretical guesswork. Validation Workflow [#validation-workflow] 1\. Provision Test Environment [#1-provision-test-environment] We instantiate an isolated, on-device runtime environment mirroring the target OS and hardware configuration. 2\. Distributed Workload Execution [#2-distributed-workload-execution] The compilation artifacts, model metadata, and test vectors are dispatched to a distributed device farm. We execute the model on over **200 physical devices** to capture real-world metrics. 3\. Telemetry Analysis and Winner Selection [#3-telemetry-analysis-and-winner-selection] We aggregate the performance data to select the **"Winning Model"** for each device identifier. This determines which compiled binary variant (quantization level, backend, optimization profile) performs best on each specific device. 4\. Automatic Distribution [#4-automatic-distribution] When a user installs your app, the Melange Runtime automatically fetches the "Winning Model" for their device. This creates a seamless, high-performance experience without any manual configuration from the developer. Advanced Telemetry Report (Premium) [#advanced-telemetry-report-premium] We execute profiling for all users to guarantee the best performance of on-device AI applications. However, detailed profiling results are currently available for **Pro+** and **Enterprise** users only. For enterprise customers, we provide detailed profiling reports broken down by **model ร— runtime ร— quantization type ร— chipset ร— device**, enabling granular performance analysis across your entire deployment target matrix. Please [contact us](mailto:contact@zetic.ai) for more information. *** Next Steps [#next-steps] * [Understanding Model Keys](/model-deployment/understanding-model-keys): Model identifiers and versioning * [Device Compatibility](/performance/device-compatibility): Supported NPU chipsets * [Benchmark Methodology](/performance/benchmark-methodology): How benchmarks are measured # Understanding Model Keys (/model-deployment/understanding-model-keys) ZETIC Melange uses two types of keys to manage model access and authentication: **Model Identifiers** and **Personal Keys**. Model Identifier [#model-identifier] Every model in Melange is referenced using the format `username/repository_name`. This identifier is used when initializing the model in your application. ``` username/my-model-repository ``` Repository Structure [#repository-structure] A Repository acts as a version control container for a specific model lineage. It maintains distinct versions of your artifacts. ``` username/my-model-repository โ”œโ”€โ”€ v1 (uploaded 2024-01-15) โ”œโ”€โ”€ v2 (uploaded 2024-02-20) โ† default โ””โ”€โ”€ v3 (uploaded 2024-03-10) โ† latest ``` Versioning Policy [#versioning-policy] * **Implicit Latest**: Clients automatically pull the most recent upload unless pinned. * **Default Pinning**: Administrators can designate a specific stable version as `default` for production channels. * **Immutable History**: All uploaded versions are preserved for rollback capabilities. Using Model Identifiers in Code [#using-model-identifiers-in-code] ```kotlin val model = ZeticMLangeModel( context, BuildConfig.PERSONAL_KEY, "USERID/REPOSITORY_NAME" // Automatically resolves to 'default' or 'latest' ) ``` ```swift let model = try ZeticMLangeModel( personalKey: personalKey, name: "USERID/REPOSITORY_NAME" // Automatically resolves to 'default' or 'latest' ) ``` ```dart final model = await ZeticMLangeModel.create( personalKey: personalKey, name: 'USERID/REPOSITORY_NAME', // Automatically resolves to default or latest ); ``` *** Personal Key [#personal-key] The **Personal Key** is your persistent credential for SDK authentication and API access. Treat it as a high-value secret. Generating a Personal Key [#generating-a-personal-key] 1. Log in to the [Melange Dashboard](https://melange.zetic.ai). 2. Access **Settings** then **Personal Key**. 3. Select **Generate New Key**. 4. **Copy immediately**: the key is displayed only once. Copy Personal Key Store your personal key in a password manager immediately. It cannot be retrieved after the generation dialog is closed. If lost, you must generate a new key. Best Practices [#best-practices] * **Never hardcode** personal keys directly in source code. Use environment variables or build configuration. * **Rotate keys** periodically for security. * **Use separate keys** for development and production environments. Store the key in `local.properties` or `BuildConfig`: ```kotlin val model = ZeticMLangeModel( context, BuildConfig.PERSONAL_KEY, // From build configuration "USERID/REPOSITORY_NAME" ) ``` Store the key in a configuration file excluded from source control: ```swift let model = try ZeticMLangeModel( personalKey: Configuration.personalKey, // From config name: "USERID/REPOSITORY_NAME" ) ``` Store the key in a build-time configuration or secure runtime configuration: ```dart final model = await ZeticMLangeModel.create( personalKey: appConfig.personalKey, name: 'USERID/REPOSITORY_NAME', ); ``` *** Deployment Status [#deployment-status] Monitor the compilation and optimization status of your models on the dashboard: | State | Description | Usable? | | -------------- | ----------------------------------------------------- | ----------------- | | **N/A** | Repository initialized; awaiting upload | No | | **Failed** | Validation or compilation error | No | | **Converting** | Graph lowering and quantization in progress | No | | **Optimizing** | Functional binary available; throughput tuning active | Yes (sub-optimal) | | **Ready** | Fully compiled, tuned, and validated for production | Yes | Models in the **Optimizing** state are executable but may not yet utilize the full NPU throughput. Use **Ready** artifacts for performance benchmarking. *** Next Steps [#next-steps] * [Web Dashboard](/model-deployment/dashboard): Manage repositories and models * [Performance-Adaptive Deployment](/model-deployment/performance-adaptive-deployment): How Melange optimizes for each device # Pre-built Models (/model-preparation/hugging-face-models) ZETIC Melange offers pre-built models that are ready to use without any model preparation or upload steps. Dashboard Pre-built Models [#dashboard-pre-built-models] The [Melange Dashboard](https://melange.zetic.ai) includes a library of pre-built models that have been tested and optimized by the Melange team. * Browse the model library on the Dashboard * Select a model and copy its model key * Use the key directly in your app with `ZeticMLangeModel` or `ZeticMLangeLLMModel` Hugging Face Models (Coming Soon) [#hugging-face-models-coming-soon] Hugging Face model support is currently under development and will be available soon. Melange will support loading models directly from [Hugging Face](https://huggingface.co/) using only the repository ID. *** Next Steps [#next-steps] For implementation details, see the API reference for each model class: * [ZeticMLangeModel (Android)](/api-reference/android/ZeticMLangeModel) / [(iOS)](/api-reference/ios/ZeticMLangeModel): General model inference * [ZeticMLangeLLMModel (Android)](/api-reference/android/ZeticMLangeLLMModel) / [(iOS)](/api-reference/ios/ZeticMLangeLLMModel): LLM inference * [ZeticMLangeHFModel (Android)](/api-reference/android/ZeticMLangeHFModel) / [(iOS)](/api-reference/ios/ZeticMLangeHFModel): Hugging Face model inference * [LLM Inference](/llm-inference/overview): Run large language models on-device # ONNX Models (/model-preparation/onnx) ONNX (Open Neural Network Exchange) is a widely supported format that enables you to use models from PyTorch, TensorFlow, Keras, scikit-learn, and other frameworks with ZETIC Melange. PyTorch (Recommended) [#pytorch-recommended] When starting from a `torch.nn.Module`, exporting directly with `torch.onnx` produces the cleanest ONNX graph for Melange. Prefer this path over multi-step conversions (for example, PyTorch โ†’ TensorFlow โ†’ ONNX), which often introduce extra ops and shape mismatches. If you're going from PyTorch straight to Melange and don't need ONNX for other tooling, the [PyTorch Exported Program (`.pt2`)](/model-preparation/pytorch-export) path is simpler. Use ONNX when you specifically need ONNX. ```python import torch # Load your model torch_model = YourModel() # Replace with your model class torch_model.eval() # Prepare a sample input that matches your model's expected shape sample_input = torch.randn(1, 3, 224, 224) # Export to ONNX torch.onnx.export( torch_model, (sample_input,), "model.onnx", ) ``` For more details, see the [torch.onnx documentation](https://pytorch.org/docs/stable/onnx.html). TensorFlow / Keras [#tensorflow--keras] Use `tf2onnx` to convert TensorFlow and Keras models to ONNX format. Installation [#installation] ```bash pip install tf2onnx ``` From a SavedModel Directory [#from-a-savedmodel-directory] ```bash python -m tf2onnx.convert --saved-model saved_model_dir --output model.onnx --opset 13 ``` From a Keras Model (Python API) [#from-a-keras-model-python-api] ```python import tensorflow as tf import tf2onnx # Load your model model = tf.keras.models.load_model("my_model.h5") # Convert to ONNX spec = (tf.TensorSpec((1, 224, 224, 3), tf.float32, name="input"),) output_path = "model.onnx" model_proto, _ = tf2onnx.convert.from_keras(model, input_signature=spec, output_path=output_path) ``` From a TFLite Model [#from-a-tflite-model] ```bash python -m tf2onnx.convert --tflite model.tflite --output model.onnx --opset 13 ``` We recommend using opset 12 or higher for the best compatibility with Melange's compiler. *** Scikit-Learn [#scikit-learn] Use `skl2onnx` to convert scikit-learn models to ONNX format. Installation [#installation-1] ```bash pip install skl2onnx ``` Conversion [#conversion] ```python from skl2onnx import convert_sklearn from skl2onnx.common.data_types import FloatTensorType initial_type = [('float_input', FloatTensorType([None, 4]))] onx = convert_sklearn(model, initial_types=initial_type) with open("model.onnx", "wb") as f: f.write(onx.SerializeToString()) ``` *** Saving Sample Inputs [#saving-sample-inputs] After converting your model, save sample inputs as NumPy files for upload: ```python import numpy as np # Create a sample input matching your model's expected shape sample_input = np.random.randn(1, 224, 224, 3).astype(np.float32) np.save("input.npy", sample_input) ``` *** Simplifying ONNX Models [#simplifying-onnx-models] If you encounter conversion issues, use `onnx-simplifier` to reduce complex subgraphs: ```bash pip install onnxsim onnxsim input_model.onnx output_model.onnx ``` Simplifying your ONNX model can resolve many compilation issues by removing redundant operations and folding constant expressions. *** Other Frameworks [#other-frameworks] For other frameworks that support ONNX export, refer to the [ONNX Tutorials](https://github.com/onnx/tutorials#converting-to-onnx-format). *** Next Steps [#next-steps] * [Supported Formats](/model-preparation/supported-formats): Verify input order and shapes * [Web Dashboard](/model-deployment/dashboard): Upload your ONNX model # PyTorch Export (/model-preparation/pytorch-export) This guide covers how to export PyTorch models for use with ZETIC Melange. PyTorch Exported Program (.pt2) [#pytorch-exported-program-pt2] PyTorch 2.0+ introduces the `torch.export` API, which produces a fully serialized computation graph. This is the recommended format for Melange. PyTorch Exported Program requires **PyTorch >= 2.9**. Earlier versions may produce incompatible graphs or fail during export. Verify your version with `python -c "import torch; print(torch.__version__)"`. ```python import torch import numpy as np # Load your model torch_model = YourModel() # Replace with your model class torch_model.eval() # Prepare sample inputs sample_input = torch.randn(1, 3, 224, 224) # Match your model's input shape # (1) Export the model exported_program = torch.export.export(torch_model, (sample_input,)) torch.export.save(exported_program, "model.pt2") # (2) Save your sample inputs for Melange np_input = sample_input.detach().numpy() np.save("input.npy", np_input) ``` For more details, refer to the [torch.export documentation](https://docs.pytorch.org/docs/stable/user_guide/torch_compiler/export.html). *** Saving Inputs [#saving-inputs] Both export methods require saving sample inputs as NumPy `.npy` files. These inputs serve two purposes: 1. **Shape definition**: They tell Melange the exact tensor dimensions to compile for. 2. **Validation**: They are used during the compilation process to verify correctness. If your model has multiple inputs, save each one separately: ```python np.save("input_0.npy", input_tensor_0.detach().numpy()) np.save("input_1.npy", input_tensor_1.detach().numpy()) ``` The order of inputs matters. See [Supported Formats](/model-preparation/supported-formats) for details on input ordering. *** Next Steps [#next-steps] * [Supported Formats](/model-preparation/supported-formats): Verify input order and shapes * [Web Dashboard](/model-deployment/dashboard): Upload your exported model # Supported Model Formats (/model-preparation/supported-formats) Melange supports two model formats for on-device deployment. Choose the format that best matches your training framework. Supported Formats [#supported-formats] | Format | Extension | Status | Recommended | | ------------------------ | --------- | --------- | ----------- | | PyTorch Exported Program | `.pt2` | Supported | Yes | | ONNX | `.onnx` | Supported | Yes | Choosing a Format [#choosing-a-format] * **PyTorch users**: Use [PyTorch Exported Program](/model-preparation/pytorch-export) (`.pt2`). Requires **PyTorch >= 2.9**. * **TensorFlow / Keras / scikit-learn users**: Convert to [ONNX](/model-preparation/onnx) format. Input Requirements [#input-requirements] Regardless of format, all models require: 1. **NumPy input files** (`.npy`): Sample inputs that define the expected tensor shapes and data types. 2. **Fixed input shapes**: NPU compilation hard-codes input shapes for maximum throughput. Even if your original model supports dynamic sizes, the Melange-compiled model will accept only the exact shape provided during upload. NPU compilation hard-codes input shapes to maximize throughput. Even if your original model supports dynamic sizes, the accelerated Melange model will accept **only the exact shape** of the sample input provided during upload. Graph Constraints [#graph-constraints] Melange compiles your model into a **static computation graph**, which places constraints on the graph itself beyond just input shapes. Address these before exporting. Fixed Shapes Throughout the Graph [#fixed-shapes-throughout-the-graph] Every tensor shape inside the graph โ€” not only the inputs โ€” must resolve to a constant at export time. If any intermediate operation produces a shape that depends on runtime values, compilation will fail. Common sources of dynamic internal shapes: * Data-dependent control flow (`if` / `while` conditioned on tensor values) * Operations like `nonzero`, `masked_select`, or slicing with indices that aren't compile-time constants * Variable-length sequences without padding Refactor these into fixed-shape equivalents (for example, pad to a maximum length and apply a mask instead of gathering a dynamic subset) so that all shapes become constants during export. No Complex Number Support [#no-complex-number-support] The Melange backend does **not support complex dtypes** (`complex64`, `complex128`). If your model uses complex tensors โ€” for instance, FFT outputs โ€” rewrite those portions to carry real and imaginary parts as separate float tensors. For a complex tensor of shape `(N,)`, use a float tensor of shape `(N, 2)` (or two tensors of shape `(N,)` for the real and imaginary parts) and implement complex arithmetic with real-valued operations. Verifying Input Order and Shapes [#verifying-input-order-and-shapes] Melange compiles your model into a static hardware graph. This means consistent **input order** and **input shapes** are mandatory for execution. Why Order Matters [#why-order-matters] The internal computation graph expects data in specific slots (e.g., `input_zero` at index 0, `input_one` at index 1). If you swap them during upload or inference, the model will produce garbage results or crash. Inspecting with Netron [#inspecting-with-netron] We recommend using [Netron](https://github.com/lutzroeder/netron) to visualize your model's input signature: 1. Open your model (`.pt2` or `.onnx`) in Netron. 2. Locate the **Input Nodes** at the top of the graph. 3. Note the vertical order: the top-most input is **Index 0**, the next is **Index 1**, and so on. You **must** provide inputs in this exact order when: 1. Uploading the model and sample inputs via the Melange Dashboard. 2. Calling the `run()` function in your Android/iOS app. Checking input sequence with Netron *** Next Steps [#next-steps] * [PyTorch Export](/model-preparation/pytorch-export): Export PyTorch models to `.pt2` * [ONNX Models](/model-preparation/onnx): Convert TensorFlow, Keras, and scikit-learn models * [Pre-built Models](/model-preparation/hugging-face-models): Use ready-to-run models from Dashboard or Hugging Face # Benchmark Methodology (/performance/benchmark-methodology) Melange ensures optimal on-device performance through rigorous benchmarking on physical hardware. This page explains our methodology. Overview [#overview] Unlike traditional approaches that rely on static rules or theoretical specifications, Melange performs **on-target performance measurement** to empirically determine the optimal model for every device. Device Farm [#device-farm] We maintain a distributed device farm of over **200 physical devices** spanning: * Qualcomm Snapdragon * MediaTek Dimensity * Samsung Exynos * Apple A-series and M-series chips Each device runs the exact OS version and driver configuration that real users encounter. What We Measure [#what-we-measure] For each model and device combination, we capture: | Metric | Description | | ------------------------------- | ----------------------------------------------------- | | **Inference Latency** | Millisecond-precision end-to-end inference time | | **Throughput** | Frames per second (vision) or tokens per second (LLM) | | **SNR (Signal-to-Noise Ratio)** | Accuracy degradation compared to the original model | Validation Workflow [#validation-workflow] 1\. Provision Test Environment [#1-provision-test-environment] An isolated, on-device runtime environment is instantiated mirroring the target OS and hardware configuration. 2\. Distributed Workload Execution [#2-distributed-workload-execution] Compilation artifacts, model metadata, and test vectors are dispatched to the device farm. The model is executed on each device to capture real-world metrics. 3\. Telemetry Analysis and Winner Selection [#3-telemetry-analysis-and-winner-selection] Performance data is aggregated to select the **"Winning Model"** for each device identifier. This determines which compiled binary variant: quantization level, backend, and optimization profile: performs best on each specific device. 4\. Automatic Distribution [#4-automatic-distribution] When a user installs your app, the Melange Runtime automatically fetches the winning model for their device. No developer configuration is needed. Why Physical Devices Matter [#why-physical-devices-matter] Theoretical performance metrics often fail in practice due to: * **Driver fragmentation**: Different GPU/NPU driver versions behave differently * **Thermal throttling**: Sustained workloads cause performance degradation * **Memory constraints**: Real-world memory pressure affects behavior * **OS-level scheduling**: Background processes impact inference timing By measuring on physical devices, we capture all of these real-world factors. Advanced Telemetry (Premium) [#advanced-telemetry-premium] Profiling is executed for all users to guarantee optimal performance. Detailed profiling reports are available for **Pro+** and **Enterprise** tier users. For enterprise customers, we provide detailed profiling reports broken down by **model ร— runtime ร— quantization type ร— chipset ร— device**, enabling granular performance analysis across your entire deployment target matrix. Please [contact us](mailto:contact@zetic.ai) for more information. *** Next Steps [#next-steps] * [Device Compatibility](/performance/device-compatibility): Supported NPU chipsets * [Performance-Adaptive Deployment](/model-deployment/performance-adaptive-deployment): How results are applied * [Inference Mode Selection](/how-to-guides/inference-mode-selection): Manual mode override # Benchmarks (/performance/benchmarks) Melange delivers hardware-accelerated inference by automatically selecting the optimal backend (CPU, GPU, or NPU) for each device. The benchmarks on this page demonstrate the real-world performance gains achieved through this approach. Benchmark Methodology [#benchmark-methodology] All benchmarks are measured on a **physical device farm of 200+ real devices**, not emulators or simulators. This ensures the numbers reflect actual production performance, accounting for real-world factors like driver fragmentation, thermal throttling, and memory constraints. Each model is profiled across CPU, GPU, and NPU backends on every device. Melange then selects the fastest backend automatically at runtime. For a deeper look at how Melange profiles and selects the optimal model binary per device, see [Performance-Adaptive Deployment](/model-deployment/performance-adaptive-deployment). *** YOLOv11 Object Detection [#yolov11-object-detection] The following table shows inference latency for YOLOv11 across a representative set of devices. The **Speedup** column compares the fastest accelerated backend (GPU or NPU) against the CPU baseline. | Device | SoC | CPU | GPU | NPU | Speedup | | ----------------- | -------- | --------- | -------- | --------- | ------- | | Galaxy A34 | MediaTek | 172.08 ms | 96.38 ms | 249.41 ms | x1.79 | | Galaxy S22 5G | Qualcomm | 79.76 ms | 36.99 ms | 8 ms | x9.97 | | Galaxy S23 | Qualcomm | 89.56 ms | 27.5 ms | 5.24 ms | x17.09 | | Galaxy S24+ | Qualcomm | 60.43 ms | 21.46 ms | 3.92 ms | x15.42 | | Galaxy S25 | Qualcomm | 53.69 ms | 17.22 ms | 3.72 ms | x14.43 | | iPhone 12 | Apple | 123.12 ms | 22.73 ms | 3.51 ms | x35.08 | | iPhone 14 | Apple | 111.29 ms | 15.75 ms | 3.75 ms | x29.68 | | iPhone 15 Pro Max | Apple | 96.36 ms | 7.72 ms | 2.05 ms | x47.00 | | iPhone 16 | Apple | 102.09 ms | 7.9 ms | 1.9 ms | x53.73 | On some devices (e.g., Galaxy A34 with MediaTek SoC), the NPU backend is slower than GPU due to limited NPU driver support. Melange detects this automatically and routes inference to the faster backend. Key Takeaways [#key-takeaways] * **NPU acceleration delivers up to 53x speedup** over CPU on supported devices (iPhone 16). * **Apple Neural Engine** consistently outperforms all other NPU implementations, achieving sub-2ms inference on recent iPhones. * **Qualcomm NPU** shows strong performance on flagship devices (Galaxy S22 and later), with 8ms or faster inference. * **MediaTek NPU** support varies: Melange automatically falls back to GPU when NPU is slower, as seen on the Galaxy A34. Full Benchmark Report [#full-benchmark-report] The complete benchmark report with additional devices, models, and detailed profiling data is available on the Melange Dashboard: [View YOLOv11 Benchmark Report](https://melange.zetic.ai/p/Steve/YOLOv11_comparison?tab=report\&version=1) *** How to Read These Numbers [#how-to-read-these-numbers] * **CPU**: Inference using standard CPU execution. This is the baseline that any mobile device can run. * **GPU**: Inference using the device's GPU compute capabilities (Metal on iOS, OpenCL/Vulkan on Android). * **NPU**: Inference using the dedicated Neural Processing Unit (Neural Engine on iOS, Hexagon/APU on Android). * **Speedup**: Ratio of CPU latency to the fastest accelerated backend latency. Higher is better. You can profile your own models across the full device farm by deploying through the [Melange Dashboard](https://melange.zetic.ai). Melange automatically benchmarks and selects the optimal backend for each target device. *** See Also [#see-also] * [Performance-Adaptive Deployment](/model-deployment/performance-adaptive-deployment): How Melange selects the optimal binary per device * [Inference Mode Selection](/how-to-guides/inference-mode-selection): Controlling speed vs. accuracy tradeoffs # Device Support (/performance/device-compatibility) ZETIC Melange broadly supports mobile devices across Android and iOS. We are continuously expanding device and NPU support. NPU Acceleration [#npu-acceleration] Melange automatically leverages NPU hardware when available for faster inference. Supported NPU platforms include: * **Qualcomm Snapdragon** (Hexagon HTP/DSP) * **Apple Neural Engine** * **Google Tensor** (Enterprise) * **MediaTek Dimensity** (Enterprise) * **Samsung Exynos** (Enterprise) Google Tensor, MediaTek, and Samsung Exynos NPU support are available for enterprise customers. [Contact us](mailto:contact@zetic.ai) for access. GPU and CPU Fallback [#gpu-and-cpu-fallback] When NPU hardware is not available, Melange automatically falls back to GPU or CPU execution on both Android and iOS. Your app will work correctly on any device meeting the minimum requirements โ€” NPU simply provides the best performance. Minimum Requirements [#minimum-requirements] Android [#android] * Minimum SDK 24 (Android 7.0) * Physical device (emulators do not have NPU hardware) iOS [#ios] * iOS 16.6+ * Physical device required (simulators do not have Neural Engine) We are actively expanding support to additional platforms and chipsets. If you need support for a specific device, [contact us](mailto:contact@zetic.ai). *** Next Steps [#next-steps] * [Benchmark Methodology](/performance/benchmark-methodology): How performance is measured * [Performance-Adaptive Deployment](/model-deployment/performance-adaptive-deployment): Automatic device optimization * [Performance Optimization](/how-to-guides/performance-optimization): Tips for best results # Quick Start: Your First Inference in 5 Minutes (/quick-start) Get a working on-device AI inference running on your phone in under 5 minutes. No account required for the demo. What You'll Build [#what-youll-build] By the end of this guide, you will have a working app that runs **YOLOv11 object detection** directly on your device's NPU. The model runs entirely on-device with zero cloud dependency. We provide a pre-configured demo model key `Steve/YOLOv11_comparison` so you can try Melange immediately. No sign-up, no dashboard, no waiting. Prerequisites [#prerequisites] * **Android Studio** Arctic Fox or later * A physical Android device (emulators do not have NPU hardware) * Minimum SDK 24 (Android 7.0) * **Xcode** 14 or later * A physical iOS device (iPhone 8 or later recommended) * iOS 16.6+ * **Flutter** 3.35.0 or later * **Dart** 3.11.5 or later * A physical Android or iOS device * Android minimum SDK 24 or iOS deployment target 16.6+ *** Option A: Run the Demo Model (Recommended) [#option-a-run-the-demo-model-recommended] Use the pre-configured YOLOv11 model to get started immediately. Add the SDK [#add-the-sdk] Add the Melange dependency to your app-level `build.gradle`: ```groovy // build.gradle (app level) android { ... packagingOptions { jniLibs { useLegacyPackaging true } } } dependencies { implementation("com.zeticai.mlange:mlange:1.10.0") } ``` The `useLegacyPackaging true` setting is **required**. Without it, the native NPU drivers will not load correctly and you will get a JNI library loading error. Add the Melange Swift Package via Xcode: 1. Open your project in Xcode 2. Go to **File** then **Add Package Dependencies** 3. Enter the package URL: `https://github.com/zetic-ai/ZeticMLangeiOS` 4. Set the dependency rule to **Exact Version** `1.10.0` 5. Select your app target and click **Add Package** Add the Flutter package to your `pubspec.yaml`: ```yaml dependencies: zetic_mlange: ^1.10.0 ``` Then fetch packages: ```bash flutter pub get ``` Complete the Android or iOS platform setup in the [Flutter setup guide](/platform-integration/flutter/setup). Initialize and Run Inference [#initialize-and-run-inference] ```kotlin // MainActivity.kt val model = ZeticMLangeModel(this, PERSONAL_KEY, "Steve/YOLOv11_comparison") val outputs = model.run(inputs) ``` Replace `PERSONAL_KEY` with any string for the demo (e.g., `"demo"`), and prepare your `inputs` as an `Array` matching the model's expected input shape. ```swift // ViewController.swift let model = try await ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "Steve/YOLOv11_comparison") let outputs = try model.run(inputs: inputs) ``` Replace `PERSONAL_KEY` with any string for the demo (e.g., `"demo"`), and prepare your `inputs` as a `[Tensor]` matching the model's expected input shape. ```dart import 'package:zetic_mlange/zetic_mlange.dart'; final model = await ZeticMLangeModel.create( personalKey: 'demo', name: 'Steve/YOLOv11_comparison', ); final outputs = model.run(inputs); ``` Prepare `inputs` as a `List` matching the model's expected input shape. Build and Run [#build-and-run] Build the project and run it on your physical device. The first launch will download and cache the optimized model binary for your specific hardware. Subsequent launches will be instant. *** Option B: Use Your Own Model [#option-b-use-your-own-model] Ready to deploy your own model? Follow these steps: Prepare your model [#prepare-your-model] Export to ONNX or PyTorch Exported Program format. See [Model Preparation](/model-preparation/supported-formats). Upload and compile [#upload-and-compile] Use the [Melange Dashboard](/model-deployment/dashboard) to upload your model and get your keys. Integrate [#integrate] Replace the demo model name with your own `MODEL_NAME` and `PERSONAL_KEY`. *** What Just Happened? [#what-just-happened] When you ran the code above, Melange executed a three-step workflow behind the scenes: 1. **Model Download**: The SDK fetched the pre-compiled, hardware-optimized model binary from the Melange CDN. This binary was already compiled for your specific device's NPU during the model preparation phase. 2. **NPU Context Creation**: Melange initialized the appropriate hardware accelerator (Qualcomm HTP, MediaTek APU, Samsung DSP, or Apple Neural Engine) and loaded the model into NPU memory using zero-copy memory mapping. 3. **Inference Execution**: Your input tensor was processed through the NPU-accelerated computation graph, and the output tensor was returned. No data left the device. *** Next Steps [#next-steps] Now that you have a working inference, explore further: Full source code for demo applications (YOLOv11, Face Detection, Whisper, and more). Build complete apps with pre/post-processing pipelines. Learn how to prepare and optimize your own models. Full SDK documentation for Android and iOS. # Prerequisites (/quick-start/prerequisites) Before you start building with ZETIC Melange, make sure you have the following tools and requirements ready. Development Environment [#development-environment] | Requirement | Details | | ------------------- | -------------------------------------------- | | **IDE** | Android Studio Arctic Fox or later | | **Physical device** | Required: emulators do not have NPU hardware | | **Minimum SDK** | API 24 (Android 7.0) | | **Build system** | Gradle with Groovy or Kotlin DSL | | **Language** | Kotlin (recommended) or Java | | Requirement | Details | | ------------------- | ---------------------------------------------- | | **IDE** | Xcode 14 or later | | **Physical device** | Required: simulators do not have Neural Engine | | **Minimum OS** | iOS 16.6+ | | **Package manager** | Swift Package Manager (built into Xcode) | | **Language** | Swift | | Requirement | Details | | ------------------------- | ----------------------------------------------------------------------------------- | | **Flutter** | 3.35.0+ | | **Dart** | 3.11.5+ | | **Physical device** | Required: Android emulators and iOS simulators do not provide the same NPU hardware | | **Android minimum SDK** | API 24 (Android 7.0) | | **iOS deployment target** | iOS 16.6+ | | **Package dependency** | `zetic_mlange: ^1.10.0` | Melange Account [#melange-account] 1. **Sign up** at [melange.zetic.ai](https://melange.zetic.ai) using your Google or GitHub account. It is free and requires no credit card. 2. **Generate a Personal Key** from the Dashboard under **Settings** then **Personal Key**. For the quick start demo, you can skip account setup and use the pre-configured demo model key `Steve/YOLOv11_comparison`. Model Preparation (Optional) [#model-preparation-optional] If you plan to deploy your own model, you will also need: * A trained model in a [supported format](/model-preparation/supported-formats): `.pt2` (recommended) or `.onnx` * Sample inputs saved as NumPy `.npy` files Network Access [#network-access] The Melange SDK requires internet access for: * **Initial model download**: The optimized model binary is downloaded on first use and cached locally. * **Dashboard access**: For model management and key generation. After the initial download, inference runs entirely offline on-device. *** Next Steps [#next-steps] * [Quick Start](/quick-start): Run your first inference in 5 minutes * [Upload Your Model](/quick-start/upload-your-model): Deploy your own model * [Android Setup](/platform-integration/android/setup): Full Android integration guide * [iOS Setup](/platform-integration/ios/setup): Full iOS integration guide * [Flutter Setup](/platform-integration/flutter/setup): Full Flutter integration guide # Upload Your Model (/quick-start/upload-your-model) Ready to deploy your own model? This guide walks you through preparing and uploading your custom model to Melange. Step 1: Prepare Your Model [#step-1-prepare-your-model] Export your model to a supported format: * **PyTorch Exported Program** (`.pt2`): Recommended * **ONNX** (`.onnx`): Supported Save sample inputs as NumPy `.npy` files alongside your model. For detailed export instructions, see [Model Preparation](/model-preparation/supported-formats). Step 2: Upload Your Model [#step-2-upload-your-model] 1. Log in to the [Melange Dashboard](https://melange.zetic.ai). 2. Create a new repository by clicking the **+** button. 3. Click **Upload** and provide your model file and input files. 4. Wait for the model to compile. Status will progress from **Converting** to **Optimizing** to **Ready**. For details, see [Web Dashboard](/model-deployment/dashboard). Ensure your input tensor shapes exactly match the shapes of the `.npy` files you upload. Melange compiles models with fixed input shapes for NPU optimization. *** Next Steps [#next-steps] Once your model reaches **Ready** status, integrate it into your app: * [Android Integration Guide](/platform-integration/android/setup): Set up inference on Android * [iOS Integration Guide](/platform-integration/ios/setup): Set up inference on iOS * [Understanding Model Keys](/model-deployment/understanding-model-keys): Versioning and key management * [Inference Mode Selection](/how-to-guides/inference-mode-selection): Optimize for speed or accuracy # Changelog (/release-notes/changelog) This page tracks release notes and version history for the ZETIC Melange SDK and platform. *** 1.9.1 [#191] ZeticMLange Flutter 1.9.1 aligns the RAG API with Android and iOS by moving RAG generation to `RagPipeline`. What Changed [#what-changed] * Added Flutter `RagPipeline` as the public RAG generation entry point. * Removed model-level Flutter RAG helper methods from `ZeticMLangeLLMModel`. * Updated Flutter RAG documentation and examples to the composition-based API. Install [#install] | Platform | Install | | -------- | ---------------------- | | Flutter | `zetic_mlange: ^1.9.1` | *** 1.9.0 [#190] ZeticMLange 1.9.0 expands on-device AI across Android, iOS, and Flutter with vision-language support, function calling, RAG, and stronger runtime reliability. What Changed [#what-changed-1] * LFM-VL support for local image + text vision-language models on Android, iOS, and Flutter. * Function calling APIs for connecting model output to app-defined tools. * RAG APIs for retrieval and context-grounded generation workflows. * KV state persistence on native Android and iOS for save, load, and reset flows. * More reliable model loading and inference through backend selection caching, lifecycle cleanup, native handle ownership, and Flutter image-copy improvements. Install [#install-1] | Platform | Install | | -------- | ------------------------------------------------------- | | Android | `implementation("com.zeticai.mlange:mlange:1.9.0")` | | iOS | `https://github.com/zetic-ai/ZeticMLangeiOS` at `1.9.0` | | Flutter | `zetic_mlange: ^1.9.0` | *** Earlier 2025 Updates [#earlier-2025-updates] LLM Inference Engine [#llm-inference-engine] * Added support for `google/gemma-3-4b-it`. * Added support for `LiquidAI/LFM2.5-1.2B-Instruct`. * KV cache cleanup policy configuration with `LLMKVCacheCleanupPolicy`. * Download progress callback support. * Speed inference mode with `LLMModelMode.RUN_SPEED`. Hugging Face Integration [#hugging-face-integration] * `ZeticMLangeHFModel` for loading models directly from Hugging Face. * Automatic download, compilation, and caching. General SDK [#general-sdk] * PyTorch Exported Program (`.pt2`) support. * Inference mode selection with `RUN_AUTO`, `RUN_SPEED`, and `RUN_ACCURACY`. * Performance-adaptive deployment across real devices. * Docs site migration to `docs.zetic.ai`. For the latest updates and announcements, follow us on [GitHub](https://github.com/zetic-ai) and join our [Discord community](https://discord.gg/q6vW4UscRY). *** SDK Repositories [#sdk-repositories] | Platform | Repository | Install | | -------- | ------------------------------------------------------------ | --------------------------------------------------- | | Android | [Maven Central](https://central.sonatype.com/) | `implementation("com.zeticai.mlange:mlange:1.9.0")` | | iOS | [ZeticMLangeiOS](https://github.com/zetic-ai/ZeticMLangeiOS) | Swift Package Manager `1.9.0` | | Flutter | [pub.dev](https://pub.dev/packages/zetic_mlange) | `zetic_mlange: ^1.9.1` | # Migration Guide: MLange โ†’ Melange (/release-notes/migration-mlange-to-melange) ZETIC has rebranded **MLange** to **Melange**. This is a branding-only change: there are no code changes, API modifications, or breaking changes required on your end. What Changed [#what-changed] The product name has been updated across all documentation, marketing materials, and the web dashboard. The underlying SDK, API surface, package names, and class names remain exactly the same. | Area | Before | After | | ------------------------ | ----------------- | ------------------ | | Product name | MLange | Melange | | Documentation references | "ZETIC MLange" | "ZETIC Melange" | | Dashboard URL | `mlange.zetic.ai` | `melange.zetic.ai` | What Stayed the Same [#what-stayed-the-same] | Area | Value | | ------------------- | ----------------------------------------- | | Android package | `com.zeticai.mlange` | | iOS package | `ZeticMLange` | | Android class names | `ZeticMLangeModel`, `ZeticMLangeLLMModel` | | iOS class names | `ZeticMLangeModel`, `ZeticMLangeLLMModel` | | Gradle dependency | `com.zeticai.mlange:mlange:+` | | SPM repository | `github.com/zetic-ai/ZeticMLangeiOS` | Do I Need to Change My Code? [#do-i-need-to-change-my-code] **No.** All SDK package names, class names, method signatures, and dependency coordinates remain identical. Your existing code will continue to work without any modifications. The class names retain the original `MLange` spelling (e.g., `ZeticMLangeModel`) for backward compatibility. This is intentional and will not change. What You Might Want to Update [#what-you-might-want-to-update] While no code changes are required, you may want to update references in your own documentation or comments: * Internal docs or READMEs that reference "MLange" can be updated to "Melange" for consistency. * User-facing strings in your app (e.g., "Powered by ZETIC MLange") can be updated to "Powered by ZETIC Melange." If you encounter any documentation that still references the old "MLange" branding, the technical content remains accurate: only the product name has changed. Summary [#summary] This rebrand is cosmetic only. No action is required from developers. All existing integrations, API keys, dashboard URLs, and SDK dependencies continue to work without modification. *** See Also [#see-also] * [Quick Start](/quick-start): Quick start guide using the current Melange branding * [Android Integration Guide](/platform-integration/android/setup): Android setup with Melange * [iOS Integration Guide](/platform-integration/ios/setup): iOS setup with Melange # Community (/resources/community) Connect with other developers building on-device AI applications with ZETIC Melange. Discord [#discord] Join our Discord server for real-time discussions, support, and announcements: [**Join Discord**](https://discord.gg/q6vW4UscRY) * Get help with integration issues * Share your projects and use cases * Discuss on-device AI best practices * Get early access to new features GitHub [#github] Follow our open-source repositories and contribute: * [github.com/zetic-ai](https://github.com/zetic-ai): Organization page * [ZETIC\_Melange\_apps](https://github.com/zetic-ai/ZETIC_Melange_apps): Sample applications Use GitHub Issues to: * Report bugs * Request features * Ask technical questions Hugging Face [#hugging-face] Explore and use pre-validated models: * [huggingface.co/zetic-ai](https://huggingface.co/zetic-ai) Contact [#contact] For direct communication with the ZETIC team: * **Email**: [contact@zetic.ai](mailto:contact@zetic.ai) * **Enterprise inquiries**: [contact@zetic.ai](mailto:contact@zetic.ai) *** Related [#related] * [GitHub Repositories](/resources/github): Open-source code and samples * [Support](/resources/support): Get technical help # GitHub Repositories (/resources/github) Explore ZETIC Melange's open-source repositories for SDKs, sample applications, and templates. Organization [#organization] * [github.com/zetic-ai](https://github.com/zetic-ai): ZETIC AI GitHub organization SDKs [#sdks] | Repository | Description | | ------------------------------------------------------------ | ------------------------------- | | [ZeticMLangeiOS](https://github.com/zetic-ai/ZeticMLangeiOS) | iOS SDK (Swift Package Manager) | The Android SDK is distributed via Maven Central: `com.zeticai.mlange:mlange`. Sample Applications [#sample-applications] | Repository | Description | | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [ZETIC\_Melange\_apps](https://github.com/zetic-ai/ZETIC_Melange_apps) | Complete sample apps for Android and iOS including object detection, face detection, face landmark, face emotion recognition, audio classification, and speech recognition | LLM Templates [#llm-templates] Build on-device chat applications with these starter templates: | Repository | Platform | | -------------------------------------------------------------------------------------------------- | ---------------- | | [zetic-llm-android-kotlin-template](https://github.com/zetic-ai/zetic-llm-android-kotlin-template) | Android (Kotlin) | | [zetic-llm-ios-swift-template](https://github.com/zetic-ai/zetic-llm-ios-swift-template) | iOS (Swift) | | [zetic-llm-flutter-template](https://github.com/zetic-ai/zetic-llm-flutter-template) | Flutter | | [zetic-llm-react-native-template](https://github.com/zetic-ai/zetic-llm-react-native-template) | React Native | Hugging Face [#hugging-face] * [huggingface.co/zetic-ai](https://huggingface.co/zetic-ai): Pre-validated models for Melange *** Related [#related] * [Community](/resources/community): Join the developer community * [Support](/resources/support): Get help # Support (/resources/support) Need help with ZETIC Melange? Here are the best ways to get support. Self-Service Resources [#self-service-resources] Start with these resources to find answers quickly: | Resource | Best For | | ------------------------------------------------------------------- | ---------------------------------- | | [Common Errors](/troubleshooting/common-errors) | Most frequently encountered issues | | [Android Issues](/troubleshooting/android-issues) | Android-specific problems | | [iOS Issues](/troubleshooting/ios-issues) | iOS-specific problems | | [Model Conversion Issues](/troubleshooting/model-conversion-issues) | Upload and compilation problems | | [FAQ](/troubleshooting/faq) | General questions about Melange | Community Support [#community-support] Discord [#discord] The fastest way to get help from the community and the ZETIC team: [**Join Discord**](https://discord.gg/q6vW4UscRY) GitHub Issues [#github-issues] For bug reports and feature requests, use GitHub Issues on the relevant repository: * [ZETIC\_Melange\_apps](https://github.com/zetic-ai/ZETIC_Melange_apps/issues): Sample app issues * [ZeticMLangeiOS](https://github.com/zetic-ai/ZeticMLangeiOS/issues): iOS SDK issues Direct Support [#direct-support] Email [#email] For technical support, enterprise inquiries, or partnership discussions: * **Technical support**: [bug\_report@zetic.ai](mailto:bug_report@zetic.ai) * **Enterprise / Sales**: [contact@zetic.ai](mailto:contact@zetic.ai) When Contacting Support [#when-contacting-support] To help us resolve your issue faster, include: * Your platform (Android/iOS) and OS version * Device model and chipset (if known) * The model format you are using (`.pt2`, `.onnx`) * Error messages or stack traces * Steps to reproduce the issue *** Related [#related] * [Community](/resources/community): Join the developer community * [GitHub Repositories](/resources/github): Open-source code and samples # Android Issues (/troubleshooting/android-issues) This page covers Android-specific issues you may encounter when integrating ZETIC Melange. JNI Library Loading Failure [#jni-library-loading-failure] **Symptoms:** * `java.lang.UnsatisfiedLinkError: dlopen failed` * `java.lang.UnsatisfiedLinkError: couldn't find native library` * App crashes immediately on `ZeticMLangeModel` initialization **Cause:** The native C++ NPU driver libraries are not being extracted correctly from the APK. **Solution:** Add `useLegacyPackaging` to your app-level `build.gradle`: ```groovy android { ... packagingOptions { jniLibs { useLegacyPackaging true } } } ``` ```kotlin android { ... packaging { jniLibs { useLegacyPackaging = true } } } ``` After adding this, perform a **clean build** (Build then Clean Project, then Rebuild Project). This setting tells the Android build system to extract native libraries from the APK instead of loading them compressed. Melange's NPU drivers require this to function correctly. *** Build Configuration Issues [#build-configuration-issues] Minimum SDK Version [#minimum-sdk-version] **Symptom:** Build fails with SDK version incompatibility. **Solution:** Ensure your `minSdkVersion` is 24 or higher: ```groovy android { defaultConfig { minSdkVersion 24 } } ``` ProGuard / R8 Rules [#proguard--r8-rules] **Symptom:** App crashes in release builds but works in debug. **Solution:** If you use code shrinking, ensure the Melange classes are preserved. Add to your `proguard-rules.pro`: ``` -keep class com.zeticai.mlange.** { *; } ``` *** Network and Download Issues [#network-and-download-issues] **Symptom:** Model initialization fails with network errors. **Solutions:** * Ensure the device has internet connectivity. The SDK downloads the model binary on first use. * Check that your app has the `INTERNET` permission in `AndroidManifest.xml`: ```xml ``` * If behind a corporate firewall, ensure access to Melange API endpoints is not blocked. *** Emulator Limitations [#emulator-limitations] **Symptom:** Inference runs but performance is unexpectedly slow, or NPU fallback messages appear in logs. **Cause:** Android emulators do not have NPU hardware. **Solution:** Always test on a physical device for accurate performance results. The emulator will use CPU fallback, which is significantly slower. *** Threading Issues [#threading-issues] **Symptom:** `NetworkOnMainThreadException` or UI freezes during model initialization. **Solution:** Initialize and run models on a background thread: ```kotlin lifecycleScope.launch(Dispatchers.IO) { val model = ZeticMLangeModel(this@MainActivity, PERSONAL_KEY, MODEL_NAME) val outputs = model.run(inputs) withContext(Dispatchers.Main) { // Update UI } } ``` *** Still Having Issues? [#still-having-issues] * Check [Common Errors](/troubleshooting/common-errors) for cross-platform issues * Visit the [FAQ](/troubleshooting/faq) * Join the [Discord community](https://discord.gg/q6vW4UscRY) * Email [contact@zetic.ai](mailto:contact@zetic.ai) # Common Errors (/troubleshooting/common-errors) This page covers the most frequently encountered errors when integrating ZETIC Melange, along with their causes and solutions. *** 1\. Model Key Not Found / Authentication Failure [#1-model-key-not-found--authentication-failure] **Symptoms:** * `RuntimeException: Model not found` (Android) * `Error: Failed to download model` (iOS) * HTTP 401 or 403 errors in logs **Cause:** The model key or personal key is invalid, expired, or does not match. **Solutions:** * Verify that your **personal key** is correct. Copy it directly from the [Melange Dashboard](https://melange.zetic.ai). * Verify that your **model key** matches an existing compiled model. The key format is typically `Username/ModelName` (e.g., `Steve/YOLOv11_comparison`). * Ensure the model compilation has completed successfully on the dashboard before attempting to use the key. * Check that your device has network connectivity. The SDK needs to download the model binary on first use. The Melange Dashboard provides **ready-to-use source code** with your keys already pre-filled. Use the copy button to avoid typos. *** 2\. Input Shape Mismatch [#2-input-shape-mismatch] **Symptoms:** * `RuntimeException: Input shape mismatch` (Android) * Runtime crash or unexpected output values * Model returns all zeros or NaN values **Cause:** The input tensor dimensions or data type do not match what the model expects. **Solutions:** * Check the model's expected input shape on the Melange Dashboard or in the model's documentation. * Ensure your input tensor has the correct number of dimensions. For example, YOLOv11 expects `[1, 3, 640, 640]` (batch, channels, height, width). * Verify the data type matches (typically `Float32`). * Make sure pixel values are normalized correctly (usually `0.0` to `1.0` for vision models). ```kotlin // Common mistake: forgetting to add the batch dimension // Wrong: shape [3, 640, 640] // Correct: shape [1, 3, 640, 640] ``` Different models expect different input formats. Some models use NCHW layout (batch, channels, height, width) while others use NHWC (batch, height, width, channels). Check your model's specification. *** 3\. JNI / Library Loading Failure (Android) [#3-jni--library-loading-failure-android] **Symptoms:** * `java.lang.UnsatisfiedLinkError: dlopen failed` * `java.lang.UnsatisfiedLinkError: couldn't find native library` * App crashes immediately on `ZeticMLangeModel` initialization **Cause:** The native C++ NPU driver libraries are not being extracted correctly from the APK. This happens when the `useLegacyPackaging` flag is missing from your Gradle configuration. **Solution:** Add the following to your app-level `build.gradle`: ```groovy android { ... packagingOptions { jniLibs { useLegacyPackaging true } } } ``` ```kotlin android { ... packaging { jniLibs { useLegacyPackaging = true } } } ``` After adding this, perform a **clean build** (Build then Clean Project, then Rebuild Project). This setting tells the Android build system to extract native libraries from the APK instead of loading them compressed. Melange's NPU drivers require this to function correctly. *** 4\. NPU Not Available / Falls Back to CPU [#4-npu-not-available--falls-back-to-cpu] **Symptoms:** * Inference works but is slower than expected * Log messages indicating CPU fallback: `"NPU not available, falling back to CPU"` * No performance improvement compared to standard frameworks **Cause:** The device's NPU is either not supported, not available, or the model was not compiled with NPU targets for the device's chipset. **Solutions:** * **Check device compatibility.** Not all Android devices have accessible NPUs. Melange supports Qualcomm Snapdragon (HTP/DSP), MediaTek (APU), Samsung Exynos (DSP), and Apple Neural Engine. Older or budget devices may not have NPU hardware. * **Verify the model was compiled with NPU targets.** On the Melange Dashboard, check that your model's compilation includes NPU-optimized binaries for your target device's chipset. * **Use a physical device.** Emulators and simulators do not have NPU hardware. Always test on a real device. * **Check for driver availability.** Some devices require specific system library versions for NPU access. Ensure your device's firmware is up to date. Even when the NPU is not available, Melange will still run inference using CPU fallback. Your app will work correctly, just without the NPU performance boost. *** 5\. Model Conversion Failure (Unsupported Operations) [#5-model-conversion-failure-unsupported-operations] **Symptoms:** * Model upload fails on the Melange Dashboard * Dashboard reports: `"Unsupported operation"` or `"Conversion failed"` * Compilation completes but produces incorrect results **Cause:** The model contains operations that are not yet supported by the Melange compiler or cannot be mapped to NPU instructions. **Solutions:** * **Check supported formats.** Melange supports: * PyTorch Exported Program (`.pt2`) * ONNX Model (`.onnx`) * **Simplify the ONNX model.** Use `onnx-simplifier` to reduce complex subgraphs: ```bash pip install onnxsim onnxsim input_model.onnx output_model.onnx ``` * **Check the ONNX opset version.** Export with a commonly supported opset (opset 12 is recommended): ```python model.export(format="onnx", opset=12, simplify=True) ``` * **Avoid dynamic shapes.** Export with static input dimensions: ```python model.export(format="onnx", dynamic=False, imgsz=640) ``` * **Contact support.** If you encounter unsupported operations, reach out to [contact@zetic.ai](mailto:contact@zetic.ai) with your model file. The team is continuously expanding operation support. For the most reliable conversion, export your model to ONNX format with `opset=12`, `simplify=True`, and `dynamic=False`. *** Still Having Issues? [#still-having-issues] If your issue is not listed above: * Check the platform-specific troubleshooting guides: [Android Issues](/troubleshooting/android-issues) and [iOS Issues](/troubleshooting/ios-issues) * See the full list of [Error Codes](/api-reference/error-codes) for detailed error descriptions * Visit the [FAQ](/troubleshooting/faq) for general questions * Join the [Discord community](https://discord.gg/q6vW4UscRY) for real-time help * Email [contact@zetic.ai](mailto:contact@zetic.ai) for technical support # FAQ (/troubleshooting/faq) General [#general] Melange is an on-device AI deployment platform that automates NPU acceleration for mobile applications. It takes your trained AI model, compiles it for mobile NPU hardware, and provides SDKs for Android and iOS to run inference directly on-device. Yes, Melange offers a free tier to get started. No credit card is required. Sign up at [melange.zetic.ai](https://melange.zetic.ai) using your Google or GitHub account. Paid plans (Pro, Pro+, Enterprise) are available for teams that need more devices, bandwidth, and support. See [Pricing and Plans](/introduction/pricing-and-plans). No. All inference runs entirely on-device. Your input data is never sent to any server. The only network call is the initial model binary download, which is cached locally. *** Models [#models] Melange supports: * **PyTorch Exported Program** (`.pt2`): Recommended (requires PyTorch >= 2.9) * **ONNX** (`.onnx`): Supported Yes. Melange supports direct integration with public Hugging Face models. See [Hugging Face Models](/model-preparation/hugging-face-models). Yes. You can upload any model in a supported format through the [Web Dashboard](/model-deployment/dashboard). Contact [contact@zetic.ai](mailto:contact@zetic.ai) with your model details. The team is continuously expanding operation support. You can also try simplifying the model with `onnxsim` or exporting with a different opset version. *** Platform Support [#platform-support] * **Android**: Full SDK support with NPU acceleration * **iOS**: Full SDK support with Neural Engine acceleration * **Flutter**: LLM template available; full SDK coming soon * **React Native**: LLM template available; full SDK coming soon Melange supports NPU acceleration on devices with Qualcomm Snapdragon (HTP/DSP), Google Pixel (Tensor), MediaTek (APU, Enterprise only), Samsung Exynos (DSP, Enterprise only), and Apple Neural Engine. See [Device Compatibility](/performance/device-compatibility) for details. Emulators and simulators do not have NPU hardware. The SDK will fall back to CPU execution on emulators, but performance will be significantly slower. Always test on physical devices. *** Performance [#performance] Melange benchmarks your model on 200+ physical devices to select the optimal compiled binary for each device. See [Performance-Adaptive Deployment](/model-deployment/performance-adaptive-deployment). Possible reasons: * Using an emulator instead of a physical device * Device NPU is not supported (CPU fallback is being used) * Model is still in "Optimizing" state (not fully tuned yet) * First inference is slower due to model loading Yes. Melange provides inference modes (`RUN_AUTO`, `RUN_SPEED`, `RUN_ACCURACY`) to balance speed and accuracy. See [Inference Mode Selection](/how-to-guides/inference-mode-selection). *** Account and Billing [#account-and-billing] Log in to the [Melange Dashboard](https://melange.zetic.ai), go to **Settings** then **Personal Key**, and generate a new key. See [Understanding Model Keys](/model-deployment/understanding-model-keys). You cannot retrieve a lost key. Generate a new one from the Dashboard. The old key will continue working until revoked. *** Getting Help [#getting-help] Join for real-time help from the community and team. Reach us for technical support. Check the guides for specific error solutions. # iOS Issues (/troubleshooting/ios-issues) This page covers iOS-specific issues you may encounter when integrating ZETIC Melange. Swift Package Manager Issues [#swift-package-manager-issues] Package Resolution Failure [#package-resolution-failure] **Symptom:** Xcode fails to resolve the Melange package with errors like "Failed to resolve dependencies" or "Unable to fetch repository." **Solutions:** * Verify the package URL is correct: `https://github.com/zetic-ai/ZeticMLangeiOS` * Check your internet connection and try again. * In Xcode, go to **File** then **Packages** then **Reset Package Caches**, then resolve again. * If using a VPN or corporate proxy, ensure GitHub is accessible. Version Conflicts [#version-conflicts] **Symptom:** Package version conflicts with other dependencies. **Solution:** Try specifying a specific version or branch when adding the package dependency instead of using "Up to Next Major Version." *** Linker Errors [#linker-errors] Undefined Accelerate Symbols [#undefined-accelerate-symbols] **Symptom:** Build fails with linker errors similar to: ``` Undefined symbol: _cblas_sgemm$NEWLAPACK$ILP64 Undefined symbol: _vDSP_maxv Undefined symbol: _vDSP_measqv Undefined symbol: _vDSP_sve Undefined symbol: _vDSP_vadd Undefined symbol: _vDSP_vdiv Undefined symbol: _vDSP_vmul Undefined symbol: _vDSP_vsadd Undefined symbol: _vDSP_vsmsa Undefined symbol: _vDSP_vsmul Undefined symbol: _vDSP_vsub ``` **Cause:** `ZeticMLange` uses Apple's **Accelerate** framework (`vDSP`, BLAS). Swift Package Manager does not link system frameworks automatically, so the app target needs to link it explicitly. **Solution:** Add `Accelerate.framework` to your app target's linked libraries: 1. Select your app target in Xcode. 2. Open **General** โ†’ **Frameworks, Libraries, and Embedded Content** (or **Build Phases** โ†’ **Link Binary With Libraries**). 3. Click **+**, search for `Accelerate.framework`, and add it. See [Setup โ†’ Link Accelerate.framework](/platform-integration/ios/setup) for step-by-step instructions. This manual step is a temporary workaround. A future SDK release will auto-link Accelerate through SPM so this troubleshooting entry no longer applies. *** Code Signing Issues [#code-signing-issues] **Symptom:** Build fails with signing errors related to the Melange framework. **Solutions:** * Ensure you have a valid development certificate and provisioning profile. * In your target's **Signing & Capabilities**, verify the team and bundle identifier are correct. * If using automatic signing, try toggling it off and on again. *** Simulator Limitations [#simulator-limitations] **Symptom:** Model runs on simulator but performance is unexpectedly slow, or you see degraded results. **Cause:** iOS Simulators do not have Neural Engine hardware. **Solution:** Always test on a physical device for accurate performance and results. The simulator will use CPU fallback. Performance measurements on the simulator are not representative of real device performance. Always benchmark on physical hardware. *** Model Initialization Errors [#model-initialization-errors] **Symptom:** `ZeticMLangeModel` initializer throws an error. **Solutions:** * Verify your **Personal Key** is correct. Copy it from the [Melange Dashboard](https://melange.zetic.ai). * Verify your **Model Key** matches a compiled model that has reached "Ready" status. * Ensure the device has network connectivity for the initial model download. * Check that you are using a physical device, not a simulator (for NPU-dependent functionality). *** App Transport Security [#app-transport-security] **Symptom:** Network requests fail with ATS-related errors. **Solution:** The Melange SDK communicates over HTTPS, which should work with default ATS settings. If you have customized ATS settings in your `Info.plist`, ensure HTTPS connections are allowed. *** Memory Warnings [#memory-warnings] **Symptom:** App receives memory warnings or crashes when loading large models. **Solutions:** * Ensure you are not holding references to multiple large model instances simultaneously. * Release model instances when they are no longer needed. * For LLM models, call `cleanUp()` to release the KV cache when done. *** Still Having Issues? [#still-having-issues] * Check [Common Errors](/troubleshooting/common-errors) for cross-platform issues * Visit the [FAQ](/troubleshooting/faq) * Join the [Discord community](https://discord.gg/q6vW4UscRY) * Email [contact@zetic.ai](mailto:contact@zetic.ai) # Model Conversion Issues (/troubleshooting/model-conversion-issues) This page covers issues you may encounter when uploading and converting models with ZETIC Melange. Unsupported Operations [#unsupported-operations] **Symptoms:** * Model upload fails on the Melange Dashboard * Dashboard reports "Unsupported operation" or "Conversion failed" **Solutions:** Simplify ONNX Models [#simplify-onnx-models] Use `onnx-simplifier` to reduce complex subgraphs: ```bash pip install onnxsim onnxsim input_model.onnx output_model.onnx ``` Check ONNX Opset Version [#check-onnx-opset-version] Export with a commonly supported opset (opset 12-13 recommended): ```python # PyTorch to ONNX torch.onnx.export(model, input, "model.onnx", opset_version=13) ``` ```bash # TFLite to ONNX python -m tf2onnx.convert --tflite model.tflite --output model.onnx --opset 13 ``` Avoid Dynamic Shapes [#avoid-dynamic-shapes] Export with static input dimensions: ```python # YOLO example model.export(format="onnx", dynamic=False, imgsz=640) ``` For the most reliable conversion, export your model to ONNX format with `opset=13`, simplification enabled, and static shapes. *** Input Shape Mismatches [#input-shape-mismatches] **Symptom:** Model compiles but produces incorrect results or crashes at inference time. **Cause:** The input shapes provided during upload do not match the shapes used during inference. **Solutions:** * Use [Netron](https://github.com/lutzroeder/netron) to inspect your model's expected input shapes. * Ensure the `.npy` input files match the model's expected dimensions exactly. * Verify input data types (typically `Float32`). *** Input Order Errors [#input-order-errors] **Symptom:** Model produces garbage results despite correct shapes. **Cause:** Inputs are provided in the wrong order. **Solution:** Verify input order using Netron: 1. Open your model in [Netron](https://github.com/lutzroeder/netron). 2. Check the top-most input node: this is Index 0. 3. The next input node is Index 1, and so on. 4. Ensure the inputs you upload on the Dashboard (and your `run()` inputs in the app) follow this order. Input order must be consistent between upload (on the Melange Dashboard) and inference (`run()` calls). Swapping inputs will produce incorrect results. *** Format-Specific Issues [#format-specific-issues] PyTorch Exported Program (.pt2) [#pytorch-exported-program-pt2] * Requires PyTorch 2.9+ * Some custom operators may not be supported by `torch.export` * Try tracing with simpler inputs if export fails ONNX (.onnx) [#onnx-onnx] * Use `onnxsim` to simplify before upload * Check for unsupported custom operators * Verify opset version compatibility (12-13 recommended) *** Compilation Status: Failed [#compilation-status-failed] **Symptom:** Model shows "Failed" status on the Dashboard. **Solutions:** 1. Check the error message on the Dashboard for specific details. 2. Verify your model file is not corrupted. 3. Try re-exporting the model with a simpler configuration. 4. Contact [contact@zetic.ai](mailto:contact@zetic.ai) with your model file for support. *** Still Having Issues? [#still-having-issues] * Check [Common Errors](/troubleshooting/common-errors) for general issues * Join the [Discord community](https://discord.gg/q6vW4UscRY) * Email [contact@zetic.ai](mailto:contact@zetic.ai) with your model file for direct support # Audio Classification (YAMNet) (/tutorials/audio-classification-yamnet) Build an on-device audio classification application using YAMNet with ZETIC Melange. This tutorial walks you through converting the TensorFlow model, deploying it to Melange, and running inference on Android and iOS. What You Will Build [#what-you-will-build] An on-device audio classification application that identifies audio events from 521 categories in real time, using YAMNet accelerated by NPU hardware. Prerequisites [#prerequisites] * A ZETIC Melange account with a Personal Key ([sign up at melange.zetic.ai](https://melange.zetic.ai)) * Python 3.8+ with `tensorflow`, `tensorflow_hub`, `tf2onnx`, and `numpy` installed * Android Studio or Xcode for mobile deployment What is YAMNet? [#what-is-yamnet] YAMNet is a deep neural network that predicts audio events from the AudioSet-YouTube corpus. * Trained on the AudioSet dataset with **521 audio event classes** * Model on TensorFlow Hub: [YAMNet](https://www.tensorflow.org/hub/tutorials/yamnet) Step 1: Convert YAMNet to ONNX [#step-1-convert-yamnet-to-onnx] We provide a pre-built model for you โ€” you can skip Steps 1โ€“3 and jump straight to [Step 4](#step-4-implement-zeticmlangemodel) using [`google/Sound Classification(YAMNET)`](https://melange.zetic.ai/p/google/Sound%20Classification%28YAMNET%29) from the Melange Dashboard. Load the YAMNet model from TensorFlow Hub and convert it to ONNX format: ```python import tensorflow as tf import tensorflow_hub as hub import tf2onnx import numpy as np model = hub.load('https://tfhub.dev/google/yamnet/1') concrete_func = model.signatures['serving_default'] input_shape = [1, 16000] sample_input = np.random.randn(*input_shape).astype(np.float32) input_tensor = tf.convert_to_tensor(waveform, dtype=tf.float32) tf.saved_model.save(model, "yamnet_saved_model", signatures=concrete_func) # python -m tf2onnx.convert --saved-model yamnet_saved_model --output yamnet.onnx --opset 13 ``` Step 2: Prepare Sample Input [#step-2-prepare-sample-input] Generate a sample audio waveform to use as input for model deployment: ```python import numpy as np sample_rate = 16000 duration = 1 waveform = np.sin(2 * np.pi * 440 * np.linspace(0, duration, sample_rate)) waveform = waveform.astype(np.float32) waveform = np.expand_dims(waveform, axis=0) np.save('waveform.npy', waveform) ``` Step 3: Generate Melange Model [#step-3-generate-melange-model] Upload the model and inputs via the [Melange Dashboard](/model-deployment/dashboard): * Model file: `yamnet.onnx` * Input: `waveform.npy` Step 4: Implement ZeticMLangeModel [#step-4-implement-zeticmlangemodel] For detailed application setup, please follow the [Android Integration Guide](/platform-integration/android/setup) guide. ```kotlin val yamnetModel = ZeticMLangeModel(this, PERSONAL_KEY, "google/Sound Classification(YAMNET)") val waveform: FloatArray = preprocess(audioData) val inputs = arrayOf( Tensor.of( data = waveform, dataType = DataType.Float32, shape = intArrayOf(1, 16000), ) ) val outputs = yamnetModel.run(inputs) ``` For detailed application setup, please follow the [iOS Integration Guide](/platform-integration/ios/setup) guide. ```swift let yamnetModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "google/Sound Classification(YAMNET)") let waveform: [Float] = preprocess(audioData) let inputs = [ Tensor( data: waveform.withUnsafeBufferPointer { Data(buffer: $0) }, dataType: BuiltinDataType.float32, shape: [1, 16000] ) ] let outputs = try yamnetModel.run(inputs: inputs) ``` For detailed application setup, please follow the [Flutter Integration Guide](/platform-integration/flutter/setup) guide. ```dart import 'package:zetic_mlange/zetic_mlange.dart'; final yamnetModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'google/Sound Classification(YAMNET)', ); final waveform = preprocess(audioData); final inputs = [ Tensor.float32View( waveform, shape: const [1, 16000], ), ]; final outputs = yamnetModel.run(inputs); ``` Step 5: Preprocess and Postprocess Audio [#step-5-preprocess-and-postprocess-audio] We provide an audio feature extractor as an Android and iOS module for handling audio preprocessing and result interpretation. ```kotlin // (1) Preprocess audio data and get processed float array val inputs = preprocess(audioData) // ... run model ... // (2) Postprocess model outputs val results = postprocess(outputs) ``` ```swift import ZeticMLange // (1) Preprocess audio data and get processed float array let inputs = preprocess(audioData) // ... run model ... // (2) Postprocess model outputs let results = postprocess(&outputs) ``` ```dart // (1) Preprocess audio data and get model tensors final inputs = preprocessAudio(audioData); // ... run model ... // (2) Postprocess model outputs final results = postprocessYamnet(outputs); ``` Complete Audio Classification Implementation [#complete-audio-classification-implementation] ```kotlin // (0) Initialize model val yamnetModel = ZeticMLangeModel(this, PERSONAL_KEY, "google/Sound Classification(YAMNET)") // (1) Preprocess audio val inputs = preprocess(audioData) // (2) Run model val outputs = yamnetModel.run(inputs) // (3) Postprocess results val predictions = postprocess(outputs) ``` ```swift // (0) Initialize model let yamnetModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "google/Sound Classification(YAMNET)") // (1) Preprocess audio let inputs = preprocess(audioData) // (2) Run model let outputs = try yamnetModel.run(inputs: inputs) // (3) Postprocess results let predictions = postprocess(outputs) ``` ```dart // (0) Initialize model final yamnetModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'google/Sound Classification(YAMNET)', ); // (1) Preprocess audio final inputs = preprocessAudio(audioData); // (2) Run model final outputs = yamnetModel.run(inputs); // (3) Postprocess results final predictions = postprocessYamnet(outputs); ``` *** Conclusion [#conclusion] With ZETIC Melange, implementing on-device audio classification with NPU acceleration is straightforward and efficient. YAMNet provides robust audio event detection capabilities across 521 categories. The simple pipeline of audio preprocessing and classification makes it easy to integrate into your applications. We are continuously adding new models to our examples and [HuggingFace](https://huggingface.co/zetic-ai) page. Stay tuned, and [contact us](mailto:contact@zetic.ai) for collaborations! # Face Detection (/tutorials/face-detection) Build an on-device face detection application using Google's MediaPipe Face Detection model with ZETIC Melange. This tutorial covers converting the model, deploying it, and running inference on Android and iOS. We provide [Face Detection demo application](https://github.com/zetic-ai/ZETIC_Melange_apps/tree/main/face_detection) source code for both Android and iOS. What You Will Build [#what-you-will-build] A real-time face detection application that identifies face locations in camera frames using the MediaPipe Face Detection model, accelerated on-device with NPU hardware. Prerequisites [#prerequisites] * A ZETIC Melange account with a Personal Key ([sign up at melange.zetic.ai](https://melange.zetic.ai)) * Python 3.8+ with `tf2onnx` installed * The [Face Detection TFLite model](https://github.com/patlevin/face-detection-tflite) (`face_detection_short_range.tflite`) * Android Studio or Xcode for mobile deployment What is Face Detection? [#what-is-face-detection] The Face Detection model in Google's MediaPipe is a high-performance machine learning model designed for real-time face detection in images and video streams. * Official documentation: [Face Detector - Google AI](https://ai.google.dev/edge/mediapipe/solutions/vision/face_detector) Step 1: Convert the Model to ONNX [#step-1-convert-the-model-to-onnx] We prepared a pre-built model for you โ€” you can skip Steps 1โ€“2 and jump straight to [Step 3](#step-3-implement-zeticmlangemodel) using [`google/MediaPipe-Face-Detection`](https://melange.zetic.ai/p/google/MediaPipe-Face-Detection) from the Melange Dashboard. Prepare the Face Detection model and convert it from TFLite to ONNX format: ```bash pip install tf2onnx python -m tf2onnx.convert --tflite face_detection_short_range.tflite --output face_detection_short_range.onnx --opset 13 ``` Step 2: Generate Melange Model [#step-2-generate-melange-model] Upload the model and inputs via the [Melange Dashboard](/model-deployment/dashboard): * Model file: `face_detection_short_range.onnx` * Input: `faces.npy` Step 3: Implement ZeticMLangeModel [#step-3-implement-zeticmlangemodel] For detailed application setup, please follow the [Android Integration Guide](/platform-integration/android/setup) guide. ```kotlin val model = ZeticMLangeModel(this, PERSONAL_KEY, "google/MediaPipe-Face-Detection") val pixels: FloatArray = preprocess(bitmap) val inputs = arrayOf( Tensor.of( data = pixels, dataType = DataType.Float32, shape = intArrayOf(1, 128, 128, 3), ) ) val outputs = model.run(inputs) ``` For detailed application setup, please follow the [iOS Integration Guide](/platform-integration/ios/setup) guide. ```swift let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "google/MediaPipe-Face-Detection") let pixels: [Float] = preprocess(uiImage) let inputs = [ Tensor( data: pixels.withUnsafeBufferPointer { Data(buffer: $0) }, dataType: BuiltinDataType.float32, shape: [1, 128, 128, 3] ) ] let outputs = try model.run(inputs: inputs) ``` For detailed application setup, please follow the [Flutter Integration Guide](/platform-integration/flutter/setup) guide. ```dart import 'dart:typed_data'; import 'package:zetic_mlange/zetic_mlange.dart'; final model = await ZeticMLangeModel.create( personalKey: personalKey, name: 'google/MediaPipe-Face-Detection', ); final pixels = preprocess(image); final inputs = [ Tensor.float32View( pixels, shape: const [1, 128, 128, 3], ), ]; final outputs = model.run(inputs); ``` Step 4: Use the Face Detection Wrapper [#step-4-use-the-face-detection-wrapper] We provide a Face Detection feature extractor as an Android and iOS module. The Face Detection feature extractor extension will be released as an open-source repository soon. ```kotlin // (0) Initialize Face Detection wrapper val feature = FaceDetectionWrapper() // (1) Preprocess bitmap and get processed float array val inputs = feature.preprocess(bitmap) // ... run model ... // (2) Postprocess to bitmap val resultBitmap = feature.postprocess(outputs) ``` ```swift import ZeticMLange import ext // (0) Initialize Face Detection wrapper let feature = FaceDetectionWrapper() // (1) Preprocess UIImage and get processed float array let inputs = feature.preprocess(image) // ... run model ... // (2) Postprocess to UIImage let resultBitmap = feature.postprocess(&outputs) ``` ```dart // Use app-defined Dart preprocessing and postprocessing helpers. // (1) Preprocess image and get model tensors final inputs = preprocessFaceDetectionImage(image); // ... run model ... // (2) Postprocess model outputs for your UI final detections = postprocessFaceDetection(outputs); ``` Complete Face Detection Implementation [#complete-face-detection-implementation] ```kotlin // (0) Initialize model and feature val model = ZeticMLangeModel(this, PERSONAL_KEY, "google/MediaPipe-Face-Detection") val faceDetection = FaceDetectionWrapper() // (1) Preprocess image val faceDetectionInputs = faceDetection.preprocess(imagePtr) // (2) Process model val faceDetectionOutputs = model.run(faceDetectionInputs) // (3) Postprocess model run result val faceDetectionPostprocessed = faceDetection.postprocess(faceDetectionOutputs) ``` ```swift // (0) Initialize model and feature let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "google/MediaPipe-Face-Detection") let faceDetection = FaceDetectionWrapper() // (1) Preprocess image let faceDetectionInputs = faceDetection.preprocess(uiImage) // (2) Process model let faceDetectionOutputs = try model.run(inputs: faceDetectionInputs) // (3) Postprocess model run result let faceDetectionPostprocessed = faceDetection.postprocess(&faceDetectionOutputs) ``` ```dart // (0) Initialize model final model = await ZeticMLangeModel.create( personalKey: personalKey, name: 'google/MediaPipe-Face-Detection', ); // (1) Preprocess image final faceDetectionInputs = preprocessFaceDetectionImage(image); // (2) Process model final faceDetectionOutputs = model.run(faceDetectionInputs); // (3) Postprocess model run result final faceDetectionPostprocessed = postprocessFaceDetection(faceDetectionOutputs); ``` *** Conclusion [#conclusion] With ZETIC Melange, building on-device face detection applications with NPU acceleration is straightforward. We have developed a custom OpenCV module and an ML application pipeline, making the implementation remarkably simple and efficient. We are continually uploading new models to our examples and [HuggingFace](https://huggingface.co/zetic-ai) page. Stay tuned, and [contact us](mailto:contact@zetic.ai) for collaborations! # Face Emotion Recognition (/tutorials/face-emotion-recognition) Build an on-device face emotion recognition application using a two-model pipeline with ZETIC Melange. This tutorial chains Face Detection with the EMO-AffectNet (ResNet-50) model to classify facial emotions in real time on Android and iOS. We provide the source code for the [Face Emotion Recognition demo application](https://github.com/zetic-ai/ZETIC_Melange_apps/tree/main/face_emotion_recognition) for both Android and iOS. What You Will Build [#what-you-will-build] A real-time emotion classification application that first detects faces, then classifies each detected face into one of seven emotion categories using the EMO-AffectNet model, all running on-device with NPU acceleration. Prerequisites [#prerequisites] * A ZETIC Melange account with a Personal Key ([sign up at melange.zetic.ai](https://melange.zetic.ai)) * Python 3.8+ with `torch`, `tf2onnx`, and `numpy` installed * The [Face Detection TFLite model](https://github.com/patlevin/face-detection-tflite/tree/main/fdlite/data) * The [EMO-AffectNet model weights](https://huggingface.co/ElenaRyumina/face_emotion_recognition) * Android Studio or Xcode for mobile deployment What is EMO-AffectNet? [#what-is-emo-affectnet] EMO-AffectNet is a ResNet-50 based deep convolutional neural network trained for facial emotion recognition. It classifies faces into 7 emotion categories: Angry, Disgust, Fear, Happy, Neutral, Sad, and Surprise. * Model on Hugging Face: [face\_emotion\_recognition](https://huggingface.co/ElenaRyumina/face_emotion_recognition) Model Pipelining [#model-pipelining] For accurate emotion recognition, we need to first detect the face region and then pass the cropped face image to the emotion model. The pipeline consists of: 1. **Face Detection**: Use the Face Detection model to accurately detect face regions in the image. Extract the face area from the original image. 2. **Face Emotion Recognition**: Input the extracted face image into the EMO-AffectNet model to classify the emotion. Step 1: Prepare the Models [#step-1-prepare-the-models] We prepared pre-built models for you โ€” you can skip Steps 1โ€“2 and jump straight to [Step 3](#step-3-implement-zeticmlangemodel) using these models from the Melange Dashboard: * [`google/MediaPipe-Face-Detection`](https://melange.zetic.ai/p/google/MediaPipe-Face-Detection) * [`ElenaRyumina/FaceEmotionRecognition`](https://melange.zetic.ai/p/ElenaRyumina/FaceEmotionRecognition) Face Detection Model [#face-detection-model] Convert the Face Detection TFLite model to ONNX format: ```bash pip install tf2onnx python -m tf2onnx.convert --tflite face_detection_short_range.tflite --output face_detection_short_range.onnx --opset 13 ``` Face Emotion Recognition Model [#face-emotion-recognition-model] Export the EMO-AffectNet model using PyTorch Exported Program. You can find the ResNet50 class [here](https://huggingface.co/ElenaRyumina/face_emotion_recognition/blob/main/run_webcam.ipynb). ```python import torch import torch.nn as nn import numpy as np emo_affectnet = ResNet50(7, channels=3) emo_affectnet.load_state_dict(torch.load('FER_static_ResNet50_AffectNet.pt')) emo_affectnet.eval() model_cpu = emo_affectnet.cpu() exported_model = torch.export.export(model_cpu, (cur_face,)) np_cur_face = cur_face.detach().numpy() np.save("data/cur_face.npy", np_cur_face) output_model_path = "models/FER_static_ResNet50_AffectNet.pt2" torch.export.save(exported_model, output_model_path) ``` Step 2: Generate Melange Model Keys [#step-2-generate-melange-model-keys] Upload both models and their inputs via the [Melange Dashboard](/model-deployment/dashboard): * Face detection model `face_detection_short_range.onnx` with input `input.npy` * Emotion recognition model `FER_static_ResNet50_AffectNet.pt2` with input `input.npy` Step 3: Implement ZeticMLangeModel [#step-3-implement-zeticmlangemodel] For detailed application setup, please follow the [Android Integration Guide](/platform-integration/android/setup) guide. ```kotlin val faceEmotionRecognitionModel = ZeticMLangeModel(this, PERSONAL_KEY, "ElenaRyumina/FaceEmotionRecognition") val pixels: FloatArray = preprocess(croppedFaceBitmap) val inputs = arrayOf( Tensor.of( data = pixels, dataType = DataType.Float32, shape = intArrayOf(1, 3, 224, 224), ) ) val outputs = faceEmotionRecognitionModel.run(inputs) ``` For detailed application setup, please follow the [iOS Integration Guide](/platform-integration/ios/setup) guide. ```swift let faceEmotionRecognitionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "ElenaRyumina/FaceEmotionRecognition") let pixels: [Float] = preprocess(croppedFaceImage) let inputs = [ Tensor( data: pixels.withUnsafeBufferPointer { Data(buffer: $0) }, dataType: BuiltinDataType.float32, shape: [1, 3, 224, 224] ) ] let outputs = try faceEmotionRecognitionModel.run(inputs: inputs) ``` For detailed application setup, please follow the [Flutter Integration Guide](/platform-integration/flutter/setup) guide. ```dart import 'package:zetic_mlange/zetic_mlange.dart'; final faceEmotionRecognitionModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'ElenaRyumina/FaceEmotionRecognition', ); final pixels = preprocess(croppedFaceImage); final inputs = [ Tensor.float32View( pixels, shape: const [1, 3, 224, 224], ), ]; final outputs = faceEmotionRecognitionModel.run(inputs); ``` Step 4: Use the Feature Extractors [#step-4-use-the-feature-extractors] We provide Face Detection and Face Emotion Recognition feature extractors as Android and iOS modules. The Face Emotion Recognition feature extractor extension will be released as an open-source repository soon. ```kotlin // (0) Initialize Face Emotion Recognition wrapper val feature = FaceEmotionRecognitionWrapper() // (1) Preprocess bitmap and get processed float array val inputs = feature.preprocess(bitmap) // ... run model ... // (2) Postprocess to bitmap val resultBitmap = feature.postprocess(outputs) ``` ```swift import ZeticMLange // (0) Initialize Face Emotion Recognition wrapper let feature = FaceEmotionRecognitionWrapper() // (1) Preprocess UIImage and get processed float array let inputs = feature.preprocess(image) // ... run model ... // (2) Postprocess to UIImage let resultBitmap = feature.postprocess(&outputs) ``` ```dart // Use app-defined Dart preprocessing and postprocessing helpers. // (1) Preprocess image and get model tensors final inputs = preprocessFaceEmotionImage(image); // ... run model ... // (2) Postprocess model outputs for your UI final emotions = postprocessFaceEmotion(outputs); ``` Complete Face Emotion Recognition Pipeline Implementation [#complete-face-emotion-recognition-pipeline-implementation] The complete implementation requires pipelining two models: Face Detection followed by Face Emotion Recognition. **Step 1: Face Detection** ```kotlin // (0) Initialize face detection model val faceDetectionModel = ZeticMLangeModel(this, PERSONAL_KEY, "google/MediaPipe-Face-Detection") val faceDetection = FaceDetectionWrapper() // (1) Preprocess image val faceDetectionInputs = faceDetection.preprocess(bitmap) // (2) Run face detection model val faceDetectionOutputs = faceDetectionModel.run(faceDetectionInputs) // (3) Postprocess to get face regions val faceDetectionPostprocessed = faceDetection.postprocess(faceDetectionOutputs) ``` **Step 2: Face Emotion Recognition** ```kotlin // (0) Initialize face emotion recognition model val faceEmotionRecognitionModel = ZeticMLangeModel(this, PERSONAL_KEY, "ElenaRyumina/FaceEmotionRecognition") val faceEmotionRecognition = FaceEmotionRecognitionWrapper() // (1) Preprocess with detected face regions val faceEmotionRecognitionInputs = faceEmotionRecognition.preprocess(bitmap, faceDetectionPostprocessed) // (2) Run face emotion recognition model val faceEmotionRecognitionOutputs = faceEmotionRecognitionModel.run(faceEmotionRecognitionInputs) // (3) Postprocess to get emotions val faceEmotionRecognitionPostprocessed = faceEmotionRecognition.postprocess(faceEmotionRecognitionOutputs) ``` **Step 1: Face Detection** ```swift // (0) Initialize face detection model let faceDetectionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "google/MediaPipe-Face-Detection") let faceDetection = FaceDetectionWrapper() // (1) Preprocess image let faceDetectionInputs = faceDetection.preprocess(bitmap) // (2) Run face detection model let faceDetectionOutputs = try faceDetectionModel.run(inputs: faceDetectionInputs) // (3) Postprocess to get face regions let faceDetectionPostprocessed = faceDetection.postprocess(faceDetectionOutputs) ``` **Step 2: Face Emotion Recognition** ```swift // (0) Initialize face emotion recognition model let faceEmotionRecognitionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "ElenaRyumina/FaceEmotionRecognition") let faceEmotionRecognition = FaceEmotionRecognitionWrapper() // (1) Preprocess with detected face regions let faceEmotionRecognitionInputs = faceEmotionRecognition.preprocess(bitmap, faceDetectionPostprocessed) // (2) Run face emotion recognition model let faceEmotionRecognitionOutputs = try faceEmotionRecognitionModel.run(inputs: faceEmotionRecognitionInputs) // (3) Postprocess to get emotions let faceEmotionRecognitionPostprocessed = faceEmotionRecognition.postprocess(faceEmotionRecognitionOutputs) ``` **Step 1: Face Detection** ```dart // (0) Initialize face detection model final faceDetectionModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'google/MediaPipe-Face-Detection', ); // (1) Preprocess image final faceDetectionInputs = preprocessFaceDetectionImage(image); // (2) Run face detection model final faceDetectionOutputs = faceDetectionModel.run(faceDetectionInputs); // (3) Postprocess to get face regions final faceDetectionPostprocessed = postprocessFaceDetection(faceDetectionOutputs); ``` **Step 2: Face Emotion Recognition** ```dart // (0) Initialize face emotion recognition model final faceEmotionRecognitionModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'ElenaRyumina/FaceEmotionRecognition', ); // (1) Preprocess with detected face regions final faceEmotionRecognitionInputs = preprocessFaceEmotionImage(image, faceDetectionPostprocessed); // (2) Run face emotion recognition model final faceEmotionRecognitionOutputs = faceEmotionRecognitionModel.run(faceEmotionRecognitionInputs); // (3) Postprocess to get emotions final faceEmotionRecognitionPostprocessed = postprocessFaceEmotion(faceEmotionRecognitionOutputs); ``` *** Conclusion [#conclusion] With ZETIC Melange, building multi-model pipelines for on-device AI is simple and efficient. The Face Detection to Face Emotion Recognition pipeline demonstrates how you can construct straightforward model chains for real-time facial analysis with NPU acceleration. We are continually uploading new models to our examples and [HuggingFace](https://huggingface.co/zetic-ai) page. Stay tuned, and [contact us](mailto:contact@zetic.ai) for collaborations! # Face Landmark Detection (/tutorials/face-landmark) Build an on-device face landmark detection application using a two-model pipeline with ZETIC Melange. This tutorial demonstrates how to chain Face Detection and Face Landmark models together for accurate facial landmark extraction on Android and iOS. We provide [Face Landmark demo application](https://github.com/zetic-ai/ZETIC_Melange_apps/tree/main/face_landmark) source code for both Android and iOS. What You Will Build [#what-you-will-build] A real-time face landmark detection application that first detects faces, then extracts detailed facial landmarks from each detected face region. This two-step pipeline ensures accurate landmark placement by feeding properly cropped face images to the landmark model. Prerequisites [#prerequisites] * A ZETIC Melange account with a Personal Key ([sign up at melange.zetic.ai](https://melange.zetic.ai)) * Python 3.8+ with `tf2onnx` installed * The [Face Detection and Face Landmark TFLite models](https://github.com/patlevin/face-detection-tflite) * Android Studio or Xcode for mobile deployment What is Face Landmark? [#what-is-face-landmark] The Face Landmark model in Google's MediaPipe is a highly efficient machine learning model used for real-time face detection and landmark extraction. * Official documentation: [Face Landmarker - Google AI](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker) Model Pipelining [#model-pipelining] For accurate use of the face landmark model, it is necessary to pass an image of the correct facial area to the model. To accomplish this, we construct a pipeline with the Face Detection model: 1. **Face Detection**: Use the Face Detection model to accurately detect face regions in the image. Extract that part of the original image using the detected face region information. 2. **Face Landmark**: Input the extracted face image into the Face Landmark model to analyze facial landmarks. Step 1: Convert the Models to ONNX [#step-1-convert-the-models-to-onnx] We prepared pre-built models for you โ€” you can skip Steps 1โ€“2 and jump straight to [Step 3](#step-3-implement-zeticmlangemodel) using these models from the Melange Dashboard: * [`google/MediaPipe-Face-Detection`](https://melange.zetic.ai/p/google/MediaPipe-Face-Detection) * [`google/MediaPipe-Face-Landmark`](https://melange.zetic.ai/p/google/MediaPipe-Face-Landmark) Prepare both models from GitHub and convert them to ONNX format. **Face Detection model:** ```bash pip install tf2onnx python -m tf2onnx.convert --tflite face_detection_short_range.tflite --output face_detection_short_range.onnx --opset 13 ``` **Face Landmark model:** ```bash python -m tf2onnx.convert --tflite face_landmark.tflite --output face_landmark.onnx --opset 13 ``` Step 2: Generate Melange Models [#step-2-generate-melange-models] Upload both models and their inputs via the [Melange Dashboard](/model-deployment/dashboard): * `face_detection_short_range.onnx` with input `input.npy` * `face_landmark.onnx` with input `input.npy` Step 3: Implement ZeticMLangeModel [#step-3-implement-zeticmlangemodel] The Face Detection model feeds cropped face regions into this model; see [Face Detection](/tutorials/face-detection) for its own code. For detailed application setup, please follow the [Android Integration Guide](/platform-integration/android/setup) guide. ```kotlin val faceLandmarkModel = ZeticMLangeModel(this, PERSONAL_KEY, "google/MediaPipe-Face-Landmark") val pixels: FloatArray = preprocess(croppedFaceBitmap) val inputs = arrayOf( Tensor.of( data = pixels, dataType = DataType.Float32, shape = intArrayOf(1, 192, 192, 3), ) ) val outputs = faceLandmarkModel.run(inputs) ``` For detailed application setup, please follow the [iOS Integration Guide](/platform-integration/ios/setup) guide. ```swift let faceLandmarkModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "google/MediaPipe-Face-Landmark") let pixels: [Float] = preprocess(croppedFaceImage) let inputs = [ Tensor( data: pixels.withUnsafeBufferPointer { Data(buffer: $0) }, dataType: BuiltinDataType.float32, shape: [1, 192, 192, 3] ) ] let outputs = try faceLandmarkModel.run(inputs: inputs) ``` For detailed application setup, please follow the [Flutter Integration Guide](/platform-integration/flutter/setup) guide. ```dart import 'package:zetic_mlange/zetic_mlange.dart'; final faceLandmarkModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'google/MediaPipe-Face-Landmark', ); final pixels = preprocess(croppedFaceImage); final inputs = [ Tensor.float32View( pixels, shape: const [1, 192, 192, 3], ), ]; final outputs = faceLandmarkModel.run(inputs); ``` Step 4: Use the Face Landmark Wrapper [#step-4-use-the-face-landmark-wrapper] We provide a Face Landmark feature extractor as an Android and iOS module. The Face Landmark feature extractor extension will be released as an open-source repository soon. ```kotlin // (0) Initialize Face Landmark wrapper val feature = FaceLandmarkWrapper() // (1) Preprocess bitmap and get processed float array val inputs = feature.preprocess(bitmap) // ... run model ... // (2) Postprocess to bitmap val resultBitmap = feature.postprocess(outputs) ``` ```swift import ZeticMLange // (0) Initialize Face Landmark wrapper let feature = FaceLandmarkWrapper() // (1) Preprocess UIImage and get processed float array let inputs = feature.preprocess(image) // ... run model ... // (2) Postprocess to UIImage let resultBitmap = feature.postprocess(&outputs) ``` ```dart // Use app-defined Dart preprocessing and postprocessing helpers. // (1) Preprocess image and get model tensors final inputs = preprocessFaceLandmarkImage(image); // ... run model ... // (2) Postprocess model outputs for your UI final landmarks = postprocessFaceLandmark(outputs); ``` Complete Face Landmark Pipeline Implementation [#complete-face-landmark-pipeline-implementation] The complete implementation requires pipelining two models: Face Detection followed by Face Landmark. **Step 1: Face Detection** ```kotlin // (0) Initialize face detection model val faceDetectionModel = ZeticMLangeModel(this, PERSONAL_KEY, "google/MediaPipe-Face-Detection") val faceDetection = FaceDetectionWrapper() // (1) Preprocess image val faceDetectionInputs = faceDetection.preprocess(bitmap) // (2) Run face detection model val faceDetectionOutputs = faceDetectionModel.run(faceDetectionInputs) // (3) Postprocess to get face regions val faceDetectionPostprocessed = faceDetection.postprocess(faceDetectionOutputs) ``` **Step 2: Face Landmark** ```kotlin // (0) Initialize face landmark model val faceLandmarkModel = ZeticMLangeModel(this, PERSONAL_KEY, "google/MediaPipe-Face-Landmark") val faceLandmark = FaceLandmarkWrapper() // (1) Preprocess with detected face regions val faceLandmarkInputs = faceLandmark.preprocess(bitmap, faceDetectionPostprocessed) // (2) Run face landmark model val faceLandmarkOutputs = faceLandmarkModel.run(faceLandmarkInputs) // (3) Postprocess to get landmarks val faceLandmarkPostprocessed = faceLandmark.postprocess(faceLandmarkOutputs) ``` **Step 1: Face Detection** ```swift // (0) Initialize face detection model let faceDetectionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "google/MediaPipe-Face-Detection") let faceDetection = FaceDetectionWrapper() // (1) Preprocess image let faceDetectionInputs = faceDetection.preprocess(bitmap) // (2) Run face detection model let faceDetectionOutputs = try faceDetectionModel.run(inputs: faceDetectionInputs) // (3) Postprocess to get face regions let faceDetectionPostprocessed = faceDetection.postprocess(faceDetectionOutputs) ``` **Step 2: Face Landmark** ```swift // (0) Initialize face landmark model let faceLandmarkModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "google/MediaPipe-Face-Landmark") let faceLandmark = FaceLandmarkWrapper() // (1) Preprocess with detected face regions let faceLandmarkInputs = faceLandmark.preprocess(bitmap, faceDetectionPostprocessed) // (2) Run face landmark model let faceLandmarkOutputs = try faceLandmarkModel.run(inputs: faceLandmarkInputs) // (3) Postprocess to get landmarks let faceLandmarkPostprocessed = faceLandmark.postprocess(faceLandmarkOutputs) ``` **Step 1: Face Detection** ```dart // (0) Initialize face detection model final faceDetectionModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'google/MediaPipe-Face-Detection', ); // (1) Preprocess image final faceDetectionInputs = preprocessFaceDetectionImage(image); // (2) Run face detection model final faceDetectionOutputs = faceDetectionModel.run(faceDetectionInputs); // (3) Postprocess to get face regions final faceDetectionPostprocessed = postprocessFaceDetection(faceDetectionOutputs); ``` **Step 2: Face Landmark** ```dart // (0) Initialize face landmark model final faceLandmarkModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'google/MediaPipe-Face-Landmark', ); // (1) Preprocess with detected face regions final faceLandmarkInputs = preprocessFaceLandmarkImage(image, faceDetectionPostprocessed); // (2) Run face landmark model final faceLandmarkOutputs = faceLandmarkModel.run(faceLandmarkInputs); // (3) Postprocess to get landmarks final faceLandmarkPostprocessed = postprocessFaceLandmark(faceLandmarkOutputs); ``` *** Conclusion [#conclusion] With ZETIC Melange, building multi-model pipelines for on-device AI is straightforward. The Face Detection to Face Landmark pipeline demonstrates how you can chain models together for accurate, real-time facial analysis with NPU acceleration. We are continually adding new models to our examples and [HuggingFace](https://huggingface.co/zetic-ai) page. Stay tuned and [contact us](mailto:contact@zetic.ai) to collaborate on exciting projects! # Object Detection (YOLOv8 / YOLOv11) (/tutorials/object-detection-yolo) Build a real-time on-device object detection application using YOLOv8 or YOLOv11 with ZETIC Melange. This tutorial walks you through exporting the model, deploying it to Melange, and running inference on Android and iOS. We provide the source code for the [YOLOv11 demo application](https://github.com/zetic-ai/ZETIC_Melange_apps/tree/main/yolov8) for both Android and iOS. If the input model key is changed to YOLOv8, you can experience YOLOv8 as well. What You Will Build [#what-you-will-build] An on-device object detection application that identifies and localizes objects in real-time camera frames using YOLOv8 or YOLOv11 models accelerated by NPU hardware. Prerequisites [#prerequisites] * A ZETIC Melange account with a Personal Key ([sign up at melange.zetic.ai](https://melange.zetic.ai)) * Python 3.8+ with `ultralytics`, `opencv-python`, and `numpy` installed * Android Studio or Xcode for mobile deployment What is YOLOv11? [#what-is-yolov11] YOLOv11 is the latest version of the acclaimed real-time object detection and image segmentation model by Ultralytics. * Official documentation: [YOLOv11 Docs](https://docs.ultralytics.com) * Currently, only detector mode is supported. Additional features will be supported later. Step 1: Export the Model [#step-1-export-the-model] We prepared pre-built models for you โ€” you can skip the export step and jump straight to [Step 3](#step-3-implement-zeticmlangemodel): * **YOLOv11**: `Steve/YOLOv11_comparison` * **YOLOv8**: [`Ultralytics/YOLOv8n`](https://melange.zetic.ai/p/Ultralytics/YOLOv8n) โ€” browse on the Melange Dashboard Export the YOLOv11 model to ONNX format. You will get `yolo11n.onnx` after running this script: ```python from ultralytics import YOLO import torch model = YOLO("yolo11n.pt") model.export(format="onnx", opset=12, simplify=True, dynamic=False, imgsz=640) ``` Step 2: Prepare Input Sample [#step-2-prepare-input-sample] Prepare your input from an image file: ```python import cv2 import numpy as np def preprocess_image(image_path, target_size=(640, 640)): img = cv2.imread(image_path) img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) img = cv2.resize(img, target_size) img = img.astype(np.float32) / 255.0 img = np.transpose(img, (2, 0, 1)) img = np.expand_dims(img, axis=0) return img ``` Step 3: Generate Melange Model [#step-3-generate-melange-model] Upload the model and inputs via the [Melange Dashboard](/model-deployment/dashboard): * Model file: `yolo11n.onnx` * Input: `images.npy` Step 4: Implement ZeticMLangeModel [#step-4-implement-zeticmlangemodel] Initialize the Melange model in your mobile application and run inference. For detailed application setup, please follow the [Android Integration Guide](/platform-integration/android/setup) guide. ```kotlin val model = ZeticMLangeModel(this, PERSONAL_KEY, MODEL_NAME) val pixels: FloatArray = preprocess(bitmap) val inputs = arrayOf( Tensor.of( data = pixels, dataType = DataType.Float32, shape = intArrayOf(1, 3, 640, 640), ) ) val outputs = model.run(inputs) ``` For detailed application setup, please follow the [iOS Integration Guide](/platform-integration/ios/setup) guide. ```swift let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: MODEL_NAME, version: VERSION) let pixels: [Float] = preprocess(uiImage) let inputs = [ Tensor( data: pixels.withUnsafeBufferPointer { Data(buffer: $0) }, dataType: BuiltinDataType.float32, shape: [1, 3, 640, 640] ) ] let outputs = try model.run(inputs: inputs) ``` For detailed application setup, please follow the [Flutter Integration Guide](/platform-integration/flutter/setup) guide. ```dart import 'package:zetic_mlange/zetic_mlange.dart'; final model = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, version: version, ); final pixels = preprocess(image); final inputs = [ Tensor.float32View( pixels, shape: const [1, 3, 640, 640], ), ]; final outputs = model.run(inputs); ``` Step 5: Use the YOLOv8 Pipeline [#step-5-use-the-yolov8-pipeline] We provide a YOLOv8 feature extractor as an Android and iOS module. This feature extractor works with both YOLOv8 and YOLOv11 models. We are using the Melange extension module here. ```kotlin val model = ZeticMLangeModelWrapper(this, PERSONAL_KEY, MODEL_NAME) val pipeline = ZeticMLangePipeline( feature = YOLOv8(this, model = model), inputSource = CameraSource(this, preview.holder, preferredSize), ) pipeline.loop { result -> // visualize YOLO result here } ``` ```swift import ZeticMLange import ext let model = try ZeticMLangeModelWrapper(PERSONAL_KEY, MODEL_NAME) let pipeline = ZeticMLangePipeline(feature: model, inputSource: CameraSource()) pipeline.startLoop() while true { let frame = pipeline.latestResult // visualize YOLO result here } pipeline.stopLoop() ``` ```dart final model = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, ); for await (final frame in cameraFrames) { final inputs = preprocessYoloFrame(frame); final outputs = model.run(inputs); final detections = postprocessYolo(outputs); // visualize YOLO result here renderDetections(detections); } ``` *** Conclusion [#conclusion] With ZETIC Melange, you can build on-device AI object detection applications with NPU acceleration in minutes. We continuously upload models to our examples and [HuggingFace](https://huggingface.co/zetic-ai) page. Please stay tuned and [contact us](mailto:contact@zetic.ai) for collaborations! # Speech Recognition (Whisper) (/tutorials/speech-recognition-whisper) Build an on-device speech recognition application using OpenAI's Whisper model with ZETIC Melange. This tutorial covers splitting Whisper into encoder and decoder components, deploying them to Melange, and running the full speech-to-text pipeline on Android and iOS. What You Will Build [#what-you-will-build] An on-device speech-to-text application that processes audio through Whisper's three-component architecture (Feature Extractor, Encoder, Decoder) with NPU acceleration for real-time transcription. Prerequisites [#prerequisites] * A ZETIC Melange account with a Personal Key ([sign up at melange.zetic.ai](https://melange.zetic.ai)) * Python 3.10+ with the required Python packages installed: ```bash pip install torch transformers numpy ``` * Android Studio or Xcode for mobile deployment What is Whisper? [#what-is-whisper] Whisper is a state-of-the-art speech recognition model developed by OpenAI that offers: * **Multilingual support**: Recognizes speech in multiple languages * **Multiple capabilities**: Performs speech recognition, language detection, and translation * **Open source**: Available through [Hugging Face](https://huggingface.co/openai/whisper-tiny) Architecture Overview [#architecture-overview] The Whisper implementation consists of three main components: 1. **Feature Extractor**: Processes raw audio into Mel Spectrogram features 2. **Encoder**: Processes Mel Spectrogram to generate audio embeddings 3. **Decoder**: Generates text tokens from the audio embeddings Step 1: Prepare Sample Inputs for Exporting [#step-1-prepare-sample-inputs-for-exporting] We provide pre-built models for you โ€” you can skip Steps 1โ€“5 and jump straight to [Step 6](#step-6-implement-zeticmlangemodel) using these models from the Melange Dashboard: * [`OpenAI/whisper-tiny-encoder`](https://melange.zetic.ai/p/OpenAI/whisper-tiny-encoder) * [`OpenAI/whisper-tiny-decoder`](https://melange.zetic.ai/p/OpenAI/whisper-tiny-decoder) To convert the model for deployment, we need to export the PyTorch model with sample inputs that match the expected tensor shapes. ```python import numpy as np import torch from transformers import WhisperForConditionalGeneration model_name = "openai/whisper-tiny" model = WhisperForConditionalGeneration.from_pretrained(model_name) # Whisper expects 30 s of 80-bin log-mel features: (batch, 80, 3000). # Exporting only needs the right shape/dtype โ€” the values don't matter. input_features = torch.randn(1, 80, 3000) MAX_TOKEN_LENGTH = model.config.max_target_positions dummy_decoder_input_ids = torch.tensor([[0 for _ in range(MAX_TOKEN_LENGTH)]]) dummy_encoder_hidden_states = torch.randn(1, 1500, model.config.d_model).float() dummy_decoder_attention_mask = torch.ones_like(dummy_decoder_input_ids) ``` Step 2: Export Encoder to Exported Program [#step-2-export-encoder-to-exported-program] Wrap and export the Whisper encoder: ```python from transformers import WhisperModel import torch.nn as nn class WhisperEncoderWrapper(nn.Module): def __init__(self, whisper_model): super().__init__() self.enc = whisper_model.model.encoder def forward(self, input_features): return self.enc(input_features=input_features, return_dict=False)[0] with torch.no_grad(): encoder = WhisperEncoderWrapper(model).eval() exported_encoder = torch.export.export(encoder, (input_features,)) torch.export.save(exported_encoder, "whisper_encoder.pt2") ``` Step 3: Export Decoder to Exported Program [#step-3-export-decoder-to-exported-program] Wrap and export the Whisper decoder: ```python class WhisperDecoderWrapper(nn.Module): def __init__(self, whisper_model): super().__init__() self.decoder = whisper_model.model.decoder self.proj_out = whisper_model.proj_out def forward(self, input_ids, encoder_hidden_states, decoder_attention_mask): hidden = self.decoder( input_ids=input_ids, encoder_hidden_states=encoder_hidden_states, attention_mask=decoder_attention_mask, use_cache=False, return_dict=False, )[0] return self.proj_out(hidden) with torch.no_grad(): decoder = WhisperDecoderWrapper(model).eval() exported_decoder = torch.export.export( decoder, (dummy_decoder_input_ids, dummy_encoder_hidden_states, dummy_decoder_attention_mask), ) torch.export.save(exported_decoder, "whisper_decoder.pt2") ``` Step 4: Save Input Samples [#step-4-save-input-samples] Save all input tensors as `.npy` files for model upload: ```python import numpy as np # Save encoder inputs np.save("whisper_input_features.npy", input_features.cpu().numpy()) # Save decoder inputs np.save( "whisper_decoder_input_ids.npy", dummy_decoder_input_ids.cpu().numpy().astype(np.int64), ) np.save( "whisper_encoder_hidden_states.npy", dummy_encoder_hidden_states.cpu().numpy().astype(np.float32), ) np.save( "whisper_decoder_attention_mask.npy", dummy_decoder_attention_mask.cpu().numpy().astype(np.int64), ) ``` Step 5: Generate Melange Models [#step-5-generate-melange-models] Upload both models and their inputs via the [Melange Dashboard](/model-deployment/dashboard): * Encoder model `whisper_encoder.pt2` with input `whisper_input_features.npy` * Decoder model `whisper_decoder.pt2` with inputs (in order): `whisper_decoder_input_ids.npy`, `whisper_encoder_hidden_states.npy`, `whisper_decoder_attention_mask.npy` The decoder model requires three input files. Make sure to provide them in the correct order as shown above. See [Supported Formats](/model-preparation/supported-formats) for details on input ordering. Step 6: Implement ZeticMLangeModel [#step-6-implement-zeticmlangemodel] For detailed application setup, please follow the [Android Integration Guide](/platform-integration/android/setup) guide. ```kotlin val encoderModel = ZeticMLangeModel(this, PERSONAL_KEY, "OpenAI/whisper-tiny-encoder") val decoderModel = ZeticMLangeModel(this, PERSONAL_KEY, "OpenAI/whisper-tiny-decoder") val inputFeatures: FloatArray = whisper.melSpectrogram(audioData) val encoderInputs = arrayOf( Tensor.of(inputFeatures, DataType.Float32, intArrayOf(1, 80, 3000)) ) val encoderOutputs = encoderModel.run(encoderInputs) val encoderHidden = encoderOutputs[0] val inputIds = LongArray(1 * 448) val attnMask = LongArray(1 * 448) { 1L } val decoderInputs = arrayOf( Tensor.of(inputIds, DataType.Int64, intArrayOf(1, 448)), encoderHidden, Tensor.of(attnMask, DataType.Int64, intArrayOf(1, 448)), ) val decoderOutputs = decoderModel.run(decoderInputs) ``` For detailed application setup, please follow the [iOS Integration Guide](/platform-integration/ios/setup) guide. ```swift let encoderModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "OpenAI/whisper-tiny-encoder") let decoderModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "OpenAI/whisper-tiny-decoder") let inputFeatures: [Float] = whisper.melSpectrogram(audioData) let encoderInputs = [ Tensor( data: inputFeatures.withUnsafeBufferPointer { Data(buffer: $0) }, dataType: BuiltinDataType.float32, shape: [1, 80, 3000] ) ] let encoderOutputs = try encoderModel.run(inputs: encoderInputs) let encoderHidden = encoderOutputs[0] let inputIds: [Int64] = Array(repeating: 0, count: 1 * 448) let attnMask: [Int64] = Array(repeating: 1, count: 1 * 448) let decoderInputs = [ Tensor( data: inputIds.withUnsafeBufferPointer { Data(buffer: $0) }, dataType: BuiltinDataType.int64, shape: [1, 448] ), encoderHidden, Tensor( data: attnMask.withUnsafeBufferPointer { Data(buffer: $0) }, dataType: BuiltinDataType.int64, shape: [1, 448] ), ] let decoderOutputs = try decoderModel.run(inputs: decoderInputs) ``` For detailed application setup, please follow the [Flutter Integration Guide](/platform-integration/flutter/setup) guide. ```dart import 'dart:typed_data'; import 'package:zetic_mlange/zetic_mlange.dart'; final encoderModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'OpenAI/whisper-tiny-encoder', ); final decoderModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'OpenAI/whisper-tiny-decoder', ); final inputFeatures = whisper.melSpectrogram(audioData); final encoderInputs = [ Tensor.float32View( inputFeatures, shape: const [1, 80, 3000], ), ]; final encoderOutputs = encoderModel.run(encoderInputs); final encoderHidden = encoderOutputs[0]; final inputIds = Int64List(1 * 448); final attnMask = Int64List.fromList(List.filled(1 * 448, 1)); final decoderInputs = [ Tensor.int64List(inputIds, shape: const [1, 448]), encoderHidden, Tensor.int64List(attnMask, shape: const [1, 448]), ]; final decoderOutputs = decoderModel.run(decoderInputs); ``` Step 7: Use the Whisper Feature Wrapper [#step-7-use-the-whisper-feature-wrapper] The `WhisperFeatureWrapper` handles audio-to-Mel-Spectrogram conversion and token decoding. You can find `WhisperDecoder` and `WhisperEncoder` implementations in [ZETIC Melange apps](https://github.com/zetic-ai/ZETIC_Melange_apps/tree/main/whisper). Complete Speech Recognition Implementation [#complete-speech-recognition-implementation] ```kotlin // Initialize components val whisper = WhisperFeatureWrapper() val encoder = ZeticMLangeModel(this, PERSONAL_KEY, "OpenAI/whisper-tiny-encoder") val decoder = ZeticMLangeModel(this, PERSONAL_KEY, "OpenAI/whisper-tiny-decoder") // Process audio val features = whisper.process(audioData) // Run encoder encoder.process(features) // Generate tokens using decoder val generatedIds = decoder.generateTokens(outputs) // Convert tokens to text val text = whisper.decodeToken(generatedIds.toIntArray(), true) ``` ```swift // Initialize components let wrapper = WhisperFeatureWrapper() let encoder = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "OpenAI/whisper-tiny-encoder") let decoder = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "OpenAI/whisper-tiny-decoder") // Process audio to features let features = wrapper.process(input.audio) // Run encoder let outputs = encoder.process(features) // Generate tokens using decoder let generatedIds = decoder.process(outputs) // Convert tokens to text let text = wrapper.decodeToken(generatedIds, true) return WhisperOutput(text: text) ``` ```dart // Initialize components final encoder = await ZeticMLangeModel.create( personalKey: personalKey, name: 'OpenAI/whisper-tiny-encoder', ); final decoder = await ZeticMLangeModel.create( personalKey: personalKey, name: 'OpenAI/whisper-tiny-decoder', ); // Process audio to features final features = whisperToMelSpectrogram(audioData); // Run encoder final encoderOutputs = encoder.run(features); // Generate tokens using decoder final generatedIds = generateWhisperTokens(decoder, encoderOutputs); // Convert tokens to text final text = decodeWhisperTokens(generatedIds); ``` *** Conclusion [#conclusion] With ZETIC Melange, implementing on-device speech recognition with NPU acceleration is straightforward and efficient. Whisper provides robust multilingual speech recognition and translation capabilities. The three-component pipeline (Feature Extractor, Encoder, Decoder) is cleanly abstracted through the Melange SDK. We are continuously adding new models to our examples and [HuggingFace](https://huggingface.co/zetic-ai) page. Stay tuned, and [contact us](mailto:contact@zetic.ai) for collaborations! # Supertonic Text-to-Speech (/tutorials/supertonic-text-to-speech) This guide shows how to run Supertonic 3 text-to-speech on a physical Android or iOS device with ZETIC Melange. Melange loads the four model stages and selects the on-device runtime; your app prepares the Supertonic inputs, runs the stages in order, and writes the output samples as a WAV file. This guide targets **ZeticMLange Android 1.10.0** and **ZeticMLange iOS 1.10.0**. The examples use the Supertonic 3 models validated by ZETIC with eight denoising steps and 44.1 kHz output. How the pipeline works [#how-the-pipeline-works] | Stage | Melange model | Inputs | Output | | ------------------ | ---------------------------------- | -------------------------------------------------------------------------------------- | --------------------- | | Duration predictor | `palm/supertonic3-dp`, version `1` | Text IDs, duration style, text mask | Predicted duration | | Text encoder | `palm/supertonic3-te`, version `1` | Text IDs, text-to-latent style, text mask | Text embedding | | Vector estimator | `palm/supertonic3-ve`, version `1` | Noisy latent, text embedding, style, latent mask, text mask, current step, total steps | Denoised latent | | Vocoder | `palm/supertonic3-vo`, version `1` | Denoised latent | Float32 audio samples | The vector estimator output becomes its next input for eight iterations. The vocoder then converts the final latent tensor into audio. Prerequisites [#prerequisites] * Complete the [Android setup](/platform-integration/android/setup) or [iOS setup](/platform-integration/ios/setup). * Use a physical device. Melange inference requires device acceleration that is not available in a simulator or emulator. * Get a **Personal Key** from the [Melange Dashboard](https://melange.zetic.ai). Inject it through local or CI configuration; never commit it to source control. * Download [`unicode_indexer.json`](https://huggingface.co/Supertone/supertonic-3/blob/main/onnx/unicode_indexer.json) and the [`M1.json`](https://huggingface.co/Supertone/supertonic-3/blob/main/voice_styles/M1.json) voice style from the official Supertonic 3 model repository. Review the [model license](https://huggingface.co/Supertone/supertonic-3/blob/main/LICENSE) before distribution. The Supertonic JSON assets are not included in the Melange SDK. Bundle your approved copies with the application. Do not put your Personal Key in this asset directory. Add the assets [#add-the-assets] The helper code in this guide loads both files from a `supertonic` resource directory. Place the files under the application target: ```text app/src/main/assets/supertonic/M1.json app/src/main/assets/supertonic/unicode_indexer.json ``` Load them through `context.assets.open("supertonic/")`. Add a `supertonic` folder containing both files to the application target: ```text supertonic/M1.json supertonic/unicode_indexer.json ``` Preserve the folder as a bundle resource. Verify that this lookup succeeds for each file: ```swift Bundle.main.url( forResource: "unicode_indexer", withExtension: "json", subdirectory: "supertonic" ) ``` `unicode_indexer.json` must contain 65,536 integer entries. `M1.json` must contain `style_dp` with shape `[1, 8, 16]` and `style_ttl` with shape `[1, 50, 256]`. Configure the four models [#configure-the-four-models] Use fixed version `1` for every stage so that all four deployed models share the validated tensor contract. ```kotlin import android.content.Context import com.zeticai.mlange.core.model.ModelMode import com.zeticai.mlange.core.model.ZeticMLangeModel private fun loadSupertonicModels( context: Context, personalKey: String, ): List { val models = mutableListOf() try { listOf( "palm/supertonic3-dp", "palm/supertonic3-te", "palm/supertonic3-ve", "palm/supertonic3-vo", ).forEach { name -> models += ZeticMLangeModel( context = context.applicationContext, personalKey = personalKey, name = name, version = 1, modelMode = ModelMode.RUN_AUTO, ) } return models } catch (error: Exception) { models.forEach(ZeticMLangeModel::close) throw error } } ``` ```swift import ZeticMLange private func loadSupertonicModels( personalKey: String ) async throws -> [ZeticMLangeModel] { let names = [ "palm/supertonic3-dp", "palm/supertonic3-te", "palm/supertonic3-ve", "palm/supertonic3-vo" ] var models: [ZeticMLangeModel] = [] do { for name in names { models.append(try await ZeticMLangeModel( personalKey: personalKey, name: name, version: 1, modelMode: .RUN_AUTO )) } return models } catch { models.forEach { $0.close() } throw error } } ``` Model initialization can download artifacts on first use. Keep it off the main thread and close any models that were already created if a later stage fails to initialize. Prepare the tensors [#prepare-the-tensors] Use these constants for the validated deployment: ```text sample rate: 44,100 Hz text bucket: 128 latent dimension: 144 latent chunk size: 512 ร— 6 = 3,072 samples denoising steps: 8 speech speed: 1.05 language tag: en ``` The platform helper layer must do the following before inference: 1. Normalize the input text and wrap it as `text`. 2. Map each Unicode scalar through `unicode_indexer.json`, pad the IDs to 128 elements, and create a `[1, 1, 128]` Float32 text mask. 3. Reject input whose normalized, language-wrapped representation exceeds 128 Unicode scalars. Split longer text at sentence boundaries and synthesize each chunk separately. 4. Decode and flatten `style_dp` and `style_ttl` from `M1.json`. 5. Compute `validLength = clamp(ceil((duration / 1.05 ร— 44,100) / 3,072), 1, 128)`. Fill the valid frames of a `[1, 144, 128]` latent tensor with Gaussian noise and create a `[1, 1, 128]` latent mask. The validated Android deployment may expose the text-ID input as Int32 or Int64. Check the first input buffer size independently for the duration predictor and text encoder, then create a `[1, 128]` tensor with the matching integer type. The validated iOS deployment uses Int32 IDs. For the preprocessing algorithm and WAV encoding details, refer to Supertonic's public [Java helper](https://github.com/supertone-inc/supertonic/blob/main/java/Helper.java) and [iOS example](https://github.com/supertone-inc/supertonic/tree/main/ios/ExampleiOSApp). Use the constants and model input order in this guide when running the Melange-hosted models. Run the stages [#run-the-stages] The following excerpts show the Melange-specific orchestration. They assume your helper layer provides: * `tokenizer.encode(text)` โ†’ padded IDs and text mask * `style.dp` and `style.ttl` โ†’ flattened Float32 style arrays * `sampleNoise(duration)` โ†’ noisy latent, latent mask, and valid latent length * tensor conversion and Float32 output decoding helpers * a 16-bit mono WAV writer ```kotlin private fun runSupertonic( models: List, text: String, ): FloatArray { require(models.size == 4) val (dp, te, ve, vo) = models val encoded = tokenizer.encode(text) val mask = Tensor.of(encoded.mask, shape = intArrayOf(1, 1, 128)) val styleDp = Tensor.of(style.dp, shape = intArrayOf(1, 8, 16)) val styleTtl = Tensor.of(style.ttl, shape = intArrayOf(1, 50, 256)) val idsForDp = textIdsTensorFor(dp, encoded.ids) val durationTensor = dp.run(arrayOf(idsForDp, styleDp, mask)).firstOrNull() ?: throw SupertonicException("Duration predictor returned no output") val duration = durationTensor.toFloatArray().firstOrNull() ?: throw SupertonicException("Duration predictor output is empty") val idsForTe = textIdsTensorFor(te, encoded.ids) val textEmbedding = te.run(arrayOf(idsForTe, styleTtl, mask)).firstOrNull() ?: throw SupertonicException("Text encoder returned no output") val noise = sampleNoise(duration) var noisy = Tensor.of(noise.values, shape = intArrayOf(1, 144, 128)) val latentMask = Tensor.of(noise.mask, shape = intArrayOf(1, 1, 128)) val totalSteps = Tensor.of(floatArrayOf(8f), shape = intArrayOf(1)) repeat(8) { step -> val currentStep = Tensor.of(floatArrayOf(step.toFloat()), shape = intArrayOf(1)) noisy = ve.run( arrayOf( noisy, textEmbedding, styleTtl, latentMask, mask, currentStep, totalSteps, ), ).firstOrNull() ?: throw SupertonicException("Vector estimator returned no output") } val wavTensor = vo.run(arrayOf(noisy)).firstOrNull() ?: throw SupertonicException("Vocoder returned no output") val wav = wavTensor.toFloatArray() val validSamples = minOf(wav.size, noise.validLength * 3_072) if (validSamples <= 0) { throw SupertonicException("Vocoder output is empty") } return wav.copyOf(validSamples) } ``` ```swift private func runSupertonic( models: [ZeticMLangeModel], text: String ) throws -> [Float] { guard models.count == 4 else { throw SupertonicError.invalidModelCount } let dp = models[0] let te = models[1] let ve = models[2] let vo = models[3] let encoded = try tokenizer.encode(text) let ids = Tensor.int32(encoded.ids, shape: [1, 128]) let mask = Tensor.float32(encoded.mask, shape: [1, 1, 128]) let styleDP = Tensor.float32(style.dp, shape: [1, 8, 16]) let styleTTL = Tensor.float32(style.ttl, shape: [1, 50, 256]) let durationOutput = try dp.run(inputs: [ids, styleDP, mask]) guard let duration = try durationOutput.first?.floatArray().first else { throw SupertonicError.emptyOutput } guard let textEmbedding = try te.run(inputs: [ids, styleTTL, mask]).first else { throw SupertonicError.emptyOutput } let noise = sampleNoise(duration: duration) var noisy = Tensor.float32(noise.values, shape: [1, 144, 128]) let latentMask = Tensor.float32(noise.mask, shape: [1, 1, 128]) let totalSteps = Tensor.float32([8], shape: [1]) for step in 0..<8 { let currentStep = Tensor.float32([Float(step)], shape: [1]) guard let denoised = try ve.run(inputs: [ noisy, textEmbedding, styleTTL, latentMask, mask, currentStep, totalSteps ]).first else { throw SupertonicError.emptyOutput } noisy = denoised } guard let wavTensor = try vo.run(inputs: [noisy]).first else { throw SupertonicError.emptyOutput } let wav = try wavTensor.floatArray() let validSamples = min(wav.count, noise.validLength * 3_072) guard validSamples > 0 else { throw SupertonicError.emptyOutput } return Array(wav.prefix(validSamples)) } ``` These are orchestration excerpts, not standalone files. Keep bounds, tensor type, asset-shape, and empty-output validation in your helper implementation instead of replacing those checks with force unwraps. Run without blocking the UI [#run-without-blocking-the-ui] Create, run, and close all four models on a worker thread. The one-shot pattern below favors predictable cleanup. For repeated synthesis, you may keep a pipeline alive and reuse it, but serialize access and close it when its owner is destroyed. ```kotlin suspend fun synthesize( context: Context, personalKey: String, text: String, ): File = withContext(Dispatchers.IO) { val models = loadSupertonicModels(context, personalKey) try { val samples = runSupertonic(models, text) WavWriter.write( directory = context.filesDir, samples = samples, sampleRate = 44_100, fileName = "supertonic-result.wav", ) } finally { models.forEach(ZeticMLangeModel::close) } } ``` ```swift func synthesize( personalKey: String, text: String ) async throws -> URL { try await Task.detached(priority: .userInitiated) { let models = try await loadSupertonicModels(personalKey: personalKey) defer { models.forEach { $0.close() } } let samples = try runSupertonic(models: models, text: text) return try WavWriter.write( samples: samples, sampleRate: 44_100, fileName: "supertonic-result.wav" ) }.value } ``` Update UI state or start audio playback only after returning to the main actor/thread. Troubleshooting [#troubleshooting] | Symptom | Check | | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | A model fails to load | Verify the Personal Key, exact model name, version `1`, network access for the first download, and physical-device support. | | `M1.json` or `unicode_indexer.json` is missing | Confirm the files are included in the built application under the `supertonic` resource directory. | | Input is longer than the bucket | Split text so each normalized, language-wrapped chunk is at most 128 Unicode scalars. | | An input size or type does not match | Recheck the shapes and stage input order above. On Android, also select Int32 or Int64 text IDs from each model's first input buffer size. | | The app freezes while loading or synthesizing | Move model construction, inference, and WAV file I/O off the main/UI thread. | | Audio is empty or distorted | Decode the vocoder output as Float32, trim it to `validLength ร— 3,072`, clamp samples to `[-1, 1]`, and encode mono PCM16 at 44.1 kHz. | Related resources [#related-resources] * [ZeticMLangeModel for Android](/api-reference/android/ZeticMLangeModel) * [ZeticMLangeModel for iOS](/api-reference/ios/ZeticMLangeModel) * [Tensor for Android](/api-reference/android/Tensor) * [Tensor for iOS](/api-reference/ios/Tensor) * [Official Supertonic repository](https://github.com/supertone-inc/supertonic) * [Official Supertonic 3 model and asset repository](https://huggingface.co/Supertone/supertonic-3) # Tensor (/api-reference/android/Tensor) This page reflects `ZeticMLange Android 1.10.0`. The `Tensor` class is the unified data container passed to and returned from `ZeticMLangeModel.run()`. It wraps a direct `ByteBuffer` together with its `DataType` and shape, and provides typed accessors for reading and writing the underlying data. Package [#package] ``` com.zeticai.mlange.core.tensor ``` Import [#import] ```kotlin import com.zeticai.mlange.core.tensor.Tensor import com.zeticai.mlange.core.tensor.DataType ``` *** Companion Factories [#companion-factories] The `Tensor.of(...)` factories are the recommended way to create tensors from typed arrays or NIO buffers. Each overload infers a sensible default `DataType` matching the source type. `Tensor.of(ByteBuffer, ...)` accepts both direct and non-direct buffers. A non-direct buffer is copied into direct storage. The `Tensor(...)` constructor itself requires a direct buffer. ```kotlin Tensor.of( data: FloatArray, dataType: DataType = DataType.Float32, shape: IntArray = intArrayOf(data.size), immediateRelease: Boolean = false, ): Tensor ``` Overloads are provided for the following source types: | Source | Default `DataType` | | ----------------------------- | ------------------ | | `ByteBuffer` | `DataType.Int8` | | `ByteArray` | `DataType.Int8` | | `FloatArray`, `FloatBuffer` | `DataType.Float32` | | `DoubleArray`, `DoubleBuffer` | `DataType.Float64` | | `IntArray`, `IntBuffer` | `DataType.Int32` | | `LongArray`, `LongBuffer` | `DataType.Int64` | | `ShortArray`, `ShortBuffer` | `DataType.Int16` | | `CharArray`, `CharBuffer` | `DataType.Int16` | Each overload also has a variant that accepts `shape: Array` in addition to `shape: IntArray`. ```kotlin val input = Tensor.of( data = floatArrayOf(/* ... */), dataType = DataType.Float32, shape = intArrayOf(1, 3, 640, 640), ) ``` `Tensor.of(data, dataType, shape)` (generic array) [#tensorofdata-datatype-shape-generic-array] ```kotlin inline fun of( data: Array, dataType: DataType, shape: Array, immediateRelease: Boolean = false, ): Tensor ``` Supported element types: `Int`, `Float`, `Long`, `Double`, `Char`, `Short`. `Tensor.random(dataType, shape, immediateRelease)` [#tensorrandomdatatype-shape-immediaterelease] Creates a tensor filled with random bytes. Handy for smoke-testing a model graph. ```kotlin fun random( dataType: DataType, shape: IntArray, immediateRelease: Boolean = false, ): Tensor ``` ```kotlin val fakeInput = Tensor.random(DataType.Float32, intArrayOf(1, 3, 640, 640)) ``` *** Constructor [#constructor] `Tensor(data, dataType, shape, immediateRelease)` [#tensordata-datatype-shape-immediaterelease] Creates a tensor directly from a direct `ByteBuffer`. Prefer the [`Tensor.of(...)`](#companion-factories) factories above; use this constructor only when you already hold a direct `ByteBuffer` (for example, wrapping a native allocation). ```kotlin Tensor( data: ByteBuffer, dataType: DataType = DataType.Int8, shape: IntArray = intArrayOf(data.capacity()), immediateRelease: Boolean = false, ) ``` | Parameter | Type | Description | | ------------------ | ------------ | -------------------------------------------------------------------------------------------------------------------------- | | `data` | `ByteBuffer` | A direct `ByteBuffer` holding the raw tensor bytes. Non-direct buffers will throw. | | `dataType` | `DataType` | Element type of the tensor. See [DataType](#datatype). Defaults to `DataType.Int8`. | | `shape` | `IntArray` | Tensor shape. The product of its dimensions times `dataType.size` must equal `data.capacity()`. | | `immediateRelease` | `Boolean` | When `true`, registers the tensor with `TensorCleaner` so its native memory is released as soon as it becomes unreachable. | **Throws:** `IllegalArgumentException` if `data` is not a direct `ByteBuffer`, or if `shape` does not match `data.capacity()`. ```kotlin val buffer = ByteBuffer.allocateDirect(4 * 3 * 4).order(ByteOrder.nativeOrder()) val tensor = Tensor(buffer, DataType.Float32, intArrayOf(1, 3, 2, 2)) ``` *** Methods [#methods] `count()` [#count] Returns the number of elements in the tensor (total bytes divided by `dataType.size`). ```kotlin fun count(): Int ``` `size()` [#size] Returns the size of the tensor in bytes (equivalent to `data.capacity()`). ```kotlin fun size(): Int ``` `data()` [#datat] Reifies the underlying `ByteBuffer` as a typed view. Useful for reading inference outputs as floats, ints, etc. ```kotlin inline fun data(): T ``` Supported `T`: | Type | Returns | | ------------------------------------------------------------------------------------- | ----------------------------- | | `ByteBuffer` | The underlying buffer itself. | | `IntBuffer`, `LongBuffer`, `FloatBuffer`, `DoubleBuffer`, `CharBuffer`, `ShortBuffer` | A typed view on the buffer. | **Throws:** `IllegalArgumentException` if `T` is not one of the supported types. ```kotlin val floatView: FloatBuffer = tensor.data() ``` `get(index)` [#gettindex] Reads a single element at `index` as the requested numeric type. ```kotlin inline fun get(index: Int): T ``` Supported `T`: `Int`, `Long`, `Float`, `Double`, `Short`, `Byte`. ```kotlin val first: Float = tensor.get(0) ``` `set(index, value)` [#setindex-value] Writes `value` into the tensor at `index`. The write type is inferred from the runtime type of `value`. ```kotlin fun set(index: Int, value: T) ``` Supported value types: `Int`, `Long`, `Float`, `Double`, `Char`, `Short`. ```kotlin tensor.set(0, 1.5f) ``` `order(order)` [#orderorder] Changes the byte order of the underlying buffer. ```kotlin fun order(order: ByteOrder) ``` ```kotlin tensor.order(ByteOrder.LITTLE_ENDIAN) ``` `from(tensor)` [#fromtensor] Copies another tensor's bytes into this tensor. The source tensor must have the same shape. ```kotlin fun from(tensor: Tensor) ``` **Throws:** `IllegalArgumentException` if the source tensor's shape does not match. ```kotlin output.from(reusedOutputTensor) ``` `copy(data, shape, dataType, immediateRelease)` [#copydata-shape-datatype-immediaterelease] Copies bytes from `data` into this tensor in place. The total byte size implied by `shape` and `dataType` must equal this tensor's current size. ```kotlin fun copy( data: ByteBuffer = this.data, shape: IntArray = this.shape, dataType: DataType = this.dataType, immediateRelease: Boolean = this.immediateRelease, ) ``` **Throws:** `IllegalArgumentException` if the source or destination buffer is smaller than the expected byte count. When `data` is non-direct, `copy(...)` first copies it into temporary direct storage. *** DataType [#datatype] `DataType` is a sealed interface describing the element type and byte size of a tensor. ```kotlin import com.zeticai.mlange.core.tensor.DataType ``` | Variant | Element size (bytes) | | --------------------------------------------------------------------- | -------------------- | | `DataType.Float32` | 4 | | `DataType.Float64` | 8 | | `DataType.Float16`, `DataType.BFloat16` | 2 | | `DataType.UInt8`, `DataType.Int8`, `DataType.QInt8`, `DataType.QInt4` | 1 | | `DataType.UInt16`, `DataType.Int16`, `DataType.QInt16` | 2 | | `DataType.UInt32`, `DataType.Int32`, `DataType.QInt32` | 4 | | `DataType.UInt64`, `DataType.Int64` | 8 | | `DataType.Boolean` | 1 | | `DataType.Unknown(size)` | Caller-provided | Use `DataType.from(name)` to look up a type by its lowercase name (e.g. `"float32"`, `"int8"`, `"bool"`), and `DataType.toName(type)` for the reverse. *** Properties [#properties] `data` [#data] The underlying direct `ByteBuffer` backing the tensor. ```kotlin val data: ByteBuffer ``` Prefer the typed [`data()`](#datat) accessor when you need a typed view. *** Full Working Example [#full-working-example] Create tensors that match the model's expected shape and data type, then pass them to `ZeticMLangeModel.run(...)`. ```kotlin import com.zeticai.mlange.core.model.ZeticMLangeModel import com.zeticai.mlange.core.tensor.Tensor class MainActivity : AppCompatActivity() { private lateinit var model: ZeticMLangeModel override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) model = ZeticMLangeModel(this, PERSONAL_KEY, "Steve/YOLOv11_comparison") } private fun runFrame(input: Tensor) { // (1) Run inference with tensors matching the model input order val outputs: Array = model.run(arrayOf(input)) // (2) Read outputs as typed arrays val logits: FloatArray = outputs[0].data() } } ``` *** Memory Management Recommendation [#memory-management-recommendation] `Tensor` wraps a direct `ByteBuffer` that lives in off-heap native memory. How you manage that memory depends on **who owns the underlying buffer**: 1. **Reuse tensors when practical.** If your preprocessing output has the same shape every frame, keep the source arrays stable and update their contents before creating or reusing `Tensor` values for `run(...)`. 2. **Only set `immediateRelease = true` on tensors you allocate yourself.** When you build a tensor with [`Tensor.of(...)`](#companion-factories) (for example, a per-frame camera input) and pass it to `run()`, the model copies your bytes into its own input buffer. Your tensor is useless after `run()` returns, so registering it with `TensorCleaner` via `immediateRelease = true` lets off-heap memory be reclaimed as soon as the object becomes unreachable โ€” which prevents `OutOfMemoryError: Direct buffer memory` under tight loops. See Also [#see-also] * [ZeticMLangeModel (Android)](/api-reference/android/ZeticMLangeModel): Uses `Tensor` as the input and output type of `run()` * [Tensor (iOS)](/api-reference/ios/Tensor): iOS equivalent * [Enums and Constants](/api-reference/android/enums-and-constants): Other Android SDK enums # ZeticMLangeHFModel (/api-reference/android/ZeticMLangeHFModel) This page reflects `ZeticMLange Android 1.10.0`. `ZeticMLangeHFModel` loads compatible models directly from Hugging Face repositories. Import [#import] ```kotlin import com.zeticai.mlange.core.model.ZeticMLangeHFModel ``` Constructor [#constructor] ```kotlin ZeticMLangeHFModel( context: Context, repoId: String, userAccessToken: String? = null, manifestDir: String? = null, index: Int = 0, ) ``` | Parameter | Type | Default | Description | | ----------------- | --------- | ------- | -------------------------------------------------------- | | `context` | `Context` | - | Android context used for cache and file access. | | `repoId` | `String` | - | Hugging Face repository ID. | | `userAccessToken` | `String?` | `null` | Optional Hugging Face access token. | | `manifestDir` | `String?` | `null` | Optional local manifest directory. | | `index` | `Int` | `0` | Model index when a repository contains multiple entries. | ```kotlin val model = ZeticMLangeHFModel( context = context, repoId = "zetic-ai/yolov11n", ) ``` `run(inputs)` [#runinputs] ```kotlin fun run(inputs: Array = emptyArray()): Array ``` Runs inference and returns output tensors. Lifecycle [#lifecycle] ```kotlin val isClosed: Boolean fun close() ``` Call `close()` when the model is no longer needed. Related [#related] * [ZeticMLangeModel](/api-reference/android/ZeticMLangeModel) * [Tensor](/api-reference/android/Tensor) * [Hugging Face Models](/model-preparation/hugging-face-models) # ZeticMLangeLLMModel (/api-reference/android/ZeticMLangeLLMModel) This page reflects `ZeticMLange Android 1.10.0`. `ZeticMLangeLLMModel` loads an on-device LLM from the Melange registry and supports text generation, token streaming, function calling, image response for LFM-VL models, and KV state persistence. Import [#import] ```kotlin import com.zeticai.mlange.core.model.llm.ZeticMLangeLLMModel ``` Constructor [#constructor] ```kotlin ZeticMLangeLLMModel( context: Context, personalKey: String, name: String, version: Int? = null, modelMode: LLMModelMode = LLMModelMode.RUN_AUTO, cacheHandlingPolicy: ModelCacheHandlingPolicy = ModelCacheHandlingPolicy.REMOVE_OVERLAPPING, initOption: LLMInitOption = LLMInitOption(), onDownload: ((Float) -> Unit)? = null, ) ``` | Parameter | Type | Default | Description | | --------------------- | -------------------------- | -------------------- | ------------------------------------------------- | | `context` | `Context` | - | Android context used for cache and file access. | | `personalKey` | `String` | - | Personal key for accessing the model. | | `name` | `String` | - | Model name in `account_name/project_name` format. | | `version` | `Int?` | `null` | Model version. `null` loads the latest version. | | `modelMode` | `LLMModelMode` | `RUN_AUTO` | Backend selection strategy. | | `cacheHandlingPolicy` | `ModelCacheHandlingPolicy` | `REMOVE_OVERLAPPING` | Managed artifact cache cleanup policy. | | `initOption` | `LLMInitOption` | `LLMInitOption()` | LLM initialization options. | | `onDownload` | `((Float) -> Unit)?` | `null` | Download progress callback from `0.0` to `1.0`. | ```kotlin val model = ZeticMLangeLLMModel( context = context, personalKey = PERSONAL_KEY, name = "account_name/project_name", initOption = LLMInitOption(nCtx = 4096), ) ``` Text Generation [#text-generation] `run(text)` [#runtext] Starts generation for a prompt. ```kotlin fun run(text: String): LLMRunResult ``` ```kotlin val result = model.run("Explain on-device AI in one paragraph.") ``` `waitForNextToken()` [#waitfornexttoken] Waits for the next generated token. ```kotlin fun waitForNextToken(): LLMNextTokenResult ``` ```kotlin while (true) { val next = model.waitForNextToken() if (next.isFinal || next.token.isEmpty()) break append(next.token) } ``` Vision-Language Response [#vision-language-response] Use `respond(...)` with an LFM-VL-capable model. ```kotlin data class Image( val rgb: ByteArray, val width: Int, val height: Int, ) fun respond( systemPrompt: String = "", userText: String, image: ZeticMLangeLLMModel.Image, ): Flow ``` ```kotlin val image = ZeticMLangeLLMModel.Image(rgbBytes, width, height) model.respond( systemPrompt = "Answer briefly.", userText = "What is in this image?", image = image, ).collect { token -> append(token) } ``` Function Calling [#function-calling] ```kotlin var functionCallingSystemPrompt: String? fun registerTool(spec: LLMToolSpec, executor: LLMToolExecutor) fun unregisterTool(name: String): Boolean fun clearTools() fun registeredTools(): List fun runWithTools(text: String): Flow ``` After registering a tool, call `runWithTools(...)` and collect its `Flow`. `run(...)` is only available when no tools are registered. ```kotlin model.registerTool( LLMToolSpec( name = "lookup", description = "Look up local app data.", parametersJson = """{"type":"object","properties":{"query":{"type":"string"}}}""", ), ) { call -> LLMToolResult(content = """{"result":"Found"}""") } model.runWithTools("Use lookup to answer the question.").collect { token -> append(token) } ``` KV State Persistence [#kv-state-persistence] ```kotlin fun saveKVState(path: String) fun loadKVState(path: String) fun resetKVState() ``` Use these APIs to persist or reset the current LLM state for resume flows. Lifecycle [#lifecycle] ```kotlin val isClosed: Boolean fun cleanUp() fun resetSession() fun close() fun deinit() ``` Call `cleanUp()` or `resetSession()` before starting a fresh conversation. Call `close()` when the model is no longer needed. Related [#related] * [Function Calling](/llm-inference/function-calling) * [RAG](/llm-inference/rag) * [Vision-Language Inference](/llm-inference/vision-language) * [Enums and constants](/api-reference/android/enums-and-constants) # ZeticMLangeModel (/api-reference/android/ZeticMLangeModel) This page reflects `ZeticMLange Android 1.10.0`. `ZeticMLangeModel` loads a general on-device model from the Melange registry and runs tensor inference on Android. Import [#import] ```kotlin import com.zeticai.mlange.core.model.ZeticMLangeModel ``` Constructor [#constructor] ```kotlin ZeticMLangeModel( context: Context, personalKey: String, name: String, version: Int? = null, modelMode: ModelMode = ModelMode.RUN_AUTO, onDownload: ((Float) -> Unit)? = null, cacheHandlingPolicy: ModelCacheHandlingPolicy = ModelCacheHandlingPolicy.REMOVE_OVERLAPPING, ) ``` | Parameter | Type | Default | Description | | --------------------- | -------------------------- | -------------------- | ------------------------------------------------- | | `context` | `Context` | - | Android context used for cache and file access. | | `personalKey` | `String` | - | Personal key for accessing the model. | | `name` | `String` | - | Model name in `account_name/project_name` format. | | `version` | `Int?` | `null` | Model version. `null` loads the latest version. | | `modelMode` | `ModelMode` | `RUN_AUTO` | Backend selection strategy. | | `onDownload` | `((Float) -> Unit)?` | `null` | Download progress callback from `0.0` to `1.0`. | | `cacheHandlingPolicy` | `ModelCacheHandlingPolicy` | `REMOVE_OVERLAPPING` | Managed artifact cache cleanup policy. | ```kotlin val model = ZeticMLangeModel( context = context, personalKey = PERSONAL_KEY, name = "account_name/project_name", modelMode = ModelMode.RUN_AUTO, ) ``` `run(inputs)` [#runinputs] Runs inference with tensors matching the model input order. ```kotlin fun run(inputs: Array = emptyArray()): Array ``` | Parameter | Type | Description | | --------- | --------------- | ------------------------------------------------------------------ | | `inputs` | `Array` | Input tensors matching the model's expected shapes and data types. | **Returns:** output tensors in model output order. ```kotlin val outputs = model.run(arrayOf(inputTensor)) val firstOutput = outputs[0] ``` Lifecycle [#lifecycle] ```kotlin val isClosed: Boolean fun close() ``` Call `close()` when the model is no longer needed. ```kotlin model.close() ``` Related [#related] * [Tensor](/api-reference/android/Tensor) * [Enums and constants](/api-reference/android/enums-and-constants) * [Cache Management](/api-reference/cache-management) # Enums and Constants (/api-reference/android/enums-and-constants) This page reflects `ZeticMLange Android 1.10.0`. General Model Types [#general-model-types] | Type | Values | | -------------------------- | -------------------------------------------------------------------- | | `ModelMode` | `RUN_AUTO`, `RUN_FP32`, `RUN_QUANTIZED`, `RUN_SPEED`, `RUN_ACCURACY` | | `APType` | `CPU`, `GPU`, `NPU`, `NA` | | `ModelCacheHandlingPolicy` | `REMOVE_OVERLAPPING`, `KEEP_EXISTING` | `Target` contains backend identifiers used by selected general-model artifacts. Most apps use `ModelMode` and let backend selection choose the target. LLM Types [#llm-types] | Type | Values | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `LLMModelMode` | `RUN_AUTO`, `RUN_SPEED`, `RUN_ACCURACY` | | `LLMTarget` | `LLAMA_CPP`, `LITERT_LM`, `MLLM` | | `LLMQuantType` | `GGUF_QUANT_ORG`, `GGUF_QUANT_F16`, `GGUF_QUANT_BF16`, `GGUF_QUANT_Q8_0`, `GGUF_QUANT_Q4_K_M`, `GGUF_QUANT_Q3_K_M`, `GGUF_QUANT_Q2_K`, `GGUF_QUANT_Q6_K` | | `LLMKVCacheCleanupPolicy` | `CLEAN_UP_ON_FULL`, `DO_NOT_CLEAN_UP` | | `KVStateStatus` | Status values returned by KV state persistence operations. | `LLMInitOption` [#llminitoption] ```kotlin data class LLMInitOption( val kvCacheCleanupPolicy: LLMKVCacheCleanupPolicy = LLMKVCacheCleanupPolicy.CLEAN_UP_ON_FULL, val nCtx: Int = 2048, ) ``` | Field | Default | Description | | ---------------------- | ------------------ | --------------------------------------------------------- | | `kvCacheCleanupPolicy` | `CLEAN_UP_ON_FULL` | In-memory KV-cache behavior when context storage is full. | | `nCtx` | `2048` | Requested context size. The runtime may normalize it. | Result Types [#result-types] | Type | Purpose | | -------------------- | ---------------------------------------------------------------------------- | | `LLMRunResult` | Result from starting LLM generation. Includes status and prompt token count. | | `LLMNextTokenResult` | Token result returned by `waitForNextToken()`. | Related [#related] * [ZeticMLangeModel](/api-reference/android/ZeticMLangeModel) * [ZeticMLangeLLMModel](/api-reference/android/ZeticMLangeLLMModel) * [Tensor](/api-reference/android/Tensor) # RagPipeline (/api-reference/flutter/RagPipeline) This page reflects `zetic_mlange 1.10.0`. Flutter RAG uses the same composition style as Android and iOS: a `RagPipeline` combines a retriever, an existing `ZeticMLangeLLMModel`, and a `RagProfile`. Import [#import] ```dart import 'package:zetic_mlange/zetic_mlange.dart'; ``` `RagPipeline` [#ragpipeline] ```dart RagPipeline({ required RagRetriever retriever, required ZeticMLangeLLMModel llm, required RagProfile profile, RagPipelineConfig config = const RagPipelineConfig(), }) Stream respond({required String query, String? system}) ``` `respond(...)` retrieves context, starts generation, and streams generated tokens. ```dart final rag = RagPipeline( retriever: retriever, llm: model, profile: const RagProfile.qwen25(), ); await for (final token in rag.respond(query: 'What does ZeticMLange do?')) { append(token); } ``` `RagRetriever` [#ragretriever] ```dart abstract interface class RagRetriever { FutureOr> retrieve(String query, {required int topK}); } ``` Implement this interface for app-owned retrieval, remote vector stores, local databases, or other retrieval systems. `RetrievedChunk` [#retrievedchunk] ```dart const RetrievedChunk({ required String text, double? score, String? source, Map? metadata, }) ``` `text` is included in the augmented prompt. `score`, `source`, and `metadata` are optional retrieval metadata. `RagPipelineConfig` [#ragpipelineconfig] ```dart const RagPipelineConfig({ int topK = 5, int maxContextTokens = 2048, String systemPrefix = 'You are a helpful assistant. Use the context below to answer.', }) ``` `LocalRagPipeline` [#localragpipeline] ```dart static Future create({ required RagProfile profile, required String embedderGgufPath, LocalRagConfig config = const LocalRagConfig(), }) Future indexDocs(List docs) Future> retrieve(String query, {required int topK}) void close() ``` `LocalRagPipeline` implements `RagRetriever`, so it can be passed directly to `RagPipeline`. ```dart final localRag = await LocalRagPipeline.create( profile: const RagProfile.qwen25(backboneGgufPath: backbonePath), embedderGgufPath: embedderPath, ); await localRag.indexDocs([ const RagDocument(text: 'Local context.', source: 'docs'), ]); final rag = RagPipeline( retriever: localRag, llm: model, profile: const RagProfile.qwen25(backboneGgufPath: backbonePath), ); ``` `RagDocument` [#ragdocument] ```dart const RagDocument({required String text, required String source}) ``` Use `RagDocument` when indexing documents into `LocalRagPipeline`. `LocalRagConfig` [#localragconfig] ```dart const LocalRagConfig({ int topK = 5, int chunkSizeChars = 1024, int chunkOverlapChars = 128, int embedderNCtx = 512, RagEmbedderPooling embedderPooling = RagEmbedderPooling.mean, }) ``` Related [#related] * [RAG guide](/llm-inference/rag) * [ZeticMLangeLLMModel](/api-reference/flutter/ZeticMLangeLLMModel) * [Enums and constants](/api-reference/flutter/enums-and-constants) # Tensor (/api-reference/flutter/Tensor) This page reflects `zetic_mlange 1.10.0`. `Tensor` is the Dart data container used for model inputs and outputs. It stores bytes, a `DataType`, and a shape. Import [#import] ```dart import 'package:zetic_mlange/zetic_mlange.dart'; ``` *** Constructors [#constructors] `Tensor` [#tensor] ```dart Tensor({ required Uint8List data, DataType dataType = DataType.int8, List? shape, }) ``` Creates a tensor with defensive-copy semantics. If `shape` is omitted, the tensor shape defaults to `[data.lengthInBytes]`. `Tensor.view` [#tensorview] ```dart Tensor.view({ required Uint8List data, DataType dataType = DataType.int8, List? shape, }) ``` Creates a tensor view over the provided byte list without copying. Mutating the source `Uint8List` also mutates the tensor bytes. For every constructor and factory, `shape` must describe exactly `data.lengthInBytes / dataType.bytesPerElement` elements. A mismatch throws `MlangeException`. *** Typed Factories [#typed-factories] ```dart Tensor.float32List(Float32List data, {List? shape}) Tensor.float32View(Float32List data, {List? shape}) Tensor.float64List(Float64List data, {List? shape}) Tensor.float64View(Float64List data, {List? shape}) Tensor.int32List(Int32List data, {List? shape}) Tensor.int32View(Int32List data, {List? shape}) Tensor.int64List(Int64List data, {List? shape}) Tensor.int64View(Int64List data, {List? shape}) Tensor.bytes(List data, {DataType dataType = DataType.int8, List? shape}) Tensor.bytesView(Uint8List data, {DataType dataType = DataType.int8, List? shape}) Tensor.random(DataType dataType, List shape, {Random? random}) ``` Use `List` factories when you want Dart to copy the source data. Use `View` factories when you already own a correctly laid-out typed list and want to avoid an extra Dart-side allocation. *** Properties [#properties] | Property | Type | Description | | -------------- | ----------- | -------------------------------------- | | `data` | `Uint8List` | Raw tensor bytes. | | `dataType` | `DataType` | Tensor element data type. | | `shape` | `List` | Tensor shape. | | `byteLength` | `int` | Number of bytes in `data`. | | `elementCount` | `int` | Number of elements implied by `shape`. | *** Methods [#methods] ```dart int count() int size() Uint8List asUint8List() Float32List asFloat32List() Float64List asFloat64List() Int32List asInt32List() Int64List asInt64List() ``` Typed accessors validate that `dataType` matches the requested view. | Method | Description | | ----------------- | ----------------------------------------------------------------------------- | | `count()` | Number of elements based on `data.lengthInBytes ~/ dataType.bytesPerElement`. | | `size()` | Number of bytes in `data`. | | `asUint8List()` | Raw byte view without dtype validation. | | `asFloat32List()` | Typed `Float32List` view. Requires `DataType.float32`. | | `asFloat64List()` | Typed `Float64List` view. Requires `DataType.float64`. | | `asInt32List()` | Typed `Int32List` view. Requires `DataType.int32`. | | `asInt64List()` | Typed `Int64List` view. Requires `DataType.int64`. | *** Example [#example] ```dart final input = Tensor.float32View( Float32List.fromList([1.0, 2.0, 3.0, 4.0]), shape: const [1, 4], ); print(input.count()); // 4 print(input.asFloat32List()); ``` # ZeticMLangeHFModel (/api-reference/flutter/ZeticMLangeHFModel) This page reflects `zetic_mlange 1.10.0`. `ZeticMLangeHFModel` loads compatible models directly from Hugging Face repositories. Import [#import] ```dart import 'package:zetic_mlange/zetic_mlange.dart'; ``` `create` [#create] ```dart static Future create( String repoId, { String? userAccessToken, String? manifestDir, int index = 0, CacheHandlingPolicy cacheHandlingPolicy = CacheHandlingPolicy.removeOverlapping, }) ``` | Parameter | Type | Default | Description | | --------------------- | --------------------- | ------------------- | ----------------------------------------------------------------- | | `repoId` | `String` | - | Hugging Face repository ID. | | `userAccessToken` | `String?` | `null` | Optional Hugging Face access token. | | `manifestDir` | `String?` | `null` | Optional local manifest directory. | | `index` | `int` | `0` | Model index when a repository contains multiple entries. | | `cacheHandlingPolicy` | `CacheHandlingPolicy` | `removeOverlapping` | Cache behavior when this model overlaps an existing cached model. | ```dart final model = await ZeticMLangeHFModel.create('zetic-ai/yolov11n'); ``` `run` [#run] ```dart List run([List inputs = const []]) ``` Runs inference and returns output tensors. Lifecycle [#lifecycle] ```dart bool get isClosed void close() ``` Call `close()` when the model is no longer needed. Related [#related] * [ZeticMLangeModel](/api-reference/flutter/ZeticMLangeModel) * [Tensor](/api-reference/flutter/Tensor) * [Hugging Face Models](/model-preparation/hugging-face-models) # ZeticMLangeLLMModel (/api-reference/flutter/ZeticMLangeLLMModel) This page reflects `zetic_mlange 1.10.0`. `ZeticMLangeLLMModel` loads an on-device LLM and supports text generation, token streaming, function calling, and image response for LFM-VL models. Import [#import] ```dart import 'package:zetic_mlange/zetic_mlange.dart'; ``` `create` [#create] ```dart static Future create({ required String personalKey, required String name, int? version, LLMModelMode modelMode = LLMModelMode.runAuto, ModelCacheHandlingPolicy cacheHandlingPolicy = CacheHandlingPolicy.removeOverlapping, LLMInitOption? initOption, LLMKVCacheCleanupPolicy kvCacheCleanupPolicy = LLMKVCacheCleanupPolicy.cleanUpOnFull, MlangeProgressCallback? onDownload, }) ``` | Parameter | Type | Default | Description | | ---------------------- | -------------------------- | ------------------- | ------------------------------------------------------ | | `personalKey` | `String` | - | Personal key for accessing the model. | | `name` | `String` | - | Model name in `account_name/project_name` format. | | `version` | `int?` | `null` | Model version. `null` loads the latest version. | | `modelMode` | `LLMModelMode` | `runAuto` | Backend selection strategy. | | `cacheHandlingPolicy` | `ModelCacheHandlingPolicy` | `removeOverlapping` | Managed artifact cache cleanup policy. | | `initOption` | `LLMInitOption?` | `null` | LLM initialization options. | | `kvCacheCleanupPolicy` | `LLMKVCacheCleanupPolicy` | `cleanUpOnFull` | Convenience default used when `initOption` is omitted. | | `onDownload` | `MlangeProgressCallback?` | `null` | Download progress callback from `0.0` to `1.0`. | ```dart final model = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: 'account_name/project_name', initOption: const LLMInitOption(nCtx: 4096), ); ``` Text Generation [#text-generation] `run` [#run] Starts generation for a prompt. ```dart LLMRunResult run(String text) ``` ```dart model.run('Explain on-device AI in one paragraph.'); ``` Reading Generated Tokens [#reading-generated-tokens] Waits for the next generated token. ```dart LLMNextTokenResult waitForNextToken() ``` ```dart while (true) { final next = model.waitForNextToken(); if (next.isFinished) break; append(next.token); } ``` Vision-Language Response [#vision-language-response] Use `respond(...)` with an LFM-VL-capable model. ```dart Future respond({ String systemPrompt = '', required String userText, required ZeticMLangeLLMImage image, }) ``` ```dart final image = ZeticMLangeLLMImage( rgb: rgbBytes, width: width, height: height, ); final response = await model.respond( systemPrompt: 'Answer briefly.', userText: 'What is in this image?', image: image, ); ``` Function Calling [#function-calling] ```dart String? functionCallingSystemPrompt void registerTool(LLMToolSpec spec, LLMToolExecutor executor) bool unregisterTool(String name) void clearTools() List registeredTools() ``` ```dart model.registerTool( const LLMToolSpec( name: 'lookup', description: 'Look up local app data.', parametersJson: '{"type":"object","properties":{"query":{"type":"string"}}}', ), (call) => const LLMToolResult(content: '{"result":"Found"}'), ); await for (final token in model.runWithTools( 'Use lookup to answer the question.', )) { append(token); } ``` `runWithTools` returns a `Stream`. Consume it to completion before starting another model operation. Lifecycle [#lifecycle] ```dart bool get isClosed void cleanUp() Future resetSession() Future close() @Deprecated('Use close() instead.') Future deinit() ``` Call `cleanUp()` or `resetSession()` before starting a fresh conversation. Call `close()` when the model is no longer needed. Related [#related] * [Function Calling](/llm-inference/function-calling) * [RAG](/llm-inference/rag) * [RAG API Reference](/api-reference/flutter/RagPipeline) * [Vision-Language Inference](/llm-inference/vision-language) * [Enums and constants](/api-reference/flutter/enums-and-constants) # ZeticMLangeModel (/api-reference/flutter/ZeticMLangeModel) This page reflects `zetic_mlange 1.10.0`. `ZeticMLangeModel` loads a general on-device model from the Melange registry and runs tensor inference from Dart. Import [#import] ```dart import 'package:zetic_mlange/zetic_mlange.dart'; ``` `create` [#create] ```dart static Future create({ required String personalKey, required String name, int? version, ModelMode modelMode = ModelMode.runAuto, MlangeProgressCallback? onProgress, ZeticMLangeCacheHandlingPolicy cacheHandlingPolicy = CacheHandlingPolicy.removeOverlapping, }) ``` | Parameter | Type | Default | Description | | --------------------- | -------------------------------- | ------------------- | ------------------------------------------------- | | `personalKey` | `String` | - | Personal key for accessing the model. | | `name` | `String` | - | Model name in `account_name/project_name` format. | | `version` | `int?` | `null` | Model version. `null` loads the latest version. | | `modelMode` | `ModelMode` | `runAuto` | Backend selection strategy. | | `onProgress` | `MlangeProgressCallback?` | `null` | Download progress callback from `0.0` to `1.0`. | | `cacheHandlingPolicy` | `ZeticMLangeCacheHandlingPolicy` | `removeOverlapping` | Managed artifact cache cleanup policy. | ```dart final model = await ZeticMLangeModel.create( personalKey: personalKey, name: 'account_name/project_name', ); ``` `run` [#run] Runs inference with tensors matching the model input order. ```dart List run([List inputs = const []]) ``` | Parameter | Type | Description | | --------- | -------------- | ------------------------------------------------------------------ | | `inputs` | `List` | Input tensors matching the model's expected shapes and data types. | **Returns:** output tensors in model output order. ```dart final outputs = model.run([inputTensor]); final firstOutput = outputs.first; ``` Lifecycle [#lifecycle] ```dart bool get isClosed void close() ``` Call `close()` when the model is no longer needed. ```dart model.close(); ``` Related [#related] * [Tensor](/api-reference/flutter/Tensor) * [Enums and constants](/api-reference/flutter/enums-and-constants) * [Cache Management](/api-reference/cache-management) # Enums and Constants (/api-reference/flutter/enums-and-constants) This page reflects `zetic_mlange 1.10.0`. General Model Types [#general-model-types] | Type | Values | | --------------------- | --------------------------------------------------------------- | | `ModelMode` | `runAuto`, `runFp32`, `runQuantized`, `runSpeed`, `runAccuracy` | | `APType` | `cpu`, `gpu`, `npu`, `na` | | `CacheHandlingPolicy` | `removeOverlapping`, `keepExisting` | `Target` contains backend identifiers used by selected general-model artifacts. Most Flutter apps use `ModelMode`. LLM Types [#llm-types] | Type | Values | | ------------------------- | -------------------------------------------------------------------------------------- | | `LLMModelMode` | `runAuto`, `runSpeed`, `runAccuracy` | | `LLMTarget` | `llamaCpp`, `litertLm`, `mllm` | | `LLMQuantType` | GGUF quantization variants such as `ggufQuantF16`, `ggufQuantQ4KM`, and `ggufQuantQ80` | | `LLMKVCacheCleanupPolicy` | `cleanUpOnFull`, `doNotCleanUp` | `LLMInitOption` [#llminitoption] ```dart final class LLMInitOption { const LLMInitOption({ this.kvCacheCleanupPolicy = LLMKVCacheCleanupPolicy.cleanUpOnFull, this.nCtx = 2048, }); } ``` | Field | Default | Description | | ---------------------- | --------------- | --------------------------------------------------------- | | `kvCacheCleanupPolicy` | `cleanUpOnFull` | In-memory KV-cache behavior when context storage is full. | | `nCtx` | `2048` | Requested context size. The runtime may normalize it. | Result and Error Types [#result-and-error-types] | Type | Purpose | | ------------------------ | ------------------------------------------------------ | | `LLMRunResult` | Result from starting LLM generation. | | `LLMNextTokenResult` | Token result returned by `waitForNextToken()`. | | `MlangeException` | Dart exception for validation and native SDK failures. | | `MlangeProgressCallback` | Download progress callback from `0.0` to `1.0`. | Related [#related] * [ZeticMLangeModel](/api-reference/flutter/ZeticMLangeModel) * [ZeticMLangeLLMModel](/api-reference/flutter/ZeticMLangeLLMModel) * [Tensor](/api-reference/flutter/Tensor) # Tensor (/api-reference/ios/Tensor) This page reflects `ZeticMLange iOS 1.10.0`. The `Tensor` class is the unified data container passed to and returned from `ZeticMLangeModel.run(inputs:)`. It wraps raw `Data`, its `DataType`, and its shape. Import [#import] ```swift import ZeticMLange ``` *** Initializer [#initializer] `Tensor(data:dataType:shape:)` [#tensordatadatatypeshape] Creates a tensor from raw bytes, an element type, and a shape. ```swift public init(data: Data, dataType: any DataType, shape: [Int]) ``` | Parameter | Type | Description | | ---------- | -------------- | -------------------------------------------------------------------------------------------- | | `data` | `Data` | Raw tensor bytes. Length must equal the product of `shape` dimensions times `dataType.size`. | | `dataType` | `any DataType` | Element type. Use a case from [`BuiltinDataType`](#builtindatatype) such as `.float32`. | | `shape` | `[Int]` | Tensor shape. | ```swift let bytes = Data(count: 1 * 3 * 640 * 640 * MemoryLayout.size) let tensor = Tensor( data: bytes, dataType: BuiltinDataType.float32, shape: [1, 3, 640, 640] ) ``` *** `Tensor(data:)` [#tensordata] Creates a tensor from raw bytes with defaults of `BuiltinDataType.int8` and `shape = [data.count]`. ```swift public convenience init(data: Data) ``` | Parameter | Type | Description | | --------- | ------ | -------------------------------------------------------------------------------- | | `data` | `Data` | Raw bytes. Interpreted as a 1-D `int8` tensor whose length matches `data.count`. | ```swift let tensor = Tensor(data: rawBytes) ``` *** Methods [#methods] `count()` [#count] Returns the number of elements in the tensor (total bytes divided by `dataType.size`). ```swift public func count() -> Int ``` `size()` [#size] Returns the size of the tensor in bytes (equivalent to `data.count`). ```swift public func size() -> Int ``` *** DataType [#datatype] `DataType` is a protocol that exposes the byte size of a tensor element. ```swift public protocol DataType { var size: Int { get } } ``` `BuiltinDataType` [#builtindatatype] The standard element types supported by Melange. Each case conforms to `DataType` and reports its byte size through `size`. ```swift public enum BuiltinDataType: String, DataType, CaseIterable { case float32 case float64 case float16 case bfloat16 case uint8 case uint16 case uint32 case uint64 case int8 case int16 case int32 case int64 case boolean case qint8 case qint16 case qint32 case qint4 } ``` | Case | Element size (bytes) | | ------------------------------ | -------------------- | | `.float32` | 4 | | `.float64` | 8 | | `.float16`, `.bfloat16` | 2 | | `.uint8`, `.int8`, `.qint8` | 1 | | `.uint16`, `.int16`, `.qint16` | 2 | | `.uint32`, `.int32`, `.qint32` | 4 | | `.uint64`, `.int64` | 8 | | `.boolean` | 1 | | `.qint4` | 1 | `Unknown` [#unknown] Fallback `DataType` returned when a type name is not recognized. Caller supplies the element size. ```swift public struct Unknown: DataType { public let size: Int } ``` `dataType(from:)` [#datatypefrom] Resolves a `DataType` from its lowercase string name (for example `"float32"` or `"int8"`). Unknown names return `Unknown(size: 0)`. ```swift public func dataType(from string: String) -> DataType ``` ```swift let type = dataType(from: "float32") // BuiltinDataType.float32 ``` *** Properties [#properties] `data` [#data] The underlying raw bytes of the tensor. ```swift public let data: Data ``` `dataType` [#datatype-1] The element type of the tensor. ```swift public let dataType: any DataType ``` `shape` [#shape] The shape of the tensor. ```swift public let shape: [Int] ``` *** Equatable [#equatable] Two tensors are equal if their `data` and `shape` match. ```swift public static func == (lhs: Tensor, rhs: Tensor) -> Bool ``` *** Full Working Example [#full-working-example] ```swift import ZeticMLange class ViewController: UIViewController { override func viewDidLoad() { super.viewDidLoad() do { let model = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: "Steve/YOLOv11_comparison" ) // (1) Build an input tensor from preprocessed pixel bytes let pixels: [Float] = preparePixels() let data = pixels.withUnsafeBufferPointer { Data(buffer: $0) } let input = Tensor( data: data, dataType: BuiltinDataType.float32, shape: [1, 3, 640, 640] ) // (2) Run inference let outputs = try model.run(inputs: [input]) // (3) Read outputs as a typed array of Floats let output = outputs[0] let floatCount = output.count() let floats: [Float] = output.data.withUnsafeBytes { raw in Array(raw.bindMemory(to: Float.self).prefix(floatCount)) } } catch { print("Melange error: \(error)") } } } ``` The `data` length must equal the product of `shape` times `dataType.size`. A mismatch will cause `run(inputs:)` to fail at the model boundary. See Also [#see-also] * [ZeticMLangeModel (iOS)](/api-reference/ios/ZeticMLangeModel): Uses `Tensor` as the input and output type of `run(inputs:)` * [Tensor (Android)](/api-reference/android/Tensor): Android equivalent * [Enums and Constants](/api-reference/ios/enums-and-constants): Other iOS SDK enums # ZeticMLangeHFModel (/api-reference/ios/ZeticMLangeHFModel) This page reflects `ZeticMLange iOS 1.10.0`. `ZeticMLangeHFModel` loads compatible models directly from Hugging Face repositories. Import [#import] ```swift import ZeticMLange ``` Initializer [#initializer] ```swift public init( _ repoId: String, userAccessToken: String? = nil, index: Int = 0, cacheHandlingPolicy: ModelCacheHandlingPolicy = .REMOVE_OVERLAPPING ) async throws ``` | Parameter | Type | Default | Description | | --------------------- | -------------------------- | --------------------- | -------------------------------------------------------- | | `repoId` | `String` | - | Hugging Face repository ID. | | `userAccessToken` | `String?` | `nil` | Optional Hugging Face access token. | | `index` | `Int` | `0` | Model index when a repository contains multiple entries. | | `cacheHandlingPolicy` | `ModelCacheHandlingPolicy` | `.REMOVE_OVERLAPPING` | Managed artifact cache cleanup policy. | ```swift let model = try await ZeticMLangeHFModel("zetic-ai/yolov11n") ``` `run(inputs:)` [#runinputs] ```swift public func run(inputs: [Tensor] = []) throws -> [Tensor] ``` Runs inference and returns output tensors. Lifecycle [#lifecycle] ```swift public private(set) var isClosed: Bool public func close() ``` Call `close()` when the model is no longer needed. Related [#related] * [ZeticMLangeModel](/api-reference/ios/ZeticMLangeModel) * [Tensor](/api-reference/ios/Tensor) * [Hugging Face Models](/model-preparation/hugging-face-models) # ZeticMLangeLLMModel (/api-reference/ios/ZeticMLangeLLMModel) This page reflects `ZeticMLange iOS 1.10.0`. `ZeticMLangeLLMModel` loads an on-device LLM from the Melange registry and supports text generation, token streaming, function calling, image response for LFM-VL models, and KV state persistence. Import [#import] ```swift import ZeticMLange ``` Initializer [#initializer] ```swift public init( personalKey: String, name: String, version: Int? = nil, modelMode: LLMModelMode = .RUN_AUTO, cacheHandlingPolicy: ModelCacheHandlingPolicy = .REMOVE_OVERLAPPING, initOption: LLMInitOption = LLMInitOption(), onDownload: ((Float) -> Void)? = nil ) async throws ``` | Parameter | Type | Default | Description | | --------------------- | -------------------------- | --------------------- | ------------------------------------------------- | | `personalKey` | `String` | - | Personal key for accessing the model. | | `name` | `String` | - | Model name in `account_name/project_name` format. | | `version` | `Int?` | `nil` | Model version. `nil` loads the latest version. | | `modelMode` | `LLMModelMode` | `.RUN_AUTO` | Backend selection strategy. | | `cacheHandlingPolicy` | `ModelCacheHandlingPolicy` | `.REMOVE_OVERLAPPING` | Managed artifact cache cleanup policy. | | `initOption` | `LLMInitOption` | `LLMInitOption()` | LLM initialization options. | | `onDownload` | `((Float) -> Void)?` | `nil` | Download progress callback from `0.0` to `1.0`. | ```swift let model = try await ZeticMLangeLLMModel( personalKey: PERSONAL_KEY, name: "account_name/project_name", initOption: LLMInitOption(nCtx: 4096) ) ``` Text Generation [#text-generation] `run(_:)` [#run_] Starts generation for a prompt. ```swift public func run(_ text: String) throws ``` ```swift try model.run("Explain on-device AI in one paragraph.") ``` Reading Generated Tokens [#reading-generated-tokens] Waits for the next generated token. ```swift public func waitForNextToken() -> LLMNextTokenResult ``` ```swift while true { let next = model.waitForNextToken() if next.isFinished { break } append(next.token) } ``` Vision-Language Response [#vision-language-response] Use `respond(...)` with an LFM-VL-capable model. ```swift public func respond( systemPrompt: String = "", userText: String, image: ZeticMLangeLLMModel.Image ) throws -> AsyncThrowingStream ``` ```swift let image = try ZeticMLangeLLMModel.Image( rgb: rgbBytes, width: width, height: height ) for try await token in try model.respond( systemPrompt: "Answer briefly.", userText: "What is in this image?", image: image ) { append(token) } ``` Function Calling [#function-calling] ```swift public var functionCallingSystemPrompt: String? public func registerTool(_ spec: LLMToolSpec, executor: @escaping LLMToolExecutor) throws public func unregisterTool(name: String) throws -> Bool public func clearTools() throws public func registeredToolSpecs() throws -> [LLMToolSpec] ``` ```swift try model.registerTool( LLMToolSpec( name: "lookup", description: "Look up local app data.", parametersJson: #"{"type":"object","properties":{"query":{"type":"string"}}}"# ) ) { call in LLMToolResult(content: #"{"result":"Found"}"#) } for try await token in try model.runWithTools("Use lookup to answer the question.") { append(token) } ``` `runWithTools(_:)` returns an `AsyncThrowingStream`. Consume it to completion before starting another model operation. KV State Persistence [#kv-state-persistence] ```swift public func saveKVState(path: String) throws public func loadKVState(path: String) throws public func resetKVState() throws ``` Use these APIs to persist or reset the current LLM state for resume flows. Lifecycle [#lifecycle] ```swift public private(set) var isClosed: Bool public func cleanUp() throws public func close() public func forceDeinit() ``` Call `cleanUp()` before starting a fresh conversation. Call `close()` when the model is no longer needed. Related [#related] * [Function Calling](/llm-inference/function-calling) * [RAG](/llm-inference/rag) * [Vision-Language Inference](/llm-inference/vision-language) * [Enums and constants](/api-reference/ios/enums-and-constants) # ZeticMLangeModel (/api-reference/ios/ZeticMLangeModel) This page reflects `ZeticMLange iOS 1.10.0`. `ZeticMLangeModel` loads a general on-device model from the Melange registry and runs tensor inference on iOS. Import [#import] ```swift import ZeticMLange ``` Initializer [#initializer] ```swift public init( personalKey: String, name: String, version: Int? = nil, modelMode: ModelMode = .RUN_AUTO, cacheHandlingPolicy: ModelCacheHandlingPolicy = .REMOVE_OVERLAPPING, onDownload: ((Float) -> Void)? = nil ) async throws ``` | Parameter | Type | Default | Description | | --------------------- | -------------------------- | --------------------- | ------------------------------------------------- | | `personalKey` | `String` | - | Personal key for accessing the model. | | `name` | `String` | - | Model name in `account_name/project_name` format. | | `version` | `Int?` | `nil` | Model version. `nil` loads the latest version. | | `modelMode` | `ModelMode` | `.RUN_AUTO` | Backend selection strategy. | | `cacheHandlingPolicy` | `ModelCacheHandlingPolicy` | `.REMOVE_OVERLAPPING` | Managed artifact cache cleanup policy. | | `onDownload` | `((Float) -> Void)?` | `nil` | Download progress callback from `0.0` to `1.0`. | ```swift let model = try await ZeticMLangeModel( personalKey: PERSONAL_KEY, name: "account_name/project_name" ) ``` `run(inputs:)` [#runinputs] Runs inference with tensors matching the model input order. ```swift public func run(inputs: [Tensor]) throws -> [Tensor] ``` | Parameter | Type | Description | | --------- | ---------- | ------------------------------------------------------------------ | | `inputs` | `[Tensor]` | Input tensors matching the model's expected shapes and data types. | **Returns:** output tensors in model output order. ```swift let outputs = try model.run(inputs: [inputTensor]) let firstOutput = outputs[0] ``` Lifecycle [#lifecycle] ```swift public private(set) var isClosed: Bool public func close() ``` Call `close()` when the model is no longer needed. ```swift model.close() ``` Related [#related] * [Tensor](/api-reference/ios/Tensor) * [Enums and constants](/api-reference/ios/enums-and-constants) * [Cache Management](/api-reference/cache-management) # Enums and Constants (/api-reference/ios/enums-and-constants) This page reflects `ZeticMLange iOS 1.10.0`. General Model Types [#general-model-types] | Type | Values | | -------------------------- | ------------------------------------------------------------------------- | | `ModelMode` | `.RUN_AUTO`, `.RUN_FP32`, `.RUN_QUANTIZED`, `.RUN_SPEED`, `.RUN_ACCURACY` | | `APType` | `.CPU`, `.GPU`, `.NPU`, `.NA` | | `ModelCacheHandlingPolicy` | `.REMOVE_OVERLAPPING`, `.KEEP_EXISTING` | `Target` contains backend identifiers used by selected general-model artifacts. Most apps use `ModelMode` and let backend selection choose the target. LLM Types [#llm-types] | Type | Values | | ------------------------- | -------------------------------------------------------------------------------------------------- | | `LLMModelMode` | `.RUN_AUTO`, `.RUN_SPEED`, `.RUN_ACCURACY` | | `LLMTarget` | `.LLAMA_CPP`, `.MLLM` | | `LLMQuantType` | GGUF quantization variants such as `.GGUF_QUANT_F16`, `.GGUF_QUANT_Q4_K_M`, and `.GGUF_QUANT_Q8_0` | | `LLMKVCacheCleanupPolicy` | `.CLEAN_UP_ON_FULL`, `.DO_NOT_CLEAN_UP` | | `KVStateStatus` | Status values returned by KV state persistence operations. | | `KVStateError` | Error thrown when a KV state operation fails. | `LLMInitOption` [#llminitoption] ```swift public struct LLMInitOption { public let kvCacheCleanupPolicy: LLMKVCacheCleanupPolicy public let nCtx: Int public let runConfigId: Int64 } ``` | Field | Default | Description | | ---------------------- | ------------------- | --------------------------------------------------------- | | `kvCacheCleanupPolicy` | `.CLEAN_UP_ON_FULL` | In-memory KV-cache behavior when context storage is full. | | `nCtx` | `2048` | Requested context size. The runtime may normalize it. | | `runConfigId` | `0` | Run configuration id used by compatible LFM-VL packages. | Result Types [#result-types] | Type | Purpose | | -------------------- | ---------------------------------------------- | | `LLMNextTokenResult` | Token result returned by `waitForNextToken()`. | Related [#related] * [ZeticMLangeModel](/api-reference/ios/ZeticMLangeModel) * [ZeticMLangeLLMModel](/api-reference/ios/ZeticMLangeLLMModel) * [Tensor](/api-reference/ios/Tensor) # Advanced Configuration (/platform-integration/android/advanced-configuration) This guide covers advanced configuration options available in the Melange Android SDK. Inference Mode Selection [#inference-mode-selection] Melange supports multiple inference modes to balance speed and accuracy. By default, the SDK uses `ModelMode.RUN_AUTO`, which selects the fastest configuration while maintaining high-quality results (SNR > 20dB). ```kotlin // Default (Auto): balanced speed and accuracy val model = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = ModelMode.RUN_AUTO ) // Speed-first: minimum latency val modelFast = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = ModelMode.RUN_SPEED ) // Accuracy-first: maximum precision val modelAccurate = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = ModelMode.RUN_ACCURACY ) ``` For a detailed explanation of each mode, see [Inference Mode Selection](/how-to-guides/inference-mode-selection). Model Version Pinning [#model-version-pinning] By default, the SDK loads the latest model version. You can pin to a specific version for production stability: ```kotlin val model = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, version = 2 // Pin to a specific version ) ``` Multi-Model Pipelines [#multi-model-pipelines] For applications that chain multiple models (e.g., detection followed by classification), initialize each model separately and pass outputs as inputs: ```kotlin // Initialize pipeline models val detectionModel = ZeticMLangeModel(this, PERSONAL_KEY, "detection_model") val classificationModel = ZeticMLangeModel(this, PERSONAL_KEY, "classification_model") // Run pipeline val detectionOutputs = detectionModel.run(inputs) // Process detection outputs and prepare classification inputs val classificationOutputs = classificationModel.run(classificationInputs) ``` For a complete pipeline example, see [Multi-Model Pipelines](/how-to-guides/multi-model-pipeline). Threading Considerations [#threading-considerations] Model initialization performs a network call on first use. Always initialize models on a background thread to avoid blocking the UI. ```kotlin lifecycleScope.launch(Dispatchers.IO) { val model = ZeticMLangeModel(this@MainActivity, PERSONAL_KEY, MODEL_NAME) val outputs = model.run(inputs) withContext(Dispatchers.Main) { // Update UI with results } } ``` *** Next Steps [#next-steps] * [Inference Mode Selection](/how-to-guides/inference-mode-selection): Detailed mode comparison * [Performance Optimization](/how-to-guides/performance-optimization): Tips for best performance * [ZeticMLangeModel API Reference](/api-reference/android/ZeticMLangeModel): Full API documentation # Basic Inference (/platform-integration/android/basic-inference) This guide shows how to run inference on Android after completing the [SDK setup](/platform-integration/android/setup). Prerequisites [#prerequisites] * Melange SDK added to your project ([Android Setup](/platform-integration/android/setup)) * A compiled model on the [Melange Dashboard](https://melange.zetic.ai) * Your **Personal Key** and **Model Key** Running Inference [#running-inference] ```kotlin // (1) Load model // This handles model download (if needed) and NPU context creation val model = ZeticMLangeModel(CONTEXT, PERSONAL_KEY, MODEL_NAME) // (2) Prepare model inputs // Ensure input shapes match your model's requirement (e.g., Float32 arrays) val inputs: Array = // Prepare your inputs // (3) Run Inference // Executes the fully automated hardware graph. // No manual delegate configuration or memory syncing required. val outputs = model.run(inputs) ``` ```java // (1) Load model // This handles model download (if needed) and NPU context creation ZeticMLangeModel model = new ZeticMLangeModel(CONTEXT, PERSONAL_KEY, MODEL_NAME); // (2) Prepare model inputs // Ensure input shapes match your model's requirement (e.g., Float32 arrays) Tensor[] inputs = // Prepare your inputs; // (3) Run Inference // Executes the hardware-accelerated graph. This is a blocking call. Tensor[] outputs = model.run(inputs); ``` Understanding the Flow [#understanding-the-flow] 1. **Model Download**: On first use, the SDK downloads the pre-compiled, hardware-optimized model binary from the Melange CDN. This binary is specific to your device's NPU chipset. 2. **NPU Context Creation**: Melange initializes the appropriate hardware accelerator (Qualcomm HTP, MediaTek APU, Samsung DSP) and loads the model into NPU memory using zero-copy memory mapping. 3. **Inference Execution**: Your input tensor is processed through the NPU-accelerated computation graph, and the output tensor is returned. No data leaves the device. Always ensure your input tensor shapes exactly match what the model expects. A shape mismatch will throw a `RuntimeException`. Check the model's input specification on the Melange Dashboard. Sample Application [#sample-application] Please refer to the [ZETIC Melange Apps](https://github.com/zetic-ai/ZETIC_Melange_apps) repository for complete sample applications and more details. *** Next Steps [#next-steps] * [Advanced Configuration](/platform-integration/android/advanced-configuration): Inference modes and pipeline usage * [Custom Preprocessing](/how-to-guides/custom-preprocessing): Implement input preprocessing * [Multi-Model Pipelines](/how-to-guides/multi-model-pipeline): Chain models together # Setup (/platform-integration/android/setup) This guide targets **`ZeticMLange Android 1.10.0`** โ€” the recommended version and the one all API reference pages are written against. This guide walks you through adding the ZETIC Melange SDK to your Android project. Melange abstracts the complexity of NPU execution: you do not need to write any C++ or OpenCL code. Prerequisites [#prerequisites] * **Android Studio** Arctic Fox or later * A physical Android device (emulators do not have NPU hardware) * Minimum SDK 24 (Android 7.0) * A **Personal Key** from the [Melange Dashboard](https://melange.zetic.ai) Your project structure should look like this: Add Melange Dependency [#add-melange-dependency] Integrate the Melange AAR (Android Archive) which contains the Unified HAL for Android devices. **build.gradle.kts** ```kotlin android { ... packaging { jniLibs { useLegacyPackaging = true } } } dependencies { implementation("com.zeticai.mlange:mlange:1.10.0") } ``` **build.gradle** ```groovy android { ... packagingOptions { jniLibs { useLegacyPackaging true } } } dependencies { implementation 'com.zeticai.mlange:mlange:1.10.0' } ``` The `useLegacyPackaging` setting is **required**. It ensures the native C++ NPU drivers (JNI) are correctly bundled without compression. Without it, you will get a `java.lang.UnsatisfiedLinkError` at runtime. Sync and Build [#sync-and-build] 1. Click **Sync Now** in the Gradle notification bar. 2. Build your project to verify the dependency resolves correctly. Verify Setup [#verify-setup] Add a simple initialization test to confirm the SDK is working: ```kotlin import com.zeticai.mlange.core.model.ZeticMLangeModel // Test initialization (use in onCreate or a background thread) val model = ZeticMLangeModel(this, PERSONAL_KEY, MODEL_NAME) ``` If the initialization completes without error, your setup is ready. The constructor performs a network call on first use to download the model binary. Call it from a background thread or use Kotlin coroutines to avoid blocking the UI. *** Next Steps [#next-steps] Run your first inference on Android. Inference modes, pipelines, and optimization. Full ZeticMLangeModel API documentation. Flutter SDK integration. React Native SDK integration (preview). # Advanced Configuration (/platform-integration/ios/advanced-configuration) This guide covers advanced configuration options available in the Melange iOS SDK. Inference Mode Selection [#inference-mode-selection] Melange supports multiple inference modes to balance speed and accuracy. By default, the SDK uses `RUN_AUTO`, which selects the fastest configuration while maintaining high-quality results (SNR > 20dB). ```swift // Default (Auto): balanced speed and accuracy let modelDefault = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_AUTO ) // Speed-first: minimum latency let modelFast = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_SPEED ) // Accuracy-first: maximum precision let modelAccurate = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_ACCURACY ) ``` For a detailed explanation of each mode, see [Inference Mode Selection](/how-to-guides/inference-mode-selection). Model Version Pinning [#model-version-pinning] By default, the SDK loads the latest model version. You can pin to a specific version for production stability: ```swift let model = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, version: 2 // Pin to a specific version ) ``` Multi-Model Pipelines [#multi-model-pipelines] For applications that chain multiple models, initialize each model separately and pass outputs as inputs: ```swift // Initialize pipeline models let detectionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "detection_model") let classificationModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "classification_model") // Run pipeline let detectionOutputs = try detectionModel.run(inputs: inputs) // Process detection outputs and prepare classification inputs let classificationOutputs = try classificationModel.run(inputs: classificationInputs) ``` For a complete pipeline example, see [Multi-Model Pipelines](/how-to-guides/multi-model-pipeline). Error Handling [#error-handling] Wrap model operations in do-catch blocks to handle initialization and inference errors gracefully: ```swift do { let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) let outputs = try model.run(inputs: inputs) } catch { print("Melange error: \(error)") // Handle error: network failure, invalid key, shape mismatch, etc. } ``` *** Next Steps [#next-steps] * [Inference Mode Selection](/how-to-guides/inference-mode-selection): Detailed mode comparison * [Performance Optimization](/how-to-guides/performance-optimization): Tips for best performance * [ZeticMLangeModel API Reference](/api-reference/ios/ZeticMLangeModel): Full API documentation # Basic Inference (/platform-integration/ios/basic-inference) This guide shows how to run inference on iOS after completing the [SDK setup](/platform-integration/ios/setup). Prerequisites [#prerequisites] * Melange SDK added to your project ([iOS Setup](/platform-integration/ios/setup)) * A compiled model on the [Melange Dashboard](https://melange.zetic.ai) * Your **Personal Key** and **Model Key** Running Inference [#running-inference] ```swift import ZeticMLange // (1) Load model // This handles model download (if needed) and Neural Engine context creation let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) // (2) Prepare model inputs // Ensure input shapes match your model's requirement (e.g., Float32 arrays) let inputs: [Tensor] = [] // Prepare your inputs // (3) Run Inference // Executes the fully automated hardware graph. // No manual delegate configuration or memory syncing required. let outputs = try model.run(inputs: inputs) ``` Understanding the Flow [#understanding-the-flow] 1. **Model Download**: On first use, the SDK downloads the pre-compiled, hardware-optimized model binary from the Melange CDN. This binary is optimized for Apple Neural Engine. 2. **Neural Engine Context Creation**: Melange initializes the Neural Engine and loads the model into NPU memory using zero-copy memory mapping. 3. **Inference Execution**: Your input tensor is processed through the NPU-accelerated computation graph, and the output tensor is returned. No data leaves the device. Always ensure your input tensor shapes exactly match what the model expects. A shape mismatch will throw an error. Check the model's input specification on the Melange Dashboard. Full Working Example [#full-working-example] ```swift import ZeticMLange class ViewController: UIViewController { override func viewDidLoad() { super.viewDidLoad() do { // Load model let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "Steve/YOLOv11_comparison") // Prepare inputs let inputs: [Tensor] = [] // Prepare your inputs // Run inference let outputs = try model.run(inputs: inputs) // Process outputs for output in outputs { // Process each output tensor } } catch { print("Melange error: \(error)") } } } ``` Sample Application [#sample-application] Please refer to the [ZETIC Melange Apps](https://github.com/zetic-ai/ZETIC_Melange_apps) repository for complete sample applications and more details. *** Next Steps [#next-steps] * [Advanced Configuration](/platform-integration/ios/advanced-configuration): Inference modes and options * [Custom Preprocessing](/how-to-guides/custom-preprocessing): Implement input preprocessing * [Multi-Model Pipelines](/how-to-guides/multi-model-pipeline): Chain models together # Setup (/platform-integration/ios/setup) This guide targets **`ZeticMLange iOS 1.10.0`** โ€” the recommended version and the one all API reference pages are written against. This guide walks you through adding the ZETIC Melange SDK to your iOS project. Melange provides a unified Swift interface that handles compilation, optimization, and execution on the Apple Neural Engine automatically. Prerequisites [#prerequisites] * **Xcode** 14 or later * A physical iOS device (iPhone 8 or later recommended) * iOS 16.6+ * A **Personal Key** from the [Melange Dashboard](https://melange.zetic.ai) Simulators do not have Neural Engine hardware. Always test on a physical device for accurate performance results. Add Melange Package [#add-melange-package] We use Swift Package Manager (SPM) to automatically resolve and link the binary dependencies required for NPU acceleration. 1. Open your project in Xcode. 2. Go to **File** then **Add Package Dependencies**. 3. Enter the package URL: `https://github.com/zetic-ai/ZeticMLangeiOS` 4. Set the dependency rule to **Exact Version** `1.10.0` (or **Up to Next Major Version** from `1.10.0`). 5. Click **Add Package**. Link Accelerate.framework [#link-accelerateframework] `ZeticMLange` depends on Apple's **Accelerate** framework (for `vDSP` and BLAS kernels). SPM does not link this system framework automatically, so you need to add it manually: 1. Select your app target in Xcode. 2. Open **General** โ†’ **Frameworks, Libraries, and Embedded Content** (or equivalently, **Build Phases** โ†’ **Link Binary With Libraries**). 3. Click **+**, search for `Accelerate.framework`, and add it. Skipping this step causes linker errors such as `Undefined symbol: _vDSP_vmul` or `_cblas_sgemm$NEWLAPACK$ILP64` at build time. See [iOS Issues โ†’ Undefined Accelerate Symbols](/troubleshooting/ios-issues#undefined-accelerate-symbols) if you hit these errors. This manual step is a temporary workaround. A future SDK release will auto-link Accelerate through SPM so this step can be removed. Select Target [#select-target] Link the `ZeticMLange` library to your specific application target: 1. Select your target in the **Add to Target** column. 2. Click **Add Package**. Verify Setup [#verify-setup] Add a simple initialization test to confirm the SDK is working: ```swift import ZeticMLange // Test initialization let model = try await ZeticMLangeModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) ``` If the initialization completes without error, your setup is ready. The initializer performs a network call on first use to download the model binary. The binary is cached locally after the first download, so subsequent initializations are fast. *** Next Steps [#next-steps] Run your first inference on iOS. Inference modes, options, and optimization. Full ZeticMLangeModel API documentation. # Advanced Configuration (/platform-integration/flutter/advanced-configuration) This guide covers advanced configuration options available in the Melange Flutter SDK. Inference Mode Selection [#inference-mode-selection] Melange supports multiple inference modes to balance speed and accuracy. By default, the Flutter SDK uses `ModelMode.runAuto`. ```dart // Default: automatic runtime selection final modelDefault = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, modelMode: ModelMode.runAuto, ); // Speed-first final modelFast = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, modelMode: ModelMode.runSpeed, ); // Accuracy-first final modelAccurate = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, modelMode: ModelMode.runAccuracy, ); ``` For a detailed explanation of each mode, see [Inference Mode Selection](/how-to-guides/inference-mode-selection). Model Version Pinning [#model-version-pinning] By default, the SDK loads the latest model version. Pin a specific version for production stability: ```dart final model = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, version: 2, ); ``` Hugging Face Models [#hugging-face-models] Use `ZeticMLangeHFModel` for supported Hugging Face repositories: ```dart final hfModel = await ZeticMLangeHFModel.create( 'owner/repository', userAccessToken: hfToken, manifestDir: 'optional-manifest-directory', index: 0, ); final outputs = hfModel.run(inputs); hfModel.close(); ``` `manifestDir` is Android-only. iOS accepts the Dart parameter through the Flutter API and ignores it. LLM Models [#llm-models] Use `ZeticMLangeLLMModel` for text generation: ```dart final llm = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: llmModelName, initOption: const LLMInitOption( nCtx: 4096, kvCacheCleanupPolicy: LLMKVCacheCleanupPolicy.cleanUpOnFull, ), onDownload: (progress) { print('Loading LLM ${(progress * 100).round()}%'); }, ); llm.run('Explain on-device AI in one paragraph.'); while (true) { final next = llm.waitForNextToken(); if (next.isFinished) { break; } print(next.token); } llm.cleanUp(); await llm.close(); ``` `waitForNextToken()` blocks until the native runtime returns the next token. Run generation from a worker isolate or keep the UI responsive by scheduling updates carefully when building an interactive chat screen. Threading Considerations [#threading-considerations] Flutter model creation is asynchronous. Keep model initialization out of frame-critical paths and update UI state after the `Future` completes: ```dart Future loadModel() async { setState(() => isLoading = true); try { model = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, onProgress: (progress) { setState(() => loadingProgress = progress); }, ); } finally { setState(() => isLoading = false); } } ``` *** Next Steps [#next-steps] * [Inference Mode Selection](/how-to-guides/inference-mode-selection): Detailed mode comparison * [Performance Optimization](/how-to-guides/performance-optimization): Tips for best performance * [Flutter API Reference](/api-reference/flutter/ZeticMLangeModel): Full Dart API documentation # Basic Inference (/platform-integration/flutter/basic-inference) This guide shows how to run inference from Dart after completing the [Flutter setup](/platform-integration/flutter/setup). Prerequisites [#prerequisites] * `zetic_mlange` added to your Flutter project * Platform setup completed for Android or iOS * A compiled model on the [Melange Dashboard](https://melange.zetic.ai) * Your **Personal Key** and model name in `account_name/project_name` format Running Inference [#running-inference] ```dart import 'dart:typed_data'; import 'package:zetic_mlange/zetic_mlange.dart'; // (1) Load model // This handles model download, cache reuse, and native runtime initialization. final model = await ZeticMLangeModel.create( personalKey: personalKey, name: 'account_name/project_name', onProgress: (progress) { print('Loading ${(progress * 100).round()}%'); }, ); // (2) Prepare model inputs // Ensure shape and dtype match your model's input specification. final input = Tensor.float32View( Float32List.fromList([/* preprocessed values */]), shape: const [1, 3, 224, 224], ); // (3) Run inference final outputs = model.run([input]); // (4) Read output data final scores = outputs.first.asFloat32List(); model.close(); ``` Understanding the Flow [#understanding-the-flow] 1. **Model Download**: On first use, the native SDK downloads the hardware-optimized model artifact for the current platform. 2. **Native Runtime Creation**: The Flutter SDK calls the Android or iOS SDK through FFI and receives a native model handle. 3. **Tensor Copy and Execution**: Dart `Tensor` bytes are copied into native input buffers, the model runs on the selected backend, and output tensors are returned to Dart as typed views. Input tensor shapes and data types must match the model's input specification. A mismatch throws `MlangeException` or a native SDK error. Tensor Inputs [#tensor-inputs] Use the typed constructors that match your preprocessing output: ```dart final floatInput = Tensor.float32View( floatValues, shape: const [1, 3, 224, 224], ); final intInput = Tensor.int64View( tokenIds, shape: const [1, 128], ); ``` Use `View` constructors when your typed list is already in the correct layout and can be shared without an extra Dart-side copy. Use `List` constructors when you want defensive copy semantics. Model Lifetime [#model-lifetime] Keep the model instance alive while you reuse it, then release the native handle when the screen or service is done: ```dart if (!model.isClosed) { model.close(); } ``` Sample Applications [#sample-applications] Please refer to the [ZETIC Melange Apps](https://github.com/zetic-ai/ZETIC_Melange_apps) repository for complete Flutter sample applications. *** Next Steps [#next-steps] * [Advanced Configuration](/platform-integration/flutter/advanced-configuration): Model modes, version pinning, Hugging Face models, and LLMs * [Custom Preprocessing](/how-to-guides/custom-preprocessing): Implement input preprocessing * [ZeticMLangeModel API Reference](/api-reference/flutter/ZeticMLangeModel): Full Dart API documentation # Setup (/platform-integration/flutter/setup) This guide targets **`zetic_mlange 1.10.0`** โ€” the Flutter FFI SDK for Android and iOS. This guide walks you through adding the ZETIC Melange Flutter SDK to your app. The Flutter package exposes a Dart API and delegates model loading, cache management, and execution to the Android and iOS SDKs through FFI. Prerequisites [#prerequisites] * **Flutter** 3.35.0 or later * **Dart** 3.11.5 or later * A physical Android or iOS device * A **Personal Key** from the [Melange Dashboard](https://melange.zetic.ai) * A compiled model on the Melange Dashboard, identified by `account_name/project_name` Emulators and simulators do not provide the same NPU hardware as physical devices. Use a real device for inference and performance testing. Add Melange Dependency [#add-melange-dependency] Add the Flutter package to your app: ```yaml dependencies: zetic_mlange: ^1.10.0 ``` Then fetch packages: ```bash flutter pub get ``` Configure Android [#configure-android] Set Android `minSdk` to 24 or later, build for `arm64-v8a`, and enable legacy JNI packaging in your app module. **android/app/build.gradle.kts** ```kotlin android { defaultConfig { minSdk = maxOf(flutter.minSdkVersion, 24) ndk { abiFilters += "arm64-v8a" } } packaging { jniLibs { useLegacyPackaging = true } } } ``` `useLegacyPackaging` is required so native model runtime libraries are packaged in a form Android can load at runtime. Without it, Android may fail with `UnsatisfiedLinkError` when the FFI layer loads native runtime libraries. Configure iOS [#configure-ios] Set the iOS deployment target to 16.6 or later. The Flutter pod downloads and links the `ZeticMLange.xcframework` release and Apple's `Accelerate` framework through CocoaPods. **ios/Podfile** ```ruby platform :ios, '16.6' target 'Runner' do use_frameworks! flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__)) end ``` Then install pods: ```bash cd ios pod install ``` The Flutter SDK uses the published iOS framework release. You do not need to copy a local `ZeticMLange.xcframework` into the Flutter package before running `pod install`. Verify Setup [#verify-setup] Import the package and create a model: ```dart import 'package:zetic_mlange/zetic_mlange.dart'; final model = await ZeticMLangeModel.create( personalKey: personalKey, name: 'account_name/project_name', onProgress: (progress) { print('Loading ${(progress * 100).round()}%'); }, ); ``` If the model initializes without an exception, your Flutter setup is ready. Model creation downloads the optimized model artifact on first use. Run it outside latency-sensitive UI paths and show progress with `onProgress` when needed. *** Next Steps [#next-steps] Run your first inference from Dart. Configure model modes, versions, Hugging Face models, and LLMs. Full Flutter API documentation. # React Native (/platform-integration/react-native) React Native support for ZETIC Melange is currently in development. This page will be updated with full integration instructions when the React Native SDK is released. Current Status [#current-status] The React Native bridge is under active development. A preview template is available for LLM use cases. LLM React Native Template [#llm-react-native-template] For LLM inference on React Native, a preview template is available: * [React Native LLM Template](https://github.com/zetic-ai/zetic-llm-react-native-template): Build a chat app with on-device LLM inference The React Native LLM template provides a working starting point for on-device LLM applications. Check the repository README for setup instructions. Roadmap [#roadmap] * React Native bridge for general model inference * JavaScript/TypeScript API matching the Kotlin/Swift SDK interface * Example apps for common use cases (image classification, object detection) Stay Updated [#stay-updated] * Join the [Discord community](https://discord.gg/q6vW4UscRY) for updates * Watch the [GitHub repository](https://github.com/zetic-ai) for releases * Contact [contact@zetic.ai](mailto:contact@zetic.ai) for early access *** Related [#related] * [Android Setup](/platform-integration/android/setup): Native Android integration * [iOS Setup](/platform-integration/ios/setup): Native iOS integration * [LLM Inference](/llm-inference/overview): On-device LLM capabilities