Basic Inference

This guide shows how to run inference on Android after completing the SDK setup.

Prerequisites

• Melange SDK added to your project (Android Setup)
• A compiled model on the Melange Dashboard
• Your Personal Key and Model Key

Running Inference

// (1) Load model
// This handles model download (if needed) and NPU context creation
val model = ZeticMLangeModel(CONTEXT, PERSONAL_KEY, MODEL_NAME)

// (2) Prepare model inputs
// Ensure input shapes match your model's requirement (e.g., Float32 arrays)
val inputs: Array<Tensor> = // Prepare your inputs

// (3) Run Inference
// Executes the fully automated hardware graph.
// No manual delegate configuration or memory syncing required.
val outputs = model.run(inputs)

// (1) Load model
// This handles model download (if needed) and NPU context creation
ZeticMLangeModel model = new ZeticMLangeModel(CONTEXT, PERSONAL_KEY, MODEL_NAME);

// (2) Prepare model inputs
// Ensure input shapes match your model's requirement (e.g., Float32 arrays)
Tensor[] inputs = // Prepare your inputs;

// (3) Run Inference
// Executes the hardware-accelerated graph. This is a blocking call.
Tensor[] outputs = model.run(inputs);

Understanding the Flow

1. Model Download: On first use, the SDK downloads the pre-compiled, hardware-optimized model binary from the Melange CDN. This binary is specific to your device's NPU chipset.
2. NPU Context Creation: Melange initializes the appropriate hardware accelerator (Qualcomm HTP, MediaTek APU, Samsung DSP) and loads the model into NPU memory using zero-copy memory mapping.
3. Inference Execution: Your input tensor is processed through the NPU-accelerated computation graph, and the output tensor is returned. No data leaves the device.

Zero-Copy Input Path

For hot loops (e.g. per-frame camera inference), skip the per-call byte copy by filling the model's own input buffers in place:

val inputBuffers = model.getInputBuffers()

for (i in inputBuffers.indices) {
    inputBuffers[i].from(sourceTensors[i])
}

val outputs = model.run()

The tensors returned from run() already wrap model-owned output buffers — read them directly for zero-copy post-processing, but consume them before the next run() call since the buffers are reused. See getInputBuffers() for details.

Sample Application

Next Steps

• Advanced Configuration: Inference modes and pipeline usage
• Custom Preprocessing: Implement input preprocessing
• Multi-Model Pipelines: Chain models together