Prepare Model and Input(s)

Guide to preparing TorchScript, PyTorch Exported Program or ONNX models and NumPy inputs for ZETIC.MLange

Save Model and Input(s)

The inputs for MLange are:

Model: TorchScript, PyTorch Exported Program, or ONNX format
Input(s): NumPy array format

Deprecated Format

TorchScript (.pt) support will be deprecated soon. We strongly recommend using PyTorch Exported Program (.pt2) or ONNX for future compatibility and better optimization.

To use PyTorch nn.Module, please trace your model first:

import torch
import numpy as np

torch_model = torch.nn.Module(...)

# Trace your PyTorch model
torchscript_model = torch.jit.trace(your_torch_model, TORCH_INPUTS)

# (1) Save your traced model
torch.jit.save(torchscript_model, OUTPUT_TORCHSCRIPT_MODEL_PATH)

# (2) Save your sample inputs to use
np_input = TORCH_INPUT.detach().numpy()
np.save("INPUT.npy", np_input)

For more details, refer to the torch.jit.save documentation.

To use PyTorch 2.0+ ExportedProgram, please export your model:

import torch
import numpy as np

torch_model = torch.nn.Module(...)

# (1) Export the model
exported_program = torch.export.export(torch_model, (TORCH_INPUTS,))
torch.export.save(exported_program, "model.pt2")

# (2) Save your sample inputs to use
np_input = TORCH_INPUT.detach().numpy()
np.save("INPUT.npy", np_input)

For more details, refer to the torch.export documentation.

TensorFlow / Keras

Use tf2onnx to convert TensorFlow models:

import tensorflow as tf
import tf2onnx

# Load your model
model = tf.keras.models.load_model("my_model.h5")

# Convert to ONNX
spec = (tf.TensorSpec((1, 224, 224, 3), tf.float32, name="input"),)
output_path = "model.onnx"

model_proto, _ = tf2onnx.convert.from_keras(model, input_signature=spec, output_path=output_path)

Scikit-Learn

Use skl2onnx for Scikit-Learn models:

from sklearn.ensemble import RandomForestClassifier
from skl2onnx import to_onnx
import numpy as np

# Train a model
model = RandomForestClassifier()
model.fit(X_train, y_train)

# Convert to ONNX
onx = to_onnx(model, X_train[:1].astype(np.float32))
with open("model.onnx", "wb") as f:
    f.write(onx.SerializeToString())

For other frameworks, refer to the ONNX Tutorials.

Verify Input Order & Shapes

ZETIC.MLange compiles your model into a static hardware graph. This means consistent Input Order and Input Shapes are mandatory for execution.

The internal computation graph expects data in specific slots (e.g., input_zero at index 0, input_one at index 1). If you swap them during upload or inference, the model will produce garbage results or crash.

How to Check with Netron

We highly recommend using Netron to visualize your model's input signature.

Open your model (.pt, .onnx, etc.) in Netron.
Locate the Input Nodes at the top of the graph.
Note the vertical order: The top-most input is Index 0, the next is Index 1, and so on.

Strict Input Sequence

You MUST provide inputs in this exact order when:

Uploading the model via CLI (-i input0.npy -i input1.npy).
Calling the run() function in your Android/iOS app.

Checking input sequence with Netron

Fixed Input Shapes

Performance Note: NPU compilation hard-codes input shapes to maximize throughput. Even if your original model supports dynamic sizes, the accelerated MLange model will accept only the exact shape of the sample input provided/generated during upload.