LLM Inference
Vision-Language Inference
Run image and text prompts with ZeticMLange LFM-VL models.
Vision-language inference lets an on-device LLM answer questions about an RGB image plus a text prompt.
The Android and iOS examples target the planned ZeticMLange 1.13.0 Session API. Flutter's package has its own version and API. Check the loaded model's capabilities before sending media.
Image Input
Images are passed as RGB bytes with explicit width and height.
val image = LLMMedia.Image(
rgb = rgbBytes,
width = width,
height = height,
)let image = try LLMImage(
rgb: rgbBytes,
width: width,
height: height
)final image = ZeticMLangeLLMImage(
rgb: rgbBytes,
width: width,
height: height,
);The RGB byte length must be width * height * 3.
Respond To An Image
model.respond(
systemPrompt = "Answer briefly.",
userText = "What is in this image?",
media = listOf(image),
).collect { token ->
append(token)
}for try await token in try model.respond(
systemPrompt: "Answer briefly.",
userText: "What is in this image?",
media: [.image(image)]
) {
append(token)
}final response = await model.respond(
systemPrompt: 'Answer briefly.',
userText: 'What is in this image?',
image: image,
);Use a model whose capabilities report image support. Unsupported backends return an error. The Android example imports com.zeticai.mlange.core.model.llm.LLMMedia.