Melange
LLM Inference

Vision-Language Inference

Run image and text prompts with ZeticMLange LFM-VL models.

Vision-language inference lets an on-device LLM answer questions about an RGB image plus a text prompt.

The Android and iOS examples target the planned ZeticMLange 1.13.0 Session API. Flutter's package has its own version and API. Check the loaded model's capabilities before sending media.

Image Input

Images are passed as RGB bytes with explicit width and height.

val image = LLMMedia.Image(
    rgb = rgbBytes,
    width = width,
    height = height,
)
let image = try LLMImage(
  rgb: rgbBytes,
  width: width,
  height: height
)
final image = ZeticMLangeLLMImage(
  rgb: rgbBytes,
  width: width,
  height: height,
);

The RGB byte length must be width * height * 3.

Respond To An Image

model.respond(
    systemPrompt = "Answer briefly.",
    userText = "What is in this image?",
    media = listOf(image),
).collect { token ->
    append(token)
}
for try await token in try model.respond(
  systemPrompt: "Answer briefly.",
  userText: "What is in this image?",
  media: [.image(image)]
) {
  append(token)
}
final response = await model.respond(
  systemPrompt: 'Answer briefly.',
  userText: 'What is in this image?',
  image: image,
);

Use a model whose capabilities report image support. Unsupported backends return an error. The Android example imports com.zeticai.mlange.core.model.llm.LLMMedia.

API Reference

On this page