MultimodalPipeline

MultimodalPipeline runs the supported LFM2.5-Audio embedding-prefill profile. Supply embeddings from a compatible external audio encoder; raw audio inputs, backend selection, and generation tuning are not part of the public API.

Import

import com.zeticai.mlange.core.model.multimodal.MultimodalPipeline
import com.zeticai.mlange.core.model.multimodal.MultimodalProfile

MultimodalProfile

MultimodalProfile.lfmAudioPrefill(
    backboneGgufPath: String,
)

lfmAudioPrefill(...) is the only supported public multimodal profile. Its embedding must be compatible with the selected LFM2.5-Audio backbone.

Create a Pipeline

companion object {
    fun create(profile: MultimodalProfile): MultimodalPipeline
}

Creation throws ZeticMLangeException when the pipeline cannot be initialized, including a missing or incompatible backbone or a missing multimodal backend dependency.

val pipeline = MultimodalPipeline.create(
    MultimodalProfile.lfmAudioPrefill(
        backboneGgufPath = backbonePath,
    ),
)

Stream a Response

fun respond(
    systemPrompt: String,
    userText: String,
    embedding: FloatArray,
): Flow<String>

respond(...) emits UTF-8 text fragments as they are generated. A generation failure terminates the Flow with ZeticMLangeException; cancelling collection stops generation and completes the flow normally.

pipeline.respond(
    systemPrompt = "Answer briefly.",
    userText = "What did you hear?",
    embedding = audioEmbedding,
).collect { token ->
    append(token)
}

Cancelling the collector stops native generation and completes the flow normally. Consume or cancel a response before starting another operation on the same pipeline.

Reset and Close

fun resetSession()
fun close()

resetSession() clears the conversation state so the next response starts fresh. Call close() when the pipeline is no longer needed.

Related

• LLM inference overview
• Enums and constants