ZeticMLangeLLMModel

ZeticMLangeLLMModel is the iOS entry point for on-device LLM inference. The current API has two initializer families:

• Automatic selection by LLMModelMode
• Explicit selection by LLMTarget, LLMQuantType, and APType

Use the automatic initializer first. Use the explicit initializer when you need a fixed GGUF quantization or want to force Apple CPU or GPU.

Import

import ZeticMLange

Initializers

Automatic Selection (Recommended)

This initializer selects the runtime and quantization automatically from model metadata.

init(
    personalKey: String,
    name: String,
    version: Int? = nil,
    modelMode: LLMModelMode = .RUN_AUTO,
    dataSetType: LLMDataSetType? = nil,
    cacheHandlingPolicy: ZeticMLangeCacheHandlingPolicy = .REMOVE_OVERLAPPING,
    initOption: LLMInitOption = LLMInitOption(),
    onDownload: ((Float) -> Void)? = nil
) throws

let model = try ZeticMLangeLLMModel(
    personalKey: PERSONAL_KEY,
    name: "google/gemma-3-4b-it",
    modelMode: .RUN_AUTO,
    initOption: LLMInitOption(
        kvCacheCleanupPolicy: .CLEAN_UP_ON_FULL,
        nCtx: 4096
    )
)

Explicit Runtime Selection

Use this initializer when you want to choose the runtime family, GGUF quantization, and processor type directly.

init(
    personalKey: String,
    name: String,
    version: Int? = nil,
    target: LLMTarget,
    quantType: LLMQuantType,
    apType: APType = .CPU,
    cacheHandlingPolicy: ZeticMLangeCacheHandlingPolicy = .REMOVE_OVERLAPPING,
    initOption: LLMInitOption = LLMInitOption(),
    onDownload: ((Float) -> Void)? = nil
) throws

let model = try ZeticMLangeLLMModel(
    personalKey: PERSONAL_KEY,
    name: "google/gemma-3-4b-it",
    target: .LLAMA_CPP,
    quantType: .GGUF_QUANT_Q4_K_M,
    apType: .GPU,
    initOption: LLMInitOption(
        kvCacheCleanupPolicy: .DO_NOT_CLEAN_UP,
        nCtx: 4096
    )
)

initOption

initOption now contains LLM runtime initialization settings.

public struct LLMInitOption {
    public let kvCacheCleanupPolicy: LLMKVCacheCleanupPolicy
    public let nCtx: Int
}

apType Support

apType is relevant when you use the explicit initializer and choose target = .LLAMA_CPP.

Methods

run(_:)

Starts generation for a prompt.

func run(_ text: String) throws -> LLMRunResult

Returns: LLMRunResult

waitForNextToken()

Blocks until the next token is available.

func waitForNextToken() -> LLMNextTokenResult

Returns: LLMNextTokenResult

cleanUp()

Resets the current conversation state without destroying the model instance.

func cleanUp() throws

forceDeinit()

Fully releases the underlying target model.

func forceDeinit()

Multimodal (Beta)

runWithEmbeddings(_:)

Prefill the decoder with a flat embedding sequence (e.g. audio encoder output, or a chat template assembled by the SDK layer). Positions continue from the current KV-cache length, so this composes with prior run(_:) / runWithEmbeddings(_:) turns.

func runWithEmbeddings(_ embeddings: [Float]) throws -> LLMRunResult

Returns: LLMRunResult. After this call returns, the embedding batch is queued. Drive token decode + sampling with waitForNextToken() as you would for run(_:).

Throws: A ZeticMLangeError with the LLMTargetModelErrorDomain domain when the loaded target is not LLaMACppTargetModel.

tokenize(_:parseSpecial:)

Tokenize text using the model's vocabulary. With parseSpecial = true, special tokens (e.g. <|audio_bos|>, <|im_start|>) in the input are recognized as single tokens rather than split by BPE.

func tokenize(_ text: String, parseSpecial: Bool) throws -> [Int32]

Returns: [Int32] of token ids. Empty on failure. Throws on non-llama.cpp backends.

tokenEmbeddings(_:)

Look up per-token embedding vectors from the model's tok_embd tensor and return them concatenated into a flat [tokenIds.count * n_embd] buffer. Quantized rows are dequantized to Float.

func tokenEmbeddings(_ tokenIds: [Int32]) throws -> [Float]

Returns: [Float] of length tokenIds.count * n_embd. Empty on failure. Throws on non-llama.cpp backends.

specialTokenId(_:)

Resolve a special token by its surface form (e.g. "<|audio_bos|>") to its vocabulary id.

func specialTokenId(_ name: String) throws -> Int32

Returns: Token id, or -1 if the string does not resolve to a single special token in this model's vocab. Throws on non-llama.cpp backends.

Multimodal Helpers

The SDK ships supporting types in the ZeticMLange module:

See the Multimodal page for usage examples.

Full Examples

Automatic Selection

import ZeticMLange

let model = try ZeticMLangeLLMModel(
    personalKey: PERSONAL_KEY,
    name: "google/gemma-3-4b-it",
    modelMode: .RUN_AUTO,
    initOption: LLMInitOption(
        kvCacheCleanupPolicy: .CLEAN_UP_ON_FULL,
        nCtx: 4096
    )
)

try model.run("Explain on-device AI in one paragraph.")

var output = ""
while true {
    let result = model.waitForNextToken()
    if result.generatedTokens == 0 { break }
    output.append(result.token)
}

try model.cleanUp()
model.forceDeinit()

Explicit Apple GPU Selection

import ZeticMLange

let model = try ZeticMLangeLLMModel(
    personalKey: PERSONAL_KEY,
    name: "google/gemma-3-4b-it",
    target: .LLAMA_CPP,
    quantType: .GGUF_QUANT_Q4_K_M,
    apType: .GPU,
    initOption: LLMInitOption(
        kvCacheCleanupPolicy: .DO_NOT_CLEAN_UP,
        nCtx: 4096
    )
)

Notes

The initializer can download model artifacts on first use. Create the model off the main thread if you want to avoid blocking the UI.

See Also

• ZeticMLangeLLMModel (Android): Android equivalent
• LLM Inference Overview: Recommended initialization paths
• Streaming Token Generation: Streaming patterns and cleanup
• Enums and Constants: LLM enums and config types