ZeticMLangeLLMModel

This page reflects ZeticMLange iOS 1.11.0.

ZeticMLangeLLMModel loads an on-device LLM from the Melange registry and supports text generation, token streaming, function calling, image response for LFM-VL models, and KV state persistence.

Import

import ZeticMLange

Initializer

public init(
  personalKey: String,
  name: String,
  version: Int? = nil,
  modelMode: LLMModelMode = .RUN_AUTO,
  cacheHandlingPolicy: ModelCacheHandlingPolicy = .REMOVE_OVERLAPPING,
  initOption: LLMInitOption = LLMInitOption(),
  onDownload: ((Float) -> Void)? = nil
) async throws

Parameter	Type	Default	Description
`personalKey`	`String`	-	Personal key for accessing the model.
`name`	`String`	-	Model name in `account_name/project_name` format.
`version`	`Int?`	`nil`	Model version. `nil` loads the latest version.
`modelMode`	`LLMModelMode`	`.RUN_AUTO`	Backend selection strategy.
`cacheHandlingPolicy`	`ModelCacheHandlingPolicy`	`.REMOVE_OVERLAPPING`	Managed artifact cache cleanup policy.
`initOption`	`LLMInitOption`	`LLMInitOption()`	LLM initialization options.
`onDownload`	`((Float) -> Void)?`	`nil`	Download progress callback from `0.0` to `1.0`.

let model = try await ZeticMLangeLLMModel(
  personalKey: PERSONAL_KEY,
  name: "account_name/project_name",
  initOption: LLMInitOption(nCtx: 4096)
)

Text Generation

`run(_:)`

Starts generation for a prompt.

public func run(_ text: String) throws

try model.run("Explain on-device AI in one paragraph.")

Reading Generated Tokens

Waits for the next generated token.

public func waitForNextToken() -> LLMNextTokenResult

while true {
  let next = model.waitForNextToken()
  if next.isFinished { break }
  append(next.token)
}

Vision-Language Response

Use respond(...) with an LFM-VL-capable model.

public func respond(
  systemPrompt: String = "",
  userText: String,
  image: ZeticMLangeLLMModel.Image
) throws -> AsyncThrowingStream<String, Error>

let image = try ZeticMLangeLLMModel.Image(
  rgb: rgbBytes,
  width: width,
  height: height
)

for try await token in try model.respond(
  systemPrompt: "Answer briefly.",
  userText: "What is in this image?",
  image: image
) {
  append(token)
}

Function Calling

public var functionCallingSystemPrompt: String?

public func registerTool(_ spec: LLMToolSpec, executor: @escaping LLMToolExecutor) throws
public func unregisterTool(name: String) throws -> Bool
public func clearTools() throws
public func registeredToolSpecs() throws -> [LLMToolSpec]

try model.registerTool(
  LLMToolSpec(
    name: "lookup",
    description: "Look up local app data.",
    parametersJson: #"{"type":"object","properties":{"query":{"type":"string"}}}"#
  )
) { call in
  LLMToolResult(content: #"{"result":"Found"}"#)
}

for try await token in try model.runWithTools("Use lookup to answer the question.") {
  append(token)
}

runWithTools(_:) returns an AsyncThrowingStream. Consume it to completion before starting another model operation.

KV State Persistence

public func saveKVState(path: String) throws
public func loadKVState(path: String) throws
public func resetKVState() throws

Use these APIs to persist or reset the current LLM state for resume flows.

Schedule an LLM artifact download without initializing the LLM runtime. Await scheduling while the app is foregrounded so iOS can register the background URL-session task before the app moves to the background.

static func downloadInBackground(
  personalKey: String,
  name: String,
  version: Int? = nil,
  modelMode: LLMModelMode = .RUN_AUTO,
  cacheHandlingPolicy: ModelCacheHandlingPolicy = .KEEP_EXISTING
) async throws -> BackgroundDownloadHandle

static func getBackgroundDownloadStatus(
  _ handle: BackgroundDownloadHandle
) async throws -> BackgroundDownloadStatus

static func stopBackgroundDownload(
  _ handle: BackgroundDownloadHandle
) async throws -> BackgroundDownloadStopResult

static func removeDownloadedModel(
  name: String,
  version: Int? = nil
) async throws -> ModelRemovalResult

let handle = try await ZeticMLangeLLMModel.downloadInBackground(
  personalKey: PERSONAL_KEY,
  name: "account_name/project_name"
)

let status = try await ZeticMLangeLLMModel.getBackgroundDownloadStatus(handle)
let stopResult = try await ZeticMLangeLLMModel.stopBackgroundDownload(handle)
let removal = try await ZeticMLangeLLMModel.removeDownloadedModel(
  name: "account_name/project_name"
)

Configure the iOS host app to forward background URL-session events through ZeticMLangeModel.handleBackgroundURLSessionEvents(...); that forwarding API is on the general-model class and supports LLM downloads as well. See iOS advanced configuration. Stopping preserves an already installed model; if removal.isInUse is true, close active LLM instances and retry removal.

Lifecycle

public private(set) var isClosed: Bool
public func cleanUp() throws
public func close()
public func forceDeinit()

Call cleanUp() before starting a fresh conversation. Call close() when the model is no longer needed.