ZeticMLangeLLMModel

This page reflects ZeticMLange iOS 1.9.0.

ZeticMLangeLLMModel loads an on-device LLM from the Melange registry and supports text generation, token streaming, function calling, image response for LFM-VL models, and KV state persistence.

Import

import ZeticMLange

Initializer

public init(
  personalKey: String,
  name: String,
  version: Int? = nil,
  modelMode: LLMModelMode = .RUN_AUTO,
  apType: APType? = nil,
  quantType: LLMQuantType? = nil,
  cacheHandlingPolicy: ModelCacheHandlingPolicy = .REMOVE_OVERLAPPING,
  initOption: LLMInitOption = LLMInitOption(),
  onDownload: ((Float) -> Void)? = nil
) async throws

Parameter	Type	Default	Description
`personalKey`	`String`	-	Personal key for accessing the model.
`name`	`String`	-	Model name in `account_name/project_name` format.
`version`	`Int?`	`nil`	Model version. `nil` loads the latest version.
`modelMode`	`LLMModelMode`	`.RUN_AUTO`	Backend selection strategy.
`apType`	`APType?`	`nil`	Optional processor filter.
`quantType`	`LLMQuantType?`	`nil`	Optional quantization filter.
`cacheHandlingPolicy`	`ModelCacheHandlingPolicy`	`.REMOVE_OVERLAPPING`	Managed artifact cache cleanup policy.
`initOption`	`LLMInitOption`	`LLMInitOption()`	LLM initialization options.
`onDownload`	`((Float) -> Void)?`	`nil`	Download progress callback from `0.0` to `1.0`.

let model = try await ZeticMLangeLLMModel(
  personalKey: PERSONAL_KEY,
  name: "account_name/project_name",
  initOption: LLMInitOption(nCtx: 4096)
)

Text Generation

`run(_:)`

Starts generation for a prompt.

public func run(_ text: String) throws -> LLMRunResult

let result = try model.run("Explain on-device AI in one paragraph.")

`waitForNextToken()`

Waits for the next generated token.

public func waitForNextToken() -> LLMNextTokenResult

while true {
  let next = model.waitForNextToken()
  if next.isFinished { break }
  append(next.token)
}

Vision-Language Response

Use respond(...) with an LFM-VL-capable model.

public func respond(
  systemPrompt: String = "",
  userText: String,
  image: ZeticMLangeLLMModel.Image
) throws -> AsyncThrowingStream<String, Error>

let image = ZeticMLangeLLMModel.Image(
  rgb: rgbBytes,
  width: width,
  height: height
)

for try await token in try model.respond(
  systemPrompt: "Answer briefly.",
  userText: "What is in this image?",
  image: image
) {
  append(token)
}

Function Calling

public var functionCallingSystemPrompt: String?

public func registerTool(_ spec: LLMToolSpec, executor: @escaping LLMToolExecutor) throws
public func unregisterTool(name: String) throws -> Bool
public func clearTools() throws
public func registeredToolSpecs() throws -> [LLMToolSpec]

try model.registerTool(
  LLMToolSpec(
    name: "lookup",
    description: "Look up local app data.",
    parametersJson: #"{"type":"object","properties":{"query":{"type":"string"}}}"#
  )
) { call in
  LLMToolResult(content: #"{"result":"Found"}"#)
}

try model.run("Use lookup to answer the question.")

KV State Persistence

public func saveKVState(path: String) throws
public func loadKVState(path: String) throws
public func resetKVState() throws

Use these APIs to persist or reset the current LLM state for resume flows.

Lifecycle

public private(set) var isClosed: Bool
public func cleanUp() throws
public func close()
public func forceDeinit()

Call cleanUp() before starting a fresh conversation. Call close() when the model is no longer needed.

ZeticMLangeLLMModel

On this page