RagPipeline

This page reflects zetic_mlange 1.9.1.

Flutter RAG uses the same composition style as Android and iOS: a RagPipeline combines a retriever, an existing ZeticMLangeLLMModel, and a RagProfile.

Import

import 'package:zetic_mlange/zetic_mlange.dart';

`RagPipeline`

RagPipeline({
  required RagRetriever retriever,
  required ZeticMLangeLLMModel llm,
  required RagProfile profile,
  RagPipelineConfig config = const RagPipelineConfig(),
})

Stream<String> respond({required String query, String? system})

respond(...) retrieves context, starts generation, and streams generated tokens.

final rag = RagPipeline(
  retriever: retriever,
  llm: model,
  profile: const RagProfile.qwen25(),
);

await for (final token in rag.respond(query: 'What does ZeticMLange do?')) {
  append(token);
}

`RagRetriever`

abstract interface class RagRetriever {
  FutureOr<List<RetrievedChunk>> retrieve(String query, {required int topK});
}

Implement this interface for app-owned retrieval, remote vector stores, local databases, or other retrieval systems.

`RetrievedChunk`

const RetrievedChunk({
  required String text,
  double? score,
  String? source,
  Map<String, String>? metadata,
})

text is included in the augmented prompt. score, source, and metadata are optional retrieval metadata.

`RagPipelineConfig`

const RagPipelineConfig({
  int topK = 5,
  int maxContextTokens = 2048,
  String systemPrefix =
      'You are a helpful assistant. Use the context below to answer.',
})

`LocalRagPipeline`

static Future<LocalRagPipeline> create({
  required RagProfile profile,
  required String embedderGgufPath,
  LocalRagConfig config = const LocalRagConfig(),
})

Future<void> indexDocs(List<RagDocument> docs)
Future<List<RetrievedChunk>> retrieve(String query, {required int topK})
void close()

LocalRagPipeline implements RagRetriever, so it can be passed directly to RagPipeline.

final localRag = await LocalRagPipeline.create(
  profile: const RagProfile.qwen25(backboneGgufPath: backbonePath),
  embedderGgufPath: embedderPath,
);

await localRag.indexDocs([
  const RagDocument(text: 'Local context.', source: 'docs'),
]);

final rag = RagPipeline(
  retriever: localRag,
  llm: model,
  profile: const RagProfile.qwen25(backboneGgufPath: backbonePath),
);

`RagDocument`

const RagDocument({required String text, required String source})

Use RagDocument when indexing documents into LocalRagPipeline.

`LocalRagConfig`

const LocalRagConfig({
  int topK = 5,
  int chunkSizeChars = 1024,
  int chunkOverlapChars = 128,
  int embedderNCtx = 512,
  RagEmbedderPooling embedderPooling = RagEmbedderPooling.mean,
})

RagPipeline

On this page