The eight stages¶
Generated by scripts/gen_catalog_doc.py from the component and benchmark
registries. Do not edit by hand: run the script.
A RAG pipeline in Lasagna is eight stages behind typed contracts. A component fills one stage, and the stages around it cannot tell which one is there. That is what lets a sweep change one stage and hold the rest fixed.
ingest, chunk, embed, retrieve, enhance the query, rerank, agentic loop, generate
| Stage | Runs at | Swap it to change | Built in | Sweep key |
|---|---|---|---|---|
| Document processor | ingest | how files are parsed into text | 4 | config_overrides |
| Chunker | ingest | how documents are split | 9 | chunkers: |
| Embedder | query time | what vectors are stored | 6 | embedders: |
| Retriever | query time | how candidates are fetched | 5 | config_overrides |
| Query enhancer | query time | how the query is rewritten before retrieval | 8 | query_enhancers: |
| Reranker | query time | the ordering of retrieved candidates | 10 | rerankers: |
| Agentic pipeline | query time | the multi-step retrieval loop | 7 | config_overrides |
| Generator | answer time | how the answer is written from the retrieved passages | 6 | generators: |
Every stage can be named in a config_overrides entry with its stage name, for
example {name: ircot, agentic_pipeline: ircot}. Five stages also have a list key
that sweeps several components at once. A document processor does not sweep:
benchmark corpora are loaded as text, so check a processor on your own files.
Registering a component is one decorator. See Write a component.
Document processor¶
Abstract base class for document processors/extractors.
Base class: BaseDocumentProcessor in lasagna.core.base_components. It runs at ingest. Swap it to change how files are parsed into text.
def extract(file_path: str, **kwargs) -> ProcessedDocument
Function form with @lasagna.document_processor: f(file_path) -> str.
The 4 built-in components are listed in the component catalog.
Chunker¶
Abstract base class for document chunking strategies.
Base class: BaseChunker in lasagna.core.base_components. It runs at ingest. Swap it to change how documents are split.
def chunk(text: str, **kwargs) -> List[Chunk]
Function form with @lasagna.chunker: f(text) -> list[str].
The 9 built-in components are listed in the component catalog.
Embedder¶
Abstract base class for embedding models.
Base class: BaseEmbedder in lasagna.core.base_components. It runs at query time. Swap it to change what vectors are stored.
def encode(texts: List[str], **kwargs) -> np.ndarray
Function form with @lasagna.embedder: f(texts) -> one vector per text.
The 6 built-in components are listed in the component catalog.
Retriever¶
Abstract base class for retrieval strategies.
Base class: BaseRetriever in lasagna.core.base_components. It runs at query time. Swap it to change how candidates are fetched.
def retrieve(query: str, collection: str, top_k: int = 5, **kwargs) -> List[ScoredDocument]
Function form with @lasagna.retriever: f(query, top_k) -> list[str] or documents.
The 5 built-in components are listed in the component catalog.
Query enhancer¶
Abstract base class for query enhancement strategies.
Base class: BaseQueryEnhancer in lasagna.core.base_components. It runs at query time. Swap it to change how the query is rewritten before retrieval.
def enhance(query: str, **kwargs) -> List[str]
Function form with @lasagna.query_enhancer: f(query) -> list[str].
The 8 built-in components are listed in the component catalog.
Reranker¶
Abstract base class for reranking models.
Base class: BaseReranker in lasagna.core.base_components. It runs at query time. Swap it to change the ordering of retrieved candidates.
def rerank(query: str, documents: List[ScoredDocument], top_k: int = 5, **kwargs) -> List[ScoredDocument]
Function form with @lasagna.reranker: f(query, text) -> float, or f(query, documents, top_k) -> documents.
The 10 built-in components are listed in the component catalog.
Agentic pipeline¶
Abstract base class for pluggable agentic RAG pipelines.
Base class: BaseAgenticPipeline in lasagna.core.base_components. It runs at query time. Swap it to change the multi-step retrieval loop.
async def run(input: AgenticPipelineInput) -> AgenticRAGResult
Function form with @lasagna.agentic_pipeline: none: a loop is a class.
The 7 built-in components are listed in the component catalog.
Generator¶
Abstract base class for answer generation: the last stage.
Base class: BaseGenerator in lasagna.core.base_components. It runs at answer time. Swap it to change how the answer is written from the retrieved passages.
def generate(query: str, contexts: List[str], *, llm: Any = None, instructions: str = '', **kwargs) -> str
Function form with @lasagna.generator: f(query, contexts) -> str (the answer).
The 6 built-in components are listed in the component catalog.