Components¶
Generated by scripts/gen_catalog_doc.py from the component and benchmark
registries. Do not edit by hand: run the script.
55 components ship with the package, across eight stages. Name one in a
sweep, or run one with lasagna benchmark run <benchmark> --reranker <name>. The same
list prints from lasagna component list.
The last column is what the component needs beyond pip install lasagna-rag. "An LLM" means the run's LLM, set up as in Installation. "Downloads a model" means the weights come from the Hugging Face Hub on first use (see Model licenses). Extras are listed in Installation.
Document processor¶
4 built in. Swap this stage to change how files are parsed into text. Contract: BaseDocumentProcessor.
| Name | What it does | Source | Needs |
|---|---|---|---|
default |
Built-in extraction (pypdf and pdfplumber, OCR when enabled) | Nothing extra | |
docling |
Docling layout analysis, TableFormer tables and OCR, exported as Markdown | arXiv:2408.09869 | pip install 'lasagna-rag[docling]' |
markitdown |
Microsoft MarkItDown: PDF, Office, HTML and more to Markdown | github.com | pip install 'lasagna-rag[markitdown]' |
pymupdf4llm |
PyMuPDF4LLM: PDF to Markdown with headers and tables (AGPL-3.0) | github.com | pip install 'lasagna-rag[pymupdf4llm]' (AGPL-3.0) |
Chunker¶
9 built in. Swap this stage to change how documents are split. Contract: BaseChunker.
| Name | What it does | Source | Needs |
|---|---|---|---|
contextual |
Chunks prefixed with LLM-written document context (Contextual Retrieval) | anthropic.com | An LLM |
fixed |
Fixed-size windows with overlap, in characters, words or tokens | Nothing extra | |
hierarchical |
Hierarchical parent-child chunker | Nothing extra | |
markdown_header |
Markdown sections with their header path in metadata (LangChain MarkdownHeaderTextSplitter) | github.com | Nothing extra |
proposition |
Propositions written by an LLM as retrieval units (Dense X Retrieval) | arXiv:2312.06648 | An LLM |
recursive |
Recursive separator splitting: paragraph, line, sentence, word (LangChain RecursiveCharacterTextSplitter) | github.com | Nothing extra |
semantic |
Semantic chunker | The embedding model | |
sentence_window |
Sliding windows of window_size whole sentences | Nothing extra | |
structure_aware |
Structure-aware chunker | The embedding model |
Embedder¶
6 built in. Swap this stage to change what vectors are stored. Contract: BaseEmbedder.
| Name | What it does | Source | Needs |
|---|---|---|---|
bge_m3 |
BGE-M3 dense vectors; sparse and multi-vector outputs via FlagEmbedding | arXiv:2402.03216 | Downloads BAAI/bge-m3 on first use; [flag-embedding] adds its sparse and multi-vector outputs |
cohere |
Cohere Embed v3/v4 with search_query/search_document input types | docs.cohere.com | pip install cohere and COHERE_API_KEY (or CO_API_KEY) |
default |
The configured embedding model | Downloads a model on first use | |
openai |
OpenAI text-embedding-3 small/large, with Matryoshka dimensions | platform.openai.com | openai package and OPENAI_API_KEY |
sentence_transformers |
Any sentence-transformers model (embedding_model), with its model card's query/document prefixes | arXiv:1908.10084 | Downloads a model on first use |
voyage |
Voyage AI voyage-3 family with query/document input types | docs.voyageai.com | pip install voyageai and VOYAGE_API_KEY |
Retriever¶
5 built in. Swap this stage to change how candidates are fetched. Contract: BaseRetriever.
| Name | What it does | Source | Needs |
|---|---|---|---|
bm25 |
Okapi BM25 over the session's chunks (bm25_k1, bm25_b) | doi:10.1561/1500000019 | Nothing extra |
dense |
Dense vector search with the configured embedder (the default first stage) | arXiv:2004.04906 | Nothing extra |
hybrid_convex |
Convex combination of normalised dense and BM25 scores (hybrid_alpha) | arXiv:2210.11934 | Nothing extra |
hybrid_rrf |
Dense and BM25 fused by Reciprocal Rank Fusion (rrf_k) | doi:10.1145/1571941.1572114 | Nothing extra |
splade |
SPLADE learned sparse retrieval (splade_model) | arXiv:2109.10086 | Downloads a model on first use |
Query enhancer¶
8 built in. Swap this stage to change how the query is rewritten before retrieval. Contract: BaseQueryEnhancer.
| Name | What it does | Source | Needs |
|---|---|---|---|
combined |
Diverse LLM reformulations of the query (RAG Fusion style) | An LLM | |
hyde |
HyDE: the query plus a hypothetical answer passage, searched and fused | arXiv:2212.10496 | An LLM |
multi_query |
RAG-Fusion: LLM-generated search queries, fused by reciprocal rank | arXiv:2402.03367 | An LLM |
query2doc |
Query2doc: the query followed by an LLM-written pseudo-document | arXiv:2303.07678 | An LLM |
query_decomposition |
Least-to-most decomposition: the sub-questions a question needs, searched and fused | arXiv:2205.10625 | An LLM |
query_expansion |
LLM paraphrases of the query, searched and fused | An LLM | |
rewrite |
Rewrite-Retrieve-Read: search queries rewritten by an LLM | arXiv:2305.14283 | An LLM |
step_back |
Step-back prompting: the question and a more generic step-back question | arXiv:2310.06117 | An LLM |
Reranker¶
10 built in. Swap this stage to change the ordering of retrieved candidates. Contract: BaseReranker.
| Name | What it does | Source | Needs |
|---|---|---|---|
cohere |
Cohere Rerank API (pip install cohere, COHERE_API_KEY) | pip install cohere and COHERE_API_KEY (or CO_API_KEY) |
|
colbert |
ColBERT late-interaction reranker with token-level MaxSim | arXiv:2004.12832 | Downloads a model on first use |
combined |
Cross-encoder followed by MMR diversity | Downloads a model on first use | |
cross_encoder |
Cross-encoder reranker (the configured reranker_model) | arXiv:1901.04085 | Downloads a model on first use |
ensemble |
Weighted ensemble of cross-encoders | Downloads a model on first use | |
flashrank |
FlashRank ONNX cross-encoders on CPU (pip install flashrank) | pip install flashrank |
|
llm_listwise |
RankGPT: LLM permutations over windows sliding from the bottom up | arXiv:2304.09542 | An LLM |
llm_pointwise |
LLM relevance grade per passage on a 0-3 scale (llm_reranker_model) | arXiv:2309.10621 | An LLM |
mmr |
MMR diversity reranking | doi:10.1145/290941.291025 | Nothing extra |
monot5 |
monoT5: P(true) from a seq2seq relevance model (monot5_model) | arXiv:2003.06713 | Downloads a model on first use |
Agentic pipeline¶
7 built in. Swap this stage to change the multi-step retrieval loop. Contract: BaseAgenticPipeline.
| Name | What it does | Source | Needs |
|---|---|---|---|
adaptive |
Adaptive-RAG: a complexity classifier picks no, single-step or multi-step retrieval | arXiv:2403.14403 | An LLM |
default |
Built-in structured RAG graph (13 nodes, CRAG, quality gates) | An LLM | |
flare |
FLARE: retrieve when the tentative next sentence has low-confidence tokens | arXiv:2305.06983 | An LLM |
ircot |
IRCoT: chain-of-thought sentences interleaved with retrieval, each the next query | arXiv:2212.10509 | An LLM |
legacy_react |
Legacy ReAct agent with monolithic search tool | arXiv:2210.03629 | An LLM |
self_ask |
Self-Ask: follow-up questions answered by retrieval until the final answer | arXiv:2210.03350 | An LLM |
self_rag |
Self-RAG (prompted critic): retrieve on demand, critique each passage's answer | arXiv:2310.11511 | An LLM |
Generator¶
6 built in. Swap this stage to change how the answer is written from the retrieved passages. Contract: BaseGenerator.
| Name | What it does | Source | Needs |
|---|---|---|---|
chain_of_note |
A reading note per passage, then the answer or 'unknown' | Yu et al. 2023, Chain-of-Note: Enhancing Robustness in Retrieval-Augmented LMs | An LLM |
chain_of_thought |
Step-by-step reasoning, then the final answer | Wei et al. 2022, Chain-of-Thought Prompting Elicits Reasoning in LLMs | An LLM |
cited |
Answers citing their supporting passages as [n] | Gao et al. 2023, Enabling Large Language Models to Generate Text with Citations (ALCE) | An LLM |
closed_book |
The question alone, no passages: the no-retrieval baseline | An LLM | |
concise |
One concise sentence from the context (the harness's default prompt) | An LLM | |
long_context |
The concise prompt over all ten retrieved passages, in rank order | An LLM |