Skip to content

Stage isolation

A comparison tells you something only when the two arms differ in the one thing you changed. Lasagna splits a pipeline into eight stages and checks, at run time, that a swap stayed inside its stage.

The eight stages

document_processor -> chunker -> embedder -> retriever -> query_enhancer
                                             -> reranker -> agentic_pipeline -> generator

Each stage has a fixed interface. A reranker takes a query and candidates and returns them ranked. A chunker takes text and returns chunks. Because the interface is fixed, you can swap one component for another and leave the other seven stages alone. The catalog lists the stages and their contracts, and the write a component guide shows how to fill one.

What the check does

Each task records a stage manifest. The manifest is a digest of every setting and every component that reached each stage. After the sweep, Lasagna compares the manifests of all arms against what each arm said it would change.

Two things count as a failure:

  • Undeclared change. A stage differs between two arms, but neither arm named it. A setting leaked across a stage boundary, for example through a profile default.
  • Inert change. An arm names a change that its manifest does not show. The setting or component never reached the pipeline, so the arm's number is the baseline's under another name.

The run prints both. lasagna results validate refuses a bundle with an undeclared change, and lasagna results view shows the check on its page.

What the check does not see

The manifest digests configuration, not behavior. A component that changes state outside its stage while it runs, such as patching another module, leaves every digest unchanged and passes the check. Two other guards cover that case:

  • The stage contracts, which say what a component may touch.
  • The conformance tests. lasagna component validate <module:Class> runs them against your component. See Contributing components.

Strict stages

A stage that cannot run fails the task. If you ask for query expansion and no LLM is configured, or an agentic search errors, the task fails. It does not fall back to the baseline and report the baseline's score under the arm's name.

Settings that do not count

Some settings change how a run is operated, not what it computes. A cache backend, a connection pool size and a log level are examples. They stay out of the manifest, so they never read as a change to the system under test.