Reference

TabSchema, TabSynth, TabHead

The three phases of the paper under their own names.

TabAgent joins a feature card, an optional TabSynth and a TabHead into one model that cross_validate can clone, fit and score. The concepts are on The three phases.

from tabagent import TabAgent, TabHead, TabSynth, datasets

card = datasets.card("skillsbench/skill-select")
agent = TabAgent(card, head=TabHead("ranking"), synth=TabSynth(k=2))

TabAgent

TabAgent(card: CardLike, head: TabHead | None = None, synth: Any = None, seed: int = 0)

A feature card, an optional TabSynth and a TabHead, trained and applied as one model.

Parameters:

Name	Type	Description	Default
card	CardLike	the FeatureCard (or a DecisionTable -> FeatureCard callable, called on each training table).	required
head	TabHead | None	the TabHead to fit; default TabHead(card.spec.kind) with the paper's CatBoost preset.	None
synth	Any	a TabSynth (or anything with its augment method) applied to the training rows, or None.	None
seed	int	the head's seed (cross_validate replaces it per seed through clone). After fit: card_ (the fitted card), head_ (the fitted head), n_synthetic_ and feature_stats_ (the training distribution of the head's inputs, for drift monitoring; see tabagent.drift).	0

Examples:

from tabagent import TabAgent, TabHead, TabSynth, datasets

name = "skillsbench/skill-select"
table = datasets.load(name, download=True)
agent = TabAgent(datasets.card(name), head=TabHead("ranking"), synth=TabSynth(k=2))
agent.fit(table)
agent.decide(table.subset(table.groups[:1]).drop_labels(), k=5)  # top-5 candidates of the first task

fit

fit(table: DecisionTable) -> TabAgent

Fit the card's statistics, synthesize (if set) and fit the head on the labeled table.

decide

decide(
    data: DecisionTable | DataFrame,
    *args: Any,
    k: int | None = None,
    defer_below: float | None = None,
    fallback: Callable[[DataFrame], Any] | None = None,
    confidence_method: str = "largest_gap",
) -> DataFrame

Decode a decision for every slate in data.

Parameters:

Name	Type	Description	Default
data	DecisionTable | DataFrame	a DecisionTable or a DataFrame with one row per (decision, candidate) (see as_table). Further positional arguments raise a TypeError that explains the difference from TabAgentHead.decide(prefix, candidates, k=...).	required
k	int | None	keep the top k candidates of each ranking (keyword-only).	None
defer_below	float | None	mark decisions whose confidence is below this value deferred (the confidence-gated fallback, App. N.3). For rankings the confidence is tabhead.confidence(method= confidence_method); for single-class decisions it is the predicted class's probability.	None
fallback	Callable[[DataFrame], Any] | None	called with the deferred decisions' rows (typically the original LLM component); its return value is stored in result.attrs["fallback"].	None

Returns:

Type	Description
DataFrame	One row per selected (decision, candidate) with score and rank (ties ordered by candidate id), or one row per decision with choice for single-class decisions; with defer_below, also confidence and deferred.

score

score(data: DecisionTable | DataFrame) -> ndarray

p(candidate is chosen) per row; the predicted class per row for single-class decisions.

clone

clone(seed: int | None = None) -> TabAgent

An unfitted copy with the same card, head settings and synthesizer (and seed, when given).

save

save(path: str | Path, metrics: dict | None = None) -> Path

Write a portable, pickle-free artifact directory (see tabagent.artifact); metrics are recorded in its manifest.

load classmethod

load(path: str | Path, verify: bool = True, trust_code: bool = False) -> TabAgent

Load an artifact written by save from a directory or hf://OWNER/REPO[@REVISION]. verify checks every file against the manifest's SHA-256; trust_code allows card features computed by modules outside tabagent (python: sources).

TabHead

TabHead(
    kind: str = "ranking",
    preset: str = "paper",
    estimator: Callable[[int], Any] | None = None,
    featurize: bool = True,
    text_max_features: int = TFIDF_MAX_FEATURES,
    text_repr: str = "auto",
    threshold: float = 0.5,
    seed: int = 0,
    **catboost_overrides,
)

The TabAgent head (paper Sec. 3.3): a classifier over the feature card's columns.

fit and score operate on feature frames (the card's output); decode turns scores into decisions.

Parameters:

Name	Type	Description	Default
kind	str	the decision pattern (ranking, multilabel, single, policy); sets loss and decoder.	'ranking'
preset	str	CatBoost configuration (see PRESETS; paper is the paper's, Sec. 4.2). Not used with estimator.	'paper'
estimator	Callable[[int], Any] | None	seed -> classifier factory for alternative heads (fit/predict_proba/classes_).	None
featurize	bool	set False for heads that consume raw columns themselves (for example TabSTAR).	True
text_max_features	int	TF-IDF vocabulary size per text column.	TFIDF_MAX_FEATURES
text_repr	str	tfidf, lsa (TF-IDF reduced by SVD, see LSA_DIMS) or auto (LSA above LSA_AUTO_TEXT_COLUMNS text columns); the fitted choice is text_repr_.	'auto'
threshold	float	the multi-label decision threshold (App. J).	0.5
seed	int	the model seed.	0
catboost_overrides		CatBoost parameters that replace the preset's (not allowed with estimator).	{}

Examples:

from tabagent import TabHead

TabHead("ranking")  # CatBoost, the paper's configuration (300 iterations, learning rate 0.08, depth 6)
TabHead("ranking", preset="catboost-default")  # CatBoost's own schedule

fit

fit(X: DataFrame, y, types: Mapping[str, FeatureType]) -> TabHead

Fit on the feature frame X with labels y; types declares each column's type.

score

score(X: DataFrame) -> ndarray

p(positive) per row, or the predicted class for single-class decisions.

confidence

confidence(scores, decision_ids, candidates, method: str = 'largest_gap', r=None) -> Series

Per-decision confidence of a ranking (App. N.3, "Confidence-gated fallback").

margin_at_r score(R-th) - score((R+1)-th), where R = |G(t)| (needs r: a mapping or Series from decision id to R; an oracle quantity, used for analysis) largest_gap the largest drop between consecutive ranked scores (cutoff-free, deployable)

risk_coverage

risk_coverage(confidence_values, correct) -> DataFrame

Selective quality when deferring the least-confident decisions (App. N.3, Fig. 14): for each coverage level (fraction kept), the mean correctness of the kept decisions.

TabSynth

TabSynth(
    k: int | None = None,
    text_cols: Sequence[str] | None = None,
    paraphrase: Callable[[str, int], str] = det_paraphrase_v1,
    max_source_rows: int | None = None,
    seed: int = DEFAULT_SEED,
    where=None,
    *,
    recompute: bool = False,
    entity_swap: bool = False,
    allocation: str = "uniform",
    max_variants: int = 12,
)

Algorithm 1 of the paper (App. H).

Parameters:

Name	Type	Description	Default
k	int | None	variants per real row (the paper uses two to four, Sec. 3.2). None uses the card's synth_k (DEFAULT_K when the card sets none).	None
text_cols	Sequence[str] | None	free-text columns to paraphrase; defaults to the card's synth_text.	None
paraphrase	Callable[[str, int], str]	(text, j) -> text; the default is the paper's deterministic rule-based paraphraser.	det_paraphrase_v1
max_source_rows	int | None	optionally paraphrase a seeded random subset of the real rows (for very large cards).	None
seed	int	seeds only the max_source_rows subsample; the paraphrases themselves are deterministic.	DEFAULT_SEED
where		paraphrase only the source rows matching {column: value} or a frame -> bool mask callable. This is domain-matched synthesis (App. B.6): e.g. where={"domain": "Shopping"} adds paraphrases of Shopping rows only, for the model evaluated on Shopping.	None
recompute	bool	build each variant from the table's raw row with the paraphrased text, then apply the card, so text-derived features (length, lexical and embedding similarity) match the paraphrase instead of being copied. Needs table and card in augment and text columns that are raw table columns.	False
entity_swap	bool	swap entities (swap_entities) before paraphrasing variant j, seeded by j.	False
allocation	str	"uniform" (k variants per row) or "inverse_sqrt_frequency" (inverse_sqrt_frequency_allocation: per-decision budgets averaging k, between 1 and max_variants). Needs table in augment. After augment: stats_ (counts and the source groups), k_ (the k used), source_rows_ and source_groups_ (see augment).	'uniform'

Examples:

from tabagent import TabAgent, TabSynth, datasets

card = datasets.card("skillsbench/skill-select")
agent = TabAgent(card, synth=TabSynth(k=2))  # two paraphrases per real training row, inside each fit

augment

augment(X: DataFrame, y: ndarray, table=None, card=None) -> tuple[DataFrame, ndarray]

Return the synthetic rows (feature frame, labels) for the real training rows X, y.

X is the card's feature frame of the real training rows; table and card supply the default text columns and k and the source group of each row (recorded in stats_ for provenance). After the call, source_rows_ holds the row position in X that each synthetic row paraphrases, and source_groups_ its group (when table is given).

TabSchema

TabSchema(
    llm: LLM | None = None,
    config: TabSchemaConfig | None = None,
    adapter=None,
    store: RunStore | None = None,
    sandbox: SubprocessSandbox | None = None,
    stage_llms: dict[str, LLM] | None = None,
)

The FeatureOrchestrator: discovers a feature card for one decision component of an agent.

Parameters:

Name	Type	Description	Default
llm	LLM | None	the model for every stage (default: auto_llm(config.model), from environment keys).	None
config	TabSchemaConfig | None	a TabSchemaConfig.	None
adapter		reads the agent's logs into trajectories and decision points (default GenericJsonAdapter).	None
store	RunStore | None	the run directory (default: a new one under $TABAGENT_CACHE/runs).	None
sandbox	SubprocessSandbox | None	runs LLM-written extraction code (default SubprocessSandbox).	None
stage_llms	dict[str, LLM] | None	LLM backends by model name, for the models config.stage_models and config.fallback_model name (default: built from the environment with auto_llm). With config.max_cost_usd set, the model needs a price in config.prices; otherwise the budget could not be enforced and the constructor raises ValueError.	None

Examples:

from tabagent import TabSchema, TabSchemaConfig
from tabagent.adapters import detect_adapter

adapter = detect_adapter("./agent_logs")
schema = TabSchema(config=TabSchemaConfig(n_analyze=20), adapter=adapter)  # the LLM comes from env keys
result = schema.discover("./agent_logs", "ToolShortlist")
result.card  # the discovered FeatureCard; the run directory holds extract.py and the reports

discover

discover(
    trajectories: list[Trajectory] | str | Path,
    target: str,
    kind: str = "ranking",
    name: str | None = None,
) -> DiscoveryResult

Run the discovery loop for the decision component target.

trajectories are Trajectory objects or a path the adapter reads; only successful ones are used. kind is the decision pattern and name the card name. Every intermediate artifact is written to the run directory, with run.json (the run's progress and anatomy, tabagent runs show); a run that stops leaves a partial report.md there, and rerunning with the same directory resumes it.

TabSchemaConfig dataclass

TabSchemaConfig(
    model: str | None = None,
    n_analyze: int = 20,
    sample_seed: int = DEFAULT_SEED,
    sample_strategy: str = "coverage",
    max_code_retries: int = 3,
    json_retries: int = 1,
    accept: tuple[str, ...] = ("accept", "conditional"),
    compile_min_success: float = 0.95,
    max_null_rate: float = 0.5,
    max_columns_per_feature: int = 64,
    max_extractor_columns: int = 512,
    compile_repair: bool = True,
    drop_redundant_text: bool = True,
    dependency_features: tuple[str, ...] = (
        "candidate_prior",
        "context_candidate_lex",
        "context_candidate_sim",
        "co_usage",
    ),
    pair_embeddings: bool | None = None,
    workers: int = 8,
    timeout_s: float = 30,
    max_cost_usd: float | None = None,
    prices: dict[str, tuple[float, float]] = dict(),
    max_prompt_chars: int = 400000,
    stage_models: dict[str, str] = dict(),
    fallback_model: str | None = None,
    context_text: bool = True,
    last_step_text: bool = True,
    positional_guard: bool = True,
    relative_features: bool = True,
    verify: bool = True,
    verify_adapter: bool = True,
    verify_samples: int = DEFAULT_SAMPLES,
    goldens: int = DEFAULT_GOLDENS,
    audit: bool = True,
    audit_samples: int = 12,
    audit_max_incorrect: float = 0.25,
    usefulness: bool = True,
    prune_harmful: bool = False,
    usefulness_max_decisions: int = DEFAULT_MAX_DECISIONS,
    usefulness_max_rows: int = DEFAULT_MAX_ROWS,
    llm_retries: int = 5,
    request_timeout_s: float | None = None,
)

Settings of a TabSchema discovery run.

Parameters:

Name	Type	Description	Default
model	str | None	the LLM (default: the paper's GPT-5.4 on OpenAI, or the provider whose key is in the environment).	None
n_analyze	int	decision points the FeatureAnalyzer reads.	20
sample_seed	int	seed of that sample.	DEFAULT_SEED
sample_strategy	str	how the sample is drawn (see tabagent.tabschema.sampling): coverage (one per task first, covering the sources, positions in the run, prefix lengths and labels of the onboarding data) or tasks (one per task, the first decision of each run).	'coverage'
max_code_retries	int	CodeExecutor retries with the error fed back (three, App. D.1).	3
json_retries	int	re-asks when an answer is not parseable JSON.	1
accept	tuple[str, ...]	MetaJudge outcomes compiled into the card: accepted and conditionally accepted features by default. ("accept",) keeps only accepted ones. On the next-tool-call decision of public τ²-bench logs, compiling the conditional features too raised head P@R by 0.066 and 0.029 in two discovery runs (10 and 20 analyzed decision points); adding the rejected ones changed it by under 0.004 more.	('accept', 'conditional')
compile_min_success	float	share of onboarding decision points the compiled extractor must run on.	0.95
max_null_rate	float	extractor columns emptier than this are dropped.	0.5
max_columns_per_feature	int	caps on the columns one planned feature expands into	64
max_extractor_columns	int	caps on the columns one planned feature expands into (code that returns a dict per row, such as word counts, gets one column per key) and on the extractor's columns in all; past them the columns that vary most across decision points are kept. On many decision points memory caps the columns other than presence flags further, to max(64, 20,000,000 / decision points) (512 at 39,000 points, 200 at 100,000); presence flags are decided from statistics over every point and only ever capped by these two; a compile note says when the memory cap binds.	512
compile_repair	bool	when the CodeValidator flags accepted features in the compiled code, ask the CodeExtractor once more with its objections (the paper's retry prompt), and keep the new code only if it runs and fewer features are flagged.	True
drop_redundant_text	bool	drop a compiled text column that duplicates another kept text column (the appended context text included) word for word on every onboarding decision point: the same lowercased tokens, numbers and order, so a text and the list of its tokens collapse into one. Overlap, containment or the same words in another order never drop a column. Of duplicates a categorical form is kept over a text one. Within the columns of one planned feature, a column that restates another in a derived form on every row (its tokens, bag of words or n-grams as a JSON list or dict, or the same words in another order) is dropped too: the head represents text itself.	True
dependency_features	tuple[str, ...]	dependency statistics added to the card. co_usage is added when an extractor column lists earlier candidates (co_usage_context).	('candidate_prior', 'context_candidate_lex', 'context_candidate_sim', 'co_usage')
pair_embeddings	bool | None	add the e5 context-candidate similarity; None adds it when the embed extra is installed.	None
workers	int	parallel analyzer calls.	8
timeout_s	float	sandbox time limit per trajectory.	30
max_cost_usd	float | None	LLM budget; needs a price for the model in prices.	None
prices	dict[str, tuple[float, float]]	{model: (USD per 1M input tokens, USD per 1M output tokens)}.	dict()
max_prompt_chars	int	a decision-point prefix longer than this (as JSON) reaches the FeatureAnalyzer with its longest strings clipped.	400000
stage_models	dict[str, str]	a model per stage, overriding model: keys are stage roles (analyzer, extractor, validator, aggregator, meta_judge, judge_relevance, ...) or judges for all three judges. For example {"extractor": "azure/gpt-6-sol"} when a provider's content filter stops one model's code answers. Every model needs a price in prices when max_cost_usd is set.	dict()
fallback_model	str | None	asked once, for any stage, when the provider's content filter stops an answer.	None
context_text	bool	append to the extractor a fixed, non-LLM function that returns the decision's context as text (context_text), so every card has a context text for the context-candidate similarity features, TabSynth and the BM25 baseline, even when the LLM's features are all flags and counts.	True
last_step_text	bool	append to the extractor a fixed, non-LLM function that returns the text of the prefix's last step (last_step_text): the step that handed control to the decision, which the context text can dilute among long payloads. It is a context text too (TabSynth, the context-candidate similarities).	True
positional_guard	bool	for a decision with candidates, ask the CodeExtractor to compute in its per-candidate function the features it wrote as columns that index the candidates by position (compile.positional_columns, decided from the columns' values), and drop those columns once it does.	True
relative_features	bool	let the recipe search (auto.select) try, for a ranking or multi-label decision, each numeric per-candidate feature's and dependency statistic's rank and gap to the best within the decision (card.with_relative) as a challenger, kept only when it scores strictly better in the inner validation. The discovered card never holds them, so the usefulness estimate and a TabAgent built on the card directly do not use them; the saved head records the choice. The release's checks beyond the paper (see docs/guides/verify-generated-code.md):	True
verify	bool	run the compiled extractor, sandboxed, on verify_samples decision points and drop the columns that are nondeterministic or change when outcome fields or the run's future change; report how it copes with short and incomplete prefixes (tabagent.tabschema.verify).	True
verify_samples	int	decision points of those checks (coverage-sampled).	DEFAULT_SAMPLES
verify_adapter	bool	check, before any LLM call, that the adapter's prefixes do not change with the run's future (verify.FutureCheck); a prefix that does stops discovery with PrefixLeakError. Turning it off (or verify) skips the check with a warning; an adapter whose prefixes legitimately hold the decision step's own input (its prompt) declares those keys in decision_input_keys instead.	True
goldens	int	golden tests recorded in extractor_tests.json and saved with the head (0: none).	DEFAULT_GOLDENS
audit	bool	ask the OutputAuditor (stage auditor) whether the extracted values are correct on audit_samples decision points, and drop features judged incorrect on more than audit_max_incorrect of the points (at least 3) where they were judged (tabagent.tabschema.audit).	True
usefulness	bool	estimate each feature's contribution by drop-column grouped cross-validation on the onboarding rows, on at most usefulness_max_decisions decision points (tabagent.tabschema.usefulness); a report only, unless prune_harmful also drops the features whose removal clearly helps. Operations:	True
llm_retries	int	retries of an LLM call that fails transiently (a rate limit, a dropped connection, a server error, an empty answer), with exponential backoff that honors the provider's Retry-After; past them the run stops with LLMUnavailable, resumable from its run directory. A call that times out is retried once, then handled like an answer cut at the output limit.	5
request_timeout_s	float | None	seconds one LLM request may take, for the backends auto_llm builds; None scales it with the stage's output limit: max(300, 120 + max_tokens / 40), room for the longest answer at 40 tokens per second (520 s for the per-decision-point stages' 16,000 tokens, 920 s for the whole-run stages' 32,000). Every field is checked when the config is made (types and ranges), with a message naming the field; from_dict builds one from a mapping (a JSON or YAML file) and names unknown keys.	None

verify_head

verify_head(path: str | Path, sandbox: Any = None, require_goldens: bool = False) -> VerifyReport

Check a head saved by onboard before deploying it: the model artifact matches its manifest, the card pins the hash of extract.py and the code matches it, the tests in extractor_tests.json cover the card's extractor features, and re-running them in the sandbox ("subprocess", "docker", "docker:<image>" or a sandbox instance) reproduces every recorded feature, twice, with the outcome fields making no difference. A head without golden prefixes passes with a warning unless require_goldens. path is a directory or hf://OWNER/REPO[@REVISION]. A path that is not a head, or an invalid tests file, raises ValueError. CLI: tabagent verify-head PATH (exit status 1 when a check fails, 2 on such an error).