Reference

Datasets¶

Each public benchmark rebuilt from its pinned upstream release, with the paper's card and folds.

Nothing is redistributed: load rebuilds each table from the upstream release at a pinned revision, verified by SHA-256. Every dataset, its source, license and the paper result it backs is on Public datasets.

from tabagent import datasets

datasets.available()  # what can be built, with licenses and paper rows
table = datasets.load("skillsbench/skill-select", download=True)
card = datasets.card("skillsbench/skill-select")  # the card behind the paper's number
folds = datasets.folds("skillsbench/skill-select")  # the paper's task-level fold assignment

datasets ¶

Public benchmark datasets as TabAgent decision tables.

from tabagent import datasets
datasets.available()                                    # what can be built, with licenses and paper rows
table = datasets.load("skillsbench/skill-select", download=True)
card = datasets.card("skillsbench/skill-select")         # the card behind the paper's number
folds = datasets.folds("skillsbench/skill-select")       # the paper's task-level fold assignment

Nothing is redistributed: load rebuilds each table from the upstream release at a pinned revision (verified by SHA-256) and caches the result under $TABAGENT_CACHE. Two sources need a step from the user: LLMRouterBench (MIT per its README; accept_license=True confirms that your use complies with the upstream terms) and AppWorld (obtained only through the official appworld download data command).

available ¶

available() -> DataFrame

Every registered dataset, one row each: its decision kind and description, benchmark, license, access (DatasetInfo.access), redistribution status and the paper result it backs.

names ¶

names() -> list[str]

The registered dataset names.

info ¶

info(name: str) -> DatasetInfo

The DatasetInfo of name; KeyError for an unknown name.

load ¶

load(
    name: str,
    *,
    download: bool = False,
    accept_license: bool = False,
    rebuild: bool = False,
    **build_kwargs,
) -> DecisionTable

Build (or read from cache) the decision table name, ready for its bundled card.

download fetches missing source files (each verified against the pinned SHA-256); accept_license confirms the upstream terms of datasets that ask for it; rebuild ignores a cached table. build_kwargs go to the dataset's builder (for example root= for AppWorld's data directory).

card ¶

card(name: str) -> FeatureCard

The bundled feature card of dataset name: the card behind its paper result.

folds ¶

folds(name: str) -> dict

The pinned {group: fold} assignment of the paper's cross-validation for dataset name.