A provenance & contamination firewall for scientific ML.

It answers the question a reviewer, clinician, or regulator actually asks — "can I trust this number?" — by catching the two things that silently corrupt results, and then signing the verdict so it can't be quietly altered:

1. Data leakage — is the same information in both the training and test set (so the reported accuracy is fake)?
2. Label provenance — where did the "ground truth" come from? Was it attested by a human, or generated by another model (so you may be measuring a model against itself)?

Output: a tamper-evident, Ed25519-signed manifest of the dataset's lineage and integrity verdict — a reproducibility certificate you can attach to a paper, a submission, or a model card.

Proof at a glance — runs in one command · 18 passing tests · proven on real public data: on Pima Indians Diabetes a leaky split reports AUC 0.99 while the honest AUC is 0.82 — CleanRoom catches the gap and signs the verdict (tamper the manifest and verification fails). Try it: python -m lineage_cleanroom demo · see demos/.

Built for Built with Claude: Life Sciences (Anthropic + Gladstone Institutes). Lineage = data lineage + cell lineage; cleanroom = the contamination-control environment of every lab.

A computational biologist in a genomic-immunology lab — the kind of lab Alex Marson runs. She runs pooled CRISPR / Perturb-seq screens in primary human T cells and trains models to prioritize hits and predict cell states. Before she trusts an AUC or puts a figure in a paper, she needs to know her train/test split isn't leaking (the same donor or batch on both sides) and that her gold labels were attested by a human, not imputed by another model. Today she checks this by hand, if she remembers — and the leak is usually invisible until a reviewer, or a failed replication, finds it.

Lineage CleanRoom is the check she's missing: she points it at her dataset and gets a signed verdict in seconds — working software she can run without the author in the room. It is additive to Claude Science, which records an artifact's history but does not prevent leakage or certify label provenance.

Data leakage is not a corner case. A 2023 survey found it affecting 294 papers across 17 scientific fields, cataloguing eight leakage types from textbook errors to open research problems (Kapoor & Narayanan, Leakage and the reproducibility crisis in ML-based science, Patterns 2023 — paper). Lineage CleanRoom targets the tractable, detectable subset (exact-duplicate and group leakage); see Known limitations for the types it deliberately does not claim to catch.

pip install -r requirements.txt

# audit a real dataset split ("I already split my data — is it leaking?")
python -m lineage_cleanroom scan \
  --data dataset.csv --label activity --split-col split \
  --group donor --provenance label_source --out ./audit

# audit a single-cell .h5ad (AnnData) split — column roles stay yours to name
pip install -r requirements-h5ad.txt          # optional extra (anndata)
python -m lineage_cleanroom scan \
  --data screen.h5ad --label state --split-col split \
  --group donor --provenance label_source --out ./audit

# demonstration: catch-the-leak on synthetic data
python -m lineage_cleanroom demo

# tests
python -m pytest lineage_cleanroom/test_cleanroom.py -q

Lineage CleanRoom -- CONTAMINATION [FAIL]
CONTAMINATION DETECTED - LEAKAGE - 63.5% of test rows overlap training
signed manifest verifies: True

The demo shows the payoff: a naive predictor reports AUC 0.9995; CleanRoom finds the leak; the honest, group-aware AUC is 0.65; and the signed manifest rejects any tampering with the numbers.

Not just synthetic. On the Pima Indians Diabetes dataset (real, OpenML), patient-level leakage inflates AUC 0.9865 → 0.8246 honest (+0.16); oversample-before-split leaks 193 exact-duplicate rows; and on Breast Cancer Wisconsin the provenance gate catches 51 model-generated evaluation labels. Each demo commits a signed manifest as evidence and is CI-verified. See demos/.

• Leakage gate (features) — exact feature-duplicate rows across the split + groups (donor / batch / genomic region) that span train and test.
• Provenance gate (label origin) — classifies each label's source (human / model:<name> / heuristic / unknown), enforces that gold/eval labels are human-attested only, and can forbid model-generated labels in training (anti-autophagy).

It certifies the label's origin, not its correctness — the trust root is the human attestation, recorded and cryptographically signed so it is tamper-evident and travels with the data.

Modular (SOLID), low-coupling, dependency-injected core, observability via an injected telemetry sink. See lineage_cleanroom/ — ingest · leakage · provenance · manifest · report · pipeline · cli · telemetry. Deterministic and offline; the audit needs no model training. Format adapters map any source to the same SplitView contract — CSV in the core, and single-cell .h5ad (AnnData) via lineage_cleanroom/adapters/ behind an optional, lazily-imported dependency, so the core stays dependency-light.

The engine is domain-agnostic — it operates on any (features, label, group, provenance) table and knows nothing about biology. The same leakage plagues ~17 fields (Kapoor & Narayanan); only the group changes: patient / donor (medicine), subject (neuroimaging), site or sensor (remote sensing), molecular scaffold (cheminformatics), author (NLP), time window (finance). Each field also has its own harder leakage (temporal, spatial, scaffold) this tool does not yet claim to catch — see below.

Honest about what it does not do (see DECISIONS.md for the reasoning):

Detects: exact feature-duplicate rows across the split; group-spanning leakage (donor / batch / region) when a group column is provided; non-human-attested gold labels.

Does not (yet) detect:

• Near-duplicate / augmentation leakage without a group column. We investigated distance-based detection and found it cannot separate moderate augmentation from legitimately-similar samples without false positives (DEC-5). Robust handling needs provenance tracking of the augmentation, not raw distance.
• Temporal leakage, preprocessing-before-split leakage, and target leakage (a feature that encodes the label).
• Anything you don't declare: group leakage needs a group column; provenance needs a provenance column. Garbage in → not audited (the tool says so rather than pretending).

Roadmap: the single-cell .h5ad (AnnData) adapter has shipped (optional dependency); next are more format adapters (Parquet, FASTA, PPI edge-list — PPI needs its own graph-leakage detector, not just a loader), provenance-based augmentation lineage, a one-screen HTML report, and — the real test — running on live lab datasets. Longer term, if adopted, the natural shape is a pre-flight gate in the pipeline / CI plus closing the harder leakage types (temporal, preprocessing, target) catalogued as open research problems.

This project makes no novel technical claim — and says so on purpose.

• Signed provenance / attestation of ML artifacts is mature: Sigstore model-transparency (Google / OpenSSF), SLSA, and in-toto attestations do this rigorously, with transparency logs. Our Ed25519 manifest is a lightweight, dependency-light, self-contained instance for a single audit — for production supply-chain trust, use Sigstore.
• Leakage / data-integrity checks exist in deepchecks and cleanlab; group-aware splitting ships in scikit-learn (GroupKFold).

What Lineage CleanRoom contributes is integration and convenience for the scientific workflow: one command that runs a leakage gate and a human-attested-gold provenance gate and emits a self-contained signed verdict — not a new algorithm. Acknowledging the prior art openly is itself in the spirit of the tool.

CHANGELOG.md · DECISIONS.md — a provenance tool keeps its own lineage of decisions.

MIT — see LICENSE.