Is this text or file AI-generated? A research tool that answers in plain English.
Checks whether a piece of text or a file (image / video / audio / PDF) carries one of the known "watermarks" that AI companies hide in their output - and tells you the answer in plain English. Six text-watermark schemes reproduced bit-for-bit against their official papers, plus forensic C2PA file-provenance verification.
The core is a single Rust binary. First install Rust if cargo --version doesn't already
work.
On macOS/Linux, run the rustup bootstrap:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | shOn Windows, download and run rustup-init.exe, then reopen your
shell so cargo is on PATH (in PowerShell you can install it via winget):
winget install --id Rustlang.Rustup -eThen build the release binary:
cargo build --releaseThe binary lands at target/release/ai-watermark-detector (ai-watermark-detector.exe on
Windows). Run it with ./target/release/ai-watermark-detector --help
(.\target\release\ai-watermark-detector.exe --help in PowerShell).
To put it on your PATH, either cargo install --path . or, on macOS/Linux, make install
(copies it to ~/.local/bin). Then:
ai-watermark-detector --help
make demo # build a demo corpus and score watermarked vs. human textRun it on some text and you get a verdict like this:
=== AI watermark screen: WATERMARK SIGNAL FOUND ===
This text carries the 'kgw' statistical watermark for the key you supplied.
Chance a random unwatermarked text would score this high: about less than 1 in a trillion.
or, on a file:
file: photo.jpg
plain answer: AI-generated, and the provenance is cryptographically TRUSTED (signer verified).
Three possible text verdicts, in words:
- WATERMARK SIGNAL FOUND - the text matches the watermark for the key you gave it.
- NO WATERMARK SIGNAL - no trace of that watermark. (This does not prove a human wrote it - only that this specific watermark+key isn't present.)
- TOO SHORT TO TELL - there isn't enough text to make a reliable statistical call (you need roughly 100+ words).
There are two completely different kinds of AI watermark, and they behave differently:
| Text watermarks | File watermarks (C2PA) | |
|---|---|---|
| What it is | An invisible statistical bias in word choice | A signed receipt attached to the file |
| Can you check it yourself? | Only if you have the secret key | Yes - verified with a public key |
| Who has the key? | The AI vendor keeps it private, forever | Anyone (it's public-key crypto, like HTTPS) |
| This tool's role | Research reproduction of the mechanism | A real, forensic check on real files |
Why the secret key matters: a text watermark leaves no visible mark. At each word the AI used a secret number (the "key") to nudge its choices. The only trace is a faint bias spread across many words. To measure that bias you must reproduce the exact same keyed nudge - which requires the key. Without the key there is literally nothing to detect; watermarked text is mathematically identical to unwatermarked text. That's a deliberate cryptographic design choice, not a limitation we can code around. It's why this tool cannot tell you "this ChatGPT/Claude/Gemini paragraph is AI" from raw text alone. It can prove the watermarking mechanism works, on text you watermark yourself with a known key - which is what real watermarking research does.
Files are different: C2PA is a signed receipt (like the padlock in your browser), so this tool really does verify it and tell you TRUSTED / UNTRUSTED / TAMPERED.
| Command | What it does | Needs a key? | Needs c2patool? |
|---|---|---|---|
score |
Scores token IDs against a text watermark (kgw/synthid/exp/unigram/sweet/exp-edit) | Yes (the vendor's, or your own) | No |
check |
Verifies C2PA provenance on files/folders/URLs, one verdict per file | No | Yes |
scan |
Same check, aggregated into a per-format coverage table | No | Yes |
commits |
Scans a git repo's history for AI coding-agent markers (Copilot/Cursor/Devin/...) | No | No (uses git) |
Run ai-watermark-detector <command> --help for the full argument list; a compact reference
is in CLI reference below.
# 1. Build the one binary (needs Rust: https://rustup.rs)
cargo build --release
# 2. Make a tiny demo corpus watermarked with a KNOWN key, then score it
./target/release/gen_corpus /tmp/demo kgw
./target/release/ai-watermark-detector score \
--config config.example.json --scheme kgw --token-file /tmp/demo/watermarked/sample_00.txt
# -> WATERMARK SIGNAL FOUND
# 3. Score an unwatermarked sample for contrast
./target/release/ai-watermark-detector score \
--config config.example.json --scheme kgw --token-file /tmp/demo/human/sample_00.txt
# -> NO WATERMARK SIGNAL
# 4. Check a file's provenance (needs c2patool; see Setup)
./target/release/ai-watermark-detector check photo.jpgOn Windows, use target\release\ai-watermark-detector.exe, a path like
%TEMP%\demo instead of /tmp/demo, and backslash path separators (or run the commands
verbatim from Git Bash / WSL).
Text - six statistical schemes from the two big research families. We reproduce each one bit-for-bit against its official published code, so you can prove the mechanism on text you watermark yourself:
Scheme (--scheme) |
Family | Used by / origin | One-line description |
|---|---|---|---|
kgw |
green-list | matches Anthropic's description of Claude | Splits the vocabulary into "preferred/not" each step and leans toward preferred words. |
synthid |
sampling | the family Gemini uses | Google's "tournament" g-value scheme (Nature 2024); we match the strong weighted detector. |
exp |
sampling | Aaronson / OpenAI-style | Gumbel "exponential" trick; per-word score -ln(1-u). |
unigram |
green-list | Zhao et al. 2024 | One fixed global split - the most edit-robust of the green-list schemes. |
sweet |
green-list | Lee et al. 2024 | KGW but only on high-information words; the standard code watermark. |
exp-edit |
sampling | Kuditipudi et al. 2024 | Edit-distance statistic that survives insertions/deletions others don't. |
Files - real forensic provenance via C2PA on jpg/png/mp4/wav/pdf: reads the
signed manifest, validates the signing certificate against a trust list, flags AI-generation
markers and Google SynthID assertions, and detects tampering. This is the only check
here that gives an authoritative answer on real vendor output (e.g. Google Imagen/Veo
images that ship with C2PA).
Images (pixel-domain), contributor tool - tools/image_watermark.py does Stable
Signature (Meta, forensic - real 48-bit extractor + exact p-value) and a labelled
SynthID-Image heuristic (no public decoder exists, so it's clearly marked not
forensic). See the pixel-domain section below.
- AI text from a real product (Claude / Gemini / ChatGPT): No - the vendor's key is private (and ChatGPT/Grok ship no text watermark at all). No public tool can do this from raw text; that's the cryptographic wall described above.
- AI files with C2PA (many Google/Adobe/Microsoft pipelines): Often yes -
checkgives a real forensic verdict. - Any watermark where you have the key: Yes, exactly - that's the research use, and if a vendor ever publishes a key you can plug it straight in (see "If a key becomes public").
You only need Rust to run the shipped detector. Python is contributor-only (for proving the Rust engine matches the official reference code).
Install Rust from https://rustup.rs, then:
cargo build --release
./target/release/ai-watermark-detector --helpThe binary is self-contained: no Python, no pip. Works on macOS, Linux, and Windows
(the same cargo build --release; on Windows the binary is
target\release\ai-watermark-detector.exe).
| OS | Install |
|---|---|
| macOS | brew install c2patool |
| Linux | cargo install c2patool |
| Windows | cargo install c2patool (or download c2patool.exe from Releases) |
The detector finds both c2patool and c2patool.exe on your PATH. If it's missing, the
tool fails closed with the exact install command for your platform. The text score
command never needs it.
Use a virtualenv. On macOS/Linux:
python3 -m venv .venv
. .venv/bin/activate
pip install -r tools/requirements.txtOn Windows (PowerShell):
python3 -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r tools\requirements.txttools/requirements.txt covers all contributor tools (validation, images, tokenizer).
Every Python tool starts with a #!/usr/bin/env python3 shebang and fails closed with a
clear pip install -r ... message if a dependency is missing - it never crashes with a raw
ImportError.
Simple rule: if you run it to get an answer, it's Rust. If it proves the Rust engine is correct, it's Python.
| You want to… | Run | Language | Needs |
|---|---|---|---|
| Score text for a watermark | ai-watermark-detector score |
Rust | just the binary |
| Check a file's provenance | ai-watermark-detector check / scan |
Rust | binary + c2patool |
| Prove the engine matches official code | tools/validate.py |
Python | tools/requirements.txt |
| Measure attack robustness / ROC | tools/attacks.py, tools/power_analysis.py |
Python | tools/requirements.txt |
| Detect image watermarks | tools/image_watermark.py |
Python | tools/requirements.txt |
| Turn text into token IDs | tools/tokenize_text.py |
Python | tools/requirements.txt |
Everything under tools/ is Python and contributor-only; everything you invoke as
./target/release/... is Rust and is the shipped product.
Why two languages? The detector ships in Rust so it's a single fast, dependency-free
binary you can curl | install and run offline. The tools/ are Python because their job
is to diff our detector against the official reference implementations - Hugging Face
transformers, MarkLLM, and Meta's Stable Signature model - which exist only as
Python/PyTorch. Reimplementing those in Rust would mean checking our code against our own
code, which proves nothing; running the vendors' actual Python is what makes the validation
trustworthy. Users never touch Python; contributors use it to prove the Rust is correct.
Both Rust subcommands print human-readable text by default (verdict first, then
details) and can emit machine-readable output with --format json|xml|yaml:
# human (default): verdict-first, plain English
./target/release/ai-watermark-detector score --config config.example.json --scheme kgw --token-file ids.txt
# machine-readable: pick json, xml, or yaml for scripts / pipelines
./target/release/ai-watermark-detector score --config config.example.json --scheme kgw --token-file ids.txt --format json
./target/release/ai-watermark-detector score --config config.example.json --scheme kgw --token-file ids.txt --format yaml
./target/release/ai-watermark-detector score --config config.example.json --scheme kgw --token-file ids.txt --format xml(--json is still accepted as a shortcut for --format json.) See
CLI reference for the full list of fields each command emits.
This tool becomes an exact scorer the moment the scheme, key, and tokenizer all match
the text - the detectors are reproduced bit-for-bit against the official implementations
(validated in the correctness suite and tools/validate.py). You already get this on a
corpus you watermark yourself, since you own the key. If a vendor ever publishes their
production key, you plug it straight into the config:
{ "ngram_len": 5, "keys": [<the published key ints>], "gamma": 0.25, "vocab_size": <vocab> }then tokenize with the vendor's exact tokenizer and score:
python3 tools/tokenize_text.py --hf <vendor/tokenizer> --file suspect.txt > ids.txt
./target/release/ai-watermark-detector score --config config.example.json --scheme <kgw|synthid|...> --token-file ids.txtHonest caveat: "100% accurate" means the statistic is computed exactly (bit-for-bit with the vendor's math). Detection is still statistical - long watermarked text gives astronomically small p-values (effectively certain), but very short or heavily-edited text can stay ambiguous no matter what. For files, C2PA is already a today-verifiable public-key check - no waiting on any key.
cargo build --release
./target/release/ai-watermark-detector --helpThe CLI has three subcommands: score (text watermark), check (file provenance),
scan (per-format coverage).
Every command prints human-readable text by default and accepts --format text|json|xml|yaml
for machine-readable output (--json is a shortcut for --format json).
score - score token IDs against a text watermark:
| Argument | Description |
|---|---|
--tokens <IDS> |
Comma/space-separated token IDs (mutually exclusive with --token-file). |
--token-file <PATH> |
File of whitespace/comma-separated token IDs. |
--config <PATH> |
JSON config: ngram_len, keys, and scheme extras (see Config). Required. |
--scheme <NAME> |
synthid (default), kgw, exp, unigram, sweet, or exp-edit. |
--threshold <F> |
Minimum confidence for a local "positive" (default 0.95). |
--format <FMT> |
text (default), json, xml, yaml. |
check - verify C2PA provenance for files, folders, or URLs:
| Argument | Description |
|---|---|
<FILES>... |
Files, folders (walked recursively), or http(s):// URLs. |
--fetch-samples |
Download real publicly-signed sample files and check them. |
--samples-dir <DIR> |
Where fetched samples are stored (default samples). |
--trust-anchors <PEM> |
PEM trust-anchor list (path or URL) for real cert-chain validation. |
--format <FMT> |
text (default), json, xml, yaml. |
scan - same check, aggregated into a per-format coverage table:
| Argument | Description |
|---|---|
<FILES>... |
Files, folders (walked recursively), or http(s):// URLs. |
--trust-anchors <PEM> |
PEM trust-anchor list (path or URL) for cert-chain validation. |
--format <FMT> |
text (default), json, xml, yaml. |
commits - scan a git repo's history for AI coding-agent provenance:
| Argument | Description |
|---|---|
<REPO> |
Path to the git repository (default .). |
--limit <N> |
How many commits back to inspect (default 200). |
--all |
List every commit, not just AI-authored ones. |
--format <FMT> |
text (default), json, xml, yaml. |
This reads the commit author, committer, and message trailers for self-declared agent markers
(Co-authored-by: Copilot, the Agent-Logs-Url trailer, copilot-swe-agent[bot], plus Cursor,
Devin, OpenAI Codex, Aider, Google Jules, Gemini, Windsurf, Amazon Q, Cline, Continue, Amp,
OpenCode, and Claude Code identities, and the generic Generated-By trailer). A match is HIGH
confidence because the tooling writes it, but absence is not proof of human authorship. It is
deterministic: no ML, just git log.
Source files themselves carry no signed provenance, but if an author adds an
SPDX-AI-Disclosure tag or a tool header (@cursor-generated,
Generated by GitHub Copilot) in a top-of-file comment, check/scan surface it as a
source-disclosure signal. Unlike commit trailers, these survive rebases and squashes.
The score result carries scheme, tokens, positions, usable_positions,
green_or_ones, total, mean_g, weighted_mean_g (SynthID only), grounded_p_value
(schemes with a closed-form null), z, approx_p_value, reliable, screen_positive, and
a warning. check/scan emit a results list of per-file verdict, has_manifest,
ai_source_type, algorithmic_source, synthid_assertion, soft_binding, soft_binding_kind,
assertions, status, and signals (each with confidence, source, detail, and an
optional tool).
The detector consumes token IDs, not words (see "Why token IDs?" below). Pick the
scheme with --scheme:
# KGW green-list scheme (Kirchenbauer et al.)
./target/release/ai-watermark-detector score \
--config config.example.json --scheme kgw --token-file tokens.txt
# Gemini-style SynthID scheme (default)
./target/release/ai-watermark-detector score \
--config config.example.json --scheme synthid --token-file tokens.txt --jsonconfig.example.json carries settings for both schemes:
{
"ngram_len": 5,
"keys": [654, 400, 836, ...],
"gamma": 0.25,
"vocab_size": 8000
}ngram_len,keys- used by both schemes.gamma- KGW green-list fraction (ignored by SynthID).vocab_size- KGW vocabulary size for the green/red split (ignored by SynthID; if omitted it is inferred from the maximum observed token id).sampling_table(optional, SynthID) - the Bernoulli table exported from Google's official processor. When present, the SynthID scheme matches Google bit-for-bit; when absent, a standalone reproduction is used. See real-world validation A below.green_oracle(optional, KGW) - alast_token -> green_token_idsmap exported from HF's reallefthashprocessor. When present, the KGW scheme matches HF bit-for-bit; when absent, a standalone reproduction is used.
Both schemes hash token sequences, not visible words. The same visible text produces different token sequences under different tokenizers, so a text-only detector cannot reproduce a watermark score without the exact tokenizer.
0 comments
log in to comment.