A JIT-compiling jq in Rust using Cranelift — up to ~100x faster, passing 100% of the official jq test suite (509/509).

The author can't read or write Rust — essentially every line was written by an AI. See How this was built.

• Targets full jq language compatibility — passes the official jq test suite in full (509/509); remaining divergences from jq are treated as bugs, aside from a few deliberate tradeoffs (see Known limitations)
• JIT compilation via Cranelift for hot execution paths
• Raw byte fast paths — 70+ filter shapes operate directly on raw bytes, skipping JSON parsing entirely; 180+ shapes in total route to specialized fast paths
• Streaming JSON parser for memory-efficient NDJSON processing
• Parallel per-record execution — --parallel shards a stateless filter over an NDJSON stream across a worker pool, byte-identically to sequential mode (see Parallel execution)
• Memory-mapped file I/O — mmap-based file reading with no upfront allocation
• Optimized value representation with compact strings, mimalloc, and inline Cranelift codegen
• jqx extensions — shell command execution (exec/execv), CSV/TSV parsing (fromcsv/fromcsvh/fromtsv/fromtsvh), function-result memoization (memoize), and the in-place path-update contract (mutate)

A few divergences from jq are deliberate engineering tradeoffs rather than bugs:

• Number representation — values are IEEE 754 doubles throughout (there is no decimal-number backend). A number that is only carried and printed keeps its original literal, so 1e1000, 13911860366432393 and friends survive tostring / tojson / fromjson round-trips and the @csv / @tsv / @text formatters exactly as jq's decimal build prints them. The moment a value is computed — arithmetic, or a comparison — it collapses to the nearest double, so 1e1000 + 0 saturates and 13911860366432393 == 13911860366432392 is true where jq's optional decimal build (have_decnum) says false. have_decnum therefore reports false: it is the honest answer for the comparison and arithmetic side.
• Regex anchors — the regex engine is the linear-time Rust regex crate (no lookaround or backtracking, which is what keeps matching ReDoS-resistant). As a result $ matches only end-of-text, not also just before a trailing newline as jq's Oniguruma does; ^ is likewise start-of-text only.

On a 2M-line NDJSON file (typical ETL/data pipeline workload):

Measured on an Apple M4 Max (macOS 26.4.1, arm64), jq 1.7.1 and jaq 3.0.0, best of 3 runs over 2M NDJSON objects. Speedups are machine-dependent — jaq is itself a fast Rust reimplementation of jq, and jq-jit stays 15–167x ahead of it across these filters. For per-version results across the full filter suite, see docs/benchmark-history.md. Run bash bench/comprehensive.sh to reproduce on your own hardware.

The author cannot read or write Rust. The entire architecture and essentially all of the code were written by Claude (Anthropic) via Claude Code — the libjq-bytecode backend, the choice of Cranelift, every optimization. The author made no code or design decisions; aside from a couple of small community fixes, every line is Claude's.

The human's role was twofold: setting the goal, and — once the tool first reached a working state — keeping a bug-hunting loop running with high-level directives ("go find bugs and fix them," "keep going"). Claude did the work on both ends: discovering the bugs (building its own test infrastructure, using Project Euler problems as a detector) and fixing them. The human never read the implementation and never pointed to a specific bug. How far that goes is the experiment: a real-world, performance-critical tool maintained for months by someone who never sees the code. See docs/development-story.md for the full account, including the (small) number of messages the human actually sent.

Pre-built binaries from the latest release are available via a personal tap:

brew install m5d215/tap/jq-jit

The Homebrew tap covers macOS arm64 and Linux x86_64. Windows x86_64 ships as a .zip on the releases page (see below); for any other platform, build from source.

Download the archive for your platform from the releases page and extract the binary onto your PATH. macOS / Linux ship as .tar.gz, Windows as .zip (jq-jit.exe).

• Rust toolchain (edition 2021)

jq-jit has no runtime C dependencies: parsing, evaluation, and JIT codegen are all pure Rust. (Earlier versions linked against libjq and libonig; those dependencies were removed as of 1.3.0.)

cargo build --release

The binary is output to target/release/jq-jit.

jq-jit [OPTIONS] <FILTER> [FILE...]

-a / --ascii-output and --stream are accepted but not yet implemented (tracked in #126); jq-jit exits with an explicit error rather than falling through silently.

# Identity
echo '{"name": "jq"}' | jq-jit '.'

# Field access
echo '{"name": "jq", "version": 1}' | jq-jit '.name'

# Process NDJSON file
jq-jit 'select(.age > 30)' data.jsonl

# Multiple filters
echo '[1,2,3]' | jq-jit 'map(. * 2)'

# Using variables
jq-jit --arg name "test" '.[$name]' data.json

# Positional arguments
jq-jit -n '$ARGS.positional' --args foo bar baz

jq-jit ships with a small set of extensions beyond standard jq, collectively referred to as jqx. They are part of the default build on main (no separate branch or feature flag).

When input is non-null, it is passed to the command's stdin (strings as-is, other values JSON-encoded). Trailing newlines are trimmed.

# Run a command and use its output
jq-jit -n 'exec("git rev-parse @") | "commit: \(.[:7])"'

# Pipe input to a command
echo '"hello"' | jq-jit 'exec("tr a-z A-Z")'  # → "HELLO"

# Stream a generator through a single process
jq-jit -n 'exec(range(0;10); "sed s/^/+/")'

# Capture exit code and stderr
jq-jit -n 'execv("ls /nope") | if .exitcode != 0 then .stderr else .stdout end'

All values are returned as strings. Parsing is RFC 4180 compliant (handles quoted fields, escaped quotes, commas/newlines within fields).

# Parse CSV into arrays
echo 'name,age\nAlice,30' | jq-jit -R 'fromcsv'

# Parse CSV file with headers into objects
jq-jit -Rsc 'fromcsvh' < data.csv

# Use custom headers
jq-jit -Rsc 'fromcsvh(["name","age"])' < no-header.csv

# Combine with exec
jq-jit -n 'exec("cat data.csv") | fromcsvh | select(.age | tonumber > 25)'

The wrapped expression must be exactly one path-update operator at the leaf — composite forms (if/then/else, reduce, ,, |) must distribute the marker inward by wrapping each leaf separately. Anything else is a parse error.

# Reduce accumulator stays in-place across iterations
jq-jit -n '[range(0; 1000) | 0] | reduce range(0; 1000) as $i (.; mutate(.[$i] = $i*$i))'

# Operator-assign forms work the same
echo '{"n":0}' | jq-jit 'mutate(.n += 1)'

mutate(...) is a contract, not a performance switch. The runtime check is strict: refcount = 1 → in-place mutation, refcount > 1 → silent copy-on-write fallback. The result is always equal to the unmarked form.

Since v1.8.0, the JIT's static escape analysis (#697, #699, #700) engages the in-place fast path on the unmarked form for most shapes encountered in practice — including the original P126 / P135 motivating cases. In those shapes, mutate(...) and the unmarked form produce identical wall-clock times. The contract remains useful as a documentation marker, a regression guard against future JIT changes, and an observability anchor for --trace-mutate.

Use --trace-mutate to verify per-invocation whether the in-place path engaged:

jq-jit --trace-mutate -n '[1,2,3] | mutate(.[0] = 99)'
# stderr: [trace:mutate] kind=assign container=array refcount=1 mode=in_place

As of v1.8.0, trace events are only emitted through the interpreter path (e.g. --force-interp); JIT-mode tracing is tracked in #702.

mutate is a jq-jit extension; stock jq does not provide it. See issue #666 for the motivating benchmarks and the persistent-vector follow-up tracked in #695.

Each lexical occurrence of memoize(...) gets its own cache; entries persist for the lifetime of the program (across NDJSON input records). Keys compare by structural equality, matching jq's == semantics (objects are order-independent, arrays element-wise). The body is run to completion on first call — multi-output generators are materialized so subsequent calls re-yield the same sequence.

Self-recursive memoization Just Works: jq's def binds the same name inside the body, so recursive calls go through the memoized wrapper:

# Fibonacci — exponential without memo, linear with it
jq-jit -n 'def fib: memoize(if . < 2 then . else ((. - 1) | fib) + ((. - 2) | fib) end); 80 | fib'

# Collatz chain length — subgraph revisits become O(1)
jq-jit -n 'def collatz: memoize(if . == 1 then 0 else (if . % 2 == 0 then ./2 else 3*. + 1 end | collatz) + 1 end); 27 | collatz'

# 2-arg form: memoize a transformation by record id, ignoring the rest
jq-jit -c 'memoize(.value * 2; .id)' <<< '{"id":1,"value":10}'

Eviction defaults to "unbounded for program lifetime" up to a per-slot cap of 1,000,000 entries; control it with --memo-max-entries N on the CLI. Past the cap, new inserts are silently dropped (the program continues, just without caching new entries). Body errors do not poison the cache — the next call re-evaluates.

Run with --debug-memo to print per-slot cache stats (hits / misses / entries) to stderr at program exit. Useful for confirming a memoize(...) is actually hitting:

jq-jit --debug-memo -n 'def fib: memoize(if . < 2 then . else ((. - 1) | fib) + ((. - 2) | fib) end); 30 | fib'
# memoize stats: 1 slot(s)
#   slot          hits        misses       entries
#      0            28            31            31

The 1-arg form keys only by the current input. If your body closes over a $var that varies between calls, results will be stale — pull the var into the key explicitly with the 2-arg form: memoize(. + $x; [., $x]).

--jsonc lets the JSON input parser accept // line comments and /* */ block comments, for running filters directly over JSONC-style config files (tsconfig.json, editor settings, .jsonc) without a comment-stripping preprocessing step:

jq-jit --jsonc '.compilerOptions.strict' tsconfig.json

Scope is deliberately comments only — trailing commas, single-quoted strings, unquoted keys, and the rest of JSON5 remain parse errors. Comments are stripped once at the input boundary (string-aware, so "http://x" is untouched); with the flag off, the input path is byte-identical to before.

Notes:

• Applies to top-level JSON input (stdin and input files, including the input / inputs builtins). fromjson, --argjson, and --slurpfile stay strict JSON.
• Not combinable with --unbuffered streaming JSON stdin (the incremental reader never holds the whole buffer); --jsonc --unbuffered with file inputs works normally.

For a stateless filter over an NDJSON document stream, every input record is independent: there is nothing to share between records, so they can run on a worker pool with the output reassembled in record order. --parallel[=N] does exactly that — something single-threaded jq structurally cannot.

# Spread the filter across all cores
jq-jit -c --parallel 'select(.level == "error") | {ts, msg}' < events.ndjson

# Pin the worker count
jq-jit -c --parallel=4 '{id, total: (.items | map(.qty * .price) | add)}' < orders.ndjson

Output is byte-identical to sequential mode: each worker runs the same compiled filter and serializer, and a single writer drains a reorder buffer in stream order. Per-record errors still surface in record order, and -e still reflects the last output value across the whole stream.

--parallel only engages when the filter is provably independent per record. Anything that observes cross-record state, stream position, shared mutable state, side effects, or non-determinism transparently falls back to sequential execution — including input/inputs, limit/first over the input stream, input_line_number, $__loc__, memoize, mutate, exec/execv, debug/stderr, halt/halt_error, and now. The check follows user-defined function calls, so a stream read hidden behind a def is still detected. It also only applies to plain buffered compact/pretty output (not -r/-S/-j/-C/--seq/--unbuffered or -n null-input); other modes run sequentially. Tiny inputs run inline regardless, since thread setup would dominate.

The speedup tracks the per-record work: light projections are bounded by the serial input scan and output write (Amdahl), while heavier per-record filters scale close to the worker count.