Push-to-talk dictation for Hyprland, sway, i3 and other Linux desktops using Confucius4-R2T2 streaming ASR. Press a hotkey, speak, press again — the text (with punctuation) is typed into whatever window has focus.

CTRL+space ──▶ 🎤 speak ──▶ CTRL+space ──▶ "What time is it? It works." typed into focus

• Runs 100% locally. The model lives on your own GPU — no cloud APIs, no accounts, no API keys, no telemetry. Audio never leaves the machine, and it keeps working with the network unplugged (the installer downloads open model weights once, everything after that is offline).
• Really fast. The server stays warm in memory (vLLM), so dictation starts the instant you press the key, streams at ~0.5s chunk latency while you talk, and the punctuated text lands in your window ~1-1.5s after you press stop. No round-trips to a datacenter, ever.
• Open source. MIT-licensed client and installer; the bundled server changes are Apache-2.0 with every modification documented (server/NOTICE). No paid tiers, nothing phoning home.

🤖 LLM agent? Read llms.txt instead — the whole project as a dense machine-readable reference.

Why R2T2? · How it works · Requirements · Install · Configure · Window manager support · Use · Files · Troubleshooting · Updating · Server patches · Acknowledgments · License

Dictation lives or dies on recognition quality, and R2T2 is the strongest ASR engine we found that runs entirely on a consumer GPU. From upstream's README: "Experimental results show that R2T2 achieves state-of-the-art (SOTA) performance in both latency and recognition quality among a range of open-source models, while remaining competitive with leading closed-source systems."

The evidence behind that claim, from their benchmarks at 160 ms chunks:

• Near-offline accuracy. On LibriSpeech-clean, R2T2 scores 2.13 WER vs 1.67 for the same model decoded fully offline — and naively streaming its Qwen3-ASR base collapses to 22.30. That gap is what R2T2's Longest-Stable-Prefix training closes. It beats same-latency open models across the suite (WhisperRT 4.70, Nemotron 3.71, Voxtral 2.49 on LS-clean; TED-LIUM 3.34, SPGI 3.00, Earnings22 9.36).
• Small and fast. ~4 GB of weights on a vLLM backend — an utterance decodes in well under a second on a mid-range NVIDIA card, which is why your text lands ~1s after you stop talking.
• Ships the server we need. A ready-made WebSocket server (Apache-2.0) that this project reuses and lightly patches — a small integration instead of an inference rewrite.
• English + Chinese, including mixed-language sentences; other languages (French, German, Spanish, Japanese, …) work but are not the optimization target. Context/hotword prompts are natively supported.

One architectural note: whisperless does not rely on R2T2's streaming output — the text you receive is a full offline decode of the utterance. Streaming lets the server process audio as you talk; the deliverable is offline-grade.

Caveats worth knowing: the weights are under NetEase's license, not an OSI one (License); benchmark numbers above are upstream-reported. In our own English-heavy use it has held up, including the author's thick accent, which whisper-style models routinely mangle — that alone settled the model choice (details of our patches).

whisperless.service (systemd user, always warm)     ~/.local/bin/whisperless
  Confucius4-R2T2 ws_server.py               pw-record 16 kHz s16 → WS frames →
  vLLM on your GPU, ~8-9GB VRAM              ws://127.0.0.1:8272/asr_stream_api_v1
  boots in ~2 min, latency ~0.5s/chunk       → collects incremental text → wtype

Two moving parts: their server (unmodified except the patches below, kept warm by systemd so dictation starts instantly) and a small Python client that captures the mic (pw-record, or arecord on non-PipeWire systems), streams it to the server over a WebSocket, and types the final transcript (wtype on Wayland, ydotool/ xdotool fallbacks — see "Window manager support"). At stop, the server re-decodes the whole utterance offline.

• Linux with a Wayland compositor or X11 session, systemd user session
• Text injector: wtype (Wayland) or ydotool or xdotool (X11) — at least one; the installer installs wtype if none is present
• Mic capture: PipeWire (pw-record) or ALSA (arecord)
• NVIDIA GPU: ~7GB VRAM measured floor (GPU_MEM_UTIL=0.42); the default config holds ~9.4GB (fine on a 16GB card alongside a desktop; 8GB cards work with GPU_MEM_UTIL≈0.40 and MAX_MODEL_LEN=4096, untested)
• Network for the one-time setup (~15GB: wheels + model weights)
• Non-NVIDIA GPUs are untested; the vLLM backend requires CUDA

Arch Linux: download whisperless-0.1.0-1-any.pkg.tar.zst from Releases and sudo pacman -U it (client + user service unit; the GPU server itself is set up by the installer step below). An AUR submission is prepared in aur/ pending AUR account availability.

git clone https://github.com/msdavid/whisperless && cd whisperless
./install.sh

🤖 Alternative: ask your coding agent. This repo is agent-friendly — llms.txt is a dense, machine-readable reference to everything here (protocol, patches, failure modes, config semantics). Point your coding agent at the repo and ask it to read llms.txt and install; it can also tailor the setup (custom hotkey, different install destination, distro quirks) and debug any step that fails.

That's it — safe on a fresh machine. The installer is idempotent (re-run anytime; it re-applies server files and the unit, but keeps your config, venv and weights) and handles everything:

Flags: SKIP_MODELS=1 ./install.sh (weights already present), SKIP_SMOKE=1 ./install.sh.

First boot takes 1-2 minutes (CUDA graph capture); later restarts ~30s. The service auto-restarts on failure and starts at login.

Everything lives in ~/.config/whisperless/config.conf (shell-style KEY=value):

SERVER_DIR="$HOME/apps/r2t2/Confucius4-R2T2"  # checkout; venv + models live here
PORT=8272                  # server websocket port
BIND_IP=127.0.0.1          # keep loopback unless serving other machines
GPU_INDEX=0                # CUDA device for the server
GPU_MEM_UTIL=0.55          # vLLM memory budget (lower it if other GPU apps need room)
MAX_MODEL_LEN=8192         # KV cache context cap (raises vLLM VRAM reservation)
HOTKEY="CTRL + space"      # keybind (lua syntax; converted automatically for sway/i3/.conf)
VOICE_LANG=English         # English | Chinese | zhen (auto zh+en) | other supported languages
MAX_SECS=120               # auto-finalize dictation after N seconds
SETTLE_DELAY=0.15          # pause before typing; prevents WM shortcut races
GRACE_TAIL=0.25            # extra capture after stop so the last word isn't clipped
TRAILING_SPACE=1           # append a space after inserted text
NOTIFY=1                   # desktop notifications (0 = silent)
MIC_TARGET=                # pw-record --target; empty = default source (wpctl status)
TYPER=auto                 # text injection: auto | wtype | ydotool | xdotool
WM=auto                    # bind target: auto | hyprland | sway | i3 | other
BORDER=1                   # Hyprland: focused-window border turns red while recording
BORDER_COLOR="rgba(ff3333ff)"          # active border while recording
BORDER_INACTIVE_COLOR="rgba(ff333355)" # inactive borders while recording

Client-side values (VOICE_LANG…BORDER_INACTIVE_COLOR) apply on the next dictation. Server-side values (PORT, GPU_*) need ./install.sh re-run (it re-renders the unit and restarts the service).

To change the hotkey or install destination, just re-run ./install.sh — it asks again (Enter keeps the current value) and re-binds. Non-interactive installs (no TTY) skip the prompts and use the configured values; pass HOTKEY="SUPER + I" to override in scripts.

Mic tips: wpctl status lists sources; set MIC_TARGET=<node id> to pick one (only applies to pw-record; arecord uses the ALSA default device).

The server and client are compositor-agnostic; only the hotkey binding is WM-specific:

Detection order: WM= config override → hyprctl/swaymsg/i3-msg presence → $XDG_CURRENT_DESKTOP.

Text injection chain (client, first that succeeds wins — TYPER= pins one):

• Wayland session: wtype → ydotool (needs a running ydotoold daemon)
• X11 session: ydotool → xdotool (--clearmodifiers, which also guards against stuck modifiers on X11)

Mic capture chain: pw-record (PipeWire) → arecord (ALSA). MIC_TARGET selects the PipeWire source; MIC_TARGET=60-style ids don't map to ALSA devices.

1. Focus any text field
2. Press CTRL+space → notification confirms recording, the focused window's border turns red (Hyprland, BORDER=1)
3. Speak
4. Press CTRL+space again → the punctuated transcript is typed, the border restores, a notification shows the text

Recording auto-stops after MAX_SECS. Rapid double-presses are safe (toggle uses a pidfile + signals, not press/release binds). The stop keeps capturing a short grace tail and drains buffered mic audio before finalizing, so trailing words survive.

Replacing hyprvoice: on Hyprland the installer removes any hyprvoice-toggle bind it finds (that's the migration path from the old whisperx flow — CTRL+space now runs whisperless). The hyprvoice scripts themselves are left on disk but unbound.

Border feedback reads the current border colors at recording start and restores them at the end (exact for single-color 0° gradients; multi-stop gradients degrade to their first stop, logged). BORDER=0 disables it. On non-Hyprland compositors the border feature is silently skipped.

Runtime paths: ~/apps/r2t2/Confucius4-R2T2/ (checkout, .venv, models/, checkpoints/vad/), ~/.config/systemd/user/whisperless.service, ~/.config/whisperless/config.conf, ~/.local/bin/whisperless, ~/.cache/whisperless.log (client log).

systemctl --user status r2t2          # server health
journalctl --user -u r2t2 -f          # server logs (includes per-chunk latency)
tail -f ~/.cache/whisperless.log      # client log (transcript chunks, errors+tracebacks)
~/.local/bin/whisperless --test f.wav # 16 kHz mono s16 wav through the pipeline
systemctl --user restart r2t2         # after changing server-side config or weights

• "whisperless failed" with empty text → check ~/.cache/whisperless.log; errors log the exception type + traceback.
• "no text injector found" → install wtype (Wayland) or ydotool/xdotool, or set TYPER= explicitly; the client logs which backend it used for every insert.
• sway/i3 modifier caveats → the installer lowercases the hotkey (SUPER + K → $mod+k); if your compose needs exact case (e.g. Shift), edit the bindsym line manually.
• Shortcut keys firing while typing → the SETTLE_DELAY guard; increase it.
• Last word missing → increase GRACE_TAIL.
• Terminal punctuation missing → make sure you're running the patched server (./install.sh re-applies it); plain upstream streaming never emits it.
• Server won't start, KV cache error → lower GPU_MEM_UTIL and MAX_MODEL_LEN, then re-run ./install.sh.
• vLLM + Blackwell (RTX 50xx) → needs the repo's current vLLM (≥0.8.5); the installer installs whatever pyproject.toml resolves today (tested with 0.14.0).

Upstream changes are not pulled automatically. To update:

cd ~/apps/r2t2/Confucius4-R2T2 && git pull        # bring in upstream changes
./install.sh                                       # re-applies patches + restarts

Re-verify the patches still apply cleanly — see below, and re-check the diff against the package's server/ws_server.py.

server/ws_server.py in this package is the source of truth; the installer copies it over the clone. Applied changes (also marked # patched inline):

1. Engine init — GPU_MEM_UTIL/MAX_MODEL_LEN read from env (upstream hardcodes 0.95 and 64k context: on a 16GB GPU shared with a desktop that fails KV-cache allocation at boot), max_new_tokens=1024 (upstream 1 — breaks the offline finalize's decode budget), max_num_batched_tokens=2048.
2. EOS finalize — the streaming finish never emits terminal punctuation (LSP append-only design; upstream's own example.py behaves the same). At EOS the endpoint re-decodes the full utterance offline via transcribe() and returns the complete punctuated transcript as the final reset:True message (falls back to the streaming finish on error).
3. Unit environment — PYTHONUNBUFFERED=1 (their debug prints are otherwise swallowed by journal buffering).

Client consequence: reset:True messages replace the accumulated text.

• Confucius4-R2T2 — NetEase Youdao's streaming ASR model and its WebSocket server (Apache-2.0)
• Qwen3-ASR — the architecture it builds on
• FireRedVAD — streaming VAD
• The old hyprvoice flow that motivated this rewrite

This project was vibe coded: a human made the design decisions, ran the testing and reviewed everything; a coding agent wrote most of the code, docs and installer. llms.txt is the agent-facing map of the whole system. Use it the way you would any fast-moving project: read what it runs before trusting it.