A fake llama-server that makes Unsloth Studio treat a remote model as a local GGUF — so Unsloth's gated local-only features (agents, MCP tools, RAG, coding agents) all switch on, while inference actually runs on any OpenAI-compatible endpoint you point it at.

Point it at a remote llama-server, LMStudio, vLLM, TabbyAPI, a LiteLLM gateway — anything that speaks POST /v1/chat/completions.

• Pure Python stdlib. Zero dependencies. One file (llama_bridge.py).
• Nothing installed or patched. It's just the binary Unsloth launches; reverting is launching Unsloth normally.
• Streaming, reasoning, and tool calls flow through verbatim.

• Why this exists
• How it works
• Requirements
• Install
• Configure
• Run
• Using Unsloth's agents & coding agents
• Choosing a shell model
• Verify & debug
• Troubleshooting
• Limitations & approximations
• Security notes
• Deploy on another machine
• Uninstall / revert
• FAQ
• License

Unsloth Studio unlocks its most useful features — the Agents tab, MCP tools, RAG, and driving external coding agents (Claude Code, Codex, OpenCode) — only for models it considers local. For an external/custom provider it relays a bare /v1/chat/completions and runs no agent or tool loop, so none of that lights up.

This bridge makes Unsloth believe a remote model is local. You get Unsloth's full local feature set, but the tokens are generated on whatever GPU host you already run — a workstation, a home server, a gateway in front of several models.

Unsloth resolves its llama-server binary from the LLAMA_SERVER_PATH environment variable (and supports a shebang wrapper script). Point that at llama_bridge.py and Unsloth launches it as the server, waits for it on http://127.0.0.1:<port>, and from then on runs every local feature itself on top of it.

Unsloth launches a server in a few different modes; the script branches on argv:

The endpoints Unsloth expects from a llama.cpp server that a plain chat upstream does not provide are synthesized locally:

Unsloth Studio ──launches──▶ llama_bridge.py  (127.0.0.1:<port>)
      │  /health /props /tokenize …  ──────────▶  synthesized locally
      │  /v1/chat/completions (SSE) ───────────▶  forwarded to your upstream
      ▼                                                     │
 agents · MCP · RAG · coding agents            OpenAI-compatible GPU host
 (Unsloth runs these itself)                   (llama-server / vLLM / LiteLLM / …)

• macOS with Unsloth Studio installed (/Applications/Unsloth.app).
• Python 3 (ships with macOS / Xcode Command Line Tools).
• An OpenAI-compatible chat endpoint reachable from this machine.
• Any small local GGUF to load in Unsloth as the "shell" (its weights are ignored; see Choosing a shell model).

git clone <this-repo> unsloth-llama-bridge
cd unsloth-llama-bridge
./install.sh          # creates .env from .env.example
$EDITOR .env          # set LLAMA_BRIDGE_UPSTREAM (and LLAMA_BRIDGE_KEY if needed)

.env is gitignored — your host and key never get committed.

All configuration is environment variables, most conveniently set in .env (loaded by both launch.sh and llama_bridge.py):

./launch.sh

This quits any running Unsloth, exports LLAMA_SERVER_PATH (plus a launchctl setenv fallback for Dock/Spotlight launches), and relaunches Unsloth. Then, in Unsloth:

1. Load any local GGUF. Its weights are ignored.
2. Chat. Replies come from your upstream model, with reasoning streamed.
3. Turn on Agents / MCP / RAG — they work, because Unsloth believes the model is local.

Two things become available once Unsloth thinks the model is local:

• The Agents tab and MCP tools, orchestrated by Unsloth itself. Configure MCP servers inside Unsloth; the bridge just streams the model's tool_calls through and Unsloth executes them.

• Coding agents. Unsloth's unsloth start <agent> launches an external coding CLI pointed at the local model (its real cloud keys unset):

  unsloth start claude     # Claude Code on your upstream model
  unsloth start codex
  unsloth start opencode

  The CLI must be installed and on your PATH. Every model call it makes flows through the bridge to your upstream.

Any GGUF works for plain chat. For the highest-fidelity tool calling, load a shell GGUF whose chat template and tool-call format match what your upstream emits — e.g. a Qwen3-family GGUF when the upstream serves a Qwen model with the qwen3_coder tool format. Unsloth configures its server-side tool parsing from the loaded template, so a mismatch can garble tool calls even though plain chat is fine.

A small shell (a few hundred MB to a couple GB) is plenty — the weights are never used for inference.

Watch the log; every forwarded request appears:

tail -f ~/unsloth-bridge.log     # look for: chat → upstream model=… …

Standalone smoke test, no Unsloth required:

# start the bridge as if Unsloth launched it
python3 llama_bridge.py -m /any/shell.gguf --host 127.0.0.1 --port 5599 --alias test &

curl -s localhost:5599/health
curl -s localhost:5599/props | head -c 200
curl -sN localhost:5599/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"stream":true,"messages":[{"role":"user","content":"capital of France?"}]}'

You should see data: SSE chunks stream back from your upstream.

• /tokenize and /apply-template are estimates (~4 chars/token) for the context meter; real templating/tokenizing happens upstream.
• /props reports the captured modalities and chat template — cosmetic, since the upstream does the real templating.
• Only /v1/chat/completions is forwarded. Agents that require the OpenAI Responses API (/v1/responses) rely on Unsloth's gateway translating to chat completions first; if a specific coding agent errors, that path may need adding.
• Reasoning is on and streamed by design. If short answers get cut, that's the upstream's thinking budget, not the bridge.

• Secrets live only in .env, which is gitignored. Never git add -f .env.
• The bearer key is sent only to LLAMA_BRIDGE_UPSTREAM. Point the bridge at a host you trust.
• The bridge binds to whatever host/port Unsloth passes (loopback by default).
• No telemetry, no third-party calls — the single outbound connection is to your configured upstream.

git clone <this-repo> && cd unsloth-llama-bridge
./install.sh && $EDITOR .env      # per-machine upstream + key
./launch.sh

Nothing is installed system-wide; all machine-specific config is in that host's own .env.

Just launch Unsloth normally, without LLAMA_SERVER_PATH:

launchctl unsetenv LLAMA_SERVER_PATH   # clears the Dock-launch fallback

Nothing was installed or patched, so there's nothing else to undo. Delete the repo folder if you're done with it.

Does this modify Unsloth? No. It's only the process Unsloth launches as its llama-server. Reverting is launching Unsloth normally.

Do I need the shell GGUF's weights to match my upstream? No — the weights are never used. Only the chat template / tool format matters, and only for tool-call fidelity.

Can the upstream be another llama-server? Yes. Any OpenAI-compatible /v1/chat/completions endpoint works.

Does RAG really stay local? Yes — the embedding server (--embedding) execs the real local llama-server; only chat is forwarded upstream.

Will this work on Linux/Windows? It's written and tested for macOS + Unsloth Studio. The core is portable Python, but launch.sh and the default paths are macOS-specific.

MIT.