Continuous batching
A lead agent splits a job across workers, and every one of them decodes on the same weights in the same batch. Their shared prompt prefix lives once in the KV cache. No model swapping, no routing, no second copy.
the betrayer harness
One model. Many agents. Your machine.
Judas is one C++ binary that loads a single local model and runs a whole team of agents on it at once. Model server, agent harness, terminal UI and web app in one process, with every file change snapshotted so you can hand the model real permissions and still undo anything.
Measured on one AMD RX 6800 (16 GB, Vulkan) with a 35B-A3B MoE model.
01 / watch it work
Hand Judas a task. The lead splits it, the workers run in parallel on the same loaded model, and every write lands in the snapshot journal.
--json flag to a small Python CLI. The lead plans, two workers write the code and the tests at the same time, every edit is snapshotted, and the lead runs the tests: 5 passed, 3.8 minutes end to end.$ judas run "add a --json flag to export and update the tests" model one local GGUF, loaded once, 8 slots lead plan: 3 parallel jobs, 1 review lead spawn worker-1 (flag) worker-2 (tests) worker-3 (docs) worker-1 read_file src/export.cpp worker-2 read_file tests/export_test.cpp worker-1 edit_file src/export.cpp snapshot saved worker-3 edit_file docs/export.md snapshot saved worker-2 shell sudo apt install fixtures blocked in code: sudo is never allowed, in any mode worker-2 shell ctest (bubblewrap sandbox) reviewer read the diff, 3 files changed, no objections done review the session in the web app, or revert it step by step
Illustrative session. Agent names and tool output are examples of the shape of a run, not a recording.
02 / continuous batching
Every agent decodes on the same weights, in the same batch. Their shared prompt prefix is stored once in the KV cache, so adding an agent adds work, not another copy of the model.
write src/export.cppwrite docs/export.mdwrite tests/export_test.cppwrite CMakeLists.txt- if (fmt == "csv") return write_csv(rows); + if (fmt == "csv") return write_csv(rows); + if (fmt == "json") return write_json(rows);
03 / undo for everything
Every write goes through a snapshot journal, using reflink copies where the filesystem supports them. Diff or revert one step, or roll a whole session back from the web app or the CLI.
That is what makes high permissions reasonable: the model can be wrong in the open, and you can take it back.
04 / everything in the box
A lead agent splits a job across workers, and every one of them decodes on the same weights in the same batch. Their shared prompt prefix lives once in the KV cache. No model swapping, no routing, no second copy.
Every write goes through a snapshot journal, using reflink copies where the filesystem supports them. Diff or revert one step, or roll a whole session back from the web app or the CLI.
Skills are plain Markdown files that load on trigger words. After long turns a background review saves lessons as skills or memory notes, and an idle curator archives the ones that stopped earning their place.
/v1/chat/completions and /v1/models for any client or editor plugin, plus a richer Judas API with server-sent events for sessions, agents, snapshots and tasks.
A web app with Chat, Cowork, Code, an agent board, tasks, memory and snapshots. A terminal UI. And judas run "task" for scripts and cron.
MCP servers run sandboxed and fail closed. Hooks fire before and after tools and edits. A heartbeat can watch services and post to a Matrix room when something needs you.
Model server, agent harness, terminal UI and web app are the same process. Nothing to orchestrate, nothing to keep in sync.
judas models lists the GGUF files already on your machine, with size, quant, context and whether each fits your GPU. Or pull one from Hugging Face in a single step.
05 / safety
Local agents are most useful when they can actually do things. Judas is designed so full-auto mode is something you can turn on.
Commands run in bubblewrap with your secret directories hidden. No bwrap, no shell: it refuses instead of running unconfined.
Only paths you allow, and never Judas's own config, secrets or snapshot store.
sudo, su, doas, pkexec, passwd and auth database writes are blocked in code in every mode, before any prompt or approval.
A repo's own config can tighten permissions, never loosen them.
Downloaded skills that pipe the internet into a shell, upload files or hide invisible text are quarantined, not loaded.
Bound to localhost, no wildcard CORS, and requests from other sites or foreign hostnames are refused.
06 / install
Linux, a C++23 compiler, CMake, Ninja, SQLite, OpenSSL, toml++ and bubblewrap. Judas links a llama.cpp you build for your GPU.
git clone https://github.com/ggml-org/llama.cpp ~/src/llama.cpp
cmake -S ~/src/llama.cpp -B ~/src/llama.cpp/build -G Ninja \
-DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON
cmake --build ~/src/llama.cpp/build --target llama llama-common
git clone https://github.com/ggml-org/llama.cpp ~/src/llama.cpp
cmake -S ~/src/llama.cpp -B ~/src/llama.cpp/build -G Ninja \
-DCMAKE_BUILD_TYPE=Release -DGGML_HIP=ON -DGPU_TARGETS=gfx1030
cmake --build ~/src/llama.cpp/build --target llama llama-common
git clone https://github.com/ggml-org/llama.cpp ~/src/llama.cpp
cmake -S ~/src/llama.cpp -B ~/src/llama.cpp/build -G Ninja \
-DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build ~/src/llama.cpp/build --target llama llama-common
git clone https://github.com/00jnr00/judas-public judas && cd judas
./install.sh
judas models finds GGUF models already on the machine: your folders, LM Studio, the Hugging Face cache and Ollama. It shows size, quant, context and whether each one fits the GPU.
judas models # what you already have
judas models search WORDS # look on Hugging Face
judas models pull <hf-repo> --add --use
Sample layout only. Real rows show your own files and numbers.
systemctl --user start judas
judas # terminal UI, or open http://127.0.0.1:8079
The installer never enables or starts the service on its own: loading a model takes the whole GPU, so you choose when.
07 / hardware
Run every day on an AMD RX 6800. All numbers on this page.
Builds and passes the unit tests on gfx1030. Serving a model under ROCm is not yet verified; tools/test-rocm.sh --model checks it.
Judas only calls libllama, so a CUDA llama.cpp should work. Nobody has tried it yet.
The sandbox needs Linux. Elsewhere you would run with sandbox = "none".
If you run it on hardware not listed here, an issue with your GPU, backend, model and tok/s helps everyone.
Judas is built and maintained by one person. If it saves you time or cloud bills, a coffee helps.
Support on Ko-fiThe API has no authentication yet. Anyone who can reach the port can run commands as your user through the agents. Keep it on 127.0.0.1, or on a private VPN you fully trust. Never on a LAN or the internet.
Judas is early software. Expect rough edges.