Wyrm
Receipts
Every figure this site quotes for Wyrm, with the file, the line, the commit and the date it was read from.
Nothing here is a screenshot. Each number below names the line in the Wyrm source repository it was read from, at commit f54302a, and the command that prints it again. The benchmarks run offline: the firewall receipt in about a tenth of a second, the LoCoMo floor from one public dataset with no model in the loop, the suite with npm test. The recording on the Wyrm page is that firewall run, captured as it printed.
This page is a snapshot, so it carries its dates. The firewall run was recorded 2026-09-06. Every figure below was read again from its own line on 2026-09-06, at commit f54302a. That re-reading is a script in this site's repository, scripts/refresh-receipts.mjs: it re-runs the benchmark, re-reads every cited line, and refuses to write anything when a line no longer carries the figure it claims. A gate fails the build on that, and on this snapshot going 45 days unverified.
Every figure,and its line
Paths are relative to the repository root. The commit and date are git blame for the cited line at HEAD, so a figure that moves takes its receipt with it. Where the same number is written twice, the second citation follows.
2,697
Tests, green
packages/mcp-server/CHANGELOG.md:159 · 419b8b1 · 2026-09-03also CHANGELOG.md:67 · 0e78fb8 · 2026-09-03also README.md:79 · 2c75331 · 2026-09-03
$ npm test
The 9.0.0 release line in the package-scoped changelog, which the root file names as the authoritative one. 9.1.0 to 9.1.2 did not restate the count, so this is the newest committed figure. The README test badge still reads 2347 from 8.6.0 and is stale.
226
Test suites
packages/mcp-server/CHANGELOG.md:159 · 419b8b1 · 2026-09-03also CHANGELOG.md:67 · 0e78fb8 · 2026-09-03also README.md:79 · 2c75331 · 2026-09-03
$ npm test
Same line as the test count (9.0.0 release).
33
Verbs advertised on the frozen surface
packages/mcp-server/tests/tool-surface-integrity.test.ts:168 · 3d64618 · 2026-07-20also packages/mcp-server/bench/listtools-size.mjs:73 · 447eaf8 · 2026-09-03also packages/mcp-server/src/handlers/survivors.ts:34 · 3d64618 · 2026-07-20
$ wyrm-mcp, then tools/list on the default profile
The pin was grown 32 to 33 by operator decision on 2026-07-20 when wyrm_thread landed (19 survivors + 12 shims + wyrm_run + wyrm_thread). README.md:347 still reads 32-verb; that prose predates the growth and is stale.
137
Tools callable through the alias spine
packages/mcp-server/src/index.ts:113 · b7e13c2 · 2026-06-06also README.md:347 · c06a3f3 · 2026-07-02
$ WYRM_PROFILE=legacy wyrm-mcp, then tools/list
100%
Firewall recall, same action repeated (16 of 16)
packages/mcp-server/bench/firewall-receipt.json:13 · 4a67c24 · 2026-07-11also packages/mcp-server/BENCHMARKS.md:68 · e53f426 · 2026-06-27
$ npm run bench:firewall
100%
Firewall precision, 0 false blocks on 16 novel actions
packages/mcp-server/bench/firewall-receipt.json:16 · 4a67c24 · 2026-07-11also packages/mcp-server/BENCHMARKS.md:69 · e53f426 · 2026-06-27
$ npm run bench:firewall
~0.1 ms
checkVerdict latency, p50
packages/mcp-server/bench/firewall-receipt.json:30 · 0063715 · 2026-07-24also packages/mcp-server/BENCHMARKS.md:71 · e53f426 · 2026-06-27
$ npm run bench:firewall
0.1148 ms in the committed receipt; 0.4216 ms in the run recorded on the Wyrm page. Latency is advisory and outside the receipt's content hash.
52.4%
LoCoMo recall@5, no-LLM FTS floor
packages/mcp-server/BENCHMARKS.md:21 · cc4462c · 2026-06-26
$ node bench/locomo-real.mjs bench/data/locomo10.json --k 10
60.0%
LoCoMo recall@5, bundled local model
packages/mcp-server/BENCHMARKS.md:24 · a09f572 · 2026-08-14
$ npm run eval:bundled
59.9%
LoCoMo recall@10, no-LLM FTS floor
packages/mcp-server/BENCHMARKS.md:21 · cc4462c · 2026-06-26
$ node bench/locomo-real.mjs bench/data/locomo10.json --k 10
72.0%
LoCoMo recall@10, bundled local model
packages/mcp-server/BENCHMARKS.md:24 · a09f572 · 2026-08-14
$ npm run eval:bundled
33 → 47 → 52%
recall@1: local baseline, NIM embeddings, NIM reranker
packages/mcp-server/README.md:81 · a09f572 · 2026-08-14
$ NVIDIA_API_KEY=... node bench/nim-retrieval.mjs
The one figure that needs a hosted call. NIM is an explicit opt-in and the egress is disclosed on every recall.
622
Jailbreak payloads red-teamed (garak corpus)
README.md:100 · 26b244d · 2026-07-17
$ node bench/injection-redteam.mjs
0
False positives on benign prose
README.md:100 · 26b244d · 2026-07-17
$ node bench/injection-redteam.mjs
Untrusted-lane content is withheld from briefs by construction (0 escapes); the detector flags 80.7% of the rest.
−38.9%
Tokens saved on a 12-agent fleet run, brief distributed once
CHANGELOG.md:222 · db64c3f · 2026-06-13also packages/mcp-server/bench/token-economy-baseline.json:142 · b3c7d69 · 2026-07-16also packages/mcp-server/bench/token-economy-baseline.json:190 · b3c7d69 · 2026-07-16
$ node bench/token-economy.mjs
S3 (18,343 tokens) against S3b (11,210) in the committed baseline: the orchestrator primes once per role instead of once per agent. Measured at 7.0 FEATHERWEIGHT on the meter that ships in the repo; the baseline was last regenerated 2026-07-16.
Run it yourself
The benchmarks live beside the code in the Wyrm source repository, under packages/mcp-server. Build once; each command then prints its figures, and the firewall run writes bench/firewall-receipt.json with a content hash that a second run on the same code reproduces byte for byte. The public mirror carries the methodology and the changelog; the LoCoMo data is a public download.
npm install && npm run build
npm run bench:firewall
curl -sL -o bench/data/locomo10.json \ https://raw.githubusercontent.com/snap-research/locomo/main/data/locomo10.json node bench/locomo-real.mjs bench/data/locomo10.json --k 10
npm run eval:bundled
npm test