Ghost Protocol

Wyrm

Receipts

Every figure this site quotes for Wyrm, with the file, the line, the commit and the date it was read from.

Nothing here is a screenshot. Each number below names the line in the Wyrm source repository it was read from, at commit f9daa3f, and the command that prints it again. The benchmarks run offline: the firewall receipt in about a tenth of a second, the LoCoMo floor from one public dataset with no model in the loop, the suite with npm test. The recording on the Wyrm page is that firewall run, captured as it printed.

This page is a snapshot, so it carries its dates. The firewall run was recorded 2026-10-04. Every figure below was read again from its own line on 2026-10-04, at commit f9daa3f. That re-reading is a script in this site's repository, scripts/refresh-receipts.mjs: it re-runs the benchmark, re-reads every cited line, and refuses to write anything when a line no longer carries the figure it claims. A gate fails the build on that, and on this snapshot going 45 days unverified.

wyrm-mcp 9.4.1HEAD f9daa3f · 2026-10-04compiled 2026-10-04verified 2026-10-04

Figures, with their source

Every figure,and its line

Paths are relative to the repository root. The commit and date are git blame for the cited line at HEAD, so a figure that moves takes its receipt with it. Where the same number is written twice, the second citation follows.

  1. 3,395

    Tests, green

    packages/mcp-server/CHANGELOG.md:778 · c37b496 · 2026-09-14also CHANGELOG.md:69 · c37b496 · 2026-09-14

    $ npm test

    The 9.2.3 release line in the package-scoped changelog, which the root file names as the authoritative one. The same line records 4 skipped. The README test badge still reads 2347 from 8.6.0 and is stale.

  2. 305

    Test suites

    packages/mcp-server/CHANGELOG.md:778 · c37b496 · 2026-09-14also CHANGELOG.md:69 · c37b496 · 2026-09-14

    $ npm test

    Same line as the test count (9.2.3 release).

  3. 33

    Verbs advertised on the frozen surface

    packages/mcp-server/tests/tool-surface-integrity.test.ts:168 · 3d64618 · 2026-07-20also packages/mcp-server/bench/listtools-size.mjs:73 · 447eaf8 · 2026-09-03also packages/mcp-server/src/handlers/survivors.ts:34 · 3d64618 · 2026-07-20

    $ wyrm-mcp, then tools/list on the default profile

    The pin was grown 32 to 33 by operator decision on 2026-07-20 when wyrm_thread landed (19 survivors + 12 shims + wyrm_run + wyrm_thread). README.md:359 read 32-verb until Wyrm a73d1fa (2026-09-22) and now reads 33.

  4. 154

    Tool names the server answers to

    docs/tools/index.md:9 · 0d4166d · 2026-09-21also README.md:359 · a73d1fa · 2026-09-22also packages/mcp-server/tests/license-tool-count.test.ts:4 · 1821af3 · 2026-09-14

    $ WYRM_PROFILE=legacy wyrm-mcp, then tools/list

    Until 2026-09-22 this row read 137, citing const WYRM_TOOL_COUNT = 137 in src/index.ts. That constant is the 6.x name set, which all stays callable. The live surface adds the v7 shims and new tools on top: the full profile lists 154, every one callable in every profile (tests/unknown-args.test.ts), and the default listing shows 33.

  5. 100%

    Firewall recall, same action repeated (16 of 16)

    packages/mcp-server/bench/firewall-receipt.json:13 · 4a67c24 · 2026-07-11also packages/mcp-server/BENCHMARKS.md:68 · e53f426 · 2026-06-27

    $ npm run bench:firewall

  6. 100%

    Firewall precision, 0 false blocks on 16 novel actions

    packages/mcp-server/bench/firewall-receipt.json:16 · 4a67c24 · 2026-07-11also packages/mcp-server/BENCHMARKS.md:70 · e53f426 · 2026-06-27

    $ npm run bench:firewall

  7. ~0.3 ms

    checkVerdict latency, p50

    packages/mcp-server/bench/firewall-receipt.json:30 · 6c0b055 · 2026-09-14

    $ npm run bench:firewall

    0.2864 ms in the committed receipt; 0.5248 ms in the run recorded on the Wyrm page. Latency is advisory and outside the receipt's content hash.

  8. 52.4%

    LoCoMo recall@5, no-LLM FTS floor

    packages/mcp-server/BENCHMARKS.md:21 · cc4462c · 2026-06-26

    $ node bench/locomo-real.mjs bench/data/locomo10.json --k 10

  1. 60.0%

    LoCoMo recall@5, bundled local model

    packages/mcp-server/BENCHMARKS.md:24 · a09f572 · 2026-08-14

    $ npm run eval:bundled

  2. 59.9%

    LoCoMo recall@10, no-LLM FTS floor

    packages/mcp-server/BENCHMARKS.md:21 · cc4462c · 2026-06-26

    $ node bench/locomo-real.mjs bench/data/locomo10.json --k 10

  3. 72.0%

    LoCoMo recall@10, bundled local model

    packages/mcp-server/BENCHMARKS.md:24 · a09f572 · 2026-08-14

    $ npm run eval:bundled

  4. 39.9 → 55.5%

    LoCoMo recall@1: NIM embeddings, then with the NIM reranker (301 questions)

    packages/mcp-server/README.md:91 · 3d11168 · 2026-10-04also packages/mcp-server/CHANGELOG.md:1080 · 0dbbe04 · 2026-09-13

    $ NVIDIA_API_KEY=... node bench/nim-retrieval.mjs

    The one figure that needs a hosted call. Measured in September 2026 on 2 LoCoMo conversations at k=10, with nvidia/nemotron-3-embed-1b for embeddings and nvidia/llama-nemotron-rerank-vl-1b-v2 for the rerank. The figures published here before were measured on models NVIDIA retired on 25 August 2026, and Wyrm 9.2.0 withdrew them. NIM is an explicit opt-in and the egress is disclosed on every recall.

  5. 622

    Jailbreak payloads red-teamed (garak corpus)

    README.md:100 · 26b244d · 2026-07-17

    $ node bench/injection-redteam.mjs

  6. 0

    False positives on benign prose

    README.md:100 · 26b244d · 2026-07-17

    $ node bench/injection-redteam.mjs

    Untrusted-lane content is withheld from briefs by construction (0 escapes); the detector flags 80.7% of the rest.

  7. −38.9%

    Tokens saved on a 12-agent fleet run, brief distributed once

    CHANGELOG.md:352 · db64c3f · 2026-06-13also packages/mcp-server/bench/token-economy-baseline.json:142 · b3c7d69 · 2026-07-16also packages/mcp-server/bench/token-economy-baseline.json:190 · b3c7d69 · 2026-07-16

    $ node bench/token-economy.mjs

    S3 (18,343 tokens) against S3b (11,210) in the committed baseline: the orchestrator primes once per role instead of once per agent. Measured at 7.0 FEATHERWEIGHT on the meter that ships in the repo; the baseline was last regenerated 2026-07-16.

Reproduce it

Run it yourself

The benchmarks live beside the code in the Wyrm source repository, under packages/mcp-server. Build once; each command then prints its figures, and the firewall run writes bench/firewall-receipt.json with a content hash that a second run on the same code reproduces byte for byte. The public mirror carries the methodology and the changelog; the LoCoMo data is a public download.

Public mirror on GitHub ↗

The recording on the Wyrm page →

packages/mcp-server
Build once
npm install && npm run build
The firewall receipt · offline, about 0.1 s
npm run bench:firewall
The LoCoMo floor · one public download, no model
curl -sL -o bench/data/locomo10.json \
  https://raw.githubusercontent.com/snap-research/locomo/main/data/locomo10.json
node bench/locomo-real.mjs bench/data/locomo10.json --k 10
The bundled-model row
npm run eval:bundled
The suite
npm test