ClawQLCloudflare zone-audit board — tokens, memory, concurrency.
Same 12-step zone audit on a curated OpenAPI → replay mock. ClawQL known-op leads tokens and memory. Executor v2 (promoted beta) is the main Executor column; v1 stable is measured on the same workload. HTTP concurrency is multiplex; stdio concurrency is a separate process-per-client table.
Measured Oct 2026 · host-bound · 10 000 sessions × 3 reps · curated Cloudflare OpenAPI
Jump: tokens · HTTP concurrency · stdio concurrency · pets multiturn
Tokens, success, memory, session latency
Weighted mean tokens (definition + discovery + result + 5× code). Task success 1.0 on every arm below. Memory is process-tree anon MB. Label session transport when comparing latency.
| Arm | Weighted tokens | Anon MB | Session p50 | Transport | Success |
|---|---|---|---|---|---|
ClawQL known-opExecute by operationId + where — primary ClawQL path | 1 833 | ~121 | 23.8 ms | HTTP | 1.0 |
ClawQL programOne execute_program plan (ADR 0015 v0) | 5 187 | ~126 | 11.2 ms | HTTP | 1.0 |
ClawQL compact search | 5 925 | ~123 | — | HTTP | 1.0 |
ClawQL default search | 17 821 | ~125 | — | HTTP | 1.0 |
Executor v2 (promoted)Main Executor comparison column · OpenAPI tools → mock | 5 328 | ~2 103 | ~216 ms | HTTP | 1.0 |
Executor v1 (stable 1.6.10)Same mock workload · mcp + daemon · not in HTTP concurrency table | 8 236 | ~535 | ~84 ms | stdio | 1.0 |
MCPJungle | 55 121 | ~102 | — | HTTP | 1.0 |
agentgateway | 54 705 | ~99 | — | HTTP | 1.0 |
Direct RESTNo gateway — raw calls to mock | 47 361 | — | — | HTTP | 1.0 |
Artifact: docs/benchmarks/cloudflare-multiturn/board-final-summary.md. Executor v2 is the promoted Executor column; v1 is the stable-release second column on the same mock.
Clients sharing one Streamable HTTP server
Climb until p99 or errors break (no predetermined client caps). Clients sustained at 100 / 250 / 500 ms p99 cutoffs · REPS=3 where noted.
| Arm | @100 ms | @250 ms | @500 ms | Peak RPS |
|---|---|---|---|---|
ClawQL Bun --smol | 82 | 211 | 434 | 1 854 |
ClawQL Node 256 | 33 | 112 | 224 | 675 |
agentgateway† Host OOM at 128 before p99 cleared 500 ms — last sustained point | 33 | 88 | 112† | 833 |
MCPJungle | 34 | 92 | 192 | 476 |
Executor v2 | 2 | 10 | 22 | 73 |
Artifact: docs/benchmarks/cloudflare-multiturn/http-concurrency-curves.json. Executor v1 is not in this table (stdio MCP).
One MCP process per client
Fair topology for Executor v1 stable. ClawQL and Executor v1 each spawn one stdio MCP process per client. Memory is the sum across all processes (+ shared daemon for v1).
| Arm | @100 ms | @250 ms | @500 ms | Peak RPS |
|---|---|---|---|---|
ClawQL known-op (stdio)Total anon @100 ms ≈ 3.4 GB (all client processes) | 31 | 74 | 122 | 1 145 |
Executor v1 stable (stdio)Total anon @100 ms ≈ 2.0 GB · single-client board ≈ 535 MB | 8 | 19 | 42 | 123 |
Artifact: docs/benchmarks/cloudflare-multiturn/stdio-concurrency-curves.json. Do not mix these cells with the HTTP multiplex table.
Claim bounds
What we claim
On this Cloudflare board, ClawQL known-op has the fewest weighted tokens (1 833) and ~121 MB anon. Executor v2 is the promoted Executor peer (5 328 tokens, ~2.1 GB). Executor v1 stable on the same mock is 8 236 tokens and ~535 MB — the beta excuse does not reverse the token story.
Two concurrency topologies
HTTP table = N clients sharing one Streamable HTTP server. Stdio table = one MCP process per client (fair for Executor v1). Do not mix the cells.
Session latency transports
ClawQL known-op 23.8 ms and program 11.2 ms are HTTP board figures. Executor v1 ~84 ms is stdio. Executor v2 ~216 ms is HTTP. Label the transport when comparing.
Most efficient governed MCP gateway — measured, not marketed.
Equal-arm board: same multiturn harness, same 40-tool fat catalog, same task success, 10 000 × 12-turn flows. ClawQL is the only measured arm that cuts tokens versus a fair direct-HTTP baseline — via gateway result projection. Executor is version-scoped on that same harness: v1.6.10 pretty-prints (+18 666); v2 β compact envelopes (−5 075). ClawQL still leads v2 by 9 388. ClawQL transport matrix (Node/Bun × stdio/HTTP + gRPC) uses the same recipe — compare Executor v2 latency to ClawQL HTTP, not Bun stdio. Peers have no gRPC MCP on this board.
Fewest I/O tokens among measured MCP aggregators and live Executor (v1 and v2 β), same harness, equal task success. · Measured Oct 2026 · host-bound · 10 000 flows × 12 turns · 40-tool fat catalog
| Capability | ClawQL | Executor | MCPJungle | agentgateway |
|---|---|---|---|---|
Auth / capability gate on the hot path | Yes | — | Optional | Optional |
Operation risk / policyBench runs ALLOW_NO_ENFORCEMENT; the gate is still present. | Yes | — | — | — |
Ephemeral audit ring | Yes | — | — | — |
Output redaction hooksAvailable in ClawQL; unused on the pets fixture. | Yes | — | — | — |
Result projection (where / fields)Primary token win on this board — ClawQL shrinks fat results at the gateway. | Gateway-enforced | Pass-through | — | — |
Multi-tool catalog discovery | Server-side search | Code / N/A | Full tools/list | Full tools/list |
Upstream HTTP to equal mock | Yes | Sandbox (upstream=0) | Yes | Yes |
Persistent agent memory (vault) | Yes | — | — | — |
Semantic enterprise search | Onyx | — | — | — |
WORM / Merkle audit trail | Yes | Basic log | — | — |
Unlimited executions (no meter) | Yes | Capped + overage | Self-host | Self-host |
12-turn I/O tokens vs fair baselinecombined @12 = definitions + Σ discovery + Σ results. Baseline defs pinned to agentgateway tools/list. | −14 463 (−41%) | +18 666 (v1) / −5 075 (v2 β) | +320 | Match |
12-turn I/O tokens — equal task success
combined @12 = definitions + Σ discovery + Σ results. Earlier equal-arm pets catalog board (separate fixture from the Cloudflare zone-audit board above).
| Arm | Defs | Σ Disc | Σ Res | Combined | vs direct | Flow p50 | Flow p999 | RSS | Task |
|---|---|---|---|---|---|---|---|---|---|
Direct HTTPNo gateway process. Defs = agentgateway serialization of the same 40 ops. | 5 520 | 0 | 29 482 | 35 002 | baseline | 2.2 | 9.4 | — | PASS |
ClawQL Node stdiostdio · transport matrix artifact | 297 | 11 568 | 8 674 | 20 539 | −14 463 | 18.7 | 49.5 | 665 | PASS |
ClawQL Bun stdiostdio · same tokens; lowest ClawQL flow p50 on this host | 297 | 11 568 | 8 674 | 20 539 | −14 463 | 16.0 | 39.5 | 344 | PASS |
ClawQL Node HTTPStreamable HTTP · fair latency peer for Executor v2 | 297 | 11 568 | 8 674 | 20 539 | −14 463 | 45.9 | 391.1 | 740 | PASS |
ClawQL Bun HTTPStreamable HTTP · Bun RSS ≈ Bun stdio | 297 | 11 568 | 8 674 | 20 539 | −14 463 | 38.9 | 364.5 | 340 | PASS |
ClawQL Node gRPCgRPC MCP · ClawQL-only on this board (peers have no gRPC MCP). Slower than HTTP for small tool calls. | 281 | 11 568 | 8 674 | 20 523 | −14 479 | 138.7 | 292.2 | 700 | PASS |
ClawQL Bun gRPCgRPC MCP · Bun keeps RSS low; latency still ~7× stdio on this payload size | 281 | 11 568 | 8 674 | 20 523 | −14 479 | 137.3 | 254.7 | 335 | PASS |
Executor 1.6.10stdio (`executor mcp`) · pretty-print tax. Discovery=0. Same runExecutorArm loop as v2. | 90 | 0 | 53 578 | 53 668 | +18 666 | 64.4 | 106.6 | 213 | PASS |
Executor 2.0.0-β.15HTTP /mcp · compare latency to ClawQL Node/Bun HTTP (not Bun stdio). Compact fix credited; ClawQL still −9 388 tokens. | 181 | 0 | 29 746 | 29 927 | −5 075 | 73.5 | 143.2 | 927 | PASS |
MCPJungle | 5 840 | 0 | 29 482 | 35 322 | +320 | 15.4 | 135.0 | 153 | PASS |
agentgateway | 5 520 | 0 | 29 482 | 35 002 | match | 13.6 | 330.9 | 214 | PASS |
Artifacts: gateway-multi-multiturn-10k.json · gateway-multi-multiturn-executor-v2-equal-10k.json
Interleaved e2e and governed gateway cost
n=10 000 interleaved · composition slim. Columns show p50 / p95 / p99 / p999 (ms).
| Arm | e2e p50–p999 | Gateway cost p50/p99/p999 | RSS med | List tok | Invoked | Result |
|---|---|---|---|---|---|---|
| ClawQL | 1.02 / 1.95 / 3.82 / 7.41 | 0.70 / 3.48 / 6.87 | 427 | 1 189 | 178 | 59 |
| Executor | 2.36 / 5.12 / 6.61 / 11.49 | 2.36 / 6.61 / 11.49 | 210 | 2 243 | 90 | 59 |
| MCPJungle | 1.34 / 2.72 / 4.36 / 6.86 | 1.02 / 3.99 / 6.52 | 152 | 46 | 46 | 28 |
| agentgateway | 1.17 / 2.38 / 4.21 / 7.29 | 0.84 / 3.81 / 6.87 | 215 | 37 | 37 | 28 |
Artifact: gateway-multi-cmp-10k.json.
How to read the multiturn board
What we claim
Fewest tokens among measured MCP aggregators and live Executor (v1 and v2 β) with equal task success on the same multiturn harness. Lowest governed gateway cost on the n=10 000 interleaved board.
What we do not claim
Scale-out concurrency wins. Memory wins. Mixing Executor v2 latency with ClawQL Bun stdio. gRPC wins vs peers that lack gRPC MCP. Category-wide supremacy until Bifrost / mcp-gateway are on the board.
Executor version scope
v1.6.10: pretty-printed, +18 666 vs baseline. v2 β.15: compact, −5 075 — credit the fix. ClawQL still leads v2 by 9 388. Video scenes that cite the pretty-print tax must label v1.6.10 on-screen, not only in a footer.
ClawQL transport matrix
Node/Bun × stdio/HTTP/gRPC on the same search→execute recipe (10 000 × 12). Tokens match across transports (~20.5k). Fastest: Bun stdio (16.0 ms p50). HTTP is the fair peer for Executor v2 (ClawQL Node HTTP 45.9 vs Executor v2 73.5). gRPC is ClawQL-only here and slower on this small-payload board (~138 ms) — prefer it for large CallTool bodies.
Next ClawQL lever
Discovery is 56% of ClawQL’s combined @12. Leaner search hits (~300 tok/turn) or skip-search when the agent already knows the operationId would cut ~8k more — without touching projection.
Open the full artifacts
Static boards and docs hold the equal-arm contract and regenerate commands.
- Cloudflare multiturn artifactsZone-audit board summary, HTTP + stdio concurrency JSON
- Executor latency boardEarlier side-by-side latency artifacts
- Multi-gateway method (docs)Equal-arm pets catalog contract, regenerate commands
- Context-compression essayWhy Layer 2 projection compounds the same way on both sides