← clawql.com · Both Sides of Context Compression

Executor.sh comparison (executor-cmp-001)

Side-by-side token comparison against executor.sh on a GitHub PR list — the benchmark behind the PragmaticVectors post.

Tokenizer: cl100k_base. focus=input (model output omitted from headline ratios).

Headline numbers (live, vercel/next.js)

Layer	Executor	ClawQL	Notes
Layer 1	1,044 (homepage) / 115 (live execute)	394 (codemode)	394 does not beat live 115
Layer 2	143,466 (pulls.list, full REST)	907 (execute + fields)	158× on tool result alone
Combined	144,510 (pub L1) / 143,581 (live L1)	1,301	111× / 110×

Layer 2 is 100% of Executor's input and 70% of ClawQL's on this task.

Reproduce

# Fixture (CI shape)
npm run benchmark:executor-comparison

# Live GitHub + ClawQL
BENCHMARK_LIVE=1 CMP_GITHUB_REPO=vercel/next.js CMP_PER_PAGE=30 \
  npm run benchmark:executor-comparison

# Live both arms (Executor CLI + ClawQL)
BENCHMARK_LIVE=1 CMP_GITHUB_REPO=vercel/next.js CMP_PER_PAGE=30 \
  EXECUTOR_BIN=/path/to/executor EXECUTOR_CWD=/path/to/executor-cwd \
  EXECUTOR_GITHUB_PULLS_PATH=github.user.githubMain.pulls.list \
  npm run benchmark:executor-comparison

Multi-turn: npm run benchmark:executor-comparison:multiturn · npm run benchmark:executor-comparison:uniform

Raw artifacts (repo)

• Methodology README
• executor-cmp-001.live.json
• Uniform N=5
• Mixed N=5
• OpenBench task

What not to claim

• 862× multi-provider workflow — different metric; do not blend into this comparison
• Fixture 953 vs 23 — harness sanity only, not for headlines
• ClawQL 883 / 3,548 — context tiers, not Executor chart parity