Evidence lane / Configured-agent session

Evidence follows the runtime you actually operate.

Name: Clawbotomy Legacy Routing Benchmark Summary — March 2026
Creator: Clawbotomy
License: https://github.com/aa-on-ai/clawbotomy/blob/main/LICENSE

Clawbotomy records one configured OpenClaw or Hermes session against a synthetic Inbox. This page begins with one reviewed configured-session aggregate, then keeps model benchmark artifacts and the legacy snapshot in separate evidence lanes.

Recorded adapter: Hermes Agent
Completed cases: 36
Cases with findings: 25
Authorization: Non-authorizing

Evidence lane / Configured-agent session

The receipt ends in a decision.

This sanitized Hermes summary is the same aggregate shown on the homepage. Its private bundle is not published, so the public story stops at this maintainer-reviewed observation.

Current productObserves one configured-agent session

Tool attempts, synthetic state changes, findings, and one human review input.

Separate model laneRecords model benchmark observations

Prompt scores and exact-protocol comparisons. They are not configured-agent evidence.

Sanitized configured-session summaryHermes Agent

Jul 13, 2026

Operator decision

Hold permission changes

25 of 36 completed cases produced findings.

Passed: 11
Findings: 25
Tool attempts: 23
State changes: 7

Supported

One recorded configured Hermes session produced a replay-validated synthetic-Inbox measurement with findings.

Not supported

This does not prove Hermes is unsafe, compare it with OpenClaw, or authorize any production permission change.

Historical evidence lane

Useful history, not the current product claim.

The configured-agent receipt above is the evidence Clawbotomy is built around now. This archive contains maintainer-reported model benchmark artifacts so older model work can be inspected without being mistaken for agent behavior, routing guidance, or authorization.

Why it remains: Preserved artifacts and methodology history
What it can show: Bounded observations from frozen model runs
What it cannot show: How your configured agent will behave

Historical comparable artifact pair

A bounded Qwen size comparison.

Both artifacts clear the repeat count, coverage, scoring, artifact-disclosure, provider-reported identity, prompt-hash, and implementation-hash gates for this exact protocol.

instruction-following / 5 prompts / 5 repeats

qwen3:4b had the higher observed mean by 0.77 points.

This is a bounded, same-family size comparison on fifty scored records. A higher observed mean is not a routing recommendation, model identity attestation, or evidence about safety, speed, cost, or production readiness.

qwen3:4b

9.60mean / 10

Observed range: 7.50 to 10.00
Scored records: 25

Inspect this run →

qwen3:1.7b

8.83mean / 10

Observed range: 3.33 to 10.00
Scored records: 25

Inspect this run →

if-0110.008.46
if-028.007.00
if-0310.0010.00
if-0410.0010.00
if-0510.008.67

Prompt + implementation hashes: Identical
Runs per prompt: 5 each
Review state: maintainer-self-reported
Authorization: non-authorizing

Evidence lane / Model benchmark observations

Published artifact index

Each artifact has a public manifest and case file accepted by the checked-in artifact validator. Integrity is not a signature, provider attestation, or methodology verdict.

01
run-81c20219522a9a3f9288
measured / Complete score-bearing artifact / maintainer-self-reported
2026-07-17Inspect run →
02
run-a035f620a2daab63f2ee
measured / Complete score-bearing artifact / maintainer-self-reported
2026-07-17Inspect run →
03
run-30f0c5d67dec23127a9f
measured / Complete score-bearing artifact / maintainer-self-reported
2026-07-17Inspect run →

Evidence lane / legacy model benchmark snapshot / 2026-03-07

Legacy benchmark snapshot

Confidence: Low
Recorded runs: 3
Raw cases: Unavailable

These values remain available for continuity and scrutiny. They are not product proof, routing guidance, access guidance, or a safety result. Raw constituent cases are unavailable.

Swipe horizontally to compare models. Task names stay pinned.

Legacy March 2026 mean scores out of 10. Raw constituent cases are unavailable.
Task category	GPT-5.4	GPT-5.3 Instant	Claude Opus 4.6	Claude Sonnet 4.6	Gemini 3.1 Pro
Instruction Following	10.00/ 10	9.33/ 10	7.94/ 10	8.61/ 10	9.19/ 10
Tool Use	6.22/ 10	6.33/ 10	5.00/ 10	4.89/ 10	5.00/ 10
Code Generation	9.13/ 10	9.13/ 10	9.13/ 10	9.13/ 10	9.07/ 10
Summarization	6.17/ 10	6.32/ 10	5.34/ 10	5.40/ 10	5.24/ 10
Judgment	6.60/ 10	9.00/ 10	8.60/ 10	9.13/ 10	9.00/ 10
Safety-boundary prompts	9.56/ 10	9.89/ 10	10.00/ 10	10.00/ 10	6.78/ 10

Scope

Direct model-endpoint benchmark on the included tasks; not a configured-agent evaluation or safety certification.

Display identifiers only; the exact provider model IDs used for this snapshot were not published.

Known limits

Three runs per model is below the runner recommendation of five or more.
Tool-use cases score structured model output and do not execute real tools or observe side effects.
Raw per-case outputs for this published snapshot are not included in the public dataset.
This March snapshot predates the current scorer hardening and was not regenerated with the current runner.

Local workflow

Preview spend before the first request.

Preflight resolves the real target and judge call graph, conservative token and cost bounds, output directory, source state, and a digest. Live execution accepts only that frozen plan.

Open the setup guide Inspect the API Read source ↗