Current release / Local-first workflow

Documentation

Plan intended Inbox powers in the browser, then run OpenClaw, Hermes, or a fixed reference control against the same synthetic Inbox. Configured-agent receipts remain private unless separately reviewed for publication. Every result remains non-authorizing.

Evidence lanes

Synthetic reference-control evidence

Deterministic fixture behavior from a checked-in positive or negative control, or one embedded declarative policy.

Configured-agent session evidence

Observed tool attempts, synthetic state changes, assertions, and process status from one configured-agent session or one explicitly sampled cohort.

Deterministic bundle verification

Internal file integrity and deterministic host replay under one exact verifier source.

Exact-pin runtime compatibility

Provider-free compatibility checks for the exact runtime identities in the current pin inventory.

Model benchmark observations

Task-specific model endpoint observations from exact frozen benchmark plans and public evidence artifacts.

Legacy model benchmark snapshot

The March 2026 maintainer-reported summary without raw constituent case artifacts.

Exact-pin runtime compatibility is separate from configured-agent behavior. Model benchmark observations and the legacy snapshot are separate from both.

Configured-agent run

Start at /evaluate. The fixed launcher accepts only the checked-in OpenClaw and Hermes bridges and writes a separate private attempt receipt so process failure never masquerades as a finding.

Evaluate a local OpenClaw model

npm run agent:evaluate -- \
  --adapter openclaw \
  --plan ./clawbotomy-inbox-support-agent.json \
  --model ollama/qwen3:1.7b \
  --openclaw-bin "$OPENCLAW_BIN" \
  --expected-openclaw-runtime-sha256 "$OPENCLAW_RUNTIME_SHA256" \
  --expected-provider-runtime-sha256 "$OPENCLAW_PROVIDER_RUNTIME_SHA256"

Evaluate the pinned Hermes runtime

npm run agent:evaluate -- \
  --adapter hermes \
  --plan ./clawbotomy-inbox-support-agent.json \
  --hermes-root "$HERMES_ROOT" \
  --hermes-home "$HERMES_HOME"

Validate the completed protocol bundle

npm run inbox -- validate .clawbotomy/inbox-runs/<runId>

Exit 0 is passed. Exit 2 is a complete run with findings. Exit 1 remains a process anomaly; only one independently validated and replayed new bundle may retain its measured status. The browser viewer requires the launcher receipt that binds each displayed bundle, renders only closed-contract metadata, and never uploads the selected private files. It is an inspector after terminal validation, not the canonical verifier.

Inbox reference run

Download a plan from /preflight. The local runner expands it into isolated cases, executes fixed in-memory tools, and writes replayable private evidence. It never connects to a mailbox or loads the configuration reference.

Run the downloaded plan against the bounded reference agent

npm run inbox -- run \
  --plan ./clawbotomy-inbox-support-agent.json \
  --agent bounded

Run the allowlisted declarative policy adapter

npm run inbox -- run \
  --plan ./clawbotomy-inbox-support-agent.json \
  --adapter declarative-policy/v1 \
  --adapter-config ./inbox-policy.json

Validate and replay the private bundle

INBOX_BUNDLE=.clawbotomy/inbox-runs/inbox-...

npm run inbox -- validate "$INBOX_BUNDLE"
npm run inbox -- replay "$INBOX_BUNDLE"
npm run inbox -- summarize "$INBOX_BUNDLE"

Bounded and overreach remain reference-only controls. Adapter evidence applies only to the exact embedded policy document. Neither path executes a deployed agent. Every path is non-authorizing and leaves production access unchanged.

Quick start

Clone and install

Clone the public repository, then run npm install. Clawbotomy is currently source software, not a published npm package.

Freeze the plan without provider calls

Choose a new private plan and bundle path. Preflight freezes source state, credential presence, models, cases, judge requests, pricing, and conservative request and cost bounds into one digest.

Review, then authorize the exact plan

Copy the plan digest, planned request count, and conservative cost upper bound from preflight. The bound uses the exact judge envelope and a plan-bound response ceiling. Live mode refuses selector overrides and aborts on drift.

Validate the private evidence offline

Verify the bundle digest, complete lifecycle, records, and request totals. Inspect individual responses and judge traces before aggregates.

Export only by separate decision

A confirmed export creates a redacted, separately hashed repository artifact. It never deploys, commits, pushes, or grants a model tool access.

Freeze a zero-request plan

RUN_ID=sonnet-if-smoke-001
PLAN_PATH=.clawbotomy/plans/$RUN_ID.json
BUNDLE_DIR=.clawbotomy/runs/$RUN_ID
export ANTHROPIC_API_KEY=your_key_here

node bench/index.js \
  --models sonnet \
  --tasks instruction-following \
  --runs 1 \
  --bundle-dir "$BUNDLE_DIR" \
  --write-plan "$PLAN_PATH" \
  --preflight

Authorize only the reviewed digest and ceilings

PLAN_DIGEST=copy_the_20_character_plan_digest
MAX_REQUESTS=copy_the_planned_provider_request_count
MAX_COST_USD=copy_the_conservative_cost_upper_bound

node bench/index.js \
  --plan "$PLAN_PATH" \
  --confirm-plan "$PLAN_DIGEST" \
  --max-requests "$MAX_REQUESTS" \
  --max-cost-usd "$MAX_COST_USD" \
  --live

Validate and inspect offline

npm run evidence -- validate "$BUNDLE_DIR"
npm run evidence -- summarize "$BUNDLE_DIR"

Optional, separate public export

PRIVATE_BUNDLE_DIGEST=copy_the_64_character_bundle_digest

npm run evidence -- export "$BUNDLE_DIR" \
  --confirm-public "$PRIVATE_BUNDLE_DIGEST"

Credential presence is part of the plan digest. Keep it unchanged between preflight and live execution. If public export may be needed, start from a clean, committed source state.

What exists today

/evaluate

Configured-agent workspace

Connects the accepted OpenClaw or Hermes bridge, explains pass/findings/infrastructure status, and derives an allowlisted local-only projection from private files selected by the operator.

/preflight

Browser-local plan

Records intended Inbox capabilities and required scenarios before any agent runs. The exported plan remains non-authorizing and makes no permission decision.

/bench

Model artifacts + legacy snapshot

Lists maintainer-reported model benchmark artifacts alongside a clearly separated March 2026 legacy snapshot that lacks raw case records.

/evidence/index.json

Public evidence registry

Indexes explicit maintainer-reported benchmark exports accepted by the checked-in artifact validator. Every entry preserves its scope and remains non-authorizing.

Interpretation rules

Scores summarize performance on the included prompts, scorer, models, and run settings. They do not prove general safety or reliability.

Provisional values are placeholders for policy exploration. Do not present them as benchmark results.

Do not convert an aggregate score into an access or routing decision.

A valid bundle digest shows that recorded files match the validator's expected bytes. It does not establish methodological correctness or general model safety.

No benchmark result authorizes tool access, write access, deployment, or autonomous operation.

Evidence state

The bundle lifecycle and schemas are implemented, and public/evidence/index.json lists the current maintainer-reported model benchmark artifacts accepted by the checked-in artifact validator. Each export remains non-authorizing. The separate March 2026 snapshot remains legacy evidence without raw case artifacts. No run auto-publishes: export writes local repository files only, which still require review and a separate commit, push, or deploy decision. Inspect the complete runner on GitHub.