Reproducible measurement
Measure the workload you actually run.
Capsule can reduce the model-visible representation of repeated or oversized evidence, but the result depends on the action, payload, query, repetition, and safety gates. The benchmark suite makes those conditions explicit.
Start with a paired baseline
- Choose a real workload such as a terminal log, a large file read, a project query, or a web result.
- Run the same input through the baseline and Capsule paths with the same environment and output requirement.
- Compare model-visible bytes or token estimates, recovery correctness, and latency. Keep failures and passthrough cases in the sample.
- Repeat after a warm replay when the feature supports content-addressed or unchanged-result reuse.
Workload map
| Workload | Command | What it measures | Important limit |
|---|---|---|---|
| Web result lease | npm run benchmark:web:zero-copy | First-call and exact-recovery payload size. | It does not observe provider-side cache or billing counters. |
| Get-Content replay | npm run benchmark:get-content:history | Safe native selection, unchanged-file replay, and recovery. | Small, unsafe, or explicit full reads intentionally pass through. |
| Project context | npm run benchmark:project | Symbol/dependency evidence versus broad file exploration. | Results vary with repository shape and query specificity. |
| Terminal output | npm run benchmark:io | Deduplication, filtering, and exact archived output. | Failures and low-confidence projections retain more context. |
Interpret the result honestly
A percentage is a property of one measured model-visible payload. It is not automatically a percentage of hidden reasoning tokens, subscription limits, provider cache behavior, or invoice cost. Report the workload, Capsule version, command, sample size, and fallback rate beside the number.
Use the repository's benchmark methodology and inspect the corresponding benchmark source. For GitHub discovery measurements, run npm run audit:github-visibility:write -- path/to/snapshot.json or npm run audit:github-visibility:search:write -- path/to/search.json; compare a later search snapshot with npm run audit:github-visibility:search:baseline -- path/to/search.json. Ask measurement questions in the measurement discussion.
Evidence and recovery
Capsule stores the original result locally before emitting a compact view. A positive benchmark therefore needs two assertions: the visible payload is smaller for that case, and the exact result remains recoverable when requested. If either condition fails, the conservative path wins.