Evidence
This folder holds measurements that back decisions about walkthrough speed and behavior. Each entry is a dated write-up with its raw data next to it.
Every number comes from a real run and can be checked again:
- traces through the queries in Diagnose a run with the CLI;
- runs through their CI run IDs;
- drafts through their Drive store versions.
Each write-up states its method, what was excluded and why, and its limits.
| Date | Question | Write-up | Data | Decision |
|---|---|---|---|---|
| 2026-10-05 | Which reasoning effort should the verification agent run at? | Effort level comparison: ████████ | JSON | Pin high: #116 |
Adding an entry
- Use the same ticket and the same inputs for every run you compare. Record those inputs, and keep the trace ID of every run, including runs you exclude.
- Write the per-run numbers to
<date>-<topic>.json. Read them from Langfuse withnpx langfuse-cli api observations list --trace-id <id> …; do not copy them by hand. - In
<date>-<topic>.md, cover the method, the results table, the exclusions and why, a reading of the results, and the limits. - Add a row to the table above and link the entry from the issue or PR it decides.