What we measure
The product exists because of one observation: an agent acts on whatever context it is given, and context goes stale. A memo is written on Monday, a decision flips on Tuesday, and on Wednesday the agent confidently ships the Monday answer.
Three things matter to us:
- Stale context. An agent acting on instructions that no longer apply. This is the failure we exist to prevent, and it is measurable.
- The cost of a missed instruction. The work does not vanish when the agent acts on yesterday's value: the action is wrong, the undo is extra, and the redo is extra again. That decomposition is our working model. We have not yet measured the multiplier, so this note does not quote one.
- Receipts. Every delivery logs a proof hash of exactly what the agent saw, stamped
[AIRMEMO · authorized · memo_4821]. The receipt is the evidence trail that turns a "trust me" failure into a reviewable one.
The first note: stale context is measurable
Dated 2026-09-16. Source: our own benchmark matrix, committed with the code. This is the note behind the Security & Trust Whitepaper (landing/src/app/security/page.mdx): a stale-context study by Meetless scored 10 frontier models 0/6: all confidently shipped stale facts while reading zero files. We wanted our own numbers, on the models our users actually run.
The protocol. 11 models, 6 in-force facts each, two arms per model. The stale arm sees the full note history with no bulletin. The pushed arm sees the same history plus the AirMemo bulletin naming each winning value. The only difference between the arms is the product mechanism under test. The full protocol and the committed results live in the repo: benchmark/README.md and docs/benchmark-results.md. Each fact carries a dated note trail so "stale" means "superseded", not "forgotten".
What we measured.
| model | stale | pushed | delta |
|---|---|---|---|
| deepseek-v4-flash | 1/6 | 4/6 | +3 |
| deepseek-v4-pro | 1/6 | 4/6 | +3 |
| ornith (local 9B) | 1/6 | 4/6 | +3 |
| glm-5.3 | 2/6 | 4/6 | +2 |
| mistral-large-3 | 2/6 | 4/6 | +2 |
| glm-5.3-flash | 1/6 | 2/6 | +1 |
| gpt-oss:20b | 3/6 | 4/6 | +1 |
| gemma4:31b | 4/6 | 4/6 | 0 |
| gpt-oss:120b | 3/6 | 3/6 | 0 |
| kimi-k3 | 1/6 | 1/6 | 0 |
| qwen3.5:397b | 4/6 | 4/6 | 0 |
The finding. 23 of 66 correct on the stale arm, 38 of 66 with the bulletin pushed: +15 across the matrix. Seven of 11 models ship stale facts, 4 tie, and none regress: no model scores worse with the in-force memo in context. The gap reproduces across vendors (DeepSeek, GLM, Mistral, GPT-OSS) and across model sizes (9B local to 675B hosted), so it is a context-mechanism failure, not a model-quality failure. The strongest movers go from 1/6 to 4/6.
Two caveats, both written in the results doc. The pushed arm's 4/6 ceiling is scorer strictness, not model ability: appended explanations are scored as incorrect, so the +deltas are a lower bound. And the matrix is score-only: latency was not recorded by the runner, and the column is omitted rather than invented.
The cadence
Notes publish when the work is real. When the next measurement lands, it lands here, dated and committed. Until then, the shelf stays as it is.