Bring your own model

The matrix on the mechanism page is one model answering six questions twice: once with the notes as they stood, once with the operations bulletin in the prompt. You do not have to take that number on faith. Run the same two arms against your own model, in this tab, and read the receipt.

Your key stays in this tab: the panel holds it for the run, sends it to the provider you picked and nowhere else, and shows back only a stub of it. Nothing here is measured about you, and only the row you choose to publish leaves the browser.

The questions and both prompts are public files. Read them before you paste a key: the questions and their keys, the stale-context prompt, and the pushed-bulletin prompt.

What this page does, before you pick anything

  • Your key never leaves this tab. This page holds it in memory for this run, sends it only to the provider's own endpoint as the request's Authorization header, and never posts it.
  • Your run's content - the prompts and whatever the model answers - goes to exactly one place: OpenRouter at openrouter.ai, the provider you picked above. It does not come back to us.
  • Scoring happens here, in this tab, with the same scorer the published matrix used - landing/src/bench/scorer.js is the port of benchmark/run.py.
  • Publishing is the one thing you opt into, and it is a separate click: the row - model, endpoint host, both scores, the SHAs, a timestamp - goes to our pilot inbox. No key. No prompt. No answers.

1. Pick the provider

Both arms run the same model. This is the model id the endpoint expects, written exactly as the provider spells it.

Held memory or sessionStorage only, cleared when the tab closes. One key per provider: this field shows OpenRouter’s key and nothing else.

Don't paste a key on a page you haven't audited. View-source first: the runner is all here. And never a key with more than chat scope.

What “it is all here” means, file by file
  • /bench/v1/facts.json - the questions and the answer keys
  • /bench/v1/prompts-stale.json - the prompt and corpus the model runs on
  • /bench/v1/prompts-pushed.json - the same, with the current ops bulletin
  • landing/src/bench/providers.js - the presets and the request/curl builders
  • landing/src/bench/scorer.js - the scorer, ported from benchmark/run.py
  • landing/src/bench/session.js - the run, the receipt, the paste-back
  • landing/src/bench/receipt.js - the row we publish, and this promise
  • landing/src/components/BenchPanel.jsx - this panel
  • the questions-with-keys and the two prompt files are served from /bench/v1/facts.json, /bench/v1/prompts-stale.json, /bench/v1/prompts-pushed.json

2. The one call this page may make

this page is not armed for openrouter.ai yet, and the header would refuse the call - open the link below first

Arm this page for openrouter.ai and reload - the link carries bench_endpoint, and /bench is the only route the server reads it on. Until the header on that page allows openrouter.ai, the run button stays off.

3. Run it

The questions are loading…