# PipeRoll Seismograph > An independent observatory for AI model behavioural drift. It runs a fixed, > private probe battery every day against the model APIs that agent fleets depend > on, and publishes detected behavioural change as witnessed readings: > "model X changed on date Y, in dimension Z." It measures each model against its > OWN past, never against other models - it is not a leaderboard and never ranks, > rates, scores, or certifies a model. The seismograph is the second instrument of PipeRoll, the agent-incident measurement institution (the first is the verified incident registry at piperoll.org). Roster: 50 models across 16 labs (44 measured daily on the canary battery; all 50 measured weekly on the deep battery). Baseline running since 2026-08-20. Latest daily reading: 2026-11-09; 82 daily readings witnessed. ## What the readings mean - Reading: a per-model, per-dimension aggregate from one battery run (pass rates, token and latency distributions for active dimensions). No raw model text, no per-probe detail is ever published. - Pass-rate Movement: a change against the model's rolling baseline that survives Benjamini-Hochberg FDR correction at alpha 0.01. A pass-rate Watch is nominally significant (p < 0.01) but unconfirmed. - Thinking-token findings use relative-shift thresholds, not significance tests: Watch at 50%, Movement at 100%; the detector also handles thinking appearing on a previously zero-thinking channel under its stated rule. - Latency uses relative-shift thresholds and appears as Context, never a standalone behavioural-drift claim. Coverage Watches in the feed indicate unassessable series and appear as coverage status on the board. - Watches remain unconfirmed and may be noise. Under-alerting is policy: the feed publishes detected movement; announcements require editor sign-off. - Drift means a model changing against itself over time, at pinned minimum-thinking settings. Dimensions: capability, instruction-following, structured-output, tool-call, sycophancy, verbosity. ## How it works - The probe battery is private; its sha256 is public and witnessed with every reading, so the series is provably unchanged without revealing the probes. - Every reading is hash-anchored in the Rekor public transparency log at measurement time, so baselines provably predate the events they are later cited against. - Two tiers: a fixed frozen ruler (drift) plus a seeded procedural dynamic control (an anti-gaming tripwire; a persistent fixed-over-dynamic gap is a Divergence finding). Grading is mechanical, deterministic code - no LLM judge. - Readings are the published artifact; the private raw sessions are the witnessed anchor, and readings are their reproducible derivations. ## The readings (machine-readable, updated every run) - Movement + watch feed: https://seismo.piperoll.org/advisories.json - current findings and coverage status plus an append-only log of non-latency Movements. Current Watches and latency Context are refreshed each run, not appended to that log. The feed is witnessed each run. - Full series: https://seismo.piperoll.org/series.json - per-model, per-dimension time series behind the board (pass rates, token and latency distributions). - Divergence (fixed-vs-dynamic gaps): https://seismo.piperoll.org/divergence.json - the anti-gaming tripwire; a persistent gap is a Divergence finding. - All three are also in the repository, and every reading is a witnessed document in witness/ (Rekor bundles). No raw model text or per-probe detail is published. ## Key resources - Methodology charter (ratified 2026-09-15): https://github.com/piperoll/seismograph/blob/main/CHARTER.md - How it runs (README): https://github.com/piperoll/seismograph/blob/main/README.md - Repository (code, readings, witness bundles): https://github.com/piperoll/seismograph - PipeRoll incident registry (the first instrument): https://piperoll.org - The conditions board (this site): current levels, a trailing window, and every movement/advisory - public and free forever.