Section 7 · Learn · Generation
Model pinning and reproducibility
Recording the exact dated model snapshot used, rather than a floating alias that silently changes.
Recording the exact dated model snapshot used, rather than a floating alias that silently changes.
What it is
Providers expose both moving aliases and dated snapshots. An alias resolves to whatever is current at call time.
Why it is needed
If a judge sets a published number, a floating reference means re-running months later silently uses a different judge, and the number moves with no explanation in the output.
How it works
Resolve the alias to its snapshot at run start and record that, not the name that was requested.
At a glance
| Reference | Reported version | Reproducible |
|---|---|---|
| gemini-flash-lite-latest | Gemini Flash-Lite Latest | no — floating |
| gemini-3.1-flash-lite | 3.1-flash-lite-05-2026 | yes — dated snapshot |
| gemini-3.6-flash | 3.6-flash-07-2026 | yes — dated snapshot |
In this project
Verified against the provider's model list: a -latest alias reports a floating label as its version, while a dated model reports a snapshot. Every query and judgment records name@snapshot, and a run config captures both model references, the candidate depth, the batch size, the seed and the corpus file.
Trade-offs
Advantages
- Re-runs are comparable
- The judge behind a published number is identifiable
Limitations
- Snapshots are eventually retired
- Pinning forgoes upstream improvements
Common mistakes
- Using a -latest alias anywhere a number is published
- Recording the requested name rather than the resolved snapshot
When to use it
Reach for it when
- Any model call whose output becomes a published figure
How it interacts with the rest of the system
LLM-as-judge and blind judgingConstrained decoding and structured output