Evaluation Core cutover
BREAKING-SCHEMA:
omk evalnow writes and reads only Evaluation Core artifacts.
There is no legacy reader, dual write, shadow run, schema migration, or automatic conversion. Evaluation reports created before this cutover cannot be opened in Studio, resumed, compared with Gold, or used by omk evolve. Keep the old OMK version if you need to inspect those files.
New runs are directories addressed by a Core runId. Each directory publishes a manifest plus the exact sealed Run Plan, Execution Bundle, Evaluation Bundle, Analysis Bundle, and Evaluation Report. A missing document, digest mismatch, broken lineage, or unresolved content reference fails closed.
Operational changes:
- pass a Core
runId, not a report path, toomk eval --resume; - pass a Core
runIdtoomk eval gold compare, with explicit--target,--evaluator, and--metricselectors; - Studio lists only Core evaluation runs; doctor and observe documents remain independent;
- managed evidence and evolve acceptance are admitted only from authenticated Core projections;
- managed records use schema v3. Schema-v2 records are rejected rather than migrated; reinstall the artifact and run a new Core evaluation to establish evidence;
- diagnostic post-processing projects only authenticated Core failures, missing evidence, exclusions, and stable reason codes; it does not read legacy result rows or invent recommendations;
- independent
--repeatruns publish run-level variance as Series analysis; without a preregistered Series decision, the release gate fails closed and member runs are not admitted as managed evidence; --dry-runassembles Runtime and prepares a sealed plan without opening a Target or Evaluator.
Migration is deliberately operational, not programmatic: archive or remove old flat report files if desired, then run the evaluation again. OMK does not infer a Core Run Plan, lineage, digests, or decision evidence from a legacy report. Historical score curves are likewise not reconstructed from legacy files.
This cutover changes the storage and application schema, not the measurement construct. Frozen evaluator prompts, the five scoring layers, Bootstrap confidence intervals, Krippendorff alpha, and length-debias semantics are unchanged.