Chapter 2.7 follows chapter 2.6, The token ledger and the timing record, and precedes chapter 2.8, The model-serving routes, within part 2, the harness.
Closeout, the work that records what happened after a batch ends, is a fixed chain of programs. A run set, a group of batch outputs kept together, receives the closeout record. The chain runs in the order stored in CLOSEOUT_CHAIN in run_batch.py. Each stage has a key, title, description, script, and arguments. The chain runs after the cell loop, including when the batch stops. Its record preserves an outcome for a stopped batch.
2.7.1 Closeout stages of the measurement harness
The closeout list has eight stages.
source: 5. Experiment/1. Harness/scripts/run_batch.py lines 567 to 601
The stages, in order, are sweep_dispositions, aggregate_metrics, update_monitoring, gen_iteration_summary, gen_run_set_report, gen_run_set_analysis, cleanup_run_set, and publish_experiment_results. Publication is the final stage of this list.
source: 5. Experiment/1. Harness/scripts/run_batch.py lines 567 to 601, tuple CLOSEOUT_CHAIN
Figure D-H6-1. The closeout chain of a batch and the files each stage writes
%% figure D-H6-1 flowchart TB subgraph driver["run_batch.py, the batch driver"] d1["_write_active_batch()"] --> d2["_write_batch_outcome()"] d2 --> d3["_closeout()"] d3 --> d4["_closeout_locked()"] d4 --> d5["set_run_index_status()"] end subgraph chain["The closeout chain, in the stored order"] direction TB c1["aggregate_metrics"] --> c2["update_monitoring"] c2 --> c3["gen_iteration_summary"] c3 --> c4["gen_run_set_report"] c4 --> c5["gen_run_set_analysis"] c5 --> c6["cleanup_run_set"] c6 --> c7["publish_experiment_results"] end subgraph states["The state each stage is recorded in"] direction LR s1["pending"] s2["running"] s3["done"] s4["skipped"] s5["failed"] end d4 -- "runs each stage in order" --> chain chain -- "each stage writes its state" --> progress["closeout_progress.json"] progress -- "one of" --> states
source: operations/site-ia/A3-diagram-specification.md
source: operations/site-ia/A6-page-briefs.md, section ”# H6.”
Closeout stages in execution order.
2.7.2 Closeout programs of the measurement harness
The closeout program set includes aggregate_metrics.py, aggregate_timings.py, update_monitoring.py, publish_experiment_results.py, publish_lock.py, which implements the publish lock, a file lock that prevents simultaneous publication processes from changing shared allocation, lane_coordinator.py, metrics_db.py, attest_batch_outcome.py, backfill_closeout.py, cleanup_run_set.py, gen_run_set_report.py, gen_run_set_analysis.py, gen_cell_narrative.py, gen_iteration_summary.py, analyze_coverage.py, classify_instrument_errors.py, rescore_tiered.py, capture_evidence.py, evidence_shots.mjs, repair_invocation_errors.py, reflow_markdown.py, and lint_report_prose.py. The publish results program uses _require_empty_index and _synchronise_tip to check publication state. The closeout progress file records batch_status, which says how the batch ended, status, which says how the chain ended, and a stages list.
2.7.3 Separation of isolated model-serving lanes
A lane, an isolated model-serving process with its own network address started for a batch, is kept separate from other lanes. A lane coordinator, the process that assigns and protects lane work, keeps concurrent batches from colliding. It coordinates allocation while the batch starts and keeps allocation state shared.
The publish lock protects selection of shared allocation while a run set is created. The lock prevents concurrent lanes from selecting the same allocation. The lane coordinator and publish lock therefore protect the boundary between batches.
Figure D-H6-2. Two endpoint lanes and the single publish lock they share
%% figure D-H6-2 flowchart LR n1_1["lane_coordinator.py"] --> n1_2["LaneSpec"] n1_2 --> n1_3["build_lane_env()"] n1_3 --> n1_4["run_lanes()"] n2_1["write_lane_provenance()"] --> n2_2["ISOLATION-CONTRACT.md"] n1_4 --> n2_1
source: operations/site-ia/A3-diagram-specification.md
source: operations/site-ia/A6-page-briefs.md, section ”# H6.”
Lane coordinator and publish lock boundary.
2.7.4 Closeout progress of the measurement harness
On 2026-09-12, the closeout chain had finished for 105 of 188 run sets according to the status key in the per-run-set closeout progress files.
source: per-run-set files, status key, read on 2026-09-12
2.7.5 Disputed cells of the measurement harness
The closeout dispute concerns three cells recorded in a run set’s closeout progress. The progress record separates batch_status, which describes how the batch ended, from status, which describes how closeout ended. Its stages list preserves the stage-level outcomes, allowing the disputed cells to be compared with the batch and chain records.
source: page 10 section 13 line 309
2.7.5.1 Publish guard that halted the operator’s own commits of the measurement harness
Observed publish event
The publish guard halted the operator’s own commits during closeout. The event appeared at the publication boundary while the closeout work was being committed.
Operational effect
The halted commits left publication incomplete for the affected closeout. The progress record retained the batch outcome and the chain status so the unfinished publication could be distinguished from the batch result.
Allocation cause
The publish guard used the publish lock to protect shared allocation state. The lane coordinator provides the separation needed when separate batches reach that state at the same time.
Closeout response
The closeout record is the basis for checking each stage and determining what remains to be published. The operator can compare the stage list with the batch status and status before retrying the guarded publication.
2.7.6 Run loss of the measurement harness
A stopped run set once left no trace in the campaign records because closeout was reached only after the cell loop’s normal path. The finally-path arrangement now runs the closeout chain when the batch stops, preserving the outcome for later reporting. source: run_batch.py lines 12 to 24