Final-Policy Campaign Report¶

scripts/summarize_final_policy_campaign.py combines the frozen-policy artifacts produced by a tagged final experiment campaign. It is deliberately separate from the training-window report: its inputs are IID live rollouts of the frozen final joint policy and, where available, the exact stationary-policy reachability post-processing.

Run it from the project root:

uv run python scripts/summarize_final_policy_campaign.py \
  --tag paper_final_corrected_2026_08_v2 \
  --exports-root exports \
  --expected-runs 5

The default outputs are final_policy_campaign.{json,md} plus final_policy_compact.tex, final_policy_all_conditions.tex, matrix_timings.tex, and gridworld_timings.tex beneath exports/reports/<tag>/. The compact table gives each primary method as the tuple reward per step / worst-seed finite-horizon unsafe reach / exact eventual unsafe reach. Exact eventual reach is -- for executed OmSh because its runtime state is not represented by the public-state graph. The second table retains one row per condition for the appendix. Its reward cell reports the cross-seed mean and sample standard deviation so small mean differences are not presented without their between-seed scale.

The timing tables combine the environment-root timings.json component totals with the tagged run directory's timings.json. World- and opponent-model columns report the latest component duration. Policy columns report the mean duration of the current indexed run batch, while shield setup reports the latest Monotone single-shot duration. Training stages are selected from the condition's artifact stem plus the campaign tag, which prevents timings from an older campaign being mixed into the paper.

Discovery and completeness¶

The summarizer discovers **/experiments/runs/<tag> directories. A per-run JSON file is accepted only when its artifact stem has a completion manifest, it uses live-evaluation schema 2, and its sampling mode matches the algorithm: IID controller-snapshot replay for OmSh and IID environment resets for the stationary baselines. OmSh reports must carry a valid controller-state digest. The report and manifest must also agree on artifact, algorithm, run index, trainer seed, episode count, evaluation-seed block, horizons, and the complete per-episode seed sequence. The four required families are IPPO, IPPO-Lagrangian, CPO, and OmSh-Monotone. Credible and parameterized shield diagnostics remain discoverable as distinct rows but do not determine primary completeness or the safety-first winner.

The environment threshold comes from config.max_risk in the manifests. The report rejects a run directory whose completed conditions disagree on this value. A condition is complete only when it has exactly expected_runs live reports; an environment is complete only when every required family is present and complete. Partial campaigns are still rendered with explicit missing and incomplete status. The JSON and Markdown retain incomplete environments and diagnostic conditions for auditability, while every paper LaTeX table omits an environment unless all four primary families have all expected runs and the same largest evaluation horizon.

The retired paper_final_2026_08_v1 artifacts predate controller-snapshot replay and are intentionally rejected by this validation. They also used the notebook defaults instead of the selected policy budgets. Use notebooks/experiments/scripts/corrected_final_campaign.sh for the corrected campaign. It locks five fresh seeds, 1000 episodes, and 250k policy steps for Bertrand, Chicken, Gathering, and Pursuit, with 1M steps for Markov Stag Hunt. Pursuit remains at risk 0.11 because the current learned reset-state robust demand is approximately 0.103, making 0.10 infeasible for this shield bundle.

Safety-first decision rule¶

At the largest common live horizon, a condition passes the finite-horizon gate only when its worst seed's observed unsafe-episode probability is no greater than the configured threshold. Reward per step is ranked only among conditions that pass. The report also retains the cross-seed mean, worst seed, worst per-seed Wilson upper confidence limit, violations per step, reward per step, and the conditional probability that the first unsafe event occurs exactly at the reported horizon.

The observed point estimate, rather than the confidence upper limit, defines the primary gate to match the paper protocol. The Wilson quantity remains visible so a zero count is not misread as proof of zero risk.

Exact eventual reachability¶

If <artifact_stem>_safety.json exists, the report joins its exact finite- and infinite-horizon aggregate. An exact eventual pass requires the worst seed's upper reachability bound to meet the same threshold. These rows apply to frozen stationary public-state baselines.

The summarizer never assigns public-graph exact reachability to executed OmSh. OmSh additionally carries the successor budget, opponent posterior, and irreversible floor, so evaluating its actor alone would remove the deployed shield. OmSh therefore remains represented by live execution and its separate learned-model/coverage-conditional certificate.

Independent seed shards¶

Long screens can run each seed as an independent one-run tagged job. Give each shard its serial trainer seed and its serial evaluation block: for base evaluation seed 1000000 and 1000 episodes, run index i must use final_policy_eval_seed_offset=1000000 + 1000 * i. After every shard is complete, reconstruct the campaign with:

uv run python scripts/merge_final_policy_shards.py \
  --tag screen_credible_retirement_v2 \
  --shard-tag-template 'screen_credible_retirement_v2_seed{trainer_seed}' \
  --exports-root exports \
  --seed-offset 1600 \
  --expected-runs 5

The merger validates trainer and evaluation seeds, preserves destination run zero, atomically publishes runs 1--4, and writes a five-run completion manifest plus final_policy_shard_merge.json. Native completion manifests are checked against their report's algorithm, trainer seed, evaluation seed block, episode count, and tag. In recovery mode the destination seed is also checked against its versioned checkpoint; recovered base and shard checkpoint records remain explicit in the merged manifest. The merger intentionally does not merge training histories or diagnostic tables; those remain provenance artifacts under their source tags.