Paper-Final Reward--Safety Campaign (August 2026)¶

Retired: these artifacts used notebook-default policy budgets rather than the selected budgets, and OmSh evaluation allowed its posterior and monotone floor to adapt across nominal replicates. Do not use these values in the paper. Live-evaluation schema 2 rejects this tag; replace it with paper_final_corrected_2026_08_v2.

This note records the provenance and interpretation of the fresh paper-final campaign. It is the durable engineering companion to the generated report; the JSON export remains the authoritative machine-readable result.

Scope and provenance¶

• experiment tag: paper_final_2026_08_v1
• trainer seeds: 1200--1204
• frozen-policy evaluation: 1,000 independent episodes per seed at horizons 1, 10, 50, 100, and 200
• primary methods: IPPO, IPPO-Lagrangian, CPO, and OmSh-Monotone
• retained environments: Bertrand, Chicken, Markov Stag Hunt, two-agent Pursuit-Evasion, and two-agent Gathering
• report root: /vol/gpudata/oja24-omsh/exports/reports/paper_final_2026_08_v1

OmSh-Credible is implemented but is no longer a primary method. Under the retained three-level risk schedule, its retained-level set is never smaller than the monotone set at a shared posterior and it also spends tail-risk budget. It therefore cannot enlarge the robust feasible-action set; historical credible runs remain diagnostics rather than evidence for a better frontier. The oracle true-shield condition is likewise excluded because it reads the live opponent policy.

Safety-first decision rule¶

At horizon 200, a condition passes the primary finite-horizon gate only if its worst seed's observed unsafe-episode probability is at most the environment's configured threshold. Reward per step is compared only among passing methods. The report also retains the worst seed-level Wilson upper confidence limit, so a zero observed count is not presented as proof of zero risk.

For frozen stationary public-state baselines, exact graph post-processing gives eventual unsafe reachability U_inf. This is a continuing-process diagnostic, not the same objective as expected episodic cost. OmSh cannot be evaluated by that public-state kernel without deleting its carried budget, posterior, and monotone floor; it is instead evaluated through the live deployed shield and its separate coverage-conditional anytime certificate.

Final result¶

All five retained environments are complete. The safety-first winner is the highest-reward method whose worst observed seed satisfies the configured finite-horizon threshold; a method that earns more reward but fails the gate is not eligible.

The complete frozen-policy result at horizon 200 is below. U is the probability of reaching an unsafe state within the evaluation window. The Wilson column is the worst seed's 95% upper confidence limit. U_inf is exact eventual reachability for a frozen stationary public-state baseline; it is not applicable to executed OmSh because the deployed controller carries budget, posterior, and floor state.

No stationary baseline passes the continuing-process threshold: every exact U_inf is above its environment threshold. In particular, the seemingly safe Markov Stag Hunt Lagrangian condition has worst-seed U_200=0.068 but exact U_inf=1. OmSh's anytime theorem is a different, coverage-conditional claim; these learned-model experiments do not statistically validate its required time-uniform outer-kernel coverage.

Pursuit failure diagnosis¶

Four OmSh Pursuit seeds have U_200=0; trainer seed 1203 has U_200=0.217, which determines the finite-horizon failure. The seed raised its monotone floor from level 0 to level 2 late in training, at approximately 2.25 million of 3 million policy steps. The exact learned-versus-true shield audit reports false-safe state/action rates of 2.45%, 3.61%, and 3.66% at levels 0, 1, and 2. The aggregate learned world model has transition-TV mean 0.129 and maximum 0.817; opponent-model NLL is 2.027, Brier score 0.968, and ECE 0.363.

Runtime telemetry reports no graph-lookup coverage holes and no infeasible budget events, while mean action availability is only 0.246. This points to learned-kernel/opponent-model error combined with irreversible floor selection, not a failure to find an action satisfying the certificate inside the learned model. The result is also why a nominal threshold change from 0.11 to 0.10 is not an empirical repair: 0.10 is structurally infeasible at reset, whereas the observed 0.217 failure is caused by model/certificate mismatch.

Recovery provenance¶

Pursuit job 7115 atomically wrote its fifth live report, fifth checkpoint, 264.8 MB history, manifest, and timing sidecars before its Marimo kernel died during post-run diagnostics. The outer batch wrapper then remained stuck for more than eleven hours. After verifying all five reports and checkpoints for all four primary families, the downstream dependencies were changed from afterok:7115 to afterany:7115 and only the hung wrapper was cancelled. The Pursuit visualizer and both corrected consolidated reporters then ran from the complete artifacts. Thus the CANCELLED scheduler state is post-artifact orchestration provenance, not a missing or truncated experiment seed.

Pursuit threshold and three-agent audit¶

The retained two-agent Pursuit condition uses p_tot=0.11. A 0.10 target is structurally infeasible for the learned shield because reset-support demand is approximately 0.102817; 0.11 is the first clean hundredth above that value, not a numerical-tolerance workaround.

Three-agent Pursuit is not a reward--safety result. Exact enumeration reached 85,184 states and 415,886,464 stochastic successor edges. The materialized Python graph exhausted a 205 GiB job while writing a 48,704,852,093-byte atomic temporary pickle, before a valid pickle terminator or atomic publication. Its partial baseline shards remain visible in the audit JSON/Markdown, while the paper-ready LaTeX renderers omit the incomplete environment. The next attempt requires compact or procedural transition storage and streaming augmentation, not another increase in memory for the same representation.