Reward-safety frontier study (October 2026)¶

Three post-AAMAS studies the user approved on 18 Oct 2026. They ask whether OmSh can beat IPPO-Lagrangian and CPO on the reward-safety frontier, and whether novelty-driven exploration for the world model could help. The rules below were written before any run was launched.

All three run on CX3 (PBS, one core per job), because the DoC cpucluster is busy. They reuse the published WM and OM, restored from the AAMAS archive, and the published seeds, evaluation streams, horizons and OmSh arguments (monotone floor, patience 512, opponent-level-gap log, both Congestion roles shielded).

Starting point¶

Table 1's single operating point per method, from the paper:

• OmSh is the highest-reward gate-passing method in Congestion, Markov Stag Hunt, Pursuit and Gathering.
• It loses Bertrand to IPPO-Lagrangian (0.159 against 0.189) and Chicken narrowly (1.013 against 1.015). There IPPO-Lagrangian's exact U_∞ is 1.0, and IPPO-TrueShielded earns the same 0.159 in Bertrand, so the gap comes from certifying eventual safety, not from model error.
• OmSh realises U_H = 0 in five environments against p = 0.10–0.20. It spends almost none of its budget.
• The published IPPO-Lagrangian and CPO used cost limit 0, the strict end (6.3 in Congestion).
• Follow-up F4 (followups-f1-f2-f4.md) gave the shield the exact kernel with the learned levels. Reward moved by +0.065 [−0.070, 0.200] in Pursuit, −0.0006 in Gathering and not at all elsewhere.

A. Frontier sweep¶

Question. Where do the methods sit as their trade-off parameter varies, and does OmSh's reward rise when it is allowed more risk?

Design (notebooks/experiments/scripts/cx3_frontier.sh, 138 jobs, 5 seeds each):

• OmSh and IPPO-TrueShielded over the budget p ∈ {0.01, 0.05, 0.10, 0.20, 0.30, 0.50}.
• Pursuit uses {0.14, 0.16, 0.20, 0.30, 0.50}, since its floor-0 reset demand is 0.132.
• The shield bundle does not depend on p (its cache key is the graph and the IOP), so every level shares one certificate per environment.
• Tags: frontier_omsh_p<p>_v1 and frontier_truesh_p<p>_v1.
• IPPO-Lagrangian and CPO over the expected episodic cost limit c = k·p for the same p grid, plus the published limit 0.
• k is the median episodic cost of an unsafe episode in the published constrained runs: 1 in Bertrand, Chicken, Gathering and Markov Stag Hunt, and 63 in Congestion, where a collision persists for the rest of the 64-step episode. That gives Congestion {0, 3.15, 6.3, 12.6, 18.9, 31.5}, with 6.3 the published limit.
• Pursuit's baselines already fail at limit 0 (worst U_200 ≥ 0.93), and a looser limit cannot lower their risk, so only limit 0 is re-run there.
• Tags: frontier_lag_cl<c>_v1 and frontier_cpo_cl<c>_v1.
• IPPO once per environment (tag frontier_ippo_v1).
• Code. Five notebooks gained a cost_limit config key (default 0.0, the trainers' default), and IPPO-Lagrangian now receives it, so the published runs are unchanged. The jobs run from the CX3 worktree ~/Projects/omsh_frontier, pinned to the launch commit.

Readout (scripts/summarize_frontier.py). Each (method, level) point is the cross-seed mean frozen-policy reward R and the worst-seed U_H, at the largest common horizon (200; Congestion 64).

• Frontier. For each threshold τ ∈ T = {0.01, 0.05, 0.10, 0.16, 0.20, 0.30, 0.50}, method M's frontier point F_M(τ) is its highest-R level with worst-seed U_H ≤ τ. M has no point at τ if no level qualifies.
• OmSh against each of IPPO-Lagrangian and CPO at each τ:
• only OmSh has a point: win; only the baseline: loss; neither: not compared;
• both: the paired-by-seed difference R_OmSh − R_B of the two frontier points, with a 95% t interval (4 df). Lower limit above 0 is a win, upper limit below 0 a loss, otherwise a tie.
• Picking each method's best level by mean reward biases both sides upward, so the paired test is descriptive, not a significance claim.
• Environment verdict against each baseline over T: dominates (at least one win, no loss), dominated (at least one loss, no win) or mixed.
• Budget use. OmSh's ΔR = R(p = 0.5) − R(lowest feasible p), paired by seed with a 95% t interval. Budget-limited if the lower limit is above 0, otherwise not budget-limited: the shield is then not what limits reward at these budgets.
• Secondary, eventual safety. Exact U_∞ of every baseline level from scripts/evaluate_final_policy_checkpoints.py, and the same frontier with U_∞ ≤ τ required of the baselines. OmSh's U_∞ is not computable exactly (the shield is history-dependent), so this table is descriptive.
• Reproduction. The published levels (OmSh, TrueSh, IPPO at the published p; IPPO-Lagrangian and CPO at the published limit) are compared with the AAMAS archive. Differences are reported. The frontier uses the CX3 values throughout, so every point comes from one platform and one commit.
• TrueSh is an oracle reference. It is plotted but not counted in the verdicts.

A, tertiary readout: matched eventual criterion (fixed 19 Oct 2026, before the audits ran)¶

Added at the user's request, so that OmSh's eventual-safety column has a number. The baselines' U_∞ is exact on the simulator. OmSh's matching quantity is Theorem 3(ii) instantiated in the original game: P(τ_U ≤ N) ≤ q_0 + E[Σ_{t<N} η_t], with η_t the step excess under the true law.

• Measurement.
• scripts/audit_certificate_dominance.py runs on every OmSh frontier level, with 200 frozen evaluation episodes per seed at the largest horizon.
• It writes exports/certificate_dominance_frontier/<env>_<tag>.json, holding the plug-in bound q_0 + mean episode excess and a one-sided 95% normal-approximation upper limit.
• Qualification.
• An OmSh level qualifies at τ if max(worst-seed U_H, worst-seed-and-role 95% upper bound) ≤ τ.
• A baseline level qualifies if max(worst-seed U_H, worst-seed U_∞) ≤ τ.
• The comparison and verdict rules are those of the primary readout (vs_<baseline>_anytime).
• Limits.
• The bound covers the evaluation window (N = 200; 64 in Congestion). In Pursuit, episodes end well before N.
• Where no excess is observed, the upper limit equals q_0. That shows only that no excess was seen, not that η_t = 0 almost surely.
• The comparison sets an upper bound for OmSh against an exact probability for the baselines, which is conservative against OmSh.

B. Exact-kernel opponent-model control (Pursuit, Gathering)¶

Question. Could a better world model improve reward through the opponent model? This bounds what novelty-driven WM exploration could gain in reward.

Levels 1–2 are trained by imagined rollouts in the learned graph, and level 0 by behaviour cloning on the collection data. F4 swapped only the shield's kernel, so this route is untested.

Design.

• scripts/build_exact_kernel_wm_graph.py writes wm_exact_kernel/env_transition_graph.pkl.
• It has the exact graph's successors and probabilities (true_shield/true_transition_graph.pkl), with each edge's reward taken from the learned graph, matched on (state bits, joint action, successor bits).
• The exact graph stores zero rewards, so the reward head stays learned. This isolates the transition kernel, which is what more exploration improves.
• The script reports how many edges needed a fallback reward and fails if the adapter's observation-Markov check fails.
• OM training is unseeded, so a retrained OM differs from the deployed one even on the same graph. In the clean timing run, retraining moved Pursuit OmSh from 0.251 to 0.294. The design therefore uses replicates and a same-platform control, fixed before launch (18 Oct 2026):
• train_om with om_graph=learned om_tag=learned_r{1,2} retrains the OM twice in the learned graph, and om_graph=exact_kernel om_tag=exact_kernel_r{1,2} trains it twice in the exact-kernel graph. Both use the same buffer, level-0 data and hyperparameters, and write om_<tag>/, so the published om/ is untouched.
• OmSh with shield_kernel=exact shield_om=<tag> uses that OM for the certificate and the runtime posterior. The shield kernel is exact in every arm, so the arms differ only in the graph the OM levels were trained in.
• Conditions, on CX3 at the published p and trainer seeds (notebooks/experiments/scripts/cx3_frontier_bc.sh):
• (i) published OmSh (sweep A's published level);
• (ii) F4: exact kernel with the published levels (tag frontier_b_f4_v1);
• (ii′) exact kernel with levels retrained in the learned graph, two OM replicates;
• (iii) exact kernel with levels trained in the exact-kernel graph, two OM replicates.

Readout, per environment, with 95% t intervals over the five trainer seeds:

• ΔR_OM = R̄(iii) − R̄(ii′). Each arm's reward is averaged over its two OM replicates for each trainer seed, then differenced by seed. This is the effect of training the levels on exact rather than learned transitions.
• Retraining noise: the replicate difference within each arm, R(r1) − R(r2), paired by seed.
• ΔR_total = R̄(iii) − R(i): the reward a perfect transition kernel could buy over the published OmSh, retraining included.

Decision. WM exploration has reward headroom through the OM in an environment if ΔR_OM's lower limit is above 0, ΔR_OM exceeds the larger absolute mean replicate difference, and every (iii) replicate passes the gate. Otherwise no detectable headroom, and study C is read for certification only.

C. Novelty-driven collection for the certified shield (Gathering)¶

Question. Does count-based novelty exploration during data collection make the statistically certified (count-centred) shield deployable?

Stage 2B found it infeasible at every sample budget, and 80% of Gathering's certificate-critical rows were never visited (certified-ambiguity-study.md). Gathering's arbitrary-opponent certificate on the exact model is feasible (reset demand about 0), so visitation is the whole problem there. In Pursuit it is already infeasible at 0.886, so exploration cannot help there.

Design.

• scripts/certified_ambiguity_study.py --collector novelty --novelty-beta β collects the stream with the same all-agent IPPO collector plus a count bonus.
• Every agent's training reward gets β/√N(s, a_joint), where N counts that (state, joint action) row in the stream so far. The rows are joint, so all agents explore, as the user proposed.
• The recorded transitions keep the true rewards, so the kernel estimate is unaffected.
• The certified shield is arbitrary-opponent, so no opponent model is fitted to the novelty data. In a deployed OmSh, the novelty stream would feed the kernel only and level 0 would keep reward-driven data.
• Gathering, seeds 0–4 (the Stage 2B seeds), at the study's default budgets and coverage allocations. Three arms: the PPO collector (re-run on CX3), novelty β = 0.1 (primary) and β = 1.0 (sensitivity).
• scripts/attribute_certificate_infeasibility.py runs on every cell, at the full stream and δ_cov = 0.05, with the same collector and seed, so it regenerates the same stream.
• Tags: certified_frontier_c_<arm>_v1 and attribution_frontier_c_<arm>_v1, with arm one of ppo, novelty_b0p1 and novelty_b1.

Readout at the full 750k-transition budget and δ_cov = 0.05 (the middle of the study's three), paired by seed:

• Primary. Certified feasibility at p_total = 0.2. Success if at least 3 of 5 novelty-β = 0.1 seeds are feasible.
• Secondary, as paired differences with 95% t intervals:
• certified reset demand (censored at 1, so it can only show a difference where both sides are below 1);
• the zero-radius count-centred demand;
• the number of certificate-critical rows never visited;
• the visited-row fraction.
• Cost. Unsafe steps during collection, per arm. Novelty seeks unsafe states too, since they are novel.

Launch record (Sun 18 Oct 2026)¶

• CX3 setup.
• The published wm/, om/, shield/ and true_shield/ of the six environments were copied from aamas/cpucluster/exports/ (then named aamas_2026_08_10) to ~/Projects/omsh/exports on CX3, through the workstation. The tar used POSIX format, so nanosecond mtimes survive and the shield caches stay valid. The two sides' sha256 lists agree on all 209 files.
• The worktrees ~/Projects/omsh_frontier (branch frontier, f78e001) and ~/Projects/omsh_bc (branch frontier_bc, 33515a4) have no upstream and link exports to that directory.
• The shared venv ~/bucket/.venv-omsh was synced once from the pinned lockfile before launch. Both commits have the same lockfile.
• A: 138 jobs, 4406035–4406173, starts staggered one minute apart from 20:25.
• B: 20 jobs, 4406283–4406302.
• Graph builds are 4406283 (Pursuit) and 4406293 (Gathering).
• Each OM job feeds the OmSh job submitted right after it, through afterok.
• C: 30 jobs, 4406303–4406332, alternating certified study and attribution per (arm, seed).
• A, secondary readout (19 Oct 2026, 01:45). Five exact-U_∞ post-processing jobs, 4409647–4409651 (Bertrand, Chicken, Congestion, Pursuit, Gathering), run scripts/evaluate_frontier_baselines.py from ~/Projects/omsh at 6945e42. Markov Stag Hunt's is 4410156 (19 Oct, 03:05), submitted once its baselines had finished.
• A, tertiary readout (19 Oct 2026, 12:20). 35 risk-dominance audit jobs, 4412460–4412494, one per (environment, OmSh budget), launched by notebooks/experiments/scripts/cx3_frontier_audit.sh from ~/Projects/omsh at 24ab7bb.
• Results to DoC (20 Oct 2026, 05:10). The following were copied to the DoC exports root /vol/bitbucket/oja24/omsh: 142 directories and 537 files, sha256-identical on both sides, with *.lock files left out.
• B's outputs: the frontier_b_* tags, wm_exact_kernel/ and the four om_<tag>/ of Pursuit and Gathering;
• certificate_dominance_frontier/;
• the final reports/frontier_study_2026_10/, which replaced the 19 Oct copy.

With the 19 Oct A and C copy, every study result is on DoC, except C's regenerable shared/ caches. - Runner caveat. CX3's build_param_suffix does not truncate. The executed-notebook names of the long-argument jobs (all of A, and B's OmSh) exceed 255 characters, so live snapshots warn and the final notebook write fails after every cell has run. - Histories, final-policy reports and manifests are written by the cells, so they are unaffected, and the job log keeps the output. - Judge completion by the condition manifests, not the PBS exit status. - B's OM jobs, which the afterok chain depends on, have short names. - Environment-level OM timings on CX3 are last-writer-wins across the four OM replicates and are not used.

Results¶

A. Bertrand (complete 18 Oct 2026, 23:45)¶

Frozen-policy reward per step and worst-seed U_200 over 5 seeds. Source: exports/reports/frontier_study_2026_10/ on CX3.

• Frontier verdicts.
• Against IPPO-Lagrangian: mixed. OmSh wins at τ = 0.01, where no IPPO-Lagrangian level qualifies, and loses at every τ ≥ 0.05 by 0.030 [0.017, 0.042].
• Against CPO: dominates, winning at every τ.
• Budget use: not budget-limited. ΔR(p 0.5 − 0.01) = 0 exactly.
• OmSh's runs are identical across budgets, so its admissible actions never change between p = 0.01 and 0.5. A likely reason: in the repeated game, positive per-step risk under the retained levels compounds towards eventual risk 1, so the actions OmSh can certify are the risk-0 ones at any budget in this range.
• IPPO-TrueShielded does change at p = 0.5 (0.1623), so under the live opponents some intermediate eventual risks exist.
• Either way, the budget does not bind OmSh, and the gap to IPPO-Lagrangian comes from certifying eventual rather than 200-step safety.
• IPPO-Lagrangian is equally insensitive to its cost limit: its episodic cost, about 0.024, stays under every limit tried except 0.
• Under the criterion OmSh certifies, eventual safety, there is no gap to close. Every IPPO-Lagrangian level has U_∞ = 1, and with U_∞ also required OmSh dominates (see "A, secondary").
• A finite-horizon shield would buy the 200-step reward only by giving up the paper's infinite-horizon guarantee. It is not a lever for this paper.
• Its only use would be an ablation that quantifies the price of eventual safety. The right response is to evaluate on the criterion the paper guarantees.
• Reproduction. The published-level CX3 means equal the paper's values to the displayed precision: OmSh 0.159, IPPO-Lagrangian 0.189 with U 0.024, CPO 0.0015, IPPO 1.913. The per-seed check against the archive runs on DoC once the results are streamed there.

A. Chicken (complete 19 Oct 2026, 00:35)¶

• Frontier verdicts.
• Against IPPO-Lagrangian: dominates. OmSh wins at τ ≤ 0.10, where IPPO-Lagrangian's risk of 0.132 disqualifies every level. It ties at τ ≥ 0.16: ΔR = −0.0026 [−0.0068, 0.0016].
• Against CPO: no difference, a tie at every τ (ΔR = +0.004 [−0.021, 0.029] against CPO's best level, c = 0.3).
• Budget use: not budget-limited (ΔR = 0 exactly), as in Bertrand.
• Insensitive baseline. IPPO-Lagrangian's risk again does not respond to its cost limit.
• Reproduction. The published-level means equal the paper's values (OmSh 1.013, IPPO-Lagrangian 1.015 with U 0.132, CPO 1.004, IPPO 2.182).

A. Congestion (complete 19 Oct 2026, 01:25)¶

Frozen-policy reward per step and worst-seed U_64. Both roles are shielded for OmSh and TrueSh.

• Frontier verdicts.
• Against IPPO-Lagrangian: dominates. OmSh wins at every τ, by 0.786 [0.782, 0.791].
• Against CPO: dominated. At c ≥ 12.6, CPO reaches 0.9841 with U_64 = 0.000, above OmSh's best level (p = 0.5, 0.9807) by 0.0035 [0.0017, 0.0052] at every τ.
• This changes a published comparison. The paper's Congestion win, 0.980 against CPO's 0.825, is against CPO at the matched limit c = 6.3. CPO with a looser limit reaches higher reward at zero observed risk. Its exact U_∞ at c = 12.6 is part of the secondary readout.
• Budget use: not budget-limited. ΔR(p 0.5 − 0.01) = +0.029 [−0.039, 0.096]. Only p = 0.01 is lower (0.952); every p from 0.05 to 0.5 gives 0.978–0.981.
• IPPO-Lagrangian is not monotone in its limit. Its reward falls to 0.03–0.06 at c = 12.6 and 31.5, and its risk grows with the limit.
• Reproduction. The published-level means equal the paper's: OmSh 0.980, TrueSh 0.197, IPPO-Lagrangian 0.165 at c = 6.3, CPO 0.825 at c = 6.3, IPPO 0.288.

A. Pursuit (complete 19 Oct 2026, 01:25)¶

• Frontier verdicts. OmSh dominates both baselines at every τ. No baseline level passes any threshold.
• Budget use: not budget-limited. ΔR(p 0.5 − 0.14) = −0.60 [−2.23, 1.04].
• Reward is not monotone in p. It peaks at p = 0.2 (0.331, U 0).
• At p = 0.5 OmSh starts taking real risk (U 0.169), and its reward drops to −0.33.
• The oracle spends its budget almost exactly. TrueSh's realised U_200 tracks p (0.135 at 0.14, 0.476 at 0.5), and its reward falls as p rises. OmSh, certifying against its imagined levels, stays at U = 0 up to p = 0.3.
• Reproduction.
• OmSh 0.251 (p = 0.16), IPPO-Lagrangian −3.742 and IPPO −6.306 equal the paper's values.
• CPO does not: −2.281 (U 0.985) against the published −3.152 (U 0.969). CPO reproduced exactly in the other four finished environments and, in the clean DoC timing run, in all six. So this looks like platform-sensitive CPO training in Pursuit. The per-seed comparison on DoC will show which seeds diverge.
• The verdict is unaffected: every CPO seed fails the gate on both platforms.

A. Gathering (complete 19 Oct 2026, 02:15)¶

• Frontier verdicts.
• Against IPPO-Lagrangian: dominates, a win at every τ. At τ ≥ 0.16 OmSh's margin is 0.016–0.017 [0.006, 0.027].
• Against CPO: dominates. OmSh wins at τ ≤ 0.05, where no CPO level qualifies, and ties at τ ≥ 0.1 against CPO's best qualifying levels (ΔR = −0.0086 [−0.0226, 0.0055]).
• Budget use: not budget-limited. ΔR(p 0.5 − 0.01) = +0.006 [−0.011, 0.022]. OmSh's realised risk rises with p (to 0.050) but its reward does not.
• The oracle converts budget into reward. TrueSh's reward rises from 0.037 to 0.070 as p goes from 0.01 to 0.5, with realised risk about half of p.
• OmSh at the same budgets stays at 0.03–0.04, which is the opponent-model cost F4 found.
• Reproduction. The published-level means equal the paper's: OmSh 0.033 with U 0.012, IPPO-Lagrangian 0.020, CPO 0.028 at c = 0, TrueSh 0.047, IPPO 0.095.

A. Markov Stag Hunt (complete 19 Oct 2026, 08:55)¶

• Frontier verdicts.
• Against IPPO-Lagrangian: dominates, winning at every τ by about 0.60.
• Against CPO: dominates. OmSh wins at τ ≤ 0.3, where no CPO level qualifies (CPO's worst U is at least 0.318). At τ = 0.5 it ties CPO's best qualifying level, c = 0.3: ΔR = +0.005 [−0.028, 0.038].
• With U_∞ also required. Every IPPO-Lagrangian and CPO level has U_∞ = 1.000, so OmSh dominates both at every τ.
• Budget use: not budget-limited. ΔR(p 0.5 − 0.01) = −0.010 [−0.018, −0.003]; a larger budget does not raise reward.
• OmSh matches the oracle (0.587–0.606 against 0.587–0.604, both at U ≤ 0.003) and IPPO's unsafe 0.605.
• Reproduction.
• OmSh 0.592 (p = 0.2), IPPO-Lagrangian 0.007 and IPPO 0.605 equal the paper's values.
• CPO (0.4753 against 0.474) and TrueSh (0.6012 against 0.602) differ in the third decimal, so the per-seed DoC check applies.

A. Summary across the six environments¶

• Primary readout (200-step risk, 64 in Congestion).
• OmSh dominates IPPO-Lagrangian in five environments; Bertrand is mixed.
• It dominates CPO in four, ties it in Chicken, and is dominated by CPO in Congestion. There, CPO with a looser cost limit (c ≥ 12.6) reaches 0.9841 at zero observed risk, against OmSh's 0.9807.
• With eventual safety also required of the baselines, which is what OmSh certifies, OmSh dominates both baselines in five environments.
• Congestion stays the exception: tuned CPO keeps U_∞ ≤ 0.0005.
• OmSh is not budget-limited anywhere. Raising p from 0.01 to 0.5 never raises its reward significantly. Its reward is set by what it can certify and by the opponent model, not by unspent budget.
• The oracle, by contrast, converts budget into reward in Gathering and Congestion.
• In Pursuit, OmSh at p = 0.2 earns 0.331 against 0.251 at the published p = 0.16. That is descriptive only, not a predeclared contrast.
• The cost limit moves the baselines unevenly.
• IPPO-Lagrangian's reward and risk barely change with its limit in Bertrand, Chicken and MSH, and change non-monotonically in Congestion.
• CPO responds clearly in Congestion (0.197 → 0.984), Gathering and MSH (0.475 → 0.601).

A, secondary: exact eventual risk of the baseline levels (19 Oct 2026, 02:15)¶

From scripts/evaluate_frontier_baselines.py (jobs 4409647–4409651): the worst-seed exact eventual unsafe probability (upper bound) of every IPPO-Lagrangian and CPO level.

• Bertrand and Chicken. U_∞ = 1.000 at every level of both baselines. With U_∞ also required, OmSh dominates both. In Bertrand this reverses the primary mixed verdict against IPPO-Lagrangian.
• Pursuit. IPPO-Lagrangian 0.997, CPO 0.983. OmSh dominates both, as before.
• Congestion.
• IPPO-Lagrangian's U_∞ rises with its limit, from 0.0007 (c = 0) to 0.146 (c = 31.5).
• CPO stays at U_∞ ≤ 0.0014 at every limit, and at most 0.0005 for c ≥ 12.6. So even with U_∞ required, CPO at c ≥ 12.6 (0.9841) dominates OmSh (0.9807).
• The Congestion reversal is therefore not an artefact of the 64-step horizon.
• Gathering. Both baselines have U_∞ = 1.000 at every level. With U_∞ also required, OmSh dominates both, which turns the primary ties against CPO at τ ≥ 0.1 into wins.
• What U_∞ does not cover. OmSh's own U_∞ is not computable exactly, because its shield is history-dependent. The comparison asks only that the baselines also meet U_∞ ≤ τ.

A, tertiary: matched eventual criterion (complete 19 Oct 2026, 13:05)¶

35 audits (jobs 4412460–4412494, 5–24 min each) instantiate Theorem 3(ii) on every OmSh frontier level. They use 200 episodes per seed over the evaluation horizon. The table gives the worst seed and role's plug-in q_0 + Ê[Σ η_t], with the 95% upper limit in brackets.

• Consistency. At the published budgets the plug-ins equal the paper's tab:dominance_audit (Pursuit 0.2986, Gathering 0.2002, p elsewhere), as expected from the bit-identical runs.
• Verdicts (vs_<baseline>_anytime). An OmSh level qualifies by its 95% upper bound, a baseline by max(U_H, U_∞).
• OmSh dominates IPPO-Lagrangian and CPO in Bertrand, Chicken, Pursuit, Gathering and MSH.
• In Congestion it dominates IPPO-Lagrangian and is dominated by CPO. It ties at τ = 0.01 and trails CPO at c ≥ 12.6 by 0.004 [0.002, 0.006] for τ ≥ 0.05.
• In Pursuit, OmSh qualifies only from τ = 0.3, at p = 0.14 (0.254). No baseline qualifies at any τ.
• Tolerance. "Within τ" uses a 10⁻⁹ floating-point tolerance (TOLERANCE in scripts/summarize_frontier_study.py). Bounds with no excess come out as p plus about 10⁻¹⁵ of accumulated rounding, which the paper's audit also treats as noise. The tolerance changes no primary or secondary verdict.

Paper additions (19 Oct 2026, omsh-paper 6250aa8)¶

At the user's request:

• Table 1 (final_policy_compact.tex) and the appendix results table (environment_results.tex) give OmSh's U_∞ entry as the worst-seed 95% upper limit of the instantiated Theorem 3(ii) bound at the published p, marked ‡:
• 0.200 in Bertrand, Chicken and MSH; 0.310 in Pursuit; 0.201 in Gathering; 0.100 in Congestion;
• so the appendix table's "U_∞ ≤ p" check now fails for OmSh in Pursuit and Gathering.
• A new appendix subsection, "Reward–safety frontier" (appsec:frontier), covers the protocol, the criteria (a)–(c), the three tables written by scripts/export_frontier_tables.py from frontier_study.json (frontier_omsh.tex, frontier_baselines.tex, frontier_verdicts.tex) and the findings.
• Results and Conclusion no longer say OmSh lies on the frontier in all six environments. They now say it dominates both constrained baselines in five once eventual safety is required of every method, and that tuned CPO beats it in Congestion.
• To regenerate the tables: python scripts/export_frontier_tables.py --summary <frontier_study.json> --output-dir ../omsh-paper/figures/tables. The summary JSON is on CX3 and DoC under reports/frontier_study_2026_10/.

Reproduction of the published levels (19 Oct 2026, 10:10)¶

The A and C results were copied to the DoC exports root /vol/bitbucket/oja24/omsh: 147 directories, 5,660 files, sha256-identical on both sides. C's regenerable shared/ graph caches were left out.

scripts/summarize_frontier_study.py --published-root <AAMAS archive>/cpucluster/exports then compared every CX3 run at its published level, seed by seed, with the paper's (reports/frontier_study_2026_10_doc/ on DoC).

• Bit-identical on every seed (maximum per-seed reward difference 0):
• OmSh, IPPO-Lagrangian and IPPO in all six environments;
• IPPO-TrueShielded in Bertrand, Chicken and Gathering;
• CPO in Bertrand, Chicken and Congestion.
• Not identical: CPO and IPPO-TrueShielded, and only in the three stochastic environments.

• Likely cause: the platform, not the code. The clean DoC timing run reproduced CPO exactly in all six environments from the same code. The likely cause is CX3's floating point in CPO's second-order step and TrueSh's value iteration, where transition probabilities are not 0/1. This is an inference, not a verified cause.
• No verdict depends on these gaps.
• In Pursuit, every CPO seed fails the gate on both platforms.
• TrueSh is an oracle reference. OmSh's 0.251 is above both TrueSh values (−0.93 and −1.67), so the paper's statement that the oracle earns less than OmSh in Pursuit holds on both platforms. Its magnitude is platform-sensitive.
• Not checked: Congestion's two-role TrueSh (archived tag true_shielded_both_roles_v1) is not under the archive's cpucluster/exports.

B. Pursuit (complete 19 Oct 2026, 12:15)¶

Arms (ii)–(iii) shield with the exact kernel. Arm (i) is the published OmSh, with the learned kernel. All run at p = 0.16 on the published trainer seeds, and reward is per step at H = 200.

• ΔR_OM = R̄(iii) − R̄(ii′) = +0.005 [−0.037, 0.047].
• Replicate gaps: learned −0.006 [−0.052, 0.040]; exact-kernel −0.020 [−0.062, 0.021].
• ΔR_total = R̄(iii) − R(i) = +0.097 [−0.062, 0.257].
• Decision: no detectable headroom. Training the opponent levels on exact rather than learned transitions does not change Pursuit's reward. So better world-model exploration has no reward to buy through the opponent model here. This matches F4, which found little through the shield's kernel either.
• Descriptive, not a predeclared contrast. All four retrained OMs (0.338–0.358) beat the published OM, whether with the exact kernel (F4, 0.315) or the learned one (0.251).
• OM training is unseeded, so the published OM looks like an unlucky draw. The clean timing run's retrained OM also gave 0.294.
• Variance across OM training runs therefore matters more for reward than the world model's transition accuracy. Seeding OM training, or choosing among several OMs on held-out data, is the more promising lever.

B. Gathering (complete 20 Oct 2026, 05:10)¶

Same arms and readout as Pursuit, at p = 0.2. The OM jobs took 27–28 h each on CX3. Reward is per step at H = 200.

• ΔR_OM = R̄(iii) − R̄(ii′) = +0.0008 [−0.0032, 0.0047].
• Replicate gaps: learned −0.0022 [−0.0052, 0.0009]; exact-kernel +0.0008 [−0.0030, 0.0046].
• ΔR_total = R̄(iii) − R(i) = −0.0015 [−0.0089, 0.0059].
• Every arm passes the gate. The largest Wilson upper limit is 0.027 (F4), against p = 0.2.
• Decision: no detectable headroom. Training the opponent levels on exact transitions does not change Gathering's reward either.
• Descriptive, not a predeclared contrast. Pursuit's retraining effect does not replicate here.
• The four retrained OMs (0.030–0.032) earn no more than the published OM (0.033). Replicates differ by up to 0.002, more than either kernel change moves reward (at most 0.001).
• F4's evaluation is identical to the published run's on seeds 2200 and 2201, in both reward and unsafe rate.

B. Summary across Pursuit and Gathering¶

• No detectable headroom in either environment.
• Training the opponent levels on exact rather than learned transitions changes reward by +0.005 [−0.037, 0.047] in Pursuit and +0.0008 [−0.0032, 0.0047] in Gathering.
• The upper limits rule out gains of more than about 14% (Pursuit) and 15% (Gathering) of the learned-graph arm's reward.
• What this bounds. An exact transition kernel is the most that a better-explored world model could give the OM. So, within this resolution, WM exploration buys no reward through the opponent model.
• C found no certification gain either. Together, the two studies give novelty-driven WM exploration no measured benefit at these budgets.
• OM-training variance is environment-specific.
• In Pursuit, the published OM looks like an unlucky draw: retrained OMs earn 0.338–0.358 against 0.251.
• In Gathering, retraining does not help: 0.030–0.032 against 0.033.
• So seeding OM training, or selecting among several OMs on held-out data, is supported by Pursuit alone. It would need its own predeclared test, for example k OMs per environment, selected on held-out data and evaluated on fresh trainer seeds.
• Safety. Every B arm in both environments passes the gate.

C. Novelty collection (complete 18 Oct 2026, 22:55)¶

Gathering, seeds 0–4, at the full 750k-transition stream and δ_cov = 0.05. Differences are paired by seed against the PPO collector, with 95% t intervals. Source: exports/reports/frontier_study_2026_10/frontier_study.{md,json} on CX3.

• Primary: not met. No novelty seed is feasible, at either β.
• Certified and zero-radius estimated reset demand are 1.0 in every cell, so their differences are censored at 0.
• Coverage rises but certification does not. β = 1.0 triples the visited-row fraction. Yet the fraction of rows still at radius one stays at 0.130–0.133 in every arm, and most critical rows stay unvisited.
• Why. Under the study's Weissman radius (δ_row = δ_cov / 1.7M kernel rows), a row's radius stays at one until about 13 visits, and falls below 0.1 only after about 1,700 visits (support sizes 2–5).
• Count novelty rewards new rows, so it spreads visits thinly. Certification needs depth on the critical rows.
• Even perfectly targeted, about 8,600 critical rows × 1,700 visits is about 15M transitions, 20 times the 750k budget.
• Cost. Collection-phase unsafe transitions rise by about a third at β = 0.1 and about double at β = 1.0, since unsafe states are novel too.
• Reading. At this budget, count-based novelty exploration does not make Gathering's certified shield deployable.
• A useful exploration objective would have to concentrate visits on the critical rows until their radii shrink (depth, not breadth), or the certificate would need a tighter per-row bound.
• Whether novelty helps reward through a better WM and OM is study B's question.