Retired OmSh Benchmarks (August 2026)¶

The active OmSh paper suite retains Gathering, Markov Stag Hunt, Pursuit, Bertrand, and Chicken. Ice Duel, Congestion, Dynamic Public Goods Game (DPGG), and Inspection were removed from the active suite after environment-specific tuning because their best monotone OmSh conditions did not establish a useful constrained-reward advantage over the non-oracle baselines. The final five-seed credible-floor screen also found no safety-qualified reward rescue, so the retirement is final for the current paper suite.

Retirement removes these environments from the OmSh package exports, notebooks, discovery registry, exact-graph routing, launchers, and active test matrix. It does not erase their history. The complete implementations and notebooks immediately before retirement are recoverable from commit ad793868, and historical result artifacts remain on cpucluster.

Decision evidence¶

The final screen used 1,000 frozen-policy episodes per seed at horizons 1,10,50,100,200. U200 max is the worst observed seed and Wilson max is the largest seed-level 95% upper confidence limit. All conditions pass the environment's 0.20 finite-window gate, so reward decides among the passing conditions.

This is a benchmark-selection decision, not a claim that CPO has an anytime guarantee. Exact frozen-policy evaluation gives eventual unsafe reachability one for Congestion, DPGG, and Inspection CPO, while Ice Duel CPO remains below its eventual gate. Executed OmSh is stateful and has a separate learned-model certificate, so it is not assigned a public-state exact value. The retired environments either structurally collapse OmSh reward under eventual-risk shielding or fail to give a useful empirical constrained-reward comparison; the retained suite still contains continuing/infinite-horizon targets.

Posterior-credible shielding was the final rescue condition. For the current three-level model and delta <= 0.05, its candidate floor cannot exceed the MAP monotone candidate at the same posterior, and under a matched total risk target it also reserves delta from the model risk budget. That pointwise ordering is preserved by the one-level patience operator when the rules see the same posterior sequence: whenever the credible rule requests a raise, the MAP rule requests the same next increment. Credible shielding therefore has no intrinsic feasible-set or optimal-reward advantage over monotone shielding under matched histories. Separate learners can nevertheless diverge onto different state and posterior histories, and extra conservatism can sometimes regularise finite-sample optimisation. Six five-seed credible-only runs therefore test Ice Duel, pure and mixed Congestion, DPGG, and pure and learned-budget Inspection under tag screen_credible_retirement_v2. The merged report is /vol/gpudata/oja24-omsh/exports/reports/screen_credible_retirement_v2/final_policy_campaign.json. Its primary-campaign complete field is intentionally false because this was a credible-only retirement screen rather than a rerun of IPPO, Lagrangian, CPO, and Monotone; every one of the six requested credible conditions does contain five seeds.

rl-engine handoff¶

All four environments already live in ~/Projects/amethyst/rl-engine:

• src/rle/environments/multi_agent/ice_duel
• src/rle/environments/multi_agent/congestion
• src/rle/environments/multi_agent/dpgg
• src/rle/environments/multi_agent/inspection

Each implementation uses rl-engine's RenderEnvMixin and reusable renderer controllers, so no additional port or renderer replacement is required.

Artifact policy¶

Historical evaluation and final-run exports remain under the main OmSh export tree. Tuning run trees and their generated reports were moved intact to /vol/gpudata/oja24-optimisations/exports/tuning so optimisation evidence is clearly separated from final evaluation artifacts. Nothing was deleted.