eval_250k_v1 Pilot Campaign¶
Scope¶
This is the first fresh-policy screening campaign after the reward-aware action replacement, successor-budget, and opponent-uncertainty changes. It compares IPPO (unshielded), IPPO-Lagrangian, ICPO/CPO, and the primary OmSh condition. True shielding is deliberately excluded. Each condition receives 250,000 environment steps and seed indices 0, 1, and 2.
The primary OmSh condition is pure primitive action selection, robust egalitarian successor-budget allocation, monotone_floor opponent handling, and Bayesian expected-reward ranking. All policies start from fresh initialization. OmSh reuses the existing environment-specific world-model graph and imagined-opponent stack because this campaign did not intentionally change the state, action, dynamics, or safety-label contracts. The Pursuit exception discovered during the campaign is recorded below.
The configured max_risk is 0.20 except for Pursuit, where it is 0.25. Promotion also has a separate hard empirical gate: neither observed per-step violations nor unsafe-episode frequency may exceed 0.40 in any seed, considering both the whole run and the final 20% window.
The constrained baselines retain the paper's documented cost_limit=0.0. This makes the pilot faithful to the existing experimental protocol, but it is not a chance-constraint-budget match: ICPO and IPPO-Lagrangian constrain a discounted cost return, while OmSh constrains modelled unsafe reachability. Any later claim about a matched reward-safety frontier must therefore tune the baseline cost limits and OmSh risk budgets as separate method-specific trade-off parameters rather than equating their numeric values.
Jobs¶
The jobs run on the DoC CPU cluster amd96 partition with 32 GiB requested memory each. They use repository commit b174193c and write under each environment's experiments/runs/eval_250k_v1/ directory.
| Job | Environment | Agents |
|---|---|---|
| 6714 | Inspection | 2 |
| 6715 | Bertrand | 2 |
| 6716 | Chicken | 2 |
| 6717 | Congestion | 2 |
| 6718 | DPGG | 2 |
| 6719 | Gathering | 2 |
| 6720 | Ice Duel | 2 |
| 6721 | Markov Stag Hunt | 2 |
| 6722 | Pursuit | 2 |
The original Inspection canary, job 6709, failed before training because the deployed Marimo batch helper imported an API removed by Marimo 0.23.16. The helper now supports both the old and current export APIs; the setup repository fix is commit cf2dd2a. Job 6714 is the successful replacement canary. This failure produced no experiment histories.
Reporting¶
Commit 3aeb5154 adds scripts/summarize_evaluation.py. It reports whole-run and final-window values after aggregating episodes within each seed, paired OmSh-minus-baseline deltas, seed dispersion and extrema, the empirical safety gate, and detailed shield telemetry. The environment rollup distinguishes strict improvement on both reward and safety from Pareto dominance, where OmSh may tie a zero-violation baseline on safety while improving reward. Three-seed results are screening evidence, not a basis for strong significance claims.
The Markdown footer derives its seed count from the completed condition summaries. It must not describe a five-seed promotion as a three-seed screen, and it reports a range when completed conditions contain different seed counts.
The report command is:
Status¶
All nine replacement jobs entered training successfully on 2026-08-10. The
Pursuit result from job 6722 is invalid as an OmSh performance estimate. Its
legal and supposedly exact transition graphs both omitted the live
environment's independent per-agent action-slip outcomes. This shared omission
made their reported transition TV artificially zero while observed safety was
badly miscalibrated. Commit 67f09f58 repairs the stochastic support and exact
probabilities. GPU jobs 272262 and 272263 rebuild the two-agent Pursuit WM and OM
in dependency order before a fresh Pursuit comparison. See
pursuit-stochastic-dynamics.md.
The first completed pilots also identified targeted parameterization tests:
| Job | Environment | Variant | Reason |
|---|---|---|---|
| 6729 | Inspection | egalitarian mixed action | only half of pure actions were available while the risk budget remained unused |
| 6730 | Gathering | learned budget, pure action | the late carried budget collapsed despite substantial empirical safety slack |
| 6731 | Bertrand | egalitarian mixed action | only half of pure actions were available while the risk budget remained unused |
| 6734 | Chicken | egalitarian mixed action | only half of pure actions were available while the risk budget remained unused |
| 6733 | Congestion | egalitarian mixed action | only one third of pure actions were available while the risk budget remained unused |
These tuning runs use seed indices 100--102 and are not held-out evaluation. Job 6732 was an empty failed Chicken submission caused by a Marimo kernel semaphore startup failure; 6734 is its clean replacement. Longer runs, larger agent counts, and the two-shielded-agent experiment remain gated on the pilot and targeted-variant results.