Skip to content

Optimisation change register

This is the impact-ordered register of durable implementation and environment changes made during the OmSh optimisation campaign. It excludes discarded experiments, documentation-only refreshes, dependency maintenance, and work performed by other agents.

1. Reward-aware parameterised shielding

  • Added stochastic primitive-action shielding and learned successor-budget allocation as independent, composable policy modes.
  • Kept feasibility projection in the shield and made replacement select the highest-reward member of the budget-admissible action set, rather than the minimum-risk action irrespective of reward.
  • Made shielded PPO train on the action actually executed after projection and replacement.
  • Wired the variants through the standard environment experiments with condition-isolated artifacts and diagnostics.
  • Principal commits: 552a9963, c80a6a03, c77bfd4f.

2. Coordinated multi-agent OmSh

  • Added MultiAgentTransitionShield, which resolves every protected action from the same pre-state and advances the base environment exactly once.
  • Gave each protected role its own posterior, reasoning-level floor, safety budget, induced-model bundle, and transition update.
  • Added role-conditioned opponent-model training and per-agent episode export; the initial supported evaluation is two-agent Markov Stag Hunt.
  • Principal commit: 3c363643.

3. Exact anytime adversarial diagnostic

  • Added exact finite-state value iteration against arbitrary opponent actions and a runtime wrapper for an infinite-horizon eventual-violation diagnostic.
  • Hardened reset-support semantics, role/action validation, convergence and residual reporting, cache provenance, and report wording so learned-model, induced-model, and exact-game claims cannot be conflated.
  • Principal commits: ce1a9c85, 00e55813.

4. Evaluation integrity and observability

  • Added isolated experiment tags, disjoint seed blocks, atomic manifests, condition-aware artifact names, and stale-bundle rejection.
  • Added seed-first aggregation for whole-run and final-20% windows; reward per step and episode; violations per step; unsafe-episode frequency; per-seed paired deltas; worst-seed/window safety gating; and shield budget, risk, coverage, intervention, calibration, and action-availability diagnostics.
  • Added correction-tolerant readers for legacy/partial logs without treating unfinished conditions as completed results.
  • Principal commits: 8141a757, b174193c, 3aeb5154, a62f9b11, 09f446dd, c7a34091, 2f2e9ac4, 7345b19e.

5. Pursuit correctness and learning signal

  • Corrected the learned and exact transition supports to include independent guard and intruder action slips with their live probabilities.
  • Restored the intended sparse interception rewards (+10 to the guard and -10 to the intruder); earlier zero-reward Pursuit returns are invalid.
  • Normalised the reward-head loss by the known reward magnitude so sparse +10/-10 targets do not dominate the shared world-model representation, while leaving predictions, checkpoints, and environment rewards in raw units.
  • Principal commits: 67f09f58, 02eaf442, 962abd78.

6. Ice Duel world-model coverage

  • Added uniform-random live-environment coverage alongside PPO collection and audited every state-action pair against exact dynamics.
  • Reduced exhaustive learned-transition total-variation error and prevented stale learned shield bundles from surviving a model rebuild.
  • Principal commits: 7cef3ef3, 7345b19e.

7. Baseline and launcher correctness

  • Corrected CPO/Lagrangian training semantics and hardened IPPO action handling before comparative evaluation.
  • Restored explicit n_agents configuration in every experiment notebook after compatibility cleanup exposed a launch regression; added an AST-wide launcher contract test.
  • Principal commits: 0e871be5, 519924e7, 17fad92c.