Ablations¶

Ablations change one ingredient of OmSh and keep everything else at the campaign protocol. Each one writes under its own tag or export root, so it never overwrites the primary condition. Compare ablations on matched trainer seeds and with frozen final-policy evaluation.

Ablation	What changes	How it is selected
Replacement rule	Pure versus mixed actions, egalitarian versus learned budgets	`shield_action_mode`
Opponent uncertainty	Monotone floor versus credible, all-level, or Bayesian handling	`shield_opponent_mode`, `shield_patience`
Padding radius	Certify against a TV ball around each level	`shield_padding_radius`
Model strength	10% training budget for the world or opponent model	`notebooks/ablations/`
World-model family	Successor kernel versus product-Bernoulli	`wm_variant`
Private-reward inference	Hide the opponent's reward from pre-training	`reward_game_variant`, `reward_inference_mode`

Replacement Rule¶

The primary condition uses pure actions with egalitarian successor budgets. The other three cells of the pure/mixed × egalitarian/learned-budget design run on the same notebook:

uv run python notebooks/matrix/chicken/experiments_mo.py \
  --run-ippo-shielded=true --shield-action-mode=egalitarian_mixed \
  --experiment-tag=<variant_tag> --seed-offset=2200 --num-runs=5 \
  --timesteps=250000 --final-policy-eval-episodes=1000 \
  --final-policy-eval-seed-offset=2200000

Use the same seed_offset as the control so each variant seed pairs with the control seed that shares its initialization, then run scripts/summarize_replacement_ablation.py. Randomised modes can spend the whole budget, removing the slack that absorbs model error, so check the safety gate before reading reward.

Opponent Uncertainty¶

shield_opponent_mode=credible_floor (published as OmSh-Credible), all_levels, and bayesian_mixture change which opponent levels the shield must tolerate. shield_patience changes how much posterior evidence a floor raise needs. The default is 64. Patience 512, 1024, 2048, and 4096 were screened against 64 on disjoint seeds under a predeclared rule; scripts/summarize_patience_screen.py applies it. See the screen protocol.

Padding Radius¶

shield_padding_radius=r trains with the padded certificate described in Guarantees and Robustness. Log the opponent gap in the same run so coverage can be checked afterwards:

uv run python notebooks/gridworlds/gathering/experiments_mo.py \
  --run-ippo-shielded=true --shield-padding-radius=0.15 \
  --log-opponent-level-gap=true --experiment-tag=<padded_tag> ...

scripts/summarize_padded_runs.py pairs padded runs with the campaign runs and marks a run as covered when the sampled gap never exceeded r minus the world-model error.

Model Strength¶

notebooks/ablations/<env>/ mirrors the three-stage pipeline for Pursuit (two agents) and Markov Stag Hunt, with two conditions:

`--ablation`	World model	Opponent model
`weak_wm_strong_om`	10% budget: 75,000 collection steps, 6 epochs, 200-update successor floor	Canonical
`strong_wm_weak_om`	Canonical	10% budget: 3 level-0 epochs and reduced imagined-update counts

uv run python notebooks/ablations/pursuit/train_wm_mo.py --ablation=weak_wm_strong_om
uv run python notebooks/ablations/pursuit/train_om_mo.py --ablation=weak_wm_strong_om
uv run python notebooks/ablations/pursuit/experiments_mo.py --ablation=weak_wm_strong_om

Artifacts live under exports/ablations/<env>/<ablation>/ with the normal wm/, om/, shield/, and experiments/ layout. The experiment notebook trains only the shielded condition and loads the canonical baselines for comparison. Its end-of-run plotting holds the full history in memory, so give cluster jobs at least 16 GB. A weak model can make the shield infeasible at the campaign budget; that is a result, not a failure to fix.

World-Model Family¶

wm_variant=bit_mlp retrains the product-Bernoulli baseline in place of the default successor kernel. Because the model-free baselines do not consume a world model, a campaign rerun with a different wm_variant only changes the OmSh rows. scripts/compare_final_policy_campaigns.py uses the model-free rows as a reproduction check. Remember that the feasible budget depends on the world model (see Safety Budgets).

Private-Reward Inference¶

In Chicken and Bertrand, reward_game_variant=private_heterogeneous gives player_1 a distinct payoff table while leaving the focal payoff and unsafe event unchanged. reward_inference_mode then controls what the focal agent's pre-training buffer may use:

`reward_inference_mode`	Opponent reward in `wm/D.pkl`
`observed` (default)	The true private reward.
`inverse_qre`	Removed, then replaced by a pseudo-reward fitted from joint actions alone with a windowed inverse quantal-response model.

Inferred rewards are accepted only with opponent_rollout_horizon=1, and the observed control uses the same horizon so the comparison isolates inference. model_variant=<name> places every artifact under model_variants/<name>/ so these runs cannot overwrite the primary campaign. Pass the campaign's algorithm settings explicitly (seed_offset, timesteps, final_policy_eval_episodes, and final_policy_eval_seed_offset), because the notebook defaults differ. Summarize with scripts/summarize_reward_inference_ablation.py.

See the private-reward ablation note for the estimator, identification limits, and launch commands.