Configuration and CLI¶

Marimo notebooks expose selected dictionary keys as CLI options. Use --kebab-case=value; keys map case-insensitively to the notebook's snake-case configuration. Unknown keys fail early.

Experiment Keys¶

Key	Meaning	Default
`timesteps`	Target environment steps per algorithm run.	250k (Congestion), 750k (matrix games), 3M (other gridworlds)
`num_runs`	Independent algorithm seeds.	3 (Congestion 5)
`rollouts`	Steps collected per policy update.	2048
`learning_epochs`	Optimizer passes over a rollout.	4
`mini_batches`	Rollout partitions per epoch.	4
`learning_rate`	Learner step size.	2e-4 (matrix games), 3e-4 (gridworlds)
`max_risk`	Initial eventual-unsafe reachability budget.	0.2; Congestion 0.1; Pursuit 0.16 (0.11 reproduces the product-Bernoulli campaign)
`use_wandb`	Enable Weights & Biases run logging.	On for Congestion and Markov Stag Hunt, otherwise off
`n_agents`	Environment team size where supported.	Environment-specific
`experiment_tag`	Isolate histories and diagnostics below `experiments/runs/<tag>/`.	Empty (canonical run)
`seed_offset`	Non-negative first trainer seed for a disjoint run block.	0
`model_variant`	Isolate every artifact below `model_variants/<name>/`.	Empty
`reward_game_variant`	Bertrand/Chicken payoff variant: `default` or `private_heterogeneous`.	`default`

timesteps is rounded up to at least one complete rollout/update. For a small smoke run, reduce rollouts as well as timesteps. The campaign protocol overrides several of these defaults; see Reproduce the Campaign Protocol.

Stage Switches¶

run_ippo
run_ippo_lag
run_cpo
run_ippo_shielded
run_ippo_true_shielded
run_exact_anytime_diagnostic
just_vis

All run_* values default to false. The exact-anytime switch runs an offline certificate diagnostic rather than an algorithm. Select only the stages a scripted run should execute. Cluster algorithm launchers own their training switches and reject caller overrides.

Shield Keys¶

Key	Values and meaning	Default
`shield_action_mode`	`pure`, `egalitarian_mixed`, `learned_budget_pure`, or `learned_budget_mixed`. Non-pure modes select parameterized PPO automatically.	`pure`
`shield_opponent_mode`	`monotone_floor`, `credible_floor`, `all_levels`, or `bayesian_mixture`.	`monotone_floor`
`shield_reward_mode`	`bayesian` or `robust`; this ranks actions only after the safety test.	`bayesian`
`shield_patience`	Consecutive supported posterior updates needed for one floor raise; `0` is eager.	64
`shield_padding_radius`	Total-variation radius in `[0, 1]` around every retained level's row. `0` is the deployed shield; rejected with `bayesian_mixture`.	0
`shielded_agent_ids`	Comma-separated roles to shield (Congestion and Markov Stag Hunt). Several roles require `pure` and one opponent model per role.	`player_0`
`shield_credible_delta`	Posterior-tail allowance for `credible_floor`.	0.05
`shield_credible_schedule`	`summable` or pointwise `fixed`.	`summable`
`shield_version`	Free-form version token for the shield artifact name.	`v0`
`shield_debug_mode`	`off`, `minimal`, or `verbose` runtime fields.	`minimal`
`shield_trace_limit`	Maximum in-memory verbose decision records.	256
`shield_missing_coverage_log_limit`	Cap on repeated missing-coverage log messages.	5
`true_policy_refresh_interval_updates`	Policy-update interval for rebuilding the exact-policy shield.	1 (Congestion and three-agent Pursuit 64)

run_exact_anytime_diagnostic=true uses max_risk as both p_total and p_shield because the serialized exact-graph route reports delta_cov=0 relative to its matching environment contract and declared reset support. Its artifacts are written below true_shield/exact_anytime/; the switch does not consume timesteps, num_runs, or PPO configuration.

The pure/monotone-floor/Bayesian-reward/unpadded/v0 condition retains the canonical ippo_shielded_* artifact names. Other action, opponent, reward, padding, or explicit shield-version conditions receive deterministic suffixes. Coordinated runs default their version to the agent set, for example agents_0_1.

Final-Policy Evaluation Keys¶

Key	Meaning	Default
`final_policy_eval_episodes`	Frozen evaluation episodes per run after training; `0` disables it.	0 (Congestion 2,000)
`final_policy_eval_horizons`	Horizons `H` for `U_H = P(T_unsafe <= H)`.	`(1, 10, 50, 100, 200)`
`final_policy_eval_seed_offset`	First evaluation seed; keep it disjoint across campaigns.	1,000,000
`final_policy_eval_confidence`	Wilson interval level.	0.95

See Final-Policy Evaluation.

Diagnostics Keys¶

Key	Meaning	Default
`verbose_data`	Add bounded raw traces and sampled policy/IOP comparisons; aggregate diagnostics remain always on.	false
`log_opponent_level_gap`	Log the closest-retained-level opponent gap on sampled rollout states, without the heavy traces of `verbose_data`.	false
`diagnostic_interval_updates`	Update interval for sampled live-policy/IOP diagnostics.	1
`diagnostic_state_sample`	Maximum sampled rollout states per diagnostic comparison.	8192
`data_trace_stride`	Keep one raw shield decision per this many decisions.	100
`max_trace_rows`	Hard cap on exported trace rows.	100,000

These controls reuse collected rollouts; enabling them does not add environment steps.

World-Model Keys¶

Key	Meaning	Default
`wm_variant`	`successor` (masked categorical kernel) or `bit_mlp` (product-Bernoulli baseline).	`successor`
`n_steps`	Replay transitions collected for world-model training.	25k (matrix games), 250k (Congestion), 750k (other gridworlds), 1M (three-agent Pursuit)
`n_agents`	Team size for configurable environments.	Environment-specific
`reward_inference_mode`	Bertrand/Chicken: `observed` keeps the opponent's reward, `inverse_qre` infers it from joint actions.	`observed`
`reward_inference_window_size`	Window size for the inverse-QRE fit.	250

Do not confuse WM n_steps with experiment timesteps. The dependency-aware cluster launcher routes them to different notebooks.

Opponent-Model Keys¶

Key	Meaning
`focal_idx`	Role to model as focal; `1` writes `om/agents/player_1/`. Congestion and Markov Stag Hunt.
`n_levels`	Number of reasoning levels, where the notebook exposes it (Congestion; default 3).
`om_`, `agent_updates`, `opp_updates`, `imagination_`	Network, level-0 fitting, and imagined policy-improvement budgets.
`opponent_rollout_horizon`	Imagined opponent response depth; the private-reward ablation requires `1`.

The detailed neural and imagination defaults live in each OM notebook's configuration block. On the cluster, om_focal_indices=0,1 creates one OM job per role.

Value Coercion and Validation¶

CLI values representing integers, floats, booleans, or None are converted to their Python values. Both --n-agents=3 and n_agents=3 work in the relevant runner context. Unknown keys and duplicate normalized keys fail early, which protects long cluster jobs from silently ignoring a typo. Tuple values such as final_policy_eval_horizons=(1,2,10,64) must be quoted for the shell.

Example baseline smoke run:

uv run python notebooks/matrix/chicken/experiments_mo.py \
  --run-ippo=true \
  --num-runs=1 --timesteps=64 --rollouts=64 --use-wandb=false