Opponent Modelling¶
Opponent modelling predicts the joint action of every non-focal agent. The
runtime object is ImaginedOpponent, a stack of neural OpponentMLP policies.
Level 0: Observed Behavior¶
The opponent buffer stores public states and observed opponent-team actions.
For multiple opponents, their tuple is encoded as one base-A class, so a
level outputs A ** (n_agents - 1) logits.
fit_level0(...) uses cross-entropy and reserves a 10% validation holdout by
default. It reports sample count, cross-entropy, accuracy, error rate,
multiclass Brier score, expected calibration error, mean confidence, action
counts, and the confusion matrix for both splits. A one-sample buffer cannot
form a holdout and reports unavailable holdout metrics as NaN in memory and
null in the JSON export.
Higher Levels: Imagined Responses¶
Each higher level is warm-started from the preceding policy and trained with
soft-rollout policy improvement on GraphWorldModelAdapter. The adapter samples
only legal joint actions and learned environment successors. A counted root
state sample may expand into several imagined environment steps, so OM progress
budgets are not directly comparable to real environment timesteps.
The OM stage writes level0_fit_report.{json,txt}, om_report.{json,txt}, and
pi_training_metrics.csv. The staircase report includes training history,
level entropies, and pairwise policy discrepancy over sampled diagnostic states.
Runtime Mixture¶
mix_alpha starts uniform over levels. After observing an opponent action, the
model:
- evaluates its likelihood under every level,
- combines likelihood and the current mixture,
- applies temperature smoothing, and
- moves toward the posterior with an exponential update.
By default, the learned shield does not simply average all level certificates.
It tracks a monotone floor and requires the selected action to be feasible for
every level at or above that floor. shield_patience requires sustained
posterior evidence before raising the floor. The optional all_levels and
bayesian_mixture shield modes provide conservative and posterior-predictive
ablations without changing how mix_alpha itself is updated.
Artifact¶
The trained stack is saved as:
It is paired with the same environment configuration and
wm/env_transition_graph.pkl used during imagined training. Changing agent
count, public-state width, action size, or graph semantics requires compatible
artifacts to be rebuilt.