Opponent Modelling¶
Opponent modelling estimates how the non-focal agents are likely to act.
The active opponent model is an imagined opponent stack. In code, the main runtime object is ImaginedOpponent.
Level 0¶
Level 0 is trained from real transition data.
The training data contains:
- a focal-agent state representation,
- the observed opponent team action,
- an encoded joint action class when there is more than one opponent.
The model is a neural network policy, OpponentMLP, trained with cross-entropy to predict the observed opponent action.
Higher Levels¶
Higher levels are trained with imagined rollouts in the learned world-model graph.
The intuition is:
- level 0 imitates observed opponent behavior,
- level 1 reasons against the lower level,
- later levels continue the staircase of imagined responses.
Each level is still a neural opponent policy. The project does not store a separate base graph for every opponent level; opponent probabilities are applied as derived annotations on top of the environment transition graph.
Mixture Over Levels¶
At runtime, the opponent model keeps a mixture over reasoning levels.
When the shield observes an opponent action, it updates the level mixture using the likelihood of that action under each level. The shield then uses the current mixture and a monotone level floor to decide which levels must be tolerated.
MBOM in This Repo¶
Older notes use MBOM to describe the mixture-based imagined opponent model. In the current codebase, read MBOM as the ImaginedOpponent stack plus:
MBOMConfigfor level count and training settings,OMBufferfor opponent-model training samples,GraphWorldModelAdapterfor imagined graph rollouts,- soft-rollout policy improvement for higher levels.