Joint Graphs¶

Joint graphs are the common transition representation used by exact dynamics, world modelling, opponent modelling, and shielding.

Data Model¶

A Graph contains:

• nodes: state key → flattened public state bits,
• edges: legal transitions, and
• start_keys: possible reset states.

Each Edge stores src, dst, the full joint_action, learned or exact env_prob, optional opp_prob, their product prob, and rewards in environment agent order.

State keys identify the public Markov state. The graph-backed opponent path is strict: two nodes may not contain the same public bits under different latent keys because an observation-only policy could not tell them apart.

Graph Lifecycle¶

exact builder
    |
    v
legal_joint_graph.pkl
    |
    +-- world-model scoring --> env_transition_graph.pkl
                                      |
                                      +-- opponent level 0 --> focal MDP L0
                                      +-- opponent level 1 --> focal MDP L1
                                      +-- ...

Opponent probabilities are derived when building a level-specific shield; they are not a separate canonical base graph artifact.

Focal-MDP Induction¶

The joint graph contains actions for every agent. induce_focal_mdp(...) marginalizes the opponent components using env_prob * opp_prob and groups the result by focal action. Sound value iteration then works on the induced graph.

Successor Sets¶

The legal graph's successor set Succ(s, a) for each (state, joint_action) group is what the learned world model distributes probability over. The default successor kernel reads these sets directly from wm/legal_joint_graph.pkl, which the WM notebook builds before training, so a learned row can never place mass outside the legal topology. The same sets define the support of the count-centred ambiguity sets.

Coverage and Scale¶

The graph must expose every legal joint action at every reachable decision state. Stochastic successor fan-out may create several edges for one state/action group. Graph size grows exponentially with agent count. The exact Congestion graph illustrates the growth:

The graph itself is not the only cost. Opponent-model imagination and induced-kernel construction enumerate the joint opponent action space, so six-vehicle Congestion is limited by opponent-model training time rather than graph memory. Three-agent Pursuit and Markov Stag Hunt are large enough that graph-sized stages need deliberate memory planning; see Cluster Execution.