Skip to content

Joint Graph Architecture

The project now treats environment structure as a single joint state-action graph.

Core model

  • legal_joint_graph.pkl stores exact legal states and legal joint-action transitions only.
  • env_transition_graph.pkl stores that same topology plus learned env_prob and rewards.
  • Opponent probabilities are added later as derived edge annotations (opp_prob, combined prob) and are not persisted as a separate base artifact.
  • Cached legal_joint_graph.pkl artifacts are only valid for the current public-state encoding. Notebook loaders should validate cached node widths against the current environment state width and automatically rebuild stale caches after env encoding changes.
  • Graph and metadata pickles are staged in the destination directory and atomically replaced after a successful flush. A truncated legacy pickle is treated as a cache miss and rebuilt, so an interrupted multi-gigabyte graph write cannot poison retries.

This matches the intended split of responsibility:

  • environment knowledge: exact legal state/action structure
  • world model: learns P_env(s' | s, a_joint) and rewards
  • opponent model: learns opponent action probabilities

Exact legality

Legality is no longer discovered by generic sampling in the default pipeline.

  • The entrypoint is src.joint_graph.build_legal_joint_graph(...).
  • Environment-specific exact builders live in src/joint_graph/exact/.
  • Current exact builders cover the matrix environments plus the supported gridworlds used in the main notebooks.

GraphWorldModelAdapter now defaults to strict legality. Missing states or missing (state, joint_action) groups should be treated as data/integration errors, not silently converted into self-loops.

Active pipeline

World modelling:

  1. build or load legal_joint_graph.pkl
  2. learn environment transition probabilities and rewards
  3. save env_transition_graph.pkl

Opponent modelling:

  1. load env_transition_graph.pkl
  2. learn opponent action distributions on top of that exact legal structure

Shielding:

  1. load env_transition_graph.pkl
  2. augment with opponent policy probabilities
  3. induce the focal MDP
  4. run Sound Value Iteration

The shield wrapper is now TransitionShield and is explicitly based on the learned environment-transition graph path.

Removed concepts

  • src.true_dynamics has been removed from the active architecture.
  • perfect_dynamics.pkl is no longer part of the default pipeline.
  • The old sampling-oriented manage_dynamics(...) flow has been replaced by exact legal graph construction.

The training and experiment notebooks use the joint-graph architecture and the two-artifact workflow above.