Autoregressive Joint Opponent Model¶

The active imagined-opponent interface retains one labelled joint opponent and one Bayesian posterior over team reasoning levels. With multiple opponents, its scalable parameterisation changes the flat A**N-class output head to the exact chain rule

p(a_1, ..., a_N | s, m) = product_j p(a_j | s, a_1, ..., a_{j-1}, m).

AutoregressiveJointOpponent uses a shared recurrent decoder, action embeddings, and labelled position embeddings. The decoder order is an internal probability factorisation; environment actions remain simultaneous. Position labels preserve the existing environment-order semantics rather than introducing independent or exchangeable per-agent models.

Exact interfaces¶

• observed_log_probs(...) teacher-forces the complete observed opponent tuple and returns its exact joint log-likelihood in work linear in the number of opponents. The Bayesian level posterior therefore keeps the existing team-level update without requiring an A**N output vector. The current updater also exports exact joint Brier, confidence, and argmax diagnostics, so that telemetry still invokes the enumerated compatibility path; it can be disabled or replaced by observed-action log scoring if posterior-update throughput becomes the bottleneck.
• sample_actions(...) draws one correlated joint action sequentially from the exact chain-rule policy.
• joint_log_probs(...) enumerates and exactly normalises every labelled joint action. This compatibility path is used by the current graph augmentation, full-distribution diagnostics, and soft-rollout policy-improvement interfaces.
• forward(...) returns these exact joint log-probabilities as compatibility logits. No probability thresholding, top-k truncation, independence assumption, or Monte Carlo marginal replaces the joint policy.

The representation therefore removes the exponential final layer and sparse joint-class likelihood at level 0, but it does not pretend that exact shield construction is polynomial. Current higher-level rollout improvement and induced-kernel construction still enumerate the legal joint action space. End-to-end scaling additionally requires structured transition/reward/safety factors or another exact marginalisation representation.

Configuration and checkpoints¶

MBOMConfig records architecture, action_size_opp, and n_opponents, and validates n_actions_opp == action_size_opp ** n_opponents. Its auto mode resolves one opponent to the legacy flat categorical MLP and multiple opponents to the autoregressive joint model. The single-opponent case has no exponential joint head to remove, and preserving the flat path keeps the established two-agent paper pipeline and its fresh-training parameterisation unchanged. Its negative log-likelihood deliberately retains the existing cross_entropy kernel as well: expanding that operation into an equivalent log_softmax and gather changed the float32 loss by one ulp in the compatibility test. The resolved choice is stored in new checkpoints. A checkpoint containing no architecture field is likewise treated as a pre-autoregressive flat checkpoint and reconstructed with OpponentMLP, so existing experiment artifacts remain loadable without silently changing their policy.

Verification¶

Unit and integration tests cover automatic single-/multi-opponent architecture selection, base-A action round trips, exact normalisation, equality between teacher-forced and enumerated likelihoods, gradients, parameter scaling, new and legacy checkpoints, Bayesian level updates, and a multi-opponent soft-rollout update. scripts/benchmark_congestion_scalability.py adds a reproducible 2/4/6/8-agent workload reporting exact graph size, fit and likelihood time, held-out joint metrics, parameter count, peak RSS, and the numerical difference between teacher-forced and enumerated probabilities. The eight-agent run requires --skip-graph; the explicit guard prevents accidental construction of the 86,100,003-edge materialised graph while still exercising eight-agent rollouts, autoregressive fitting, exact observed-tuple scoring, and the enumerated likelihood oracle.

Level-0 classification diagnostics retain a dense confusion matrix through 512 joint classes. Above that threshold, Level0SplitMetrics records the exact matrix shape plus sorted nonzero (observed, predicted, count) entries. This changes only storage: every observed confusion count is preserved, while an eight-agent, three-action OM no longer turns 2,187 classes into 4,782,969 Python integers. MBOMConfig.dense_confusion_max_actions stores the threshold in new checkpoints.

DoC CPU scalability run¶

Jobs 7260--7262 ran commit 0f368898 together on amd24 node auklet10, each with 8 CPUs and 32 GiB. Job 7302 subsequently ran the eight-agent no-graph condition from commit 89107106 with 4 CPUs and 32 GiB. Every run used 10,000 pretraining transitions, 8 epochs, hidden size 96, batch size 512, seed 2026, and the same flat-output control. Full JSON artifacts are under /vol/gpudata/oja24-omsh/exports/benchmarks/convention_merge/2026-08-15-ar-exact/ on cpucluster.

Teacher-forced and enumerated log probabilities agreed exactly at the observed actions in all four runs; maximum normalisation error was 4.17e-7. The recurrent core is larger than the flat control through six agents, but adding opponents changes only position embeddings and decoder work instead of multiplying the output head; by eight agents it is less than one third the size of the flat head. Exact observed-tuple scoring remains below 5 ms for 256 states at eight agents, whereas asking for all 2,187 opponent tuples takes about 9.57 seconds. The 379-second eight-agent fit includes exact dense argmax/calibration diagnostics over every joint class; the teacher-forced likelihood kernel itself remains linear in opponent count. The comparable held-out cross-entropies show that the smoke test preserved useful joint predictive capacity; they are not a claim that one optimiser or fixed eight-epoch budget is optimal.