Multiagent Probabilistic Shielding¶
OMSH studies safe decision-making when a focal reinforcement-learning agent shares an environment with other agents whose behavior must be estimated.
The repository combines three layers:
- World modelling starts from an exact legal joint-state graph and learns transition probabilities and rewards from experience.
- Opponent modelling fits a neural level-0 policy from observed actions, then trains higher reasoning levels through imagined rollouts.
- Probabilistic shielding conservatively bounds eventual unsafe reachability and replaces a focal action when it cannot be certified within the current safety budget.
The runtime learners are IPPO, IPPO-Lagrangian, and ICPO. Experiments compare those baselines with learned-shielded IPPO and a true-transition shield baseline. The learned-shield path supports pure actions, pairwise mixed primitive actions, egalitarian or learned successor-budget allocation, and several opponent-uncertainty modes. Safety is defined explicitly per environment and is reported separately from reward.
Experiment runs now export reviewer-facing episode tables, optimizer and
shield diagnostics, calibration summaries, checkpoints, and completion
manifests by default. Tagged campaigns isolate these artifacts from canonical
runs and can be summarized across environments with
scripts/summarize_evaluation.py.
Where to Begin¶
- Get Started installs the project, validates the checkout, and runs one small baseline notebook.
- Conceptual Overview explains how public state, joint graphs, opponent levels, and safety certificates fit together.
- Training Pipeline gives the local WM → OM → experiment workflow and artifact dependencies.
- Cluster Execution documents the supported Slurm and PBS launchers.
- Environment Reference lists agent counts, action spaces, export roots, and current safety objectives.
- API Reference documents the curated reusable Python interfaces.
Documentation Scope¶
The main pages are operational and user-facing. Durable contracts, design rationale, cache semantics, and resolved failure modes live in the maintainer notes. Both sets are built by Zensical and validated with strict link checking.