Safety Budgets¶

max_risk is the shield’s initial episode budget for eventual unsafe reachability. It is not a one-step failure probability and not a reward penalty.

Feasibility¶

At reset, at least one focal action must satisfy the robust certificate at the initial state and opponent-level floor. If the best action needs more budget than max_risk, shield construction or reset fails with both the best eventual risk demand and best immediate unsafe probability. Raising max_risk can make the configuration feasible, but changes the safety contract rather than merely silencing an error.

Successor Budgets¶

For the executed action, the shield constructs one successor-budget vector that works across every accessible opponent level. Each successor receives at least its robust floor-wise certificate. In the default mode, any remaining allowance is filled egalitarianly while all expected-budget constraints remain satisfied.

The learned-budget modes let the policy propose how to distribute the remaining allowance. The wrapper projects that proposal into the same robust feasible polytope; a numerical or feasibility failure falls back to a certified allocation rather than relaxing safety. The action stored and scored by PPO is the exact raw proposal whose projection determined the transition.

After the environment transition, the budget becomes the value assigned to the realized successor. Unknown successors receive budget 1.0, which normally forces a later infeasibility rather than assuming unobserved behavior is safe.

Level Changes¶

Raising the opponent-level floor does not directly change the numeric budget. It changes which level-specific transition distributions the next action must tolerate. Because the floor never falls, a run cannot relax its opponent assumption after briefly committing to a higher level.

Learner Visibility¶

The focal policy observes both the current budget and the level floor as two extra continuous features. The base world model, opponent model, and graph keys continue to use only the public environment state; shield bookkeeping is not folded into those artifacts.

In bayesian_mixture mode, the normalized opponent posterior is also appended because the feasible set depends directly on it. Pairwise-mixed and learned-budget modes use a factored categorical/raw-Gaussian focal action space; non-focal agents retain their original discrete spaces.