Pursuit: per-transition world-model radii from slip-rate confidence¶

robust-margin.md found that a uniform TV radius cannot instantiate the anytime theorem in Pursuit. The floor-0 margin is 0.0022 at budget 0.16, while the learned kernel's error on reachable rows reaches 0.43. This note covers the alternative: per-row radii under the structural assumption known rules, unknown rates. Code is in src/shield/ambiguity/rates.py (generic) and scripts/pursuit_rate_radii.py (Pursuit). Tests are in tests/test_pursuit_rate_radii.py.

How the slip rates enter the kernel¶

Pursuit (src/environments/gridworlds/pursuit/env.py) has exactly one source of randomness. Each step, each agent independently slips with probability slip_guard_prob (player_0) or slip_intruder_prob (player_1); both default to 0.1. A slip replaces the requested action with a uniform draw from _legal_slip_actions, the cardinal moves that leave the cell. The commented-out "slip to Stay" variant is inactive. Collision states self-loop.

With theta = (theta_guard, theta_intruder) and K_b the joint law given which agents slipped (b in {0,1}^2), every joint row is

Gamma_theta = sum_b prod_j theta_j^{b_j} (1 - theta_j)^{1 - b_j} K_b.

So the row is multilinear in theta: affine in each rate separately. K_b is guard_successors(..., slip_probabilities=b), the exact builder's own per-agent law evaluated at slip probabilities 0 and 1. The decomposition reproduces the cached exact graph true_shield/true_transition_graph.pkl at theta = 0.1 to 1.1e-16 on all 48,400 joint rows (571,152 entries), with identical supports. It also matches a fresh exact-builder graph at the estimate. For theta in the open box, the support does not depend on theta, and the learned graph lists exactly that support: no entry is missing and none has zero mass.

Per agent, a slip is visible only when it does not reproduce the requested move. If the requested move is itself legal (k = 1) one of the |L| slip draws repeats it, so the visible-deviation probability is theta * q with q = 1 - k / |L|. A requested Stay or blocked move gives q = 1. The grid has no dead ends, so q takes the values 1/2, 2/3, 3/4 and 1. The full-transition likelihood ratio between two rates equals the ratio of these deviation indicators, because the slip direction is uniform whatever the rate. So the per-class counts (q, n, s) are sufficient, and the two rates have separate likelihoods.

Data¶

wm/D.pkl holds the world model's 750,000 training transitions. They come from collect_rollouts_ppo: BufferWrapper around IPPO training of both agents from scratch, each episode reset with seed base + episode index. Each transition yields one thinned observation per agent. Diagnostics, which are checks and not part of the certificate:

• none of the 1.5M agent-steps lands outside {requested move} ∪ legal slips;
• a likelihood-ratio test of one rate against one rate per visibility class finds no heterogeneity;
• a chi-square test finds the visible slip directions uniform over the alternatives.

Confidence box¶

Each rate gets an interval at delta_WM / 2 (a union bound over the two rates). The box is their product. Two intervals are computed:

• Time-uniform mixture (primary). M(theta) = int_0^1 L(t) dt / L(theta), with uniform prior and the thinned likelihood L. Under the true rate, L(t)/L(theta*) is a non-negative mean-one martingale for each t. The visibility q_t is fixed before step t's slip coin, and the coin and the slip direction are fresh draws from the environment RNG. The mixture over t is therefore one too, and by Ville's inequality {theta : M(theta) < 1/delta} covers with probability >= 1 - delta at every sample size simultaneously. This matters because the training data were collected adaptively: the visibility classes depend on states and actions chosen by a learning policy, and the realised count of each class is random. _log_mixture_integral bounds the integral from below, using chords of the concave log likelihood plus dropped tails. A smaller integral can only widen the set.
• Clopper–Pearson (fixed sample). Exact binomial on the fully visible subset (q = 1). By optional skipping those slip coins are i.i.d. Bernoulli(theta). But the subset's size is data-dependent, so CP's nominal level assumes it was fixed in advance. It is reported for comparison and is about 30% narrower.

Why one interval covers every deployment step¶

The box is built once, from pre-deployment data, and never updated. Let E = {theta* in box}, so P(E) >= 1 - delta_WM. On E, every row satisfies TV(c, Gamma_theta*) <= max_{theta in box} TV(c, Gamma_theta) = eps(row). That is a deterministic consequence of E, so A1 holds at every row and every deployment step at once. Deployment needs no union bound over time and no time-uniform correction. The time-uniformity used above concerns the training sample, not deployment.

The remaining modelling assumption is the i.i.d. one from anytime-ambiguity-sets.md ("Why i.i.d. applies"): a time-homogeneous simulator, fresh slip randomness each step, and the same rates in data collection and deployment. Here "fresh" means numpy RandomState streams reseeded per episode with consecutive seeds, which are treated as independent.

Radii¶

Joint rows. TV(c, Gamma_theta) is convex along each coordinate, because it is a sum of |c - affine| terms. A function convex in each coordinate separately attains its maximum over a box at a vertex: maximise one coordinate at a time. So eps_joint(row) = max over the 4 vertices (vertex_tv_radius), which is exact and not a bound. Two centres are used:

• (a) parametric, c = Gamma_thetahat. This is the structurally faithful centre. Its bundle is built from the exact builder's graph at thetahat (build_true_transition_graph on Pursuit(slip_*_prob = thetahat), serialised, then build_transition_shield_bundle with the campaign IOP stack).
• (b) learned, c = the neural kernel wm/env_transition_graph.pkl. Its bundle is the cached shield/transition_shield_bundle.pkl.

Focal rows, two valid marginalisations:

• convexity: sum_{a-i} pi_l(a-i | s) eps_joint(s, a, a-i), as induce_row_radius computes it (pinned against it in the tests);
• direct: P_{l,theta} = sum_{a-i} pi_l Gamma_theta is also multilinear in theta, so max_vertex TV(compiled row, P_{l,vertex}) is the exact supremum. It is about 25% smaller, because the same rate drives every joint row of the mixture, which the convexity bound ignores.

With a real opponent mu, TV(P_{mu,theta*}, centre) <= TV(mu, pi_l) + eps_focal, because a common kernel cannot increase TV. So the solver's additional uniform radius on top of base_radius is exactly the slack left for A2 (eps_OM).

opponent_probabilities takes pi_l from the shield's own augmentation (build_iop_induced_graph(..., return_joint_graph=True) on the parametric graph). The script checks that the result reproduces both bundles' focal rows. Rows are aligned to compile_bundle's (level, state, action) order through the compiled state keys.

Results¶

These numbers come from exports/gridworlds/pursuit/2, campaign IOP stack, budget p = 0.16, player_0. The run takes about 5 minutes on 9 local processes. Each solve takes seconds, and building the thetahat graph and bundle takes about 2 minutes the first time (cached in the work directory).

Estimate (n = 750,000 transitions per agent):

Fit diagnostics are clean:

• 0 of 1.5M agent-steps fall outside the slip support;
• rate homogeneity across the 6 classes: p = 0.55 (guard), 0.85 (intruder);
• direction uniformity: p = 0.89, 0.87.

Boxes (each rate at delta_WM / 2):

Radii. These are the mixture box at 0.05, over the 1,012 safe states, all of which are reachable: 25,300 joint rows and 15,180 focal rows across 3 levels.

Collision self-loops (2.3% of rows) have radius 0. Neither centre's listed support misses any Gamma_theta mass at any box vertex. The smallest opponent probability is 7.2e-4, so no opponent action is ever dropped from a focal row's support.

Reset demand and extra radius for A2 (budget 0.16; "–" means the per-row radii alone exceed the budget):

The learned rows are certified-infeasible brackets. The solver stopped once the lower value passed the budget, so only lower bounds are reported: the smallest across boxes and marginalisations, 0.1601 for the direct radius at floor 0 and 0.174–0.176 for convexity. The radius-0 row reproduces the thetahat bundle's own floor bounds at reset. Extra radii are the certified end of MarginSolver.margin's bracket, at about 0.5% relative resolution.

Data scaling (mixture box at 0.05, thetahat fixed, counts scaled):

Reading¶

• Structure solves A1. Knowing the rules turns a world-model error of up to 0.43 (learned kernel) into per-row radii of at most 0.0032. At the training data size, the floor-0 certificate fits the 0.16 budget with the direct marginalisation (demand 0.1588) or with a fixed-sample CP box. It just misses with the convexity bound (0.1611). Doubling the data makes the time-uniform box at 0.05 fit with either marginalisation. The radii shrink as 1/sqrt(n).
• A2 is the binding constraint at floor 0, at any data size. As n grows the A2 slack rises only to the thetahat certificate's own uniform margin, 0.00215. That is the same fragility as the exact-kernel margin in robust-margin.md. Today the slack is 0.0003 (time-uniform) to 0.0009 (CP, direct). Floors 1 and 2 leave 0.003–0.005 and about 0.02.
• The neural kernel is not a usable centre. Its error to Gamma_thetahat dominates the 0.002-scale rate uncertainty, and the padded certificate around it is infeasible at every floor. Its per-row radii are essentially the audited learned-versus-exact TV. For Pursuit, the parametric world model is both more accurate and certifiable.
• Direct versus convexity. The direct focal radius is about 25% smaller on average, and the difference decides floor-0 feasibility at the current data size.

Caveats¶

• Budget accounting. The tables use p = 0.16 as the shield budget, so the end-to-end statement is P(unsafe) <= 0.16 + delta_WM (+ the A2 failure probability). If 0.16 is the total, p_shield = 0.16 - delta_WM (anytime-ambiguity-sets.md). At delta_WM = 0.05 (p_shield = 0.11), floors 0 and 1 are infeasible even at radius 0 (nominal 0.1524, 0.1404). Floor 2 fits with extra 0.0033 (mixture, convexity) to 0.0046 (CP, direct). At delta_WM = 0.01 (p_shield = 0.15), floor 0 is still infeasible, floor 1 fits with 0.0002–0.0012 extra, and floor 2 with about 0.017. These are the /reserved cases in the JSON. delta_WM costs far more as budget than as width: going from 0.05 to 0.01 widens the box by about 7%.
• The structural assumption is assumed, not tested by the interval: the known slip mechanism, the same rates in every state and over time, independence across agents, and the same rates in data and deployment. The diagnostics are consistent with it.
• Clopper–Pearson treats the visible-subset size as fixed, while the data were collected adaptively. The mixture interval is the one with a proof.
• Reset support is a single deterministic start (start_positions is fixed), and it is identical in the exact, thetahat and learned graphs.
• Not what is deployed. The deployed bundle is the learned one, unpadded. This certificate belongs to a padded shield over the parametric bundle with per-row radii. src/shield/padding.py pads with one uniform radius; MarginSolver(base_radius=...) already takes per-row radii, but the runtime padded_expectation and the successor-budget fill would need per-row radii too.
• Opponent gap is still unmeasured. The extra radius is the slack available for A2, not evidence that A2 holds.

Alternatives not taken¶

• A joint (elliptical) confidence region for both rates would be smaller than the Bonferroni box. But a coordinate-wise convex function need not peak at a vertex of a non-box region, so the radius would need a different maximiser.
• A fixed-sample interval on all the data (Poisson–binomial CP, or a Wald interval with the observed information) would be about 2x narrower than the time-uniform one. It is not valid under adaptive collection without the martingale argument.
• Per-row radii could be cut further by optimising delta_WM against the budget reserve, or by splitting it unequally between the two rates.

Running it¶

From the repository root, or from any directory that holds a copy of exports/gridworlds/pursuit/2 (wm/D.pkl, wm/env_transition_graph.pkl, om/, shield/, true_shield/true_transition_graph.pkl, with mtimes preserved for the learned bundle's staleness check):

uv run python scripts/pursuit_rate_radii.py --jobs 9

The output is exports/robust_margin/pursuit_rate_radii.json. The work directory exports/robust_margin/pursuit_rate_radii/ caches the thetahat graph and bundle, keyed on thetahat and input stamps. The JSON's summary table lists every case. radii, boxes, verification, focal_centre_check and data hold the rest.

Corrected-reward bundle (8 Oct 2026)¶

Rerun on the Pursuit bundle built from the opponent models with successor-conditioned imagined rewards (CX3 4294588, ~/Projects/omsh_rerun; docs/AI/experiments/reward-fix-rerun.md). Mixture box at δ = 0.05, direct radii: - the box width is 0.0036 / 0.0035 per rate, and the level-row radii are at most 0.0028 (as before); - floor-0, 1 and 2 reset demands are 0.1358, 0.1280 and 0.1085, leaving 0.0070, 0.0100 and 0.0167 of opponent radius at p = 0.16 (before: 0.1588, 0.1467, 0.0983, with 3.2e-4, 0.0039, 0.0201); - the exact kernel's floor-0 margin is 0.0087 (before: 0.0022); - around the learned kernel (error up to 0.418), the padded certificate is still infeasible at every floor; - with δ_WM reserved (budget 0.11), floors 0 and 1 are infeasible, and floor 2 now admits 5e-4.

Paper D.2 updated (omsh-paper, 8 Oct evening).