Floor-patience screen (predeclared, 2026-10-04)¶

Why¶

The monotone floor raises one level once the Bayesian posterior's mode has stayed above it for shield_patience consecutive observed opponent actions. The campaign value is 64. The posterior is unstable: its mode switches level 500–4,700 times per run. So in Bertrand, Chicken, Congestion, Gathering and Markov Stag Hunt the floor reaches level 2 within the first few thousand steps of every seed, before the opponents have learned. Pursuit raises in stages, and one of its seeds never raises.

Replaying the recorded posterior-mode sequences through the floor rule offline reproduces the observed floors at patience 64. At larger patience the raises come later and are staggered across training:

• At 4,096, Chicken's raises spread over 6k–18k steps, and Markov Stag Hunt seed 0 reaches level 2 at 79k.
• At 16,384, Chicken climbs in stages between 18k and 48k.

The replay ignores feedback, because a different floor changes training. So it guides the choice; it does not predict the outcome.

Protocol¶

• Arms: shield_patience=64 (control) and shield_patience=4096 (variant). Each arm has its own tag: floor_patience_screen_p64_v1 and floor_patience_screen_p4096_v1.
• Seeds: disjoint from the campaign. seed_offset=7300, final_policy_eval_seed_offset=7300000, five seeds.
• Everything else follows the campaign protocol per environment. Timesteps, budgets (Pursuit 0.16, Congestion 0.10, otherwise 0.20), 1,000 evaluation episodes (Congestion 2,000, horizons 1, 2, 10, 64), Congestion shields both roles, Markov Stag Hunt shields player_0, and log_opponent_level_gap=True.

Where it runs¶

On CX3 (PBS, CPU-only), not cpucluster, because no timing from the screen goes into the paper and the cpucluster per-user CPU cap was taken by another project.

• Inputs: the campaign's wm/, om/ and shield/transition_shield_bundle*.pkl for each environment, streamed from cpucluster with tar (mtimes preserved, so the shield cache's staleness check passes). The CX3 checkout was fast-forwarded to main, and $BUCKET/.venv-omsh was re-synced with uv sync --frozen.
• Jobs: qsub -N ps<P>_<env> -l select=1:ncpus=8:mem=<M>gb -l walltime=23:59:00 with -v "REL_PATH=...,PY_ARGS='...'". The single quotes keep the commas in the horizons tuple and in shielded_agent_ids.
• Outputs: written under CX3's exports/<env>/experiments/runs/floor_patience_screen_p*_v1/. Copy them back before analysis.
• Two start-up races in ~/py.sh. Each job runs git pull and then uv sync --active --all-groups against the shared checkout and venv.
• git pull: simultaneous starts race on the ref lock (cannot lock ref), and the job dies. Fix: stagger starts with qsub -a, two minutes apart.
• uv sync: the venv first pointed at the login node's /usr/bin Python, which compute nodes may not have. The first sync on such a node deleted and recreated the venv with a uv-managed CPython 3.11.14, so jobs starting during the rebuild found no bin/activate. The venv now uses the managed interpreter, so later syncs are no-ops.
• Job IDs: of the first batch, 4266840 (ps64_bertrand) and 4266852 (ps4096_gathering) survived. The rest reran as 4267817–4267827 and 4267953–4267954.

Decision rule, fixed before any result¶

The variant replaces patience 64 in the primary condition only if, on the screen seeds, all three of these hold:

1. it passes the finite-horizon gate (worst-seed U_H <= p) in all six environments;
2. its mean reward per step is at least the control's in at least four of the six environments;
3. in every environment where the control's mean reward exceeds the best gate-passing baseline's campaign reward (IPPO-Lagrangian or CPO), the variant's does too. OmSh must keep its place on the safety–reward frontier.

If it is promoted, the variant reruns on the campaign seeds (2200, and 5300 for Congestion) and replaces OmSh-Monotone in the main tables, with the patience change stated. If it is not promoted, it is reported as an appendix ablation. For both arms, report the step of the first raise, the final floor, the below-floor rate and the closest-retained-level gap.

Analysis¶

scripts/summarize_patience_screen.py reads both tags in every environment and writes exports/patience_screen/summary.{json,md}. For each arm it reports:

• frozen-policy reward and worst-seed U_H at the largest horizon;
• the first raise step and final floor per seed;
• the below-floor rate and the closest-retained-level gap.

It applies the decision rule above only once all 12 arms have five seeds each. Campaign baseline rewards for condition 3 are hard-coded from Table final_policy_compact. A job with a long argument list exits with status 1 after writing every result, because the runner's notebook-export filename is too long. Judge completeness by the run directory, not the exit code.

Result (Mon 5 Oct 2026, 09:40): not promoted¶

All 12 arms completed with five seeds (CX3 jobs 4269809–4269820; summary in exports/patience_screen/summary.{json,md}; paper Table tab:patience_screen).

Patience 64 remains primary, and the screen is reported as an appendix ablation.

• Lock-on: patience 4096 removes the early lock-on. The median first raise moves from 128–500 steps (21k in Pursuit) to 4.5k–55k steps, or never (Pursuit, Gathering, 4 of 5 Bertrand seeds).
• Fit: the closest retained level is as close or closer to the opponents everywhere (Gathering 0.70 → 0.35, Congestion 0.26 → 0.18).
• Bertrand and Chicken: rewards are identical across arms because every reachable state has certificate value 0 and the levels give the same one-step constraints there. The floor does not change the admissible set.
• Replication: the control arm replicates the campaign OmSh results on disjoint seeds.

Follow-up sweep: patience 512, 1024, 2048 (declared Mon 5 Oct 2026, before any result)¶

The user asked to map the range between the two arms above. The 4096 result was already known, so this is a new, declared follow-up, not a re-analysis of the first screen.

• Arms: shield_patience ∈ {512, 1024, 2048}. Tags are floor_patience_screen_p{P}_v1, and every other setting is identical to the 64 and 4096 arms.
• Seeds: the same screen seeds (7300–7304, evaluation offset 7300000), so each arm is paired seed for seed with the existing patience-64 control.
• Rule, fixed now: apply the three conditions above to each P ∈ {512, 1024, 2048, 4096} separately against the patience-64 control:
• gate in all six environments;
• reward at least the control's in at least four;
• frontier kept wherever the control beats the best gate-passing campaign baseline.

If one or more P pass, promote the largest passing P, the one that best avoids early lock-on. It then reruns on the campaign seeds and replaces patience 64 in the main tables, with the sweep reported. If none pass, patience 64 stays primary and the full sweep is reported in the appendix. - Caveat, stated in advance: in Bertrand and Chicken the floor did not change the admissible set at 4096, so the rewards tied exactly. Ties count as "not worse", so condition 2 in effect asks for no loss in two of Congestion, Pursuit, Gathering and Markov Stag Hunt. Testing four values raises the chance that one passes by noise, so a promotion is reported with every arm's numbers. - Submission: notebooks/experiments/scripts/cx3_patience_screen.sh, run with ssh cx3 "PATIENCES='512 1024 2048' bash -s" < …. The follow-up arms are CX3 jobs 4275155–4275173, submitted Mon 17:05. scripts/summarize_patience_screen.py --patiences 512 1024 2048 4096 applies the rule to each variant and promotes the largest passing value.

Follow-up result (Tue 6 Oct 2026, 04:20): patience 512 meets the rule¶

All 18 follow-up arms completed with five seeds. The summary is in exports/patience_screen/summary.{json,md}, and the paper's Table tab:patience_screen now has all five patience values.

• Markov Stag Hunt loses about 0.02–0.03 reward at every higher patience (0.592–0.600 against 0.619). Congestion is flat to within 0.0015.
• Caveat: 512 delays the median first raise only to about 1–2k steps (Pursuit 118k, one seed never). Early lock-on largely remains; it disappears from 2048.
• Next step: per the protocol, 512 would rerun on the campaign seeds and replace patience 64 in the main tables. With the Fri 9 Oct deadline, that is the user's decision; the paper currently states that the main results use patience 64.

Patience 512 on the campaign seeds (Tue 6 Oct 2026)¶

The user chose to rerun the campaign at 512 as extra results; the patience-64 campaign stays usable.

• Runs: CX3 jobs 4278442–4278447, tag successor_campaign_p512_v1. Campaign seeds and budgets; only shield_patience=512 and log_opponent_level_gap=True differ, and the logging makes no random draws.
• Audits: dependent risk-dominance audits 4278459–4278464.
• Comparison: scripts/compare_patience_campaigns.py pairs the runs seed for seed; output in exports/patience_screen/campaign_p64_vs_p512.{json,md}.

• Reward and safety: reward is unchanged in four environments and significantly higher in Pursuit and Gathering. Every gate passes, and OmSh's position against the baselines is unchanged or better.
• Pursuit audit: worst-run plug-in bound 0.368 against 0.291 at p64 (paper Table dominance_audit). Four of five p512 runs are lower (0.178–0.271); the worst run sits at floor 1.
• Lock-on: first raises move from a few hundred steps to about 1–2.5k (Pursuit 43k), and more seeds end below the top level.
• Archive: /vol/bitbucket/oja24/omsh/results/campaign-patience512-2026-10-06/.