Skip to content

Reviewer-facing diagnostic exports

Experiment reruns export compact diagnostics by default. There is no export_diagnostics switch: aggregates are part of the normal artifact contract. verbose_data=True is reserved for bounded raw traces and sampled comparisons that require extra model evaluations.

Cost boundary

Always-on exports reuse values already computed by training, model evaluation, Bayesian filtering, and shield construction. They do not add environment steps. This includes episode tables, optimizer metrics, action-mask distortion, Bayesian opponent-model mode changes, observed-action and unsafe-transition calibration, world/OM reports, shield feasibility sweeps, and checkpoints.

When an exact transition graph already exists, the learned transition graph is compared with it in one linear graph pass. The learned-versus-oracle shield comparison builds an exact-dynamics induced bundle under the same IOP levels once and caches it at shield/oracle_iop_transition_shield_bundle.pkl. It never causes the exact graph itself to be built.

The learned-versus-oracle export first verifies that the cached learned shield bundle still matches the current learned transition graph and IOP checkpoint. After a WM or OM rebuild it omits the stale comparison until normal OmSh setup has rebuilt the learned bundle; it never relabels old shield values as post-rebuild diagnostics.

verbose_data=True enables:

  • a strided, bounded shield decision trace (data_trace_stride, max_trace_rows);
  • bounded live-opponent-policy versus IOP comparisons every diagnostic_interval_updates, using at most diagnostic_state_sample rollout states.

These options add no rollouts. For example:

uv run python notebooks/marimo_cli.py notebooks/matrix/chicken/experiments_mo.py \
  --verbose-data true --data-trace-stride 100 --max-trace-rows 100000

Artifact layout

Within an environment export directory:

  • wm/wm_report.{json,txt}: held-out state/reward scores, state calibration bins/ECE, and per-agent reward error;
  • wm/transition_tv_report.{json,txt}: learned-versus-exact conditional kernel TV when the exact graph already exists;
  • om/level0_fit_report.{json,txt}: action prediction CE, accuracy, Brier, ECE, action counts, and confusion matrices;
  • om/om_report.{json,txt} and om/pi_training_metrics.csv: staircase training history, entropy, and pairwise OM-level policy discrepancy;
  • shield/shield_report.{json,txt}: SVI convergence/coverage and risk-budget feasibility by robust level floor;
  • shield/learned_vs_oracle_report.{json,txt}: false-safe/false-unsafe rates, safe-set precision/recall/Jaccard, action-risk error, selected-action agreement, and reward regret under exact dynamics and the same IOP stack;
  • experiments/tables/*_episodes.csv.gz: every EpisodeLog field per seed and episode;
  • experiments/tables/*_om_level_events.csv.gz: exact (timestep, old, new) Bayesian mode changes;
  • experiments/metrics/*_updates.{json,csv.gz}: optimizer, mask distortion, and policy-drift metrics;
  • experiments/diagnostics/*.{json,txt}: aggregate experiment and calibration reports;
  • experiments/checkpoints/*.pkl: final policy and optimizer states;
  • experiments/traces/*.jsonl.gz: bounded verbose traces, only when enabled;
  • diagnostics/finite_horizon_bound.{json,txt}: the paper's finite-horizon expression populated when all required estimates are available;
  • experiments/plots/om_bayesian_levels*.png: unconstrained Bayesian posterior mode per seed, overlaid with the separate monotone shield floor.

EpisodeLog pickles are schema-evolution tolerant in the diagnostic exporter. An instance written before a metric field was added does not acquire that attribute when unpickled; the exporter preserves its available fields and reports newer metrics as unavailable instead of aborting exact-shield setup. Diagnostic runs shorter than one episode also export empty reward/safety plots on a finite [0, 1] timestep axis; they do not attempt the degenerate [0, 0] axis that can make Matplotlib fail during rasterization. When plot horizons are truncated to the smallest run, an algorithm with no completed episodes is excluded from the horizon minimum; otherwise a one-step diagnostic would truncate every populated comparison history to timestep zero.

Interpretation and naming

The Bayesian series is called the Bayesian OM posterior mode. It is not a ground-truth reasoning level and is intentionally allowed to decrease. Its exact changes, occupancy, confidence, entropy, and time below the shield floor are recorded separately from monotone floor raises.

Opponent calibration is top-label calibration of predicted opponent actions. Unsafe-transition calibration compares one-step predicted unsafe probability with observed unsafe transitions. The code does not report “reasoning-level calibration,” because no labelled reasoning levels exist.

The paper-facing runtime table includes episode-reachability Brier score and one-step unsafe-transition Brier/ECE alongside intervention, predicted risk, budget, availability, and infeasibility. Observed reachability at declared horizons remains a separate frozen-policy statistic; the implementation does not relabel an eventual certificate value as a horizon-specific prediction. The paper-facing world-model table is generated from wm/wm_report.json and reports the held-out transition count, state-bit accuracy, positive-class F1, state Brier/ECE, and reward MAE. When wm/transition_tv_report.json exists, it also reports exact-graph state-action coverage and mean/maximum conditional transition TV; absent exact comparisons remain explicitly unavailable. Positive F1 is included because bit accuracy alone can obscure rare positive state bits.

Names containing sampled_empirical are maxima or summaries over recorded rollout states only. graph_exhaustive covers every pair in the enumerated graph but is not automatically an original-game certificate. shield_report states which SVI values are certified uniform bounds inside the learned induced model. The finite-horizon plug-in report never promotes sampled empirical maxima to the uniform assumptions required by the theorem; it reports missing terms instead.