Troubleshooting¶

Zensical or a Python Command Is Missing¶

Install every dependency group and run commands through the project environment:

uv sync --all-groups --all-extras
uv run zensical build --strict

A Notebook Rejects a CLI Key¶

Notebook keys use snake case internally and kebab case on the command line. Check Configuration and CLI. Unknown and duplicate normalized keys are errors by design. In cluster pipelines, remember that n_steps is WM-only while most training keys are experiment-only.

A Baseline Run Tries to Load Shield Artifacts¶

All standard experiment algorithms default to disabled. A plain IPPO-only call only needs --run-ippo=true. If it still reaches a shield stage, verify that a wrapper or saved command is not explicitly enabling that stage.

Learned Shield Artifacts Are Missing¶

Run the matching WM and OM notebooks first. The learned shield requires:

wm/env_transition_graph.pkl
om/iop_stack.pt

Use the same environment parameters and agent count at every stage.

Initial Safety Bound Is Infeasible¶

The message reports the best robust eventual-unsafe demand and best immediate unsafe probability. This is not a missing-file error: no focal action satisfies the selected max_risk across the accessible opponent levels. Review the environment safety objective, start state, opponent artifacts, and intended risk contract before changing the budget.

Graph Width, Metadata, or Opponent Dimension Mismatch¶

Cached graph metadata normally triggers an automatic rebuild. If downstream artifacts were produced from older state/action semantics, move the affected environment export root aside and rerun WM → OM → experiments. Keeping the old directory as a backup is safer than deleting it until the rebuilt outputs have been verified.

Missing Coverage Appears in Shield Telemetry¶

The shield sends missing transition mass to an unsafe sink and treats unknown successors conservatively. Frequent missing coverage usually means incompatible state encoding, stale graph/model artifacts, or incomplete legal topology. It should not be suppressed as a cosmetic logging issue.

A Large Exact Graph Runs Out of Memory¶

Graph size grows exponentially in agent count. Three-agent Pursuit and high-agent Congestion can be much larger than two-agent runs. Confirm the intended count, inspect cached metadata, use the configured graph guards, and avoid building several copies concurrently.

A Short JAX Run Still Takes Time¶

The first run compiles JAX functions, and timesteps is rounded up to a full rollout. Reduce both --timesteps and --rollouts for a smoke test. The persistent test compilation cache helps repeated pytest runs but does not make every new notebook shape compile-free.

Weights & Biases Prompts or Fails¶

Pass --use-wandb=false unless remote logging is intended and authenticated. Markov Stag Hunt currently enables it by default; most other experiment notebooks do not.

Cluster Dependencies Are Missing¶

Use DRY_RUN=1 and inspect the submission output. Slurm pipeline launchers need a submit command compatible with sbatch --parsable; PBS launchers must recover a full numeric job ID before creating afterok children. The launcher aborts instead of continuing without a dependency.

Notebook Outputs Land Under the Wrong Directory¶

Maintained marimo notebooks call the project-root helper. A custom notebook should import and call notebooks.project_root.set_cwd_to_project_root() before creating relative export paths.