Skip to content

Results archive and exports roots

Archive layout (6 Nov 2026)

At the user's request, /vol/bitbucket/oja24/_archive/omsh/ holds only the paper archives:

  • aamas/, the AAMAS submission archive (renamed from aamas_2026_08_10/);
  • arxiv/, the target for the arXiv version's CX3 re-run (../experiments/cx3-rerun-2026-11.md).

Everything else moved to the sibling /vol/bitbucket/oja24/_archive/omsh_extra/, on the same filesystem, so the moves were instant renames:

  • followups_2026_10/ and timing_clean_amd48_2026_10/, unchanged;
  • legacy_deleted_20261106/, the record of the deleted legacy archive: its paths.txt, verify.txt, the script that built it and a listing.

The legacy archive itself (pre-AAMAS environments and tags, and the August Pursuit-3 bundle, about 60 GB) was deleted with the user's approval on 6 Nov 2026. Nothing inside or outside the archive linked to any moved path (find -lname over the archive, the DoC exports root, gpudata and ~/Projects).

arXiv archive (9 Nov 2026)

/vol/bitbucket/oja24/_archive/omsh/arxiv/ holds the CX3 re-run of the paper (../experiments/cx3-rerun-2026-11.md), archived with the user's approval on 9 Nov 2026. Its README.md gives the layout and caveats. - Contents: - cx3/omsh_arxiv/ and cx3/omsh_arxiv_pb/, each with exports and the untracked notebooks; - task/ (ledger, logs, snapshots, launchers) and pbs_logs/; - tables/ (every regenerated table, the figures and the summaries); - _meta/ (scripts, manifests, verification) and _logs/. - How it was copied: staged on the workstation (/var/home/omar/omsh_archive_staging_20261109) and pushed with rsync over the shell1 jump. - Verification: - CX3 to staging: sha256 manifests from a CX3 PBS job, 0 differences over 13,247 files. - Staging to archive: tar streamed back and hashed on the workstation (_meta/verify_doc_20261109.txt). - DoC Slurm was not used. The CPU cluster's per-user memory limit was holding another project's pending jobs, and the GPU cluster attaches a GPU to every job.

Since the data shift (9 Oct 2026)

The AAMAS submission data is archived, verified and removed from the live trees.

  • AAMAS archive: /vol/bitbucket/oja24/_archive/omsh/aamas/, on DoC storage. Its README.md gives the layout and the full verification record (_meta/verify_*).
  • cpucluster/exports/ is the former /vol/gpudata/oja24-omsh/exports (the paper's table host), with absolute symlinks resolved.
  • cx3/<tree>/{exports,untracked}/ holds the CX3 clone and the worktrees omsh_rerun, omsh_omfix and omsh_omold; their symlinks are relative.
  • results/ holds the per-experiment archives described below, frozen.
  • bitbucket/{exports-bitmlp,reports}/ holds the WM ablation and the Stage 1/2B studies.
  • The old clones' untracked run outputs and the analysis scratch are kept alongside.
  • Clean-timing archive (14 Oct 2026): /vol/bitbucket/oja24/_archive/omsh_extra/timing_clean_amd48_2026_10/. It holds the clean timing run's exports, Slurm logs, executed notebooks, sacct records, regenerated tables and checksum verification (its README.md).
  • It supersedes the paper's timing tables. The 24 sidecars those tables were generated from were moved to aamas/_superseded_timings/, with the same relative paths, not deleted. These are timings.json and timings_hardware.json at each reported environment's root and in experiments/runs/successor_campaign_v1/, under cpucluster/exports/.
  • Every other timing file in the AAMAS archive is untouched.
  • Legacy archive (pre-AAMAS environments and tags, Pursuit-3): deleted on 6 Nov 2026 (above).
  • DoC exports root: /vol/bitbucket/oja24/omsh, linked as ~/Projects/omsh/exports in the fresh DoC clone. It is shared DoC storage, so runs on cpucluster or gpucluster need no separate results copy.
  • Follow-up archive (18 Oct 2026): /vol/bitbucket/oja24/_archive/omsh_extra/followups_2026_10/.
  • It holds the F1, F2 and F4 runs and reports, the published models restored into their root, the 14 executed notebooks and the clone's app log.
  • It is checksum-verified (its README.md), and its regenerated summary is byte-identical to the live report.
  • The live root /vol/bitbucket/oja24/omsh_followups and the clone ~/Projects/omsh_followups were deleted at the user's request after a second checksum check. The branch followups was already merged into main.
  • CX3 frontier-study exports (18 Oct 2026): ~/Projects/omsh/exports on CX3, shared by the pinned worktrees omsh_frontier and omsh_bc. It holds the six environments' published models, copied from the AAMAS archive with all 209 sha256 sums matching, and the study's runs (../experiments/frontier-study-2026-10.md).
  • The completed A and C results (147 directories, 5,660 files) were copied to the DoC exports root on 19 Oct 2026, sha256-identical. They are the frontier_* tags under each environment's experiments/runs/, and reports/{certified,attribution}_frontier_c_* and reports/frontier_study_2026_10*.
  • B's results followed on 20 Oct 2026: 142 directories and 537 files, sha256-identical.
    • They are the frontier_b_* tags, wm_exact_kernel/ and om_{learned,exact_kernel}_r{1,2}/ of gridworlds/{pursuit,gathering}/2, plus certificate_dominance_frontier/.
    • The final reports/frontier_study_2026_10/ replaced the 19 Oct copy.
    • Every frontier-study result is now on DoC, except C's regenerable shared/ caches.
  • Clean-timing exports root (deleted 15 Oct 2026): /vol/bitbucket/oja24/omsh_timing, the exports of the clone ~/Projects/omsh_timing. Both were deleted at the user's request after a second checksum check against the clean-timing archive. The cancelled amd96 attempt's partial outputs stay under /vol/bitbucket/oja24/omsh (tag timing_rerun_amd96_v1).
  • DoC venv: /vol/gpudata/oja24-omsh/.venv. venv=omsh resolves to it through $BUCKET/.venv-omsh. It is built once with uv venv --python 3.11 and then uv sync --active --all-groups; the runner needs bin/activate before its own sync.

To restore an artefact, copy it back from the archive with rsync -a. Keeping mtimes keeps the shield bundle caches valid, since their metadata records the graph and IOP mtimes and sizes.

Copying from CX3

CX3 cannot see /vol/bitbucket, so CX3 results must still be copied to DoC, through the workstation. Only ssh cx3 (its ControlMaster) reaches CX3; the -J shell1 … login.cx3 route no longer authenticates:

ssh cx3 'cd ~/Projects/omsh/exports && tar cf - --exclude="*.lock" <paths>' \
  | ssh cpucluster 'mkdir -p /vol/bitbucket/oja24/omsh/<paths-parent> && tar xf - -C /vol/bitbucket/oja24/omsh'

Then verify with a checksum dry run, rsync -a -n -c -i SRC/ DEST/; it must list no >f lines.

Per-experiment archives (6–9 Oct 2026, now frozen in the AAMAS archive)

The user asked (6 Oct 2026) for a copy of every experiment's results in /vol/bitbucket/oja24/omsh/results/<experiment>-<yyyy-mm-dd>/, while the live exports sat on gpudata and CX3. Those folders are now in the archive's results/:

  • Layout: each mirrors export-relative paths under exports/ and has a README.txt with what was run, where (cluster and job IDs), seeds, protocol document and outcome.
  • Contents:
  • floor-patience-screen-2026-10-06: all five patience arms and the summary.
  • congestion-scaling-2026-10-05: the n = 3 pipeline, the n = 4 OM, OmSh and audit, and the n = 4/6 TrueShielded runs.
  • mixed-values-2026-10-04: mixed against pure certificate values.
  • campaign-patience512-2026-10-06, padded-patience512-2026-10-07 and reward-fix-rerun-2026-10-07: the p512 campaign, the padded runs and the reward-fix reruns.
  • true-shielded-both-roles-2026-10-09: IPPO-TrueShielded with both Congestion roles shielded, still the bypass fallback (0.197, U 0).