Level-0 Opponent-Model Fitting¶
ImaginedOpponent.fit_level0 now reserves a validation holdout before training so level-0 performance is measured on unseen data instead of only the optimization set.
- Default split: 90% train, 10% holdout.
- Split timing: the partition is sampled once at the start of
fit_level0, then each epoch trains only on the training subset. - Return value:
Level0FitMetrics, with per-splitcross_entropy,accuracy,error_rate, and sample counts. - Display helper:
print_level0_fit_metrics(...)renders the metrics as arich.Tableso the training notebooks can show the split-wise results consistently. - Small-buffer behavior: if the buffer has only one sample, training uses that sample and the holdout metrics are reported as
NaNbecause a true holdout is impossible.
This is mainly meant to quantify level-0 imitation error honestly before higher-level imagined best-response training builds on top of it.