Manaka supervised training
Measurement record for the distillation line. ManakaZero's main path is self-play from random weights; supervised training exists to answer a narrower question: can this evaluator family learn useful shogi structure at all, and which changes improve it under a fixed teacher?
The teacher is Knowledge_distilled_dataset_by_NAGISA_V3 (about 300M source positions; the curated corpus is smaller after deduplication).
Main result
The early gen2 weights, after roughly one pass over the teacher corpus, reached about:
- policy top1: 36%;
- value-sign agreement: 72%.
So the network learned a substantial signal. But policy-only play was still weak; most of the practical strength came from adding search on top.
The old “−140 Elo versus material counting” result belongs to the early v2 weights and their policy head. It should not be treated as a current-architecture rating: policy_from and later architecture knobs were added afterwards.
Search compensated for the weak policy
Against the same Material Lv1 reference, the early gen2 line improved rapidly as simulations increased:
| simulations | Elo |
|---|---|
| 1 | −140 |
| 32 | +108 |
| 64 | +234 |
| 128 | +364 |
| 256 | +444 |
This established the central separation: policy/evaluator quality and search strength are different axes. Architecture work therefore uses policy-only games when the goal is to isolate the evaluator.
One pass was close to the useful limit of this corpus
A second pass over the same teacher data produced much smaller validation gains than the first. The leading explanations were:
- outcome label
zcarries only one final result per game, so its independent information is far smaller than the number of positions; - the corpus has limited opening/start-position diversity;
- only positions on played game lines are present.
This is why q (the teacher search value) is blended with z, and why later work focused on data diversity and architecture rather than simply repeating epochs.
Data loading
Streaming mode was added so the full corpus does not need to reside in RAM. In the original measurement it cut memory from hundreds of GB to a few GB at some throughput cost. Current training throughput is maintained in the benchmark, not here.
What later experiments superseded
The original supervised page predated most architecture work. Use these newer records for architecture conclusions:
- network history — A–E policy-only round robin and later aggregation ideas;
- twelve-shape validation — CE versus top1 on a common holdout;
- KP/HalfKP experiment — native tokens versus NNUE feature spaces.
In particular, do not compare old gen2 Elo directly with later shape Elo unless the weights, opponent and search conditions match.
Current use of supervised training
Supervised runs are useful for:
- checking whether a new shape can learn before spending self-play compute;
- comparing shapes on a fixed corpus and validation set;
- producing controlled ablations where only one architecture knob changes.
They are not a substitute for game-based equal-time evaluation. CE, top1 and Elo can order shapes differently.