DLManaka self-play roadmap

This page tracks implementation status. Paper summaries live in papers; historical failure analysis lives in the plateau record; the latest documented priorities live in the gen11 improvement plan.

Status

item	status	note
value-head value_weight test	done, rejected	doubling the weight did not stop decay
Gumbel AlphaZero search	implemented	sequential + batched search
self-play record generation	implemented	dl-manaka-selfplay
playout cap randomization	implemented	full search on a subset of moves
replay window / gating	implemented in later loop work	used by the gen11 line
RGSC / regret-guided starts	candidate	not implemented in the current codebase
uncertainty-guided branching	candidate	not implemented
RMCTS replacement	on hold	alternative search, not combined with Gumbel

Current self-play path

parent weights
  → Gumbel MCTS self-play
  → sparse policy + z/q targets
  → replay/shard dataset
  → train child
  → parent/internal gate
  → external anchor measurement
  → promote or reject

Search is both the move selector and the teacher. The important lesson from the first plateau was that a weak search produces weak training targets even when the network itself has unused strength.

Search budget

Batched search made deeper teachers practical. Later measurements showed strong gains through 512 simulations but little additional playing gain at 1024. The current improvement plan therefore treats 512 simulations as the relevant teacher-depth experiment, rather than assuming deeper is always better.

Playout cap randomization reduces generation cost by using the full search only on a fraction of moves and recording those moves as policy targets. Cheap moves still advance the game but are not treated as equally strong teacher labels.

Candidate: regret-guided start positions

RGSC starts games from positions where the network's earlier evaluation disagreed strongly with the eventual outcome. The attraction is targeted data: spend self-play compute where the evaluator was demonstrably wrong instead of repeatedly sampling already-understood openings.

If implemented, introduce it as a separate controlled change. A simplified first version can prioritize measured prediction error without adding a dedicated regret network.

Candidate: uncertainty-guided branching

The UGE line branches from uncertain positions, plays several continuations and averages their outcomes to reduce value-label variance. This directly relates to the observed noise in z and the value-head decay hypothesis.

It is more expensive than changing the z/q blend, so the cheaper value-target experiment comes first.

Combination rules

• Do not introduce RGSC and uncertainty branching in the same run; both reweight difficult positions and their effects would be confounded.
• Keep external anchor measurement after every meaningful training change. Internal versus can prefer a sibling-specific model.
• Treat search, data selection and value-target changes as separate axes where possible.
• Any new teacher-generation shortcut must verify teacher quality, not just games/hour.

Next steps

1. Finish the cheap value-target (value_q_ratio) side experiment.
2. Measure 512-simulation teacher quality and the virtual-loss trade-off.
3. If the value side remains the limitation, test regret-guided starts before the heavier uncertainty-branching system.
4. Revisit alternative search algorithms only if Gumbel search itself becomes the measured bottleneck.