Manaka network history

This page records architecture changes and their outcomes. For the current structure, see architecture; for the original diagnosis, see structural critique.

Prototype

The baseline shape was pair16_d128_o3_h512: native tokens in two perspectives, power sums through order 3, a 16-channel full-rank pair table, one hidden layer, value and policy heads, plus policy_from in v3.

The design rule was unchanged throughout: every block before W1 had to remain a sum over tokens so it could be updated incrementally.

Proposals A–E

The 2026-08-24 critique produced five independent experiments.

proposal	change	target	inference cost
A	vary/remove pair_dim and order	identify which baseline blocks matter	configuration-dependent
B	hidden2	more head depth	extra dense layer
C	moment_dim	cross-channel second-order terms	wider W1 + update work
D	translation-tied pair training	sample efficiency	zero after export
E	field_dim	square-addressed context	wider W1 + policy field

B — second hidden layer

h2N inserts a second ReLU layer after W1. It cannot break incremental update because it is downstream of the accumulator, but its full cost is paid at every evaluated node.

C — second moments

mN maintains M = Σ v(z)v(z)ᵀ. Because this is still a sum over tokens it updates incrementally, while exposing cross-channel interactions absent from the elementwise symmetric-polynomial block.

D — translation tying

Training represented a pair row as free[pair] + tied[class(pair)], where the tied class uses piece classes and relative displacement. Export bakes the two into the ordinary flat pair table, so the engine and MNW1 inference layout do not change.

E — square field

fN maintains a square-indexed field A[s] = Σ K[token,s]. It gives W1 spatially addressed information and lets policy read the destination square directly. This was the only A–E proposal designed to preserve board layout after aggregation.

A–E measurements

The first comparison used one-epoch supervised weights and Simulations = 0, so the round robin measured policy quality without search. All eight variants were trained independently with the same seed and corpus.

Policy-only round robin, 2026-08-25

configuration	approximate Elo	val policy top1	val value sign
field4	+50	37.1%	69.0%
field2	+31	36.6%	68.2%
base	+12	36.0%	69.6%
order2	~0	36.4%	69.4%
pairtie	~−1	36.3%	70.2%
moment16	−8	36.3%	69.0%
pair0	−35	36.0%	70.5%
hidden2-256	−49	35.4%	69.8%

The useful conclusions are simple:

• Square fields worked. field2 and field4 were the only clear improvements over the prototype, and widening the field helped further.
• The pair table mattered. Removing it cost strength.
• More head depth did not help. hidden2=256 finished last and also costs inference time.
• Second moments did not help in this configuration.
• Lowering order 3→2 changed little in this policy-only test.
• Translation tying was approximately neutral; its main appeal remains zero inference cost.

These results apply only to those weights. Later validation records use different one-epoch runs, so a different ranking is not automatically a contradiction. See the common validation record.

Speed must be measured separately

Architecture gains are not free. hidden2, moments and fields widen or deepen work after the accumulator and therefore reduce nodes per second. Exact current throughput belongs in the benchmark, not in this historical experiment page.

The decision criterion for a production shape is therefore equal-time Elo, not policy-only Elo or validation alone.

Later aggregation proposals

The A–E result shifted attention away from “add more head depth” and toward how information is aggregated before W1. The later proposals were:

proposal	idea	status
F	lseN: log-sum-exp pair channels	implemented
G	ks: preserve discarded RMS magnitude	implemented
H	hashed split aggregation	not implemented here
I	per-token marginal pair sums	not implemented
J	one-move-ahead pair deltas	not implemented
K	output buckets / gated heads	not implemented
L	product pooling between perspectives	not implemented
M	training-only pair factorizer	not implemented

F — log-sum-exp pair channels

Ordinary pair channels sum 780 active pair rows and can dilute one decisive relation. lseN reallocates N existing pair channels to

log Σ exp(θ_pair)

without widening the table. exp(θ) is baked into the exported table, so incremental update remains add/subtract; only readout takes a logarithm. Weight format v5 added this support.

The implementation keeps θ clamped because repeated subtraction from a positive f32 sum must remain numerically reversible enough for the engine/trainer agreement tests.

G — preserve scale

ks gives the head log(rms(block)) in addition to the normalized direction. Shogi always has forty pieces, so block magnitude is position-dependent signal rather than merely set-size noise.

What has been measured since

The 2026-08-26 common validation run includes lse4, ks and lse4ks. None establishes a clear general win: CE and top1 rank the shapes differently. See the validation record.

pair64full also appears there and is the strongest example of metric disagreement: high top1 but poor CE. This is why later architecture choices must not be made from one validation metric alone.

Newer switches

The current shape grammar also contains:

• kp / halfkp — alternate NNUE feature spaces;
• nnue — export-compatible NNUE arithmetic;
• split — separate native-token power sums for own and opponent pieces while sharing parameters.

These belong to the current architecture rather than the original F–M proposal set. Their definitions are documented in architecture; KP/HalfKP measurements are in the feature-space record.

Why DLManaka is different

Manaka and DLManaka use the same 2,187-label policy space, but their first layers make different performance trade-offs.

Manaka

Manaka collapses token contributions into a compact accumulator before the head. A move changes only a few tokens, so the accumulator can be patched instead of rebuilt. This makes single-position CPU evaluation cheap, but aggregation can discard spatial detail.

DLManaka

DLManaka keeps a 9×9 spatial feature map through a deep residual tower. Its convolutions, nonlinearities and squeeze-excitation blocks make every later activation depend on surrounding or global state. After the first nonlinearity there is no compact additive delta that can reproduce the new network state.

Keeping every layer's feature maps per search node would also cost far more memory than Manaka's accumulator. DLManaka therefore evaluates whole positions and relies on GPU batching instead of incremental update.

What the comparison means

The design trade is structural:

	Manaka	DLManaka
main target	CPU	GPU
position state	compact additive accumulator	spatial feature maps
incremental update	yes	no
batching	optional / not central	essential
spatial information	compressed unless a field is enabled	preserved through the tower

Current speed figures are deliberately omitted here. They change with implementations and devices; use the common benchmark for an aligned measurement.

The architecture question is therefore not “which evaluator is universally faster”, but how much spatial/policy quality Manaka can recover while keeping enough of its incremental-update advantage to win at equal time.