1# @ajax/mesh-ref-gpt2
2
3GPT-2 124M in the cluster float mode on the CPU: the CPU reference of programs/gpt2x (the exact-mode GPU
4kernels) and the exact executor of programs/gpt2 on a CPU. Every value is a fixed function of the weights
5and the token prefix — the order of every sum is fixed and the GPU kernels follow it operation for
6operation — so the logits and the KV cache are bitwise the same on every executor, and a row computed
7alone (decode) equals the same row computed in a batch (prefill).
8
9`mesh_gpt2_forward` runs rows through the model and appends their K and V to the cache;
10`mesh_gpt2_forward_layers` calls back after each layer (a prefill can stream its KV); `mesh_gpt2_generate`
11is greedy generation with one digest over every step's logits. Work is split over POSIX threads by
12(row, column block), by row and by vocabulary row; the results never depend on the thread count.
13
14**Ported from** `crates/mesh-ref/src/gpt2.rs` (`forward`, `forward_layers`, `generate`, `par_linear`,
15`par_rows`, `project_vocab`). Host only.
16
17**Tested** on synthetic weights against golden data from the Rust: the row operations on hashed data,
18the weights, a greedy generation (tokens, logits digest, KV digest), and gpt2.rs's
19`decode_rows_equal_prefill_rows` — three tokens one at a time and all at once give the same KV and
20logits, both equal to the Rust's.
21
22## Differences
23
24- Threads come from a shared counter over independent units rather than Rust's scoped chunking; outputs
25  go to caller buffers; out-of-memory and context overflow return false instead of panicking.