1# @ajax/mesh-prog-gpt2x
2
3GPT-2 124M's exact-mode GPU kernels: the window (PUT, COPY), EMBED, NORM, GEMM and GEMV with their
4epilogues (bias, GELU, residual), KV, ATTN and LOGITS, every op following the CPU reference operation
5for operation (FTZ, correctly rounded, unfused, fixed sum orders; exp, tanh and sqrt from + - * /), so
6the logits and the KV cache are bitwise the reference's on every GPU. The host runs the model as
7stages of tasks. Layout: @ajax/mesh-abi-view-gpt2x.
8
9- **Ported from:** alpha-rust-mesh `programs/gpt2x/src/lib.rs`.
10- **Tests:** on the simulator, op by op against @ajax/mesh-ref-gpt2-*'s row operations, bitwise, on
11  synthetic data: the scalar functions over a sweep, the window, EMBED, NORM, GEMM and GEMV with each
12  epilogue off the tile grid, KV and causal ATTN for 12 heads, LOGITS for the first and last blocks.