1# @ajax/mesh-ref-gpt2-shape
2
3GPT-2 124M's shape — 12 layers, 12 heads of 64, width 768, feed-forward 3072, vocabulary 50 257, context
41024, the node's 256 threads — and the flat weight buffer's layout: every tensor in checkpoint order and
5shape (`wte`, `wpe`, the twelve tensors of each block, `ln_f`), the same on the CPU and in the GPU's
6scratch. `mesh_gpt2_tensor` gives each tensor's checkpoint name, offset and count.
7
8**Ported from** `crates/mesh-ref/src/gpt2.rs` (the constants, `BlockOff`, `Layout`, `Layout::tensors`).
9Freestanding. **Tested** by gpt2.rs's `layout_covers_the_checkpoint` and a golden digest of every
10tensor's name, offset and count from the Rust.
11
12## Differences
13
14- `tensors()` returns a `Vec<(String, usize, usize)>`; here `mesh_gpt2_tensor(i, ...)` fills a buffer.