GPT-2's KV cache: per layer, K and V rows of the full width for each position, and its digest
$ sexc add @ajax/mesh-ref-gpt2-kv- total 3
- drwxr-xr-x-../
- drwxr-xr-x-Oct 27 01:59v/# 1 version
- drwxr-xr-x-Oct 27 01:59files/# the files of 0.1.0
- latest
- 0.1.0, published Oct 27 01:59 by ajax
- owners
- ajax
- license
- MIT
- dependencies
- @ajax/mesh-ftz ^0.1.0, @ajax/mesh-ref-gpt2-shape ^0.1.0, @ajax/mesh-ref-gpt2-logits ^0.1.0
- dependents
- @ajax/mesh-ref-gpt2
- keywords
- alphaos mesh reference gpt2 determinism
- homepage
- https://github.com/thomasdavis/alpha-c-mesh/tree/main/packages/mesh-ref-gpt2-kv
- repository
- https://github.com/thomasdavis/alpha-c-mesh
1# @ajax/mesh-ref-gpt2-kv 2 3GPT-2's KV cache: per layer, the K and V rows (768 wide, heads concatenated) of every position so far, 4and its digest (FNV-1a 64 over each layer's K then V rows). Rows are only appended; during a forward pass 5layer l's rows land before layer l + 1's, and `mesh_gpt2_kv_len` reports layer 0's count, as the Rust 6`Kv::len` does. 7 8**Ported from** `crates/mesh-ref/src/gpt2.rs` (`Kv`). Host only. **Tested** through the generation and 9prefill/decode golden data of `@ajax/mesh-ref-gpt2`. 10 11## Differences 12 13- `Vec<Vec<f32>>` per layer becomes one growable buffer per layer with a shared capacity and per-layer 14 row counts.