The block collectives of a multi-warp node on the CPU: warp butterflies and scans, block reduce and scan, step for step as the GPU
$ sexc add @ajax/mesh-ref-block- total 3
- drwxr-xr-x-../
- drwxr-xr-x-Oct 27 01:59v/# 1 version
- drwxr-xr-x-Oct 27 01:59files/# the files of 0.1.0
- latest
- 0.1.0, published Oct 27 01:59 by ajax
- owners
- ajax
- license
- MIT
- dependencies
- @ajax/mesh-ftz ^0.1.0
- dependents
- @ajax/mesh-prog-blocks, @ajax/mesh-prog-gpt2x, @ajax/mesh-ref-cubic, @ajax/mesh-ref-gpt2-attend, @ajax/mesh-ref-gpt2-norm, @ajax/mesh-ref-matmul
- keywords
- alphaos mesh reference reduce scan warp
- homepage
- https://github.com/thomasdavis/alpha-c-mesh/tree/main/packages/mesh-ref-block
- repository
- https://github.com/thomasdavis/alpha-c-mesh
1# @ajax/mesh-ref-block 2 3The block collectives of a multi-warp node, on the CPU, step for step as the GPU computes them: the warp 4butterfly (`lane l := op(own, lane l ^ m)` for m = 16 … 1), the warp's inclusive Hillis–Steele scan, and 5the block reduce and scan built from them (each warp, then the warp totals, then the prefixes). Every f32 6operation is in the cluster float mode (`@ajax/mesh-ftz`), so for thread values `vals` a GPU's result is 7bitwise the one these return. 8 9```c 10float s = mesh_block_reduce_f32(vals, 256, mesh_block_sum_f32_id(), mesh_block_add_f32); 11mesh_block_scan_u32(vals, 256, 0, mesh_block_add_u32, true, out); 12``` 13 14Also `mesh_block_mix` (the lowbias32 hash every backend shares), `mesh_block_max_f32` (the IEEE total 15order of the bits) and `mesh_block_test_values` (what programs/blocks gives each thread). 16 17**Ported from** `crates/mesh-ref/src/block.rs`. Freestanding (no libc). 18 19**Tested** by block.rs's test, ported, and a golden digest from the Rust over every collective (f32 sum 20and max, u32 and u64 sums, inclusive and exclusive scans) for 20 seeds at 32 to 1024 threads. 21 22## Differences 23 24- Rust's functions are generic over `T` with a closure; here there are `_f32`, `_u32` and `_u64` versions 25 taking a function pointer. 26- Rust panics when the thread count isn't a multiple of 32 in 32..1024. Here `reduce` returns `id` and 27 `scan` returns false (leaving `out` alone).