skip to content
[readonly]

The block collectives of a multi-warp node on the CPU: warp butterflies and scans, block reduce and scan, step for step as the GPU

sh
$ sexc add @ajax/mesh-ref-block
  • total 3
  • drwxr-xr-x-../
  • drwxr-xr-x-Oct 27 01:59v/# 1 version
  • drwxr-xr-x-Oct 27 01:59files/# the files of 0.1.0
latest
0.1.0, published Oct 27 01:59 by ajax
owners
ajax
license
MIT
dependencies
@ajax/mesh-ftz ^0.1.0
dependents
@ajax/mesh-prog-blocks, @ajax/mesh-prog-gpt2x, @ajax/mesh-ref-cubic, @ajax/mesh-ref-gpt2-attend, @ajax/mesh-ref-gpt2-norm, @ajax/mesh-ref-matmul
keywords
alphaos mesh reference reduce scan warp
homepage
https://github.com/thomasdavis/alpha-c-mesh/tree/main/packages/mesh-ref-block
repository
https://github.com/thomasdavis/alpha-c-mesh
==> README.md <==
1# @ajax/mesh-ref-block
2
3The block collectives of a multi-warp node, on the CPU, step for step as the GPU computes them: the warp
4butterfly (`lane l := op(own, lane l ^ m)` for m = 16 … 1), the warp's inclusive Hillis–Steele scan, and
5the block reduce and scan built from them (each warp, then the warp totals, then the prefixes). Every f32
6operation is in the cluster float mode (`@ajax/mesh-ftz`), so for thread values `vals` a GPU's result is
7bitwise the one these return.
8
9```c
10float s = mesh_block_reduce_f32(vals, 256, mesh_block_sum_f32_id(), mesh_block_add_f32);
11mesh_block_scan_u32(vals, 256, 0, mesh_block_add_u32, true, out);
12```
13
14Also `mesh_block_mix` (the lowbias32 hash every backend shares), `mesh_block_max_f32` (the IEEE total
15order of the bits) and `mesh_block_test_values` (what programs/blocks gives each thread).
16
17**Ported from** `crates/mesh-ref/src/block.rs`. Freestanding (no libc).
18
19**Tested** by block.rs's test, ported, and a golden digest from the Rust over every collective (f32 sum
20and max, u32 and u64 sums, inclusive and exclusive scans) for 20 seeds at 32 to 1024 threads.
21
22## Differences
23
24- Rust's functions are generic over `T` with a closure; here there are `_f32`, `_u32` and `_u64` versions
25  taking a function pointer.
26- Rust panics when the thread count isn't a multiple of 32 in 32..1024. Here `reduce` returns `id` and
27  `scan` returns false (leaving `out` alone).