The gpt2 service protocol's wire: frames, key=value messages, checksummed KV layers
$ sexc add @ajax/mesh-infer-wire- total 3
- drwxr-xr-x-../
- drwxr-xr-x-Oct 27 05:23v/# 1 version
- drwxr-xr-x-Oct 27 05:23files/# the files of 0.1.0
- latest
- 0.1.0, published Oct 27 05:23 by ajax
- owners
- ajax
- license
- MIT
- dependencies
- @ajax/mesh-buf ^0.1.0, @ajax/mesh-rsfmt ^0.1.0, @ajax/mesh-chan ^0.1.0, @ajax/mesh-check ^0.1.0
- dependents
- @ajax/mesh-infer-request, @ajax/mesh-inferd
- keywords
- mesh alphaos inference gpt2 protocol frames
- homepage
- https://github.com/thomasdavis/alpha-c-mesh/tree/main/packages/mesh-infer-wire
- repository
- https://github.com/thomasdavis/alpha-c-mesh
1# @ajax/mesh-infer-wire 2 3The gpt2 service protocol's wire: frames, key=value messages, checksummed KV layers. 4 5The wire of the inference services (programs/gpt2; docs/plan-disagg-inference.md). 6 7Every gpt2 worker listens on TCP; frames are a u32 length (LE, of what follows), a u8 kind, a payload. A request (K_GEN, K_DECODE, K_PREFILL) carries a Msg: `key=value` lines (sorted by key), a blank line, a body. KV travels as K_LAYER frames (layer, rows, a checksum, the bytes); K_FIRST ends a prefill; K_RESULT and K_ERROR answer. 8 9```c 10#include "mesh/infer_wire.h" 11``` 12 13## Ported from 14 15[alpha-rust-mesh](https://github.com/rsafier/alpha-rust-mesh) (Richard Safier, MIT): crates/mesh-host/src/infer.rs: `K_*`, `MAX_FRAME`, `REQUEST_WAIT`, `PROGRAM_*`, `write_frame`, `read_frame`, `read_frame_timed`, `Msg` (set, put, get, need, num, encode, decode), `layer_sum`, `layer_frame`, `parse_layer`. 16 17## Tests 18 19- `tests/wire.c`: infer.rs's `messages_and_layers_round_trip` (over a socketpair), and every error text. 20 21## Differences 22 23- A message holding a NUL byte is refused as not UTF-8 (C strings cannot carry it); Rust would accept it. The workers never send one. 24 25## Licence 26 27MIT. See the repository's LICENSE.