skip to content
[readonly]

A member's blob store: job payloads content-addressed by sha256 and referenced per job, bounded, evicting only forgotten jobs' references, verified on read

sh
$ sexc add @ajax/mesh-blobs-store
  • total 3
  • drwxr-xr-x-../
  • drwxr-xr-x-Oct 27 05:23v/# 1 version
  • drwxr-xr-x-Oct 27 05:23files/# the files of 0.1.0
latest
0.1.0, published Oct 27 05:23 by ajax
owners
ajax
license
MIT
dependencies
@ajax/mesh-check ^0.1.0, @ajax/mesh-strbuf ^0.1.0, @ajax/mesh-omap ^0.1.0, @ajax/mesh-jobs-id ^0.1.0, @ajax/mesh-ref-sha256d ^0.1.0, @ajax/mesh-blobs-durable ^0.1.0
dependents
@ajax/mesh-blobd, @ajax/mesh-jobd
keywords
mesh alphaos blobs content-addressed storage
homepage
https://github.com/thomasdavis/alpha-c-mesh/tree/main/packages/mesh-blobs-store
repository
https://github.com/thomasdavis/alpha-c-mesh
==> README.md <==
1# @ajax/mesh-blobs-store
2
3A member's blob store: job items' payloads, kept off the job log, content-addressed by their sha256
4and **referenced per job**.
5
6- **Layout.** A job's blob is the file `<dir>/<job>/<sha256 hex>`, either whole under its name or
7  absent (`@ajax/mesh-blobs-durable` writes it). Every read checks the bytes against the name. A
8  corrupt or missing file is removed, reads as absent, and is reported lost.
9- **One blob, several jobs.** Two jobs that render the same scene compute byte-identical payloads.
10  Each job holds its own reference, a hard link of the file another job's reference names, so the
11  bytes are on disk once. Forgetting or evicting one job removes only its references. A blob goes with
12  its last reference. (In the live fleet, forgetting job 1.2 once took job 2.1's committed blobs with
13  it; this layout prevents that.)
14- **Bounded.** At most `cap` bytes, each blob counted once. Room is made only by evicting *forgotten*
15  jobs' references. A store full of listed jobs' blobs refuses: `job forget` a finished job to make
16  room.
17- **In memory.** Without a directory, blobs are kept in memory under the same cap.
18
19## Ported from
20
21[alpha-rust-mesh](https://github.com/rsafier/alpha-rust-mesh) (Richard Safier, MIT):
22`crates/mesh-host/src/blobs.rs`: `Live`, `Make`, `Reserved`, `Store` (`open`, `reserve`, `written`,
23`put_mem`, `get`, `take_lost`, `drop_job`, `jobs`, `all`, the counts).
24
25## Differences
26
27The Rust keeps its index in a `HashMap`, so `all()` and `drop_job()` list hashes in arbitrary order.
28Here they come in hash order.
29
30## Tests
31
32`tests/store.c`: the Rust's four tests:
33- `write_rename_durability`;
34- `hash_verification`;
35- `a_blob_of_two_jobs_outlives_either`, on disk and in memory, with the link race;
36- `eviction`, on disk and in memory.
37
38## Licence
39
40MIT. See the repository's LICENSE.