The job log file: append-only records fsynced before what depends on them, a torn write cut back, replay at start, compaction by rewrite
$ sexc add @ajax/mesh-jobd-store- total 3
- drwxr-xr-x-../
- drwxr-xr-x-Oct 27 05:24v/# 1 version
- drwxr-xr-x-Oct 27 05:24files/# the files of 0.1.0
- latest
- 0.1.0, published Oct 27 05:24 by ajax
- owners
- ajax
- license
- MIT
- dependencies
- @ajax/mesh-check ^0.1.0, @ajax/mesh-strbuf ^0.1.0, @ajax/mesh-jobs-log ^0.1.0, @ajax/mesh-jobs-out ^0.1.0
- dependents
- @ajax/mesh-jobd
- keywords
- mesh alphaos jobs log fsync durability
- homepage
- https://github.com/thomasdavis/alpha-c-mesh/tree/main/packages/mesh-jobd-store
- repository
- https://github.com/thomasdavis/alpha-c-mesh
1# @ajax/mesh-jobd-store 2 3The job log file, an append-only file of job log records (`@ajax/mesh-jobs-log`). 4 5- **Appends.** Records are fsynced before anything that depends on them leaves (`mj_durable`). A 6 write or sync that fails is fatal to the caller (fail-stop). A write that failed half-way is first 7 cut back to the last whole record, so a torn tail is never followed by a record (review finding 4). 8- **Start.** The log is replayed, a torn last record is dropped, and the file is rewritten compacted. 9- **Compaction.** When the file grows past twice its last compact size plus 8 MiB, it is rewritten 10 whole as the replica's own log, through a renamed file, so a failed compaction leaves the log as it 11 was. 12- **In memory.** Without a path, nothing is written. 13 14## Ported from 15 16[alpha-rust-mesh](https://github.com/rsafier/alpha-rust-mesh) (Richard Safier, MIT): 17`crates/mesh-host/src/jobd.rs`: `Store` (`open`, `rewrite`, `wants_compact`, its `Disk`). 18 19## Tests 20 21`tests/store.c`: the Rust's `a_torn_write_is_cut_back` (a failed half-write cut back, a torn tail at 22start dropped and rewritten), compaction, and the in-memory store. 23 24## Licence 25 26MIT. See the repository's LICENSE.