skip to content
[readonly]

The job log file: append-only records fsynced before what depends on them, a torn write cut back, replay at start, compaction by rewrite

sh
$ sexc add @ajax/mesh-jobd-store
  • total 3
  • drwxr-xr-x-../
  • drwxr-xr-x-Oct 27 05:24v/# 1 version
  • drwxr-xr-x-Oct 27 05:24files/# the files of 0.1.0
latest
0.1.0, published Oct 27 05:24 by ajax
owners
ajax
license
MIT
dependencies
@ajax/mesh-check ^0.1.0, @ajax/mesh-strbuf ^0.1.0, @ajax/mesh-jobs-log ^0.1.0, @ajax/mesh-jobs-out ^0.1.0
dependents
@ajax/mesh-jobd
keywords
mesh alphaos jobs log fsync durability
homepage
https://github.com/thomasdavis/alpha-c-mesh/tree/main/packages/mesh-jobd-store
repository
https://github.com/thomasdavis/alpha-c-mesh
==> README.md <==
1# @ajax/mesh-jobd-store
2
3The job log file, an append-only file of job log records (`@ajax/mesh-jobs-log`).
4
5- **Appends.** Records are fsynced before anything that depends on them leaves (`mj_durable`). A
6  write or sync that fails is fatal to the caller (fail-stop). A write that failed half-way is first
7  cut back to the last whole record, so a torn tail is never followed by a record (review finding 4).
8- **Start.** The log is replayed, a torn last record is dropped, and the file is rewritten compacted.
9- **Compaction.** When the file grows past twice its last compact size plus 8 MiB, it is rewritten
10  whole as the replica's own log, through a renamed file, so a failed compaction leaves the log as it
11  was.
12- **In memory.** Without a path, nothing is written.
13
14## Ported from
15
16[alpha-rust-mesh](https://github.com/rsafier/alpha-rust-mesh) (Richard Safier, MIT):
17`crates/mesh-host/src/jobd.rs`: `Store` (`open`, `rewrite`, `wants_compact`, its `Disk`).
18
19## Tests
20
21`tests/store.c`: the Rust's `a_torn_write_is_cut_back` (a failed half-write cut back, a torn tail at
22start dropped and rewritten), compaction, and the in-memory store.
23
24## Licence
25
26MIT. See the repository's LICENSE.