Add mkcapsule --resolve: build-time claim registry for capsule block collisions (FABRIC-3.md §XXIV)
Traced what a "Block NNNN" collision actually means before designing a fix for it: capsule_loader.c's block-write path routes through the generic block-subsystem API, which kernel_main.c registers as two devices in a fixed order -- the volatile ramdrive first (LBN 2048-3071), then Artemis's real virtio-blk device immediately after (LBN 3072+, backed by disk/artemis.img). Every capsule this project has lands in Artemis's persistent range, not the ramdrive, and blk_update()'s dirty-marking + repl.c's idle-loop flush write that content through to the real disk file on every boot. A block-number collision is therefore a silent, persistent overwrite of real disk content surviving reboots, not a transient RAM mixup. The existing collision gate (check_block_conflicts(), already a hard non-interactive build failure) already catches capsule-vs-capsule collisions across the whole flat range. The real gap: zero visibility into blocks something other than a capsule owns (Artemis's own non-capsule persistent data), and no device-boundary/capacity awareness at all. Added, scoped step by step before writing any code: - tools/capsule-claims.txt -- derived, auto-created/regenerated, git-ignored. Lets --resolve tell "this capsule's own content changed" apart from "genuinely new collision with something else." - tools/capsule-reserved.txt -- human-authored, git-tracked, seeded with nothing yet rather than guessed at. Checked by both the plain build gate (new check_reserved_conflicts()) and --resolve. - tools/patches/ -- git-tracked, one file per accepted interactive renumber; a structured old->new block list, not a generic diff, since that's the only thing a renumber ever changes. - mkcapsule --resolve <dir> -- the only interactive mkcapsule mode, a deliberate separate invocation from the plain build path (which stays non-interactive so CI never blocks on a prompt). Suggests a renumbering that preserves a capsule's own existing block spacing, prompts y/N, rewrites the .4th source in place on acceptance. Found and fixed a real bug during verification: the registry's empty-block-list case (workload-5.4th, zero Block headers) serialized with a stray trailing space that the reader parsed back as a phantom block 0, causing spurious re-registration every run -- caught by testing idempotency directly, not assuming a clean first run meant it worked. Verified: isolated collision tests confirm both accept and reject paths, confirm a resolved collision doesn't re-prompt the other side, confirm reserved-range collisions are caught by both --resolve and the plain build gate. Full 3-architecture rebuild via the real Makefile.starkernel succeeded clean; amd64 boots with an unchanged dict_hash/capsule_hash from every prior boot this session. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
c8ba8832c4
commit
cd2fda4351
+105
@@ -3096,3 +3096,108 @@ the current `capsules/` tree or `doe.4th`'s own CSV header -- corrected to descr
|
||||
actually there. Verified: `mkcapsule --lint` clean, all 3 architectures build with zero new
|
||||
warnings, amd64 boots with an unchanged dict_hash/capsule_hash from every prior boot this session.
|
||||
|
||||
## XXIV. `mkcapsule --resolve`: block-number collisions are silent persistent-disk corruption,
|
||||
not a transient RAM mixup -- traced fully, then given a real build-time fix (2026-09-12)
|
||||
|
||||
**Scoping "add 3 workloads into the std79 factorial" surfaced a much bigger, longstanding
|
||||
question.** Discussing which workloads to add and how (a genuine 5th factor was identified in
|
||||
the process -- workload identity × placement mode, both worth varying independently, not
|
||||
guessed at) led directly to Captain Bob's own instinct: "block collision is becoming an issue."
|
||||
That was worth tracing to ground truth before designing anything, not just reassured against.
|
||||
|
||||
**`Block NNNN` is not a compile-time label -- it's a real, addressed, persistent storage
|
||||
write, confirmed by reading the actual write path rather than assumed from the file format
|
||||
convention.** `capsule_loader.c`'s block-write function (`write_ramdrive_block`, a legacy name --
|
||||
its behavior is fully generic) calls `blk_get_buffer()`/`blk_update()`, the same block-subsystem
|
||||
API every device uses. `kernel_main.c`'s own boot sequence registers exactly two devices, in a
|
||||
fixed, confirmed order: the volatile ramdrive first (`capsule_blk_init`, LBN 2048-3071, resets
|
||||
every boot), then Artemis's real virtio-blk device immediately after (LBN 3072 onward, ~22,998
|
||||
blocks per an earlier boot log) -- backed by the actual `disk/artemis.img` this repo has carried
|
||||
in git all session. Every capsule this project has (`acl-std79.4th` at 4023+, the workload
|
||||
capsules at 4406+/4606+, `messaging.4th` at 5003+) lands in Artemis's range, not the ramdrive.
|
||||
`blk_update()` marks the target block dirty in Artemis's real BAM; `repl.c`'s idle loop
|
||||
(`blk_vm_flush_all()`, ~1 Hz) writes dirty blocks through to the actual disk file. **A block-
|
||||
number collision between two capsules is therefore a silent, persistent overwrite of real disk
|
||||
content, on every boot, surviving reboots until something else happens to rewrite that block --**
|
||||
not a transient RAM mixup that clears on restart, which is what "collision" had been read as
|
||||
before this was traced.
|
||||
|
||||
**The existing collision gate (`check_block_conflicts()`, already a hard, non-interactive build
|
||||
failure -- not new) does catch this, for capsule-vs-capsule collisions specifically.** It's blind
|
||||
to device topology entirely (treats every `Block NNNN` across every `.4th` file as one flat
|
||||
integer space), which happens to be an accurate model of the real system's own flat LBN
|
||||
addressing. The real gap: it only knows about `.4th` files it scans. It has zero visibility into
|
||||
Artemis's own *non-capsule* persistent data (identity records, home-blocks fence, etc. -- real
|
||||
data, not a `.4th` source file), and no awareness of the ramdrive/Artemis device boundary or
|
||||
capacity limits at all.
|
||||
|
||||
**Built, scoped step by step with Captain Bob before any code, per the same discipline as every
|
||||
other fix this session:** a proper build-time linker for capsule block claims, not a runtime
|
||||
mechanism -- "mkcapsule is our secret sauce linker" was the framing that stuck. Three pieces,
|
||||
deliberately separated by persistence model:
|
||||
|
||||
- **`tools/capsule-claims.txt`** -- fully derived, auto-created/regenerated, git-ignored.
|
||||
Capsule name → content hash → claimed blocks. Exists so the interactive resolver (below) can
|
||||
tell "this capsule's own content changed" apart from "this is a genuinely new collision with
|
||||
something else" -- the plain build gate doesn't need this distinction (it just re-scans and
|
||||
hard-fails every time), only the interactive tool does.
|
||||
- **`tools/capsule-reserved.txt`** -- human-authored, git-tracked, seeded with nothing yet
|
||||
(deliberately -- guessing at Artemis's real block usage would be worse than admitting it isn't
|
||||
known yet). Format: `START-END description` per line. Both the plain build gate and
|
||||
`--resolve` check every capsule's blocks against it.
|
||||
- **`tools/patches/`** -- git-tracked, one file per *accepted* interactive renumber. Not a
|
||||
generic line-diff (a renumber only ever changes `Block N` integers, never content, so a
|
||||
structured `old -> new` list is more directly useful than a diff would be) -- a permanent
|
||||
decision record, separate from the disposable registry.
|
||||
|
||||
**`mkcapsule --resolve <dir>`** is the only interactive mkcapsule mode, and deliberately a
|
||||
*separate* invocation from the plain `make` build path -- a human runs it on purpose when adding
|
||||
something new; CI/unattended builds never see a prompt. Auto-creates the claims registry from a
|
||||
fresh scan if missing. For each capsule: unchanged (same hash, same blocks as last run) is a
|
||||
silent no-op; changed-but-still-free is a silent re-claim (logged, no prompt -- moving your own
|
||||
capsule's own content around isn't a conflict); a genuine collision (with another capsule's
|
||||
*current* blocks, or a reserved range) computes a suggested renumbering that preserves the
|
||||
capsule's own existing block spacing (shifts the whole set by one delta, rather than reassigning
|
||||
each block independently, so `workload-3.4th`'s 4606,4615,4625,...,4665 convention survives a
|
||||
renumber intact) and prompts y/N. Accepting rewrites the `.4th` source in place and writes the
|
||||
patch record; rejecting or finding no free shift within a bounded search leaves everything
|
||||
untouched and marks the run failed.
|
||||
|
||||
**The plain build gate gained the reserved-range check too** (`check_reserved_conflicts()`,
|
||||
same hard-fail, non-interactive shape as the existing capsule-vs-capsule gate) -- closing half
|
||||
of the visibility gap immediately, with the other half (auto-discovering Artemis's own real
|
||||
block usage) left honestly unseeded rather than guessed at.
|
||||
|
||||
**A real bug found and fixed during verification, not just a clean first pass:** the registry's
|
||||
own empty-block-list case (`workload-5.4th`, a raw code capsule with zero `Block N` headers)
|
||||
serialized with a stray trailing space, which the reader parsed back as a phantom block `0` --
|
||||
causing every re-run to spuriously "re-register" that one capsule forever. Caught by testing
|
||||
idempotency directly (run `--resolve` twice with nothing changed, expect zero registrations the
|
||||
second time) rather than assuming a clean first run meant the mechanism was correct. Fixed in
|
||||
both the writer (no trailing space when a capsule has zero blocks) and the reader (defensively
|
||||
skip whitespace-only tokens regardless).
|
||||
|
||||
**Verified end to end, not just compiled:** an isolated test directory with a deliberate
|
||||
capsule-vs-capsule collision confirmed both the reject path (declines, exits non-zero, zero
|
||||
file changes) and the accept path (rewrites the source in place, writes the patch, and --
|
||||
correctly -- does *not* re-prompt for the second capsule in the same collision once the first
|
||||
capsule's move already cleared it, since the live in-memory block state updates immediately
|
||||
after an accepted renumber). A second isolated test confirmed reserved-range collisions are
|
||||
caught identically by both `--resolve` and the plain build gate. Full 3-architecture rebuild via
|
||||
the real `Makefile.starkernel` (not just a standalone `gcc` compile) succeeded clean; amd64
|
||||
boot-verified with an unchanged dict_hash/capsule_hash from every prior boot this session.
|
||||
|
||||
**Explicitly deferred, not built:** a CI-side idea Bob raised -- on a missing registry or a
|
||||
fresh conflict, check whether an existing patch in `tools/patches/` already resolves it before
|
||||
failing the build outright. Reasonable, but needs its own careful design (reliably matching a
|
||||
stored patch to a specific conflict) rather than folding into this pass. Also noted for later,
|
||||
not now: Bob's idea of an LLM capsule someday improving the renumbering suggestion itself, once
|
||||
`--resolve` has real mileage on it.
|
||||
|
||||
**Next step, per Bob's own call:** `mkcapsule.c` has grown to 1600+ lines across genuinely
|
||||
separable jobs (scanning/parsing, lint, manifest, conflict resolution, C-source codegen) and is
|
||||
due for a decomposition into smaller logical units -- an ETL shape (extract: scan/parse capsule
|
||||
files; transform: lint/manifest/conflict-resolve; load: C-source generation) was the starting
|
||||
idea, to be done as its own clean pass once the workload-into-factorial design (the actual
|
||||
reason this was scoped) resumes.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user