Add mkcapsule --resolve: build-time claim registry for capsule block collisions (FABRIC-3.md §XXIV)
Build / build-amd64-iso (push) Waiting to run
Build / build-aarch64-iso (push) Waiting to run
Build / build-riscv64-img (push) Waiting to run

Traced what a "Block NNNN" collision actually means before designing
a fix for it: capsule_loader.c's block-write path routes through the
generic block-subsystem API, which kernel_main.c registers as two
devices in a fixed order -- the volatile ramdrive first (LBN
2048-3071), then Artemis's real virtio-blk device immediately after
(LBN 3072+, backed by disk/artemis.img). Every capsule this project
has lands in Artemis's persistent range, not the ramdrive, and
blk_update()'s dirty-marking + repl.c's idle-loop flush write that
content through to the real disk file on every boot. A block-number
collision is therefore a silent, persistent overwrite of real disk
content surviving reboots, not a transient RAM mixup.

The existing collision gate (check_block_conflicts(), already a hard
non-interactive build failure) already catches capsule-vs-capsule
collisions across the whole flat range. The real gap: zero visibility
into blocks something other than a capsule owns (Artemis's own
non-capsule persistent data), and no device-boundary/capacity
awareness at all.

Added, scoped step by step before writing any code:

- tools/capsule-claims.txt -- derived, auto-created/regenerated,
  git-ignored. Lets --resolve tell "this capsule's own content
  changed" apart from "genuinely new collision with something else."
- tools/capsule-reserved.txt -- human-authored, git-tracked, seeded
  with nothing yet rather than guessed at. Checked by both the plain
  build gate (new check_reserved_conflicts()) and --resolve.
- tools/patches/ -- git-tracked, one file per accepted interactive
  renumber; a structured old->new block list, not a generic diff,
  since that's the only thing a renumber ever changes.
- mkcapsule --resolve <dir> -- the only interactive mkcapsule mode,
  a deliberate separate invocation from the plain build path (which
  stays non-interactive so CI never blocks on a prompt). Suggests a
  renumbering that preserves a capsule's own existing block spacing,
  prompts y/N, rewrites the .4th source in place on acceptance.

Found and fixed a real bug during verification: the registry's
empty-block-list case (workload-5.4th, zero Block headers) serialized
with a stray trailing space that the reader parsed back as a phantom
block 0, causing spurious re-registration every run -- caught by
testing idempotency directly, not assuming a clean first run meant
it worked.

Verified: isolated collision tests confirm both accept and reject
paths, confirm a resolved collision doesn't re-prompt the other side,
confirm reserved-range collisions are caught by both --resolve and
the plain build gate. Full 3-architecture rebuild via the real
Makefile.starkernel succeeded clean; amd64 boots with an unchanged
dict_hash/capsule_hash from every prior boot this session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo
This commit is contained in:
Robert Allan James
2026-09-12 21:56:37 -04:00
co-authored by Claude Sonnet 5
parent c8ba8832c4
commit cd2fda4351
7 changed files with 9651 additions and 2 deletions
+105
View File
@@ -3096,3 +3096,108 @@ the current `capsules/` tree or `doe.4th`'s own CSV header -- corrected to descr
actually there. Verified: `mkcapsule --lint` clean, all 3 architectures build with zero new
warnings, amd64 boots with an unchanged dict_hash/capsule_hash from every prior boot this session.
## XXIV. `mkcapsule --resolve`: block-number collisions are silent persistent-disk corruption,
not a transient RAM mixup -- traced fully, then given a real build-time fix (2026-09-12)
**Scoping "add 3 workloads into the std79 factorial" surfaced a much bigger, longstanding
question.** Discussing which workloads to add and how (a genuine 5th factor was identified in
the process -- workload identity × placement mode, both worth varying independently, not
guessed at) led directly to Captain Bob's own instinct: "block collision is becoming an issue."
That was worth tracing to ground truth before designing anything, not just reassured against.
**`Block NNNN` is not a compile-time label -- it's a real, addressed, persistent storage
write, confirmed by reading the actual write path rather than assumed from the file format
convention.** `capsule_loader.c`'s block-write function (`write_ramdrive_block`, a legacy name --
its behavior is fully generic) calls `blk_get_buffer()`/`blk_update()`, the same block-subsystem
API every device uses. `kernel_main.c`'s own boot sequence registers exactly two devices, in a
fixed, confirmed order: the volatile ramdrive first (`capsule_blk_init`, LBN 2048-3071, resets
every boot), then Artemis's real virtio-blk device immediately after (LBN 3072 onward, ~22,998
blocks per an earlier boot log) -- backed by the actual `disk/artemis.img` this repo has carried
in git all session. Every capsule this project has (`acl-std79.4th` at 4023+, the workload
capsules at 4406+/4606+, `messaging.4th` at 5003+) lands in Artemis's range, not the ramdrive.
`blk_update()` marks the target block dirty in Artemis's real BAM; `repl.c`'s idle loop
(`blk_vm_flush_all()`, ~1 Hz) writes dirty blocks through to the actual disk file. **A block-
number collision between two capsules is therefore a silent, persistent overwrite of real disk
content, on every boot, surviving reboots until something else happens to rewrite that block --**
not a transient RAM mixup that clears on restart, which is what "collision" had been read as
before this was traced.
**The existing collision gate (`check_block_conflicts()`, already a hard, non-interactive build
failure -- not new) does catch this, for capsule-vs-capsule collisions specifically.** It's blind
to device topology entirely (treats every `Block NNNN` across every `.4th` file as one flat
integer space), which happens to be an accurate model of the real system's own flat LBN
addressing. The real gap: it only knows about `.4th` files it scans. It has zero visibility into
Artemis's own *non-capsule* persistent data (identity records, home-blocks fence, etc. -- real
data, not a `.4th` source file), and no awareness of the ramdrive/Artemis device boundary or
capacity limits at all.
**Built, scoped step by step with Captain Bob before any code, per the same discipline as every
other fix this session:** a proper build-time linker for capsule block claims, not a runtime
mechanism -- "mkcapsule is our secret sauce linker" was the framing that stuck. Three pieces,
deliberately separated by persistence model:
- **`tools/capsule-claims.txt`** -- fully derived, auto-created/regenerated, git-ignored.
Capsule name → content hash → claimed blocks. Exists so the interactive resolver (below) can
tell "this capsule's own content changed" apart from "this is a genuinely new collision with
something else" -- the plain build gate doesn't need this distinction (it just re-scans and
hard-fails every time), only the interactive tool does.
- **`tools/capsule-reserved.txt`** -- human-authored, git-tracked, seeded with nothing yet
(deliberately -- guessing at Artemis's real block usage would be worse than admitting it isn't
known yet). Format: `START-END description` per line. Both the plain build gate and
`--resolve` check every capsule's blocks against it.
- **`tools/patches/`** -- git-tracked, one file per *accepted* interactive renumber. Not a
generic line-diff (a renumber only ever changes `Block N` integers, never content, so a
structured `old -> new` list is more directly useful than a diff would be) -- a permanent
decision record, separate from the disposable registry.
**`mkcapsule --resolve <dir>`** is the only interactive mkcapsule mode, and deliberately a
*separate* invocation from the plain `make` build path -- a human runs it on purpose when adding
something new; CI/unattended builds never see a prompt. Auto-creates the claims registry from a
fresh scan if missing. For each capsule: unchanged (same hash, same blocks as last run) is a
silent no-op; changed-but-still-free is a silent re-claim (logged, no prompt -- moving your own
capsule's own content around isn't a conflict); a genuine collision (with another capsule's
*current* blocks, or a reserved range) computes a suggested renumbering that preserves the
capsule's own existing block spacing (shifts the whole set by one delta, rather than reassigning
each block independently, so `workload-3.4th`'s 4606,4615,4625,...,4665 convention survives a
renumber intact) and prompts y/N. Accepting rewrites the `.4th` source in place and writes the
patch record; rejecting or finding no free shift within a bounded search leaves everything
untouched and marks the run failed.
**The plain build gate gained the reserved-range check too** (`check_reserved_conflicts()`,
same hard-fail, non-interactive shape as the existing capsule-vs-capsule gate) -- closing half
of the visibility gap immediately, with the other half (auto-discovering Artemis's own real
block usage) left honestly unseeded rather than guessed at.
**A real bug found and fixed during verification, not just a clean first pass:** the registry's
own empty-block-list case (`workload-5.4th`, a raw code capsule with zero `Block N` headers)
serialized with a stray trailing space, which the reader parsed back as a phantom block `0` --
causing every re-run to spuriously "re-register" that one capsule forever. Caught by testing
idempotency directly (run `--resolve` twice with nothing changed, expect zero registrations the
second time) rather than assuming a clean first run meant the mechanism was correct. Fixed in
both the writer (no trailing space when a capsule has zero blocks) and the reader (defensively
skip whitespace-only tokens regardless).
**Verified end to end, not just compiled:** an isolated test directory with a deliberate
capsule-vs-capsule collision confirmed both the reject path (declines, exits non-zero, zero
file changes) and the accept path (rewrites the source in place, writes the patch, and --
correctly -- does *not* re-prompt for the second capsule in the same collision once the first
capsule's move already cleared it, since the live in-memory block state updates immediately
after an accepted renumber). A second isolated test confirmed reserved-range collisions are
caught identically by both `--resolve` and the plain build gate. Full 3-architecture rebuild via
the real `Makefile.starkernel` (not just a standalone `gcc` compile) succeeded clean; amd64
boot-verified with an unchanged dict_hash/capsule_hash from every prior boot this session.
**Explicitly deferred, not built:** a CI-side idea Bob raised -- on a missing registry or a
fresh conflict, check whether an existing patch in `tools/patches/` already resolves it before
failing the build outright. Reasonable, but needs its own careful design (reliably matching a
stored patch to a specific conflict) rather than folding into this pass. Also noted for later,
not now: Bob's idea of an LLM capsule someday improving the renumbering suggestion itself, once
`--resolve` has real mileage on it.
**Next step, per Bob's own call:** `mkcapsule.c` has grown to 1600+ lines across genuinely
separable jobs (scanning/parsing, lint, manifest, conflict resolution, C-source codegen) and is
due for a decomposition into smaller logical units -- an ETL shape (extract: scan/parse capsule
files; transform: lint/manifest/conflict-resolve; load: C-source generation) was the starting
idea, to be done as its own clean pass once the workload-into-factorial design (the actual
reason this was scoped) resumes.