diff --git a/FABRIC-3.md b/FABRIC-3.md index aa7f796f..b389db67 100644 --- a/FABRIC-3.md +++ b/FABRIC-3.md @@ -1676,3 +1676,179 @@ grows large enough for it to matter again. `printf '$(KERNEL_ARGS)\n' > starforth.cfg` breaks when `KERNEL_ARGS` starts with `--` (dash's `printf` misreads it as an option). Worked around live during X.2's debug-level testing with a harmless prefix token (`KERNEL_ARGS="x --log-level=debug"`); not fixed in the Makefile itself. + +## XI. FORTH-79/83 identity lockdown, Hera→Artemis storage-attach messaging, and a real +"identity attach doesn't complete" bug — CLOSED, 2026-09-07/09 + +Three threads that turned out to be entangled: (1) a new opt-in ACL lockdown personality for +`MINT`, restricting a minted identity to FORTH-79/83 standard words only; (2) migrating the +storage-attach step of a USB identity's attach from a direct C call inside Hera's own idle loop +to a real cross-VM message round-trip through Artemis; (3) a "why does attaching an identity +without Zuse ever attaching first take many real minutes, or never complete" bug that took an +extraordinarily long live-debugging session — many QEMU boots, live register/memory forensics +via QEMU's own GDB stub, a fresh subagent dive — to actually root-cause, because the true cause +turned out to be two independent, unrelated bugs stacked on top of each other, each one masking +clean evidence of the other until the first was fixed. + +### XI.1 — `MINT` restricted-personality lockdown — CLOSED, commit `f4ded3e` + +Captain Bob's directive: "starting with that 00 user we created, we're going to give access +only to FORTH 79 and 83 standard words. everything else is locked down." New +`capsules/acl-std79.4th`: `ACL-STD79-LIST` (a real, hand-built allowlist of ~200 FORTH-79/83 +standard-word execution tokens), `ACL-STD79-ALLOWED?`, and `ACL-LOCKDOWN-STD79` — a one-shot +walk of the entire dictionary from a fixed marker (`ACL-WALK-MARK`) that denies (`ACL-ALLOW! +0`) and permanently pins (`ACL-PIN`) every word not on the allowlist, allows+pins every word +that is. `MintPersonality` (`capsule_mint.h`) gives `capsule_mint_identity()` a +`MINT_PERSONALITY_STD79_LOCKDOWN` option alongside the existing default; `mama_word_mint()` +(`mama_forth_words.c`) pops a trailing `restrict?` flag to select it. `MINT_RESTRICTED_ +PERSONALITY` (`capsule_mint.c`, block 4998) is the new identity's own first-boot script: `EXEC` +the lockdown capsule, run `ACL-LOCKDOWN-STD79`, print a welcome line. + +**Two real, self-referential bugs found live, same class, same fix shape** — `ACL-LOCKDOWN- +STD79`'s own walk denies `ACL-STD79-ALLOWED?`/`ACL-ALLOW!`/`ACL-PIN` themselves mid-walk (none +of the four are FORTH-79/83 standard words), so the walk's very next iteration tries to call an +already-denied word on itself — "VM fault — emergency console disabled; halting", reproduced +live twice, once per missing word, before all four were allowlisted together in one protected +block (block 4047) with an explicit comment explaining why they can never be denied. + +Verified live on a disposable test identity: a standard word (`1 2 + .`) still works after +lockdown; a non-standard "superpower" word (`VM-EXEC`) is denied with `ACL: denied 'VM-EXEC'`. + +### XI.2 — Hera→Artemis storage-attach message round-trip — CLOSED, commit `63b8b3b` + +Previously, `sk_repl_idle()` called `blk_subsys_attach_device()` directly, synchronously, +inside Hera's own idle-tick loop, the moment a USB Mass Storage device's BOT setup completed. +Bob's decision, given a choice of designs: "1 - hera keeps polling and artemis does the work", +"2 - the payload is FORTH to be executed by the subscriber if it makes it through the ACLs", +"3 - YES WIREBIND identities" (all identity attaches, not just Zuse's genesis), "hera does +sig-check, hera decides birth" (storage registration moves to Artemis; identity verification +and birth stay Hera's), "wait for ack, safer for identity data" (birth is deferred until +storage-attach is confirmed, not run optimistically underneath it). + +Mechanism: Hera's idle loop now sends `S" HERA-BLK-ATTACH-REQ" S" Artemis" +VM-EXEC` (direct `VM-EXEC`, not her own `MSG-SEND` — she still cannot load +`common:messaging.4th` herself, see `kernel_main.c`'s own long-standing comment on why, unchanged +by any of this). `HERA-BLK-ATTACH-REQ` (`capsules/artemis/init.4th`) runs a new `BLK-ATTACH` +primitive (`block_words.c`, wraps `blk_subsys_attach_device()`), builds a reply string, and +sends it back via Artemis's own real `MSG-SEND` — delivered to Hera by the same `MSG-TICK` pump +that already drives every other cross-VM message. A new `BLK-ATTACH-ACK` word +(`sk_word_blk_attach_ack()`, `repl.c`) receives it and only then runs the previously-synchronous +`capsule_zuse_boot_try_attach()`/`capsule_wirebind_try_attach()` identity-birth calls. + +**Real bug caught live on the very first boot test:** `capsules/artemis/init.4th`'s +`ACK-APPEND-NUM` fed a single cell straight into `<# #S #>`, which expects a **double**-cell +number (`{lo, hi}`) — every other numeric-append helper in this codebase (`doe.4th`, `lib.4th`, +`messaging.4th`) does `0 SWAP <# #S #>` first for exactly this reason. Manifested as `DUP: Stack +underflow` in Artemis on the very first `HERA-BLK-ATTACH-REQ` call. Fixed to match the +established `0 SWAP` convention. + +**Separate collision found during the same testing, fixed generally rather than special-cased:** +Hera's own universal `MSG-TICK` pump (unconditional, every idle tick, into every live VM) hit +"VM-EXEC: ERROR in " every tick, forever, the first time it ever ran against a +FORTH-79/83-locked-down identity (§XI.1) — `MSG-TICK` is not a standard word, so ACL correctly +denies it. Considered and rejected: allowlisting `MSG-TICK` itself (its own call graph — +`MSG-ARENA`/`STADIUM-*`/`CH-*`/`VM-EXEC`, transitively — would have to be allowlisted too, +handing a "standard words only" identity real messaging and cross-VM execution power); a +per-personality "skip the pump" flag (doesn't generalize — any future personality missing +`MSG-TICK`, for any reason, hits the identical failure). Fixed generally: the pump now checks +the *target* VM's own dictionary (`vm_find_word(target, "MSG-TICK", 8)` + `acl_allow`) fresh +every tick, the same source of truth the interpreter's own ACL enforcement already uses, and +skips silently if the word isn't there or isn't allowed — works for any VM, any reason, no +personality-specific plumbing. + +### XI.3 — "Identity attaches without Zuse never completes" — root-caused as two separate bugs, both CLOSED, commits `1a26355` and `1839a2b` + +The reproduction: `make -f Makefile.starkernel ARCH=amd64 clean qemu ZUSEDISK=` (Zuse's own +drive never attached this boot at all), then hotplug any other identity's drive (e.g. +`00-thumb-ident.img`) via QMP once `Heartbeat running.` appears. The same recipe *with* Zuse's +drive attached first (either the Makefile's own default, or hotplugged before the other +identity) always worked, fast. This asymmetry — and only this asymmetry — was the whole +investigation's anchor for a very long time, and it was almost, but not quite, the right frame +(see the correction at the end of this subsection). + +**What did *not* turn out to be the cause**, despite each being investigated at length with +real evidence gathered before being ruled out: TCG/QEMU host-timer-injection starvation from +tight MMIO-polling loops with no yield hint (`arch_relax()`/`PAUSE` was added to +`xhci_wait_bit()`, `xhci_bot_wait_for_idle()`, and `virtio_blk.c`'s `vblk_io()` regardless — +real, defensible hardening, kept, just not the fix); a virtqueue reentrancy hazard in +`vblk_io()` (an `arch_disable_interrupts()`/`arch_enable_interrupts()` critical-section wrap +was tried and made the symptom measurably *worse*, then reverted — a genuinely useful negative +result, since it ruled out "QEMU services virtio-blk independently of guest interrupt state" as +an assumption safe to lean on further here); the migration-check/attach-message interleaving +itself (a defer-while-pending guard was added to `sk_repl_idle()` and is harmless, kept, but +didn't fix the underlying problem either). + +**Bug 1 — a pathological full-device scan, entirely self-inflicted the same day.** +`blk_migration_idle_check()` (`block_subsystem.c`) was generalized earlier in this same session +to walk every attached device slot uniformly instead of hardcoding `first_disk_slot()` +(Artemis's own disk) — a real fix for a real device-agnosticism gap, reasoned about and applied +correctly on its own terms. But its per-slot scan can only early-exit once it finds a devblock +that is *both* "hot" (claimed and worn past `MIGRATION_WEAR_THRESHOLD`) *and* "free" — and a +just-attached, never-claimed USB identity drive can never satisfy the "hot" half by design +(claiming only ever happens via `blk_firsttouch_claim()`, itself hardcoded to +`first_disk_slot()`). So the scan ran to completion — the drive's entire ~16,000 devblocks, +almost all cache misses (`DISK_CACHE_SLOTS` is 8) over slow emulated USB/BOT — every single +idle tick, forever, blocking `sk_repl_idle()` (console, xHCI polling, the storage-attach +message round-trip, everything) each time. A fresh subagent, briefed cold on the bug after the +live-debugging session had run for hours without converging, found this by reading code rather +than more live forensics — the single highest-leverage move in the whole investigation. + +Fix (`block_subsystem.c`): a new `has_ever_claimed` flag on `blk_dev_slot_t`, set in +`blk_set_meta()` (the one choke point every `BLK_FLAG_CLAIMED` transition passes through), +skips the scan entirely, O(1), for any slot nothing has ever claimed. A new +`migration_scan_lbn` resume cursor bounds *any* slot's per-tick cost to +`MIGRATION_SCAN_BUDGET` (256) devblocks examined, picking up where the previous tick left off +instead of restarting from `start_lbn` every time — restores the function's own long-standing +"coarse cadence, cheap early-exit" design intent for every device, not just the one it used to +hardcode. **Verified live:** CPU stayed flat across an 8+ minute run instead of climbing without +bound. + +**Bug 2 — found only after Bug 1 was fixed, because Bug 1's own symptom (unbounded CPU churn) +had been masking clean evidence of it the entire time.** With the scan fixed, the attach +sequence became fast enough to observe cleanly: the full `HERA-BLK-ATTACH-REQ` → `BLK-ATTACH` → +`MSG-SEND` → `MSG-DELIVER` → `BLK-ATTACH-ACK` round-trip (§XI.2) completed correctly and +quickly, every time — the messaging mechanism itself was never broken. What actually happened +next: `capsule_wirebind_try_attach()` (`capsule_wirebind.c`) silently, correctly-by-the-old- +design refused, because it gated on `mama_vm->zuse_cert_installed` — a flag only ever set when +Zuse's own drive attaches and authenticates *this specific boot* +(`capsule_zuse_boot_try_attach()` → `install_and_activate()` → `vm_zuse_cert_install()`). No +Zuse this boot, no cert installed, every other identity's cert verification refuses, silently +(no log line existed on this path at all before this investigation added and then removed a +temporary one to catch it). + +That gate conflated two genuinely different capabilities: minting a *new* identity (needs +Zuse's live *private* seed, a real privileged operation, correctly gated on her live session) +and *verifying* an *existing* identity's cert (needs nothing but her already-*public* key, +which was already being persisted independently of her live session the whole time — +`zuse_genesis_marker_t`, written once at genesis into the kernel's own top-of-device metadata +fence on Artemis's resident storage, deliberately holding only the public key, never the seed, +per that type's own doc comment). Captain Bob, on being asked whether this coupling was +intentional: "zuse is zuse, [each identity] is [itself]" — one identity's attach must not +depend on another identity's live presence. + +Fix: a new `capsule_zuse_boot_load_root_pubkey()` (`capsule_zuse_boot.c`) reads the genesis +marker and populates two new VM fields, `zuse_root_pubkey_known`/`zuse_root_pubkey` (`vm.h`) — +deliberately separate from `zuse_cert_installed`/`zuse_cert_seed`/`zuse_cert_pubkey`, which stay +untouched and still gate `MINT` exactly as before. Called once from `kernel_main.c` as soon as +Artemis's own storage attaches, unconditionally, independent of whether Zuse's own drive is +ever attached this boot. `capsule_wirebind_verify_cert()`/`capsule_wirebind_try_attach()` +(`capsule_wirebind.c`) now check `zuse_root_pubkey_known` instead of `zuse_cert_installed`. + +**Verified live, amd64:** identity 00 attaches and completes WIREBIND in **19 seconds** with +Zuse's own drive never attached this boot at all — previously unbounded (many real minutes, or +effectively never) before Bug 1's fix, then still silently stuck (Bug 2, undiagnosed) even +after it. Zuse's own attach flow re-verified unaffected (regression check, amd64). Three-arch +`clean qemu` acceptance (amd64/aarch64/riscv64) passed with both fixes included. + +**Framing correction, for anyone reading this section to understand the investigation's own +shape, not just its conclusion:** "identity attaches slowly/never when Zuse never attaches +first" was a real, reproducible, correctly-observed symptom, but "Zuse must attach first" was +never actually the load-bearing variable — Artemis's own virtio-blk disk is *always* attached +first, unconditionally, before Hera's idle loop even exists (`kernel_main.c`, ~line 591). The +real variable, in both bugs, was "has *any* second device's storage-attach path ever run +successfully this boot" (Bug 1: does the scan ever get a chance to find something claimed) and +"is Zuse's public key available *at all*, this boot" (Bug 2: entirely independent of ordering, +just of her ever attaching at all before this point). Both bugs happened to correlate with the +"Zuse first" framing without being caused by it — a genuine trap for exactly the kind of +live-forensics-driven investigation this section is a record of, and worth naming plainly +rather than editing out of the historical record.