Fix real WIREBIND crash: stale active-VM pointer dispatched after blocking read (FABRIC-3.md §XII.3)
Build / build-amd64-iso (push) Waiting to run
Build / build-aarch64-iso (push) Waiting to run
Build / build-riscv64-img (push) Waiting to run

The interpreter_enabled guard added in the previous commit (662ef44) was a
real but incomplete fix -- re-running the exact repro against it still
panicked (this time as a raw #PF page fault), proving something deeper
was wrong.

Root cause, found via targeted console_puts probes (not GDB --
starkernel_kernel.elf's symbols don't correspond to the actual running
starkernel_loader.efi binary for this monolithic build, same gotcha
already on record from the 2026-08-18 aarch64 investigation):

sk_repl_run()'s main loop captures `active` once, before calling
sk_console_readline(), which then blocks for the next full line. If the
identity `active` points at is killed while that read is still blocked,
the bailout meant to catch this (sk_console_identity_present()) only
checks a generic "is anyone attached" boolean, not "is the specific
identity active belonged to still attached" -- a fast detach of one
identity followed by attach of a different one never produces an
observable gap in that boolean, so the bailout never fires. The stale
`active`, now pointing at freed memory, gets dispatched into.

Fix: re-resolve `active` fresh from g_repl_active_vm immediately before
dispatch, right after sk_console_readline() returns. One line, no
registry lookup, no dereference of the stale pointer -- closes the race
regardless of whether the bailout catches it first.

Verified: rebuilt amd64 clean, reproduced the exact same attach/USE/
detach/attach/USE sequence against the fixed build -- clean switch, no
fault, exerciser runs correctly afterward.

FABRIC-3.md §XII.2 also corrected to stop claiming the interpreter_
enabled guard alone closed the crash -- it didn't, per the above.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo
This commit is contained in:
Robert Allan James
2026-09-10 17:19:01 -04:00
co-authored by Claude Sonnet 5
parent 662ef44e59
commit 70421bdd43
10 changed files with 45475 additions and 22 deletions
+57 -21
View File
@@ -2070,37 +2070,73 @@ its own caller, not something `capsule_exec_init()` did on every caller's behalf
`capsule_birth_mama()` calls `vm_exec_fn` directly (not `capsule_exec_init()`), so this fix has no
effect on Mama's own boot path.
### XII.2 — `USE` into a WIREBIND identity before its interpreter is enabled panics the whole
machine — CLOSED
### XII.2 — `USE` into a WIREBIND identity before its interpreter is enabled — real guard added,
but NOT what was actually crashing the machine (see §XII.3) — CLOSED as its own narrow fix
Found live: attaching identity `03` via QMP hotplug, then immediately sending
`S" 03" USE` the moment the log printed `WIREBIND: 03 attached and ready -- USE it to begin`,
produced not an ACL fault but a full kernel panic --
`[StarKernel HAL] PANIC: interpreter invoked before bootstrap completion` -- and the machine
halted completely (`System halted.`, QEMU process left spinning uselessly). Root cause: a WIREBIND
identity's VM starts with `interpreter_enabled = 0` (`vm_bootstrap.c:162`) and only gets
`vm_enable_interpreter()` called as a later, separate step of the same birth sequence. The
"attached and ready" message prints as soon as the VM is registered, before that later step is
guaranteed to have run. `mama_word_use()` (`USE`) checked the target for dead/stillborn state but
never checked interpreter readiness before redirecting the whole console's input to it -- the very
next line typed would hit `vm_assert_interpreter_enabled()` inside `vm_interpret()` and call
`host->panic()`, which is a hard, unrecoverable halt, not the per-session-recoverable ACL-fault
path a redirected VM otherwise gets (§XI.3's fix). This is a real race independent of automation
speed: a human typing `USE` right after seeing "ready" hits it exactly the same way.
halted completely (`System halted.`, QEMU process left spinning uselessly).
**Fix:** `mama_word_use()` (`mama_forth_words.c`) now checks `((VM *)entry.vm_ptr)->interpreter_
**First-pass diagnosis (correct as far as it went, but incomplete):** a WIREBIND identity's VM
starts with `interpreter_enabled = 0` (`vm_bootstrap.c:162`), and `mama_word_use()` (`USE`)
checked the target for dead/stillborn state but never checked interpreter readiness before
redirecting. Added a guard: `mama_word_use()` now checks `((VM *)entry.vm_ptr)->interpreter_
enabled` before redirecting, refusing with `USE: <name> not ready yet -- still bootstrapping, try
again` instead of performing the redirect. The caller just retries a moment later. Did not touch
the birth sequence itself (i.e., did not try to make the "attached and ready" message wait for
`vm_enable_interpreter()` to actually run first) -- narrower, lower-risk fix at the point of actual
failure.
again` instead of performing the redirect. This guard is real, correct, and worth keeping -- it
closes a genuine (if narrow) gap in `USE`'s own target-readiness check.
**Both fixes verified:** amd64 rebuild clean, zero new warnings.
**But re-running the identical repro against the fixed build panicked again** -- same message,
different symptom the second time (a raw `#PF` page fault at `RIP=CR2=0xa0000`, not a clean
`panic()` call) -- proving the guard above was not the actual root cause of what the campaign was
hitting. See §XII.3 for the real root cause, found via targeted live debug probes (not GDB --
`starkernel_kernel.elf` is not the running binary for this monolithic build, its symbols do not
correspond to `starkernel_loader.efi`'s runtime addresses, the same gotcha already recorded from
the 2026-08-18 aarch64 investigation; `console_puts`-based probes compiled into the actual running
image instead).
**Status:** exerciser campaign resuming with the fixed build. The 24-case exerciser itself (see
### XII.3 — Real root cause: `active` VM pointer captured before a blocking read, dispatched after
the VM it points at was freed — CLOSED
**Reproduced deterministically:** attach `00`, `USE` into it (confirmed live), detach `00` (kills
both its console and user VMs, `sf_free()`s the structs), immediately attach `01`, then send
`S" 01" USE` -- panics or page-faults depending on exactly what memory got reused, every time.
**Root cause, confirmed via three targeted `console_puts` probes (added, used, reverted -- per
standing practice):** `sk_repl_run()`'s main loop captures `active = g_repl_active_vm ? ... : vm`
**once**, at the top of each iteration, *before* calling `sk_console_readline()` -- which then
blocks, potentially for a long time, waiting for a full line of input. If the identity `active`
pointed at is killed while that read is still blocked (`n == 0`, nothing typed yet), the intended
safety net -- `sk_console_readline()`'s own `n < 0` bailout -- checks `sk_console_identity_
present()`, a **generic "is anyone currently attached" boolean**, not "is the specific identity
`active` belonged to still attached." A fast detach-of-one-identity-then-attach-of-a-different-one
while the read is blocked never produces an observable gap in that boolean -- presence reads true
continuously (`00` present, gap, `01` present, polled too coarsely or too late to ever see the
gap) -- so the bailout never fires. The stale `active`, now pointing at freed memory, gets passed
straight to `sk_repl_dispatch_line()`. Confirmed live: a probe at the top of the loop showed
`g_repl_active_vm` correctly reset to `NULL` the instant `00`'s teardown ran (the kill-path fix
from FABRIC-2.md §F.10, still correct), but the *local* `active` in the loop iteration that was
already blocked in `sk_console_readline()` never re-read it -- it dispatched the stale snapshot
once the next line finally arrived, regardless of what had happened to the global in the meantime.
**Fix:** `sk_repl_run()` (`repl.c`) now re-resolves `active` fresh from `g_repl_active_vm`
immediately before `sk_repl_dispatch_line()`, right after `sk_console_readline()` returns --
one extra line, no registry lookup, no risk of touching the freed struct at all (it's a plain
global re-read, not a dereference of the stale pointer). This closes the race regardless of
whether the `n < 0` bailout catches it first; that bailout's own generic-boolean weakness is
untouched here and could still misfire for other purposes, but nothing downstream of it can crash
the machine anymore -- worth its own follow-up if it comes up again.
**Verified:** rebuilt amd64 clean (zero new warnings), reproduced the exact same attach/USE/detach/
attach/USE sequence against the fixed build -- `USE: now using 01` succeeds cleanly, no fault, no
panic, the 24-case exerciser runs correctly against `01` afterward with identical output to `00`
and `zuse`.
**Status:** exerciser campaign resuming with both fixes in place. The 24-case exerciser itself (see
scratchpad `std79-exerciser.fth`, not yet committed -- FORTH-79 standard words only, biased toward
mixed/double-precision arithmetic and shift/negate boundary cases per cross-ISA risk) had already
produced one interesting, unconfirmed data point before the panic: `-123 456 M*` printed
`DOUBLE-OVERFLOW` via `D.` on amd64, identically for both `zuse` and `rajames` -- worth checking
produced one interesting, unconfirmed data point before the first panic: `-123 456 M*` printed
`DOUBLE-OVERFLOW` via `D.` on amd64, identically for `zuse`, `rajames`, and `00` -- worth checking
against the other two architectures once the campaign completes; not yet root-caused or reported
as a bug on its own.
+1 -1
View File
@@ -1,5 +1,5 @@
# Capsule Block Manifest — Auto-generated
<!-- Generated by mkcapsule --manifest 2026-09-10T20:06:42Z -->
<!-- Generated by mkcapsule --manifest 2026-09-10T21:16:32Z -->
<!-- DO NOT EDIT — re-run mkcapsule --manifest to refresh. -->
<!-- Hand-written justifications and immutability notes live -->
<!-- in MANIFEST.md alongside this auto-generated index. -->
BIN
View File
Binary file not shown.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+19
View File
@@ -1251,6 +1251,25 @@ void sk_repl_run(VM *vm)
continue;
}
/* Found live 2026-09-10: `active` was captured once, above, before
* sk_console_readline() blocked for this line -- but that call can
* block for an arbitrarily long time, during which the VM `active`
* points at can be killed (WIREBIND detach) and its memory freed.
* The n<0 bailout above is supposed to catch a logout mid-read, but
* it only fires on sk_console_identity_present() -- a generic "is
* ANYONE attached" boolean, not "is the specific identity `active`
* belonged to still attached" -- so a fast detach-then-reattach of
* a *different* identity while this call was blocked (n still 0)
* never trips it: presence reads true throughout, no gap is ever
* observed. The stale `active` then gets dispatched into freed
* memory. Confirmed live via targeted probes: g_repl_active_vm is
* correctly reset to NULL by the kill/teardown path the moment it
* happens, but this loop iteration's *local* `active` was already
* snapshotted and never re-read. Re-resolve fresh from the global
* right before dispatch -- cheap, and closes the race regardless
* of whether the bailout above catches it first. */
active = g_repl_active_vm ? g_repl_active_vm : vm;
sk_repl_dispatch_line(active, input);
/* ABORT stops mid-line but leaves the flag set for the caller to