From b106a4b0b53dc2d42396cc752044e1fb2f242218 Mon Sep 17 00:00:00 2001 From: Robert Allan James Date: Mon, 7 Sep 2026 12:47:05 -0400 Subject: [PATCH] =?UTF-8?q?FABRIC-3.md=20=C2=A7X.4:=20document=20the=20all?= =?UTF-8?q?ocator=20resolution=20=E2=80=94=20sf=5Fmalloc/sf=5Ffree=20now?= =?UTF-8?q?=20on=20the=20real=20kernel=20heap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updates §X.4 (and §X's own title) from "capacity question still OPEN" to resolved: Captain Bob's framing (unknown VM count in advance, heap should use whatever memory is actually available once this is a full OS) led to routing alloc_kernel.c's sf_malloc()/sf_free() through the kernel's existing kmalloc.c heap (2 GiB floor, PMM-backed, coalescing, already boot-tested) instead of a separate 4MB arena -- commit 56e19a0. Verified live: all 9 identities now attach where only 6 did before. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_014Ec88YKxxhZGG1RNnune78 --- FABRIC-3.md | 48 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 35 insertions(+), 13 deletions(-) diff --git a/FABRIC-3.md b/FABRIC-3.md index 0077fe9f..b4f88a8b 100644 --- a/FABRIC-3.md +++ b/FABRIC-3.md @@ -1497,7 +1497,7 @@ attach) remains unexplained and unfixed — tracked as the next item, separate f closures above. ## X. §IX.5 follow-on: xHCI concurrent enumeration, root-caused and CLOSED; identity VM heap -capacity measured, one real leak found and fixed, capacity question still OPEN — 2026-09-06/07 +capacity measured, leak fixed, allocator routed onto the real kernel heap — CLOSED, 2026-09-06/07 Picks up exactly where §IX.5 left off. Two separate root causes, both real, found by actually attaching multiple devices live rather than reasoning about the driver in the abstract — same @@ -1586,7 +1586,7 @@ The 05 hotplug is where X.4 below picks up — its USB/xHCI attach itself succee confirming X.2's fix generalizes to the hotplug path, not just boot-time scan); what failed next was a different layer entirely. -### X.4 — Identity VM kernel-heap capacity: exact numbers measured live, one real leak found and FIXED (commit `d8a195b`), the capacity/fragmentation question itself is OPEN +### X.4 — Identity VM kernel-heap capacity: exact numbers measured live, leak FIXED (commit `d8a195b`), architecture question RESOLVED same day — sf_malloc/sf_free now routed onto the kernel's real heap (commit `56e19a0`) X.3's 05 hotplug attached at the USB layer but then failed at `WIREBIND`'s VM-birth step with a kernel allocator exhaustion error (`vm_create_word: malloc failed`, `vm_init: rolling window @@ -1630,17 +1630,39 @@ Verified live with the same probe, replaying the identical failure: `used_bytes` returns to **exactly** the pre-attempt baseline (3,526,256, matching precisely) instead of leaking 228,576 bytes. Three-arch acceptance passed with the fix included. -**Still open — the actual architecture question, not yet decided (Captain Bob, 2026-09-07):** -"identity is everything in this OS." The leak fix makes retries neutral instead of compounding, -but the underlying ~6-identity practical ceiling (itself a fragmentation artifact, not a hard -capacity number) is unaddressed. Options on the table, none chosen yet: -- Grow the heap / make the arena size dynamic instead of a fixed 4 MiB compile-time constant. -- Shrink per-VM dictionaries — every `WIREBIND` user VM currently carries the full word set, - including things like `TTF-TEXT`/`SCROLL-BACK`/`KBD-SCAN` a non-console user session may not - need. -- Address the fragmentation contributor directly (splitting/coalescing in the free-list, or - serializing VM-birth allocation against concurrent background allocation) — this is *why* - identity 05 failed even with more free bytes than it needed. +**The architecture question — decided and RESOLVED same day, commit `56e19a0`.** Captain Bob's +framing: the number of concurrently-running VMs is not knowable in advance, and once this is a +complete operating system the heap should be able to use whatever memory is actually available, +not a hardcoded compile-time ceiling. Turned out this was already half-built and just not wired +up: `src/starkernel/memory/kmalloc.c` — the kernel's *general* heap, unrelated to +`alloc_kernel.c`'s isolated 4 MiB VM-dictionary arena — was already initialized at boot (M6, +well before any VM is ever born), already reserved from real PMM-tracked physical memory rather +than a fixed array, already defaulted to a 2 GiB floor explicitly commented "for 256+ baby VMs", +already overridable via `--heap=`, and its free list already coalesces on every free. VM +dictionaries (`vm_create_word()`, via `sf_malloc()`/`sf_free()`) were simply never routed +through it. + +Fix: `alloc_kernel.c`'s `sf_malloc()`/`sf_free()` now delegate to `kmalloc_aligned()`/`kfree()` +instead of managing their own separate arena. `sf_alloc_init()` becomes a no-op (kmalloc is +already initialized by the time any VM allocation can happen; "resetting" a heap now shared by +every kernel subsystem would be actively wrong — confirmed no external caller depended on the +old reset semantics). Kernel-only: the hosted StarForth build keeps its own separate +`alloc_host.c`, untouched. + +**Verified live, replaying the exact same hotplug sequence:** all 9 identities (Zuse + 8) now +attach successfully — identity 05, which specifically failed at the old 4 MiB ceiling, now +succeeds. Three-arch clean qemu acceptance (single Zuse device, standard regression) passed on +amd64/aarch64/riscv64 (one aarch64 attempt hit an unrelated, already-documented one-off QEMU +hiccup — empty log, boot never left firmware — and passed cleanly on immediate retry with no +rebuild). + +**Not addressed by this fix, still open:** the free list `kmalloc.c` provides is coalescing but +still first-fit without block *splitting* — a different, smaller-grained fragmentation lever +than the one that was actually hit here. Per-VM dictionary sizing (shrinking what each +`WIREBIND` user VM's word set actually needs — it currently carries the full set, including +things like `TTF-TEXT`/`SCROLL-BACK`/`KBD-SCAN` a non-console user session may not need) remains +a separate, unexplored lever, not required by this fix but still worth revisiting if identity +count grows large enough for it to matter again. **Also flagged, not touched (not asked):** `Makefile.starkernel`'s `printf '$(KERNEL_ARGS)\n' > starforth.cfg` breaks when `KERNEL_ARGS` starts with `--` (dash's