FABRIC-3.md §X.4: document the allocator resolution — sf_malloc/sf_free now on the real kernel heap
Build / build-riscv64-img (push) Waiting to run
Build / build-amd64-iso (push) Waiting to run
Build / build-aarch64-iso (push) Waiting to run

Updates §X.4 (and §X's own title) from "capacity question still OPEN"
to resolved: Captain Bob's framing (unknown VM count in advance, heap
should use whatever memory is actually available once this is a full
OS) led to routing alloc_kernel.c's sf_malloc()/sf_free() through the
kernel's existing kmalloc.c heap (2 GiB floor, PMM-backed, coalescing,
already boot-tested) instead of a separate 4MB arena -- commit 56e19a0.
Verified live: all 9 identities now attach where only 6 did before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Ec88YKxxhZGG1RNnune78
This commit is contained in:
Robert Allan James
2026-09-07 12:47:05 -04:00
co-authored by Claude Sonnet 5
parent 56e19a00f9
commit b106a4b0b5
+35 -13
View File
@@ -1497,7 +1497,7 @@ attach) remains unexplained and unfixed — tracked as the next item, separate f
closures above.
## X. §IX.5 follow-on: xHCI concurrent enumeration, root-caused and CLOSED; identity VM heap
capacity measured, one real leak found and fixed, capacity question still OPEN — 2026-09-06/07
capacity measured, leak fixed, allocator routed onto the real kernel heap — CLOSED, 2026-09-06/07
Picks up exactly where §IX.5 left off. Two separate root causes, both real, found by actually
attaching multiple devices live rather than reasoning about the driver in the abstract — same
@@ -1586,7 +1586,7 @@ The 05 hotplug is where X.4 below picks up — its USB/xHCI attach itself succee
confirming X.2's fix generalizes to the hotplug path, not just boot-time scan); what failed next
was a different layer entirely.
### X.4 — Identity VM kernel-heap capacity: exact numbers measured live, one real leak found and FIXED (commit `d8a195b`), the capacity/fragmentation question itself is OPEN
### X.4 — Identity VM kernel-heap capacity: exact numbers measured live, leak FIXED (commit `d8a195b`), architecture question RESOLVED same day — sf_malloc/sf_free now routed onto the kernel's real heap (commit `56e19a0`)
X.3's 05 hotplug attached at the USB layer but then failed at `WIREBIND`'s VM-birth step with a
kernel allocator exhaustion error (`vm_create_word: malloc failed`, `vm_init: rolling window
@@ -1630,17 +1630,39 @@ Verified live with the same probe, replaying the identical failure: `used_bytes`
returns to **exactly** the pre-attempt baseline (3,526,256, matching precisely) instead of
leaking 228,576 bytes. Three-arch acceptance passed with the fix included.
**Still open — the actual architecture question, not yet decided (Captain Bob, 2026-09-07):**
"identity is everything in this OS." The leak fix makes retries neutral instead of compounding,
but the underlying ~6-identity practical ceiling (itself a fragmentation artifact, not a hard
capacity number) is unaddressed. Options on the table, none chosen yet:
- Grow the heap / make the arena size dynamic instead of a fixed 4 MiB compile-time constant.
- Shrink per-VM dictionaries — every `WIREBIND` user VM currently carries the full word set,
including things like `TTF-TEXT`/`SCROLL-BACK`/`KBD-SCAN` a non-console user session may not
need.
- Address the fragmentation contributor directly (splitting/coalescing in the free-list, or
serializing VM-birth allocation against concurrent background allocation) — this is *why*
identity 05 failed even with more free bytes than it needed.
**The architecture question — decided and RESOLVED same day, commit `56e19a0`.** Captain Bob's
framing: the number of concurrently-running VMs is not knowable in advance, and once this is a
complete operating system the heap should be able to use whatever memory is actually available,
not a hardcoded compile-time ceiling. Turned out this was already half-built and just not wired
up: `src/starkernel/memory/kmalloc.c` — the kernel's *general* heap, unrelated to
`alloc_kernel.c`'s isolated 4 MiB VM-dictionary arena — was already initialized at boot (M6,
well before any VM is ever born), already reserved from real PMM-tracked physical memory rather
than a fixed array, already defaulted to a 2 GiB floor explicitly commented "for 256+ baby VMs",
already overridable via `--heap=`, and its free list already coalesces on every free. VM
dictionaries (`vm_create_word()`, via `sf_malloc()`/`sf_free()`) were simply never routed
through it.
Fix: `alloc_kernel.c`'s `sf_malloc()`/`sf_free()` now delegate to `kmalloc_aligned()`/`kfree()`
instead of managing their own separate arena. `sf_alloc_init()` becomes a no-op (kmalloc is
already initialized by the time any VM allocation can happen; "resetting" a heap now shared by
every kernel subsystem would be actively wrong — confirmed no external caller depended on the
old reset semantics). Kernel-only: the hosted StarForth build keeps its own separate
`alloc_host.c`, untouched.
**Verified live, replaying the exact same hotplug sequence:** all 9 identities (Zuse + 8) now
attach successfully — identity 05, which specifically failed at the old 4 MiB ceiling, now
succeeds. Three-arch clean qemu acceptance (single Zuse device, standard regression) passed on
amd64/aarch64/riscv64 (one aarch64 attempt hit an unrelated, already-documented one-off QEMU
hiccup — empty log, boot never left firmware — and passed cleanly on immediate retry with no
rebuild).
**Not addressed by this fix, still open:** the free list `kmalloc.c` provides is coalescing but
still first-fit without block *splitting* — a different, smaller-grained fragmentation lever
than the one that was actually hit here. Per-VM dictionary sizing (shrinking what each
`WIREBIND` user VM's word set actually needs — it currently carries the full set, including
things like `TTF-TEXT`/`SCROLL-BACK`/`KBD-SCAN` a non-console user session may not need) remains
a separate, unexplored lever, not required by this fix but still worth revisiting if identity
count grows large enough for it to matter again.
**Also flagged, not touched (not asked):** `Makefile.starkernel`'s
`printf '$(KERNEL_ARGS)\n' > starforth.cfg` breaks when `KERNEL_ARGS` starts with `--` (dash's