admin/LithosAnanake
1
0
Fork 0
Code Issues Pull Requests Actions 9 Packages Projects Releases Wiki Activity

Fix aarch64 Stadium/COOL O(ncells) scan; rerun std79 DoE clean, 81/81 (FABRIC-3.md §XVI)
Build / build-amd64-iso (push) Waiting to run
Details
Build / build-aarch64-iso (push) Waiting to run
Details
Build / build-riscv64-img (push) Waiting to run
Details

Browse Source 
Root cause of the 90+ minute aarch64 VM-birth stall found in §XV: stadium_admit()'s
eviction-fallback scan iterated the entire stadium_ncells array filtered by owner,
not the calling VM's own resident cells as its own doc comment claimed. Combined
with stadium_grant_quota() always splitting from Hera's shrinking free list and
stadium_word_dispatch() calling stadium_admit() per distinct word a VM's capsule
executes, this compounded into a real O(n) blowup — catastrophic specifically on
aarch64 because its -m 4096 (vs 1024 on amd64/riscv64) inflates the kmalloc heap
kmalloc_init() bisects down to, which inflates stadium_ncells 4x (335,544 vs 83,886
cells, measured from boot logs).

Fixed by threading a real per-VM doubly-linked resident-cell list
(StadiumVMQuota.resident_head + stadium_resident_next[]/stadium_resident_prev[])
so the fallback scan is bounded by that VM's own resident count, not the global
cell array size.

Verified with a full rerun of the 3x9x3 std79 DoE campaign from scratch: one
continuous boot per architecture, all 9 identities simultaneously live throughout
(the 3-boot aarch64/riscv64 batching workaround is no longer needed). 81/81 trials
correct, 0 mismatches, DOE-RUN header sequence md5-identical across all three raw
logs. Identity 04's attach on aarch64, which stalled 90+ minutes before, now
completes in ~34s; full boot-to-DoE-complete in ~290s.

Corrects an earlier misreading (carried into §XV, std79-doe.fth's comments, and
the project memory note) that described the symptom as a runaway "335,000+ cycles"
dispatch counter — those were cell array indices, not an event count.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo
This commit is contained in:
Robert Allan James
2026-09-11 16:37:44 -04:00
co-authored by Claude Sonnet 5
parent e2abc56306
commit 9eff122090
23 changed files with 94109 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/starkernel/vm/stadium.c
+100 -17
View File
@@ -80,6 +80,17 @@ typedef struct {
VMUuid vm_id;
int in_use;
size_t free_head;
size_t resident_head; /* FABRIC-3.md aarch64 Stadium/COOL scaling fix:
* head of this VM's own resident-cell list, threaded
* through stadium_resident_next[]/stadium_resident_prev[]
* (global, indexed by cell, below). Lets
* stadium_admit()'s eviction-fallback scan walk only
* this VM's own residents -- what its doc already
* claimed ("scoped to that SAME VM's own resident
* patrons only") -- instead of filtering the entire
* stadium_ncells array by owner, which is what the
* scan actually did before this fix. STADIUM_CELL_NONE
* when empty. */
uint64_t reservoir; /* item 4.1, FABRIC-0.md §17.7 -- Q48.16, heat this VM's
* quota holds but no resident patron has claimed.
* Invariant: Σ(resident patron heat) + reservoir ==
@@ -92,6 +103,45 @@ typedef struct {
* (2026-08-15: the bound is computed from RAM, not a compile-time constant). */
static StadiumVMQuota *stadium_quotas = (StadiumVMQuota *)0;
/* Doubly-linked resident-list threading, one entry per cell, parallel to
* stadium_owner[] -- kmalloc'd at stadium_boot_init() to stadium_ncells
* entries each. Only meaningful for cells currently resident (bitmap bit
* set); STADIUM_LINK_NONE-equivalent (via link_to_size()/size_to_link(),
* defined below) terminates each end. Doubly-linked because stadium_evict()
* removes by cell_index directly (not by list-walk), so O(1) unlink needs a
* prev pointer, not just next. */
static uint32_t *stadium_resident_next = (uint32_t *)0;
static uint32_t *stadium_resident_prev = (uint32_t *)0;
/* resident_list_push - O(1) insert cell_index at the front of slot's own
* resident list. Caller must have already set bitmap/owner for cell_index. */
static void resident_list_push(int slot, size_t cell_index) {
size_t old_head = stadium_quotas[slot].resident_head;
stadium_resident_prev[cell_index] = size_to_link(STADIUM_CELL_NONE);
stadium_resident_next[cell_index] = size_to_link(old_head);
if (old_head != STADIUM_CELL_NONE) {
stadium_resident_prev[old_head] = size_to_link(cell_index);
}
stadium_quotas[slot].resident_head = cell_index;
}
/* resident_list_remove - O(1) splice cell_index out of slot's own resident
* list. Caller must call this BEFORE clearing cell_index's bitmap bit. */
static void resident_list_remove(int slot, size_t cell_index) {
size_t prev = link_to_size(stadium_resident_prev[cell_index]);
size_t next = link_to_size(stadium_resident_next[cell_index]);
if (prev != STADIUM_CELL_NONE) {
stadium_resident_next[prev] = size_to_link(next);
} else {
stadium_quotas[slot].resident_head = next;
}
if (next != STADIUM_CELL_NONE) {
stadium_resident_prev[next] = size_to_link(prev);
}
}
/* Returns the quota slot index for vm_id, or -1 if none is granted. */
static int quota_slot_for_vm(VMUuid vm_id) {
size_t i;
@@ -137,11 +187,15 @@ int stadium_boot_init(void) {
StadiumCell *cells = (StadiumCell *)kmalloc(ncells * STADIUM_CELL_BYTES);
uint8_t *bitmap = (uint8_t *)kmalloc(bitmap_bytes);
uint16_t *owner = (uint16_t *)kmalloc(ncells * sizeof(uint16_t));
if (!cells || !bitmap || !owner) {
uint32_t *resident_next = (uint32_t *)kmalloc(ncells * sizeof(uint32_t));
uint32_t *resident_prev = (uint32_t *)kmalloc(ncells * sizeof(uint32_t));
if (!cells || !bitmap || !owner || !resident_next || !resident_prev) {
console_println("Stadium: kmalloc failed for boot-time allocation");
if (cells) kfree(cells);
if (bitmap) kfree(bitmap);
if (owner) kfree(owner);
if (cells) kfree(cells);
if (bitmap) kfree(bitmap);
if (owner) kfree(owner);
if (resident_next) kfree(resident_next);
if (resident_prev) kfree(resident_prev);
return -1;
}
@@ -168,6 +222,8 @@ int stadium_boot_init(void) {
kfree(cells);
kfree(bitmap);
kfree(owner);
kfree(resident_next);
kfree(resident_prev);
return -1;
}
@@ -193,15 +249,20 @@ int stadium_boot_init(void) {
cells[i].header.link = size_to_link((i + 1 < ncells) ? (i + 1) : STADIUM_CELL_NONE);
cells[i].header.contains = STADIUM_CONTAINS_NONE;
owner[i] = 0;
/* Not on any resident list yet -- nothing is resident until
* stadium_admit() puts it there. */
resident_next[i] = size_to_link(STADIUM_CELL_NONE);
resident_prev[i] = size_to_link(STADIUM_CELL_NONE);
}
}
{
size_t i;
for (i = 0; i < max_vm_count; i++) {
quotas[i].vm_id = vm_uuid_none();
quotas[i].in_use = 0;
quotas[i].free_head = STADIUM_CELL_NONE;
quotas[i].reservoir = 0;
quotas[i].vm_id = vm_uuid_none();
quotas[i].in_use = 0;
quotas[i].free_head = STADIUM_CELL_NONE;
quotas[i].resident_head = STADIUM_CELL_NONE;
quotas[i].reservoir = 0;
}
}
quotas[0].vm_id = vm_uuid_hera();
@@ -215,6 +276,8 @@ int stadium_boot_init(void) {
stadium_cell_array = cells;
stadium_bitmap = bitmap;
stadium_owner = owner;
stadium_resident_next = resident_next;
stadium_resident_prev = resident_prev;
stadium_ncells = ncells;
stadium_quotas = quotas;
stadium_max_vm_count_val = max_vm_count;
@@ -378,8 +441,9 @@ int stadium_evict(size_t cell_index) {
if (header->contains != STADIUM_CONTAINS_NONE) return -1;
stadium_dispatch(cell_index, (StadiumBehaviour)header->behaviour);
bitmap_clear(cell_index);
slot = stadium_owner[cell_index];
resident_list_remove((int)slot, cell_index);
bitmap_clear(cell_index);
/* item 4.1, §17.7: the departing patron's remaining heat must flow back
* to its owner's reservoir before the cell returns to the free list, or
@@ -417,7 +481,6 @@ int stadium_evict(size_t cell_index) {
*/
size_t stadium_admit(VMUuid vm_id, const StadiumPatronHeader *candidate) {
int slot;
size_t i;
size_t idx;
size_t least_dense_index = STADIUM_CELL_NONE;
uint64_t least_dense_value = 0;
@@ -450,22 +513,41 @@ size_t stadium_admit(VMUuid vm_id, const StadiumPatronHeader *candidate) {
* reservoir credit and free-list return land on the VM that actually
* admitted this patron, not whatever owner[idx] held at boot. */
stadium_owner[idx] = (uint16_t)slot;
resident_list_push(slot, idx);
return idx;
}
for (i = 0; i < stadium_ncells; i++) {
StadiumPatronHeader *h;
/* FABRIC-3.md aarch64 Stadium/COOL scaling fix: this used to be a
* for (i = 0; i < stadium_ncells; i++) scan of the ENTIRE global array,
* filtered down to this VM's own owned cells by an `owner[i] != slot`
* check inside the loop -- O(stadium_ncells) on every call that reaches
* here, not O(this VM's own resident count) as the function's own doc
* above already claimed ("scoped to that SAME VM's own residents").
* Root-caused live (2026-09-11/12): word-execution admission
* (stadium_word_dispatch(), stadium_words.c) calls stadium_admit() once
* per distinct word a VM's own capsule executes, and each VM's granted
* quota is a small, halving fraction of Hera's original pool -- once
* that quota's free list empties (a handful of words in), EVERY further
* new word this VM ever executes hit this scan. At stadium_ncells in
* the hundreds of thousands (aarch64's -m 4096 vs. amd64/riscv64's
* -m 1024 --> a 4x larger kmalloc heap --> a 4x larger Stadium, see
* FABRIC-3.md SXV/SXVI), repeated hundred-thousand-cell scans compounded
* into the observed 90+ minute stall. Walking this VM's own
* resident_head list instead bounds the cost by how many cells this VM
* actually owns (<= its granted quota, itself <= a small constant
* multiple of DICTIONARY_SIZE in practice), independent of
* stadium_ncells. */
for (idx = stadium_quotas[slot].resident_head; idx != STADIUM_CELL_NONE;
idx = link_to_size(stadium_resident_next[idx])) {
StadiumPatronHeader *h = &stadium_cell_array[idx].header;
if (!bitmap_get(i)) continue;
if (stadium_owner[i] != (uint16_t)slot) continue;
h = &stadium_cell_array[i].header;
if (h->flags & STADIUM_FLAG_PIN) continue;
if (h->contains != STADIUM_CONTAINS_NONE) continue;
{
uint64_t d = stadium_density(i);
uint64_t d = stadium_density(idx);
if (least_dense_index == STADIUM_CELL_NONE || d < least_dense_value) {
least_dense_index = i;
least_dense_index = idx;
least_dense_value = d;
}
}
@@ -492,6 +574,7 @@ size_t stadium_admit(VMUuid vm_id, const StadiumPatronHeader *candidate) {
* currently a no-op in practice, but it must not be assumed to stay a
* no-op: this is the correctness statement, not a redundant write. */
stadium_owner[idx] = (uint16_t)slot;
resident_list_push(slot, idx);
return idx;
}