Fix stadium_grant_quota() donor floor; rerun std79 DoE clean, 81/81 (FABRIC-3.md §XVII)
Build / build-amd64-iso (push) Waiting to run
Build / build-aarch64-iso (push) Waiting to run
Build / build-riscv64-img (push) Waiting to run

capsule_birth.c hardcoded every new VM's initial Stadium quota grant to split
from Hera specifically. Since a grant always halves whatever the donor
currently has, Hera's own free list converges toward empty after a bounded
number of grants — independent of whether the Stadium as a whole still had
spare capacity, since VMs she'd granted to earlier typically still held
nearly all of their own share untouched. Past that point every subsequent
VM birth's Stadium grant would be silently refused (soft-failed, non-fatal
by existing design), even with plenty of capacity sitting idle elsewhere.

Fixed by adding an O(1)-maintained free_count to StadiumVMQuota (incremented
in stadium_evict(), decremented at both of stadium_admit()'s free-list-pop
sites, set/adjusted in stadium_grant_quota()'s own split — this also let
grant_quota drop its old O(free-list length) counting walk in favor of an
O(1) read) and stadium_best_donor(), an O(live VM count) scan over quota
slots returning whichever in-use VM currently has the most free cells.
capsule_birth.c's birth path now splits from that VM instead of
unconditionally vm_uuid_hera().

Verified with another full rerun of the 3x9x3 std79 DoE campaign from
scratch — same discipline as the prior Stadium fix (any defect repair
reruns the whole DoE from the top) — one continuous boot per architecture,
all 9 identities simultaneously live throughout. 81/81 trials correct, 0
mismatches, DOE-RUN header sequence md5-identical to every prior run.
aarch64 ~280s total (vs ~290s for the O(ncells)-scan fix alone — confirms
no regression). Both known Stadium defects are now closed together on one
clean campaign rerun.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo
This commit is contained in:
Robert Allan James
2026-09-11 17:20:07 -04:00
co-authored by Claude Sonnet 5
parent 9eff122090
commit e51a8d229e
16 changed files with 57128 additions and 39 deletions
+45 -6
View File
@@ -2610,10 +2610,49 @@ exist in the whole stalled log) carried into that section's live write-up and in
`experiments/std79-doe/std79-doe.fth`'s own comment block and the
`project_std79_doe_complete.md` memory note; all three have been corrected to point here.
**Latent secondary issue, recorded not fixed (out of scope for this pass):** the free-list
halving in `stadium_grant_quota()` has no floor -- after enough successive VM births Hera's own
free list drops below 2 cells and every subsequent grant is silently refused (`half == 0` -> -1),
which `capsule_birth.c` currently treats as non-fatal. Same root mechanism as the fix above, a
different (silent-failure-to-birth, not stall) symptom, not yet a problem at any VM count actually
exercised so far.
**Latent secondary issue, recorded not fixed at the time (out of scope for that pass) -- now also
fixed, see §XVII below:** the free-list halving in `stadium_grant_quota()` has no floor -- after
enough successive VM births Hera's own free list drops below 2 cells and every subsequent grant
is silently refused (`half == 0` -> -1), which `capsule_birth.c` currently treats as non-fatal.
Same root mechanism as the fix above, a different (silent-failure-to-birth, not stall) symptom.
## XVII. stadium_grant_quota() donor-floor fix (2026-09-11/12)
§XVI's fix closed the O(stadium_ncells) scan; this follow-up closes the second, related defect
flagged there but not fixed at the time: `stadium_grant_quota()`'s split always came from Hera
specifically (`capsule_birth.c` hardcoded `stadium_grant_quota(vm_id, vm_uuid_hera())`), and
because the split halves whatever the donor currently has left, Hera's own free list converges
toward empty after a bounded number of grants -- independent of whether the Stadium as a whole
still has plenty of spare capacity, since VMs she granted to earlier typically still hold nearly
all of their own share untouched. Past that point every subsequent VM birth would have its
Stadium grant silently refused (soft-failed, non-fatal per the existing design), even though
capacity existed elsewhere in the system.
**Fix:** added `free_count` to `StadiumVMQuota` (`src/starkernel/vm/stadium.c`), an O(1)-maintained
length of each VM's own free list -- incremented in `stadium_evict()` when a cell returns to a
quota's free list, decremented at both of `stadium_admit()`'s free-list-pop sites, set/adjusted in
`stadium_grant_quota()`'s own split. This also let `stadium_grant_quota()` drop its old
O(from_vm_id's free-list length) counting walk in favor of an O(1) read.
Added `stadium_best_donor()` (`stadium.c`/`stadium.h`): an O(stadium_max_vm_count()) scan over
quota *slots* (bounded by live VM population, not cell count) returning whichever in-use VM
currently holds the most free cells. `capsule_birth.c`'s birth path now calls this instead of
hardcoding `vm_uuid_hera()`, falling back to Hera only if no VM holds a quota yet (should not
happen in practice, since `stadium_birth_hera()` always runs first).
**Verification:** another full rerun of the entire 3x9x3 std79 DoE campaign from scratch --
same discipline as §XVI ("any defect repair requires rerunning the DoE from the top") -- one
continuous boot per architecture, all 9 identities simultaneously live throughout:
`experiments/std79-doe/results-20260911-donor-floor-fix/`.
- amd64: ~161s total.
- aarch64: ~280s total (consistent with §XVI's ~290s -- confirms this fix didn't regress the
O(ncells) scan fix's performance).
- riscv64: ~162s total.
All three logs' `DOE-RUN,run_id,id_idx,id_label,rep` header sequences are md5-identical to each
other and to §XVI's run, and the canonical 28-number per-trial result string appears exactly 27
times in each log. **81/81 trials correct, 0 mismatches** -- both known Stadium defects (§XVI's
O(ncells) scan and this section's donor floor) are now closed, verified together on one clean
campaign rerun.
+1 -1
View File
@@ -1,5 +1,5 @@
# Capsule Block Manifest — Auto-generated
<!-- Generated by mkcapsule --manifest 2026-09-11T20:32:11Z -->
<!-- Generated by mkcapsule --manifest 2026-09-11T21:16:00Z -->
<!-- DO NOT EDIT — re-run mkcapsule --manifest to refresh. -->
<!-- Hand-written justifications and immutability notes live -->
<!-- in MANIFEST.md alongside this auto-generated index. -->
BIN
View File
Binary file not shown.
+19 -12
View File
@@ -4,10 +4,12 @@ The formal successor to `experiments/std79-exerciser/`'s ad hoc campaign (see FA
requested as a genuine randomized full-factorial design matching this project's own DoE
methodology (`capsules/doe.4th`'s Fisher-Yates run-matrix shuffle) rather than convenience
batching — and written entirely in FORTH, not host-orchestrated shell scripting. See
FABRIC-3.md §XV for the design writeup and §XVI for the real aarch64 Stadium/COOL scaling bug
that was found, root-caused, and fixed (`src/starkernel/vm/stadium.c`, commit pending) — the
3-boot batching workaround below is now historical only; a single boot with all 9 identities
simultaneously live works cleanly on all three architectures as of the fix.
FABRIC-3.md §XV for the design writeup, §XVI for the real aarch64 Stadium/COOL scaling bug that
was found, root-caused, and fixed (`src/starkernel/vm/stadium.c`), and §XVII for a second,
related Stadium bug (`stadium_grant_quota()`'s donor-floor) found while writing up §XVI and
fixed in a follow-up pass — the 3-boot batching workaround below is now historical only; a
single boot with all 9 identities simultaneously live works cleanly on all three architectures
as of both fixes.
`std79-doe.fth` is not a capsule loaded via `EXEC` — feed it as raw text to a running REPL
(e.g. `socat - UNIX-CONNECT:<serial_sock> < std79-doe.fth`), same as the original exerciser, then
@@ -25,11 +27,16 @@ boots if ever needed for an unrelated reason. Every boot must use the *same* see
`{aarch64,riscv64}-doe-batch{1,2,3}-raw.log` (27 trials each, split across 3 boots of 3
identities, the then-necessary workaround). **Result: 81/81 trials correct, 0 mismatches.**
`results-20260911-stadium-fix/` holds the *post-fix* rerun — one boot per architecture, all 9
identities simultaneously live in every boot, same seed throughout. **Result: 81/81 trials
correct, 0 mismatches, byte-identical output to the original campaign** — and the DOE-RUN
header sequence (run_id/id_idx/id_label/rep assignment) is md5-identical across all three raw
logs, confirming the master shuffle is genuinely architecture-independent. Total per-architecture
wall-clock (boot + all 9 identity attaches + full 27-trial run, one continuous QEMU session):
amd64 ~165s, aarch64 ~290s, riscv64 ~161s — aarch64's identity 04 attach alone, which stalled
90+ minutes before the fix, now completes in ~34s.
`results-20260911-stadium-fix/` holds the rerun after §XVI's O(ncells)-scan fix — one boot per
architecture, all 9 identities simultaneously live in every boot, same seed throughout.
**Result: 81/81 trials correct, 0 mismatches, byte-identical output to the original campaign** —
and the DOE-RUN header sequence (run_id/id_idx/id_label/rep assignment) is md5-identical across
all three raw logs, confirming the master shuffle is genuinely architecture-independent. Total
per-architecture wall-clock (boot + all 9 identity attaches + full 27-trial run, one continuous
QEMU session): amd64 ~165s, aarch64 ~290s, riscv64 ~161s — aarch64's identity 04 attach alone,
which stalled 90+ minutes before the fix, now completes in ~34s.
`results-20260911-donor-floor-fix/` holds a further rerun after §XVII's donor-floor fix (same
discipline: any Stadium defect repair reruns the whole DoE from the top). **Result: 81/81
trials correct, 0 mismatches**, DOE-RUN header sequence md5-identical to every prior run. Total
wall-clock: amd64 ~161s, aarch64 ~280s (confirms no regression from §XVI's fix), riscv64 ~162s.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+18
View File
@@ -495,6 +495,24 @@ size_t stadium_admit(VMUuid vm_id, const StadiumPatronHeader *candidate);
*/
int stadium_grant_quota(VMUuid new_vm_id, VMUuid from_vm_id);
/*
* stadium_best_donor - FABRIC-3.md SXVI donor-floor fix: the currently
* in-use VM with the largest free (unclaimed) cell count right now, i.e.
* the VM stadium_grant_quota()'s `from_vm_id` argument should be for a new
* VM's initial grant. Callers should NOT hardcode vm_uuid_hera() here --
* always splitting from Hera specifically converges her own free list
* toward empty after a bounded number of grants (each halves what remains)
* even while other, previously-granted VMs still hold nearly all of their
* own share untouched, silently starving later births though the Stadium
* as a whole has plenty of spare capacity. O(stadium_max_vm_count()) --
* a scan over quota slots (bounded by live VM population), not cells.
*
* @return The VM with the most free cells, or vm_uuid_none() if no VM
* holds a quota yet (Stadium not initialized, or called before
* Hera's own boot-time grant/admission).
*/
VMUuid stadium_best_donor(void);
/*
* stadium_cell_heat_get - Read a resident cell's own heat (FABRIC-0.md item
* 4.2's fourth ruling). Requires cell_index to be resident AND owned by
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+19 -4
View File
@@ -636,10 +636,25 @@ CapsuleRunResult capsule_birth_baby(
* STADIUM-ADMIT during that campaign refused unconditionally (quota
* slot < 0), 100% of trials, on all three architectures. Trade-off this
* introduces: a VM that dies stillborn below (IDENTITY exec fails) has
* still consumed half of Hera's free list, with no rollback -- accepted
* because stadium_grant_quota() failure was already non-fatal and a
* stillbirth here is the rare case, not the common one. */
(void)stadium_grant_quota(vm_id, vm_uuid_hera());
* still consumed half of its donor's free list, with no rollback --
* accepted because stadium_grant_quota() failure was already non-fatal
* and a stillbirth here is the rare case, not the common one.
*
* Donor is whichever live VM currently holds the most free cells
* (stadium_best_donor(), FABRIC-3.md SXVI) -- NOT unconditionally
* vm_uuid_hera() as this used to hardcode. Always splitting from Hera
* specifically converges HER free list toward empty after a bounded
* number of grants (each halves what remains), silently refusing every
* later birth once she runs dry even while VMs she granted to earlier
* still hold nearly all of their own share untouched -- the Stadium as
* a whole nowhere near full. Falls back to vm_uuid_hera() only if no VM
* holds a quota yet (stadium_best_donor() returns vm_uuid_none()), which
* should not happen here since stadium_birth_hera() always runs first. */
{
VMUuid donor = stadium_best_donor();
if (vm_uuid_equal(donor, vm_uuid_none())) donor = vm_uuid_hera();
(void)stadium_grant_quota(vm_id, donor);
}
/* FABRIC-2.md SS B, VM-COOL: admit this VM as a patron of its own
* quota -- identity 0 (same convention stadium_birth_hera() uses for
+64 -16
View File
@@ -80,6 +80,17 @@ typedef struct {
VMUuid vm_id;
int in_use;
size_t free_head;
size_t free_count; /* FABRIC-3.md SXVI donor-floor fix: O(1)-maintained
* length of this VM's own free list -- lets
* stadium_grant_quota()'s caller pick whichever VM
* actually has the most spare capacity to split
* from, instead of always Hera specifically (the
* old policy silently starved new VM births once
* HERA's own list ran thin, even while other VMs
* granted a generous initial half sat on nearly all
* of it unused). Kept in exact lockstep with every
* free_head push/pop below and in stadium_evict()/
* stadium_admit(). */
size_t resident_head; /* FABRIC-3.md aarch64 Stadium/COOL scaling fix:
* head of this VM's own resident-cell list, threaded
* through stadium_resident_next[]/stadium_resident_prev[]
@@ -261,6 +272,7 @@ int stadium_boot_init(void) {
quotas[i].vm_id = vm_uuid_none();
quotas[i].in_use = 0;
quotas[i].free_head = STADIUM_CELL_NONE;
quotas[i].free_count = 0;
quotas[i].resident_head = STADIUM_CELL_NONE;
quotas[i].reservoir = 0;
}
@@ -268,6 +280,7 @@ int stadium_boot_init(void) {
quotas[0].vm_id = vm_uuid_hera();
quotas[0].in_use = 1;
quotas[0].free_head = 0;
quotas[0].free_count = ncells;
/* item 4.1, §17.7: at quota-grant time, before any resident patron
* exists, the reservoir holds the VM's entire conserved share -- mirrors
* Hera holding the fleet's whole Q48_ONE before any other VM is born. */
@@ -466,6 +479,7 @@ int stadium_evict(size_t cell_index) {
header->link = size_to_link(stadium_quotas[slot].free_head);
header->contains = STADIUM_CONTAINS_NONE;
stadium_quotas[slot].free_head = cell_index;
stadium_quotas[slot].free_count++;
return 0;
}
@@ -507,6 +521,7 @@ size_t stadium_admit(VMUuid vm_id, const StadiumPatronHeader *candidate) {
if (stadium_quotas[slot].free_head != STADIUM_CELL_NONE) {
idx = stadium_quotas[slot].free_head;
stadium_quotas[slot].free_head = link_to_size(stadium_cell_array[idx].header.link);
stadium_quotas[slot].free_count--;
stadium_cell_array[idx].header = *candidate;
bitmap_set(idx);
/* Item 4.2 fix (§25.7): record ownership so stadium_evict()'s
@@ -567,6 +582,7 @@ size_t stadium_admit(VMUuid vm_id, const StadiumPatronHeader *candidate) {
* it straight back off. */
idx = stadium_quotas[slot].free_head;
stadium_quotas[slot].free_head = link_to_size(stadium_cell_array[idx].header.link);
stadium_quotas[slot].free_count--;
stadium_cell_array[idx].header = *candidate;
bitmap_set(idx);
/* Same fix as the free-list-pop path above -- stadium_evict() just wrote
@@ -581,13 +597,14 @@ size_t stadium_admit(VMUuid vm_id, const StadiumPatronHeader *candidate) {
/*
* FABRIC-0.md item 4.1a: one-time initial quota grant, not item 1.3's
* (still-unbuilt) recurring transfer. See stadium.h's doc for the full
* argument. Two passes over from_vm_id's free list: the first counts it
* (need the length before deciding where to split), the second detaches the
* first `half` cells into new_vm_id's own list, reassigning owner as it goes.
* argument. from_slot's free-list length is now read directly from its
* O(1)-maintained free_count (FABRIC-3.md SXVI donor-floor fix) rather than
* walked -- the walk below only detaches the first `half` cells into
* new_vm_id's own list, reassigning owner as it goes.
*/
int stadium_grant_quota(VMUuid new_vm_id, VMUuid from_vm_id) {
int from_slot, new_slot, i;
size_t count, half, idx, last_new, new_head, remainder_head;
size_t half, idx, last_new, new_head, remainder_head;
if (!stadium_initialized) return -1;
if (quota_slot_for_vm(new_vm_id) >= 0) return -1;
@@ -601,14 +618,7 @@ int stadium_grant_quota(VMUuid new_vm_id, VMUuid from_vm_id) {
}
if (new_slot < 0) return -1;
count = 0;
idx = stadium_quotas[from_slot].free_head;
while (idx != STADIUM_CELL_NONE) {
count++;
idx = link_to_size(stadium_cell_array[idx].header.link);
}
half = count / 2;
half = stadium_quotas[from_slot].free_count / 2;
if (half == 0) return -1; /* fewer than 2 free cells -- nothing to split */
new_head = stadium_quotas[from_slot].free_head;
@@ -624,15 +634,53 @@ int stadium_grant_quota(VMUuid new_vm_id, VMUuid from_vm_id) {
remainder_head = idx;
stadium_cell_array[last_new].header.link = size_to_link(STADIUM_CELL_NONE);
stadium_quotas[from_slot].free_head = remainder_head;
stadium_quotas[from_slot].free_count -= half;
stadium_quotas[new_slot].vm_id = new_vm_id;
stadium_quotas[new_slot].in_use = 1;
stadium_quotas[new_slot].free_head = new_head;
stadium_quotas[new_slot].reservoir = Q48_ONE;
stadium_quotas[new_slot].vm_id = new_vm_id;
stadium_quotas[new_slot].in_use = 1;
stadium_quotas[new_slot].free_head = new_head;
stadium_quotas[new_slot].free_count = half;
stadium_quotas[new_slot].resident_head = STADIUM_CELL_NONE;
stadium_quotas[new_slot].reservoir = Q48_ONE;
return 0;
}
/*
* stadium_best_donor - FABRIC-3.md SXVI donor-floor fix: which currently
* in-use VM has the most free (unclaimed) cells right now. Callers granting
* a new VM's initial quota (capsule_birth.c) should split from THIS VM, not
* unconditionally from Hera -- the old always-Hera policy meant her own free
* list converged toward empty after a bounded number of grants (each grant
* halves what remains) while every VM she had already granted to typically
* still held almost all of its own share untouched, silently starving every
* later birth even though the Stadium as a whole was nowhere near full.
* O(stadium_max_vm_count_val) -- a linear scan over quota SLOTS (bounded by
* live VM population, not cell count), not the cells themselves.
*
* @return The VM with the largest free_count, or vm_uuid_none() if no VM
* holds a quota yet (Stadium not initialized, or called before
* Hera's own boot-time grant).
*/
VMUuid stadium_best_donor(void) {
size_t i;
size_t best_count = 0;
int found = 0;
VMUuid best = vm_uuid_none();
if (!stadium_initialized) return best;
for (i = 0; i < stadium_max_vm_count_val; i++) {
if (!stadium_quotas[i].in_use) continue;
if (!found || stadium_quotas[i].free_count > best_count) {
best_count = stadium_quotas[i].free_count;
best = stadium_quotas[i].vm_id;
found = 1;
}
}
return best;
}
/* Shared by stadium_cell_heat_get()/_set(): resident AND owned by vm_id's
* own quota slot. Returns the quota slot on success, -1 on any refusal. */
static int owned_resident_slot(VMUuid vm_id, size_t cell_index) {