Robert Allan JamesandClaude Sonnet 5
71bcb72a59
sk_repl_idle()'s messaging pump (repl.c) walked the entire live-VM registry every idle beat (~1Hz) and dispatched a full VM-EXEC "MSG-TICK" -- a dictionary lookup plus a 32-slot arena scan -- into every live VM, every tick, unconditionally, forever. Fine at Tripod's original 3-VM scale; a full 9-identity turn-attractor campaign exposed it as a genuine wall on riscv64 specifically (its TCG makes each dispatch cost more): the same campaign that completed in 194s/ 339s on amd64/aarch64 never finished on riscv64 at 9 VMs across three attempts, while 8 VMs there was fine in 159s. Ruled out capacity explanations before touching anything: bumping riscv64's QEMU RAM 1024->4096 changed nothing (reverted), and a live STADIUM-RES@/MSG-STATUS probe with all 9 VMs attached showed no depletion. Host memory pressure was also ruled out directly (one background task did get OOM-killed once during the investigation, but the identical stall reproduced again with 9.2GB free). The real mistake was three premature kills under 4 minutes with no way to tell "slow" from "stuck" from outside the guest -- fixed by having run_doe_batch.sh sample the qemu process's own /proc/<pid>/stat utime every 60s; with that signal, riscv64 at 9 VMs was unambiguously alive (climbing utime, no hang), just disproportionately slow going from 8 VMs (159s) to 9 (600s+ and climbing). This was never really a riscv64-only bug: an O(N) per-second walk over the full VM population doesn't scale to the hundreds of VMs this fleet is headed toward, on any architecture -- riscv64 just made it visible first, at N=9, because its per-dispatch cost is highest. Fixed by round-robin batching: the pump now dispatches to at most SK_MSG_PUMP_BATCH (4) live VMs per idle beat via a persistent cursor that resumes where the previous beat left off, instead of all of them every time. Bounds both the scan and dispatch cost to O(K) regardless of total VM count; any single VM's queue now drains roughly every ceil(N/K) beats instead of every beat, still bounded and still matching the pump's own existing best-effort contract. No new C primitives, no messaging/Stadium changes. Verified: clean build on all 3 architectures, then the full 3x9x3 campaign re-run on all 3 (not just riscv64) per the standing rule that a defect repair requires a clean re-run everywhere before anything counts as closed: amd64 198s 0 faults 99/99 tokens x8 656/656 K-conserved aarch64 339s 0 faults 99/99 tokens x8 659/659 K-conserved riscv64 178s 0 faults 99/99 tokens x8 659/659 K-conserved riscv64 went from "never completes" to faster than aarch64, same campaign, same seed, same identity set. No regression on amd64/ aarch64. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo