LithosAnanake/experiments/bare_metal/runs/doe-riscv64-20260912-174936.csv at 61755fde782bbe05b38ce023a4a394957c5cc8e8

admin/LithosAnanake

Fork 0

Files

T

Robert Allan JamesandClaude Sonnet 5 71bcb72a59

Build / build-amd64-iso (push) Waiting to run

Details

Build / build-aarch64-iso (push) Waiting to run

Details

Build / build-riscv64-img (push) Waiting to run

Details

Fix O(N) idle-loop messaging pump; full 3x9x3 turn-attractor campaign clean on all 3 architectures (FABRIC-3.md §XXII)

sk_repl_idle()'s messaging pump (repl.c) walked the entire live-VM
registry every idle beat (~1Hz) and dispatched a full VM-EXEC
"MSG-TICK" -- a dictionary lookup plus a 32-slot arena scan -- into
every live VM, every tick, unconditionally, forever. Fine at Tripod's
original 3-VM scale; a full 9-identity turn-attractor campaign
exposed it as a genuine wall on riscv64 specifically (its TCG makes
each dispatch cost more): the same campaign that completed in 194s/
339s on amd64/aarch64 never finished on riscv64 at 9 VMs across three
attempts, while 8 VMs there was fine in 159s.

Ruled out capacity explanations before touching anything: bumping
riscv64's QEMU RAM 1024->4096 changed nothing (reverted), and a live
STADIUM-RES@/MSG-STATUS probe with all 9 VMs attached showed no
depletion. Host memory pressure was also ruled out directly (one
background task did get OOM-killed once during the investigation,
but the identical stall reproduced again with 9.2GB free). The real
mistake was three premature kills under 4 minutes with no way to
tell "slow" from "stuck" from outside the guest -- fixed by having
run_doe_batch.sh sample the qemu process's own /proc/<pid>/stat utime
every 60s; with that signal, riscv64 at 9 VMs was unambiguously alive
(climbing utime, no hang), just disproportionately slow going from 8
VMs (159s) to 9 (600s+ and climbing).

This was never really a riscv64-only bug: an O(N) per-second walk
over the full VM population doesn't scale to the hundreds of VMs this
fleet is headed toward, on any architecture -- riscv64 just made it
visible first, at N=9, because its per-dispatch cost is highest.

Fixed by round-robin batching: the pump now dispatches to at most
SK_MSG_PUMP_BATCH (4) live VMs per idle beat via a persistent cursor
that resumes where the previous beat left off, instead of all of them
every time. Bounds both the scan and dispatch cost to O(K) regardless
of total VM count; any single VM's queue now drains roughly every
ceil(N/K) beats instead of every beat, still bounded and still
matching the pump's own existing best-effort contract. No new C
primitives, no messaging/Stadium changes.

Verified: clean build on all 3 architectures, then the full 3x9x3
campaign re-run on all 3 (not just riscv64) per the standing rule
that a defect repair requires a clean re-run everywhere before
anything counts as closed:

  amd64   198s  0 faults  99/99 tokens x8  656/656 K-conserved
  aarch64 339s  0 faults  99/99 tokens x8  659/659 K-conserved
  riscv64 178s  0 faults  99/99 tokens x8  659/659 K-conserved

riscv64 went from "never completes" to faster than aarch64, same
campaign, same seed, same identity set. No regression on amd64/
aarch64.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo

2026-09-12 19:29:14 -04:00

8.8 KiB

Raw Blame History

1	tick_number	elapsed_ns	tick_interval_ns	word_executions_delta	hot_word_count	avg_word_heat_q48	window_width	actual_window_size	jitter_bits	apic_ticks	time_trust_q48	vm_call_depth_max	hera_heat_q48	hermes_heat_q48	artemis_heat_q48	fleet_k_q48	fleet_conserved
2	436	4360000	10000	256	85	632	512	512	0	3425	65536	1	62506	1503	1527	65536	1
3	437	4370000	10000	256	86	621	1024	1024	0	3427	65536	1	62506	1503	1527	65536	1
4	438	4380000	10000	256	88	606	1024	1024	0	3431	65536	1	62506	1503	1527	65536	1
5	439	4390000	10000	256	89	585	2048	2048	0	3432	65536	1	62506	1503	1527	65536	1
6	440	4400000	10000	256	90	576	2048	2048	0	3434	65536	1	62506	1503	1527	65536	1
7	441	4410000	10000	256	90	571	4096	4096	0	3435	65536	1	62506	1503	1527	65536	1
8	442	4420000	10000	256	90	566	4096	4096	0	3436	65536	1	62506	1503	1527	65536	1
9	443	4430000	10000	256	90	564	4096	4096	0	3437	65536	1	62506	1503	1527	65536	1
10	444	4440000	10000	256	90	559	4096	4096	0	3438	65536	1	62506	1503	1527	65536	1
11	445	4450000	10000	256	90	554	2048	2048	0	3439	65536	1	62506	1503	1527	65536	1
12	446	4460000	10000	256	90	546	2048	2048	0	3440	65536	1	62506	1503	1527	65536	1
13	447	4470000	10000	256	91	547	2048	2048	0	3442	65536	1	62506	1503	1527	65536	1
14	448	4480000	10000	256	91	548	1024	1024	0	3443	65536	1	62506	1503	1527	65536	1
15	449	4490000	10000	256	91	550	1024	1024	0	3445	65536	1	62506	1503	1527	65536	1
16	450	4500000	10000	256	91	551	512	512	0	3447	65536	1	62506	1503	1527	65536	1
17	451	4510000	10000	256	91	552	512	512	0	3448	65536	1	62506	1503	1527	65536	1
18	452	4520000	10000	256	92	554	256	256	0	3449	65536	1	62506	1503	1527	65536	1
19	453	4530000	10000	256	92	555	256	256	0	3450	65536	1	62506	1503	1527	65536	1
20	454	4540000	10000	256	93	557	256	256	0	3451	65536	1	62506	1503	1527	65536	1
21	455	4550000	10000	256	94	525	256	256	0	3452	65536	1	62506	1503	1527	65536	1
22	456	4560000	10000	256	96	527	256	256	0	3453	65536	1	62506	1503	1527	65536	1
23	457	4570000	10000	256	104	528	256	256	0	3454	65536	1	62506	1503	1527	65536	1
24	458	4580000	10000	256	104	529	256	256	0	3456	65536	1	62506	1503	1527	65536	1
25	459	4590000	10000	256	104	531	256	256	0	3457	65536	1	62506	1503	1527	65536	1
26	460	4600000	10000	256	104	532	256	256	0	3458	65536	1	62506	1503	1527	65536	1
27	461	4610000	10000	256	104	533	256	256	0	3458	65536	1	62506	1503	1527	65536	1
28	462	4620000	10000	256	104	535	256	256	0	3459	65536	1	62506	1503	1527	65536	1
29	463	4630000	10000	256	104	528	256	256	0	3460	65536	1	62506	1503	1527	65536	1
30	464	4640000	10000	256	104	497	256	256	0	3461	65536	1	62506	1503	1527	65536	1
31	465	4650000	10000	256	104	479	256	256	0	3462	65536	1	62506	1503	1527	65536	1
32	466	4660000	10000	256	104	467	256	256	0	3466	65536	1	62506	1503	1527	65536	1
33	467	4670000	10000	256	104	468	256	256	0	3467	65536	1	62506	1503	1527	65536	1
34	468	4680000	10000	256	104	469	256	256	0	3468	65536	1	62506	1503	1527	65536	1
35	469	4690000	10000	256	104	471	256	256	0	3469	65536	1	62506	1503	1527	65536	1
36	470	4700000	10000	256	104	472	256	256	0	3470	65536	1	62506	1503	1527	65536	1
37	471	4710000	10000	256	104	473	256	256	0	3471	65536	1	62506	1503	1527	65536	1
38	472	4720000	10000	256	104	470	256	256	0	3472	65536	1	62506	1503	1527	65536	1
39	473	4730000	10000	256	104	471	256	256	0	3473	65536	1	62506	1503	1527	65536	1
40	474	4740000	10000	256	104	472	256	256	0	3474	65536	1	62506	1503	1527	65536	1
41	475	4750000	10000	256	104	473	256	256	0	3476	65536	1	62506	1503	1527	65536	1
42	476	4760000	10000	256	104	474	256	256	0	3477	65536	1	62506	1503	1527	65536	1
43	477	4770000	10000	256	104	476	256	256	0	3478	65536	1	62506	1503	1527	65536	1
44	478	4780000	10000	256	104	477	256	256	0	3479	65536	1	62506	1503	1527	65536	1
45	479	4790000	10000	256	104	478	256	256	0	3479	65536	1	62506	1503	1527	65536	1
46	480	4800000	10000	256	109	479	256	256	0	3481	65536	1	62506	1503	1527	65536	1
47	481	4810000	10000	256	109	480	256	256	0	3482	65536	1	62506	1503	1527	65536	1
48	482	4820000	10000	256	109	481	256	256	0	3483	65536	1	62506	1503	1527	65536	1
49	483	4830000	10000	256	109	483	256	256	0	3485	65536	1	62506	1503	1527	65536	1
50	484	4840000	10000	256	109	484	256	256	0	3486	65536	1	62506	1503	1527	65536	1
51	485	4850000	10000	256	109	485	256	256	0	3488	65536	1	62506	1503	1527	65536	1
52	486	4860000	10000	256	109	486	256	256	0	3489	65536	1	62506	1503	1527	65536	1
53	487	4870000	10000	256	109	487	256	256	0	3490	65536	1	62506	1503	1527	65536	1
54	488	4880000	10000	256	109	489	256	256	0	3491	65536	1	62506	1503	1527	65536	1
55	489	4890000	10000	256	111	490	256	256	0	3492	65536	1	62506	1503	1527	65536	1
56	133	1330000	18446744073705991616	32917	38	367	256	256	4895412794951727409	3495	65536	0	62506	1503	1527	65536	1
57	490	4900000	3570000	83731	111	491	256	256	4704899453134307328	3497	65536	1	62506	1503	1527	65536	1
58	491	4910000	10000	256	111	492	256	256	0	3499	65536	1	62506	1503	1527	65536	1
59	492	4920000	10000	256	111	493	256	256	0	3500	65536	1	62506	1503	1527	65536	1
60	493	4930000	10000	256	111	494	256	256	0	3501	65536	1	62506	1503	1527	65536	1
61	494	4940000	10000	256	111	496	256	256	0	3502	65536	1	62506	1503	1527	65536	1
62	495	4950000	10000	256	111	497	256	256	0	3502	65536	1	62506	1503	1527	65536	1
63	496	4960000	10000	256	111	498	256	256	0	3504	65536	1	62506	1503	1527	65536	1
64	497	4970000	10000	256	111	499	256	256	0	3504	65536	1	62506	1503	1527	65536	1
65	498	4980000	10000	256	111	500	256	256	0	3506	65536	1	62506	1503	1527	65536	1
66	499	4990000	10000	256	111	501	256	256	0	3507	65536	1	62506	1503	1527	65536	1
67	500	5000000	10000	256	111	503	256	256	0	3508	65536	1	62506	1503	1527	65536	1
68	501	5010000	10000	256	111	504	256	256	0	3509	65536	1	62506	1503	1527	65536	1
69	502	5020000	10000	256	111	505	256	256	0	3511	65536	1	62506	1503	1527	65536	1
70	503	5030000	10000	256	111	506	256	256	0	3512	65536	1	62506	1503	1527	65536	1
71	504	5040000	10000	256	115	507	256	256	0	3513	65536	1	62506	1503	1527	65536	1
72	505	5050000	10000	256	116	508	256	256	0	3515	65536	1	62506	1503	1527	65536	1
73	506	5060000	10000	256	117	510	256	256	0	3516	65536	1	62506	1503	1527	65536	1
74	507	5070000	10000	256	117	511	256	256	0	3518	65536	1	62506	1503	1527	65536	1
75	508	5080000	10000	256	117	512	256	256	0	3519	65536	1	62506	1503	1527	65536	1
76	509	5090000	10000	256	117	513	256	256	0	3520	65536	1	62506	1503	1527	65536	1
77	510	5100000	10000	256	117	514	256	256	0	3522	65536	1	62506	1503	1527	65536	1
78	511	5110000	10000	256	117	516	256	256	0	3523	65536	1	62506	1503	1527	65536	1
79	512	5120000	10000	256	117	517	256	256	0	3524	65536	1	62506	1503	1527	65536	1
80	513	5130000	10000	256	117	518	256	256	0	3525	65536	1	62506	1503	1527	65536	1
81	514	5140000	10000	256	117	519	256	256	0	3526	65536	1	62506	1503	1527	65536	1
82	515	5150000	10000	256	117	520	256	256	0	3527	65536	1	62506	1503	1527	65536	1
83	516	5160000	10000	256	117	521	256	256	0	3528	65536	1	62506	1503	1527	65536	1
84	517	5170000	10000	256	117	523	256	256	0	3529	65536	1	62506	1503	1527	65536	1
85	518	5180000	10000	256	117	524	256	256	0	3530	65536	1	62506	1503	1527	65536	1
86	519	5190000	10000	256	117	525	256	256	0	3534	65536	1	62506	1503	1527	65536	1
87	520	5200000	10000	256	117	526	256	256	0	3537	65536	1	62506	1503	1527	65536	1
88	521	5210000	10000	256	117	527	256	256	0	3538	65536	1	62506	1503	1527	65536	1
89	522	5220000	10000	256	117	528	256	256	0	3538	65536	1	62506	1503	1527	65536	1
90	523	5230000	10000	256	117	530	256	256	0	3540	65536	1	62506	1503	1527	65536	1
91	524	5240000	10000	256	117	531	256	256	0	3541	65536	1	62506	1503	1527	65536	1
92	525	5250000	10000	256	117	532	256	256	0	3542	65536	1	62506	1503	1527	65536	1
93	526	5260000	10000	256	118	533	256	256	0	3543	65536	1	62506	1503	1527	65536	1
94	527	5270000	10000	256	118	534	256	256	0	3544	65536	1	62506	1503	1527	65536	1
95	528	5280000	10000	256	118	535	256	256	0	3545	65536	1	62506	1503	1527	65536	1
96	529	5290000	10000	256	118	537	256	256	0	3546	65536	1	62506	1503	1527	65536	1
97	530	5300000	10000	256	118	538	256	256	0	3547	65536	1	62506	1503	1527	65536	1
98	531	5310000	10000	256	118	539	256	256	0	3547	65536	1	62506	1503	1527	65536	1
99	532	5320000	10000	256	118	540	256	256	0	3549	65536	1	62506	1503	1527	65536	1
100	533	5330000	10000	256	120	541	256	256	0	3550	65536	1	62506	1503	1527	65536	1

8.8 KiB Raw Blame History

8.8 KiB

Raw Blame History