Robert Allan JamesandClaude Sonnet 5
8471d529bc
The claim ("coalesces but doesn't split") was never actually checked
against kmalloc.c -- it was carried over from alloc_kernel.c's own doc
comment about itself and mis-applied to a different file. Asked to fix
it, re-reading kmalloc.c showed allocate_from_block() already has a
complete, unconditional splitting implementation. Nothing was broken;
correcting the record instead of "fixing" working code.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Ec88YKxxhZGG1RNnune78
1679 lines
123 KiB
Markdown
1679 lines
123 KiB
Markdown
# FABRIC-3.md — bare metal boot
|
||
|
||
**Status:** Living working document, opened 2026-09-04 as the successor to `FABRIC-2.md`
|
||
(now closed/archival — see its own header). Topic for this document, per direct instruction:
|
||
**bare metal boot** — getting LithosAnanke to actually boot on real hardware, not just QEMU.
|
||
`FABRIC-2.md` §I.6 (Milestone 8) already named this as the one item that pass couldn't close
|
||
from a coding session at all, for exactly this reason — it needs a real machine and a human
|
||
physically present. This document is where that work, and everything downstream of it, gets
|
||
tracked.
|
||
|
||
**How to use this document going forward.** New findings, new punch-list items, and new
|
||
decisions for bare-metal-boot work get added here, not to `FABRIC-2.md`. Same discipline every
|
||
prior document in this series used: write the decision and its reasoning down before building,
|
||
close items with a dated note citing real evidence, never silently drop a stale claim.
|
||
|
||
---
|
||
|
||
## I.1 — Task 1: merge `v2.0.1` into `master`, verify build/function equivalence
|
||
|
||
**Written up before executing**, per direct instruction and this series' own standing
|
||
discipline.
|
||
|
||
**Why this is task 1.** `FABRIC-2.md`'s entire 7-step closure pass (§I.1–§I.5, §I.7, plus
|
||
today's FABRIC-series rename) happened on the `v2.0.1` branch, not `master`. Before any real
|
||
bare-metal-boot work starts, that work needs to land where `.claude/CLAUDE.md` says the
|
||
project's sole production line actually lives: `master`. Doing this first, cleanly, before
|
||
starting new work avoids ever having two divergent lines to reconcile later.
|
||
|
||
**Investigated before writing this up, not assumed:**
|
||
- `git merge-base --is-ancestor master v2.0.1` — **true**. `master` (local HEAD `d2a0305`) is
|
||
a strict ancestor of `v2.0.1` (HEAD `b031b80`) — `v2.0.1` is exactly `master` plus 47 commits
|
||
forward, no divergent history on either side. This means the "merge" is a pure **fast-forward**,
|
||
not a real three-way merge — nothing to resolve, no conflict possible.
|
||
- `origin/master` carries exactly one commit beyond local `master` (`58c59e8`, "Initial
|
||
commit") that local `master` hadn't fetched yet — confirmed already contained in `v2.0.1`'s
|
||
own history (`git merge-base --is-ancestor 58c59e8 v2.0.1` — true), so it introduces no
|
||
discrepancy either.
|
||
- `master`'s own tree still has the *old* `FABRIC.md`/`FABRIC-2.md`/`FABRIC-3.md` naming
|
||
(unrenamed) — expected, since today's rename commit (`b031b80`) only exists on `v2.0.1` so
|
||
far. The fast-forward brings the rename to `master` along with everything else; nothing
|
||
separate needs doing for it.
|
||
|
||
**Plan:**
|
||
1. Fast-forward `master` to `v2.0.1`'s tip (`git checkout master && git merge --ff-only v2.0.1`)
|
||
— refuses loudly instead of silently doing a real merge if the ancestor relationship somehow
|
||
isn't what the investigation above found, so this step re-verifies its own precondition.
|
||
2. Push `master` to `origin`.
|
||
3. **Verify build/function equivalence on a genuinely clean tree**, not by inference: `git clean`
|
||
(after confirming nothing untracked-but-wanted is present), then the full acceptance sequence
|
||
`.claude/CLAUDE.md` already mandates for any kernel change — `clean qemu` on all three
|
||
architectures, in the foreground, one at a time, each reaching `ok>` and shutting down
|
||
cleanly. Since the tree is byte-identical to `v2.0.1`'s post-fast-forward, this is expected
|
||
to reproduce exactly what `v2.0.1`'s own last acceptance pass already showed — the point of
|
||
re-running it here is to confirm that expectation holds on `master` itself, not to assume it
|
||
from the fast-forward alone.
|
||
4. Return to `v2.0.1` as the working branch afterward (`.claude/CLAUDE.md`'s own rule: always
|
||
return to the correct working branch after any out-of-branch work), unless told otherwise.
|
||
|
||
**DONE 2026-09-04, exactly as planned:**
|
||
1. Committed the write-up above on `v2.0.1` first (`72c14cb`), pushed. This became `v2.0.1`'s
|
||
new tip.
|
||
2. `git checkout master && git merge --ff-only v2.0.1` — **Fast-forward**, `d2a0305..72c14cb`,
|
||
confirming the investigated ancestor relationship held exactly as expected; no conflict, no
|
||
merge commit.
|
||
3. `git push origin master` — `origin/master` moved `58c59e8..72c14cb`.
|
||
4. **Verified on a genuinely clean `master` tree**, not inferred from the fast-forward:
|
||
- Hosted build (`make clean && make`): clean compile, zero warnings, same as `v2.0.1`.
|
||
- Full 3-arch kernel acceptance (`clean qemu`, amd64/aarch64/riscv64, each in the foreground):
|
||
all three reached `(zuse) ok>`/`ok>` and shut down cleanly, zero build errors, zero
|
||
unexpected warnings — identical outcome to `v2.0.1`'s own last acceptance pass, confirmed
|
||
directly rather than assumed. Logs: `logs/20260904-113208/amd64/`,
|
||
`logs/20260904-113320/aarch64/`, `logs/20260904-113552/riscv64/`.
|
||
5. `master` and `v2.0.1` are now identical (`72c14cb` on both, `origin` and local). Returned to
|
||
`v2.0.1` as the working branch per plan step 4.
|
||
|
||
**Task 1 closed.** `master` genuinely is the production line again, current through today's
|
||
FABRIC-series rename and the full `FABRIC-2.md` §I closure. Bare-metal-boot work (this
|
||
document's actual topic) starts from here.
|
||
|
||
## I.2 — Task 2: version correction — the `v2.0.1` bump and `v2.0.0` tag were premature
|
||
|
||
**Direct instruction, 2026-09-04**: the `LITHOS_VERSION` bump to `2.0.1` (and the branch name
|
||
that followed it) got ahead of the real state — per `Makefile.starkernel`'s own versioning
|
||
policy (`v2.0.0` = QEMU release, even major/LTS; `v2.0.1` = the SER5 hardware-track *line*,
|
||
RDRAND backend + thumbdrive image goal), claiming `2.0.1` implies hardware-track progress that
|
||
was never actually verified on real hardware — that verification is precisely `FABRIC-3.md`'s
|
||
whole open topic (§I.6 in the closed `FABRIC-2.md`). The current `master` HEAD is, correctly,
|
||
still a `v2.0.0`-class QEMU-only release. "Nothing harmful" — a version-label correction, not a
|
||
functional rollback.
|
||
|
||
**Found and fixed while correcting this, not left half-done:**
|
||
- A real gap in the FABRIC-series rename from earlier today: `Makefile.starkernel`,
|
||
`Kconfig.kernel`, `scripts/bleach_zuse_img.sh`, four `proof/*.thy` files, and
|
||
`src/starkernel/arch/amd64/isr.S` all still had stale `FABRIC.md`/`FABRIC-2.md`/`FABRIC-3.md`
|
||
citations — the original sweep's file-list only matched `--include=*.md/*.c/*.h/*.4th`, which
|
||
silently skipped every file without one of those four extensions. Found by re-grepping with
|
||
the extensions excluded instead of included. Fixed with the same safe placeholder-substitution
|
||
technique the original rename used (each file, one pass, ordered `FABRIC-3→2→1→0` placeholders
|
||
then resolved) — verified no double-shifted or broken references remained afterward.
|
||
`.claude/settings.local.json`'s own historical Bash-permission-grant log (literal past command
|
||
strings naming the file as it was called *at the time*) was deliberately left alone — rewriting
|
||
it would falsify an audit trail, not fix a stale citation.
|
||
- `ClaudeEXPORT/memories.json`/`conversations.json` also still reference the old names — left
|
||
untouched on purpose, same reasoning as the memory note on that archive: it's a frozen export,
|
||
mining material, not live documentation to keep in sync.
|
||
|
||
**Changes:**
|
||
1. `Makefile.starkernel`: `LITHOS_VERSION ?= 2.0.1` → `2.0.0`.
|
||
2. The rename-gap fix above (7 files).
|
||
3. Verified 3-arch boot (`clean qemu`, amd64/aarch64/riscv64, each in the foreground): all three
|
||
show `LithosAnanke v2.0.0` in the boot banner (confirmed directly in each serial log, not
|
||
assumed from the Makefile edit alone), zero build errors, zero unexpected warnings, clean
|
||
shutdown.
|
||
4. Moved the existing `v2.0.0` git tag (previously at `2efd7fe`, the original QEMU-release
|
||
milestone commit — that commit and its own message stay fully intact in history, only the
|
||
tag pointer moves) to the current `master`/`v2.0.1`-branch HEAD, per explicit instruction —
|
||
the prior tag placement was itself part of the same "got ahead of myself" correction, not a
|
||
separate decision. No remote tag existed yet (`git ls-remote --tags origin` was empty for
|
||
`v2.0.0`), so no destructive remote operation was needed, only a local move-and-push.
|
||
5. **Follow-up, same day**: `v2.0.1` (the working branch this and Task 1 happened on) deleted,
|
||
local and `origin` — confirmed a strict ancestor of `master`'s new HEAD first, so nothing
|
||
was lost. `master` is the repo's only branch from here on.
|
||
|
||
---
|
||
|
||
## II. Three architectures, three different hardware scopes
|
||
|
||
Per direct instruction, 2026-09-04. The real-hardware targets are **not** symmetric across
|
||
architectures — each gets its own section below because the actual scope of "done" is
|
||
different for each:
|
||
|
||
- **amd64 — genericity is the goal, not just the SER5.** The Beelink SER5 is the machine in
|
||
hand and the development/reference target, but the real requirement is broader: this needs
|
||
to boot on *any* x86_64 machine — laptop, desktop, tower, or mini PC — not just one vendor's
|
||
quirks. SER5-only success is necessary but not sufficient; anything that works only because
|
||
of an SER5-specific assumption (a particular ACPI table shape, a specific UEFI
|
||
implementation's quirks) is a bug against this goal, not a deferred nice-to-have.
|
||
- **aarch64 — Raspberry Pi 5, and only the Raspberry Pi 5.** No genericity requirement across
|
||
aarch64 boards — this is the one and only target for this architecture.
|
||
- **riscv64 — Milk-V Mars, and only the Milk-V Mars.** Same as aarch64: one specific board,
|
||
not a generic riscv64-SBC goal.
|
||
|
||
**How to use sections III–V below.** Same discipline as everything else in this series: plan
|
||
before building, one section at a time, iterating — not all three architectures in parallel,
|
||
and not front-loading a complete plan before any real hardware is in front of us. Each section
|
||
starts with what's already true (existing repo infrastructure, already-decided policy) and
|
||
what's still genuinely unknown, not assumed.
|
||
|
||
## III. amd64 — generic x86_64 bare metal (reference hardware: Beelink SER5)
|
||
|
||
**Already true, not to be re-derived:**
|
||
- `ROADMAP.md`'s "Board-by-board hardware rollout" already names this `v2.2.0`'s gate: the
|
||
generic GPT/FAT32 thumbdrive image (`make -f Makefile.starkernel ARCH=amd64 thumbdrive`,
|
||
already built — `Makefile.starkernel:1018`) flashes to and boots on the real SER5 via its
|
||
real UEFI, reaching POST + `ok>`, with the amd64 RDRAND entropy backend
|
||
(`src/starkernel/rng/rng.c`, already built and part of `master`) serving live entropy.
|
||
- `iso-usb` (`Makefile.starkernel:1060`) is the alternate, novice-friendly path (UEFI
|
||
isohybrid ISO for tools like GNOME Disks "Restore Disk Image...") — same underlying image,
|
||
different flashing UX.
|
||
- `FABRIC-2.md` §I.6's own 8-step physical-boot sequence (build ISO, identify the target
|
||
device, flash it, physically boot, decide an observation method, confirm POST, confirm
|
||
`ok>`, document) is the closest thing to an existing plan — but it predates the genericity
|
||
requirement and was written with no hardware in hand yet.
|
||
|
||
**Decided in conversation, 2026-09-04:**
|
||
- **Observation: HDMI (interactive) + serial (logged transcript), both.** The kernel's own
|
||
VT100 framebuffer console (`console.c`/`vt100.c`/`framebuffer.c`) already gives a real
|
||
interactive display over HDMI — no new code needed there. Serial capture, if the SER5
|
||
exposes a UART header, uses the Raspberry Pi's own GPIO UART as the USB-serial bridge
|
||
(already available hardware, not a purchase blocker) — this needs the SER5's own UART pins
|
||
physically identified first (not yet confirmed it has an accessible header at all).
|
||
- **Genericity is verified by standards-compliance, not a second machine** — no second x86_64
|
||
box is available right now. The bar is: nothing in the boot path may depend on an
|
||
SER5-specific assumption (a particular ACPI table shape, a specific UEFI implementation's
|
||
quirk) — argued by code audit against real UEFI/ACPI standards, not by testing on a second
|
||
board, until one becomes available. This is a real constraint on the punch list below (item
|
||
6), not a deferred nice-to-have.
|
||
- **Secure Boot: already disabled on this SER5.** No signed-loader work needed for this pass —
|
||
"Secure Boot disabled in firmware setup" is the supported path, documented as such rather
|
||
than built around.
|
||
|
||
**Punch list, this cadence's actual next steps:**
|
||
1. Build the generic thumbdrive image: `make -f Makefile.starkernel ARCH=amd64 thumbdrive`.
|
||
2. Flash it to a USB stick (`dd`, per the target's own existing usage message).
|
||
3. Physically inspect the SER5 for an exposed UART header/pins; if present, wire the
|
||
Raspberry Pi's GPIO UART to it as the serial bridge. If absent, HDMI-only for this pass —
|
||
not a blocker, just a scope note for step 7's log.
|
||
4. Connect HDMI + keyboard to the SER5.
|
||
5. Boot the SER5 from the flashed stick (firmware boot-order menu as needed — Secure Boot
|
||
already disabled, confirmed above, so no signing prompt expected).
|
||
6. **Code audit pass — DONE, 2026-09-04.** Reviewed `src/starkernel/boot/uefi_loader.c` and
|
||
every file in `arch/amd64/` (`arch.c`, `apic.c`, `ioapic.c`, `interrupts.c`, `i8042.c`,
|
||
`timer.c`) for anything that assumes SER5-specific hardware rather than standard UEFI/ACPI.
|
||
Report only, per this project's own "identify, don't fix unless asked" rule — nothing
|
||
below was changed.
|
||
|
||
**Clean (verified standards-correct, not just "worked in QEMU"):**
|
||
- `uefi_loader.c`: COM1 presence is probed (scratch-register readback + bounded THRE spin)
|
||
before any code trusts it, so a board with no legacy serial decode — the documented SER5
|
||
case — degrades to silently dropping bytes rather than hanging. The ACPI table search
|
||
does two explicit passes (2.0 GUID first, 1.0 only as fallback) rather than a single
|
||
first-match loop, avoiding a real bug class (silently preferring the RSDT-only 1.0
|
||
pointer). `GetMemoryMap`/`ExitBootServices` follows the UEFI "golden path" (nothing
|
||
between the two calls) with the required `EFI_INVALID_PARAMETER` retry loop. GOP
|
||
framebuffer degrades to `PixelBltOnly` cleanly when no linear framebuffer exists.
|
||
- `apic.c`: LAPIC presence, x2APIC-vs-xAPIC mode, and the APIC timer's own tick rate are
|
||
all determined at runtime (MSR read, empirical TSC-timed calibration) — no hardcoded
|
||
bus-frequency assumption anywhere in the timer path.
|
||
- `ioapic.c`: I/O APIC base and GSI base come from the ACPI MADT, not a hardcoded address.
|
||
- `timer.c`'s PM_TMR_BLK port: already fixed correctly (`fadt_find_pm_tmr_port()`,
|
||
2026-08-11 per this file's own comment) — discovered from the FADT, not assumed from a
|
||
single machine type's legacy port.
|
||
|
||
**Findings (real, not fixed — reported per policy):**
|
||
1. **HPET base address hardcoded, same bug class as the already-fixed PM_TMR port.**
|
||
`timer.c`'s `HPET_PHYS_BASE 0xFED00000` is never cross-checked against the ACPI `"HPET"`
|
||
table (confirmed absent — no `"HPET"` signature lookup exists anywhere in the tree,
|
||
unlike the FADT/`"FACP"` walk `fadt_find_pm_tmr_port()` already does and could extend to
|
||
reach it with the same RSDP→XSDT→table-by-signature walk). 0xFED00000 is the
|
||
near-universal legacy convention, not a guaranteed one — this file's own comment already
|
||
documents the *identical* failure mode for the PM Timer port and how it was fixed;
|
||
the same reasoning was never applied to HPET.
|
||
2. **LAPIC relocation: MADT data already parsed one file over, never plumbed through.**
|
||
`apic_init()`'s own doc comment self-flags `boot_info` as unused, "reserved for future
|
||
use (e.g., reading the ACPI MADT for the physical LAPIC base address)" — always uses
|
||
`LAPIC_DEFAULT_PHYS = 0xFEE00000`. Narrower than "unimplemented": `ioapic.c`'s
|
||
`MadtHeader.local_apic_address` field is already read into a parsed struct by
|
||
`parse_madt()`, just never passed to `apic.c`. `parse_madt()` also only handles MADT
|
||
entry types 1 (I/O APIC) and 2 (Interrupt Source Override) — type 5 (Local APIC Address
|
||
Override), the entry that matters on a system that actually relocates it, isn't parsed
|
||
at all.
|
||
3. **i8042 PS/2 controller poked unconditionally, no presence check.** `i8042_init()` and
|
||
`i8042_drain_stale()` read/write ports 0x60/0x64 with no probe of any kind — contrast
|
||
with this same codebase's own `raw_serial_init()`, which explicitly probes COM1's
|
||
scratch register before trusting it exists. The ACPI FADT carries an IA-PC
|
||
boot-architecture flags field that reports 8042 presence; it is never consulted. Same
|
||
root gap in `pic_disable()` (`interrupts.c`): unconditionally masks the legacy 8259 PIC
|
||
at ports 0x21/0xA1 with no corresponding FADT check.
|
||
4. **`arch_cold_reset()` uses the legacy i8042 reset-pulse hack instead of the ACPI Reset
|
||
Register.** Confirmed live, not dormant: called directly from `BYE`
|
||
(`mama_forth_words.c:1437`, Hera-only cold restart). Pulses port 0x64 with 0xFE — the
|
||
pre-ACPI convention that assumes the same i8042 controller finding #3 already flags —
|
||
instead of the FADT's `RESET_REG`/`RESET_VALUE` fields, which are the actual
|
||
standards-defined generic reset mechanism (confirmed absent from the tree — no
|
||
`RESET_REG` reference exists anywhere) and don't depend on i8042 existing at all.
|
||
Degrades gracefully (falls through to a `cli;hlt` spin loop, doesn't hang) if the
|
||
pulse has no effect — the honest risk here is a silent no-op reset on an i8042-less
|
||
board, not a hang.
|
||
|
||
None of these four are certain to actually bite on the real SER5 — the near-universal
|
||
conventions they assume (0xFED00000, 0xFEE00000, i8042-at-0x60/0x64) hold on the large
|
||
majority of real PC-compatible x86_64 systems including consumer mini-PCs, and only
|
||
finding 4 has a confirmed live call path. Recorded here because the decision above
|
||
requires arguing genericity from standards compliance, not from "it booted" — these are
|
||
the concrete gaps between what the code assumes and what UEFI/ACPI actually guarantees.
|
||
7. Capture the boot: confirm POST reaches the same `1012/0/0` result QEMU shows, confirm
|
||
`ok>`/`zuse)ok>`, confirm `rng: backend = rdrand` (live entropy, not the QEMU-only
|
||
`virtio-rng` path), save the serial transcript (if wired) the same way `logs/` already
|
||
keeps QEMU's.
|
||
8. Mint a Zuse identity on a second thumbdrive on the real SER5, confirm it re-attaches —
|
||
the same real-hardware round-trip `ROADMAP.md`'s `v2.2.0` gate already names.
|
||
9. Update this section with results — pass/fail per step, any SER5-specific or genuinely
|
||
generic-UEFI finding either way, before moving to aarch64.
|
||
|
||
## IV. aarch64 — Raspberry Pi 5
|
||
|
||
**Already true:** `ROADMAP.md` names this `v2.4.0`'s gate: boots on the real board, aarch64
|
||
peripheral-RNG backend live, Zuse mint/attach on real media. The peripheral-RNG backend itself
|
||
is **not yet built** — today's `rng_get_bytes()` (`src/starkernel/rng/rng.c`) only has a
|
||
`virtio-rng` path, real on QEMU, meaningless on real Pi 5 hardware (no virtio device there).
|
||
|
||
**Decided in conversation, 2026-09-04:**
|
||
- **Observation: HDMI-only for this board's own bring-up.** No second Pi, no dedicated
|
||
USB-serial adapter available. The Milk-V Mars could in principle serve as a GPIO-UART
|
||
bridge once it arrives (same 40-pin-header shape as the SER5 plan), but using it to observe
|
||
the Pi 5 before the Mars has been independently validated itself would be a chicken-and-egg
|
||
dependency, not a real plan. Revisit serial capture later if genuinely needed, once at least
|
||
one board is proven working — not a blocker for this pass.
|
||
- **Both boards (Pi 5, Milk-V Mars) arrive 2026-09-17.** Real runway exists to finish the
|
||
design/code work below *before* any hardware is in hand — "plan well before doing," per
|
||
direct instruction.
|
||
|
||
**Still genuinely open, not yet decided:**
|
||
- **A pinned GPIO VM, theory-stage** — see `FABRIC-4.md` §2. Raised in conversation, not yet
|
||
scoped; downstream of §IV.1's own native-boot-flow work (a GPIO VM needs GPIO addresses
|
||
from the DTB the same way the rest of this bring-up does).
|
||
|
||
### IV.1 — Boot-chain decision: UEFI vs. native, researched 2026-09-04
|
||
|
||
**Researched, not assumed** (web search, current as of this session):
|
||
|
||
**UEFI option investigated and found weak.** A real UEFI+ACPI firmware for Pi 5 exists —
|
||
[`rpi5-uefi`](https://github.com/worproject/rpi5-uefi) (TF-A + EDK2, SBBR-compliant). But:
|
||
it's **archived as of 2025-02-04**, support ended because newer Pi EEPROM firmware broke
|
||
compatibility with it; its own README says ACPI support is "under development and limited to
|
||
a few devices"; RP1 Ethernet/GPIO/PWM/EEPROM don't work under it. This kernel's whole
|
||
aarch64 boot path (`boot/uefi_loader.c`, `BootInfo->acpi_table`) assumes UEFI+ACPI the same
|
||
way amd64 and the QEMU aarch64 target do — but that assumption may not hold on a real,
|
||
current-firmware Pi 5 at all.
|
||
|
||
**Native boot flow — the real alternative, researched concretely:**
|
||
- Boot partition needs `bcm2712-rpi-5-b.dtb`, `config.txt`, and the kernel image itself —
|
||
Pi 5 firmware defaults to loading `kernel_2712.img`, falling back to `kernel8.img` if that's
|
||
absent.
|
||
- `config.txt` needs `os_check=0` for a non-Linux image, or the firmware assumes Linux and
|
||
loads from `0x200000` instead of the classic Pi bare-metal load address `0x80000`.
|
||
- Entry protocol: `x0` = 32-bit DTB pointer (upper 32 bits of the 64-bit register
|
||
unspecified — must mask before use), `x1`–`x3` reserved/zero. **No UEFI PE loader, no ACPI
|
||
at all** — a completely different entry shape from `boot/uefi_loader.c`.
|
||
- Framebuffer: the VideoCore **mailbox property interface** (channel 8) — a real, different
|
||
mechanism from UEFI GOP, no precedent anywhere in this codebase today.
|
||
|
||
**Decision, per direct instruction 2026-09-04: native boot flow.** Not UEFI. The archived,
|
||
partially-working UEFI project is too fragile a foundation to build a real-hardware release
|
||
on top of.
|
||
|
||
**What this actually means for the codebase, named honestly rather than estimated small:**
|
||
- A **new, non-UEFI entry path** for aarch64 real hardware — this kernel's boot sequence
|
||
currently assumes `uefi_loader.c`'s PE-loader shape unconditionally on aarch64; a Pi 5
|
||
native boot needs its own entry point (linked at `0x80000`, receiving `x0` = DTB pointer
|
||
directly, no `BootInfo` from UEFI at all).
|
||
- A **DTB-driven `BootInfo` equivalent** replacing ACPI-sourced data for this path — memory
|
||
map, peripheral addresses (UART, etc.) all come from the devicetree instead.
|
||
- **One real, genuine piece of reusable groundwork**: `starkernel/hal/fdt.c`/`fdt.h`, the
|
||
minimal FDT reader already built for riscv64's `timebase-frequency` lookup
|
||
(`arch/riscv64/timer.c`), is directly extensible for this — parsing `bcm2712-rpi-5-b.dtb`
|
||
for peripheral addresses is the same kind of lookup, not a new mechanism.
|
||
- A **new mailbox-property-interface framebuffer driver** — genuinely new code, no existing
|
||
precedent in this codebase, needed before the VT100 console framework
|
||
(`console.c`/`vt100.c`/`framebuffer.c`) has anything to draw onto for this board.
|
||
- This is a real architectural fork for aarch64, not a small per-board addition — QEMU
|
||
aarch64 keeps its existing UEFI+ACPI path unchanged; Pi 5 real hardware gets a second,
|
||
parallel entry path. **Not yet scoped into a punch list** — that's the next step, once this
|
||
fork's own shape (how much of `kernel_main.c`'s post-entry sequence can stay shared between
|
||
the two paths vs. needs its own branch) is thought through.
|
||
|
||
### IV.2 — Peripheral RNG: unresolved, not just under-researched
|
||
|
||
`ROADMAP.md` names an "aarch64 peripheral-RNG backend" as part of `v2.4.0`'s gate. Researched
|
||
directly rather than assumed still-TODO: Broadcom's `iproc-rng200` block (real, on Pi 4/BCM2711
|
||
as `brcm,bcm2711-rng200`) has **no `bcm2712` compatible-string entry anywhere in current
|
||
mainline Linux** (checked the actual driver's `of_device_id` table directly). The RP1
|
||
companion chip's own published peripheral list (GPIO/USB/Ethernet/DMA/ADC/PLLs/SRAM/
|
||
UARTs/SPIs) doesn't mention an RNG either. Two real possibilities, not yet distinguished:
|
||
BCM2712 still has the RNG200 block but Linux hasn't wired it into a devicetree binding yet, or
|
||
it genuinely isn't exposed to the ARM cores this generation. No public register address exists
|
||
to target right now — this needs either a Broadcom datasheet (if one becomes available) or
|
||
direct hardware probing once the board is in hand (scan the known BCM2711 RNG200 offset region
|
||
on the BCM2712 memory map and see if anything responds — risky without a datasheet confirming
|
||
it's safe to touch, so likely a "board in hand, careful probe" task, not a today task).
|
||
Deliberately **not a blocker for the first native boot** — reaching `ok>` doesn't require a
|
||
live entropy backend; `rng_get_bytes()` already has a "no entropy backend available" WARNING
|
||
path (`rng.c`) rather than a hard failure, so this can land after boot succeeds.
|
||
|
||
### IV.3 — Punch list: design/code work, no hardware needed (before 2026-09-17)
|
||
|
||
Traced against real code before writing this, not estimated: `boot_info->acpi_table`'s only
|
||
aarch64-relevant consumers today are `pci_init()` (`kernel_main.c:589`, unconditional, not
|
||
amd64-gated — relevant because RP1 is PCIe-attached on real Pi 5 hardware) and this session's
|
||
own `running_under_hypervisor()` (`arch/aarch64/timer.c`, already degrades safely to "not a
|
||
hypervisor" when `acpi_table` is `NULL` — no fix needed there).
|
||
`ioapic_init()`/`i8042_init()` are `#ifdef ARCH_AMD64`-gated, irrelevant here.
|
||
`arch/aarch64/apic.c` (GIC init) already only ever tries `boot_info->dtb`, never
|
||
`acpi_table` — its own doc comment already anticipated DTB-based discovery, just blocked
|
||
until now because QEMU's own UEFI firmware never publishes one; Pi 5 native boot removes that
|
||
blocker for free.
|
||
|
||
1. **Entry stub — DONE 2026-09-04.** New
|
||
`src/starkernel/arch/aarch64/native_rpi5_entry.S` / `include/starkernel/rpi5_native_entry.h`:
|
||
`rpi5_native_start` masks `x0` down to the documented 32-bit DTB-pointer range (§IV.1: the
|
||
firmware leaves the upper 32 bits of the register unspecified), stores it into
|
||
`g_rpi5_dtb_ptr` for item 2's still-open constructor to read, then switches `sp` to a
|
||
dedicated 2 MiB BSS stack (this path has no EDK2 boot stack to inherit — there is no EDK2 at
|
||
all here, unlike every other entry path this codebase has). Intentionally halts (`wfe`/`b`
|
||
loop) afterward rather than tail-calling into item 2's constructor, which doesn't exist yet.
|
||
**Not yet linked at `0x80000`** — that needs its own linker script/build target (item 6's own
|
||
`config.txt` work is the sibling piece; the separate-image build itself is not scoped into
|
||
this item). Now tail-calls item 2's constructor (below) instead of halting — updated
|
||
2026-09-04 when that item landed. Verified 3-arch boot to `ok>`/`zuse)ok>` —
|
||
`Makefile.starkernel`'s `KERNEL_ASM` wildcards every `*.S` in `arch/aarch64/`, so this file
|
||
compiles and links into the existing QEMU/UEFI acceptance build as dead code (unreferenced
|
||
symbol, nothing there ever branches to it), same as `rpi5_dtb.c`/`rpi5_mailbox.c` before it.
|
||
2. **DTB → `BootInfo` constructor — DONE 2026-09-04.** New
|
||
`include/starkernel/rpi5_native_boot.h` / `src/starkernel/arch/aarch64/rpi5_native_boot.c`:
|
||
`rpi5_native_boot()` populates the *existing* `BootInfo` struct from the devicetree instead
|
||
of UEFI protocols (`dtb` = the real pointer, `acpi_table`/`runtime_services` = `NULL`,
|
||
`kernel_stack_base` = `NULL`/BSS-fallback — aarch64 has no `kernel_entry.S` trampoline at
|
||
all, so item 1's own BSS stack already *is* the stack `kernel_main_impl` runs on),
|
||
`args` via `/chosen`'s `bootargs` fed straight into the *existing*
|
||
`cmdline_parse_ascii()` (pure C99, no UEFI coupling — confirmed before reusing it, not
|
||
assumed), `framebuffer` via item 3's `rpi5_mailbox_get_framebuffer()` at a fixed
|
||
1920x1080x32 default (no EDID query exists in this codebase — flagged, not guessed past
|
||
this comment, revisit once real hardware and a real attached display are in hand), then
|
||
calls the **existing, unmodified** `kernel_main()` — this is the crux of why most of M1–M9
|
||
stays shared. `memory_map` comes from `/memory`'s own `reg`, honoring the *root* node's
|
||
`#address-cells`/`#size-cells` (confirmed against `bcm2712.dtsi`'s actual root node — `<2>`/
|
||
`<2>` — not assumed; a hardcoded-wrong cell width here would compile clean and boot clean in
|
||
QEMU while silently corrupting the real memory map on real silicon, so this was verified
|
||
from the source rather than recalled). Required a new `fdt_find_node_by_device_type()`
|
||
(`fdt.c`/`fdt.h`) since `/memory` is identified by `device_type = "memory"` per DT spec
|
||
§3.4, not `compatible`. Required a Makefile fix: `boot/cmdline.c` was only in `LOADER_SRCS_BASE`
|
||
(the `.efi` target), not `KERNEL_SRCS_BASE` (the separate `.elf` target `arch/aarch64/*.c`
|
||
also wildcards into) — added it there too, a real link failure caught before it could ship.
|
||
**`/reserved-memory` carving — DONE 2026-09-04.** Originally deferred here as needing
|
||
interval-splitting logic written blind against hardware not yet in hand — revisited once an
|
||
actual reservation was confirmed to exist rather than assumed either way: fetched
|
||
`bcm2712-ds.dtsi` directly and found a real `reserved-memory` node with one static child
|
||
(`atf@0`, `reg`-addressed, ARM Trusted Firmware's own region) and one dynamic child
|
||
(`linux,cma`, `size`/`alloc-ranges` only, no fixed address — skipped, nothing fixed to carve
|
||
and no allocator this early to service it against anyway). `collect_reserved_ranges()`
|
||
walks `reserved-memory`'s children via two new `fdt.c` primitives —
|
||
`fdt_find_node_by_name()` (needed since `/reserved-memory` has neither `compatible` nor
|
||
`device_type` per DT spec §3.5.4) and `fdt_next_child_node()` (one exported symbol, not the
|
||
two-primitive general sibling-walker originally sketched — collapsed after review, since
|
||
this codebase's only real use is "iterate one node's direct children," not general tree
|
||
navigation) — reading each child's own `#address-cells`/`#size-cells` with a fallback to
|
||
root's only if absent (confirmed necessary, not just defensive:
|
||
`reserved-memory`'s own declared `<2>`/`<1>` genuinely differs from root's `<2>`/`<2>`).
|
||
`emit_region_with_carveouts()` clips a sorted reserved-range list against each RAM region,
|
||
emitting alternating `EfiConventionalMemory` gaps and `EfiReservedMemoryType` carve-outs
|
||
(insertion sort, not `qsort` — freestanding, no libc). `no-map`/`reusable` flags are not
|
||
distinguished; every static reservation is excluded from `EfiConventionalMemory` regardless.
|
||
`RPI5_MAX_MEMMAP_ENTRIES` is the exact worst-case count
|
||
(`RAM_REGIONS * (2*RESERVED_RANGES + 1)`), recomputed rather than estimated — the
|
||
`rpi5_mailbox.c` buffer-size bug is the standing lesson for this pattern. Verified 3-arch
|
||
boot to `ok>`/`zuse)ok>` — compile-only, same caveat as every item in this list: nothing in
|
||
the existing UEFI/QEMU path calls `rpi5_native_boot()`, so this cannot be exercised until
|
||
real hardware.
|
||
3. **Mailbox-property-interface framebuffer driver — DONE 2026-09-04.** New
|
||
`include/starkernel/rpi5_mailbox.h` / `src/starkernel/arch/aarch64/rpi5_mailbox.c`:
|
||
`rpi5_mailbox_get_framebuffer()` builds and sends one property-tag buffer (phys size, virt
|
||
size, depth, pixel order, virtual offset, allocate-buffer, get-pitch), populating an
|
||
`Rpi5FramebufferInfo` kept in exact field-for-field sync with `uefi.h`'s `FramebufferInfo`
|
||
so `console.c`/`vt100.c`/`framebuffer.c` need no changes downstream. Register layout
|
||
(`+0x00`/`+0x18` MBOX0 read/status, `+0x20`/`+0x38` MBOX1 write/status) confirmed against a
|
||
Pi-5-specific bare-metal reference (main.lv), independently cross-checked against this
|
||
codebase's own `rpi5_dtb.c` translated base address — two independent sources agreeing. A
|
||
real buffer-overflow bug was found and fixed before compiling (the static request buffer
|
||
was sized 32 words against an actual 35-word requirement, recomputed exactly rather than
|
||
re-estimated; resized to 40 words for margin). **Two things flagged, not guessed, as
|
||
genuinely unverified against real hardware:** the `TAG_ALLOCATE_BUFFER` tag's request-size
|
||
field value (set to the response size, matching common practice across surveyed reference
|
||
implementations, not a single spec-quoted number); and whether the allocate-buffer
|
||
response address needs the classic `& 0x3FFFFFFF` bus-alias masking on Pi 5 specifically —
|
||
kept defensively even though the same Pi-5-specific send-side reference found no bus-alias
|
||
bit in play there. Verified 3-arch boot to `ok>`/`zuse)ok>` (compile-only — no caller yet;
|
||
that's the entry-stub/DTB-constructor items above, still open).
|
||
4. **`fdt.c`/`fdt.h` extension — DONE 2026-09-04.** Added `fdt_find_node_by_compatible()`
|
||
(matches any entry in a node's NUL-separated `compatible` list, first match in document
|
||
order) and `fdt_find_prop_in_node()` (scoped to that one node's own direct properties only
|
||
— stops at the first child node or the node's own end, never descends or continues into a
|
||
sibling). Same minimal, non-tree-building style as the existing reader — no new state, no
|
||
allocation, one linear scan per call. Verified 3-arch boot to `ok>`.
|
||
|
||
**4a. UART + mailbox address lookup — DONE 2026-09-04.** New
|
||
`include/starkernel/rpi5_dtb.h` / `src/starkernel/arch/aarch64/rpi5_dtb.c`:
|
||
`rpi5_uart_base()`/`rpi5_mailbox_base()`, each `fdt_find_node_by_compatible()` (`"arm,pl011"`
|
||
/ `"brcm,bcm2835-mbox"`) → `fdt_find_prop_in_node(..., "reg", ...)`. **A real translation
|
||
gap found and fixed before this could have been silently wrong**: confirmed directly
|
||
against `bcm2712.dtsi` (raspberrypi/linux) that both peripherals live under one `soc`
|
||
simple-bus node whose `ranges` property adds a fixed `0x10_0000_0000` offset to every
|
||
child `reg` value — `fdt.c`'s reader deliberately does not apply `ranges` translation
|
||
generally (not a general devicetree library), so this file applies that one, fixed,
|
||
SoC-wide offset explicitly by name (`BCM2712_SOC_RANGES_OFFSET`), documented with the exact
|
||
devicetree excerpt that confirmed it. Verified 3-arch boot to `ok>` (compile-only — these
|
||
two functions have no caller yet; that's the entry-stub/framebuffer-driver items above,
|
||
still open).
|
||
5. **`pci_init()` DTB path — RE-SCOPED 2026-09-04, this item's own original text was wrong.**
|
||
Investigated before writing any code (per this document's own discipline) rather than
|
||
building the originally-assumed "swap the ACPI MCFG lookup for a DTB one, same flat-ECAM
|
||
access underneath" version — that assumption does not hold, confirmed against three primary
|
||
sources, not guessed:
|
||
- `bcm2712-rpi-5-b.dts` (real board file): RP1 is attached under `pcie2`
|
||
(`pcie2: pcie@1000120000`, `compatible = "brcm,bcm2712-pcie"`) via
|
||
`rp1_target: &pcie2 { status = "okay"; };`.
|
||
- `pcie2`'s own `reg` is `<0x10 0x00120000 0x00 0x9310>` — a ~37 KB window, far too small to
|
||
be a flat 256 MB ECAM region (256 buses × 1 MB) the way `pci.c`'s existing `ecam_*`
|
||
functions assume.
|
||
- The real Linux driver for `"brcm,bcm2712-pcie"` (`drivers/pci/controller/pcie-brcmstb.c`)
|
||
confirms why: `brcm_pcie_map_bus()` computes a standard ECAM-shaped offset
|
||
(`PCIE_ECAM_OFFSET(bus, devfn, 0)`) but does **not** read/write it directly — it writes
|
||
that offset to an `IDX_ADDR` index register, then accesses the config data through a small
|
||
windowed `DATA_ADDR` region. Root-complex-local accesses (`devfn == 0` on the root bus)
|
||
skip the indirection and hit `base + PCIE_ECAM_REG(where)` directly; every downstream
|
||
device access (RP1 included) goes through the index/data window.
|
||
- This is a real **third config-access mechanism**, not a bigger version of "find the ECAM
|
||
base in the DTB instead of ACPI." `pci.c`'s `cfg_read32`/`cfg_write32` family
|
||
(`pci.c:188–232`) today dispatches exactly two ways, selected by `#ifdef ARCH_AMD64` vs.
|
||
everything else (flat `ecam_*`) — and that `#ifdef` selection is itself now wrong for this
|
||
case: QEMU aarch64 (flat ECAM via ACPI MCFG, real and working) and Pi 5 aarch64
|
||
(Broadcom indirect windowing) are the **same build** (`ARCH=aarch64`), so the dispatch
|
||
must become a runtime choice, not a compile-time one — a real change to a shared file all
|
||
three architectures currently boot clean through, not a small addition.
|
||
- **Secondary finding, corrected 2026-09-05 — not a current blocker.** This item originally
|
||
claimed native boot's lack of a UEFI identity map means "**any** PCIe work on this path...
|
||
needs its own explicit `vmm_map_range()` call." Traced against actual source rather than
|
||
assumed: `arch_mmu_init()` is a stub on both non-amd64 ISAs (aarch64 and riscv64 alike,
|
||
`arch/{aarch64,riscv64}/arch.c`), `vmm.c`'s own `load_cr3()` is a no-op outside
|
||
`__x86_64__`, and — confirmed directly by this session's own riscv64 satp fix (§V.3
|
||
item 7) — `satp` is explicitly left at Bare mode on that ISA. **No non-amd64 ISA ever
|
||
turns its own MMU on today**, native or QEMU/UEFI boot alike; with translation off,
|
||
VA==PA everywhere and there is nothing for a mapping call to do. The concern is real but
|
||
conditional — it becomes a genuine prerequisite only once a future milestone actually
|
||
brings up aarch64/riscv64 paging (`arch_mmu_init()`'s own stub comment: "deferred to a
|
||
later milestone"), not before.
|
||
- **Also checked while re-scoping, not previously stated: no silent-wrong-answer risk
|
||
today.** With no ACPI table on the native Pi 5 boot path, `pci_init()`'s existing MCFG
|
||
parse fails cleanly (`rc != 0`) and aarch64 takes no ECAM fallback (unlike riscv64's own
|
||
QEMU-constant fallback) — it already prints `"PCI: MCFG parse failed on aarch64 — no
|
||
ECAM"` and stops, rather than misreading RP1's 37 KB indirect-window `reg` as if it were
|
||
flat 256 MB ECAM. No diagnostic gap to close here.
|
||
- **Still not implemented, and the runtime-dispatch prerequisite re-examined 2026-09-05 —
|
||
not separable from the Broadcom mode itself.** Considered building the "make dispatch
|
||
runtime, not compile-time" half alone (reusing the DTB node-scoped lookup primitive that
|
||
closed the aarch64 GIC base and riscv64 PLIC base findings, §IV.3/§V.3 item 7 and item 3)
|
||
— rejected on inspection, not scope: `portio_read32()` and its siblings use `outl`/`inl`
|
||
inline asm that cannot exist in an aarch64 or riscv64 translation unit at all, so any
|
||
`case PCI_CFG_MODE_PORTIO:` referencing them must itself stay `#ifdef ARCH_AMD64`-gated.
|
||
That leaves exactly one reachable case per architecture's build either way — an enum
|
||
wearing the same compile-time selection, fixing nothing. The dispatch can only become
|
||
genuinely runtime once a second real non-amd64 mechanism is actually compiled in, which
|
||
means this "prerequisite" and the Broadcom mode below are one change, not two sequenceable
|
||
ones.
|
||
- **Broadcom indirect access: not implemented, reason is unverifiability and incompleteness,
|
||
not size.** No QEMU model exists for `"brcm,bcm2712-pcie"` — no boot on this system can
|
||
execute a new branch built for it, so unlike the GIC/PLIC DTB-discovery fixes (guard
|
||
exercised, success branch pending hardware) this would be *zero* branches ever exercised
|
||
before 2026-09-17. Separately, config-space access alone is not sufficient for RP1 to
|
||
enumerate: `pcie-brcmstb.c`'s real driver does controller bring-up (link training, PERST,
|
||
inbound/outbound window setup) before `brcm_pcie_map_bus()` returns anything but
|
||
`0xFFFFFFFF` — implementing only the `IDX_ADDR`/`DATA_ADDR` index-window mechanism would
|
||
compile and boot cleanly while silently never working, exactly the half-finished
|
||
implementation this project's own conventions rule out. Recording the real shape remains
|
||
this pass's deliverable; building it needs real Pi 5 + RP1 hardware to verify against, not
|
||
available until 2026-09-17.
|
||
6. **`config.txt` contents — DONE, 2026-09-04.** Written to `boot_media/rpi5/config.txt`
|
||
(new directory — `configs/` is Kconfig defconfigs, `img/` is banners/docs, neither fits).
|
||
Researched against the official current `config.txt` reference
|
||
(raspberrypi.com/documentation/computers/config_txt.html) and one real, working Pi 5
|
||
bare-metal project's own checked-in `config.txt` (`leopoldch/BatMetal`), not assumed from
|
||
general Pi knowledge. Final contents, with the reasoning as comments in the file itself:
|
||
- `kernel=kernel_2712.img` — the real Pi-5-specific default filename firmware looks for
|
||
first (falls back to `kernel8.img` if absent); naming our image this exactly removes
|
||
ambiguity, so the item's own `kernel8.img`/`os_check=0` fallback alternative wasn't
|
||
needed.
|
||
- `os_check=0` — official docs name "bare-metal development" as the explicit use case for
|
||
this flag. Kept despite BatMetal's own config.txt *not* setting it and still booting —
|
||
recorded as an open discrepancy rather than silently resolved; no hardware in hand yet to
|
||
confirm which behavior is actually gated.
|
||
- `device_tree=bcm2712-rpi-5-b.dtb` — pins the DTB explicitly rather than relying on
|
||
board-revision auto-selection (confirmed via WebSearch of `config.txt` syntax docs).
|
||
- `enable_uart=1` — **correction to this item's own original text, which didn't have
|
||
this.** Added per review: costs nothing when nothing is listening, and is the only
|
||
diagnostic channel that survives a framebuffer failure on the first real boot (the
|
||
mailbox framebuffer path, `rpi5_native_boot.c`, has two documented-unverified details
|
||
and has never run on silicon). Enables the Pi 5's dedicated JST debug UART, not the
|
||
GPIO 14/15 header (`dtoverlay=uart0`, deliberately left out — needs a USB-serial
|
||
adapter this HDMI-only bring-up doesn't have). `rpi5_uart_base()` already exists
|
||
and is currently uncalled — this is the hook a UART fallback console would use.
|
||
- `arm_64bit=1` — **dropped, correction to this item's own original text.** Official
|
||
docs: "Models that only support a 64-bit kernel ignore this flag" — confirmed inert on
|
||
Pi 5, not wrong to include, just meaningless.
|
||
- `dtparam=pciex1` — researched (BatMetal's own comment: "Required for RP1 access"), not
|
||
included. Community-reported only, not confirmed against official docs, and surprising
|
||
if true (RP1 is normally always-on). Not needed yet — item 5's real RP1/PCIe work is
|
||
still re-scoped, not implemented. Revisit if that work needs it.
|
||
- `boot_media/rpi5/README.md` also added, noting the one remaining gap this item's
|
||
`config.txt` exposes but doesn't itself close: `kernel_2712.img` names a file nothing
|
||
currently builds — item 1's separate-image build target (linking at `0x80000`) is still
|
||
outstanding future work, same gap that item's own text already flagged.
|
||
|
||
7. **Code audit pass — DONE, 2026-09-04.** Reviewed `arch/aarch64/apic.c`, `arch.c`,
|
||
`timer.c`, `interrupts.c` for the same class of QEMU-virt-vs-real-hardware assumption
|
||
§III item 6's amd64 audit looked for. Report-only by default, per this project's own
|
||
"identify, don't fix unless asked" rule — **all three findings below were fixed in code
|
||
across two follow-up rounds, each named specifically by direct instruction after this
|
||
audit landed**: the GIC base address (severe), the stale VBAR_EL2 doc comment (doc-only),
|
||
and the hardcoded HVC PSCI conduit. §V.3's own parallel audit found a fourth finding of
|
||
this same class (riscv64's `satp`-clear reasoning, lowest severity of the whole set) that
|
||
remains report-only, not asked for. See each finding's own
|
||
own text for what changed.
|
||
|
||
**Finding — severe, confirmed live, blocked reaching `ok>` on real hardware as the code
|
||
stood — FIXED 2026-09-04, per direct instruction (this item's own findings are normally
|
||
report-only; this one was explicitly asked for by name).** `apic.c`'s own file header
|
||
already self-documented `GICD_BASE_PA`/`GICC_BASE_PA` (`0x08000000`/`0x08010000`) as
|
||
"QEMU-virt-machine constants... a deliberate, recorded exception," reasoned correctly at
|
||
the time it was written: no DTB was ever available to discover them from, because QEMU's
|
||
own aarch64 UEFI firmware doesn't forward one. That premise no longer held — the native
|
||
boot path (item 2, DONE) receives a real DTB directly and calls the same, unmodified
|
||
`kernel_main()` M4 sequence, which calls `apic_init(boot_info)` unconditionally
|
||
(confirmed: `kernel_main.c:413`, no arch- or boot-path gating) — `apic_init()` itself
|
||
ignored `boot_info` entirely and always programmed the QEMU addresses. The real BCM2712
|
||
GIC-400 is at `0x10_7fff9000` (distributor) / `0x10_7fffa000` (CPU interface 0) —
|
||
confirmed directly against `bcm2712.dtsi`'s own `axi`/`gicv2` nodes
|
||
(`compatible = "arm,gic-400"`, `#address-cells = <2>`/`#size-cells = <2>`, `axi`'s own
|
||
`ranges` identity-mapping this region — no offset needed, unlike `rpi5_dtb.c`'s
|
||
1-cell/offset `soc` peripherals), not recalled.
|
||
**Fix**: `apic.c` now tries `gic_bases_from_dtb()` first — `fdt_valid(dtb)` →
|
||
`fdt_find_node_by_compatible(dtb, "arm,gic-400")` → `fdt_find_prop_in_node(..., "reg",
|
||
...)`, reading the first two 2-address-cell/2-size-cell entries (GICD, then GICC — the
|
||
standard arm,gic-400 binding order) — and falls back to the QEMU constants only when no
|
||
DTB or no matching node is found, so the existing QEMU/UEFI path's behaviour (verified via
|
||
the mandatory 3-arch boot) is unchanged. `GICD_BASE_PA`/`GICC_BASE_PA` became
|
||
`s_gicd_base`/`s_gicc_base` (module-static `uintptr_t`, not `#define`s, since they're no
|
||
longer compile-time constants on this path) — every MMIO call site (`apic_init()`,
|
||
`apic_spi_enable()`, `apic_read_iar()`, `apic_eoi_intid()`) now reads through them.
|
||
The 3-arch acceptance run exercised the *guard*, not just compiled it: `apic_init()`'s
|
||
`gic_bases_from_dtb()` call runs unconditionally on every aarch64 boot, including this
|
||
one, and its `fdt_valid()` check correctly returned 0 on this system's QEMU/UEFI firmware
|
||
— confirmed via the boot log's own `GICv2: no DTB GIC node -- using QEMU virt-machine
|
||
defaults` line, followed by the unchanged `distributor+CPU interface enabled, PPI 30`.
|
||
What remains genuinely unexercised is the *success* branch (a real DTB with a matching
|
||
`"arm,gic-400"` node) — that needs a devicetree this build never has, so it stays
|
||
unverified until real Pi 5 hardware, same caveat as every other native-path item in this
|
||
list.
|
||
**Finding — doc-only, not a functional gap — FIXED (comment corrected) 2026-09-04, per
|
||
direct instruction naming it specifically.** `arch_interrupts_init()`'s own doc comment
|
||
in `interrupts.c` claimed "`VBAR_EL1` is written unconditionally regardless of the
|
||
detected level... if EL2, this is a known gap" — checked against `isr.S`
|
||
(`aarch64_install_vectors`, lines 124–146) and found **stale**: the actual implementation
|
||
already branches on `aarch64_current_el()` and writes `vbar_el2`/`vbar_el1` correctly,
|
||
with `el2_mode_flag` threading the same answer through the IRQ trampoline for
|
||
`ELR_EL1`/`SPSR_EL1` vs `ELR_EL2`/`SPSR_EL2` selection. Rewrote the doc comment and the
|
||
matching runtime console message (which made the identical stale claim — "not yet
|
||
wired, see item 0.5/0.7") to describe what `isr.S` actually does, and fixed a
|
||
same-vintage one-word staleness in `arch.c`'s own `aarch64_install_vectors` extern
|
||
comment ("installs `VBAR_EL1`" → EL-aware) caught while touching this. No behavior
|
||
changed anywhere — this was purely three comments correcting themselves to match code
|
||
that was already right. Which EL the Pi 5's ATF hands the kernel off at under the native
|
||
path is itself still unconfirmed (the `atf@0` reserved-memory region found during item
|
||
2's work only establishes that EL3 firmware exists, not which EL the kernel lands at) —
|
||
moot for this specific fix now that both paths are correctly wired regardless of which EL
|
||
is chosen, but worth knowing for other EL-dependent code.
|
||
**Finding — same defect class as amd64's `arch_cold_reset()` finding, on the QEMU side of
|
||
this arch specifically — FIXED 2026-09-04, per direct instruction naming it
|
||
specifically.** `arch/aarch64/arch.c`'s `arch_cold_reset()` issued PSCI `SYSTEM_RESET` via
|
||
`HVC` unconditionally — a fix already documented in-file as a QEMU-specific workaround,
|
||
because QEMU's AAVMF firmware has no genuine EL3/TrustZone secure monitor to answer an
|
||
`SMC`. The Pi 5's real ATF (confirmed present via the `atf@0` reservation) is exactly the
|
||
kind of genuine EL3 firmware PSCI's `SMC` conduit assumes — confirmed further this pass:
|
||
`bcm2712.dtsi`'s own `/psci` node declares `method = "smc"` directly, not inferred from
|
||
ATF's mere presence. **Fix**: new `aarch64_psci_conduit_init(dtb)` (`arch.c`) —
|
||
`fdt_valid(dtb)` → `fdt_find_node_by_compatible(dtb, "arm,psci-1.0")` →
|
||
`fdt_find_prop_in_node(..., "method", ...)` — sets a module-static `s_psci_use_smc` flag
|
||
to 1 only when the DTB's own `method` property reads exactly `"smc"`; every other
|
||
outcome (no DTB, no PSCI node, `method="hvc"`, property absent) leaves it at its default
|
||
0, preserving the exact HVC behaviour this system's QEMU boot already relies on.
|
||
`arch_cold_reset()` now branches on that flag between the `smc #0`/`hvc #0` trap
|
||
instructions; the function ID and calling convention are identical either way, matching
|
||
the file's own existing comment about that. Called once from `apic_init()`
|
||
(`arch/aarch64/apic.c`), the one point in boot with `boot_info->dtb` already in hand —
|
||
collocated in `arch.c`, not `apic.c`, because `arch_cold_reset()` is the actual consumer
|
||
and has no `boot_info` of its own by the time it runs (called from deep in VM execution,
|
||
via `BYE`). Riscv64's equivalent (`arch/riscv64/arch.c`, SBI SRST) remains the one cold-
|
||
reset implementation across all three architectures that never needed this kind of fix —
|
||
still worth naming as the standard the other two moved toward, one now fixed, amd64's
|
||
i8042-pulse version still open. 3-arch acceptance exercised the guard (this system's QEMU
|
||
AAVMF forwards no DTB, so `aarch64_psci_conduit_init()` hits its own "no DTB" branch and
|
||
`arch_cold_reset()` keeps using HVC, unchanged) but not the `SMC` success branch itself —
|
||
that stays unverified until real Pi 5 hardware runs `BYE`.
|
||
|
||
**Hardware-dependent, after 2026-09-17 (not started until then):**
|
||
8. Build the boot media (SD card: `config.txt`, `bcm2712-rpi-5-b.dtb`, kernel image).
|
||
9. Connect HDMI + keyboard (observation decision above).
|
||
10. Boot; confirm `ok>`/`zuse)ok>` reached.
|
||
11. Mint a Zuse identity on real media, confirm re-attach — the `v2.4.0` gate's own
|
||
requirement, same shape as amd64's.
|
||
12. Update this section with results before moving to riscv64's own hardware-dependent steps.
|
||
|
||
## V. riscv64 — Milk-V Mars
|
||
|
||
**Already true:** `ROADMAP.md` names this (generically, "Milk-V") as part of `v2.5.0`'s gate:
|
||
boots on the real board, the Zkr (RNDR) entropy backend live. Same gap as aarch64:
|
||
`rng_get_bytes()` has no riscv64 hardware-RNG path today, only `virtio-rng`.
|
||
|
||
### V.1 — Boot chain: resolved, researched 2026-09-04
|
||
|
||
**Resolved, not left open.** The Mars is a documented mainline U-Boot board target in its own
|
||
right ([U-Boot docs — Milk-V Mars](https://docs.u-boot.org/en/latest/board/starfive/milk-v_mars.html)),
|
||
and it uses **the exact same U-Boot binaries as the StarFive VisionFive 2** — same SoC
|
||
(StarFive JH7110), board identity detected at SPL time, devicetree patched accordingly, no
|
||
separate Mars-specific firmware. This directly answers §V's own previously-open question:
|
||
**U-Boot + OpenSBI + devicetree, not UEFI** — same fork this kernel already decided for
|
||
aarch64 (§IV.1), now confirmed for riscv64 too.
|
||
|
||
**Boot chain, concretely:**
|
||
1. BootROM (ZSBL), StarFive's on-chip loader at `0x2A000000`, selects boot media by GPIO pins.
|
||
2. U-Boot SPL (FSBL) — initializes DRAM, configures PLLs.
|
||
3. OpenSBI (`fw_dynamic.bin`) — M-mode runtime services.
|
||
4. U-Boot main, S-mode, depends on OpenSBI.
|
||
5. Boot media: QSPI flash (recommended) or UART XMODEM (recovery). SD/eMMC boot modes are
|
||
deprecated in current U-Boot.
|
||
|
||
**Entry protocol, from real VisionFive 2 bare-metal work (same SoC, directly applicable per
|
||
§VI's own cross-reference):**
|
||
- Entry point `0x40000000`.
|
||
- Core identification via the `mhartid` CSR — the SiFive S7 monitor core is hart 0, the four
|
||
U74 application cores are harts 1–4 (matches the QEMU riscv64 target's own hart numbering
|
||
convention already assumed elsewhere in this codebase — worth double-checking, not
|
||
assuming, once real hardware is in hand).
|
||
- UART at `0x10000000`, 115200 baud, already initialized by firmware before handoff.
|
||
- Custom bare-metal images package via `vf2-imager` (invokes U-Boot's `mkimage`) into a FIT
|
||
image — same tooling should apply to the Mars, unconfirmed until tried.
|
||
- **Not yet found**: what registers carry the DTB pointer/hart ID at the actual kernel entry
|
||
point under this specific chain (the source consulted covered the image-packaging tooling,
|
||
not the OpenSBI→kernel handoff register convention). **Resolved 2026-09-04**: standard
|
||
RISC-V SBI boot protocol, confirmed via OpenSBI's own docs — `a0`=hart ID, `a1`=DTB pointer,
|
||
S-mode entry. Not chain-specific guesswork; this is the universal convention OpenSBI's
|
||
`FW_DYNAMIC` firmware type uses regardless of vendor, so it applies to this chain directly.
|
||
|
||
**What this means for the codebase** — same shape of fork as aarch64 (§IV.1): a non-UEFI
|
||
entry path, a DTB-driven `BootInfo` equivalent (the existing `starkernel/hal/fdt.c` reader
|
||
extends here too, same as for the Pi 5), no ACPI.
|
||
|
||
**Decided in conversation, 2026-09-04: observation is HDMI-only**, same reasoning and same
|
||
constraint as the Pi 5 (§IV) — no bridge hardware available for this board's own first
|
||
bring-up either; the Mars has its own HDMI 2.0 output (§VI).
|
||
|
||
### V.2 — Peripheral RNG and Zkr: still genuinely open
|
||
|
||
- Zkr/RNDR instruction availability on the Mars's actual CPU (riscv64 Scalar Crypto extension
|
||
support varies by implementation) — not yet confirmed; the VisionFive 2 bare-metal research
|
||
above didn't surface this either, would need its own targeted look (or a real-hardware
|
||
probe of `misa`/the Zkr extension discovery mechanism). Deliberately **not a blocker for
|
||
first boot**, same reasoning as §IV.2's aarch64 RNG gap — `rng_get_bytes()` already
|
||
WARNs rather than hard-fails with no backend.
|
||
- **Whether the Mars needs the same pinned-GPIO-VM treatment as the Pi 5** — explicitly
|
||
**not decided either way**, per direct instruction ("same for Milk-V (? not sure here)").
|
||
See `FABRIC-4.md` §2. The Mars does have its own 40-pin GPIO header (§VI), so the open
|
||
question is the VM architecture around it, not whether the hardware exists.
|
||
|
||
### V.3 — Punch list: design/code work, no hardware needed (before 2026-09-17)
|
||
|
||
Traced against real code before writing this, same discipline as §IV.3: `pci_init()`
|
||
(`kernel_main.c:589`, unconditional) is the one real `acpi_table` consumer relevant here too
|
||
— the Mars's M.2 E-Key slot (§VI) is PCIe-attached, same shape of gap as the Pi 5's RP1.
|
||
`riscv64/timer.c` is **already** fully DTB-driven (both `timebase-frequency` and this
|
||
session's own hypervisor-detection check) — no further work needed there; it was built DTB-
|
||
first from the start, unlike aarch64's timer which needed a new ACPI-based check today.
|
||
|
||
**One real, already-flagged risk found while tracing this — FIXED 2026-09-05, see item 3
|
||
below.** `arch/riscv64/apic.c`'s own doc comment said the PLIC base address was "a constant,
|
||
not discovered from `boot_info->dtb`" — and `arch/riscv64/plic.c`'s own doc comment
|
||
(predating this document) already warned `PLIC_BASE`/`PLIC_CONTEXT_S` are "QEMU-virt-
|
||
specific... not assumed stable across" other configurations. That warning was concrete: the
|
||
JH7110's real PLIC address on the Mars was not confirmed to match QEMU-virt's, and the
|
||
interrupt controller would not have worked correctly if it didn't.
|
||
|
||
1. **Entry stub**: new native riscv64 entry point at `0x40000000` (§V.1), receiving `a0`=hart
|
||
ID, `a1`=DTB pointer directly (now-confirmed SBI convention) — no UEFI, no PE loader.
|
||
2. **DTB → `BootInfo` constructor**: same shape as aarch64's (§IV.3 item 2) — `dtb`=real
|
||
pointer, `acpi_table`=`NULL`, memory map from DTB `/memory`+`/reserved-memory`, `args` from
|
||
`/chosen`/`bootargs`.
|
||
3. **PLIC base address: make it DTB-discovered — DONE 2026-09-05.** `plic_init()`
|
||
(`arch/riscv64/plic.c`) now takes `boot_info->dtb`, passed through from `apic_init()`
|
||
(`arch/riscv64/apic.c`), and tries `fdt_find_node_by_compatible(dtb, "sifive,plic-1.0.0")`
|
||
→ `fdt_find_prop_in_node(..., "reg", ...)` before falling back to the QEMU-virt constant —
|
||
exactly the primitive built for the aarch64 GIC base fix (§IV.3 item 7), reused unchanged.
|
||
`s_plic_base` is now a runtime `uintptr_t`, not a `#define`, same shape as apic.c's
|
||
`s_gicd_base`/`s_gicc_base`. **Correction, same day**: this document's own text at this
|
||
point originally predicted the success branch would run under QEMU, on the strength of a
|
||
stale claim in `plic.c`'s own pre-fix header ("riscv64 DTB access does work, unlike
|
||
aarch64") — checked against the actual boot log rather than trusted, and it doesn't hold
|
||
for this system's QEMU/UEFI riscv64 firmware specifically: `timer.c`'s own
|
||
`timebase-frequency` read falls back too (`"Timer: RISC-V time CSR @ 10000000 Hz
|
||
(FALLBACK, no devicetree)"`), and the new PLIC line confirms the same thing —
|
||
`"PLIC: no DTB PLIC node -- using QEMU virt-machine default (base=0x0c000000)"`. So this
|
||
fix is evidentially in the same position as the GIC fix: the guard (no-DTB fallback) is
|
||
exercised and correct, the success branch (a real DTB with a matching PLIC node) remains
|
||
unverified until real hardware. The stale claim that misled this paragraph's first draft
|
||
lived only in `plic.c`'s old header and was already replaced by this fix's own comment —
|
||
not re-introduced. The Mars's real JH7110 `reg` value stays unconfirmed either way — the
|
||
compatible string is architectural (SiFive PLIC-1.0.0 is a standard binding), not
|
||
board-specific, but no
|
||
hardware exists yet to check the Mars's own DTB against it.
|
||
4. **Framebuffer for HDMI output**: JH7110's display path is genuinely unresearched this
|
||
pass — unlike the Pi 5's mailbox interface (well-documented, reused across many Pi bare-
|
||
metal projects), no equivalent research done yet for JH7110's own display controller.
|
||
Flagged here rather than assumed simple.
|
||
5. **`pci_init()` DTB path — the "shares §IV.3 item 5's code" framing is now wrong, corrected
|
||
2026-09-04.** §IV.3 item 5's own investigation found the Pi 5's RP1 sits behind a
|
||
Broadcom-specific indirect config-access window (`"brcm,bcm2712-pcie"`, confirmed against
|
||
`pcie-brcmstb.c`), not flat ECAM — a SoC-specific mechanism with no reason to assume it
|
||
matches JH7110's own PCIe controller. The "one implementation, two consumers" plan this line
|
||
originally stated no longer holds; the Mars's M.2 E-Key slot needs its **own** primary-source
|
||
check of JH7110's actual PCIe controller (compatible string, `reg` window size, and whether
|
||
its Linux driver uses flat ECAM or its own indirect scheme) before any implementation
|
||
decision, same discipline §IV.3 item 5 itself just applied. Not yet done.
|
||
6. **Boot image packaging**: `vf2-imager`/`mkimage`-based FIT image (§V.1) — confirm this
|
||
tooling's actual invocation once building the first real image, not just cited from
|
||
VisionFive 2 research.
|
||
|
||
7. **Code audit pass — DONE, 2026-09-04.** Reviewed `arch/riscv64/apic.c`, `plic.c`,
|
||
`arch.c`, `interrupts.c`, `timer.c` for the same class of QEMU-virt-vs-real-hardware
|
||
assumption §III item 6's amd64 audit and item 7 above's aarch64 audit looked for.
|
||
Report-only by default — the `satp`-clear finding below was fixed in code, per direct
|
||
instruction naming it specifically after this audit landed. The other findings (PLIC
|
||
base, already tracked as item 3 above; the clean SBI/SBI-SRST findings) needed no code
|
||
change either way.
|
||
|
||
**Clean, and the most portable pattern of all three architectures**: the SBI timer path
|
||
(`apic.c`) is genuinely hardware-independent — it probes for the TIME extension at
|
||
runtime and reports loudly rather than assuming it's present, so it depends on nothing
|
||
but OpenSBI itself, present on both QEMU and (per §V.1's own research) the Mars's real
|
||
U-Boot+OpenSBI chain. `arch_cold_reset()` (`arch.c`) uses the SBI SRST extension — a real
|
||
standards-defined mechanism, not a board-specific hack — making it the one of the three
|
||
architectures' cold-reset implementations that does **not** need a finding here (contrast
|
||
amd64's i8042-pulse hack and aarch64's HVC-hardcoded PSCI call, both flagged above/in
|
||
§III). `timer.c` is confirmed DTB-first as this section's own preamble already stated:
|
||
`timebase-frequency` is read via `fdt_prop_u32()` with a *named* QEMU-only fallback
|
||
(`RISCV_TIMEBASE_HZ_FALLBACK`) used only when the DTB is absent or invalid.
|
||
**Finding — already tracked, this pass confirms rather than discovers it**: `plic.c`'s
|
||
`PLIC_BASE`/`PLIC_CONTEXT_S` hardcoded-to-QEMU-virt situation is exactly item 3 above,
|
||
already flagged as "a real punch-list item, not a hypothetical" before this audit ran.
|
||
Nothing new to add beyond confirming the file's own header comment is accurate and the
|
||
risk is real, not overstated.
|
||
**Finding — minor, reasoning didn't transfer to the native path — FIXED 2026-09-04, per
|
||
direct instruction naming it specifically.** `arch_early_init()`'s explicit `satp` clear
|
||
(Bare-mode switch) was justified in its own comment entirely by behavior *observed under
|
||
QEMU's EDK2 RISC-V firmware* (confirmed `satp.MODE=10`/Sv57 live, kernel identity-mapped
|
||
within it). Under the native boot path (U-Boot+OpenSBI, no UEFI/EDK2 at all per §V.1)
|
||
that specific observation cannot apply — OpenSBI's S-mode handoff conventionally already
|
||
leaves `satp=0` (Bare mode), most likely making the unconditional switch a harmless no-op
|
||
there, but on an unverified assumption rather than a checked one.
|
||
**Fix**: rather than build a general Sv39/Sv48/Sv57 page-table walker just to *prove* the
|
||
running address is identity-mapped before switching (out of proportion to this finding's
|
||
own severity), `arch_early_init()` now reads `satp.MODE` first and **skips the switch
|
||
entirely when it's already 0** — the expected case on a from-Bare native boot, needing no
|
||
safety argument at all since there's nothing to switch away from. The unconditional
|
||
`csrw satp, x0`/`sfence.vma` pair still runs, unchanged, for the confirmed QEMU/EDK2 case
|
||
(`mode != 0`), where the identity-mapping argument remains the actual justification it
|
||
always was. If the native path somehow reaches the non-zero branch anyway (OpenSBI
|
||
leaving a non-Bare `satp`, not confirmed either way), the code says so explicitly in its
|
||
own comment now rather than silently relying on the QEMU-only observation. Verified against
|
||
the riscv64 boot log's own `satp.MODE` print, not assumed: this system's QEMU/EDK2 boot
|
||
shows `satp.MODE = 0x000000000000000a` (Sv57) at kernel entry, so the `mode != 0` branch
|
||
ran and printed `satp cleared -- Bare mode, explicit` exactly as before the fix — the
|
||
unconditional-switch path is unchanged on the one boot path that exercises it. amd64 and
|
||
aarch64 don't compile this file at all, so their part of the 3-arch acceptance run is pure
|
||
non-regression on files untouched by this fix, not evidence about the guard itself.
|
||
|
||
**Hardware-dependent, after 2026-09-17:**
|
||
8. Build and flash the boot image to QSPI flash (or attempt UART XMODEM recovery boot if QSPI
|
||
flashing isn't set up yet — both are real supported paths per §V.1).
|
||
9. Connect HDMI + keyboard.
|
||
10. Boot; confirm `ok>`/`zuse)ok>` reached.
|
||
11. Mint a Zuse identity on real media, confirm re-attach — the `v2.5.0` gate's own
|
||
requirement.
|
||
12. Update this section with results.
|
||
|
||
---
|
||
|
||
## VI. Hardware identification reference
|
||
|
||
Per-board SoC/CPU facts, consolidated here so later sections don't have to re-derive them.
|
||
Researched 2026-09-04 (web search, sources cited); anything not directly confirmed against
|
||
the actual unit in hand is flagged as such rather than assumed.
|
||
|
||
### amd64 — Beelink SER5 (reference/development machine)
|
||
|
||
- **CPU: AMD Ryzen 7 family.** Beelink has shipped the "SER5" name with several different
|
||
Ryzen 7 SKUs over its product life (5700U, 5800H, 7735HS all confirmed to exist under this
|
||
branding) — **exact SKU on this unit not yet confirmed**; check `dmesg`/BIOS/the physical
|
||
unit when convenient (`cat /proc/cpuinfo` or the BIOS splash screen under Linux/before
|
||
LithosAnanke boots, since LithosAnanke itself has no CPU-identification word yet). Not
|
||
load-bearing for this document's own genericity requirement (§III) — the boot path must not
|
||
depend on which SKU this is, by design — but worth pinning down for this reference's own
|
||
accuracy.
|
||
- **Architecture generation**: Zen2 (5700U/5800H) or Zen3 (7735HS) depending on the SKU above
|
||
— matters for any future CPU-feature-detection work (e.g. RDRAND is present on all of
|
||
these; that part's already confirmed live via `rng: backend = rdrand`, §III).
|
||
- Sources: [Gentoo wiki — SER5 5560U](https://wiki.gentoo.org/wiki/Beelink_SER5_AMD_Ryzen_5_5560U_Mini_PC),
|
||
[Starry Hope — SER5](https://www.starryhope.com/minipcs/models/beelink-ser5-mini-pc/),
|
||
[Starry Hope — SER5 Pro](https://www.starryhope.com/minipcs/models/beelink-ser5-pro-mini-pc/),
|
||
[Minixpc — SER5 Max](https://minixpc.com/blogs/news/beelink-ser5-max-review-powered-by-amd-ryzen7-5800h-processor).
|
||
|
||
### aarch64 — Raspberry Pi 5 (sole target)
|
||
|
||
- **SoC: Broadcom BCM2712.**
|
||
- **CPU**: quad-core 64-bit Arm Cortex-A76, 2.4 GHz, 512 KB per-core L2 cache, 2 MB shared L3.
|
||
- **GPU**: VideoCore VII, 12-core, 800 MHz, OpenGL ES 3.1 + Vulkan 1.2 (not relevant to
|
||
LithosAnanke's own framebuffer work — that goes through the mailbox property interface,
|
||
§IV.1 — but recorded here for completeness).
|
||
- **RAM**: LPDDR4X-4267, board variants at 1/2/4/8/16 GB, 32-bit memory interface, ~17 GB/s
|
||
bandwidth.
|
||
- **I/O**: RP1 companion chip (PCIe 2.0 x4-attached) handles GPIO, USB 2.0/3.0, Gigabit
|
||
Ethernet, CSI/DSI, analog video — confirmed separately (§IV.2) to have no RNG peripheral in
|
||
its own published peripheral list.
|
||
- **Cortex-A76 and `FEAT_RNG` (ARMv8.5 `RNDR`/`RNDRRS`)**: not confirmed present — A76 is not
|
||
among the cores that typically implement this feature (more common on newer cores like
|
||
Cortex-X2/A710); if this matters for any future entropy-source decision, verify via `ID_AA64ISAR0_EL1`
|
||
directly on the real board rather than assuming either way.
|
||
- Sources: [CNX Software — Pi 5 launch](https://www.cnx-software.com/2023/09/28/raspberry-pi-5-sbc-broadcom-bcm2712-quad-core-cortex-a76-soc/),
|
||
[Raspberry Pi — Processors doc](https://www.raspberrypi.com/documentation/computers/processors.html),
|
||
[sbcwiki — BCM2712](https://sbcwiki.com/docs/soc-manufacturers/broadcom/bcm2712/boards/rasperrypi-5/).
|
||
|
||
### riscv64 — Milk-V Mars (sole target)
|
||
|
||
- **SoC: StarFive JH7110**, 28 nm.
|
||
- **CPU**: 4× SiFive U74-MC application cores (RV64GC) + 1× SiFive S7 monitor core, up to
|
||
1.5 GHz.
|
||
- **RAM**: up to 8 GB LPDDR4; storage via eMMC slot + microSD slot.
|
||
- **I/O**: 3× USB 3.0, 1× USB 2.0, HDMI 2.0 (4K), Gigabit Ethernet with PoE support, M.2 E-Key
|
||
(WiFi/BT), 4-lane + 2-lane MIPI CSI, 40-pin GPIO header.
|
||
- **Physical**: designed to Raspberry Pi 3B dimensions — cases/heatsinks/fans for that form
|
||
factor are compatible.
|
||
- **Multimedia**: H.264/H.265 4K@60fps decode, H.265 1080p@30fps encode (not relevant to
|
||
LithosAnanke's own bring-up, recorded for completeness).
|
||
- Same JH7110 SoC as the StarFive VisionFive 2 — any VisionFive 2 bring-up material found
|
||
while researching §V's own boot-chain question is likely directly applicable here too, worth
|
||
checking first before assuming Mars-specific research is needed from scratch.
|
||
- Sources: [milkv.io — Mars overview](https://milkv.io/docs/mars/overview),
|
||
[milkv.io — Mars product page](https://milkv.io/mars),
|
||
[TinyComputers.io — Mars review](https://tinycomputers.io/posts/milk-v-mars-review.html).
|
||
|
||
### Noted for later, not yet in scope — BeagleBone Black
|
||
|
||
Added to this reference per direct instruction 2026-09-04, **recorded only — no work scoped
|
||
around it yet.** Genuinely different from the three targets above: the BeagleBone Black's
|
||
SoC is a **32-bit ARM** part, not aarch64 — a fourth architecture this kernel has no support
|
||
for at all today (amd64/aarch64/riscv64 only), not another board under an existing one.
|
||
|
||
- **SoC: TI Sitara AM335x.**
|
||
- **CPU**: single-core ARM Cortex-A8, 1 GHz, armv7-a (32-bit) — up to ~2000 MIPS.
|
||
- **RAM**: 512 MB DDR3L. Storage: 4 GB eMMC (default boot source) + microSD (secondary/
|
||
overridable to primary).
|
||
- **Other on-die units**: PowerVR SGX530 3D GPU; 2× PRU (Programmable Realtime Unit) 32-bit
|
||
200 MHz microcontrollers — real-time I/O coprocessors, no equivalent on any of the three
|
||
boards above; crypto accelerators.
|
||
- **Boot modes**: eMMC, microSD, serial, USB.
|
||
- Sources: [element14 — BBB product page](https://www.element14.com/community/docs/DOC-84108/l/beaglebone-black-development-board-with-1ghz-am335x-arm%C3%A3%C3%A2-cortex-a8-processor),
|
||
[TI.com — BEAGL-BONE-BLACK](https://www.ti.com/tool/BEAGL-BONE-BLACK).
|
||
|
||
### Noted for later, not yet in scope — Zynq-7000 (Puzhi PZ7010/PZ7020 "StarLite")
|
||
|
||
Added per direct instruction 2026-09-04, **recorded only — no work scoped around it yet.**
|
||
Unlike BeagleBone Black above, this one isn't a random addition: `ROADMAP.md` already names
|
||
**Zynq FPGA as the next big milestone beyond v2.5.0** — "the step where the battle-tested
|
||
amd64/aarch64/riscv64 story rides on configurable silicon," and the three-product split
|
||
decided alongside it names "hardware steady-state machinery with sealed executions,
|
||
HOL-proven" as the FPGA-native product this board would ultimately serve. This entry just
|
||
puts a concrete, purchasable board under that already-named milestone.
|
||
|
||
- **Board**: Puzhi PZ7010-StarLite (XC7Z010) or PZ7020-StarLite (XC7Z020) — same board design,
|
||
two SoC variants. 90×60mm, black PCB, immersion gold finish.
|
||
- **SoC: Xilinx/AMD Zynq-7000**, combining a **Processing System (PS)** — dual-core ARM
|
||
Cortex-A9, up to 667 MHz (`-1` speed grade) or 800 MHz (`-2`, XC7Z020 only) — with
|
||
**Programmable Logic (PL)**, 28 nm Artix-7/Kintex-7-based FPGA fabric. Genuinely a fifth
|
||
architecture class in this reference: ARMv7-A again (like BeagleBone Black), but a
|
||
different core (Cortex-A9 vs. A8) *and* an FPGA fabric with no equivalent on any board
|
||
above — this is the "configurable silicon" milestone `ROADMAP.md` already flagged as
|
||
reshaping the hardware story (soft/hard CPU cores, PL fabric, non-standard memory map,
|
||
custom peripherals), not a small per-board addition even in concept.
|
||
- **PS details** (identical between both variants): 256 KB on-chip memory, DDR3 controller,
|
||
32 KB I-cache + 32 KB D-cache per core, 512 KB shared L2.
|
||
- **PL resources (the actual XC7Z010 vs. XC7Z020 difference)**: XC7Z010 — 4,400 logic slices,
|
||
17,600 6-input LUTs, 35,200 flip-flops, 270 KB block RAM, 80 DSP slices. XC7Z020 — 13,300
|
||
logic slices, 53,200 LUTs, 106,400 flip-flops, 630 KB block RAM, 220 DSP slices.
|
||
- **RAM/storage**: 512 MB/1 GB DDR3, QSPI flash, EEPROM, SD boot.
|
||
- **I/O**: JTAG, UART, HDMI out, Gigabit Ethernet, USB 2.0 host, 40-pin expansion; MIPI CSI on
|
||
the 7020 variant only.
|
||
- Sources: [Puzhi — PZ7010-StarLite](https://www.en.puzhi.com/Product/AMD-FPGA-Development-Board/Zynq-7000-SoC/PZ7010-StarLite),
|
||
[Puzhi — PZ7020-StarLite](https://www.en.puzhi.com/Product/AMD-FPGA-Development-Board/Zynq-7000-SoC/PZ7020-StarLite),
|
||
[Xilinx/AMD — Zynq-7000 SoC Data Sheet (DS190)](https://www.mouser.com/datasheet/2/903/ds190-Zynq-7000-Overview-1595492.pdf),
|
||
[PCBSync — XC7Z010 vs XC7Z020 comparison](https://pcbsync.com/xilinx-xc7z010/).
|
||
|
||
## VII. xHCI/BOT driver: single-device architecture must go multi-device — CLOSED 2026-09-05
|
||
|
||
**Closure note, 2026-09-05:** implemented and live-verified per §VII.6's punch list, items 1-5
|
||
and 7 (item 6, file consolidation, was explicitly optional and not done; item 8, resume
|
||
minting, is the next task). Per-slot registry (`xhci_msc_slot_t`/`dev->msc_slots`, sized off
|
||
`dev->max_slots`) now holds every persistent per-device field; the boot-time scan no longer
|
||
`break`s after the first connected port; a dropped connect/disconnect during a busy Command
|
||
Ring is now queued (`dev->pending_events[]`) and drained instead of lost;
|
||
`blkio_usb.c`/`repl.c`'s own singletons became per-slot registries the same way.
|
||
|
||
**A second, more severe bug found only through live multi-device testing, not scoped in the
|
||
original plan:** `transfer_purpose`/`pending_transfer_slot_id` and `next_action`/
|
||
`next_action_slot_id` were *also* single scalars shared across the whole controller, not just
|
||
the fields named in §VII.2's original inventory. With two devices genuinely enumerating
|
||
concurrently, a second device's own EP0/bulk completion could silently overwrite the first
|
||
device's still-outstanding one before its real hardware completion arrived — the first device
|
||
then stalled forever with no error, no timeout, nothing to see in the log except the *other*
|
||
device's own progress. Confirmed live: a two-device boot (Zuse's drive + a blank second drive)
|
||
hung indefinitely at the exact point this predicts, recovered by moving both fields to
|
||
`xhci_msc_slot_t` (`transfer_purpose`) and driving `next_action` dispatch as a per-slot scan
|
||
at the end of `xhci_poll_events()`, and — critically — reading the Transfer Event TRB's own
|
||
real Slot ID field (`XHCI_EVT_SLOT_ID(trb->control)`, xHCI 1.2 table 6-38) instead of trusting
|
||
external bookkeeping for which device a completion belongs to.
|
||
|
||
**Verified live, all three architectures, `clean qemu` (mandatory acceptance), single-device
|
||
path unchanged:** amd64/aarch64/riscv64 all reach `(zuse) ok>` with the existing single
|
||
`ZUSEDISK` attach path, no regression.
|
||
|
||
**Verified live, amd64, the actual fix (item 7b):** Zuse's drive + a second blank drive
|
||
(`bob-thumb-ident.img`) both attached from boot (`-device usb-storage,bus=xhci0.0,...` twice).
|
||
Diagnostic probes (written, run, reverted — not committed) confirmed both slots independently
|
||
progress through Enable Slot → Address Device → GET_DEVICE_DESC → GET_CONFIG_DESC →
|
||
Configure Endpoint → SET_CONFIGURATION without either corrupting or stalling the other; Zuse's
|
||
own device completed the full chain, attached, and authenticated (`Zuse: identity confirmed
|
||
from attached thumbdrive`) while the second device was still independently mid-enumeration —
|
||
concurrent progress, not serialized-and-got-lucky. The second (blank) drive's own full
|
||
attach-to-completion was not separately confirmed to finish within the bounded test window
|
||
(TCG timing, not a correctness question) — not a red flag, since the property being tested
|
||
(no cross-device corruption/starvation) was already established by both devices advancing
|
||
independently.
|
||
|
||
**Not done, out of scope for this pass:** true simultaneous minting of all 8 identity drives
|
||
(§VII.6 item 8) — next task. Sequential QMP hot-swap (the original mint script) remains valid
|
||
too; simultaneous attach is now also an option, not the only one.
|
||
|
||
|
||
**Status 2026-09-05: design + punch list only. No code has been touched. Halted pending
|
||
Captain Bob's explicit "okay" to begin implementation** — this section exists to write the
|
||
decision down before building, per this project's own standing methodology
|
||
([[feedback_fabric_series_effective_methodology]]).
|
||
|
||
### VII.1 — How this surfaced
|
||
|
||
While scoping the identity-thumbdrive minting workflow (8 real WIREBIND/Zuse identities to
|
||
be minted from blank images in `disk/thumbdrives/`), the plan called for hot-swapping a
|
||
single USB slot via QMP because the kernel's xHCI/BOT stack tracks exactly one attached mass-
|
||
storage device at a time. Bob interrupted before that script ran: *"Wait! We can only have
|
||
one usb slot. is that a QEMU limitation?"* It is not — confirmed live via `qemu-xhci`'s own
|
||
multi-port root hub and this driver's own `port_slot_id[XHCI_MAX_TRACKED_PORTS]` (32 tracked
|
||
ports, `include/starkernel/xhci.h:515`), which is already multi-port-aware. Everything
|
||
*downstream* of port/slot correlation is a deliberate single-device singleton, dating to
|
||
Milestones 2e–2h. Bob's ruling: *"We MUST fix that! It was a deliberate choice at that time
|
||
and is no longer true, much in the same way a build flag for console."* — same class of
|
||
correction as the headless-console change (§ pending its own FABRIC writeup): a simplification
|
||
that was right when made, and is stale now that the project needs it to not be true.
|
||
|
||
### VII.2 — Exact scope of the singleton (full inventory, code-verified, not recalled)
|
||
|
||
`port_slot_id[]` (`xhci_driver.h:58`) is the only field in `xhci_dev_t` that is already an
|
||
array. Every field downstream of a successful Address Device is a single reused scalar, each
|
||
with an explicit "single-device scope" comment written at the time it was added:
|
||
|
||
**Connect / enumerate state machine:**
|
||
- `connect_state`, `pending_connect_port_id`, `pending_connect_slot_id` (`xhci_driver.h:59,70-79`)
|
||
- `input_ctx`, `device_ctx`, `ep0_ring` (`xhci_driver.h:91-95`) — "only ever addresses one
|
||
device at a time"
|
||
|
||
**Control-transfer / descriptor state machine:**
|
||
- `transfer_purpose`, `pending_transfer_slot_id` (`xhci_driver.h:113-126`)
|
||
- `device_descriptor[18]`, `config_descriptor[128]` (`xhci_driver.h:127-128`) — "reused, not
|
||
per-slot"
|
||
- `bulk_in_ep_addr`/`bulk_out_ep_addr`, `bulk_in_ring`/`bulk_out_ring`, endpoint max-packet
|
||
fields (`xhci_driver.h:139-157`)
|
||
|
||
**BOT (Bulk-Only Transport / SCSI) state machine** (`xhci_driver.h:159-274`):
|
||
- `bot_cbw`/`bot_csw`, `bot_data_buf[1024]`, `bot_last_tag`, `bot_next_tag`
|
||
- `bot_cmd_kind`, `bot_last_status`, `bot_expected_data_len`
|
||
- `bot_read10_*`/`bot_write10_*` staging fields
|
||
- `bot_tur_retries`, `bot_tur_chain_target`
|
||
- `bot_cap_last_lba`/`bot_cap_block_size`
|
||
- Full stall-recovery sub-state-machine (`bot_stall_recoveries`, `stall_dci`, `stall_ep_addr`,
|
||
`bot_reset_clear_remaining`, `stall_retry_action`) — "this driver runs exactly one bulk
|
||
transfer at a time... there is no concurrency to serialize"
|
||
- `bot_msc_attach_pending`/`bot_msc_attach_slot_id`/`bot_msc_attached`/`bot_msc_detach_pending`
|
||
— the flags `repl.c`'s own `static blkio_dev_t usb_blk_dev` (single-device scope,
|
||
`repl.c:244`) and the two module-level pointers `g_homeblocks_dev` (`repl.c:122`) /
|
||
`g_attached_blk_dev` (`repl.c:139`) all depend on.
|
||
|
||
**A third, previously-unlisted singleton point, found while tracing `xhci.c` for this
|
||
writeup:** `xhci_scan_ports_for_already_connected()` (`src/starkernel/usb/xhci.c:1343-1362`,
|
||
called once from `xhci_bringup()`) walks every tracked port looking for one already connected
|
||
at boot (needed because a device present on the QEMU command line before controller reset
|
||
never generates a Port Status Change *event* — confirmed against the xHCI event model, not
|
||
assumed) — and **`break`s after the first hit**, with its own comment stating why: "this
|
||
driver's real use case is exactly one thumbdrive already attached at boot, not several." A
|
||
second device already plugged in at boot (e.g. the mint workflow's Zuse-drive-plus-target-
|
||
drive scenario, both present from `qemu-xhci` at launch) would never be discovered by this
|
||
scan at all — not merely mishandled once found, genuinely invisible. This must be fixed
|
||
alongside the state-shape changes below, and is now folded into the punch list.
|
||
|
||
**A structural finding that narrows the fix, not widens it:** `xhci_handle_port_connected()`
|
||
(`xhci.c:1300-1327`) already gates new Enable Slot requests on `connect_state == XHCI_CONN_IDLE`
|
||
and *drops* (does not queue) a second simultaneous connect with `"xhci: enable slot already
|
||
pending -- dropped"` — the same pattern repeats for disconnect (`xhci.c:1531-1539`,
|
||
`"disable slot skipped -- command ring busy"`). This confirms §VII.3's category-2 in-flight
|
||
state genuinely never needs true concurrency: the driver's own connect/disconnect handling is
|
||
already built to serialize one command at a time and simply discard what it can't yet serve
|
||
— the fix for these two sites is to *not drop the second event*, not to make the driver
|
||
concurrent (see punch-list item 3).
|
||
|
||
**Also confirmed, a positive finding:** `block_subsystem.c`'s own attach layer
|
||
(`blk_subsys_attach_device()`, `block_subsystem.c:677`, and its `first_disk_slot()` device-slot
|
||
table) is **already multi-device-capable** — Artemis (virtio) and a USB MSC device already
|
||
coexist as independent attached devices there today. The bottleneck is confined to the xHCI
|
||
driver itself and the three singleton points immediately above it in `repl.c`
|
||
(`usb_blk_dev`/`g_homeblocks_dev`/`g_attached_blk_dev`) — `block_subsystem.c` itself needs no
|
||
change for this fix. This makes the fix smaller than initially framed in §VII.1/§VII.3.
|
||
|
||
Also confirmed: this driver runs on **all three architectures**, not just amd64 — `qemu-xhci`
|
||
is instantiated for amd64, aarch64, and riscv64 alike (`Makefile.starkernel` lines 819, 903,
|
||
993), and `xhci_find_and_map()`/`xhci_bringup()` are called unconditionally from
|
||
`kernel_main.c:653-656` (no `ARCH_*` guard). Any fix here is a three-architecture change, same
|
||
acceptance bar as everything else in this document.
|
||
|
||
### VII.3 — The distinction that makes this tractable: persistent vs. in-flight state
|
||
|
||
Not everything above needs to become concurrent. Two different categories:
|
||
|
||
1. **Per-slot persistent state — must become per-slot (arrays), full stop.** Endpoint
|
||
addresses/max-packet, bulk rings, device/config descriptors, the `blkio_dev_t` (and its
|
||
`usb_blk_dev` analogue in `repl.c`), and the MSC attach flags. If these stay singular, a
|
||
second device attaching while a first is already open **corrupts the first's live state** —
|
||
this is the actual bug being fixed, not a cosmetic one.
|
||
|
||
2. **Per-transfer in-flight state — stays single, gains a slot selector.** `connect_state`,
|
||
`transfer_purpose`, `pending_transfer_slot_id`, `bot_cbw`/`bot_csw`, `bot_cmd_kind`, the
|
||
stall-recovery sub-state-machine. Nothing in any real use case (present or planned) reads
|
||
two drives at the literal same instant — the driver stays "one transfer in flight at a
|
||
time," it just needs to know *which slot* that one transfer belongs to, instead of
|
||
assuming there is only ever one possible answer.
|
||
|
||
This is the difference between a bounded, scoped refactor (make category 1 arrays, thread a
|
||
slot index through category 2) and a full concurrent rewrite of the driver's core state
|
||
machine. Only category 1 needs to change shape.
|
||
|
||
### VII.4 — Concurrency bound
|
||
|
||
Revised 2026-09-05 (Bob, second pass): the earlier framing on this page — "a small N,
|
||
matching a real xHCI root hub port count," floated with a proposed default of 4 — is
|
||
rejected. Bob's ruling: *"it might again, been true at one time, but we REALLY need to not
|
||
paint ourselves into ANY scale issues by taking an easier way out."* This is the exact
|
||
pattern this section itself exists to correct (a deliberate simplification, right when made,
|
||
now stale) — hardcoding a second small ceiling right next to the one just being removed would
|
||
just relocate the same mistake, not fix it.
|
||
|
||
**Corrected direction, and a correction to this page's own first draft of that direction:**
|
||
the per-slot registry (category 1 state, §VII.3) must not carry a small fixed-size array at
|
||
all — but the fix is not "start reading `HCSPARAMS1.MaxSlots`," because **that register is
|
||
already read, today**: `xhci_find_and_map()` (`xhci.c:68-71`) already does
|
||
`dev->max_slots = XHCI_HCSPARAMS1_MAX_SLOTS(hcs1)`, and `xhci_bringup()` already uses it
|
||
correctly — the DCBAA (`xhci.c:188-198`) is allocated at exactly `(dev->max_slots + 1) *
|
||
sizeof(uint64_t)` via `kmalloc_aligned()` (a real, hardware-sized heap allocation, not a fixed
|
||
array), and `dev->op->config` (`xhci.c:299`) is programmed with
|
||
`XHCI_CONFIG_MAX_SLOTS_EN(dev->max_slots)` to tell the controller the same number back. **The
|
||
slot-correlation layer of this driver is already correctly scaled to real hardware and needs
|
||
no change.** The bug is narrower and more specific than "the driver doesn't know how big the
|
||
hardware is": `dev->max_slots` is known and used correctly for the DCBAA, but the category-1
|
||
fields (§VII.3) never use it at all — they're scalars regardless of what `max_slots` says,
|
||
because they were written before per-slot addressing was a hardware value from anywhere
|
||
convenient to size against.
|
||
|
||
**Corrected punch-list direction:** size the new per-slot registry for category-1 state off
|
||
`dev->max_slots` — the same field already driving the DCBAA allocation — via the same
|
||
`kmalloc_aligned()`-at-bringup pattern the DCBAA itself already establishes as this driver's
|
||
precedent for hardware-sized allocation, rather than introducing a second, separately-derived
|
||
bound or a fixed-size array at all. (`XHCI_MAX_TRACKED_PORTS`, 32, `xhci.h:515`, is a
|
||
*different* table — port-to-slot correlation, sized as "comfortably covers any real root hub,"
|
||
by its own comment's admission a chosen convenience constant, not hardware-derived — and
|
||
should not be reused or treated as precedent for this one.)
|
||
|
||
### VII.5 — Bob's live suggestion: consolidate into one file, pass state explicitly
|
||
|
||
Raised mid-scoping: *"maybe refactor the driver into one single driver file and calling from
|
||
the dependency? Simplify the code."* Worth noting precisely what's already true and what
|
||
isn't, before this becomes a punch-list item:
|
||
|
||
- **Already true:** nearly every command-builder function (`xhci_cmd_*`, `xhci_bot_*`,
|
||
`xhci_ep0_*`) already takes an explicit `xhci_dev_t *dev` parameter — this is already
|
||
dependency-injection-shaped at the call-site level, not a global-variable design.
|
||
- **Not true — the actual singleton:** `xhci_poll_events()` itself takes **no** arguments and
|
||
reads a module-static `xhci_dev_t *g_xhci_dev` (`xhci.c:1482`) set once by `xhci_bringup()`;
|
||
`xhci_get_dev()` (returning that same static) is used by exactly two callers —
|
||
`kernel_main.c`'s bring-up block and `repl.c`'s `sk_repl_idle()` (`repl.c:243`) — and
|
||
everything else (the entire event-driven dispatch chain inside `xhci_poll_events()` itself)
|
||
already has `dev` in hand as a real parameter, it just came from that one static originally.
|
||
This static-singleton *retrieval* pattern is not actually the multi-device blocker (only one
|
||
physical xHCI *controller* is ever supported or claimed to be — that premise is unchanged
|
||
and correct, `virtio_blk.c` uses the identical single-controller-static precedent) — the
|
||
blocker is entirely inside `xhci_dev_t`'s own fields (§VII.2/§VII.3), one controller struct
|
||
holding scalar per-*device* state instead of per-slot arrays. Bob's "calling from the
|
||
dependency" framing is already satisfied at the controller level; the fix below applies the
|
||
same idea one level down, to the device slots living inside that one controller.
|
||
- **File consolidation (`xhci_driver.h` + `src/starkernel/usb/xhci.c` → one file):** a
|
||
separate, smaller question from the state-shape fix above, genuinely optional. Current split
|
||
is ordinary header/implementation separation, already followed throughout `src/starkernel/`
|
||
(nothing else in the tree merges its public API header and its `.c` file). Recommend keeping
|
||
the split and judging "simpler" by whether the *singleton* is gone, not by file count — but
|
||
this is Bob's call, not a default to override. Tracked as punch-list item 6 (optional).
|
||
|
||
### VII.6 — Punch list (numbered = proposed execution order; nothing here has been started;
|
||
each item names the exact functions/fields it touches, traced against real code, not estimated)
|
||
|
||
1. **New `xhci_msc_slot_t` per-slot record type, registry array sized off `dev->max_slots`.**
|
||
Define one struct bundling every category-1 field currently scalar in `xhci_dev_t`
|
||
(`xhci_driver.h:91-95` connect/enumerate: `input_ctx`, `device_ctx`, `ep0_ring`,
|
||
`ep0_ring_cycle`, `ep0_ring_enq`; `xhci_driver.h:127-157` control/bulk: `device_descriptor`,
|
||
`config_descriptor`, `config_total_length`, `bulk_in_ep_addr`/`bulk_out_ep_addr`,
|
||
`bulk_in_max_packet`/`bulk_out_max_packet`, `bulk_in_ring`/`bulk_out_ring` +
|
||
cycle/enqueue state; `xhci_driver.h:285-294` MSC attach: `bot_msc_attach_pending`,
|
||
`bot_msc_attached`) plus a `uint32_t slot_id` tag (0 = unused slot). Add
|
||
`xhci_msc_slot_t *msc_slots;` and `uint32_t msc_slot_count;` to `xhci_dev_t`, replacing the
|
||
individual fields listed above (removed from the struct, not left dead alongside the new
|
||
ones — this project's own no-dead-code convention). Allocate in `xhci_bringup()`
|
||
immediately after `dev->max_slots` is known (`xhci.c:69`, already set by
|
||
`xhci_find_and_map()` which always runs first) via `kmalloc_aligned(sizeof(xhci_msc_slot_t)
|
||
* (dev->max_slots + 1), ...)`, zeroed — the exact same sizing input and allocation call
|
||
`xhci.c:188-198`'s DCBAA already uses, so this item adds no new sizing policy, just a
|
||
second allocation using the existing one's already-correct input.
|
||
2. **`xhci_msc_slot_for(dev, slot_id)` lookup, replacing every direct field access.** One
|
||
function: bounds-check `slot_id <= dev->max_slots`, return `&dev->msc_slots[slot_id]` (or
|
||
NULL out of range). Every one of the ~15 category-1 fields' current access sites across
|
||
`xhci.c` (`xhci_cmd_address_device()` at `xhci.c:453` onward, `xhci_cmd_configure_endpoint()`
|
||
at `xhci.c:554`, the `xhci_bot_*`/`xhci_ep0_*` families at `xhci.c:753-1266`, and the
|
||
completion handlers inside `xhci_poll_events()` at `xhci.c:1480-2175`) changes from
|
||
`dev->field` to `xhci_msc_slot_for(dev, slot_id)->field` — every one of these call sites
|
||
already receives `slot_id` as a parameter today (confirmed: every `xhci_cmd_*`/`xhci_bot_*`/
|
||
`xhci_ep0_*` function signature already takes `uint32_t slot_id`), so this item is a
|
||
mechanical field-access rewrite, not a new parameter-threading exercise.
|
||
3. **Stop dropping simultaneous connect/disconnect instead of building real concurrency.**
|
||
`xhci_handle_port_connected()` (`xhci.c:1300-1327`) currently drops a second connect with
|
||
`"enable slot already pending -- dropped"` when `connect_state != XHCI_CONN_IDLE`; the
|
||
disconnect path (`xhci.c:1531-1539`) drops the same way with `"disable slot skipped --
|
||
command ring busy"`. Per §VII.2's finding, this in-flight state genuinely stays
|
||
single-outstanding (one Command Ring, one command at a time, matching real xHCI command
|
||
submission) — the fix is a **pending-request queue of depth `dev->max_slots`** (a small
|
||
array of `{port_id, is_connect}` entries, not a state-machine rewrite): a dropped connect/
|
||
disconnect is queued instead of discarded, and drained one entry at a time as
|
||
`connect_state` returns to `XHCI_CONN_IDLE` (the same point that already exists at
|
||
`xhci.c:1594`, `:1610`, `:1611` — add a "drain one queued request" call there). This
|
||
directly fixes the currently-real bug where two thumbdrives connected close together (well
|
||
within human/QMP timing) can silently lose the second one today, independent of the
|
||
category-1 field-shape fix in items 1-2.
|
||
4. **Fix `xhci_scan_ports_for_already_connected()`'s single-hit `break` (`xhci.c:1343-1362`).**
|
||
Remove the `break` at `xhci.c:1360` so the boot-time already-connected scan drives *every*
|
||
tracked port with `CCS` set through `xhci_handle_port_connected()`, not just the first.
|
||
Since item 3's queue now exists, a second (or third) already-connected device found here
|
||
queues cleanly instead of needing its own separate handling — this item has no correctness
|
||
force of its own once item 3 lands, it is purely "stop deliberately stopping early."
|
||
5. **Update `repl.c`'s three singleton points to arrays over `xdev->max_slots`.**
|
||
`static blkio_dev_t usb_blk_dev` (`repl.c:244`), `static blkio_dev_t *g_homeblocks_dev`
|
||
(`repl.c:122`), and `static blkio_dev_t *g_attached_blk_dev` (`repl.c:139`) all currently
|
||
assume one attached USB MSC device. Convert `usb_blk_dev` to an array indexed the same way
|
||
as item 1's `msc_slots` (by `slot_id`, bounds-checked against `xdev->max_slots`);
|
||
`g_homeblocks_dev`/`g_attached_blk_dev`'s callers (`sk_get_homeblocks_dev()`/
|
||
`sk_get_attached_blk_dev()`, whichever functions wrap `repl.c:127`/`:142` today) need to
|
||
become "does a homeblocks/attached device exist for *this* identity/slot" rather than "the
|
||
one homeblocks/attached device" — the real consumers are `capsule_zuse_boot_try_attach()`
|
||
and `capsule_wirebind_try_attach()` (`repl.c:300`, `:311`), which is exactly the layer the
|
||
identity-login (WIREBIND/Zuse) flow depends on and the reason this whole fix exists. Per
|
||
§VII.2, `block_subsystem.c`'s own `blk_subsys_attach_device()`/`first_disk_slot()` layer
|
||
needs **no** change — it already supports multiple simultaneously-attached devices; only
|
||
the `repl.c` glue feeding it is the singleton.
|
||
6. *(Optional, Bob's call per §VII.5, not required for correctness)* — consolidate
|
||
`xhci_driver.h` + `src/starkernel/usb/xhci.c` into one file, if items 1-5 together don't
|
||
already read as "simple enough" on their own. Recommendation stands: keep the split, judge
|
||
simplicity by whether the singleton is gone (it will be), not by file count.
|
||
7. **Three-architecture acceptance** (`clean qemu`, amd64 → aarch64 → riscv64, sequential,
|
||
foreground, per this project's non-negotiable acceptance bar) confirming both directions:
|
||
(a) **no regression** — existing single-device behavior (the mandatory `ZUSEDISK_QEMU_ARGS`
|
||
attach path every current boot already exercises) is unchanged when only one device is ever
|
||
attached; (b) **the actual fix** — two devices attached simultaneously (a second
|
||
`-device usb-storage,bus=xhci0.0,drive=...` added to the existing QEMU launch args, or via
|
||
QMP `device_add` against a second port on `xhci0.0` without an intervening `device_del`)
|
||
both enumerate, both reach `SET_CONFIGURATION`, both open as BOT/SCSI devices via
|
||
`blkio_usb_open_msc()`, and neither's `msc_slots[]` entry (item 1) shows corruption from the
|
||
other's descriptors/rings/BOT state — verified by log inspection (each device's own
|
||
`bot_cap_last_lba`/`bot_cap_block_size` reported correctly and independently), not just
|
||
"both attached without crashing."
|
||
8. Only after item 7 passes: resume the identity-thumbdrive minting workflow — either via true
|
||
simultaneous multi-device attach (Zuse's drive + a target drive both present at once, the
|
||
scenario item 4's fix specifically enables) or the original QMP hot-swap sequencing (still
|
||
valid, just no longer the only option) — Bob's call at that point, not assumed now.
|
||
|
||
**Halt point — lifted 2026-09-05:** Bob's go-ahead ("okay, go ahead and implement it") authorized
|
||
implementation; see the closure note at the top of this section for what shipped and what was
|
||
found along the way.
|
||
|
||
## VIII. Headless-until-login console policy, and the KILL/g_repl_active_vm dangling-pointer
|
||
fix — CLOSED 2026-09-05
|
||
|
||
Two smaller, unrelated fixes landed the same day as the xHCI work above and are documented
|
||
here together since they were committed in the same pass, per direct instruction.
|
||
|
||
### VIII.1 — Headless-until-login console
|
||
|
||
**Decision, live-directed by Bob mid-session:** *"you have to be sure that there is no
|
||
console for the running system unless a 'thumbdrive' is present. no thumbdrive, no prompt.
|
||
when a user logs in by inserting a thumbdrive a vm with the users identity. the USE word will
|
||
allow that user's console to attach to the human name for that vm"* and *"see, it should be
|
||
headless if no console vm is present."* Zuse's own login is nothing special except that she
|
||
has no ACLs (*"there should be nothing special about zuse as a user except zuse has no
|
||
acl's"*) — her attach path uses the exact same gate as a regular WIREBIND user.
|
||
|
||
**Design:** `EMERGENCY_CONSOLE_ENABLED` (`Kconfig.heartbeat`, default flipped `y`→`n`) is
|
||
reused and extended from its narrower original scope (VM-fault-handler recovery only) to also
|
||
gate boot-time console visibility. When `n` (production default): the kernel boots completely
|
||
silent — no banner, no prompt, nothing on the console — until a real identity logs in via an
|
||
attached thumbdrive (WIREBIND user or Zuse's own attach/genesis-mint, identical treatment),
|
||
and a runtime VM error halts rather than dropping to a recovery REPL (no fallthrough surface).
|
||
When `y` (debug/recovery builds only, opt-in): behavior is unchanged from before this fix —
|
||
console shows immediately at boot, REPL errors recover and continue.
|
||
|
||
**Mechanism:** `sk_console_mark_login()`/`sk_console_login_occurred()` (`repl.c`/`repl.h`) —
|
||
one shared flag, set by either login path (`capsule_wirebind.c`'s regular-user console-VM
|
||
birth call site, `capsule_zuse_boot.c`'s `install_and_activate()`, the shared success point
|
||
for both genesis-mint and re-attach). `sk_repl_headless_wait(VM *mama)` (new, `repl.c`) spins
|
||
on `heartbeat_service()` + `sk_repl_idle()` (so USB/WIREBIND detection keeps running) until
|
||
the flag is set, called from `kernel_main.c` immediately before the existing `sk_repl(mama)`
|
||
call, gated `#if !EMERGENCY_CONSOLE_ENABLED`. `sk_repl_run()`/`sk_repl_step()`'s error
|
||
branches were also wired to a previously-dead `sk_fault_handler()` (defined but never called
|
||
anywhere — confirmed via grep before flipping the default exposed the resulting
|
||
`-Werror=unused-function`), matching `sk_repl()`'s own doc comment's original, never-
|
||
implemented intent: `#if !EMERGENCY_CONSOLE_ENABLED` halts on error, `#else` recovers.
|
||
|
||
**Verified live, both directions, amd64:** positive — with a thumbdrive attached, console is
|
||
silent until login, first prompt shown is always pre-authenticated (`(zuse) ok>` or the
|
||
equivalent WIREBIND-user prompt), never a bare `ok>`. Negative — with `ZUSEDISK=` empty (no
|
||
thumbdrive at all), a 60-second bounded boot produced zero occurrences of "Emergency CLI" or
|
||
any `ok>` prompt in the serial log, and no leaked `qemu-system-*` process. Not yet verified on
|
||
aarch64/riscv64 specifically for the headless behavior (the 3-arch runs done this session all
|
||
used an attached Zuse drive) — the code path is architecture-generic (no `ARCH_*` gating), so
|
||
this is a documentation gap, not a known defect.
|
||
|
||
### VIII.2 — `KILL`/`g_repl_active_vm` dangling-pointer fix
|
||
|
||
**Bug** (flagged report-only in `project_dictionary_superpower_audit_20260903`, fixed now by
|
||
direct instruction): `mama_word_kill()`'s call site never checked whether the VM being killed
|
||
was the one the console's `USE` word had currently attached to
|
||
(`g_repl_active_vm`/`sk_repl_get_active_vm()`). `EJECT`/`UNCLEAN` (`capsule_wirebind.c`)
|
||
already reset-before-kill for this exact reason; plain `KILL` did not, leaving
|
||
`g_repl_active_vm` dangling — the next command dispatched through the console would fault
|
||
against a freed/retired VM.
|
||
|
||
**Fix** (`mama_forth_words.c`, `mama_word_kill()`): widened the existing `VMRegistryEntry
|
||
entry` lookup's scope slightly and added, immediately after the existing Hera-guarded
|
||
`vm_physics_retire()` call: `if (entry.vm_ptr && sk_repl_get_active_vm() == (VM *)
|
||
entry.vm_ptr) { sk_repl_set_active_vm((VM *) 0); }` — detaches the console back to Hera's own
|
||
dictionary before the killed VM's memory can be reused, same as the existing
|
||
`EJECT`/`UNCLEAN` precedent.
|
||
|
||
**Verified:** three-arch `clean qemu` acceptance (this session's runs, alongside the xHCI and
|
||
headless-console work) — no regression in existing `KILL`/`USE` behavior. Not separately
|
||
stress-tested against the specific dangling-pointer scenario (kill the actively-`USE`'d VM,
|
||
then dispatch another command) — the fix is small and directly addresses the confirmed gap,
|
||
but that specific repro was not re-run live after the fix landed.
|
||
|
||
## IX. Four bugs found live minting/verifying the 8 identity thumbdrives — CLOSED 2026-09-06
|
||
|
||
All four surfaced by actually exercising the system this project's own documentation exists
|
||
to describe — the thumbdrive-identity workflow §VII/§VIII made possible — not by code review.
|
||
Each is documented here with the same discipline: what broke, why, the fix, and what was
|
||
verified afterward.
|
||
|
||
### IX.1 — Zuse/WIREBIND cross-contamination on detach
|
||
|
||
**Bug:** with §VII's multi-device fix in place, live-verifying all 8 minted identities (boot
|
||
with Zuse attached, hot-plug each target drive one at a time, confirm `WIREBIND: <username>
|
||
attached and ready`) surfaced a real cross-contamination bug: detaching drive `00` printed
|
||
`Zuse: session ended -- reattach to re-authenticate` even though Zuse's own drive was never
|
||
touched. Root cause: `capsule_zuse_boot_logout()` (`capsule_zuse_boot.c:164`, called
|
||
unconditionally from `repl.c`'s per-slot detach loop for every departing device) had no way
|
||
to tell "some other device detached" from "my own device detached" — its only guard was a
|
||
bare boolean (`g_zuse_attached_this_device`), not a device comparison, despite its own header
|
||
doc comment already claiming that no-op existed. The exact same defect class existed in the
|
||
sibling function, `capsule_wirebind_unclean_detach()` (`capsule_wirebind.c:267`) — also no
|
||
device parameter, so an unrelated device detaching while a WIREBIND user's own stayed
|
||
attached would have incorrectly torn down that user's session too (not separately triggered
|
||
live, found by inspection once the first instance was understood).
|
||
|
||
**Fix:** both functions now take the departing `struct blkio_dev *dev` and compare it against
|
||
their own tracked device (`g_zuse_attached_dev`, new — mirrors `capsule_wirebind.c`'s own
|
||
pre-existing `g_wirebind_attached_dev` precedent exactly) before acting; every other value is
|
||
a no-op. `repl.c`'s detach loop passes the actual departing `usb_blk_dev` pointer it already
|
||
had in scope. `EJECT` (`mama_forth_words.c`'s `mama_word_eject()`, an explicit operator
|
||
command with no specific detaching device to check against) now calls a new accessor,
|
||
`capsule_zuse_boot_attached_dev()`, passing her own tracked device straight back in — trivially
|
||
matches when she's genuinely attached, correctly no-ops via the existing boolean guard
|
||
otherwise.
|
||
|
||
**Verified:** live, amd64 — a full 8-identity sequential attach/detach cycle (Zuse detached
|
||
first, then bob/rajames, 00-06 one at a time) with zero cross-contamination; `(zuse) ok>`
|
||
correctly persisted through unrelated detaches before this fix's own follow-on (§IX.3) closed
|
||
the remaining gap where her session ending correctly still left a bare prompt visible.
|
||
|
||
### IX.2 — Dictionary-entry memory leak (`vm_cleanup()`), causing a real kernel PANIC
|
||
|
||
**Bug:** repeating the WIREBIND birth/kill cycle 8-9 times in one boot (exactly what
|
||
identity verification does) produced a genuine kernel PANIC on the 8th-9th cycle every time:
|
||
cascading `vm_create_word: malloc failed` errors during capsule loading into the newly-born
|
||
VM, followed by `PANIC: Stadium: eviction selected patron zero (Hera) -- governor invariant
|
||
broken` and a full system halt. Root cause, found by tracing the allocation path: every
|
||
dictionary word (`vm_create_word()`, `dictionary_management.c:429`) allocates its own
|
||
`DictEntry` via `sf_malloc()` — separate from the VM's own `vm->memory` arena entirely — but
|
||
`vm_cleanup()` (**both** `src/starkernel/vm/vm_core.c:286` kernel and `src/vm_bootstrap.c:370`
|
||
hosted) freed `vm->memory`, the hotwords cache, rolling window, SSM state, and call stack, but
|
||
never walked the VM's own dictionary chain (`vm->latest`) to free the individual `DictEntry`
|
||
allocations — confirmed via grep that no such free existed anywhere in the codebase. Every
|
||
word a VM ever defined (hundreds per birth, given capsule loading) leaked permanently on kill.
|
||
Never noticed before this session: the hosted binary normally only calls `vm_cleanup()` once
|
||
at process exit (the OS reclaims everything anyway), and kernel VMs were normally born once
|
||
and kept alive for a whole boot, not repeatedly born and killed.
|
||
|
||
**Fix:** both `vm_cleanup()` implementations now walk `vm->latest` via each entry's own
|
||
`->link`, freeing `entry->transition_metrics` (a second, separate per-entry `sf_malloc()`,
|
||
`dictionary_management.c:455` — also unfreed anywhere) before the entry itself, saving the
|
||
next-link before each free since freeing destroys it.
|
||
|
||
**Verified:** live, amd64, the exact repeated-cycle scenario that panicked — no longer panics
|
||
alone (see §IX.3, this fix alone was insufficient until the allocator itself was also fixed).
|
||
|
||
### IX.3 — `sf_malloc`/`sf_free` was a bump allocator with a deliberate no-op free
|
||
|
||
**Bug, found immediately after §IX.2's fix didn't change the panic at all:** the kernel's own
|
||
`sf_malloc`/`sf_free` (`src/starkernel/vm/alloc_kernel.c`) is a fixed 4MB static bump arena
|
||
whose `sf_free()` was a **documented, deliberate no-op** — "VM is long-lived (no restart)...
|
||
free is a no-op... no fragmentation issues in practice." That premise was true until this
|
||
session's own repeated-birth-kill workflow. §IX.2's fix, while correct application-level
|
||
hygiene, was functionally inert against this specific allocator: calling `sf_free()`
|
||
diligently changes nothing when `sf_free()` itself throws the block away regardless.
|
||
|
||
**Fix:** `sf_malloc`/`sf_free` now implement a real free list. Every allocation carries a
|
||
fixed header (`size` + a free-list `next` link, live only while free) immediately before the
|
||
returned pointer. `sf_free()` pushes the block onto `g_free_list`; `sf_malloc()` searches it
|
||
first-fit before falling back to bump-allocating a fresh block from the arena, exactly as
|
||
before. Deliberately no splitting/coalescing — this workload's repeated allocations are for
|
||
the same capsules loaded into a fresh VM each time, so freed blocks from a just-killed VM's
|
||
dictionary are typically an exact or near-exact fit for the next VM's own, making first-fit
|
||
reuse the right simplicity tradeoff. `sf_realloc()` (pre-existing "orphans the old block"
|
||
behavior) was deliberately left untouched — out of scope for this fix, no evidence it's
|
||
exercised in the failure path.
|
||
|
||
**Verified:** live, amd64, the full 8-identity repeated birth/kill cycle that panicked at the
|
||
identical point on every prior run (confirmed twice) — no panic, all 9 sessions (Zuse + 8)
|
||
confirmed in one continuous boot. Three-arch `clean qemu` acceptance also re-run clean after
|
||
§IX.1/§IX.3/§IX.4 landed together.
|
||
|
||
### IX.4 — Headless-until-login gate didn't re-engage after a mid-boot logout
|
||
|
||
**Bug:** §VIII's original headless-console gate (`sk_console_mark_login()`/`sk_console_
|
||
login_occurred()`) was a one-way sticky flag, gating only the very first login of the boot
|
||
before `sk_repl(mama)` was ever called. Once tripped once, it never reset — a later full
|
||
logout (nobody attached at all) fell through to a bare, unauthenticated `ok>` instead of going
|
||
silent again, for the rest of the boot. Found live during the same 8-identity verification
|
||
run, flagged directly: *"we should never see an unauthorized, no session forth prompt."*
|
||
Fixing just the top-level loop wasn't enough either: the console is typically sitting blocked
|
||
inside `sk_console_readline()` waiting for input at the moment a logout happens (via hot-
|
||
unplug detected from *inside* that same call's own idle-service loop), so a check only at the
|
||
top of `sk_repl_run()`'s loop wouldn't take effect until a line was actually typed next.
|
||
|
||
**Fix:** `sk_console_mark_login()`/`sk_console_login_occurred()` retired entirely, replaced by
|
||
a live check, `sk_console_identity_present()` (`repl.c`) — mirrors `sk_print_prompt()`'s own
|
||
`zuse_session`/WIREBIND-username check exactly, evaluated fresh every time rather than latched
|
||
once. Three call sites updated: `sk_repl_headless_wait()`'s own loop condition; a new check at
|
||
the top of `sk_repl_run()`'s main loop (re-enters headless wait whenever nobody is currently
|
||
attached, not just before the very first login); and, critically, inside `sk_console_
|
||
readline()`'s own idle branch itself — when the identity present at prompt-print time logs out
|
||
while the call is still blocked waiting for input with nothing typed (`n == 0`), it now
|
||
returns -1 instead of re-printing a (correctly bare, but still unauthenticated) prompt and
|
||
continuing to wait. Callers with `reanchor_prompt` nonzero (the REPL's own top-level prompt
|
||
sites) check for -1 and loop back without printing `" ok"`, landing back at the top-of-loop
|
||
check, which re-enters silence immediately. `shim.c`'s `fgets()` (`reanchor_prompt == 0`)
|
||
never receives -1.
|
||
|
||
**Verified:** live, amd64, full 8-identity sequential cycle (Zuse's own detach, then each of 8
|
||
WIREBIND users' own detach) — grepped the entire session's serial log for every prompt-prefix
|
||
occurrence: 9 total `ok>` lines, every single one carrying a name (`(zuse)`, `(rajames)`,
|
||
`(00)`...`(06)`) — zero bare, unauthenticated prompts anywhere in the run. Three-arch
|
||
`clean qemu` acceptance re-run clean with this fix included.
|
||
|
||
### IX.5 — `MINT` now verifies its own write
|
||
|
||
**Motivation:** a separate live test (attaching Zuse + 8 identities simultaneously without
|
||
ever detaching any, exercising genuine 3+-device concurrent attach for the first time) hit a
|
||
4th-device USB enumeration failure (`xhci: not a Mass Storage/SCSI/BOT device`) — a real,
|
||
still-open defect (not yet root-caused, tracked as follow-on work, not part of this closure).
|
||
That failure raised a fair question: could `MINT` itself silently report success without the
|
||
identity actually being readable back afterward? `capsule_mint_identity()`
|
||
(`capsule_mint.c`) previously trusted every `blkio_write()`'s own `BLK_OK` return as
|
||
sufficient proof; it never confirmed any of it was actually readable back.
|
||
|
||
**Fix:** a new `verify_mint()` (`capsule_mint.c`) runs immediately after all writes and the
|
||
flush, before returning `MINT_OK`: re-runs `homeblocks_sig_check()` — the exact same check a
|
||
real later attach performs — then separately re-reads the identity record devblock and checks
|
||
magic/version/CRC/full-content match against what was written. A new result code,
|
||
`MINT_ERR_VERIFY_FAILED` (`capsule_mint.h`), is returned (and reported by the `MINT` word's
|
||
own existing console switch, `mama_forth_words.c`) if any check fails. Logged entirely via
|
||
`log_message()` (`LOG_ERROR` per specific mismatch, `LOG_INFO` on success) rather than
|
||
`console_println()`, per direct instruction — no added unconditional console/serial noise;
|
||
the `MINT` word's own final pass/fail line still reports to the console as it already did for
|
||
every other `MintResult` case.
|
||
|
||
**Not yet re-verified against a real failure** — every mint performed after this fix landed
|
||
(the original 8 identities were already minted before it existed) has succeeded, so the
|
||
failure-path logging itself hasn't been exercised live yet, only compiled. Three-arch
|
||
`clean qemu` acceptance passed with this change included.
|
||
|
||
**Still open, not addressed by this closure:** the 3+-simultaneous-device USB enumeration
|
||
failure that motivated §IX.5 (`xhci: not a Mass Storage/SCSI/BOT device` on a 4th concurrent
|
||
attach) remains unexplained and unfixed — tracked as the next item, separate from the four
|
||
closures above.
|
||
|
||
## X. §IX.5 follow-on: xHCI concurrent enumeration, root-caused and CLOSED; identity VM heap
|
||
capacity measured, leak fixed, allocator routed onto the real kernel heap — CLOSED, 2026-09-06/07
|
||
|
||
Picks up exactly where §IX.5 left off. Two separate root causes, both real, found by actually
|
||
attaching multiple devices live rather than reasoning about the driver in the abstract — same
|
||
lesson §VII and §IX already taught this project twice over.
|
||
|
||
### X.1 — The "4th-device enumeration failure" was never a driver bug — CLOSED, commit `30c26ad`
|
||
|
||
Root cause, confirmed live via QEMU's own `info usb` monitor command *before touching any
|
||
kernel code*: the default `qemu-xhci` controller (`p2=4,p3=4`) exposes only **4 real dual-role
|
||
ports**, not 8 as the parameter names suggest (p2/p3 count the *same* physical ports from the
|
||
USB2/USB3 register sets, not additive). Every device attach in this project used
|
||
`-device usb-storage,bus=xhci0.0,drive=...` with no explicit `port=`; past the 4th device, QEMU
|
||
silently auto-inserts an internal USB2 hub and everything past that is invisible to a driver
|
||
with no hub-descent logic. The driver's `"not a Mass Storage/SCSI/BOT device"` warning was
|
||
**correct** — the device really was a hub. The per-slot xHCI/BOT machinery from §VII was
|
||
inspected and found structurally sound for real top-level-port devices; no live corruption bug
|
||
was in it.
|
||
|
||
**Fix, entirely in the QEMU test harness, not the kernel:** new `XHCI_PORTS` Make variable
|
||
(`Makefile.starkernel`, default 16, overridable) sizes `p2`/`p3` on all three arches with real
|
||
headroom above the current 9-device identity roster — sized deliberately, not exactly matched
|
||
to today's count, per the standing ruling against hardcoding a bound to today's scale (§VII.4).
|
||
`ZUSEDISK_QEMU_ARGS` now gives Zuse's drive an explicit `port=1`; `QEMU_EXTRA`'s own doc comment
|
||
shows the pattern for attaching more devices.
|
||
|
||
**Found and fixed in passing:** `ZUSEDISK`'s Makefile default (`disk/zuse.img`) was stale —
|
||
deleted from git at `c3db963`, superseded by `disk/thumbdrives/zuse-thumb-ident.img`, but the
|
||
default was never updated, so a plain `make qemu` silently booted with no Zuse device attached
|
||
at all. Corrected to the real minted image.
|
||
|
||
**Flagged, not touched (not asked):** `scripts/bleach_zuse_img.sh` and `disk/README.md` still
|
||
reference the deleted `disk/zuse.img` path.
|
||
|
||
Three-arch `clean qemu` acceptance (single Zuse device, port 1) passed on amd64/aarch64/riscv64
|
||
with this change.
|
||
|
||
### X.2 — The real driver bug: Configure Endpoint completions dropped under concurrency — CLOSED, commit `e10fb76`
|
||
|
||
With X.1's port fix in place, actually booting the real kernel with Zuse + all 8 identity
|
||
drives attached concurrently (9 devices, each its own real port) surfaced a second, genuine bug:
|
||
only **1 of 9** devices ever reached `blkio_usb: MSC device ready`. No error and no success for
|
||
the other 8 — silence, not a diagnosable failure.
|
||
|
||
Root-caused live via temporary WARN-level diagnostic probes (written, captured, fully reverted
|
||
— the committed fix contains no probe code) at four points in `xhci.c`: the initial port scan,
|
||
the connect handler, the Command Completion Event handler, and the deferred per-slot dispatch
|
||
loop. The probes disproved the first hypothesis (a stalled `pending_events[]` connect queue —
|
||
all 9 devices actually completed Enable Slot + Address Device correctly) and found the real one:
|
||
`xhci_poll_events()`'s deferred dispatch loop submitted `XHCI_NEXT_ACTION_CONFIGURE_ENDPOINT` (a
|
||
Command Ring op) **unconditionally** for every slot with that action pending in a single pass —
|
||
unlike Enable Slot/Address Device/Disable Slot, which are correctly gated behind
|
||
`dev->connect_state == XHCI_CONN_IDLE` before ever submitting. With 2+ devices enumerating at
|
||
once, this let multiple Configure Endpoint commands sit outstanding on the Command Ring
|
||
simultaneously. Their completion is correlated purely via the single shared `dev->connect_state`
|
||
field, not the completion event's own Slot ID — so whichever slot's completion happened to land
|
||
while `connect_state` still read `AWAIT_CONFIGURE_ENDPOINT` got correctly chained into
|
||
`SET_CONFIG`; every other slot's completion arrived after `connect_state` had already moved on
|
||
and was silently swallowed by the handler's generic "unrelated command completion" catch-all —
|
||
no error path exists for that catch-all, which is exactly why it produced silence rather than a
|
||
diagnosable failure.
|
||
|
||
**Fix:** the same single-in-flight discipline this driver already uses for every other Command
|
||
Ring op. If the Ring isn't free when a slot's Configure Endpoint action comes due, put the
|
||
action back on that slot instead of racing a second command onto a busy Ring; the next tick's
|
||
dispatch pass retries it.
|
||
|
||
**Verified live:** the same 9-device concurrent boot now produces 9 `MSC device ready` lines and
|
||
zero xHCI errors (was 1 of 9). Three-arch single-device regression (the standard acceptance
|
||
case) still passed on all three arches.
|
||
|
||
### X.3 — Live 9-device hotplug walkthrough, on request: dynamic attach path also verified, not just boot-time scan
|
||
|
||
Separate from X.1/X.2's boot-time-attach testing, Captain Bob directed a live walkthrough: one
|
||
QEMU instance up with nothing attached, then Zuse, bob/rajames, and 00 through 06 hotplugged one
|
||
at a time via QMP `drive_add`+`device_add` on distinct `XHCI_PORTS`, watching the kernel's live
|
||
reaction after each. This exercises `xhci_poll_events()`'s Port Status Change *event* path
|
||
(true hotplug), not `xhci_scan_ports_for_already_connected()`'s boot-time initial-scan path
|
||
(X.1/X.2's scenario) — a genuinely different code path through the same driver.
|
||
|
||
Every hotplug through 04 succeeded cleanly: `blkio_usb: MSC device ready`, correct distinct
|
||
per-device identity readback (a temporary probe read each drive's own `full_name`/`username`
|
||
straight off its own devblock via `blkio_read()`, confirming e.g. slot 3 really said `"00"`, not
|
||
Zuse's or bob's data — no cross-contamination), and a live `WIREBIND` birth per identity. Six
|
||
identities (including Zuse, who costs no extra VM) were live simultaneously with zero errors.
|
||
The 05 hotplug is where X.4 below picks up — its USB/xHCI attach itself succeeded fine (further
|
||
confirming X.2's fix generalizes to the hotplug path, not just boot-time scan); what failed next
|
||
was a different layer entirely.
|
||
|
||
### X.4 — Identity VM kernel-heap capacity: exact numbers measured live, leak FIXED (commit `d8a195b`), architecture question RESOLVED same day — sf_malloc/sf_free now routed onto the kernel's real heap (commit `56e19a0`)
|
||
|
||
X.3's 05 hotplug attached at the USB layer but then failed at `WIREBIND`'s VM-birth step with a
|
||
kernel allocator exhaustion error (`vm_create_word: malloc failed`, `vm_init: rolling window
|
||
malloc failed`, `PARITY:BIRTH_FAILED`) — graceful, no crash, boot continued to `(zuse) ok>`, but
|
||
a real scalability wall. Measured precisely (not estimated) via a temporary allocator-stats
|
||
probe in `capsule_run.c`'s `capsule_parity_log_birth`/`capsule_parity_log_birth_failed` (written,
|
||
captured, fully reverted — no probe code committed), replaying the same hotplug sequence:
|
||
|
||
- Kernel heap arena (`alloc_kernel.c`, `SF_ARENA_SIZE`) is a **fixed 4 MiB (4,194,304 bytes)** —
|
||
does not grow.
|
||
- Boot baseline (Hera + Hermes + Artemis, the 3 system VMs): **781,904 bytes**.
|
||
- Zuse costs **nothing extra** — she authenticates into the existing Hera session rather than
|
||
birthing a new VM.
|
||
- Every other identity (`WIREBIND`) costs **exactly 457,392 bytes, with zero variance**,
|
||
confirmed identical across bob, 00, 01, 02, 03, 04. This is two VM births per identity: a
|
||
"default personality" console VM (228,576 bytes) then the real user session (228,816 bytes).
|
||
- **6 identities fit** (bob + 00 + 01 + 02 + 03 + 04): arena at 3,526,256 / 4,194,304 used,
|
||
668,048 free.
|
||
- **The 7th identity (05) failed — not from clean capacity exhaustion.** 668,048 bytes were
|
||
free, more than the usual 457,392 needed, but the attempt died partway through (peak
|
||
3,876,624, only 350,368 bytes into itself), with individual 128-136 byte word-registration
|
||
allocations failing. 662 `sf_free()` calls happened *during* that single failed attempt —
|
||
background heartbeat/physics activity from the 6 already-live VMs churning the same arena
|
||
concurrently. The allocator's free-list is documented as first-fit with **no splitting or
|
||
coalescing**, so fragmentation from that concurrent churn is the likely proximate cause, not
|
||
raw capacity.
|
||
|
||
**The bug found and FIXED here:** the failure was *leaking* memory, permanently, on top of the
|
||
fragmentation problem. `capsule_wirebind_try_attach()` births its two VMs (console, then user)
|
||
in sequence; when the second failed, the console VM that had *already succeeded* was never torn
|
||
down — an orphan nothing ever points a real user at, since `WIREBIND` only ever hands the caller
|
||
the user VM's id. Confirmed live: after the failed attempt, `used_bytes` settled at 3,754,832 —
|
||
228,576 bytes *above* the pre-attempt baseline, i.e. one whole console VM's worth, permanently
|
||
gone. Every failed attach was making the arena strictly worse for the next attempt, not neutral.
|
||
|
||
Fix: `capsule_vm_kill(username)` on the now-orphaned console VM before returning from the
|
||
failure path in `capsule_wirebind.c` — the same teardown `capsule_wirebind_eject()`/
|
||
`capsule_wirebind_unclean_detach()` already use elsewhere in that file (`vm_cleanup()` +
|
||
`sf_free()`, confirmed live in §IX.2/§IX.3 to actually reclaim per-word dictionary allocations).
|
||
Verified live with the same probe, replaying the identical failure: `used_bytes` after the fix
|
||
returns to **exactly** the pre-attempt baseline (3,526,256, matching precisely) instead of
|
||
leaking 228,576 bytes. Three-arch acceptance passed with the fix included.
|
||
|
||
**The architecture question — decided and RESOLVED same day, commit `56e19a0`.** Captain Bob's
|
||
framing: the number of concurrently-running VMs is not knowable in advance, and once this is a
|
||
complete operating system the heap should be able to use whatever memory is actually available,
|
||
not a hardcoded compile-time ceiling. Turned out this was already half-built and just not wired
|
||
up: `src/starkernel/memory/kmalloc.c` — the kernel's *general* heap, unrelated to
|
||
`alloc_kernel.c`'s isolated 4 MiB VM-dictionary arena — was already initialized at boot (M6,
|
||
well before any VM is ever born), already reserved from real PMM-tracked physical memory rather
|
||
than a fixed array, already defaulted to a 2 GiB floor explicitly commented "for 256+ baby VMs",
|
||
already overridable via `--heap=`, and its free list already coalesces on every free. VM
|
||
dictionaries (`vm_create_word()`, via `sf_malloc()`/`sf_free()`) were simply never routed
|
||
through it.
|
||
|
||
Fix: `alloc_kernel.c`'s `sf_malloc()`/`sf_free()` now delegate to `kmalloc_aligned()`/`kfree()`
|
||
instead of managing their own separate arena. `sf_alloc_init()` becomes a no-op (kmalloc is
|
||
already initialized by the time any VM allocation can happen; "resetting" a heap now shared by
|
||
every kernel subsystem would be actively wrong — confirmed no external caller depended on the
|
||
old reset semantics). Kernel-only: the hosted StarForth build keeps its own separate
|
||
`alloc_host.c`, untouched.
|
||
|
||
**Verified live, replaying the exact same hotplug sequence:** all 9 identities (Zuse + 8) now
|
||
attach successfully — identity 05, which specifically failed at the old 4 MiB ceiling, now
|
||
succeeds. Three-arch clean qemu acceptance (single Zuse device, standard regression) passed on
|
||
amd64/aarch64/riscv64 (one aarch64 attempt hit an unrelated, already-documented one-off QEMU
|
||
hiccup — empty log, boot never left firmware — and passed cleanly on immediate retry with no
|
||
rebuild).
|
||
|
||
**Correction, 2026-09-07 (same day):** an earlier draft of this section claimed `kmalloc.c`'s
|
||
free list "coalesces but doesn't split." That was wrong — never actually verified before being
|
||
written, just carried over from `alloc_kernel.c`'s own doc comment about *itself* lacking
|
||
splitting, mis-applied to `kmalloc.c` too. Re-read on request: `allocate_from_block()` already
|
||
implements real, unconditional block splitting (`can_split()`/`min_splittable_payload()`) — a
|
||
free block larger than a request needs, with enough leftover to be worth it, gets its remainder
|
||
carved off and threaded back onto the free list as its own block. Combined with
|
||
`coalesce_neighbors()` on every `kfree()`, this allocator already has both halves of real
|
||
fragmentation resistance. Nothing to fix here.
|
||
|
||
**Not addressed, still open:** per-VM dictionary sizing (shrinking what each `WIREBIND` user
|
||
VM's word set actually needs — it currently carries the full set, including things like
|
||
`TTF-TEXT`/`SCROLL-BACK`/`KBD-SCAN` a non-console user session may not need) remains a separate,
|
||
unexplored lever, not required by anything above but still worth revisiting if identity count
|
||
grows large enough for it to matter again.
|
||
|
||
**Also flagged, not touched (not asked):** `Makefile.starkernel`'s
|
||
`printf '$(KERNEL_ARGS)\n' > starforth.cfg` breaks when `KERNEL_ARGS` starts with `--` (dash's
|
||
`printf` misreads it as an option). Worked around live during X.2's debug-level testing with a
|
||
harmless prefix token (`KERNEL_ARGS="x --log-level=debug"`); not fixed in the Makefile itself.
|