Files
LithosAnanake/docs/lithosananke/ROADMAP.md
T
Robert Allan James 49a3faa331 G.1: xHCI bulk-endpoint stall recovery (per F.14), built + verified
Full BOT-spec stall recovery per FABRIC-3.md F.14: new STALL_ERROR handling,
Reset Endpoint + Set TR Dequeue Pointer commands, CLEAR_FEATURE(ENDPOINT_HALT),
escalating to Bulk-Only Mass Storage Reset, capped retries
(XHCI_BOT_STALL_MAX_RECOVERIES=2) mirroring bot_tur_retries, clean terminal
failure via xhci_stall_fail().

Purely additive recovery path off the non-success transfer-event branch; the
normal path is unchanged. Builds clean on amd64/aarch64/riscv64. QEMU amd64
boot regression passes: zero stalls, BOT attach (READ CAPACITY10 -> READ10 ->
home-blocks) completes, normal-path xHCI trace identical to baseline. Live
stall injection is not provable under qemu-xhci; deferred to v2.5.0 hardware.

FABRIC-3.md G.1 documented; ROADMAP release-versioning policy folded in.
2026-08-29 00:58:59 -04:00

16 KiB
Raw Blame History

LithosAnanke Roadmap

Branch: master (post-split: this repo's master is the sole LithosAnanke production line; the old lithosananke branch belonged to the pre-split combined monorepo and no longer exists here — see .claude/CLAUDE.md's "On the branch topology" note) Current: M7.1 in progress — see FABRIC.md/FABRIC-2.md/FABRIC-3.md for real status


Release Versioning Policy (decided 2026-08-28)

This is the standing release-versioning policy for LithosAnanke. It is authoritative; the FABRIC documents track implementation against it, and any permanent change to the policy belongs here.

  • X.0.0 — QEMU release. A whole-number minor in the X.0 position is the QEMU-only release: the point where the three-architecture (amd64/aarch64/riscv64) story under QEMU is complete and defensible on its own terms. Nothing real-hardware-only is required to cut an X.0.0.
  • X.5.0 — Hardware bare-metal release. A X.5 release is the real-hardware release: the point where the same story transfers to bare metal on real boards (SER5, RasPi 6, Milk-V, Zynq FPGA). Real-hardware-only work — the per-arch RNG drivers behind the unified entropy entry point, and real-board boot validation — gates X.5.0, not X.0.0.
  • Even major numbers are LTS. A release whose major number (X) is even is a Long-Term-Support release — maintained and supported long-term (bug fixes, security, backports) rather than a point-in-time cut. Odd major numbers are non-LTS development lines.

Applied to the two planned releases: the v2.0.0 cut (even major, so LTS) is the QEMU release; v2.5.0 is the hardware bare-metal release that transfers v2.0.0's QEMU story to real boards. See FABRIC-3.md §G for the release-gate punch lists.


Milestone Overview

M0  UEFI Boot           ████████████████████ COMPLETE
M1  PMM                 ████████████████████ COMPLETE
M2  VMM                 ████████████████████ COMPLETE
M3  IDT                 ████████████████████ COMPLETE
M4  APIC                ████████████████████ COMPLETE
M5  Timer               ████████████████████ COMPLETE
M6  Heap                ████████████████████ COMPLETE
M7  VM Parity           ████████████████████ COMPLETE
M7.1 Capsules           ████████████████░░░░ LIVE — see FABRIC.md/FABRIC-2.md/FABRIC-3.md
M8  REPL                ░░░░░░░░░░░░░░░░░░░░ OBSOLETE — see FABRIC.md §25.5 item 4.4
M9  Block I/O           ░░░░░░░░░░░░░░░░░░░░ PLANNED
M10 Networking          ░░░░░░░░░░░░░░░░░░░░ FUTURE

Phase 1: Boot Foundation (M0-M6)

M0: UEFI Boot + Serial Output

Goal: Kernel prints "LithosAnanke booting..." to serial

Deliverables:

  • UEFI loader (uefi_loader.c)
  • BootInfo handoff (memory map, ACPI, framebuffer)
  • ExitBootServices
  • Serial console output

Validation:

StarKernel UEFI Loader
Loading...
RAW SERIAL UP

M1: Physical Memory Manager

Goal: Track and allocate physical page frames

Deliverables:

  • Bitmap allocator
  • Parse UEFI memory map
  • pmm_alloc_pages() / pmm_free_pages()
  • Statistics reporting

Validation:

PMM initialized.
PMM statistics:
  Total pages: 249450
  Free pages : 247921

M2: Virtual Memory Manager

Goal: 4-level paging, identity + higher-half mapping

Deliverables:

  • PML4 → PDPT → PD → PT setup
  • CR3 switch
  • vmm_map() / vmm_unmap()
  • Self-test

Validation:

VMM initialized (mapped RAM, CR3 switched)
VMM self-test: mapped OK at 0xffff800000000000

M3: Interrupt Descriptor Table

Goal: CPU exceptions and IRQ handling

Deliverables:

  • 256-entry IDT
  • ISR stubs (asm)
  • Exception handlers (div-by-zero, page fault, etc.)
  • IRQ routing framework

Validation:

IDT installed.

M4: APIC Timer

Goal: Local APIC initialization and timer IRQs

Deliverables:

  • Local APIC enable
  • APIC timer configuration
  • Spurious interrupt vector

Validation:

APIC: init...
APIC enabled (SIVR=0xFF).
APIC: init done

M5: Timer Calibration

Goal: Accurate time measurement

Deliverables:

  • TSC frequency detection
  • HPET calibration (when available)
  • PM Timer fallback
  • Relative vs. absolute trust levels

Validation:

Timer: init...
Timer: trust=1 (0=NONE,1=REL,2=ABS), TSC=0 Hz
Timer: init done

M6: Kernel Heap

Goal: kmalloc() / kfree() working

Deliverables:

  • Heap initialization
  • Allocation tracking
  • Statistics

Validation:

Kernel heap initialized.
Heap statistics:
  Total bytes: 16777176
  Free  bytes: 16777176

Phase 2: VM Integration (M7)

M7: VM Parity Validation

Goal: StarForth VM boots with reproducible dictionary hash

Deliverables:

  • VMHostServices abstraction
  • VM arena allocation (5 MB)
  • FORTH-79 word registration (295 words)
  • Parity checkpoint logging
  • Heartbeat thread start

Validation:

VM: bootstrap parity...
[HAL][host] VMHostServices table registered
VM arena allocated: 0xffff900000000000 (5 MB)
Registering FORTH-79 Standard word set...
PARITY:M7.1a word_count=295 here=0x30 latest_id=294 hash=0x684bbf2fa1d96d55
PARITY:OK
VM: parity bootstrap complete
Starting heartbeat...
APIC Timer: started

Commit: 6f350bc — M7: StarForth VM integration with parity validation


Phase 3: Capsule Architecture (M7.1)

M7.1: Init Capsule System

Goal: Content-addressed, immutable init capsules for VM birth

Status: Design Complete (see M7.1.md) — this section predates the actual capsule/Tripod/Stadium work and is not current. See FABRIC.md/FABRIC-2.md for real status (per .claude/CLAUDE.md's standing redirect) — Tripod (Hera/Hermes/Artemis), word-level ACL Phases 17, and the Stadium physics-economy migration are all live, not just designed. Most recently: item 4.6 (Artemis's block-heat arena migrated to Stadium admission-on-allocate) verified passing its 30-rep stress campaign on all three architectures 2026-08-18, after fixing a quota-grant ordering bug (capsule_birth.c) that had made every admission fail; see FABRIC-2.md Section H. The same pass also fixed a long-standing aarch64-only crash on BYE (wrong PSCI conduit — SMC has nothing to answer it under AAVMF/no-EL3 boot; switched to HVC), see FABRIC-2.md Section I.

Core Concepts:

Concept Description
DOMAIN Mama-only construction space — never visible to babies
PERSONALITY Baby-only identity — result of executing (p) INIT
(p) Production Truth-bearing capsules that birth VMs
(e) Experiment Mama-only workloads for DoE

Birth Protocol:

  1. Mama selects one production (p) capsule by content hash
  2. Mama validates eligibility (ACTIVE, PRODUCTION, not REVOKED)
  3. Mama allocates new VM
  4. INIT blocks copied to execution window (RAM blocks 02047)
  5. INIT blocks executed sequentially
  6. Execution window cleared
  7. VM begins life with PERSONALITY imprinted
  8. Mama logs PARITY:BIRTH vm_id=N capsule_id=X mode=p ...
  9. Mama increments capsule.birth_count

Deliverables:

  • CapsuleDesc struct (64 bytes, cache-aligned)
  • CapsuleDirHeader struct
  • xxHash64 implementation (freestanding)
  • capsule_validate() function
  • Birth protocol (PARITY:BIRTH logging)
  • DoE run logging (CapsuleRunRecord)
  • mkcapsule build tool

Key Design Decisions:

  • Content-addressed: capsule_id == content_hash
  • (p) Production vs (e) Experiment modes
  • One truth per VM — no shared/implicit base INITs
  • Mama holds all truths — (e) capsules never touch babies
  • Twins/variants are just VMs with same/similar capsules

Phase 4: Interactive Forth (M8)

⚠ OBSOLETE (2026-08-11). This entire M8 section is superseded by FABRIC.md §25.5 item 4.4 (REPL). The console/REPL design has moved past what's scoped here — do not use this section as a source for REPL or CANVAS design decisions. Kept in place for history only.

M8: REPL + Interactive Forth

Status: Scoped 2026-08-11 (this section). Substrate (interrupt-driven keyboard input, glyph rendering) already complete via Stadium's Console fabric work (FABRIC.md §25.5 items 4.3.54.3.7f) — see "What's already done" below. This scoping pass covers what's left: wiring the existing keyboard-event stream into the REPL's character input path.

Goal: Type Forth at the kernel keyboard (not just serial), get ok prompt. The REPL already works interactively over serial today (sk_repl_run(), src/starkernel/repl.c) — that is not what this milestone adds. What's missing is a second input source feeding the same loop.

What's already done, read from the code before scoping this, not assumed:

  • Interrupt-driven keyboard input on all three architectures (amd64 i8042, aarch64/riscv64 virtio-keyboard-pci + GIC/PLIC), converged behind one shared interface: KEY-EVENT ( -- keycode pressed -1 | 0 ) in src/word_source/keyboard_words.c, keycode in the Linux input keycode namespace. FABRIC.md items 4.3.54.3.5f, verified live on all three architectures with real keypresses.
  • Glyph rendering, both paths: the stroke-drawn font (TEXT, capsule-based, 113-glyph v1 repertoire, FABRIC.md 4.3.64.3.6g) and TrueType (TTF-TEXT, proportional spacing via real hmtx advance widths, FABRIC.md 4.3.74.3.7f). Neither is required for the REPL text itself, though — see the VT100 finding below.
  • The REPL's on-screen text already renders independently of both of the above. console_fb_init() (src/starkernel/hal/console.c) calls vt100_init(), which mirrors every console_putc()/console_puts() call to the framebuffer via the baked-in font_8x16.c raster font. This is already live — the REPL prompt and its output appear on screen today, serial-driven, no CANVAS/glyph-capsule dependency. This confirms §27.6's mockup: a fixed VT100 REPL strip (already working) plus a separate large CANVAS region above it (still unsized — see the last deliverable below).

What's actually missing, confirmed by reading the code, not assumed: src/starkernel/repl.c's sk_readline() reads characters via console_getc() (src/starkernel/hal/console.c), which is serial-UART-only — it never looks at KEY-EVENT. Separately, the FORTH-79 words KEY/?TERMINAL (src/word_source/io_words.c:84-108, shared/vendored source) are dead in the kernel build: KEY calls libc getchar(), which the kernel shim (src/starkernel/vm/host/shim.c:1206) hardcodes to return -1 (EOF); ?TERMINAL is a literal stub that always returns false. Neither path has ever been wired to a real keyboard.

Deliverables:

  • Minimal scancode→ASCII layout: a small lookup table (US QWERTY, unshifted/shifted rows only — no dead keys, no locale switching) driven by KEY-EVENT's Linux-namespace keycodes, plus shift-state tracking (KEY_LEFTSHIFT/KEY_RIGHTSHIFT press/release) since KEY-EVENT itself carries no modifier state today (confirmed by reading keyboard_words.c — deferred at 4.3.5f, not implemented anywhere).
  • Merge keyboard into console_getc()'s poll: extend it (or add a second function sk_readline() polls alongside it) to also drain KEY-EVENT and translate presses to ASCII via the layout table above. Serial stays a co-equal input source, not replaced — the entire acceptance/DoE harness injects FORTH commands over the serial socket (socat ... UNIX-CONNECT:$SERIAL_SOCK) and greps the serial log for ok>; breaking that path breaks make qemu, DOE_INJECT, and every screendump verification technique used throughout 4.3.x. Both sources feed the same byte stream into the existing sk_readline() — no new line-editing code needed, backspace/echo already work (sk_readline(), repl.c:83-141).
  • Wire KEY/?TERMINAL (io_words.c) to the same merged source under #ifdef __STARKERNEL__, for FORTH-79 compliance — any capsule calling these directly (not just the REPL) should get real input too, not a permanently-EOF stub. Small addition once the merge above exists.
  • CANVAS viewport sizing (§27.6's explicit deferral to this milestone): the large graphics region above the VT100 REPL strip needs an actual scrollable framebuffer viewport size decided (e.g. 640×480, per the original mockup) — layout/presentation, not input plumbing; do this last, after keyboard input works, since it doesn't block interactivity.

Explicitly out of scope for this milestone (minimal bar, matches "Line editing (backspace, minimal)"): extended (0xE0-prefixed) i8042 scancodes, virtio-input autorepeat (value=2) as actual character repeat, arrow-key cursor movement/history, non-US keyboard layouts. Deferred to a later item if ever needed — not blocking "type Forth at the kernel, get ok."

Validation:

LithosAnanke v0.3.0
ok 1 2 + .
3 ok

(via a real keypress on all three architectures, same live-keypress verification standard FABRIC.md 4.3.5f already established — not a serial-injected synthetic test, though serial injection must still work identically afterward.)


Phase 5: Persistence (M9)

M9: Block Storage

Goal: Read/write blocks to disk

Deliverables:

  • Block device driver — implemented via virtio_blk.c (virtio block device), not the AHCI/SATA driver originally scoped here; see .claude/CLAUDE.md's M9 note.
  • Block device abstraction — block_subsystem.c's unified LBN address space, backend-agnostic across RAM/RAMDRIVE/DISK/USB via the blkio_dev vtable.
  • BLOCK / BUFFER / UPDATE / FLUSH words — live, src/word_source/block_words.c.
  • Persistent dictionary — no evidence found of word definitions surviving reboot; not independently verified as done, left unchecked rather than assumed from the rest of this list being live.

Future Milestones

M10: Networking

  • VirtIO-net driver
  • TCP/IP stack (minimal)
  • DHCP client

M11: Process Model

  • Forth tasks
  • Scheduling
  • IPC

M12: Self-Hosting

  • Compile Forth on LithosAnanke
  • Edit/assemble/link cycle

Validation Commands

# Build
make -f Makefile.starkernel ARCH=amd64 STARFORTH_ENABLE_VM=1

# Run QEMU
make -f Makefile.starkernel ARCH=amd64 STARFORTH_ENABLE_VM=1 qemu

# Clean
make -f Makefile.starkernel ARCH=amd64 clean-kernel

Branch Relationship

master (hosted StarForth)
    │
    ├── lithosananke (kernel branch)
    │       │
    │       └── M7 complete, M7.1 design complete
    │
    └── starkernel-junkyard (legacy, can delete)

Success Criteria

LithosAnanke is successful when:

  1. M7 Parity — VM dictionary hash reproducible across boots
  2. M7.1 Capsules — Birth protocol enforced, provenance logged
  3. M8 REPL — Interactive Forth at bare metal — superseded, this criterion is now met via Stadium's Console fabric work (FABRIC.md §25.5 item 4.4), not a standalone M8; see the OBSOLETE banner above Phase 4
  4. M9 Persistence — State survives reboot

The foundation is laid. The necessity is clear.