diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index 1b71e57..e004a70 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -10,11 +10,12 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co > **Superseded subsystem docs (Captain Bob, 2026-08-15):** `.claude/TRIPOD.md`, > `.claude/HERMES.md`, `.claude/ARTEMIS.md`, and `.claude/CONSOLE.md` are all superseded — -> `FABRIC.md` (design history) and `FABRIC-2.md` (current/living, read this one first) are -> the sole authoritative source for Tripod/Hermes/Artemis/Console work now. The four -> subsystem docs remain in the repo as historical record only; each carries its own -> superseded-header pointing here. Do not treat them as current, do not read them for design -> authority, and do not cite them in place of `FABRIC.md`/`FABRIC-2.md`. +> `FABRIC.md` and `FABRIC-2.md` (both design history/archival as of 2026-08-25) and +> `FABRIC-3.md` (current/living, read this one first) are the sole authoritative source for +> Tripod/Hermes/Artemis/Console work now. The four subsystem docs remain in the repo as +> historical record only; each carries its own superseded-header pointing here. Do not treat +> them as current, do not read them for design authority, and do not cite them in place of +> `FABRIC.md`/`FABRIC-2.md`/`FABRIC-3.md`. > **Scope:** This repo is LithosAnanke — the bare-metal UEFI kernel that boots StarForth > directly on hardware. StarForth (the hosted FORTH-79 VM) has its own separate repository diff --git a/FABRIC-2.md b/FABRIC-2.md index df93036..14487ce 100644 --- a/FABRIC-2.md +++ b/FABRIC-2.md @@ -1,5 +1,25 @@ # FABRIC-2.md — the Stadium, continued +**Status: CLOSED, 2026-08-25.** This document is archival. It is not edited further and no +new work is added to it — everything still open as of the closure date was carried forward +into **`FABRIC-3.md`**, which is now the living document. Read `FABRIC-3.md` first for +current status; come back here only for the historical design argument and decision record +behind it (Milestones 2/2a–2h's xHCI/USB/block-subsystem work, the Stadium substrate build-out, +the ACL-TTL DoE campaigns, and everything else recorded below). + +**Why it's closed, not just superseded.** At ~4,400 lines and 51 open items scattered among +hundreds of resolved ones, continuing to append here made the still-open work hard to find — +the same reasoning `FABRIC.md` itself was closed for at 7,595 lines. Closed on request rather +than at a natural milestone boundary; nothing here was rushed to closure, the document just +stopped growing at this point. + +**The 2026-08-25 closure audit.** Every `- [ ]` item in the document was extracted +mechanically (`grep -c` confirmed 51, then a script pulled each item's own text without +carrying along unrelated resolved narrative) and carried into `FABRIC-3.md` §A, organized +under the same section groupings they already had here. None dropped, none re-prioritized. + +**Original opening note, kept for history:** + **Status:** Living working document, opened 2026-08-12 as the successor to `FABRIC.md` (now closed/archival — see its own header). This document does not repeat `FABRIC.md`'s design argument or history; it restates only outcomes, with pointers back to the section diff --git a/FABRIC-3.md b/FABRIC-3.md new file mode 100644 index 0000000..8af2e37 --- /dev/null +++ b/FABRIC-3.md @@ -0,0 +1,317 @@ +# FABRIC-3.md — the Stadium, continued again + +**Status:** Living working document, opened 2026-08-25 as the successor to `FABRIC-2.md` +(now closed/archival — see its own header). This document does not repeat `FABRIC-2.md`'s +design argument or history; it restates only outcomes, with pointers back to the section +that derived them. Read `FABRIC.md` for the original "why," `FABRIC-2.md` for everything +derived through 2026-08-25, this document for what's left as of that date onward. + +**Provenance.** Everything in Section A below is a full, non-sampled carry-forward of every +open (`- [ ]`) item in `FABRIC-2.md` as of 2026-08-25 — 51 items, confirmed by +`grep -c '^- \[ \]' FABRIC-2.md`, none dropped (Section A itself holds 49: the other 2, +`FABRIC-2.md` §F.3's own two checkbox lines, were pure summaries cross-referencing items +already listed individually elsewhere — 4.4s/1.11/4.3/§17.4 and 5.1/ACL-RWT re-measurement, +both of which are carried forward as their own individual items above — not distinct content, +confirmed by diffing item text programmatically before writing this document, not assumed). +Extracted mechanically (a script pulling each checkbox item's own text, stopping at the first +blank line rather than the next checkbox, to avoid pulling in unrelated already-resolved +narrative that happened to sit between two open items in the source document) and spot-checked +against the original. Item numbers/labels are +carried forward unchanged, for traceability — this is not a renumbering or a re-prioritization. +Section groupings match `FABRIC-2.md`'s own (documentation debt, xHCI WRITE(10), Milestone +3–9 punch lists, etc.) — items are relocated, not reorganized. + +**How to use this document going forward.** New findings, new punch-list items, and new +decisions get added here, not to `FABRIC-2.md`. Follow the same discipline `FABRIC-2.md` +§(intro) established for how work gets picked up, closed, and recorded. + +--- + +## A. Carried forward from FABRIC-2.md (51 items, all still open as of 2026-08-25) + +### From FABRIC-2.md §A — Blocked or scoped, not started + +- [ ] **1.11 — Dirty-event granularity.** Leaning region-based. Blocked on item 4.3 — settled + as part of the console migration, not speculatively before it. *Refs (FABRIC.md):* §17.5, + §23.2, §23.4 #1. + +- [ ] **4.3 — Console.** Umbrella item; settles 1.11 as part of the work. Nearly everything + under it (4.3.1–4.3.7f, 4.4–4.4ac) is done — the parent stays open only because 4.4s below + is still blocked and nothing has formally closed the umbrella. *Refs (FABRIC.md):* §17.5, + §27. + +- [ ] **4.4s — `(user)` prompt segment.** Scoped, blocked, not started. Extends 4.4's prompt + format. *Refs (FABRIC.md):* §27.8, 4.4. + +- [ ] **5.1 — Re-run the DoE on the new substrate.** A green POST suite is not evidence that + determinism holds under the Stadium migration — needs its own campaign. Not started. + +### From FABRIC-2.md §D — Design questions still genuinely open + +- [ ] **§17.4 — the framebuffer utility's internal heat/decay dynamics are undesigned.** + Explicitly "Open, deferred": not a Stadium patron, but what physics (if any) governs it + internally was never designed. Not blocking anything. Blocked on item 1.11 specifically + (dirty-event granularity), not "the framebuffer work" in general — see `FABRIC-2.md` §D's + own 2026-08-13 refinement of this item before assuming it's ripe. + +### From FABRIC-2.md §E — Documentation debt + +- [ ] **ACL-RWT DoE overhead re-measurement.** The measured overhead numbers in + `.claude/CLAUDE.md` ("+0.0054%–+0.0088%") were all captured at `-O0`, before item 4.5 + enabled real compiler optimization. Nobody has re-measured, or even confirmed the old and + new numbers are comparable at all. Flagged in passing during item 4.5f, never formally + scoped. **Ruled 2026-08-19 (`FABRIC-2.md` §F.3): wait for Artemis to land before + re-running** — Artemis has since landed; this is now unblocked but still not started. + +### From FABRIC-2.md §J — Maintainability sweep (2026-08-18) + +- [ ] `docs/lithosananke/ROADMAP.md` and `M7.1.md` — stale `Branch: lithosananke` (no such + branch exists post-split), `M7.1.md`'s "Status: Design Complete" (shipped and live, not + just designed), `ROADMAP.md`'s self-contradiction (M8 marked OBSOLETE in one place, still + a live success criterion in another), and its stale "AHCI driver" claim for M9 (real + implementation is `virtio_blk.c`) — not fixed, flagged. + +- [ ] Top-level `ROADMAP.md` (StarForth-era, "Phase 0 Complete... Phase 1 Starting," dated + 2025-12-14) — badly stale, no historical/superseded banner to warn a reader. Not fixed. + +- [ ] `docs/03-architecture/word-acl/DESIGN.md` says ACL Phase 7 (LithosAnanke kernel parity) + is still "remaining" — direct contradiction with `.claude/CLAUDE.md`, which states Phase 7 + is independently verified complete. Not fixed. + +- [ ] `VM-FLEET-ATTRACTOR-DESIGN-20260705.md` claims `doe-campaign.4th` is "broken and being + superseded" — unverified against repeated successful `L8-DOE` runs (a different FORTH entry + point; not confirmed either way). + +- [ ] Isabelle/HOL: the pipeline-metrics model/C-struct mismatch this sweep surfaced — + flagged in the `.thy` file itself, not independently tracked elsewhere, not fixed. + +### From FABRIC-2.md §X, Milestone 2 — USB hardware stack + +- [ ] Decide and implement where the hotplug event surfaces to the rest of the kernel — + likely a callback registered by whatever owns the home-blocks logic, not xHCI code calling + into `block_subsystem.c` directly (matching the existing "kernel/Artemis decoupling + boundary" pattern already documented in `block_subsystem.c`). **Partially addressed by + Milestone 2h's `blkio_usb.c`/connect-time attach wiring (`FABRIC-2.md`, 2026-08-25) — worth + re-checking whether that closes this item outright before treating it as still fully open.** + +- [ ] Implement CBW/data/CSW for SCSI WRITE(10) — this is where the earlier "read/write, + unquestionable" requirement actually gets satisfied. Still the single biggest functional + gap in the xHCI driver — blocks writing to a real USB thumb drive at all (`blkio_usb.c` is + read-only today specifically because of this). + +- [ ] Implement basic error/stall recovery (CSW failure status, endpoint stall clear) — at + minimum enough to not wedge the controller on a single bad transfer. + +### From FABRIC-2.md §X, Milestone 3 — Block subsystem extensions + +- [ ] Implement the CA-signed-cert verification path (Milestone 6 dependency — the cert chain + validator doesn't exist yet either). + +- [ ] Implement the first-touch allocation function: given a verified identity pubkey and a + requested block count, either read an existing range from the drive's map or claim a new + one at `g.total_user_lbn` and write it back. *(Single-block relocation itself — the + mechanism this would allocate ranges for — is done: `blk_subsys_relocate_block()`/ + `RELOCATE-BLOCK`, `FABRIC-2.md`, commit `36d832f`. This item is about the identity→range + allocation that decides what to relocate blocks* into*, still unbuilt.)* + +- [ ] Design the on-drive block-map format (Section U item 4) — what it records (block ranges + claimed? individual block liveness? something else), how it's serialized. + +- [ ] Implement writing the block-map to a drive. + +- [ ] Implement reading/validating the block-map from a drive on insertion. + +- [ ] Design the migration state machine (Section U item 5) — states, transition triggers. + Session direction, 2026-08-25: **ACL manages *when* to relocate** (capacity pressure, or a + compudynamics heat/cold signal); migration itself is expected to be rare, not routine. The + `physics_hotwords_cache.c`-reuse question is settled differently than originally framed — + see this document's new §B below (Stadium unification), which reframes block/word placement + as a `compudynamics.c`-driven decision generically, not a `physics_hotwords_cache.c` + (`DictEntry*`-hardcoded) reuse question specifically. + +- [ ] Decide and implement unclean-removal handling (Section U's explicitly flagged open + question — never answered) — at minimum, detect a mid-flush disconnect via Milestone 2e's + disconnect signal and decide what state that leaves affected blocks in. + +### From FABRIC-2.md §X, Milestone 4 — Drive/credential security + +- [ ] Design the home-blocks drive signature format (Section U item 7) — reusing + `CAPSULE_MAGIC_PACK`'s pattern (magic + version in a fixed header field) as the confirmed + precedent, applied to a drive's reserved header block instead of a capsule. + +- [ ] Implement the signature check, called before any write path touches a newly-inserted + drive. + +- [ ] Implement the warn-and-refuse behavior for blank/foreign/unrecognized media. + +- [ ] Extend `acl_pinned`'s one-way-ratchet mechanism (already exists, already proven, just + needs applying) to gate zuse credential minting specifically — confirm whether this + literally reuses the existing `acl_pinned` bit on some relevant `DictEntry`, or needs its + own analogous one-way flag on the credential data itself (the credential isn't a dictionary + word, so the existing bit may not directly apply — open question, not yet resolved). + +### From FABRIC-2.md §X, Milestone 5 — Console/VM key-match binding + +- [ ] Settle the still-open question: reuse `ACL-PIN`/`acl_allow` directly, or build a + separate key-matching primitive — `ACL-PIN` gates word execution specifically and nothing + today gates console-session-to-VM ownership, so this decision needs to happen before any + code gets written here. + +- [ ] Design the key/lock data shape (what the console presents, what the VM carries, how + they're compared). + +- [ ] Wire drive insertion (Milestone 2e's hotplug signal, post-identity-authentication) to a + call into `capsule_birth_baby()` (confirmed a real, callable, on-demand birth path already) + to spin up or re-attach that identity's VM. + +- [ ] Implement the actual attach/bind step — extending `sk_repl_set_active_vm()` (confirmed + to exist, currently an unguarded raw pointer-set) with the key-match check from above, so a + console can only bind to the one VM whose lock matches its key. + +- [ ] Implement detach behavior on console disconnect or VM teardown. + +### From FABRIC-2.md §X, Milestone 6 — Kernel/capsule PKI signing chain + +- [ ] Generate (offline, outside the kernel/repo entirely) the real root CA keypair — "stays + unrevocable," never embedded, never loaded by any kernel code. + +- [ ] Generate the "snakeoil" intermediate certificate, signed by that real root CA (this is + a real CA-signed intermediate, not a self-signed/untrusted cert despite the name — + "snakeoil" names its informal/private-project status). + +- [ ] Embed the already-CA-signed snakeoil intermediate as a capsule blob at build time + (mechanically proven already via the font-capsule precedent — no new embedding + infrastructure needed, just a new payload). **Bootstrapping resolved: no kernel-boot-time + verification of a hardcoded CA public key is needed at all** — trust is established once, + at build time, by whoever holds the real root CA and produces the build. + +- [ ] Add a signing step to the `mkcapsule` build tool (or a separate signing tool) that + produces a signature alongside each capsule's existing xxHash64. + +- [ ] Extend `MANIFEST_AUTO.md`'s generation to add a signature-status column, matching the + existing xxHash64 column's generation pattern. + +- [ ] Implement magic-number-based content-type detection (Section U item 14) — a shared + primitive, also usable for Milestone 4's foreign-drive check. + +### From FABRIC-2.md §X, Milestone 7 — Contributor capsules / trust tiers + +- [ ] Create the `capsules/contrib/` directory (mechanically trivial, matches existing + subdirectory convention — the directory itself is not the work). + +- [ ] Add a `FLAG_CONTRIB` bit to `mkcapsule.c`'s flag system, assigned by path match + (`contrib/` prefix), same pattern as how `init.4th` already gets `FLAG_MAMA_INIT`. + +- [ ] Decide and implement one of the four spitballed trust-tier directions (signature- + authority tiers / block-namespace sandboxing / QEMU-vs-real-hardware conditional + enforcement) — none chosen yet, this is a real decision point, not just an implementation + task. + +- [ ] If block-namespace sandboxing is chosen: extend `mkcapsule`'s existing conflict- + detection logic to also reject a `contrib/`-path capsule claiming blocks outside its + reserved range. + +### From FABRIC-2.md §X, Milestone 8 — Bare-metal boot from physical USB + +- [ ] Build a fresh `starkernel.iso` via `make -f Makefile.starkernel ARCH=amd64 clean` + the + ISO-build step. +- [ ] Identify the exact block device path for the target USB drive on the host doing the + flashing (`lsblk`/`dmesg` after insertion — care needed, wrong device = data loss). +- [ ] `dd if=build/amd64/kernel/starkernel.iso of=/dev/sdX bs=4M status=progress` (or + equivalent) — confirm `dd` is the right tool for an El Torito ISO vs. needing `isohybrid` + first (open question, not yet verified). +- [ ] Physically boot the real machine from the flashed drive (BIOS/UEFI boot-order menu, + Secure Boot may need disabling — unknown until tried). +- [ ] Capture what happens with no serial-socket log available (real hardware has no + `qemu-serial-*.sock` to `socat` into) — decide the observation method. +- [ ] Confirm POST reaches the same 1012/0/0 result on real hardware as every QEMU acceptance + run. +- [ ] Confirm `ok>` prompt is reachable and a basic command (e.g. `HEARTBEAT-TICKS@ .`) works + identically to QEMU. +- [ ] Document the result (pass/fail, and if fail, what diverged from QEMU) — first real + external validation this project has ever had outside QEMU TCG emulation. + +### From FABRIC-2.md §X, Milestone 9 — Networking / capsule distribution server + +- [ ] (Deferred) Revisit and punch-list this milestone once Milestone 7 closes, not before. + +--- + +## B. Stadium unification — words/VMs/blocks/messages on the same engine + +Raised 2026-08-25: "words are stadium patrons, VMs are patrons, blocks are patrons, messages +are stadium patrons, all should be operated on by THE SAME ENGINE." Investigated before +designing anything — the real state is more nuanced than "everything's a stub," verified via +direct reads and `git log`, not assumed: + +**`FABRIC.md` §18.3 already decided the mapping** (not invented here): blocks → `MIGRATE`, +messages → `DELIVER`, ACLs → `EXPIRE`, words and VMs both → `COOL`. `stadium_evict()` +(`src/starkernel/vm/stadium.c`) — real, tested infrastructure: bitmap tracking, pin/`contains` +refusal, the Hera-patron-zero panic guard, heat-conservation back to the owner's reservoir on +every reap — calls `stadium_dispatch()` for the actual payload action when a patron departs. + +**Per-behaviour status, as of 2026-08-25:** + +- **`MIGRATE` (blocks)** — zero consumer, genuinely stub (`stadium_dispatch()`'s case prints + `"MIGRATE (stub)"` and returns). This session already built the real mechanical primitive + it needs: `blk_subsys_relocate_block()`/`RELOCATE-BLOCK` (`FABRIC-2.md`, commit `36d832f`), + live-verified (redirect + content survive an abrupt kill and reboot) but never wired to + `stadium_dispatch()` — it's a separate, parallel, already-working mechanism today, not + routed through Stadium at all. +- **`COOL` (words *and* VMs, same tag)** — half real. **Words are fully live**, but via a + *separate, bespoke* mechanism, `stadium_word_dispatch()` (`stadium_words.c`, item 4.1), + wired directly into the real VM word-execution hot path (`vm_core.c:690,885,896`) — it does + **not** go through the generic `stadium_dispatch()` switch at all. `ONTOLOGY.md` §IX + claiming words are "not yet migrated" is itself stale documentation drift (same class of + bug as the "glibc" misattribution corrected earlier this session — flagged as a small, + separate fix below, not blocking). **VM cooling has no evidence of ever being wired + anywhere** — still genuinely stub. +- **`DELIVER` (Hermes messages)** — `FABRIC.md` (~line 3452) records this explicitly as + "Open, surfaced not resolved": Hermes's message/channel heat already integrates with + Stadium's reservoir accounting (`STADIUM-HEAT@`, `STADIUM-RES-PULL/PUSH`), but which Hermes + lifecycle event maps to `DELIVER` vs. `EXPIRE` was never decided, let alone wired. Real, + substantial, Hermes-specific integration work. +- **`EXPIRE` (ACL TTL expiry)** — no evidence of any wiring anywhere; `ACL.4th`/ + `acl_recheck()` has zero Stadium involvement today. Also substantial, separate work. + +**Why `DELIVER`/`EXPIRE` aren't being resolved in the same pass as `MIGRATE`:** each is a +full subsystem integration (Hermes lifecycle mapping; ACL-to-Stadium wiring where none has +ever existed) in its own right — attempting all four stubs at once risks exactly the rushed, +shipped-but-incomplete outcome the no-stubs rule (below) exists to prevent. `MIGRATE` gets +resolved for real because this session already has a tested primitive underneath it; the +other three become honest, explicit punch-list items instead of being touched speculatively. + +**Punch list:** + +- [ ] Wire `STADIUM_BEHAVIOUR_MIGRATE` in `stadium_dispatch()` to call + `blk_subsys_relocate_block()` for real. **Needs investigation first**: what LBN pair does a + block-patron's cell header actually carry at eviction time, and has anything ever admitted + a block patron onto a Stadium cell in the first place? (Item 3.5's admission code in + `stadium.c`/`stadium_words.c` needs tracing before wiring — if block-patron *admission* + itself doesn't exist yet, that's a bigger prerequisite than just the dispatch stub, and + should be reported back rather than built speculatively.) +- [ ] Scope VM-`COOL`: confirm whether `capsule_vm_kill()`/Tripod's existing VM lifecycle has + any real hook point equivalent to `stadium_evict()`, or needs the same kind of admission + work `MIGRATE` might need. Resolve for real if small; otherwise punch-list explicitly. +- [ ] `DELIVER` (Hermes) — scope as its own future item once `MIGRATE`/VM-`COOL` land; do not + re-decide the `DELIVER`-vs-`EXPIRE` message/channel mapping here, `FABRIC.md`'s own note + already flags it as genuinely open, not casually assumable. +- [ ] `EXPIRE` (ACL) — scope as its own future item; ACL has never touched Stadium, this is + new integration work, not a wire-up of something partially built. +- [ ] Fix `ONTOLOGY.md` §IX's "words (dictionary, warehouse-resident today, not yet migrated)" + line — stale, words are fully migrated and live via `stadium_word_dispatch()`. Small, + independent doc fix, not blocking any of the above. + +--- + +## C. Standing rule: no stubs or TODOs, ever + +Stated directly, 2026-08-25, after the `stadium_dispatch()` stub investigation above: +**"I've never allowed stubs before."** Saved as a persistent memory +(`feedback_no_stubs_or_todos.md`) so this applies across sessions, not just this one. Full +statement: no stub function that prints a placeholder and returns, no `TODO`-and-move-on +comment in place of real logic, in any language, ever committed as if it were finished work. +Small, honest increments are still fine and encouraged — each increment just has to be a +complete, real implementation of whatever slice it covers, never a placeholder for a later +slice. A pre-existing stub found while working nearby (as here) gets flagged and resolved, +not built on top of or left in place.