admin/LithosAnanake
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
bcc72d00bb08fcf2a153ae0a9af5b9223ec09124
LithosAnanake/docs/working/scratch/TASK_LIST.md
T

12 KiB
Raw Blame History

StarForth Physics Runtime — TASK_LIST.md

Read this first if you are an AI assistant (Claude, etc.).

0. PRELUDE FOR AI AGENTS (MANDATORY)

You are assisting Captain Bob on the StarForth project.
This codebase implements a physics-driven FORTH-79 VM with non-standard architecture.
Your job is to implement small, isolated tasks from this file — not to redesign the system.

0.1. Codebase Context

Primary files involved in this work include (paths may vary slightly):

  • src/vm.c
  • src/main.c
  • src/physics_runtime.c
  • src/physics_hotwords_cache.c
  • src/physics_metadata.c
  • src/physics_pipelining_metrics.c
  • src/doe_metrics.c (or equivalent)
  • src/platform_time.c / platform_time.h
  • src/log.c / log.h
  • src/test_runner/... (test harness)

You must inspect existing behavior before changing anything.

0.2. Hard Rules

  1. ONE TASK AT A TIME

    • Only work on a single checkbox task from this file per session.
    • Do not “helpfully combine” multiple tasks.

  2. NO COMMITS WITHOUT APPROVAL

    • You are not allowed to commit any changes.
    • All changes must be presented as diffs or full files for Captain Bob to review first.

  3. INCREMENTAL, TESTED CHANGES

    • Every change must:
      • Compile cleanly (no new warnings).
      • Run existing tests that are relevant to the touched subsystem.
    • If tests fail: STOP, report precisely what failed, and do not patch around it.

  4. DO NOT REWRITE ARCHITECTURE

    • Do not:
      • “Simplify” the design.
      • Replace custom physics with generic algorithms.
      • Introduce new dependencies.
      • Convert C99 code to C++, Rust, Go, etc.
    • This architecture is intentionally weird. Respect it.

  5. KEEP CHANGES LOCAL

    • Don’t refactor unrelated files “for style.”
    • Don’t rename functions or structs unless explicitly required by a task.
    • Don’t move code between files unless a task says so.

  6. NO HIDDEN MAGIC

    • All behavior must remain:
      • Deterministic.
      • Explicit.
      • Transparent.
    • No background threads, no async I/O, no “smart” runtimes unless explicitly specified.

  7. WHEN IN DOUBT, ASK

    • If you are unsure about:
      • Semantics.
      • Intent.
      • Factor definitions for DOE.
      • Interpretation of this file.
    • Then: STOP and ask Captain Bob for clarification.

1. Working Rules (For Any Human or AI Helper)

  • ALWAYS test each change independently.
  • NEVER commit without Captain Bob’s explicit approval.
  • NEVER batch multiple tasks into one commit.
  • NEVER “refactor” outside the exact task.
  • NEVER change VM semantics without written approval.
  • If uncertain, STOP and ask.
  • All tasks must compile with zero warnings (at existing warning level).

2. Immediate Stability Tasks (Do These First)

These are purely safety / clarity tasks. No behavior changes unless absolutely necessary.

  • Add clarifying comments to physics-related code paths (heat, decay, window, prefetch) explaining intent, not implementation detail.
  • Audit the word execution path in vm.c for:
    • Null checks on prev_word, transition_metrics, and dictionary entries.
    • Safe handling of word_id range / bounds.
  • Verify that rolling_window, pipeline_metrics, and execution_heat paths cannot dereference NULL or uninitialized pointers.
  • Add compile-time guards (#if ENABLE_*) around physics features so they can be disabled without breaking the build.
  • Ensure DOE mode (--doe-experiment) writes only the CSV line to stdout as its final result (logs can exist but must not break CSV parsing).

3. Unified VM Signal & Wire Identification (Documentation + Light Scaffolding)

Goal: make explicit where the VM could emit signals, without fully implementing the event bus yet.

  • In a new doc PHYSICS_SIGNALS.md, list all potential signal points:
    • Execution signals:
      • before execution
      • after execution
      • heat increment
      • linear decay applied
      • rolling window update
      • prefetch attempt
      • prefetch hit/miss
      • transition recorded
    • Temporal signals:
      • heartbeat tick start/end
      • heartbeat rate change
      • inference cycle start/end
      • window stabilized/unstable
    • DOE / metrics signals:
      • DOE run start
      • DOE run end
      • metrics snapshot taken
  • Cross-reference each signal with its current location in code (file + function name).
  • Confirm in the doc which signals are:
    • MUST-HAVE
    • HIGH-VALUE
    • FUTURE / OPTIONAL

(No runtime changes yet; this is design + mapping only.)

4. VM Event Bus Foundation (Minimal, Non-Intrusive)

Goal: introduce a small, efficient event bus API without wiring it into everything at once.

  • Add vm_event_type_t enum in a new header vm_events.h:
    • Include only a minimal set to start:
      • VM_EVENT_HEAT_UPDATED
      • VM_EVENT_DECAY_APPLIED
      • VM_EVENT_WINDOW_UPDATED
      • VM_EVENT_PREFETCH_STATS
      • VM_EVENT_INFERENCE_COMPLETED
      • VM_EVENT_TICK_RATE_CHANGED
  • Add vm_event_t struct in vm_events.h with:
    • vm_event_type_t type
    • uint64_t timestamp_ns
    • A small tagged union for payloads (keep it conservative).
  • Implement vm_publish_event(VM *vm, const vm_event_t *ev) in a new vm_events.c:
    • For now: no queues, just a single optional callback hook.
  • Add registration API:
    • void vm_set_event_listener(VM *vm, void (*listener)(VM *, const vm_event_t *));
  • Write a simple test listener in the test harness that logs events when enabled.

Important: Do not wire every possible signal yet — just prove out the API.

5. Adaptive Heartbeat: Measurement Foundations

Goal: add pressure metrics without changing tick behavior yet.

  • Identify and document in PHYSICS_RUNTIME.md the signals that should contribute to “pressure”:
    • heat_rate (change in total heat over time)
    • window_variance or window stability proxy
    • prefetch_error_rate (1 - hit_ratio)
    • stale_word_ratio (stale / total)
  • Implement a small physics_pressure_state struct in physics_runtime.c (or a new module) to hold:
    • last_total_heat
    • last_tick_ns
    • smoothed_pressure (Q48.16 or double)
  • Add a function:
    • Q48_16 physics_compute_pressure(const VM *vm);
    • For now, just compute a simple weighted linear combination and log it.
  • Ensure pressure computation is side-effect-free and can be called safely from vm_tick().

No adaptive tick rate yet — just measurement.

6. Adaptive Heartbeat: Rate Control (Wired but Disabled by Default)

Goal: implement the math and the hooks, but keep behavior unchanged unless explicitly enabled.

  • In the heartbeat/worker code, introduce:
    • uint64_t base_tick_ns;
    • uint64_t min_tick_ns;
    • uint64_t max_tick_ns;
  • Implement a function:
    • uint64_t vm_compute_tick_interval_ns(const VM *vm);
    • Use:
      • pressure metric
      • clamp to [min_tick_ns, max_tick_ns]
      • return base_tick_ns if adaptive mode is disabled.
  • Add a config flag (compile-time or runtime) to enable/disable adaptive heartbeat.
  • When adaptive mode is enabled, call vm_compute_tick_interval_ns() in the tick loop.
  • Emit VM_EVENT_TICK_RATE_CHANGED when the tick interval changes.

By default, keep adaptive mode OFF so tests remain deterministic until approved.

7. Feedback Loops Standardization (FL1–FL4)

Goal: Make the four core loops explicit and documented:

  • FL1: Heat accumulation & decay
  • FL2: Hotwords cache promotion
  • FL3: Inference → decay slope tuning
  • FL4: Inference → rolling window tuning / prefetch ROI

Tasks:

  • In PHYSICS_RUNTIME.md, create a section “Feedback Loops (FL1–FL4)” describing:
    • Inputs
    • Internal state
    • Outputs
    • Where in code the loop is implemented.
  • Add inline comments in the corresponding C functions labeling them:
    • /* FL1: Heat accumulation & decay */
    • /* FL2: Hotwords cache promotion */
    • /* FL3: Inference → decay slope */
    • /* FL4: Inference → window tuning */
  • Ensure FL1–FL4 each have at least one optional vm_publish_event call (behind a compile-time or runtime flag).
  • Confirm that enabling/disabling physics features does not leave loops partially active.

8. DOE Subsystem Overhaul

Goal: Make DOE runs clean, deterministic, and data-rich.

8.1 CLI and Mode Handling

  • Confirm --doe-experiment flag is handled in cli.c / main.c.
  • Add an option --doe-header to print CSV header row once.
  • Ensure DOE mode:
    • Resets all physics metrics
    • Does not start the REPL
    • Exits with status 0 on success

8.2 Metrics Extraction

  • In doe_metrics.c (or similar), define a clear DoeMetrics struct with named fields:
    • workload_duration_ns
    • cpu_temp_delta
    • cpu_freq_delta
    • total_heat
    • hot_word_count
    • stale_word_count
    • effective_window_size
    • prefetch_attempts
    • prefetch_hits
    • prefetch_hit_ratio
    • decay_slope_q48
    • tick_interval_ns (if adaptive enabled)
    • any other core fields currently in the CSV row
  • Implement:
    • DoeMetrics metrics_from_vm(const VM *vm, uint64_t workload_duration_ns, int32_t cpu_temp_delta, int32_t cpu_freq_delta);
  • Implement:
    • void metrics_write_csv_header(FILE *fp);
    • void metrics_write_csv_row(FILE *fp, const DoeMetrics *m);
  • Ensure CSV field order is stable and documented in a comment.

9. Metrics & Logging Hygiene

Goal: Make logs and metrics compatible and non-chaotic.

  • Standardize log prefixes for physics/inference/DOE messages.
  • Ensure DOE CSV is single-line per run, no stray prints.
  • Verify logs go to stderr when in DOE mode, and CSV to stdout (or vice versa, but clearly separated).
  • Add an option to disable all logs except errors during DOE.

10. Future Pub/Sub & Multi-VM (Design-Only for Now)

Goal: Prepare for pub/sub and multi-VM sync without implementing it fully.

  • Create PUBSUB_DESIGN.md documenting:
    • VM event bus as the internal publisher
    • Potential external subscribers:
      • MamaForth
      • BastardForth
      • Monitoring/visualization tools
    • Message types for future:
      • heat sync
      • entropy deltas
      • inference parameter sync
      • tick-rate harmonization
  • Sketch how multiple VMs could register into a global registry and share physics signals.

11. Testing & Validation

  • Add unit tests for:
    • event dispatch with a dummy listener
    • pressure metric computation
    • tick interval computation (with mock pressure)
  • Add regression tests for:
    • --doe-experiment mode:
      • produces exactly one CSV line per run
      • no crashes, no leaks
  • Add a simple script (documented, not necessarily committed) to:
    • run N DOE runs
    • append to a CSV file
    • verify consistent column count.

12. Documentation

  • PHYSICS_RUNTIME.md — overview of physics components and feedback loops.
  • PHYSICS_SIGNALS.md — list of signals and where they live in the code.
  • PUBSUB_DESIGN.md — future-facing multi-VM/pub-sub design.
  • Update any existing high-level docs to mention:
    • adaptive heartbeat
    • physics runtime
    • DOE integration as first-class citizens.

13. Final Integration Checklist

  • Adaptive heartbeat compiles and runs with the feature disabled by default.
  • Event bus compiles and runs even with no listeners.
  • DOE mode works end-to-end and is documented.
  • No regressions in existing tests.
  • All new behavior is behind flags and does not surprise users.
  • Captain Bob has reviewed and approved each major step before any commit.

Remember:
You are not here to make it “normal.”
You are here to help build something no one’s seen before — one tiny, deterministic step at a time. Sec 1 & 2 done.

Reference in New Issue View Git Blame Copy Permalink