Add 3x9 factorial analysis of std79-doe heartbeat/physics telemetry
Build / build-amd64-iso (push) Waiting to run
Build / build-aarch64-iso (push) Waiting to run
Build / build-riscv64-img (push) Waiting to run

Correlates every HB-ON/HB-OFF heartbeat-tick CSV row (results-20260912-
with-heartbeat-csv/*-doe-raw.log) back to which trial (run_id, id_idx,
id_label, rep) was active when it printed, despite the async tick
printer splicing rows mid-token -- including mid a DOE-RUN marker itself
-- into the trial loop's own console output on the shared serial line.

Pipeline (analysis-20260912/, see its own README.md):
- correlate_doe.py: two-pass reconstruction per architecture (remove
  atomic CSV-row spans to rebuild the clean trial-output stream, map
  each removed row's offset back to the nearest preceding run_id
  marker); identity/rep looked up from a known-clean prior run's
  run_id mapping rather than re-parsed, since one aarch64 marker
  (trial 12, rajames rep 1) lost its id_idx digit to a zero-separator
  collision with an adjacent CSV field and is unrecoverable from that
  log alone -- its rows fold into trial 11 instead, documented as a
  known limitation.
- combine.py: merges all three architectures into combined.csv (768
  rows), decoding Q48.16 fields to floats and jitter_bits' IEEE754 bit
  pattern to real jitter_ns.
- analysis.R: per-cell (architecture x identity) means/SD and two-way
  ANOVA for each of 12 telemetry metrics, one boxplot SVG per metric,
  written up as ANALYSIS.md.

Key findings: identity significantly affects word-heat/window-sizing
metrics (expected -- different identities execute different word
sets), architecture significantly affects timing metrics (APIC
ticks/tick, timing variance, fleet heat -- expected, different QEMU
targets), zero architecture x identity interaction on any metric.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo
This commit is contained in:
Robert Allan James
2026-09-12 12:46:34 -04:00
co-authored by Claude Sonnet 5
parent 403a7639e1
commit 934be5a257
19 changed files with 5872 additions and 0 deletions
@@ -0,0 +1,55 @@
# std79-doe 3×9 factorial analysis (heartbeat/physics telemetry)
Read `ANALYSIS.md` for the report itself (key findings, per-metric cell means,
two-way ANOVA tables, boxplot SVGs). This file just documents the pipeline.
## Pipeline
1. **Source data:** `../results-20260912-with-heartbeat-csv/{amd64,aarch64,riscv64}-doe-raw.log`
— raw QEMU serial logs from the `HB-ON`/`HB-OFF`-instrumented std79 DoE campaign
(see `../README.md` and FABRIC-3.md §XV/§XVI/§XVII for the campaign itself).
2. **`correlate_doe.py <raw_log> <out_csv>`** — reconstructs which DoE trial
(`run_id`, `id_idx`, `id_label`, `rep`) was active for every heartbeat-tick CSV
row, despite the async tick printer splicing rows mid-token into the trial
loop's own console output on the shared serial line (see the script's own
docstring for the two-pass reconstruction method, and its `RUN_ID_MAP` for
why identity/rep are looked up from a known-clean prior run rather than
re-parsed from each log — one aarch64 marker was unrecoverable, see below).
Run once per architecture:
```
python3 correlate_doe.py ../results-20260912-with-heartbeat-csv/amd64-doe-raw.log amd64-correlated.csv
python3 correlate_doe.py ../results-20260912-with-heartbeat-csv/aarch64-doe-raw.log aarch64-correlated.csv
python3 correlate_doe.py ../results-20260912-with-heartbeat-csv/riscv64-doe-raw.log riscv64-correlated.csv
```
3. **`combine.py <in1.csv> [in2.csv ...] <out.csv>`** — merges the three
per-architecture correlated CSVs into `combined.csv` (committed, 768 rows),
decoding Q48.16 fixed-point fields to plain floats and `jitter_bits`' raw
IEEE754 bit pattern to a real `jitter_ns` value:
```
python3 combine.py amd64-correlated.csv aarch64-correlated.csv riscv64-correlated.csv combined.csv
```
4. **`analysis.R`** (run from this directory) — reads `combined.csv`, computes
per-cell (architecture × identity) summary statistics and a two-way ANOVA
for each of the 12 varying telemetry metrics, renders one boxplot SVG per
metric to `svg/`, and writes `ANALYSIS.md`:
```
Rscript analysis.R
```
Requires `dplyr`, `tidyr`, `ggplot2`, `svglite` (all present in this
environment already).
## Known limitation
aarch64's raw log lost 1 of 27 `DOE-RUN` markers to interleaving beyond
recovery — trial 12 (identity `rajames`, rep 1). Its rows fold into trial 11
(identity `rajames`, rep 0) in `combined.csv`, so aarch64's `rajames` cell
isn't perfectly separable between those two replicates specifically. Every
other cell, on every architecture, is unaffected — see `ANALYSIS.md`'s own
note and `correlate_doe.py`'s docstring for the full explanation.