Robert Allan JamesandClaude Sonnet 5
6573a6d1d5
These are data in their own right (the per-trial-labeled telemetry, one step before merging into combined.csv), not disposable scratch -- keep them alongside it rather than only documenting how to regenerate them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo
58 lines
2.9 KiB
Markdown
58 lines
2.9 KiB
Markdown
# std79-doe 3×9 factorial analysis (heartbeat/physics telemetry)
|
||
|
||
Read `ANALYSIS.md` for the report itself (key findings, per-metric cell means,
|
||
two-way ANOVA tables, boxplot SVGs). This file just documents the pipeline.
|
||
|
||
## Pipeline
|
||
|
||
1. **Source data:** `../results-20260912-with-heartbeat-csv/{amd64,aarch64,riscv64}-doe-raw.log`
|
||
— raw QEMU serial logs from the `HB-ON`/`HB-OFF`-instrumented std79 DoE campaign
|
||
(see `../README.md` and FABRIC-3.md §XV/§XVI/§XVII for the campaign itself).
|
||
|
||
2. **`correlate_doe.py <raw_log> <out_csv>`** — reconstructs which DoE trial
|
||
(`run_id`, `id_idx`, `id_label`, `rep`) was active for every heartbeat-tick CSV
|
||
row, despite the async tick printer splicing rows mid-token into the trial
|
||
loop's own console output on the shared serial line (see the script's own
|
||
docstring for the two-pass reconstruction method, and its `RUN_ID_MAP` for
|
||
why identity/rep are looked up from a known-clean prior run rather than
|
||
re-parsed from each log — one aarch64 marker was unrecoverable, see below).
|
||
Output committed as `{amd64,aarch64,riscv64}-correlated.csv` (255-257 rows
|
||
each, data in their own right — the per-trial-labeled telemetry, one step
|
||
before merging) — never discarded as scratch, regenerable exactly via:
|
||
|
||
```
|
||
python3 correlate_doe.py ../results-20260912-with-heartbeat-csv/amd64-doe-raw.log amd64-correlated.csv
|
||
python3 correlate_doe.py ../results-20260912-with-heartbeat-csv/aarch64-doe-raw.log aarch64-correlated.csv
|
||
python3 correlate_doe.py ../results-20260912-with-heartbeat-csv/riscv64-doe-raw.log riscv64-correlated.csv
|
||
```
|
||
|
||
3. **`combine.py <in1.csv> [in2.csv ...] <out.csv>`** — merges the three
|
||
committed per-architecture correlated CSVs into `combined.csv` (committed,
|
||
768 rows), decoding Q48.16 fixed-point fields to plain floats and
|
||
`jitter_bits`' raw IEEE754 bit pattern to a real `jitter_ns` value:
|
||
|
||
```
|
||
python3 combine.py amd64-correlated.csv aarch64-correlated.csv riscv64-correlated.csv combined.csv
|
||
```
|
||
|
||
4. **`analysis.R`** (run from this directory) — reads `combined.csv`, computes
|
||
per-cell (architecture × identity) summary statistics and a two-way ANOVA
|
||
for each of the 12 varying telemetry metrics, renders one boxplot SVG per
|
||
metric to `svg/`, and writes `ANALYSIS.md`:
|
||
|
||
```
|
||
Rscript analysis.R
|
||
```
|
||
|
||
Requires `dplyr`, `tidyr`, `ggplot2`, `svglite` (all present in this
|
||
environment already).
|
||
|
||
## Known limitation
|
||
|
||
aarch64's raw log lost 1 of 27 `DOE-RUN` markers to interleaving beyond
|
||
recovery — trial 12 (identity `rajames`, rep 1). Its rows fold into trial 11
|
||
(identity `rajames`, rep 0) in `combined.csv`, so aarch64's `rajames` cell
|
||
isn't perfectly separable between those two replicates specifically. Every
|
||
other cell, on every architecture, is unaffected — see `ANALYSIS.md`'s own
|
||
note and `correlate_doe.py`'s docstring for the full explanation.
|