Robert Allan JamesandClaude Sonnet 5
934be5a257
Correlates every HB-ON/HB-OFF heartbeat-tick CSV row (results-20260912- with-heartbeat-csv/*-doe-raw.log) back to which trial (run_id, id_idx, id_label, rep) was active when it printed, despite the async tick printer splicing rows mid-token -- including mid a DOE-RUN marker itself -- into the trial loop's own console output on the shared serial line. Pipeline (analysis-20260912/, see its own README.md): - correlate_doe.py: two-pass reconstruction per architecture (remove atomic CSV-row spans to rebuild the clean trial-output stream, map each removed row's offset back to the nearest preceding run_id marker); identity/rep looked up from a known-clean prior run's run_id mapping rather than re-parsed, since one aarch64 marker (trial 12, rajames rep 1) lost its id_idx digit to a zero-separator collision with an adjacent CSV field and is unrecoverable from that log alone -- its rows fold into trial 11 instead, documented as a known limitation. - combine.py: merges all three architectures into combined.csv (768 rows), decoding Q48.16 fields to floats and jitter_bits' IEEE754 bit pattern to real jitter_ns. - analysis.R: per-cell (architecture x identity) means/SD and two-way ANOVA for each of 12 telemetry metrics, one boxplot SVG per metric, written up as ANALYSIS.md. Key findings: identity significantly affects word-heat/window-sizing metrics (expected -- different identities execute different word sets), architecture significantly affects timing metrics (APIC ticks/tick, timing variance, fleet heat -- expected, different QEMU targets), zero architecture x identity interaction on any metric. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EXieurDfDSsDFdnSyusuWo
56 lines
2.6 KiB
Markdown
56 lines
2.6 KiB
Markdown
# std79-doe 3×9 factorial analysis (heartbeat/physics telemetry)
|
||
|
||
Read `ANALYSIS.md` for the report itself (key findings, per-metric cell means,
|
||
two-way ANOVA tables, boxplot SVGs). This file just documents the pipeline.
|
||
|
||
## Pipeline
|
||
|
||
1. **Source data:** `../results-20260912-with-heartbeat-csv/{amd64,aarch64,riscv64}-doe-raw.log`
|
||
— raw QEMU serial logs from the `HB-ON`/`HB-OFF`-instrumented std79 DoE campaign
|
||
(see `../README.md` and FABRIC-3.md §XV/§XVI/§XVII for the campaign itself).
|
||
|
||
2. **`correlate_doe.py <raw_log> <out_csv>`** — reconstructs which DoE trial
|
||
(`run_id`, `id_idx`, `id_label`, `rep`) was active for every heartbeat-tick CSV
|
||
row, despite the async tick printer splicing rows mid-token into the trial
|
||
loop's own console output on the shared serial line (see the script's own
|
||
docstring for the two-pass reconstruction method, and its `RUN_ID_MAP` for
|
||
why identity/rep are looked up from a known-clean prior run rather than
|
||
re-parsed from each log — one aarch64 marker was unrecoverable, see below).
|
||
Run once per architecture:
|
||
|
||
```
|
||
python3 correlate_doe.py ../results-20260912-with-heartbeat-csv/amd64-doe-raw.log amd64-correlated.csv
|
||
python3 correlate_doe.py ../results-20260912-with-heartbeat-csv/aarch64-doe-raw.log aarch64-correlated.csv
|
||
python3 correlate_doe.py ../results-20260912-with-heartbeat-csv/riscv64-doe-raw.log riscv64-correlated.csv
|
||
```
|
||
|
||
3. **`combine.py <in1.csv> [in2.csv ...] <out.csv>`** — merges the three
|
||
per-architecture correlated CSVs into `combined.csv` (committed, 768 rows),
|
||
decoding Q48.16 fixed-point fields to plain floats and `jitter_bits`' raw
|
||
IEEE754 bit pattern to a real `jitter_ns` value:
|
||
|
||
```
|
||
python3 combine.py amd64-correlated.csv aarch64-correlated.csv riscv64-correlated.csv combined.csv
|
||
```
|
||
|
||
4. **`analysis.R`** (run from this directory) — reads `combined.csv`, computes
|
||
per-cell (architecture × identity) summary statistics and a two-way ANOVA
|
||
for each of the 12 varying telemetry metrics, renders one boxplot SVG per
|
||
metric to `svg/`, and writes `ANALYSIS.md`:
|
||
|
||
```
|
||
Rscript analysis.R
|
||
```
|
||
|
||
Requires `dplyr`, `tidyr`, `ggplot2`, `svglite` (all present in this
|
||
environment already).
|
||
|
||
## Known limitation
|
||
|
||
aarch64's raw log lost 1 of 27 `DOE-RUN` markers to interleaving beyond
|
||
recovery — trial 12 (identity `rajames`, rep 1). Its rows fold into trial 11
|
||
(identity `rajames`, rep 0) in `combined.csv`, so aarch64's `rajames` cell
|
||
isn't perfectly separable between those two replicates specifically. Every
|
||
other cell, on every architecture, is unaffected — see `ANALYSIS.md`'s own
|
||
note and `correlate_doe.py`'s docstring for the full explanation.
|