185 lines
8.0 KiB
TeX
185 lines
8.0 KiB
TeX
%% SCRAP: hardware/PGO_GUIDE
|
|
%% SOURCE: docs/working/hardware/PGO_GUIDE.adoc
|
|
%% STATUS: CURRENT
|
|
%% FITS: dev-guide/ch-profiling
|
|
%% EDITORIAL: lifted — prose rewritten to press voice
|
|
|
|
\section{Profile-Guided Optimization}
|
|
|
|
Profile-guided optimization (PGO) feeds runtime profiling data back into the
|
|
compiler to direct code generation. StarForth's PGO pipeline exercises every
|
|
major code path, producing a binary tuned to real-world execution patterns.
|
|
For typical workloads PGO yields a \textbf{5--15\% improvement} over a
|
|
standard \texttt{-O3} build by predicting hot paths correctly, packing hot
|
|
code for instruction-cache locality, inlining frequently called functions more
|
|
aggressively, and laying out code to match common execution order.
|
|
|
|
\subsection{Quick Start}
|
|
|
|
A complete PGO build runs from a single target:
|
|
|
|
\begin{lstlisting}[language=bash]
|
|
make pgo
|
|
\end{lstlisting}
|
|
|
|
The target executes a six-stage pipeline: clean the environment, build with
|
|
instrumentation (\texttt{-fprofile-generate}), run the comprehensive
|
|
profiling workload, collect the resulting \texttt{.gcda} files, rebuild with
|
|
the profile data (\texttt{-fprofile-use}), and clean up temporary artifacts.
|
|
Build time is roughly two to three minutes depending on hardware.
|
|
|
|
\subsection{PGO Targets}
|
|
|
|
\begin{itemize}
|
|
\item \texttt{make pgo} --- maximum performance for typical workloads.
|
|
Runs the full profiling workload and produces an optimized binary with
|
|
assembly optimizations and direct threading enabled, then removes profile
|
|
data automatically.
|
|
\item \texttt{make pgo-perf} --- everything in the standard build plus
|
|
\texttt{perf} capture during workload execution, retaining frame pointers
|
|
for accurate stack traces. Produces \texttt{pgo-perf.data} for call-graph
|
|
and hotspot analysis. Requires \texttt{sudo} and
|
|
\texttt{linux-tools-generic}.
|
|
\item \texttt{make pgo-valgrind} --- runs Callgrind during the benchmark
|
|
workload, collecting instruction counts, cache misses, and branch
|
|
mispredictions, and builds an optimized binary retaining debug symbols.
|
|
Produces \texttt{pgo-callgrind.out}. Callgrind imposes a 10--100$\times$
|
|
slowdown, so the workload is limited to 100 benchmark iterations.
|
|
\end{itemize}
|
|
|
|
\subsection{Benchmark Comparison}
|
|
|
|
The \texttt{make bench-compare} target builds both a regular and a PGO binary,
|
|
runs identical benchmarks, and reports the timing difference. A representative
|
|
run shows the regular build at 0.45\,s elapsed and the PGO build at 0.38\,s ---
|
|
roughly a 1.18$\times$ speedup.
|
|
|
|
\subsection{The Profiling Workload}
|
|
|
|
The workload script (\texttt{scripts/pgo-workload.sh}) drives seven major code
|
|
paths: the full unit test suite (over 400 cases across all word categories),
|
|
stress tests (deep call stacks, stack exhaustion, large definitions),
|
|
integration tests (complete Forth programs and real-world usage), benchmarks
|
|
(5000 iterations of hot-path operations), an interactive REPL workload, block
|
|
I/O operations, and lightweight word-frequency profiling for hot-word
|
|
identification. The workload can be run independently against any binary:
|
|
|
|
\begin{lstlisting}[language=bash]
|
|
./scripts/pgo-workload.sh ./build/starforth
|
|
\end{lstlisting}
|
|
|
|
\subsection{Profile Data}
|
|
|
|
GCC emits two file types. The \texttt{.gcno} coverage notes are written at
|
|
compile time and removed before the optimization build; the \texttt{.gcda}
|
|
coverage data is written at runtime and consumed by \texttt{-fprofile-use}.
|
|
Data lands in the directory where the instrumented binary runs; the build
|
|
system searches \texttt{./}, \texttt{src/}, \texttt{src/*/}, and
|
|
\texttt{build/}. A healthy run produces 100 or more \texttt{.gcda} files, one
|
|
per source file that executed; substantially fewer indicates unexercised code
|
|
paths.
|
|
|
|
\subsection{Compiler Flags}
|
|
|
|
The instrumentation stage uses moderate optimization to keep the profiling
|
|
build fast:
|
|
|
|
\begin{lstlisting}[language=bash]
|
|
CFLAGS="-O2 -DUSE_ASM_OPT=1 -fprofile-generate"
|
|
LDFLAGS="-fprofile-generate -lgcov"
|
|
\end{lstlisting}
|
|
|
|
The optimization stage applies maximum optimization, direct threading, and
|
|
graceful handling of inconsistent profile data:
|
|
|
|
\begin{lstlisting}[language=bash]
|
|
CFLAGS="-O3 -DUSE_ASM_OPT=1 -DUSE_DIRECT_THREADING=1 \
|
|
-fprofile-use -fprofile-correction -Wno-error=coverage-mismatch"
|
|
LDFLAGS="-fprofile-use"
|
|
\end{lstlisting}
|
|
|
|
For \texttt{perf} analysis, \texttt{-fno-omit-frame-pointer} preserves the
|
|
frame-pointer register at a 2--3\% performance cost in exchange for accurate
|
|
call graphs.
|
|
|
|
\subsection{Troubleshooting}
|
|
|
|
\begin{itemize}
|
|
\item \emph{No profile data found} --- the workload failed to exercise the
|
|
relevant code, the \texttt{.gcda} files are missing, or the source changed
|
|
between stages. Expand the workload and rebuild with \texttt{make pgo},
|
|
which cleans first.
|
|
\item \emph{Coverage mismatch} --- source was modified between
|
|
instrumentation and optimization. Run \texttt{make pgo} from scratch.
|
|
\item \emph{Permission denied writing .gcda} --- the working directory is
|
|
not writable; restore write permissions and rebuild.
|
|
\item \emph{PGO slower than the regular build} --- the profile does not match
|
|
real usage, or the workload was unrealistic. Customize the workload and
|
|
confirm with \texttt{make bench-compare}.
|
|
\end{itemize}
|
|
|
|
\subsection{Best Practices}
|
|
|
|
Profile representative workloads that cover the bulk of real usage rather than
|
|
trivial operations or rarely executed error paths. Re-run PGO after major code
|
|
changes (more than roughly 10\% of the codebase) and always verify gains with
|
|
\texttt{make bench-compare}. PGO composes well with \texttt{-march=native},
|
|
link-time optimization, direct threading, and hand-written assembly ---
|
|
\texttt{make pgo} enables all of these automatically.
|
|
|
|
\subsection{How PGO Works}
|
|
|
|
During instrumentation the compiler inserts counters at control-flow edges;
|
|
the runtime increments them and writes \texttt{.gcda} files on exit. During
|
|
optimization the compiler reads the profile, separates hot from cold paths,
|
|
and makes informed decisions: inlining hot functions, packing hot code for
|
|
I-cache locality, arranging branches for correct prediction, devirtualizing
|
|
calls where the profile fixes the type, and unrolling hot loops. Hot code is
|
|
laid out sequentially while cold code is outlined to a separate section.
|
|
|
|
\begin{table}[h]
|
|
\centering
|
|
\begin{tabular}{lll}
|
|
\toprule
|
|
Heuristic & Default & With PGO \\
|
|
\midrule
|
|
Inline threshold & 600 units & Adjusted per call site \\
|
|
Loop unroll factor & 4 & Up to 8 for hot loops \\
|
|
Branch prediction & Static (50/50) & Dynamic (from profile) \\
|
|
Function outlining & Disabled & Cold code outlined \\
|
|
Register allocation & Balanced & Favors hot paths \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\caption{GCC optimization heuristics with and without profile data.}
|
|
\end{table}
|
|
|
|
\subsection{Build Target Comparison}
|
|
|
|
\begin{table}[h]
|
|
\centering
|
|
\begin{tabular}{lllll}
|
|
\toprule
|
|
Target & Optimization & Speed & Build Time & Use Case \\
|
|
\midrule
|
|
\texttt{debug} & \texttt{-O0 -g} & 1.0$\times$ & 30\,s & Development \\
|
|
\texttt{all} & \texttt{-O2} & 3.5$\times$ & 45\,s & Default, balanced \\
|
|
\texttt{fast} & \texttt{-O3} + ASM & 5.2$\times$ & 60\,s & Production, no LTO \\
|
|
\texttt{fastest} & \texttt{-O3} + ASM + DT + LTO & 6.8$\times$ & 90\,s & Maximum \\
|
|
\texttt{pgo} & fastest + PGO & 7.5--8.0$\times$ & 3\,m & Absolute maximum \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\caption{Relative performance and build cost across targets (ASM: assembly
|
|
optimizations; DT: direct threading; LTO: link-time optimization).}
|
|
\end{table}
|
|
|
|
\subsection{Platform Notes}
|
|
|
|
On x86\_64 all PGO features are available, with best results under
|
|
\texttt{-march=native}; \texttt{perf} hardware counters and Valgrind are fully
|
|
supported. On ARM64 (Raspberry Pi 4, Apple Silicon) PGO is fully supported ---
|
|
use \texttt{-march=armv8-a+crc+simd -mtune=cortex-a72} on the Pi 4; \texttt{perf}
|
|
support varies by kernel and Valgrind support is limited on some platforms.
|
|
PGO requires native execution to profile, so cross-compilation must run the
|
|
instrumented binary on target hardware, transfer the \texttt{.gcda} files
|
|
back, and cross-compile the optimized binary with that data.
|