LithosAnanake/docs/working/experiments/02-experiments/physics-optimization/proposals.md

<!-- Moved from docs/02-experiments/physics-optimization/proposals.md to docs/working/experiments/02-experiments/physics-optimization/proposals.md on 2026-06-16 (docs reorg Phase 2) -->
# Physics-Driven Optimization Proposals for StarForth

## Executive Summary

The hot-words cache experiment proves that **physics-driven optimization** (metrics collection → real-time decisions → zero manual tuning) delivers measurable performance improvements while remaining verifiable and maintainable.

This document proposes **9 additional optimization opportunities** in StarForth using similar methodologies. Each leverages the execution_heat tracking already in place and requires no external libraries or floating-point arithmetic.

**Note on JIT Compilation:** JIT is NOT recommended for StarForth. See section below.

---

## Why Not JIT?

### JIT Violates StarForth's Core Constraints
1. **Runtime Code Generation** → Dynamic memory management → Heap complexity
2. **Formal Verification Nightmare** → Generated code can't be proven correct
3. **No Floating-Point Allowed** → JIT compilation itself requires FP overhead
4. **L4Re Incompatibility** → Microkernel doesn't allow arbitrary code generation
5. **Security Risk** → JIT compilation surfaces are attack vectors

### Why Physics-Driven Beats JIT
| Aspect | JIT | Physics-Driven |
|--------|-----|-----------------|
| **Verification** | ❌ Hard (generated code) | ✅ Easy (static analysis) |
| **L4Re Compatible** | ❌ No | ✅ Yes |
| **Floating-Point Required** | ✅ Yes | ❌ No (Q48.16) |
| **Memory Overhead** | 📈 High (code cache) | 📉 Low (metadata only) |
| **Incremental** | ❌ All-or-nothing | ✅ Apply where ROI is highest |
| **Profitability** | 📊 30-100× improvement | 📊 1.78-3× improvement (sufficient!) |

**Verdict:** Physics-driven approach is superior for StarForth's constraints. We gain 1.78× speedup with zero verification burden and perfect L4Re compatibility. That's a winning trade.

---

## Opportunity #1: Return Stack Prediction & Colon Word Inlining

### Problem
When a colon definition calls another word, we execute a full dictionary lookup + function call. For frequently-called small words, this overhead dominates.

**Example:**
```forth
: SQUARE ( n -- n^2 )  DUP * ;
: DISTANCE ( x y -- r )  SQUARE SWAP SQUARE + SQRT ;
```

Every call to `DISTANCE` triggers:
- Dictionary lookup for SQUARE (even though it's predictable)
- Return stack push/pop overhead
- Small function body execution

### Solution: Execution Heat-Based Inlining

**Metrics to Track:**
- `word->execution_heat` (already tracked!)
- `definition->call_count` (how many words call this colon definition)
- `definition->size_bytes` (total body size)

**Decision Rule:**
```c
// Inline a colon definition if:
// 1. It's called frequently (execution_heat > 100)
// 2. It's small enough (size_bytes < 256)
// 3. Total expanded size won't exceed L-cache (8KB)
if (word->execution_heat > 100 &&
    definition->size_bytes < 256 &&
    total_expanded < 8192) {
    // Inline the word body instead of calling it
    vm_compile_inlined(definition);
}
```

**Expected Performance Impact:**
- Eliminate dictionary lookups for inlined words
- Remove return stack overhead
- Expected gain: **1.3–2.0×** for call-heavy programs

**Implementation Complexity:** Medium (requires compile-time analysis)

**Verification:** Static (can prove inlining correctness via Isabelle)

---

## Opportunity #2: Block I/O Prefetching Based on Access Patterns

### Problem
Block storage I/O is expensive. When a program accesses `BLOCK 10`, it often follows with `BLOCK 11`, `BLOCK 12`, etc.

**Current:** Each `BLOCK` word triggers I/O independently (no lookahead).

### Solution: Execution Heat-Tracked Access Patterns

**Metrics to Track:**
- `block->last_access_ns` (monotonic timestamp)
- `block->access_sequence` (last N block IDs accessed)
- `block->next_block_heat[MAX_BLOCKS]` (how often block X follows block Y)

**Decision Rule:**
```c
// When accessing block N, if we predict block M comes next:
// Prefetch block M into buffer cache
if (predicted_next_block != NULL &&
    predicted_next_block->access_heat > PREFETCH_THRESHOLD) {
    blk_prefetch_into_cache(predicted_next_block);
}

// Transition heat calculation (Markov-style):
transition_heat[from_block][to_block]++;
```

**Expected Performance Impact:**
- Eliminate blocking I/O wait for sequential access patterns
- Cache warm start for predictable block sequences
- Expected gain: **1.5–3.0×** for block-heavy programs

**Implementation Complexity:** Medium (requires Markov state tracking)

**Verification:** Dynamic (observable via block I/O metrics)

---

## Opportunity #3: Stack Operation Fusion & Pattern Detection

### Problem
Common patterns like `DUP DROP`, `SWAP ROT`, `OVER SWAP` are executed frequently but could be fused into single operations.

**Current:**
```forth
DUP DROP    \ 2 lookups, 2 executions
SWAP ROT    \ 2 lookups, 2 executions
OVER SWAP   \ 2 lookups, 2 executions
```

**Expected:** ~6-10 nanoseconds per operation pair

### Solution: Pattern-Driven Fusion at Compile Time

**Metrics to Track:**
- Track consecutive word execution pairs: `word[i]` → `word[i+1]`
- `pair_heat[WORD_A][WORD_B]` (how often does A precede B?)
- Threshold: fusion candidate if `pair_heat > 100`

**Decision Rule:**
```c
// During compilation, detect hot patterns:
if (pair_heat[DUP][DROP] > 100) {
    // Fuse into single primitive: DUP_DROP (no-op)
    register_word(vm, "DUP-DROP", forth_DUP_DROP);
}

if (pair_heat[SWAP][ROT] > 100) {
    // Fuse: SWAP followed by ROT = SWAP_ROT primitive
    register_word(vm, "SWAP-ROT", forth_SWAP_ROT);
}
```

**Expected Performance Impact:**
- Eliminate dictionary lookups for fused patterns
- Reduce call overhead for small operations
- Expected gain: **1.2–1.5×** for stack-heavy programs

**Implementation Complexity:** Low (pattern detection + primitive registration)

**Verification:** Static (proven by composition of verified primitives)

---

## Opportunity #4: Memory Allocation Pattern Prediction

### Problem
Dynamic memory allocation (ALLOCATE, RESIZE) has overhead. Patterns (e.g., "allocate 256 bytes, then 256 bytes again") repeat.

**Current:** Every allocation goes through full allocator path

### Solution: Heat-Tracked Allocation Sizes

**Metrics to Track:**
- `allocation_heat[SIZE]` (how many times is SIZE requested?)
- `allocation_pattern_sequence` (last N allocation sizes)
- `allocation_time[SIZE]` (execution time for allocations)

**Decision Rule:**
```c
// Pre-allocate a pool for hot sizes:
if (allocation_heat[256] > 1000) {
    // Keep 10 pre-allocated 256-byte blocks warm
    allocator_warmpool_size(256, 10);
}

// For allocation-heavy patterns, reuse buffers:
if (SEQUENCE == [256, 256, 512, 256, 256, 512]) {
    // Detected repeating pattern - cache allocations
    allocator_pattern_cache(pattern);
}
```

**Expected Performance Impact:**
- Eliminate allocator overhead for pre-warmed sizes
- Reduce fragmentation via pattern-aware pooling
- Expected gain: **1.3–1.8×** for allocation-heavy programs

**Implementation Complexity:** Medium (requires allocator integration)

**Verification:** Dynamic (observable via allocation metrics)

---

## Opportunity #5: Control Flow Branch Prediction

### Problem
`IF/THEN/ELSE` branches have no prediction. In loops, the same branch is taken repeatedly (predictable).

**Current:**
```forth
: SEARCH ( n -- ? )
  BEGIN
    DUP 10 <
    IF  ... ELSE ... THEN
  UNTIL
```

Every iteration re-evaluates the condition and potentially mis-speculates.

### Solution: Execution Heat-Tracked Branch Patterns

**Metrics to Track:**
- Track outcomes of `IF/THEN/ELSE` at each location
- `branch_heat[location][taken]` (how often does branch X go True vs. False?)
- `branch_sequence` (what's the typical pattern?)

**Decision Rule:**
```c
// For hot branches, set a prediction hint:
if (branch_heat[location][TRUE] > 90 &&
    branch_heat[location][FALSE] < 10) {
    // Mark this branch as "likely true"
    vm->current_executing_entry->flags |= BRANCH_LIKELY_TRUE;
}

// Inner interpreter can use hint for:
// - CPU branch prediction (x86 PREFETCHNTA hints)
// - Reordering of code paths
// - Speculative stack preparation
```

**Expected Performance Impact:**
- Reduce branch mispredictions in CPU pipeline
- Reorder code for better CPU cache behavior
- Expected gain: **1.1–1.3×** (CPU-dependent)

**Implementation Complexity:** Medium (requires CPU-specific assembly work)

**Verification:** Dynamic (observable via perf branch-misses)

---

## Opportunity #6: Vocabulary Search Path Optimization

### Problem
Multi-vocabulary systems (FORTH, EDITOR, SYSTEM) search vocabularies in order. If most lookups hit FORTH, other vocabularies waste cycles.

**Current:** Always search: EDITOR → SYSTEM → FORTH

### Solution: Dynamic Search Order Based on Execution Heat

**Metrics to Track:**
- `vocab_hit_rate[EDITOR]`, `vocab_hit_rate[SYSTEM]`, `vocab_hit_rate[FORTH]`
- `vocab_lookup_time[VOCAB]` (latency per vocabulary)
- Track which vocabulary actually had the hit

**Decision Rule:**
```c
// Reorder search path based on hit rates:
// If FORTH has 85% hits, search it first
sorted_vocabs = sort_by_hit_rate([EDITOR, SYSTEM, FORTH]);

// Update search order dynamically:
if (hit_rate[FORTH] > hit_rate[EDITOR]) {
    swap_search_order(FORTH, EDITOR);
}
```

**Expected Performance Impact:**
- Reduce dictionary lookup depth for hot vocabulary
- Fewer misses on wrong vocabularies
- Expected gain: **1.2–1.6×** (vocab-dependent)

**Implementation Complexity:** Low (requires search path reordering)

**Verification:** Static (observable via lookup metrics)

---

## Opportunity #7: String Operation Batching

### Problem
String operations (TYPE, EMIT, PARSE) are called frequently on small strings. Each call has interpretation overhead.

**Current:**
```forth
." Hello " ." World " CR
```

Compiles to 3 separate operations with 3 lookups each.

### Solution: Compile-Time String Fusion

**Metrics to Track:**
- Detect consecutive string literals: `." A " ." B "` pattern
- `string_op_heat[TYPE]` (how often are consecutive string ops used?)
- String size statistics

**Decision Rule:**
```c
// During compilation, fuse consecutive string operations:
if (word_sequence == [EMIT, EMIT, EMIT] &&
    all_immediate) {
    // Fuse into single STRING_OUTPUT operation
    // Batch output: less interpreter overhead
    emit_fused_string_output(concatenated_string);
}
```

**Expected Performance Impact:**
- Eliminate overhead for small string operations
- Improve I/O efficiency via batching
- Expected gain: **1.1–1.4×** (I/O bound dependent)

**Implementation Complexity:** Low (compile-time pattern detection)

**Verification:** Static (observable via string operation metrics)

---

## Opportunity #8: Arithmetic Operation Reordering & Associativity

### Problem
Commutative operations (+ - * AND OR) are executed in fixed order. Some orders have better CPU cache behavior.

**Example:**
```forth
A B +  C +  D +    \ Left associative
A B C D + + +      \ Right associative (different cache behavior)
```

### Solution: Heat-Tracked Operand Ordering

**Metrics to Track:**
- `operand_heat[RESULT]` (how hot is the result of this operation?)
- Track which operand sizes are common
- CPU cache line utilization for different orderings

**Decision Rule:**
```c
// For hot commutative operations, try both orderings:
if (add_heat[location] > 1000 &&
    operand_size_a != operand_size_b) {
    // Try reordering for better cache behavior
    // A=large, B=small: load small first (better prefetch)
    if (size_b < size_a) {
        emit_operation(B, A, ADD);  // Reordered
    }
}
```

**Expected Performance Impact:**
- Improve CPU cache line reuse
- Reduce memory latency for hot arithmetic
- Expected gain: **1.05–1.15×** (CPU-dependent)

**Implementation Complexity:** Medium (requires cache analysis)

**Verification:** Dynamic (observable via perf cache-misses)

---

## Opportunity #9: Word Placement Optimization (Code Layout)

### Problem
Dictionary layout is insertion-order (no optimization). Hot words might be far from each other, causing CPU cache misses.

**Current:** Words stored in linked-list order they were defined

### Solution: Heat-Driven Memory Compaction

**Metrics to Track:**
- `word->execution_heat` (already tracked!)
- `word->co_execution_heat[OTHER_WORD]` (how often are these words called together?)
- `word->memory_address` (track actual memory layout)

**Decision Rule:**
```c
// During garbage collection / compaction:
// 1. Calculate which words are frequently called together
// 2. Place hot word clusters adjacent in memory
// 3. Minimize cache line misses for hot word sequences

hot_cluster = [EXIT, LIT, CR]  // Frequently co-executed
memory_compact_together(hot_cluster);  // Colocate in memory

// Result: prefetch-friendly dictionary layout
```

**Expected Performance Impact:**
- Reduce CPU instruction cache misses
- Better temporal locality for hot code paths
- Expected gain: **1.05–1.2×** (CPU architecture-dependent)

**Implementation Complexity:** High (requires memory layout changes, GC integration)

**Verification:** Dynamic (observable via perf icache-misses)

---

## Implementation Roadmap

### Phase 1: Foundation (Already Complete ✅)
- ✅ Execute heat tracking in VM
- ✅ Hot-words cache (1.78× speedup proven)
- ✅ Bayesian inference for confidence
- ✅ Q48.16 fixed-point arithmetic

### Phase 2: Low-Hanging Fruit (Recommended Next)
1. **Stack Operation Fusion** (#3) - Highest ROI, lowest complexity
2. **Vocabulary Search Reordering** (#6) - Minimal complexity
3. **Return Stack Prediction** (#1) - Medium complexity, high impact

### Phase 3: Medium Complexity (Future)
4. **Block I/O Prefetching** (#2) - Medium complexity
5. **String Operation Batching** (#7) - Low/medium complexity
6. **Memory Allocation Patterns** (#4) - Moderate complexity

### Phase 4: Advanced (Future)
7. **Control Flow Branch Prediction** (#5) - High complexity, CPU-specific
8. **Arithmetic Operation Reordering** (#8) - Medium complexity
9. **Word Placement Optimization** (#9) - High complexity (requires GC)

---

## Unified Metrics Infrastructure

All 9 opportunities require a **common metrics collection framework**:

```c
typedef struct {
    // Execution frequency (already have this)
    uint64_t execution_heat;

    // Co-execution patterns (NEW)
    uint64_t co_execution_heat[DICT_SIZE];

    // Timing information (ENHANCED)
    uint64_t total_time_ns;
    uint64_t call_count;

    // Cache behavior (NEW)
    uint32_t cache_line_hits;
    uint32_t cache_line_misses;

    // Sequence patterns (NEW)
    circular_buffer_t recent_execution;

} PhysicsMetrics;
```

**One-time investment** in unified metrics → all 9 opportunities unlocked.

---

## Expected Cumulative Performance Gain

### Conservative Estimate
```
Baseline:                1.0×
+ Hot-words cache:       1.78×
+ Stack fusion:          2.13× (1.78 × 1.2)
+ Vocab reordering:      2.56× (2.13 × 1.2)
+ Return prediction:     3.30× (2.56 × 1.3)
──────────────────────────────
Total (Phase 2):         3.3× speedup
```

### Ambitious Estimate (All 9)
```
Cumulative with all optimizations:  5–8× speedup possible
```

### But Remember
- Not all programs benefit from all optimizations
- Each optimization has CPU/workload dependency
- Diminishing returns apply (law of diminishing returns)
- **1.78× from cache alone is already substantial!**

---

## Why Physics-Driven Beats Alternatives

| Approach | Speedup | Verification | L4Re Compatible | Complexity |
|----------|---------|--------------|-----------------|------------|
| **JIT Compilation** | 30–100× | ❌ Impossible | ❌ No | Very High |
| **Static Optimization** | 1.05–1.2× | ✅ Easy | ✅ Yes | Medium |
| **Physics-Driven** (current) | 1.78× | ✅ Easy | ✅ Yes | Low–Medium |
| **Physics-Driven** (all 9) | 5–8× | ✅ Easy | ✅ Yes | Medium–High |

**Verdict:** Physics-driven is optimal for StarForth's constraints. We get measurable gains (1.78×+), complete verifiability, and L4Re compatibility all in one package.

---

## References

- **Experiment Report**: `docs/PHYSICS_HOTWORDS_CACHE_EXPERIMENT.md`
- **Reproduction Guide**: `docs/REPRODUCE_PHYSICS_EXPERIMENT.md`
- **Physics Architecture**: `include/physics_metadata.h`, `include/physics_hotwords_cache.h`
- **CLAUDE.md**: Core design patterns

---

## Summary: Strategic Direction

1. **Physics-Driven Optimization** is the right approach for StarForth
2. **Hot-words cache** (1.78×) is proven and ready for production
3. **9 additional opportunities** can stack to 5–8× total gain
4. **Low-hanging fruit** (stack fusion, vocab reordering) should come next
5. **JIT is wrong direction** (violates constraints, harder to verify)
6. **Incremental approach** matches StarForth's design philosophy

The physics model is the foundation. Build on it, measure everything, verify rigorously.