aarch64: arm the ARM Generic Timer -- CNTP/CNTHP TVAL+CTL, per-tick re-arm
Punch list §25 item 0.7 complete. This is what finally makes items 0.5 and 0.6 provably work end to end. apic_timer_start(): writes CNTP_TVAL_EL0 (or CNTHP_TVAL_EL2 at EL2 -- aarch64_current_el(), same EL-aware discipline as 0.4-0.6) to s_timer_period_tsc, then CTL.ENABLE=1/IMASK=0, followed by an ISB. The ISB is not decorative: confirmed against Linux's own arch_timer_reg_write_cp15() (arch/arm64/include/asm/arch_timer.h) that only the *control* register write needs synchronising before the enable/mask state is guaranteed visible to the interrupt pipeline -- TVAL/CVAL writes do not carry the same requirement, which is why apic_timer_rearm() omits it. TVAL is architecturally 32-bit but MSR-to-system-register is always a 64-bit instruction form -- passing a uint32_t operand directly failed to build (-Wasm-operand-widths). Fixed by truncating to 32 bits ourselves then zero-extending back to 64 for the operand, which supplies explicit, provably-correct zeros in the RES0 upper field rather than depending on unverified hardware behaviour -- Linux's own driver never exercises this path (it always uses the 64-bit CVAL form instead), so there was no local source to confirm the alternative against. apic_timer_rearm() (new): re-writes TVAL only, no ISB needed. TVAL is relative to "now," not an absolute deadline like riscv64's SBI interface (item 0.3), so there is no drift-correction bookkeeping -- each write means "N ticks from this instant." Wired into aarch64_irq_handler() and called *first*, before heartbeat_tick(), matching riscv64_timer_rearm()'s ordering discipline exactly: the ARM Generic Timer does not auto-reload, so a return path that skips this leaves the interrupt condition latched, which the GIC would redeliver the instant it's EOI'd -- a real storm, the same class of failure item 0.6's verification investigated (and that time found absent, because nothing was armed yet). Verified: builds clean; every generated instruction checked against disassembly, not just reviewed by eye (both EL branches, correct TVAL/CTL register names, single shared ISB in apic_timer_start(), no ISB in apic_timer_rearm()). Boots to ok> with no regression, dict_hash 0x3d4e1daf289da94f unchanged. Rate measured directly against real wall-clock time via QEMU's own -d int trap trace (same method as riscv64's item 0.3), two independent windows: 1,090 interrupts over 11.05 s (98.679 Hz) and 4,031 over 40.88 s (98.614 Hz) -- consistent across both, so this is a real, small, systematic bias (~1.3-1.4% slow), not measurement noise from polling granularity, which would have shrunk with the longer window and did not. Attributed to genuine per-interrupt service latency: TVAL is rewritten mid-ISR, so the trampoline save/restore, GICC_IAR read, EL branch and GICC_EOIR write all lengthen the effective period slightly versus the nominal 10 ms, inherent to any relative-countdown re-arm scheme. Reported as measured, not smoothed over. The interrupt sustained continuously across both windows with no stall and no storm, which is the primary evidence re-arm-every-tick is correct; the small rate bias is overhead, not a defect. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
co-authored by
Claude Sonnet 5
Claude Sonnet 5
parent
cabb0e8bd4
commit
43aa0a2a36
@@ -2112,7 +2112,7 @@ on until there is a tick on all three architectures (§16.1, §16.5).*
|
||||
the prompt with no regression. Scope is one interrupt; a general GIC driver is out of
|
||||
scope and must not be written.
|
||||
|
||||
- [ ] **0.7 — aarch64: arm the generic timer.**
|
||||
- [x] **0.7 — aarch64: arm the generic timer.**
|
||||
`apic_timer_start()` / `apic_timer_stop()` using the register set chosen in 0.4, re-armed
|
||||
each tick.
|
||||
*Done when:* `heartbeat_ticks()` advances on aarch64 at the configured rate.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Capsule Block Manifest — Auto-generated
|
||||
<!-- Generated by mkcapsule --manifest 2026-08-03T23:46:50Z -->
|
||||
<!-- Generated by mkcapsule --manifest 2026-08-04T02:14:10Z -->
|
||||
<!-- DO NOT EDIT — re-run mkcapsule --manifest to refresh. -->
|
||||
<!-- Hand-written justifications and immutability notes live -->
|
||||
<!-- in MANIFEST.md alongside this auto-generated index. -->
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
tick_number,elapsed_ns,tick_interval_ns,cache_hits_delta,bucket_hits_delta,word_executions_delta,hot_word_count,avg_word_heat_q48,window_width,actual_window_size,predicted_label_hits,jitter_bits,apic_ticks,time_trust_q48,variance_q48,vm_call_depth_max,hera_heat_q48,hermes_heat_q48,artemis_heat_q48
|
||||
1,10000,10000,0,0,183,4,45,731,731,0,0,0,65536,0,0,65536,0,0
|
||||
2,20000,10000,0,0,173,5,44,582,582,0,0,0,65536,0,0,65536,0,0
|
||||
3,30000,10000,0,0,184,6,60,582,582,0,0,0,65536,0,0,65536,0,0
|
||||
4,40000,10000,0,0,174,8,71,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
5,50000,10000,0,0,150,9,78,582,582,0,0,0,65536,0,0,65536,0,0
|
||||
6,60000,10000,0,0,159,11,73,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
7,70000,10000,0,0,188,15,71,685,685,0,0,0,65536,0,0,65536,0,0
|
||||
8,80000,10000,0,0,214,16,83,685,685,0,0,0,65536,0,0,65536,0,0
|
||||
9,90000,10000,0,0,216,19,82,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
10,100000,10000,0,0,222,21,84,567,567,0,0,0,65536,0,0,65536,0,0
|
||||
11,110000,10000,0,0,207,23,86,731,731,0,0,0,65536,0,0,65536,0,0
|
||||
12,120000,10000,0,0,189,26,83,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
13,130000,10000,0,0,221,28,80,731,731,0,0,0,65536,0,0,65536,0,0
|
||||
14,140000,10000,0,0,204,29,83,685,685,0,0,0,65536,0,0,65536,0,0
|
||||
15,150000,10000,0,0,203,30,82,731,731,0,0,0,65536,0,0,65536,0,0
|
||||
16,160000,10000,0,0,201,32,83,567,567,0,0,0,65536,0,0,65536,0,0
|
||||
17,170000,10000,0,0,190,34,86,731,731,0,0,0,65536,0,0,65536,0,0
|
||||
18,180000,10000,0,0,170,35,73,582,582,0,0,0,65536,0,0,65536,0,0
|
||||
19,190000,10000,0,0,154,28,44,731,731,0,0,0,65536,0,0,65536,0,0
|
||||
20,200000,10000,0,0,154,28,41,567,567,0,0,0,65536,0,0,65536,0,0
|
||||
21,210000,10000,0,0,153,32,39,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
22,220000,10000,0,0,167,35,42,582,582,0,0,0,65536,0,0,65536,0,0
|
||||
23,230000,10000,0,0,176,37,41,685,685,0,0,0,65536,0,0,65536,0,0
|
||||
24,240000,10000,0,0,175,37,39,731,731,0,0,0,65536,0,0,65536,0,0
|
||||
25,250000,10000,0,0,196,38,37,731,731,0,0,0,65536,0,0,65536,0,0
|
||||
26,260000,10000,0,0,192,41,37,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
27,270000,10000,0,0,158,40,38,685,685,0,0,0,65536,0,0,65536,0,0
|
||||
28,280000,10000,0,0,158,40,35,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
29,290000,10000,0,0,164,41,37,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
30,300000,10000,0,0,154,43,39,685,685,0,0,0,65536,0,0,65536,0,0
|
||||
31,310000,10000,0,0,172,46,35,582,582,0,0,0,65536,0,0,65536,0,0
|
||||
32,320000,10000,0,0,180,49,34,567,567,0,0,0,65536,0,0,65536,0,0
|
||||
33,330000,10000,0,0,145,51,35,567,567,0,0,0,65536,0,0,65536,0,0
|
||||
34,340000,10000,0,0,203,53,37,582,582,0,0,0,65536,0,0,65536,0,0
|
||||
35,350000,10000,0,0,154,55,39,731,731,0,0,0,65536,0,0,65536,0,0
|
||||
36,360000,10000,0,0,144,57,37,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
37,370000,10000,0,0,144,57,35,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
38,380000,10000,0,0,132,57,34,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
39,390000,10000,0,0,180,57,32,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
40,400000,10000,0,0,220,63,29,4096,0,0,0,0,65536,0,0,65536,0,0
|
||||
41,410000,10000,0,0,215,64,28,4096,24,0,0,0,65536,0,0,65536,0,0
|
||||
42,420000,10000,0,0,208,68,29,4096,24,0,0,0,65536,0,0,65536,0,0
|
||||
43,430000,10000,0,0,177,72,27,4096,24,0,0,0,65536,0,0,65536,0,0
|
||||
44,440000,10000,0,0,199,77,28,4096,24,0,0,0,65536,0,0,65536,0,0
|
||||
45,450000,10000,0,0,223,82,26,4096,30,0,0,0,65536,0,0,65536,0,0
|
||||
46,460000,10000,0,0,202,85,24,4096,67,0,0,0,65536,0,0,65536,0,0
|
||||
47,470000,10000,0,0,218,91,25,4096,198,0,0,0,65536,0,0,65536,0,0
|
||||
48,480000,10000,0,0,221,92,26,4096,350,0,0,0,65536,0,0,65536,0,0
|
||||
49,490000,10000,0,0,225,97,26,4096,528,0,0,0,65536,0,0,65536,0,0
|
||||
50,500000,10000,0,0,214,100,27,4096,694,0,0,0,65536,0,0,65536,0,0
|
||||
51,510000,10000,0,0,227,102,28,4096,883,0,0,0,65536,0,0,65536,0,0
|
||||
52,520000,10000,0,0,208,102,29,4096,1043,0,0,0,65536,0,0,65536,0,0
|
||||
53,530000,10000,0,0,207,102,30,4096,1202,0,0,0,65536,0,0,65536,0,0
|
||||
54,540000,10000,0,0,200,102,31,4096,1319,0,0,0,65536,0,0,65536,0,0
|
||||
55,550000,10000,0,0,205,103,32,4096,1464,0,0,0,65536,0,0,65536,0,0
|
||||
56,560000,10000,0,0,211,103,32,4096,1620,0,0,0,65536,0,0,65536,0,0
|
||||
57,570000,10000,0,0,193,110,33,4096,1712,0,0,0,65536,0,0,65536,0,0
|
||||
58,580000,10000,0,0,190,5,67,4096,1775,0,0,0,65536,0,0,65536,0,0
|
||||
|
File diff suppressed because it is too large
Load Diff
@@ -5,7 +5,7 @@
|
||||
*/
|
||||
|
||||
/**
|
||||
* apic.c (aarch64) - Minimal GICv2 driver (punch-list item 0.6).
|
||||
* apic.c (aarch64) - Minimal GICv2 driver + ARM Generic Timer (items 0.6/0.7).
|
||||
*
|
||||
* "Minimal" is deliberate and stays deliberate: exactly one interrupt (the
|
||||
* non-secure EL1 physical timer PPI, or its EL2 hypervisor-timer counterpart
|
||||
@@ -203,15 +203,14 @@ void apic_eoi(void)
|
||||
{
|
||||
}
|
||||
|
||||
/*
|
||||
* @brief Configure the periodic timer (AArch64 stub).
|
||||
/**
|
||||
* @brief Compute the expected system-counter ticks per heartbeat period.
|
||||
*
|
||||
* On x86-64 this calibrates and programs the APIC timer. On AArch64 the
|
||||
* equivalent is the ARM Generic Timer's EL1 Physical Timer (CNTP_CTL_EL0 /
|
||||
* CNTP_TVAL_EL0); that driver is deferred. This stub computes
|
||||
* @c s_timer_period_tsc (the expected number of @c CNTPCT_EL0 ticks
|
||||
* between heartbeats) which is used by @c heartbeat_init() to seed the
|
||||
* rolling-window expected-delta value.
|
||||
* Computes @c s_timer_period_tsc (the expected number of @c CNTPCT_EL0
|
||||
* ticks between heartbeats), used both by @c heartbeat_init() to seed the
|
||||
* rolling-window @c expected_delta and by @c apic_timer_start() /
|
||||
* @c apic_timer_rearm() (item 0.7) as the @c TVAL step written on every
|
||||
* arm and re-arm.
|
||||
*
|
||||
* If either @p tsc_hz or @p tick_hz is zero, a fallback of 10,000,000
|
||||
* ticks is stored (approximately 160 ms at 62.5 MHz, a safe non-zero
|
||||
@@ -232,29 +231,108 @@ int apic_timer_init(uint64_t tsc_hz, uint32_t tick_hz)
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Start periodic timer delivery (AArch64 stub).
|
||||
* @brief Arm the ARM Generic Timer and enable its interrupt (item 0.7).
|
||||
*
|
||||
* On x86-64 this unmasks the APIC timer LVT entry and re-arms the
|
||||
* initial count register. On AArch64 the ARM Generic Timer's EL1
|
||||
* Physical Timer would be enabled here; the driver is deferred.
|
||||
* This stub is a no-op that satisfies the common call site in
|
||||
* @c kernel_main() without fault.
|
||||
* Writes @c CNTP_TVAL_EL0 (or @c CNTHP_TVAL_EL2 at EL2 — @c
|
||||
* aarch64_current_el(), same EL-aware discipline as items 0.4–0.6) to
|
||||
* @c s_timer_period_tsc, then sets @c CTL.ENABLE=1, @c CTL.IMASK=0.
|
||||
*
|
||||
* @c TVAL is a signed 32-bit *relative* countdown — writing it sets
|
||||
* "fire @c s_timer_period_tsc ticks from this instant," unlike riscv64's
|
||||
* SBI interface (item 0.3), which takes an absolute deadline the caller
|
||||
* must compute. No "now" reading is needed here at all.
|
||||
*
|
||||
* The @c ISB after the @c CTL write is required, not decorative: per the
|
||||
* architecture, and confirmed against Linux's own
|
||||
* @c arch_timer_reg_write_cp15() (arch/arm64/include/asm/arch_timer.h),
|
||||
* only the *control* register write needs synchronising before the
|
||||
* enable/mask state is guaranteed visible to the interrupt pipeline —
|
||||
* @c TVAL/@c CVAL writes do not carry the same requirement, which is why
|
||||
* @c apic_timer_rearm() below omits it.
|
||||
*
|
||||
* @c s_timer_period_tsc must already be set by @c apic_timer_init(). If
|
||||
* it is still 0, @c TVAL=0 fires immediately and continuously until
|
||||
* @c apic_timer_rearm() runs — a correctly-functioning if oddly-paced
|
||||
* timer, not a hang.
|
||||
*/
|
||||
void apic_timer_start(void)
|
||||
{
|
||||
/* ARM generic timer EL1 physical timer — stub */
|
||||
/* CNTx_TVAL_EL is architecturally 32-bit (bits[63:32] RES0 on write),
|
||||
* but MSR to a system register is always a 64-bit (Xn) instruction
|
||||
* form -- there is no 32-bit encoding to match a narrower operand.
|
||||
* Truncate to 32 bits ourselves, then zero-extend back to 64 for the
|
||||
* operand: this supplies explicit, provably-correct zeros in the RES0
|
||||
* field rather than depending on how the hardware treats whatever
|
||||
* upper bits a narrower operand happened to leave there -- a
|
||||
* distinction we could not confirm against a local authoritative
|
||||
* source (unlike the GICv2 offsets and CTL bit layout above,
|
||||
* cross-checked against Linux headers on this system; TVAL is not,
|
||||
* since Linux's own driver only ever uses the 64-bit CVAL form and
|
||||
* never exercises this path). */
|
||||
uint64_t tval = (uint64_t)(uint32_t)s_timer_period_tsc;
|
||||
|
||||
if (aarch64_current_el() == 2) {
|
||||
__asm__ volatile ("msr cnthp_tval_el2, %0" :: "r"(tval));
|
||||
__asm__ volatile ("msr cnthp_ctl_el2, %0" :: "r"((uint64_t)1) : "memory");
|
||||
} else {
|
||||
__asm__ volatile ("msr cntp_tval_el0, %0" :: "r"(tval));
|
||||
__asm__ volatile ("msr cntp_ctl_el0, %0" :: "r"((uint64_t)1) : "memory");
|
||||
}
|
||||
__asm__ volatile ("isb" ::: "memory");
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Stop periodic timer delivery (AArch64 stub).
|
||||
* @brief Re-arm the ARM Generic Timer for the next period (item 0.7).
|
||||
*
|
||||
* On x86-64 this sets the APIC timer LVT mask bit. On AArch64 the
|
||||
* ARM Generic Timer would be disabled here. Stub — no-op until the
|
||||
* GIC / Generic Timer driver is implemented.
|
||||
* Writes a fresh @c TVAL — the entirety of a periodic re-arm, since @c TVAL
|
||||
* is relative to "now" rather than an absolute deadline. Unlike riscv64's
|
||||
* SBI re-arm (item 0.3), there is no drift-correction bookkeeping: each
|
||||
* write means "@c s_timer_period_tsc ticks from this instant," so time
|
||||
* spent servicing the interrupt before this call simply lengthens that one
|
||||
* period rather than accumulating error across ticks.
|
||||
*
|
||||
* **This is the one call that must never be skipped.** The ARM Generic
|
||||
* Timer does not auto-reload: once @c TVAL crosses zero, @c CTL.ISTATUS
|
||||
* latches and stays asserted — a level condition — until software writes
|
||||
* a new @c TVAL or clears @c CTL.ENABLE. A return path that skips this
|
||||
* leaves the interrupt permanently pending, which the GIC would deliver
|
||||
* again the instant it is EOI'd: a real interrupt storm, not a
|
||||
* hypothetical one (this is exactly the failure this driver was checked
|
||||
* against during item 0.6's own verification, and confirmed absent there
|
||||
* only because nothing armed the timer yet).
|
||||
*
|
||||
* Called from @c aarch64_irq_handler() (interrupts.c) before
|
||||
* @c heartbeat_tick() — same ordering discipline as
|
||||
* @c riscv64_timer_rearm() (item 0.3): re-arm first, so a fault in the
|
||||
* rest of the handler cannot also cost the next tick.
|
||||
*/
|
||||
void apic_timer_rearm(void)
|
||||
{
|
||||
/* uint64_t, zero-extended -- see apic_timer_start() for why. */
|
||||
uint64_t tval = (uint64_t)(uint32_t)s_timer_period_tsc;
|
||||
|
||||
if (aarch64_current_el() == 2) {
|
||||
__asm__ volatile ("msr cnthp_tval_el2, %0" :: "r"(tval));
|
||||
} else {
|
||||
__asm__ volatile ("msr cntp_tval_el0, %0" :: "r"(tval));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Stop timer delivery by clearing CTL.ENABLE (item 0.7).
|
||||
*
|
||||
* Per the architecture, @c ENABLE=0 unconditionally stops the count and
|
||||
* de-asserts the interrupt condition regardless of @c IMASK — a single
|
||||
* write is sufficient, no separate mask step needed.
|
||||
*/
|
||||
void apic_timer_stop(void)
|
||||
{
|
||||
/* Stub */
|
||||
if (aarch64_current_el() == 2) {
|
||||
__asm__ volatile ("msr cnthp_ctl_el2, %0" :: "r"((uint64_t)0) : "memory");
|
||||
} else {
|
||||
__asm__ volatile ("msr cntp_ctl_el0, %0" :: "r"((uint64_t)0) : "memory");
|
||||
}
|
||||
__asm__ volatile ("isb" ::: "memory");
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -24,12 +24,14 @@ extern void aarch64_install_vectors(void);
|
||||
* arch/riscv64/interrupts.c. */
|
||||
extern int aarch64_current_el(void);
|
||||
|
||||
/* Defined in apic.c (item 0.6). Same extern-in-place convention: GICv2
|
||||
* register-level details belong to the file that owns the MMIO base
|
||||
* addresses, not the shared apic.h. */
|
||||
/* Defined in apic.c (items 0.6/0.7). Same extern-in-place convention: GICv2
|
||||
* and generic-timer register-level details belong to the file that owns
|
||||
* the MMIO base addresses / EL-aware register selection, not the shared
|
||||
* apic.h. */
|
||||
extern uint32_t apic_read_iar(void);
|
||||
extern uint32_t apic_timer_ppi(void);
|
||||
extern void apic_eoi_intid(uint32_t intid);
|
||||
extern void apic_timer_rearm(void);
|
||||
|
||||
/* INTID 1023 = "spurious" (GICv2 spec): the CPU interface has nothing
|
||||
* pending, typically because another interrupt at the same or higher
|
||||
@@ -39,7 +41,7 @@ extern void apic_eoi_intid(uint32_t intid);
|
||||
#define GIC_INTID_SPURIOUS 1023u
|
||||
|
||||
/**
|
||||
* @brief Dispatch an IRQ taken at current-EL with SPx (punch-list item 0.6).
|
||||
* @brief Dispatch an IRQ taken at current-EL with SPx (punch-list item 0.7).
|
||||
*
|
||||
* Called from @c irq_spx_trampoline in @c isr.S, which has already saved the
|
||||
* full caller-saved integer and FP/SIMD register sets plus @c ELR_ELx /
|
||||
@@ -54,20 +56,21 @@ extern void apic_eoi_intid(uint32_t intid);
|
||||
* an unrecognised real INTID still needs the CPU interface unblocked for
|
||||
* the next one.
|
||||
*
|
||||
* The only INTID actually routed anywhere is the timer PPI
|
||||
* (@c apic_timer_ppi() — 30 at EL1, 26 at EL2, decided once in
|
||||
* @c apic_init()), which calls @c heartbeat_tick(). **Unreachable in
|
||||
* practice at this item**: @c apic_timer_start() (item 0.7) is still the
|
||||
* no-op stub, so nothing ever arms @c CNTP_CTL_EL0 / @c CNTHP_CTL_EL2 and
|
||||
* the GIC never sees the timer line assert. Item 0.7's tick-advance
|
||||
* acceptance is what proves this dispatch actually runs (same deferral
|
||||
* item 0.5 already made, C2).
|
||||
* For the timer PPI (@c apic_timer_ppi() — 30 at EL1, 26 at EL2, decided
|
||||
* once in @c apic_init()): @c apic_timer_rearm() runs **first**, before
|
||||
* @c heartbeat_tick() — the ARM Generic Timer does not auto-reload, so
|
||||
* skipping this leaves the interrupt condition permanently latched (see
|
||||
* @c apic_timer_rearm()'s own doc comment). Ordering the re-arm ahead of
|
||||
* the rest of the dispatch means a fault in @c heartbeat_tick() cannot
|
||||
* also cost the next tick — same discipline as @c riscv64_timer_rearm()
|
||||
* (item 0.3).
|
||||
*/
|
||||
void aarch64_irq_handler(void)
|
||||
{
|
||||
uint32_t intid = apic_read_iar();
|
||||
|
||||
if (intid == apic_timer_ppi()) {
|
||||
apic_timer_rearm();
|
||||
heartbeat_tick();
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user