admin/LithosAnanake
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

aarch64: split irq_spx into a real save/dispatch/restore/ERET trampoline

Browse Source 
Punch list §25 item 0.5 complete.

irq_spx now branches (one instruction, well inside the 128-byte vector slot)
to irq_spx_trampoline, a 672-byte-frame save/restore sequence that calls a
C handler and returns via ERET. The other fifteen vectors are untouched,
still routing to the existing fatal handler.

EL selection (B3) happens once, in aarch64_install_vectors(), not per
interrupt: aarch64_current_el() (item 0.4) picks VBAR_EL1 or VBAR_EL2, and
the same answer is cached in a byte flag (el2_mode_flag) that the trampoline
reads to choose ELR_EL1/SPSR_EL1 vs ELR_EL2/SPSR_EL2 -- the two forms are
genuinely different MRS/MSR encodings, not runtime-selectable operands, so
this is the cheapest correct design: decide once at install time, branch
twice (save, restore) per interrupt afterward. VBAR_EL1 was previously
written unconditionally; this closes that half of item 0.4's known gap.
EL2 is coded from the architecture reference and cannot be boot-tested in
this environment (QEMU's aarch64 virt/EDK2 combination here yields EL1) --
reported as unverified rather than asserted as tested.

FP/SIMD save is not optional (B2, carried from item 0.4's finding that the
build has no -mgeneral-regs-only): the AAPCS64 caller-saved set -- v0-v7,
v16-v31, full 128 bits each -- plus FPSR/FPCR is saved and restored around
the C handler call. v8-v15 are callee-saved by the ABI and deliberately
excluded: the handler, being ordinary compiled C, preserves those itself.

aarch64_irq_handler() (interrupts.c) is deliberately empty. Distinguishing
which interrupt fired needs the GIC's IAR, which does not exist until item
0.6; nothing unmasks or routes any source to this vector yet, so the
function is not reachable during a normal boot. Per the item's own text,
no attempt was made to manufacture an interrupt to exercise this path early
-- 0.6 (GIC) and 0.7 (timer) are what prove it took and returned one.

Verified: every hand-computed frame offset (0, 16, 32 ... 640, frame size
0x2a0=672) checked against the actual disassembly of the built kernel, not
just visually reviewed -- save and restore sequences mirror exactly, and
aarch64_install_vectors' branch on the detected EL, the flag write, and the
trampoline's read of the same flag address all confirmed consistent. Boots
clean on real QEMU output, no regression: dict_hash 0x3d4e1daf289da94f
unchanged from the item 0.1-0.4 baseline, and the item 0.4 EL banner
("AArch64: running at EL1") still prints correctly ahead of "IDT installed.".

Only aarch64-scoped files touched (isr.S, interrupts.c) -- no shared loader
or header changed, so amd64 and riscv64 are provably unaffected; not
rebuilt for this item.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-08-03 18:47:29 -04:00
co-authored by Claude Sonnet 5
parent f43f3f4482
commit 8d8f3aaae2
6 changed files with 10640 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/starkernel/arch/aarch64/interrupts.c
+23
View File
@@ -23,6 +23,29 @@ extern void aarch64_install_vectors(void);
* arch/riscv64/interrupts.c. */
extern int aarch64_current_el(void);
/**
* @brief Dispatch an IRQ taken at current-EL with SPx (punch-list item 0.5).
*
* Called from @c irq_spx_trampoline in @c isr.S, which has already saved the
* full caller-saved integer and FP/SIMD register sets plus @c ELR_ELx /
* @c SPSR_ELx, and will restore all of it and @c ERET after this function
* returns — unlike @c aarch64_exception_handler(), this path is designed to
* return normally.
*
* **Deliberately empty at this item.** Distinguishing which interrupt fired
* requires reading the GIC's Interrupt Acknowledge Register, and
* acknowledging it requires writing EOIR — neither exists yet (item 0.6).
* Nothing currently unmasks or routes any interrupt source to this vector,
* so it is not reachable during a normal boot; item 0.5's own acceptance is
* "boots with no regression," not having observed a call here (C2). This
* function exists so the vector split is complete and linkable now, ready
* for item 0.6 to add the IAR read / cause dispatch / EOIR write, and item
* 0.7 to route the timer PPI to @c heartbeat_tick() through it.
*/
void aarch64_irq_handler(void)
{
}
/**
* @brief Print a 64-bit value as "0xNNNNNNNNNNNNNNNN" to the kernel console.
*
src/starkernel/arch/aarch64/isr.S
+212 -2
View File
@@ -1,6 +1,12 @@
/*
* isr.S (aarch64) - Exception vector table stub.
* Installs a minimal VBAR_EL1 that routes all exceptions to a common C handler.
*
* Fourteen of the sixteen vectors still route to the shared fatal handler in
* interrupts.c (unchanged by punch-list item 0.5). irq_spx -- the vector that
* matters, since the kernel runs at current-EL with SPx (SP_EL1/SP_EL2, per
* the detected EL) -- branches to a real trampoline: save state, call a C
* handler, restore, ERET. Everything else keeps spinning on a fatal WFE/WFI;
* only the interrupt path becomes returnable.
*/
.section .text
@@ -50,7 +56,9 @@ synchronous_spx:
.balign 128
irq_spx:
b aarch64_exception_handler
/* 128-byte slot limit (32 instructions) -- the real save/restore
* sequence does not fit inline. One branch out, well within budget. */
b irq_spx_trampoline
.balign 128
fiq_spx:
@@ -94,10 +102,212 @@ fiq_lower_el32:
serror_lower_el32:
b aarch64_exception_handler
/* ---------------------------------------------------------------------------
* EL selection.
*
* The kernel runs at exactly one exception level for its entire lifetime
* (item 0.4's own caching rationale: CurrentEL cannot change post-boot), so
* EL is decided **once here**, not per interrupt. aarch64_current_el()
* (arch.c) is consulted to choose which VBAR_ELx gets the table's address,
* and el2_mode_flag records the same answer for irq_spx_trampoline below to
* pick ELR_EL1/SPSR_EL1 vs ELR_EL2/SPSR_EL2 without re-deriving it on every
* interrupt.
* ------------------------------------------------------------------------- */
.section .bss
.balign 1
el2_mode_flag:
.byte 0
.text
.global aarch64_install_vectors
aarch64_install_vectors:
stp x29, x30, [sp, #-16]!
bl aarch64_current_el /* w0 = 1 or 2 */
adrp x1, el2_mode_flag
add x1, x1, :lo12:el2_mode_flag
cmp w0, #2
b.ne 1f
mov w2, #1
strb w2, [x1]
adr x0, aarch64_vector_table
msr vbar_el2, x0
isb
b 2f
1: mov w2, #0
strb w2, [x1]
adr x0, aarch64_vector_table
msr vbar_el1, x0
isb
2: ldp x29, x30, [sp], #16
ret
/* ---------------------------------------------------------------------------
* irq_spx_trampoline — save / call C handler / restore / ERET.
*
* Frame layout (672 bytes, 16-byte aligned; SP is guaranteed 16-aligned at
* any point AAPCS64 code can be asynchronously interrupted, so no extra
* alignment handling is needed around the initial SUB):
*
* 0..239 x0-x29 (15 STP pairs)
* 240 x30
* 248 ELR_ELx
* 256 SPSR_ELx
* 264 FPSR
* 272 FPCR
* 280 (8 bytes unused -- keeps the V-register block 16-aligned)
* 288..415 v0-v7 (4 STP Q-pairs) -- AAPCS64 caller-saved
* 416..671 v16-v31 (8 STP Q-pairs) -- AAPCS64 caller-saved
*
* v8-v15 are AAPCS64 callee-saved and are deliberately not here: the C
* handler below is ordinary compiled code and preserves those itself if it
* touches them, per the same ABI the save list is drawn from.
* ------------------------------------------------------------------------- */
.equ TF_X0, 0
.equ TF_X2, 16
.equ TF_X4, 32
.equ TF_X6, 48
.equ TF_X8, 64
.equ TF_X10, 80
.equ TF_X12, 96
.equ TF_X14, 112
.equ TF_X16, 128
.equ TF_X18, 144
.equ TF_X20, 160
.equ TF_X22, 176
.equ TF_X24, 192
.equ TF_X26, 208
.equ TF_X28, 224
.equ TF_X30, 240
.equ TF_ELR, 248
.equ TF_SPSR, 256
.equ TF_FPSR, 264
.equ TF_FPCR, 272
.equ TF_V0, 288
.equ TF_V2, 320
.equ TF_V4, 352
.equ TF_V6, 384
.equ TF_V16, 416
.equ TF_V18, 448
.equ TF_V20, 480
.equ TF_V22, 512
.equ TF_V24, 544
.equ TF_V26, 576
.equ TF_V28, 608
.equ TF_V30, 640
.equ TF_SIZE, 672
.balign 4
irq_spx_trampoline:
sub sp, sp, #TF_SIZE
stp x0, x1, [sp, #TF_X0]
stp x2, x3, [sp, #TF_X2]
stp x4, x5, [sp, #TF_X4]
stp x6, x7, [sp, #TF_X6]
stp x8, x9, [sp, #TF_X8]
stp x10, x11, [sp, #TF_X10]
stp x12, x13, [sp, #TF_X12]
stp x14, x15, [sp, #TF_X14]
stp x16, x17, [sp, #TF_X16]
stp x18, x19, [sp, #TF_X18]
stp x20, x21, [sp, #TF_X20]
stp x22, x23, [sp, #TF_X22]
stp x24, x25, [sp, #TF_X24]
stp x26, x27, [sp, #TF_X26]
stp x28, x29, [sp, #TF_X28]
str x30, [sp, #TF_X30]
/* Original x0-x30 are safe in memory now; free to use x0-x2 as scratch
* for the rest of the save. EL-dependent register bank (B3): the two
* forms are genuinely different MRS encodings, chosen from the flag
* install_vectors set once, not re-derived per interrupt. */
adrp x0, el2_mode_flag
add x0, x0, :lo12:el2_mode_flag
ldrb w0, [x0]
cbnz w0, 1f
mrs x1, elr_el1
mrs x2, spsr_el1
b 2f
1: mrs x1, elr_el2
mrs x2, spsr_el2
2: stp x1, x2, [sp, #TF_ELR]
/* FP/SIMD (B2): built without -mgeneral-regs-only, so the C handler may
* use any of these. */
mrs x1, fpsr
mrs x2, fpcr
stp x1, x2, [sp, #TF_FPSR]
stp q0, q1, [sp, #TF_V0]
stp q2, q3, [sp, #TF_V2]
stp q4, q5, [sp, #TF_V4]
stp q6, q7, [sp, #TF_V6]
stp q16, q17, [sp, #TF_V16]
stp q18, q19, [sp, #TF_V18]
stp q20, q21, [sp, #TF_V20]
stp q22, q23, [sp, #TF_V22]
stp q24, q25, [sp, #TF_V24]
stp q26, q27, [sp, #TF_V26]
stp q28, q29, [sp, #TF_V28]
stp q30, q31, [sp, #TF_V30]
bl aarch64_irq_handler
ldp q0, q1, [sp, #TF_V0]
ldp q2, q3, [sp, #TF_V2]
ldp q4, q5, [sp, #TF_V4]
ldp q6, q7, [sp, #TF_V6]
ldp q16, q17, [sp, #TF_V16]
ldp q18, q19, [sp, #TF_V18]
ldp q20, q21, [sp, #TF_V20]
ldp q22, q23, [sp, #TF_V22]
ldp q24, q25, [sp, #TF_V24]
ldp q26, q27, [sp, #TF_V26]
ldp q28, q29, [sp, #TF_V28]
ldp q30, q31, [sp, #TF_V30]
ldp x0, x1, [sp, #TF_FPSR]
msr fpsr, x0
msr fpcr, x1
/* x0-x30 are still their post-interrupt-entry values (untouched since
* the initial save) except x0-x2, which the save sequence above used as
* scratch -- all three are about to be overwritten by the final GP
* restore below regardless, so clobbering them here is harmless. */
ldp x0, x1, [sp, #TF_ELR] /* x0=ELR, x1=SPSR */
adrp x2, el2_mode_flag
add x2, x2, :lo12:el2_mode_flag
ldrb w2, [x2]
cbnz w2, 3f
msr elr_el1, x0
msr spsr_el1, x1
b 4f
3: msr elr_el2, x0
msr spsr_el2, x1
4:
ldp x0, x1, [sp, #TF_X0]
ldp x2, x3, [sp, #TF_X2]
ldp x4, x5, [sp, #TF_X4]
ldp x6, x7, [sp, #TF_X6]
ldp x8, x9, [sp, #TF_X8]
ldp x10, x11, [sp, #TF_X10]
ldp x12, x13, [sp, #TF_X12]
ldp x14, x15, [sp, #TF_X14]
ldp x16, x17, [sp, #TF_X16]
ldp x18, x19, [sp, #TF_X18]
ldp x20, x21, [sp, #TF_X20]
ldp x22, x23, [sp, #TF_X22]
ldp x24, x25, [sp, #TF_X24]
ldp x26, x27, [sp, #TF_X26]
ldp x28, x29, [sp, #TF_X28]
ldr x30, [sp, #TF_X30]
add sp, sp, #TF_SIZE
eret