Skip to content

2026-10-26 f64-isa: bit-exact double precision for the Maxwell translator, and the rare-ISA audit

Branch claude/f64-isa-2026-10-26-macos1 (Linux cloud session; Metal checks are written but not run). Owner area: gfx/src/shader/.

What changed

  • Double precision is translated. DADD, DMUL, DFMA, DSETP, DMNMX, DSET and the 64-bit forms of F2F, F2I and I2F no longer fail with "double precision is not supported (Metal has no f64)". A binary64 is a register pair (even register = low word) carried as a ulong of raw bits; every operation is a call into a small integer library that is emitted once per shader that needs it.
  • One source, two languages. src/shader/soft_double_core.inc holds the algorithms in the subset shared by C and Metal (uint 32-bit, ulong 64-bit, no pointers). soft_double.h compiles it as C (the host reference); soft_double_msl.h stringifies the same tokens (#__VA_ARGS__ after macro expansion) into sd_msl_source, which the translator appends to the shader. The MSL is therefore not a re-implementation: the test vectors that pass for the C build validate the text that reaches the Metal compiler.
  • Semantics: IEEE 754 round to nearest even and the three directed modes (the Maxwell rounding field: 0 RN, 1 RM, 2 RP, 3 RZ), full denormal support (no flush), single-rounding fused multiply-add, exact zero signs (a cancelling sum is +0, -0 under RM), every NaN result is the canonical 0x7FFFFFFFFFFFFFFF (the hardware does not propagate payloads), F2I saturates and maps NaN to 0, DMNMX lets the non-NaN operand win and orders -0 below +0.
  • Operand forms: register pair, two constant-buffer words, the 20-bit immediate that fills the top of the high word; the neg/abs modifier bits of every form; DFMA's cbuf-in-C form (0x53); the DSETP/DSET compare codes and combine operators.
  • NAPPLE_F64=off restores the old refusal (emulation level switch for A/B debugging a suspect shader). ShaderKey is untouched.
  • Rare-ISA audit (40 items, table below): video ALU ops, IMAD32I, FCHK, PIXLD, LEPC, CAL/RET/JMP/JCAL/JMX, CCTL/CCTLL implemented or made structural; the rest refused by name.
  • Layout fixes from envytools (MIT, the Maxwell disassembler tables; facts only): IMAD read bits 49/51 as negations and 53 as .HI; the hardware has .X at 49, .SAT at 50, .HI at 54, and no negations. IMAD now follows the tables (.SAT/.X/.CC fail cleanly). The FSET cbuf mask no longer swallows DSET (0x49).

Verification

Check Status Evidence
Reference vs the CPU's native double tested (Linux x86-64) sh gfx/tests/t_softdouble.sh: 281,926,702 checks, 0 mismatches (123-value edge corpus, all pairs and every 3rd-by-3rd triple; 1.5 M random iterations of eight generators: any bits, moderate exponents, near exponents, sparse mantissas for exact and tie cases, subnormal, near overflow, cancellation, fma exactness c = -(a*b) +- ulps; all four rounding modes). Per op: DADD 18.06 M, DSUB 18.06 M, DMUL 18.06 M, DFMA 18.83 M, DSETP (16 conditions) 72.2 M, DMNMX 9.03 M, RINT 6.0 M, F2I (8/16/32/64 bit, signed/unsigned) 54.0 M, F2F f64>f32 6.0 M, f32>f64 31.6 M, I2F s64/u64 -> f64 18.0 M and -> f32 12.0 M. 0 mismatches each.
The test can fail tested With the ties-to-even line of sd_round_pack removed: 3,002 edge and 10,168 random mismatches.
No undefined behaviour tested The same run built with -fsanitize=undefined,address -fno-sanitize-recover: clean (200 k iterations).
Emitted MSL text vs the C build tested sh gfx/tests/t_f64_msl_cxx.sh: the dumped sd_msl_source compiled as C++14 (MSL's base) and compared with the C build on 4.8 M calls: 0 mismatches. Proves the stringification and typing; not the Metal compiler.
Translator text tests tested sh gfx/tests/t_xlate_linux.sh: ok, 103 Metal compile checks skipped (no Metal here). Covers every f64 form (DADD/DMUL/DFMA/DSETP/DMNMX/DSET in reg, cbuf, imm; F2F/F2I/I2F 32/64 mixes), the video ops, IMAD32I, FCHK, PIXLD, LEPC, calls/jumps, the refusals, and that a shader without doubles carries no library.
GPU execution unbuilt/unrun: Mac check gfx/tests/t_f64.m (picked up by make -C gfx tests): compiles sd_msl_source in a compute kernel and compares every function, all rounding modes, 8 x 131072 vectors each, with the C reference and (add/sub/mul/fma) the CPU double. Syntax-checked against stub Metal headers only.
Metal compile of the generated shaders Mac check make -C gfx test runs MSL_COMPILES on all 103 translations (skipped on Linux).
Other Linux gfx tests tested sh gfx/tests/t_getters_linux.sh: 33 checks, 0 failed.

Performance note (stretch)

Static cost of one call, in LLVM IR instructions after -O2 (branches excluded): add 135, mul 146, fma 236, compare 20, min/max 23, rint 34, int->f64 31, f64->int 55, f64->f32 84, f32->f64 31, int64->f32 39. Host timing on random normal operands (x86-64, -O2, mispredicted branches included): add 13 ns, mul 9 ns, fma 21 ns, compare 7 ns, f64->f32 3 ns. On the GPU each op is a few dozen to ~250 dependent 32/64-bit integer instructions (64-bit adds and shifts are emulated by the Apple GPU as 32-bit pairs, so expect 2-4x the IR count in ALU cycles); a shader doing doubles is slow per double but doubles are rare in game shaders. The whole library is emitted whole (about 11 KB of MSL) in every shader with a double op; the Metal compiler drops what is unused.

The 40 items the catalog scan lists as missing (tier rare)

Status: implemented = translated now (layout from envytools envydis gm107 unless noted); structural = control flow the pc state machine now handles; no-op = no effect on the emulated machine; refused = fails with a named error because no application shader reaches it. "Test" is in gfx/tests/t_xlate.m (Linux-runnable text assertions; the emitted MSL compile is the Mac check). The scan marks an item implemented when its mnemonic appears in a comment in maxwell_translate.c: every implemented/structural/no-op row has one; refused rows deliberately have none.

Mnemonic Status Where / behaviour Test
VADD implemented video_op: va + vb in 64-bit, saturate (bit 55) to the dest format (bit 54), secondary op with Rc (MRG_16H/16L/8B0/8B2, ACC, MIN, MAX, PASS) VADD u32 reg ACC / s8 imm MRG sat / u16 h1 MIN
VMAD implemented (va*vb >> 0/7/15 by bits 51-52) + Rc, Rc signed when an operand is; the .PO flag is not decoded VMAD shr7 / signed shr15
VABSDIFF implemented va - vb
VSHL, VSHR implemented count = imm16 (8 bits with bit 50), .W (bit 49) wraps modulo 32 else clamps at 32; arithmetic right shift when A is signed VSHL clamp / VSHR wrap
VSET implemented Rd = (va cmp vb) ? 1 : 0, compare code from bits 43, 44, 47, then the secondary op VSET lt / ge
VSETP implemented Pd/Pq from the compare, combined with a predicate (bits 39-42) by AND/OR/XOR (bits 45-46) VSETP gt
VABSDIFF4 refused four-lane byte absolute difference with lane masks; no use found refusal
IMAD32I implemented Rd = +-(Ra * imm32 [>>32 for .HI]) + +-Rd; signed per bits 54/57; .CC refused IMAD32I, IMAD32I.HI
IMADSP refused multiply-add with sub-word selectors; compilers emit XMAD refusal
IDP refused integer dot product exists from SM 6.0; the Switch GPU is SM 5.3 refusal
IDE refused interrupt enable/disable (system) refusal
FCHK implemented FCHK.DIVIDE Pd: true unless both operands are normal and the quotient exponent keeps clear of the ends; conservative (true sends the shader to its own slow path, which the compiler emitted and is exact). Pd at bits 3-5 FCHK.DIVIDE
PIXLD implemented single-sample semantics: COVMASK/COVERED = 1, OFFSET/CENT_OFFSET/MY_INDEX = 0, Pd true except for the last three. Wrong for MSAA targets (the translator has no sample-count input) PIXLD.COVMASK / MY_INDEX
LEPC implemented Rd = the address of the instruction in the translator's branch-target units LEPC
JMP structural absolute 32-bit target, a label in the switch (pc) machine JMP/JCAL/JMX
JMX structural pc = Ra + imm32; like BRX every instruction becomes a label JMP/JCAL/JMX
CAL structural pushes the return address on a 16-deep cal stack, relative target CAL/RET
JCAL structural same with an absolute target JMP/JCAL/JMX
RET structural pops cal; with an empty stack the program ends CAL/RET
PEXIT, LONGJMP, PLONGJMP refused CUDA exception/unwinding plumbing; graphics shaders never contain them refusal
B2R, R2B refused barrier-state registers (BAR.RED reductions) refusal
CCTL, CCTLL no-op cache invalidate/write-back CCTL, CCTLL
BPT refused breakpoint/trap refusal
RAM, SAM, RTT refused trap-return plumbing refusal
ISBERD refused internal stage buffer read (tessellation/geometry; the translator has neither stage) refusal
GETCRSPTR, SETCRSPTR, GETLMEMBASE, SETLMEMBASE refused call-return-stack pointer / local-memory base (generic addressing, exceptions) refusal
STP refused store to the internal stage buffer (tessellation control) refusal
TXA refused texture attribute query, no semantics known refusal
SURED untouched surface family, a later package (still the "surface access" refusal) existing
DSET implemented the FSET layout on doubles; the result is one 32-bit register (all ones, or 1.0f with BF) DSET rr / cbuf BF / imm

Counts: 18 implemented or structural or no-op (VADD, VMAD, VABSDIFF, VSHL, VSHR, VSET, VSETP, IMAD32I, FCHK, PIXLD, LEPC, JMP, JMX, CAL, JCAL, RET, CCTL, CCTLL) + DSET = 19; 20 refused by name; SURED untouched (later package).

Not done / limits

  • The Maxwell F2F float-narrowing rounding is read from bits 39-40 when bit 42 is clear; envytools lists only 0 ("round") and 3 ("pass") for that field, so a directed-rounding F2F.F32.F64 may use another encoding. Default rounding is right.
  • F2F f64>f16 goes through f32 (two roundings).
  • VMAD .PO, VADD bits 56-57 and the .CC bits of the video family are not decoded (nothing in either corpus uses the family).
  • FTZ bits of F2F/F2I on a double are ignored (doubles have no flush).
  • Nothing here was run on a Mac: the translator output has not been through newLibraryWithSource, and no corpus shader contains a double (so no real word confirms the D-op layouts; they match envytools).