Skip to content

nx-shader (2026-10-25): translator items from the nxnative review (G7, G13, G14, VMNMX, G11), SMO glitch triage

Branch claude/nx-shader-2026-10-25-titles. Paths: gfx/src/shader/, gfx/src/mtl_shader.m, gfx/tests/t_xlate.m, gfx/tests/t_pipekey.c (new). nxnative and Ryujinx (MIT) were read as references; nothing copied.

Changes

  • G7 varyings start at (0,0,0,1). Already true on main (maxwell_translate.c, prologue: out.pos and every written out.vN = float4(0,0,0,1)); the review cited a stale reading. Now asserted in t_xlate.
  • G13 NaN maths.
  • FSETP/FSET/FCMP ordered NE was emitted as MSL a != b, which is true when either is NaN; Maxwell's ordered NE is false. Now (a < b) || (a > b). Real bug (the GPU check below failed before the fix).
  • FMNMX emits precise::fmin/fmax (the non-NaN operand wins whatever the library's math mode).
  • NAPPLE_GFX_MATH=safe|relaxed|fast (default safe on every platform, macOS unchanged). Measured on this Mac (M-series): fmin/fmax/saturate/clamp give the Maxwell answer for NaN in all three modes (hardware behaviour), isnan() survives, but relaxed already folds x != x to false. The translator never emits x != x (it uses isnan), so relaxed is a candidate for iOS (nxnative ships relaxed there) once measured on a device; not made default.
  • G14 SR_Y_DIRECTION. Was -1 always. Evidence: NVN's default origin is lower-left; nxnative's LAS notes say the register is -1 under lower-left (LASRecomp answers -1 for LAS); Ryujinx answers a constant +1 (correct for the upper-left titles it runs). Now -1 under the guest's lower-left origin, +1 under upper-left: ShaderKey.upper_left, set in build_pipe from the pipeline key (origin_flip && !flip_y; with NAPPLE_ORIGIN_FLIP=0 the key lacks the origin and the answer stays -1). LAS (lower-left, 85 of its corpus shaders read it) is unchanged; SMO's corpus never reads it.
  • VMNMX (unknown word 3b34056060470100 at 0x3e8 of Animal Crossing v0 fragment 23e43e94e45bb1d5): decoded with Ryujinx's InstVmnmx field layout = R0 = max.u32(max.u32(R1, R4), R10). Implemented in full (byte/half/word selects, signedness, imm16, SAT, MRG16H/L, MRG8B0/B2, ACC, MIN, MAX with Rc); the mixed-signedness case is exact in 64 bits (Ryujinx approximates it). Verified: AC v0 headless 60 s with NAPPLE_SHADER_DUMP, before 1 failed shader (this one), after shaders built 258 of 258 queued (0 failed); the dumped binary translates and compiles.
  • G11 first-use stutter (mtl_shader.m):
  • Persisted pipeline keys (pipelines.bin) now survive TRANSLATOR_VERSION bumps (only the key size must match). Before, every bump deleted the list, so each new build started with a cold prewarm. A stale key costs only a background build; draws always look up their own key.
  • Seed manifests: <cache>/pipelines-seed.bin and NAPPLE_GFX_PIPE_SEED=<file>[:<file>] (same format as pipelines.bin) feed the prewarm on a fresh install.
  • Key normalisation before hashing (pipe_key_normalize): blend state of disabled or unwritten targets, write masks / rt kinds of absent targets, attributes the vertex program never reads and streams only they use. Fragment variants keep only the varyings they read in vs_out_mask (ShaderInfo.attr_read_mask, indexed_inputs), so one fragment program paired with several vertex programs compiles once.
  • Speculation (opt-in, NAPPLE_GFX_SPECULATE=1): a program registered with no persisted key gets its most likely VS/FS variant compiled on two utility-QoS queues; a draw needing a variant being speculated waits for that build (no double work, nothing skipped). Off by default because SMO registers whole shader archives: cold 120 s run compiled 6673 speculative variants, 101 of them used.
  • Not done: Metal 4 base pipelines + specialisation (nxnative, Mac only), submission look-ahead (needs a hook in mtl_replay.m, see the integrator file).
  • TRANSLATOR_VERSION 8 (generated MSL changed).

Tests (all in make -C gfx test, ALL TESTS PASSED)

  • t_xlate: GPU-executed checks on a 1x1 RGBA32Uint target (translated fragment shader, registers r0..r3 as the colour): fmin(NaN,1), fmax(1,NaN), NaN.NE, 2.NE.1 in safe/relaxed/fast; SR_Y_DIRECTION for both origins; the AC VMNMX word plus byte/half/sign/imm/merge/acc and mixed-sign cases; G7 text check. Mutation-checked: reverting each change makes the matching check fail (7 failures).
  • t_pipekey (new): 5 draws differing only in irrelevant state build 1 pipeline (5 without normalisation), blending on builds a 2nd; 2 speculative variants compiled at registration and both used by the first draw (the test turns speculation on; 0 with it off); pass 2 (re-exec) finds the 2 persisted keys after the file header was rewritten to an old translator version.
  • Corpus guard unchanged: t_las 679/679 translated and compiled, t_smo 128/128.

SMO exploded geometry: ROOT CAUSE FOUND (not in the translator; fix belongs to api_cmd.c, owned by ac-render)

  • Cause: the command-list graveyard releases recordings idle for more than GRAVE_IDLE_S = 30 s (gfx/src/api_cmd.c), and SMO later calls those handles again. The calls are dropped (CallCommands: stale or invalid command handle ... ignored), so a model's display list runs without the state or draws it set up. In the repro (user save, Cascade Kingdom, NAPPLE_INPUT_SCRIPT='30:A:0.3,45:A:0.3,60:A:0.3', Mario idles and falls asleep), the parrot that lands on sleeping Mario after ~2 minutes is drawn from lists recorded long before: with them dropped it becomes the huge slabs/wedges.
  • A/B on the same data and binary (in-place SMO data, 215 s headless, frames 8000 and 9000), changing only GRAVE_IDLE_S:
  • 30 (current): 21 stale-handle warnings, frame 9000 covered by a huge blue/grey wedge (/Users/neto/nxs-evidence/idle30-{8000,9000}.png).
  • 1000000 (no idle release): 0 warnings, frames 8000/9000 correct, the parrot sits on Mario (/Users/neto/nxs-evidence/noidle-{8000,9000}.png).
  • Cost: live recordings at 215 s are 265523 without idle release vs 42239 with it (~7.5 KB each, about +1.7 GB): the reason the 30 s window exists (perf-las note). A fix must keep a handle callable while the game can still call it, and bound memory another way.
  • Fix options for ac-render: (a) never release a recording whose handle has not been finalized and whose command memory was not overwritten by a newer recording that the game actually calls; (b) release idle recordings only above a memory budget, oldest first, and log it; (c) keep idle recordings in a compact form (they are host copies). The "shattered cap brim" / "white ribbons" on the title menu are likely the same mechanism (menu idle > 30 s): recheck after the fix.
  • Ruled out on the way: the skinned VS fa3fa31651fa2f19 translation (read line by line, correct); the cloth/rope compute kernel fa336d3ab1ecc436 is a red herring for this glitch (its compute c0 UBO table 0x210 + 16 i and SSBO table 0x310 + 16 i match the layout the kernel reads: UBO 6/7/8/12, SSBO 0-2 written, 4-7 read). Kernel profile (Ryujinx opcode table): FMUL/FFMA (all .FTZ), MUFU RCP/SIN/LG2/EX2/RSQ/SQRT, RRO, LDG.128, STG.32/64, 19 BRA; all of it translated the way Ryujinx does.
  • New diagnostic: NAPPLE_SHADER_DUMP_HASH=<hash> with NAPPLE_SHADER_DUMP=<dir> writes that program's Maxwell code even when it translates.

45 s boots with the in-place data mode (branch claude/mk8d-64, napple build --data-only --update-romfs in-place, ~250 MB each)

Runtime = mk8d-64 + this branch merged in a scratch worktree (romfs.map support lives on mk8d-64). LAS 45 s headless: exit 0, 0 CRASH, 18 shaders 0 failed. SMO 45 s: exit 0, 0 CRASH, 144 shaders 0 failed. SMO 90 s / 215 s repro runs: exit 0.

Measurements (SMO, same route, 120 s headless, fresh save-root cache)

run pipelines built worst gap blocking builds notes
before G11 (TRANSLATOR v8) 1032 (514 shaders) 10.2 s 18.5 s Metal's own compiler cache partly warm from v7 runs
G11, speculation on 563 7.7 s 14.1 s 7094 shader compiles (6673 speculative, 101 used): background load
G11, speculation off (default) 565 (525 shaders) 1.0 s 0.5 s not comparable on time: the system Metal compiler cache was warm from the previous run

The robust number is the pipeline count: normalisation halves it (1032 -> 565) on the same route. Blocking times depend on the system compiler cache, which was not cleared (system state). Exit code 0 in all three runs.