Skip to content

media-mac (2026-10-27, batch b6): VideoToolbox DecoderBackend, NVDEC/VIC stream shape from a real title

Branch claude/media-mac-2026-10-27-b6 (Mac session, base 34ce68b). Verified on this Mac with MK8D (headless, audio off): the title's NVDEC channel decodes its 512x608 preview video through VideoToolbox. The preview box on screen still does not show the picture (precise remaining cause below).

What changed

  1. services/decoder_vt.c: a real DecoderBackend (videotoolbox, ABI 1) over vt_h264.c, installed by a constructor; NAPPLE_DECODER=null opts out. configure() starts a fresh VtH264 (vt_h264 keeps the first SPS it saw), decode() decodes then flushes so the request's own frame is returned, reset()/close() forward. Host test tests/test_decoder_vt.c (make -C runtime test): encodes a clip with VideoToolbox, feeds Annex B access units through the backend as NVDEC would, checks size/luma/chroma per frame and a frame checksum, determinism over two runs, VP9 refused, garbage/empty requests (107 checks).
  2. nvdrv media engine against MK8D's real command stream (nvdrv_media_core.[ch], nvdrv_media_regs.h): the old engine never executed anything for MK8D because the SDK writes every method through the THI FIFO (INCR 0x10 count 2 pairs). Now: THI method/data decode; DPB slot registers corrected (luma 0x10c+idx, chroma 0x11d+idx; the old 0x140/0x160 were guesses, the display-buffer fallback moved to unused 0x130/0x131 for the tests); nvdec_synth_h264 builds the SPS/PPS from the picture-info structure (words 18..45, matched against MK8D's 512x608 CABAC stream: stream length at 18, 32x38 MBs at 25/26, pitches 0x200 at 35/36, bit-field word 44, weighted_bipred at 45) because the slice NAL units carry no parameter sets; VIC register numbers corrected (input 0x100/0x101, config 0x1c1, output 0x1c8/0x1c9) and an NV12 -> NV12 copy when the config structure is unreadable; media_write_prepare hook (gfx dirty tracking) set by media_gfx_glue.c.
  3. Debug aid: NAPPLE_MEDIA_TRACE=1 prints the first NVDEC decodes (backend, size, rc) and all failures.
  4. Tests: test_svc_nvdrv_media 115 -> 121 (THI pairs, synthesised SPS/PPS incl. reconfigure on a size change, VIC copy) and now installs the null backend explicitly (the runtime test builds link the VT constructor).

Evidence (MK8D, base game + update + DLC, headless, NAPPLE_AUDIO_DEVICE=0)

Before: nvdec/vic channels opened at 19 s and 309 submits ran, but stats.executes stayed 0 (no [nvdec] line ever). After: every submit executes; NAPPLE_MEDIA_TRACE shows backend videotoolbox codec 3 ... rc 0 512x608 for the IDR (43262 bytes) and every following P frame, then vic copy with non-zero luma. The 150 s golden route exits 0, 0 gfx warnings, 0 Metal validation, no crash (frame diffs against goldens remain, as in every earlier run).

Not fixed: the preview box

The panel went from black to green (Y=0, U=V=0 sampled as YUV), i.e. the game now gets its fences and draws the panel, but the texture it samples does not contain our frames. Findings that pin the cause: - The VIC output surfaces (nvmap handles [+0x4c000] luma, [+0x26000] chroma: NV12 512x608) are NOT covered by any NVN memory pool (NAPPLE_MAPLOG + a pool log: none contains them) and no NVN texture lives in them; MapVirtual maps only one unrelated range. So the game does not sample these surfaces through gfx's pool/texture path, and nothing in the media core can reach its texture. - The VIC config structure address (0x610110 << 8 = 0x61011000) lies outside every pinned handle (nv_resolve misses), so the real VIC configuration (output format/size/swizzle, scaling) is unread; the NV12 copy is a stand-in. - Candidates for the missing link, to be settled with a Mac trace of the game's own use of the output handle: (a) the game maps the nvmap handle into the GPU address space (nvhost-as-gpu MAP_BUFFER_EX) and a GPU copy/compute pass reads it (needs the nvgpu AS in nvdrv and gfx to alias it), (b) it CPU-copies from the handle into a texture at a moment this runtime serves different data, (c) the VIC config selects block-linear output and the copy needs the swizzle. Which one is a gfx + nvdrv.c question (outside this area).

Not done / stays unverified

  • Movie audio (Vorbis/Opus): the existing 37-check test_svc_movie_audio and the code path were left as they are; MK8D's audio decoder objects configure but no audible run is possible (audio is off by rule), so the Mac check stays open.
  • JPEG: test_svc_jpeg (17 checks) and test_media (ImageIO round trip) already cover decode/encode; no change.
  • VP9 on the VideoToolbox backend (refused: libvpx path stays in movie.c); non-1:1 pitch (guest pitch > width) is not handled by the backend ABI (MK8D's pitch equals the width).