GPU sizing + model compatibility matrix #10

New issue

Closed

opened 2026-09-01 20:40:06 +00:00 by Mark · 0 comments

Mark commented

2026-09-01 20:40:06 +00:00

Owner

Smoke-tested on RTX 2060 (6GB usable, base model = ~1GB VRAM, small = ~2GB). Production deployment probably has different hardware.

Need to know:

medium (~5GB VRAM) — does it work on 2060? Does it actually improve accuracy on real-world content?
large-v3 (~10GB VRAM) — needs 12GB+ card
large-v3-turbo (~6GB) — sweet spot?
CPU fallback — what's the slowdown ratio? (rough estimate: 30-50x slower)
Apple Silicon (MPS backend) — supported by WhisperX?
Mixed precision: int8 vs float16 vs float32 — accuracy/speed tradeoffs

What needs building: references/gpu-sizing.md with the actual measured numbers from running each model size on at least one GPU tier, plus the CPU estimate.

Smoke-tested on RTX 2060 (6GB usable, base model = ~1GB VRAM, small = ~2GB). Production deployment probably has different hardware. **Need to know:** - `medium` (~5GB VRAM) — does it work on 2060? Does it actually improve accuracy on real-world content? - `large-v3` (~10GB VRAM) — needs 12GB+ card - `large-v3-turbo` (~6GB) — sweet spot? - CPU fallback — what's the slowdown ratio? (rough estimate: 30-50x slower) - Apple Silicon (MPS backend) — supported by WhisperX? - Mixed precision: int8 vs float16 vs float32 — accuracy/speed tradeoffs **What needs building:** `references/gpu-sizing.md` with the actual measured numbers from running each model size on at least one GPU tier, plus the CPU estimate.