Mark/vllm_mm

Fork 0

[CRITICAL] vLLM engine core initialization fails on RTX 2060 (CUDA OOM) #2

New issue

Closed

opened 2026-04-17 00:33:06 +00:00 by Mark · 1 comment

Mark commented

2026-04-17 00:33:06 +00:00

Owner

vLLM engine core initialization fails on RTX 2060 due to CUDA OOM during model load.

Environment

GPU: NVIDIA GeForce RTX 2060 (12 GB VRAM, compute capability 7.5 / sm_75)
vLLM version: 0.19.0
Backend container: localhost/vllm_mm_backend:latest

Symptoms

Every vLLM model fails to start with:

RuntimeError: Engine core initialization failed. Failed core proc(s): {}

Root Cause

CUDA out-of-memory during gpu_model_runner.py weight loading.

Manual test with Qwen/Qwen2-7B-Instruct inside the backend container shows the explicit OOM:

ERROR  [gpu_model_runner.py:4818] Failed to load model - not enough GPU memory.
CUDA out of memory. Tried to allocate 130.00 MiB.
GPU 0 has a total capacity of 11.55 GiB of which 139.00 MiB is free.

Additional factors:

sm_75 does not support torch.bfloat16, so vLLM falls back to torch.float16, nearly doubling weight size vs. BF16.
vLLM V1 engine compiles/captures cudagraphs upfront, consuming significant VRAM before model weights are even loaded.
--cpu-offload-gb is passed correctly but the engine core dies before offloading can take effect (during the initial GPU model runner initialization).

Affected Models

Qwen/Qwen3-30B-A3B (with --cpu-offload-gb 8.0)
Qwen/Qwen3.5-35B-A3B (with --cpu-offload-gb 24.0)
Qwen/Qwen2-7B-Instruct (no offload)
openai/gpt-oss-20b (already deleted due to mxfp4 sm_80+ requirement)

What Works

llama.cpp + GGUF Q4_K_M runs correctly on the same GPU (verified with Qwen3.5-9B-Q4_K_M.gguf and speculative decoding).

Proposed Fix / Next Steps

Use quantized vLLM models (AWQ, GPTQ, or FP8) that fit within ~10 GB VRAM.
Or test with a much smaller dense model (<3B parameters in fp16).
Or investigate vLLM flags to reduce startup VRAM (e.g. disable cudagraphs, reduce max_num_seqs, lower gpu_memory_utilization further).

Until then, vLLM backend is non-functional on this hardware.

vLLM engine core initialization fails on RTX 2060 due to CUDA OOM during model load. ## Environment - GPU: NVIDIA GeForce RTX 2060 (12 GB VRAM, compute capability 7.5 / sm_75) - vLLM version: 0.19.0 - Backend container: `localhost/vllm_mm_backend:latest` ## Symptoms Every vLLM model fails to start with: ``` RuntimeError: Engine core initialization failed. Failed core proc(s): {} ``` ## Root Cause CUDA out-of-memory during `gpu_model_runner.py` weight loading. Manual test with `Qwen/Qwen2-7B-Instruct` inside the backend container shows the explicit OOM: ``` ERROR [gpu_model_runner.py:4818] Failed to load model - not enough GPU memory. CUDA out of memory. Tried to allocate 130.00 MiB. GPU 0 has a total capacity of 11.55 GiB of which 139.00 MiB is free. ``` Additional factors: - sm_75 does not support `torch.bfloat16`, so vLLM falls back to `torch.float16`, nearly doubling weight size vs. BF16. - vLLM V1 engine compiles/captures cudagraphs upfront, consuming significant VRAM before model weights are even loaded. - `--cpu-offload-gb` is passed correctly but the engine core dies **before** offloading can take effect (during the initial GPU model runner initialization). ## Affected Models - `Qwen/Qwen3-30B-A3B` (with `--cpu-offload-gb 8.0`) - `Qwen/Qwen3.5-35B-A3B` (with `--cpu-offload-gb 24.0`) - `Qwen/Qwen2-7B-Instruct` (no offload) - `openai/gpt-oss-20b` (already deleted due to `mxfp4` sm_80+ requirement) ## What Works - **llama.cpp + GGUF Q4_K_M** runs correctly on the same GPU (verified with `Qwen3.5-9B-Q4_K_M.gguf` and speculative decoding). ## Proposed Fix / Next Steps 1. Use **quantized vLLM models** (AWQ, GPTQ, or FP8) that fit within ~10 GB VRAM. 2. Or test with a much smaller dense model (<3B parameters in fp16). 3. Or investigate vLLM flags to reduce startup VRAM (e.g. disable cudagraphs, reduce `max_num_seqs`, lower `gpu_memory_utilization` further). Until then, **vLLM backend is non-functional on this hardware**.

Mark added the

blocked:no-procedure

label

2026-05-15 21:00:02 +00:00

Mark commented

2026-05-15 23:06:55 +00:00

Author

Owner

Ops Pipeline — No Executable Procedure

Research did not produce any executable commands.

Here is the ops assessment for vLLM Ops Issue #2.

HOST: blackhole (localhost — RTX 2060, 12GB VRAM, sm_75, CUDA 13.0)

STATUS: EXECUTABLE

EVIDENCE:

The CUDA OOM during vLLM engine init has two contributing causes on this host:

VRAM auto-tuning has a threshold gap. In backend/app/services/backends/vllm_backend.py, the VRAM profile bands are:
- _VRAM_LOW = 6 (applies conservative tuning: 0.65 gpu_mem, enforce_eager, 8192 ctx)
- _VRAM_MEDIUM = 8 (applies mode

## Ops Pipeline — No Executable Procedure Research did not produce any executable commands. Here is the ops assessment for vLLM Ops Issue #2. --- **HOST:** blackhole (localhost — RTX 2060, 12GB VRAM, sm_75, CUDA 13.0) **STATUS:** EXECUTABLE **EVIDENCE:** The CUDA OOM during vLLM engine init has two contributing causes on this host: 1. **VRAM auto-tuning has a threshold gap.** In `backend/app/services/backends/vllm_backend.py`, the VRAM profile bands are: - `_VRAM_LOW = 6` (applies conservative tuning: 0.65 gpu_mem, enforce_eager, 8192 ctx) - `_VRAM_MEDIUM = 8` (applies mode

Mark added

and removed

labels

2026-05-28 07:23:05 +00:00

Mark added the

in-progress

label

2026-06-05 18:48:13 +00:00