Dual 3090s: the bottleneck isn't the GPU

Two fresh benches pin down the single-3090 ceiling

Two Qwen3.8-27B benchmarks landed this week, and they tell different stories about the same card. The first, from InsiderLLM’s Mark Bartlett, ran Qwen3.8-27B against Qwen3.6-27B on a single RTX 3090 with llama.cpp (build b10088). Single-user decode came in at 41.49 tok/s for 3.8 against 41.77 for 3.6 — essentially flat, with a 0.67% slowdown in the new weights. Peak VRAM was 17.94 GB. That is the lower bound: what you get when you load the model in the conservative toolchain and ask for one stream of text.

The second, a community repo called syv-ai/qwen38-27b-rtx3090, runs the same weights on the same card under vLLM with a tuned multi-token-prediction (MTP) drafter. That drafter is a small companion model that proposes batches of tokens for the main model to verify, letting generation skip ahead. Single-user decode reads 113.6 tok/s at default sampling, 118 greedy. Swap the MTP drafter for a DFlash2 block drafter (one that proposes seven tokens per pass instead of four chained) and the same card delivers 132 tok/s greedy, and up to 381 tok/s on prompts that reproduce a 25k-token document. That is the upper bound: every layer of the stack hand-tuned for this card and this model.

3.2×the same Qwen3.8-27B on the same RTX 3090: 41 tok/s in vanilla llama.cpp, 132 tok/s with the tuned DFlash2 drafter in vLLM.

The spread between those two numbers is the same model, on the same card, with the same 24 GB of VRAM. The difference is the stack.

What 48 GB actually buys you

A reader with two RTX 3090s (48 GB VRAM total), a Ryzen 5 5600X and llama.cpp 0.1.2-dev on Windows 10 without WSL is asking whether their Qwen3.8-27B performance is normal. The two benchmarks give a precise answer. The Qwen3.8-27B weights in Unsloth’s UD-Q4_K_XL quant measure 17.9 GB on disk and use 17.94 GB of VRAM at peak on a single 3090. The second card gives 48 GB total — enough headroom to run the model alongside a draft model on a separate GPU, or to push context beyond what a single 3090’s KV cache (the working memory the model uses to track long context) can hold. The hardware is not where the bottleneck sits for a build like this.

If you are on vanilla llama.cpp on Windows, you are on the slow path. The InsiderLLM bench used Linux; llama.cpp’s CUDA backend has historically had rough edges on native Windows. The bigger gap is that vanilla llama.cpp does not run the MTP speculation path that the syv-ai repo uses to triple single-user throughput. There is also a known issue with llama.cpp’s tensor split across multiple GPUs — exactly the path a dual-3090 owner needs. Llama.cpp stays within 6% of vLLM in the single-stream case, but only when the right knobs are turned. Speculation is lossless: speculative decoding samples the same distribution as no speculation at all.

What to try first

For the dual-3090 owner, three moves are worth more than any others:

• Establish your own baseline before changing anything. A single 3090 with llama.cpp should land near 41 tok/s at depth zero with default settings. If you are seeing less than 35, the bottleneck is configuration, not hardware — sanity-check -ngl, the draft-model path and prompt-processing GPU offload. The same finding held for an earlier Qwen 3.8 build that turned out to be a stack issue rather than a model issue.
• Switch the drafter if you want single-user speed. The syv-ai repo’s DFlash2 setup takes a single 3090 from 41 tok/s to 132 tok/s greedy. It is Docker-based and Linux-first; on Windows without WSL, this is a harder path. The real answer is that this kind of gain justifies setting up WSL or moving the model to a Linux box — it is not reachable from the existing Windows-native toolchain.
• Treat the second card as headroom, not parallel speed. With llama.cpp’s tensor split path having known issues, the second 3090 buys you room to run longer contexts (a 200k context run that would not fit on one card can fit across two) or to host the model and a draft model on separate cards. It does not buy a 2× decode speedup in the way naive GPU parallelism suggests.

Is my performance normal? is really two questions: what is normal, and what is possible. Normal, on vanilla llama.cpp on Windows, is somewhere south of the InsiderLLM 41 tok/s figure. Possible, on the same hardware with the right stack, is three times that. The spread is not closed by better hardware. It is closed by moving off the slowest path in the stack. Two RTX 3090s is more Qwen3.8-27B than almost anyone needs; the ceiling on the build is the toolchain, not the silicon.