Published · Phase 1

Qwen3.8 27B Hardware Requirements

For Qwen3.8-27B, use a 24–32 GB GPU for 4-bit local inference, 48 GB for comfortable 8-bit/long-context work, and 64–80 GB for BF16-class experiments. RTX 5090 is the best speed/value recommendation at 4 bit.

Editorial review: complete · Updated 2026-08-30

Direct answer

Direct answer guidance: Buy 32 GB for 4-bit speed, 48 GB for 8-bit headroom, and 80+ GB only if BF16 or several resident services are required.

Model / quant variants

Model / quant variants guidance: llama.cpp is excellent for 4–6 bit GGUF. vLLM is the preferred API/batching runtime for the official Transformers checkpoint and compatible quantizations.

Approximate weight footprint

Approximate weight footprint guidance: The official BF16 repository is about 55.6 GB. Plan roughly 16–19 GB at 4 bit, 29–33 GB at 8 bit, and 58–65 GB at BF16 after packaging/runtime overhead, then add KV cache.

Hardware tiers

Hardware tiers guidance: The official BF16 repository is about 55.6 GB. Plan roughly 16–19 GB at 4 bit, 29–33 GB at 8 bit, and 58–65 GB at BF16 after packaging/runtime overhead, then add KV cache. A 16 GB GPU can use smaller quantizations with tight context but is not the recommended experience; cloud GPUs are sensible for occasional BF16 evaluation.

Observed configurations

Observed configurations guidance: A 5090-class GPU can deliver roughly 70–130 tok/s at 4 bit; DGX Spark examples are closer to 25–40 tok/s but support higher precision and much larger context.

Context / concurrency effects

Context / concurrency effects guidance: On 32 GB, start at 16K. On 48–80 GB, 32K+ becomes easier, but TTFT still grows and must be measured.

Performance expectations

Performance expectations guidance: A 5090-class GPU can deliver roughly 70–130 tok/s at 4 bit; DGX Spark examples are closer to 25–40 tok/s but support higher precision and much larger context.

Best-value configurations

Best-value configurations guidance: 4-bit on a 32 GB consumer GPU is the value point. Each precision tier above it should be justified by measured task quality. Buy 32 GB for 4-bit speed, 48 GB for 8-bit headroom, and 80+ GB only if BF16 or several resident services are required.

What will not fit / weak evidence

  • Quantization file size is not identical to peak runtime memory.
  • Vision inputs and large caches can move a configuration into the next hardware tier.

Planner CTA

Compare this recommendation against your model, context, concurrency, latency, and budget in the ComputeSage Planner.

Sources / methodology

Recommendations combine official specifications, public model/runtime documentation, adjacent benchmark observations, and clearly labeled engineering estimates. Review the ComputeSage evidence methodology.