Published · Phase 3

Choose Hardware for a Local LLM

A useful LLM hardware calculator should recommend a tier from model size, precision, context, concurrency, latency, and budget—not merely say whether weights fit. Use the result as a shortlist, then validate the top configuration with a representative run.

Editorial review: complete · Updated 2026-08-30

Direct answer

Direct answer guidance: Recommend RTX 5090 below ~28 GB total, 48 GB below ~42 GB, RTX PRO/80–96 GB below ~85 GB, DGX Spark below ~105 GB, and multi-GPU/cloud above that—subject to latency and support needs.

Model / quant variants

Model / quant variants guidance: Select llama.cpp for GGUF/local flexibility and vLLM for concurrent serving. Runtime choice can change memory and throughput enough to alter the hardware recommendation.

Approximate weight footprint

Approximate weight footprint guidance: Calculate weight bytes from parameters and precision, add 10–20% runtime overhead, estimate KV cache for context × sequences, add multimodal/adapters/draft models, then require 10–20% spare capacity.

Hardware tiers

Hardware tiers guidance: Calculate weight bytes from parameters and precision, add 10–20% runtime overhead, estimate KV cache for context × sequences, add multimodal/adapters/draft models, then require 10–20% spare capacity. If the recommendation is too expensive, reduce precision/context, choose a smaller model, use retrieval, or rent cloud hardware for occasional jobs.

Observed configurations

Observed configurations guidance: Estimate speed from the closest public model/hardware/runtime results, then publish a range widened for differences in precision, prompt, and batching. Approximation is useful when the uncertainty is explicit.

Context / concurrency effects

Context / concurrency effects guidance: Make context and concurrent sequences explicit inputs. Using a model's maximum context by default will systematically overstate hardware needs and understate latency.

Performance expectations

Performance expectations guidance: Estimate speed from the closest public model/hardware/runtime results, then publish a range widened for differences in precision, prompt, and batching. Approximation is useful when the uncertainty is explicit.

Best-value configurations

Best-value configurations guidance: Compare capital cost, energy, support, utilization, cloud/API alternative, and operator time. A cheaper system that misses latency or fit requirements is not lower cost. Recommend RTX 5090 below ~28 GB total, 48 GB below ~42 GB, RTX PRO/80–96 GB below ~85 GB, DGX Spark below ~105 GB, and multi-GPU/cloud above that—subject to latency and support needs.

What will not fit / weak evidence

  • Display estimates as ranges and list the assumptions that would move the recommendation up or down a tier.
  • Recompute when the model revision, quantization, runtime, context, or concurrency changes.

Planner CTA

Compare this recommendation against your model, context, concurrency, latency, and budget in the ComputeSage Planner.

Sources / methodology

Recommendations combine official specifications, public model/runtime documentation, adjacent benchmark observations, and clearly labeled engineering estimates. Review the ComputeSage evidence methodology.