GPU Memory Sizing Guide: H100 vs H200 for 70B Inference and KV Cache

A model that fits on paper can still fail when real users arrive. The missing line is often not the model weights: it is the key-value cache, runtime workspace, or a parallel layout that cannot distribute memory as evenly as the spreadsheet assumes.

This GPU memory sizing guide answers a practical procurement question: how many H100 or H200 GPUs should you budget for a 70-billion-parameter inference service? It includes a reusable calculation, a concurrency table, two original diagrams, and an acceptance checklist. All designs are hypothetical; the arithmetic was executed in Python, but no GPU benchmarks or deployment tests were run for this article.

X Follow @ipexptobe

AI GPU Cluster Network Calculator: Ports, Bandwidth and Switch Sizing for 8–512 GPUs

Most AI-cluster articles describe products. Engineers usually need a different answer: how many ports, switches, links and watts are implied by the GPU count? This page starts that answer with a practical calculator and a repeatable sizing method for early design conversations.

The calculator below is intentionally conservative: it performs visible arithmetic for GPU count, network links and switch-port demand. It does not pretend to be a validated reference architecture, a quote, or a benchmark. Use it to catch impossible assumptions before a vendor design review.

X Follow @ipexptobe

NVIDIA InfiniBand for AI Clusters: NDR vs XDR, Rail Design and Acceptance Tests

A GPU cluster can have every cable connected and still be unready for training. The useful engineering question is not simply “NDR or XDR?” It is whether the port map, subnet management, routing policy, GPU-to-NIC alignment and failure behavior describe the same system.

This focused AI infrastructure installment develops a hypothetical 64-server, 512-GPU InfiniBand design, then turns it into a commissioning workflow. Calculations below are theoretical capacity accounting, executed with Python—not benchmark results. The example assumes eight GPUs and eight independent 400Gb/s network ports per server; it is not a claim that every eight-GPU server has that topology.

X Follow @ipexptobe