NVIDIA H100 Specifications
Architecture
Hopper
Memory
80 GB HBM3
AI Performance
4 PFLOPS (FP8)
Specifications compiled from NVIDIA official documentation, architecture whitepapers, and public cloud pricing. Prices vary by vendor and availability.
If you want to understand the current AI boom, you have to understand the NVIDIA H100. Before the RTX Spark brought AI to laptops, the H100 was the engine room of the entire industry. It is the GPU that trained models like GPT-4, Claude, and Gemini.
But it is not a graphics card. You cannot plug it into a gaming PC. It is a specialized data-center accelerator designed for one purpose: moving massive amounts of data through matrices of math as fast as physically possible.
Here is the fact-checked breakdown of its architecture, why it costs tens of thousands of dollars, and how it connects to the TSMC manufacturing nodes we discussed previously.
What is the Hopper Architecture?
The H100 is built on NVIDIA's Hopper architecture, the successor to Ampere. It is manufactured on a custom TSMC 4N process (a customized version of 5nm). It packs 80 billion transistors into a single die.
Unlike consumer GPUs (like the RTX 5070), the H100 does not have RT (Ray Tracing) cores. It is designed entirely around Tensor Cores and CUDA cores for AI and High-Performance Computing (HPC) workloads.
The Transformer Engine: Why It Matters for AI
The defining feature of the H100 is its Transformer Engine.
Modern AI models (like Large Language Models) are based on the Transformer architecture. They rely heavily on matrix multiplication. The H100 includes dedicated hardware that can perform these calculations in FP8 (8-bit floating point) precision.
Memory and Bandwidth: The Real Bottleneck
AI models are essentially huge blocks of numbers. The GPU needs to feed data to its compute cores constantly. If the memory bandwidth is too slow, the GPU sits idle waiting for data.
The H100 uses 80 GB of HBM3 memory with a staggering 3.35 TB/s bandwidth. To put that in perspective, a high-end consumer gaming GPU has about 1 TB/s of bandwidth. The H100 has more than three times that.
This massive bandwidth is why the H100 can handle models with hundreds of billions of parameters.
How It Connects: NVLink and Clusters
No single H100 can train a frontier model. They must be linked together in massive AI clusters.
The H100 uses NVLink 4.0, which provides a direct, 900 GB/s connection between GPUs. This bypasses the much slower PCIe bus. In a DGX H100 server (NVIDIA's reference design), eight H100 GPUs are connected via NVLink to act as a single massive compute unit.
The Economics: Why Does It Cost $30,000?
The H100 is expensive to manufacture. The TSMC 4N process is advanced, the HBM3 memory is custom and requires complex packaging, and the massive die size (814 mm²) means yields are lower.
- Hardware Cost: The physical chip, memory, and board components are estimated to cost several thousand dollars.
- R&D and Software: NVIDIA spent billions developing the Hopper architecture and the CUDA software ecosystem that runs on it.
- Supply and Demand: Because AI demand exploded, the market price for H100s surged far above the initial MSRP. Companies were willing to pay whatever it took to get compute.
Export Restrictions and the Geopolitical Chip War
Because the H100 is critical for advanced AI, the United States government imposed export controls on the chip, restricting its sale to certain countries (notably China). This led NVIDIA to create restricted-performance variants (like the H800) specifically for those markets.
These restrictions are a major factor in the global race to build independent semiconductor supply chains.
The Bottom Line
The NVIDIA H100 is the defining piece of hardware of the early 2020s AI era. It is an engineering marvel, a massively expensive piece of silicon, and the physical engine behind the generative AI revolution. While newer chips like the Blackwell B200 and the RTX Spark are now taking the spotlight, the H100 remains the workhorse that started it all.
Frequently Asked Questions
Can I use an NVIDIA H100 for gaming?
Technically, it can render graphics, but it is a terrible experience. It lacks the display outputs and the RT/geometry hardware optimized for gaming. It is not designed for it, and a $500 consumer GPU will provide a better gaming experience.
What is the difference between H100 SXM and PCIe?
The SXM version plugs into a specialized motherboard (like the HGX baseboard) and uses NVLink for high-speed GPU-to-GPU communication. It runs at a higher power limit (up to 700W). The PCIe version plugs into a standard server PCIe slot, runs at a lower power limit (350W), and uses PCIe for communication, which is slower for AI scaling.
How many H100s do I need to train a model?
It depends on the model size. Fine-tuning a small model might take one or two GPUs. Training a frontier model (like GPT-4) requires tens of thousands of H100s linked together over months.
Sources & Fact-Check
This article was drafted with AI assistance and reviewed by a human editor. All specifications, pricing, and performance figures were cross-referenced against the following primary sources: