NVIDIA Hopper Architecture at a Glance

When NVIDIA unveiled the Hopper architecture at GTC 2022, CEO Jensen Huang did not mince words. He called it "a giant leap over Ampere" and "our largest generational leap ever". That was not marketing hyperbole. Hopper, and the H100 GPU built on it, became the engine of the generative AI revolution. It trained ChatGPT, powered Claude, and turned NVIDIA into one of the most valuable companies on Earth.

But what exactly made Hopper so special? It was not a single breakthrough, but a combination of five innovations that worked together to solve the bottlenecks of large-scale AI. Here is the complete breakdown of the architecture that changed computing.

What is Hopper? (And Why the Name Matters)

Hopper is the codename for NVIDIA's ninth-generation data center GPU architecture, succeeding Ampere (A100). It is manufactured on a custom TSMC 4N process and packs 80 billion transistors into a single die.

The name is a tribute to Grace Hopper, the computer scientist and U.S. Navy rear admiral who invented the first compiler and helped develop COBOL. She is also credited with popularizing the term "bug" after a moth was found in a computer relay. NVIDIA had already used her first name for its Arm-based Grace CPU, and the Hopper GPU is designed to pair with it in the Grace Hopper Superchip.

The Five Innovations That Defined Hopper

NVIDIA's own documentation highlights five breakthrough features that fuel the H100 and H200. Each one addresses a different bottleneck in AI and high-performance computing.

1. The Transformer Engine (FP8 Precision)

The most important innovation for AI is the Transformer Engine. Modern AI models are built on the Transformer architecture, which relies on massive matrix multiplications. Hopper introduced FP8 (8-bit floating point) precision to its fourth-generation Tensor Cores, but with a twist: it can dynamically choose between FP8 and FP16 precision on the fly.

2. HBM3 Memory (3.35 TB/s)

The H100 SXM5 was the world's first GPU to use HBM3 memory, delivering over 3 TB/s of bandwidth—roughly double the A100's 1.5 TB/s. A larger 50 MB L2 cache also reduces trips to DRAM, further improving efficiency. This massive bandwidth is essential for feeding the Tensor Cores with data fast enough to keep them busy.

3. NVLink 4 and NVSwitch (900 GB/s)

Training frontier models requires thousands of GPUs working in sync. PCIe is far too slow for this. Hopper introduced NVLink 4, delivering 900 GB/s of GPU-to-GPU bandwidth—over 7x faster than PCIe Gen5. Combined with the third-generation NVSwitch, it allows up to 256 H100 GPUs to be connected in a single, non-blocking domain. We covered this in detail in our NVLink 4 explainer.

4. DPX Instructions (Dynamic Programming)

Hopper is not just for AI. It also targets high-performance computing (HPC). The new DPX instructions accelerate dynamic programming algorithms—such as Smith-Waterman for genomics and Floyd-Warshall for robotics routing—by up to 7x over the A100. This makes Hopper a versatile platform for both AI and scientific research.

5. Confidential Computing

The H100 is the world's first GPU with built-in Confidential Computing. This allows data to remain encrypted even while it is being processed in GPU memory, protecting sensitive workloads in multi-tenant cloud environments. It is a critical feature for enterprises and governments that need to run AI on private data without exposing it to the cloud provider.

The H100 Variants: One Chip, Three Forms

NVIDIA sells the H100 in three distinct form factors, each tuned for different server environments. The specs differ significantly.

Specification	H100 SXM5	H100 PCIe	H100 NVL
Memory	80 GB HBM3	80 GB HBM2e	94 GB HBM3 (per GPU)
Bandwidth	3.35 TB/s	2.0 TB/s	~3.9 TB/s
CUDA Cores	16,896	14,592	14,592
TDP	700 W	350 W	~400 W

The SXM5 is the high-performance module used in HGX and DGX systems. The PCIe card fits standard servers but sacrifices memory bandwidth (HBM2e instead of HBM3) and core count. The NVL is a dual-GPU card designed specifically for large language model inference, offering 94GB of HBM3 per GPU and 188GB per card.

Hopper vs. Ampere: The Generational Leap

NVIDIA claimed the H100 delivered up to 9x faster AI training and 30x faster inference than the A100 on large language models. These are not small gains; they are the difference between training a model in months versus weeks, or running inference at a cost that makes a product viable versus unviable.

The chart below summarizes the key improvements.

Metric	Ampere (A100)	Hopper (H100 SXM5)
Memory	40/80 GB HBM2e	80 GB HBM3
Bandwidth	1.5 - 2.0 TB/s	3.35 TB/s
NVLink	600 GB/s (NVLink 3)	900 GB/s (NVLink 4)
AI Precision	FP16, BF16, TF32	FP8 (with Transformer Engine)
Transistors	54 billion	80 billion

The Legacy: How Hopper Built the AI Boom

Hopper was not just a hardware upgrade. It arrived at the exact moment the AI industry needed it. The Transformer architecture (the "T" in GPT) was scaling rapidly, and the A100 was beginning to show its limits for the largest models. Hopper's combination of FP8, massive memory bandwidth, and fast interconnect made it possible to train models with hundreds of billions of parameters.

The H100 became the most sought-after piece of technology in the world. Cloud providers, AI labs, and enterprises scrambled to secure supply. It powered the training of GPT-4, Claude, and Gemini, and it remains the workhorse of production AI inference even in 2026, as newer Blackwell systems come online.

As NVIDIA moves to Blackwell and Rubin, Hopper's influence endures. The Transformer Engine, FP8, and NVLink 4 are now standard features across the industry, and the H100 is the benchmark against which every new AI chip is measured.

The Bottom Line

NVIDIA's Hopper architecture was a masterclass in targeted engineering. It did not try to be everything to everyone. It identified the five bottlenecks holding back large-scale AI—precision, memory bandwidth, interconnect speed, HPC flexibility, and security—and solved them simultaneously. The result was the H100, the chip that turned generative AI from a research curiosity into a global industry.

Frequently Asked Questions

Is Hopper the same as H100?

Hopper is the architecture. H100 is the specific GPU product built on that architecture. The H200 is also built on Hopper, but with more and faster memory.

Why is it called Hopper?

It is named after Grace Hopper, a pioneering computer scientist and U.S. Navy rear admiral who invented the first compiler. NVIDIA also used her first name for its Grace CPU, which pairs with Hopper in the Grace Hopper Superchip.

What is the Transformer Engine?

It is a combination of hardware and software in Hopper that dynamically chooses between FP8 and FP16 precision during AI training. It halves memory usage and doubles throughput for Transformer models, which are the foundation of modern LLMs.

Can I buy an H100 for gaming?

Technically, it can render graphics, but it is not designed for it. It lacks display outputs and gaming-specific hardware. A consumer RTX 5090 will provide a vastly better gaming experience at a fraction of the cost.

Sources & Fact-Check

This article was drafted with AI assistance and reviewed by a human editor. All specifications, pricing, and performance figures were cross-referenced against the following primary sources:

• NVIDIA Technical Blog — NVIDIA Hopper Architecture In-Depth (March 2022)
• NVIDIA — H100 Tensor Core GPU Datasheet
• NVIDIA — H100 GPU Product Page
• TechPowerUp — H100 SXM5 80 GB Specifications
• Morph — NVIDIA H100: Specs, Price, and Comparison (July 2026)