Introduction
Every time we hear about ChatGPT, Gemini, Claude, or any modern AI model, another term appears alongside it:
GPU.
Sometimes we even hear about Google TPUs.
This made me wonder:
If my laptop already has a CPU, why do companies spend millions of dollars buying GPUs and TPUs?
The answer lies in how these processors are designed.
Although all three process information, they solve problems very differently.
Let's understand why.
Imagine Three Engineers...
Suppose a company receives 10,000 customer invoices that need to be verified.
Three engineers volunteer.
👨 Engineer 1 (CPU)
He is extremely intelligent.
He checks one invoice carefully.
Finishes.
Moves to the next.
His strength isn't speed.
His strength is decision making.
👨👨👨 Engineer 2 (GPU)
Instead of working alone...
He brings 5,000 assistants.
Each assistant verifies one invoice.
The work finishes much faster.
🏭 Engineer 3 (TPU)
Google hires a specialist.
Instead of hiring more people...
They build an assembly line where invoices continuously move through dedicated stations.
Each station performs exactly one mathematical operation.
No unnecessary decisions.
Only maximum throughput.
That is essentially the difference between CPU, GPU and TPU.
Now let's see what happens internally.
🧠 CPU — Built for Decisions, Not Repetition
A CPU (Central Processing Unit) is designed to execute many different kinds of instructions efficiently.
Internally it consists of:
- Control Unit (CU)
- Arithmetic Logic Unit (ALU)
- Multiple CPU Cores
- Cache hierarchy (L1 → L2 → L3)
- Registers
- Main Memory Interface
How a CPU Executes an Instruction
Every instruction follows a cycle.
Instruction ↓ Fetch ↓ Decode ↓ Execute ↓ Store
Let's understand each stage.
1. Fetch
The Control Unit fetches an instruction from RAM.
Example:
ADD A, B
2. Decode
The CPU determines:
- What operation?
- Which registers?
- Which memory locations?
3. Execute
The ALU performs:
- arithmetic
- logical comparison
- branching
4. Store
The result is written back into memory.
Then the CPU starts again.
Millions of times every second.
Why CPUs Are So Good
Because they optimize for:
✔ Low latency
✔ Branch prediction
✔ Context switching
✔ Complex operating systems
✔ Database transactions
✔ Application execution
This is why Oracle Database, Linux, browsers and web servers primarily run on CPUs.
But Then Deep Learning Arrived...
Training a neural network isn't about making complicated decisions.
Instead, it performs the same mathematical operation...
again...
and again...
and again...
Imagine multiplying two huge matrices.
A × B
Not once.
Millions of times.
A CPU quickly becomes the bottleneck.
GPU — Designed for Massive Parallelism
A GPU wasn't originally built for AI.
It was built for graphics.
Rendering a 4K image means calculating millions of pixels simultaneously.
To solve this problem...
NVIDIA designed GPUs with thousands of smaller cores.
How GPU Architecture Works
Instead of a few powerful cores...
A GPU contains:
- Streaming Multiprocessors (SMs)
Each SM contains:
- CUDA cores
- Shared memory
- Registers
Multiple SMs share:
- L2 Cache
- High Bandwidth Memory (HBM)
Thousands of threads execute simultaneously.
Imagine multiplying a matrix.
CPU:
1 ↓ 2 ↓ 3
GPU:
1 2 3 4 5 6 7 ... All together
This is called SIMD/SIMT parallelism.
Why GPUs Changed AI
Almost every deep learning computation eventually becomes:
Matrix Multiplication
Examples:
- Convolution
- Attention
- Embeddings
- Transformers
GPUs excel at these operations.
This is why companies like OpenAI use thousands of NVIDIA GPUs for training.
Then Even GPUs Became a Limitation...
Although GPUs are excellent...
They still contain logic designed for graphics.
Google asked:
What if we removed everything unnecessary...
...and built hardware only for tensors?
TPU — Built Specifically for AI
TPU stands for:
Tensor Processing Unit
Unlike CPUs and GPUs...
A TPU is designed almost entirely around matrix multiplication.
Inside a TPU
Instead of CUDA cores...
A TPU contains:
- Systolic Array
- Multiply-Accumulate (MAC) Units
- Weight Buffer
- Activation Buffer
- On-chip SRAM
- High Bandwidth Memory
Rather than instructions moving around...
Data flows continuously through the array.
Think of it like an automobile assembly line.
Each station performs one operation.
The output immediately moves to the next station.
This dramatically reduces memory movement.
And memory movement is often slower than computation itself.
Why TPUs Are So Fast
Because neural networks mostly perform:
Tensor × Tensor ↓ Matrix Multiplication ↓ Activation ↓ Repeat
TPUs optimize exactly this workload.
Nothing more.
Nothing less.
Python Example
TensorFlow automatically uses available GPUs.
import tensorflow as tf print(tf.config.list_physical_devices("GPU"))
If a compatible GPU exists...
TensorFlow offloads tensor operations automatically.
Quick Comparison
| CPU | GPU | TPU |
|---|---|---|
| Few powerful cores | Thousands of smaller cores | Thousands of MAC units |
| Sequential | Parallel | Tensor optimized |
| Operating systems | Graphics & AI | Large-scale AI |
| Low latency | High throughput | Maximum AI efficiency |
So... What Happens When You Ask ChatGPT Something?
Prompt ↓ CPU (Tokenization, Networking, Scheduling) ↓ GPU (Matrix multiplication, Transformer inference) ↓ Generated Response
For Google Gemini:
Prompt ↓ TPU ↓ Tensor Operations ↓ Response
Final Thoughts
When I first learned about CPUs, GPUs, and TPUs, I thought the difference was simply "more cores."
But the real difference lies in how they are architected to solve problems.
- CPUs are designed for decision making.
- GPUs are designed for parallel computation.
- TPUs are designed for tensor computation.
Understanding this also explains why modern AI became possible.
It wasn't just because algorithms improved.
It was because hardware evolved alongside them.
