🧠 Why Can't We Train ChatGPT on a CPU? Understanding CPU vs GPU vs TPU

 

Introduction

Every time we hear about ChatGPT, Gemini, Claude, or any modern AI model, another term appears alongside it:

GPU.

Sometimes we even hear about Google TPUs.

This made me wonder:

If my laptop already has a CPU, why do companies spend millions of dollars buying GPUs and TPUs?

The answer lies in how these processors are designed.

Although all three process information, they solve problems very differently.

Let's understand why.

---

Imagine Three Engineers...

Suppose a company receives 10,000 customer invoices that need to be verified.

Three engineers volunteer.

👨 Engineer 1 (CPU)

He is extremely intelligent.

He checks one invoice carefully.

Finishes.

Moves to the next.

His strength isn't speed.

His strength is decision making.

---

👨‍👨‍👨 Engineer 2 (GPU)

Instead of working alone...

He brings 5,000 assistants.

Each assistant verifies one invoice.

The work finishes much faster.

---

🏭 Engineer 3 (TPU)

Google hires a specialist.

Instead of hiring more people...

They build an assembly line where invoices continuously move through dedicated stations.

Each station performs exactly one mathematical operation.

No unnecessary decisions.

Only maximum throughput.

---

That is essentially the difference between CPU, GPU and TPU.

Now let's see what happens internally.

---

🧠 CPU — Built for Decisions, Not Repetition

A CPU (Central Processing Unit) is designed to execute many different kinds of instructions efficiently.

Internally it consists of:

• Control Unit (CU)
• Arithmetic Logic Unit (ALU)
• Multiple CPU Cores
• Cache hierarchy (L1 → L2 → L3)
• Registers
• Main Memory Interface

---

How a CPU Executes an Instruction

Every instruction follows a cycle.

Instruction

↓

Fetch

↓

Decode

↓

Execute

↓

Store

Let's understand each stage.

1. Fetch

The Control Unit fetches an instruction from RAM.

Example:

ADD A, B

---

2. Decode

The CPU determines:

• What operation?
• Which registers?
• Which memory locations?

---

3. Execute

The ALU performs:

• arithmetic
• logical comparison
• branching

---

4. Store

The result is written back into memory.

Then the CPU starts again.

Millions of times every second.

---

Why CPUs Are So Good

Because they optimize for:

✔ Low latency

✔ Branch prediction

✔ Context switching

✔ Complex operating systems

✔ Database transactions

✔ Application execution

This is why Oracle Database, Linux, browsers and web servers primarily run on CPUs.

---

But Then Deep Learning Arrived...

Training a neural network isn't about making complicated decisions.

Instead, it performs the same mathematical operation...

again...

and again...

and again...

Imagine multiplying two huge matrices.

A × B

Not once.

Millions of times.

A CPU quickly becomes the bottleneck.

---

GPU — Designed for Massive Parallelism

A GPU wasn't originally built for AI.

It was built for graphics.

Rendering a 4K image means calculating millions of pixels simultaneously.

To solve this problem...

NVIDIA designed GPUs with thousands of smaller cores.

---

How GPU Architecture Works

Instead of a few powerful cores...

A GPU contains:

• Streaming Multiprocessors (SMs)

Each SM contains:

• CUDA cores
• Shared memory
• Registers

Multiple SMs share:

• L2 Cache
• High Bandwidth Memory (HBM)

Thousands of threads execute simultaneously.

---

Imagine multiplying a matrix.

CPU:

1
↓

2

↓

3

GPU:

1 2 3 4 5 6 7 ...

All together

This is called SIMD/SIMT parallelism.

---

Why GPUs Changed AI

Almost every deep learning computation eventually becomes:

Matrix Multiplication

Examples:

• Convolution
• Attention
• Embeddings
• Transformers

GPUs excel at these operations.

This is why companies like OpenAI use thousands of NVIDIA GPUs for training.

---

Then Even GPUs Became a Limitation...

Although GPUs are excellent...

They still contain logic designed for graphics.

Google asked:

What if we removed everything unnecessary...

...and built hardware only for tensors?

---

TPU — Built Specifically for AI

TPU stands for:

Tensor Processing Unit

Unlike CPUs and GPUs...

A TPU is designed almost entirely around matrix multiplication.

---

Inside a TPU

Instead of CUDA cores...

A TPU contains:

• Systolic Array
• Multiply-Accumulate (MAC) Units
• Weight Buffer
• Activation Buffer
• On-chip SRAM
• High Bandwidth Memory

Rather than instructions moving around...

Data flows continuously through the array.

Think of it like an automobile assembly line.

Each station performs one operation.

The output immediately moves to the next station.

This dramatically reduces memory movement.

And memory movement is often slower than computation itself.

---

Why TPUs Are So Fast

Because neural networks mostly perform:

Tensor × Tensor

↓

Matrix Multiplication

↓

Activation

↓

Repeat

TPUs optimize exactly this workload.

Nothing more.

Nothing less.

---

Python Example

TensorFlow automatically uses available GPUs.

import tensorflow as tf

print(tf.config.list_physical_devices("GPU"))

If a compatible GPU exists...

TensorFlow offloads tensor operations automatically.

---

Quick Comparison

CPU	GPU	TPU
Few powerful cores	Thousands of smaller cores	Thousands of MAC units
Sequential	Parallel	Tensor optimized
Operating systems	Graphics & AI	Large-scale AI
Low latency	High throughput	Maximum AI efficiency

---

So... What Happens When You Ask ChatGPT Something?

Prompt

↓

CPU
(Tokenization,
Networking,
Scheduling)

↓

GPU
(Matrix multiplication,
Transformer inference)

↓

Generated Response

For Google Gemini:

Prompt

↓

TPU

↓

Tensor Operations

↓

Response

---

Final Thoughts

When I first learned about CPUs, GPUs, and TPUs, I thought the difference was simply "more cores."

But the real difference lies in how they are architected to solve problems.

• CPUs are designed for decision making.
• GPUs are designed for parallel computation.
• TPUs are designed for tensor computation.

Understanding this also explains why modern AI became possible.

It wasn't just because algorithms improved.

It was because hardware evolved alongside them.

🤖 GPT vs Gemini: A Practical Comparison of the Latest AI Models

 With rapid advances in generative AI, choosing the "best" model is no longer about benchmarks alone.

It’s about context length, reasoning style, multimodality, ecosystem fit, and cost.

In this blog, I compare the latest GPT and Gemini models from a practical, system-level perspective — not marketing claims.

---

🧠 Latest Models at a Glance

🔹 OpenAI – GPT-5.2

GPT-5.2 is OpenAI’s current flagship model, optimized for:

• Structured reasoning

• Agentic workflows

• Coding and analytical tasks

• Enterprise and developer use cases

It is widely integrated across:

• ChatGPT

• Microsoft Copilot

• OpenAI APIs

• Third-party platforms

---

🔹 Google – Gemini 3

Gemini 3 is Google’s most advanced multimodal model, designed for:

• Very large context understanding

• Native multimodal reasoning

• Deep integration with Google Search and Workspace

Variants include:

• Gemini 3 Pro

• Gemini 3 Pro DeepThink

• Gemini 3 Flash (fast and cost-efficient)

---

🔍 Core Capability Comparison

Area	GPT-5.2	Gemini 3
Reasoning & logic	Strong structured reasoning	Strong long-context reasoning
Context window	Large	Extremely large (up to ~1M tokens)
Multimodal support	Text + image + tools	Text + image + video + audio
Coding workflows	Excellent step-by-step logic	Good, especially visual explanations
Enterprise readiness	Mature APIs & tooling	Deep Google ecosystem integration
Agent frameworks	Strong (agents, tools, planning)	Growing (task orchestration focus)

---

🧠 Reasoning Style: A Key Difference

One noticeable difference lies in how these models reason.

• GPT-5.2 excels at:

  • Step-by-step logical reasoning

  • Structured explanations

  • Tool-based and agentic workflows

• Gemini 3 shines when:

  • Handling long documents

  • Mixing modalities (text + image + video)

  • Working inside Google-native products

Neither is "smarter" in isolation — they are optimized for different problem spaces.

---

🧩 Multimodality & Context Handling

Gemini’s standout feature is its very large context window, making it ideal for:

• Long documents

• Large codebases

• Multi-file reasoning

• Video + text understanding

GPT-5.2, while supporting multimodality, focuses more on controlled reasoning and task execution than raw context length.

---

🛠️ Developer & Enterprise Perspective

From a system design viewpoint:

GPT-5.2 works best when:

• Building AI agents

• Designing RAG pipelines

• Creating structured workflows

• Integrating with enterprise tooling

Gemini 3 works best when:

• Operating within Google Cloud / Workspace

• Handling multimodal data at scale

• Performing search-heavy or document-heavy tasks

---

💰 Cost & Performance Considerations

In real deployments:

• Gemini Flash variants are optimized for speed and cost

• GPT-5.2 Pro prioritizes accuracy and reasoning depth

This reinforces a growing trend:

Model choice is becoming a cost–latency–accuracy tradeoff, not a leaderboard race.

---

🧠 The Bigger Insight: Models vs Systems

A key takeaway from comparing GPT and Gemini is this:

Strong AI applications are built by systems, not models alone.

The same task can succeed or fail depending on:

• Prompt design

• Retrieval strategy (RAG)

• Reasoning flow (CoT)

• Validation layers

• Cost controls

This is why understanding AI architecture matters more than memorizing model names.

---

🌱 Final Thoughts

GPT-5.2 and Gemini 3 represent two different philosophies:

• GPT → structured reasoning, tooling, workflows

• Gemini → multimodal understanding, long context, ecosystem depth

The right choice depends on what you are building, not which model trends on social media.

Explore related blogs

RAG Explained

Chain-of-Thought Reasoning

Prompt Engineering

AI Agents vs Agentic AI

🎯 Fine-Tuning vs In-Context Learning: Two Ways to Teach AI

When we think of “teaching AI,” most of us imagine feeding it massive datasets and retraining it from scratch.

But today’s Large Language Models (LLMs) can learn new tasks without retraining — simply by observing examples.

That difference lies between Fine-Tuning and In-Context Learning (ICL) — two distinct ways AI learns and adapts.

Let’s simplify both and understand when to use which.

🧠 Fine-Tuning: Traditional Model Training

Fine-tuning is like teaching an AI through long-term memory.
You take a pre-trained model (like GPT or Llama), add new labeled examples, and retrain it so it absorbs new knowledge permanently.

Example:
If you want an AI to analyze customer complaints in your company’s tone and format, you’d fine-tune it on your existing chat logs and desired outputs.

What happens internally:

• The model’s internal parameters are adjusted.

• It learns patterns specific to your data.

• The new behavior becomes part of its memory.

🧾 Advantages:
✅ High accuracy for domain-specific tasks
✅ Model “remembers” the skill permanently
✅ Works offline — no need for external context

⚠️ Limitations:
❌ Expensive and time-consuming
❌ Needs a large, labeled dataset
❌ Harder to update frequently

---

⚙️ In-Context Learning: The Modern Shortcut

In-Context Learning (ICL) is like teaching AI through short-term memory.
Instead of retraining, you show examples directly within the prompt — and the model adapts instantly for that session.

Example:
You tell the AI:

“Here are two examples of email replies.
Now, write one more in the same style.”

The model doesn’t modify its parameters — it just learns from context and imitates the pattern temporarily.

What happens internally:

• The examples are embedded in the model’s working memory.

• It predicts new text based on patterns in those examples.

• Once the session ends, the model “forgets” them.

🧾 Advantages:
✅ No retraining needed
✅ Very flexible and quick
✅ Works well for personalization and prototyping

⚠️ Limitations:
❌ Not persistent — forgets after session
❌ Limited by prompt size
❌ May misinterpret poorly structured examples

---

🔍 Key Differences at a Glance

Feature	Fine-Tuning	In-Context Learning
Learning Type	Long-term (parameter update)	Short-term (context-based)
Data Requirement	Large labeled dataset	Few examples in prompt
Speed	Slow	Fast
Cost	High	Low
Persistence	Permanent	Temporary
Best For	Domain adaptation, specialization	Quick task customization, demos

---

📘 Real-World Use Cases

Use Case	Best Method	Why
Customer support chatbots	Fine-tuning	Needs consistent tone and responses
Email writing assistance	In-context	Each prompt changes style dynamically
Legal or medical AI tools	Fine-tuning	Requires domain accuracy
AI writing assistants	In-context	Learns tone/style per session

---

💬 How These Methods Complement Each Other

You don’t always have to choose one.
A powerful setup often uses both:

• Fine-tune a base model for your domain (e.g., healthcare).

• Then use in-context learning to personalize it (e.g., specific doctor’s writing style).

That’s how modern AI systems combine long-term learning and short-term adaptability.

---

🌱 Final Thoughts

Fine-Tuning teaches AI what to know.
In-Context Learning teaches AI how to adapt.

One builds deep expertise; the other builds flexibility.
Together, they make AI not just intelligent — but adaptive and responsive to real-world needs.

🌟 Prompt Engineering: The Art of Talking to AI Like a Pro

In my recent blog on AI hallucinations, I wrote about how AI sometimes makes up facts when it doesn’t understand context properly.
But have you ever wondered why that happens?

Most of the time — it’s not the AI’s fault. It’s because of how we talk to it.
That’s where Prompt Engineering comes in — the skill of asking the right question, in the right way, to get the right answer.

Think of it like giving directions to a cab driver.
If you say “take me somewhere nice,” you’ll end up anywhere.
But if you say “take me to the beach near Marine Drive,” you’ll reach exactly where you want to go.

That’s exactly what prompt engineering is all about.

---

🧠 What Exactly Is Prompt Engineering?

Prompt engineering means designing inputs (prompts) that guide AI systems like ChatGPT, Gemini, or Llama to generate accurate, relevant, and useful responses.

AI models don’t “think” like humans — they predict.
They predict the next word based on the previous ones, using patterns learned from massive amounts of data.
So, the more specific and structured your input, the better the AI can predict your desired outcome.

Example 👇
❌ Bad Prompt: “Tell me about data.”
✅ Good Prompt: “Explain data preprocessing in machine learning with simple examples like removing null values and scaling features.”

The difference?
The second one gives context, role, and clarity — three key ingredients for a perfect prompt.

---

🧩 The Core Principles of Effective Prompting

Here’s a framework that works like magic — especially when you’re working with LLMs or AI tools daily:

1. Clarity: Be specific. Tell the AI what you want, what format you expect, and how long it should be.

2. Context: Provide background info. For example — who the audience is, what the tone should be, or if it’s for a blog, report, or code output.

3. Format: Mention output format — “in table form,” “bullet points,” “Python code,” etc.

4. Iteration: Don’t expect perfection in one go. Refine, rephrase, and guide.

5. Role-based prompting: Tell the AI who it should be.

   Example: “You are a Data Science professor. Explain neural networks to beginners using real-life analogies.”

   
   
   

    

---

🧮 Types of Prompts (with Examples)

Type	Purpose	Example
Instruction Prompt	Direct command	“Summarize this blog in 3 bullet points.”
Role-based Prompt	Assign a role	“You’re a cloud architect explaining OCI networking.”
Chain of Thought Prompt	Step-by-step reasoning	“Explain your reasoning step by step before answering.”
Zero-shot Prompt	No examples	“Translate this paragraph into French.”
Few-shot Prompt	Uses examples	“Here are 3 Q&A examples. Now answer the 4th one similarly.”

---

⚠️ Common Prompting Mistakes (and How to Avoid Them)

Even experienced users make these errors:

• Using vague or broad instructions.

• Asking multiple unrelated questions in one go.

• Forgetting to define tone or target audience.

• Not testing the prompt before using it in a workflow.

• Assuming AI understands context without being told.

A good way to avoid these is to think like an AI — imagine you have no background information except what’s in the prompt.
If you remove that context, will the answer still make sense?

---

🤖 Why Prompt Engineering Matters

Here’s why this skill is quickly becoming essential — not just for data scientists, but for everyone working with AI:

• It helps reduce hallucinations (when AI makes things up).

• It improves factual accuracy and context relevance.

• It saves time by reducing rework.

• It’s a foundation skill for Agentic AI, Retrieval-Augmented Generation (RAG), and custom LLM apps.

In short — good prompts = smarter AI.

---

💡 My Takeaway

After learning about this during my Data Science degree and experimenting daily with AI tools, I realized — prompt engineering isn’t just about writing better commands.
It’s a new kind of communication — a bridge between humans and machines.

If we can master how to talk to AI, we can make it understand us better.

Liked this post? Read my previous one on ‘Hallucinations in LLMs: Why AI Sometimes Makes Things Up’ — to understand why prompt quality matters even more. 

🤖 Expert Systems: The First Wave of Artificial Intelligence

When people think of AI today, they imagine chatbots, self-driving cars, or generative models like ChatGPT. But decades before all this, Expert Systems were the first real attempt at making machines “think” like humans.

🔍 What is an Expert System?

An Expert System is a computer program designed to mimic the decision-making ability of a human expert in a specific domain.

• It doesn’t just store facts.

• It applies rules and logic to those facts to solve problems — almost like consulting a virtual expert.

Think of it as the Google Maps of the 1970s AI world: you gave it a problem, and it tried to guide you to the solution.

---

⚙️ How Expert Systems Work

Expert Systems typically have three main components:

1. Knowledge Base 🧠

   • A collection of facts and rules.

   • Example: “If fever + cough → Possible flu.”

2. Inference Engine 🔗

   • The “reasoning brain” that applies the rules to known facts and derives conclusions.

3. User Interface 🖥️

• Allows the human user to interact, ask questions, and receive advice.

---

🌟 Real-World Examples of Expert Systems

• MYCIN (1970s) – Diagnosed bacterial infections and recommended antibiotics.

• DENDRAL – Helped chemists identify molecular structures.

• CLIPS – Used in NASA projects for decision-making.

• Modern echoes – Many medical diagnostic tools and troubleshooting apps still use expert-system logic.

                  

---

✅ Advantages of Expert Systems

• Store and preserve expert knowledge.

• Work 24/7 without fatigue.

• Useful in highly specialized fields (medicine, engineering, troubleshooting).

---

❌ Limitations of Expert Systems

• Very domain-specific (good only in one field).

• Rigid: can’t learn new things without manual updates.

• Struggle with uncertainty, creativity, and “common sense.”

                   

---

🚀 Why Expert Systems Still Matter

Even though modern AI (like Machine Learning and Deep Learning) has largely replaced Expert Systems, they laid the foundation for:

• Rule-based reasoning

• Knowledge representation

• Human–computer interaction

In a way, today’s AI assistants combine the best of both: the logical rules of Expert Systems and the learning power of Machine Learning.

                

---

✨ Conclusion
Expert Systems remind us that AI’s journey didn’t start with neural networks or ChatGPT. It began with the humble dream of capturing human expertise in code — a dream that still inspires AI research today.

Expert Systems laid foundation for many AI advancements. To understand the broader field of AI that evolved from here, read my post on Artificial Intelligence Explained