AI Engineering Course - A free and complete AI Engineering Course to learn AI Engineering step by step - from Machine Learning, Neural Networks, and Transformers to LLMs, Fine-Tuning, RAG, AI Agents, LLM Inference, Evaluation, AI Safety, and AI System Design.

 ![AI Engineering Course - Learn AI Engineering step by step](https://github.com/amitshekhariitbhu/ai-engineering-course/blob/main/assets/banner.png)

AI Engineering Course

This AI Engineering Course is helpful for anyone who wants to become:

AI Engineer

Gen AI Engineer

LLM Engineer

Agentic AI Engineer

AI Agent Engineer

Forward Deployed Engineer

AI Solutions Architect

AI Platform Engineer

Applied AI Engineer

Machine Learning Engineer

MLOps Engineer

LLMOps Engineer

Prepared and maintained by the Founder of Outcome School: Amit Shekhar

Follow Amit Shekhar

Follow Outcome School

I teach at Outcome School

This AI Engineering Course is completely free to read. If you want more practical and in-depth learning with me in live classes, I also teach a paid live program at Outcome School:

AI and Machine Learning (Live Classes)

Note: This AI Engineering Course will continue to grow as I write more blogs and create more videos on new topics. Keep learning.

About This AI Engineering Course
What is AI Engineering?
Who is This AI Engineering Course For?
What Will We Learn in This AI Engineering Course?
Prerequisites for This AI Engineering Course
How to Use This AI Engineering Course
AI Engineering Course Curriculum at a Glance
AI Engineering Learning Path
Module 0: Must Know
Module 1: Machine Learning Foundations
Module 2: Deep Learning and Neural Networks
Module 3: Generative AI and the Transformer Architecture
Module 4: How LLMs Generate Text
Module 5: Modern LLM Architecture
Module 6: Types of Language Models
Module 7: Training, Fine-Tuning, and Alignment
Module 8: Prompt Engineering and Context Engineering
Module 9: Vector Search and Retrieval-Augmented Generation (RAG)
Module 10: AI Agents and Agentic Systems
Module 11: Agentic Engineering and Agent Frameworks
Module 12: LLM Inference Engineering
Module 13: Evaluation and Observability
Module 14: AI Safety and Security
Module 15: Multimodal AI and Generative Models
Module 16: AI Infrastructure, Deployment, and System Design
Module 17: Frontier Ideas in AI
Module 18: Prepare for AI Engineering Interviews
AI Engineering Key Concepts Glossary
AI Engineering Course FAQs
License

About This AI Engineering Course

This AI Engineering Course is a free, structured, and step-by-step curriculum to learn AI Engineering from scratch. It has 18 modules and 146+ in-depth lessons, and every lesson is a detailed blog that explains one concept in simple words with examples, diagrams, and math wherever it is needed.

In simple words, this is the course that I wish I had when I started learning AI Engineering. We start with the basics of Machine Learning, and slowly move to how a Transformer works from the inside, how an LLM generates text, how we fine-tune and align an LLM, how we build RAG systems and AI Agents, how we serve an LLM fast and cheap in production, how we evaluate and secure it, and finally how we design a complete AI system end to end.

Every lesson in this AI Engineering Course is:

Free to read. No sign-up. No paywall.
Written for beginners. No jargon. No assumptions.
Detailed. We go from "why do we need it" to "how does it work step by step" to "where is it used in the real world".
In order. Each lesson builds on top of the previous one.

What is AI Engineering?

AI Engineering is the discipline of building real-world applications and systems on top of AI models, especially Large Language Models (LLMs).

An AI Engineer does not always train a model from scratch. Instead, an AI Engineer knows how these models work from the inside, picks the right model, gives it the right context, connects it to tools and data, makes it fast and cheap to run, measures if it is doing a good job, keeps it safe, and ships it to real users.

In simple words:

AI Engineering = Understanding the model + Building on top of the model + Running the model in production.

This is exactly what we will learn in this AI Engineering Course.

Who is This AI Engineering Course For?

This AI Engineering Course is for:

Software Engineers who want to move into AI Engineering.
Backend, Mobile, and Frontend Developers who want to build AI-powered products.
Machine Learning Engineers and Data Scientists who want to go deep into LLMs, RAG, and AI Agents.
Students and freshers who want to start a career in AI.
Engineering Managers and Tech Leads who want to understand how modern AI systems are built.
Anyone preparing for AI Engineer interviews.

What Will We Learn in This AI Engineering Course?

In this AI Engineering Course, we will learn:

Machine Learning foundations: supervised, unsupervised, and reinforcement learning, loss functions, regularization, precision and recall.
Deep Learning: gradient descent, backpropagation, cross-entropy, dropout, normalization, RNNs, PyTorch, and TensorFlow.
The Transformer architecture: tokenization, BPE, embeddings, self attention, the math behind Q, K, and V, multi-head attention, causal masking, RoPE, and the feed-forward network.
How LLMs generate text: temperature, top-k and top-p sampling, token streaming, and the lost in the middle problem.
Modern LLM architecture: Mixture of Experts (MoE), Grouped Query Attention (GQA), sliding window attention, attention sinks, Flash Attention, and DeepSeek-V4.
Types of language models: SLMs, Large Reasoning Models, Recursive Language Models, Diffusion Language Models, and System One Models.
Fine-tuning and alignment: fine-tuning, LoRA, prefix tuning, knowledge distillation, continual learning, RLHF, InstructGPT, PPO, DPO, and GRPO.
Prompt Engineering and Context Engineering: chain-of-thought, tree of thoughts, prompt chaining, prompt caching, and context compaction.
RAG and Vector Search: vector databases, ANN search, semantic search, hybrid search, rerankers, ColBERT, chunking, Contextual Retrieval, HyDE, query rewriting, multi-query retrieval, caching, Agentic RAG, GraphRAG, and Vectorless RAG.
AI Agents: function calling, the agent loop, ReAct, Plan-and-Execute, Reflection, memory, MCP, Agent Skills, multi-agent systems, SubAgents, orchestration, computer-use agents, and browser agents.
Agentic Engineering: harness engineering, loop engineering, graph engineering, LangChain, LangGraph, Claude Code, and Cursor.
LLM Inference Engineering: prefill vs decode, KV cache, paged attention, continuous batching, speculative decoding, Medusa, EAGLE, quantization, GGUF, llama.cpp, Ollama, vLLM, SGLang, and TensorRT-LLM.
Evaluation and Observability: LLM evaluation, LLM as a judge, AI agent evaluation, and agent observability.
AI Safety and Security: guardrails, prompt injection, and watermarking.
Multimodal AI and Generative Models: Vision Transformers, image embeddings, diffusion models, GANs, and VAEs.
AI Infrastructure and System Design: GPUs, CUDA kernels, TPUs, LPUs, cloud vs on-device deployment, LLM routing, and designing a real-time voice AI agent.
Frontier Ideas in AI: JEPA, world models, and recursive self-improvement.
AI Engineering interview preparation.

Prerequisites for This AI Engineering Course

Basic programming knowledge, preferably Python. Most code examples in the lessons are in Python.
High-school level math. We explain the linear algebra, calculus, and probability we need right inside the lessons, step by step.
Curiosity. That's all.

No prior background in AI or Machine Learning is required.

How to Use This AI Engineering Course

This AI Engineering Course is designed so that anyone can follow it in the given order, even without any prior background in AI.

Follow the modules in order. Each module builds on top of the previous one.
Inside each module, read the lessons in the given order. Every lesson explains one concept in simple words with examples.
Do not skip Module 1 and Module 2, even if you are in a hurry. Everything else in AI Engineering is built on top of them.
If you already know a topic, read the "We will cover the following" list of that lesson. If you can explain every point, move to the next one.
After every module, try to explain the concepts to a friend in your own words. If you can explain it, you have learned it.

Let's get started.

AI Engineering Course Curriculum at a Glance

Module	Topic	Lessons
0	Must Know	1
1	Machine Learning Foundations	9
2	Deep Learning and Neural Networks	10
3	Generative AI and the Transformer Architecture	15
4	How LLMs Generate Text	5
5	Modern LLM Architecture	7
6	Types of Language Models	5
7	Training, Fine-Tuning, and Alignment	11
8	Prompt Engineering and Context Engineering	6
9	Vector Search and Retrieval-Augmented Generation (RAG)	15
10	AI Agents and Agentic Systems	16
11	Agentic Engineering and Agent Frameworks	8
12	LLM Inference Engineering	17
13	Evaluation and Observability	4
14	AI Safety and Security	3
15	Multimodal AI and Generative Models	6
16	AI Infrastructure, Deployment, and System Design	10
17	Frontier Ideas in AI	3
18	Prepare for AI Engineering Interviews	1

AI Engineering Learning Path

flowchart TD
    A[Machine Learning Foundations] --> B[Deep Learning and Neural Networks]
    B --> C[Generative AI and Transformers]
    C --> D[How LLMs Generate Text]
    D --> E[Modern LLM Architecture]
    E --> F[Types of Language Models]
    F --> G[Training, Fine-Tuning, and Alignment]
    G --> H[Prompt and Context Engineering]
    H --> I[RAG and Vector Search]
    I --> J[AI Agents and Agentic Systems]
    J --> K[Agentic Engineering and Frameworks]
    K --> L[LLM Inference Engineering]
    L --> M[Evaluation and Observability]
    M --> N[AI Safety and Security]
    N --> O[Multimodal AI and Generative Models]
    O --> P[AI Infrastructure and System Design]
    P --> Q[Frontier Ideas in AI]
    Q --> R[AI Engineering Interviews]

Module 0: Must Know

Before jumping into the details, we must know the six words that come up in every AI Engineering conversation:

LLM
RAG
MCP
Agent
Fine-tuning
Quantization

Learn about all six in one video: AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, Quantization

Now, we know the big picture. In the next modules, we will learn each of these in depth, one concept at a time.

Module 1: Machine Learning Foundations

In this module, we will learn what Machine Learning is, the different ways a machine can learn, and the basic terms we will keep using in every later module of this AI Engineering Course.

By the end of this module, we will know how a model learns from data, how we measure it, and how we stop it from overfitting.

Lessons in this module:

1.1 What is Machine Learning?

In this blog, we will learn what is Machine Learning.

Let's get started: What is Machine Learning?

1.2 Supervised vs Unsupervised Learning

In this blog, we will learn about Supervised vs Unsupervised Learning in Machine Learning.

We will cover the following:

Supervised Learning
Unsupervised Learning
Differences Between Supervised and Unsupervised Learning

Let's get started: Supervised vs Unsupervised Learning

1.3 Linear Regression vs Logistic Regression

In this blog, we will learn about Linear Regression vs Logistic Regression in Machine Learning.

We will cover the following:

Linear Regression
Logistic Regression
Differences Between Linear Regression and Logistic Regression

Let's get started: Linear Regression vs Logistic Regression

1.4 What is Feature Engineering in Machine Learning?

In this blog, we will learn about the Feature Engineering for Machine Learning.

Let's get started: What is Feature Engineering in Machine Learning?

Watch the video: Feature Engineering in Machine Learning

Watch the video: One-hot Encoding in Machine Learning

1.5 Precision vs Recall

In this blog, we will learn about Precision vs Recall, the two numbers we use to measure how good a system is at making yes-or-no decisions. We will also see how Precision and Recall differ from each other and when to use which one, what the four possible outcomes of any such decision are, why these two numbers pull against each other, and how the cost of a mistake decides which one we must care about more.

We will cover the following:

The problem we are trying to solve
The four possible outcomes
What is Precision?
What is Recall?
Precision vs Recall
When to use which one?
A quick recap of the formulas
Summary

Let's get started: Precision vs Recall

1.6 What Are L1 and L2 Loss Functions?

In this blog, we will learn about the L1 and L2 Loss functions.

We will cover the following:

L1 Loss Function
L2 Loss Function
How to decide between L1 and L2 Loss Function?

Let's get started: What Are L1 and L2 Loss Functions?

1.7 What is Regularization in Machine Learning? L1 vs L2 Explained

In this blog, we will learn about the Regularization In Machine Learning.

We will cover the following:

What is overfitting?
L1 Regularization or Lasso Regularization
L2 Regularization or Ridge Regularization

Let's get started: What is Regularization in Machine Learning? L1 vs L2 Explained

1.8 What is Reinforcement Learning?

In this blog, we will learn about Reinforcement Learning, the branch of machine learning where an agent learns to make decisions by interacting with an environment and getting rewards or penalties for its actions.

We will cover the following:

The Big Picture
What is Reinforcement Learning?
A Simple Real-World Analogy
The Building Blocks of RL
The Reinforcement Learning Loop
Reinforcement Learning vs Supervised vs Unsupervised Learning
Episode, Return, and Discount Factor
Exploration vs Exploitation
Common Families of RL Algorithms
Where Is Reinforcement Learning Used?
Why Reinforcement Learning Is Hard
Quick Summary

Let's get started: What is Reinforcement Learning?

1.9 What is Contrastive Learning? How It Works Step by Step

In this blog, we will learn about Contrastive Learning. We will also see how it works step-by-step and where it is used in the real world.

We will cover the following:

What is Contrastive Learning?
Why do we need Contrastive Learning?
The key idea behind Contrastive Learning.
Positive pairs and Negative pairs.
How does Contrastive Learning work step-by-step?
Loss functions used in Contrastive Learning.
Popular Contrastive Learning methods.
Real-world use cases of Contrastive Learning.

Let's get started: What is Contrastive Learning? How It Works Step by Step

Videos and more resources for Module 1:

Feature Engineering in Machine Learning (Video)
One-hot Encoding in Machine Learning (Video)

Module 2: Deep Learning and Neural Networks

In this module, we will learn how a neural network actually learns. We will understand the math behind gradient descent and backpropagation step by step, and the techniques that make training stable.

By the end of this module, we will be able to explain how a neural network trains, from the forward pass to the weight update, and why normalization and dropout matter.

Lessons in this module:

2.1 What is Bias In Artificial Neural Network?

In this blog, we will learn what is Bias In Artificial Neural Network.

Let's get started: What is Bias In Artificial Neural Network?

2.2 How Does Gradient Descent Work?

In this blog, we will learn about the math behind gradient descent with a step-by-step numeric example.

We will cover the following:

The Big Picture
What is a Loss Function
What is Gradient Descent
The Intuition Behind Gradient Descent
The Math Behind Gradient Descent
Step-by-Step Numeric Example
Gradient Descent with Multiple Parameters
The Role of Learning Rate
Types of Gradient Descent
Gradient Descent in Python
Putting It All Together

Let's get started: How Does Gradient Descent Work?

Watch the video: Epoch, Batch, Batch Size, Iteration

2.3 How Does Backpropagation Work? The Math Explained Step by Step

In this blog, we will learn about the math behind backpropagation in neural networks.

We will cover the following:

What is Backpropagation?
The Chain Rule of Calculus
Forward Pass
Loss Calculation
Backward Pass (Backpropagation)
Step-by-Step Numeric Example
Weight Update Using Gradient Descent
Backpropagation in Python

Let's get started: How Does Backpropagation Work? The Math Explained Step by Step

2.4 What is Cross-Entropy Loss?

In this blog, we will learn about the math behind Cross-Entropy Loss with a step-by-step numeric example.

We will cover the following:

The Big Picture
What is Cross-Entropy
The Cross-Entropy Loss Formula
Why We Take the Negative Log
Binary Cross-Entropy Loss
Categorical Cross-Entropy Loss
Step-by-Step Numeric Example
Cross-Entropy Loss for Language Models
The Gradient of Cross-Entropy Loss
Quick Summary

Let's get started: What is Cross-Entropy Loss?

Watch the video: Softmax Activation Function in Machine Learning

2.5 What is Dropout in Neural Networks and How Does It Work?

In this blog, we will learn about Dropout in Neural Networks. We will understand what it is, the problem it solves, how it works step by step with a simple example, and where it is used.

We will cover the following:

What is Dropout?
The problem of Overfitting
Why do we need Dropout?
How does Dropout work?
A step-by-step example
Dropout during training vs testing
Dropout in code
Variants of Dropout
Advantages of Dropout
Where Dropout is used

Let's get started: What is Dropout in Neural Networks and How Does It Work?

2.6 Batch Normalization vs Layer Normalization

In this blog, we are going to learn about Batch Normalization vs Layer Normalization. We will also see how Batch Normalization and Layer Normalization differ from each other and when to use which one.

We will cover the following:

What is Normalization?
Why do we need Normalization?
What is Batch Normalization?
What is Layer Normalization?
Batch Normalization vs Layer Normalization
When to use which one?

Let's get started: Batch Normalization vs Layer Normalization

2.7 What is RMSNorm? Root Mean Square Layer Normalization Explained

In this blog, we will learn about RMSNorm, a faster and simpler alternative to Layer Normalization that powers most modern Large Language Models like Llama, Mistral, Gemma, Qwen, PaLM, and DeepSeek.

We will cover the following:

Why normalization is needed in deep networks
A quick recap of Layer Normalization (LayerNorm)
What RMSNorm is and how it works
The math behind RMSNorm with a concrete numeric example
LayerNorm vs RMSNorm - the key differences
Why modern LLMs prefer RMSNorm
A code example
Where RMSNorm fits in a Transformer
Quick Summary

Let's get started: What is RMSNorm? Root Mean Square Layer Normalization Explained

2.8 What is a Recurrent Neural Network (RNN)?

In this blog, we will learn about the Recurrent Neural Network.

Let's get started: What is a Recurrent Neural Network (RNN)?

2.9 How does PyTorch work?

In this blog, we will learn about how PyTorch works. We will also see what a tensor is, how the computation graph and autograd work together to train a model, and why PyTorch uses the GPU to become one of the most popular tools in the real world.

We will cover the following:

What is PyTorch?
What is a Tensor?
The problem PyTorch solves
What is a Computation Graph?
What is Autograd?
A complete training example
What is the GPU and why does PyTorch use it?
Why is PyTorch so popular?

Let's get started: How does PyTorch work?

2.10 How Does The Machine Learning Library TensorFlow Work?

In this blog, we will learn how the Machine Learning library TensorFlow works.

Let's get started: How Does The Machine Learning Library TensorFlow Work?

Videos and more resources for Module 2:

Epoch, Batch, Batch Size, Iteration (Video)

Note: We have now learned the foundations of Machine Learning and Deep Learning. This course is free. If you want to learn these foundations in a more practical and in-depth way with me in live classes, check out our paid live program: AI and Machine Learning at Outcome School.

Module 3: Generative AI and the Transformer Architecture

In this module, we will learn what Generative AI is and how the Transformer, the architecture behind every modern LLM, works from the inside. We will go from tokens to embeddings to attention, one piece at a time.

By the end of this module, we will be able to draw the Transformer from memory and explain every block inside it, including the math behind Q, K, and V.

Lessons in this module:

3.1 What is Generative AI?

In this blog, we will learn about what Generative AI is. We will understand what the word "generate" actually means, how Generative AI is different from the older AI, how it learns from a huge amount of examples, how it creates something completely new, where we use it every day, and the limitations we must know.

We will cover the following:

What is Generative AI?
Generative AI = Generative + AI
What does "generate" mean here?
How is Generative AI different from the old AI?
How does Generative AI learn?
How does Generative AI actually create something new?
What can Generative AI create?
What is a model in Generative AI?
The complete flow of Generative AI
Where do we use Generative AI every day?
The limitations we must know
Summary

Let's get started: What is Generative AI?

3.2 What are Autoregressive Models?

In this blog, we will learn about Autoregressive Models, the family of models that generate one piece at a time by predicting the next step from the past.

We will cover the following:

What is an Autoregressive Model?
The Chain Rule of Probability
The Generation Loop
Step-by-Step Numeric Example
Why GPT-style Models are Autoregressive
Why Autoregressive Models Need Causal Masking
The Connection with KV Cache
Autoregressive vs Non-Autoregressive Generation
Popular Autoregressive Models we should know
Pros and Cons of Autoregressive Models
Quick Summary

Let's get started: What are Autoregressive Models?

3.3 Tokenization in LLMs

In this blog, we will learn about Tokenization in LLMs, the very first step that turns our text into numbers before a Large Language Model can process it. We will also see what a token is, the three approaches to break text into tokens, how BPE works, how tokens are decoded back to text, and how tokenization affects LLMs in practice.

We will cover the following:

What is Tokenization?
Why do we need Tokenization?
What is a Token?
Approach 1: Character-level Tokenization
Approach 2: Word-level Tokenization
Approach 3: Subword-level Tokenization
How does Byte Pair Encoding (BPE) work?
Vocabulary and Token IDs
Decoding: From Tokens back to Text
Tokenization in action with code
Special Tokens
How Tokenization affects LLMs in practice
Where it works well and where it fails

Let's get started: Tokenization in LLMs

Watch the video: Tokenization in Large Language Models (LLMs)

3.4 What is Byte Pair Encoding (BPE) in LLMs?

In this blog, we will learn about BPE (Byte Pair Encoding) - the tokenization algorithm used by most modern Large Language Models (LLMs) to break text into smaller pieces before processing it.

We will cover the following:

What is Tokenization?
The Problem: How to Break Text into Tokens?
What is BPE (Byte Pair Encoding)?
How BPE Works: Step by Step
How BPE Tokenizes New Text
Why BPE is Used in Modern LLMs

Let's get started: What is Byte Pair Encoding (BPE) in LLMs?

3.5 What are Embeddings?

In this blog, we will learn about Embeddings, one of the most important ideas behind modern AI like search engines, recommendations, and chatbots. We will also see why a computer cannot compare meaning on its own, how an embedding turns meaning into numbers so that similar things sit close together, how we measure that closeness, and where we use embeddings in the real world.

We will cover the following:

The problem: computers cannot compare meaning
What are Embeddings?
A simple example with two numbers
Why we need many more than two numbers
How do we measure closeness?
Where do embeddings come from?
The famous word math example
Embeddings are not only for words
What we use embeddings for
Things we must be careful about
Summary

Let's get started: What are Embeddings?

Watch the video: Embeddings in Machine Learning

3.6 How do RNNs and Transformers differ?

In this blog, we will learn about how RNNs and Transformers differ. These are the two main ways a computer reads a sequence like a sentence, and we will learn why one of them reads word by word, why the other reads everything at once, and when to use which one.

We will cover the following:

What both of them are for
What is an RNN?
The problem with an RNN
What is a Transformer?
The key difference in one line
RNN vs Transformer side by side
Let's tabulate the difference
When to use which one?
Summary

Let's get started: How do RNNs and Transformers differ?

3.7 How Does the Transformer Architecture Work?

In this blog, we will learn about the Transformer architecture by decoding it piece by piece - understanding what each component does, how they work together, and why this architecture powers every modern Large Language Model (LLM).

We will cover the following:

Why the Transformer was needed
The two halves of the architecture
Tokenization, Embedding, and Positional Encoding
The Attention Mechanism and Multi-Head Attention
Feed-Forward Networks, Residual Connections, and Layer Normalization
How the Encoder and Decoder work
How data flows through the entire architecture
The three variants of the Transformer
Why the Transformer is so powerful

Let's get started: How Does the Transformer Architecture Work?

3.8 Encoder vs Decoder in Transformers

In this blog, we will learn about Encoder vs Decoder in Transformers, the two building blocks behind almost every modern AI model that works with language. We will also see how the Encoder and the Decoder differ from each other, how each one works with simple examples, why one of them reads in both directions while the other one looks only backward, what the three types of Transformers are, and when to use which one.

We will cover the following:

What is a Transformer?
A word about tokens
What is an Encoder?
What is a Decoder?
The one big difference
The three types of Transformers
Let's tabulate the difference
When to use which one?
Summary

Let's get started: Encoder vs Decoder in Transformers

3.9 What is Self Attention in Transformers and How Does It Work?

In this blog, we will learn about Self Attention in Transformers. We will understand what it is, how it works step by step, and why it is the heart of modern Large Language Models like BERT and GPT.

We will cover the following:

What is Self Attention?
Why do we need Self Attention?
Query, Key, and Value vectors
Step-by-step working of Self Attention
A simple example walk-through
Why Self Attention works so well
Multi-Head Self Attention
Where Self Attention is used

Let's get started: What is Self Attention in Transformers and How Does It Work?

3.10 How Does Attention Work? The Math Behind Q, K, and V Step by Step

In this blog, we will learn about the math behind Attention: Query(Q), Key(K), and Value(V) with a step-by-step numeric example.

We will cover the following:

The Attention Formula
Setting Up: From Words to Vectors
Creating Q, K, and V Matrices
Computing Attention Scores (Q x K^T)
Scaling the Scores
Applying Softmax
Computing the Final Output (Attention Weights x V)
Putting It All Together

Let's get started: How Does Attention Work? The Math Behind Q, K, and V Step by Step

Watch the video: Softmax Activation Function in Machine Learning

3.11 Why Do We Scale Attention by √dₖ? The Math Behind the Scaling Factor

In this blog, we will learn about why we scale the dot product attention by √dₖ in the Transformer architecture with a step-by-step numeric example.

We will cover the following:

The Attention Formula (Quick Recap)
What Happens Without Scaling?
Why Do Dot Products Grow with dₖ?
Understanding Variance of the Dot Product
Proving It Step by Step: Variance of the Dot Product is dₖ
What Large Dot Products Do to Softmax
Why √dₖ is the Right Scaling Factor
Seeing It with Real Numbers
Putting It All Together

Let's get started: Why Do We Scale Attention by √dₖ? The Math Behind the Scaling Factor

3.12 What is Causal Masking in Attention and Why Do LLMs Need It?

In this blog, we will learn about causal masking in attention.

We will cover the following:

Without Causal Masking
With Causal Masking
Implementation of Causal Masking
The Causal Mask Matrix

Let's get started: What is Causal Masking in Attention and Why Do LLMs Need It?

3.13 What is Multi-Head Attention in Transformers and How Does It Work?

In this blog, we will learn about Multi-Head Attention in Transformers. We will understand what it is, how it works step by step, and why it gives Transformers their power to understand language so well.

We will cover the following:

What is Multi-Head Attention?
A quick recap of Self Attention
Why do we need Multi-Head Attention?
Step-by-step working of Multi-Head Attention
A simple example walk-through
Where Multi-Head Attention is used
Advantages of Multi-Head Attention

Let's get started: What is Multi-Head Attention in Transformers and How Does It Work?

3.14 What is Cross Attention in Transformers and How Does It Work?

In this blog, we will learn about Cross Attention in Transformers. We will understand what it is, how it works step by step, how it is different from Self Attention, and where it is used.

We will cover the following:

What is Cross Attention?
Why do we need Cross Attention?
Query, Key, and Value in Cross Attention
Self Attention vs Cross Attention
Step-by-step working of Cross Attention
A simple example walk-through
Where Cross Attention is used
Importance of Cross Attention

Let's get started: What is Cross Attention in Transformers and How Does It Work?

3.15 What is RoPE (Rotary Position Embedding)? The Math Behind It

In this blog, we will learn about the math behind Rotary Position Embedding (RoPE) and why it is used in modern Large Language Models.

We will cover the following:

The Big Picture
Why a Transformer Needs Position Information
Older Approaches and Their Problems
The Core Idea Behind RoPE
The 2D Rotation Math
How RoPE Is Applied to Q and K
Why the Dot Product Captures Relative Position
A Small Numeric Example
Real-World Use Cases
Quick Summary

Let's get started: What is RoPE (Rotary Position Embedding)? The Math Behind It

3.16 What is the Feed-Forward Network in LLMs and What Does It Do?

In this blog, we will learn about Feed-Forward Networks in LLMs - understanding what they are, how they work inside the Transformer architecture, why every Transformer layer needs one, and what role they play in making Large Language Models so powerful.

We will cover the following:

What is a Feed-Forward Network?
Understanding Feed-Forward Networks with a Real-World Analogy
Where Does the Feed-Forward Network Sit in a Transformer?
How Does a Feed-Forward Network Work - Step by Step
The Expand-then-Contract Pattern
Why Does the FFN Expand and Then Contract?
ReLU and Activation Functions
What Does the Feed-Forward Network Actually Learn?
How Much of the Model is the Feed-Forward Network?
Feed-Forward Networks in Mixture of Experts
Why Feed-Forward Networks Are So Important

Let's get started: What is the Feed-Forward Network in LLMs and What Does It Do?

Videos and more resources for Module 3:

Softmax Activation Function in Machine Learning (Video)
Inside ChatGPT: What Happens After You Hit Enter (Read)
Tokenization in Large Language Models (LLMs) (Video)
Embeddings in Machine Learning (Video)
Positional Embeddings in LLMs (Read)

Module 4: How LLMs Generate Text

In this module, we will learn how an LLM picks the next token, how we control its creativity, how it knows when to stop, how the output reaches the user token by token, and where the context window fails.

By the end of this module, we will know exactly what happens between the prompt and the final answer, and which knobs change the output.

Lessons in this module:

4.1 How does Temperature control LLM output?

In this blog, we will learn about how Temperature controls LLM output, the single number that decides whether an AI model gives us a safe and predictable answer or a creative and surprising one. We will also see how an LLM picks one token at a time, how it gives a score to every possible next token, how those scores become probabilities, how Temperature quietly changes those probabilities before the pick happens, what happens at low, high, and zero Temperature, why it is called Temperature, and when to use which value based on our use case.

We will cover the following:

What is Temperature in LLMs?
How does an LLM pick the next token?
From scores to probabilities
Where does Temperature come into the picture?
Step-by-step example with numbers
Low Temperature
High Temperature
Temperature = 1 and Temperature = 0
Why is it called Temperature?
When to use which Temperature?
Common mistakes while using Temperature

Let's get started: How does Temperature control LLM output?

4.2 How do Top-k and Top-p Sampling work?

In this blog, we will learn about how Top-k and Top-p Sampling work, the two most common ways an LLM decides which word to write next when it is replying to us. We will also see how an LLM picks one token at a time, why always picking the best token gives boring text, why picking from every token gives silly text, how Top-k keeps a fixed number of tokens, how Top-p keeps tokens based on their total probability, how both of them work with temperature, and when to use which one.

We will cover the following:

How an LLM picks the next token
The problem with always picking the best token
The problem with picking from every token
What is Top-k Sampling?
Step-by-step example of Top-k Sampling
The problem with Top-k Sampling
What is Top-p Sampling?
Step-by-step example of Top-p Sampling
Top-k vs Top-p Sampling
How Top-k and Top-p work with Temperature
When to use which one

Let's get started: How do Top-k and Top-p Sampling work?

4.3 Stop Tokens in LLMs

In this blog, we will learn about Stop Tokens in LLMs, the special tokens that tell a Large Language Model when to stop generating text. We will also see how an LLM generates text one token at a time, why it needs to stop, how the model learns to produce a Stop Token, how a Stop Token differs from a Stop Sequence, and how chat models use Stop Tokens to end their turn.

We will cover the following:

What is a Token?
How does an LLM generate text?
Why does an LLM need to stop?
What is a Stop Token?
How does the model learn to produce a Stop Token?
Stop Token vs Stop Sequence
What happens if the model never stops?
Stop Tokens in Chat Models
Common mistakes with Stop Tokens

Let's get started: Stop Tokens in LLMs

4.4 How does Token Streaming work?

In this blog, we will learn about how Token Streaming works. We will also see why we need it, how the server and the browser talk to each other to make it happen, and where it is used in real systems like ChatGPT and Claude.

We will cover the following:

What is token streaming
A quick recap of how an LLM generates text
Why we need streaming at all
What is SSE
How the HTTP connection stays open
The format of a streamed message
A full walkthrough from server to screen
The [DONE] marker that ends the stream
SSE vs WebSockets
Token streaming in the real world

Let's get started: How does Token Streaming work?

4.5 What is the Lost in the Middle Problem in LLMs and How to Fix It?

In this blog, we will learn about the Lost in the Middle problem in LLMs, the strange behaviour where a model reads a very long text, uses the beginning and the end very well, and quietly ignores whatever is sitting in the middle. We will also see what a context window really means, how the accuracy forms a U-shaped curve, why the middle gets forgotten, how this silently breaks RAG systems and long conversations, how we can test our own model for it, and what we can do to fix it.

We will cover the following:

What is a context window
What is the Lost in the Middle problem
Let's understand it with an example
The U-shaped curve
Why does this happen
Where this hurts us in real life
How to test for the Lost in the Middle problem
How to solve the Lost in the Middle problem
Key points to remember

Let's get started: What is the Lost in the Middle Problem in LLMs and How to Fix It?

Watch the video: Why is the context window limited in LLMs?

Videos and more resources for Module 4:

Why is the context window limited in LLMs? (Video)

Note: We now know how LLMs generate text from the inside. If you want to go more practical and in-depth on these topics with me in live classes, check out our paid live program: AI and Machine Learning at Outcome School.

Module 5: Modern LLM Architecture

In this module, we will learn the improvements that modern LLMs add on top of the basic Transformer to become bigger, faster, and able to handle longer inputs. At the end, we will see all of these ideas together inside a real model.

By the end of this module, we will be able to read the architecture section of any new open-weight model and understand every design choice in it.

Lessons in this module:

5.1 Evolution of LLM Architecture

In this blog, we will learn about the Evolution of LLM Architecture, the step-by-step journey of how the design of large language models changed from simple word-by-word readers to the massive AI models we use today. We will also see why the early models kept forgetting, how attention solved that problem, how the Transformer removed the slow parts, how making models bigger made them smarter, how Mixture of Experts made big models cheaper to run, what problems still remain, and what is coming next.

We will cover the following:

What is an LLM Architecture?
Stage 1: Reading one word at a time (RNN)
Stage 2: Attention
Stage 3: The Transformer
Stage 4: Scaling
Stage 5: Mixture of Experts (MoE)
Stage 6: New Directions
Summary of the evolution

Let's get started: Evolution of LLM Architecture

5.2 What is Mixture of Experts (MoE) and How Does It Work?

In this blog, we will learn about the Mixture of Experts (MoE) architecture - understanding what experts are, how the router picks them, why MoE makes large models faster and cheaper, and why it powers many of today's most powerful Large Language Models (LLMs).

We will cover the following:

Why Mixture of Experts was needed
What an "expert" really means
The router and how it picks experts
Where MoE sits inside a Transformer
Sparse activation and why it saves compute
Load balancing across experts
Advantages and challenges of MoE
Why MoE powers many modern LLMs

Let's get started: What is Mixture of Experts (MoE) and How Does It Work?

5.3 What is Grouped Query Attention (GQA) and Why Do LLMs Use It?

In this blog, we will learn about Grouped-Query Attention (GQA) and how it differs from Multi-Head Attention (MHA).

We will cover the following:

The Big Picture
Quick Recap: Multi-Head Attention (MHA)
The Problem with Multi-Head Attention
What is Multi-Query Attention (MQA)?
What is Grouped-Query Attention (GQA)?
How Grouped-Query Attention Works
GQA is a Generalization of MHA and MQA
GQA vs MHA vs MQA
Real-World Use Cases
A Note on Terminology
Uptraining: Converting MHA to GQA
Quick Summary

Let's get started: What is Grouped Query Attention (GQA) and Why Do LLMs Use It?

5.4 How does Sliding Window Attention work?

In this blog, we will learn about how sliding window attention works. We will also see why normal attention becomes slow and expensive for long text, and how sliding window attention comes to the rescue.

We will cover the following:

What is attention?
The problem with normal attention
What is sliding window attention?
A simple step-by-step walkthrough
How information still travels far away
Comparing normal attention and sliding window attention
Where sliding window attention is used
Advantages and trade-offs

Let's get started: How does Sliding Window Attention work?

5.5 How do Attention Sinks work?

In this blog, we will learn about how attention sinks work in Large Language Models. We will also see why streaming long conversations becomes a problem, why the first tokens become a sink that the naive fix breaks, and how StreamingLLM uses attention sinks in the real world to keep models running.

We will cover the following:

What is a Large Language Model
What is attention
The problem of streaming with long conversations
The naive fix and why it fails
What is an attention sink
Why the first tokens become a sink
A step-by-step numeric walkthrough
The fix in code
StreamingLLM and modern attention sinks
Importance of attention sinks

Let's get started: How do Attention Sinks work?

5.6 What is Flash Attention and Why Is It So Fast?

In this blog, we will learn about Flash Attention by decoding it piece by piece - understanding why standard attention is slow, what makes Flash Attention fast, how it uses GPU memory cleverly, and why it is used in almost every modern Large Language Model (LLM).

We will cover the following:

A quick recap of standard attention
Why standard attention is slow
How GPU memory actually works (HBM vs SRAM)
The core idea behind Flash Attention
Tiling: breaking the work into small blocks
Online softmax: computing softmax without the full matrix
Recomputation in the backward pass
Flash Attention 2
Flash Attention 3
Advantages and impact of Flash Attention

Let's get started: What is Flash Attention and Why Is It So Fast?

5.7 What is DeepSeek-V4 and How Does It Work? Architecture Explained

In this blog, we will learn about DeepSeek-V4, the new family of open Mixture-of-Experts language models that natively supports a one-million-token context with dramatically lower inference cost.

We will cover the following:

The Big Picture
Two Models: DeepSeek-V4-Pro and DeepSeek-V4-Flash
Hybrid Attention with CSA and HCA
Manifold-Constrained Hyper-Connections (mHC)
Muon Optimizer
FP4 Quantization-Aware Training
Pre-Training
Post-Training: Specialist Training and On-Policy Distillation
Reasoning Modes
Putting It All Together
Quick Summary

Let's get started: What is DeepSeek-V4 and How Does It Work? Architecture Explained

Module 6: Types of Language Models

In this module, we will learn that not every language model is a large, text-generating LLM. We will see the smaller, reasoning, recursive, diffusion-based, and decision-only models and when to use which one.

By the end of this module, we will know which type of model to pick for a given problem.

Lessons in this module:

6.1 What are Small Language Models (SLMs) and When Should We Use Them?

In this blog, we will learn about Small Language Models (SLMs), what counts as small, why they matter, where they shine, and the trade-offs we must keep in mind.

We will cover the following:

SLM = Small + Language Model
What is a Language Model?
What Counts as "Small"?
Popular SLMs we should know
How SLMs Stay Capable Despite Being Small
Why SLMs Matter
SLM vs LLM
The Size Spectrum
Where SLMs Shine - Use Cases
Trade-offs of SLMs
When to Pick an SLM
Quick Summary

Let's get started: What are Small Language Models (SLMs) and When Should We Use Them?

6.2 What are Large Reasoning Models (LRMs) and How Are They Different from LLMs?

In this blog, we will learn about Large Reasoning Models (LRMs), how they are different from standard Large Language Models, how they think before they answer, how they are trained, and when we must use them.

We will cover the following:

The Big Picture
What is a Large Reasoning Model (LRM)?
LLM vs LRM
How does an LRM actually think?
Test-time compute: thinking longer makes them smarter
How are LRMs trained?
Input and Output: training phase vs prediction phase
When to use an LRM, and when to use a regular LLM
Popular LRMs we should know
Common Mistakes when using LRMs
Quick Summary

Let's get started: What are Large Reasoning Models (LRMs) and How Are They Different from LLMs?

6.3 What are Recursive Language Models (RLMs) and How Do They Work?

In this blog, we will learn about Recursive Language Models (RLMs), a new way of using language models to handle very large inputs that do not fit in the model's context window.

We will cover the following:

What is a Recursive Language Model (RLM)?
Why do we need RLMs?
How an RLM works
How the model writes and runs code
Why RLMs work better
Recursion inside RLMs
How RLMs differ from simple chunking
Advantages of RLMs
Limitations of RLMs
When to use RLMs
RLM vs RAG
A real use case

Let's get started: What are Recursive Language Models (RLMs) and How Do They Work?

6.4 How do Diffusion Language Models (DLMs) work?

In this blog, we will learn about Diffusion Language Models (DLMs), a new way to make models write text. They promise to generate words in a different way than the LLMs we use today, and that too much faster in many cases.

We will cover the following:

What is a Diffusion Language Model?
How do today's language models write text?
The problem with the usual approach
Where the diffusion idea comes from
What does "noise" mean for text?
The two phases: forward and reverse
How a DLM actually generates text, step by step
A tiny end-to-end example
A simple code-style walk-through
DLMs vs the usual language models
Advantages of DLMs
Limitations of DLMs
The current state

Let's get started: How do Diffusion Language Models (DLMs) work?

6.5 Jev and System One Models Explained

In this blog, we will learn about Jev and System One Models, a new kind of AI model that does not write text at all but only makes fast decisions that our software can use directly. We will also see what System One and System Two thinking mean, why using a normal LLM for small decisions is slow and unreliable, how Jev answers many questions in a single pass instead of one word at a time, what a typed answer is, how the probability attached to every answer is made honest, why Jev cannot hallucinate, and where it works well and where it fails.

We will cover the following:

What is a System One Model?
System One vs System Two Thinking
The Problem with Using an LLM for Decisions
What is Jev?
How Jev Works
Typed Answers: Choice, Score, and Yes/No
Calibration and RLCD
Why Jev Cannot Hallucinate
Jev vs LLM
Where Jev Works Well and Where It Fails
When to Use Which One

Let's get started: Jev and System One Models Explained

Module 7: Training, Fine-Tuning, and Alignment

In this module, we will learn how a pre-trained model is adapted to our own task, how it is made smaller, and how it is taught to follow instructions and human preferences.

By the end of this module, we will know when to fine-tune, how LoRA makes it cheap, and how RLHF, PPO, DPO, and GRPO align a model.

Lessons in this module:

7.1 How does fine-tuning work?

In this blog, we will learn about how Fine-tuning works. We will also see why we need it, how it works step by step with a simple example, how full fine-tuning and LoRA differ and when to use which one based on our use case.

We will cover the following:

What is Fine-tuning?
Why do we need Fine-tuning?
How does Fine-tuning work step by step?
A simple worked example with numbers
Full Fine-tuning vs LoRA
When to use Fine-tuning
Tips before we Fine-tune
Summary

Let's get started: How does fine-tuning work?

7.2 What is LoRA (Low-Rank Adaptation) and How Does It Fine-Tune LLMs?

In this blog, we will learn about LoRA - Low-Rank Adaptation of Large Language Models.

We will cover the following:

The Big Picture
Why Full Fine-Tuning Is Expensive
The Core Idea Behind LoRA
How LoRA Works Step by Step
A Small Numeric Example
Where LoRA Is Applied in a Transformer
Merging LoRA Back Into the Model
Real-World Use Cases
Quick Summary

Let's get started: What is LoRA (Low-Rank Adaptation) and How Does It Fine-Tune LLMs?

7.3 How does Prefix Tuning work?

In this blog, we will learn about Prefix Tuning, a cheap way to adapt a large language model to a new task without changing the model itself. It saves memory, saves money, and lets one big model serve many different tasks at once. We will also see why full fine-tuning is so expensive, how the prefix is added and trained without changing the model, how Prefix Tuning differs from full fine-tuning and prompt tuning and when to use which one based on our use case, where it falls short and how it compares with LoRA, and where it is used in the real world.

We will cover the following:

What is a large language model?
The problem: why full fine-tuning is expensive
What is Prefix Tuning?
Prefix Tuning = Prefix + Tuning
How does Prefix Tuning work?
The prefix is not real words
Where the prefix is added
How the prefix is trained
How small is the prefix really?
A simple code example
Prefix Tuning vs Full Fine-Tuning
Prefix Tuning vs Prompt Tuning
Advantages of Prefix Tuning
Limitations of Prefix Tuning
Where Prefix Tuning is used

Let's get started: How does Prefix Tuning work?

7.4 How does Knowledge Distillation work?

In this blog, we will learn about how Knowledge Distillation works. We will also see why we need it, how a small model learns from a big model, and how this lets us run powerful AI on a phone, on an edge device, and at low cost.

We will cover the following:

What is Knowledge Distillation?
Why we need Knowledge Distillation
Hard labels vs soft labels
Dark knowledge
Temperature in the softmax
The distillation loss
A step-by-step training walkthrough
Types of Knowledge Distillation
Real examples of Knowledge Distillation
Wrapping up Knowledge Distillation

Let's get started: How does Knowledge Distillation work?

7.5 What is Continual Learning in LLMs? Solving Catastrophic Forgetting

In this blog, we will learn about Continual Learning in LLMs. We will understand what it is, why we need it, the big problem of catastrophic forgetting, the approaches used to solve it, and where it is used in the real world.

We will cover the following:

What is Continual Learning?
Why do we need Continual Learning in LLMs?
The big problem: Catastrophic Forgetting
Approaches to Continual Learning in LLMs
Challenges in Continual Learning
Real-world use cases

Let's get started: What is Continual Learning in LLMs? Solving Catastrophic Forgetting

7.6 What is Deep RL from Human Preferences? The Paper That Started RLHF

In this blog, we are going to learn about Deep Reinforcement Learning from Human Preferences, the 2017 paper that started it all. It taught machines what we want by simply asking us to pick which of two behaviors looks better. This is the origin of RLHF, the technique behind ChatGPT.

We will cover the following:

The building blocks we must know first
The big picture: what the paper does
Why it was needed: the reward problem
Trajectory segments, or clips
The human comparison
The reward model and the preference math
Training the reward model
Training the agent with reinforcement learning
The loop and the smart bits
The results: the backflip and beyond
The legacy: this is RLHF
What this looks like today
Quick Summary

Let's get started: What is Deep RL from Human Preferences? The Paper That Started RLHF

7.7 What is InstructGPT? How GPT-3 Learned to Follow Instructions

In this blog, we are going to learn about InstructGPT, the model that taught GPT-3 to actually follow our instructions, and the work that led directly to ChatGPT.

We will cover the following:

What is the InstructGPT paper?
The building blocks we must know first
The big picture: what InstructGPT does
Why GPT-3 was not enough
Helpful, Honest, and Harmless
The three-step method
Step 1: Supervised Fine-Tuning
Step 2: The Reward Model
Step 3: Reinforcement Learning with PPO
The alignment tax
The results
What alignment looks like today
Quick Summary

Let's get started: What is InstructGPT? How GPT-3 Learned to Follow Instructions

7.8 What is RLHF? Reinforcement Learning from Human Feedback Explained

In this blog, we will learn about Reinforcement Learning from Human Feedback (RLHF), the training technique that turns a raw pre-trained LLM into a helpful, honest, and safe assistant by teaching it from human preferences.

We will cover the following:

What is RLHF
Why we need RLHF
The Big Picture
Stage 1: Supervised Fine-Tuning (SFT)
Stage 2: Training the Reward Model
Stage 3: RL Fine-Tuning with PPO
The KL Penalty
Putting It All Together
Reward Hacking
Common Mistakes
Best Practices
Quick Summary

Let's get started: What is RLHF? Reinforcement Learning from Human Feedback Explained

7.9 What is Proximal Policy Optimization (PPO) and How Does It Work?

In this blog, we are going to learn about Proximal Policy Optimization (PPO). We will also see how PPO works step-by-step and how it is used in training Large Language Models with RLHF.

We will cover the following:

What is Reinforcement Learning?
What is a Policy?
The problem with simple policy updates.
What is Proximal Policy Optimization (PPO)?
The key idea behind PPO: Clipping.
The PPO objective function in simple words.
How PPO works step-by-step.
PPO in Large Language Models (RLHF).
Advantages of PPO.
Disadvantages of PPO.

Let's get started: What is Proximal Policy Optimization (PPO) and How Does It Work?

7.10 What is Direct Preference Optimization (DPO) and How Does It Work?

In this blog, we are going to learn about Direct Preference Optimization (DPO). We will also see how DPO works step-by-step and how it differs from RLHF (PPO).

We will cover the following:

What is RLHF and why do we need it?
The problem with RLHF.
What is Direct Preference Optimization (DPO)?
What is preference data?
The key idea behind DPO.
The DPO loss function in simple words.
How DPO works step-by-step.
DPO vs RLHF (PPO).
Advantages of DPO.
Disadvantages of DPO.

Let's get started: What is Direct Preference Optimization (DPO) and How Does It Work?

7.11 What is Group Relative Policy Optimization (GRPO) and How Does It Work?

In this blog, we are going to learn about Group Relative Policy Optimization (GRPO). We will also see how GRPO works step-by-step and when to use it based on our use case.

We will cover the following:

What is GRPO?
Why do we need GRPO?
The problem with PPO.
How does GRPO work?
Step-by-step example.
The GRPO objective in simple words.
Advantages of GRPO.
Practical things to keep in mind.
When to use GRPO.
Conclusion.

Let's get started: What is Group Relative Policy Optimization (GRPO) and How Does It Work?

Note: We have now learned how LLMs are trained, fine-tuned, and aligned. If you want to learn these in a more practical and in-depth way with me in live classes, check out our paid live program: AI and Machine Learning at Outcome School.

Module 8: Prompt Engineering and Context Engineering

In this module, we will learn how to talk to an LLM so that it gives better answers, and how to manage everything that goes into its context window.

By the end of this module, we will be able to design prompts and contexts that make an LLM reliable, fast, and cheap.

Lessons in this module:

8.1 How does Chain-of-Thought (CoT) Prompting work?

In this blog, we will learn about how Chain-of-Thought (CoT) Prompting works. We will also see why a model that jumps straight to the answer often gets it wrong, how making it reason step by step fixes this, the difference between zero-shot and few-shot CoT, and where this technique is truly useful.

We will cover the following:

What is a prompt?
What is an LLM?
The problem: when the model jumps straight to the answer
What is Chain-of-Thought (CoT) Prompting?
A simple example without CoT and with CoT
Zero-shot CoT vs Few-shot CoT
A step-by-step walkthrough of a reasoning chain
Why does Chain-of-Thought Prompting work?
Where Chain-of-Thought Prompting is useful
Things to keep in mind

Let's get started: How does Chain-of-Thought (CoT) Prompting work?

8.2 How does Tree of Thoughts work?

In this blog, we will learn about how Tree of Thoughts works. We will also see the problem with the single straight path of Chain of Thought, how Tree of Thoughts lets the model explore many reasoning paths, judge them, and go back from a bad step, and where it works well and where it fails.

We will cover the following:

How does an LLM answer?
What is Chain of Thought?
The problem with Chain of Thought
What is Tree of Thoughts?
What is a thought?
Step 1: Breaking the problem into thoughts
Step 2: Generating many thoughts
Step 3: Judging the thoughts
Step 4: Searching through the tree
A complete example: the Game of 24
A simple code sketch of Tree of Thoughts
Chain of Thought vs Tree of Thoughts
Where Tree of Thoughts works well and where it fails

Let's get started: How does Tree of Thoughts work?

8.3 How does Prompt Chaining work?

In this blog, we will learn about how Prompt Chaining works. We will also see why we need it, how it works step by step by passing the output of one prompt into the next, and where it is used in the real world to solve bigger tasks reliably.

We will cover the following:

What is a prompt?
What is Prompt Chaining?
Why do we need Prompt Chaining?
How does Prompt Chaining work step by step?
A real example of Prompt Chaining
Code example of Prompt Chaining
Common patterns in Prompt Chaining
Advantages of Prompt Chaining
Things to take care of while using Prompt Chaining
When to use Prompt Chaining

Let's get started: How does Prompt Chaining work?

8.4 How does Prompt Caching work?

In this blog, we will learn about how Prompt Caching works. We will also see why we need it, how it actually works inside a large language model, and where it is used in real systems like AI assistants and agents.

We will cover the following:

What is a prompt
A quick recap of how an LLM reads a prompt
What is Prompt Caching
Why we need Prompt Caching
The core idea behind Prompt Caching
The exact-prefix rule
Cache write vs cache read and TTL
What we should put in the cache
The benefits of Prompt Caching
Prompt Caching in the real world

Let's get started: How does Prompt Caching work?

8.5 What is Context Engineering?

In this blog, we will learn about Context Engineering - what it is, why it has become the most important skill for building reliable AI applications, how it differs from Prompt Engineering, the components that make up the context, common patterns like RAG, few-shot examples, tools, and memory, and the best practices and common mistakes to keep in mind.

We will cover the following:

What is Context Engineering?
The Big Picture
Why Context Engineering matters
Prompt Engineering vs Context Engineering
The components of the context
Common patterns in Context Engineering
Common mistakes to avoid
Best practices
Quick Summary

Let's get started: What is Context Engineering?

8.6 How does context compaction work?

In this blog, we will learn about how context compaction works in Large Language Models. We will also see why long conversations overflow the context window, how summarization shrinks the older messages without losing the important points, and where compaction is used in real AI agents.

We will cover the following:

What is a Large Language Model
What is the context window
What is context
The problem of long conversations
The naive fix and why it fails
What is context compaction
How summarization powers compaction
A step-by-step walkthrough
Context compaction in code
Compaction in real AI agents
Why context compaction is important

Let's get started: How does context compaction work?

Module 9: Vector Search and Retrieval-Augmented Generation (RAG)

In this module, we will learn how to give an LLM knowledge that it was never trained on. We will start with how vectors are stored and searched, then move to retrieval techniques, and finally to the advanced forms of RAG.

By the end of this module, we will be able to build a production-grade RAG pipeline and pick the right retrieval technique for our data.

Lessons in this module:

9.1 How does a Vector Database work?

In this blog, we will learn about how a Vector Database works. This is one of the most important pieces behind modern AI search, recommendations, and tools like ChatGPT that answer questions from our own documents.

We will cover the following:

What is a Vector Database?
A quick recap of embeddings
Why normal databases fall short
What a Vector Database actually stores
How do we measure similarity?
Cosine similarity
Dot product
Euclidean distance
The nearest neighbour problem
Why brute force is too slow
Approximate Nearest Neighbour (ANN) and indexing
HNSW explained simply
IVF explained simply
PQ explained simply
A small code example
Real-world applications of Vector Databases

Let's get started: How does a Vector Database work?

9.2 How does Approximate Nearest Neighbor (ANN) search work?

In this blog, we will learn about Approximate Nearest Neighbor (ANN) Search, the idea that lets apps find "similar" things in a huge collection in the blink of an eye. It powers search engines, recommendation systems, face matching, and the memory behind modern AI chatbots. We will also see why the naive approach fails, how trees, hashing, clustering, and graphs make the search fast, and where ANN search is used in the real world.

We will cover the following:

What is Nearest Neighbor Search?
How do we turn things into numbers (vectors)?
How do we measure "closeness"?
The naive approach and why it fails
What is Approximate Nearest Neighbor (ANN) Search?
The trade-off: speed vs accuracy
Approach 1: Trees (KD-Tree)
Approach 2: Hashing (LSH)
Approach 3: Clustering (IVF)
Approach 4: Graphs (HNSW)
A simple code example
Where ANN Search is used
Picking the right method

Let's get started: How does Approximate Nearest Neighbor (ANN) search work?

9.3 How does Semantic Search work?

In this blog, we will learn about how Semantic Search works. We will also see why we need it, how it actually works step by step, and where it is used in real systems like search engines, AI assistants, and recommendations.

We will cover the following:

What is keyword search
Where keyword search fails
What is Semantic Search
What is an embedding
How similar meanings sit close together
How we measure closeness with cosine similarity
What is a vector database
The full Semantic Search flow
Approximate nearest neighbor for speed at scale
Semantic Search in the real world

Let's get started: How does Semantic Search work?

9.4 How does Hybrid Search work?

In this blog, we will learn about how Hybrid Search works. We will also see why we need it, the two kinds of search it combines, how their results are merged together, and where it is used in real systems like RAG.

We will cover the following:

What is keyword search
Why keyword search alone is not enough
What is semantic search
Why semantic search alone is not enough
What is Hybrid Search
How Hybrid Search runs both searches
How the two result lists are combined
Reciprocal Rank Fusion (RRF)
Weighted score combination and normalization
Hybrid Search in the real world

Let's get started: How does Hybrid Search work?

9.5 How does a Reranker work?

In this blog, we will learn about how a Reranker works. We will also see where it sits in a search and RAG pipeline, why we need it, and how it makes our answers more accurate.

We will cover the following:

What is a Reranker
Where a Reranker sits in a search / RAG pipeline
The two-stage retrieval idea
Why first-stage retrieval is fast but not precise
Bi-encoder vs Cross-encoder
How a Reranker scores documents step by step
The accuracy vs latency and cost trade-off
Late-interaction models like ColBERT
Real examples of Rerankers
Why Rerankers matter for RAG

Let's get started: How does a Reranker work?

9.6 What is ColBERT? Late Interaction Retrieval Explained

In this blog, we are going to learn about ColBERT, a retrieval method that keeps the fine-grained, word-by-word matching of a slow BERT reranker but makes it fast enough to search millions of passages, using a clever trick called late interaction.

We will cover the following:

What is the ColBERT paper?
The building blocks we must know first
The big picture: what ColBERT does
The two old extremes
Late interaction: the key idea
Encoding the query and document
The MaxSim operation
Why max, not average
Ranking documents
Training: positives and negatives
The loss with small numbers
Fast retrieval at scale
The cost of a bigger index
The Results
Where ColBERT led
Quick Summary

Let's get started: What is ColBERT? Late Interaction Retrieval Explained

9.7 How to Chunk Documents for RAG? Chunking Strategies Explained

In this blog, we will learn about Chunking Strategies for RAG, the art of cutting a big document into smaller pieces so that an AI system can find the right piece at the right time. We will also see what RAG is, why chunking is needed at all, what happens when we chunk badly, the most useful chunking strategies one by one, how to pick the chunk size and the overlap, and where each strategy works well and where it fails.

We will cover the following:

What is RAG?
What is a chunk?
Why do we need chunking?
How retrieval actually works
What happens when we chunk badly
Fixed-size chunking
Chunking by sentence
Recursive chunking
Document structure based chunking
Semantic chunking
Contextual chunking
Small-to-big chunking
Agentic chunking
Chunk overlap
How to choose the chunk size
Comparison of all the strategies
Common mistakes
Conclusion

Let's get started: How to Chunk Documents for RAG? Chunking Strategies Explained

9.8 How does Contextual Retrieval work?

In this blog, we will learn about how Contextual Retrieval works. It is a way to improve RAG by adding a short note to every chunk of a document before we store it. The note explains where the chunk comes from, so the system can find the right chunk even when the chunk alone does not say enough.

We will cover the following:

What is a chunk?
How does normal retrieval find chunks?
The problem: chunks lose their context
What is Contextual Retrieval?
How does Contextual Retrieval work step by step?
Contextual Embeddings
Contextual BM25
Combining both and adding reranking
How much does it help?
How is the cost kept low?
A simple code example
When to use it and when not to use it

Let's get started: How does Contextual Retrieval work?

9.9 How does HyDE work in RAG?

In this blog, we will learn about how HyDE works in RAG, which is the clever trick of searching with a fake answer. We will also see why searching with the plain question is weak, why a fake answer searches better, how HyDE works step by step with a worked example, and when to use it in the real world.

We will cover the following:

What is RAG in simple words
What is the search problem in RAG
Why searching with the question is weak
What is HyDE
Why searching with a fake answer works better
How HyDE works step by step
A worked example of HyDE
A simple code example of HyDE
Advantages of HyDE
Disadvantages of HyDE
When to use HyDE
Summary

Let's get started: How does HyDE work in RAG?

9.10 How do Query Rewriting and Multi-Query Retrieval work?

In this blog, we will learn about how Query Rewriting and Multi-Query Retrieval work, which are two simple ways to improve the search step of a RAG system. We will also see why the user's question is often not a good search query, how Reciprocal Rank Fusion (RRF) merges the results of many searches, and when to use which one.

We will cover the following:

Why the user's question is not a good search query
What is Query Rewriting
How Query Rewriting works
Types of Query Rewriting
Query Rewriting in code
The problem that Query Rewriting does not solve
What is Multi-Query Retrieval
How Multi-Query Retrieval works
How to merge the results with Reciprocal Rank Fusion (RRF)
Multi-Query Retrieval in code
Query Rewriting vs Multi-Query Retrieval
Where they work well and where they fail

Let's get started: How do Query Rewriting and Multi-Query Retrieval work?

9.11 How does an Embedding Cache work?

In this blog, we will learn about how an Embedding Cache works. We will also see what an embedding is, why an Embedding Cache saves us a lot of money and time, how the cache key is built, and where it is used in real systems like RAG and semantic search.

We will cover the following:

What is an embedding
A quick recap of how we get an embedding
What is an Embedding Cache
Why we need an Embedding Cache
The core idea behind an Embedding Cache
The cache key, a hash of the text plus the model
The request flow, a hit and a miss
Eviction, LRU and TTL
Where the cache lives, memory or disk
The benefits of an Embedding Cache
An Embedding Cache in the real world

Let's get started: How does an Embedding Cache work?

9.12 How does Semantic Caching work?

In this blog, we will learn about how Semantic Caching works. We will also see why traditional caching falls short for AI apps, how Semantic Caching uses embeddings and similarity to reuse past answers, and how setting the right threshold makes it work in the real world.

We will cover the following:

What is a cache?
The problem with traditional caching for AI apps
What is Semantic Caching?
What are embeddings?
What is similarity between embeddings?
How does Semantic Caching work step by step?
A numeric walkthrough
Setting the similarity threshold
Advantages of Semantic Caching
Things to keep in mind

Let's get started: How does Semantic Caching work?

9.13 What is Agentic RAG? How It Works and When to Use It

In this blog, we will learn about Agentic RAG - what it is, why standard RAG falls short, the agentic RAG loop, the three building blocks, the common patterns, when to use it, and the limitations to keep in mind.

We will cover the following:

The Big Picture
A Quick Recap of RAG
A Quick Recap of AI Agent
Why Standard RAG Falls Short
What is Agentic RAG
The Agentic RAG Loop
The Three Building Blocks
A Walkthrough with a Real Example
Common Patterns of Agentic RAG
Standard RAG vs Agentic RAG
When to Use Agentic RAG
Limitations of Agentic RAG
Quick Summary

Let's get started: What is Agentic RAG? How It Works and When to Use It

Watch the video: Agentic RAG Explained

9.14 What is GraphRAG? How Knowledge Graphs Improve RAG

In this blog, we will learn about GraphRAG and how it improves retrieval by using a knowledge graph along with vector search.

We will cover the following:

What is GraphRAG?
Why normal RAG is not enough
The big picture of GraphRAG
How GraphRAG builds the knowledge graph
How GraphRAG answers a question
Local search vs Global search
When to use GraphRAG
Trade-offs of GraphRAG
Quick Summary

Let's get started: What is GraphRAG? How Knowledge Graphs Improve RAG

9.15 What is Vectorless RAG? RAG Without Embeddings or a Vector Database

In this blog, we will learn about Vectorless RAG, a way of answering questions from our own documents without converting those documents into numbers and without using any vector database. We will also see how the normal RAG works, why the vector part creates problems, how a Vectorless RAG system reads a document the way a human reads a book, how the tree structure and the search happen step by step, what the advantages and disadvantages are, and when to use which one.

We will cover the following:

What is an LLM
What is RAG
How the normal Vector RAG works
Problems with Vector RAG
What is Vectorless RAG
How Vectorless RAG works
An example of Vectorless RAG
Other Vectorless approaches
Advantages of Vectorless RAG
Disadvantages of Vectorless RAG
Vector RAG vs Vectorless RAG
When to use which one

Let's get started: What is Vectorless RAG? RAG Without Embeddings or a Vector Database

Videos and more resources for Module 9:

AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, Quantization (Video)
Agentic RAG Explained (Video)

Note: We now know how to build RAG systems. If you want to build them hands-on with more practical and in-depth learning with me in live classes, check out our paid live program: AI and Machine Learning at Outcome School.

Module 10: AI Agents and Agentic Systems

In this module, we will learn how an LLM goes from answering questions to actually doing work. We will start with a single agent, see how it uses tools and memory, and then move to systems where many agents work together.

By the end of this module, we will be able to design a single agent, give it tools and memory, and scale it to a multi-agent system.

Lessons in this module:

10.1 What is an AI Agent? How It Works

In this blog, we will learn about the AI Agent - what it is, how it is different from a plain LLM, its five core parts, how it works end to end, the main types, and when to use one.

We will cover the following:

The Big Picture
What is an AI Agent
AI Agent vs Plain LLM vs Chatbot
The Five Core Parts
How an AI Agent Works End to End
A Concrete Example: Research Agent
Types of AI Agents
What AI Agents Can Do Today
When to Use an AI Agent
Common Failure Modes
Quick Summary

Let's get started: What is an AI Agent? How It Works

Watch the video: AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, Quantization

10.2 How does Function Calling work in LLMs?

In this blog, we will learn about how Function Calling works in LLMs. We will see what it is, why we need it, the key insight behind it, and how it powers AI agents and assistants step by step.

We will cover the following:

What is Function Calling
Why We Need Function Calling
The Key Insight: The Model Does Not Run the Function
How Function Calling Works Step by Step
A Concrete Example: get_weather(city)
The Conversation Loop
Multi-Step and Parallel Function Calling
Relation to Structured Outputs and JSON Mode
Real-World Use: The Backbone of AI Agents
Quick Summary

Let's get started: How does Function Calling work in LLMs?

10.3 What is an AI Agent Loop?

In this blog, we will learn about the AI Agent Loop - what it is, why an AI Agent needs it, the think-act-observe cycle that powers it, how the loop knows when to stop, and the common ways the loop fails.

We will cover the following:

The Big Picture
What is the AI Agent Loop
Why an AI Agent Needs a Loop
The Think-Act-Observe Cycle
The Loop Step by Step
The Loop in Real Code
Parallel Tool Calls in One Turn
How the Loop Knows When to Stop
Common Loop Failures
Quick Summary

Let's get started: What is an AI Agent Loop?

10.4 What is a ReAct Agent? How It Thinks and Acts, Explained

In this blog, we will learn about the ReAct Agent - what it is, how it is built, its anatomy, how it thinks and acts, and how to handle its common failure modes.

We will cover the following:

What is a ReAct Agent
ReAct Agent vs AI Agent
Anatomy of a ReAct Agent
The ReAct Prompt Template
How a ReAct Agent Thinks and Acts
A Full Trace Example
Implementing a ReAct Agent
Common Failure Modes and How to Fix Them
Quick Summary

Let's get started: What is a ReAct Agent? How It Thinks and Acts, Explained

10.5 What is a Plan-and-Execute Agent and How Does It Work?

In this blog, we will learn about the Plan-and-Execute Agent - what it is, its anatomy, how it plans and runs the steps, how it differs from a ReAct Agent, and how to handle its common failure modes.

We will cover the following:

What is a Plan-and-Execute Agent
Plan-and-Execute Agent vs AI Agent
Anatomy of a Plan-and-Execute Agent
How a Plan-and-Execute Agent Works
A Full Trace Example
Plan-and-Execute Agent vs ReAct Agent
Common Failure Modes and How to Fix Them
Quick Summary

Let's get started: What is a Plan-and-Execute Agent and How Does It Work?

10.6 What is a Reflection Agent? How It Generates, Critiques, and Revises

In this blog, we will learn about the Reflection Agent - what it is, how it is built, its anatomy, how it generates, critiques, and revises its own work, and how to handle its common failure modes.

We will cover the following:

What is a Reflection Agent
Reflection Agent vs AI Agent
Anatomy of a Reflection Agent
How a Reflection Agent Works
A Full Trace Example
Reflection Agent vs ReAct Agent
Common Failure Modes and How to Fix Them
Quick Summary

Let's get started: What is a Reflection Agent? How It Generates, Critiques, and Revises

10.7 How Does AI Agent Memory Work? The Memory Stack Explained

In this blog, we will learn about AI Agent Memory - why agents need it, the memory stack, the four core operations (write, read, update, forget), how memory flows at runtime, and the common mistakes.

We will cover the following:

The Big Picture
Why AI Agents Need Memory
The Memory Stack
The Four Core Operations
How Memory Flows at Runtime
What to Store and What Not to Store
Common Mistakes and How to Fix Them
Quick Summary

Let's get started: How Does AI Agent Memory Work? The Memory Stack Explained

10.8 How does ChatGPT remember you?

In this blog, we will learn how ChatGPT remembers us across chats even though the model itself remembers nothing. We will see why LLMs forget by default, how saved memories, chat history, and Custom Instructions work, and how these memories are added to the prompt before every reply.

We will cover the following:

Why LLMs forget by default
The context window
Memory within one chat
What ChatGPT Memory is
Saved memories
Reference chat history
Custom Instructions
How memories enter the prompt
An end-to-end example
Viewing and deleting memories
Temporary Chat
Where memory helps and where it fails

Let's get started: How does ChatGPT remember you?

10.9 What is MCP (Model Context Protocol)?

In this blog, we will learn about MCP (Model Context Protocol). We will also see the problem it solves, the pieces it is made of, how a request travels from the AI model all the way to a tool and back, and what we must be careful about while using it.

We will cover the following:

The problem before MCP
What is MCP?
MCP = Model + Context + Protocol
The USB-C analogy
How it is different from a normal API
The three parts of MCP
How it all works step by step
How the connection happens
A real example
Importance of MCP
Things we must be careful about
Summary

Let's get started: What is MCP (Model Context Protocol)?

10.10 What are Agent Skills?

In this blog, we will learn about Agent Skills. We will also see why we need them, what is inside a Skill, how the description makes an agent pick up the right Skill by itself, how progressive disclosure keeps the context window free, how Agent Skills differ from MCP, and how we can create our own Skill.

We will cover the following:

The problem before Agent Skills
What are Agent Skills?
What is inside a Skill?
The description is the trigger
Progressive disclosure, the main idea
A Skill can carry real code
Where Skills live
How do we create our own Skill?
Agent Skills vs MCP
A real example
Importance of Agent Skills
Things we must be careful about
Summary

Let's get started: What are Agent Skills?

10.11 What is OKF (Open Knowledge Format)?

In this blog, we will learn about OKF (Open Knowledge Format). We will also see why the knowledge about our data stays scattered across many places, how OKF writes that knowledge down as a folder of plain markdown files which any AI agent or any tool can read, and where it fits along with MCP and Agent Skills in the real world.

We will cover the following:

The problem: our knowledge is scattered
What is OKF?
OKF = Open + Knowledge + Format
What is inside an OKF bundle?
The frontmatter and the one required field
Cross-links turn files into a graph
Why plain markdown files?
How an agent actually uses it
OKF, MCP, and Agent Skills
What ships with OKF today
Summary

Let's get started: What is OKF (Open Knowledge Format)?

10.12 What are Multi-Agent Systems and When Should We Use Them?

In this blog, we will learn about Multi-Agent Systems - what they are, the three pillars that hold them together, the common agent roles, how agents communicate and coordinate, the trade-offs, and when to use them.

We will cover the following:

The Big Picture
What is a Multi-Agent System
The Three Pillars
Common Agent Roles
How Agents Communicate
How Agents Coordinate
Multi-Agent vs Single Agent - The Trade-offs
Common Mistakes
When to Use a Multi-Agent System
Quick Summary

Let's get started: What are Multi-Agent Systems and When Should We Use Them?

10.13 What are AI SubAgents? Why We Need Them and How They Work

In this blog, we will learn about AI SubAgents. We will understand what they are, why we need them, how they work, and how to use them to build AI systems that can handle big and complex tasks.

We will cover the following:

What is an AI Agent?
What are AI SubAgents?
Why do we need SubAgents?
How do SubAgents work?
Example use case
Benefits of using SubAgents
Challenges with SubAgents
Best practices

Let's get started: What are AI SubAgents? Why We Need Them and How They Work

10.14 How AI Agents Communicate

In this blog, we will learn about how AI agents communicate. We will understand why agents need to communicate, the main ways they talk to each other, the message format, and the protocols that make agents work together to finish complex tasks.

We will cover the following:

What is agent communication?
Why do agents need to communicate?
What agents need in order to communicate
How a message flows between agents
The ways AI agents communicate
Direct Communication
Centralized Communication
Broadcast Communication
Shared Memory Communication
What a message looks like
The rules agents follow to talk
Challenges when agents communicate
Best Practices

Let's get started: How AI Agents Communicate

10.15 What is AI Orchestration? How It Works and the Common Patterns

In this blog, we will learn about AI Orchestration. We will understand what it is, why we need it, how it is different from AI Agents, and the common patterns we use to coordinate multiple LLMs, tools, and steps together to build real AI products.

We will cover the following:

What is AI Orchestration?
Why do we need AI Orchestration?
AI Orchestration vs AI Agents
Components of AI Orchestration
How AI Orchestration works
Patterns of AI Orchestration
Sequential Pattern
Parallel Pattern
Conditional Pattern
Loop Pattern
Orchestrator-Worker Pattern
Tools for AI Orchestration
Challenges in AI Orchestration
Best Practices

Let's get started: What is AI Orchestration? How It Works and the Common Patterns

10.16 What is Sakana Fugu? The Technical Report Explained

In this blog, we are going to learn about Sakana Fugu, a family of AI models that work like a conductor for a team of other AI models.

We will cover the following:

What is Sakana Fugu?
Why Fugu was needed
The big picture: what Fugu does
Collective Intelligence
The two Fugus - Fugu and Fugu-Ultra
How Fugu picks the right model - the lightweight selection head
Teaching Fugu who is best - supervised fine-tuning
Polishing Fugu on real tasks - evolutionary strategies
How Fugu-Ultra conducts an orchestra - the Conductor
Teaching Fugu-Ultra to conduct - GRPO
Stopping the agents from copying each other
How well does Fugu perform?
The clever strategies Fugu discovered on its own
Quick Summary

Let's get started: What is Sakana Fugu? The Technical Report Explained

10.17 How do Computer-Use Agents work?

In this blog, we will learn about how computer-use agents work.

We will cover the following:

What is a computer-use agent?
Why do we need a computer-use agent?
The perceive, think, act loop
How does the agent see the screen?
How does the agent decide what to do?
How does the agent take actions?
A step-by-step walkthrough with an example
The system prompt and tools
Safety and guardrails
Limitations of computer-use agents
Conclusion

Let's get started: How do Computer-Use Agents work?

10.18 How do Browser Agents work?

In this blog, we will learn about how browser agents work. We will also see how a browser agent sees a web page, how it takes actions, and how the observe-think-act loop completes a task from start to finish.

We will cover the following:

What is an AI agent?
Why do we need Browser Agents?
What is a Browser Agent?
The parts of a Browser Agent
How does a Browser Agent see a web page?
How does a Browser Agent take actions?
The observe-think-act loop
A complete example: ordering a book
A simple code example
Screenshots vs page structure
Problems and how they are handled
Where it works well and where it fails

Let's get started: How do Browser Agents work?

Module 11: Agentic Engineering and Agent Frameworks

In this module, we will learn the engineering practices for building reliable agents, and then see how the popular frameworks and coding agents are built.

By the end of this module, we will know how to engineer the harness, the loop, and the graph around an agent, and how real coding agents work under the hood.

Lessons in this module:

11.1 What is Harness Engineering?

In this blog, we will learn about Harness Engineering in AI. We will understand what a harness is, why we need it, and how it is used in AI Agents and evaluation systems.

We will cover the following:

What is a Harness in AI?
Why do we need Harness Engineering?
Components of an AI Harness
Harness Engineering for AI Agents
Harness Engineering for Evaluation
Best Practices in Harness Engineering
Putting It All Together

Let's get started: What is Harness Engineering?

11.2 What is Loop Engineering?

In this blog, we will learn about Loop Engineering, the practice of designing the repeating cycle that an AI agent runs again and again until a task is actually finished. We will also see why we need it, what one turn of the loop looks like, the parts of the loop that we must control, how it is different from prompt engineering and context engineering, the common ways a loop breaks, the techniques that fix those breaks, and where it works well and where it fails.

We will cover the following:

What is Loop Engineering?
Loop Engineering = Loop + Engineering
Why do we need Loop Engineering?
What is a loop in an AI agent?
The simplest loop and its problems
The parts of the loop that we must engineer
Prompt Engineering vs Context Engineering vs Loop Engineering
Common ways a loop breaks
Techniques of Loop Engineering
A complete example
Where it works well and where it fails

Let's get started: What is Loop Engineering?

11.3 What is Graph Engineering?

In this blog, we will learn about Graph Engineering, the practice of building an AI system as a graph of small steps connected by clear paths instead of one giant prompt or one endless loop. We will also see why we need it, what nodes and edges actually mean, how the state travels through the graph, how conditional edges take decisions, how cycles let the system do the work again, how parallel branches save time, how checkpoints let us pause and resume, and where it works well and where it fails.

We will cover the following:

What is Graph Engineering?
Graph = Nodes + Edges
Why do we need Graph Engineering?
The three building blocks: Node, Edge, and State
Let's build our first graph
Conditional edges: taking decisions inside the graph
Cycles: doing the work again when needed
One full run, step by step
Parallel branches: doing many things at the same time
Checkpoints: pause and resume the graph
Human in the loop
Handling errors inside a graph
Graph Engineering vs Loop Engineering
Where Graph Engineering works well
Where Graph Engineering fails
Best practices in Graph Engineering
Conclusion

Let's get started: What is Graph Engineering?

11.4 AI Is Only as Good as Our Definition of Done

In this blog, we will learn about one simple idea, AI is only as good as our definition of done, which means the clear line we draw between a task that is finished and a task that is not. We will also see what a definition of done really is, why AI feels magical on the tasks that a machine can check, why it feels unreliable on the tasks that only a human can judge, how the loop that AI runs behind the scenes creates this exact gap, and how we can write a definition of done that turns a vague task into a checkable one.

We will cover the following:

What is a definition of done
Tasks where the definition of done is exact
Tasks where the definition of done is fuzzy
Why AI is so strong exactly where we can measure
How to write a better definition of done

Let's get started: AI Is Only as Good as Our Definition of Done

11.5 How does LangChain work?

In this blog, we will learn about how LangChain works. We will also see why we need it, what chains, prompts, memory, and output parsers are, how retrieval and agents fit in, and how the full flow works together in the real world.

We will cover the following:

What is LangChain?
Why do we need LangChain?
The core idea behind LangChain
LLM and Prompt Template
What is a Chain?
Output Parser
Memory
Retrieval and RAG
Tools and Agents
A complete flow of how LangChain works

Let's get started: How does LangChain work?

11.6 How does LangGraph work?

In this blog, we will learn about how LangGraph works. We will also see why we need it, what graphs, state, nodes, and edges are, how tools work and who actually calls them, how memory and human-in-the-loop fit in through a complete example, and when to use it in the real world.

We will cover the following:

What is LangGraph?
Why do we need LangGraph?
What is a Graph in LangGraph?
What is State in LangGraph?
Nodes and Edges
Conditional Edges
A complete example
Tools and who calls them
Memory and persistence
Human-in-the-loop
When to use LangGraph

Let's get started: How does LangGraph work?

11.7 How does Claude Code work?

In this blog, we will learn about how Claude Code works. We will also see what Claude Code is, why a normal AI chatbot is not enough, how the agent loop drives it, what tools give it its eyes and hands, how it searches a big project and verifies its own work, and how CLAUDE.md, permissions, plan mode, subagents, and hooks fit together.

We will cover the following:

What is Claude Code?
The problem with a normal AI chatbot
The agent loop
The tools of Claude Code
Example: Claude Code fixing a bug
How does Claude Code search a big project?
How does Claude Code verify its own work?
CLAUDE.md: The project memory
Permissions: How we stay in control
Plan mode, subagents, and hooks
Putting it all together

Let's get started: How does Claude Code work?

11.8 How does Cursor work?

In this blog, we will learn about how Cursor works. We will also see what Cursor is, why it is a code editor with an AI brain on top, how it indexes our codebase into embeddings and searches it by meaning, how Tab autocomplete, Chat, and Agent mode work, how Cursor applies changes through a diff, why it uses different models for different jobs, and how it tries to keep our code private.

We will cover the following:

What is Cursor?
Cursor = Code Editor + AI
The big idea behind Cursor
How does Cursor understand our code?
How does Cursor index our codebase?
How does Tab autocomplete work?
How does the Chat work?
How does the Agent mode work?
How does Cursor apply the changes?
Why does Cursor use different models?
How does Cursor keep our code private?
The complete flow of Cursor

Let's get started: How does Cursor work?

Note: We have now learned how AI Agents work and how they are built. If you want to learn these in a more practical and in-depth way with me in live classes, check out our paid live program: AI and Machine Learning at Outcome School.

Module 12: LLM Inference Engineering

In this module, we will learn how to make LLMs faster and cheaper to run. We will start with what happens during inference, then learn the caching, batching, and speculation techniques, then quantization, and finally the serving engines that put it all together.

By the end of this module, we will understand TTFT, TPOT, and throughput, and know which optimization fixes which bottleneck.

Lessons in this module:

[LLM Inference Optimization](https://outcomeschool.com/blog/llm-infere

ai-engineering-course

AI Engineering Course

Prepared and maintained by the Founder of Outcome School: Amit Shekhar

Follow Amit Shekhar

Follow Outcome School

I teach at Outcome School

Table of Contents

About This AI Engineering Course

What is AI Engineering?

Who is This AI Engineering Course For?

What Will We Learn in This AI Engineering Course?

Prerequisites for This AI Engineering Course

How to Use This AI Engineering Course

AI Engineering Course Curriculum at a Glance

AI Engineering Learning Path

Module 0: Must Know

Module 1: Machine Learning Foundations

1.1 What is Machine Learning?

1.2 Supervised vs Unsupervised Learning

1.3 Linear Regression vs Logistic Regression

1.4 What is Feature Engineering in Machine Learning?

1.5 Precision vs Recall

1.6 What Are L1 and L2 Loss Functions?

1.7 What is Regularization in Machine Learning? L1 vs L2 Explained

1.8 What is Reinforcement Learning?

1.9 What is Contrastive Learning? How It Works Step by Step

Module 2: Deep Learning and Neural Networks

2.1 What is Bias In Artificial Neural Network?

2.2 How Does Gradient Descent Work?

2.3 How Does Backpropagation Work? The Math Explained Step by Step

2.4 What is Cross-Entropy Loss?

2.5 What is Dropout in Neural Networks and How Does It Work?

2.6 Batch Normalization vs Layer Normalization

2.7 What is RMSNorm? Root Mean Square Layer Normalization Explained

2.8 What is a Recurrent Neural Network (RNN)?

2.9 How does PyTorch work?

2.10 How Does The Machine Learning Library TensorFlow Work?

Module 3: Generative AI and the Transformer Architecture

3.1 What is Generative AI?

3.2 What are Autoregressive Models?

3.3 Tokenization in LLMs

3.4 What is Byte Pair Encoding (BPE) in LLMs?

3.5 What are Embeddings?

3.6 How do RNNs and Transformers differ?

3.7 How Does the Transformer Architecture Work?

3.8 Encoder vs Decoder in Transformers

3.9 What is Self Attention in Transformers and How Does It Work?

3.10 How Does Attention Work? The Math Behind Q, K, and V Step by Step

3.11 Why Do We Scale Attention by √dₖ? The Math Behind the Scaling Factor

3.12 What is Causal Masking in Attention and Why Do LLMs Need It?

3.13 What is Multi-Head Attention in Transformers and How Does It Work?

3.14 What is Cross Attention in Transformers and How Does It Work?

3.15 What is RoPE (Rotary Position Embedding)? The Math Behind It

3.16 What is the Feed-Forward Network in LLMs and What Does It Do?

Module 4: How LLMs Generate Text

4.1 How does Temperature control LLM output?

4.2 How do Top-k and Top-p Sampling work?

4.3 Stop Tokens in LLMs

4.4 How does Token Streaming work?

4.5 What is the Lost in the Middle Problem in LLMs and How to Fix It?

Module 5: Modern LLM Architecture

5.1 Evolution of LLM Architecture

5.2 What is Mixture of Experts (MoE) and How Does It Work?

5.3 What is Grouped Query Attention (GQA) and Why Do LLMs Use It?

5.4 How does Sliding Window Attention work?

5.5 How do Attention Sinks work?

5.6 What is Flash Attention and Why Is It So Fast?

5.7 What is DeepSeek-V4 and How Does It Work? Architecture Explained

Module 6: Types of Language Models

6.1 What are Small Language Models (SLMs) and When Should We Use Them?

6.2 What are Large Reasoning Models (LRMs) and How Are They Different from LLMs?

6.3 What are Recursive Language Models (RLMs) and How Do They Work?

6.4 How do Diffusion Language Models (DLMs) work?

6.5 Jev and System One Models Explained

Module 7: Training, Fine-Tuning, and Alignment

7.1 How does fine-tuning work?

7.2 What is LoRA (Low-Rank Adaptation) and How Does It Fine-Tune LLMs?

7.3 How does Prefix Tuning work?

7.4 How does Knowledge Distillation work?

7.5 What is Continual Learning in LLMs? Solving Catastrophic Forgetting