How AI Models (LLMs) Work

Quick Overview: What Are LLMs?

Large Language Models (LLMs) like ChatGPT, Claude, Gemini, and Llama are autoregressive neural networks that model the probability of sequences of tokens (words/subwords). They power every modern chatbot you use.

The lecture breaks training into five key components: architecture (transformers – skipped here), training loss/algorithm, data, evaluation, and systems/hardware optimization.

Today’s LLMs follow two main phases:

• Pre-training – Model the entire internet (GPT-2/3 era)
• Post-training – Turn raw models into helpful, safe AI assistants (ChatGPT era)

Pre-Training: The Foundation

Core Task & Loss

LLMs are trained with next-token prediction using cross-entropy loss. This is equivalent to maximizing the log-likelihood of real text from the internet.

Tokenization → Embed tokens → Transformer → Linear layer + softmax → Predict next token.

Why Tokenizers Matter

Byte-Pair Encoding (BPE) is the most common method. It merges common character pairs into single tokens (average ~3-4 letters per token). This keeps sequences short while handling typos, rare words, and non-English languages.

Key insight: Vocabulary size = output dimension of the model. Good tokenizers dramatically improve math, code, and multilingual performance.

Data Pipeline (The Real Secret Sauce)

Start with Common Crawl (~250 billion pages). Then:

• Extract clean text from HTML
• Filter NSFW, PII, low-quality content
• Deduplicate (headers, boilerplate, duplicate books)
• Heuristic + model-based quality filtering (Wikipedia references → high-quality classifier)
• Domain up/down-weighting (boost code & books, reduce entertainment)
• Final high-quality overfitting phase (Wikipedia + curated data)

Result: From ~150B tokens (early models) → 15 trillion tokens today (Llama 3 scale).

Evaluation & Scaling Laws

Old metric: Perplexity (exponentiated average loss). New standard: Massive benchmarks like MMLU, HELM, Open LLM Leaderboard.

Scaling Laws (Chinchilla paper): Performance improves predictably with more compute, data, and parameters. Optimal ratio ≈ 20 tokens per parameter for training (150+ for inference cost).

Larger models + more data always win — no overfitting at these scales.

Post-Training: From Raw Model → AI Assistant

Step 1: Supervised Fine-Tuning (SFT)

Fine-tune on high-quality human (or LLM-generated) question-answer pairs. You don’t need much data — 2,000–10,000 examples often suffice because the model already “knows” language from pre-training.

Goal: Teach the model to respond in the desired style/format.

Step 2: Alignment with Human Preferences

Use RLHF (PPO) or the simpler modern method DPO:

• Generate two answers per prompt
• Humans (or stronger LLMs) pick the better one
• Optimize the model to produce more “preferred” outputs

DPO is now preferred because it’s just maximum likelihood — no messy reinforcement learning loops.

Systems: Making It Run on Real Hardware

Models are huge — systems are now the bottleneck. Key tricks covered:

• Low-precision training (FP16/BF16 for compute, FP32 for weights)
• Operator fusion (torch.compile = 2× speedup)
• Memory hierarchy awareness & communication optimization

Training Llama 3 405B took ~70 days on 16,000 H100s (~$75M+).

Actionable Learning Guide – Start Building Today

Follow these steps to go from “I watched the video” to “I understand (and can experiment with) LLMs”:

Pro tip: The lecture says data is 70% of the magic. Start by cleaning your own small dataset — that skill scales to production models.

Key Takeaways

• Pre-training = model the internet with next-token prediction
• Data cleaning & scaling laws matter more than fancy architectures
• Post-training (SFT + DPO) turns raw models into safe, helpful assistants
• Systems optimization (low precision, fusion) is what actually lets models run
• You can replicate 90% of the breakthroughs with open tools today

Related

For a frontier-lab view of scaling open models (token efficiency, long context, agent swarms), see How Moonshot Makes Open Models Great (Kimi CEO Yang Zhilin) · coding agent path: OpenCode + Kimi.