A system that learns patterns from data. Loosely inspired by the brain. Really just statistics done at enormous scale.
What it is: pattern recognition, with math
Your brain sees a mango. Color neurons say "yellow." Shape neurons say "oval." Smell neurons say "sweet." Your brain concludes "mango." A neural network does the same thing, with math instead of neurons.
INPUT HIDDEN LAYERS OUTPUT
(raw data) (pattern finders) (answer)
"yellow" ───→ [neuron] ──→ [neuron] ──→ Mango (92%)
"oval" ───→ [neuron] ──→ [neuron] ──→ Lemon (6%)
"sweet" ───→ [neuron] ──→ [neuron] ──→ Orange (2%)
Each connection has a weight, a number indicating how important that input is. Training adjusts these weights until the network predicts correctly.
How it works: three concepts
- Training. Show the network millions of examples; let it adjust weights. A kid learning fruits by seeing 10,000 photos.
- Parameters. All those weights collectively. "Llama 3.1 has 405B parameters" means 405 billion adjustable knobs.
- Inference. Using the trained network to predict on new data. Training is school. Inference is the job.
7B = 7 billion parameters
70B = 70 billion
405B = 405 billion (Llama 3.1 largest dense model)
~1.8T = ~1.8 trillion (GPT-4 estimate, MoE)
FACTUAL NOTE · MIXTURE OF EXPERTS The widely cited "GPT-4 has 1.8 trillion parameters" figure refers to total parameters across a Mixture-of-Experts (MoE) architecture. Only ~280B are active per token; the model routes each token through 2 of 16 expert sub-networks. So GPT-4 is not a 1.8T dense model the way Llama 405B is a 405B dense model. The distinction matters for VRAM, latency, and cost reasoning.
Why more parameters ≠ better
- Overfitting. Too many parameters and the model memorizes training data instead of learning patterns.
- Diminishing returns. 7B → 70B is a massive jump. 70B → 700B is a much smaller one.
- Data quality beats size. A 7B model on clean medical data outperforms a 1T model on random internet text, on medical tasks.
PM decisions
- Model size selection. Bigger isn't better, it's a tradeoff. Covered in detail in Lesson 3 and Appendix A.
- Where the model runs. API (someone else's GPU) vs self-hosted (your GPU) vs fine-tuned variant. Each has cost and IP implications.
Failure modes
- Reaching for the biggest model "to be safe", and 3-5x'ing your unit economics. Current frontier-tier gaps are roughly Haiku → Sonnet 3x, Haiku → Opus 5x.
- Assuming model size correlates linearly with quality on your task. It often doesn't.
The case
Jack's first instinct is to use the largest model available: "the desk deserves the best." He prototypes the summarizer on Opus for two weeks. Quality is great. Then finance sees the projected bill at scale and pushes back hard. Jack runs the same eval on Haiku and Sonnet; the gap on news summarization is small, well within editor tolerance. He migrates to Sonnet, reserves Opus for genuinely hard reasoning tasks elsewhere on the roadmap, and accepts the lesson: model selection is a product decision, not a prestige decision. The two weeks weren't wasted, they generated the cost-versus-quality data that made the case for Sonnet defensible. But they could have been avoided by running the eval-against-cost comparison up front.
Further reading
- 3Blue1Brown · Neural Networks (video series)
- Andrej Karpathy · Neural Networks, Zero to Hero