Deep Learning Guide · 01
How AI Learns
AI ‘learning’ is not the same as a person studying and grasping meaning. It is a computational process in which a model handles many examples, measures how wrong it was, and repeatedly changes its internal weights so the next prediction is a little less wrong.
The idea in three sentences
- Data is the world a model can see; the training objective determines what it is rewarded for noticing in that world.
- A model distributes patterns across many weights rather than storing a book of answers, although it can memorize some training material.
- Low error on training data does not prove that a model will be accurate or safe in unfamiliar reality.
01 · What Learning Means
Instead of writing every rule, we ask a system to find the shape of rules in examples.
Traditional programs and learned models both have human-defined purposes and computer-executed calculations. The difference is where the detailed mapping from input to output comes from.
The rules are visible in code.
A developer writes procedures such as ‘if this condition holds, do that.’ Similar inputs follow explicit rules, and failures are often traceable to a particular instruction.
Example: calculate tax from a rate tableThe rules are distributed across weights.
A developer chooses the architecture, data, and error measure; a training algorithm adjusts many weights. The result is flexible, but a particular judgment is harder to trace to one example or rule.
Example: classify an animal from pixel patterns02 · Five Ingredients
Training outcomes come from five interacting elements—not model size alone.
The same architecture can become a very different model depending on what it sees, what counts as correct, and how it is adjusted.
- 01
Data
Provides input examples and, when needed, labels. Representation, accuracy, provenance, duplication, and consent matter as much as volume.
- 02
Model
A family of functions that can map inputs to outputs. Layers, connections, and scale shape the kinds of patterns it can represent.
- 03
Loss function
Turns the distance between prediction and target into a number. Values that are not measured are not automatically protected by training.
- 04
Optimizer
Decides how far and in which direction to change weights to reduce loss. Gradient descent and its variants are widely used.
- 05
Compute and time
Accelerators, memory, and electricity make it possible to repeat over more examples and larger models. More compute does not guarantee quality or safety.
03 · Training Loop
Predict, measure the error, make a small correction, and repeat.
Real training is distributed across many devices, but its core logic can be compressed into this loop.
- 01Training dataFeed a batch of examples
- 02Forward passPredict with current weights
- 03LossMeasure distance from targets
- 04BackpropagationCalculate each weight's effect
- 05UpdateMove toward lower loss
- 06RepeatContinue with the next batch
A tiny fictional example: predicting delivery time
Suppose a simple model using only distance predicts 25 minutes for a 10-kilometer delivery, while the recorded time is 35 minutes.
Output from the current weights
Value in the training record
The loss function measures the miss
Adjust weights toward a longer next prediction
A real model would combine distance with traffic, weather, time, and many other features. It does not try to fit one case exactly; it searches for weights that reduce error across many cases.
- Batch
- A group of examples used together for one update
- Step
- One completed prediction, loss calculation, and weight update
- Epoch
- One pass through the prepared training data
- Convergence
- A state where further repetition no longer reduces loss much
04 · Backpropagation
Backpropagation distributes the output error backward through the network.
A neural network moves from inputs through intermediate layers to an output. A forward pass creates a prediction; backpropagation retraces that computation to calculate how changing each weight would change the loss.
Input
Numerical representations of pixels, token fragments, or sound
Intermediate representations
Combine simple features into more abstract patterns
Output
Class probabilities, a next token, or another generated value
Error signal
Starts at the result and calculates contribution along earlier connections
new weight = old weight − learning rate × gradientThe gradient indicates the direction in which a weight would make loss rise or fall. The learning rate controls the size of each move.
Learning is slow.
It may be stable but take far too many updates to reach a useful region.
Loss falls steadily.
Training curves help balance speed against stability.
It jumps past better values.
Loss can oscillate, grow, or make training fail.
05 · Ways to Learn
Not every AI system learns in the same way.
Methods differ in what is predicted and where feedback comes from. Real products often combine several stages.
Supervised learning
Pairs inputs with human-provided labels. It is common in spam classification, defect detection, and numerical prediction.
Who labeled the answers, and by what rule?Self-supervised learning
Creates prediction tasks from the data itself, such as the next or a hidden token. It is central to language-model pretraining.
How do errors and biases in public data repeat?Reinforcement learning
Learns a policy that seeks higher reward through actions in an environment. It is used in games, robotics, and some post-training.
Which values and loopholes are missing from the reward?Post-training
Uses instruction examples, preference comparisons, and safety rules to adjust how an already pretrained model responds.
Whose preferences become the default behavior?06 · Language Models
Language models learn representations by predicting the next token across enormous text collections.
A conversational AI is not initially trained to answer questions politely. We should distinguish broad pretraining on language patterns from post-training that shapes instruction-following behavior.
- 01
Collect and clean
Documents and code are gathered and processed for duplication, quality, personal information, and hazardous content. Inclusion and exclusion shape knowledge and bias.
- 02
Tokenize
Text becomes sequences of token IDs the model can process. A token may be a whole word, part of a word, or punctuation.
- 03
Predict the next token
Given earlier tokens, training raises the probability of the token that actually follows. Almost every position in a document can become a training example.
- 04
Repeat at scale
Many devices process batches and aggregate gradients. Across many steps, patterns of grammar, facts, code, and problem-solving form in the weights.
- 05
Train instructions and preferences
Examples of good answers and comparisons among responses shape helpful dialogue, refusals, and safety behavior. Preference is not a complete test of truth.
- 06
Evaluate and deploy
Held-out tasks, safety tests, and real-use monitoring reveal capability and failure. New misuse and changing environments still require review after deployment.
07 · Generalization & Failure
The goal is not to fit the training material, but to work on cases never seen before.
Training performance alone cannot distinguish a model that memorized answers from one that found broader patterns. Data is therefore split by role.
- 01
Training set
Used to update model weights
- 02
Validation set
Compares architecture, learning rate, and stopping choices
- 03
Test set
Used at the end to estimate performance on unseen problems
Six problems that can remain while loss falls
Overfitting
The model memorizes accidents and noise in training examples, then performs poorly on new data.
Biased sample
Errors grow for groups and situations that are missing, rare, or incorrectly represented.
Shortcut learning
The model uses easy background, format, or marker cues instead of the intended principle.
Data leakage
Test problems or close copies enter training, making measured performance look unrealistically high.
Distribution shift
The deployed environment changes, so patterns that worked during training no longer hold.
Objective mismatch
Measured loss falls while qualities people actually value—fairness, safety, or usefulness—get worse.
Questions to ask when a training result is announced
- 01
Are data provenance, consent, time period, and representation disclosed?
- 02
Were training, validation, and test materials genuinely separated?
- 03
Are failures reported by group, language, and environment—not only as one average?
- 04
Do loss and reward cover the values the real system must protect?
- 05
Can researchers outside the developer reproduce or independently evaluate the result?
- 06
Can errors, misuse, and environmental change be monitored and stopped after deployment?
Check your understanding
Does loss near zero mean an AI is safe?
No. Loss measures error against a defined dataset and objective. Situations outside the data, unmeasured values, and real-world permissions require separate evaluation.
Does backpropagation directly tell the model the correct rule?
No. It calculates the direction and magnitude of each weight's effect on the current loss. An optimizer uses that signal to make small updates.
Does a language model search its training documents like a database?
A base model normally calculates next-token probabilities from learned weights. It may memorize some strings, but generation is not generally a document lookup. A separate search tool may also be connected.
Does a strong test score guarantee success in deployment?
No. The test must represent real users and conditions, and deployment introduces distribution shifts, interactions, and misuse that require continuing observation.
08 · Study Path
Interact first, add mathematics next, and only then build a complete model.
You do not need to finish every resource. Start at the level that matches your goal and step back when unfamiliar terms block understanding.
Beginner · See it move
30 minutes–3 hoursTensorFlow Playground
Change data, layers, and learning rate in a browser and watch training loss and test behavior respond.
Open learning resourceGoogle Machine Learning Crash Course
Learn loss, gradient descent, data splits, and neural networks through short explanations, videos, and exercises.
Open learning resourceIntermediate · Connect code and math
Days–weeksDive into Deep Learning
An open book combining equations and executable code for linear models, generalization, optimization, and neural networks.
Open learning resourcePyTorch · Automatic Differentiation
A small code tutorial showing how computational graphs and automatic differentiation derive gradients from a forward computation.
Open learning resourceAdvanced · Understand the full stack
Several weeks or moreDeep Learning · MIT Press
A free online textbook covering ML foundations, overfitting and validation, optimization, and deep networks systematically.
Open learning resourceStanford CS336 · Language Modeling from Scratch
Covers the complete path from data collection and tokenization through Transformers, distributed training, evaluation, and post-training.
Open learning resourceThe InstructGPT paper
A representative study connecting supervised fine-tuning, human preference comparisons, and reinforcement learning to shape instruction-following behavior.
Open learning resourceContinue to inference, the stage where a trained model handles new input. The essential distinction is simple: training changes weights; inference uses them.