What is a neural network?
A neural network is layers of simple units that each combine their inputs, apply a nonlinear activation, and pass the result on. By adjusting the connection weights, the network learns to map inputs to outputs. Backpropagation is the algorithm that computes how to adjust those weights.
Why it matters
Neural networks are the foundation of deep learning and everything modern in AI, including the LLMs later in this track. Understanding neurons, layers, activations, and backpropagation conceptually is what lets you reason about why a network trains, stalls, or diverges instead of treating it as magic.
What to learn
- The neuron: weighted sum plus activation
- Layers and how depth builds representations
- Activation functions and why nonlinearity matters
- The loss function as the thing to minimize
- Gradient descent and learning rate
- Backpropagation as the chain rule at scale
- Epochs, batches, and iterations
Common pitfall
Treating the network as a black box and tweaking randomly when it will not learn. Without the intuition for gradients and the loss, you cannot tell whether the learning rate is wrong, the data is unscaled, or the architecture is broken. Understanding the basics turns frustrating guesswork into directed debugging.
Resources
Primary (free):
- 3Blue1Brown — Neural networks · video
- Google — Neural networks crash course · docs
- Andrej Karpathy — Neural networks: zero to hero · video
Practice
On paper or in code, trace a single forward pass through a tiny two-layer network: compute the weighted sums, apply activations, and get an output. Then describe in words how backpropagation would adjust one weight to reduce the loss. Done when the forward pass and the idea of the backward pass both make sense.
Outcomes
- Explain a neuron, a layer, and an activation function.
- Describe how gradient descent minimizes a loss.
- Explain backpropagation as the chain rule applied at scale.
- Debug a non-learning network with intuition, not guesses.