Data · Guide

Mathematics for ML

Linear algebra, calculus and probability — the working subset you need to read a paper and debug a model that will not converge.

— min read Data

How Much Do You Actually Need

Less than a mathematics degree, more than most bootcamps admit. The bar is: read a paper without stalling, and work out why a model is not learning.

You will almost never derive anything by hand. Libraries do the arithmetic. What the mathematics buys you is the ability to reason about why something behaves the way it does — why the loss plateaued, why the features had to be scaled, why that layer's gradients vanished.

Three areas carry nearly all the weight. Linear algebra is how data and models are represented. Calculus is how they learn. Probability is how you deal with the fact that none of it is certain.

Learn them alongside the models rather than as a prerequisite course. Abstract linear algebra is dull; understanding that a neural network layer is a matrix multiply followed by a nonlinearity makes it stick.

Linear Algebra

Everything in machine learning is an array of numbers. A sample is a vector, a batch is a matrix, an image batch is a tensor. Almost every operation is a matrix multiplication.

ObjectShapeIn practice
ScalarA learning rate, a loss value
VectornOne sample's features; one embedding
Matrixm × nA batch of samples; a layer's weights
Tensor3D+Images (batch, height, width, channels)

The dot product is the single most important operation, because it measures alignment. Two vectors pointing the same way give a large positive number; perpendicular gives zero. That is exactly what cosine similarity uses, which is why it underpins search, recommendations and retrieval-augmented generation.

Shapes — the thing that actually breaks
# a batch of 32 samples, 10 features each
X  = shape(32, 10)

# a layer mapping 10 features to 64 units
W  = shape(10, 64)

# inner dimensions must match: 10 == 10
XW = shape(32, 64)

# the error you will see a thousand times:
# mat1 and mat2 shapes cannot be multiplied (32x10 and 64x10)

Matrix multiplication is not commutative: AB and BA are different, and often only one is even defined. Nearly every shape error is the inner dimensions failing to meet.

When a model will not run, print the shapes. When it runs but will not learn, print the shapes anyway — a silent broadcast can produce a valid tensor that means nothing.

Two more worth knowing by name: the transpose, which flips rows and columns and is how you make dimensions line up; and eigenvectors, the directions a transformation only stretches, which is what PCA finds when it picks the axes carrying the most variance.

Calculus & Gradients

Calculus is how a model learns. A derivative answers one question: if I nudge this parameter, which way does the error move, and how sharply?

With many parameters the derivative becomes a gradient — a vector holding one partial derivative per parameter. It points in the direction of steepest increase, so training walks the opposite way. That is the whole of gradient descent: compute the gradient, step against it, repeat.

Gradient descent, in five lines of pseudocode
for epoch in range(n):
    preds = model(X)
    loss  = error(preds, y)
    grad  = d_loss_d_weights(loss)   # which way is uphill
    weights -= learning_rate * grad  # step downhill

The learning rate is the step size, and it explains most training failures. Too large and the loss oscillates or diverges; too small and it crawls or settles in a poor spot. It is the first thing to change when training misbehaves.

The chain rule is what makes deep networks trainable. To know how a weight in the first layer affects the final loss, you multiply the derivatives along the path between them. That is all backpropagation is: the chain rule, applied backwards through the network, reusing the intermediate results.

Multiplying many small derivatives together drives the product toward zero — the vanishing gradient, where early layers stop learning. Multiplying large ones explodes it. ReLU, residual connections and normalisation all exist to manage this product.

Probability

Machine learning outputs beliefs, not facts. A classifier does not say "spam" — it says 0.94, and what you do at 0.94 is a decision you have to make.

IdeaWhy it matters here
DistributionThe shape of the data decides which model and metric are sensible
Conditional probabilityP(spam given these words) is literally what a classifier estimates
Bayes' theoremUpdating a belief with evidence; the base rate usually dominates
ExpectationThe long-run average — what a loss function is minimising
VarianceHow much predictions move if you resample the training data
IndependenceAssumed constantly, true rarely; Naive Bayes is named after it

The normal distribution shows up everywhere because of the central limit theorem: averages of many independent things tend toward it regardless of what they started as. That is why so many methods assume it — and why they mislead on the heavy-tailed data that is common in practice.

Cross-entropy, the standard classification loss, is a probability idea: it measures how surprised the model is by the true answer. Confident and right costs almost nothing; confident and wrong costs a great deal. That asymmetry is what pushes a model toward honest probabilities rather than confident guesses.

The bias–variance trade-off is the frame for underfitting and overfitting. High bias means the model is too simple to capture the pattern; high variance means it has memorised this particular sample. Every regularisation technique is buying one with the other.

Interview Questions

What is a gradient, and what does gradient descent do with it?

The gradient is the vector of partial derivatives of the loss with respect to each parameter; it points uphill. Gradient descent steps in the opposite direction by a distance set by the learning rate, and repeats.

Why do gradients vanish in deep networks?

Backpropagation multiplies derivatives along the path from loss to weight. Many factors below one drive the product toward zero, so early layers receive almost no signal. ReLU, residual connections and normalisation are all responses to this.

Why does cosine similarity dominate embedding search?

It compares direction rather than magnitude, so two texts about the same subject score as similar even when one is far longer. It is the dot product of the normalised vectors.

Explain the bias–variance trade-off.

Bias is error from a model too simple to represent the pattern; variance is error from sensitivity to the particular training sample. Reducing one usually raises the other, and the aim is the total, not either alone.

Why is the normal distribution assumed so often?

The central limit theorem: sums and averages of many independent contributions tend toward it. It is convenient and often reasonable — and badly wrong on heavy-tailed data like income or latency.

Your loss is oscillating and will not come down. First thing you change?

The learning rate — lower it. Oscillation is the classic sign of steps too large to settle into a minimum.

Quick Quiz

1. Multiplying shapes (32, 10) by (10, 64) gives…
2. The gradient points…
3. Backpropagation is essentially…
4. Cosine similarity compares…
5. A model that memorised its training sample has high…