Statistics
Descriptive measures, hypothesis testing, Bayesian thinking, and the sampling traps that quietly invalidate a result.
Why Statistics Decides Everything
Every dataset you touch is a sample, not the world. Statistics is the discipline of saying how much of what you observed is signal and how much is the luck of which rows you happened to get.
This is the skill that separates a data scientist from someone who can call fit(), and an analyst from someone who can write SQL. A model with no notion of uncertainty will confidently report a pattern that vanishes next month.
You need less of it than a statistics degree and more of it than most bootcamps admit. The four sections below are the working set.
Descriptive Statistics
Describing what you have, before inferring anything about what you do not. The first question is always the shape of the distribution, because every summary number after that depends on it.
| Measure | What it tells you | When it misleads |
|---|---|---|
| Mean | The balance point | Dragged hard by outliers and skew |
| Median | The middle value | Ignores the size of extremes entirely |
| Mode | The most common value | Meaningless on continuous data |
| Std deviation | Typical distance from the mean | Assumes a roughly symmetric shape |
| Percentiles | Where a value sits in the pack | Needs enough data to be stable |
Income, latency and session length are all right-skewed, which is why mean salary and mean response time are two of the most misleading numbers in common use. Report the median and the tail percentiles instead: p50, p95, p99.
Hypothesis Testing
Inference is the move from sample to population. You assume nothing is happening — the null hypothesis — and ask how surprising your data would be if that were true.
A p-value answers exactly one question: if there were no real effect, how often would I see a result at least this extreme? A small p-value means the data sits awkwardly with the null. It does not tell you the probability that your hypothesis is true, and it says nothing at all about whether the effect is big enough to care about.
| Term | Plain meaning |
|---|---|
| Null hypothesis | Nothing is going on |
| p-value | How surprising this data would be if nothing were going on |
| Confidence interval | A range of effect sizes the data is compatible with |
| Type I error | Crying wolf — a false positive |
| Type II error | Missing a real effect — a false negative |
| Statistical power | Your chance of detecting an effect that is really there |
Prefer the confidence interval to the p-value whenever you can. "Between 1% and 9% lift" tells a decision-maker something actionable; "p = 0.04" does not.
Bayesian Thinking
The Bayesian move is to start from what you already believed and update it with evidence, rather than testing a single hypothesis in isolation.
The classic illustration: a test is 99% accurate for a disease that affects 1 in 10,000 people. You test positive. Your actual chance of having it is under 1%, because the false positives from the enormous healthy population swamp the true positives from the tiny sick one. The base rate dominates the test accuracy.
In practice the Bayesian framing wins where you have real prior knowledge or need a probability you can act on directly — "there is an 85% chance B beats A" is easier to decide on than a p-value.
Bias & Sampling Traps
Most bad analysis is not bad mathematics. It is correct mathematics on a sample that was never representative.
| Trap | What goes wrong | Typical example |
|---|---|---|
| Survivorship bias | You only measured what remained | Studying customers who did not churn |
| Selection bias | The sample was never random | Surveying only people who opened the email |
| Simpson's paradox | A trend reverses once you split by a group | Overall rate up, every segment down |
| Confounding | A third variable drives both | Ice cream sales and drownings |
| Multiple comparisons | Test twenty things, one looks significant | Slicing until something has p < 0.05 |
Interview Questions
What does a p-value actually mean?
The probability of seeing data at least this extreme if the null hypothesis were true. It is not the probability the null is true, and not the probability your finding is real. Candidates who state either of those are usually rejected on the spot.
Mean or median for salary?
Median. Salary is right-skewed, so a few very large values pull the mean above what a typical person earns. Report the median plus percentiles.
Explain Type I and Type II error.
Type I is a false positive — you acted on an effect that was not there. Type II is a false negative — you missed one that was. Lowering your significance threshold trades the first for the second.
What is Simpson's paradox?
A trend that holds in every subgroup reverses when the groups are combined, usually because group sizes differ. It is the reason you always check whether an aggregate result survives segmentation.
A test is 99% accurate for a 1-in-10,000 disease and you test positive. Should you worry?
Not much. The healthy population is so much larger that its 1% false positives outnumber the true positives from the sick population by roughly a hundred to one. Base rate beats test accuracy.