Every term, defined
All the wording that carries a dotted underline anywhere on this page, gathered in one place. Type in the search box on the left to narrow it — it matches the definitions too, not just the terms.
Why the tail is the whole argument
Four symmetric curves centred on zero, three of them with the same variance. What separates them is not the middle but the edge. Switch the vertical axis to logarithmic and look at six units out: the normal has all but disappeared while the Cauchy is still going, a gap of six orders of magnitude.
How they turn into each other
Almost none of these were invented separately. Follow an arrow to see the assumption that converts one into another — most of applied statistics lives on these edges.
Picking one
Start from the thing you measured, not from the curve you like.
| A single yes or no | Bernoulli |
| Successes in a fixed number of tries | Binomial |
| Successes drawn from a finite pool, no replacement | Hypergeometric |
| Events per hour, rare and independent | Poisson |
| Events per hour, but more erratic than Poisson allows | Negative binomial |
| Tries until the first success | Geometric |
| Time until the next event, no memory | Exponential |
| Time until the k-th event | Gamma / Erlang |
| Time until failure, with ageing or burn-in | Weibull |
| A sum of many small independent effects | Normal |
| A product of many small effects; prices, incomes | Log-normal |
| A proportion or an unknown probability | Beta |
| Anything between two bounds, nothing more known | Uniform |
| The largest value in each block or year | Gumbel |
| Wealth, city size, file size — the top dominates | Pareto |
| A mean estimated from a small sample | Student's t |
| A sample variance, or a goodness-of-fit total | Chi-square |
| A ratio of two variances | F |
Past one dimension
These need more than an x-axis, so they get a sentence instead of a curve.
- Multinomial — the binomial with more than two outcomes. Counts across k categories from n draws; every marginal is a binomial.
- Multivariate normal — the normal with a covariance matrix instead of a variance. The backbone of regression, Kalman filters and Gaussian processes.
- Dirichlet — the beta with more than two parts. A distribution over probability vectors that sum to one; the conjugate prior for the multinomial, and the base of topic models.
- Wishart — a distribution over covariance matrices; the multivariate chi-square.
- Von Mises — the normal wrapped onto a circle, for angles, wind directions and times of day.
- Copulas — a way to keep each variable's own marginal distribution while specifying the dependence between them separately. Their misuse in credit models is a well-documented chapter of 2008.
- Mixtures — two or more distributions weighted together. Any bump-shaped-but-not-quite dataset is usually a mixture, and fitting a single curve to it produces a mean nobody in the data actually has.
- The empirical distribution — your data, each point weighted 1/n. Resampling from it is the bootstrap, and it assumes nothing at all.