📈 Section 4 · Statistics 🟡 Intermediate MODULE 20

Probability Distributions

⏱️ 50 min

📖 Normal, Binomial & Uniform

🧩 3 Quiz Questions

🏗️ 1 Challenge · 3 tasks

Your progress in Section 440%

🎯 Where this fits: Lesson 19 described data you already have. This lesson describes the shapes that data-generating processes tend to follow before you've even collected the data — the "typical pattern" a coin flip, a measurement error, or a random sample tends to fall into. Understanding a few common probability distributions — normal, binomial, and uniform — is the foundation for hypothesis testing in Lesson 21 and for how most machine learning models reason about uncertainty later in this course.

Section 1

What Is a Probability Distribution?

A probability distribution describes how likely each possible value (or range of values) of a random variable is. The exact mathematical description splits into two cases, depending on whether the variable is discrete or continuous.

🔢

Discrete → PMF

A discrete variable (dice rolls, number of defects) has a Probability Mass Function — it gives the exact probability of each individual outcome, and those probabilities sum to 1.

📏

Continuous → PDF

A continuous variable (height, temperature, response time) has a Probability Density Function — the probability of any single exact value is 0; probability comes from the area under the curve over a range.

📈

CDF

A Cumulative Distribution Function, for either kind, gives P(X ≤ x) — the probability the variable is at or below some value. scipy.stats exposes this as .cdf().

🧩

Why it matters

Knowing which distribution a process follows lets you answer "how likely is this outcome, or one more extreme?" — exactly the question hypothesis testing in Lesson 21 depends on.

Section 2

The Normal (Gaussian) Distribution

The normal distribution is the familiar symmetric bell curve. It shows up constantly in nature and measurement — heights, test scores, measurement error — because of the Central Limit Theorem, which (loosely) says that averages of many independent random effects tend toward a normal shape, regardless of the shape of the individual effects.

Normal Distribution PDF

f(x) = (1 / (σ√(2π))) · e^(−(x−μ)² / (2σ²))

Fully described by just two parameters: the mean μ (where the peak sits) and the standard deviation σ (how wide the curve is).

✨

The Empirical Rule — 68-95-99.7

For any normal distribution: about 68% of values fall within 1 standard deviation of the mean (μ ± σ), about 95% fall within 2 standard deviations (μ ± 2σ), and about 99.7% fall within 3 standard deviations (μ ± 3σ). This is a standard, well-known result — worth memorizing, since it lets you sanity-check "is this value unusual?" without running any code at all.

normal_distribution.py

PYTHON

import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

# Generate 10,000 random samples from a normal distribution: mean=100, std=15
rng = np.random.default_rng(seed=42)
samples = rng.normal(loc=100, scale=15, size=10000)

print("Sample mean:", np.mean(samples).round(2))
print("Sample std: ", np.std(samples, ddof=1).round(2))

# scipy.stats.norm — the theoretical distribution, not random samples
x = np.linspace(55, 145, 300)
pdf = stats.norm.pdf(x, loc=100, scale=15)

# What fraction of values fall below 115 (one std above the mean)?
p_below_115 = stats.norm.cdf(115, loc=100, scale=15)
print("P(X <= 115):", p_below_115.round(4))   # ~0.8413

fig, ax = plt.subplots(figsize=(8, 5))
ax.hist(samples, bins=50, density=True, alpha=0.5, color="#4f9eff", label="Random samples")
ax.plot(x, pdf, color="#2de8c0", linewidth=2.5, label="Theoretical PDF")
ax.legend()
plt.show()

📝

numpy.random vs scipy.stats — sampling vs. the exact math

rng.normal() draws random numbers that follow the distribution — useful for simulation. stats.norm.pdf() and stats.norm.cdf() compute the exact theoretical curve and probabilities — useful for calculation. density=True on ax.hist() rescales the histogram so it's directly comparable to the PDF curve on the same axes, since a raw histogram's bar heights depend on bins and sample count.

Section 3

The Binomial Distribution

The binomial distribution is discrete — it models the number of "successes" out of n independent yes/no trials, each with the same success probability p. Coin flips, pass/fail tests, and conversion counts on a fixed number of visitors are all classic binomial situations.

Binomial PMF

P(X = k) = C(n, k) · p^k · (1−p)^(n−k)

The probability of exactly k successes in n trials, where C(n, k) is "n choose k" — the number of ways to pick which k trials succeed.

binomial_distribution.py

PYTHON

# A fair coin, flipped 10 times — how many heads is likely?
n, p = 10, 0.5

# Exact probability of getting exactly 6 heads
p_exactly_6 = stats.binom.pmf(6, n, p)
print("P(exactly 6 heads):", p_exactly_6.round(4))

# Probability of 6 or fewer heads (CDF)
p_at_most_6 = stats.binom.cdf(6, n, p)
print("P(6 or fewer heads):", p_at_most_6.round(4))

# Simulate it: 10,000 experiments of "flip a coin 10 times, count heads"
rng = np.random.default_rng(seed=42)
experiments = rng.binomial(n=n, p=p, size=10000)
print("Simulated mean heads:", experiments.mean().round(2), " (theory: n*p =", n * p, ")")

✨

Binomial mean and variance have simple closed forms

A binomial distribution's mean is always n * p, and its variance is always n * p * (1 - p) — no summing required. For 10 fair coin flips, the expected number of heads is exactly 5, which is why rng.binomial(10, 0.5, size=10000).mean() lands very close to 5.0.

Section 4

The Uniform Distribution

The uniform distribution is the simplest of the three: every value in a range is equally likely. A perfectly fair die roll (discrete uniform) or a random float between 0 and 1 (continuous uniform) are the standard examples — it's the shape you get from a "pick any value in this range, with no preference" process.

Continuous Uniform PDF, on [a, b]

f(x) = 1 / (b − a) for a ≤ x ≤ b, else 0

Flat and constant across the whole range [a, b] — a rectangle, not a curve.

uniform_distribution.py

PYTHON

# 10,000 random draws, uniformly between 20 and 30
rng = np.random.default_rng(seed=42)
samples = rng.uniform(low=20, high=30, size=10000)

print("Sample mean:", np.mean(samples).round(2), " (theory: (a+b)/2 = 25.0)")

# scipy.stats.uniform is parameterized as (loc, scale) = (a, b - a), not (a, b)
x = np.linspace(18, 32, 200)
pdf = stats.uniform.pdf(x, loc=20, scale=10)   # scale = b - a = 10

fig, ax = plt.subplots(figsize=(7, 4))
ax.hist(samples, bins=30, density=True, alpha=0.5, color="#fbbf24")
ax.plot(x, pdf, color="#fb923c", linewidth=2.5)
plt.show()

⚠️

scipy.stats.uniform's parameters are easy to get wrong

Unlike rng.uniform(low=20, high=30), stats.uniform takes loc (the start, a) and scale (the width of the range, b − a) — not the end point b directly. For a range of 20 to 30, that's loc=20, scale=10, not scale=30. This same loc/scale convention applies across most of scipy.stats, including norm, where it maps to mean and standard deviation instead.

🧩 Knowledge Check — Lesson 20

3 questions on probability distributions before you move on.

1. According to the empirical rule, roughly what percentage of a normal distribution's values fall within 2 standard deviations of the mean?

2. Which distribution correctly models "the number of heads in 20 independent coin flips"?

3. What is the key difference between a discrete variable's PMF and a continuous variable's PDF?

💪

Try It Yourself — Lesson 20

Simulate and compare · Intermediate Level

Adult male height is often modeled as roughly normal with mean 175 cm and standard deviation 7 cm — an illustrative approximation for practice, not a precise population statistic.

Task 1: Simulate 5,000 heights 📏

Use rng.normal(loc=175, scale=7, size=5000) to generate simulated heights. Compute the sample mean and standard deviation with NumPy — how close do they land to the true 175 and 7?

Task 2: Apply the empirical rule by hand, then check it 🧮

Without running code, predict what range covers ~95% of heights (mean ± 2×std). Then verify with stats.norm.cdf(189, 175, 7) - stats.norm.cdf(161, 175, 7) — how close is it to 0.95?

Task 3: Find an unusually tall cutoff 🚩

Use stats.norm.ppf(0.99, loc=175, scale=7) — the inverse of cdf() — to find the height below which 99% of the simulated population falls. What does that tell you about how rare someone taller than that cutoff would be?

💡 Show hints if you're stuck

Task 1: heights = rng.normal(loc=175, scale=7, size=5000), then heights.mean(), heights.std(ddof=1)
Task 2: The empirical rule predicts roughly [175 - 14, 175 + 14] = [161, 189] for ~95%.
Task 3: ppf stands for "percent point function" — it's the inverse of the CDF, converting a probability back into a value.

Finished this lesson?

Mark it complete to track your progress.

Module 20 of 23 Section 4 — Statistics & Probability for Data Science

Probability Distributions

What Is a Probability Distribution?

The Normal (Gaussian) Distribution

The Binomial Distribution

The Uniform Distribution

Lesson 20 Complete!