🤖 Section 5 · Machine Learning 🟡 Intermediate MODULE 28

K-Means Clustering — Unsupervised Learning

⏱️ 45 min

📖 Finding Groups With No Labels

🧩 3 Quiz Questions

🏗️ 1 Challenge · 3 tasks

Your progress in Section 571%

🎯 Back to unsupervised learning, from Lesson 24. Every model so far — linear regression, logistic regression, decision trees, random forests — was supervised: it learned from data that already had the correct answer attached. K-Means clustering is different. It's handed data with no labels at all, and its job is to discover natural groups on its own — useful for things like customer segmentation, where you often don't know the "right" groups in advance.

Section 1

What Is Clustering?

Clustering is the task of grouping data points so that points in the same group are more similar to each other than to points in other groups — without ever being told what the groups should be. There's no y to fit against, only X.

🛍️

Customer segmentation

Grouping customers by spending habits and demographics to target marketing differently per group.

🖼️

Image compression

Reducing an image's color palette by clustering similar pixel colors together.

🚨

Anomaly detection

Points that don't fit well into any cluster can be flagged as unusual or worth investigating.

🔬

Exploratory grouping

When you suspect a dataset contains distinct subgroups but don't yet know what they are — clustering can reveal a starting hypothesis.

📝

There's no "correct" answer to check against

This is the biggest mental shift from Lessons 25-27: with no true labels, there's no accuracy or F1-score to compute. Evaluating a clustering result is more about whether the groups are useful and make sense than about matching a ground truth — a genuinely different kind of problem from supervised classification.

Section 2

The K-Means Algorithm

K-Means is the most widely used clustering algorithm, and its idea is refreshingly simple: pick K "centroids" (group centers), then repeatedly reassign points to their nearest centroid and recompute the centroids, until nothing changes.

Choose K

Decide how many clusters to look for — this is the one number you must supply up front (Section 3 covers how to pick it).

Initialize K random centroids

Place K points (the "centroids") somewhere in the feature space, either randomly or via a smarter initialization scikit-learn handles automatically.

Assign each point to its nearest centroid

Every data point joins the cluster of whichever centroid it's closest to (typically by straight-line/Euclidean distance).

Update centroids to the mean of their cluster

Each centroid moves to the average position of all the points now assigned to it.

Repeat steps 3-4 until convergence

Keep reassigning and recentering until the assignments stop changing (or a maximum iteration count is hit).

✨

"Centroid" just means "the average point"

A centroid isn't necessarily an actual data point — it's the mean position of every point currently assigned to that cluster, in every feature's dimension at once. As points get reassigned between clusters, the centroids drift toward the true center of whatever points end up grouped together.

Section 3

Choosing K — the Elbow Method

Unlike a classifier's number of classes (which the labeled data tells you), K-Means requires you to decide K yourself, before fitting. The elbow method is the standard heuristic for picking a reasonable value.

Inertia (Within-Cluster Sum of Squares)

Inertia = Σ (distance from each point to its cluster's centroid)²

Lower inertia means points sit closer to their centroids — tighter, more compact clusters. Inertia always decreases as K increases (more clusters can always fit the data at least as tightly), which is exactly why you can't just pick the K with the lowest inertia — that would always be K = number of data points.

Fit KMeans for a range of K values

Typically K = 1 through 10, recording each model's .inertia_ after fitting.

Plot inertia vs. K

Inertia drops sharply at first as K grows, then the rate of improvement slows down.

Find the "elbow"

The point where the curve bends and adding more clusters stops giving much benefit — that bend, shaped like an elbow, suggests a reasonable K.

📉 Illustrative elbow plot shape

Inertia typically falls steeply from K=1 to K=2, continues dropping from K=2 to K=3, then the drop from K=3 onward becomes much smaller and flatter — that visual "elbow" around K=3 would be a reasonable signal to pick K=3, in this illustrative shape. The real elbow position depends entirely on the actual dataset being clustered — this is a description of the general pattern to look for, not a specific claimed result.

⚠️

The elbow method is a heuristic, not an exact rule

Real elbow plots are often less crisp than the textbook picture — the "bend" can be subtle or ambiguous. When that happens, domain knowledge (e.g. "we plan to run exactly 4 marketing campaigns, so K=4 is practical regardless") is a perfectly valid way to pick K alongside, or instead of, a fuzzy elbow.

Section 4

sklearn.cluster.KMeans

This example clusters illustrative customers by two features: age and a 0-100 spending score. Both features get standardized first, for a reason explained right after the code.

kmeans_customers.py

PYTHON

import numpy as np
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

# Illustrative customer data: [age, spending_score (0-100)]
X = np.array([
    [25, 80], [27, 75], [22, 85], [30, 70],
    [43, 20], [45, 15], [48, 22], [50, 18],
    [62, 55], [65, 60], [60, 50], [58, 58],
])

# Scale features first — see the info box right below for why this matters
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# n_init='auto' (or n_init=10 on older scikit-learn) runs the algorithm
# multiple times with different random starting centroids and keeps the best result
kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
labels = kmeans.fit_predict(X_scaled)

print("Cluster assignments:", labels)
print("Points per cluster:", np.bincount(labels))
print("Inertia:", kmeans.inertia_)

# Centroids are in SCALED space — inverse_transform to read them in original units
centroids_original = scaler.inverse_transform(kmeans.cluster_centers_)
print("Centroids [age, spending_score]:\n", centroids_original.round(1))

⚠️

Always scale features before K-Means

K-Means groups points by raw distance — and age (ranging roughly 20-65) and spending score (0-100) sit on very different numeric scales. Without scaling, the feature with the larger range would dominate the distance calculation almost entirely, effectively ignoring the other feature. StandardScaler (from Lesson 25's ecosystem) puts every feature on a comparable scale first, exactly like it's used before other distance-sensitive algorithms.

📝

fit_predict() vs. separate fit() and predict()

kmeans.fit_predict(X_scaled) is a convenience shortcut for calling .fit(X_scaled) followed by .predict(X_scaled) — it fits the model AND returns the cluster label for each training point in one call. For genuinely new points collected later, kmeans.predict(new_points) assigns them to whichever already-fitted centroid is nearest, without moving the centroids again.

Section 5

Finding K With the Elbow Method, in Code

Putting Section 3's idea into practice: fit KMeans across a range of K values and inspect the resulting inertias.

elbow_method.py

PYTHON

import matplotlib.pyplot as plt

inertias = []
K_range = range(1, 8)

for k in K_range:
    km = KMeans(n_clusters=k, random_state=42, n_init=10)
    km.fit(X_scaled)
    inertias.append(km.inertia_)

print(list(zip(K_range, [round(i, 2) for i in inertias])))

fig, ax = plt.subplots(figsize=(7, 5))
ax.plot(list(K_range), inertias, marker='o', color="#4f9eff")
ax.set_xlabel("Number of clusters (K)")
ax.set_ylabel("Inertia")
ax.set_title("Elbow Method for Choosing K")
plt.show()

# Once K is chosen (say, K=3 based on the elbow), fit the final model
final_model = KMeans(n_clusters=3, random_state=42, n_init=10)
final_labels = final_model.fit_predict(X_scaled)

🎨 Illustrative scatter plot with cluster colors

plt.scatter(X[:,0], X[:,1], c=labels, cmap='viridis') followed by plt.scatter(centroids_original[:,0], centroids_original[:,1], marker='x', s=200, color='red') would plot each customer as a point colored by its assigned cluster, with red X markers at each centroid — visually confirming whether the three clusters correspond to sensible, separated groups (in this illustrative data: younger big spenders, older low spenders, and a middle group).

🧩 Knowledge Check — Lesson 28

3 questions on K-Means clustering before you move on.

1. What is a "centroid" in K-Means?

2. Why is it a bad idea to pick K by simply choosing whichever value gives the lowest inertia?

3. Why should features typically be scaled (e.g. with StandardScaler) before running K-Means?

💪

Try It Yourself — Lesson 28

Cluster your own data · Intermediate Level

Reuse the customer X array from Section 4 for all three tasks.

Task 1: Try K=2 and K=4 🔢

Fit KMeans with n_clusters=2, then again with n_clusters=4, both on the scaled data. Print the cluster sizes with np.bincount(labels) for each. Do either grouping seem more "natural" than the K=3 result from Section 4?

Task 2: Skip the scaling, on purpose ⚠️

Fit KMeans(n_clusters=3, random_state=42, n_init=10) directly on the raw, UNscaled X instead of X_scaled. Compare the resulting cluster assignments to the scaled version from Section 4. Are they the same? What does that tell you about the warning in Section 4?

Task 3: Predict a brand-new customer 🧑

Using the fitted kmeans model from Section 4, predict which cluster a new 35-year-old customer with a spending score of 90 belongs to. Remember: scale the new point with the SAME already-fitted scaler before calling .predict() — never call .fit_transform() again on new data.

💡 Show hints if you're stuck

Task 1: KMeans(n_clusters=2, random_state=42, n_init=10).fit_predict(X_scaled) — repeat with n_clusters=4.
Task 2: KMeans(n_clusters=3, random_state=42, n_init=10).fit_predict(X) — pass the raw X, not X_scaled.
Task 3: new_point = scaler.transform([[35, 90]]); kmeans.predict(new_point) — note .transform(), not .fit_transform(), since the scaler was already fit on the training data.

Finished this lesson?

Mark it complete to track your progress.

Module 28 of 30 Section 5 — Machine Learning with Scikit-Learn