🧠 Classical ML · Tier 2 🟡 Intermediate MODULE 04

Unsupervised: Clustering

⏱️ 3 hours

🫧 Unsupervised

🧩 5 Quiz Questions

📦 5 Sections

Classical ML — Module 4 of 100%

🎯 What you'll learn: How K-Means works (the EM loop), choosing K with the elbow method and silhouette score, hierarchical clustering and dendrograms, DBSCAN for arbitrary-shape clusters with noise, and a customer segmentation project tying it all together.

4.1 · K-Means

Partition Data into K Groups

K-Means is an iterative EM algorithm. It alternates between two steps until convergence: (1) Assign each point to its nearest centroid, and (2) Move each centroid to the mean of its assigned points. It minimises the within-cluster sum of squared distances (inertia).

The algorithm is sensitive to initialisation. sklearn uses k-means++ by default, which seeds centroids far apart from each other, dramatically reducing the chance of poor convergence.

K-Means — fit, predict, key attributesPYTHON

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
import numpy as np

# Always scale before K-Means — it uses Euclidean distance
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

kmeans = KMeans(
    n_clusters=5,
    init='k-means++',  # smart seeding (default)
    n_init=10,         # run 10 times, keep best inertia
    random_state=42
)
labels = kmeans.fit_predict(X_scaled)

print(f"Inertia: {kmeans.inertia_:.1f}")        # within-cluster SSE
print(f"Centroids shape: {kmeans.cluster_centers_.shape}")
print(f"Iterations to converge: {kmeans.n_iter_}")

# Predict cluster for new data
new_cluster = kmeans.predict(scaler.transform(X_new))

4.2 · Choosing K

Elbow Method and Silhouette Score

K-Means needs K as input. There's no single correct answer, but two metrics help:

Elbow method: plot inertia vs K. Look for where adding another cluster gives diminishing returns — the "elbow" of the curve. Subjective but quick.

Silhouette score: for each point, measures how similar it is to its own cluster vs. the nearest other cluster. Ranges −1 (wrong cluster) to +1 (well-separated). Higher = better. More objective than the elbow method.

Finding optimal K with both methodsPYTHON

from sklearn.metrics import silhouette_score

inertias, sil_scores = [], []
K_range = range(2, 12)

for k in K_range:
    km = KMeans(n_clusters=k, random_state=42, n_init=10)
    labels = km.fit_predict(X_scaled)
    inertias.append(km.inertia_)
    sil_scores.append(silhouette_score(X_scaled, labels))

best_k = K_range[np.argmax(sil_scores)]
print(f"Best K by silhouette: {best_k}")
print(f"Best silhouette score: {max(sil_scores):.3f}")

# Rule of thumb: silhouette > 0.5 = reasonable clusters
# silhouette > 0.7 = strong structure in the data

⚠️

K-Means assumes spherical, equal-sized clusters

K-Means will always produce K clusters, even when the true structure is very different — e.g., elongated shapes, varying densities, or clusters with outliers. If your data doesn't look like roughly spherical blobs, consider DBSCAN or Gaussian Mixture Models instead.

4.3 · Hierarchical Clustering

Build a Tree of Clusters

Hierarchical clustering builds a dendrogram — a tree showing how clusters merge (agglomerative, bottom-up) or split (divisive, top-down). You don't choose K upfront; instead you cut the dendrogram at any level to get any number of clusters. Useful for exploring data when you don't know K.

Agglomerative clustering and dendrogramPYTHON

from sklearn.cluster import AgglomerativeClustering
from scipy.cluster.hierarchy import dendrogram, linkage
import matplotlib.pyplot as plt

# linkage='ward' minimises within-cluster variance (most common)
agg = AgglomerativeClustering(
    n_clusters=4,
    linkage='ward'
)
labels = agg.fit_predict(X_scaled)

# Visualise with a scipy dendrogram
Z = linkage(X_scaled, method='ward')
plt.figure(figsize=(12, 5))
dendrogram(Z, truncate_mode='lastp', p=20)
plt.title('Hierarchical Clustering Dendrogram')
plt.xlabel('Sample index')
plt.ylabel('Distance (Ward)')
plt.show()
# The horizontal line where you cut = the K you get

4.4 · DBSCAN

Density-Based Clustering — Handles Noise and Odd Shapes

DBSCAN (Density-Based Spatial Clustering of Applications with Noise) groups points that are densely packed together and marks isolated points as noise (−1). It requires no K and finds arbitrarily shaped clusters. The two key parameters:

eps: the maximum distance between two points to be considered neighbours
min_samples: minimum points in a neighbourhood to form a core point

DBSCAN — noise detection and arbitrary clustersPYTHON

from sklearn.cluster import DBSCAN
from sklearn.neighbors import NearestNeighbors

# Tip: estimate eps with k-distance graph
# eps ≈ where k-distance curve shows a "knee"
nbrs = NearestNeighbors(n_neighbors=5).fit(X_scaled)
distances, _ = nbrs.kneighbors(X_scaled)
distances = np.sort(distances[:, -1])  # distance to 5th neighbour

db = DBSCAN(eps=0.5, min_samples=5)
labels = db.fit_predict(X_scaled)

n_clusters = len(set(labels)) - (1 if -1 in labels else 0)
n_noise    = list(labels).count(-1)
print(f"Clusters: {n_clusters}, Noise points: {n_noise}")

K-Means

✓ Fast, scalable, simple

✓ Works well on spherical blobs

✗ Must choose K upfront

✗ Sensitive to outliers

Hierarchical

✓ No K needed upfront

✓ Dendrogram is interpretable

✗ O(n²) — slow on large data

✗ Cannot update incrementally

DBSCAN

✓ Finds arbitrary shapes

✓ Labels noise explicitly

✗ Sensitive to eps, min_samples

✗ Struggles with varying density

4.5 · Project: Customer Segmentation

RFM Clustering: Who Are Your Best Customers?

Customer segmentation with RFM (Recency, Frequency, Monetary value) is a classic K-Means application. Each customer becomes a point in 3D feature space; clusters become business segments like "loyal high-value" or "at-risk dormant".

RFM customer segmentation end-to-endPYTHON

import pandas as pd
import numpy as np
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score

# Build RFM features from transaction data
snapshot_date = df['date'].max() + pd.Timedelta(days=1)
rfm = df.groupby('customer_id').agg(
    Recency   = ('date',   lambda x: (snapshot_date - x.max()).days),
    Frequency = ('order_id', 'nunique'),
    Monetary  = ('amount',   'sum')
).reset_index()

# Log-transform monetary to reduce skew
rfm['Monetary'] = np.log1p(rfm['Monetary'])

# Scale and cluster
X = rfm[['Recency', 'Frequency', 'Monetary']]
X_sc = StandardScaler().fit_transform(X)

km = KMeans(n_clusters=4, random_state=42, n_init=10)
rfm['Segment'] = km.fit_predict(X_sc)
print(f"Silhouette: {silhouette_score(X_sc, rfm.Segment):.3f}")

# Profile each segment
print(rfm.groupby('Segment')['Recency','Frequency','Monetary'].mean().round(1))

💡

Naming clusters is a business skill

K-Means outputs numbers (0, 1, 2…). Your job is to look at the centroid means and assign meaningful names: "Champions" (low recency, high frequency, high spend), "At Risk" (high recency, previously high frequency), "New Customers" (low recency, low frequency), etc. This translation from math to business narrative is what makes the work useful.

🧩 Module 4 Check

5 questions · instant feedback

1. What does K-Means minimise during training?

2. A silhouette score of −0.2 for a point means:

3. Which clustering algorithm explicitly identifies and labels outliers as noise?

4. What does k-means++ initialisation improve compared to random centroid placement?

5. You have customer transaction data with natural crescent-shaped purchase pattern clusters. Which algorithm is most appropriate?

← Module 3: SVMs Module 5: Dimensionality Reduction →