4.2 — Choosing K: Elbow Method and Silhouette Score

K-Means requires you to specify k up front — choosing it well is its own problem, separate from running the algorithm itself.

Inertia (within-cluster sum of squares, WCSS)

The sum of squared distances from each point to its assigned centroid. Inertia always decreases as k increases — more clusters can only fit the data at least as well — so you can't just pick the k that minimizes it. That's always the largest k you try, which isn't a useful answer.

The elbow method

Plot inertia against k. Early increases in k reduce inertia sharply; past some point, additional clusters give diminishing returns and the curve flattens. The "elbow" — where the curve bends — is a candidate k.

Silhouette score

A more principled per-point metric. For each point, compare its average distance to points in its own cluster (cohesion) against its average distance to points in the nearest other cluster (separation). The score ranges from -1 to 1:

• Near 1 — well-clustered
• Near 0 — sitting on a cluster boundary
• Negative — likely misassigned to the wrong cluster

Average across all points to get a single score per choice of k; pick the k that maximizes it.

Silhouette plots

Go further than the average alone by showing per-point silhouette values for each cluster, sorted within each cluster. This reveals more than a single number can: a cluster might have a decent average score while containing many borderline or negative points — a defect the elbow method or a bare average can't surface.

from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

inertias, sil_scores = [], []
for k in range(2, 10):
    km = KMeans(n_clusters=k, random_state=42).fit(X)
    inertias.append(km.inertia_)
    sil_scores.append(silhouette_score(X, km.labels_))

best_k = range(2, 10)[sil_scores.index(max(sil_scores))]

Takeaways

• Inertia alone never tells you the right k — it's monotonically decreasing by construction.
• The elbow plot gives a rough range; silhouette score gives a more defensible specific value within that range.
• Use both together — neither metric alone is fully reliable, and a silhouette plot can catch problems a single average score hides.