K-Means Clustering

11.1Unsupervised learning and clustering

In Module 1 you met three types of machine learning. Modules 3 to 10 were all supervised: every training example came with the right answer (a price, "spam", "left"). Now we move to unsupervised learning, where there are no answers at all.

Without answers, we can't predict a label. Instead, we ask a different question: what natural groups exist in this data? Finding those groups is called clustering.

In this module, we'll help a UK online shop understand its customers. It knows what each customer spends and how often they order, but nobody has ever grouped them into "types". K-Means will find those types for us.

11.2How K-Means works

K-Means is the most popular clustering algorithm. The "K" is the number of groups you want, and "means" refers to averages: each group is represented by the average (centre) of its points, called a centroid.

That's the whole algorithm. It's simple, fast and works surprisingly well. Try it step by step:

Try a few "new random starts" with k = 4. Sometimes the centroids find the four obvious groups; occasionally two centroids get stuck sharing one group. That leads to an important practical point in lesson 11.4.

11.3Choosing the number of clusters

K-Means needs you to choose k up front. But if nobody knows the groups, how do you know how many there are? There are two common tools.

The elbow method

Inertia measures how tightly packed the clusters are: the total squared distance from every point to its centroid. Smaller is tighter. But inertia always falls as k increases (with one cluster per point, it would be zero!), so we don't just pick the smallest. Instead, we look for the point where adding another cluster stops helping much: the elbow of the curve.

The silhouette score

The silhouette score asks, for every point: am I much closer to my own cluster than to the next nearest one? It runs from −1 to 1. Close to 1 means clear, well-separated clusters. Close to 0 means clusters overlap. Pick the k with the highest score.

11.4Scaling and starting positions

Always scale your features

K-Means uses distances, just like KNN. If annual spend is in hundreds of pounds and orders per year is in single figures, spend will completely dominate, and the clusters will be based on spend alone. Scale your features first (Module 2).

Starting positions can change the result

As you may have seen in the lab, a bad starting position can leave K-Means stuck with a poor answer. scikit-learn protects you in two ways, and both are switched on by default:

11.5From clusters to customer segments

K-Means only gives each customer a number: cluster 0, 1, 2 or 3. The numbers mean nothing on their own. The real value comes when you describe each cluster by looking at its averages, then give it a name a business can use.

Here are the four clusters found in the Python example, sorted by spend:

Now a marketing team has four clear groups, each with a different plan. That's the kind of insight that makes clustering so valuable in data analyst and BI roles.

11.6When K-Means fails

K-Means assumes clusters are roughly round blobs of similar size. When groups have other shapes, it struggles, because it always splits the space with straight boundaries between centroids.

Other weaknesses of K-Means: outliers can drag a centroid away from its group, and you must choose k in advance.

11.7Customer segmentation in Python

We'll create 1,200 customers with three features: annual spend, orders per year and days since their last order. (These are close to the "RFM" features, recency, frequency and monetary value, that retailers really use.) We hide four groups in the data, then see if K-Means can find them without being told.

import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

# 1. 1,200 customers of a UK online shop (no labels!)
rng = np.random.default_rng(11)
groups = [  # (how many, spend £, orders per year, days since last order)
    (300, 1800, 24, 12),   # these hidden groups are what we hope to find
    (400,  350,  6, 40),
    (300,  900,  3, 200),
    (200,  150,  2, 320),
]
rows = []
for size, spend, orders, days in groups:
    rows.append(pd.DataFrame({
        "annual_spend":  rng.normal(spend, spend * 0.25, size).clip(20),
        "orders":        rng.normal(orders, orders * 0.3, size).clip(1).round(),
        "days_since":    rng.normal(days, days * 0.25, size).clip(1).round(),
    }))
df = pd.concat(rows, ignore_index=True)

# 2. Scale: spend is in hundreds, orders in single figures
X = StandardScaler().fit_transform(df)

# 3. Try k = 2 to 7 and compare inertia and silhouette
for k in range(2, 8):
    km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(X)
    sil = silhouette_score(X, km.labels_)
    print(f"k={k}  inertia={km.inertia_:7.0f}  silhouette={sil:.3f}")

# 4. Fit the chosen model and describe each cluster
km = KMeans(n_clusters=4, n_init=10, random_state=42).fit(X)
df["cluster"] = km.labels_
profile = df.groupby("cluster").agg(
    customers=("annual_spend", "size"),
    avg_spend=("annual_spend", "mean"),
    avg_orders=("orders", "mean"),
    avg_days_since=("days_since", "mean"),
).round(0).sort_values("avg_spend", ascending=False)
print(profile)

k=2  inertia=   1441  silhouette=0.598
k=3  inertia=    710  silhouette=0.579
k=4  inertia=    459  silhouette=0.623
k=5  inertia=    356  silhouette=0.566
k=6  inertia=    303  silhouette=0.550
k=7  inertia=    258  silhouette=0.548
         customers  avg_spend  avg_orders  avg_days_since
cluster                                                  
2              294     1823.0        24.0            12.0
1              288      921.0         3.0           204.0
0              421      361.0         6.0            43.0
3              197      164.0         2.0           323.0

What the output tells us

• Inertia drops sharply from k = 2 to 4 (1,441 → 459), then only slowly. That's the elbow.
• The silhouette score peaks at k = 4 (0.623). Both tools point to four groups.
• The cluster profiles almost exactly match the four hidden groups we built into the data (for example, the top cluster averages £1,823 spend, 24 orders and 12 days, against the hidden values of £1,800, 24 and 12). K-Means found them with no labels at all.
• The cluster numbers (0 to 3) are arbitrary. Run it with a different random_state and the groups might be numbered differently, which is why we sort and name them ourselves.

11.8Strengths, weaknesses and real-world uses

SummaryKey takeaways

Check your understanding

Your machine learning roadmap

1. 01
   Introduction to Machine LearningWhat ML is and how it works
2. 02
   Preparing Data for Machine LearningFeatures, encoding, scaling, train/test split
3. 03
   Linear RegressionPredicting numbers
4. 04
   Logistic RegressionPredicting yes or no
5. 05
   Evaluating ModelsAccuracy, precision, recall, overfitting, cross-validation
6. 06
   K-Nearest NeighboursLearning from similar examples
7. 07
   Decision TreesFlowcharts that learn
8. 08
   Random Forest and Gradient BoostingMany models working together
9. 09
   Support Vector MachinesFinding the best boundary
10. 10
    Naive BayesProbability-based classification
11. 11
    Clustering with K-MeansYou are here
12. 12
    Dimensionality Reduction with PCASimplifying big datasets
13. 13
    Capstone ProjectBuild and present a full ML project