Clustering

Grouping similar things together when nobody told you what the groups were.

Key Points

• Unsupervised: you don't provide labels. The algorithm finds groups based on similarity in the feature space.
• K-means: partition into k clusters by minimizing within-cluster distance. Requires you to specify k upfront.
• DBSCAN: density-based. Finds clusters of arbitrary shape and automatically marks outliers as noise. Doesn't need k upfront.
• Hierarchical clustering: builds a tree of nested clusters (dendrogram). You decide where to cut the tree to get your groups.
• Choosing the number of clusters is one of the hardest parts. The elbow method and silhouette score help — but they're not definitive.

What it is

Clustering is an unsupervised technique that groups data points so that points in the same group are more similar to each other than to points in other groups. Unlike classification, there are no predefined labels — the algorithm discovers the structure on its own. K-means, DBSCAN, and hierarchical clustering are the most common methods.

Why it exists

Sometimes you don't know what the groups are — you just suspect the data has structure. Customer segmentation is the classic case. You have behavioral data for millions of customers and you want to find natural groups that behave similarly, without pre-defining what those groups look like. Clustering lets the data suggest the answer.

Intuition

How it works

1. Choose k

   Decide how many clusters you want. Use the elbow method or silhouette score to guide the choice.

2. Initialize centroids

   Pick k random points from the dataset as starting cluster centers.

3. Assign points

   Each data point is assigned to the nearest centroid.

4. Update centroids

   Recalculate each centroid as the mean of all points assigned to it.

5. Repeat

   Repeat steps 3 and 4 until assignments stop changing or a maximum number of iterations is reached.

When to use

• Customer segmentation — find natural groups with similar purchasing or behavioral patterns.
• Document grouping — cluster news articles or support tickets by topic.
• Image compression — K-means can reduce colors by replacing each pixel with its cluster centroid.
• Anomaly detection — points that don't fit any cluster (in DBSCAN) are flagged as outliers.

When not to use

• When clusters are not actually present in the data. Clustering will always produce groups even if the data is uniformly distributed.
• K-means doesn't handle non-spherical cluster shapes well. DBSCAN handles arbitrary shapes.

Common mistakes

• Not scaling features before clustering. K-means uses Euclidean distance, so a feature measured in thousands will dominate over one measured in single digits.
• Treating the number of clusters as a definitive answer. The elbow method gives you a hint, not a ground truth.
• Evaluating clustering quality by eye on 2D plots after PCA reduction. The clusters in original feature space may not look clean in 2D.

Notes

• Cluster labels are arbitrary — cluster 1 and cluster 2 are just labels. What matters is which points are grouped together, not the number assigned to the group.
• K-means results are sensitive to initialization. Running with multiple random starts (k-means++) helps avoid bad local minima.
• Gaussian Mixture Models (GMM) are a probabilistic extension of K-means — instead of hard assignment, each point has a probability of belonging to each cluster.

In practice

Clustering is used in customer segmentation, recommendation systems (finding users with similar taste), document organization, image segmentation, and bioinformatics (grouping genes by expression pattern). It also appears as a preprocessing step — run clustering first, then build a model per cluster.

Related concepts