Hierarchical Clustering

Build a family tree of your data instead of committing to one fixed number of clusters upfront.

Key Points

• Unlike K-Means, you don't need to choose the number of clusters before running it.
• Agglomerative (bottom-up) is far more common than divisive (top-down).
• Produces a dendrogram — a tree diagram showing how clusters merge at increasing distances.
• The 'linkage' method (single, complete, average, Ward) determines how the distance between clusters is measured.
• Computationally expensive on large datasets compared to K-Means.

What it is

Hierarchical clustering builds a tree-like structure (dendrogram) of nested clusters by either repeatedly merging the closest pairs of points/clusters (agglomerative, bottom-up) or repeatedly splitting the data (divisive, top-down). You decide the number of clusters afterward by cutting the tree at a chosen height.

Why it exists

K-Means forces you to commit to a fixed number of clusters upfront, which is often unknown. Hierarchical clustering exists to reveal the full nested structure of the data at every possible granularity, letting you decide how many clusters make sense after seeing the whole picture.

Intuition

How it works

1. Start with every point as its own cluster

   Treat each data point as a singleton cluster.

2. Merge the closest pair

   Find the two closest clusters (using the chosen linkage method) and merge them into one.

3. Repeat

   Keep merging the closest pair of clusters until only one cluster containing all points remains.

4. Cut the dendrogram

   Choose a height to 'cut' the resulting tree, which determines the final number of clusters.

When to use

• Use when you don't know the number of clusters in advance and want to explore structure at multiple levels.
• Use on small-to-medium datasets where a dendrogram is interpretable and computation is feasible.
• Use when a nested/hierarchical relationship between groups genuinely makes sense (e.g. taxonomy, organizational structure).

When not to use

• Do not use on very large datasets — it's computationally expensive (typically O(n^2) or worse).
• Do not use single-linkage on noisy data — it's prone to 'chaining,' where clusters merge through thin bridges of noise.

Common mistakes

• Choosing linkage method arbitrarily without understanding how it changes cluster shape (single linkage chains, Ward tends toward compact clusters).
• Cutting the dendrogram at an arbitrary height without a principled reason.
• Running it on a large dataset and being surprised by how slow it is.

In practice

Used in bioinformatics (gene expression clustering), document/topic organization, and any exploratory analysis where understanding nested group structure matters more than getting one fixed number of clusters quickly.

Related concepts