Hierarchical Clustering in Machine Learning
Introduction
Hierarchical Clustering is another important algorithm in Machine Learning used to group data into clusters. Unlike K-Means, it does not require you to specify the number of clusters in advance.
In this lesson, you will learn how Hierarchical Clustering works, its types, and how to interpret dendrograms.
What is Hierarchical Clustering?
Hierarchical Clustering is an unsupervised learning algorithm that builds a hierarchy of clusters.
It groups data step by step, creating a tree-like structure.
Example
Grouping customers based on purchasing behavior without knowing how many groups exist.
Types of Hierarchical Clustering
Agglomerative Clustering (Bottom-Up)
- Starts with each data point as its own cluster
- Merges closest clusters step by step
- Continues until all points are grouped
Divisive Clustering (Top-Down)
- Starts with one large cluster
- Splits it into smaller clusters
- Continues until each data point is separate
Agglomerative clustering is more commonly used.
How Hierarchical Clustering Works
- Treat each data point as a cluster
- Compute distances between clusters
- Merge the closest clusters
- Repeat until one cluster remains
Distance Measures
Distance plays a key role in clustering.
Common Methods
- Euclidean Distance
- Manhattan Distance
- Cosine Similarity
Linkage Methods
Linkage defines how distance between clusters is calculated.
Types
- Single Linkage (minimum distance)
- Complete Linkage (maximum distance)
- Average Linkage (average distance)
What is a Dendrogram?
A dendrogram is a tree-like diagram used to visualize hierarchical clustering.
It shows how clusters are merged step by step.
Key Insight
You can decide the number of clusters by cutting the dendrogram at a certain level.
Advantages of Hierarchical Clustering
- No need to predefine number of clusters
- Easy to visualize with dendrogram
- Works well with small datasets
Limitations of Hierarchical Clustering
- Computationally expensive
- Not suitable for large datasets
- Sensitive to noise and outliers
Implementation in Python
from scipy.cluster.hierarchy import dendrogram, linkage
import matplotlib.pyplot as plt
X = [[1,2], [1,4], [10,2], [10,4]]
Z = linkage(X, method=’ward’)
dendrogram(Z)
plt.show()
Real-World Applications
- Customer segmentation
- Document clustering
- Gene analysis
- Image segmentation
When to Use Hierarchical Clustering
- When number of clusters is unknown
- When dataset is small
- When visualization is important
Conclusion
Hierarchical Clustering provides a flexible way to group data and understand relationships between clusters. It is especially useful when you want a visual representation of clustering.
In the next lesson, you will learn about Dimensionality Reduction using PCA.
FAQs
What is Hierarchical Clustering?
It is an algorithm that builds a hierarchy of clusters.
What is a dendrogram?
It is a tree diagram that shows how clusters are formed.
What is the difference between K-Means and Hierarchical Clustering?
K-Means requires predefined clusters, while hierarchical does not.
Which type of hierarchical clustering is commonly used?
Agglomerative clustering is the most commonly used.
Is hierarchical clustering good for large datasets?
No, it is better suited for smaller datasets.
Internal Link
To explore more courses and improve your skills, click here for more free courses