#Lecture 007 — Clustering: K-Means

Everything since Lecture 005 has been supervised — labeled data with a right answer. K-Means is our first unsupervised algorithm: it finds groups in data that has no labels at all.

Contents

Prerequisites · Connection · Objectives · Lesson · Code Examples · Common Mistakes · Best Practices · Mini Challenge · Practice Questions · Answers · Takeaways

#Prerequisites

• Lecture 001 — the unsupervised learning category
• Lecture 004 — feature scaling (K-Means is distance-based, like KNN)

#Connection With Previous Lectures

Lecture 001 introduced clustering as part of unsupervised learning; this lecture is the first algorithm in that category, and it reuses the "distance-based models need scaled features" rule from Lecture 004 and the "choosing K" theme from KNN in Lecture 006.

#Learning Objectives

By the end of this lecture you will be able to:

• Describe the four steps of the K-Means algorithm
• Explain what the elbow method is used for
• List two of K-Means' key limitations

#Lesson

K-Means groups unlabeled data into k clusters based on similarity (Euclidean distance). The algorithm:

1. Randomly initialize k centroids.
2. Assign each point to its nearest centroid.
3. Recompute each centroid as the mean of its assigned points.
4. Repeat steps 2–3 until centroids stop moving (or a max iteration count is hit).

Choosing the right k is a key challenge — the elbow method (plotting within-cluster error against k and looking for the "elbow," the point where adding more clusters stops helping much) is a common heuristic. K-Means assumes roughly spherical, similarly-sized clusters and is sensitive to outliers and to the random initial centroid placement — hence running it multiple times with different random seeds (n_init).

#Code Examples

#Example 1 — Clustering Synthetic Data

from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans

X, _ = make_blobs(n_samples=500, n_features=2, centers=3, random_state=23)
X_scaled = StandardScaler().fit_transform(X)

kmeans = KMeans(n_clusters=3, random_state=23, n_init=10)
labels = kmeans.fit_predict(X_scaled)

Note there's no y here at all — fit_predict only ever sees X. That's the defining difference from every supervised algorithm in Lectures 005–006.

#Common Beginner Mistakes

1. Assuming K-Means will "discover" the correct number of clusters on its own — k must be chosen beforehand (via the elbow method, domain knowledge, or silhouette score), and K-Means will happily produce k clusters even from data that has no real cluster structure at all.

2. Skipping feature scaling — K-Means is distance-based like KNN, so unscaled features distort which points look "close" to each other.

3. Running K-Means once and trusting the result — a bad random initialization can land on a poor local solution; n_init (multiple runs, keeping the best) guards against this.

#Best Practices

• Always scale features before K-Means.
• Use the elbow method (or silhouette score) to choose k, rather than guessing.
• Run with n_init > 1 so a single unlucky initialization doesn't determine the result.

#Mini Challenge

Run Example 1 with n_clusters set to 2, 3, and 5, and compare how the resulting clusters change — which value actually matches the three blobs the data was generated from?

#Practice Questions

1. What is the main difference between K-Means and the supervised algorithms from Lectures 005–006, in terms of what data it needs?
2. Describe the four steps of the K-Means algorithm in your own words.
3. What is the elbow method used for?
4. Name two limitations of K-Means.

#Practice Answers

1. K-Means needs no labels (y) — it only clusters based on the features (X).
2. Initialize k centroids randomly, assign each point to its nearest centroid, recompute each centroid as the mean of its points, repeat until centroids stop moving.
3. Choosing a good value of k, by plotting within-cluster error against k and looking for the point where adding more clusters stops helping much.
4. Any two of: assumes roughly spherical/similarly-sized clusters, sensitive to outliers, sensitive to random initial centroid placement, k must be chosen in advance.

#Key Takeaways

• K-Means is unsupervised — it clusters X with no y at all
• Four-step loop: initialize centroids, assign points, recompute centroids, repeat
• The elbow method (or silhouette score) helps choose k
• K-Means assumes spherical clusters and is sensitive to outliers and initialization