#Lecture 004 — Data Preprocessing

Raw data is messy — missing values, inconsistent formats, categorical text instead of numbers, features on wildly different scales. Preprocessing turns that into a clean, consistent form a model can actually learn from. Skipping this step is one of the most common reasons beginner models perform poorly.

Contents

Prerequisites · Connection · Objectives · Lesson · Code Examples · Common Mistakes · Best Practices · Mini Challenge · Practice Questions · Answers · Takeaways

#Prerequisites

• Lecture 003 — pandas basics and the scikit-learn fit()/transform() pattern

#Connection With Previous Lectures

Lecture 003 showed train_test_split producing clean X/y data. In practice, real data never starts that clean — this lecture is the step that happens before any model in Lectures 005–010 can be trained on it.

#Learning Objectives

By the end of this lecture you will be able to:

• List the four core preprocessing steps and what each one fixes
• Explain why feature scaling matters more for some algorithms than others
• Explain data leakage and how to avoid it when splitting and scaling

#Lesson

#Handling Missing Values

Impute with the column's mean/median/mode, or drop the affected rows/columns if too much is missing to impute reliably.

#Encoding Categorical Variables

Models need numbers, not text. One-hot encoding turns a category column (e.g., "city") into one 0/1 column per category; label encoding assigns each category an integer — simpler but implies an order that may not exist, so it's mainly used for naturally ordered categories.

#Feature Scaling

Normalize or standardize numeric features so they're on comparable ranges. This is critical for distance-based models (KNN, SVM — Lecture 006) and for any model trained with gradient descent (linear/logistic regression, neural networks) — unscaled features let one column dominate the distance or gradient purely because of its larger numeric range.

#Train/Test Split & Data Leakage

Split data into training and test sets before fitting any scaler or encoder. Data leakage is when information from the test set "leaks" into training — most commonly by fitting a scaler on the full dataset before splitting, which lets the model's performance estimate be quietly inflated by information it shouldn't have had access to.

#Code Examples

#Example 1 — A Typical Preprocessing Pipeline

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer

df = pd.read_csv("data.csv")

# Impute missing numeric values with the column mean
imputer = SimpleImputer(strategy="mean")
df[["age", "income"]] = imputer.fit_transform(df[["age", "income"]])

# One-hot encode a categorical column
df = pd.get_dummies(df, columns=["city"])

# Split before scaling to avoid leakage
X_train, X_test, y_train, y_test = train_test_split(
    df.drop("target", axis=1), df["target"], test_size=0.2, random_state=42
)

# Fit the scaler on train only, then transform both
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

Note the asymmetry on the last two lines: fit_transform on the training set, but plain transform on the test set. That one-word difference is the whole fix for leakage.

#Common Beginner Mistakes

1. Fitting a scaler/encoder on the entire dataset before splitting into train/test — this leaks test-set statistics into training and inflates apparent performance. Always fit preprocessing objects on the training set only, then transform (not fit_transform) the test set.

2. Skipping feature scaling for distance-based models — carried over directly into Lecture 006's KNN/SVM pitfalls.

3. One-hot encoding a column with hundreds of unique categories — this can explode the number of columns; high-cardinality categorical features often need a different strategy (e.g., grouping rare categories, or target/frequency encoding).

#Best Practices

• Always split before fitting any preprocessing object.
• Inspect missing-value counts per column before choosing an imputation strategy.
• Prefer one-hot encoding for unordered categories, label encoding only for genuinely ordered ones.

#Mini Challenge

Take any CSV with a numeric column that has missing values and a text category column, and write the four-step pipeline from Example 1 against it (impute → encode → split → scale).

#Practice Questions

1. Name the four core preprocessing steps covered in this lecture.
2. Why does feature scaling matter more for KNN than for a decision tree?
3. What is data leakage, and what's the most common way beginners cause it?
4. When should you prefer label encoding over one-hot encoding?

#Practice Answers

1. Handling missing values, encoding categorical variables, feature scaling, and the train/test split.
2. KNN classifies by distance between points, so a feature on a much larger numeric scale dominates the distance calculation unless all features are scaled comparably; decision trees split one feature at a time and aren't affected by scale.
3. Test-set information influencing the training process, most commonly by fitting a scaler or encoder on the full dataset (including the test portion) before splitting.
4. When the categories have a genuine order (e.g., "low/medium/high") rather than being unordered labels.

#Key Takeaways

• Preprocessing: handle missing values, encode categories, scale features, then split (fit scalers/encoders on train only)
• Distance-based and gradient-trained models need scaled features; tree-based models don't
• Data leakage quietly inflates reported performance — always fit on train, transform on test
• fit_transform vs. transform is the one-word difference that prevents leakage