Data Preparation & Feature Engineering

2.1Why data preparation matters

Imagine a chef preparing a meal. Before any cooking starts, they wash the vegetables, throw away anything rotten, and chop everything to the right size. If they skip this, even the best recipe in the world will produce a bad meal.

Machine learning is the same. The algorithm is the recipe, but the data is the ingredients. Real-world data is almost always messy: it has gaps, typos, repeated rows and strange values. Models also only understand numbers, so any text has to be converted first.

Every data preparation job follows roughly the same steps. This module walks through each one in order.

1. CleanRemove duplicates, fix typos and impossible values.
2. Fill gapsDecide what to do with missing values.
3. Handle outliersFind extreme values and decide if they're real.
4. Choose X and yPick the features and the target to predict.
5. EncodeTurn text categories into numbers.
6. SplitSeparate training data from test data.
7. ScalePut number columns on a similar range.

2.2Meet our messy dataset

Throughout this module we'll use one small example. A UK broadband company wants to predict which customers will leave (this is called churn). Here is a sample of their data, exactly as it came out of their system.

Can you spot the problems? The red cells are all issues we need to fix before any model can use this data.

2.3Fixing duplicates and inconsistent values

The easiest problems to fix come first.

Duplicates

A duplicate is a row that appears more than once. It often happens when data is copied or joined together from different systems. In pandas, one line removes them:

df = df.drop_duplicates()

Inconsistent text

To a computer, "London", "london" and "LONDON " (with a space at the end) are three completely different values. We fix this by making them all follow the same format:

df["city"] = df["city"].str.strip().str.title()   # " london" -> "London"

Impossible values

An age of 250, a price of -£40, or a date in the year 2099 can't be right. Use your common sense and knowledge of the business to set rules. Here, any age over 100 is clearly a mistake, so we treat it as missing and fix it in the next step.

2.4Dealing with missing values

Missing values are one of the most common problems in real data. A customer skipped a question on a form, a sensor stopped working, or a system didn't save a value. You have three main choices.

What value should you fill in?

For number columns, the usual choices are the mean (average) or the median (the middle value when everything is sorted). For text columns, we use the mode (the most common value).

The median is often the safer choice. Look at what happens to the salaries of six people when one of them is a very high earner:

In our dataset, the missing age is filled with the median age (37.5) and the missing bill with the median bill (£38).

# How many missing values in each column?
print(df.isnull().sum())

# Fill numbers with the median, text with the mode
df["age"] = df["age"].fillna(df["age"].median())
df["city"] = df["city"].fillna(df["city"].mode()[0])

2.5Spotting outliers

An outlier is a value that is far away from all the others. The best way to spot them is with a chart called a box plot.

When you find an outlier, don't delete it straight away. First ask: is it a mistake, or is it real?

2.6Features (X) and target (y)

Now our data is clean, we split the columns into two groups. By tradition, data scientists call the features X (capital, because it's a table of many columns) and the target y (small, because it's a single column).

Our cleaned data: blue columns are X, the amber column is y, and green cells were fixed in the steps above.

Drop columns that don't help

The Customer ID is just a name tag. "C004" tells us nothing about whether someone will leave, so we remove it. If we kept it, the model might learn silly patterns, such as "IDs ending in 4 leave more often".

2.7Encoding: turning text into numbers

Machine learning models are maths, and you can't do maths with the word "Leeds". So we need to convert text categories into numbers. This is called encoding. There are two main ways.

Label encoding: give each category a number

This works well when the categories have a natural order, such as sizes or education levels.

The order makes sense: Large (3) really is bigger than Small (1). The same idea works for our Left? column: Yes = 1, No = 0.

One-hot encoding: one yes/no column per category

For categories with no order, like cities, we create a new column for each one. Each row gets a 1 in the column for its city and 0 everywhere else.

2.8Feature scaling

Look at two features for five customers: age (roughly 20 to 60) and yearly income (roughly £20,000 to £75,000). Income numbers are about a thousand times bigger than ages.

Many algorithms compare values by their size. To them, a difference of £1,000 in income looks far more important than a difference of 30 years in age, just because the number is bigger. Scaling fixes this by putting all features on a similar range, so each one gets a fair say.

The two most common methods

Do all algorithms need scaling? No. Algorithms that measure distances or use gradients (K-Nearest Neighbours, SVM, logistic regression, neural networks) need it. Tree-based algorithms (decision trees, random forests) don't care. You'll see this in each algorithm's module.

2.9Feature engineering: creating better features

Feature engineering means creating new, more useful columns from the ones you already have. It's where your understanding of the business really pays off, and it often improves a model more than switching algorithms.

2.10Splitting your data the right way

In Module 1 you learned that we keep some data hidden as a test set. There's one important rule about when to split: split first, then scale.

Why does this matter? If you scale before splitting, the scaler looks at the test data too. Information about the "hidden" test set leaks into training, like a student peeking at the exam paper. Your test score will look better than the model really is.

2.11The full pipeline in Python

Here is everything from this module in one script, using our broadband customers. Read the comments to follow each step.

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# 1. Load the (messy) data
df = pd.DataFrame({
    "customer_id":  ["C001", "C002", "C003", "C004", "C005", "C002", "C006"],
    "age":          [34, 52, np.nan, 29, 250, 52, 41],
    "city":         ["Leeds", "London", "Newcastle", "london", "Leeds", "London", "Manchester"],
    "contract":     ["Monthly", "2-year", "1-year", "Monthly", "1-year", "2-year", "Monthly"],
    "monthly_bill": [45, 30, 38, 55, 35, 30, np.nan],
    "months":       [6, 40, 18, 3, 24, 40, 9],
    "left":         ["Yes", "No", "No", "Yes", "No", "No", "Yes"],
})

# 2. Clean
df = df.drop_duplicates()                                  # remove repeated rows
df["city"] = df["city"].str.title()                        # "london" -> "London"
df.loc[df["age"] > 100, "age"] = np.nan                    # impossible age -> missing
df["age"] = df["age"].fillna(df["age"].median())           # fill with the median
df["monthly_bill"] = df["monthly_bill"].fillna(df["monthly_bill"].median())

# 3. Features (X) and target (y)
X = df.drop(columns=["customer_id", "left"])
y = df["left"].map({"Yes": 1, "No": 0})

# 4. Encode text columns with one-hot encoding
X = pd.get_dummies(X, columns=["city", "contract"], dtype=int)

# 5. Split BEFORE scaling
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.33, random_state=42)

# 6. Scale the number columns (learn from training data only)
num_cols = ["age", "monthly_bill", "months"]
scaler = StandardScaler()
X_train[num_cols] = scaler.fit_transform(X_train[num_cols])
X_test[num_cols] = scaler.transform(X_test[num_cols])

print("Rows after cleaning:", len(df))
print("Training rows:", len(X_train), "| Test rows:", len(X_test))
print("Number of features:", X.shape[1])
print(X_train[num_cols].round(2))

Rows after cleaning: 6
Training rows: 4 | Test rows: 2
Number of features: 10
    age  monthly_bill  months
6  1.07         -0.44   -0.56
2  0.28         -0.44    0.56
4  0.28         -0.82    1.30
3 -1.64          1.71   -1.30

Notice what happened. We started with 7 rows and 7 columns of messy text and numbers. We ended with 6 clean rows and 10 number-only features (3 number columns, plus 4 city columns and 3 contract columns from one-hot encoding). The scaled values sit around 0, just like in the scaling lab. This data is now ready for a model.

SummaryKey takeaways

Check your understanding

Your machine learning roadmap

1. 01
   Introduction to Machine LearningWhat ML is and how it works
2. 02
   Preparing Data for Machine LearningYou are here
3. 03
   Linear RegressionPredicting numbers
4. 04
   Logistic RegressionPredicting yes or no
5. 05
   Evaluating ModelsAccuracy, precision, recall, overfitting, cross-validation
6. 06
   K-Nearest NeighboursLearning from similar examples
7. 07
   Decision TreesFlowcharts that learn
8. 08
   Random Forest and Gradient BoostingMany models working together
9. 09
   Support Vector MachinesFinding the best boundary
10. 10
    Naive BayesProbability-based classification
11. 11
    Clustering with K-MeansFinding groups without labels
12. 12
    Dimensionality Reduction with PCASimplifying big datasets
13. 13
    Capstone ProjectBuild and present a full ML project