Linear Regression

3.1What is linear regression?

In Module 1 you trained a model that predicted exam scores from study hours. That model was linear regression. Now we'll look at it properly.

Linear regression is a supervised learning algorithm that predicts a number. It looks at the relationship between features and a target, and draws the straight line that fits the data best. Then it uses that line to make predictions.

In this module, we'll help an estate agent in Newcastle predict house prices. Here are ten houses they sold recently:

You can already see the pattern: bigger houses cost more. Linear regression turns that pattern into an exact rule we can use to price a house that hasn't been sold yet.

3.2The equation of a straight line

You may remember this from school maths. Every straight line can be written as:

For our houses, linear regression finds this line:

In plain English: start at £30,300 and add about £1,930 for every square metre. So for a 95 m² house:

That's the whole idea. "Training" a linear regression model simply means finding the best values for m and c.

3.3Residuals: how wrong is the line?

No straight line goes through every point perfectly. The gap between a real value and the line's prediction is called a residual (or error).

For example, the 65 m² house sold for £148k. The line predicted about £155.8k, so its residual is −£7.8k. The model over-estimated that house.

A good line has small residuals overall. But how do we decide which line is "best"? That's the next lesson.

3.4Finding the best line

Linear regression uses a method called least squares. It works like this:

1. Work out the residual for every point.
2. Square each residual (multiply it by itself).
3. Take the average. This is the Mean Squared Error (MSE).
4. Pick the line with the smallest MSE.

Why square the errors? Two reasons. First, it stops positive and negative errors cancelling each other out (+5 and −5 would add up to 0, which would wrongly look perfect). Second, it punishes big mistakes much more than small ones: an error of 10 becomes 100, but an error of 2 only becomes 4.

Moving sliders by hand is slow. The computer finds the best line in a split second, using either a direct maths formula or a method called gradient descent. Gradient descent is like walking down a hill in fog: you feel which way is downhill, take a small step, and repeat until you reach the bottom (the smallest error). It's also how neural networks learn, which you can explore in the CX Learn Neural Networks track.

3.5How good is the model? MAE, RMSE and R²

Once a model is trained, we test it on data it hasn't seen and measure how well it did. For regression, there are four common measures.

A small worked example

Our model predicts the prices of three test houses:

3.6Multiple linear regression

House size isn't the only thing that affects price. The number of bedrooms and how far the house is from the city centre matter too. Multiple linear regression uses several features at once. The idea is exactly the same, the equation just gets longer:

Each feature gets its own slope, called a coefficient. We trained this model on 200 houses (you'll see the code in lesson 3.8). Here is what it learned:

A positive coefficient means the feature pushes the price up. A negative one pushes it down. This makes linear regression very easy to explain, which is one reason businesses love it.

3.7When linear regression goes wrong

Linear regression is powerful, but it makes some assumptions. When they aren't true, the predictions get worse.

1. The relationship isn't a straight line

Car stopping distances (from the UK Highway Code) don't rise in a straight line: doubling your speed more than doubles the distance.

3.8Linear regression in Python

Now let's build the full house price model with scikit-learn. We create 200 houses where we secretly know the "true" rule, so we can check whether the model discovers it: £1,600 per m², £12,000 per bedroom and −£4,000 per km, plus some random noise, just like real life.

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

# 1. Create a dataset of 200 houses (price in £1000s)
rng = np.random.default_rng(42)
n = 200
df = pd.DataFrame({
    "size_m2":     rng.integers(45, 160, n),
    "bedrooms":    rng.integers(1, 6, n),
    "distance_km": rng.uniform(0.5, 15, n).round(1),
})
df["price"] = (40 + 1.6 * df["size_m2"] + 12 * df["bedrooms"]
               - 4 * df["distance_km"] + rng.normal(0, 15, n)).round(0)

# 2. Features (X) and target (y)
X = df[["size_m2", "bedrooms", "distance_km"]]
y = df["price"]

# 3. Split into training and test sets
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42)

# 4. Train the model
model = LinearRegression()
model.fit(X_train, y_train)

# 5. What did the model learn?
print("Intercept:", round(model.intercept_, 1))
for name, coef in zip(X.columns, model.coef_):
    print(f"{name}: {coef:.2f}")

# 6. Test it on houses it has never seen
pred = model.predict(X_test)
print("MAE: ", round(mean_absolute_error(y_test, pred), 1))
print("RMSE:", round(np.sqrt(mean_squared_error(y_test, pred)), 1))
print("R²:  ", round(r2_score(y_test, pred), 3))

# 7. Predict a new house: 95 m², 3 bedrooms, 4 km from the centre
new_house = pd.DataFrame({"size_m2": [95], "bedrooms": [3], "distance_km": [4]})
print(f"Predicted price: £{model.predict(new_house)[0]:.1f}k")

Intercept: 38.5
size_m2: 1.61
bedrooms: 11.36
distance_km: -3.89
MAE:  13.1
RMSE: 16.3
R²:   0.921
Predicted price: £209.8k

What the output tells us

• The model found the hidden rule. It learned 1.61, 11.36 and −3.89, very close to the true values of 1.6, 12 and −4. It only saw the data, never the rule.
• MAE of 13.1 means predictions are, on average, about £13k away from the real price.
• RMSE of 16.3 is a bit higher than MAE, which means there are a few bigger misses.
• R² of 0.921 means the model explains about 92% of the differences in house prices. That's strong.

3.9Linear regression in the real world

In data analyst and BI roles, you'll often use linear regression not only to predict, but to explain: "for every extra £1,000 spent on social ads, sales rise by about £4,200". Being able to say this clearly is a valuable skill.

SummaryKey takeaways

Check your understanding

Your machine learning roadmap

1. 01
   Introduction to Machine LearningWhat ML is and how it works
2. 02
   Preparing Data for Machine LearningFeatures, encoding, scaling, train/test split
3. 03
   Linear RegressionYou are here
4. 04
   Logistic RegressionPredicting yes or no
5. 05
   Evaluating ModelsAccuracy, precision, recall, overfitting, cross-validation
6. 06
   K-Nearest NeighboursLearning from similar examples
7. 07
   Decision TreesFlowcharts that learn
8. 08
   Random Forest and Gradient BoostingMany models working together
9. 09
   Support Vector MachinesFinding the best boundary
10. 10
    Naive BayesProbability-based classification
11. 11
    Clustering with K-MeansFinding groups without labels
12. 12
    Dimensionality Reduction with PCASimplifying big datasets
13. 13
    Capstone ProjectBuild and present a full ML project