Capstone Project

13.1The brief

You've just joined Tyne Fibre, a (fictional) broadband provider with customers across the UK. On your first morning, this email arrives from the Head of Customer Retention:

This is a realistic brief. Notice it doesn't mention algorithms at all. Your job is to turn it into a machine learning project, and then turn the results back into business language.

13.2Your project plan

You'll follow the seven-step machine learning workflow from Module 1. Every step uses skills from earlier modules.

13.3Step 1: Define the problem

Before touching any data, answer these questions in writing. They'll become the first slide of your presentation.

13.4Step 2: Get and explore the data

Run this script to create the dataset. It builds a file called tyne_fibre_customers.csv with the same data for everyone, so you can compare your results with the reference solution.

# Creates the capstone dataset: tyne_fibre_customers.csv
import numpy as np
import pandas as pd

rng = np.random.default_rng(2026)
n = 6000
regions = ["North East", "Yorkshire", "Scotland", "London", "Midlands"]
contract = rng.choice(["Monthly", "12-month", "24-month"], n, p=[0.45, 0.30, 0.25])
tenure = np.where(contract == "Monthly", rng.integers(1, 40, n), rng.integers(1, 72, n))
speed = rng.normal(np.where(rng.random(n) < 0.2, 35, 70), 12, n).clip(5, 120).round(1)
df = pd.DataFrame({
    "customer_id":      [f"TF{100000 + i}" for i in range(n)],
    "age":              rng.integers(18, 85, n),
    "region":           rng.choice(regions, n),
    "contract":         contract,
    "tenure_months":    tenure,
    "monthly_bill":     rng.normal(38, 11, n).clip(15, 90).round(2),
    "tv_bundle":        rng.choice(["Yes", "No"], n, p=[0.35, 0.65]),
    "payment_method":   rng.choice(["Direct Debit", "Card", "Bank transfer"], n, p=[0.6, 0.3, 0.1]),
    "avg_speed_mbps":   speed,
    "outages_90d":      rng.poisson(0.8, n),
    "support_calls_90d": rng.poisson(1.5, n),
})
z = (-2.9
     + 1.3 * (df["contract"] == "Monthly") - 0.9 * (df["contract"] == "24-month")
     + 1.2 * (df["tenure_months"] < 6) - 0.7 * (df["tenure_months"] > 36)
     + 0.45 * df["support_calls_90d"].clip(0, 6)
     + 0.5 * df["outages_90d"].clip(0, 4)
     + 1.1 * (df["avg_speed_mbps"] < 40)
     + 0.03 * (df["monthly_bill"] - 38)
     + 0.6 * ((df["monthly_bill"] > 55) & (df["contract"] == "Monthly"))
     - 0.5 * (df["tv_bundle"] == "Yes")
     + 0.4 * (df["payment_method"] == "Card"))
df["churned"] = np.where(rng.random(n) < 1 / (1 + np.exp(-z)), "Yes", "No")

# Real data is messy: add some problems for you to fix
df.loc[rng.choice(n, 300, replace=False), "avg_speed_mbps"] = np.nan
df.loc[rng.choice(n, 120, replace=False), "payment_method"] = np.nan
df.loc[rng.choice(n, 8, replace=False), "age"] = 999
df = pd.concat([df, df.sample(25, random_state=1)], ignore_index=True)   # duplicates

df.to_csv("tyne_fibre_customers.csv", index=False)
print(df.shape, "| churn rate:", f"{(df['churned'] == 'Yes').mean():.1%}")

(6025, 12) | churn rate: 25.3%

The data dictionary

Your tasks

1. Load the file with pandas and check its shape, df.info() and df.describe().
2. Find the problems: count missing values with df.isnull().sum(), look for duplicates with df.duplicated().sum(), and check the minimum and maximum of every number column.
3. Work out the overall churn rate.
4. Compare the churn rate across groups, for example with df.groupby("contract")["churned"].value_counts(normalize=True), and draw some charts.

What your charts might show

13.5Step 3: Prepare the data

Now fix the problems and get the data ready for models, using everything from Module 2.

New tool: ColumnTransformer

Number columns and category columns need different treatment. scikit-learn's ColumnTransformer lets you send each group of columns through its own mini-pipeline, then joins the results back together. Put it inside a Pipeline with your model, and all the filling, scaling and encoding is learned from the training data only, even inside cross-validation. No leakage.

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

num_cols = ["age", "tenure_months", "monthly_bill", "avg_speed_mbps",
            "outages_90d", "support_calls_90d"]
cat_cols = ["region", "contract", "tv_bundle", "payment_method"]
prep = ColumnTransformer([
    ("num", Pipeline([("fill", SimpleImputer(strategy="median")),
                      ("scale", StandardScaler())]), num_cols),
    ("cat", Pipeline([("fill", SimpleImputer(strategy="most_frequent")),
                      ("onehot", OneHotEncoder(handle_unknown="ignore"))]), cat_cols),
])

# Later: Pipeline([("prep", prep), ("clf", YourModel())])

13.6Step 4: Compare models

Put your preparation pipeline in front of several different models, and score each one with 5-fold cross-validation on the training data (Module 5), using ROC AUC.

Try at least three from the track, for example:

• Logistic regression (Module 4): simple, fast and easy to explain. Your baseline.
• Random forest (Module 8): strong with default settings.
• Gradient boosting (Module 8): often the most accurate once tuned.

Then tune the best two with GridSearchCV. Record every result in a table: it will go in your presentation.

13.7Step 5: Choose who to call

This is where your project becomes genuinely useful. The model gives every customer a probability of leaving. The threshold decides who gets a call. Using Sarah's figures:

Use the chosen model's real predictions for the 1,200 test customers to find the most profitable threshold.

Notice the shape of the curve. Call too few people and you miss leavers you could have saved. Call too many and the cost of calls to happy customers eats the profit: calling everyone would lose £20,760 on these 1,200 customers. The best threshold is about 0.43, earning about £11,400 for every 1,200 customers, a little more than the default of 0.5.

13.8Step 6: Explain the model and segment the risk

What drives churn?

Sarah asked why customers leave. Use permutation importance: shuffle one column at a time in the test data and see how much the model's score drops. A big drop means the model relied on that column. It works for any model, even inside a pipeline.

What kind of offer for whom?

Sarah's third question needs clustering (Module 11). Take only the customers your model flags as at risk, scale a few descriptive features, and run K-Means. The reference solution found three groups:

This is what turns a model into a strategy: not just who to call, but what to say to each group.

13.9The reference solution

Here's a complete solution that covers steps 3 to 6. Your code doesn't need to match it; there are many good ways to do this project. Use it to check your results or get unstuck.

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, HistGradientBoostingClassifier
from sklearn.metrics import roc_auc_score, precision_score, recall_score
from sklearn.inspection import permutation_importance
from sklearn.cluster import KMeans

# ---------- 1. Load and clean ----------
df = pd.read_csv("tyne_fibre_customers.csv")
df = df.drop_duplicates()
df.loc[df["age"] > 100, "age"] = np.nan          # impossible ages -> missing
print("Rows:", len(df), "| Churn rate:", f"{(df['churned'] == 'Yes').mean():.1%}")

X = df.drop(columns=["customer_id", "churned"])
y = (df["churned"] == "Yes").astype(int)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

# ---------- 2. Preparation pipeline ----------
num_cols = ["age", "tenure_months", "monthly_bill", "avg_speed_mbps",
            "outages_90d", "support_calls_90d"]
cat_cols = ["region", "contract", "tv_bundle", "payment_method"]
prep = ColumnTransformer([
    ("num", Pipeline([("fill", SimpleImputer(strategy="median")),
                      ("scale", StandardScaler())]), num_cols),
    ("cat", Pipeline([("fill", SimpleImputer(strategy="most_frequent")),
                      ("onehot", OneHotEncoder(handle_unknown="ignore"))]), cat_cols),
])

# ---------- 3. Compare models with 5-fold cross-validation ----------
candidates = {
    "Logistic regression": LogisticRegression(max_iter=1000),
    "Random forest": RandomForestClassifier(n_estimators=300, min_samples_leaf=10,
                                            random_state=42),
    "Gradient boosting": HistGradientBoostingClassifier(random_state=42),
}
print("\nCross-validated ROC AUC:")
for name, clf in candidates.items():
    pipe = Pipeline([("prep", prep), ("clf", clf)])
    scores = cross_val_score(pipe, X_train, y_train, cv=5, scoring="roc_auc")
    print(f"  {name:20} {scores.mean():.3f} (+/- {scores.std():.3f})")

# ---------- 4. Tune the two strongest options, keep the best ----------
searches = {
    "Logistic regression": GridSearchCV(
        Pipeline([("prep", prep), ("clf", LogisticRegression(max_iter=1000))]),
        param_grid={"clf__C": [0.01, 0.1, 1, 10]}, cv=5, scoring="roc_auc"),
    "Gradient boosting": GridSearchCV(
        Pipeline([("prep", prep), ("clf", HistGradientBoostingClassifier(random_state=42))]),
        param_grid={"clf__learning_rate": [0.03, 0.1], "clf__max_depth": [3, None],
                    "clf__max_iter": [100, 300]}, cv=5, scoring="roc_auc"),
}
print("\nAfter tuning:")
for name, search in searches.items():
    search.fit(X_train, y_train)
    print(f"  {name:20} {search.best_score_:.3f}  {search.best_params_}")
best_name = max(searches, key=lambda k: searches[k].best_score_)
model = searches[best_name].best_estimator_
print("Chosen model:", best_name)

# ---------- 5. Final test and business threshold ----------
prob = model.predict_proba(X_test)[:, 1]
print("\nTest ROC AUC:", round(roc_auc_score(y_test, prob), 3))

VALUE_OF_CUSTOMER, OFFER_COST, SUCCESS_RATE = 420, 60, 0.40
def net_value(t):
    contacted = prob >= t
    caught = (contacted & (y_test == 1)).sum()
    return caught * SUCCESS_RATE * VALUE_OF_CUSTOMER - contacted.sum() * OFFER_COST

thresholds = np.arange(0.05, 0.95, 0.01)
best_t = thresholds[np.argmax([net_value(t) for t in thresholds])]
pred = (prob >= best_t).astype(int)
print(f"Best threshold: {best_t:.2f} | net value: £{net_value(best_t):,.0f}")
print(f"Precision: {precision_score(y_test, pred):.2f} | Recall: {recall_score(y_test, pred):.2f}")

# ---------- 6. What drives churn? ----------
imp = permutation_importance(model, X_test, y_test, scoring="roc_auc",
                             n_repeats=5, random_state=42)
top = pd.Series(imp.importances_mean, index=X.columns).sort_values(ascending=False)
print("\nTop drivers of churn:")
print(top.head(5).round(3).to_string())

# ---------- 7. Segment the at-risk customers ----------
at_risk = X_test[pred == 1].copy()
seg_cols = ["tenure_months", "monthly_bill", "support_calls_90d", "outages_90d"]
seg_X = StandardScaler().fit_transform(at_risk[seg_cols].fillna(at_risk[seg_cols].median()))
at_risk["segment"] = KMeans(n_clusters=3, n_init=10, random_state=42).fit_predict(seg_X)
print("\nAt-risk segments:")
print(at_risk.groupby("segment")[seg_cols].mean().round(1)
      .assign(customers=at_risk["segment"].value_counts()).to_string())

Rows: 6000 | Churn rate: 25.4%

Cross-validated ROC AUC:
  Logistic regression  0.813 (+/- 0.011)
  Random forest        0.807 (+/- 0.015)
  Gradient boosting    0.794 (+/- 0.016)

After tuning:
  Logistic regression  0.813  {'clf__C': 1}
  Gradient boosting    0.810  {'clf__learning_rate': 0.03, 'clf__max_depth': 3, 'clf__max_iter': 300}
Chosen model: Logistic regression

Test ROC AUC: 0.813
Best threshold: 0.43 | net value: £11,448
Precision: 0.63 | Recall: 0.51

Top drivers of churn:
contract             0.133
support_calls_90d    0.045
outages_90d          0.032
tenure_months        0.030
monthly_bill         0.014

At-risk segments:
         tenure_months  monthly_bill  support_calls_90d  outages_90d  customers
segment                                                                        
0                 15.3          30.7                3.1          1.6         77
1                  7.0          47.4                1.6          1.0         91
2                 29.3          47.4                2.2          1.2         78

Stretch goals

Finished early? Try one or more of these to make your project stand out:

• Create new features (Module 2), such as "bill per Mbps" or "new and on a monthly contract", and see if they help.
• Try the gradient boosting model with class_weight="balanced", or try a KNN or SVM (Modules 6 and 9).
• Use PCA (Module 12) to draw the at-risk customers in 2D, coloured by segment.
• Test how sensitive your profit estimate is: what if only 25% of contacted leavers stay, or a call costs £80?
• Build a simple Power BI or Excel dashboard showing each customer's risk score and segment.

13.10Step 7: Present your findings

Sarah isn't a data scientist. Your presentation should be short, visual and focused on decisions. Aim for about eight slides:

13.11Submission checklist and marking guide

Your finished project should include:

• A Jupyter notebook (or Python script) that runs from top to bottom without errors
• Clear headings and short notes explaining each step and decision
• At least three charts from your exploration
• A comparison table of at least three models with cross-validated scores
• A recommended threshold backed by a profit calculation
• The top drivers of churn and at least two customer segments with suggested actions
• A short presentation (around 8 slides) or a one-page summary for Sarah

SummaryWhat you've achieved

Final check: think like a data scientist

Your machine learning roadmap

1. 01
   Introduction to Machine LearningWhat ML is and how it works
2. 02
   Preparing Data for Machine LearningFeatures, encoding, scaling, train/test split
3. 03
   Linear RegressionPredicting numbers
4. 04
   Logistic RegressionPredicting yes or no
5. 05
   Evaluating ModelsAccuracy, precision, recall, overfitting, cross-validation
6. 06
   K-Nearest NeighboursLearning from similar examples
7. 07
   Decision TreesFlowcharts that learn
8. 08
   Random Forest and Gradient BoostingMany models working together
9. 09
   Support Vector MachinesFinding the best boundary
10. 10
    Naive BayesProbability-based classification
11. 11
    Clustering with K-MeansFinding groups without labels
12. 12
    Dimensionality Reduction with PCASimplifying big datasets
13. 13
    Capstone ProjectYou are here