Convolutional Neural Networks (CNNs)

8.1How a computer sees an image

To you, a photo is a face or a dog. To a computer, it's just a grid of numbers. Each tiny square, or pixel, gets a number for how bright it is.

8.2Why normal networks struggle with images

You could feed an image into the dense networks from earlier modules by flattening it: laying every pixel out in one long row. It works for tiny images, but causes three big problems:

8.3Convolution: scanning with a filter

The key idea of a CNN is the filter (also called a kernel): a tiny grid of weights, usually 3 × 3. The filter slides across the image, one position at a time. At each position it:

1. multiplies each of its 9 weights by the pixel underneath,
2. adds up the 9 results,
3. writes that one number into a new grid, called a feature map.

This sliding multiply-and-add is called convolution, which is where CNNs get their name.

Different filters detect different things. One filter finds vertical edges, another horizontal edges, another spots or corners. And here's the crucial part: in a CNN, nobody designs the filters. The weights in each filter are learned during training, exactly like any other weight. The network discovers for itself which patterns are useful.

Why this solves the three problems

8.4Try convolution yourself

Choose a filter and watch it scan a simple picture of a house. Drag the slider (or press Play) to move the filter, and see the calculation at each position.

Notice that the edge filters light up only along the edges of the house: the walls for "vertical", the roof base and floor for "horizontal". Positive values (amber) mark one direction of change, negative values (blue) the opposite. A real CNN layer runs dozens of filters at once, producing a stack of feature maps.

8.5Stride and padding

You may have noticed that the 12 × 12 image produced a smaller 10 × 10 feature map. That's because a 3 × 3 filter can't be centred on the edge pixels. Two settings control the size of the output:

For the lab: (12 − 3 + 0) ÷ 1 + 1 = 10. With padding="same", the output would stay at 12 × 12.

8.6Pooling: shrinking the feature maps

After a convolution, CNNs usually shrink the feature maps with a pooling layer. The most common kind, max pooling, splits the map into small blocks (usually 2 × 2) and keeps only the biggest number from each.

8.7Putting it together: a full CNN

A typical CNN has two parts. The first part finds patterns with repeated blocks of convolution, ReLU and pooling. The second part makes the decision with the dense layers you already know.

What each layer learns

Because each layer builds on the one before, CNNs learn a hierarchy of features, from simple to complex:

Colour images and many filters

For a colour image, each filter is actually 3 × 3 × 3: one 3 × 3 slice for each of the red, green and blue channels, all added together. And each layer uses many filters. A layer with 32 filters produces 32 feature maps, which become 32 channels for the next layer to scan.

8.8Famous CNNs

8.9Building a CNN in Keras

We'll build a small CNN for the handwritten digits, and compare it with a normal dense network trained in exactly the same way. Then we'll test both on the same digits shifted one pixel to the right, to see which one really understands shapes.

import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"          # hide TensorFlow start-up messages
import numpy as np
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
import keras
from keras import layers

keras.utils.set_random_seed(42)

# 1. Images: 1,797 handwritten digits, each 8x8 pixels with 1 channel (greyscale)
X, y = load_digits(return_X_y=True)
X = (X / 16.0).reshape(-1, 8, 8, 1)               # scale to 0-1, keep the 2D shape
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)

# 2. A small CNN
cnn = keras.Sequential([
    keras.Input(shape=(8, 8, 1)),
    layers.Conv2D(16, kernel_size=3, padding="same", activation="relu"),
    layers.MaxPooling2D(pool_size=2),             # 8x8 -> 4x4
    layers.Conv2D(32, kernel_size=3, padding="same", activation="relu"),
    layers.MaxPooling2D(pool_size=2),             # 4x4 -> 2x2
    layers.Flatten(),
    layers.Dense(64, activation="relu"),
    layers.Dense(10, activation="softmax"),       # one output per digit
])
cnn.summary(line_length=70)                       # look inside the CNN

# 3. A normal (dense) network for comparison
dense = keras.Sequential([
    keras.Input(shape=(8, 8, 1)),
    layers.Flatten(),                             # 8x8 grid -> 64 numbers in a row
    layers.Dense(128, activation="relu"),
    layers.Dense(64, activation="relu"),
    layers.Dense(10, activation="softmax"),
])

# 4. Train both the same way
for name, model in [("CNN", cnn), ("Dense network", dense)]:
    model.compile(optimizer="adam", loss="sparse_categorical_crossentropy",
                  metrics=["accuracy"])
    model.fit(X_train, y_train, epochs=30, batch_size=32, verbose=0,
              validation_split=0.1)
    _, acc = model.evaluate(X_test, y_test, verbose=0)
    # Test again with every image shifted one pixel to the right
    shifted = np.roll(X_test, shift=1, axis=2)
    shifted[:, :, 0, :] = 0
    _, acc_shift = model.evaluate(shifted, y_test, verbose=0)
    print(f"{name:14} weights={model.count_params():6,}  "
          f"accuracy={acc:.3f}  shifted images={acc_shift:.3f}")

Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Layer (type)                 ┃ Output Shape          ┃     Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ conv2d (Conv2D)              │ (None, 8, 8, 16)      │         160 │
├──────────────────────────────┼───────────────────────┼─────────────┤
│ max_pooling2d (MaxPooling2D) │ (None, 4, 4, 16)      │           0 │
├──────────────────────────────┼───────────────────────┼─────────────┤
│ conv2d_1 (Conv2D)            │ (None, 4, 4, 32)      │       4,640 │
├──────────────────────────────┼───────────────────────┼─────────────┤
│ max_pooling2d_1              │ (None, 2, 2, 32)      │           0 │
│ (MaxPooling2D)               │                       │             │
├──────────────────────────────┼───────────────────────┼─────────────┤
│ flatten (Flatten)            │ (None, 128)           │           0 │
├──────────────────────────────┼───────────────────────┼─────────────┤
│ dense (Dense)                │ (None, 64)            │       8,256 │
├──────────────────────────────┼───────────────────────┼─────────────┤
│ dense_1 (Dense)              │ (None, 10)            │         650 │
└──────────────────────────────┴───────────────────────┴─────────────┘
 Total params: 13,706 (53.54 KB)
 Trainable params: 13,706 (53.54 KB)
 Non-trainable params: 0 (0.00 B)
CNN            weights=13,706  accuracy=0.976  shifted images=0.673
Dense network  weights=17,226  accuracy=0.973  shifted images=0.469

What the output tells us

• The model summary shows the shapes shrinking and deepening, just like Figure 4: 8 × 8 × 16 feature maps, pooled to 4 × 4, then 4 × 4 × 32, pooled to 2 × 2, then flattened to 128 numbers.
• The first convolution layer has only 160 weights: 16 filters × 9 weights, plus 16 biases. That's weight sharing at work.
• On normal test images, both networks score about the same (97.6% vs 97.3%). With tiny 8 × 8 images, a dense network can still cope.
• On shifted images, the difference is clear. The dense network drops to 46.9%, because it learned "pixel number 27 should be bright". The CNN keeps 67.3%, because its filters look for shapes wherever they are. On real photos, where objects can be anywhere, this advantage is huge.
• The CNN does it with fewer weights (13,706 vs 17,226).

8.10CNNs in the real world

Since around 2020, a newer design called the Vision Transformer (based on Module 10) has matched or beaten CNNs on some large image tasks. But CNNs remain widely used, especially where speed and smaller datasets matter.

SummaryKey takeaways

Check your understanding

Your neural networks roadmap

1. 01
   Introduction to Neural NetworksWhat neural networks are and how they work
2. 02
   The Perceptron and the Artificial NeuronBuild a single neuron from scratch
3. 03
   Activation FunctionsSigmoid, tanh, ReLU and softmax
4. 04
   Loss Functions and Gradient DescentHow networks measure and reduce error
5. 05
   BackpropagationHow every weight learns its share of the blame
6. 06
   Building Networks with KerasYour first deep learning library
7. 07
   Training Better NetworksOverfitting, dropout, batch normalisation, early stopping
8. 08
   Convolutional Neural NetworksYou are here
9. 09
   Recurrent Neural Networks and LSTMsWorking with sequences
10. 10
    Transformers and AttentionThe architecture behind ChatGPT and Claude
11. 11
    Transfer Learning and Pretrained ModelsStanding on the shoulders of giants
12. 12
    Capstone ProjectBuild and present a deep learning project