Teaching computers to recognise what's in a picture is one of deep learning's biggest successes. In this module you'll build convolutional neural networks (CNNs) in Keras, understand the shapes flowing through them, load your own images from folders, and use data augmentation to get more from a small set of photos.
- Level: Intermediate
- Time: about 45 minutes
- Needs: Modules 1 to 7
By the end of this module you will be able to
- Prepare image data and scale pixels with a Rescaling layer
- Explain what Conv2D and MaxPooling2D layers do
- Set filters, kernel size, padding and strides
- Work out a CNN's output shapes and parameter counts
- Choose between Flatten and GlobalAveragePooling2D
- Load your own images from folders with image_dataset_from_directory
- Use augmentation layers to make models more robust
- Show with real code why CNNs and augmentation beat a dense network
8.1Images as tensors
As you saw in Module 2, an image is a grid of numbers. In Keras, a batch of images has the shape (images, height, width, channels): 1 channel for greyscale, 3 for colour (red, green, blue). Pixel values usually run from 0 to 255, so the first step is to scale them to 0 to 1. The neatest way is a layer inside the model:
keras.Input(shape=(32, 32, 1)), # 32x32 greyscale images
layers.Rescaling(1 / 255), # 0-255 becomes 0-1
Like the Normalization layer, it's saved with the model, so you can later pass it raw images straight from a camera or file.
8.2Why convolutional networks?
You could flatten an image into one long list and use Dense layers. But that throws away which pixels are next to each other, needs a huge number of weights, and forces the network to learn every object separately in every position.
A convolutional layer works differently. It slides small filters, typically 3 × 3 pixels, across the image. Each filter looks for one small pattern, like an edge or a corner, everywhere in the image, using the same few weights at every position. Stacking these layers lets the network build up from edges, to shapes, to whole objects.
| Layer | What it does | Typical use |
|---|---|---|
Conv2D(filters, kernel_size) | Slides filters over the image to find patterns, producing one feature map per filter | Conv2D(32, 3, padding="same", activation="relu") |
MaxPooling2D() | Shrinks each feature map by keeping the biggest value in each 2 × 2 block | After one or two Conv2D layers |
GlobalAveragePooling2D() | Averages each whole feature map into a single number | At the end, before the output layer |
Flatten() | Unrolls the feature maps into one long list | At the end, for small images |
Want to understand exactly how convolution works, with worked examples? Our Neural Networks course has a full CNN module. Here, we'll focus on building them in Keras.
8.3The Conv2D layer's settings
| Argument | Meaning | Common choice |
|---|---|---|
filters | How many different patterns to look for. Each makes one feature map | 32, then 64, then 128 as you go deeper |
kernel_size | The size of each filter | 3 (meaning 3 × 3) |
padding | "same" adds a border so the output keeps the same size; "valid" (the default) lets it shrink slightly | "same" |
strides | How many pixels the filter moves each step. 2 halves the size | 1 |
activation | Applied to each output value | "relu" |
Each filter has one weight for every pixel it covers, in every input channel, plus one bias:
So Conv2D(32, 3) on a greyscale image has (3 × 3 × 1 + 1) × 32 = 320 parameters, however big the image is. That's why CNNs need far fewer weights than dense networks.
8.4Follow the shapes
Most CNN errors come from shapes. Build a network below and watch the shapes and parameters change at every layer.
Try it: CNN shape calculator
Try switching the final step to Flatten with 224 × 224 colour images: the Dense layer's weights explode. GlobalAveragePooling2D keeps models small and is the usual modern choice.
8.5Loading your own images
For real projects, put your images in folders, one folder per class:
photos/
cats/ cat001.jpg, cat002.jpg, ...
dogs/ dog001.jpg, dog002.jpg, ...
Then one Keras function loads them, resizes them, labels them from the folder names, batches them and splits off a validation set:
train_ds = keras.utils.image_dataset_from_directory(
"photos", validation_split=0.2, subset="training", seed=1,
image_size=(128, 128), batch_size=32)
val_ds = keras.utils.image_dataset_from_directory(
"photos", validation_split=0.2, subset="validation", seed=1,
image_size=(128, 128), batch_size=32)
print(train_ds.class_names) # ['cats', 'dogs']
model.fit(train_ds, validation_data=val_ds, epochs=20)
It loads images in batches as needed, so it works even when your photos don't all fit in memory. Use the same seed for both calls so the training and validation sets don't overlap.
8.6Data augmentation
CNNs need lots of varied examples. Data augmentation creates new ones by randomly changing your training images in ways that don't change the answer: a flipped cat is still a cat. In Keras, augmentation is just more layers at the start of the model:
layers.RandomFlip("horizontal"), # mirror left-right
layers.RandomRotation(0.1), # rotate up to 10% of a full turn (36 degrees)
layers.RandomZoom(0.1), # zoom in or out up to 10%
layers.RandomTranslation(0.1, 0.1), # shift up to 10%
layers.RandomContrast(0.2), # vary the contrast
These layers are only active during training. Every epoch, the model sees a slightly different version of each image, so it can't simply memorise them. When you call evaluate() or predict(), they do nothing.
Only change what doesn't change the answer
Flipping a photo of a dog left to right is fine. Flipping the digit 6 upside down turns it into a 9! Choose augmentations that make sense for your images.
8.7A CNN with augmentation in Python
Imagine a factory camera that has to recognise four shapes of part: circle, square, triangle and star. The problem: all 400 training photos were taken with the parts upright, but on the real production line they arrive at any angle.
We'll compare a dense network, a CNN, and the same CNN with augmentation.
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
import numpy as np
import keras
from keras import layers
from PIL import Image, ImageDraw
# ---------- 1. Make an image dataset of 4 shapes ----------
# Training photos were all taken upright; in real use, shapes arrive at any angle.
SHAPES = ["circle", "square", "triangle", "star"]
def draw(shape, rng, upright, size=32):
img = Image.new("L", (size * 2, size * 2), 0)
d = ImageDraw.Draw(img)
cx, cy = rng.uniform(22, 42, 2) # position
r = rng.uniform(10, 18) # size
if upright:
a = rng.uniform(-0.15, 0.15) - np.pi / 2 # (almost) upright
else:
a = rng.uniform(0, 2 * np.pi) # any angle
ring = [(cx + r * np.cos(a + t), cy + r * np.sin(a + t))
for t in np.linspace(0, 2 * np.pi, {"square": 4, "triangle": 3, "star": 10}.get(shape, 4), endpoint=False)]
if shape == "circle":
d.ellipse([cx - r, cy - r, cx + r, cy + r], fill=255)
elif shape == "star":
d.polygon([(cx + (p[0] - cx) * (1 if i % 2 == 0 else .45), cy + (p[1] - cy) * (1 if i % 2 == 0 else .45))
for i, p in enumerate(ring)], fill=255)
else:
d.polygon(ring, fill=255)
arr = np.asarray(img.resize((size, size), Image.BILINEAR), dtype="float32")
return np.clip(arr + rng.normal(0, 25, arr.shape), 0, 255) # camera noise
def make(n_per_shape, seed, upright):
rng = np.random.default_rng(seed)
X = np.array([draw(s, rng, upright) for s in SHAPES for _ in range(n_per_shape)])[..., None]
y = np.repeat(np.arange(len(SHAPES)), n_per_shape)
return X, y
X_train, y_train = make(100, seed=1, upright=True) # 400 photos, all taken upright
X_test, y_test = make(500, seed=2, upright=False) # 2,000 photos at any angle
print("Training images:", X_train.shape, "| pixel range:", X_train.min(), "to", X_train.max())
# ---------- 2. Three models ----------
def dense_model():
return keras.Sequential([
keras.Input(shape=(32, 32, 1)),
layers.Rescaling(1 / 255), # 0-255 -> 0-1
layers.Flatten(),
layers.Dense(128, activation="relu"),
layers.Dense(4, activation="softmax")])
def cnn_model(augment=False):
aug = [layers.RandomFlip("horizontal_and_vertical"),
layers.RandomRotation(0.5, fill_mode="constant", interpolation="nearest"),
layers.RandomZoom(0.1, fill_mode="constant", interpolation="nearest")] if augment else []
return keras.Sequential([
keras.Input(shape=(32, 32, 1)),
*aug, # only active during training
layers.Rescaling(1 / 255),
layers.Conv2D(32, 3, padding="same", activation="relu"),
layers.MaxPooling2D(), # 32x32 -> 16x16
layers.Conv2D(32, 3, padding="same", activation="relu"),
layers.MaxPooling2D(), # 16x16 -> 8x8
layers.Conv2D(64, 3, padding="same", activation="relu"),
layers.GlobalAveragePooling2D(), # 8x8x64 -> 64 numbers
layers.Dense(4, activation="softmax")])
cnn_model(augment=True).summary()
for name, build in [("Dense network", dense_model),
("CNN", cnn_model),
("CNN + augmentation", lambda: cnn_model(augment=True))]:
keras.utils.set_random_seed(0)
model = build()
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy",
metrics=["accuracy"])
model.fit(X_train, y_train, epochs=100, batch_size=32, verbose=0)
train_acc = model.evaluate(X_train, y_train, verbose=0)[1]
test_acc = model.evaluate(X_test, y_test, verbose=0)[1]
print(f"{name:20} weights={model.count_params():7,} train={train_acc:.3f} test={test_acc:.3f}")
Training images: (400, 32, 32, 1) | pixel range: 0.0 to 255.0 Model: "sequential" ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ random_flip (RandomFlip) │ (None, 32, 32, 1) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ random_rotation (RandomRotation) │ (None, 32, 32, 1) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ random_zoom (RandomZoom) │ (None, 32, 32, 1) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ rescaling (Rescaling) │ (None, 32, 32, 1) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d (Conv2D) │ (None, 32, 32, 32) │ 320 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d (MaxPooling2D) │ (None, 16, 16, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_1 (Conv2D) │ (None, 16, 16, 32) │ 9,248 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_1 (MaxPooling2D) │ (None, 8, 8, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_2 (Conv2D) │ (None, 8, 8, 64) │ 18,496 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ global_average_pooling2d │ (None, 64) │ 0 │ │ (GlobalAveragePooling2D) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense (Dense) │ (None, 4) │ 260 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘ Total params: 28,324 (110.64 KB) Trainable params: 28,324 (110.64 KB) Non-trainable params: 0 (0.00 B) Dense network weights=131,716 train=1.000 test=0.459 CNN weights= 28,324 train=1.000 test=0.678 CNN + augmentation weights= 28,324 train=0.822 test=0.803
What the output tells us
- The dense network memorised its training images (100%) with 131,716 weights, but managed only 45.9% on the test images, barely better than the 25% you'd get by guessing.
- The CNN did much better (67.8%) with under a quarter of the weights (28,324), because its filters find edges and corners wherever they appear. But it had only seen upright shapes, so rotated ones still confused it.
- Augmentation lifted test accuracy to 80.3%. By randomly rotating, flipping and zooming the training images, the model learned that a star is a star at any angle. It scored lower on its own training data (82.2%), which is a good sign: it stopped memorising.
- The summary shows the augmentation layers have no weights, and the classic CNN pattern: feature maps shrink from 32 × 32 to 16 × 16 to 8 × 8 while the number of filters grows from 32 to 64.
Two details in the code
We set fill_mode="constant" so rotated images are filled with black, like the background, instead of a mirrored copy of the image. And interpolation="nearest" keeps the camera noise looking the same as in real photos. Checking that augmented images still look realistic is always worth doing.
With thousands of real photos, even better results come from starting with a model that has already learned from millions of images. That's transfer learning, in Module 10.
SummaryKey takeaways
- Image batches have shape (images, height, width, channels); scale pixels with
layers.Rescaling(1/255). Conv2Dslides small filters over the image to find patterns anywhere;MaxPooling2Dhalves the size.- Conv2D parameters = (kernel × kernel × input channels + 1) × filters, whatever the image size.
- Classic pattern: feature maps get smaller while filters get more numerous.
- End with
GlobalAveragePooling2Dto keep models small, then a softmaxDenselayer. - Load folders of images with
keras.utils.image_dataset_from_directory. - Augmentation layers (flip, rotate, zoom…) work only during training and make models more robust.
Check your understanding
Your Keras roadmap
- 01Introduction to KerasWhat Keras is, setup and your first model
- 02Tensors and Data for KerasShapes, scaling and splitting data
- 03The Sequential API and Dense LayersStacking layers into a model
- 04Compiling ModelsLosses, optimisers and metrics
- 05Training and Evaluatingfit(), validation and callbacks
- 06The Functional APIMultiple inputs, outputs and branches
- 07RegularisationDropout, batch normalisation and early stopping
- 08Image Classification with KerasYou are here
- 09Text and Sequences with KerasEmbeddings, LSTMs and text classification
- 10Pretrained Models, Saving and DeploymentFine-tuning and sharing your models
Next module
Text and Sequences with Keras