CX Learn by CadetX
Exams My Dashboard
Keras · TensorFlow · Deep Learning
KR-101 · Module 5 Beginner about 40 minutes 6 Lessons Prereq: Modules 1 to 4

Training & Evaluating

This is where your model actually learns. In this module you'll master fit(), use a validation set to watch training as it happens, read training curves like a professional, let callbacks stop training and save your best model automatically, and then evaluate and use your model properly.

  • Level: Beginner
  • Time: about 40 minutes
  • Needs: Modules 1 to 4

By the end of this module you will be able to

  • Use fit() with epochs, batch size and a validation set
  • Explain epochs, batches and steps
  • Read a History object and plot training curves
  • Diagnose underfitting, overfitting and a bad learning rate from curves
  • Use EarlyStopping, ModelCheckpoint and ReduceLROnPlateau callbacks
  • Evaluate a model once, honestly, on the test set
  • Turn predicted probabilities into class labels

5.1Training with fit()

model.fit() runs the training loop for you: it shows the model batches of examples, measures the loss, and updates the weights, over and over.

fit.py
history = model.fit(
    X_train, y_train,
    epochs=50,               # passes through the whole training set
    batch_size=32,           # examples per weight update
    validation_split=0.2,    # keep 20% aside to check progress
    verbose=1,               # 1 = progress bar, 0 = silent
)
▶ Open Google ColabPaste the code into a new Colab notebook and press Run. Keras is already installed.
ArgumentWhat it doesGood starting value
epochsHow many times to go through all the training dataSet it high and let early stopping decide (lesson 5.4)
batch_sizeHow many examples the model sees before each weight update32
validation_splitHolds back the last share of the training data to check progress after each epoch0.2
validation_dataUse your own validation set instead: validation_data=(X_val, y_val)For time series, or when you've made your own split
callbacksA list of helpers that run during trainingSee lesson 5.4
How fit splits the work. 1,437 training images are split by validation_split into 1,149 for training and 288 for validation. The 1,149 training images are cut into 36 batches of up to 32. The model updates its weights after each batch. One epoch is one pass through all 36 batches, followed by a check on the validation set.X_train: 1,437 imagesTraining: 1,149 images (80%)Validation288 (20%)validation_split=0.236 batches of up to 32 images (batch_size=32): the weights update after each oneChecked onceper epoch1 epoch = all 36 batches + a validation check
Figure 1. What happens inside fit() for the digits example in this module. Each weight update is called a step, so here one epoch is 36 steps.

A detail that catches people out

validation_split takes the last 20% of your training data, before shuffling. If your data is sorted (for example, all the class 0 examples first), the validation set will be lopsided. Shuffle first, or use train_test_split and pass validation_data yourself.

5.2Reading the training history

fit() returns a History object. Its history attribute is a dictionary holding the loss and every metric, for every epoch, for both the training data and the validation data:

plot_history.py
import matplotlib.pyplot as plt

h = history.history                  # keys: loss, accuracy, val_loss, val_accuracy
plt.plot(h["loss"], label="Training loss")
plt.plot(h["val_loss"], label="Validation loss")
plt.xlabel("Epoch"); plt.ylabel("Loss"); plt.legend()
plt.show()
▶ Open Google ColabPaste the code into a new Colab notebook and press Run. Keras is already installed.

Always plot the training and validation loss together. The gap between them tells you more than any single number. Below are four real training runs on the handwritten digits data. Can you spot what's wrong with each?

Try it: diagnose the training curves

Training lossValidation loss

What's your diagnosis?

What the curves showDiagnosisWhat to try
Both losses fall and level off close togetherHealthyYou're done, or try a slightly bigger model
Training loss keeps falling; validation loss turns and risesOverfittingEarly stopping, more data, regularisation (Module 7)
Both losses stay highUnderfittingA bigger model, more epochs, a higher learning rate
Validation loss jumps around wildlyLearning rate too highLower the learning rate

5.3Callbacks: automatic helpers

A callback is a helper that Keras runs at certain moments during training, usually at the end of every epoch. You pass a list of them to fit(). These four are the most useful:

CallbackWhat it does
EarlyStoppingStops training when the validation loss stops improving, and can roll back to the best weights. Use it almost always.
ModelCheckpointSaves the model to a file whenever it gets better, so a crash or a bad epoch never loses your best model
ReduceLROnPlateauLowers the learning rate when progress stalls, to help the model settle into a better solution
TensorBoardLogs training to view in TensorBoard's interactive charts (built into Colab)
callbacks.py
callbacks = [
    keras.callbacks.EarlyStopping(monitor="val_loss", patience=10,
                                  restore_best_weights=True),
    keras.callbacks.ModelCheckpoint("best_model.keras", monitor="val_loss",
                                    save_best_only=True),
    keras.callbacks.ReduceLROnPlateau(monitor="val_loss", factor=0.5, patience=5),
]
history = model.fit(X_train, y_train, epochs=500, validation_split=0.2,
                    callbacks=callbacks)
▶ Open Google ColabPaste the code into a new Colab notebook and press Run. Keras is already installed.

patience=10 means "wait 10 epochs without improvement before stopping", because validation loss naturally wobbles a little. With early stopping in place, you can set epochs high and let Keras decide when to stop.

5.4Evaluating and predicting

evaluate(): one honest score

model.evaluate(X_test, y_test) returns the loss and metrics on data the model has never seen. Add return_dict=True to get a labelled dictionary. Use the test set once, at the very end: if you keep tweaking your model to improve the test score, it stops being an honest measure.

predict(): from probabilities to answers

model.predict() returns what the output layer produces, which is usually probabilities. You then turn them into answers:

Output layerpredict() givesTurn it into an answer with
Dense(1, activation="sigmoid")One probability per example(probs > 0.5).astype(int)
Dense(10, activation="softmax")10 probabilities per exampleprobs.argmax(axis=1)
Dense(1)A numberUse it directly

Predicting for one example

Keras always expects a batch. To predict for a single example with shape (64,), add a batch dimension first: model.predict(x[None, :]) gives it shape (1, 64). Forgetting this is one of the most common shape errors.

5.5A complete training run in Python

We'll recognise handwritten digits (1,797 images of 8 × 8 pixels, 10 classes) using everything from this module: a validation split, three callbacks, evaluation and prediction, and reloading the best saved model.

train_digits.py
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
import numpy as np
import keras
from keras import layers
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

keras.utils.set_random_seed(42)

# Handwritten digits: 1,797 images of 8x8 pixels, flattened to 64 numbers
X, y = load_digits(return_X_y=True)
X = (X / 16.0).astype("float32")                     # scale pixels to 0-1
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

model = keras.Sequential([
    keras.Input(shape=(64,)),
    layers.Dense(128, activation="relu"),
    layers.Dense(64, activation="relu"),
    layers.Dense(10, activation="softmax"),
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])

# ---------- Callbacks: automatic helpers during training ----------
early_stop = keras.callbacks.EarlyStopping(monitor="val_loss", patience=10, min_delta=0.001,
                                           restore_best_weights=True)
callbacks = [
    early_stop,
    keras.callbacks.ModelCheckpoint("best_digits.keras", monitor="val_loss",
                                    save_best_only=True),
    keras.callbacks.ReduceLROnPlateau(monitor="val_loss", factor=0.5, patience=5,
                                      min_lr=1e-5),
]

# ---------- fit(): train, keeping 20% of the training data for validation ----------
history = model.fit(X_train, y_train, epochs=200, batch_size=32,
                    validation_split=0.2, callbacks=callbacks, verbose=0)

h = history.history
best = early_stop.best_epoch                          # counted from 0
print(f"Stopped after {len(h['loss'])} of 200 epochs; weights restored from epoch {best + 1}")
print(f"Best epoch: loss={h['loss'][best]:.3f}  val_loss={h['val_loss'][best]:.3f}  "
      f"val_accuracy={h['val_accuracy'][best]:.3f}")
print(f"Learning rate at the end: {float(model.optimizer.learning_rate.numpy()):.6f} (started at 0.001)")

# ---------- evaluate(): one honest score on the test set ----------
results = model.evaluate(X_test, y_test, verbose=0, return_dict=True)
print("Test results:", {k: round(v, 3) for k, v in results.items()})

# ---------- predict(): probabilities -> classes ----------
probs = model.predict(X_test[:5], verbose=0)
print("Predicted digits:", probs.argmax(axis=1), "| true digits:", y_test[:5])
print("Confidence:      ", probs.max(axis=1).round(3))

one_image = X_test[0]                                # shape (64,)
print("Single image needs a batch dimension:", one_image[None, :].shape)

# ---------- Load the best model saved by ModelCheckpoint ----------
best_model = keras.models.load_model("best_digits.keras")
print("Reloaded model test accuracy:",
      round(best_model.evaluate(X_test, y_test, verbose=0)[1], 3))
Output
Stopped after 84 of 200 epochs; weights restored from epoch 74
Best epoch: loss=0.002  val_loss=0.118  val_accuracy=0.962
Learning rate at the end: 0.000500 (started at 0.001)
Test results: {'accuracy': 0.978, 'loss': 0.07}
Predicted digits: [5 2 8 1 7] | true digits: [5 2 8 1 7]
Confidence:       [0.999 1.    0.794 0.854 0.999]
Single image needs a batch dimension: (1, 64)
Reloaded model test accuracy: 0.978
▶ Open Google ColabPaste the code into a new Colab notebook and press Run. Keras is already installed.

What the output tells us

  • Early stopping saved time: we allowed up to 200 epochs, but training stopped after 84 and rolled back to the best weights, from epoch 74.
  • ReduceLROnPlateau halved the learning rate from 0.001 to 0.0005 when progress stalled.
  • Test accuracy is 97.8% on 360 digits the model never saw.
  • predict() plus argmax turned probabilities into digits: all 5 correct. The confidence shows the model was less sure about the 8 (79%) and the 1 (85%) than the others.
  • A single image needed reshaping from (64,) to (1, 64).
  • ModelCheckpoint saved the best model to a file, and reloading it gives the same 97.8%.

SummaryKey takeaways

  • fit() trains the model; set epochs, batch_size and a validation set.
  • An epoch is one pass through the training data; each batch update is a step.
  • Plot training and validation loss together from history.history.
  • Rising validation loss means overfitting; both high means underfitting; jumpy means the learning rate is too high.
  • EarlyStopping, ModelCheckpoint and ReduceLROnPlateau make training safer and faster.
  • Use evaluate() on the test set once, at the end.
  • Turn predict() output into answers with a threshold or argmax, and add a batch dimension for single examples.

Check your understanding

Your Keras roadmap

  1. 01
    Introduction to KerasWhat Keras is, setup and your first model
  2. 02
    Tensors and Data for KerasShapes, scaling and splitting data
  3. 03
    The Sequential API and Dense LayersStacking layers into a model
  4. 04
    Compiling ModelsLosses, optimisers and metrics
  5. 05
    Training and EvaluatingYou are here
  6. 06
    The Functional APIMultiple inputs, outputs and branches
  7. 07
    RegularisationDropout, batch normalisation and early stopping
  8. 08
    Image Classification with KerasCNNs and data augmentation
  9. 09
    Text and Sequences with KerasEmbeddings, LSTMs and text classification
  10. 10
    Pretrained Models, Saving and DeploymentFine-tuning and sharing your models