Building a model describes its shape. Compiling it decides how it learns. In one line, you choose the loss (how mistakes are measured), the optimiser (how weights are improved) and the metrics (how progress is reported). Get these right and training goes smoothly; get them wrong and your model can fail silently.
- Level: Beginner
- Time: about 35 minutes
- Needs: Modules 1 to 3
By the end of this module you will be able to
- Explain what model.compile() does
- Choose the right loss function for any task
- Match the loss to your label format and output layer
- Explain the from_logits option
- Choose an optimiser and set the learning rate
- Add metrics like precision, recall and AUC
- Spot common compile mistakes before they waste your time
4.1What does compile() do?
Compiling prepares a model for training. You make three choices:
model.compile(
loss="binary_crossentropy", # 1. How to measure mistakes
optimizer="adam", # 2. How to improve the weights
metrics=["accuracy"], # 3. What to report while training
)
Like a fitness coach
The loss is the measurement the coach tries to improve, like your time over 5 km. The optimiser is the training plan that decides how to improve it. The metrics are extra numbers the coach reports to you, like your heart rate, which you care about but which don't drive the plan.
You must compile before calling fit(). You can also compile again at any time to change the settings, for example to lower the learning rate.
4.2Choosing a loss function
The loss is the single number the model tries to make as small as possible during training. It must suit your task:
| Task | Output layer | Loss |
|---|---|---|
| Predict a number | Dense(1) | "mse" (mean squared error), or "mae" if there are extreme outliers |
| Yes or no | Dense(1, activation="sigmoid") | "binary_crossentropy" |
| One of several classes, integer labels (0, 1, 2…) | Dense(n, activation="softmax") | "sparse_categorical_crossentropy" |
| One of several classes, one-hot labels | Dense(n, activation="softmax") | "categorical_crossentropy" |
| Several labels at once | Dense(n, activation="sigmoid") | "binary_crossentropy" |
Cross-entropy, used for classification, rewards being confidently right and punishes being confidently wrong very heavily. If the correct answer is "benign", predicting 95% benign gives a tiny loss, while predicting 5% benign gives a big one.
Strings or objects
You can write losses as strings like "binary_crossentropy", or as objects like keras.losses.BinaryCrossentropy(). Objects let you change settings, which you need for one important option.
The from_logits option
Some models leave out the final sigmoid or softmax and output raw scores, called logits. If you do that, you must tell the loss, or it will give wrong results without any error:
# Output layer with NO activation: raw scores (logits)
layers.Dense(5)
model.compile(
optimizer="adam",
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=["accuracy"],
)
This is slightly more numerically stable, and you'll see it in many tutorials and pretrained models. As a beginner, the simplest rule is: use sigmoid or softmax in the output layer, and leave from_logits at its default (False).
4.3Check your setup
The output layer, the loss and the label format must all agree. A mismatch often doesn't crash; the model just learns badly. Try some combinations:
Try it: compile checker
4.4Optimisers and the learning rate
After each batch, the optimiser nudges every weight in the direction that reduces the loss. The size of each nudge is set by the learning rate, the most important setting in deep learning.
| Optimiser | How it works | Typical learning rate |
|---|---|---|
| SGD | Plain gradient descent: a fixed step downhill. Add momentum=0.9 to build up speed | 0.01 |
| RMSprop | Gives each weight its own step size, smaller for jumpy ones | 0.001 |
| Adam | Combines momentum and per-weight step sizes. The best default | 0.001 |
| AdamW | Adam plus weight decay, which helps against overfitting; popular for large models | 0.001 |
Writing optimizer="adam" uses the default learning rate of 0.001. To choose your own, use an optimiser object:
model.compile(
optimizer=keras.optimizers.Adam(learning_rate=0.0005),
loss="binary_crossentropy",
metrics=["accuracy"],
)
| Learning rate | What you'll see |
|---|---|
| Too low | The loss falls painfully slowly |
| About right | The loss falls quickly, then levels off |
| Too high | The loss jumps around wildly, or becomes huge or nan |
4.5Metrics
Metrics are extra scores Keras reports during training and evaluation. They don't affect learning; they help you understand it. Accuracy alone can mislead, especially when one class is rare, so add metrics that fit your problem:
| Metric | Question it answers | In Keras |
|---|---|---|
| Accuracy | What share of predictions were right? | "accuracy" |
| Precision | When the model says "yes", how often is it right? | keras.metrics.Precision() |
| Recall | Of all the real "yes" cases, how many did it find? | keras.metrics.Recall() |
| AUC | How well does it rank "yes" cases above "no" cases, from 0.5 (guessing) to 1 (perfect)? | keras.metrics.AUC() |
| MAE | For numbers: how far off is it, on average? | "mae" |
Give each metric a name, like keras.metrics.Recall(name="recall"), to get tidy labels in your results.
4.6Compile choices in Python
Using the breast cancer dataset (569 tumours, 30 measurements), we'll compile a model with several metrics, compare three optimisers and four learning rates, and see what happens with a classic mistake.
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
import keras
from keras import layers
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
# Breast cancer data: 569 tumours, 30 measurements, 1 = benign, 0 = malignant
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
X.astype("float32"), y, test_size=0.25, random_state=42, stratify=y)
norm = layers.Normalization()
norm.adapt(X_train)
def build(output_layer):
keras.utils.set_random_seed(0)
return keras.Sequential([keras.Input(shape=(30,)), norm,
layers.Dense(16, activation="relu"), output_layer])
# ---------- 1. A well-compiled model with several metrics ----------
model = build(layers.Dense(1, activation="sigmoid"))
model.compile(
optimizer=keras.optimizers.Adam(learning_rate=0.001),
loss="binary_crossentropy",
metrics=["accuracy", keras.metrics.Precision(name="precision"),
keras.metrics.Recall(name="recall"), keras.metrics.AUC(name="auc")])
model.fit(X_train, y_train, epochs=30, batch_size=32, verbose=0)
results = model.evaluate(X_test, y_test, verbose=0, return_dict=True)
print({k: round(v, 3) for k, v in results.items()})
# ---------- 2. Comparing optimisers ----------
print()
for name, opt in [("SGD", keras.optimizers.SGD(learning_rate=0.01)),
("RMSprop", keras.optimizers.RMSprop(learning_rate=0.001)),
("Adam", keras.optimizers.Adam(learning_rate=0.001))]:
m = build(layers.Dense(1, activation="sigmoid"))
m.compile(optimizer=opt, loss="binary_crossentropy", metrics=["accuracy"])
h = m.fit(X_train, y_train, epochs=10, batch_size=32, verbose=0)
print(f"{name:8} loss after 10 epochs = {h.history['loss'][-1]:.3f}")
# ---------- 3. Comparing learning rates (Adam) ----------
print()
for lr in [0.00001, 0.001, 0.01, 10.0]:
m = build(layers.Dense(1, activation="sigmoid"))
m.compile(optimizer=keras.optimizers.Adam(learning_rate=lr),
loss="binary_crossentropy", metrics=["accuracy"])
h = m.fit(X_train, y_train, epochs=10, batch_size=32, verbose=0)
test_loss, test_acc = m.evaluate(X_test, y_test, verbose=0)
print(f"learning_rate={lr:<7} test loss={test_loss:8.3f} test accuracy={test_acc:.3f}")
print("Training loss by epoch at 10.0:", [round(v) for v in h.history["loss"]])
# ---------- 4. A classic mistake: softmax with ONE output neuron ----------
wrong = build(layers.Dense(1, activation="softmax"))
wrong.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
wrong.fit(X_train, y_train, epochs=10, batch_size=32, verbose=0)
print("\nSoftmax with 1 neuron always predicts:",
wrong.predict(X_test[:5], verbose=0).ravel(),
f"| test accuracy = {wrong.evaluate(X_test, y_test, verbose=0)[1]:.3f}")
{'accuracy': 0.944, 'auc': 0.995, 'loss': 0.113, 'precision': 0.956, 'recall': 0.956}
SGD loss after 10 epochs = 0.307
RMSprop loss after 10 epochs = 0.169
Adam loss after 10 epochs = 0.220
learning_rate=1e-05 test loss= 1.107 test accuracy=0.308
learning_rate=0.001 test loss= 0.261 test accuracy=0.895
learning_rate=0.01 test loss= 0.085 test accuracy=0.965
learning_rate=10.0 test loss= 490.764 test accuracy=0.965
Training loss by epoch at 10.0: [553, 574, 397, 117, 25, 6, 4, 126, 393, 67]
Softmax with 1 neuron always predicts: [1. 1. 1. 1. 1.] | test accuracy = 0.629What the output tells us
- Several metrics at once: the model reached 94.4% accuracy with precision and recall of 95.6% each, and an AUC of 0.995, showing it ranks benign tumours above malignant ones almost perfectly.
- Optimisers learn at different speeds. After 10 epochs, RMSprop and Adam had lower loss than plain SGD. With more epochs, all three would do well here; Adam is simply a reliable default.
- A learning rate of 0.00001 is far too low: after 10 epochs the model has barely moved from its random start and scores just 30.8%.
- 0.01 was the best here, with the lowest test loss. Adam's default of 0.001 was still improving.
- A learning rate of 10 is far too high. The training loss leaps around between 4 and 574, and the test loss is 490. Accuracy still looks fine because this dataset is easy, but a test loss that high means the model is wildly overconfident whenever it's wrong. Always watch the loss, not just accuracy.
- Softmax with one neuron always outputs exactly 1, so the model can only ever say "benign". Its 62.9% accuracy is simply the share of benign tumours, and there was no error message to warn you. The compile checker above catches this.
SummaryKey takeaways
compile()sets the loss, the optimiser and the metrics. Always compile beforefit().- Match the loss to the task: mse for numbers, binary_crossentropy for yes/no, sparse_categorical_crossentropy for classes with integer labels.
- If your output layer has no activation, use
from_logits=Truein the loss. - Adam with a learning rate around 0.001 is a great default; use an optimiser object to change it.
- Too low a learning rate is slow; too high makes the loss jump around or explode.
- Add precision, recall and AUC when accuracy isn't enough.
- Mismatched outputs, losses and labels often fail silently, so check them.
Check your understanding
Your Keras roadmap
- 01Introduction to KerasWhat Keras is, setup and your first model
- 02Tensors and Data for KerasShapes, scaling and splitting data
- 03The Sequential API and Dense LayersStacking layers into a model
- 04Compiling ModelsYou are here
- 05Training and Evaluatingfit(), validation and callbacks
- 06The Functional APIMultiple inputs, outputs and branches
- 07RegularisationDropout, batch normalisation and early stopping
- 08Image Classification with KerasCNNs and data augmentation
- 09Text and Sequences with KerasEmbeddings, LSTMs and text classification
- 10Pretrained Models, Saving and DeploymentFine-tuning and sharing your models
Next module
Training and Evaluating