Introduction to Neural Networks

1.1What is a neural network?

Your brain contains around 86 billion tiny cells called neurons. Each one is very simple: it receives signals from other neurons and decides whether to pass a signal on. But connected together in huge numbers, they let you recognise faces, understand speech and learn new skills.

A neural network borrows that idea. It's made of lots of simple "artificial neurons", each doing a small calculation, connected together in layers. On its own, one artificial neuron can't do much. Thousands or millions of them working together can learn to recognise images, understand language and much more.

1.2Where neural networks fit

If you've taken our Machine Learning track, you'll remember this picture. Neural networks sit at the heart of deep learning, which is a branch of machine learning.

1.3The artificial neuron

An artificial neuron does four simple things:

If that looks familiar, it should. With the sigmoid activation, a single neuron is exactly the logistic regression model from Module 4 of the Machine Learning track. A neural network is, in a sense, lots of small logistic regressions stacked together.

Let's make it real. Imagine one neuron deciding: "Should I go to the park?"

Here's the key question: who chooses the weights and bias? In the lab, you did. In a real neural network, the network learns them from examples. That's what training means. You'll see how in lesson 1.5.

1.4From one neuron to a network

One neuron can only draw a straight-line boundary, just like logistic regression. The power comes from connecting many neurons in layers, where each layer's outputs become the next layer's inputs.

What makes a network "deep"?

A network with several hidden layers is called a deep neural network, and training one is called deep learning. Modern networks can have dozens or even hundreds of layers and billions of weights.

1.5How a neural network learns

A new network starts with random weights, so its first predictions are pure guesses. Training improves them step by step, using a loop you'll recognise from the Machine Learning track:

One pass through the whole training dataset is called an epoch. Training usually takes many epochs. You'll learn exactly how backpropagation and gradient descent work in Modules 4 and 5.

1.6A short history: why now?

Neural networks aren't new. The first ideas date from the 1940s. So why did they only change the world in the last 15 years?

Three things finally came together:

1.7The main types of neural network

Different kinds of data need different network designs, called architectures. You'll learn each of these later in this track.

1.8When to use a neural network (and when not to)

Neural networks are powerful, but they're not always the best tool. As you saw in the Machine Learning track, for data in rows and columns, gradient boosting often wins.

1.9Neural networks in your everyday life

1.10Your first neural network in Python

Later in this track you'll use dedicated deep learning libraries. To start, scikit-learn has a simple neural network called MLPClassifier (MLP stands for "multi-layer perceptron", another name for a feedforward network). Let's teach it to read the handwritten digits you may remember from the Machine Learning track.

from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from sklearn.neural_network import MLPClassifier

# 1. The data: 1,797 handwritten digits, each 8x8 = 64 pixels
X, y = load_digits(return_X_y=True)
X = MinMaxScaler().fit_transform(X)          # pixel values from 0 to 1
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)

# 2. Build a network: 64 inputs -> 32 neurons -> 16 neurons -> 10 outputs
net = MLPClassifier(hidden_layer_sizes=(32, 16),
                    max_iter=1000, random_state=42)

# 3. Train it: guess, measure the error, adjust the weights, repeat
net.fit(X_train, y_train)
for epoch in [1, 5, 20, 50, 100]:
    print(f"Epoch {epoch:3}: loss = {net.loss_curve_[epoch - 1]:.3f}")

# 4. Test it on digits it has never seen
print("Test accuracy:", round(net.score(X_test, y_test), 3))

# 5. How many weights did it learn?
n_weights = sum(w.size for w in net.coefs_) + sum(b.size for b in net.intercepts_)
print("Weights and biases learned:", n_weights)
print("Prediction for the first test digit:", net.predict(X_test[:1])[0],
      "| Real answer:", y_test[0])

Epoch   1: loss = 2.338
Epoch   5: loss = 2.105
Epoch  20: loss = 0.925
Epoch  50: loss = 0.186
Epoch 100: loss = 0.069
Test accuracy: 0.973
Weights and biases learned: 2778
Prediction for the first test digit: 1 | Real answer: 1

What the output tells us

• The network has 4 layers: 64 inputs (one per pixel), two hidden layers of 32 and 16 neurons, and 10 outputs (one per digit).
• The loss falls from 2.34 (random guessing) to 0.07 as the weights are adjusted epoch after epoch.
• 97.3% test accuracy on digits it has never seen, from a network that started out knowing nothing.
• It learned 2,778 weights and biases by itself. Nobody told it what a "7" looks like.

SummaryKey takeaways

Check your understanding

Your neural networks roadmap

1. 01
   Introduction to Neural NetworksYou are here
2. 02
   The Perceptron and the Artificial NeuronBuild a single neuron from scratch
3. 03
   Activation FunctionsSigmoid, tanh, ReLU and softmax
4. 04
   Loss Functions and Gradient DescentHow networks measure and reduce error
5. 05
   BackpropagationHow every weight learns its share of the blame
6. 06
   Building Networks with KerasYour first deep learning library
7. 07
   Training Better NetworksOverfitting, dropout, batch normalisation, early stopping
8. 08
   Convolutional Neural NetworksTeaching computers to see
9. 09
   Recurrent Neural Networks and LSTMsWorking with sequences
10. 10
    Transformers and AttentionThe architecture behind ChatGPT and Claude
11. 11
    Transfer Learning and Pretrained ModelsStanding on the shoulders of giants
12. 12
    Capstone ProjectBuild and present a deep learning project