The Perceptron & Artificial Neuron

2.1What is a perceptron?

In 1958, psychologist Frank Rosenblatt built a machine called the Mark I Perceptron. It was a room-sized machine wired to a 20 × 20 grid of light sensors, and it could learn to tell simple shapes apart. Newspapers at the time were very excited about machines that could "learn".

The idea behind it is simple. A perceptron is a single artificial neuron that:

1. takes some inputs,
2. multiplies each one by a weight and adds them up, plus a bias,
3. outputs 1 ("yes") if the total is zero or more, and 0 ("no") if it's below zero.

2.2Working out a perceptron by hand

A college uses a perceptron to flag whether a student is likely to pass an exam, based on two inputs: hours studied (x₁) and hours of sleep the night before (x₂). Its weights are w₁ = 0.6 and w₂ = 0.4, and its bias is b = −5.

That's all a perceptron does. Notice how studying counts more than sleep (0.6 vs 0.4), and the bias of −5 means a student needs a fair amount of both to reach the "pass" line.

What the weights and bias do

2.3How a perceptron learns

In the example above, we chose the weights. The real magic of Rosenblatt's perceptron is that it can learn them from examples, using a very simple rule. For every training example:

1. Predict: work out the output (0 or 1).
2. Compare: error = correct answer − prediction. This can only be +1 (said no, should have said yes), −1 (said yes, should have said no) or 0 (correct).
3. Adjust: if there was an error, nudge each weight and the bias in the direction that fixes it.

If the answer was right, the error is 0 and nothing changes. If the perceptron said "no" when it should have said "yes", the weights on the active inputs go up, making "yes" more likely next time. And the opposite for a wrong "yes".

Watching it learn the AND rule

The AND rule outputs 1 only when both inputs are 1. Let's train a perceptron on it, starting with all weights at 0 and a learning rate of 0.1. Here are the first two epochs (real values from the Python code in lesson 2.7):

After 4 epochs, every answer is correct and the weights stop changing. The perceptron has learned AND by itself.

2.4The decision boundary

Remember logistic regression from the Machine Learning track? A perceptron draws the same kind of boundary: a straight line. Everything on one side gets 1, everything on the other side gets 0. The weights set the angle of the line, and the bias shifts it.

Every time the perceptron makes a mistake and updates its weights, the line moves a little. Watch it happen with 26 students:

The learning rate

The learning rate controls how big each nudge is (0.1 in the lab). Here's a surprise: for a perceptron that starts with all weights at zero, the learning rate makes no difference to where the line ends up. Every weight and the bias get scaled by the same amount, so the line is in exactly the same place. In modern networks it's a very different story: choosing a good learning rate is one of the most important decisions you'll make (Module 4).

2.5The XOR problem

Some problems can't be solved with one straight line. The most famous is XOR ("exclusive or"): the answer is 1 when exactly one input is 1, but 0 when both are 0 or both are 1. For example: "I'll go out if either Sam or Alex comes, but not both, because they don't get on."

A single perceptron can never learn XOR, however long you train it. In the Python example, it gets stuck at 2 out of 4 correct.

In 1969, Marvin Minsky and Seymour Papert published a book, Perceptrons, which set out limits like this in detail. Research funding for neural networks dried up for years, a period now called the first "AI winter".

The solution: add a hidden layer

The fix is to use more than one neuron, in layers. Two hidden neurons can each draw their own line, and an output neuron combines them:

This is the key idea behind all of deep learning: stacking layers lets a network build complex shapes out of simple straight lines. But there was one big problem left, which took until the 1980s to solve.

2.6From the perceptron to the modern neuron

If hidden layers solve XOR, why did it take so long? Because nobody knew how to train the hidden layers. The perceptron rule only works for a single neuron, where we can see its error directly. For a hidden neuron, what's the "correct answer"? Nobody tells us.

The answer (backpropagation, Module 5) needs a way to measure how a small change in a weight changes the output. The step function makes that impossible:

So modern networks replace the step function with smooth activation functions. You'll meet the most important ones in the next module.

2.7Building a perceptron in Python

A perceptron is simple enough to build from scratch in about 20 lines of Python, using NumPy. We'll teach it the AND, OR and XOR rules, then try scikit-learn's built-in Perceptron on a real dataset.

import numpy as np

# 1. A perceptron built from scratch
class Perceptron:
    def __init__(self, learning_rate=0.1, epochs=20):
        self.lr = learning_rate
        self.epochs = epochs

    def predict(self, X):
        total = X @ self.w + self.b                 # weighted sum + bias
        return np.where(total >= 0, 1, 0)           # step activation

    def fit(self, X, y):
        self.w = np.zeros(X.shape[1])               # start with all weights at 0
        self.b = 0.0
        for epoch in range(self.epochs):
            mistakes = 0
            for xi, target in zip(X, y):
                error = target - self.predict(xi)   # +1, 0 or -1
                self.w += self.lr * error * xi      # the perceptron learning rule
                self.b += self.lr * error
                mistakes += int(error != 0)
            if mistakes == 0:                       # everything correct: stop
                return epoch + 1
        return self.epochs

# 2. Teach it three logic gates
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
gates = {"AND": [0, 0, 0, 1], "OR": [0, 1, 1, 1], "XOR": [0, 1, 1, 0]}
for name, y in gates.items():
    p = Perceptron()
    epochs = p.fit(X, np.array(y))
    correct = (p.predict(X) == y).sum()
    print(f"{name:3}: {correct}/4 correct after {epochs} epochs, "
          f"weights={p.w.round(2)}, bias={p.b:.2f}")

# 3. scikit-learn's Perceptron on a real medical dataset
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Perceptron as SkPerceptron

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = Pipeline([("scale", StandardScaler()),
                  ("perceptron", SkPerceptron(random_state=42))])
model.fit(X_train, y_train)
print(f"\nBreast cancer data (30 features): test accuracy = {model.score(X_test, y_test):.3f}")

AND: 4/4 correct after 4 epochs, weights=[0.2 0.1], bias=-0.20
OR : 4/4 correct after 4 epochs, weights=[0.1 0.1], bias=-0.10
XOR: 2/4 correct after 20 epochs, weights=[-0.1  0. ], bias=0.00

Breast cancer data (30 features): test accuracy = 0.965

What the output tells us

• AND and OR are learned perfectly in just 4 epochs, starting from weights of zero.
• XOR fails. After all 20 epochs, it still only gets 2 of the 4 right. It will never do better, because no straight line can separate XOR.
• On real data, a single perceptron can do very well: 96.5% on the breast cancer dataset you may know from the Machine Learning track. Many real problems are close to straight-line separable once you have enough features.
• X @ self.w is NumPy's way of writing the weighted sum: it multiplies each input by its weight and adds them up in one step.

SummaryKey takeaways

Check your understanding

Your neural networks roadmap

1. 01
   Introduction to Neural NetworksWhat neural networks are and how they work
2. 02
   The Perceptron and the Artificial NeuronYou are here
3. 03
   Activation FunctionsSigmoid, tanh, ReLU and softmax
4. 04
   Loss Functions and Gradient DescentHow networks measure and reduce error
5. 05
   BackpropagationHow every weight learns its share of the blame
6. 06
   Building Networks with KerasYour first deep learning library
7. 07
   Training Better NetworksOverfitting, dropout, batch normalisation, early stopping
8. 08
   Convolutional Neural NetworksTeaching computers to see
9. 09
   Recurrent Neural Networks and LSTMsWorking with sequences
10. 10
    Transformers and AttentionThe architecture behind ChatGPT and Claude
11. 11
    Transfer Learning and Pretrained ModelsStanding on the shoulders of giants
12. 12
    Capstone ProjectBuild and present a deep learning project