Deep Learning
Topic 10 · Deep Learning · ⏱ about 8 h

Convolutional Networks in Depth

Why convolution works for images, the shape arithmetic you need every day (padding, stride, channels, receptive field), pooling and 1×1 convolutions, and the architectures from LeNet to ResNet and ConvNeXt.

Show:
In this topic
Beginner

Why convolution

A dense layer on a 224 × 224 × 3 image would need 150,528 weights per hidden unit, and would have to learn "an edge" separately at every position. Images have structure that a convolution exploits:

  • Locality: nearby pixels are related, so each output looks only at a small window (e.g. 3 × 3).
  • Weight sharing: the same small filter slides over every position, so a pattern learned in one place is detected everywhere. A 3 × 3 filter on RGB has 27 weights + 1 bias, whatever the image size.
  • Translation equivariance: shift the input, and the feature map shifts the same way. Pooling then adds some invariance to small shifts.

Stacking convolutions builds a hierarchy: early layers detect edges and colours, middle layers textures and parts, and late layers whole objects. The core course's architectures topic shows a filter sliding over an image; this topic is about the arithmetic and design decisions.

Beginner

Watch it: Padding and stride: how a convolution sets the output size

A short narrated animation. Press ▶ Play, or step through it with ⏮ ⏭. Tick 🔊 narrate to hear each step.

Interactive: this part runs in your browser.

Beginner

The shape arithmetic

A Conv2d(C_in, C_out, kernel_size=k, stride=s, padding=p) layer has C_out filters, each of size k × k × C_in. Each filter produces one output channel. For an input of height (or width) n:

nout=⌊n+2p−ks⌋+1,parameters=Cout (k2Cin+1).n_\text{out}=\left\lfloor\frac{n+2p-k}{s}\right\rfloor+1,\qquad \text{parameters}=C_\text{out}\,(k^2 C_\text{in}+1).
  • Padding p = (k − 1)/2 with stride 1 ("same" padding) keeps the size: a 3 × 3 conv with p = 1.
  • Stride 2 halves the size: the standard way to downsample in modern CNNs.
  • Channels usually double when the size halves (64 → 128 → 256 → 512), keeping the compute per layer similar.

The receptive field is the region of the input that can influence one output unit. Each 3 × 3 conv adds 2 pixels (times the product of earlier strides), so two stacked 3 × 3 convs see 5 × 5 and three see 7 × 7, with fewer parameters and more nonlinearity than one big filter: the VGG insight.

Try it

Try it: conv arithmetic calculator

Build a small stack of layers and see the output shape, parameters and receptive field at each step. An invalid configuration (output below 1) is flagged.

Interactive: this part runs in your browser.

Beginner

Pooling, 1×1 and depthwise convolutions

Max pooling (2 × 2, stride 2) keeps the largest value in each window: it halves the size and adds a little shift invariance. Global average pooling averages each channel over all positions, turning (B, C, H, W) into (B, C) before the final linear classifier; it replaced the huge dense layers of early networks and lets the network accept any image size.

A 1 × 1 convolution mixes channels at each position without looking at neighbours: a cheap way to change the number of channels (the "bottleneck" in ResNet). A depthwise separable convolution splits a normal convolution into a per-channel spatial filter followed by a 1 × 1 channel mix, cutting computation by roughly k² (about 9×): the basis of MobileNet and EfficientNet for phones and edge devices.

Beginner

Architectures that shaped the field

YearNetworkKey idea
1998LeNet-5Conv → pool → conv → pool → dense: read handwritten cheques.
2012AlexNetDeep CNN on GPUs with ReLU and dropout; won ImageNet by a large margin and started the deep-learning boom.
2014VGGOnly 3 × 3 convs, stacked deep: simple and uniform.
2015ResNetResidual connections and BatchNorm: 50 to 152 layers train reliably. Still a standard baseline and backbone.
2019EfficientNetScales depth, width and resolution together; depthwise convs.
2020Vision Transformer (ViT)Split the image into patches and use a transformer (see the Computer Vision track).
2022ConvNeXtA CNN modernised with transformer-era design choices; competitive with ViTs.

In practice you rarely design a CNN from scratch: you take a pretrained backbone (ResNet-50, EfficientNet, ConvNeXt) and fine-tune it (see Transfer learning).

Try it

Code example: convolution and pooling from scratch

A worked example: read it step by step (every part is explained), then run it and experiment.

Interactive: this part runs in your browser.

Try it

Code example: convolution features for scanned digits

A worked example: read it step by step (every part is explained), then run it and experiment.

Interactive: this part runs in your browser.

Code

Code: a small CNN in PyTorch

A compact, modern-style CNN for 1 × 28 × 28 images, with the shape after each stage in comments.

Python · PyTorch
import torch, torch.nn as nn
def block(cin, cout):
    return nn.Sequential(nn.Conv2d(cin, cout, 3, padding=1, bias=False), nn.BatchNorm2d(cout), nn.ReLU(),
                         nn.Conv2d(cout, cout, 3, padding=1, bias=False), nn.BatchNorm2d(cout), nn.ReLU())
model = nn.Sequential(
    block(1, 32),  nn.MaxPool2d(2),          # (B, 32, 14, 14)
    block(32, 64), nn.MaxPool2d(2),          # (B, 64, 7, 7)
    block(64, 128),                          # (B, 128, 7, 7)
    nn.AdaptiveAvgPool2d(1), nn.Flatten(),   # (B, 128)  global average pooling
    nn.Dropout(0.2), nn.Linear(128, 10))     # (B, 10)   logits
x = torch.randn(16, 1, 28, 28)
print(model(x).shape, sum(p.numel() for p in model.parameters()))
This code needs PyTorch, which cannot run inside a web page. To run it, press Copy, then paste it into a new notebook in Google Colab ↗ (free; Runtime → Change runtime type → GPU for deep learning).
Check yourself

Quiz

1. A Conv2d(3, 16, kernel_size=5, stride=1, padding=0) on a 32 × 32 image gives…

2. How many parameters does Conv2d(64, 128, 3) have (with bias)?

3. Why stack two 3 × 3 convs instead of one 5 × 5?

4. What does global average pooling do?

5. What is a 1 × 1 convolution mainly used for?

6. Which innovation made 100+ layer CNNs trainable?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready