✅ Checkpoint: Deep Learning track
A mixed quiz on the whole Deep Learning track. Pass mark: 8 of 11 on the first try.
How this checkpoint works
Answer every question without looking back. Only your first try counts. If you pass, you are ready for the Computer Vision and NLP tracks. If not, the result lists the topics to revisit.
Quiz
1. Logits have shape (B, 1) and labels (B,). What should you do before computing an element-wise loss?
2. For Y = XW, the gradient of the loss with respect to W is…
3. A 20-layer ReLU MLP's activations shrink to ~0 by layer 20 at initialisation. The best fix is…
4. Which optimiser setting is a sensible default for fine-tuning a transformer?
5. Your validation accuracy is noisy and lower than in training, and you never call model.eval(). Two layers behaving differently in eval are…
6. A model cannot reach low loss on a batch of 8 examples. Most likely…
7. Conv2d(32, 64, 3, stride=2, padding=1) on a 32 × 32 input gives…
8. Why can LSTMs learn longer dependencies than plain RNNs?
9. In attention, softmax(QKᵀ/√d)V: the softmax is taken over…
10. With 500 labels and a pretrained model trained on similar data, start with…
11. A 7-billion-parameter model is served in bf16. Roughly how much GPU memory do the weights need?
📚 Further reading
Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.
- Dive into Deep Learning (free book with code) ↗ Interactive textbook in PyTorch, by Zhang, Lipton, Li & Smola
- Understanding Deep Learning (Prince) ↗ Modern, clear and free textbook