Computer Vision: Start Here
The main vision tasks, what a junior CV engineer does, the tools of the field, and how this track and its game season are organised.
What computer vision covers
Computer vision (CV) is about getting useful information from images and video. Most real projects are one of a few tasks, and choosing the right one is the first design decision:
| Task | Output | Example |
|---|---|---|
| Classification | one label (or several) per image | Which type of document is this scan? Is this skin lesion suspicious? |
| Object detection | a box + label + score for every object | Find every handwritten field on a referral form; count cells in a microscope image. |
| Semantic segmentation | a label for every pixel | Outline the tumour in an MRI slice; mark road vs pavement. |
| Instance segmentation | a separate mask for every object | Separate touching cells; measure each wound. |
| Keypoints / pose | coordinates of landmarks | Joint positions for physiotherapy exercises. |
| Embedding / retrieval | a vector per image | Find similar X-rays; zero-shot labels with CLIP. |
| OCR | text and its location | Read the typed parts of a form. |
Video adds time (tracking, action recognition), and 3-D adds depth (point clouds, CT volumes); the same building blocks apply.
What junior CV roles involve, and the tools
Most day-to-day CV work is not inventing architectures. It is: getting and checking data (labelling guidelines, label quality, splits by patient or site), fine-tuning a pretrained model, evaluating it properly with the task's metric (mAP for detection, Dice/IoU for segmentation), finding failure cases, and making it fast and robust enough to deploy.
The standard tools: PyTorch with torchvision; timm (hundreds of pretrained image models); Ultralytics YOLO and Detectron2 for detection and segmentation; Albumentations for augmentation; OpenCV and Pillow for image processing; MONAI for medical imaging; Hugging Face transformers for ViT, CLIP and segmentation models; labelling tools such as CVAT and Label Studio.
Prerequisite: the Deep Learning track, especially CNNs in depth and transfer learning.
How this track is organised
| Topics | Game |
|---|---|
| Images & augmentation ยท Image classification in practice ยท Object detection | C1: Reading the referral forms |
| Segmentation ยท Vision transformers & CLIP ยท Robustness, fairness & deployment | C2: The wound-care app |
As in the Deep Learning track, the browser exercises are in NumPy and the Colab notebooks (09 and 10, below) use PyTorch on a GPU.
Hands-on: Colab notebooks for this track
09: fine-tune a pretrained classifier with augmentation, Grad-CAM and error analysis. 10: fine-tune an object detector and evaluate it with mAP. Copy a notebook's code into Colab, or upload the .ipynb file.
Interactive: this part runs in your browser.
Quiz
1. You need to measure the area of every individual wound in a photo, even when two touch. Which task?
2. Which task outputs a label for every pixel?
3. Most junior CV work consists mainly ofโฆ
4. Which library is built for medical imaging?
๐ Further reading
Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.
- CS231n: Deep Learning for Computer Vision (Stanford) โ Notes on backprop, CNNs and training
- timm documentation (Hugging Face) โ Pretrained image models with benchmarks
- Ultralytics YOLO docs โ Training, validation and deployment of YOLO models