Computer Vision
Topic 8 · Computer Vision · ⏱ about 6 h

Robustness, Fairness & Deploying Vision Models

Distribution shift between devices and sites, shortcut learning, adversarial examples, fairness across skin tones and groups, and taking a model to phones, edge devices and production.

Show:
In this topic
Beginner

Distribution shift is the main failure mode

A vision model that scores 95% on its test set can fail badly in deployment because the new images differ: a different camera or scanner, lighting, compression, resolution, patient population, or the way staff take photos. This distribution shift is the most common reason vision models disappoint. Defences:

  • Test on an external set from a different site, device or time period before claiming anything; report results per site.
  • Augment for the variation you expect (blur, brightness, compression, colour casts).
  • Monitor input statistics (brightness, sharpness, embedding drift) and confidence in production, and sample cases for human review.
  • Quality gates: reject blurry or badly framed photos at capture time and ask the user to retake.
Try it

Try it: the model meets a new clinic

A wound classifier trained on photos from one clinic's camera is used elsewhere. Change the conditions and see how accuracy changes, with and without training-time augmentation.

Interactive: this part runs in your browser.

Beginner

Shortcuts and adversarial examples

Shortcut learning: models latch onto easy cues correlated with the label in the training data: rulers or pen marks next to suspicious skin lesions, hospital-specific text on X-rays, the background behind animals. Find them with Grad-CAM, error analysis by subgroup, and tests where the cue is removed.

Adversarial examples: tiny, carefully computed changes to the pixels (invisible to people) can flip a prediction. In most clinical settings deliberate attacks are less likely than natural shift, but the phenomenon shows how differently networks "see". Adversarial training and input checks improve robustness at some cost to accuracy.

Beginner

Fairness in vision

Vision datasets are often unbalanced across people: dermatology image collections have historically contained mostly lighter skin tones, and face datasets have skewed by age, sex and ethnicity. Models then work worse for under-represented groups. Practices:

  • Record relevant attributes (e.g. Fitzpatrick skin type for dermatology) for evaluation, with consent and privacy safeguards.
  • Report performance per group, with confidence intervals; a small group needs enough examples for a meaningful estimate.
  • Collect more data for under-represented groups rather than only re-weighting.
  • Document limits in a model card: "not validated on skin types V–VI" is information clinicians need.
Beginner

Deploying vision models

  • Where it runs: server GPU (batch many images), CPU server, phone (Core ML, TensorFlow Lite / LiteRT, ONNX Runtime Mobile), or edge device (NVIDIA Jetson). Choose the model size for the target: MobileNet/EfficientNet-Lite or a small YOLO on phones.
  • Make it fast: export to ONNX/TensorRT, quantise to int8 with a calibration set, use the smallest input resolution that keeps accuracy (see Efficient training & deployment).
  • Same preprocessing everywhere: resize method, colour order, normalisation. Test the deployed pipeline end to end on known images.
  • Privacy: photos of patients are sensitive. On-device inference can avoid sending images to a server; store only what is needed, with consent.
  • Human in the loop: show confidence, route uncertain cases to people, and log feedback for retraining.
Try it

Code example: per-site and per-group evaluation

A worked example: read it step by step (every part is explained), then run it and experiment.

Interactive: this part runs in your browser.

Check yourself

Quiz

1. Your model scores 95% on a random test split but 78% at a new clinic. The most likely cause?

2. A quality gate at capture time…

3. You have only 25 test images of skin type VI. Reporting "accuracy 92% for type VI"…

4. Which deployment detail most often silently breaks a vision model?

5. Adversarial examples show that…

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready