Machine Learning
Topic 35 ยท Beyond the basics ยท โฑ about 10 h

Deploying Models: APIs, Docker, Tracking & CI

From a notebook to a service: saving models, batch vs real-time, FastAPI, input validation, Docker, experiment tracking, CI and monitoring.

Show:
In this topic
Beginner

From notebook to product

A model in a notebook helps nobody. Deployment means making its predictions available where decisions are made, reliably, and in a way you can update, monitor and roll back. Most ML projects that fail do so here, not in modelling.

Two common shapes. Batch prediction: a scheduled job (nightly, hourly) loads new rows, predicts, and writes results to a table that other systems read; e.g. tomorrow's no-show risks for the call list. It is simpler, cheaper and easier to debug. Real-time (online) prediction: a service answers each request within milliseconds, needed when the input only exists at that moment, e.g. a card payment at checkout. Choose batch unless the use case truly needs real-time.

Either way you need the same pieces: a saved, versioned model; the exact same preprocessing as in training (ship the whole scikit-learn Pipeline, not just the model); input validation; logging; and a way to monitor and roll back.

Beginner

Saving models and serving them with an API

Save the fitted pipeline with joblib.dump(pipe, "model.joblib") and record which code version, data and parameters produced it. (Only load model files you trust: loading a pickle can run arbitrary code.)

An API (application programming interface) lets other software ask for predictions over HTTP, usually by sending JSON to an endpoint such as POST /predict. FastAPI is the most popular Python framework for this: you describe the expected input with a Pydantic model, and FastAPI validates every request (types, required fields, ranges) and returns a clear 422 error automatically when it is wrong.

Return more than a number: the probability, the decision, and the model version, and log each request (without personal data) so you can investigate later. Add a GET /health endpoint that monitoring systems can call.

Try it

Try it: call a prediction API

A simulated POST /predict endpoint with the validation rules a real service would have. Edit the JSON or inject a mistake, send it, and read the status code and response.

Interactive: this part runs in your browser.

Try it

Code example: the logic behind the endpoint

A worked example: read it step by step (every part is explained), then run it and experiment.

Interactive: this part runs in your browser.

Beginner

Docker, CI and experiment tracking

Docker packages your code, its dependencies and the operating-system pieces it needs into an image that runs identically on your laptop, a colleague's machine and a cloud server. A short Dockerfile describes how to build it; docker build creates the image and docker run starts a container from it. This removes "it works on my machine" problems.

Continuous integration (CI), e.g. GitHub Actions, runs your tests, linting and a small training smoke test on every pull request, so broken code can't be merged. Continuous delivery then builds and deploys the image automatically after merging.

Experiment tracking (MLflow, Weights & Biases) records every training run's parameters, metrics, data version and the resulting model file, so you can answer "which run produced the model in production, and how was it trained?". A model registry stores approved versions with stages such as "staging" and "production", which makes rolling back a one-line change.

Beginner

Monitoring after launch

Models degrade silently. Monitor four layers: service health (latency, errors, uptime); data quality (missing values, out-of-range inputs, schema changes); drift in input features and in the prediction distribution (e.g. PSI, which needs no labels); and outcome metrics once labels arrive (precision, recall, calibration, per group). Every alert needs an owner and a runbook, and you need a tested way to fall back to the previous model or a simple rule. The game milestone after this topic puts all of this into practice.

Code

Code: a minimal production setup

The four files most small ML services start with.

app.py ยท FastAPI service
from fastapi import FastAPI
from pydantic import BaseModel, Field
import joblib, pandas as pd, logging

MODEL_VERSION = "noshow-gb-2025-11-01"
pipe = joblib.load("model.joblib")           # the whole sklearn Pipeline
app = FastAPI()
log = logging.getLogger("predict")

class Appointment(BaseModel):                # FastAPI validates every request against this
    lead_days: int = Field(ge=0, le=365)
    age: int = Field(ge=0, le=120)
    distance_km: float = Field(ge=0, le=500)
    sms_sent: int = Field(ge=0, le=1)
    prior_noshows: int = Field(ge=0, le=100)

@app.get("/health")
def health():
    return {"status": "ok", "model_version": MODEL_VERSION}

@app.post("/predict")
def predict(a: Appointment):
    p = float(pipe.predict_proba(pd.DataFrame([a.model_dump()]))[0, 1])
    log.info("prediction p=%.3f version=%s", p, MODEL_VERSION)   # no personal data in logs
    return {"probability": round(p, 3), "decision": "call" if p >= 0.3 else "sms", "model_version": MODEL_VERSION}

# run locally:  uvicorn app:app --reload   then open http://127.0.0.1:8000/docs
Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py model.joblib ./
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

# docker build -t noshow-api .
# docker run -p 8000:8000 noshow-api
.github/workflows/ci.yml ยท tests on every pull request
name: ci
on: [pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.11" }
      - run: pip install -r requirements.txt pytest ruff
      - run: ruff check .
      - run: pytest -q
train.py ยท experiment tracking with MLflow
import mlflow, mlflow.sklearn
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import average_precision_score

params = {"learning_rate": 0.05, "max_leaf_nodes": 31}
with mlflow.start_run(run_name="gb-baseline"):
    mlflow.log_params(params)
    model = HistGradientBoostingClassifier(**params).fit(X_train, y_train)
    ap = average_precision_score(y_val, model.predict_proba(X_val)[:, 1])
    mlflow.log_metric("val_pr_auc", ap)
    mlflow.sklearn.log_model(model, "model")     # stored with the run; register it if approved
Check yourself

Quiz

1. You need tomorrow's no-show risk for every booked appointment, once a day. The simplest suitable design isโ€ฆ

2. Why save the whole scikit-learn Pipeline rather than only the model?

3. A request has age: "forty". A well-built API returnsโ€ฆ

4. What problem does Docker mainly solve?

5. Which signal can you monitor before any labels arrive?

Go deeper

๐Ÿ“š Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready