Machine Learning
Topic 10 · ML foundations · ⏱ about 8 h

scikit-learn Essentials

The library behind most classical ML: the estimator API, splitting, preprocessing, Pipelines and ColumnTransformers, cross-validation, tuning, metrics and saving models.

Show:
In this topic
Beginner

One consistent API

scikit-learn is the standard Python library for classical machine learning: dozens of models, preprocessing tools, model selection and metrics, all with the same interface. Learn the interface once and every algorithm in this course is three lines away.

  • Estimators learn from data with fit(X, y). Hyperparameters go in the constructor: RandomForestClassifier(n_estimators=300). What was learned is stored in attributes ending in an underscore: coef_, feature_importances_.
  • Predictors add predict(X), and classifiers usually predict_proba(X); score(X, y) gives a default metric.
  • Transformers (scalers, encoders, imputers, PCA) have fit and transform, or fit_transform on training data.
from sklearn.model_selection import train_test_split from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import classification_report X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=0) model = RandomForestClassifier(n_estimators=300, random_state=0) # 1. choose and configure model.fit(X_train, y_train) # 2. learn y_pred = model.predict(X_test) # 3. predict print(classification_report(y_test, y_pred)) # swap in LogisticRegression(), HistGradientBoostingClassifier(), SVC(probability=True)... same three lines

X is a 2-D table (DataFrame or array, one row per example) and y a 1-D array of targets. random_state makes results reproducible.

Try it

Code example: fit, predict, score

A worked example: read it step by step (every part is explained), then run it and experiment.

Interactive: this part runs in your browser.

Beginner

Pipelines and ColumnTransformer

Real tables mix numeric and categorical columns, with missing values. A ColumnTransformer applies different preprocessing to different columns; a Pipeline chains preprocessing and a model into one estimator. This is the single most important habit in applied ML, for three reasons: preprocessing is fitted on training data only (inside every cross-validation fold too), so nothing leaks; the same steps run at prediction time; and you save and deploy one object.

from sklearn.pipeline import Pipeline from sklearn.compose import ColumnTransformer from sklearn.impute import SimpleImputer from sklearn.preprocessing import StandardScaler, OneHotEncoder from sklearn.linear_model import LogisticRegression prep = ColumnTransformer([ ("num", Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())]), ["lead_days", "age", "distance_km"]), ("cat", OneHotEncoder(handle_unknown="ignore"), ["clinic", "weekday"]), ]) pipe = Pipeline([("prep", prep), ("clf", LogisticRegression(max_iter=1000))]) pipe.fit(X_train, y_train) pipe.predict_proba(X_new) # raw rows in, probabilities out: preprocessing included
Try it

Try it: build a pipeline

Choose the steps and see the pipeline diagram and the code. Then try the "scale before splitting" mistake to see where leakage creeps in.

Interactive: this part runs in your browser.

Beginner

Cross-validation, tuning and metrics

from sklearn.model_selection import cross_val_score, GridSearchCV, StratifiedKFold, TimeSeriesSplit cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0) # TimeSeriesSplit for time-ordered data scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="roc_auc") print(scores.mean(), scores.std()) search = GridSearchCV(pipe, {"clf__C": [0.01, 0.1, 1, 10]}, cv=cv, scoring="average_precision") search.fit(X_train, y_train) # tunes on training folds only print(search.best_params_, search.best_score_) best = search.best_estimator_ # refitted on all training data

Parameters of a step inside a pipeline are addressed as step__parameter (two underscores). RandomizedSearchCV samples combinations when the grid is large. Scoring names include accuracy, balanced_accuracy, f1, roc_auc, average_precision, neg_mean_absolute_error (errors are negated so that higher is always better). In sklearn.metrics you will use confusion_matrix, classification_report, roc_auc_score, average_precision_score, mean_absolute_error and r2_score.

Save the final pipeline with joblib.dump(best, "model.joblib") and load it with joblib.load, using the same scikit-learn version.

Try it

Code example: Pipeline + ColumnTransformer + GridSearchCV

A worked example: read it step by step (every part is explained), then run it and experiment.

Interactive: this part runs in your browser.

Advanced

Mistakes to avoid

  • Fitting preprocessing on all the data (StandardScaler().fit(X) before splitting, or SelectKBest outside the pipeline): leakage. Put it in the Pipeline.
  • Tuning on the test set: use cross-validation or a separate validation set; touch the test set once.
  • Ignoring handle_unknown in OneHotEncoder: a new category in production crashes the model.
  • Using predict when you need a threshold: predict uses 0.5; use predict_proba and choose the threshold from costs.
  • Random splits for grouped or time data: use GroupKFold (same patient never in both) or TimeSeriesSplit.
  • Forgetting the class balance: stratify=y in splits, class_weight="balanced" where appropriate, and metrics that reflect it.
Try it

Cheat sheet: scikit-learn

The commands you will use most, on one screen. Come back to it while you work.

Interactive: this part runs in your browser.

Check yourself

Quiz

1. Which method do all scikit-learn estimators share?

2. Why put the scaler inside a Pipeline?

3. How do you name the C parameter of a step called "clf" in a GridSearchCV grid?

4. A fitted model's learned coefficients are in…

5. Your classifier should flag cases at probability ≥ 0.3. You should use…

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready