Evaluation Metrics & Cross-Validation
Measuring performance honestly — accuracy is often not enough.
The confusion matrix
For a classification model, the starting point for every evaluation is the confusion matrix, a small table that counts the four possible outcomes of a yes/no prediction. Suppose the "positive" class is the thing we are trying to detect, such as fraud or disease.
| Predicted positive | Predicted negative | |
|---|---|---|
| Actually positive | True Positive (TP): correctly detected | False Negative (FN): missed |
| Actually negative | False Positive (FP): false alarm | True Negative (TN): correctly left alone |
The simplest summary is accuracy, the fraction of all predictions that were correct, . Accuracy is easy to understand but can be badly misleading when the classes are imbalanced. If 99% of transactions are legitimate, a "model" that labels every transaction legitimate achieves 99% accuracy while catching no fraud whatsoever. We need measures that focus on the positive class.
Precision answers the question: "of all the cases the model flagged as positive, what fraction really were positive?"
High precision means few false alarms. It matters most when false alarms are costly, for example a spam filter that hides genuine emails.
Recall (also called sensitivity, or the true-positive rate) answers a different question: "of all the cases that really were positive, what fraction did the model catch?"
High recall means few misses. It matters most when missing a positive is costly, for example in cancer screening or fraud detection, where an undetected case can be very serious.
There is usually a tension between the two: flagging more cases catches more true positives (higher recall) but also raises more false alarms (lower precision). The F1 score combines them into a single number using the harmonic mean, which is only high when both are high:
Interactive: this part runs in your browser.
Watch it: Cross-validation and time-series splits
A short narrated animation. Press ▶ Play, or step through it with ⏮ ⏭. Tick 🔊 narrate to hear each step.
Interactive: this part runs in your browser.
Try it: move the threshold
A model outputs a score for each example. Everything right of the threshold is predicted positive. Watch precision and recall trade off as you slide.
Interactive: this part runs in your browser.
ROC, AUC & regression metrics
Most classifiers don't output a hard yes or no; they output a score or probability, and a threshold turns it into a decision. As the demo above showed, every threshold gives a different balance of precision and recall. Rather than judging a model at one arbitrary threshold, it is often more informative to see how it performs across all thresholds.
The ROC curve (receiver operating characteristic, a name inherited from radar engineering) does this by plotting the true-positive rate (recall) against the false-positive rate, , as the threshold sweeps from high to low. A perfect classifier jumps straight to the top-left corner, catching every positive before raising any false alarm, while a classifier that guesses at random follows the diagonal. The area under the ROC curve (AUC) condenses the curve into one number: 1.0 for a perfect classifier and 0.5 for random guessing. Because it considers every threshold, AUC measures how well the model ranks positives above negatives, independently of where you eventually set the threshold. When the positive class is very rare, the precision–recall curve is often more revealing than the ROC curve, because the ROC curve's false-positive rate can look reassuringly small simply because the number of negatives is huge.
For regression problems, where the target is a number, three measures are standard. The mean absolute error (MAE) averages the absolute size of the errors; it is easy to interpret and not dominated by a few large errors. The mean squared error (MSE) averages the squared errors, so it penalises large errors heavily; its square root, the RMSE, is in the same units as the target, which makes it easier to interpret. And reports the fraction of the variation in the target that the model explains, where 1 is perfect and 0 means no better than predicting the average.
k-fold cross-validation
A single train/validation split gives a single number, and that number depends partly on luck: which examples happened to land in the validation set. With small or medium datasets, a lucky or unlucky split can easily make one model look better than another when it isn't. K-fold cross-validation reduces this luck by using every example for validation exactly once.
The procedure is as follows. First, divide the training data into equal parts, called folds; or is typical. Then train the model times. Each time, one fold is held out as the validation set and the model is trained on the other folds. This gives validation scores, which we average to get the cross-validation estimate. The diagram shows the five rounds of 5-fold cross-validation, with the orange block being the validation fold each time.
Besides a more reliable average, cross-validation tells you how much the score varies from fold to fold, which is a useful warning sign: if the fold scores range from 70% to 90%, a reported average of 80% hides a lot of uncertainty.
The way the folds are formed should match the structure of the data. For classification, use stratified k-fold, which keeps the class proportions the same in every fold. For time series, use a time-series split, which always trains on earlier data and validates on later data. When one person or entity contributes several rows, use group k-fold, which keeps all of an individual's rows in the same fold, so the model can't score well simply by recognising people it has already seen.
Cross-validation is also the standard tool for tuning hyperparameters. You choose a set of candidate settings, compute the cross-validation score for each, and pick the best. Grid search tries every combination from a list of values. Random search tries random combinations, which is usually more efficient, because typically only a few hyperparameters really matter and random search explores more distinct values of each. Bayesian optimisation goes further by using the results so far to decide which settings to try next.
One rule still applies: even with cross-validation, keep a final test set that is never used for any decision, and evaluate on it once at the end. If you need an unbiased estimate of how well your whole tuning procedure performs, use nested cross-validation, which is explained in the Advanced section.
Interactive: this part runs in your browser.
Metric formulas and what AUC really means
Each metric answers a different question. Precision: "when I raise an alarm, am I usually right?" Recall: "do I catch most of the real cases?" AUC: "do I rank positives above negatives?" And the best threshold isn't always 0.5: it depends on how much each kind of mistake costs.
🔤 Symbols in this section
| true/false positives and negatives | |
| F-score weighting recall β times as much as precision | |
| model score | |
| a random positive and a random negative example | |
| cost of a false positive / false negative | |
| indicator: 1 if true, 0 if false |
This section gathers the precise definitions of the common metrics, explains a surprising interpretation of AUC, and shows how to choose a decision threshold from the costs of mistakes.
The four basic rates are all ratios taken from the confusion matrix:
The F1 score weights precision and recall equally. When one matters more than the other, the F-beta score lets you choose the balance:
A value of gives recall more importance, so the score is popular when missing positives is costly, while favours precision.
For imbalanced data, a good single-number summary is the Matthews correlation coefficient, which uses all four cells of the confusion matrix and is essentially a correlation between predictions and truth:
It ranges from −1 (always wrong) through 0 (no better than chance) to +1 (perfect), and unlike accuracy it cannot be inflated simply by predicting the majority class.
Some metrics judge the predicted probabilities rather than the decisions. Log-loss, , heavily punishes confident mistakes. The Brier score, , is the mean squared error of the probabilities. Both reward probabilities that are honest as well as accurate.
What AUC really measures. There is an elegant interpretation of the area under the ROC curve: it equals the probability that a randomly chosen positive example receives a higher score than a randomly chosen negative example.
The formula on the right simply checks every positive–negative pair and counts the fraction in which the positive is ranked higher (the indicator equals 1 when its condition is true). Statisticians will recognise this as the normalised Mann–Whitney U statistic. The interpretation makes clear that AUC is purely about ranking. It is unaffected by whether the scores are well-calibrated probabilities, and it says nothing about which threshold to use.
Choosing the threshold from costs. The default threshold of 0.5 is only sensible if the two kinds of mistake are equally costly, which is rarely true. Suppose a false positive costs and a false negative costs , and the model's probability is well calibrated. Predicting "positive" has an expected cost of (we are wrong if the case is negative), while predicting "negative" has an expected cost of . Predicting positive is the better choice when , which rearranges to
For example, if missing a cancer is judged nine times worse than a false alarm, the threshold should be , not 0.5: any patient with more than a 10% estimated risk should be referred.
For problems with more than two classes, per-class metrics are combined in one of three ways. A macro average computes the metric for each class and averages them, treating every class as equally important. A micro average pools all the true and false positives across classes first, treating every example as equally important. A weighted average is a macro average weighted by the size of each class.
Cross-validation theory, nested CV and comparing models
Cross-validation gives an estimate of performance, and every estimate has uncertainty. If you tune hyperparameters using the same folds you report, you're quietly picking the luckiest result. Nested CV keeps an honest outer test, and proper statistical tests tell you whether one model is really better than another.
🔤 Symbols in this section
| number of folds | |
| model trained without fold j | |
| cases where only model A / only model B is right | |
| chi-squared test statistic | |
| expected calibration error | |
| accuracy and mean confidence in bin b |
Cross-validation produces an estimate of performance, and like every estimate it has a bias and a variance. Understanding these helps you avoid reading too much into small differences, and helps you avoid one particularly common trap.
The k-fold estimate is the average of the fold errors, , where is the model trained without fold . Its bias comes from the fact that each model is trained on only a fraction of the data, so it is slightly worse than a model trained on everything; cross-validation is therefore slightly pessimistic, and using more folds reduces this. Its variance is more subtle. The fold scores are not independent, because their training sets overlap heavily, so the usual standard-error formula underestimates the real uncertainty. Leave-one-out cross-validation () is almost unbiased, but its models are nearly identical, which can make its estimate highly variable. Five or ten folds is a good compromise, and repeating k-fold several times with different random splits further reduces the variance.
The tuning trap. Suppose you try fifty hyperparameter settings and report the cross-validation score of the best one. That score is optimistically biased, because by picking the best of fifty noisy estimates you have partly selected the setting that got lucky on these particular folds. The remedy is nested cross-validation. An outer loop splits the data into folds as usual. Inside each outer training set, an inner cross-validation loop performs the entire tuning procedure and picks the best setting. That chosen model is then evaluated on the outer fold, which played no part in the tuning. The outer scores therefore measure how well the whole procedure, tuning included, performs on genuinely unseen data.
Is model A really better than model B? To compare two classifiers on the same test set, McNemar's test looks only at the examples where they disagree. Let be the number of examples that A gets right and B gets wrong, and the number that B gets right and A gets wrong. If the models were equally good, and should be similar, and the statistic measures how surprising the observed imbalance is (it is compared with a chi-squared distribution with one degree of freedom). To compare models across cross-validation folds, the corrected resampled t-test of Nadeau and Bengio adjusts the ordinary t-test for the overlap between training sets, by inflating the variance by a factor . Without such corrections, small differences look far more significant than they are.
Checking calibration. A classifier is calibrated if its probabilities match reality: among all the cases where it predicts 70%, about 70% should turn out positive. To check this, group the predictions into bins by their predicted probability and, for each bin, plot the average predicted probability against the fraction that were actually positive. This plot is called a reliability diagram, and a calibrated model lies along the diagonal. The expected calibration error summarises the plot as a weighted average gap, , where is the average predicted probability in bin and the observed fraction of positives.
Code: metrics, ROC/PR, tuning and nested CV
Everything from this topic in scikit-learn, including a check that AUC equals the pairwise ranking probability.
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import (train_test_split, StratifiedKFold,
GridSearchCV, cross_val_score)
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import (confusion_matrix, precision_score, recall_score,
f1_score, matthews_corrcoef, roc_auc_score,
average_precision_score, brier_score_loss)
X, y = make_classification(n_samples=2000, weights=[0.9, 0.1], random_state=0)
Xtr, Xte, ytr, yte = train_test_split(X, y, stratify=y, test_size=0.3, random_state=0)
model = RandomForestClassifier(n_estimators=200, random_state=0).fit(Xtr, ytr)
p = model.predict_proba(Xte)[:, 1]
for thr in [0.5, 0.3, 0.1]: # moving the threshold
pred = (p >= thr).astype(int)
print(f"thr={thr}: P={precision_score(yte, pred):.2f} R={recall_score(yte, pred):.2f} "
f"F1={f1_score(yte, pred):.2f} MCC={matthews_corrcoef(yte, pred):.2f}")
print(confusion_matrix(yte, (p >= 0.5).astype(int)))
print("ROC-AUC:", round(roc_auc_score(yte, p), 3),
" PR-AUC:", round(average_precision_score(yte, p), 3),
" Brier:", round(brier_score_loss(yte, p), 4))
# AUC == P(score of random positive > score of random negative)
pos, neg = p[yte == 1], p[yte == 0]
pairwise = (pos[:, None] > neg[None, :]).mean() + 0.5 * (pos[:, None] == neg[None, :]).mean()
print("pairwise ranking probability:", pairwise.round(3))
# Nested CV: inner loop tunes, outer loop estimates performance honestly
inner = StratifiedKFold(3, shuffle=True, random_state=1)
outer = StratifiedKFold(5, shuffle=True, random_state=2)
search = GridSearchCV(RandomForestClassifier(random_state=0),
{"max_depth": [3, 6, None], "min_samples_leaf": [1, 5]},
cv=inner, scoring="roc_auc")
nested = cross_val_score(search, X, y, cv=outer, scoring="roc_auc")
print("nested CV AUC: %.3f ± %.3f" % (nested.mean(), nested.std()))Quiz
1. For cancer screening, which metric matters most?
2. A model that always predicts "not fraud" on data with 1% fraud has accuracy…
3. An AUC of 0.5 means…
4. ROC-AUC equals the probability that…
5. A false negative costs 9× a false positive. With calibrated probabilities, the best threshold is…
6. Why use nested cross-validation?
📚 Further reading
Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.
- scikit-learn: Cross-validation ↗ Official guide
- scikit-learn: Metrics and scoring ↗ Official guide
- scikit-learn: Common pitfalls ↗ Leakage and other mistakes