Classification Metrics: Precision, Recall, F1, ROC-AUC, and PR-AUC

GCP Study Hub
September 4, 2026

Accuracy is the most basic classification metric, defined as total correct predictions divided by total predictions. Getting 14 of 18 predictions right yields 78 percent accuracy. Because accuracy does not distinguish between true positives, true negatives, false positives, and false negatives, it can be misleading on imbalanced datasets where one class dominates, and it may not match what a use case actually cares about. On the Professional Machine Learning Engineer exam, selecting a classification metric depends on class balance, the relative cost of each error type, and whether predicted probabilities need calibration.

Error rate is the complement of accuracy, calculated as total incorrect predictions divided by total predictions (4 of 18 wrong gives a 22 percent error rate). Accuracy and error rate always sum to 100 percent, and error rate is often clearer for high-performing models. Moving from 99.1 percent to 99.5 percent accuracy lowers the error rate from 0.9 percent to 0.5 percent, cutting errors nearly in half.

Confusion matrices for binary and multi-class models

A confusion matrix organizes predictions with actual classes as columns and predicted classes as rows, showing both how often a model is wrong and which errors it makes. Binary classification produces four outcomes:

  • True positive: predicted positive and actually positive.
  • False positive: predicted positive when actually negative, claiming something is present when it is not.
  • False negative: predicted negative when actually positive, missing something that is actually there.
  • True negative: predicted negative and actually negative.

The same grid extends to multi-class problems such as classifying cats, dogs, and birds. Diagonal cells hold correct predictions (for example, 85 cats, 70 dogs, and 90 birds), while off-diagonal cells show which classes are confused for one another. If the off-diagonal counts show 15 cats labeled as dogs and 10 dogs labeled as cats, with few mistakes on birds, the matrix points directly to cat-versus-dog confusion.

Precision, recall, sensitivity, and specificity

Precision equals true positives divided by all predicted positives (true positives plus false positives), answering how often a positive prediction is actually right. Recall equals true positives divided by all actual positives (true positives plus false negatives), answering how many real positive cases the model caught.

In multi-class classification, precision and recall are computed per class. In the cat, dog, and bird matrix, if the predicted cat row holds 85 true cats, 10 dogs, and 5 birds (100 total predicted cats), cat precision is 85 divided by 100, or 85 percent. If the actual cat column holds 85 cats predicted as cats, 15 predicted as dogs, and 2 predicted as birds (102 actual cats), cat recall is 85 divided by 102, or 83.3 percent.

In science, medicine, and statistics, sensitivity and specificity often replace precision and recall terminology, and they can appear in domain scenarios on the Professional Machine Learning Engineer exam. Sensitivity equals the true positive rate, making it interchangeable with recall. Specificity equals the true negative rate, measuring how many actual negatives the model correctly identifies as negative. On a disease blood test, high sensitivity is suited to screening because it catches almost everyone who is sick, while high specificity avoids false alarms and unnecessary follow-up testing for healthy people.

Decision thresholds and the precision-recall trade-off

A binary classification neural network with a sigmoid activation outputs a positive-class probability such as 0.6. Classifying anything above 0.5 as positive is only a default. Adjusting the decision threshold changes whether the model prioritizes avoiding false negatives or avoiding false positives, which depends on stakeholder context. Medical diagnosis weighs missing diseases against false alarms, fraud detection weighs blocking legitimate transactions against allowing fraud through, and hiring weighs rejecting qualified candidates against interviewing unqualified ones.

Lowering the threshold catches more positive cases while generating more false alarms. Raising it makes positive predictions more conservative while risking missed cases. Three rules govern threshold adjustments:

  • False positives harm precision by increasing the denominator of all predicted positives.
  • False negatives harm recall because they are missed cases in the denominator of all actual positives.
  • Adjusting the decision threshold to reduce one error type typically increases the other.

F1 score

F1 score combines precision and recall into a single metric using their harmonic mean: two times precision times recall, divided by precision plus recall. Because the harmonic mean heavily penalizes low values, both metrics must be decent for F1 to look good. A model with 95 percent precision and 5 percent recall gets an F1 of 9 percent, while a model with 70 percent precision and 70 percent recall gets an F1 of 70 percent. We would lean toward F1 when comparing models with one number and both false positives and false negatives matter.

ROC-AUC, precision-recall curves, and MAP

The receiver operating characteristic (ROC) curve evaluates a binary classifier across all decision thresholds by plotting true positive rate on the y-axis against false positive rate on the x-axis, both from 0 to 1. The diagonal line from (0, 0) to (1, 1) marks random guessing, no better than a coin flip, with an area under the curve (AUC) of 0.5. A strong curve rises steeply toward the top left, reaching high true positive rates at low false positive rates (for example, an AUC of 0.92). ROC-AUC equals the probability that the model correctly distinguishes a positive example from a negative example.

Precision-recall (PR) curves plot precision on the y-axis against recall on the x-axis across thresholds. They work better than ROC curves on imbalanced datasets where the positive class is rare, such as fraud or disease detection, because they focus directly on minority-class performance. The random baseline is a horizontal line at the positive class rate (13 percent precision when 13 percent of examples are positive). As recall rises, precision generally drops, and the area under the curve is summarized as average precision (PR-AUC), such as 0.78 on an imbalanced dataset, with better curves staying closer to the top right corner.

Mean average precision (MAP) extends PR curves to multi-class problems and is widely used in image classification. It averages the per-class average precision scores across all classes. Averaging 0.85 for cats, 0.72 for dogs, and 0.91 for birds produces a MAP of 0.83.

Log loss (cross-entropy loss)

Log loss, also known as cross-entropy loss, evaluates predicted probabilities rather than hard class labels, with lower scores indicating better performance. When the actual class is positive (1), predicting a probability near 0 incurs an extremely large penalty, while predicting near 1 drives the penalty near 0. When the actual class is negative (0), predicting near 0 keeps the penalty low, while predicting near 1 causes the penalty to shoot up sharply.

Because log loss heavily penalizes confident wrong predictions and rewards confident correct ones, it encourages well-calibrated probabilities that stay uncertain on difficult cases. It is useful whenever probability confidence drives downstream decisions, such as with logistic regression.

Our Professional Machine Learning Engineer course covers classification metrics alongside regression metrics and model evaluation, with practice questions that drill these distinctions.

Get tips and updates from GCP Study Hub

arrow