
Regression evaluation metrics measure how closely a trained regression model's predictions match actual target values. For each data point, the prediction error is the vertical distance between the true value and the predicted value. Although every regression metric works from those same prediction errors, each one summarizes them differently. On the Professional Machine Learning Engineer exam, choosing the right metric comes down to how a scenario weighs outlier sensitivity, unit interpretability, baseline comparison, and scale independence.
Mean Absolute Error, or MAE, follows a two-step calculation. First, take the absolute value of every prediction error and add them up. Using absolute values ensures overpredictions and underpredictions count equally, since only the magnitude of each error matters. Second, divide that sum by the number of data points to get the average.
The main advantage of MAE is direct interpretability. Because errors stay in the original scale of the target variable, an MAE of 0.3 means predictions are off by 0.3 units on average. MAE is also robust to outliers because it does not square the errors, giving a straightforward measure of typical prediction accuracy.
Mean Squared Error, or MSE, changes how errors are weighted by squaring each one before averaging. The calculation has three steps: square each prediction error, add all of the squared errors together, and divide by the number of data points.
Squaring causes larger errors to grow disproportionately compared to smaller ones, which makes MSE far more sensitive to large errors and outliers than MAE. A single prediction that is far off can dominate the MSE score. This makes MSE useful when a model needs to heavily penalize and avoid large prediction mistakes, though outliers can also make evaluation less stable. Because MSE is reported in squared units of the target variable, such as squared dollars for house prices, it is less intuitive to interpret than MAE.
Root Mean Squared Error, or RMSE, adds one final step to MSE. The first three steps are the same: square the prediction errors, add them up, and divide by the number of data points. The fourth step takes the square root of that result.
Taking the square root returns the metric to the original units of the data. When predicting house prices in dollars, RMSE is reported in dollars rather than squared dollars, so an RMSE of $15,000 indicates predictions are off by about $15,000 on average.
That square root does not remove outlier sensitivity. Because squaring happens before the square root, large mistakes are still amplified and contribute disproportionately to the final score. RMSE combines the original-unit interpretability of MAE with the outlier sensitivity of MSE, making it a common choice when both characteristics are needed.
R-squared takes a different approach from measuring raw error size. It evaluates how much better a model performs than the simplest possible baseline, which predicts the mean of all actual target values for every data point. Predicting the mean explains none of the variance in the data and corresponds to an R-squared of 0.
R-squared compares the total variance of the actual data around the mean against the remaining unexplained variance around the fitted regression predictions. The difference between the two is the variance captured by the model, expressed as a proportion of the total variance. Models that explain a large share of the variance have an R-squared closer to 1, while models that barely improve on guessing the mean stay close to 0.
R-squared is scale-independent and shows whether a model is genuinely useful compared to predicting the average every time. Its name comes from the correlation coefficient R, and in simple linear regression, R-squared is the square of the correlation between actual and predicted values.
Mean Absolute Percentage Error, or MAPE, measures prediction accuracy as a percentage of the actual value. For a single data point, the individual percentage error is the prediction error divided by the actual value. If the actual value is 100 and the prediction is 90, the error is 10, so dividing 10 by 100 yields a 10 percent error.
Overall MAPE uses a three-step process. First, calculate the individual percentage error for every data point. Second, take the absolute value of each percentage error so positive and negative signs are ignored, treating overpredictions and underpredictions equally. Third, add those absolute percentage errors together and divide by the number of data points to get the average.
Averaging individual absolute percentage errors across a dataset, where points might be off by amounts such as 1.5 percent and 11.1 percent, can yield an overall MAPE such as 5.07 percent. That means predictions are off by about 5 percent relative to actual values on average. Because MAPE is scale-independent and expresses error in percentage terms, it is especially useful for communicating performance to stakeholders.
When a Professional Machine Learning Engineer scenario asks for a regression metric, the requirements typically map to one of these trade-offs:
When a use case needs both original-unit readability and a strong penalty on large mistakes, we would lean toward RMSE over MAE or MSE. When outliers should not dominate the score, MAE is the cleaner choice, while R-squared and MAPE apply when scale independence matters most.
Our Professional Machine Learning Engineer course covers regression metrics alongside classification metrics and model evaluation, with practice questions that drill these distinctions.