
Hyperparameters are configuration values set before model training begins that govern how a model moves through training data, updates its weights, and generalizes to unseen examples. On the Professional Machine Learning Engineer exam, training scenarios often hinge on diagnosing convergence behavior from loss curves or choosing the right regularization mechanism to control overfitting.
An epoch is one full pass through the entire training dataset during model training, meaning the model sees every training example exactly once. In a tabular dataset where each row is a training example, real-world training sets often contain thousands or millions of rows, and going through every row completes a single epoch. Models typically train for multiple epochs, seeing the same data repeatedly while learning different patterns on each pass as internal parameters adjust. Setting the number of epochs before training begins determines how many complete cycles the model makes through the dataset.
Batch size is the number of training examples a model processes before updating its weights once. Instead of processing examples one at a time or waiting until the end of the entire dataset to perform an update, training groups examples into batches and updates weights after each batch.
With a small training set of 12 examples and a batch size of 4, the model groups examples 1 through 4 into the first batch, processes all four, calculates the loss across the batch, and performs its first weight update. It repeats this for examples 5 through 8 (second update) and examples 9 through 12 (third update). Completing all three batches equals one full epoch, resulting in three weight updates per epoch.
Batch size directly determines how many weight updates occur during each pass through the training data. Larger batch sizes produce fewer updates per epoch, while smaller batch sizes produce more frequent updates, balancing computational efficiency with learning stability.
The learning rate controls how much a model's weights are adjusted during each update step, directly affecting how quickly and effectively the model learns. A high learning rate, such as 0.1, takes large steps down the loss surface. That can speed up training initially, though large jumps increase the risk of overshooting or zigzagging around the minimum, causing unstable convergence or missing optimal weight values. A low learning rate, such as 0.01, takes smaller, more controlled steps that follow the loss surface more precisely and keep training stable, though convergence may take longer.
Tracking training loss across epochs shows how different learning rates behave:
Too low a learning rate makes training inefficient, while too high a learning rate makes training unstable or counterproductive.
Dropout rate is the fraction of neurons randomly turned off during training to improve generalization. Intentionally handicapping a neural network during training makes it more robust and less likely to memorize the training data, encouraging it to learn underlying patterns instead.
Dropout applies only to the hidden layers, leaving the input and output layers completely active. In a network with two hidden layers of four neurons and three neurons, a dropout rate of 0.3 randomly disables a subset of hidden neurons during training (such as one of four in the first hidden layer and one of three in the second) while keeping all input and output neurons active. This forces the remaining neurons to work harder and prevents the network from becoming overly dependent on any single neuron, improving generalization to unseen data.
Regularization hyperparameters control how much penalty is applied to large weight magnitudes, reducing overfitting and producing simpler models that generalize better. On the Professional Machine Learning Engineer exam, the central distinction between L1 and L2 regularization is how each penalty treats feature weights.
L1 regularization can push some weights all the way to zero, eliminating those features entirely so the model relies only on the most relevant features. That makes L1 useful for automatic feature selection and yields models that are simpler and easier to interpret. If the L1 penalty is too high, the model can zero out weights it needs and hurt accuracy. If it is too low, the model retains unnecessary weights and remains more complex than needed.
L2 regularization strength also penalizes large weights, but shrinks them toward zero instead of forcing them to zero. Every feature remains in the model with limited influence, producing smoother, more stable weights. Setting the L2 penalty too high squeezes weights down until the model loses useful signal, while setting it too low leaves the model too flexible and prone to overfitting.
Consider an unregularized model prone to overfitting with four weights: 4.7, 3.8, 5.2, and -1.9. With L1 regularization applied, those weights can become 1.5, 1, 1.8, and 0, removing the fourth feature completely. With L2 regularization applied instead, the weights shrink to 1.2, -0.9, 1.1, and -0.7, where none reaches zero and all four are smaller across the board.
Both techniques reduce overfitting, so we would choose between them based on the modeling goal. L1 regularization suits scenarios that call for automatic feature selection and easier interpretation, while L2 regularization suits scenarios where keeping every feature with smoother, more stable weights is preferred.
Our Professional Machine Learning Engineer course covers core training hyperparameters alongside model evaluation metrics and overfitting diagnostics, with practice questions that drill these distinctions.