Model Optimization Techniques: Quantization, Pruning, and Distillation

GCP Study Hub
September 14, 2026

Model optimization is the process of transforming a model to make it more efficient while preserving acceptable accuracy. The goal is to keep the model performing close to the original with less overhead. Common techniques include quantization, model pruning, weight clustering, and distillation. Each method shrinks or simplifies a model differently, and on the Professional Machine Learning Engineer exam, selecting between them depends on whether the constraint is storage size, inference latency, accelerator support, or retraining effort.

Why optimize a model and what you trade off

Efficiency gains from model optimization show up in three concrete ways:

  • Size reduction: An optimized model takes up less storage on disk and less memory once loaded.
  • Latency reduction: A smaller, simpler model generally runs faster, returning predictions more quickly after deployment.
  • Accelerator compatibility: Specialized hardware such as GPUs and TPUs often runs more effectively when operations are simplified or represented at lower precision.

None of these improvements come without a trade-off. Optimizing for size or latency usually introduces some accuracy loss. Whether that reduction is acceptable depends on the model, the application, and how lower accuracy affects the user experience.

Quantization: post-training options and quantization-aware training

Quantization reduces the precision of the values used to represent a model's parameters, storing numbers in smaller data types than the original model used. Lowering numerical precision produces a smaller model and faster inference because hardware executes lower-precision math more efficiently.

Quantization can be applied at two stages: after training or during training. Post-training quantization converts an existing trained model and offers three main options:

  • Dynamic range quantization: This is typically the quickest first optimization to try because it requires no sample data to set up. We would lean toward it as a reasonable default before moving to more involved methods.
  • Float-16 quantization: This option works well when targeting GPU inference, since GPUs handle float-16 operations efficiently and it provides size reduction with minimal impact on accuracy.
  • Integer quantization: This is the choice when target hardware only supports integer operations, which is common on edge accelerators such as edge TPUs. It requires sample input data to guide the conversion from floating-point to integer values accurately.

When post-training quantization causes too much accuracy loss on its own, the alternative is quantization-aware training. Instead of reducing precision after the model is already trained, quantization-aware training incorporates lower-precision values into the training process itself so the model learns to operate within those constraints.

Model pruning

Model pruning is distinct from feature pruning, which is used to address overfitting. Model pruning removes low-importance weights by setting them to zero during training, targeting the connections that contribute the least to the model's output. Increasing the number of zero-value parameters compresses the network.

Before pruning, a neural network is densely connected, with nodes linked to many others. After pruning, low-importance weights are zeroed out and removed, leaving a sparser structure made up of the connections that matter most to predictions. Because removing weights alters what the network has learned, model pruning usually requires fine-tuning afterward to bring accuracy back to an acceptable level.

Model pruning reduces storage and memory usage and can improve inference speed. However, the latency improvement only appears on hardware built to skip zero-value computations, rather than on every device.

Weight clustering

Weight clustering shrinks a model by grouping similar weight values together to reduce the number of unique values the model needs to store. Multiple original weights are replaced with shared representative values, one per cluster.

Consider a set of 30 unique weight values before clustering. Grouping similar numbers into four clusters collapses those small variations into four shared representatives, such as 0.82, 0.35, -0.2, and -0.68. Instead of storing 30 separate numbers, the model stores four values along with a reference indicating which cluster each weight belongs to. Across millions of parameters, that representation substantially reduces storage size.

Weight clustering primarily improves storage efficiency rather than inference latency, because the math still runs the same way at inference time. Like model pruning, it requires fine-tuning afterward to recover accuracy lost from replacing original weights with shared values.

Distillation

Distillation creates a smaller model by transferring knowledge from a larger, more complex model during training. A large network with strong performance, sometimes called the teacher, passes what it has learned to a compact network, sometimes called the student. The resulting student model reduces size and improves inference speed while aiming to stay close to the teacher's performance.

This approach is particularly useful when deploying to devices with limited memory or compute where running the full-sized model is impractical. Unlike quantization, model pruning, and weight clustering, distillation requires training a new model from scratch rather than modifying an existing model.

Comparing optimization techniques on the exam

Scenarios on the Professional Machine Learning Engineer exam often narrow the choice by stating the target hardware, acceptable retraining workflow, or primary bottleneck:

  • When you want a fast starting point without preparing sample data, dynamic range quantization is the direct default.
  • When targeting GPU inference with minimal accuracy loss, float-16 post-training quantization fits well.
  • When deploying to integer-only hardware such as edge TPUs, integer quantization is needed along with sample input data.
  • When post-training quantization degrades accuracy too far, quantization-aware training lets the model adapt during training.
  • When the goal is purely storage reduction and inference latency is not the focus, weight clustering targets the storage footprint, followed by fine-tuning.
  • When you want a sparser network and have hardware capable of skipping zero-value computations, model pruning reduces size and can speed up inference after fine-tuning.
  • When a constrained device cannot run the original architecture at all, distillation trains a compact student model from scratch using a larger teacher model.

Our Professional Machine Learning Engineer course covers model optimization techniques alongside training infrastructure and hardware accelerators, with practice questions that drill these distinctions.

Get tips and updates from GCP Study Hub

arrow