Custom Inference Routines: Preprocessing and Serving on Agent Platform

GCP Study Hub
November 12, 2026

Custom inference routines on Gemini Enterprise Agent Platform (formerly Vertex AI) allow custom containers to run preprocessing and postprocessing code together with a trained model at serving time. Instead of an endpoint receiving a prediction request and handing it straight to the model, your custom code runs before the model sees the input and runs again on the way out. You may also encounter this feature under its former name, custom prediction routines. For the Professional Machine Learning Engineer exam, understanding how custom inference routines work helps distinguish lightweight Python request handling from serving options built primarily for performance.

Preprocessing and postprocessing at serving time

Running custom code around a trained model lets you keep request preparation and response formatting inside the serving layer. The work divides into two stages around the model call:

  • Preprocessing covers everything that happens before the model sees the input. Typical tasks include transforming incoming data into the format the model expects and fetching additional data from external services, such as looking up a feature that the client application does not have.
  • Postprocessing runs after the model produces a raw prediction. It formats the output into the structure the calling application expects or applies business logic, such as suppressing a prediction that falls below a confidence threshold.

Implementing the predictor interface

To use a custom inference routine, you implement a predictor interface. This interface is a Python class that defines the logic executed for each prediction request. Within that class, you fill in methods for loading the model, preprocessing the input, running the prediction itself, and postprocessing the result. Agent Platform calls these methods in that order every time a request arrives.

Deployment workflow on Agent Platform

Getting a custom inference routine into production follows four steps across Docker, Artifact Registry, and Agent Platform:

  • Build a custom Docker container that holds the trained model artifacts together with your custom inference routine, including the preprocessing and postprocessing code.
  • Push the container image to Artifact Registry for image storage.
  • Upload the model to Agent Platform Model Registry, pointing it to the container image stored in Artifact Registry.
  • Deploy the model to an Agent Platform endpoint, which pulls the image, runs the container, and serves predictions through your predictor class.

When to choose custom inference routines

Custom inference routines are best suited for models that need lightweight Python preprocessing or postprocessing without requiring you to manage containers or servers. On the Professional Machine Learning Engineer exam, the choice comes down to whether you need lightweight Python request logic or serving performance. When performance is the priority instead, we would lean toward NVIDIA Triton Inference Server rather than a custom inference routine.

Our Professional Machine Learning Engineer course covers custom inference routines alongside Agent Platform endpoints and NVIDIA Triton Inference Server, with practice questions that drill these distinctions.

Get tips and updates from GCP Study Hub

arrow