
Custom inference routines on Gemini Enterprise Agent Platform (formerly Vertex AI) allow custom containers to run preprocessing and postprocessing code together with a trained model at serving time. Instead of an endpoint receiving a prediction request and handing it straight to the model, your custom code runs before the model sees the input and runs again on the way out. You may also encounter this feature under its former name, custom prediction routines. For the Professional Machine Learning Engineer exam, understanding how custom inference routines work helps distinguish lightweight Python request handling from serving options built primarily for performance.
Running custom code around a trained model lets you keep request preparation and response formatting inside the serving layer. The work divides into two stages around the model call:
To use a custom inference routine, you implement a predictor interface. This interface is a Python class that defines the logic executed for each prediction request. Within that class, you fill in methods for loading the model, preprocessing the input, running the prediction itself, and postprocessing the result. Agent Platform calls these methods in that order every time a request arrives.
Getting a custom inference routine into production follows four steps across Docker, Artifact Registry, and Agent Platform:
Custom inference routines are best suited for models that need lightweight Python preprocessing or postprocessing without requiring you to manage containers or servers. On the Professional Machine Learning Engineer exam, the choice comes down to whether you need lightweight Python request logic or serving performance. When performance is the priority instead, we would lean toward NVIDIA Triton Inference Server rather than a custom inference routine.
Our Professional Machine Learning Engineer course covers custom inference routines alongside Agent Platform endpoints and NVIDIA Triton Inference Server, with practice questions that drill these distinctions.