
The Dataflow RunInference API is an Apache Beam transform that runs machine learning predictions directly inside a Dataflow pipeline. Rather than sending each incoming element over the network to a separate prediction service, the model executes in the same pipeline step that is already processing your data. Removing that network round trip keeps prediction latency very low. For the Professional Machine Learning Engineer exam, understanding how RunInference handles execution, batching, model updates, and hardware acceleration helps separate in-pipeline inference from architectures that rely on external endpoints.
Because the RunInference transform runs inside the Dataflow pipeline step itself, elements do not need to leave the pipeline for a remote prediction call. Eliminating that network round trip is what keeps prediction latency low.
To improve throughput, RunInference also dynamically batches incoming inputs. Instead of invoking the model once for each individual element, Apache Beam groups incoming elements together and passes them to the model as a batch. Beam tunes the batch size automatically based on the load the pipeline is seeing. Larger batches make more efficient use of model compute, especially when inference runs on accelerators.
When models are retrained on a schedule, the RunInference transform automatically refreshes the model so the pipeline always uses the latest deployed version. You publish a new model artifact, and the running pipeline picks it up and swaps to the updated version in place.
Because the refresh happens without stopping the Dataflow job or redeploying the pipeline, workloads that rely on scheduled retraining can run daily model updates with no downtime.
RunInference can use GPUs on supported runners to increase inference speed. Dataflow lets you attach GPUs to worker VMs so the prediction step runs on accelerated hardware. This setup pays off for deep learning models where the computation is heavy enough to justify the accelerator. Enabling GPU acceleration on the workers speeds up inference without requiring changes to how the pipeline itself is written.
To see how these pieces fit together, consider a streaming fraud detection job running on Dataflow:
On the Professional Machine Learning Engineer exam, scenarios that combine streaming ingestion from Pub/Sub, low-latency predictions without external network round trips, scheduled model updates without stopping the job, and optional GPU acceleration for deep learning models point directly to Dataflow with the RunInference API.
Our Professional Machine Learning Engineer course covers the Dataflow RunInference API alongside Dataflow streaming pipelines and Pub/Sub ingestion, with practice questions that drill these distinctions.