
Custom training on Gemini Enterprise Agent Platform lets you bring your own training code and have the platform run it on managed infrastructure. You stay in control of the model architecture and the training logic, and the platform handles everything underneath. Custom training jobs run in one of two ways, serverless training or training clusters. For the Professional Cloud Architect exam, the choice between them comes down to cost, capacity, and how much a job can tolerate waiting for hardware or losing a node partway through.
Serverless training runs a custom training job without you provisioning or managing any infrastructure. Your code executes inside containers, and the job's configuration lives in a worker pool spec. That object declares both the compute and the container specification for the job, so a single spec describes which machine to use and which image to run on it. You are describing the hardware you want, not creating or maintaining it.
Compute is provisioned on demand and deleted once the job finishes. You pay only while the job is actually running, and nothing is left over to clean up afterward. That is why serverless training is the cheaper default for most work.
Training clusters take the opposite approach. A dedicated cluster of accelerators is reserved and remains allocated to the workload, which suits large-scale or mission-critical jobs. Because the capacity is reserved, you get assured capacity and no queue time, so jobs start immediately. Without a reservation, a large job can sit waiting for accelerators to become available.
Training clusters also automate the complex parts of running that hardware, including environment setup and hardware fault recovery, to maximize continuous cluster uptime. On a long distributed run, a single failed node would normally end the job. The cluster detects the failure and recovers the node so that training keeps going.
Serverless training is the default choice, since you pay only while the job runs and never touch the infrastructure. Training clusters are the choice when a job is large enough or important enough that waiting on capacity, or dealing with hardware failures, is not acceptable. In a Professional Cloud Architect scenario, a focus on cost and on avoiding infrastructure management points toward serverless training. Requirements for assured accelerator capacity, immediate job start, or a long distributed run that has to survive node failures point toward training clusters.
A job on a training cluster moves through a consistent set of stages, from raw data to a deployed model.
It begins with loading data. Training data might come from Cloud Storage, BigQuery, or a managed dataset inside Agent Platform. These are the raw inputs.
The next stage is the training script, which defines the actual logic of the model. You can use pre-built containers that come with frameworks such as scikit-learn, XGBoost, or PyTorch already set up. If you need more flexibility, you can build your own container and push it to Artifact Registry. That custom container holds your training code, including any custom logic or dependencies the job needs.
Once the training image is ready, you configure the compute environment.
Then you choose the execution mode. Training can run on a single VM, or it can be distributed across multiple VMs when the job is large or complex.
When training completes, the resulting model can be pushed into Agent Platform Model Registry, where model versions are tracked and managed. From there the model can be deployed to an endpoint, which creates an API that applications can call for real-time predictions.
Overall, this workflow gives full flexibility to customize both the training logic and the execution environment, while Google Cloud takes care of orchestration and scaling behind the scenes.
Our Professional Cloud Architect course covers Agent Platform custom training jobs alongside Model Registry and online prediction endpoints, with practice questions that drill these distinctions.