Agent Telemetry on Agent Runtime: Tracing Model and Tool Latency

GCP Study Hub
November 18, 2026

Setting GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY to true when deploying an ADK agent on Agent Runtime turns on traces, logs, and metrics. In Cloud Trace, each model call and each tool call becomes its own span, so the trace of a request shows where its time went. On the Professional Agentic Architect exam, latency troubleshooting scenarios focus on this distinction between trace spans that expose individual steps inside a request and other signals that only measure whole requests, record requests at a gateway, or measure processor time.

Enabling telemetry on Agent Runtime

An ADK agent often handles a single user request through multiple model calls and several tool calls. When responses become slow, knowing that an overall request took a long time does not indicate which step caused the delay. Setting GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY to true at deployment turns on traces, logs, and metrics for an ADK agent deployed on Agent Runtime.

With telemetry enabled, Cloud Trace records a separate span for each model call and each tool call inside every request. Opening slow requests in the trace view shows where time was spent across the execution path, and comparing span durations points directly to the step responsible for the delay.

Using ADK OpenTelemetry spans on Cloud Run

ADK is already instrumented with OpenTelemetry, an open standard for tracing. It emits spans such as invoke_agent, execute_tool for each tool call, and generate_content for each model call. When an ADK agent runs a multi-step workflow on Cloud Run, you can export those OpenTelemetry spans to Cloud Trace, for example with get_gcp_exporters(enable_cloud_tracing=True).

In Cloud Trace, those exported spans appear as a waterfall showing where each request spent its time, with each tool call and each model call recorded as its own span. Using OpenTelemetry also keeps the traces portable if a team later exports them to another backend.

Why other signals do not isolate slow steps

Several monitoring, logging, and profiling tools provide useful operational visibility, but they cannot identify which individual step inside a running agent request is slow. Separating these options is a common decision point on the Professional Agentic Architect exam:

  • Cloud Monitoring metrics: Metrics such as overall request latency, including Cloud Monitoring charts for a Cloud Run service, confirm that whole requests are slow. They do not show which step inside a request is responsible.
  • Logs and timestamp comparisons: Adding timestamped print statements or comparing log timestamps by hand or in BigQuery requires manual work to line up entries. This approach is also incomplete because it misses steps that write no log entries, and timestamped print statements miss model calls unless each one is wrapped by hand.
  • CPU profiling with Cloud Profiler: A CPU profiler such as Cloud Profiler shows where code spends processor time. Because an agent spends most of its time waiting on model calls and tool calls, a CPU profile does not explain where request latency went.
  • Gateway trace propagation and access logs: Trace header propagation through a gateway carries trace context between hops, which only helps if the agent itself emits spans. Without telemetry enabled on the agent, there are no agent spans to inspect. Gateway access logs also record requests at the gateway rather than the steps inside the agent.
  • More minimum instances: Increasing minimum instances removes cold starts, but it does not explain delays inside a request that is already running.
  • Gen AI evaluation service: This service measures response quality rather than latency.

Choosing the right tracing setup

When diagnosing slow multi-step requests for an ADK agent on Agent Runtime, we would set GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY to true at deployment and inspect the slow request spans in Cloud Trace. When running an ADK agent on Cloud Run with built-in instrumentation and an open standard, exporting ADK's OpenTelemetry spans to Cloud Trace provides the same per-step waterfall view while keeping traces portable across backends. Keeping trace spans distinct from aggregate metrics, manual log correlation, CPU profiles, and gateway logs covers how agent latency troubleshooting appears on the Professional Agentic Architect exam.

Our Professional Agentic Architect practice exams test agent telemetry on Agent Runtime alongside OpenTelemetry tracing on Cloud Run and Cloud Monitoring signals, with detailed explanations that drill these distinctions.

Get tips and updates from GCP Study Hub

arrow