Close httplib2 connections.
Gets a task.
list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
Lists tasks under a parent. Supports filtering (e.g. by status or assigned_agent) so a surface can show "my tasks".
list_next(previous_request, previous_response)
Retrieves the next page of results.
close()
Close httplib2 connections.
get(name, x__xgafv=None)
Gets a task.
Args:
name: string, Required. The name of the task to retrieve. Both name forms are accepted: * projects/{project}/locations/{location}/agents/{agent}/tasks/{task} * projects/{project}/locations/{location}/tasks/{task} (required)
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # A durable, addressable unit of work: scoped context + a goal with verifiers, assigned to an agent, and executed as a trajectory of steps. A Task is the resource users open, list, and manage. It is a light "header" (status, title, summary, outcome). The executed trajectory (steps) is not stored inline -- it is streamed live over CreateTaskBidiStream (fetched from the runtime), so a long-running/re-firing task carries no unbounded blob. Resource name (canonical): projects/{project}/locations/{location}/agents/{agent}/tasks/{task} Task is a sub-resource of Agent. A short alias projects/{project}/locations/{location}/tasks/{task} is also accepted (dual-path); the task id is unique within a project+location, so both forms resolve the same task. The short form is retained intentionally : * GetTask/ListTasks support both (dual) resource paths. * Subtasks store the short form in parent_task_id, so that reassignment of the parent task to another agent later will not require updating the subtasks, and the rendering of the task tree will not imply the agent relationships.
"assignedAgent": "A String", # Output only. The resource name of the `Agent` this task is assigned to, in the format `projects/{project}/locations/{location}/agents/{agent}`. It is derived from the create parent; any value set by the client is ignored.
"assignedAgentIdentity": "A String", # Output only. The resolved Agent Identity (SPIFFE ID) of the `Agent` this task is assigned to. Populated on single-task RPCs (`CreateTask`, `GetTask`, `UpdateTask`, `SuspendTask`, `ResumeTask`, `CancelTask`); not populated by `ListTasks`.
"budgetConsumptionDetail": { # Observability snapshot of budget consumption for a Task . # Output only. Current budget consumption detail of the Task.
"attributions": [ # Output only. Spend broken down by consumer.
{ # Per-consumer attribution: which unit of the Task tree incurred the spend.
"root": { # The root (owning) Task's own spend. # Output only. The root Task's own spend.
},
"spent": { # A consumer's spend, per Budget dimension. # Output only. This consumer's spend, per dimension.
"tokenCount": "A String", # Output only. Tokens spent by this consumer.
"toolCallCount": "A String", # Output only. Tool calls spent by this consumer.
},
"subTask": { # A spawned sub-Task's spend. # Output only. A spawned sub-Task's spend.
"task": "A String", # Output only. Resource name of the sub-Task that incurred the spend.
},
},
],
"tokenConsumption": { # Consumption for one budget dimension. # Output only. Token budget status. This is only populated if the Task has a token limit Budget set.
"limit": "A String", # Output only. The cap from `TaskSpec.budget` for this dimension's consumption.
"spent": "A String", # Output only. Amount consumed so far.
},
"toolCallConsumption": { # Consumption for one budget dimension. # Output only. Tool-call budget status. This is only populated if the Task has a tool call Budget limit set.
"limit": "A String", # Output only. The cap from `TaskSpec.budget` for this dimension's consumption.
"spent": "A String", # Output only. Amount consumed so far.
},
},
"createTime": "A String", # Output only. The time the task was created.
"goals": [ # Output only. Goals of the Task, including both active and completed/failed. A Task may pursue several concurrently active goals .
{ # A goal is an objective a Task works toward. A Task may pursue several concurrent goals . A goal could have one or more Verifiers as independent judges to validate the completion of it.
"displayName": "A String", # Optional. A human-readable title specifiable to the goal.
"id": "A String", # Output only. Server-assigned identifier, stable across the goal's lifetime.
"spec": "A String", # Required. The goal's objective. This is the "goal" and what the agent should pursue.
"state": "A String", # Output only. The goal's state.
"verifiers": [ # Optional. The verifier(s) that judge whether this goal is achieved. When all `required` verifiers pass, the goal will be marked `SUCCEEDED` in its `state`. If empty, the work is still checked against criteria written from the goal's `spec`, but the goal stays `ACTIVE`; add a required `auto_verification` to have it complete.
{ # A declarative success criterion for a goal. Harness-agnostic: the runtime follows the description to decide pass/fail and to repair on failure. A criterion may be checked automatically (`auto_verification`, `nl_criteria`, `command_check`) or by a human (`human_check`, resolved via a HITL elicitation at verification time). If `kind` is unset the Verifier carries no criterion and is ignored. On a Task with a local environment, verifiers do not access the environment: `nl_criteria` and `auto_verification` are judged on the record of the agent's work in the current turn, and `command_check` is not allowed.
"autoVerification": { # Judges the goal against its own `spec`, with no separate criterion to write. List it more than once to run that many independent judgements of the same objective; as with any verifier, every `required` one must pass for the goal to succeed. # Automated. Judged against the goal's own `spec`.
},
"name": "A String", # Optional. Optional label used for reporting (e.g. "tests-green").
"nlCriteria": { # An automated natural-language criterion evaluated by an autorater/agent. Wrapped in a message (rather than a bare string) so evaluation parameters (e.g. model, temperature, language) can be added without a breaking change. # Automated. A customizable natural-language rubric an autorater/agent evaluates.
"criteria": "A String", # Required. The rubric to evaluate (e.g. "Every unread high-priority thread has a summary and a draft").
},
"required": True or False, # Optional. If true, the goal cannot be marked COMPLETED unless this verifier passes. If false, the verifier is advisory (recorded, non-blocking).
"state": "A String", # Output only. The verifier's latest verdict.
},
],
},
],
"labels": { # Optional. Optional caller-supplied labels. Must follow the unified cloud labels spec (keys/values lowercase `[a-z0-9_-]`, <=63 chars). Stored as-is and returned verbatim on Get/ListTask, so the caller owns what goes in and reads back. Label names starting with `goog-` or `google-` are reserved for predefined labels. Besides user-defined billing/reporting labels, callers may stamp non-PII operational traffic annotations here under the reserved prefix (persisted on the task, so they apply to every turn and survive reconnection). Predefined: `goog-golden-query-id`: stable golden-query / CUJ id (e.g. `hello-world`); absent for organic traffic. Lets per-CUJ latency be trended independently.
"a_key": "A String",
},
"name": "A String", # Identifier. Resource name of the task.
"nextRunTime": "A String", # Output only. Next scheduled firing of a recurring template. Unset once the template is `SCHEDULE_FINISHED`, and for cron-less (event) schedules.
"parentTaskId": "A String", # Optional. If this task is a subtask, the parent task's short-form, agent-independent resource name: projects/{project}/locations/{location}/tasks/{task} The short form is used here on purpose (NOT the agent-scoped .../agents/{agent}/tasks/{task}): it stays stable when the parent is reassigned to another agent, so subtasks never need rewriting, and the task tree does not imply agent relationships .
"rootTaskId": "A String", # Output only. Lineage group key: the root of this task's tree, materialized from `parent_task_id` (a root task's own short-form name). Same short form, and for the same reason, as `parent_task_id`. One `ListTasks(filter: "root_task_id=")` returns the whole tree as a flat, paginated collection, instead of a level-by-level walk that costs one round trip per depth. The client reassembles the tree from `parent_task_id`; a page may therefore contain a task whose parent lands on a later page.
"runCount": "A String", # Output only. Runs minted so far by a recurring template, so a client can show progress against `RecurringSchedule.max_run_count`.
"scheduleState": "A String", # Output only. Scheduling lifecycle of a recurring template. Unset unless `spec.schedule.recurring` is set.
"spec": { # The centralized specification of a Task: the scoped context it runs in, when it runs, the resource guardrails, and the goal (with verifiers). # Required. The full specification of what to do, how to run it, and how to know it is done.
"budget": { # Holistic resource caps metered across the entire task. # Optional. Optional holistic budget caps metered across the entire task lifetime.
"maxToolCalls": "A String", # Optional. Maximum total tool calls across the whole task.
"maxTotalTokens": "A String", # Optional. Maximum total model tokens across the whole task.
"maxWallclock": "A String", # Optional. Maximum wall-clock duration for the task. Counts only time spent in `WORKING` or `REQUIRES_ACTION`; `IDLE`, `QUEUED`, and `SUSPENDED` do not accrue.
},
"deltaStreaming": True or False, # Optional. Whether incrementally generated content is delivered as it is produced, rather than once the step is finished. The two modes are exact duals. If false (default), every step arrives once, complete, in TaskServerEvent.step -- so a client that reads only `step` sees the whole conversation. Suited to non-chat callers that consume finished steps. If true, EVERY step instead arrives as a `step_start` / `step_delta`* / `step_stop` run sharing an index, so a chat client sees one uniform lifecycle and never a bare `step`; reassembling a run yields the same step that would otherwise have been delivered. Content the model generates progressively -- assistant text (and the citations on it) and thoughts -- streams natively across the run's `step_delta`s as it is produced. A step that arrives whole (tool calls, tool results, elicitations, user-input echoes, errors) carries no `step_delta`: the complete step rides `step_start` and `step_stop` closes the run, since nothing is generated incrementally. The field is deliberately not named for text: what streams progressively is an implementation detail that may widen, and a client asking for incremental delivery wants it for whatever supports it. Note per-step token usage rides `step_stop`, so it is reported only when this is true.
"expectedOutput": { # Optional. Optional structured deliverable contract (reuses the Interactions ResponseFormat). When set, the agent's final output must conform to it.
"audio": { # Configuration for audio output format.
"bitRate": 42, # Bit rate in bits per second (bps). Only applicable for compressed formats (MP3, Opus).
"delivery": "A String", # The delivery mode for the audio output.
"mimeType": "A String", # The MIME type of the audio output.
"sampleRate": 42, # Sample rate in Hz.
},
"image": { # Configuration for image output format.
"aspectRatio": "A String", # The aspect ratio for the image output.
"delivery": "A String", # The delivery mode for the image output.
"imageSize": "A String", # The size of the image output.
"mimeType": "A String", # The MIME type of the image output.
},
"structValue": { # `Struct` represents a structured data value, consisting of fields which map to dynamically typed values. # Multi-discriminator values is already enabled in GAOS
"fields": [ # Dynamically typed fields. List instead of map because LLMs are sensitive to ordering, and we want to give users full control.
{ # Represents a single field in a struct.
"name": "A String",
"value": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
],
},
"text": { # Configuration for text output format.
"mimeType": "A String", # The MIME type of the text output.
"schema": { # `Struct` represents a structured data value, consisting of fields which map to dynamically typed values. # The JSON schema that the output should conform to. Only applicable when mime_type is application/json. Supports a subset of JSON Schema. `minLength`, `maxLength`, `minProperties`, `maxProperties`, `uniqueItems`, `contains`, `minContains`, and `maxContains` are ignored and not enforced; validate these in your application. `minItems` and `maxItems` are enforced. Schemas that are too complex to enforce (for example, large `maxItems` on nested arrays) return a 400 INVALID_ARGUMENT error.
"fields": [ # Dynamically typed fields. List instead of map because LLMs are sensitive to ordering, and we want to give users full control.
{ # Represents a single field in a struct.
"name": "A String",
"value": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
],
},
},
"video": { # Configuration for video output format.
"aspectRatio": "A String", # The aspect ratio for the video output.
"delivery": "A String", # The delivery mode for the video output.
"duration": "A String", # The duration for the video output.
"gcsUri": "A String", # The Cloud Storage URI to store the video output. Required for Vertex if delivery mode is URI.
"resolution": "A String", # The video output resolution. Defaults to 720p.
},
},
"hitlConfig": { # Session-level HITL (human-in-the-loop) configuration. Namespaces every HITL knob that a client may set at task creation, so new settings (per-tool overrides, timeout, allowed reviewers) can land without burning more top-level TaskSpec tags. `approval_mode` carries the wire default as its own `APPROVAL_MODE_UNSPECIFIED` enum value, which the server treats as `APPROVAL_MODE_ASK` (standard HITL, always prompt a human). That is also the pre-change behavior for every existing task. # Optional. Immutable. Session-level HITL (human-in-the-loop) configuration for this task. Unset means every HITL knob takes its default; see `HitlConfig` above for the individual field defaults.
"approvalMode": "A String", # Optional. Immutable. How the client wants elicitations on this task approved. Unset (or APPROVAL_MODE_UNSPECIFIED) is treated as APPROVAL_MODE_ASK.
},
"inputs": [ # Optional. The prompt expressed as Interaction steps (e.g. a ContentStep with the user's request). Reuses the Interactions step model so the same content flows into the runtime unchanged. Required for a scheduled or PRIORITY_DEFERRED task, which starts with no caller attached. Otherwise it may be empty: * Unary CreateTask with no inputs creates the task IDLE and dispatches nothing. The task starts when a caller subscribes to it on CreateTaskBidiStream (`subscribe_task`) and sends the first input. * CreateTaskBidiStream `create_task` with no inputs is a connect-only / pre-warm create; the first input is sent later as a client step. Recommended pattern for giving a task files before it runs: 1. CreateTask with empty `inputs`. The task is IDLE; nothing runs yet. 2. UploadTaskArtifact each file to `/workspace/userdata/`. 3. CreateTaskBidiStream with `subscribe_task`, then send the first input, referring to the files by their `/workspace/userdata/` paths. This dispatches the task with the uploaded files in its workspace.
{ # A step in the interaction.
"modelOutput": { # Output generated by the model.
"content": [
{ # The content of the response.
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
],
"error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation.
"code": 42, # The status code, which should be an enum value of google.rpc.Code.
"details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
{
"a_key": "", # Properties of the object. Contains field @type with type URL.
},
],
"message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
},
},
"thought": { # A thought step.
"signature": "A String", # A signature hash for backend validation.
"summary": [ # A summary of the thought.
{ # The content of the response.
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
],
},
"toolCall": { # Tool call step.
"codeExecutionCall": { # Code execution call step.
"arguments": { # The arguments to pass to the code execution. # Required. The arguments to pass to the code execution.
"code": "A String", # The code to be executed.
"language": "A String", # Programming language of the `code`.
},
},
"fileSearchCall": { # File Search call step.
},
"functionCall": { # A function tool call step.
"arguments": { # `Struct` represents a structured data value, consisting of fields which map to dynamically typed values. # Required. The arguments to pass to the function.
"fields": [ # Dynamically typed fields. List instead of map because LLMs are sensitive to ordering, and we want to give users full control.
{ # Represents a single field in a struct.
"name": "A String",
"value": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
],
},
"name": "A String", # Required. The name of the tool to call.
},
"googleMapsCall": { # Google Maps call step.
"arguments": { # The arguments to pass to the Google Maps tool. # The arguments to pass to the Google Maps tool.
"queries": [ # The queries to be executed.
"A String",
],
},
},
"googleSearchCall": { # Google Search call step.
"arguments": { # The arguments to pass to Google Search. # Required. The arguments to pass to Google Search.
"queries": [ # Web search queries for the following-up web search.
"A String",
],
},
"searchType": "A String", # The type of search grounding enabled.
},
"id": "A String", # Required. A unique ID for this specific tool call.
"mcpServerToolCall": { # MCPServer tool call step.
"arguments": { # `Struct` represents a structured data value, consisting of fields which map to dynamically typed values. # Required. The JSON object of arguments for the function.
"fields": [ # Dynamically typed fields. List instead of map because LLMs are sensitive to ordering, and we want to give users full control.
{ # Represents a single field in a struct.
"name": "A String",
"value": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
],
},
"name": "A String", # Required. The name of the tool which was called.
"serverName": "A String", # Required. The name of the used MCP server.
},
"processingCall": { # A server-initiated processing step for media analysis (e.g. video understanding).
},
"retrievalCall": { # Retrieval call step. Used by Vertex Retrieval tools such as Parallel AI, Exa AI, Vertex AI Search, etc. RetrievalType decides which tool is used.
"arguments": { # The arguments to pass to Retrieval tools. # Required. The arguments to pass to the retrieval tool.
"queries": [ # Queries for Retrieval information.
"A String",
],
},
"retrievalType": "A String", # The type of retrieval tools.
},
"signature": "A String", # A signature hash for backend validation.
"urlContextCall": { # URL context call step.
"arguments": { # The arguments to pass to the URL context. # Required. The arguments to pass to the URL context.
"urls": [ # The URLs to fetch.
"A String",
],
},
},
},
"toolResult": { # Tool result step.
"callId": "A String", # Required. ID to match the ID from the function call block.
"codeExecutionResult": { # Code execution result step.
"isError": True or False, # Whether the code execution resulted in an error.
"result": "A String", # Required. The output of the code execution.
},
"fileSearchResult": { # File Search result step.
},
"functionResult": { # Result of a function tool call.
"isError": True or False, # Whether the tool call resulted in an error.
"name": "A String", # The name of the tool that was called.
"result": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error. # Required. The result of the tool call.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
"googleMapsResult": { # Google Maps result step.
"result": [
{ # The result of the Google Maps.
"places": [
{
"name": "A String",
"placeId": "A String",
"reviewSnippets": [
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String",
},
],
"widgetContextToken": "A String",
},
],
},
"googleSearchResult": { # Google Search result step.
"isError": True or False, # Whether the Google Search resulted in an error.
"result": [ # Required. The results of the Google Search.
{ # The result of the Google Search.
"searchSuggestions": "A String", # Web content snippet that can be embedded in a web page or an app webview.
},
],
},
"mcpServerToolResult": { # MCPServer tool result step.
"name": "A String", # Name of the tool which is called for this specific tool call.
"result": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error. # Required. The output from the MCP server call. Can be simple text or rich content.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
"serverName": "A String", # The name of the used MCP server.
},
"processingResult": { # The result of a server-initiated media processing step.
},
"retrievalResult": { # Vertex Retrieval result step. Used by Vertex Retrieval tools such as Parallel AI, Exa AI, Vertex AI Search, etc.
"isError": True or False, # Whether the retrieval resulted in an error.
},
"signature": "A String", # A signature hash for backend validation.
"urlContextResult": { # URL context result step.
"isError": True or False, # Whether the URL context resulted in an error.
"result": [ # Required. The results of the URL context.
{ # The result of the URL context.
"status": "A String", # The status of the URL retrieval.
"url": "A String", # The URL that was fetched.
},
],
},
},
"userInput": { # Input provided by the user. # DO NOT USE -- These are for 3P JSON only
"contentList": { # A list of Content. # The content of the step. An array of Content objects.
"contents": [ # The contents of the list.
{ # The content of the response.
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
],
},
"contentString": "A String", # The content of the step. A single string.
},
},
],
"maxRetries": 42, # Optional. Deprecated: tasks are not retried automatically. CreateTask and UpdateTask reject a non-zero value with INVALID_ARGUMENT.
"modelConfig": { # Selects the model the task runs on. Starts with just the model id/alias; extensible with further model parameters without a breaking change. # Optional. Optional model configuration override for the task. When set, the task runs on this model configuration for its entire lifecycle.
"effortLevel": "A String", # Optional. Reasoning/thinking depth the model runs at, case-insensitive. Gemini takes "minimal", "low", "medium" or "high", and only on models that have a thinking level; other publishers define their own set. A value the resolved model does not accept leaves its own default standing.
"maxOutputTokens": 42, # Optional. Maximum number of tokens the model may generate in each call, including any thinking tokens it spends before answering. Must not be negative; zero leaves the backend default standing. The cap also applies to the calls that summarize the task's history when its context is compacted, so a model that thinks at a high effort level may need a larger value to return a summary at all.
"model": "A String", # Optional. Publisher model resource name (an `aiplatform.googleapis.com/PublisherModel`) of the form "publishers/{publisher}/models/{model}", e.g. "publishers/anthropic/models/claude-opus-4-8" or "publishers/google/models/gemini-3-flash". The project-qualified form "projects/{project}/locations/{location}/publishers/{publisher}/models/{model}" is also accepted. When it is used, the project and location it carries take precedence over both dynamic project/location resolution and the actor config defaults. Model selection cannot be updated after task creation right now though it might be supported in the future. Empty means the actor's default gemini model.
"modelSelectionMode": "A String", # Optional. How the model in `model` was selected. Ignored when set on `LiveConfig.model_config`.
"temperature": 3.14, # Optional. Sampling temperature for the model's generation config (valid range (0.0, 2.0]); a value outside that range leaves the backend default standing.
},
"priority": "A String", # Optional. Optional priority band. Concrete tiers and quota are out of scope and deferred to a follow-up.
"schedule": { # Defines when and how a task runs. # Optional. Optional schedule controlling when/how the task runs. If unset, the task runs immediately (one-off, now).
"oneOff": { # A one-off (single) run trigger. # Run exactly once (at an absolute time or after a delay).
"delay": "A String", # Relative delay before the single run.
"runAt": "A String", # Absolute time to run at (e.g. 8 PM today).
},
"recurring": { # Defines a recurring schedule logic for repeated task executions. # Run the task spec repeatedly.
"cron": { # A recurring cron trigger. # The cron expression, e.g. "0 8 * * MON-FRI".
"expression": "A String", # Required. The cron expression, e.g. "0 8 * * MON-FRI". Any field may also be `?` or `?/step` to let the service pick the value, so that schedules do not all fire at the same moment: `?` picks one value in the field's range, and `?/step` picks a start within the first step and repeats every step from there. The pick is made once, when the schedule is created or updated, and the expression is stored with the picked values in place of `?` (e.g. "?/30 * * * *" is stored as "17-59/30 * * * *"); sending a `?` again picks new values.
"timeZone": "A String", # Optional. Optional IANA time zone name, e.g. "America/Los_Angeles". Defaults to UTC.
},
"endTime": "A String", # Optional. Do not fire after this time.
"eventSource": { # An event-driven trigger. # Identifier of a registered event source / webhook subscription whose events fire the task.
"eventSource": "A String", # Required. Identifier of a registered event source / webhook subscription whose events fire the task.
},
"maxRunCount": "A String", # Optional. Max total number of firings. Unset means unbounded.
"overlapPolicy": "A String", # Optional. What to do when a firing is due while a prior run is still active.
"startTime": "A String", # Optional. Do not fire before this time.
},
},
"scope": { # The scoped context a task runs within. # Optional. Optional scoped context: skills/tools/memory/artifacts for the task.
"allowedTools": { # A set of tool references, each a tool's GCP resource name (`projects/.../mcpServers|endpoints/...`) with optional fine-grained method, CLI tool and resource guardrails. Wraps a repeated field so it can be used as an arm of the `ContextScope.tool_scope` oneof. LINT.IfChange(tool_refs) # Keep only these tools from the agent's inherited set.
"cliToolScope": { # A CLI tool boundary restricting a subtask to designated CLI commands. # Optional. Optional CLI tool scope that the task is restricted to. Unset means all CLI tools are allowed. Set with an empty `cli_tools` list means all CLI tools are denied.
"cliTools": [ # Optional. Allowed CLI tools for the task. Example: ["gmail send_message", "calendar list_events"].
"A String",
],
},
"methods": [ # Optional. Optional fine-grained method selectors for the referenced tools. Format: "/methods/".
"A String",
],
"resources": [ # Optional. Optional resource identifiers that the task is restricted to interact with.
{ # A specific resource boundary restricting an agent's access to designated entities.
"recipients": [ # Optional. Optional recipients email boundaries for communication tools. Example: ["alice@gmail.com", "bob@google.com"].
"A String",
],
"resource": "A String", # Optional. The specific resource identifier (e.g. thread_id, doc_id, repo name). Examples: * GMAIL_THREAD: "18df123456789abc" * GOOGLE_CHAT_THREAD: "spaces/AAA/threads/BBB" * GITHUB_REPOSITORY: "GoogleCloudPlatform/kubectl"
"resourceType": "A String", # Required. The type of resource being scoped.
"selfCreated": True or False, # Optional. For cold outreach / outbound threads when resource_id doesn't exist. Grants access ONLY to resources created by this task instance.
},
],
"toolNames": [ # Optional. Tool GCP resource names. For example: * MCP server: `projects/my-project/locations/us-central1/mcpServers/jira` * Endpoint: `projects/my-project/locations/us-central1/endpoints/drive`
"A String",
],
},
"collaborativeProject": { # Scopes a task to a `CollaborativeProject` . Starts with the collaborative project resource reference; extensible with per-task collaborative project settings without a breaking change. # Optional. Optional collaborative project configuration for this task. Only a root task (`parent_task_id` unset) may set or update this field; when `parent_task_id` is set, the subtask automatically inherits its root task's collaborative project (setting a different value is rejected with `INVALID_ARGUMENT`). Updating or clearing this field via `UpdateTask` is only allowed while the task is in the `IDLE` state (attempts in any other state are rejected with `FAILED_PRECONDITION`).
"collaborativeProject": "A String", # Optional. The resource name of the `CollaborativeProject` this task is scoped to: `projects/{project}/locations/{location}/collaborativeProjects/{collaborative_project}` The `{project}` segment must match the task's own Google Cloud project. Empty when the task is not scoped to a collaborative project. Only a root task (`parent_task_id` unset) may set or update this field; when `parent_task_id` is set, the subtask automatically inherits its root task's collaborative project (setting a different value is rejected with `INVALID_ARGUMENT`). Updating or clearing this field via `UpdateTask` is only allowed while the task is in the `IDLE` state (attempts in any other state are rejected with `FAILED_PRECONDITION`).
},
"dataConnectorConfig": { # Configuration for Gemini Enterprise MCP data connectors scoped to this task. # Optional. Data connector configuration for this task.
"dataConnectors": { # Optional. Map of active connector short name / data_source (e.g., "jira", "servicenow") to its full DataConnector resource name (e.g., "projects/{project}/locations/{location}/collections/{collection}/dataConnector"). If multiple connectors share a data_source, the first configured connector for that data_source is retained.
"a_key": "A String",
},
"engineResourceName": "A String", # Optional. Full resource name of the Discovery Engine Engine (e.g., "projects/{project}/locations/{location}/collections/{collection}/engines/{engine}").
},
"environment": { # Configuration for a custom environment. # Optional. Optional server-managed sandbox environment for the task: skills and tools mounted into the sandbox plus the egress policy. Reuses the Interactions EnvironmentConfig (e.g. SKILL_REGISTRY sources to mount skills, network egress allowlist).
"env": { # Environment variables to set in the sandbox environment.
"a_key": { # An environment variable to set in the execution environment.
"credential": "A String", # Optional reference to a server-managed Credential resource by ID.
"value": "A String", # Direct string value for plain environment variables.
},
},
"environmentId": "A String", # Optional. The environment ID for the interaction. If specified, the request will update the existing environment instead of creating a new one.
"memoryMounts": [ # Optional. Memory stores mounted into the environment under `/memory/`. When merged with an agent or base environment configuration, non-conflicting mounts are preserved and request mounts take precedence on store ID or dir_name collisions.
{ # A memory store mounted into the environment under `/memory/`.
"description": "A String", # Optional. Per-mount guidance describing when the agent should read or write this memory store.
"dirName": "A String", # Optional. Directory name under `/memory//`. When omitted, defaults to the store's `memory_store_id`.
"memoryStore": "A String", # Required. Resource name or ID of the MemoryStore (e.g. `memoryStores/my-store` or `my-store`).
"mode": "A String", # Optional. Access mode granted to the environment for this store. Defaults to `READ_ONLY` when unspecified.
},
],
"networkAllowlist": { # Network egress configuration for the environment. # Allow only specific domains.
"allowlist": [ # List of allowed domains and their configurations. Set to `"disabled"` to block all network egress.
{ # A single domain allowlist rule with optional header injection.
"credential": "A String", # Optional. Reference to a server-managed Credential resource by ID.
"domain": "A String", # Domain to allow outbound requests to. Supports wildcards (e.g. '*.googleapis.com'). Use '*' to allow all domains.
"mode": "A String", # Network egress mode. Set via the `"allowlist": "disabled"` short form; must be the only rule in the allowlist and cannot be combined with `domain`, `transform` or `credential`.
"transform": { # Headers to inject into requests matching this rule. Key: header name (e.g., "Authorization"). Value: header value (e.g., "Bearer your-token").
"a_key": "A String",
},
},
],
},
"networkMode": "A String", # Network egress mode.
"sources": [
{ # A source to be mounted into the environment.
"content": "A String", # The inline content if `type` is `INLINE`.
"encoding": "A String", # Optional encoding for inline content (e.g. `base64`).
"source": "A String", # The source of the environment. For Cloud Storage, this is the Cloud Storage path. For GitHub, this is the GitHub path.
"target": "A String", # Where the source should appear in the environment.
"type": "A String",
},
],
},
"excludedTools": { # A set of tool references, each a tool's GCP resource name (`projects/.../mcpServers|endpoints/...`) with optional fine-grained method, CLI tool and resource guardrails. Wraps a repeated field so it can be used as an arm of the `ContextScope.tool_scope` oneof. LINT.IfChange(tool_refs) # Remove these tools from the agent's inherited set.
"cliToolScope": { # A CLI tool boundary restricting a subtask to designated CLI commands. # Optional. Optional CLI tool scope that the task is restricted to. Unset means all CLI tools are allowed. Set with an empty `cli_tools` list means all CLI tools are denied.
"cliTools": [ # Optional. Allowed CLI tools for the task. Example: ["gmail send_message", "calendar list_events"].
"A String",
],
},
"methods": [ # Optional. Optional fine-grained method selectors for the referenced tools. Format: "/methods/".
"A String",
],
"resources": [ # Optional. Optional resource identifiers that the task is restricted to interact with.
{ # A specific resource boundary restricting an agent's access to designated entities.
"recipients": [ # Optional. Optional recipients email boundaries for communication tools. Example: ["alice@gmail.com", "bob@google.com"].
"A String",
],
"resource": "A String", # Optional. The specific resource identifier (e.g. thread_id, doc_id, repo name). Examples: * GMAIL_THREAD: "18df123456789abc" * GOOGLE_CHAT_THREAD: "spaces/AAA/threads/BBB" * GITHUB_REPOSITORY: "GoogleCloudPlatform/kubectl"
"resourceType": "A String", # Required. The type of resource being scoped.
"selfCreated": True or False, # Optional. For cold outreach / outbound threads when resource_id doesn't exist. Grants access ONLY to resources created by this task instance.
},
],
"toolNames": [ # Optional. Tool GCP resource names. For example: * MCP server: `projects/my-project/locations/us-central1/mcpServers/jira` * Endpoint: `projects/my-project/locations/us-central1/endpoints/drive`
"A String",
],
},
"externalBrowser": { # A browser on the connecting client (e.g. the user's own, signed-in Chrome) that a task's browser actions run on. The actions are relayed to the client, so the client must stay connected while they happen. The task's other operations (files, commands) still run where `environment_config` puts them. # Optional. Optional browser on the connecting client that the task's browser actions run on, instead of a browser in a server-managed sandbox. Supported with `environment`, `no_environment`, or no `environment_config`.
},
"externalMount": { # A directory on the connecting client (e.g. the user's local machine) mounted into a task's server-managed environment. # Optional. Optional directory on the connecting client mounted into the task's server-managed environment. File operations under its `mount_path` are relayed to the client. Supported only when `environment` is set.
"mountPath": "A String", # Optional. Absolute path in the environment where the directory is mounted. Empty means `/workspace/desktop`. When set, must be `/workspace/{name}`, where `{name}` is 1-63 lowercase letters, digits, `_` or `-`, starts with a letter or digit, and is not one of the names other mounts own under `/workspace`: `skills`, `core`, `memory`, `inputs`, `artifacts`, `deliverables` or `userdata`.
"sourcePath": "A String", # Optional. Absolute path of the folder on the connecting client (the mount source). Interpreted only by the client; the server never resolves it.
},
"localEnvironment": { # Configuration for an environment that lives on the client connection rather than in a server-managed sandbox. When set (via Interaction.local_environment), the agent's filesystem and shell are treated as living on the client: the agent's built-in environment operations (e.g. reading/listing/editing files and running commands) are suspended on the server and yielded back to the client to execute, with their results returned on a subsequent turn. This is mutually exclusive with a server-managed `EnvironmentConfig` (remote_environment), since the environment is either on the client or in a server sandbox, never both. This governs only the agent's built-in environment. Client-declared function tools are always executed on the client regardless of this field. # Optional. Optional client-side ("local") environment: the agent's built-in filesystem/shell operations are yielded to the connecting client (e.g. the gogo desktop client) to execute locally, instead of running in a server-managed sandbox. Mirrors Interaction.local_environment and is mutually exclusive with `environment`. (Independent of any client-declared function tools, which are always executed on the client regardless of this field.)
},
"memoryConfig": { # Task-level memory settings: which sources memory draws from, and whether memory is active for this task at all. # Optional. Memory settings for this task.
"disableMemory": True or False, # Optional. If true, the agent neither reads existing memories into the prompt nor forms new ones from this task. Memory is on by default, and this can only turn it off, never back on: * Agent memory enabled, this true: off for this task. * Agent memory enabled, this false or unset: on. * Agent memory disabled: off, whatever this says. Because a task belongs to a single end user's interaction, a surface that knows the user can use this to turn memory off for one user of a shared agent; memory keeps working for the agent's other users. `Agent.memory_config.disable_memory` wins because it is the mutable one. It can be turned on after tasks have already been created, and a stored task spec keeps whatever it was created with — so if a spec could re-enable memory, the agent-level switch would be unable to cover the tasks that already exist. The two are therefore OR'ed, and the agent's current value is re-applied every time a session opens, including on a resume. Scoped to this task. Memories already stored for the agent are retained and stay readable by the agent's users; this governs only whether this task can access and modify them.
"knowledgeCatalogConfig": { # Configuration for enterprise Knowledge Catalog metadata discovery. # Optional. Configuration for enterprise Knowledge Catalog search.
"enabledConnectors": [ # Optional. Filter list of system/connector technology names (e.g., "bigquery", "looker", "spanner"). If empty, all indexed catalog connectors are searched.
"A String",
],
"knowledgeCatalogEnabled": True or False, # Optional. If true, enables searching enterprise metadata in Google Cloud Knowledge Catalog.
"projectId": "A String", # Optional. GCP project ID to scope Knowledge Catalog searches.
},
},
},
"subtaskConfig": { # Configuration and limits for subtasks spawned by this task. # Optional. Configuration and limits for subtasks spawned by this task.
"maxDepth": 42, # Optional. The maximum depth of subtask hierarchy allowed under this task. Must be between 0 and 5, inclusive. When unset on a root task, defaults to 5. For a subtask, it is clamped to at most parent.max_depth - 1 (defaulting to parent.max_depth - 1 if unspecified). A value of 0 means this task cannot spawn any subtasks.
},
"systemInstruction": "A String", # Optional. Optional standing instruction applied across the whole task.
},
"status": "A String", # Output only. The coarse, durable status of the task.
"statusDetail": { # Why a task is in its current status, when the status alone is not enough to act on (today: `SUSPENDED`). Clients MUST treat an unrecognized `StatusReason` as an opaque block. Copied to the DB mirror by name and number, so a field or enum value added here without the matching DB-side addition is silently dropped on write. LINT.IfChange(status_detail) # Output only. Explanation for the current state, when the state alone is not enough to act on (today: `SUSPENDED`). Cleared when the task leaves that state.
"detail": "A String", # Optional human-readable explanation, so a surface can render something without its own StatusReason -> string table. Not localized; clients needing localization should render from `reason` instead. Not a `google.rpc.Status`: RPC codes do not map onto lifecycle states, and `TaskOutcome.status` already covers terminal failures.
"reason": "A String", # Machine-readable reason. Clients switch on this to pick a recovery affordance (upsell CTA, admin contact, retry).
},
"statusUpdateTime": "A String", # Output only. The time the task's status was last updated.
"summary": "A String", # Output only. A rolling, system-generated summary of the task's progress/result.
"title": "A String", # Optional. Optional human-readable title for listing surfaces.
"updateTime": "A String", # Output only. The time the task was last updated.
}
list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
Lists tasks under a parent. Supports filtering (e.g. by status or assigned_agent) so a surface can show "my tasks".
Args:
parent: string, Required. The parent to list tasks under. Both forms are accepted and scope the listing differently: * projects/{project}/locations/{location}/agents/{agent} -- tasks assigned to that agent. * projects/{project}/locations/{location} -- all tasks in the location. (required)
filter: string, Optional. A filter expression, e.g. "status=WORKING". Lineage filters pick how much of a task tree to fetch: * "parent_task_id=" -- direct children only (progressive disclosure, one level per call). * "root_task_id=" -- the whole tree in one flat collection. Label filters support filtering by task labels: * "labels.=" -- match tasks with the given label key and value. Multiple label filters can be combined with AND. Multiple values for the same key can be combined with OR, e.g., "labels.foo=bar OR labels.foo=baz". Status filters restrict the listing to tasks in the given statuses: * "status=" -- e.g. "status=WORKING". Multiple statuses can be combined with OR, e.g., "status=WORKING OR status=IDLE". Both page normally via `page_size`/`page_token`, and both are ordered by `order_by` (default "create_time desc"), not by tree shape. A whole-tree page is therefore not a subtree: a child can arrive before its parent, so the client buffers by `parent_task_id` and renders as parents resolve.
orderBy: string, Optional. Sort order, e.g. "create_time desc". The sortable fields are `create_time` and `update_time`. Defaults to "create_time desc".
pageSize: integer, Optional. The maximum number of tasks to return. The service may return fewer. If unspecified (or <= 0), at most 50 are returned. The maximum is 100; larger values are capped to 100.
pageToken: string, Optional. A page token from a previous ListTasks call.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # Response for TaskService.ListTasks.
"nextPageToken": "A String", # A token to retrieve the next page; empty if there are no more pages.
"tasks": [ # The tasks from the requested collection.
{ # A durable, addressable unit of work: scoped context + a goal with verifiers, assigned to an agent, and executed as a trajectory of steps. A Task is the resource users open, list, and manage. It is a light "header" (status, title, summary, outcome). The executed trajectory (steps) is not stored inline -- it is streamed live over CreateTaskBidiStream (fetched from the runtime), so a long-running/re-firing task carries no unbounded blob. Resource name (canonical): projects/{project}/locations/{location}/agents/{agent}/tasks/{task} Task is a sub-resource of Agent. A short alias projects/{project}/locations/{location}/tasks/{task} is also accepted (dual-path); the task id is unique within a project+location, so both forms resolve the same task. The short form is retained intentionally : * GetTask/ListTasks support both (dual) resource paths. * Subtasks store the short form in parent_task_id, so that reassignment of the parent task to another agent later will not require updating the subtasks, and the rendering of the task tree will not imply the agent relationships.
"assignedAgent": "A String", # Output only. The resource name of the `Agent` this task is assigned to, in the format `projects/{project}/locations/{location}/agents/{agent}`. It is derived from the create parent; any value set by the client is ignored.
"assignedAgentIdentity": "A String", # Output only. The resolved Agent Identity (SPIFFE ID) of the `Agent` this task is assigned to. Populated on single-task RPCs (`CreateTask`, `GetTask`, `UpdateTask`, `SuspendTask`, `ResumeTask`, `CancelTask`); not populated by `ListTasks`.
"budgetConsumptionDetail": { # Observability snapshot of budget consumption for a Task . # Output only. Current budget consumption detail of the Task.
"attributions": [ # Output only. Spend broken down by consumer.
{ # Per-consumer attribution: which unit of the Task tree incurred the spend.
"root": { # The root (owning) Task's own spend. # Output only. The root Task's own spend.
},
"spent": { # A consumer's spend, per Budget dimension. # Output only. This consumer's spend, per dimension.
"tokenCount": "A String", # Output only. Tokens spent by this consumer.
"toolCallCount": "A String", # Output only. Tool calls spent by this consumer.
},
"subTask": { # A spawned sub-Task's spend. # Output only. A spawned sub-Task's spend.
"task": "A String", # Output only. Resource name of the sub-Task that incurred the spend.
},
},
],
"tokenConsumption": { # Consumption for one budget dimension. # Output only. Token budget status. This is only populated if the Task has a token limit Budget set.
"limit": "A String", # Output only. The cap from `TaskSpec.budget` for this dimension's consumption.
"spent": "A String", # Output only. Amount consumed so far.
},
"toolCallConsumption": { # Consumption for one budget dimension. # Output only. Tool-call budget status. This is only populated if the Task has a tool call Budget limit set.
"limit": "A String", # Output only. The cap from `TaskSpec.budget` for this dimension's consumption.
"spent": "A String", # Output only. Amount consumed so far.
},
},
"createTime": "A String", # Output only. The time the task was created.
"goals": [ # Output only. Goals of the Task, including both active and completed/failed. A Task may pursue several concurrently active goals .
{ # A goal is an objective a Task works toward. A Task may pursue several concurrent goals . A goal could have one or more Verifiers as independent judges to validate the completion of it.
"displayName": "A String", # Optional. A human-readable title specifiable to the goal.
"id": "A String", # Output only. Server-assigned identifier, stable across the goal's lifetime.
"spec": "A String", # Required. The goal's objective. This is the "goal" and what the agent should pursue.
"state": "A String", # Output only. The goal's state.
"verifiers": [ # Optional. The verifier(s) that judge whether this goal is achieved. When all `required` verifiers pass, the goal will be marked `SUCCEEDED` in its `state`. If empty, the work is still checked against criteria written from the goal's `spec`, but the goal stays `ACTIVE`; add a required `auto_verification` to have it complete.
{ # A declarative success criterion for a goal. Harness-agnostic: the runtime follows the description to decide pass/fail and to repair on failure. A criterion may be checked automatically (`auto_verification`, `nl_criteria`, `command_check`) or by a human (`human_check`, resolved via a HITL elicitation at verification time). If `kind` is unset the Verifier carries no criterion and is ignored. On a Task with a local environment, verifiers do not access the environment: `nl_criteria` and `auto_verification` are judged on the record of the agent's work in the current turn, and `command_check` is not allowed.
"autoVerification": { # Judges the goal against its own `spec`, with no separate criterion to write. List it more than once to run that many independent judgements of the same objective; as with any verifier, every `required` one must pass for the goal to succeed. # Automated. Judged against the goal's own `spec`.
},
"name": "A String", # Optional. Optional label used for reporting (e.g. "tests-green").
"nlCriteria": { # An automated natural-language criterion evaluated by an autorater/agent. Wrapped in a message (rather than a bare string) so evaluation parameters (e.g. model, temperature, language) can be added without a breaking change. # Automated. A customizable natural-language rubric an autorater/agent evaluates.
"criteria": "A String", # Required. The rubric to evaluate (e.g. "Every unread high-priority thread has a summary and a draft").
},
"required": True or False, # Optional. If true, the goal cannot be marked COMPLETED unless this verifier passes. If false, the verifier is advisory (recorded, non-blocking).
"state": "A String", # Output only. The verifier's latest verdict.
},
],
},
],
"labels": { # Optional. Optional caller-supplied labels. Must follow the unified cloud labels spec (keys/values lowercase `[a-z0-9_-]`, <=63 chars). Stored as-is and returned verbatim on Get/ListTask, so the caller owns what goes in and reads back. Label names starting with `goog-` or `google-` are reserved for predefined labels. Besides user-defined billing/reporting labels, callers may stamp non-PII operational traffic annotations here under the reserved prefix (persisted on the task, so they apply to every turn and survive reconnection). Predefined: `goog-golden-query-id`: stable golden-query / CUJ id (e.g. `hello-world`); absent for organic traffic. Lets per-CUJ latency be trended independently.
"a_key": "A String",
},
"name": "A String", # Identifier. Resource name of the task.
"nextRunTime": "A String", # Output only. Next scheduled firing of a recurring template. Unset once the template is `SCHEDULE_FINISHED`, and for cron-less (event) schedules.
"parentTaskId": "A String", # Optional. If this task is a subtask, the parent task's short-form, agent-independent resource name: projects/{project}/locations/{location}/tasks/{task} The short form is used here on purpose (NOT the agent-scoped .../agents/{agent}/tasks/{task}): it stays stable when the parent is reassigned to another agent, so subtasks never need rewriting, and the task tree does not imply agent relationships .
"rootTaskId": "A String", # Output only. Lineage group key: the root of this task's tree, materialized from `parent_task_id` (a root task's own short-form name). Same short form, and for the same reason, as `parent_task_id`. One `ListTasks(filter: "root_task_id=")` returns the whole tree as a flat, paginated collection, instead of a level-by-level walk that costs one round trip per depth. The client reassembles the tree from `parent_task_id`; a page may therefore contain a task whose parent lands on a later page.
"runCount": "A String", # Output only. Runs minted so far by a recurring template, so a client can show progress against `RecurringSchedule.max_run_count`.
"scheduleState": "A String", # Output only. Scheduling lifecycle of a recurring template. Unset unless `spec.schedule.recurring` is set.
"spec": { # The centralized specification of a Task: the scoped context it runs in, when it runs, the resource guardrails, and the goal (with verifiers). # Required. The full specification of what to do, how to run it, and how to know it is done.
"budget": { # Holistic resource caps metered across the entire task. # Optional. Optional holistic budget caps metered across the entire task lifetime.
"maxToolCalls": "A String", # Optional. Maximum total tool calls across the whole task.
"maxTotalTokens": "A String", # Optional. Maximum total model tokens across the whole task.
"maxWallclock": "A String", # Optional. Maximum wall-clock duration for the task. Counts only time spent in `WORKING` or `REQUIRES_ACTION`; `IDLE`, `QUEUED`, and `SUSPENDED` do not accrue.
},
"deltaStreaming": True or False, # Optional. Whether incrementally generated content is delivered as it is produced, rather than once the step is finished. The two modes are exact duals. If false (default), every step arrives once, complete, in TaskServerEvent.step -- so a client that reads only `step` sees the whole conversation. Suited to non-chat callers that consume finished steps. If true, EVERY step instead arrives as a `step_start` / `step_delta`* / `step_stop` run sharing an index, so a chat client sees one uniform lifecycle and never a bare `step`; reassembling a run yields the same step that would otherwise have been delivered. Content the model generates progressively -- assistant text (and the citations on it) and thoughts -- streams natively across the run's `step_delta`s as it is produced. A step that arrives whole (tool calls, tool results, elicitations, user-input echoes, errors) carries no `step_delta`: the complete step rides `step_start` and `step_stop` closes the run, since nothing is generated incrementally. The field is deliberately not named for text: what streams progressively is an implementation detail that may widen, and a client asking for incremental delivery wants it for whatever supports it. Note per-step token usage rides `step_stop`, so it is reported only when this is true.
"expectedOutput": { # Optional. Optional structured deliverable contract (reuses the Interactions ResponseFormat). When set, the agent's final output must conform to it.
"audio": { # Configuration for audio output format.
"bitRate": 42, # Bit rate in bits per second (bps). Only applicable for compressed formats (MP3, Opus).
"delivery": "A String", # The delivery mode for the audio output.
"mimeType": "A String", # The MIME type of the audio output.
"sampleRate": 42, # Sample rate in Hz.
},
"image": { # Configuration for image output format.
"aspectRatio": "A String", # The aspect ratio for the image output.
"delivery": "A String", # The delivery mode for the image output.
"imageSize": "A String", # The size of the image output.
"mimeType": "A String", # The MIME type of the image output.
},
"structValue": { # `Struct` represents a structured data value, consisting of fields which map to dynamically typed values. # Multi-discriminator values is already enabled in GAOS
"fields": [ # Dynamically typed fields. List instead of map because LLMs are sensitive to ordering, and we want to give users full control.
{ # Represents a single field in a struct.
"name": "A String",
"value": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
],
},
"text": { # Configuration for text output format.
"mimeType": "A String", # The MIME type of the text output.
"schema": { # `Struct` represents a structured data value, consisting of fields which map to dynamically typed values. # The JSON schema that the output should conform to. Only applicable when mime_type is application/json. Supports a subset of JSON Schema. `minLength`, `maxLength`, `minProperties`, `maxProperties`, `uniqueItems`, `contains`, `minContains`, and `maxContains` are ignored and not enforced; validate these in your application. `minItems` and `maxItems` are enforced. Schemas that are too complex to enforce (for example, large `maxItems` on nested arrays) return a 400 INVALID_ARGUMENT error.
"fields": [ # Dynamically typed fields. List instead of map because LLMs are sensitive to ordering, and we want to give users full control.
{ # Represents a single field in a struct.
"name": "A String",
"value": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
],
},
},
"video": { # Configuration for video output format.
"aspectRatio": "A String", # The aspect ratio for the video output.
"delivery": "A String", # The delivery mode for the video output.
"duration": "A String", # The duration for the video output.
"gcsUri": "A String", # The Cloud Storage URI to store the video output. Required for Vertex if delivery mode is URI.
"resolution": "A String", # The video output resolution. Defaults to 720p.
},
},
"hitlConfig": { # Session-level HITL (human-in-the-loop) configuration. Namespaces every HITL knob that a client may set at task creation, so new settings (per-tool overrides, timeout, allowed reviewers) can land without burning more top-level TaskSpec tags. `approval_mode` carries the wire default as its own `APPROVAL_MODE_UNSPECIFIED` enum value, which the server treats as `APPROVAL_MODE_ASK` (standard HITL, always prompt a human). That is also the pre-change behavior for every existing task. # Optional. Immutable. Session-level HITL (human-in-the-loop) configuration for this task. Unset means every HITL knob takes its default; see `HitlConfig` above for the individual field defaults.
"approvalMode": "A String", # Optional. Immutable. How the client wants elicitations on this task approved. Unset (or APPROVAL_MODE_UNSPECIFIED) is treated as APPROVAL_MODE_ASK.
},
"inputs": [ # Optional. The prompt expressed as Interaction steps (e.g. a ContentStep with the user's request). Reuses the Interactions step model so the same content flows into the runtime unchanged. Required for a scheduled or PRIORITY_DEFERRED task, which starts with no caller attached. Otherwise it may be empty: * Unary CreateTask with no inputs creates the task IDLE and dispatches nothing. The task starts when a caller subscribes to it on CreateTaskBidiStream (`subscribe_task`) and sends the first input. * CreateTaskBidiStream `create_task` with no inputs is a connect-only / pre-warm create; the first input is sent later as a client step. Recommended pattern for giving a task files before it runs: 1. CreateTask with empty `inputs`. The task is IDLE; nothing runs yet. 2. UploadTaskArtifact each file to `/workspace/userdata/`. 3. CreateTaskBidiStream with `subscribe_task`, then send the first input, referring to the files by their `/workspace/userdata/` paths. This dispatches the task with the uploaded files in its workspace.
{ # A step in the interaction.
"modelOutput": { # Output generated by the model.
"content": [
{ # The content of the response.
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
],
"error": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation.
"code": 42, # The status code, which should be an enum value of google.rpc.Code.
"details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
{
"a_key": "", # Properties of the object. Contains field @type with type URL.
},
],
"message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
},
},
"thought": { # A thought step.
"signature": "A String", # A signature hash for backend validation.
"summary": [ # A summary of the thought.
{ # The content of the response.
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
],
},
"toolCall": { # Tool call step.
"codeExecutionCall": { # Code execution call step.
"arguments": { # The arguments to pass to the code execution. # Required. The arguments to pass to the code execution.
"code": "A String", # The code to be executed.
"language": "A String", # Programming language of the `code`.
},
},
"fileSearchCall": { # File Search call step.
},
"functionCall": { # A function tool call step.
"arguments": { # `Struct` represents a structured data value, consisting of fields which map to dynamically typed values. # Required. The arguments to pass to the function.
"fields": [ # Dynamically typed fields. List instead of map because LLMs are sensitive to ordering, and we want to give users full control.
{ # Represents a single field in a struct.
"name": "A String",
"value": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
],
},
"name": "A String", # Required. The name of the tool to call.
},
"googleMapsCall": { # Google Maps call step.
"arguments": { # The arguments to pass to the Google Maps tool. # The arguments to pass to the Google Maps tool.
"queries": [ # The queries to be executed.
"A String",
],
},
},
"googleSearchCall": { # Google Search call step.
"arguments": { # The arguments to pass to Google Search. # Required. The arguments to pass to Google Search.
"queries": [ # Web search queries for the following-up web search.
"A String",
],
},
"searchType": "A String", # The type of search grounding enabled.
},
"id": "A String", # Required. A unique ID for this specific tool call.
"mcpServerToolCall": { # MCPServer tool call step.
"arguments": { # `Struct` represents a structured data value, consisting of fields which map to dynamically typed values. # Required. The JSON object of arguments for the function.
"fields": [ # Dynamically typed fields. List instead of map because LLMs are sensitive to ordering, and we want to give users full control.
{ # Represents a single field in a struct.
"name": "A String",
"value": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
],
},
"name": "A String", # Required. The name of the tool which was called.
"serverName": "A String", # Required. The name of the used MCP server.
},
"processingCall": { # A server-initiated processing step for media analysis (e.g. video understanding).
},
"retrievalCall": { # Retrieval call step. Used by Vertex Retrieval tools such as Parallel AI, Exa AI, Vertex AI Search, etc. RetrievalType decides which tool is used.
"arguments": { # The arguments to pass to Retrieval tools. # Required. The arguments to pass to the retrieval tool.
"queries": [ # Queries for Retrieval information.
"A String",
],
},
"retrievalType": "A String", # The type of retrieval tools.
},
"signature": "A String", # A signature hash for backend validation.
"urlContextCall": { # URL context call step.
"arguments": { # The arguments to pass to the URL context. # Required. The arguments to pass to the URL context.
"urls": [ # The URLs to fetch.
"A String",
],
},
},
},
"toolResult": { # Tool result step.
"callId": "A String", # Required. ID to match the ID from the function call block.
"codeExecutionResult": { # Code execution result step.
"isError": True or False, # Whether the code execution resulted in an error.
"result": "A String", # Required. The output of the code execution.
},
"fileSearchResult": { # File Search result step.
},
"functionResult": { # Result of a function tool call.
"isError": True or False, # Whether the tool call resulted in an error.
"name": "A String", # The name of the tool that was called.
"result": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error. # Required. The result of the tool call.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
},
"googleMapsResult": { # Google Maps result step.
"result": [
{ # The result of the Google Maps.
"places": [
{
"name": "A String",
"placeId": "A String",
"reviewSnippets": [
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String",
},
],
"widgetContextToken": "A String",
},
],
},
"googleSearchResult": { # Google Search result step.
"isError": True or False, # Whether the Google Search resulted in an error.
"result": [ # Required. The results of the Google Search.
{ # The result of the Google Search.
"searchSuggestions": "A String", # Web content snippet that can be embedded in a web page or an app webview.
},
],
},
"mcpServerToolResult": { # MCPServer tool result step.
"name": "A String", # Name of the tool which is called for this specific tool call.
"result": { # `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error. # Required. The output from the MCP server call. Can be simple text or rich content.
"boolValue": True or False, # Represents a boolean value.
"contentValue": { # The content of the response. # Represents rich content (text, image, etc.).
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
"listValue": { # `ListValue` is a wrapper around a repeated field of values. # Represents a repeated `Value`.
"values": [ # Repeated field of dynamically typed values.
# Object with schema name: GenaiValue
],
},
"nullValue": "A String", # Represents a null value.
"numberValue": 3.14, # Represents a double value.
"stringValue": "A String", # Represents a string value.
"structValue": # Object with schema name: GenaiStruct # Represents a structured value.
},
"serverName": "A String", # The name of the used MCP server.
},
"processingResult": { # The result of a server-initiated media processing step.
},
"retrievalResult": { # Vertex Retrieval result step. Used by Vertex Retrieval tools such as Parallel AI, Exa AI, Vertex AI Search, etc.
"isError": True or False, # Whether the retrieval resulted in an error.
},
"signature": "A String", # A signature hash for backend validation.
"urlContextResult": { # URL context result step.
"isError": True or False, # Whether the URL context resulted in an error.
"result": [ # Required. The results of the URL context.
{ # The result of the URL context.
"status": "A String", # The status of the URL retrieval.
"url": "A String", # The URL that was fetched.
},
],
},
},
"userInput": { # Input provided by the user. # DO NOT USE -- These are for 3P JSON only
"contentList": { # A list of Content. # The content of the step. An array of Content objects.
"contents": [ # The contents of the list.
{ # The content of the response.
"audio": { # An audio content block.
"channels": 42, # The number of audio channels.
"data": "A String", # The audio content.
"mimeTypeString": "A String", # Flexible MIME type string of the audio, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"sampleRate": 42, # The sample rate of the audio.
"uri": "A String", # The URI of the audio.
},
"document": { # A document content block.
"data": "A String", # The document content.
"mimeTypeString": "A String", # Flexible MIME type string of the document, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"uri": "A String", # The URI of the document.
},
"image": { # An image content block.
"data": "A String", # The image content.
"mimeTypeString": "A String", # Flexible MIME type string of the image, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the image.
},
"text": { # A text content block.
"annotations": [ # Citation information for model-generated content.
{ # Citation information for model-generated content.
"endIndex": 42, # End of the attributed segment, exclusive.
"fileCitation": { # A file citation annotation. # A file citation annotation.
"customMetadata": { # User provided metadata about the retrieved context.
"a_key": "", # Properties of the object.
},
"documentUri": "A String", # The URI of the file.
"fileName": "A String", # The name of the file.
"mediaId": "A String", # Media ID in-case of image citations, if applicable.
"pageNumber": 42, # Page number of the cited document, if applicable.
"source": "A String", # Source attributed for a portion of the text.
},
"placeCitation": { # A place citation annotation. # A place citation annotation.
"name": "A String", # Title of the place.
"placeId": "A String", # The ID of the place, in `places/{place_id}` format.
"reviewSnippets": [ # Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
{ # Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
"reviewId": "A String", # The ID of the review snippet.
"title": "A String", # Title of the review.
"url": "A String", # A link that corresponds to the user review on Google Maps.
},
],
"url": "A String", # URI reference of the place.
},
"speechMetadata": { # Speech annotation for text content. # Speech annotation for text content.
"speaker": "A String", # The speaker to associate with this turn.
"style": "A String", # Style instruction for the speech synthesis.
},
"startIndex": 42, # Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
"urlCitation": { # A URL citation annotation. # NOTE: We use these instead of the Citation message for historical reasons. A URL citation annotation.
"title": "A String", # The title of the URL.
"url": "A String", # The URL.
},
"wordInfo": { # Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution. # Word-level ASR annotation with timing and speaker info.
"endOffset": "A String", # End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"speaker": "A String", # Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig.
"startOffset": "A String", # Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word".
"text": "A String", # The transcribed word.
},
},
],
"text": "A String", # Required. The text content.
},
"video": { # A video content block.
"data": "A String", # The video content.
"mimeTypeString": "A String", # Flexible MIME type string of the video, superseding mime_type = 1. Note: Bespoke logic in the GAOS parser/serializer maps this to the "mime_type" JSON key.
"name": "A String", # A user-defined name for this content block. Can be referenced by the model in the final response.
"processingConfig": {
"static": {
"endOffset": "A String", # Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set.
"fps": 3.14, # Optional. Video frame-rate sampling density.
"startOffset": "A String", # Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative.
},
},
"processingType": "A String",
"resolution": "A String", # The resolution of the media.
"uri": "A String", # The URI of the video.
},
},
],
},
"contentString": "A String", # The content of the step. A single string.
},
},
],
"maxRetries": 42, # Optional. Deprecated: tasks are not retried automatically. CreateTask and UpdateTask reject a non-zero value with INVALID_ARGUMENT.
"modelConfig": { # Selects the model the task runs on. Starts with just the model id/alias; extensible with further model parameters without a breaking change. # Optional. Optional model configuration override for the task. When set, the task runs on this model configuration for its entire lifecycle.
"effortLevel": "A String", # Optional. Reasoning/thinking depth the model runs at, case-insensitive. Gemini takes "minimal", "low", "medium" or "high", and only on models that have a thinking level; other publishers define their own set. A value the resolved model does not accept leaves its own default standing.
"maxOutputTokens": 42, # Optional. Maximum number of tokens the model may generate in each call, including any thinking tokens it spends before answering. Must not be negative; zero leaves the backend default standing. The cap also applies to the calls that summarize the task's history when its context is compacted, so a model that thinks at a high effort level may need a larger value to return a summary at all.
"model": "A String", # Optional. Publisher model resource name (an `aiplatform.googleapis.com/PublisherModel`) of the form "publishers/{publisher}/models/{model}", e.g. "publishers/anthropic/models/claude-opus-4-8" or "publishers/google/models/gemini-3-flash". The project-qualified form "projects/{project}/locations/{location}/publishers/{publisher}/models/{model}" is also accepted. When it is used, the project and location it carries take precedence over both dynamic project/location resolution and the actor config defaults. Model selection cannot be updated after task creation right now though it might be supported in the future. Empty means the actor's default gemini model.
"modelSelectionMode": "A String", # Optional. How the model in `model` was selected. Ignored when set on `LiveConfig.model_config`.
"temperature": 3.14, # Optional. Sampling temperature for the model's generation config (valid range (0.0, 2.0]); a value outside that range leaves the backend default standing.
},
"priority": "A String", # Optional. Optional priority band. Concrete tiers and quota are out of scope and deferred to a follow-up.
"schedule": { # Defines when and how a task runs. # Optional. Optional schedule controlling when/how the task runs. If unset, the task runs immediately (one-off, now).
"oneOff": { # A one-off (single) run trigger. # Run exactly once (at an absolute time or after a delay).
"delay": "A String", # Relative delay before the single run.
"runAt": "A String", # Absolute time to run at (e.g. 8 PM today).
},
"recurring": { # Defines a recurring schedule logic for repeated task executions. # Run the task spec repeatedly.
"cron": { # A recurring cron trigger. # The cron expression, e.g. "0 8 * * MON-FRI".
"expression": "A String", # Required. The cron expression, e.g. "0 8 * * MON-FRI". Any field may also be `?` or `?/step` to let the service pick the value, so that schedules do not all fire at the same moment: `?` picks one value in the field's range, and `?/step` picks a start within the first step and repeats every step from there. The pick is made once, when the schedule is created or updated, and the expression is stored with the picked values in place of `?` (e.g. "?/30 * * * *" is stored as "17-59/30 * * * *"); sending a `?` again picks new values.
"timeZone": "A String", # Optional. Optional IANA time zone name, e.g. "America/Los_Angeles". Defaults to UTC.
},
"endTime": "A String", # Optional. Do not fire after this time.
"eventSource": { # An event-driven trigger. # Identifier of a registered event source / webhook subscription whose events fire the task.
"eventSource": "A String", # Required. Identifier of a registered event source / webhook subscription whose events fire the task.
},
"maxRunCount": "A String", # Optional. Max total number of firings. Unset means unbounded.
"overlapPolicy": "A String", # Optional. What to do when a firing is due while a prior run is still active.
"startTime": "A String", # Optional. Do not fire before this time.
},
},
"scope": { # The scoped context a task runs within. # Optional. Optional scoped context: skills/tools/memory/artifacts for the task.
"allowedTools": { # A set of tool references, each a tool's GCP resource name (`projects/.../mcpServers|endpoints/...`) with optional fine-grained method, CLI tool and resource guardrails. Wraps a repeated field so it can be used as an arm of the `ContextScope.tool_scope` oneof. LINT.IfChange(tool_refs) # Keep only these tools from the agent's inherited set.
"cliToolScope": { # A CLI tool boundary restricting a subtask to designated CLI commands. # Optional. Optional CLI tool scope that the task is restricted to. Unset means all CLI tools are allowed. Set with an empty `cli_tools` list means all CLI tools are denied.
"cliTools": [ # Optional. Allowed CLI tools for the task. Example: ["gmail send_message", "calendar list_events"].
"A String",
],
},
"methods": [ # Optional. Optional fine-grained method selectors for the referenced tools. Format: "/methods/".
"A String",
],
"resources": [ # Optional. Optional resource identifiers that the task is restricted to interact with.
{ # A specific resource boundary restricting an agent's access to designated entities.
"recipients": [ # Optional. Optional recipients email boundaries for communication tools. Example: ["alice@gmail.com", "bob@google.com"].
"A String",
],
"resource": "A String", # Optional. The specific resource identifier (e.g. thread_id, doc_id, repo name). Examples: * GMAIL_THREAD: "18df123456789abc" * GOOGLE_CHAT_THREAD: "spaces/AAA/threads/BBB" * GITHUB_REPOSITORY: "GoogleCloudPlatform/kubectl"
"resourceType": "A String", # Required. The type of resource being scoped.
"selfCreated": True or False, # Optional. For cold outreach / outbound threads when resource_id doesn't exist. Grants access ONLY to resources created by this task instance.
},
],
"toolNames": [ # Optional. Tool GCP resource names. For example: * MCP server: `projects/my-project/locations/us-central1/mcpServers/jira` * Endpoint: `projects/my-project/locations/us-central1/endpoints/drive`
"A String",
],
},
"collaborativeProject": { # Scopes a task to a `CollaborativeProject` . Starts with the collaborative project resource reference; extensible with per-task collaborative project settings without a breaking change. # Optional. Optional collaborative project configuration for this task. Only a root task (`parent_task_id` unset) may set or update this field; when `parent_task_id` is set, the subtask automatically inherits its root task's collaborative project (setting a different value is rejected with `INVALID_ARGUMENT`). Updating or clearing this field via `UpdateTask` is only allowed while the task is in the `IDLE` state (attempts in any other state are rejected with `FAILED_PRECONDITION`).
"collaborativeProject": "A String", # Optional. The resource name of the `CollaborativeProject` this task is scoped to: `projects/{project}/locations/{location}/collaborativeProjects/{collaborative_project}` The `{project}` segment must match the task's own Google Cloud project. Empty when the task is not scoped to a collaborative project. Only a root task (`parent_task_id` unset) may set or update this field; when `parent_task_id` is set, the subtask automatically inherits its root task's collaborative project (setting a different value is rejected with `INVALID_ARGUMENT`). Updating or clearing this field via `UpdateTask` is only allowed while the task is in the `IDLE` state (attempts in any other state are rejected with `FAILED_PRECONDITION`).
},
"dataConnectorConfig": { # Configuration for Gemini Enterprise MCP data connectors scoped to this task. # Optional. Data connector configuration for this task.
"dataConnectors": { # Optional. Map of active connector short name / data_source (e.g., "jira", "servicenow") to its full DataConnector resource name (e.g., "projects/{project}/locations/{location}/collections/{collection}/dataConnector"). If multiple connectors share a data_source, the first configured connector for that data_source is retained.
"a_key": "A String",
},
"engineResourceName": "A String", # Optional. Full resource name of the Discovery Engine Engine (e.g., "projects/{project}/locations/{location}/collections/{collection}/engines/{engine}").
},
"environment": { # Configuration for a custom environment. # Optional. Optional server-managed sandbox environment for the task: skills and tools mounted into the sandbox plus the egress policy. Reuses the Interactions EnvironmentConfig (e.g. SKILL_REGISTRY sources to mount skills, network egress allowlist).
"env": { # Environment variables to set in the sandbox environment.
"a_key": { # An environment variable to set in the execution environment.
"credential": "A String", # Optional reference to a server-managed Credential resource by ID.
"value": "A String", # Direct string value for plain environment variables.
},
},
"environmentId": "A String", # Optional. The environment ID for the interaction. If specified, the request will update the existing environment instead of creating a new one.
"memoryMounts": [ # Optional. Memory stores mounted into the environment under `/memory/`. When merged with an agent or base environment configuration, non-conflicting mounts are preserved and request mounts take precedence on store ID or dir_name collisions.
{ # A memory store mounted into the environment under `/memory/`.
"description": "A String", # Optional. Per-mount guidance describing when the agent should read or write this memory store.
"dirName": "A String", # Optional. Directory name under `/memory//`. When omitted, defaults to the store's `memory_store_id`.
"memoryStore": "A String", # Required. Resource name or ID of the MemoryStore (e.g. `memoryStores/my-store` or `my-store`).
"mode": "A String", # Optional. Access mode granted to the environment for this store. Defaults to `READ_ONLY` when unspecified.
},
],
"networkAllowlist": { # Network egress configuration for the environment. # Allow only specific domains.
"allowlist": [ # List of allowed domains and their configurations. Set to `"disabled"` to block all network egress.
{ # A single domain allowlist rule with optional header injection.
"credential": "A String", # Optional. Reference to a server-managed Credential resource by ID.
"domain": "A String", # Domain to allow outbound requests to. Supports wildcards (e.g. '*.googleapis.com'). Use '*' to allow all domains.
"mode": "A String", # Network egress mode. Set via the `"allowlist": "disabled"` short form; must be the only rule in the allowlist and cannot be combined with `domain`, `transform` or `credential`.
"transform": { # Headers to inject into requests matching this rule. Key: header name (e.g., "Authorization"). Value: header value (e.g., "Bearer your-token").
"a_key": "A String",
},
},
],
},
"networkMode": "A String", # Network egress mode.
"sources": [
{ # A source to be mounted into the environment.
"content": "A String", # The inline content if `type` is `INLINE`.
"encoding": "A String", # Optional encoding for inline content (e.g. `base64`).
"source": "A String", # The source of the environment. For Cloud Storage, this is the Cloud Storage path. For GitHub, this is the GitHub path.
"target": "A String", # Where the source should appear in the environment.
"type": "A String",
},
],
},
"excludedTools": { # A set of tool references, each a tool's GCP resource name (`projects/.../mcpServers|endpoints/...`) with optional fine-grained method, CLI tool and resource guardrails. Wraps a repeated field so it can be used as an arm of the `ContextScope.tool_scope` oneof. LINT.IfChange(tool_refs) # Remove these tools from the agent's inherited set.
"cliToolScope": { # A CLI tool boundary restricting a subtask to designated CLI commands. # Optional. Optional CLI tool scope that the task is restricted to. Unset means all CLI tools are allowed. Set with an empty `cli_tools` list means all CLI tools are denied.
"cliTools": [ # Optional. Allowed CLI tools for the task. Example: ["gmail send_message", "calendar list_events"].
"A String",
],
},
"methods": [ # Optional. Optional fine-grained method selectors for the referenced tools. Format: "/methods/".
"A String",
],
"resources": [ # Optional. Optional resource identifiers that the task is restricted to interact with.
{ # A specific resource boundary restricting an agent's access to designated entities.
"recipients": [ # Optional. Optional recipients email boundaries for communication tools. Example: ["alice@gmail.com", "bob@google.com"].
"A String",
],
"resource": "A String", # Optional. The specific resource identifier (e.g. thread_id, doc_id, repo name). Examples: * GMAIL_THREAD: "18df123456789abc" * GOOGLE_CHAT_THREAD: "spaces/AAA/threads/BBB" * GITHUB_REPOSITORY: "GoogleCloudPlatform/kubectl"
"resourceType": "A String", # Required. The type of resource being scoped.
"selfCreated": True or False, # Optional. For cold outreach / outbound threads when resource_id doesn't exist. Grants access ONLY to resources created by this task instance.
},
],
"toolNames": [ # Optional. Tool GCP resource names. For example: * MCP server: `projects/my-project/locations/us-central1/mcpServers/jira` * Endpoint: `projects/my-project/locations/us-central1/endpoints/drive`
"A String",
],
},
"externalBrowser": { # A browser on the connecting client (e.g. the user's own, signed-in Chrome) that a task's browser actions run on. The actions are relayed to the client, so the client must stay connected while they happen. The task's other operations (files, commands) still run where `environment_config` puts them. # Optional. Optional browser on the connecting client that the task's browser actions run on, instead of a browser in a server-managed sandbox. Supported with `environment`, `no_environment`, or no `environment_config`.
},
"externalMount": { # A directory on the connecting client (e.g. the user's local machine) mounted into a task's server-managed environment. # Optional. Optional directory on the connecting client mounted into the task's server-managed environment. File operations under its `mount_path` are relayed to the client. Supported only when `environment` is set.
"mountPath": "A String", # Optional. Absolute path in the environment where the directory is mounted. Empty means `/workspace/desktop`. When set, must be `/workspace/{name}`, where `{name}` is 1-63 lowercase letters, digits, `_` or `-`, starts with a letter or digit, and is not one of the names other mounts own under `/workspace`: `skills`, `core`, `memory`, `inputs`, `artifacts`, `deliverables` or `userdata`.
"sourcePath": "A String", # Optional. Absolute path of the folder on the connecting client (the mount source). Interpreted only by the client; the server never resolves it.
},
"localEnvironment": { # Configuration for an environment that lives on the client connection rather than in a server-managed sandbox. When set (via Interaction.local_environment), the agent's filesystem and shell are treated as living on the client: the agent's built-in environment operations (e.g. reading/listing/editing files and running commands) are suspended on the server and yielded back to the client to execute, with their results returned on a subsequent turn. This is mutually exclusive with a server-managed `EnvironmentConfig` (remote_environment), since the environment is either on the client or in a server sandbox, never both. This governs only the agent's built-in environment. Client-declared function tools are always executed on the client regardless of this field. # Optional. Optional client-side ("local") environment: the agent's built-in filesystem/shell operations are yielded to the connecting client (e.g. the gogo desktop client) to execute locally, instead of running in a server-managed sandbox. Mirrors Interaction.local_environment and is mutually exclusive with `environment`. (Independent of any client-declared function tools, which are always executed on the client regardless of this field.)
},
"memoryConfig": { # Task-level memory settings: which sources memory draws from, and whether memory is active for this task at all. # Optional. Memory settings for this task.
"disableMemory": True or False, # Optional. If true, the agent neither reads existing memories into the prompt nor forms new ones from this task. Memory is on by default, and this can only turn it off, never back on: * Agent memory enabled, this true: off for this task. * Agent memory enabled, this false or unset: on. * Agent memory disabled: off, whatever this says. Because a task belongs to a single end user's interaction, a surface that knows the user can use this to turn memory off for one user of a shared agent; memory keeps working for the agent's other users. `Agent.memory_config.disable_memory` wins because it is the mutable one. It can be turned on after tasks have already been created, and a stored task spec keeps whatever it was created with — so if a spec could re-enable memory, the agent-level switch would be unable to cover the tasks that already exist. The two are therefore OR'ed, and the agent's current value is re-applied every time a session opens, including on a resume. Scoped to this task. Memories already stored for the agent are retained and stay readable by the agent's users; this governs only whether this task can access and modify them.
"knowledgeCatalogConfig": { # Configuration for enterprise Knowledge Catalog metadata discovery. # Optional. Configuration for enterprise Knowledge Catalog search.
"enabledConnectors": [ # Optional. Filter list of system/connector technology names (e.g., "bigquery", "looker", "spanner"). If empty, all indexed catalog connectors are searched.
"A String",
],
"knowledgeCatalogEnabled": True or False, # Optional. If true, enables searching enterprise metadata in Google Cloud Knowledge Catalog.
"projectId": "A String", # Optional. GCP project ID to scope Knowledge Catalog searches.
},
},
},
"subtaskConfig": { # Configuration and limits for subtasks spawned by this task. # Optional. Configuration and limits for subtasks spawned by this task.
"maxDepth": 42, # Optional. The maximum depth of subtask hierarchy allowed under this task. Must be between 0 and 5, inclusive. When unset on a root task, defaults to 5. For a subtask, it is clamped to at most parent.max_depth - 1 (defaulting to parent.max_depth - 1 if unspecified). A value of 0 means this task cannot spawn any subtasks.
},
"systemInstruction": "A String", # Optional. Optional standing instruction applied across the whole task.
},
"status": "A String", # Output only. The coarse, durable status of the task.
"statusDetail": { # Why a task is in its current status, when the status alone is not enough to act on (today: `SUSPENDED`). Clients MUST treat an unrecognized `StatusReason` as an opaque block. Copied to the DB mirror by name and number, so a field or enum value added here without the matching DB-side addition is silently dropped on write. LINT.IfChange(status_detail) # Output only. Explanation for the current state, when the state alone is not enough to act on (today: `SUSPENDED`). Cleared when the task leaves that state.
"detail": "A String", # Optional human-readable explanation, so a surface can render something without its own StatusReason -> string table. Not localized; clients needing localization should render from `reason` instead. Not a `google.rpc.Status`: RPC codes do not map onto lifecycle states, and `TaskOutcome.status` already covers terminal failures.
"reason": "A String", # Machine-readable reason. Clients switch on this to pick a recovery affordance (upsell CTA, admin contact, retry).
},
"statusUpdateTime": "A String", # Output only. The time the task's status was last updated.
"summary": "A String", # Output only. A rolling, system-generated summary of the task's progress/result.
"title": "A String", # Optional. Optional human-readable title for listing surfaces.
"updateTime": "A String", # Output only. The time the task was last updated.
},
],
}
list_next(previous_request, previous_response)
Retrieves the next page of results. Args: previous_request: The request for the previous page. (required) previous_response: The response from the request for the previous page. (required) Returns: A request object that you can call 'execute()' on to request the next page. Returns None if there are no more items in the collection.