countTokens(endpoint, body=None, x__xgafv=None)

Perform a token counting.

Args:
  endpoint: string, Required. The name of the Endpoint requested to perform token counting. Format: `projects/{project}/locations/{location}/endpoints/{endpoint}` (required)
  body: object, The request body.
    The object takes the form of:

{ # Request message for PredictionService.CountTokens.
  "contents": [ # Optional. Input content.
    { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
      "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
        { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
          "fileData": { # URI based data. # Optional. URI based data.
            "fileUri": "A String", # Required. URI.
            "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
          },
          "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
            "args": { # Optional. Required. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
              "a_key": "", # Properties of the object.
            },
            "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
          },
          "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
            "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
            "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
              "a_key": "", # Properties of the object.
            },
          },
          "inlineData": { # Content blob. It's preferred to send as text directly rather than raw bytes. # Optional. Inlined bytes data.
            "data": "A String", # Required. Raw bytes.
            "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
          },
          "text": "A String", # Optional. Text part (can be code).
          "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
            "endOffset": "A String", # Optional. The end offset of the video.
            "startOffset": "A String", # Optional. The start offset of the video.
          },
        },
      ],
      "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
    },
  ],
  "instances": [ # Optional. The instances that are the input to token counting call. Schema is identical to the prediction schema of the underlying model.
    "",
  ],
  "model": "A String", # Optional. The name of the publisher model requested to serve the prediction. Format: `projects/{project}/locations/{location}/publishers/*/models/*`
  "systemInstruction": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn. # Optional. The user provided system instructions for the model. Note: only text should be used in parts and content in each part will be in a separate paragraph.
    "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
      { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
        "fileData": { # URI based data. # Optional. URI based data.
          "fileUri": "A String", # Required. URI.
          "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
        },
        "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
          "args": { # Optional. Required. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
            "a_key": "", # Properties of the object.
          },
          "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
        },
        "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
          "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
          "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
            "a_key": "", # Properties of the object.
          },
        },
        "inlineData": { # Content blob. It's preferred to send as text directly rather than raw bytes. # Optional. Inlined bytes data.
          "data": "A String", # Required. Raw bytes.
          "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
        },
        "text": "A String", # Optional. Text part (can be code).
        "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
          "endOffset": "A String", # Optional. The end offset of the video.
          "startOffset": "A String", # Optional. The start offset of the video.
        },
      },
    ],
    "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
  },
  "tools": [ # Optional. A list of `Tools` the model may use to generate the next response. A `Tool` is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model.
    { # Tool details that the model may use to generate response. A `Tool` is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. A Tool object should contain exactly one type of Tool (e.g FunctionDeclaration, Retrieval or GoogleSearchRetrieval).
      "functionDeclarations": [ # Optional. Function tool type. One or more function declarations to be passed to the model along with the current user query. Model may decide to call a subset of these functions by populating FunctionCall in the response. User should provide a FunctionResponse for each function call in the next turn. Based on the function responses, Model will generate the final response back to the user. Maximum 64 function declarations can be provided.
        { # Structured representation of a function declaration as defined by the [OpenAPI 3.0 specification](https://spec.openapis.org/oas/v3.0.3). Included in this declaration are the function name and parameters. This FunctionDeclaration is a representation of a block of code that can be used as a `Tool` by the model and executed by the client.
          "description": "A String", # Optional. Description and purpose of the function. Model uses it to decide how and whether to call the function.
          "name": "A String", # Required. The name of the function to call. Must start with a letter or an underscore. Must be a-z, A-Z, 0-9, or contain underscores, dots and dashes, with a maximum length of 64.
          "parameters": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. Describes the parameters to this function in JSON Schema Object format. Reflects the Open API 3.03 Parameter Object. string Key: the name of the parameter. Parameter names are case sensitive. Schema Value: the Schema defining the type used for the parameter. For function with no parameters, this can be left unset. Parameter names must start with a letter or an underscore and must only contain chars a-z, A-Z, 0-9, or underscores with a maximum length of 64. Example with 1 required and 1 optional parameter: type: OBJECT properties: param1: type: STRING param2: type: INTEGER required: - param1
            "default": "", # Optional. Default value of the data.
            "description": "A String", # Optional. The description of the data.
            "enum": [ # Optional. Possible values of the element of Type.STRING with enum format. For example we can define an Enum Direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]}
              "A String",
            ],
            "example": "", # Optional. Example of the object. Will only populated when the object is the root.
            "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
            "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
            "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
            "maxLength": "A String", # Optional. Maximum length of the Type.STRING
            "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
            "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
            "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
            "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
            "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
            "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
            "nullable": True or False, # Optional. Indicates if the value may be null.
            "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
            "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
              "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
            },
            "required": [ # Optional. Required properties of Type.OBJECT.
              "A String",
            ],
            "title": "A String", # Optional. The title of the Schema.
            "type": "A String", # Optional. The type of the data.
          },
          "response": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. Describes the output from this function in JSON Schema format. Reflects the Open API 3.03 Response Object. The Schema defines the type used for the response value of the function.
            "default": "", # Optional. Default value of the data.
            "description": "A String", # Optional. The description of the data.
            "enum": [ # Optional. Possible values of the element of Type.STRING with enum format. For example we can define an Enum Direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]}
              "A String",
            ],
            "example": "", # Optional. Example of the object. Will only populated when the object is the root.
            "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
            "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
            "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
            "maxLength": "A String", # Optional. Maximum length of the Type.STRING
            "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
            "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
            "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
            "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
            "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
            "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
            "nullable": True or False, # Optional. Indicates if the value may be null.
            "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
            "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
              "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
            },
            "required": [ # Optional. Required properties of Type.OBJECT.
              "A String",
            ],
            "title": "A String", # Optional. The title of the Schema.
            "type": "A String", # Optional. The type of the data.
          },
        },
      ],
      "googleSearchRetrieval": { # Tool to retrieve public web data for grounding, powered by Google. # Optional. GoogleSearchRetrieval tool type. Specialized retrieval tool that is powered by Google search.
      },
      "retrieval": { # Defines a retrieval tool that model can call to access external knowledge. # Optional. Retrieval tool type. System will always execute the provided retrieval tool(s) to get external knowledge to answer the prompt. Retrieval results are presented to the model for generation.
        "disableAttribution": True or False, # Optional. Deprecated. This option is no longer supported.
        "vertexAiSearch": { # Retrieve from Vertex AI Search datastore for grounding. See https://cloud.google.com/products/agent-builder # Set to use data source powered by Vertex AI Search.
          "datastore": "A String", # Required. Fully-qualified Vertex AI Search data store resource ID. Format: `projects/{project}/locations/{location}/collections/{collection}/dataStores/{dataStore}`
        },
        "vertexRagStore": { # Retrieve from Vertex RAG Store for grounding. # Set to use data source powered by Vertex RAG store. User data is uploaded via the VertexRagDataService.
          "ragCorpora": [ # Optional. Deprecated. Please use rag_resources instead.
            "A String",
          ],
          "ragResources": [ # Optional. The representation of the rag source. It can be used to specify corpus only or ragfiles. Currently only support one corpus or multiple files from one corpus. In the future we may open up multiple corpora support.
            { # The definition of the Rag resource.
              "ragCorpus": "A String", # Optional. RagCorpora resource name. Format: `projects/{project}/locations/{location}/ragCorpora/{rag_corpus}`
              "ragFileIds": [ # Optional. rag_file_id. The files should be in the same rag_corpus set in rag_corpus field.
                "A String",
              ],
            },
          ],
          "similarityTopK": 42, # Optional. Number of top k results to return from the selected corpora.
          "vectorDistanceThreshold": 3.14, # Optional. Only return results with vector distance smaller than the threshold.
        },
      },
    },
  ],
}

  x__xgafv: string, V1 error format.
    Allowed values
      1 - v1 error format
      2 - v2 error format

Returns:
  An object of the form:

    { # Response message for PredictionService.CountTokens.
  "totalBillableCharacters": 42, # The total number of billable characters counted across all instances from the request.
  "totalTokens": 42, # The total number of tokens counted across all instances from the request.
}

generateContent(model, body=None, x__xgafv=None)

Generate content with multimodal inputs.

Args:
  model: string, Required. The fully qualified name of the publisher model or tuned model endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` (required)
  body: object, The request body.
    The object takes the form of:

{ # Request message for [PredictionService.GenerateContent].
  "contents": [ # Required. The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history + latest request.
    { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
      "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
        { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
          "fileData": { # URI based data. # Optional. URI based data.
            "fileUri": "A String", # Required. URI.
            "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
          },
          "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
            "args": { # Optional. Required. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
              "a_key": "", # Properties of the object.
            },
            "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
          },
          "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
            "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
            "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
              "a_key": "", # Properties of the object.
            },
          },
          "inlineData": { # Content blob. It's preferred to send as text directly rather than raw bytes. # Optional. Inlined bytes data.
            "data": "A String", # Required. Raw bytes.
            "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
          },
          "text": "A String", # Optional. Text part (can be code).
          "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
            "endOffset": "A String", # Optional. The end offset of the video.
            "startOffset": "A String", # Optional. The start offset of the video.
          },
        },
      ],
      "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
    },
  ],
  "generationConfig": { # Generation config. # Optional. Generation config.
    "candidateCount": 42, # Optional. Number of candidates to generate.
    "frequencyPenalty": 3.14, # Optional. Frequency penalties.
    "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
    "presencePenalty": 3.14, # Optional. Positive penalties.
    "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
    "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
      "default": "", # Optional. Default value of the data.
      "description": "A String", # Optional. The description of the data.
      "enum": [ # Optional. Possible values of the element of Type.STRING with enum format. For example we can define an Enum Direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]}
        "A String",
      ],
      "example": "", # Optional. Example of the object. Will only populated when the object is the root.
      "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
      "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
      "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
      "maxLength": "A String", # Optional. Maximum length of the Type.STRING
      "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
      "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
      "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
      "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
      "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
      "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
      "nullable": True or False, # Optional. Indicates if the value may be null.
      "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
      "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
        "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
      },
      "required": [ # Optional. Required properties of Type.OBJECT.
        "A String",
      ],
      "title": "A String", # Optional. The title of the Schema.
      "type": "A String", # Optional. The type of the data.
    },
    "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
      "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
        "modelRoutingPreference": "A String", # The model routing preference.
      },
      "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
        "modelName": "A String", # The model name to use. Only the public LLM models are accepted. e.g. 'gemini-1.5-pro-001'.
      },
    },
    "seed": 42, # Optional. Seed.
    "stopSequences": [ # Optional. Stop sequences.
      "A String",
    ],
    "temperature": 3.14, # Optional. Controls the randomness of predictions.
    "topK": 3.14, # Optional. If specified, top-k sampling will be used.
    "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
  },
  "safetySettings": [ # Optional. Per request settings for blocking unsafe content. Enforced on GenerateContentResponse.candidates.
    { # Safety settings.
      "category": "A String", # Required. Harm category.
      "method": "A String", # Optional. Specify if the threshold is used for probability or severity score. If not specified, the threshold is used for probability score.
      "threshold": "A String", # Required. The harm block threshold.
    },
  ],
  "systemInstruction": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn. # Optional. The user provided system instructions for the model. Note: only text should be used in parts and content in each part will be in a separate paragraph.
    "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
      { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
        "fileData": { # URI based data. # Optional. URI based data.
          "fileUri": "A String", # Required. URI.
          "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
        },
        "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
          "args": { # Optional. Required. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
            "a_key": "", # Properties of the object.
          },
          "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
        },
        "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
          "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
          "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
            "a_key": "", # Properties of the object.
          },
        },
        "inlineData": { # Content blob. It's preferred to send as text directly rather than raw bytes. # Optional. Inlined bytes data.
          "data": "A String", # Required. Raw bytes.
          "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
        },
        "text": "A String", # Optional. Text part (can be code).
        "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
          "endOffset": "A String", # Optional. The end offset of the video.
          "startOffset": "A String", # Optional. The start offset of the video.
        },
      },
    ],
    "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
  },
  "toolConfig": { # Tool config. This config is shared for all tools provided in the request. # Optional. Tool config. This config is shared for all tools provided in the request.
    "functionCallingConfig": { # Function calling config. # Optional. Function calling config.
      "allowedFunctionNames": [ # Optional. Function names to call. Only set when the Mode is ANY. Function names should match [FunctionDeclaration.name]. With mode set to ANY, model will predict a function call from the set of function names provided.
        "A String",
      ],
      "mode": "A String", # Optional. Function calling mode.
    },
  },
  "tools": [ # Optional. A list of `Tools` the model may use to generate the next response. A `Tool` is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model.
    { # Tool details that the model may use to generate response. A `Tool` is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. A Tool object should contain exactly one type of Tool (e.g FunctionDeclaration, Retrieval or GoogleSearchRetrieval).
      "functionDeclarations": [ # Optional. Function tool type. One or more function declarations to be passed to the model along with the current user query. Model may decide to call a subset of these functions by populating FunctionCall in the response. User should provide a FunctionResponse for each function call in the next turn. Based on the function responses, Model will generate the final response back to the user. Maximum 64 function declarations can be provided.
        { # Structured representation of a function declaration as defined by the [OpenAPI 3.0 specification](https://spec.openapis.org/oas/v3.0.3). Included in this declaration are the function name and parameters. This FunctionDeclaration is a representation of a block of code that can be used as a `Tool` by the model and executed by the client.
          "description": "A String", # Optional. Description and purpose of the function. Model uses it to decide how and whether to call the function.
          "name": "A String", # Required. The name of the function to call. Must start with a letter or an underscore. Must be a-z, A-Z, 0-9, or contain underscores, dots and dashes, with a maximum length of 64.
          "parameters": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. Describes the parameters to this function in JSON Schema Object format. Reflects the Open API 3.03 Parameter Object. string Key: the name of the parameter. Parameter names are case sensitive. Schema Value: the Schema defining the type used for the parameter. For function with no parameters, this can be left unset. Parameter names must start with a letter or an underscore and must only contain chars a-z, A-Z, 0-9, or underscores with a maximum length of 64. Example with 1 required and 1 optional parameter: type: OBJECT properties: param1: type: STRING param2: type: INTEGER required: - param1
            "default": "", # Optional. Default value of the data.
            "description": "A String", # Optional. The description of the data.
            "enum": [ # Optional. Possible values of the element of Type.STRING with enum format. For example we can define an Enum Direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]}
              "A String",
            ],
            "example": "", # Optional. Example of the object. Will only populated when the object is the root.
            "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
            "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
            "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
            "maxLength": "A String", # Optional. Maximum length of the Type.STRING
            "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
            "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
            "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
            "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
            "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
            "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
            "nullable": True or False, # Optional. Indicates if the value may be null.
            "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
            "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
              "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
            },
            "required": [ # Optional. Required properties of Type.OBJECT.
              "A String",
            ],
            "title": "A String", # Optional. The title of the Schema.
            "type": "A String", # Optional. The type of the data.
          },
          "response": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. Describes the output from this function in JSON Schema format. Reflects the Open API 3.03 Response Object. The Schema defines the type used for the response value of the function.
            "default": "", # Optional. Default value of the data.
            "description": "A String", # Optional. The description of the data.
            "enum": [ # Optional. Possible values of the element of Type.STRING with enum format. For example we can define an Enum Direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]}
              "A String",
            ],
            "example": "", # Optional. Example of the object. Will only populated when the object is the root.
            "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
            "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
            "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
            "maxLength": "A String", # Optional. Maximum length of the Type.STRING
            "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
            "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
            "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
            "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
            "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
            "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
            "nullable": True or False, # Optional. Indicates if the value may be null.
            "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
            "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
              "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
            },
            "required": [ # Optional. Required properties of Type.OBJECT.
              "A String",
            ],
            "title": "A String", # Optional. The title of the Schema.
            "type": "A String", # Optional. The type of the data.
          },
        },
      ],
      "googleSearchRetrieval": { # Tool to retrieve public web data for grounding, powered by Google. # Optional. GoogleSearchRetrieval tool type. Specialized retrieval tool that is powered by Google search.
      },
      "retrieval": { # Defines a retrieval tool that model can call to access external knowledge. # Optional. Retrieval tool type. System will always execute the provided retrieval tool(s) to get external knowledge to answer the prompt. Retrieval results are presented to the model for generation.
        "disableAttribution": True or False, # Optional. Deprecated. This option is no longer supported.
        "vertexAiSearch": { # Retrieve from Vertex AI Search datastore for grounding. See https://cloud.google.com/products/agent-builder # Set to use data source powered by Vertex AI Search.
          "datastore": "A String", # Required. Fully-qualified Vertex AI Search data store resource ID. Format: `projects/{project}/locations/{location}/collections/{collection}/dataStores/{dataStore}`
        },
        "vertexRagStore": { # Retrieve from Vertex RAG Store for grounding. # Set to use data source powered by Vertex RAG store. User data is uploaded via the VertexRagDataService.
          "ragCorpora": [ # Optional. Deprecated. Please use rag_resources instead.
            "A String",
          ],
          "ragResources": [ # Optional. The representation of the rag source. It can be used to specify corpus only or ragfiles. Currently only support one corpus or multiple files from one corpus. In the future we may open up multiple corpora support.
            { # The definition of the Rag resource.
              "ragCorpus": "A String", # Optional. RagCorpora resource name. Format: `projects/{project}/locations/{location}/ragCorpora/{rag_corpus}`
              "ragFileIds": [ # Optional. rag_file_id. The files should be in the same rag_corpus set in rag_corpus field.
                "A String",
              ],
            },
          ],
          "similarityTopK": 42, # Optional. Number of top k results to return from the selected corpora.
          "vectorDistanceThreshold": 3.14, # Optional. Only return results with vector distance smaller than the threshold.
        },
      },
    },
  ],
}

  x__xgafv: string, V1 error format.
    Allowed values
      1 - v1 error format
      2 - v2 error format

Returns:
  An object of the form:

    { # Response message for [PredictionService.GenerateContent].
  "candidates": [ # Output only. Generated candidates.
    { # A response candidate generated from the model.
      "avgLogprobs": 3.14, # Output only. Average log probability score of the candidate.
      "citationMetadata": { # A collection of source attributions for a piece of content. # Output only. Source attribution of the generated content.
        "citations": [ # Output only. List of citations.
          { # Source attributions for content.
            "endIndex": 42, # Output only. End index into the content.
            "license": "A String", # Output only. License of the attribution.
            "publicationDate": { # Represents a whole or partial calendar date, such as a birthday. The time of day and time zone are either specified elsewhere or are insignificant. The date is relative to the Gregorian Calendar. This can represent one of the following: * A full date, with non-zero year, month, and day values. * A month and day, with a zero year (for example, an anniversary). * A year on its own, with a zero month and a zero day. * A year and month, with a zero day (for example, a credit card expiration date). Related types: * google.type.TimeOfDay * google.type.DateTime * google.protobuf.Timestamp # Output only. Publication date of the attribution.
              "day": 42, # Day of a month. Must be from 1 to 31 and valid for the year and month, or 0 to specify a year by itself or a year and month where the day isn't significant.
              "month": 42, # Month of a year. Must be from 1 to 12, or 0 to specify a year without a month and day.
              "year": 42, # Year of the date. Must be from 1 to 9999, or 0 to specify a date without a year.
            },
            "startIndex": 42, # Output only. Start index into the content.
            "title": "A String", # Output only. Title of the attribution.
            "uri": "A String", # Output only. Url reference of the attribution.
          },
        ],
      },
      "content": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn. # Output only. Content parts of the candidate.
        "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
          { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
            "fileData": { # URI based data. # Optional. URI based data.
              "fileUri": "A String", # Required. URI.
              "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
            },
            "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
              "args": { # Optional. Required. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
                "a_key": "", # Properties of the object.
              },
              "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
            },
            "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
              "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
              "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
                "a_key": "", # Properties of the object.
              },
            },
            "inlineData": { # Content blob. It's preferred to send as text directly rather than raw bytes. # Optional. Inlined bytes data.
              "data": "A String", # Required. Raw bytes.
              "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
            },
            "text": "A String", # Optional. Text part (can be code).
            "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
              "endOffset": "A String", # Optional. The end offset of the video.
              "startOffset": "A String", # Optional. The start offset of the video.
            },
          },
        ],
        "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
      },
      "finishMessage": "A String", # Output only. Describes the reason the mode stopped generating tokens in more detail. This is only filled when `finish_reason` is set.
      "finishReason": "A String", # Output only. The reason why the model stopped generating tokens. If empty, the model has not stopped generating the tokens.
      "groundingMetadata": { # Metadata returned to client when grounding is enabled. # Output only. Metadata specifies sources used to ground generated content.
        "groundingChunks": [ # List of supporting references retrieved from specified grounding source.
          { # Grounding chunk.
            "retrievedContext": { # Chunk from context retrieved by the retrieval tools. # Grounding chunk from context retrieved by the retrieval tools.
              "title": "A String", # Title of the attribution.
              "uri": "A String", # URI reference of the attribution.
            },
            "web": { # Chunk from the web. # Grounding chunk from the web.
              "title": "A String", # Title of the chunk.
              "uri": "A String", # URI reference of the chunk.
            },
          },
        ],
        "groundingSupports": [ # Optional. List of grounding support.
          { # Grounding support.
            "confidenceScores": [ # Confidence score of the support references. Ranges from 0 to 1. 1 is the most confident. This list must have the same size as the grounding_chunk_indices.
              3.14,
            ],
            "groundingChunkIndices": [ # A list of indices (into 'grounding_chunk') specifying the citations associated with the claim. For instance [1,3,4] means that grounding_chunk[1], grounding_chunk[3], grounding_chunk[4] are the retrieved content attributed to the claim.
              42,
            ],
            "segment": { # Segment of the content. # Segment of the content this support belongs to.
              "endIndex": 42, # Output only. End index in the given Part, measured in bytes. Offset from the start of the Part, exclusive, starting at zero.
              "partIndex": 42, # Output only. The index of a Part object within its parent Content object.
              "startIndex": 42, # Output only. Start index in the given Part, measured in bytes. Offset from the start of the Part, inclusive, starting at zero.
              "text": "A String", # Output only. The text corresponding to the segment from the response.
            },
          },
        ],
        "searchEntryPoint": { # Google search entry point. # Optional. Google search entry for the following-up web searches.
          "renderedContent": "A String", # Optional. Web content snippet that can be embedded in a web page or an app webview.
          "sdkBlob": "A String", # Optional. Base64 encoded JSON representing array of tuple.
        },
        "webSearchQueries": [ # Optional. Web search queries for the following-up web search.
          "A String",
        ],
      },
      "index": 42, # Output only. Index of the candidate.
      "safetyRatings": [ # Output only. List of ratings for the safety of a response candidate. There is at most one rating per category.
        { # Safety rating corresponding to the generated content.
          "blocked": True or False, # Output only. Indicates whether the content was filtered out because of this rating.
          "category": "A String", # Output only. Harm category.
          "probability": "A String", # Output only. Harm probability levels in the content.
          "probabilityScore": 3.14, # Output only. Harm probability score.
          "severity": "A String", # Output only. Harm severity levels in the content.
          "severityScore": 3.14, # Output only. Harm severity score.
        },
      ],
    },
  ],
  "promptFeedback": { # Content filter results for a prompt sent in the request. # Output only. Content filter results for a prompt sent in the request. Note: Sent only in the first stream chunk. Only happens when no candidates were generated due to content violations.
    "blockReason": "A String", # Output only. Blocked reason.
    "blockReasonMessage": "A String", # Output only. A readable block reason message.
    "safetyRatings": [ # Output only. Safety ratings.
      { # Safety rating corresponding to the generated content.
        "blocked": True or False, # Output only. Indicates whether the content was filtered out because of this rating.
        "category": "A String", # Output only. Harm category.
        "probability": "A String", # Output only. Harm probability levels in the content.
        "probabilityScore": 3.14, # Output only. Harm probability score.
        "severity": "A String", # Output only. Harm severity levels in the content.
        "severityScore": 3.14, # Output only. Harm severity score.
      },
    ],
  },
  "usageMetadata": { # Usage metadata about response(s). # Usage metadata about the response(s).
    "candidatesTokenCount": 42, # Number of tokens in the response(s).
    "promptTokenCount": 42, # Number of tokens in the request. When `cached_content` is set, this is still the total effective prompt size meaning this includes the number of tokens in the cached content.
    "totalTokenCount": 42, # Total token count for prompt and response candidates.
  },
}

get(name, huggingFaceToken=None, isHuggingFaceModel=None, languageCode=None, view=None, x__xgafv=None)

Gets a Model Garden publisher model.

Args:
name: string, Required. The name of the PublisherModel resource. Format: `publishers/{publisher}/models/{publisher_model}` (required)
huggingFaceToken: string, Optional. Token used to access Hugging Face gated models.
isHuggingFaceModel: boolean, Optional. Boolean indicates whether the requested model is a Hugging Face model.
languageCode: string, Optional. The IETF BCP-47 language code representing the language in which the publisher model's text information should be written in.
view: string, Optional. PublisherModel view specifying which fields to read.
Allowed values
PUBLISHER_MODEL_VIEW_UNSPECIFIED - The default / unset value. The API will default to the BASIC view.
PUBLISHER_MODEL_VIEW_BASIC - Include basic metadata about the publisher model, but not the full contents.
PUBLISHER_MODEL_VIEW_FULL - Include everything.
PUBLISHER_MODEL_VERSION_VIEW_BASIC - Include: VersionId, ModelVersionExternalName, and SupportedActions.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format

Returns:
An object of the form:

{ # A Model Garden Publisher Model.
"frameworks": [ # Optional. Additional information about the model's Frameworks.
"A String",
],
"launchStage": "A String", # Optional. Indicates the launch stage of the model.
"name": "A String", # Output only. The resource name of the PublisherModel.
"openSourceCategory": "A String", # Required. Indicates the open source category of the publisher model.
"predictSchemata": { # Contains the schemata used in Model's predictions and explanations via PredictionService.Predict, PredictionService.Explain and BatchPredictionJob. # Optional. The schemata that describes formats of the PublisherModel's predictions and explanations as given and returned via PredictionService.Predict.
"instanceSchemaUri": "A String", # Immutable. Points to a YAML file stored on Google Cloud Storage describing the format of a single instance, which are used in PredictRequest.instances, ExplainRequest.instances and BatchPredictionJob.input_config. The schema is defined as an OpenAPI 3.0.2 [Schema Object](https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.2.md#schemaObject). AutoML Models always have this field populated by Vertex AI. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.
"parametersSchemaUri": "A String", # Immutable. Points to a YAML file stored on Google Cloud Storage describing the parameters of prediction and explanation via PredictRequest.parameters, ExplainRequest.parameters and BatchPredictionJob.model_parameters. The schema is defined as an OpenAPI 3.0.2 [Schema Object](https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.2.md#schemaObject). AutoML Models always have this field populated by Vertex AI, if no parameters are supported, then it is set to an empty string. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.
"predictionSchemaUri": "A String", # Immutable. Points to a YAML file stored on Google Cloud Storage describing the format of a single prediction produced by this Model, which are returned via PredictResponse.predictions, ExplainResponse.explanations, and BatchPredictionJob.output_config. The schema is defined as an OpenAPI 3.0.2 [Schema Object](https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.2.md#schemaObject). AutoML Models always have this field populated by Vertex AI. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.
},
"publisherModelTemplate": "A String", # Optional. Output only. Immutable. Used to indicate this model has a publisher model and provide the template of the publisher model resource name.
"supportedActions": { # Actions could take on this Publisher Model. # Optional. Supported call-to-action options.
"createApplication": { # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc.. # Optional. Create application using the PublisherModel.
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
"deploy": { # Model metadata that is needed for UploadModel or DeployModel/CreateEndpoint requests. # Optional. Deploy the PublisherModel to Vertex Endpoint.
"artifactUri": "A String", # Optional. The path to the directory containing the Model artifact and any of its supporting files.
"automaticResources": { # A description of resources that to large degree are decided by Vertex AI, and require only a modest additional configuration. Each Model supporting these resources documents its specific guidelines. # A description of resources that to large degree are decided by Vertex AI, and require only a modest additional configuration.
"maxReplicaCount": 42, # Immutable. The maximum number of replicas this DeployedModel may be deployed on when the traffic against it increases. If the requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale the model to that many replicas is guaranteed (barring service outages). If traffic against the DeployedModel increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, a no upper bound for scaling under heavy traffic will be assume, though Vertex AI may be unable to scale beyond certain replica number.
"minReplicaCount": 42, # Immutable. The minimum number of replicas this DeployedModel will be always deployed on. If traffic against it increases, it may dynamically be deployed onto more replicas up to max_replica_count, and as traffic decreases, some of these extra replicas may be freed. If the requested value is too large, the deployment will error.
},
"containerSpec": { # Specification of a container for serving predictions. Some fields in this message correspond to fields in the [Kubernetes Container v1 core specification](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core). # Optional. The specification of the container that is to be used when deploying this Model in Vertex AI. Not present for Large Models.
"args": [ # Immutable. Specifies arguments for the command that runs when the container starts. This overrides the container's [`CMD`](https://docs.docker.com/engine/reference/builder/#cmd). Specify this field as an array of executable and arguments, similar to a Docker `CMD`'s "default parameters" form. If you don't specify this field but do specify the command field, then the command from the `command` field runs without any additional arguments. See the [Kubernetes documentation about how the `command` and `args` fields interact with a container's `ENTRYPOINT` and `CMD`](https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#notes). If you don't specify this field and don't specify the `command` field, then the container's [`ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#cmd) and `CMD` determine what runs based on their default behavior. See the Docker documentation about [how `CMD` and `ENTRYPOINT` interact](https://docs.docker.com/engine/reference/builder/#understand-how-cmd-and-entrypoint-interact). In this field, you can reference [environment variables set by Vertex AI](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables) and environment variables set in the env field. You cannot reference environment variables set in the Docker image. In order for environment variables to be expanded, reference them by using the following syntax: $( VARIABLE_NAME) Note that this differs from Bash variable expansion, which does not use parentheses. If a variable cannot be resolved, the reference in the input string is used unchanged. To avoid variable expansion, you can escape this syntax with `$$`; for example: $$(VARIABLE_NAME) This field corresponds to the `args` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core).
"A String",
],
"command": [ # Immutable. Specifies the command that runs when the container starts. This overrides the container's [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint). Specify this field as an array of executable and arguments, similar to a Docker `ENTRYPOINT`'s "exec" form, not its "shell" form. If you do not specify this field, then the container's `ENTRYPOINT` runs, in conjunction with the args field or the container's [`CMD`](https://docs.docker.com/engine/reference/builder/#cmd), if either exists. If this field is not specified and the container does not have an `ENTRYPOINT`, then refer to the Docker documentation about [how `CMD` and `ENTRYPOINT` interact](https://docs.docker.com/engine/reference/builder/#understand-how-cmd-and-entrypoint-interact). If you specify this field, then you can also specify the `args` field to provide additional arguments for this command. However, if you specify this field, then the container's `CMD` is ignored. See the [Kubernetes documentation about how the `command` and `args` fields interact with a container's `ENTRYPOINT` and `CMD`](https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#notes). In this field, you can reference [environment variables set by Vertex AI](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables) and environment variables set in the env field. You cannot reference environment variables set in the Docker image. In order for environment variables to be expanded, reference them by using the following syntax: $( VARIABLE_NAME) Note that this differs from Bash variable expansion, which does not use parentheses. If a variable cannot be resolved, the reference in the input string is used unchanged. To avoid variable expansion, you can escape this syntax with `$$`; for example: $$(VARIABLE_NAME) This field corresponds to the `command` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core).
"A String",
],
"deploymentTimeout": "A String", # Immutable. Deployment timeout. Limit for deployment timeout is 2 hours.
"env": [ # Immutable. List of environment variables to set in the container. After the container starts running, code running in the container can read these environment variables. Additionally, the command and args fields can reference these variables. Later entries in this list can also reference earlier entries. For example, the following example sets the variable `VAR_2` to have the value `foo bar`: ```json [ { "name": "VAR_1", "value": "foo" }, { "name": "VAR_2", "value": "$(VAR_1) bar" } ] ``` If you switch the order of the variables in the example, then the expansion does not occur. This field corresponds to the `env` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core).
{ # Represents an environment variable present in a Container or Python Module.
"name": "A String", # Required. Name of the environment variable. Must be a valid C identifier.
"value": "A String", # Required. Variables that reference a $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.
},
],
"grpcPorts": [ # Immutable. List of ports to expose from the container. Vertex AI sends gRPC prediction requests that it receives to the first port on this list. Vertex AI also sends liveness and health checks to this port. If you do not specify this field, gRPC requests to the container will be disabled. Vertex AI does not use ports other than the first one listed. This field corresponds to the `ports` field of the Kubernetes Containers v1 core API.
{ # Represents a network port in a container.
"containerPort": 42, # The number of the port to expose on the pod's IP address. Must be a valid port number, between 1 and 65535 inclusive.
},
],
"healthProbe": { # Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic. # Immutable. Specification for Kubernetes readiness probe.
"exec": { # ExecAction specifies a command to execute. # ExecAction probes the health of a container by executing a command.
"command": [ # Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.
"A String",
],
},
"periodSeconds": 42, # How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. Must be less than timeout_seconds. Maps to Kubernetes probe argument 'periodSeconds'.
"timeoutSeconds": 42, # Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. Must be greater or equal to period_seconds. Maps to Kubernetes probe argument 'timeoutSeconds'.
},
"healthRoute": "A String", # Immutable. HTTP path on the container to send health checks to. Vertex AI intermittently sends GET requests to this path on the container's IP address and port to check that the container is healthy. Read more about [health checks](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#health). For example, if you set this field to `/bar`, then Vertex AI intermittently sends a GET request to the `/bar` path on the port of your container specified by the first value of this `ModelContainerSpec`'s ports field. If you don't specify this field, it defaults to the following value when you deploy this Model to an Endpoint: /v1/endpoints/ENDPOINT/deployedModels/ DEPLOYED_MODEL:predict The placeholders in this value are replaced as follows: * ENDPOINT: The last segment (following `endpoints/`)of the Endpoint.name][] field of the Endpoint where this Model has been deployed. (Vertex AI makes this value available to your container code as the [`AIP_ENDPOINT_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables).) * DEPLOYED_MODEL: DeployedModel.id of the `DeployedModel`. (Vertex AI makes this value available to your container code as the [`AIP_DEPLOYED_MODEL_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables).)
"imageUri": "A String", # Required. Immutable. URI of the Docker image to be used as the custom container for serving predictions. This URI must identify an image in Artifact Registry or Container Registry. Learn more about the [container publishing requirements](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#publishing), including permissions requirements for the Vertex AI Service Agent. The container image is ingested upon ModelService.UploadModel, stored internally, and this original path is afterwards not used. To learn about the requirements for the Docker image itself, see [Custom container requirements](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#). You can use the URI to one of Vertex AI's [pre-built container images for prediction](https://cloud.google.com/vertex-ai/docs/predictions/pre-built-containers) in this field.
"ports": [ # Immutable. List of ports to expose from the container. Vertex AI sends any prediction requests that it receives to the first port on this list. Vertex AI also sends [liveness and health checks](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#liveness) to this port. If you do not specify this field, it defaults to following value: ```json [ { "containerPort": 8080 } ] ``` Vertex AI does not use ports other than the first one listed. This field corresponds to the `ports` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core).
{ # Represents a network port in a container.
"containerPort": 42, # The number of the port to expose on the pod's IP address. Must be a valid port number, between 1 and 65535 inclusive.
},
],
"predictRoute": "A String", # Immutable. HTTP path on the container to send prediction requests to. Vertex AI forwards requests sent using projects.locations.endpoints.predict to this path on the container's IP address and port. Vertex AI then returns the container's response in the API response. For example, if you set this field to `/foo`, then when Vertex AI receives a prediction request, it forwards the request body in a POST request to the `/foo` path on the port of your container specified by the first value of this `ModelContainerSpec`'s ports field. If you don't specify this field, it defaults to the following value when you deploy this Model to an Endpoint: /v1/endpoints/ENDPOINT/deployedModels/DEPLOYED_MODEL:predict The placeholders in this value are replaced as follows: * ENDPOINT: The last segment (following `endpoints/`)of the Endpoint.name][] field of the Endpoint where this Model has been deployed. (Vertex AI makes this value available to your container code as the [`AIP_ENDPOINT_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables).) * DEPLOYED_MODEL: DeployedModel.id of the `DeployedModel`. (Vertex AI makes this value available to your container code as the [`AIP_DEPLOYED_MODEL_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables).)
"sharedMemorySizeMb": "A String", # Immutable. The amount of the VM memory to reserve as the shared memory for the model in megabytes.
"startupProbe": { # Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic. # Immutable. Specification for Kubernetes startup probe.
"exec": { # ExecAction specifies a command to execute. # ExecAction probes the health of a container by executing a command.
"command": [ # Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.
"A String",
],
},
"periodSeconds": 42, # How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. Must be less than timeout_seconds. Maps to Kubernetes probe argument 'periodSeconds'.
"timeoutSeconds": 42, # Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. Must be greater or equal to period_seconds. Maps to Kubernetes probe argument 'timeoutSeconds'.
},
},
"dedicatedResources": { # A description of resources that are dedicated to a DeployedModel, and that need a higher degree of manual configuration. # A description of resources that are dedicated to the DeployedModel, and that need a higher degree of manual configuration.
"autoscalingMetricSpecs": [ # Immutable. The metric specifications that overrides a resource utilization metric (CPU utilization, accelerator's duty cycle, and so on) target value (default to 60 if not set). At most one entry is allowed per metric. If machine_spec.accelerator_count is above 0, the autoscaling will be based on both CPU utilization and accelerator's duty cycle metrics and scale up when either metrics exceeds its target value while scale down if both metrics are under their target value. The default target value is 60 for both metrics. If machine_spec.accelerator_count is 0, the autoscaling will be based on CPU utilization metric only with default target value 60 if not explicitly set. For example, in the case of Online Prediction, if you want to override target CPU utilization to 80, you should set autoscaling_metric_specs.metric_name to `aiplatform.googleapis.com/prediction/online/cpu/utilization` and autoscaling_metric_specs.target to `80`.
{ # The metric specification that defines the target resource utilization (CPU utilization, accelerator's duty cycle, and so on) for calculating the desired replica count.
"metricName": "A String", # Required. The resource metric name. Supported metrics: * For Online Prediction: * `aiplatform.googleapis.com/prediction/online/accelerator/duty_cycle` * `aiplatform.googleapis.com/prediction/online/cpu/utilization`
"target": 42, # The target resource utilization in percentage (1% - 100%) for the given metric; once the real usage deviates from the target by a certain percentage, the machine replicas change. The default value is 60 (representing 60%) if not provided.
},
],
"machineSpec": { # Specification of a single machine. # Required. Immutable. The specification of a single machine used by the prediction.
"acceleratorCount": 42, # The number of accelerators to attach to the machine.
"acceleratorType": "A String", # Immutable. The type of accelerator(s) that may be attached to the machine as per accelerator_count.
"machineType": "A String", # Immutable. The type of the machine. See the [list of machine types supported for prediction](https://cloud.google.com/vertex-ai/docs/predictions/configure-compute#machine-types) See the [list of machine types supported for custom training](https://cloud.google.com/vertex-ai/docs/training/configure-compute#machine-types). For DeployedModel this field is optional, and the default value is `n1-standard-2`. For BatchPredictionJob or as part of WorkerPoolSpec this field is required.
"reservationAffinity": { # A ReservationAffinity can be used to configure a Vertex AI resource (e.g., a DeployedModel) to draw its Compute Engine resources from a Shared Reservation, or exclusively from on-demand capacity. # Optional. Immutable. Configuration controlling how this resource pool consumes reservation.
"key": "A String", # Optional. Corresponds to the label key of a reservation resource. To target a SPECIFIC_RESERVATION by name, use `compute.googleapis.com/reservation-name` as the key and specify the name of your reservation as its value.
"reservationAffinityType": "A String", # Required. Specifies the reservation affinity type.
"values": [ # Optional. Corresponds to the label values of a reservation resource. This must be the full resource name of the reservation.
"A String",
],
},
"tpuTopology": "A String", # Immutable. The topology of the TPUs. Corresponds to the TPU topologies available from GKE. (Example: tpu_topology: "2x2x1").
},
"maxReplicaCount": 42, # Immutable. The maximum number of replicas this DeployedModel may be deployed on when the traffic against it increases. If the requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale the model to that many replicas is guaranteed (barring service outages). If traffic against the DeployedModel increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, will use min_replica_count as the default value. The value of this field impacts the charge against Vertex CPU and GPU quotas. Specifically, you will be charged for (max_replica_count * number of cores in the selected machine type) and (max_replica_count * number of GPUs per replica in the selected machine type).
"minReplicaCount": 42, # Required. Immutable. The minimum number of machine replicas this DeployedModel will be always deployed on. This value must be greater than or equal to 1. If traffic against the DeployedModel increases, it may dynamically be deployed onto more replicas, and as traffic decreases, some of these extra replicas may be freed.
"spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms).
},
"deployMetadata": { # Metadata information about the deployment for managing deployment config. # Optional. Metadata information about this deployment config.
"labels": { # Optional. Labels for the deployment. For managing deployment config like verifying, source of deployment config, etc.
"a_key": "A String",
},
"sampleRequest": "A String", # Optional. Sample request for deployed endpoint.
},
"deployTaskName": "A String", # Optional. The name of the deploy task (e.g., "text to image generation").
"largeModelReference": { # Contains information about the Large Model. # Optional. Large model reference. When this is set, model_artifact_spec is not needed.
"name": "A String", # Required. The unique name of the large Foundation or pre-built model. Like "chat-bison", "text-bison". Or model name with version ID, like "chat-bison@001", "text-bison@005", etc.
},
"modelDisplayName": "A String", # Optional. Default model display name.
"publicArtifactUri": "A String", # Optional. The signed URI for ephemeral Cloud Storage access to model artifact.
"sharedResources": "A String", # The resource name of the shared DeploymentResourcePool to deploy on. Format: `projects/{project}/locations/{location}/deploymentResourcePools/{deployment_resource_pool}`
"title": "A String", # Required. The title of the regional resource reference.
},
"deployGke": { # Configurations for PublisherModel GKE deployment # Optional. Deploy PublisherModel to Google Kubernetes Engine.
"gkeYamlConfigs": [ # Optional. GKE deployment configuration in yaml format.
"A String",
],
},
"multiDeployVertex": { # Multiple setups to deploy the PublisherModel. # Optional. Multiple setups to deploy the PublisherModel to Vertex Endpoint.
"multiDeployVertex": [ # Optional. One click deployment configurations.
{ # Model metadata that is needed for UploadModel or DeployModel/CreateEndpoint requests.
"artifactUri": "A String", # Optional. The path to the directory containing the Model artifact and any of its supporting files.
"automaticResources": { # A description of resources that to large degree are decided by Vertex AI, and require only a modest additional configuration. Each Model supporting these resources documents its specific guidelines. # A description of resources that to large degree are decided by Vertex AI, and require only a modest additional configuration.
"maxReplicaCount": 42, # Immutable. The maximum number of replicas this DeployedModel may be deployed on when the traffic against it increases. If the requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale the model to that many replicas is guaranteed (barring service outages). If traffic against the DeployedModel increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, a no upper bound for scaling under heavy traffic will be assume, though Vertex AI may be unable to scale beyond certain replica number.
"minReplicaCount": 42, # Immutable. The minimum number of replicas this DeployedModel will be always deployed on. If traffic against it increases, it may dynamically be deployed onto more replicas up to max_replica_count, and as traffic decreases, some of these extra replicas may be freed. If the requested value is too large, the deployment will error.
},
"containerSpec": { # Specification of a container for serving predictions. Some fields in this message correspond to fields in the [Kubernetes Container v1 core specification](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core). # Optional. The specification of the container that is to be used when deploying this Model in Vertex AI. Not present for Large Models.
"args": [ # Immutable. Specifies arguments for the command that runs when the container starts. This overrides the container's [`CMD`](https://docs.docker.com/engine/reference/builder/#cmd). Specify this field as an array of executable and arguments, similar to a Docker `CMD`'s "default parameters" form. If you don't specify this field but do specify the command field, then the command from the `command` field runs without any additional arguments. See the [Kubernetes documentation about how the `command` and `args` fields interact with a container's `ENTRYPOINT` and `CMD`](https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#notes). If you don't specify this field and don't specify the `command` field, then the container's [`ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#cmd) and `CMD` determine what runs based on their default behavior. See the Docker documentation about [how `CMD` and `ENTRYPOINT` interact](https://docs.docker.com/engine/reference/builder/#understand-how-cmd-and-entrypoint-interact). In this field, you can reference [environment variables set by Vertex AI](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables) and environment variables set in the env field. You cannot reference environment variables set in the Docker image. In order for environment variables to be expanded, reference them by using the following syntax: $( VARIABLE_NAME) Note that this differs from Bash variable expansion, which does not use parentheses. If a variable cannot be resolved, the reference in the input string is used unchanged. To avoid variable expansion, you can escape this syntax with `$$`; for example: $$(VARIABLE_NAME) This field corresponds to the `args` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core).
"A String",
],
"command": [ # Immutable. Specifies the command that runs when the container starts. This overrides the container's [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint). Specify this field as an array of executable and arguments, similar to a Docker `ENTRYPOINT`'s "exec" form, not its "shell" form. If you do not specify this field, then the container's `ENTRYPOINT` runs, in conjunction with the args field or the container's [`CMD`](https://docs.docker.com/engine/reference/builder/#cmd), if either exists. If this field is not specified and the container does not have an `ENTRYPOINT`, then refer to the Docker documentation about [how `CMD` and `ENTRYPOINT` interact](https://docs.docker.com/engine/reference/builder/#understand-how-cmd-and-entrypoint-interact). If you specify this field, then you can also specify the `args` field to provide additional arguments for this command. However, if you specify this field, then the container's `CMD` is ignored. See the [Kubernetes documentation about how the `command` and `args` fields interact with a container's `ENTRYPOINT` and `CMD`](https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#notes). In this field, you can reference [environment variables set by Vertex AI](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables) and environment variables set in the env field. You cannot reference environment variables set in the Docker image. In order for environment variables to be expanded, reference them by using the following syntax: $( VARIABLE_NAME) Note that this differs from Bash variable expansion, which does not use parentheses. If a variable cannot be resolved, the reference in the input string is used unchanged. To avoid variable expansion, you can escape this syntax with `$$`; for example: $$(VARIABLE_NAME) This field corresponds to the `command` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core).
"A String",
],
"deploymentTimeout": "A String", # Immutable. Deployment timeout. Limit for deployment timeout is 2 hours.
"env": [ # Immutable. List of environment variables to set in the container. After the container starts running, code running in the container can read these environment variables. Additionally, the command and args fields can reference these variables. Later entries in this list can also reference earlier entries. For example, the following example sets the variable `VAR_2` to have the value `foo bar`: ```json [ { "name": "VAR_1", "value": "foo" }, { "name": "VAR_2", "value": "$(VAR_1) bar" } ] ``` If you switch the order of the variables in the example, then the expansion does not occur. This field corresponds to the `env` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core).
{ # Represents an environment variable present in a Container or Python Module.
"name": "A String", # Required. Name of the environment variable. Must be a valid C identifier.
"value": "A String", # Required. Variables that reference a $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.
},
],
"grpcPorts": [ # Immutable. List of ports to expose from the container. Vertex AI sends gRPC prediction requests that it receives to the first port on this list. Vertex AI also sends liveness and health checks to this port. If you do not specify this field, gRPC requests to the container will be disabled. Vertex AI does not use ports other than the first one listed. This field corresponds to the `ports` field of the Kubernetes Containers v1 core API.
{ # Represents a network port in a container.
"containerPort": 42, # The number of the port to expose on the pod's IP address. Must be a valid port number, between 1 and 65535 inclusive.
},
],
"healthProbe": { # Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic. # Immutable. Specification for Kubernetes readiness probe.
"exec": { # ExecAction specifies a command to execute. # ExecAction probes the health of a container by executing a command.
"command": [ # Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.
"A String",
],
},
"periodSeconds": 42, # How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. Must be less than timeout_seconds. Maps to Kubernetes probe argument 'periodSeconds'.
"timeoutSeconds": 42, # Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. Must be greater or equal to period_seconds. Maps to Kubernetes probe argument 'timeoutSeconds'.
},
"healthRoute": "A String", # Immutable. HTTP path on the container to send health checks to. Vertex AI intermittently sends GET requests to this path on the container's IP address and port to check that the container is healthy. Read more about [health checks](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#health). For example, if you set this field to `/bar`, then Vertex AI intermittently sends a GET request to the `/bar` path on the port of your container specified by the first value of this `ModelContainerSpec`'s ports field. If you don't specify this field, it defaults to the following value when you deploy this Model to an Endpoint: /v1/endpoints/ENDPOINT/deployedModels/ DEPLOYED_MODEL:predict The placeholders in this value are replaced as follows: * ENDPOINT: The last segment (following `endpoints/`)of the Endpoint.name][] field of the Endpoint where this Model has been deployed. (Vertex AI makes this value available to your container code as the [`AIP_ENDPOINT_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables).) * DEPLOYED_MODEL: DeployedModel.id of the `DeployedModel`. (Vertex AI makes this value available to your container code as the [`AIP_DEPLOYED_MODEL_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables).)
"imageUri": "A String", # Required. Immutable. URI of the Docker image to be used as the custom container for serving predictions. This URI must identify an image in Artifact Registry or Container Registry. Learn more about the [container publishing requirements](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#publishing), including permissions requirements for the Vertex AI Service Agent. The container image is ingested upon ModelService.UploadModel, stored internally, and this original path is afterwards not used. To learn about the requirements for the Docker image itself, see [Custom container requirements](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#). You can use the URI to one of Vertex AI's [pre-built container images for prediction](https://cloud.google.com/vertex-ai/docs/predictions/pre-built-containers) in this field.
"ports": [ # Immutable. List of ports to expose from the container. Vertex AI sends any prediction requests that it receives to the first port on this list. Vertex AI also sends [liveness and health checks](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#liveness) to this port. If you do not specify this field, it defaults to following value: ```json [ { "containerPort": 8080 } ] ``` Vertex AI does not use ports other than the first one listed. This field corresponds to the `ports` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#container-v1-core).
{ # Represents a network port in a container.
"containerPort": 42, # The number of the port to expose on the pod's IP address. Must be a valid port number, between 1 and 65535 inclusive.
},
],
"predictRoute": "A String", # Immutable. HTTP path on the container to send prediction requests to. Vertex AI forwards requests sent using projects.locations.endpoints.predict to this path on the container's IP address and port. Vertex AI then returns the container's response in the API response. For example, if you set this field to `/foo`, then when Vertex AI receives a prediction request, it forwards the request body in a POST request to the `/foo` path on the port of your container specified by the first value of this `ModelContainerSpec`'s ports field. If you don't specify this field, it defaults to the following value when you deploy this Model to an Endpoint: /v1/endpoints/ENDPOINT/deployedModels/DEPLOYED_MODEL:predict The placeholders in this value are replaced as follows: * ENDPOINT: The last segment (following `endpoints/`)of the Endpoint.name][] field of the Endpoint where this Model has been deployed. (Vertex AI makes this value available to your container code as the [`AIP_ENDPOINT_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables).) * DEPLOYED_MODEL: DeployedModel.id of the `DeployedModel`. (Vertex AI makes this value available to your container code as the [`AIP_DEPLOYED_MODEL_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#aip-variables).)
"sharedMemorySizeMb": "A String", # Immutable. The amount of the VM memory to reserve as the shared memory for the model in megabytes.
"startupProbe": { # Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic. # Immutable. Specification for Kubernetes startup probe.
"exec": { # ExecAction specifies a command to execute. # ExecAction probes the health of a container by executing a command.
"command": [ # Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.
"A String",
],
},
"periodSeconds": 42, # How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. Must be less than timeout_seconds. Maps to Kubernetes probe argument 'periodSeconds'.
"timeoutSeconds": 42, # Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. Must be greater or equal to period_seconds. Maps to Kubernetes probe argument 'timeoutSeconds'.
},
},
"dedicatedResources": { # A description of resources that are dedicated to a DeployedModel, and that need a higher degree of manual configuration. # A description of resources that are dedicated to the DeployedModel, and that need a higher degree of manual configuration.
"autoscalingMetricSpecs": [ # Immutable. The metric specifications that overrides a resource utilization metric (CPU utilization, accelerator's duty cycle, and so on) target value (default to 60 if not set). At most one entry is allowed per metric. If machine_spec.accelerator_count is above 0, the autoscaling will be based on both CPU utilization and accelerator's duty cycle metrics and scale up when either metrics exceeds its target value while scale down if both metrics are under their target value. The default target value is 60 for both metrics. If machine_spec.accelerator_count is 0, the autoscaling will be based on CPU utilization metric only with default target value 60 if not explicitly set. For example, in the case of Online Prediction, if you want to override target CPU utilization to 80, you should set autoscaling_metric_specs.metric_name to `aiplatform.googleapis.com/prediction/online/cpu/utilization` and autoscaling_metric_specs.target to `80`.
{ # The metric specification that defines the target resource utilization (CPU utilization, accelerator's duty cycle, and so on) for calculating the desired replica count.
"metricName": "A String", # Required. The resource metric name. Supported metrics: * For Online Prediction: * `aiplatform.googleapis.com/prediction/online/accelerator/duty_cycle` * `aiplatform.googleapis.com/prediction/online/cpu/utilization`
"target": 42, # The target resource utilization in percentage (1% - 100%) for the given metric; once the real usage deviates from the target by a certain percentage, the machine replicas change. The default value is 60 (representing 60%) if not provided.
},
],
"machineSpec": { # Specification of a single machine. # Required. Immutable. The specification of a single machine used by the prediction.
"acceleratorCount": 42, # The number of accelerators to attach to the machine.
"acceleratorType": "A String", # Immutable. The type of accelerator(s) that may be attached to the machine as per accelerator_count.
"machineType": "A String", # Immutable. The type of the machine. See the [list of machine types supported for prediction](https://cloud.google.com/vertex-ai/docs/predictions/configure-compute#machine-types) See the [list of machine types supported for custom training](https://cloud.google.com/vertex-ai/docs/training/configure-compute#machine-types). For DeployedModel this field is optional, and the default value is `n1-standard-2`. For BatchPredictionJob or as part of WorkerPoolSpec this field is required.
"reservationAffinity": { # A ReservationAffinity can be used to configure a Vertex AI resource (e.g., a DeployedModel) to draw its Compute Engine resources from a Shared Reservation, or exclusively from on-demand capacity. # Optional. Immutable. Configuration controlling how this resource pool consumes reservation.
"key": "A String", # Optional. Corresponds to the label key of a reservation resource. To target a SPECIFIC_RESERVATION by name, use `compute.googleapis.com/reservation-name` as the key and specify the name of your reservation as its value.
"reservationAffinityType": "A String", # Required. Specifies the reservation affinity type.
"values": [ # Optional. Corresponds to the label values of a reservation resource. This must be the full resource name of the reservation.
"A String",
],
},
"tpuTopology": "A String", # Immutable. The topology of the TPUs. Corresponds to the TPU topologies available from GKE. (Example: tpu_topology: "2x2x1").
},
"maxReplicaCount": 42, # Immutable. The maximum number of replicas this DeployedModel may be deployed on when the traffic against it increases. If the requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale the model to that many replicas is guaranteed (barring service outages). If traffic against the DeployedModel increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, will use min_replica_count as the default value. The value of this field impacts the charge against Vertex CPU and GPU quotas. Specifically, you will be charged for (max_replica_count * number of cores in the selected machine type) and (max_replica_count * number of GPUs per replica in the selected machine type).
"minReplicaCount": 42, # Required. Immutable. The minimum number of machine replicas this DeployedModel will be always deployed on. This value must be greater than or equal to 1. If traffic against the DeployedModel increases, it may dynamically be deployed onto more replicas, and as traffic decreases, some of these extra replicas may be freed.
"spot": True or False, # Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms).
},
"deployMetadata": { # Metadata information about the deployment for managing deployment config. # Optional. Metadata information about this deployment config.
"labels": { # Optional. Labels for the deployment. For managing deployment config like verifying, source of deployment config, etc.
"a_key": "A String",
},
"sampleRequest": "A String", # Optional. Sample request for deployed endpoint.
},
"deployTaskName": "A String", # Optional. The name of the deploy task (e.g., "text to image generation").
"largeModelReference": { # Contains information about the Large Model. # Optional. Large model reference. When this is set, model_artifact_spec is not needed.
"name": "A String", # Required. The unique name of the large Foundation or pre-built model. Like "chat-bison", "text-bison". Or model name with version ID, like "chat-bison@001", "text-bison@005", etc.
},
"modelDisplayName": "A String", # Optional. Default model display name.
"publicArtifactUri": "A String", # Optional. The signed URI for ephemeral Cloud Storage access to model artifact.
"sharedResources": "A String", # The resource name of the shared DeploymentResourcePool to deploy on. Format: `projects/{project}/locations/{location}/deploymentResourcePools/{deployment_resource_pool}`
"title": "A String", # Required. The title of the regional resource reference.
},
],
},
"openEvaluationPipeline": { # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc.. # Optional. Open evaluation pipeline of the PublisherModel.
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
"openFineTuningPipeline": { # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc.. # Optional. Open fine-tuning pipeline of the PublisherModel.
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
"openFineTuningPipelines": { # Open fine tuning pipelines. # Optional. Open fine-tuning pipelines of the PublisherModel.
"fineTuningPipelines": [ # Required. Regional resource references to fine tuning pipelines.
{ # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc..
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
],
},
"openGenerationAiStudio": { # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc.. # Optional. Open in Generation AI Studio.
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
"openGenie": { # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc.. # Optional. Open Genie / Playground.
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
"openNotebook": { # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc.. # Optional. Open notebook of the PublisherModel.
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
"openNotebooks": { # Open notebooks. # Optional. Open notebooks of the PublisherModel.
"notebooks": [ # Required. Regional resource references to notebooks.
{ # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc..
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
],
},
"openPromptTuningPipeline": { # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc.. # Optional. Open prompt-tuning pipeline of the PublisherModel.
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
"requestAccess": { # The regional resource name or the URI. Key is region, e.g., us-central1, europe-west2, global, etc.. # Optional. Request for access.
"references": { # Required.
"a_key": { # Reference to a resource.
"description": "A String", # Description of the resource.
"resourceName": "A String", # The resource name of the Google Cloud resource.
"uri": "A String", # The URI of the resource.
"useCase": "A String", # Use case (CUJ) of the resource.
},
},
"resourceDescription": "A String", # Optional. Description of the resource.
"resourceTitle": "A String", # Optional. Title of the resource.
"resourceUseCase": "A String", # Optional. Use case (CUJ) of the resource.
"title": "A String", # Required.
},
"viewRestApi": { # Rest API docs. # Optional. To view Rest API docs.
"documentations": [ # Required.
{ # A named piece of documentation.
"content": "A String", # Required. Content of this piece of document (in Markdown format).
"title": "A String", # Required. E.g., OVERVIEW, USE CASES, DOCUMENTATION, SDK & SAMPLES, JAVA, NODE.JS, etc..
},
],
"title": "A String", # Required. The title of the view rest API.
},
},
"versionId": "A String", # Output only. Immutable. The version ID of the PublisherModel. A new version is committed when a new model version is uploaded under an existing model id. It is an auto-incrementing decimal number in string representation.
"versionState": "A String", # Optional. Indicates the state of the model version.
}

streamGenerateContent(model, body=None, x__xgafv=None)

Generate content with multimodal inputs with streaming support.

Args:
  model: string, Required. The fully qualified name of the publisher model or tuned model endpoint to use. Publisher model format: `projects/{project}/locations/{location}/publishers/*/models/*` Tuned model endpoint format: `projects/{project}/locations/{location}/endpoints/{endpoint}` (required)
  body: object, The request body.
    The object takes the form of:

{ # Request message for [PredictionService.GenerateContent].
  "contents": [ # Required. The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history + latest request.
    { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn.
      "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
        { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
          "fileData": { # URI based data. # Optional. URI based data.
            "fileUri": "A String", # Required. URI.
            "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
          },
          "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
            "args": { # Optional. Required. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
              "a_key": "", # Properties of the object.
            },
            "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
          },
          "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
            "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
            "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
              "a_key": "", # Properties of the object.
            },
          },
          "inlineData": { # Content blob. It's preferred to send as text directly rather than raw bytes. # Optional. Inlined bytes data.
            "data": "A String", # Required. Raw bytes.
            "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
          },
          "text": "A String", # Optional. Text part (can be code).
          "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
            "endOffset": "A String", # Optional. The end offset of the video.
            "startOffset": "A String", # Optional. The start offset of the video.
          },
        },
      ],
      "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
    },
  ],
  "generationConfig": { # Generation config. # Optional. Generation config.
    "candidateCount": 42, # Optional. Number of candidates to generate.
    "frequencyPenalty": 3.14, # Optional. Frequency penalties.
    "maxOutputTokens": 42, # Optional. The maximum number of output tokens to generate per message.
    "presencePenalty": 3.14, # Optional. Positive penalties.
    "responseMimeType": "A String", # Optional. Output response mimetype of the generated candidate text. Supported mimetype: - `text/plain`: (default) Text output. - `application/json`: JSON response in the candidates. The model needs to be prompted to output the appropriate response type, otherwise the behavior is undefined. This is a preview feature.
    "responseSchema": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. The `Schema` object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema). If set, a compatible response_mime_type must also be set. Compatible mimetypes: `application/json`: Schema for JSON response.
      "default": "", # Optional. Default value of the data.
      "description": "A String", # Optional. The description of the data.
      "enum": [ # Optional. Possible values of the element of Type.STRING with enum format. For example we can define an Enum Direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]}
        "A String",
      ],
      "example": "", # Optional. Example of the object. Will only populated when the object is the root.
      "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
      "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
      "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
      "maxLength": "A String", # Optional. Maximum length of the Type.STRING
      "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
      "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
      "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
      "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
      "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
      "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
      "nullable": True or False, # Optional. Indicates if the value may be null.
      "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
      "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
        "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
      },
      "required": [ # Optional. Required properties of Type.OBJECT.
        "A String",
      ],
      "title": "A String", # Optional. The title of the Schema.
      "type": "A String", # Optional. The type of the data.
    },
    "routingConfig": { # The configuration for routing the request to a specific model. # Optional. Routing configuration.
      "autoMode": { # When automated routing is specified, the routing will be determined by the pretrained routing model and customer provided model routing preference. # Automated routing.
        "modelRoutingPreference": "A String", # The model routing preference.
      },
      "manualMode": { # When manual routing is set, the specified model will be used directly. # Manual routing.
        "modelName": "A String", # The model name to use. Only the public LLM models are accepted. e.g. 'gemini-1.5-pro-001'.
      },
    },
    "seed": 42, # Optional. Seed.
    "stopSequences": [ # Optional. Stop sequences.
      "A String",
    ],
    "temperature": 3.14, # Optional. Controls the randomness of predictions.
    "topK": 3.14, # Optional. If specified, top-k sampling will be used.
    "topP": 3.14, # Optional. If specified, nucleus sampling will be used.
  },
  "safetySettings": [ # Optional. Per request settings for blocking unsafe content. Enforced on GenerateContentResponse.candidates.
    { # Safety settings.
      "category": "A String", # Required. Harm category.
      "method": "A String", # Optional. Specify if the threshold is used for probability or severity score. If not specified, the threshold is used for probability score.
      "threshold": "A String", # Required. The harm block threshold.
    },
  ],
  "systemInstruction": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn. # Optional. The user provided system instructions for the model. Note: only text should be used in parts and content in each part will be in a separate paragraph.
    "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
      { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
        "fileData": { # URI based data. # Optional. URI based data.
          "fileUri": "A String", # Required. URI.
          "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
        },
        "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
          "args": { # Optional. Required. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
            "a_key": "", # Properties of the object.
          },
          "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
        },
        "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
          "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
          "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
            "a_key": "", # Properties of the object.
          },
        },
        "inlineData": { # Content blob. It's preferred to send as text directly rather than raw bytes. # Optional. Inlined bytes data.
          "data": "A String", # Required. Raw bytes.
          "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
        },
        "text": "A String", # Optional. Text part (can be code).
        "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
          "endOffset": "A String", # Optional. The end offset of the video.
          "startOffset": "A String", # Optional. The start offset of the video.
        },
      },
    ],
    "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
  },
  "toolConfig": { # Tool config. This config is shared for all tools provided in the request. # Optional. Tool config. This config is shared for all tools provided in the request.
    "functionCallingConfig": { # Function calling config. # Optional. Function calling config.
      "allowedFunctionNames": [ # Optional. Function names to call. Only set when the Mode is ANY. Function names should match [FunctionDeclaration.name]. With mode set to ANY, model will predict a function call from the set of function names provided.
        "A String",
      ],
      "mode": "A String", # Optional. Function calling mode.
    },
  },
  "tools": [ # Optional. A list of `Tools` the model may use to generate the next response. A `Tool` is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model.
    { # Tool details that the model may use to generate response. A `Tool` is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. A Tool object should contain exactly one type of Tool (e.g FunctionDeclaration, Retrieval or GoogleSearchRetrieval).
      "functionDeclarations": [ # Optional. Function tool type. One or more function declarations to be passed to the model along with the current user query. Model may decide to call a subset of these functions by populating FunctionCall in the response. User should provide a FunctionResponse for each function call in the next turn. Based on the function responses, Model will generate the final response back to the user. Maximum 64 function declarations can be provided.
        { # Structured representation of a function declaration as defined by the [OpenAPI 3.0 specification](https://spec.openapis.org/oas/v3.0.3). Included in this declaration are the function name and parameters. This FunctionDeclaration is a representation of a block of code that can be used as a `Tool` by the model and executed by the client.
          "description": "A String", # Optional. Description and purpose of the function. Model uses it to decide how and whether to call the function.
          "name": "A String", # Required. The name of the function to call. Must start with a letter or an underscore. Must be a-z, A-Z, 0-9, or contain underscores, dots and dashes, with a maximum length of 64.
          "parameters": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. Describes the parameters to this function in JSON Schema Object format. Reflects the Open API 3.03 Parameter Object. string Key: the name of the parameter. Parameter names are case sensitive. Schema Value: the Schema defining the type used for the parameter. For function with no parameters, this can be left unset. Parameter names must start with a letter or an underscore and must only contain chars a-z, A-Z, 0-9, or underscores with a maximum length of 64. Example with 1 required and 1 optional parameter: type: OBJECT properties: param1: type: STRING param2: type: INTEGER required: - param1
            "default": "", # Optional. Default value of the data.
            "description": "A String", # Optional. The description of the data.
            "enum": [ # Optional. Possible values of the element of Type.STRING with enum format. For example we can define an Enum Direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]}
              "A String",
            ],
            "example": "", # Optional. Example of the object. Will only populated when the object is the root.
            "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
            "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
            "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
            "maxLength": "A String", # Optional. Maximum length of the Type.STRING
            "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
            "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
            "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
            "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
            "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
            "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
            "nullable": True or False, # Optional. Indicates if the value may be null.
            "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
            "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
              "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
            },
            "required": [ # Optional. Required properties of Type.OBJECT.
              "A String",
            ],
            "title": "A String", # Optional. The title of the Schema.
            "type": "A String", # Optional. The type of the data.
          },
          "response": { # Schema is used to define the format of input/output data. Represents a select subset of an [OpenAPI 3.0 schema object](https://spec.openapis.org/oas/v3.0.3#schema-object). More fields may be added in the future as needed. # Optional. Describes the output from this function in JSON Schema format. Reflects the Open API 3.03 Response Object. The Schema defines the type used for the response value of the function.
            "default": "", # Optional. Default value of the data.
            "description": "A String", # Optional. The description of the data.
            "enum": [ # Optional. Possible values of the element of Type.STRING with enum format. For example we can define an Enum Direction as : {type:STRING, format:enum, enum:["EAST", NORTH", "SOUTH", "WEST"]}
              "A String",
            ],
            "example": "", # Optional. Example of the object. Will only populated when the object is the root.
            "format": "A String", # Optional. The format of the data. Supported formats: for NUMBER type: "float", "double" for INTEGER type: "int32", "int64" for STRING type: "email", "byte", etc
            "items": # Object with schema name: GoogleCloudAiplatformV1Schema # Optional. SCHEMA FIELDS FOR TYPE ARRAY Schema of the elements of Type.ARRAY.
            "maxItems": "A String", # Optional. Maximum number of the elements for Type.ARRAY.
            "maxLength": "A String", # Optional. Maximum length of the Type.STRING
            "maxProperties": "A String", # Optional. Maximum number of the properties for Type.OBJECT.
            "maximum": 3.14, # Optional. Maximum value of the Type.INTEGER and Type.NUMBER
            "minItems": "A String", # Optional. Minimum number of the elements for Type.ARRAY.
            "minLength": "A String", # Optional. SCHEMA FIELDS FOR TYPE STRING Minimum length of the Type.STRING
            "minProperties": "A String", # Optional. Minimum number of the properties for Type.OBJECT.
            "minimum": 3.14, # Optional. SCHEMA FIELDS FOR TYPE INTEGER and NUMBER Minimum value of the Type.INTEGER and Type.NUMBER
            "nullable": True or False, # Optional. Indicates if the value may be null.
            "pattern": "A String", # Optional. Pattern of the Type.STRING to restrict a string to a regular expression.
            "properties": { # Optional. SCHEMA FIELDS FOR TYPE OBJECT Properties of Type.OBJECT.
              "a_key": # Object with schema name: GoogleCloudAiplatformV1Schema
            },
            "required": [ # Optional. Required properties of Type.OBJECT.
              "A String",
            ],
            "title": "A String", # Optional. The title of the Schema.
            "type": "A String", # Optional. The type of the data.
          },
        },
      ],
      "googleSearchRetrieval": { # Tool to retrieve public web data for grounding, powered by Google. # Optional. GoogleSearchRetrieval tool type. Specialized retrieval tool that is powered by Google search.
      },
      "retrieval": { # Defines a retrieval tool that model can call to access external knowledge. # Optional. Retrieval tool type. System will always execute the provided retrieval tool(s) to get external knowledge to answer the prompt. Retrieval results are presented to the model for generation.
        "disableAttribution": True or False, # Optional. Deprecated. This option is no longer supported.
        "vertexAiSearch": { # Retrieve from Vertex AI Search datastore for grounding. See https://cloud.google.com/products/agent-builder # Set to use data source powered by Vertex AI Search.
          "datastore": "A String", # Required. Fully-qualified Vertex AI Search data store resource ID. Format: `projects/{project}/locations/{location}/collections/{collection}/dataStores/{dataStore}`
        },
        "vertexRagStore": { # Retrieve from Vertex RAG Store for grounding. # Set to use data source powered by Vertex RAG store. User data is uploaded via the VertexRagDataService.
          "ragCorpora": [ # Optional. Deprecated. Please use rag_resources instead.
            "A String",
          ],
          "ragResources": [ # Optional. The representation of the rag source. It can be used to specify corpus only or ragfiles. Currently only support one corpus or multiple files from one corpus. In the future we may open up multiple corpora support.
            { # The definition of the Rag resource.
              "ragCorpus": "A String", # Optional. RagCorpora resource name. Format: `projects/{project}/locations/{location}/ragCorpora/{rag_corpus}`
              "ragFileIds": [ # Optional. rag_file_id. The files should be in the same rag_corpus set in rag_corpus field.
                "A String",
              ],
            },
          ],
          "similarityTopK": 42, # Optional. Number of top k results to return from the selected corpora.
          "vectorDistanceThreshold": 3.14, # Optional. Only return results with vector distance smaller than the threshold.
        },
      },
    },
  ],
}

  x__xgafv: string, V1 error format.
    Allowed values
      1 - v1 error format
      2 - v2 error format

Returns:
  An object of the form:

    { # Response message for [PredictionService.GenerateContent].
  "candidates": [ # Output only. Generated candidates.
    { # A response candidate generated from the model.
      "avgLogprobs": 3.14, # Output only. Average log probability score of the candidate.
      "citationMetadata": { # A collection of source attributions for a piece of content. # Output only. Source attribution of the generated content.
        "citations": [ # Output only. List of citations.
          { # Source attributions for content.
            "endIndex": 42, # Output only. End index into the content.
            "license": "A String", # Output only. License of the attribution.
            "publicationDate": { # Represents a whole or partial calendar date, such as a birthday. The time of day and time zone are either specified elsewhere or are insignificant. The date is relative to the Gregorian Calendar. This can represent one of the following: * A full date, with non-zero year, month, and day values. * A month and day, with a zero year (for example, an anniversary). * A year on its own, with a zero month and a zero day. * A year and month, with a zero day (for example, a credit card expiration date). Related types: * google.type.TimeOfDay * google.type.DateTime * google.protobuf.Timestamp # Output only. Publication date of the attribution.
              "day": 42, # Day of a month. Must be from 1 to 31 and valid for the year and month, or 0 to specify a year by itself or a year and month where the day isn't significant.
              "month": 42, # Month of a year. Must be from 1 to 12, or 0 to specify a year without a month and day.
              "year": 42, # Year of the date. Must be from 1 to 9999, or 0 to specify a date without a year.
            },
            "startIndex": 42, # Output only. Start index into the content.
            "title": "A String", # Output only. Title of the attribution.
            "uri": "A String", # Output only. Url reference of the attribution.
          },
        ],
      },
      "content": { # The base structured datatype containing multi-part content of a message. A `Content` includes a `role` field designating the producer of the `Content` and a `parts` field containing multi-part data that contains the content of the message turn. # Output only. Content parts of the candidate.
        "parts": [ # Required. Ordered `Parts` that constitute a single message. Parts may have different IANA MIME types.
          { # A datatype containing media that is part of a multi-part `Content` message. A `Part` consists of data which has an associated datatype. A `Part` can only contain one of the accepted types in `Part.data`. A `Part` must have a fixed IANA MIME type identifying the type and subtype of the media if `inline_data` or `file_data` field is filled with raw bytes.
            "fileData": { # URI based data. # Optional. URI based data.
              "fileUri": "A String", # Required. URI.
              "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
            },
            "functionCall": { # A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values. # Optional. A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] with the parameters and their values.
              "args": { # Optional. Required. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.
                "a_key": "", # Properties of the object.
              },
              "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name].
            },
            "functionResponse": { # The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction. # Optional. The result output of a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function call. It is used as context to the model.
              "name": "A String", # Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].
              "response": { # Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.
                "a_key": "", # Properties of the object.
              },
            },
            "inlineData": { # Content blob. It's preferred to send as text directly rather than raw bytes. # Optional. Inlined bytes data.
              "data": "A String", # Required. Raw bytes.
              "mimeType": "A String", # Required. The IANA standard MIME type of the source data.
            },
            "text": "A String", # Optional. Text part (can be code).
            "videoMetadata": { # Metadata describes the input video content. # Optional. Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.
              "endOffset": "A String", # Optional. The end offset of the video.
              "startOffset": "A String", # Optional. The start offset of the video.
            },
          },
        ],
        "role": "A String", # Optional. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
      },
      "finishMessage": "A String", # Output only. Describes the reason the mode stopped generating tokens in more detail. This is only filled when `finish_reason` is set.
      "finishReason": "A String", # Output only. The reason why the model stopped generating tokens. If empty, the model has not stopped generating the tokens.
      "groundingMetadata": { # Metadata returned to client when grounding is enabled. # Output only. Metadata specifies sources used to ground generated content.
        "groundingChunks": [ # List of supporting references retrieved from specified grounding source.
          { # Grounding chunk.
            "retrievedContext": { # Chunk from context retrieved by the retrieval tools. # Grounding chunk from context retrieved by the retrieval tools.
              "title": "A String", # Title of the attribution.
              "uri": "A String", # URI reference of the attribution.
            },
            "web": { # Chunk from the web. # Grounding chunk from the web.
              "title": "A String", # Title of the chunk.
              "uri": "A String", # URI reference of the chunk.
            },
          },
        ],
        "groundingSupports": [ # Optional. List of grounding support.
          { # Grounding support.
            "confidenceScores": [ # Confidence score of the support references. Ranges from 0 to 1. 1 is the most confident. This list must have the same size as the grounding_chunk_indices.
              3.14,
            ],
            "groundingChunkIndices": [ # A list of indices (into 'grounding_chunk') specifying the citations associated with the claim. For instance [1,3,4] means that grounding_chunk[1], grounding_chunk[3], grounding_chunk[4] are the retrieved content attributed to the claim.
              42,
            ],
            "segment": { # Segment of the content. # Segment of the content this support belongs to.
              "endIndex": 42, # Output only. End index in the given Part, measured in bytes. Offset from the start of the Part, exclusive, starting at zero.
              "partIndex": 42, # Output only. The index of a Part object within its parent Content object.
              "startIndex": 42, # Output only. Start index in the given Part, measured in bytes. Offset from the start of the Part, inclusive, starting at zero.
              "text": "A String", # Output only. The text corresponding to the segment from the response.
            },
          },
        ],
        "searchEntryPoint": { # Google search entry point. # Optional. Google search entry for the following-up web searches.
          "renderedContent": "A String", # Optional. Web content snippet that can be embedded in a web page or an app webview.
          "sdkBlob": "A String", # Optional. Base64 encoded JSON representing array of tuple.
        },
        "webSearchQueries": [ # Optional. Web search queries for the following-up web search.
          "A String",
        ],
      },
      "index": 42, # Output only. Index of the candidate.
      "safetyRatings": [ # Output only. List of ratings for the safety of a response candidate. There is at most one rating per category.
        { # Safety rating corresponding to the generated content.
          "blocked": True or False, # Output only. Indicates whether the content was filtered out because of this rating.
          "category": "A String", # Output only. Harm category.
          "probability": "A String", # Output only. Harm probability levels in the content.
          "probabilityScore": 3.14, # Output only. Harm probability score.
          "severity": "A String", # Output only. Harm severity levels in the content.
          "severityScore": 3.14, # Output only. Harm severity score.
        },
      ],
    },
  ],
  "promptFeedback": { # Content filter results for a prompt sent in the request. # Output only. Content filter results for a prompt sent in the request. Note: Sent only in the first stream chunk. Only happens when no candidates were generated due to content violations.
    "blockReason": "A String", # Output only. Blocked reason.
    "blockReasonMessage": "A String", # Output only. A readable block reason message.
    "safetyRatings": [ # Output only. Safety ratings.
      { # Safety rating corresponding to the generated content.
        "blocked": True or False, # Output only. Indicates whether the content was filtered out because of this rating.
        "category": "A String", # Output only. Harm category.
        "probability": "A String", # Output only. Harm probability levels in the content.
        "probabilityScore": 3.14, # Output only. Harm probability score.
        "severity": "A String", # Output only. Harm severity levels in the content.
        "severityScore": 3.14, # Output only. Harm severity score.
      },
    ],
  },
  "usageMetadata": { # Usage metadata about response(s). # Usage metadata about the response(s).
    "candidatesTokenCount": 42, # Number of tokens in the response(s).
    "promptTokenCount": 42, # Number of tokens in the request. When `cached_content` is set, this is still the total effective prompt size meaning this includes the number of tokens in the cached content.
    "totalTokenCount": 42, # Total token count for prompt and response candidates.
  },
}

Vertex AI API . publishers . models

Instance Methods

Method Details