Close httplib2 connections.
delete(projectId, datasetId, tableId, x__xgafv=None)
Deletes the table specified by tableId from the dataset. If the table contains data, all the data will be deleted.
get(projectId, datasetId, tableId, selectedFields=None, view=None, x__xgafv=None)
Gets the specified table resource by table ID. This method does not return the data in the table, it only returns the table resource, which describes the structure of this table.
getIamPolicy(resource, body=None, x__xgafv=None)
Gets the access control policy for a resource. Returns an empty policy if the resource exists and does not have a policy set.
insert(projectId, datasetId, body=None, x__xgafv=None)
Creates a new, empty table in the dataset.
list(projectId, datasetId, maxResults=None, pageToken=None, x__xgafv=None)
Lists all tables in the specified dataset. Requires the READER dataset role.
Retrieves the next page of results.
patch(projectId, datasetId, tableId, autodetect_schema=None, body=None, x__xgafv=None)
Updates information in an existing table. The update method replaces the entire table resource, whereas the patch method only replaces fields that are provided in the submitted table resource. This method supports RFC5789 patch semantics.
setIamPolicy(resource, body=None, x__xgafv=None)
Sets the access control policy on the specified resource. Replaces any existing policy. Can return `NOT_FOUND`, `INVALID_ARGUMENT`, and `PERMISSION_DENIED` errors.
testIamPermissions(resource, body=None, x__xgafv=None)
Returns permissions that a caller has on the specified resource. If the resource does not exist, this will return an empty set of permissions, not a `NOT_FOUND` error. Note: This operation is designed to be used for building permission-aware UIs and command-line tools, not for authorization checking. This operation may "fail open" without warning.
update(projectId, datasetId, tableId, autodetect_schema=None, body=None, x__xgafv=None)
Updates information in an existing table. The update method replaces the entire Table resource, whereas the patch method only replaces fields that are provided in the submitted Table resource.
close()
Close httplib2 connections.
delete(projectId, datasetId, tableId, x__xgafv=None)
Deletes the table specified by tableId from the dataset. If the table contains data, all the data will be deleted.
Args:
projectId: string, Required. Project ID of the table to delete (required)
datasetId: string, Required. Dataset ID of the table to delete (required)
tableId: string, Required. Table ID of the table to delete (required)
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
get(projectId, datasetId, tableId, selectedFields=None, view=None, x__xgafv=None)
Gets the specified table resource by table ID. This method does not return the data in the table, it only returns the table resource, which describes the structure of this table.
Args:
projectId: string, Required. Project ID of the requested table (required)
datasetId: string, Required. Dataset ID of the requested table (required)
tableId: string, Required. Table ID of the requested table (required)
selectedFields: string, List of table schema fields to return (comma-separated). If unspecified, all fields are returned. A fieldMask cannot be used here because the fields will automatically be converted from camelCase to snake_case and the conversion will fail if there are underscores. Since these are fields in BigQuery table schemas, underscores are allowed.
view: string, Optional. Specifies the view that determines which table information is returned. By default, basic table information and storage statistics (STORAGE_STATS) are returned.
Allowed values
TABLE_METADATA_VIEW_UNSPECIFIED - The default value. Default to the STORAGE_STATS view.
BASIC - Includes basic table information including schema and partitioning specification. This view does not include storage statistics such as numRows or numBytes. This view is significantly more efficient and should be used to support high query rates.
STORAGE_STATS - Includes all information in the BASIC view as well as storage statistics (numBytes, numLongTermBytes, numRows and lastModifiedTime).
FULL - Includes all table information, including storage statistics. It returns same information as STORAGE_STATS view, but may contain additional information in the future.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{
"biglakeConfiguration": { # Configuration for BigLake managed tables. # Optional. Specifies the configuration of a BigLake managed table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read and write to external storage, such as Cloud Storage. The connection_id can have the form `{project}.{location}.{connection_id}` or `projects/{project}/locations/{location}/connections/{connection_id}".
"fileFormat": "A String", # Optional. The file format the table data is stored in.
"storageUri": "A String", # Optional. The fully qualified location prefix of the external folder where table data is stored. The '*' wildcard character is not allowed. The URI should be in the format `gs://bucket/path_to_table/`
"tableFormat": "A String", # Optional. The table format the metadata only snapshots are stored in.
},
"cloneDefinition": { # Information about base table and clone time of a table clone. # Output only. Contains information about the clone. This value is set via the clone operation.
"baseTableReference": { # Required. Reference describing the ID of the table that was cloned.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"cloneTime": "A String", # Required. The time at which the base table was cloned. This value is reported in the JSON response using RFC3339 format.
},
"clustering": { # Configures table clustering. # Clustering specification for the table. Must be specified with time-based partitioning, data in the table will be first partitioned and subsequently clustered.
"fields": [ # One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. The ordering of the clustering fields should be prioritized from most to least important for filtering purposes. Additional information on limitations can be found here: https://cloud.google.com/bigquery/docs/creating-clustered-tables#limitations
"A String",
],
},
"creationTime": "A String", # Output only. The time when this table was created, in milliseconds since the epoch.
"defaultCollation": "A String", # Optional. Defines the default collation specification of new STRING fields in the table. During table creation or update, if a STRING field is added to this table without explicit collation specified, then the table inherits the table default collation. A change to this field affects only fields added afterwards, and does not alter the existing fields. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"defaultRoundingMode": "A String", # Optional. Defines the default rounding mode specification of new decimal fields (NUMERIC OR BIGNUMERIC) in the table. During table creation or update, if a decimal field is added to this table without an explicit rounding mode specified, then the field inherits the table default rounding mode. Changing this field doesn't affect existing fields.
"description": "A String", # Optional. A user-friendly description of this table.
"encryptionConfiguration": { # Configuration for Cloud KMS encryption settings. # Custom encryption configuration (e.g., Cloud KMS keys).
"kmsKeyName": "A String", # Optional. Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.
},
"etag": "A String", # Output only. A hash of this resource.
"expirationTime": "A String", # Optional. The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed. The defaultTableExpirationMs property of the encapsulating dataset can be used to set a default expirationTime on newly created tables.
"externalCatalogTableOptions": { # Metadata about open source compatible table. The fields contained in these options correspond to hive metastore's table level properties. # Optional. Options defining open source compatible table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection is needed to read the open source table from BigQuery Engine. The connection_id can have the form `..` or `projects//locations//connections/`.
"parameters": { # Optional. A map of key value pairs defining the parameters and properties of the open source table. Corresponds with hive meta store table parameters. Maximum size of 4Mib.
"a_key": "A String",
},
"storageDescriptor": { # Contains information about how a table's data is stored and accessed by open source query engines. # Optional. A storage descriptor containing information about the physical storage of this table.
"inputFormat": "A String", # Optional. Specifies the fully qualified class name of the InputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"). The maximum length is 128 characters.
"locationUri": "A String", # Optional. The physical location of the table (e.g. `gs://spark-dataproc-data/pangea-data/case_sensitive/` or `gs://spark-dataproc-data/pangea-data/*`). The maximum length is 2056 bytes.
"outputFormat": "A String", # Optional. Specifies the fully qualified class name of the OutputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"). The maximum length is 128 characters.
"serdeInfo": { # Serializer and deserializer information. # Optional. Serializer and deserializer information.
"name": "A String", # Optional. Name of the SerDe. The maximum length is 256 characters.
"parameters": { # Optional. Key-value pairs that define the initialization parameters for the serialization library. Maximum size 10 Kib.
"a_key": "A String",
},
"serializationLibrary": "A String", # Required. Specifies a fully-qualified class name of the serialization library that is responsible for the translation of data between table representation and the underlying low-level input and output format structures. The maximum length is 256 characters.
},
},
},
"externalDataConfiguration": { # Optional. Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table.
"autodetect": True or False, # Try to detect schema and format options automatically. Any option specified explicitly will be honored.
"avroOptions": { # Options for external data sources. # Optional. Additional properties to set if sourceFormat is set to AVRO.
"useAvroLogicalTypes": True or False, # Optional. If sourceFormat is set to "AVRO", indicates whether to interpret logical types as the corresponding BigQuery data type (for example, TIMESTAMP), instead of using the raw type (for example, INTEGER).
},
"bigtableOptions": { # Options specific to Google Cloud Bigtable data sources. # Optional. Additional options if sourceFormat is set to BIGTABLE.
"columnFamilies": [ # Optional. List of column families to expose in the table schema along with their types. This list restricts the column families that can be referenced in queries and specifies their value types. You can use this list to do type conversions - see the 'type' field for more details. If you leave this list empty, all column families are present in the table schema and their values are read as BYTES. During a query only the column families referenced in that query are read from Bigtable.
{ # Information related to a Bigtable column family.
"columns": [ # Optional. Lists of columns that should be exposed as individual fields as opposed to a list of (column name, value) pairs. All columns whose qualifier matches a qualifier in this list can be accessed as `.`. Other columns can be accessed as a list through the `.Column` field.
{ # Information related to a Bigtable column.
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. 'encoding' can also be set at the column family level. However, the setting at this level takes precedence if 'encoding' is set at both levels.
"fieldName": "A String", # Optional. If the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as the column field name and is used as field name in queries.
"onlyReadLatest": True or False, # Optional. If this is set, only the latest version of value in this column are exposed. 'onlyReadLatest' can also be set at the column family level. However, the setting at this level takes precedence if 'onlyReadLatest' is set at both levels.
"qualifierEncoded": "A String", # [Required] Qualifier of the column. Columns in the parent column family that has this exact qualifier are exposed as `.` field. If the qualifier is valid UTF-8 string, it can be specified in the qualifier_string field. Otherwise, a base-64 encoded value must be set to qualifier_encoded. The column field name is the same as the column qualifier. However, if the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as field_name.
"qualifierString": "A String", # Qualifier string.
"type": "A String", # Optional. The type to convert the value in cells of this column. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. 'type' can also be set at the column family level. However, the setting at this level takes precedence if 'type' is set at both levels.
},
],
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. This can be overridden for a specific column by listing that column in 'columns' and specifying an encoding for it.
"familyId": "A String", # Identifier of the column family.
"onlyReadLatest": True or False, # Optional. If this is set only the latest version of value are exposed for all columns in this column family. This can be overridden for a specific column by listing that column in 'columns' and specifying a different setting for that column.
"type": "A String", # Optional. The type to convert the value in cells of this column family. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. This can be overridden for a specific column by listing that column in 'columns' and specifying a type for it.
},
],
"ignoreUnspecifiedColumnFamilies": True or False, # Optional. If field is true, then the column families that are not specified in columnFamilies list are not exposed in the table schema. Otherwise, they are read with BYTES type values. The default value is false.
"outputColumnFamiliesAsJson": True or False, # Optional. If field is true, then each column family will be read as a single JSON column. Otherwise they are read as a repeated cell structure containing timestamp/value tuples. The default value is false.
"readRowkeyAsString": True or False, # Optional. If field is true, then the rowkey column families will be read and converted to string. Otherwise they are read with BYTES type values and users need to manually cast them with CAST if necessary. The default value is false.
},
"compression": "A String", # Optional. The compression type of the data source. Possible values include GZIP and NONE. The default value is NONE. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats. An empty string is an invalid value.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id can have the form `{project_id}.{location_id};{connection_id}` or `projects/{project_id}/locations/{location_id}/connections/{connection_id}`.
"csvOptions": { # Information related to a CSV data source. # Optional. Additional properties to set if sourceFormat is set to CSV.
"allowJaggedRows": True or False, # Optional. Indicates if BigQuery should accept rows that are missing trailing optional columns. If true, BigQuery treats missing trailing columns as null values. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.
"allowQuotedNewlines": True or False, # Optional. Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, ISO-8859-1, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.
"fieldDelimiter": "A String", # Optional. The separator character for fields in a CSV file. The separator is interpreted as a single byte. For files encoded in ISO-8859-1, any single character can be used as a separator. For files encoded in UTF-8, characters represented in decimal range 1-127 (U+0001-U+007F) can be used without any modification. UTF-8 characters encoded with multiple bytes (i.e. U+0080 and above) will have only the first byte used for separating fields. The remaining bytes will be treated as a part of the field. BigQuery also supports the escape sequence "\t" (U+0009) to specify a tab separator. The default value is comma (",", U+002C).
"nullMarker": "A String", # Optional. Specifies a string that represents a null value in a CSV file. For example, if you specify "\N", BigQuery interprets "\N" as a null value when querying a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.
"preserveAsciiControlCharacters": True or False, # Optional. Indicates if the embedded ASCII control characters (the first 32 characters in the ASCII-table, from '\x00' to '\x1F') are preserved.
"quote": """, # Optional. The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote ("). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true. To include the specific quote character within a quoted value, precede it with an additional matching quote character. For example, if you want to escape the default character ' " ', use ' "" '.
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a CSV file that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"decimalTargetTypes": [ # Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown. Example: Suppose the value of this field is ["NUMERIC", "BIGNUMERIC"]. If (precision,scale) is: * (38,9) -> NUMERIC; * (39,9) -> BIGNUMERIC (NUMERIC cannot hold 30 integer digits); * (38,10) -> BIGNUMERIC (NUMERIC cannot hold 10 fractional digits); * (76,38) -> BIGNUMERIC; * (77,38) -> BIGNUMERIC (error if value exeeds supported range). This field cannot contain duplicate types. The order of the types in this field is ignored. For example, ["BIGNUMERIC", "NUMERIC"] is the same as ["NUMERIC", "BIGNUMERIC"] and NUMERIC always takes precedence over BIGNUMERIC. Defaults to ["NUMERIC", "STRING"] for ORC and ["NUMERIC"] for the other file formats.
"A String",
],
"fileSetSpecType": "A String", # Optional. Specifies how source URIs are interpreted for constructing the file set to load. By default source URIs are expanded against the underlying storage. Other options include specifying manifest files. Only applicable to object storage systems.
"googleSheetsOptions": { # Options specific to Google Sheets data sources. # Optional. Additional options if sourceFormat is set to GOOGLE_SHEETS.
"range": "A String", # Optional. Range of a sheet to query from. Only used when non-empty. Typical format: sheet_name!top_left_cell_id:bottom_right_cell_id For example: sheet1!A1:B20
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a sheet that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"hivePartitioningOptions": { # Options for configuring hive partitioning detect. # Optional. When set, configures hive partitioning support. Not all storage formats support hive partitioning -- requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.
"fields": [ # Output only. For permanent external tables, this field is populated with the hive partition keys in the order they were inferred. The types of the partition keys can be deduced by checking the table schema (which will include the partition keys). Not every API will populate this field in the output. For example, Tables.Get will populate it, but Tables.List will not contain this field.
"A String",
],
"mode": "A String", # Optional. When set, what mode of hive partitioning to use when reading data. The following modes are supported: * AUTO: automatically infer partition key name(s) and type(s). * STRINGS: automatically infer partition key name(s). All types are strings. * CUSTOM: partition key schema is encoded in the source URI prefix. Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. Note that this field should only be true when creating a permanent external table or querying a temporary external table. Hive-partitioned loads with require_partition_filter explicitly set to true will fail.
"sourceUriPrefix": "A String", # Optional. When hive partition detection is requested, a common prefix for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout: gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. CUSTOM detection requires encoding the partitioning schema immediately after the common prefix. For CUSTOM, any of * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:STRING}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:STRING} would all be valid source URI prefixes.
},
"ignoreUnknownValues": True or False, # Optional. Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don't match any column names Google Cloud Bigtable: This setting is ignored. Google Cloud Datastore backups: This setting is ignored. Avro: This setting is ignored. ORC: This setting is ignored. Parquet: This setting is ignored.
"jsonExtension": "A String", # Optional. Load option to be used together with source_format newline-delimited JSON to indicate that a variant of JSON is being loaded. To load newline-delimited GeoJSON, specify GEOJSON (and source_format must be set to NEWLINE_DELIMITED_JSON).
"jsonOptions": { # Json Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to JSON.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8.
},
"maxBadRecords": 42, # Optional. The maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats.
"metadataCacheMode": "A String", # Optional. Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source.
"objectMetadata": "A String", # Optional. ObjectMetadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the source_uris. If ObjectMetadata is set, source_format should be omitted. Currently SIMPLE is the only supported Object Metadata type.
"parquetOptions": { # Parquet Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to PARQUET.
"enableListInference": True or False, # Optional. Indicates whether to use schema inference specifically for Parquet LIST logical type.
"enumAsString": True or False, # Optional. Indicates whether to infer Parquet ENUM logical type as STRING instead of BYTES by default.
"mapTargetType": "A String", # Optional. Indicates how to represent a Parquet map if present.
},
"referenceFileSchemaUri": "A String", # Optional. When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.
"schema": { # Schema of a table # Optional. The schema for the data. Schema is required for CSV and JSON formats if autodetect is not on. Schema is disallowed for Google Cloud Bigtable, Cloud Datastore backups, Avro, ORC and Parquet formats.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"sourceFormat": "A String", # [Required] The data format. For CSV files, specify "CSV". For Google sheets, specify "GOOGLE_SHEETS". For newline-delimited JSON, specify "NEWLINE_DELIMITED_JSON". For Avro files, specify "AVRO". For Google Cloud Datastore backups, specify "DATASTORE_BACKUP". For Apache Iceberg tables, specify "ICEBERG". For ORC files, specify "ORC". For Parquet files, specify "PARQUET". [Beta] For Google Cloud Bigtable, specify "BIGTABLE".
"sourceUris": [ # [Required] The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one '*' wildcard character and it must come after the 'bucket' name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the '*' wildcard character is not allowed.
"A String",
],
},
"friendlyName": "A String", # Optional. A descriptive name for this table.
"id": "A String", # Output only. An opaque ID uniquely identifying the table.
"kind": "bigquery#table", # The type of resource ID.
"labels": { # The labels associated with this table. You can use these to organize and group your tables. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.
"a_key": "A String",
},
"lastModifiedTime": "A String", # Output only. The time when this table was last modified, in milliseconds since the epoch.
"location": "A String", # Output only. The geographic location where the table resides. This value is inherited from the dataset.
"materializedView": { # Definition and configuration of a materialized view. # Optional. The materialized view definition.
"allowNonIncrementalDefinition": True or False, # Optional. This option declares the intention to construct a materialized view that isn't refreshed incrementally.
"enableRefresh": True or False, # Optional. Enable automatic refresh of the materialized view when the base table is updated. The default value is "true".
"lastRefreshTime": "A String", # Output only. The time when this materialized view was last refreshed, in milliseconds since the epoch.
"maxStaleness": "A String", # [Optional] Max staleness of data that could be returned when materizlized view is queried (formatted as Google SQL Interval type).
"query": "A String", # Required. A query whose results are persisted.
"refreshIntervalMs": "A String", # Optional. The maximum frequency at which this materialized view will be refreshed. The default value is "1800000" (30 minutes).
},
"materializedViewStatus": { # Status of a materialized view. The last refresh timestamp status is omitted here, but is present in the MaterializedViewDefinition message. # Output only. The materialized view status.
"lastRefreshStatus": { # Error details. # Output only. Error result of the last automatic refresh. If present, indicates that the last automatic refresh was unsuccessful.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"refreshWatermark": "A String", # Output only. Refresh watermark of materialized view. The base tables' data were collected into the materialized view cache until this time.
},
"maxStaleness": "A String", # Optional. The maximum staleness of data that could be returned when the table (or stale MV) is queried. Staleness encoded as a string encoding of sql IntervalValue type.
"model": { # Deprecated.
"modelOptions": { # Deprecated.
"labels": [
"A String",
],
"lossType": "A String",
"modelType": "A String",
},
"trainingRuns": [ # Deprecated.
{
"iterationResults": [ # Deprecated.
{
"durationMs": "A String", # Deprecated.
"evalLoss": 3.14, # Deprecated.
"index": 42, # Deprecated.
"learnRate": 3.14, # Deprecated.
"trainingLoss": 3.14, # Deprecated.
},
],
"startTime": "A String", # Deprecated.
"state": "A String", # Deprecated.
"trainingOptions": { # Deprecated.
"earlyStop": True or False,
"l1Reg": 3.14,
"l2Reg": 3.14,
"learnRate": 3.14,
"learnRateStrategy": "A String",
"lineSearchInitLearnRate": 3.14,
"maxIteration": "A String",
"minRelProgress": 3.14,
"warmStart": True or False,
},
},
],
},
"numActiveLogicalBytes": "A String", # Output only. Number of logical bytes that are less than 90 days old.
"numActivePhysicalBytes": "A String", # Output only. Number of physical bytes less than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numBytes": "A String", # Output only. The size of this table in logical bytes, excluding any data in the streaming buffer.
"numCurrentPhysicalBytes": "A String", # Output only. Number of physical bytes used by current live data storage. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numLongTermBytes": "A String", # Output only. The number of logical bytes in the table that are considered "long-term storage".
"numLongTermLogicalBytes": "A String", # Output only. Number of logical bytes that are more than 90 days old.
"numLongTermPhysicalBytes": "A String", # Output only. Number of physical bytes more than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPartitions": "A String", # Output only. The number of partitions present in the table or materialized view. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This includes storage used for time travel.
"numRows": "A String", # Output only. The number of rows of data in this table, excluding any data in the streaming buffer.
"numTimeTravelPhysicalBytes": "A String", # Output only. Number of physical bytes used by time travel storage (deleted or changed data). This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numTotalLogicalBytes": "A String", # Output only. Total number of logical bytes in the table or materialized view.
"numTotalPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This also includes storage used for time travel. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"partitionDefinition": { # The partitioning information, which includes managed table, external table and metastore partitioned table partition information. # Optional. The partition information for all table formats, including managed partitioned tables, hive partitioned tables, iceberg partitioned, and metastore partitioned tables. This field is only populated for metastore partitioned tables. For other table formats, this is an output only field.
"partitionedColumn": [ # Optional. Details about each partitioning column. This field is output only for all partitioning types other than metastore partitioned tables. BigQuery native tables only support 1 partitioning column. Other table types may support 0, 1 or more partitioning columns. For metastore partitioned tables, the order must match the definition order in the Hive Metastore, where it must match the physical layout of the table. For example, CREATE TABLE a_table(id BIGINT, name STRING) PARTITIONED BY (city STRING, state STRING). In this case the values must be ['city', 'state'] in that order.
{ # The partitioning column information.
"field": "A String", # Required. The name of the partition column.
},
],
},
"rangePartitioning": { # If specified, configures range partitioning for this table.
"field": "A String", # Required. The name of the column to partition the table on. It must be a top-level, INT64 column whose mode is NULLABLE or REQUIRED.
"range": { # [Experimental] Defines the ranges for range partitioning.
"end": "A String", # [Experimental] The end of range partitioning, exclusive.
"interval": "A String", # [Experimental] The width of each interval.
"start": "A String", # [Experimental] The start of range partitioning, inclusive.
},
},
"replicas": [ # Optional. Output only. Table references of all replicas currently active on the table.
{
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
],
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.
"resourceTags": { # [Optional] The tags associated with this table. Tag keys are globally unique. See additional information on [tags](https://cloud.google.com/iam/docs/tags-access-control#definitions). An object containing a list of "key": value pairs. The key is the namespaced friendly name of the tag key, e.g. "12345/environment" where 12345 is parent id. The value is the friendly short name of the tag value, e.g. "production".
"a_key": "A String",
},
"restrictions": { # Optional. Output only. Restriction config for table. If set, restrict certain accesses on the table based on the config. See [Data egress](https://cloud.google.com/bigquery/docs/analytics-hub-introduction#data_egress) for more details.
"type": "A String", # Output only. Specifies the type of dataset/table restriction.
},
"schema": { # Schema of a table # Optional. Describes the schema of this table.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"selfLink": "A String", # Output only. A URL that can be used to access this resource again.
"snapshotDefinition": { # Information about base table and snapshot time of the snapshot. # Output only. Contains information about the snapshot. This value is set via snapshot creation.
"baseTableReference": { # Required. Reference describing the ID of the table that was snapshot.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"snapshotTime": "A String", # Required. The time at which the base table was snapshot. This value is reported in the JSON response using RFC3339 format.
},
"streamingBuffer": { # Output only. Contains information regarding this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
"estimatedBytes": "A String", # Output only. A lower-bound estimate of the number of bytes currently in the streaming buffer.
"estimatedRows": "A String", # Output only. A lower-bound estimate of the number of rows currently in the streaming buffer.
"oldestEntryTime": "A String", # Output only. Contains the timestamp of the oldest entry in the streaming buffer, in milliseconds since the epoch, if the streaming buffer is available.
},
"tableConstraints": { # The TableConstraints defines the primary key and foreign key. # Optional. Tables Primary Key and Foreign Key information
"foreignKeys": [ # Optional. Present only if the table has a foreign key. The foreign key is not enforced.
{ # Represents a foreign key constraint on a table's columns.
"columnReferences": [ # Required. The columns that compose the foreign key.
{ # The pair of the foreign key column and primary key column.
"referencedColumn": "A String", # Required. The column in the primary key that are referenced by the referencing_column.
"referencingColumn": "A String", # Required. The column that composes the foreign key.
},
],
"name": "A String", # Optional. Set only if the foreign key constraint is named.
"referencedTable": {
"datasetId": "A String",
"projectId": "A String",
"tableId": "A String",
},
},
],
"primaryKey": { # Represents the primary key constraint on a table's columns.
"columns": [ # Required. The columns that are composed of the primary key constraint.
"A String",
],
},
},
"tableReference": { # Required. Reference describing the ID of this table.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"tableReplicationInfo": { # Replication info of a table created using `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv` # Optional. Table replication info for table created `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv`
"replicatedSourceLastRefreshTime": "A String", # Optional. Output only. If source is a materialized view, this field signifies the last refresh time of the source.
"replicationError": { # Error details. # Optional. Output only. Replication error that will permanently stopped table replication.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"replicationIntervalMs": "A String", # Optional. Specifies the interval at which the source table is polled for updates. It's Optional. If not specified, default replication interval would be applied.
"replicationStatus": "A String", # Optional. Output only. Replication status of configured replication.
"sourceTable": { # Required. Source table reference that is replicated.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
},
"timePartitioning": { # If specified, configures time-based partitioning for this table.
"expirationMs": "A String", # Optional. Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.
"field": "A String", # Optional. If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.
"requirePartitionFilter": false, # If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. This field is deprecated; please set the field with the same name on the table itself instead. This field needs a wrapper because we want to output the default value, false, if the user explicitly set it.
"type": "A String", # Required. The supported types are DAY, HOUR, MONTH, and YEAR, which will generate one partition per day, hour, month, and year, respectively.
},
"type": "A String", # Output only. Describes the table type. The following values are supported: * `TABLE`: A normal BigQuery table. * `VIEW`: A virtual table defined by a SQL query. * `EXTERNAL`: A table that references data stored in an external storage system, such as Google Cloud Storage. * `MATERIALIZED_VIEW`: A precomputed view defined by a SQL query. * `SNAPSHOT`: An immutable BigQuery table that preserves the contents of a base table at a particular time. See additional information on [table snapshots](https://cloud.google.com/bigquery/docs/table-snapshots-intro). The default value is `TABLE`.
"view": { # Describes the definition of a logical view. # Optional. The view definition.
"foreignDefinitions": [ # Optional. Foreign view representations.
{ # A view can be represented in multiple ways. Each representation has its own dialect. This message stores the metadata required for these representations.
"dialect": "A String", # Optional. Represents the dialect of the query.
"query": "A String", # Required. The query that defines the view.
},
],
"privacyPolicy": { # Represents privacy policy that contains the privacy requirements specified by the data owner. Currently, this is only supported on views. # Optional. Specifices the privacy policy for the view.
"aggregationThresholdPolicy": { # Represents privacy policy associated with "aggregation threshold" method. # Optional. Policy used for aggregation thresholds.
"privacyUnitColumns": [ # Optional. The privacy unit column(s) associated with this policy. For now, only one column per data source object (table, view) is allowed as a privacy unit column. Representing as a repeated field in metadata for extensibility to multiple columns in future. Duplicates and Repeated struct fields are not allowed. For nested fields, use dot notation ("outer.inner")
"A String",
],
"threshold": "A String", # Optional. The threshold for the "aggregation threshold" policy.
},
"differentialPrivacyPolicy": { # Represents privacy policy associated with "differential privacy" method. # Optional. Policy used for differential privacy.
"deltaBudget": 3.14, # Optional. The total delta budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of delta that is pre-defined by the contributor through the privacy policy delta_per_query field. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"deltaBudgetRemaining": 3.14, # Output only. The delta budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"deltaPerQuery": 3.14, # Optional. The delta value that is used per query. Delta represents the probability that any row will fail to be epsilon differentially private. Indicates the risk associated with exposing aggregate rows in the result of a query.
"epsilonBudget": 3.14, # Optional. The total epsilon budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of epsilon they request in their query. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"epsilonBudgetRemaining": 3.14, # Output only. The epsilon budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"maxEpsilonPerQuery": 3.14, # Optional. The maximum epsilon value that a query can consume. If the subscriber specifies epsilon as a parameter in a SELECT query, it must be less than or equal to this value. The epsilon parameter controls the amount of noise that is added to the groups — a higher epsilon means less noise.
"maxGroupsContributed": "A String", # Optional. The maximum groups contributed value that is used per query. Represents the maximum number of groups to which each protected entity can contribute. Changing this value does not improve or worsen privacy. The best value for accuracy and utility depends on the query and data.
"privacyUnitColumn": "A String", # Optional. The privacy unit column associated with this policy. Differential privacy policies can only have one privacy unit column per data source object (table, view).
},
"joinRestrictionPolicy": { # Represents privacy policy associated with "join restrictions". Join restriction gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view. # Optional. Join restriction policy is outside of the one of policies, since this policy can be set along with other policies. This policy gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view.
"joinAllowedColumns": [ # Optional. The only columns that joins are allowed on. This field is must be specified for join_conditions JOIN_ANY and JOIN_ALL and it cannot be set for JOIN_BLOCKED.
"A String",
],
"joinCondition": "A String", # Optional. Specifies if a join is required or not on queries for the view. Default is JOIN_CONDITION_UNSPECIFIED.
},
},
"query": "A String", # Required. A query that BigQuery executes when the view is referenced.
"useExplicitColumnNames": True or False, # True if the column names are explicitly specified. For example by using the 'CREATE VIEW v(c1, c2) AS ...' syntax. Can only be set for GoogleSQL views.
"useLegacySql": True or False, # Specifies whether to use BigQuery's legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery's GoogleSQL: https://cloud.google.com/bigquery/sql-reference/ Queries and views that reference this view must use the same flag value. A wrapper is used here because the default value is True.
"userDefinedFunctionResources": [ # Describes user-defined function resources used in the query.
{ # This is used for defining User Defined Function (UDF) resources only when using legacy SQL. Users of GoogleSQL should leverage either DDL (e.g. CREATE [TEMPORARY] FUNCTION ... ) or the Routines API to define UDF resources. For additional information on migrating, see: https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql#differences_in_user-defined_javascript_functions
"inlineCode": "A String", # [Pick one] An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.
"resourceUri": "A String", # [Pick one] A code resource to load from a Google Cloud Storage URI (gs://bucket/path).
},
],
},
}
getIamPolicy(resource, body=None, x__xgafv=None)
Gets the access control policy for a resource. Returns an empty policy if the resource exists and does not have a policy set.
Args:
resource: string, REQUIRED: The resource for which the policy is being requested. See [Resource names](https://cloud.google.com/apis/design/resource_names) for the appropriate value for this field. (required)
body: object, The request body.
The object takes the form of:
{ # Request message for `GetIamPolicy` method.
"options": { # Encapsulates settings provided to GetIamPolicy. # OPTIONAL: A `GetPolicyOptions` object for specifying options to `GetIamPolicy`.
"requestedPolicyVersion": 42, # Optional. The maximum policy version that will be used to format the policy. Valid values are 0, 1, and 3. Requests specifying an invalid value will be rejected. Requests for policies with any conditional role bindings must specify version 3. Policies with no conditional role bindings may specify any valid value or leave the field unset. The policy in the response might use the policy version that you specified, or it might use a lower policy version. For example, if you specify version 3, but the policy has no conditional role bindings, the response uses version 1. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).
},
}
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources. A `Policy` is a collection of `bindings`. A `binding` binds one or more `members`, or principals, to a single `role`. Principals can be user accounts, service accounts, Google groups, and domains (such as G Suite). A `role` is a named list of permissions; each `role` can be an IAM predefined role or a user-created custom role. For some types of Google Cloud resources, a `binding` can also specify a `condition`, which is a logical expression that allows access to a resource only if the expression evaluates to `true`. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies). **JSON example:** ``` { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } ``` **YAML example:** ``` bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') etag: BwWWja0YfJA= version: 3 ``` For a description of IAM and its features, see the [IAM documentation](https://cloud.google.com/iam/docs/).
"auditConfigs": [ # Specifies cloud audit logging configuration for this policy.
{ # Specifies the audit configuration for a service. The configuration determines which permission types are logged, and what identities, if any, are exempted from logging. An AuditConfig must have one or more AuditLogConfigs. If there are AuditConfigs for both `allServices` and a specific service, the union of the two AuditConfigs is used for that service: the log_types specified in each AuditConfig are enabled, and the exempted_members in each AuditLogConfig are exempted. Example Policy with multiple AuditConfigs: { "audit_configs": [ { "service": "allServices", "audit_log_configs": [ { "log_type": "DATA_READ", "exempted_members": [ "user:jose@example.com" ] }, { "log_type": "DATA_WRITE" }, { "log_type": "ADMIN_READ" } ] }, { "service": "sampleservice.googleapis.com", "audit_log_configs": [ { "log_type": "DATA_READ" }, { "log_type": "DATA_WRITE", "exempted_members": [ "user:aliya@example.com" ] } ] } ] } For sampleservice, this policy enables DATA_READ, DATA_WRITE and ADMIN_READ logging. It also exempts `jose@example.com` from DATA_READ logging, and `aliya@example.com` from DATA_WRITE logging.
"auditLogConfigs": [ # The configuration for logging of each type of permission.
{ # Provides the configuration for logging a type of permissions. Example: { "audit_log_configs": [ { "log_type": "DATA_READ", "exempted_members": [ "user:jose@example.com" ] }, { "log_type": "DATA_WRITE" } ] } This enables 'DATA_READ' and 'DATA_WRITE' logging, while exempting jose@example.com from DATA_READ logging.
"exemptedMembers": [ # Specifies the identities that do not cause logging for this type of permission. Follows the same format of Binding.members.
"A String",
],
"logType": "A String", # The log type that this config enables.
},
],
"service": "A String", # Specifies a service that will be enabled for audit logging. For example, `storage.googleapis.com`, `cloudsql.googleapis.com`. `allServices` is a special value that covers all services.
},
],
"bindings": [ # Associates a list of `members`, or principals, with a `role`. Optionally, may specify a `condition` that determines how and when the `bindings` are applied. Each of the `bindings` must contain at least one principal. The `bindings` in a `Policy` can refer to up to 1,500 principals; up to 250 of these principals can be Google groups. Each occurrence of a principal counts towards these limits. For example, if the `bindings` grant 50 different roles to `user:alice@example.com`, and not to any other principal, then you can add another 1,450 principals to the `bindings` in the `Policy`.
{ # Associates `members`, or principals, with a `role`.
"condition": { # Represents a textual expression in the Common Expression Language (CEL) syntax. CEL is a C-like expression language. The syntax and semantics of CEL are documented at https://github.com/google/cel-spec. Example (Comparison): title: "Summary size limit" description: "Determines if a summary is less than 100 chars" expression: "document.summary.size() < 100" Example (Equality): title: "Requestor is owner" description: "Determines if requestor is the document owner" expression: "document.owner == request.auth.claims.email" Example (Logic): title: "Public documents" description: "Determine whether the document should be publicly visible" expression: "document.type != 'private' && document.type != 'internal'" Example (Data Manipulation): title: "Notification string" description: "Create a notification string with a timestamp." expression: "'New message received at ' + string(document.create_time)" The exact variables and functions that may be referenced within an expression are determined by the service that evaluates it. See the service documentation for additional information. # The condition that is associated with this binding. If the condition evaluates to `true`, then this binding applies to the current request. If the condition evaluates to `false`, then this binding does not apply to the current request. However, a different role binding might grant the same role to one or more of the principals in this binding. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).
"description": "A String", # Optional. Description of the expression. This is a longer text which describes the expression, e.g. when hovered over it in a UI.
"expression": "A String", # Textual representation of an expression in Common Expression Language syntax.
"location": "A String", # Optional. String indicating the location of the expression for error reporting, e.g. a file name and a position in the file.
"title": "A String", # Optional. Title for the expression, i.e. a short string describing its purpose. This can be used e.g. in UIs which allow to enter the expression.
},
"members": [ # Specifies the principals requesting access for a Google Cloud resource. `members` can have the following values: * `allUsers`: A special identifier that represents anyone who is on the internet; with or without a Google account. * `allAuthenticatedUsers`: A special identifier that represents anyone who is authenticated with a Google account or a service account. Does not include identities that come from external identity providers (IdPs) through identity federation. * `user:{emailid}`: An email address that represents a specific Google account. For example, `alice@example.com` . * `serviceAccount:{emailid}`: An email address that represents a Google service account. For example, `my-other-app@appspot.gserviceaccount.com`. * `serviceAccount:{projectid}.svc.id.goog[{namespace}/{kubernetes-sa}]`: An identifier for a [Kubernetes service account](https://cloud.google.com/kubernetes-engine/docs/how-to/kubernetes-service-accounts). For example, `my-project.svc.id.goog[my-namespace/my-kubernetes-sa]`. * `group:{emailid}`: An email address that represents a Google group. For example, `admins@example.com`. * `domain:{domain}`: The G Suite domain (primary) that represents all the users of that domain. For example, `google.com` or `example.com`. * `principal://iam.googleapis.com/locations/global/workforcePools/{pool_id}/subject/{subject_attribute_value}`: A single identity in a workforce identity pool. * `principalSet://iam.googleapis.com/locations/global/workforcePools/{pool_id}/group/{group_id}`: All workforce identities in a group. * `principalSet://iam.googleapis.com/locations/global/workforcePools/{pool_id}/attribute.{attribute_name}/{attribute_value}`: All workforce identities with a specific attribute value. * `principalSet://iam.googleapis.com/locations/global/workforcePools/{pool_id}/*`: All identities in a workforce identity pool. * `principal://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/subject/{subject_attribute_value}`: A single identity in a workload identity pool. * `principalSet://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/group/{group_id}`: A workload identity pool group. * `principalSet://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/attribute.{attribute_name}/{attribute_value}`: All identities in a workload identity pool with a certain attribute. * `principalSet://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/*`: All identities in a workload identity pool. * `deleted:user:{emailid}?uid={uniqueid}`: An email address (plus unique identifier) representing a user that has been recently deleted. For example, `alice@example.com?uid=123456789012345678901`. If the user is recovered, this value reverts to `user:{emailid}` and the recovered user retains the role in the binding. * `deleted:serviceAccount:{emailid}?uid={uniqueid}`: An email address (plus unique identifier) representing a service account that has been recently deleted. For example, `my-other-app@appspot.gserviceaccount.com?uid=123456789012345678901`. If the service account is undeleted, this value reverts to `serviceAccount:{emailid}` and the undeleted service account retains the role in the binding. * `deleted:group:{emailid}?uid={uniqueid}`: An email address (plus unique identifier) representing a Google group that has been recently deleted. For example, `admins@example.com?uid=123456789012345678901`. If the group is recovered, this value reverts to `group:{emailid}` and the recovered group retains the role in the binding. * `deleted:principal://iam.googleapis.com/locations/global/workforcePools/{pool_id}/subject/{subject_attribute_value}`: Deleted single identity in a workforce identity pool. For example, `deleted:principal://iam.googleapis.com/locations/global/workforcePools/my-pool-id/subject/my-subject-attribute-value`.
"A String",
],
"role": "A String", # Role that is assigned to the list of `members`, or principals. For example, `roles/viewer`, `roles/editor`, or `roles/owner`. For an overview of the IAM roles and permissions, see the [IAM documentation](https://cloud.google.com/iam/docs/roles-overview). For a list of the available pre-defined roles, see [here](https://cloud.google.com/iam/docs/understanding-roles).
},
],
"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help prevent simultaneous updates of a policy from overwriting each other. It is strongly suggested that systems make use of the `etag` in the read-modify-write cycle to perform policy updates in order to avoid race conditions: An `etag` is returned in the response to `getIamPolicy`, and systems are expected to put that etag in the request to `setIamPolicy` to ensure that their change will be applied to the same version of the policy. **Important:** If you use IAM Conditions, you must include the `etag` field whenever you call `setIamPolicy`. If you omit this field, then IAM allows you to overwrite a version `3` policy with a version `1` policy, and all of the conditions in the version `3` policy are lost.
"version": 42, # Specifies the format of the policy. Valid values are `0`, `1`, and `3`. Requests that specify an invalid value are rejected. Any operation that affects conditional role bindings must specify version `3`. This requirement applies to the following operations: * Getting a policy that includes a conditional role binding * Adding a conditional role binding to a policy * Changing a conditional role binding in a policy * Removing any role binding, with or without a condition, from a policy that includes conditions **Important:** If you use IAM Conditions, you must include the `etag` field whenever you call `setIamPolicy`. If you omit this field, then IAM allows you to overwrite a version `3` policy with a version `1` policy, and all of the conditions in the version `3` policy are lost. If a policy does not include any conditions, operations on that policy may specify any valid version or leave the field unset. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).
}
insert(projectId, datasetId, body=None, x__xgafv=None)
Creates a new, empty table in the dataset.
Args:
projectId: string, Required. Project ID of the new table (required)
datasetId: string, Required. Dataset ID of the new table (required)
body: object, The request body.
The object takes the form of:
{
"biglakeConfiguration": { # Configuration for BigLake managed tables. # Optional. Specifies the configuration of a BigLake managed table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read and write to external storage, such as Cloud Storage. The connection_id can have the form `{project}.{location}.{connection_id}` or `projects/{project}/locations/{location}/connections/{connection_id}".
"fileFormat": "A String", # Optional. The file format the table data is stored in.
"storageUri": "A String", # Optional. The fully qualified location prefix of the external folder where table data is stored. The '*' wildcard character is not allowed. The URI should be in the format `gs://bucket/path_to_table/`
"tableFormat": "A String", # Optional. The table format the metadata only snapshots are stored in.
},
"cloneDefinition": { # Information about base table and clone time of a table clone. # Output only. Contains information about the clone. This value is set via the clone operation.
"baseTableReference": { # Required. Reference describing the ID of the table that was cloned.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"cloneTime": "A String", # Required. The time at which the base table was cloned. This value is reported in the JSON response using RFC3339 format.
},
"clustering": { # Configures table clustering. # Clustering specification for the table. Must be specified with time-based partitioning, data in the table will be first partitioned and subsequently clustered.
"fields": [ # One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. The ordering of the clustering fields should be prioritized from most to least important for filtering purposes. Additional information on limitations can be found here: https://cloud.google.com/bigquery/docs/creating-clustered-tables#limitations
"A String",
],
},
"creationTime": "A String", # Output only. The time when this table was created, in milliseconds since the epoch.
"defaultCollation": "A String", # Optional. Defines the default collation specification of new STRING fields in the table. During table creation or update, if a STRING field is added to this table without explicit collation specified, then the table inherits the table default collation. A change to this field affects only fields added afterwards, and does not alter the existing fields. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"defaultRoundingMode": "A String", # Optional. Defines the default rounding mode specification of new decimal fields (NUMERIC OR BIGNUMERIC) in the table. During table creation or update, if a decimal field is added to this table without an explicit rounding mode specified, then the field inherits the table default rounding mode. Changing this field doesn't affect existing fields.
"description": "A String", # Optional. A user-friendly description of this table.
"encryptionConfiguration": { # Configuration for Cloud KMS encryption settings. # Custom encryption configuration (e.g., Cloud KMS keys).
"kmsKeyName": "A String", # Optional. Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.
},
"etag": "A String", # Output only. A hash of this resource.
"expirationTime": "A String", # Optional. The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed. The defaultTableExpirationMs property of the encapsulating dataset can be used to set a default expirationTime on newly created tables.
"externalCatalogTableOptions": { # Metadata about open source compatible table. The fields contained in these options correspond to hive metastore's table level properties. # Optional. Options defining open source compatible table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection is needed to read the open source table from BigQuery Engine. The connection_id can have the form `..` or `projects//locations//connections/`.
"parameters": { # Optional. A map of key value pairs defining the parameters and properties of the open source table. Corresponds with hive meta store table parameters. Maximum size of 4Mib.
"a_key": "A String",
},
"storageDescriptor": { # Contains information about how a table's data is stored and accessed by open source query engines. # Optional. A storage descriptor containing information about the physical storage of this table.
"inputFormat": "A String", # Optional. Specifies the fully qualified class name of the InputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"). The maximum length is 128 characters.
"locationUri": "A String", # Optional. The physical location of the table (e.g. `gs://spark-dataproc-data/pangea-data/case_sensitive/` or `gs://spark-dataproc-data/pangea-data/*`). The maximum length is 2056 bytes.
"outputFormat": "A String", # Optional. Specifies the fully qualified class name of the OutputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"). The maximum length is 128 characters.
"serdeInfo": { # Serializer and deserializer information. # Optional. Serializer and deserializer information.
"name": "A String", # Optional. Name of the SerDe. The maximum length is 256 characters.
"parameters": { # Optional. Key-value pairs that define the initialization parameters for the serialization library. Maximum size 10 Kib.
"a_key": "A String",
},
"serializationLibrary": "A String", # Required. Specifies a fully-qualified class name of the serialization library that is responsible for the translation of data between table representation and the underlying low-level input and output format structures. The maximum length is 256 characters.
},
},
},
"externalDataConfiguration": { # Optional. Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table.
"autodetect": True or False, # Try to detect schema and format options automatically. Any option specified explicitly will be honored.
"avroOptions": { # Options for external data sources. # Optional. Additional properties to set if sourceFormat is set to AVRO.
"useAvroLogicalTypes": True or False, # Optional. If sourceFormat is set to "AVRO", indicates whether to interpret logical types as the corresponding BigQuery data type (for example, TIMESTAMP), instead of using the raw type (for example, INTEGER).
},
"bigtableOptions": { # Options specific to Google Cloud Bigtable data sources. # Optional. Additional options if sourceFormat is set to BIGTABLE.
"columnFamilies": [ # Optional. List of column families to expose in the table schema along with their types. This list restricts the column families that can be referenced in queries and specifies their value types. You can use this list to do type conversions - see the 'type' field for more details. If you leave this list empty, all column families are present in the table schema and their values are read as BYTES. During a query only the column families referenced in that query are read from Bigtable.
{ # Information related to a Bigtable column family.
"columns": [ # Optional. Lists of columns that should be exposed as individual fields as opposed to a list of (column name, value) pairs. All columns whose qualifier matches a qualifier in this list can be accessed as `.`. Other columns can be accessed as a list through the `.Column` field.
{ # Information related to a Bigtable column.
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. 'encoding' can also be set at the column family level. However, the setting at this level takes precedence if 'encoding' is set at both levels.
"fieldName": "A String", # Optional. If the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as the column field name and is used as field name in queries.
"onlyReadLatest": True or False, # Optional. If this is set, only the latest version of value in this column are exposed. 'onlyReadLatest' can also be set at the column family level. However, the setting at this level takes precedence if 'onlyReadLatest' is set at both levels.
"qualifierEncoded": "A String", # [Required] Qualifier of the column. Columns in the parent column family that has this exact qualifier are exposed as `.` field. If the qualifier is valid UTF-8 string, it can be specified in the qualifier_string field. Otherwise, a base-64 encoded value must be set to qualifier_encoded. The column field name is the same as the column qualifier. However, if the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as field_name.
"qualifierString": "A String", # Qualifier string.
"type": "A String", # Optional. The type to convert the value in cells of this column. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. 'type' can also be set at the column family level. However, the setting at this level takes precedence if 'type' is set at both levels.
},
],
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. This can be overridden for a specific column by listing that column in 'columns' and specifying an encoding for it.
"familyId": "A String", # Identifier of the column family.
"onlyReadLatest": True or False, # Optional. If this is set only the latest version of value are exposed for all columns in this column family. This can be overridden for a specific column by listing that column in 'columns' and specifying a different setting for that column.
"type": "A String", # Optional. The type to convert the value in cells of this column family. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. This can be overridden for a specific column by listing that column in 'columns' and specifying a type for it.
},
],
"ignoreUnspecifiedColumnFamilies": True or False, # Optional. If field is true, then the column families that are not specified in columnFamilies list are not exposed in the table schema. Otherwise, they are read with BYTES type values. The default value is false.
"outputColumnFamiliesAsJson": True or False, # Optional. If field is true, then each column family will be read as a single JSON column. Otherwise they are read as a repeated cell structure containing timestamp/value tuples. The default value is false.
"readRowkeyAsString": True or False, # Optional. If field is true, then the rowkey column families will be read and converted to string. Otherwise they are read with BYTES type values and users need to manually cast them with CAST if necessary. The default value is false.
},
"compression": "A String", # Optional. The compression type of the data source. Possible values include GZIP and NONE. The default value is NONE. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats. An empty string is an invalid value.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id can have the form `{project_id}.{location_id};{connection_id}` or `projects/{project_id}/locations/{location_id}/connections/{connection_id}`.
"csvOptions": { # Information related to a CSV data source. # Optional. Additional properties to set if sourceFormat is set to CSV.
"allowJaggedRows": True or False, # Optional. Indicates if BigQuery should accept rows that are missing trailing optional columns. If true, BigQuery treats missing trailing columns as null values. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.
"allowQuotedNewlines": True or False, # Optional. Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, ISO-8859-1, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.
"fieldDelimiter": "A String", # Optional. The separator character for fields in a CSV file. The separator is interpreted as a single byte. For files encoded in ISO-8859-1, any single character can be used as a separator. For files encoded in UTF-8, characters represented in decimal range 1-127 (U+0001-U+007F) can be used without any modification. UTF-8 characters encoded with multiple bytes (i.e. U+0080 and above) will have only the first byte used for separating fields. The remaining bytes will be treated as a part of the field. BigQuery also supports the escape sequence "\t" (U+0009) to specify a tab separator. The default value is comma (",", U+002C).
"nullMarker": "A String", # Optional. Specifies a string that represents a null value in a CSV file. For example, if you specify "\N", BigQuery interprets "\N" as a null value when querying a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.
"preserveAsciiControlCharacters": True or False, # Optional. Indicates if the embedded ASCII control characters (the first 32 characters in the ASCII-table, from '\x00' to '\x1F') are preserved.
"quote": """, # Optional. The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote ("). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true. To include the specific quote character within a quoted value, precede it with an additional matching quote character. For example, if you want to escape the default character ' " ', use ' "" '.
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a CSV file that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"decimalTargetTypes": [ # Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown. Example: Suppose the value of this field is ["NUMERIC", "BIGNUMERIC"]. If (precision,scale) is: * (38,9) -> NUMERIC; * (39,9) -> BIGNUMERIC (NUMERIC cannot hold 30 integer digits); * (38,10) -> BIGNUMERIC (NUMERIC cannot hold 10 fractional digits); * (76,38) -> BIGNUMERIC; * (77,38) -> BIGNUMERIC (error if value exeeds supported range). This field cannot contain duplicate types. The order of the types in this field is ignored. For example, ["BIGNUMERIC", "NUMERIC"] is the same as ["NUMERIC", "BIGNUMERIC"] and NUMERIC always takes precedence over BIGNUMERIC. Defaults to ["NUMERIC", "STRING"] for ORC and ["NUMERIC"] for the other file formats.
"A String",
],
"fileSetSpecType": "A String", # Optional. Specifies how source URIs are interpreted for constructing the file set to load. By default source URIs are expanded against the underlying storage. Other options include specifying manifest files. Only applicable to object storage systems.
"googleSheetsOptions": { # Options specific to Google Sheets data sources. # Optional. Additional options if sourceFormat is set to GOOGLE_SHEETS.
"range": "A String", # Optional. Range of a sheet to query from. Only used when non-empty. Typical format: sheet_name!top_left_cell_id:bottom_right_cell_id For example: sheet1!A1:B20
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a sheet that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"hivePartitioningOptions": { # Options for configuring hive partitioning detect. # Optional. When set, configures hive partitioning support. Not all storage formats support hive partitioning -- requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.
"fields": [ # Output only. For permanent external tables, this field is populated with the hive partition keys in the order they were inferred. The types of the partition keys can be deduced by checking the table schema (which will include the partition keys). Not every API will populate this field in the output. For example, Tables.Get will populate it, but Tables.List will not contain this field.
"A String",
],
"mode": "A String", # Optional. When set, what mode of hive partitioning to use when reading data. The following modes are supported: * AUTO: automatically infer partition key name(s) and type(s). * STRINGS: automatically infer partition key name(s). All types are strings. * CUSTOM: partition key schema is encoded in the source URI prefix. Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. Note that this field should only be true when creating a permanent external table or querying a temporary external table. Hive-partitioned loads with require_partition_filter explicitly set to true will fail.
"sourceUriPrefix": "A String", # Optional. When hive partition detection is requested, a common prefix for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout: gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. CUSTOM detection requires encoding the partitioning schema immediately after the common prefix. For CUSTOM, any of * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:STRING}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:STRING} would all be valid source URI prefixes.
},
"ignoreUnknownValues": True or False, # Optional. Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don't match any column names Google Cloud Bigtable: This setting is ignored. Google Cloud Datastore backups: This setting is ignored. Avro: This setting is ignored. ORC: This setting is ignored. Parquet: This setting is ignored.
"jsonExtension": "A String", # Optional. Load option to be used together with source_format newline-delimited JSON to indicate that a variant of JSON is being loaded. To load newline-delimited GeoJSON, specify GEOJSON (and source_format must be set to NEWLINE_DELIMITED_JSON).
"jsonOptions": { # Json Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to JSON.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8.
},
"maxBadRecords": 42, # Optional. The maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats.
"metadataCacheMode": "A String", # Optional. Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source.
"objectMetadata": "A String", # Optional. ObjectMetadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the source_uris. If ObjectMetadata is set, source_format should be omitted. Currently SIMPLE is the only supported Object Metadata type.
"parquetOptions": { # Parquet Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to PARQUET.
"enableListInference": True or False, # Optional. Indicates whether to use schema inference specifically for Parquet LIST logical type.
"enumAsString": True or False, # Optional. Indicates whether to infer Parquet ENUM logical type as STRING instead of BYTES by default.
"mapTargetType": "A String", # Optional. Indicates how to represent a Parquet map if present.
},
"referenceFileSchemaUri": "A String", # Optional. When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.
"schema": { # Schema of a table # Optional. The schema for the data. Schema is required for CSV and JSON formats if autodetect is not on. Schema is disallowed for Google Cloud Bigtable, Cloud Datastore backups, Avro, ORC and Parquet formats.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"sourceFormat": "A String", # [Required] The data format. For CSV files, specify "CSV". For Google sheets, specify "GOOGLE_SHEETS". For newline-delimited JSON, specify "NEWLINE_DELIMITED_JSON". For Avro files, specify "AVRO". For Google Cloud Datastore backups, specify "DATASTORE_BACKUP". For Apache Iceberg tables, specify "ICEBERG". For ORC files, specify "ORC". For Parquet files, specify "PARQUET". [Beta] For Google Cloud Bigtable, specify "BIGTABLE".
"sourceUris": [ # [Required] The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one '*' wildcard character and it must come after the 'bucket' name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the '*' wildcard character is not allowed.
"A String",
],
},
"friendlyName": "A String", # Optional. A descriptive name for this table.
"id": "A String", # Output only. An opaque ID uniquely identifying the table.
"kind": "bigquery#table", # The type of resource ID.
"labels": { # The labels associated with this table. You can use these to organize and group your tables. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.
"a_key": "A String",
},
"lastModifiedTime": "A String", # Output only. The time when this table was last modified, in milliseconds since the epoch.
"location": "A String", # Output only. The geographic location where the table resides. This value is inherited from the dataset.
"materializedView": { # Definition and configuration of a materialized view. # Optional. The materialized view definition.
"allowNonIncrementalDefinition": True or False, # Optional. This option declares the intention to construct a materialized view that isn't refreshed incrementally.
"enableRefresh": True or False, # Optional. Enable automatic refresh of the materialized view when the base table is updated. The default value is "true".
"lastRefreshTime": "A String", # Output only. The time when this materialized view was last refreshed, in milliseconds since the epoch.
"maxStaleness": "A String", # [Optional] Max staleness of data that could be returned when materizlized view is queried (formatted as Google SQL Interval type).
"query": "A String", # Required. A query whose results are persisted.
"refreshIntervalMs": "A String", # Optional. The maximum frequency at which this materialized view will be refreshed. The default value is "1800000" (30 minutes).
},
"materializedViewStatus": { # Status of a materialized view. The last refresh timestamp status is omitted here, but is present in the MaterializedViewDefinition message. # Output only. The materialized view status.
"lastRefreshStatus": { # Error details. # Output only. Error result of the last automatic refresh. If present, indicates that the last automatic refresh was unsuccessful.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"refreshWatermark": "A String", # Output only. Refresh watermark of materialized view. The base tables' data were collected into the materialized view cache until this time.
},
"maxStaleness": "A String", # Optional. The maximum staleness of data that could be returned when the table (or stale MV) is queried. Staleness encoded as a string encoding of sql IntervalValue type.
"model": { # Deprecated.
"modelOptions": { # Deprecated.
"labels": [
"A String",
],
"lossType": "A String",
"modelType": "A String",
},
"trainingRuns": [ # Deprecated.
{
"iterationResults": [ # Deprecated.
{
"durationMs": "A String", # Deprecated.
"evalLoss": 3.14, # Deprecated.
"index": 42, # Deprecated.
"learnRate": 3.14, # Deprecated.
"trainingLoss": 3.14, # Deprecated.
},
],
"startTime": "A String", # Deprecated.
"state": "A String", # Deprecated.
"trainingOptions": { # Deprecated.
"earlyStop": True or False,
"l1Reg": 3.14,
"l2Reg": 3.14,
"learnRate": 3.14,
"learnRateStrategy": "A String",
"lineSearchInitLearnRate": 3.14,
"maxIteration": "A String",
"minRelProgress": 3.14,
"warmStart": True or False,
},
},
],
},
"numActiveLogicalBytes": "A String", # Output only. Number of logical bytes that are less than 90 days old.
"numActivePhysicalBytes": "A String", # Output only. Number of physical bytes less than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numBytes": "A String", # Output only. The size of this table in logical bytes, excluding any data in the streaming buffer.
"numCurrentPhysicalBytes": "A String", # Output only. Number of physical bytes used by current live data storage. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numLongTermBytes": "A String", # Output only. The number of logical bytes in the table that are considered "long-term storage".
"numLongTermLogicalBytes": "A String", # Output only. Number of logical bytes that are more than 90 days old.
"numLongTermPhysicalBytes": "A String", # Output only. Number of physical bytes more than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPartitions": "A String", # Output only. The number of partitions present in the table or materialized view. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This includes storage used for time travel.
"numRows": "A String", # Output only. The number of rows of data in this table, excluding any data in the streaming buffer.
"numTimeTravelPhysicalBytes": "A String", # Output only. Number of physical bytes used by time travel storage (deleted or changed data). This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numTotalLogicalBytes": "A String", # Output only. Total number of logical bytes in the table or materialized view.
"numTotalPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This also includes storage used for time travel. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"partitionDefinition": { # The partitioning information, which includes managed table, external table and metastore partitioned table partition information. # Optional. The partition information for all table formats, including managed partitioned tables, hive partitioned tables, iceberg partitioned, and metastore partitioned tables. This field is only populated for metastore partitioned tables. For other table formats, this is an output only field.
"partitionedColumn": [ # Optional. Details about each partitioning column. This field is output only for all partitioning types other than metastore partitioned tables. BigQuery native tables only support 1 partitioning column. Other table types may support 0, 1 or more partitioning columns. For metastore partitioned tables, the order must match the definition order in the Hive Metastore, where it must match the physical layout of the table. For example, CREATE TABLE a_table(id BIGINT, name STRING) PARTITIONED BY (city STRING, state STRING). In this case the values must be ['city', 'state'] in that order.
{ # The partitioning column information.
"field": "A String", # Required. The name of the partition column.
},
],
},
"rangePartitioning": { # If specified, configures range partitioning for this table.
"field": "A String", # Required. The name of the column to partition the table on. It must be a top-level, INT64 column whose mode is NULLABLE or REQUIRED.
"range": { # [Experimental] Defines the ranges for range partitioning.
"end": "A String", # [Experimental] The end of range partitioning, exclusive.
"interval": "A String", # [Experimental] The width of each interval.
"start": "A String", # [Experimental] The start of range partitioning, inclusive.
},
},
"replicas": [ # Optional. Output only. Table references of all replicas currently active on the table.
{
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
],
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.
"resourceTags": { # [Optional] The tags associated with this table. Tag keys are globally unique. See additional information on [tags](https://cloud.google.com/iam/docs/tags-access-control#definitions). An object containing a list of "key": value pairs. The key is the namespaced friendly name of the tag key, e.g. "12345/environment" where 12345 is parent id. The value is the friendly short name of the tag value, e.g. "production".
"a_key": "A String",
},
"restrictions": { # Optional. Output only. Restriction config for table. If set, restrict certain accesses on the table based on the config. See [Data egress](https://cloud.google.com/bigquery/docs/analytics-hub-introduction#data_egress) for more details.
"type": "A String", # Output only. Specifies the type of dataset/table restriction.
},
"schema": { # Schema of a table # Optional. Describes the schema of this table.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"selfLink": "A String", # Output only. A URL that can be used to access this resource again.
"snapshotDefinition": { # Information about base table and snapshot time of the snapshot. # Output only. Contains information about the snapshot. This value is set via snapshot creation.
"baseTableReference": { # Required. Reference describing the ID of the table that was snapshot.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"snapshotTime": "A String", # Required. The time at which the base table was snapshot. This value is reported in the JSON response using RFC3339 format.
},
"streamingBuffer": { # Output only. Contains information regarding this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
"estimatedBytes": "A String", # Output only. A lower-bound estimate of the number of bytes currently in the streaming buffer.
"estimatedRows": "A String", # Output only. A lower-bound estimate of the number of rows currently in the streaming buffer.
"oldestEntryTime": "A String", # Output only. Contains the timestamp of the oldest entry in the streaming buffer, in milliseconds since the epoch, if the streaming buffer is available.
},
"tableConstraints": { # The TableConstraints defines the primary key and foreign key. # Optional. Tables Primary Key and Foreign Key information
"foreignKeys": [ # Optional. Present only if the table has a foreign key. The foreign key is not enforced.
{ # Represents a foreign key constraint on a table's columns.
"columnReferences": [ # Required. The columns that compose the foreign key.
{ # The pair of the foreign key column and primary key column.
"referencedColumn": "A String", # Required. The column in the primary key that are referenced by the referencing_column.
"referencingColumn": "A String", # Required. The column that composes the foreign key.
},
],
"name": "A String", # Optional. Set only if the foreign key constraint is named.
"referencedTable": {
"datasetId": "A String",
"projectId": "A String",
"tableId": "A String",
},
},
],
"primaryKey": { # Represents the primary key constraint on a table's columns.
"columns": [ # Required. The columns that are composed of the primary key constraint.
"A String",
],
},
},
"tableReference": { # Required. Reference describing the ID of this table.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"tableReplicationInfo": { # Replication info of a table created using `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv` # Optional. Table replication info for table created `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv`
"replicatedSourceLastRefreshTime": "A String", # Optional. Output only. If source is a materialized view, this field signifies the last refresh time of the source.
"replicationError": { # Error details. # Optional. Output only. Replication error that will permanently stopped table replication.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"replicationIntervalMs": "A String", # Optional. Specifies the interval at which the source table is polled for updates. It's Optional. If not specified, default replication interval would be applied.
"replicationStatus": "A String", # Optional. Output only. Replication status of configured replication.
"sourceTable": { # Required. Source table reference that is replicated.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
},
"timePartitioning": { # If specified, configures time-based partitioning for this table.
"expirationMs": "A String", # Optional. Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.
"field": "A String", # Optional. If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.
"requirePartitionFilter": false, # If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. This field is deprecated; please set the field with the same name on the table itself instead. This field needs a wrapper because we want to output the default value, false, if the user explicitly set it.
"type": "A String", # Required. The supported types are DAY, HOUR, MONTH, and YEAR, which will generate one partition per day, hour, month, and year, respectively.
},
"type": "A String", # Output only. Describes the table type. The following values are supported: * `TABLE`: A normal BigQuery table. * `VIEW`: A virtual table defined by a SQL query. * `EXTERNAL`: A table that references data stored in an external storage system, such as Google Cloud Storage. * `MATERIALIZED_VIEW`: A precomputed view defined by a SQL query. * `SNAPSHOT`: An immutable BigQuery table that preserves the contents of a base table at a particular time. See additional information on [table snapshots](https://cloud.google.com/bigquery/docs/table-snapshots-intro). The default value is `TABLE`.
"view": { # Describes the definition of a logical view. # Optional. The view definition.
"foreignDefinitions": [ # Optional. Foreign view representations.
{ # A view can be represented in multiple ways. Each representation has its own dialect. This message stores the metadata required for these representations.
"dialect": "A String", # Optional. Represents the dialect of the query.
"query": "A String", # Required. The query that defines the view.
},
],
"privacyPolicy": { # Represents privacy policy that contains the privacy requirements specified by the data owner. Currently, this is only supported on views. # Optional. Specifices the privacy policy for the view.
"aggregationThresholdPolicy": { # Represents privacy policy associated with "aggregation threshold" method. # Optional. Policy used for aggregation thresholds.
"privacyUnitColumns": [ # Optional. The privacy unit column(s) associated with this policy. For now, only one column per data source object (table, view) is allowed as a privacy unit column. Representing as a repeated field in metadata for extensibility to multiple columns in future. Duplicates and Repeated struct fields are not allowed. For nested fields, use dot notation ("outer.inner")
"A String",
],
"threshold": "A String", # Optional. The threshold for the "aggregation threshold" policy.
},
"differentialPrivacyPolicy": { # Represents privacy policy associated with "differential privacy" method. # Optional. Policy used for differential privacy.
"deltaBudget": 3.14, # Optional. The total delta budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of delta that is pre-defined by the contributor through the privacy policy delta_per_query field. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"deltaBudgetRemaining": 3.14, # Output only. The delta budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"deltaPerQuery": 3.14, # Optional. The delta value that is used per query. Delta represents the probability that any row will fail to be epsilon differentially private. Indicates the risk associated with exposing aggregate rows in the result of a query.
"epsilonBudget": 3.14, # Optional. The total epsilon budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of epsilon they request in their query. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"epsilonBudgetRemaining": 3.14, # Output only. The epsilon budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"maxEpsilonPerQuery": 3.14, # Optional. The maximum epsilon value that a query can consume. If the subscriber specifies epsilon as a parameter in a SELECT query, it must be less than or equal to this value. The epsilon parameter controls the amount of noise that is added to the groups — a higher epsilon means less noise.
"maxGroupsContributed": "A String", # Optional. The maximum groups contributed value that is used per query. Represents the maximum number of groups to which each protected entity can contribute. Changing this value does not improve or worsen privacy. The best value for accuracy and utility depends on the query and data.
"privacyUnitColumn": "A String", # Optional. The privacy unit column associated with this policy. Differential privacy policies can only have one privacy unit column per data source object (table, view).
},
"joinRestrictionPolicy": { # Represents privacy policy associated with "join restrictions". Join restriction gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view. # Optional. Join restriction policy is outside of the one of policies, since this policy can be set along with other policies. This policy gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view.
"joinAllowedColumns": [ # Optional. The only columns that joins are allowed on. This field is must be specified for join_conditions JOIN_ANY and JOIN_ALL and it cannot be set for JOIN_BLOCKED.
"A String",
],
"joinCondition": "A String", # Optional. Specifies if a join is required or not on queries for the view. Default is JOIN_CONDITION_UNSPECIFIED.
},
},
"query": "A String", # Required. A query that BigQuery executes when the view is referenced.
"useExplicitColumnNames": True or False, # True if the column names are explicitly specified. For example by using the 'CREATE VIEW v(c1, c2) AS ...' syntax. Can only be set for GoogleSQL views.
"useLegacySql": True or False, # Specifies whether to use BigQuery's legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery's GoogleSQL: https://cloud.google.com/bigquery/sql-reference/ Queries and views that reference this view must use the same flag value. A wrapper is used here because the default value is True.
"userDefinedFunctionResources": [ # Describes user-defined function resources used in the query.
{ # This is used for defining User Defined Function (UDF) resources only when using legacy SQL. Users of GoogleSQL should leverage either DDL (e.g. CREATE [TEMPORARY] FUNCTION ... ) or the Routines API to define UDF resources. For additional information on migrating, see: https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql#differences_in_user-defined_javascript_functions
"inlineCode": "A String", # [Pick one] An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.
"resourceUri": "A String", # [Pick one] A code resource to load from a Google Cloud Storage URI (gs://bucket/path).
},
],
},
}
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{
"biglakeConfiguration": { # Configuration for BigLake managed tables. # Optional. Specifies the configuration of a BigLake managed table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read and write to external storage, such as Cloud Storage. The connection_id can have the form `{project}.{location}.{connection_id}` or `projects/{project}/locations/{location}/connections/{connection_id}".
"fileFormat": "A String", # Optional. The file format the table data is stored in.
"storageUri": "A String", # Optional. The fully qualified location prefix of the external folder where table data is stored. The '*' wildcard character is not allowed. The URI should be in the format `gs://bucket/path_to_table/`
"tableFormat": "A String", # Optional. The table format the metadata only snapshots are stored in.
},
"cloneDefinition": { # Information about base table and clone time of a table clone. # Output only. Contains information about the clone. This value is set via the clone operation.
"baseTableReference": { # Required. Reference describing the ID of the table that was cloned.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"cloneTime": "A String", # Required. The time at which the base table was cloned. This value is reported in the JSON response using RFC3339 format.
},
"clustering": { # Configures table clustering. # Clustering specification for the table. Must be specified with time-based partitioning, data in the table will be first partitioned and subsequently clustered.
"fields": [ # One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. The ordering of the clustering fields should be prioritized from most to least important for filtering purposes. Additional information on limitations can be found here: https://cloud.google.com/bigquery/docs/creating-clustered-tables#limitations
"A String",
],
},
"creationTime": "A String", # Output only. The time when this table was created, in milliseconds since the epoch.
"defaultCollation": "A String", # Optional. Defines the default collation specification of new STRING fields in the table. During table creation or update, if a STRING field is added to this table without explicit collation specified, then the table inherits the table default collation. A change to this field affects only fields added afterwards, and does not alter the existing fields. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"defaultRoundingMode": "A String", # Optional. Defines the default rounding mode specification of new decimal fields (NUMERIC OR BIGNUMERIC) in the table. During table creation or update, if a decimal field is added to this table without an explicit rounding mode specified, then the field inherits the table default rounding mode. Changing this field doesn't affect existing fields.
"description": "A String", # Optional. A user-friendly description of this table.
"encryptionConfiguration": { # Configuration for Cloud KMS encryption settings. # Custom encryption configuration (e.g., Cloud KMS keys).
"kmsKeyName": "A String", # Optional. Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.
},
"etag": "A String", # Output only. A hash of this resource.
"expirationTime": "A String", # Optional. The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed. The defaultTableExpirationMs property of the encapsulating dataset can be used to set a default expirationTime on newly created tables.
"externalCatalogTableOptions": { # Metadata about open source compatible table. The fields contained in these options correspond to hive metastore's table level properties. # Optional. Options defining open source compatible table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection is needed to read the open source table from BigQuery Engine. The connection_id can have the form `..` or `projects//locations//connections/`.
"parameters": { # Optional. A map of key value pairs defining the parameters and properties of the open source table. Corresponds with hive meta store table parameters. Maximum size of 4Mib.
"a_key": "A String",
},
"storageDescriptor": { # Contains information about how a table's data is stored and accessed by open source query engines. # Optional. A storage descriptor containing information about the physical storage of this table.
"inputFormat": "A String", # Optional. Specifies the fully qualified class name of the InputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"). The maximum length is 128 characters.
"locationUri": "A String", # Optional. The physical location of the table (e.g. `gs://spark-dataproc-data/pangea-data/case_sensitive/` or `gs://spark-dataproc-data/pangea-data/*`). The maximum length is 2056 bytes.
"outputFormat": "A String", # Optional. Specifies the fully qualified class name of the OutputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"). The maximum length is 128 characters.
"serdeInfo": { # Serializer and deserializer information. # Optional. Serializer and deserializer information.
"name": "A String", # Optional. Name of the SerDe. The maximum length is 256 characters.
"parameters": { # Optional. Key-value pairs that define the initialization parameters for the serialization library. Maximum size 10 Kib.
"a_key": "A String",
},
"serializationLibrary": "A String", # Required. Specifies a fully-qualified class name of the serialization library that is responsible for the translation of data between table representation and the underlying low-level input and output format structures. The maximum length is 256 characters.
},
},
},
"externalDataConfiguration": { # Optional. Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table.
"autodetect": True or False, # Try to detect schema and format options automatically. Any option specified explicitly will be honored.
"avroOptions": { # Options for external data sources. # Optional. Additional properties to set if sourceFormat is set to AVRO.
"useAvroLogicalTypes": True or False, # Optional. If sourceFormat is set to "AVRO", indicates whether to interpret logical types as the corresponding BigQuery data type (for example, TIMESTAMP), instead of using the raw type (for example, INTEGER).
},
"bigtableOptions": { # Options specific to Google Cloud Bigtable data sources. # Optional. Additional options if sourceFormat is set to BIGTABLE.
"columnFamilies": [ # Optional. List of column families to expose in the table schema along with their types. This list restricts the column families that can be referenced in queries and specifies their value types. You can use this list to do type conversions - see the 'type' field for more details. If you leave this list empty, all column families are present in the table schema and their values are read as BYTES. During a query only the column families referenced in that query are read from Bigtable.
{ # Information related to a Bigtable column family.
"columns": [ # Optional. Lists of columns that should be exposed as individual fields as opposed to a list of (column name, value) pairs. All columns whose qualifier matches a qualifier in this list can be accessed as `.`. Other columns can be accessed as a list through the `.Column` field.
{ # Information related to a Bigtable column.
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. 'encoding' can also be set at the column family level. However, the setting at this level takes precedence if 'encoding' is set at both levels.
"fieldName": "A String", # Optional. If the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as the column field name and is used as field name in queries.
"onlyReadLatest": True or False, # Optional. If this is set, only the latest version of value in this column are exposed. 'onlyReadLatest' can also be set at the column family level. However, the setting at this level takes precedence if 'onlyReadLatest' is set at both levels.
"qualifierEncoded": "A String", # [Required] Qualifier of the column. Columns in the parent column family that has this exact qualifier are exposed as `.` field. If the qualifier is valid UTF-8 string, it can be specified in the qualifier_string field. Otherwise, a base-64 encoded value must be set to qualifier_encoded. The column field name is the same as the column qualifier. However, if the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as field_name.
"qualifierString": "A String", # Qualifier string.
"type": "A String", # Optional. The type to convert the value in cells of this column. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. 'type' can also be set at the column family level. However, the setting at this level takes precedence if 'type' is set at both levels.
},
],
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. This can be overridden for a specific column by listing that column in 'columns' and specifying an encoding for it.
"familyId": "A String", # Identifier of the column family.
"onlyReadLatest": True or False, # Optional. If this is set only the latest version of value are exposed for all columns in this column family. This can be overridden for a specific column by listing that column in 'columns' and specifying a different setting for that column.
"type": "A String", # Optional. The type to convert the value in cells of this column family. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. This can be overridden for a specific column by listing that column in 'columns' and specifying a type for it.
},
],
"ignoreUnspecifiedColumnFamilies": True or False, # Optional. If field is true, then the column families that are not specified in columnFamilies list are not exposed in the table schema. Otherwise, they are read with BYTES type values. The default value is false.
"outputColumnFamiliesAsJson": True or False, # Optional. If field is true, then each column family will be read as a single JSON column. Otherwise they are read as a repeated cell structure containing timestamp/value tuples. The default value is false.
"readRowkeyAsString": True or False, # Optional. If field is true, then the rowkey column families will be read and converted to string. Otherwise they are read with BYTES type values and users need to manually cast them with CAST if necessary. The default value is false.
},
"compression": "A String", # Optional. The compression type of the data source. Possible values include GZIP and NONE. The default value is NONE. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats. An empty string is an invalid value.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id can have the form `{project_id}.{location_id};{connection_id}` or `projects/{project_id}/locations/{location_id}/connections/{connection_id}`.
"csvOptions": { # Information related to a CSV data source. # Optional. Additional properties to set if sourceFormat is set to CSV.
"allowJaggedRows": True or False, # Optional. Indicates if BigQuery should accept rows that are missing trailing optional columns. If true, BigQuery treats missing trailing columns as null values. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.
"allowQuotedNewlines": True or False, # Optional. Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, ISO-8859-1, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.
"fieldDelimiter": "A String", # Optional. The separator character for fields in a CSV file. The separator is interpreted as a single byte. For files encoded in ISO-8859-1, any single character can be used as a separator. For files encoded in UTF-8, characters represented in decimal range 1-127 (U+0001-U+007F) can be used without any modification. UTF-8 characters encoded with multiple bytes (i.e. U+0080 and above) will have only the first byte used for separating fields. The remaining bytes will be treated as a part of the field. BigQuery also supports the escape sequence "\t" (U+0009) to specify a tab separator. The default value is comma (",", U+002C).
"nullMarker": "A String", # Optional. Specifies a string that represents a null value in a CSV file. For example, if you specify "\N", BigQuery interprets "\N" as a null value when querying a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.
"preserveAsciiControlCharacters": True or False, # Optional. Indicates if the embedded ASCII control characters (the first 32 characters in the ASCII-table, from '\x00' to '\x1F') are preserved.
"quote": """, # Optional. The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote ("). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true. To include the specific quote character within a quoted value, precede it with an additional matching quote character. For example, if you want to escape the default character ' " ', use ' "" '.
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a CSV file that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"decimalTargetTypes": [ # Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown. Example: Suppose the value of this field is ["NUMERIC", "BIGNUMERIC"]. If (precision,scale) is: * (38,9) -> NUMERIC; * (39,9) -> BIGNUMERIC (NUMERIC cannot hold 30 integer digits); * (38,10) -> BIGNUMERIC (NUMERIC cannot hold 10 fractional digits); * (76,38) -> BIGNUMERIC; * (77,38) -> BIGNUMERIC (error if value exeeds supported range). This field cannot contain duplicate types. The order of the types in this field is ignored. For example, ["BIGNUMERIC", "NUMERIC"] is the same as ["NUMERIC", "BIGNUMERIC"] and NUMERIC always takes precedence over BIGNUMERIC. Defaults to ["NUMERIC", "STRING"] for ORC and ["NUMERIC"] for the other file formats.
"A String",
],
"fileSetSpecType": "A String", # Optional. Specifies how source URIs are interpreted for constructing the file set to load. By default source URIs are expanded against the underlying storage. Other options include specifying manifest files. Only applicable to object storage systems.
"googleSheetsOptions": { # Options specific to Google Sheets data sources. # Optional. Additional options if sourceFormat is set to GOOGLE_SHEETS.
"range": "A String", # Optional. Range of a sheet to query from. Only used when non-empty. Typical format: sheet_name!top_left_cell_id:bottom_right_cell_id For example: sheet1!A1:B20
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a sheet that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"hivePartitioningOptions": { # Options for configuring hive partitioning detect. # Optional. When set, configures hive partitioning support. Not all storage formats support hive partitioning -- requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.
"fields": [ # Output only. For permanent external tables, this field is populated with the hive partition keys in the order they were inferred. The types of the partition keys can be deduced by checking the table schema (which will include the partition keys). Not every API will populate this field in the output. For example, Tables.Get will populate it, but Tables.List will not contain this field.
"A String",
],
"mode": "A String", # Optional. When set, what mode of hive partitioning to use when reading data. The following modes are supported: * AUTO: automatically infer partition key name(s) and type(s). * STRINGS: automatically infer partition key name(s). All types are strings. * CUSTOM: partition key schema is encoded in the source URI prefix. Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. Note that this field should only be true when creating a permanent external table or querying a temporary external table. Hive-partitioned loads with require_partition_filter explicitly set to true will fail.
"sourceUriPrefix": "A String", # Optional. When hive partition detection is requested, a common prefix for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout: gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. CUSTOM detection requires encoding the partitioning schema immediately after the common prefix. For CUSTOM, any of * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:STRING}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:STRING} would all be valid source URI prefixes.
},
"ignoreUnknownValues": True or False, # Optional. Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don't match any column names Google Cloud Bigtable: This setting is ignored. Google Cloud Datastore backups: This setting is ignored. Avro: This setting is ignored. ORC: This setting is ignored. Parquet: This setting is ignored.
"jsonExtension": "A String", # Optional. Load option to be used together with source_format newline-delimited JSON to indicate that a variant of JSON is being loaded. To load newline-delimited GeoJSON, specify GEOJSON (and source_format must be set to NEWLINE_DELIMITED_JSON).
"jsonOptions": { # Json Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to JSON.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8.
},
"maxBadRecords": 42, # Optional. The maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats.
"metadataCacheMode": "A String", # Optional. Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source.
"objectMetadata": "A String", # Optional. ObjectMetadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the source_uris. If ObjectMetadata is set, source_format should be omitted. Currently SIMPLE is the only supported Object Metadata type.
"parquetOptions": { # Parquet Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to PARQUET.
"enableListInference": True or False, # Optional. Indicates whether to use schema inference specifically for Parquet LIST logical type.
"enumAsString": True or False, # Optional. Indicates whether to infer Parquet ENUM logical type as STRING instead of BYTES by default.
"mapTargetType": "A String", # Optional. Indicates how to represent a Parquet map if present.
},
"referenceFileSchemaUri": "A String", # Optional. When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.
"schema": { # Schema of a table # Optional. The schema for the data. Schema is required for CSV and JSON formats if autodetect is not on. Schema is disallowed for Google Cloud Bigtable, Cloud Datastore backups, Avro, ORC and Parquet formats.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"sourceFormat": "A String", # [Required] The data format. For CSV files, specify "CSV". For Google sheets, specify "GOOGLE_SHEETS". For newline-delimited JSON, specify "NEWLINE_DELIMITED_JSON". For Avro files, specify "AVRO". For Google Cloud Datastore backups, specify "DATASTORE_BACKUP". For Apache Iceberg tables, specify "ICEBERG". For ORC files, specify "ORC". For Parquet files, specify "PARQUET". [Beta] For Google Cloud Bigtable, specify "BIGTABLE".
"sourceUris": [ # [Required] The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one '*' wildcard character and it must come after the 'bucket' name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the '*' wildcard character is not allowed.
"A String",
],
},
"friendlyName": "A String", # Optional. A descriptive name for this table.
"id": "A String", # Output only. An opaque ID uniquely identifying the table.
"kind": "bigquery#table", # The type of resource ID.
"labels": { # The labels associated with this table. You can use these to organize and group your tables. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.
"a_key": "A String",
},
"lastModifiedTime": "A String", # Output only. The time when this table was last modified, in milliseconds since the epoch.
"location": "A String", # Output only. The geographic location where the table resides. This value is inherited from the dataset.
"materializedView": { # Definition and configuration of a materialized view. # Optional. The materialized view definition.
"allowNonIncrementalDefinition": True or False, # Optional. This option declares the intention to construct a materialized view that isn't refreshed incrementally.
"enableRefresh": True or False, # Optional. Enable automatic refresh of the materialized view when the base table is updated. The default value is "true".
"lastRefreshTime": "A String", # Output only. The time when this materialized view was last refreshed, in milliseconds since the epoch.
"maxStaleness": "A String", # [Optional] Max staleness of data that could be returned when materizlized view is queried (formatted as Google SQL Interval type).
"query": "A String", # Required. A query whose results are persisted.
"refreshIntervalMs": "A String", # Optional. The maximum frequency at which this materialized view will be refreshed. The default value is "1800000" (30 minutes).
},
"materializedViewStatus": { # Status of a materialized view. The last refresh timestamp status is omitted here, but is present in the MaterializedViewDefinition message. # Output only. The materialized view status.
"lastRefreshStatus": { # Error details. # Output only. Error result of the last automatic refresh. If present, indicates that the last automatic refresh was unsuccessful.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"refreshWatermark": "A String", # Output only. Refresh watermark of materialized view. The base tables' data were collected into the materialized view cache until this time.
},
"maxStaleness": "A String", # Optional. The maximum staleness of data that could be returned when the table (or stale MV) is queried. Staleness encoded as a string encoding of sql IntervalValue type.
"model": { # Deprecated.
"modelOptions": { # Deprecated.
"labels": [
"A String",
],
"lossType": "A String",
"modelType": "A String",
},
"trainingRuns": [ # Deprecated.
{
"iterationResults": [ # Deprecated.
{
"durationMs": "A String", # Deprecated.
"evalLoss": 3.14, # Deprecated.
"index": 42, # Deprecated.
"learnRate": 3.14, # Deprecated.
"trainingLoss": 3.14, # Deprecated.
},
],
"startTime": "A String", # Deprecated.
"state": "A String", # Deprecated.
"trainingOptions": { # Deprecated.
"earlyStop": True or False,
"l1Reg": 3.14,
"l2Reg": 3.14,
"learnRate": 3.14,
"learnRateStrategy": "A String",
"lineSearchInitLearnRate": 3.14,
"maxIteration": "A String",
"minRelProgress": 3.14,
"warmStart": True or False,
},
},
],
},
"numActiveLogicalBytes": "A String", # Output only. Number of logical bytes that are less than 90 days old.
"numActivePhysicalBytes": "A String", # Output only. Number of physical bytes less than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numBytes": "A String", # Output only. The size of this table in logical bytes, excluding any data in the streaming buffer.
"numCurrentPhysicalBytes": "A String", # Output only. Number of physical bytes used by current live data storage. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numLongTermBytes": "A String", # Output only. The number of logical bytes in the table that are considered "long-term storage".
"numLongTermLogicalBytes": "A String", # Output only. Number of logical bytes that are more than 90 days old.
"numLongTermPhysicalBytes": "A String", # Output only. Number of physical bytes more than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPartitions": "A String", # Output only. The number of partitions present in the table or materialized view. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This includes storage used for time travel.
"numRows": "A String", # Output only. The number of rows of data in this table, excluding any data in the streaming buffer.
"numTimeTravelPhysicalBytes": "A String", # Output only. Number of physical bytes used by time travel storage (deleted or changed data). This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numTotalLogicalBytes": "A String", # Output only. Total number of logical bytes in the table or materialized view.
"numTotalPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This also includes storage used for time travel. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"partitionDefinition": { # The partitioning information, which includes managed table, external table and metastore partitioned table partition information. # Optional. The partition information for all table formats, including managed partitioned tables, hive partitioned tables, iceberg partitioned, and metastore partitioned tables. This field is only populated for metastore partitioned tables. For other table formats, this is an output only field.
"partitionedColumn": [ # Optional. Details about each partitioning column. This field is output only for all partitioning types other than metastore partitioned tables. BigQuery native tables only support 1 partitioning column. Other table types may support 0, 1 or more partitioning columns. For metastore partitioned tables, the order must match the definition order in the Hive Metastore, where it must match the physical layout of the table. For example, CREATE TABLE a_table(id BIGINT, name STRING) PARTITIONED BY (city STRING, state STRING). In this case the values must be ['city', 'state'] in that order.
{ # The partitioning column information.
"field": "A String", # Required. The name of the partition column.
},
],
},
"rangePartitioning": { # If specified, configures range partitioning for this table.
"field": "A String", # Required. The name of the column to partition the table on. It must be a top-level, INT64 column whose mode is NULLABLE or REQUIRED.
"range": { # [Experimental] Defines the ranges for range partitioning.
"end": "A String", # [Experimental] The end of range partitioning, exclusive.
"interval": "A String", # [Experimental] The width of each interval.
"start": "A String", # [Experimental] The start of range partitioning, inclusive.
},
},
"replicas": [ # Optional. Output only. Table references of all replicas currently active on the table.
{
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
],
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.
"resourceTags": { # [Optional] The tags associated with this table. Tag keys are globally unique. See additional information on [tags](https://cloud.google.com/iam/docs/tags-access-control#definitions). An object containing a list of "key": value pairs. The key is the namespaced friendly name of the tag key, e.g. "12345/environment" where 12345 is parent id. The value is the friendly short name of the tag value, e.g. "production".
"a_key": "A String",
},
"restrictions": { # Optional. Output only. Restriction config for table. If set, restrict certain accesses on the table based on the config. See [Data egress](https://cloud.google.com/bigquery/docs/analytics-hub-introduction#data_egress) for more details.
"type": "A String", # Output only. Specifies the type of dataset/table restriction.
},
"schema": { # Schema of a table # Optional. Describes the schema of this table.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"selfLink": "A String", # Output only. A URL that can be used to access this resource again.
"snapshotDefinition": { # Information about base table and snapshot time of the snapshot. # Output only. Contains information about the snapshot. This value is set via snapshot creation.
"baseTableReference": { # Required. Reference describing the ID of the table that was snapshot.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"snapshotTime": "A String", # Required. The time at which the base table was snapshot. This value is reported in the JSON response using RFC3339 format.
},
"streamingBuffer": { # Output only. Contains information regarding this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
"estimatedBytes": "A String", # Output only. A lower-bound estimate of the number of bytes currently in the streaming buffer.
"estimatedRows": "A String", # Output only. A lower-bound estimate of the number of rows currently in the streaming buffer.
"oldestEntryTime": "A String", # Output only. Contains the timestamp of the oldest entry in the streaming buffer, in milliseconds since the epoch, if the streaming buffer is available.
},
"tableConstraints": { # The TableConstraints defines the primary key and foreign key. # Optional. Tables Primary Key and Foreign Key information
"foreignKeys": [ # Optional. Present only if the table has a foreign key. The foreign key is not enforced.
{ # Represents a foreign key constraint on a table's columns.
"columnReferences": [ # Required. The columns that compose the foreign key.
{ # The pair of the foreign key column and primary key column.
"referencedColumn": "A String", # Required. The column in the primary key that are referenced by the referencing_column.
"referencingColumn": "A String", # Required. The column that composes the foreign key.
},
],
"name": "A String", # Optional. Set only if the foreign key constraint is named.
"referencedTable": {
"datasetId": "A String",
"projectId": "A String",
"tableId": "A String",
},
},
],
"primaryKey": { # Represents the primary key constraint on a table's columns.
"columns": [ # Required. The columns that are composed of the primary key constraint.
"A String",
],
},
},
"tableReference": { # Required. Reference describing the ID of this table.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"tableReplicationInfo": { # Replication info of a table created using `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv` # Optional. Table replication info for table created `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv`
"replicatedSourceLastRefreshTime": "A String", # Optional. Output only. If source is a materialized view, this field signifies the last refresh time of the source.
"replicationError": { # Error details. # Optional. Output only. Replication error that will permanently stopped table replication.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"replicationIntervalMs": "A String", # Optional. Specifies the interval at which the source table is polled for updates. It's Optional. If not specified, default replication interval would be applied.
"replicationStatus": "A String", # Optional. Output only. Replication status of configured replication.
"sourceTable": { # Required. Source table reference that is replicated.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
},
"timePartitioning": { # If specified, configures time-based partitioning for this table.
"expirationMs": "A String", # Optional. Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.
"field": "A String", # Optional. If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.
"requirePartitionFilter": false, # If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. This field is deprecated; please set the field with the same name on the table itself instead. This field needs a wrapper because we want to output the default value, false, if the user explicitly set it.
"type": "A String", # Required. The supported types are DAY, HOUR, MONTH, and YEAR, which will generate one partition per day, hour, month, and year, respectively.
},
"type": "A String", # Output only. Describes the table type. The following values are supported: * `TABLE`: A normal BigQuery table. * `VIEW`: A virtual table defined by a SQL query. * `EXTERNAL`: A table that references data stored in an external storage system, such as Google Cloud Storage. * `MATERIALIZED_VIEW`: A precomputed view defined by a SQL query. * `SNAPSHOT`: An immutable BigQuery table that preserves the contents of a base table at a particular time. See additional information on [table snapshots](https://cloud.google.com/bigquery/docs/table-snapshots-intro). The default value is `TABLE`.
"view": { # Describes the definition of a logical view. # Optional. The view definition.
"foreignDefinitions": [ # Optional. Foreign view representations.
{ # A view can be represented in multiple ways. Each representation has its own dialect. This message stores the metadata required for these representations.
"dialect": "A String", # Optional. Represents the dialect of the query.
"query": "A String", # Required. The query that defines the view.
},
],
"privacyPolicy": { # Represents privacy policy that contains the privacy requirements specified by the data owner. Currently, this is only supported on views. # Optional. Specifices the privacy policy for the view.
"aggregationThresholdPolicy": { # Represents privacy policy associated with "aggregation threshold" method. # Optional. Policy used for aggregation thresholds.
"privacyUnitColumns": [ # Optional. The privacy unit column(s) associated with this policy. For now, only one column per data source object (table, view) is allowed as a privacy unit column. Representing as a repeated field in metadata for extensibility to multiple columns in future. Duplicates and Repeated struct fields are not allowed. For nested fields, use dot notation ("outer.inner")
"A String",
],
"threshold": "A String", # Optional. The threshold for the "aggregation threshold" policy.
},
"differentialPrivacyPolicy": { # Represents privacy policy associated with "differential privacy" method. # Optional. Policy used for differential privacy.
"deltaBudget": 3.14, # Optional. The total delta budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of delta that is pre-defined by the contributor through the privacy policy delta_per_query field. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"deltaBudgetRemaining": 3.14, # Output only. The delta budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"deltaPerQuery": 3.14, # Optional. The delta value that is used per query. Delta represents the probability that any row will fail to be epsilon differentially private. Indicates the risk associated with exposing aggregate rows in the result of a query.
"epsilonBudget": 3.14, # Optional. The total epsilon budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of epsilon they request in their query. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"epsilonBudgetRemaining": 3.14, # Output only. The epsilon budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"maxEpsilonPerQuery": 3.14, # Optional. The maximum epsilon value that a query can consume. If the subscriber specifies epsilon as a parameter in a SELECT query, it must be less than or equal to this value. The epsilon parameter controls the amount of noise that is added to the groups — a higher epsilon means less noise.
"maxGroupsContributed": "A String", # Optional. The maximum groups contributed value that is used per query. Represents the maximum number of groups to which each protected entity can contribute. Changing this value does not improve or worsen privacy. The best value for accuracy and utility depends on the query and data.
"privacyUnitColumn": "A String", # Optional. The privacy unit column associated with this policy. Differential privacy policies can only have one privacy unit column per data source object (table, view).
},
"joinRestrictionPolicy": { # Represents privacy policy associated with "join restrictions". Join restriction gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view. # Optional. Join restriction policy is outside of the one of policies, since this policy can be set along with other policies. This policy gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view.
"joinAllowedColumns": [ # Optional. The only columns that joins are allowed on. This field is must be specified for join_conditions JOIN_ANY and JOIN_ALL and it cannot be set for JOIN_BLOCKED.
"A String",
],
"joinCondition": "A String", # Optional. Specifies if a join is required or not on queries for the view. Default is JOIN_CONDITION_UNSPECIFIED.
},
},
"query": "A String", # Required. A query that BigQuery executes when the view is referenced.
"useExplicitColumnNames": True or False, # True if the column names are explicitly specified. For example by using the 'CREATE VIEW v(c1, c2) AS ...' syntax. Can only be set for GoogleSQL views.
"useLegacySql": True or False, # Specifies whether to use BigQuery's legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery's GoogleSQL: https://cloud.google.com/bigquery/sql-reference/ Queries and views that reference this view must use the same flag value. A wrapper is used here because the default value is True.
"userDefinedFunctionResources": [ # Describes user-defined function resources used in the query.
{ # This is used for defining User Defined Function (UDF) resources only when using legacy SQL. Users of GoogleSQL should leverage either DDL (e.g. CREATE [TEMPORARY] FUNCTION ... ) or the Routines API to define UDF resources. For additional information on migrating, see: https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql#differences_in_user-defined_javascript_functions
"inlineCode": "A String", # [Pick one] An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.
"resourceUri": "A String", # [Pick one] A code resource to load from a Google Cloud Storage URI (gs://bucket/path).
},
],
},
}
list(projectId, datasetId, maxResults=None, pageToken=None, x__xgafv=None)
Lists all tables in the specified dataset. Requires the READER dataset role.
Args:
projectId: string, Required. Project ID of the tables to list (required)
datasetId: string, Required. Dataset ID of the tables to list (required)
maxResults: integer, The maximum number of results to return in a single response page. Leverage the page tokens to iterate through the entire collection.
pageToken: string, Page token, returned by a previous call, to request the next page of results
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # Partial projection of the metadata for a given table in a list response.
"etag": "A String", # A hash of this page of results.
"kind": "bigquery#tableList", # The type of list.
"nextPageToken": "A String", # A token to request the next page of results.
"tables": [ # Tables in the requested dataset.
{
"clustering": { # Configures table clustering. # Clustering specification for this table, if configured.
"fields": [ # One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. The ordering of the clustering fields should be prioritized from most to least important for filtering purposes. Additional information on limitations can be found here: https://cloud.google.com/bigquery/docs/creating-clustered-tables#limitations
"A String",
],
},
"creationTime": "A String", # Output only. The time when this table was created, in milliseconds since the epoch.
"expirationTime": "A String", # The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.
"friendlyName": "A String", # The user-friendly name for this table.
"id": "A String", # An opaque ID of the table.
"kind": "A String", # The resource type.
"labels": { # The labels associated with this table. You can use these to organize and group your tables.
"a_key": "A String",
},
"rangePartitioning": { # The range partitioning for this table.
"field": "A String", # Required. The name of the column to partition the table on. It must be a top-level, INT64 column whose mode is NULLABLE or REQUIRED.
"range": { # [Experimental] Defines the ranges for range partitioning.
"end": "A String", # [Experimental] The end of range partitioning, exclusive.
"interval": "A String", # [Experimental] The width of each interval.
"start": "A String", # [Experimental] The start of range partitioning, inclusive.
},
},
"requirePartitionFilter": false, # Optional. If set to true, queries including this table must specify a partition filter. This filter is used for partition elimination.
"tableReference": { # A reference uniquely identifying table.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"timePartitioning": { # The time-based partitioning for this table.
"expirationMs": "A String", # Optional. Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.
"field": "A String", # Optional. If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.
"requirePartitionFilter": false, # If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. This field is deprecated; please set the field with the same name on the table itself instead. This field needs a wrapper because we want to output the default value, false, if the user explicitly set it.
"type": "A String", # Required. The supported types are DAY, HOUR, MONTH, and YEAR, which will generate one partition per day, hour, month, and year, respectively.
},
"type": "A String", # The type of table.
"view": { # Information about a logical view.
"privacyPolicy": { # Represents privacy policy that contains the privacy requirements specified by the data owner. Currently, this is only supported on views. # Specifices the privacy policy for the view.
"aggregationThresholdPolicy": { # Represents privacy policy associated with "aggregation threshold" method. # Optional. Policy used for aggregation thresholds.
"privacyUnitColumns": [ # Optional. The privacy unit column(s) associated with this policy. For now, only one column per data source object (table, view) is allowed as a privacy unit column. Representing as a repeated field in metadata for extensibility to multiple columns in future. Duplicates and Repeated struct fields are not allowed. For nested fields, use dot notation ("outer.inner")
"A String",
],
"threshold": "A String", # Optional. The threshold for the "aggregation threshold" policy.
},
"differentialPrivacyPolicy": { # Represents privacy policy associated with "differential privacy" method. # Optional. Policy used for differential privacy.
"deltaBudget": 3.14, # Optional. The total delta budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of delta that is pre-defined by the contributor through the privacy policy delta_per_query field. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"deltaBudgetRemaining": 3.14, # Output only. The delta budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"deltaPerQuery": 3.14, # Optional. The delta value that is used per query. Delta represents the probability that any row will fail to be epsilon differentially private. Indicates the risk associated with exposing aggregate rows in the result of a query.
"epsilonBudget": 3.14, # Optional. The total epsilon budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of epsilon they request in their query. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"epsilonBudgetRemaining": 3.14, # Output only. The epsilon budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"maxEpsilonPerQuery": 3.14, # Optional. The maximum epsilon value that a query can consume. If the subscriber specifies epsilon as a parameter in a SELECT query, it must be less than or equal to this value. The epsilon parameter controls the amount of noise that is added to the groups — a higher epsilon means less noise.
"maxGroupsContributed": "A String", # Optional. The maximum groups contributed value that is used per query. Represents the maximum number of groups to which each protected entity can contribute. Changing this value does not improve or worsen privacy. The best value for accuracy and utility depends on the query and data.
"privacyUnitColumn": "A String", # Optional. The privacy unit column associated with this policy. Differential privacy policies can only have one privacy unit column per data source object (table, view).
},
"joinRestrictionPolicy": { # Represents privacy policy associated with "join restrictions". Join restriction gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view. # Optional. Join restriction policy is outside of the one of policies, since this policy can be set along with other policies. This policy gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view.
"joinAllowedColumns": [ # Optional. The only columns that joins are allowed on. This field is must be specified for join_conditions JOIN_ANY and JOIN_ALL and it cannot be set for JOIN_BLOCKED.
"A String",
],
"joinCondition": "A String", # Optional. Specifies if a join is required or not on queries for the view. Default is JOIN_CONDITION_UNSPECIFIED.
},
},
"useLegacySql": True or False, # True if view is defined in legacy SQL dialect, false if in GoogleSQL.
},
},
],
"totalItems": 42, # The total number of tables in the dataset.
}
list_next()
Retrieves the next page of results.
Args:
previous_request: The request for the previous page. (required)
previous_response: The response from the request for the previous page. (required)
Returns:
A request object that you can call 'execute()' on to request the next
page. Returns None if there are no more items in the collection.
patch(projectId, datasetId, tableId, autodetect_schema=None, body=None, x__xgafv=None)
Updates information in an existing table. The update method replaces the entire table resource, whereas the patch method only replaces fields that are provided in the submitted table resource. This method supports RFC5789 patch semantics.
Args:
projectId: string, Required. Project ID of the table to update (required)
datasetId: string, Required. Dataset ID of the table to update (required)
tableId: string, Required. Table ID of the table to update (required)
body: object, The request body.
The object takes the form of:
{
"biglakeConfiguration": { # Configuration for BigLake managed tables. # Optional. Specifies the configuration of a BigLake managed table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read and write to external storage, such as Cloud Storage. The connection_id can have the form `{project}.{location}.{connection_id}` or `projects/{project}/locations/{location}/connections/{connection_id}".
"fileFormat": "A String", # Optional. The file format the table data is stored in.
"storageUri": "A String", # Optional. The fully qualified location prefix of the external folder where table data is stored. The '*' wildcard character is not allowed. The URI should be in the format `gs://bucket/path_to_table/`
"tableFormat": "A String", # Optional. The table format the metadata only snapshots are stored in.
},
"cloneDefinition": { # Information about base table and clone time of a table clone. # Output only. Contains information about the clone. This value is set via the clone operation.
"baseTableReference": { # Required. Reference describing the ID of the table that was cloned.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"cloneTime": "A String", # Required. The time at which the base table was cloned. This value is reported in the JSON response using RFC3339 format.
},
"clustering": { # Configures table clustering. # Clustering specification for the table. Must be specified with time-based partitioning, data in the table will be first partitioned and subsequently clustered.
"fields": [ # One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. The ordering of the clustering fields should be prioritized from most to least important for filtering purposes. Additional information on limitations can be found here: https://cloud.google.com/bigquery/docs/creating-clustered-tables#limitations
"A String",
],
},
"creationTime": "A String", # Output only. The time when this table was created, in milliseconds since the epoch.
"defaultCollation": "A String", # Optional. Defines the default collation specification of new STRING fields in the table. During table creation or update, if a STRING field is added to this table without explicit collation specified, then the table inherits the table default collation. A change to this field affects only fields added afterwards, and does not alter the existing fields. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"defaultRoundingMode": "A String", # Optional. Defines the default rounding mode specification of new decimal fields (NUMERIC OR BIGNUMERIC) in the table. During table creation or update, if a decimal field is added to this table without an explicit rounding mode specified, then the field inherits the table default rounding mode. Changing this field doesn't affect existing fields.
"description": "A String", # Optional. A user-friendly description of this table.
"encryptionConfiguration": { # Configuration for Cloud KMS encryption settings. # Custom encryption configuration (e.g., Cloud KMS keys).
"kmsKeyName": "A String", # Optional. Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.
},
"etag": "A String", # Output only. A hash of this resource.
"expirationTime": "A String", # Optional. The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed. The defaultTableExpirationMs property of the encapsulating dataset can be used to set a default expirationTime on newly created tables.
"externalCatalogTableOptions": { # Metadata about open source compatible table. The fields contained in these options correspond to hive metastore's table level properties. # Optional. Options defining open source compatible table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection is needed to read the open source table from BigQuery Engine. The connection_id can have the form `..` or `projects//locations//connections/`.
"parameters": { # Optional. A map of key value pairs defining the parameters and properties of the open source table. Corresponds with hive meta store table parameters. Maximum size of 4Mib.
"a_key": "A String",
},
"storageDescriptor": { # Contains information about how a table's data is stored and accessed by open source query engines. # Optional. A storage descriptor containing information about the physical storage of this table.
"inputFormat": "A String", # Optional. Specifies the fully qualified class name of the InputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"). The maximum length is 128 characters.
"locationUri": "A String", # Optional. The physical location of the table (e.g. `gs://spark-dataproc-data/pangea-data/case_sensitive/` or `gs://spark-dataproc-data/pangea-data/*`). The maximum length is 2056 bytes.
"outputFormat": "A String", # Optional. Specifies the fully qualified class name of the OutputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"). The maximum length is 128 characters.
"serdeInfo": { # Serializer and deserializer information. # Optional. Serializer and deserializer information.
"name": "A String", # Optional. Name of the SerDe. The maximum length is 256 characters.
"parameters": { # Optional. Key-value pairs that define the initialization parameters for the serialization library. Maximum size 10 Kib.
"a_key": "A String",
},
"serializationLibrary": "A String", # Required. Specifies a fully-qualified class name of the serialization library that is responsible for the translation of data between table representation and the underlying low-level input and output format structures. The maximum length is 256 characters.
},
},
},
"externalDataConfiguration": { # Optional. Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table.
"autodetect": True or False, # Try to detect schema and format options automatically. Any option specified explicitly will be honored.
"avroOptions": { # Options for external data sources. # Optional. Additional properties to set if sourceFormat is set to AVRO.
"useAvroLogicalTypes": True or False, # Optional. If sourceFormat is set to "AVRO", indicates whether to interpret logical types as the corresponding BigQuery data type (for example, TIMESTAMP), instead of using the raw type (for example, INTEGER).
},
"bigtableOptions": { # Options specific to Google Cloud Bigtable data sources. # Optional. Additional options if sourceFormat is set to BIGTABLE.
"columnFamilies": [ # Optional. List of column families to expose in the table schema along with their types. This list restricts the column families that can be referenced in queries and specifies their value types. You can use this list to do type conversions - see the 'type' field for more details. If you leave this list empty, all column families are present in the table schema and their values are read as BYTES. During a query only the column families referenced in that query are read from Bigtable.
{ # Information related to a Bigtable column family.
"columns": [ # Optional. Lists of columns that should be exposed as individual fields as opposed to a list of (column name, value) pairs. All columns whose qualifier matches a qualifier in this list can be accessed as `.`. Other columns can be accessed as a list through the `.Column` field.
{ # Information related to a Bigtable column.
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. 'encoding' can also be set at the column family level. However, the setting at this level takes precedence if 'encoding' is set at both levels.
"fieldName": "A String", # Optional. If the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as the column field name and is used as field name in queries.
"onlyReadLatest": True or False, # Optional. If this is set, only the latest version of value in this column are exposed. 'onlyReadLatest' can also be set at the column family level. However, the setting at this level takes precedence if 'onlyReadLatest' is set at both levels.
"qualifierEncoded": "A String", # [Required] Qualifier of the column. Columns in the parent column family that has this exact qualifier are exposed as `.` field. If the qualifier is valid UTF-8 string, it can be specified in the qualifier_string field. Otherwise, a base-64 encoded value must be set to qualifier_encoded. The column field name is the same as the column qualifier. However, if the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as field_name.
"qualifierString": "A String", # Qualifier string.
"type": "A String", # Optional. The type to convert the value in cells of this column. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. 'type' can also be set at the column family level. However, the setting at this level takes precedence if 'type' is set at both levels.
},
],
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. This can be overridden for a specific column by listing that column in 'columns' and specifying an encoding for it.
"familyId": "A String", # Identifier of the column family.
"onlyReadLatest": True or False, # Optional. If this is set only the latest version of value are exposed for all columns in this column family. This can be overridden for a specific column by listing that column in 'columns' and specifying a different setting for that column.
"type": "A String", # Optional. The type to convert the value in cells of this column family. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. This can be overridden for a specific column by listing that column in 'columns' and specifying a type for it.
},
],
"ignoreUnspecifiedColumnFamilies": True or False, # Optional. If field is true, then the column families that are not specified in columnFamilies list are not exposed in the table schema. Otherwise, they are read with BYTES type values. The default value is false.
"outputColumnFamiliesAsJson": True or False, # Optional. If field is true, then each column family will be read as a single JSON column. Otherwise they are read as a repeated cell structure containing timestamp/value tuples. The default value is false.
"readRowkeyAsString": True or False, # Optional. If field is true, then the rowkey column families will be read and converted to string. Otherwise they are read with BYTES type values and users need to manually cast them with CAST if necessary. The default value is false.
},
"compression": "A String", # Optional. The compression type of the data source. Possible values include GZIP and NONE. The default value is NONE. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats. An empty string is an invalid value.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id can have the form `{project_id}.{location_id};{connection_id}` or `projects/{project_id}/locations/{location_id}/connections/{connection_id}`.
"csvOptions": { # Information related to a CSV data source. # Optional. Additional properties to set if sourceFormat is set to CSV.
"allowJaggedRows": True or False, # Optional. Indicates if BigQuery should accept rows that are missing trailing optional columns. If true, BigQuery treats missing trailing columns as null values. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.
"allowQuotedNewlines": True or False, # Optional. Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, ISO-8859-1, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.
"fieldDelimiter": "A String", # Optional. The separator character for fields in a CSV file. The separator is interpreted as a single byte. For files encoded in ISO-8859-1, any single character can be used as a separator. For files encoded in UTF-8, characters represented in decimal range 1-127 (U+0001-U+007F) can be used without any modification. UTF-8 characters encoded with multiple bytes (i.e. U+0080 and above) will have only the first byte used for separating fields. The remaining bytes will be treated as a part of the field. BigQuery also supports the escape sequence "\t" (U+0009) to specify a tab separator. The default value is comma (",", U+002C).
"nullMarker": "A String", # Optional. Specifies a string that represents a null value in a CSV file. For example, if you specify "\N", BigQuery interprets "\N" as a null value when querying a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.
"preserveAsciiControlCharacters": True or False, # Optional. Indicates if the embedded ASCII control characters (the first 32 characters in the ASCII-table, from '\x00' to '\x1F') are preserved.
"quote": """, # Optional. The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote ("). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true. To include the specific quote character within a quoted value, precede it with an additional matching quote character. For example, if you want to escape the default character ' " ', use ' "" '.
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a CSV file that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"decimalTargetTypes": [ # Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown. Example: Suppose the value of this field is ["NUMERIC", "BIGNUMERIC"]. If (precision,scale) is: * (38,9) -> NUMERIC; * (39,9) -> BIGNUMERIC (NUMERIC cannot hold 30 integer digits); * (38,10) -> BIGNUMERIC (NUMERIC cannot hold 10 fractional digits); * (76,38) -> BIGNUMERIC; * (77,38) -> BIGNUMERIC (error if value exeeds supported range). This field cannot contain duplicate types. The order of the types in this field is ignored. For example, ["BIGNUMERIC", "NUMERIC"] is the same as ["NUMERIC", "BIGNUMERIC"] and NUMERIC always takes precedence over BIGNUMERIC. Defaults to ["NUMERIC", "STRING"] for ORC and ["NUMERIC"] for the other file formats.
"A String",
],
"fileSetSpecType": "A String", # Optional. Specifies how source URIs are interpreted for constructing the file set to load. By default source URIs are expanded against the underlying storage. Other options include specifying manifest files. Only applicable to object storage systems.
"googleSheetsOptions": { # Options specific to Google Sheets data sources. # Optional. Additional options if sourceFormat is set to GOOGLE_SHEETS.
"range": "A String", # Optional. Range of a sheet to query from. Only used when non-empty. Typical format: sheet_name!top_left_cell_id:bottom_right_cell_id For example: sheet1!A1:B20
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a sheet that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"hivePartitioningOptions": { # Options for configuring hive partitioning detect. # Optional. When set, configures hive partitioning support. Not all storage formats support hive partitioning -- requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.
"fields": [ # Output only. For permanent external tables, this field is populated with the hive partition keys in the order they were inferred. The types of the partition keys can be deduced by checking the table schema (which will include the partition keys). Not every API will populate this field in the output. For example, Tables.Get will populate it, but Tables.List will not contain this field.
"A String",
],
"mode": "A String", # Optional. When set, what mode of hive partitioning to use when reading data. The following modes are supported: * AUTO: automatically infer partition key name(s) and type(s). * STRINGS: automatically infer partition key name(s). All types are strings. * CUSTOM: partition key schema is encoded in the source URI prefix. Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. Note that this field should only be true when creating a permanent external table or querying a temporary external table. Hive-partitioned loads with require_partition_filter explicitly set to true will fail.
"sourceUriPrefix": "A String", # Optional. When hive partition detection is requested, a common prefix for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout: gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. CUSTOM detection requires encoding the partitioning schema immediately after the common prefix. For CUSTOM, any of * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:STRING}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:STRING} would all be valid source URI prefixes.
},
"ignoreUnknownValues": True or False, # Optional. Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don't match any column names Google Cloud Bigtable: This setting is ignored. Google Cloud Datastore backups: This setting is ignored. Avro: This setting is ignored. ORC: This setting is ignored. Parquet: This setting is ignored.
"jsonExtension": "A String", # Optional. Load option to be used together with source_format newline-delimited JSON to indicate that a variant of JSON is being loaded. To load newline-delimited GeoJSON, specify GEOJSON (and source_format must be set to NEWLINE_DELIMITED_JSON).
"jsonOptions": { # Json Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to JSON.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8.
},
"maxBadRecords": 42, # Optional. The maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats.
"metadataCacheMode": "A String", # Optional. Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source.
"objectMetadata": "A String", # Optional. ObjectMetadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the source_uris. If ObjectMetadata is set, source_format should be omitted. Currently SIMPLE is the only supported Object Metadata type.
"parquetOptions": { # Parquet Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to PARQUET.
"enableListInference": True or False, # Optional. Indicates whether to use schema inference specifically for Parquet LIST logical type.
"enumAsString": True or False, # Optional. Indicates whether to infer Parquet ENUM logical type as STRING instead of BYTES by default.
"mapTargetType": "A String", # Optional. Indicates how to represent a Parquet map if present.
},
"referenceFileSchemaUri": "A String", # Optional. When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.
"schema": { # Schema of a table # Optional. The schema for the data. Schema is required for CSV and JSON formats if autodetect is not on. Schema is disallowed for Google Cloud Bigtable, Cloud Datastore backups, Avro, ORC and Parquet formats.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"sourceFormat": "A String", # [Required] The data format. For CSV files, specify "CSV". For Google sheets, specify "GOOGLE_SHEETS". For newline-delimited JSON, specify "NEWLINE_DELIMITED_JSON". For Avro files, specify "AVRO". For Google Cloud Datastore backups, specify "DATASTORE_BACKUP". For Apache Iceberg tables, specify "ICEBERG". For ORC files, specify "ORC". For Parquet files, specify "PARQUET". [Beta] For Google Cloud Bigtable, specify "BIGTABLE".
"sourceUris": [ # [Required] The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one '*' wildcard character and it must come after the 'bucket' name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the '*' wildcard character is not allowed.
"A String",
],
},
"friendlyName": "A String", # Optional. A descriptive name for this table.
"id": "A String", # Output only. An opaque ID uniquely identifying the table.
"kind": "bigquery#table", # The type of resource ID.
"labels": { # The labels associated with this table. You can use these to organize and group your tables. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.
"a_key": "A String",
},
"lastModifiedTime": "A String", # Output only. The time when this table was last modified, in milliseconds since the epoch.
"location": "A String", # Output only. The geographic location where the table resides. This value is inherited from the dataset.
"materializedView": { # Definition and configuration of a materialized view. # Optional. The materialized view definition.
"allowNonIncrementalDefinition": True or False, # Optional. This option declares the intention to construct a materialized view that isn't refreshed incrementally.
"enableRefresh": True or False, # Optional. Enable automatic refresh of the materialized view when the base table is updated. The default value is "true".
"lastRefreshTime": "A String", # Output only. The time when this materialized view was last refreshed, in milliseconds since the epoch.
"maxStaleness": "A String", # [Optional] Max staleness of data that could be returned when materizlized view is queried (formatted as Google SQL Interval type).
"query": "A String", # Required. A query whose results are persisted.
"refreshIntervalMs": "A String", # Optional. The maximum frequency at which this materialized view will be refreshed. The default value is "1800000" (30 minutes).
},
"materializedViewStatus": { # Status of a materialized view. The last refresh timestamp status is omitted here, but is present in the MaterializedViewDefinition message. # Output only. The materialized view status.
"lastRefreshStatus": { # Error details. # Output only. Error result of the last automatic refresh. If present, indicates that the last automatic refresh was unsuccessful.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"refreshWatermark": "A String", # Output only. Refresh watermark of materialized view. The base tables' data were collected into the materialized view cache until this time.
},
"maxStaleness": "A String", # Optional. The maximum staleness of data that could be returned when the table (or stale MV) is queried. Staleness encoded as a string encoding of sql IntervalValue type.
"model": { # Deprecated.
"modelOptions": { # Deprecated.
"labels": [
"A String",
],
"lossType": "A String",
"modelType": "A String",
},
"trainingRuns": [ # Deprecated.
{
"iterationResults": [ # Deprecated.
{
"durationMs": "A String", # Deprecated.
"evalLoss": 3.14, # Deprecated.
"index": 42, # Deprecated.
"learnRate": 3.14, # Deprecated.
"trainingLoss": 3.14, # Deprecated.
},
],
"startTime": "A String", # Deprecated.
"state": "A String", # Deprecated.
"trainingOptions": { # Deprecated.
"earlyStop": True or False,
"l1Reg": 3.14,
"l2Reg": 3.14,
"learnRate": 3.14,
"learnRateStrategy": "A String",
"lineSearchInitLearnRate": 3.14,
"maxIteration": "A String",
"minRelProgress": 3.14,
"warmStart": True or False,
},
},
],
},
"numActiveLogicalBytes": "A String", # Output only. Number of logical bytes that are less than 90 days old.
"numActivePhysicalBytes": "A String", # Output only. Number of physical bytes less than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numBytes": "A String", # Output only. The size of this table in logical bytes, excluding any data in the streaming buffer.
"numCurrentPhysicalBytes": "A String", # Output only. Number of physical bytes used by current live data storage. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numLongTermBytes": "A String", # Output only. The number of logical bytes in the table that are considered "long-term storage".
"numLongTermLogicalBytes": "A String", # Output only. Number of logical bytes that are more than 90 days old.
"numLongTermPhysicalBytes": "A String", # Output only. Number of physical bytes more than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPartitions": "A String", # Output only. The number of partitions present in the table or materialized view. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This includes storage used for time travel.
"numRows": "A String", # Output only. The number of rows of data in this table, excluding any data in the streaming buffer.
"numTimeTravelPhysicalBytes": "A String", # Output only. Number of physical bytes used by time travel storage (deleted or changed data). This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numTotalLogicalBytes": "A String", # Output only. Total number of logical bytes in the table or materialized view.
"numTotalPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This also includes storage used for time travel. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"partitionDefinition": { # The partitioning information, which includes managed table, external table and metastore partitioned table partition information. # Optional. The partition information for all table formats, including managed partitioned tables, hive partitioned tables, iceberg partitioned, and metastore partitioned tables. This field is only populated for metastore partitioned tables. For other table formats, this is an output only field.
"partitionedColumn": [ # Optional. Details about each partitioning column. This field is output only for all partitioning types other than metastore partitioned tables. BigQuery native tables only support 1 partitioning column. Other table types may support 0, 1 or more partitioning columns. For metastore partitioned tables, the order must match the definition order in the Hive Metastore, where it must match the physical layout of the table. For example, CREATE TABLE a_table(id BIGINT, name STRING) PARTITIONED BY (city STRING, state STRING). In this case the values must be ['city', 'state'] in that order.
{ # The partitioning column information.
"field": "A String", # Required. The name of the partition column.
},
],
},
"rangePartitioning": { # If specified, configures range partitioning for this table.
"field": "A String", # Required. The name of the column to partition the table on. It must be a top-level, INT64 column whose mode is NULLABLE or REQUIRED.
"range": { # [Experimental] Defines the ranges for range partitioning.
"end": "A String", # [Experimental] The end of range partitioning, exclusive.
"interval": "A String", # [Experimental] The width of each interval.
"start": "A String", # [Experimental] The start of range partitioning, inclusive.
},
},
"replicas": [ # Optional. Output only. Table references of all replicas currently active on the table.
{
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
],
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.
"resourceTags": { # [Optional] The tags associated with this table. Tag keys are globally unique. See additional information on [tags](https://cloud.google.com/iam/docs/tags-access-control#definitions). An object containing a list of "key": value pairs. The key is the namespaced friendly name of the tag key, e.g. "12345/environment" where 12345 is parent id. The value is the friendly short name of the tag value, e.g. "production".
"a_key": "A String",
},
"restrictions": { # Optional. Output only. Restriction config for table. If set, restrict certain accesses on the table based on the config. See [Data egress](https://cloud.google.com/bigquery/docs/analytics-hub-introduction#data_egress) for more details.
"type": "A String", # Output only. Specifies the type of dataset/table restriction.
},
"schema": { # Schema of a table # Optional. Describes the schema of this table.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"selfLink": "A String", # Output only. A URL that can be used to access this resource again.
"snapshotDefinition": { # Information about base table and snapshot time of the snapshot. # Output only. Contains information about the snapshot. This value is set via snapshot creation.
"baseTableReference": { # Required. Reference describing the ID of the table that was snapshot.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"snapshotTime": "A String", # Required. The time at which the base table was snapshot. This value is reported in the JSON response using RFC3339 format.
},
"streamingBuffer": { # Output only. Contains information regarding this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
"estimatedBytes": "A String", # Output only. A lower-bound estimate of the number of bytes currently in the streaming buffer.
"estimatedRows": "A String", # Output only. A lower-bound estimate of the number of rows currently in the streaming buffer.
"oldestEntryTime": "A String", # Output only. Contains the timestamp of the oldest entry in the streaming buffer, in milliseconds since the epoch, if the streaming buffer is available.
},
"tableConstraints": { # The TableConstraints defines the primary key and foreign key. # Optional. Tables Primary Key and Foreign Key information
"foreignKeys": [ # Optional. Present only if the table has a foreign key. The foreign key is not enforced.
{ # Represents a foreign key constraint on a table's columns.
"columnReferences": [ # Required. The columns that compose the foreign key.
{ # The pair of the foreign key column and primary key column.
"referencedColumn": "A String", # Required. The column in the primary key that are referenced by the referencing_column.
"referencingColumn": "A String", # Required. The column that composes the foreign key.
},
],
"name": "A String", # Optional. Set only if the foreign key constraint is named.
"referencedTable": {
"datasetId": "A String",
"projectId": "A String",
"tableId": "A String",
},
},
],
"primaryKey": { # Represents the primary key constraint on a table's columns.
"columns": [ # Required. The columns that are composed of the primary key constraint.
"A String",
],
},
},
"tableReference": { # Required. Reference describing the ID of this table.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"tableReplicationInfo": { # Replication info of a table created using `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv` # Optional. Table replication info for table created `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv`
"replicatedSourceLastRefreshTime": "A String", # Optional. Output only. If source is a materialized view, this field signifies the last refresh time of the source.
"replicationError": { # Error details. # Optional. Output only. Replication error that will permanently stopped table replication.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"replicationIntervalMs": "A String", # Optional. Specifies the interval at which the source table is polled for updates. It's Optional. If not specified, default replication interval would be applied.
"replicationStatus": "A String", # Optional. Output only. Replication status of configured replication.
"sourceTable": { # Required. Source table reference that is replicated.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
},
"timePartitioning": { # If specified, configures time-based partitioning for this table.
"expirationMs": "A String", # Optional. Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.
"field": "A String", # Optional. If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.
"requirePartitionFilter": false, # If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. This field is deprecated; please set the field with the same name on the table itself instead. This field needs a wrapper because we want to output the default value, false, if the user explicitly set it.
"type": "A String", # Required. The supported types are DAY, HOUR, MONTH, and YEAR, which will generate one partition per day, hour, month, and year, respectively.
},
"type": "A String", # Output only. Describes the table type. The following values are supported: * `TABLE`: A normal BigQuery table. * `VIEW`: A virtual table defined by a SQL query. * `EXTERNAL`: A table that references data stored in an external storage system, such as Google Cloud Storage. * `MATERIALIZED_VIEW`: A precomputed view defined by a SQL query. * `SNAPSHOT`: An immutable BigQuery table that preserves the contents of a base table at a particular time. See additional information on [table snapshots](https://cloud.google.com/bigquery/docs/table-snapshots-intro). The default value is `TABLE`.
"view": { # Describes the definition of a logical view. # Optional. The view definition.
"foreignDefinitions": [ # Optional. Foreign view representations.
{ # A view can be represented in multiple ways. Each representation has its own dialect. This message stores the metadata required for these representations.
"dialect": "A String", # Optional. Represents the dialect of the query.
"query": "A String", # Required. The query that defines the view.
},
],
"privacyPolicy": { # Represents privacy policy that contains the privacy requirements specified by the data owner. Currently, this is only supported on views. # Optional. Specifices the privacy policy for the view.
"aggregationThresholdPolicy": { # Represents privacy policy associated with "aggregation threshold" method. # Optional. Policy used for aggregation thresholds.
"privacyUnitColumns": [ # Optional. The privacy unit column(s) associated with this policy. For now, only one column per data source object (table, view) is allowed as a privacy unit column. Representing as a repeated field in metadata for extensibility to multiple columns in future. Duplicates and Repeated struct fields are not allowed. For nested fields, use dot notation ("outer.inner")
"A String",
],
"threshold": "A String", # Optional. The threshold for the "aggregation threshold" policy.
},
"differentialPrivacyPolicy": { # Represents privacy policy associated with "differential privacy" method. # Optional. Policy used for differential privacy.
"deltaBudget": 3.14, # Optional. The total delta budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of delta that is pre-defined by the contributor through the privacy policy delta_per_query field. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"deltaBudgetRemaining": 3.14, # Output only. The delta budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"deltaPerQuery": 3.14, # Optional. The delta value that is used per query. Delta represents the probability that any row will fail to be epsilon differentially private. Indicates the risk associated with exposing aggregate rows in the result of a query.
"epsilonBudget": 3.14, # Optional. The total epsilon budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of epsilon they request in their query. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"epsilonBudgetRemaining": 3.14, # Output only. The epsilon budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"maxEpsilonPerQuery": 3.14, # Optional. The maximum epsilon value that a query can consume. If the subscriber specifies epsilon as a parameter in a SELECT query, it must be less than or equal to this value. The epsilon parameter controls the amount of noise that is added to the groups — a higher epsilon means less noise.
"maxGroupsContributed": "A String", # Optional. The maximum groups contributed value that is used per query. Represents the maximum number of groups to which each protected entity can contribute. Changing this value does not improve or worsen privacy. The best value for accuracy and utility depends on the query and data.
"privacyUnitColumn": "A String", # Optional. The privacy unit column associated with this policy. Differential privacy policies can only have one privacy unit column per data source object (table, view).
},
"joinRestrictionPolicy": { # Represents privacy policy associated with "join restrictions". Join restriction gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view. # Optional. Join restriction policy is outside of the one of policies, since this policy can be set along with other policies. This policy gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view.
"joinAllowedColumns": [ # Optional. The only columns that joins are allowed on. This field is must be specified for join_conditions JOIN_ANY and JOIN_ALL and it cannot be set for JOIN_BLOCKED.
"A String",
],
"joinCondition": "A String", # Optional. Specifies if a join is required or not on queries for the view. Default is JOIN_CONDITION_UNSPECIFIED.
},
},
"query": "A String", # Required. A query that BigQuery executes when the view is referenced.
"useExplicitColumnNames": True or False, # True if the column names are explicitly specified. For example by using the 'CREATE VIEW v(c1, c2) AS ...' syntax. Can only be set for GoogleSQL views.
"useLegacySql": True or False, # Specifies whether to use BigQuery's legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery's GoogleSQL: https://cloud.google.com/bigquery/sql-reference/ Queries and views that reference this view must use the same flag value. A wrapper is used here because the default value is True.
"userDefinedFunctionResources": [ # Describes user-defined function resources used in the query.
{ # This is used for defining User Defined Function (UDF) resources only when using legacy SQL. Users of GoogleSQL should leverage either DDL (e.g. CREATE [TEMPORARY] FUNCTION ... ) or the Routines API to define UDF resources. For additional information on migrating, see: https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql#differences_in_user-defined_javascript_functions
"inlineCode": "A String", # [Pick one] An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.
"resourceUri": "A String", # [Pick one] A code resource to load from a Google Cloud Storage URI (gs://bucket/path).
},
],
},
}
autodetect_schema: boolean, Optional. When true will autodetect schema, else will keep original schema
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{
"biglakeConfiguration": { # Configuration for BigLake managed tables. # Optional. Specifies the configuration of a BigLake managed table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read and write to external storage, such as Cloud Storage. The connection_id can have the form `{project}.{location}.{connection_id}` or `projects/{project}/locations/{location}/connections/{connection_id}".
"fileFormat": "A String", # Optional. The file format the table data is stored in.
"storageUri": "A String", # Optional. The fully qualified location prefix of the external folder where table data is stored. The '*' wildcard character is not allowed. The URI should be in the format `gs://bucket/path_to_table/`
"tableFormat": "A String", # Optional. The table format the metadata only snapshots are stored in.
},
"cloneDefinition": { # Information about base table and clone time of a table clone. # Output only. Contains information about the clone. This value is set via the clone operation.
"baseTableReference": { # Required. Reference describing the ID of the table that was cloned.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"cloneTime": "A String", # Required. The time at which the base table was cloned. This value is reported in the JSON response using RFC3339 format.
},
"clustering": { # Configures table clustering. # Clustering specification for the table. Must be specified with time-based partitioning, data in the table will be first partitioned and subsequently clustered.
"fields": [ # One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. The ordering of the clustering fields should be prioritized from most to least important for filtering purposes. Additional information on limitations can be found here: https://cloud.google.com/bigquery/docs/creating-clustered-tables#limitations
"A String",
],
},
"creationTime": "A String", # Output only. The time when this table was created, in milliseconds since the epoch.
"defaultCollation": "A String", # Optional. Defines the default collation specification of new STRING fields in the table. During table creation or update, if a STRING field is added to this table without explicit collation specified, then the table inherits the table default collation. A change to this field affects only fields added afterwards, and does not alter the existing fields. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"defaultRoundingMode": "A String", # Optional. Defines the default rounding mode specification of new decimal fields (NUMERIC OR BIGNUMERIC) in the table. During table creation or update, if a decimal field is added to this table without an explicit rounding mode specified, then the field inherits the table default rounding mode. Changing this field doesn't affect existing fields.
"description": "A String", # Optional. A user-friendly description of this table.
"encryptionConfiguration": { # Configuration for Cloud KMS encryption settings. # Custom encryption configuration (e.g., Cloud KMS keys).
"kmsKeyName": "A String", # Optional. Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.
},
"etag": "A String", # Output only. A hash of this resource.
"expirationTime": "A String", # Optional. The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed. The defaultTableExpirationMs property of the encapsulating dataset can be used to set a default expirationTime on newly created tables.
"externalCatalogTableOptions": { # Metadata about open source compatible table. The fields contained in these options correspond to hive metastore's table level properties. # Optional. Options defining open source compatible table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection is needed to read the open source table from BigQuery Engine. The connection_id can have the form `..` or `projects//locations//connections/`.
"parameters": { # Optional. A map of key value pairs defining the parameters and properties of the open source table. Corresponds with hive meta store table parameters. Maximum size of 4Mib.
"a_key": "A String",
},
"storageDescriptor": { # Contains information about how a table's data is stored and accessed by open source query engines. # Optional. A storage descriptor containing information about the physical storage of this table.
"inputFormat": "A String", # Optional. Specifies the fully qualified class name of the InputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"). The maximum length is 128 characters.
"locationUri": "A String", # Optional. The physical location of the table (e.g. `gs://spark-dataproc-data/pangea-data/case_sensitive/` or `gs://spark-dataproc-data/pangea-data/*`). The maximum length is 2056 bytes.
"outputFormat": "A String", # Optional. Specifies the fully qualified class name of the OutputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"). The maximum length is 128 characters.
"serdeInfo": { # Serializer and deserializer information. # Optional. Serializer and deserializer information.
"name": "A String", # Optional. Name of the SerDe. The maximum length is 256 characters.
"parameters": { # Optional. Key-value pairs that define the initialization parameters for the serialization library. Maximum size 10 Kib.
"a_key": "A String",
},
"serializationLibrary": "A String", # Required. Specifies a fully-qualified class name of the serialization library that is responsible for the translation of data between table representation and the underlying low-level input and output format structures. The maximum length is 256 characters.
},
},
},
"externalDataConfiguration": { # Optional. Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table.
"autodetect": True or False, # Try to detect schema and format options automatically. Any option specified explicitly will be honored.
"avroOptions": { # Options for external data sources. # Optional. Additional properties to set if sourceFormat is set to AVRO.
"useAvroLogicalTypes": True or False, # Optional. If sourceFormat is set to "AVRO", indicates whether to interpret logical types as the corresponding BigQuery data type (for example, TIMESTAMP), instead of using the raw type (for example, INTEGER).
},
"bigtableOptions": { # Options specific to Google Cloud Bigtable data sources. # Optional. Additional options if sourceFormat is set to BIGTABLE.
"columnFamilies": [ # Optional. List of column families to expose in the table schema along with their types. This list restricts the column families that can be referenced in queries and specifies their value types. You can use this list to do type conversions - see the 'type' field for more details. If you leave this list empty, all column families are present in the table schema and their values are read as BYTES. During a query only the column families referenced in that query are read from Bigtable.
{ # Information related to a Bigtable column family.
"columns": [ # Optional. Lists of columns that should be exposed as individual fields as opposed to a list of (column name, value) pairs. All columns whose qualifier matches a qualifier in this list can be accessed as `.`. Other columns can be accessed as a list through the `.Column` field.
{ # Information related to a Bigtable column.
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. 'encoding' can also be set at the column family level. However, the setting at this level takes precedence if 'encoding' is set at both levels.
"fieldName": "A String", # Optional. If the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as the column field name and is used as field name in queries.
"onlyReadLatest": True or False, # Optional. If this is set, only the latest version of value in this column are exposed. 'onlyReadLatest' can also be set at the column family level. However, the setting at this level takes precedence if 'onlyReadLatest' is set at both levels.
"qualifierEncoded": "A String", # [Required] Qualifier of the column. Columns in the parent column family that has this exact qualifier are exposed as `.` field. If the qualifier is valid UTF-8 string, it can be specified in the qualifier_string field. Otherwise, a base-64 encoded value must be set to qualifier_encoded. The column field name is the same as the column qualifier. However, if the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as field_name.
"qualifierString": "A String", # Qualifier string.
"type": "A String", # Optional. The type to convert the value in cells of this column. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. 'type' can also be set at the column family level. However, the setting at this level takes precedence if 'type' is set at both levels.
},
],
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. This can be overridden for a specific column by listing that column in 'columns' and specifying an encoding for it.
"familyId": "A String", # Identifier of the column family.
"onlyReadLatest": True or False, # Optional. If this is set only the latest version of value are exposed for all columns in this column family. This can be overridden for a specific column by listing that column in 'columns' and specifying a different setting for that column.
"type": "A String", # Optional. The type to convert the value in cells of this column family. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. This can be overridden for a specific column by listing that column in 'columns' and specifying a type for it.
},
],
"ignoreUnspecifiedColumnFamilies": True or False, # Optional. If field is true, then the column families that are not specified in columnFamilies list are not exposed in the table schema. Otherwise, they are read with BYTES type values. The default value is false.
"outputColumnFamiliesAsJson": True or False, # Optional. If field is true, then each column family will be read as a single JSON column. Otherwise they are read as a repeated cell structure containing timestamp/value tuples. The default value is false.
"readRowkeyAsString": True or False, # Optional. If field is true, then the rowkey column families will be read and converted to string. Otherwise they are read with BYTES type values and users need to manually cast them with CAST if necessary. The default value is false.
},
"compression": "A String", # Optional. The compression type of the data source. Possible values include GZIP and NONE. The default value is NONE. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats. An empty string is an invalid value.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id can have the form `{project_id}.{location_id};{connection_id}` or `projects/{project_id}/locations/{location_id}/connections/{connection_id}`.
"csvOptions": { # Information related to a CSV data source. # Optional. Additional properties to set if sourceFormat is set to CSV.
"allowJaggedRows": True or False, # Optional. Indicates if BigQuery should accept rows that are missing trailing optional columns. If true, BigQuery treats missing trailing columns as null values. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.
"allowQuotedNewlines": True or False, # Optional. Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, ISO-8859-1, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.
"fieldDelimiter": "A String", # Optional. The separator character for fields in a CSV file. The separator is interpreted as a single byte. For files encoded in ISO-8859-1, any single character can be used as a separator. For files encoded in UTF-8, characters represented in decimal range 1-127 (U+0001-U+007F) can be used without any modification. UTF-8 characters encoded with multiple bytes (i.e. U+0080 and above) will have only the first byte used for separating fields. The remaining bytes will be treated as a part of the field. BigQuery also supports the escape sequence "\t" (U+0009) to specify a tab separator. The default value is comma (",", U+002C).
"nullMarker": "A String", # Optional. Specifies a string that represents a null value in a CSV file. For example, if you specify "\N", BigQuery interprets "\N" as a null value when querying a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.
"preserveAsciiControlCharacters": True or False, # Optional. Indicates if the embedded ASCII control characters (the first 32 characters in the ASCII-table, from '\x00' to '\x1F') are preserved.
"quote": """, # Optional. The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote ("). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true. To include the specific quote character within a quoted value, precede it with an additional matching quote character. For example, if you want to escape the default character ' " ', use ' "" '.
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a CSV file that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"decimalTargetTypes": [ # Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown. Example: Suppose the value of this field is ["NUMERIC", "BIGNUMERIC"]. If (precision,scale) is: * (38,9) -> NUMERIC; * (39,9) -> BIGNUMERIC (NUMERIC cannot hold 30 integer digits); * (38,10) -> BIGNUMERIC (NUMERIC cannot hold 10 fractional digits); * (76,38) -> BIGNUMERIC; * (77,38) -> BIGNUMERIC (error if value exeeds supported range). This field cannot contain duplicate types. The order of the types in this field is ignored. For example, ["BIGNUMERIC", "NUMERIC"] is the same as ["NUMERIC", "BIGNUMERIC"] and NUMERIC always takes precedence over BIGNUMERIC. Defaults to ["NUMERIC", "STRING"] for ORC and ["NUMERIC"] for the other file formats.
"A String",
],
"fileSetSpecType": "A String", # Optional. Specifies how source URIs are interpreted for constructing the file set to load. By default source URIs are expanded against the underlying storage. Other options include specifying manifest files. Only applicable to object storage systems.
"googleSheetsOptions": { # Options specific to Google Sheets data sources. # Optional. Additional options if sourceFormat is set to GOOGLE_SHEETS.
"range": "A String", # Optional. Range of a sheet to query from. Only used when non-empty. Typical format: sheet_name!top_left_cell_id:bottom_right_cell_id For example: sheet1!A1:B20
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a sheet that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"hivePartitioningOptions": { # Options for configuring hive partitioning detect. # Optional. When set, configures hive partitioning support. Not all storage formats support hive partitioning -- requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.
"fields": [ # Output only. For permanent external tables, this field is populated with the hive partition keys in the order they were inferred. The types of the partition keys can be deduced by checking the table schema (which will include the partition keys). Not every API will populate this field in the output. For example, Tables.Get will populate it, but Tables.List will not contain this field.
"A String",
],
"mode": "A String", # Optional. When set, what mode of hive partitioning to use when reading data. The following modes are supported: * AUTO: automatically infer partition key name(s) and type(s). * STRINGS: automatically infer partition key name(s). All types are strings. * CUSTOM: partition key schema is encoded in the source URI prefix. Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. Note that this field should only be true when creating a permanent external table or querying a temporary external table. Hive-partitioned loads with require_partition_filter explicitly set to true will fail.
"sourceUriPrefix": "A String", # Optional. When hive partition detection is requested, a common prefix for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout: gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. CUSTOM detection requires encoding the partitioning schema immediately after the common prefix. For CUSTOM, any of * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:STRING}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:STRING} would all be valid source URI prefixes.
},
"ignoreUnknownValues": True or False, # Optional. Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don't match any column names Google Cloud Bigtable: This setting is ignored. Google Cloud Datastore backups: This setting is ignored. Avro: This setting is ignored. ORC: This setting is ignored. Parquet: This setting is ignored.
"jsonExtension": "A String", # Optional. Load option to be used together with source_format newline-delimited JSON to indicate that a variant of JSON is being loaded. To load newline-delimited GeoJSON, specify GEOJSON (and source_format must be set to NEWLINE_DELIMITED_JSON).
"jsonOptions": { # Json Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to JSON.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8.
},
"maxBadRecords": 42, # Optional. The maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats.
"metadataCacheMode": "A String", # Optional. Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source.
"objectMetadata": "A String", # Optional. ObjectMetadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the source_uris. If ObjectMetadata is set, source_format should be omitted. Currently SIMPLE is the only supported Object Metadata type.
"parquetOptions": { # Parquet Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to PARQUET.
"enableListInference": True or False, # Optional. Indicates whether to use schema inference specifically for Parquet LIST logical type.
"enumAsString": True or False, # Optional. Indicates whether to infer Parquet ENUM logical type as STRING instead of BYTES by default.
"mapTargetType": "A String", # Optional. Indicates how to represent a Parquet map if present.
},
"referenceFileSchemaUri": "A String", # Optional. When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.
"schema": { # Schema of a table # Optional. The schema for the data. Schema is required for CSV and JSON formats if autodetect is not on. Schema is disallowed for Google Cloud Bigtable, Cloud Datastore backups, Avro, ORC and Parquet formats.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"sourceFormat": "A String", # [Required] The data format. For CSV files, specify "CSV". For Google sheets, specify "GOOGLE_SHEETS". For newline-delimited JSON, specify "NEWLINE_DELIMITED_JSON". For Avro files, specify "AVRO". For Google Cloud Datastore backups, specify "DATASTORE_BACKUP". For Apache Iceberg tables, specify "ICEBERG". For ORC files, specify "ORC". For Parquet files, specify "PARQUET". [Beta] For Google Cloud Bigtable, specify "BIGTABLE".
"sourceUris": [ # [Required] The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one '*' wildcard character and it must come after the 'bucket' name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the '*' wildcard character is not allowed.
"A String",
],
},
"friendlyName": "A String", # Optional. A descriptive name for this table.
"id": "A String", # Output only. An opaque ID uniquely identifying the table.
"kind": "bigquery#table", # The type of resource ID.
"labels": { # The labels associated with this table. You can use these to organize and group your tables. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.
"a_key": "A String",
},
"lastModifiedTime": "A String", # Output only. The time when this table was last modified, in milliseconds since the epoch.
"location": "A String", # Output only. The geographic location where the table resides. This value is inherited from the dataset.
"materializedView": { # Definition and configuration of a materialized view. # Optional. The materialized view definition.
"allowNonIncrementalDefinition": True or False, # Optional. This option declares the intention to construct a materialized view that isn't refreshed incrementally.
"enableRefresh": True or False, # Optional. Enable automatic refresh of the materialized view when the base table is updated. The default value is "true".
"lastRefreshTime": "A String", # Output only. The time when this materialized view was last refreshed, in milliseconds since the epoch.
"maxStaleness": "A String", # [Optional] Max staleness of data that could be returned when materizlized view is queried (formatted as Google SQL Interval type).
"query": "A String", # Required. A query whose results are persisted.
"refreshIntervalMs": "A String", # Optional. The maximum frequency at which this materialized view will be refreshed. The default value is "1800000" (30 minutes).
},
"materializedViewStatus": { # Status of a materialized view. The last refresh timestamp status is omitted here, but is present in the MaterializedViewDefinition message. # Output only. The materialized view status.
"lastRefreshStatus": { # Error details. # Output only. Error result of the last automatic refresh. If present, indicates that the last automatic refresh was unsuccessful.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"refreshWatermark": "A String", # Output only. Refresh watermark of materialized view. The base tables' data were collected into the materialized view cache until this time.
},
"maxStaleness": "A String", # Optional. The maximum staleness of data that could be returned when the table (or stale MV) is queried. Staleness encoded as a string encoding of sql IntervalValue type.
"model": { # Deprecated.
"modelOptions": { # Deprecated.
"labels": [
"A String",
],
"lossType": "A String",
"modelType": "A String",
},
"trainingRuns": [ # Deprecated.
{
"iterationResults": [ # Deprecated.
{
"durationMs": "A String", # Deprecated.
"evalLoss": 3.14, # Deprecated.
"index": 42, # Deprecated.
"learnRate": 3.14, # Deprecated.
"trainingLoss": 3.14, # Deprecated.
},
],
"startTime": "A String", # Deprecated.
"state": "A String", # Deprecated.
"trainingOptions": { # Deprecated.
"earlyStop": True or False,
"l1Reg": 3.14,
"l2Reg": 3.14,
"learnRate": 3.14,
"learnRateStrategy": "A String",
"lineSearchInitLearnRate": 3.14,
"maxIteration": "A String",
"minRelProgress": 3.14,
"warmStart": True or False,
},
},
],
},
"numActiveLogicalBytes": "A String", # Output only. Number of logical bytes that are less than 90 days old.
"numActivePhysicalBytes": "A String", # Output only. Number of physical bytes less than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numBytes": "A String", # Output only. The size of this table in logical bytes, excluding any data in the streaming buffer.
"numCurrentPhysicalBytes": "A String", # Output only. Number of physical bytes used by current live data storage. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numLongTermBytes": "A String", # Output only. The number of logical bytes in the table that are considered "long-term storage".
"numLongTermLogicalBytes": "A String", # Output only. Number of logical bytes that are more than 90 days old.
"numLongTermPhysicalBytes": "A String", # Output only. Number of physical bytes more than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPartitions": "A String", # Output only. The number of partitions present in the table or materialized view. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This includes storage used for time travel.
"numRows": "A String", # Output only. The number of rows of data in this table, excluding any data in the streaming buffer.
"numTimeTravelPhysicalBytes": "A String", # Output only. Number of physical bytes used by time travel storage (deleted or changed data). This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numTotalLogicalBytes": "A String", # Output only. Total number of logical bytes in the table or materialized view.
"numTotalPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This also includes storage used for time travel. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"partitionDefinition": { # The partitioning information, which includes managed table, external table and metastore partitioned table partition information. # Optional. The partition information for all table formats, including managed partitioned tables, hive partitioned tables, iceberg partitioned, and metastore partitioned tables. This field is only populated for metastore partitioned tables. For other table formats, this is an output only field.
"partitionedColumn": [ # Optional. Details about each partitioning column. This field is output only for all partitioning types other than metastore partitioned tables. BigQuery native tables only support 1 partitioning column. Other table types may support 0, 1 or more partitioning columns. For metastore partitioned tables, the order must match the definition order in the Hive Metastore, where it must match the physical layout of the table. For example, CREATE TABLE a_table(id BIGINT, name STRING) PARTITIONED BY (city STRING, state STRING). In this case the values must be ['city', 'state'] in that order.
{ # The partitioning column information.
"field": "A String", # Required. The name of the partition column.
},
],
},
"rangePartitioning": { # If specified, configures range partitioning for this table.
"field": "A String", # Required. The name of the column to partition the table on. It must be a top-level, INT64 column whose mode is NULLABLE or REQUIRED.
"range": { # [Experimental] Defines the ranges for range partitioning.
"end": "A String", # [Experimental] The end of range partitioning, exclusive.
"interval": "A String", # [Experimental] The width of each interval.
"start": "A String", # [Experimental] The start of range partitioning, inclusive.
},
},
"replicas": [ # Optional. Output only. Table references of all replicas currently active on the table.
{
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
],
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.
"resourceTags": { # [Optional] The tags associated with this table. Tag keys are globally unique. See additional information on [tags](https://cloud.google.com/iam/docs/tags-access-control#definitions). An object containing a list of "key": value pairs. The key is the namespaced friendly name of the tag key, e.g. "12345/environment" where 12345 is parent id. The value is the friendly short name of the tag value, e.g. "production".
"a_key": "A String",
},
"restrictions": { # Optional. Output only. Restriction config for table. If set, restrict certain accesses on the table based on the config. See [Data egress](https://cloud.google.com/bigquery/docs/analytics-hub-introduction#data_egress) for more details.
"type": "A String", # Output only. Specifies the type of dataset/table restriction.
},
"schema": { # Schema of a table # Optional. Describes the schema of this table.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"selfLink": "A String", # Output only. A URL that can be used to access this resource again.
"snapshotDefinition": { # Information about base table and snapshot time of the snapshot. # Output only. Contains information about the snapshot. This value is set via snapshot creation.
"baseTableReference": { # Required. Reference describing the ID of the table that was snapshot.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"snapshotTime": "A String", # Required. The time at which the base table was snapshot. This value is reported in the JSON response using RFC3339 format.
},
"streamingBuffer": { # Output only. Contains information regarding this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
"estimatedBytes": "A String", # Output only. A lower-bound estimate of the number of bytes currently in the streaming buffer.
"estimatedRows": "A String", # Output only. A lower-bound estimate of the number of rows currently in the streaming buffer.
"oldestEntryTime": "A String", # Output only. Contains the timestamp of the oldest entry in the streaming buffer, in milliseconds since the epoch, if the streaming buffer is available.
},
"tableConstraints": { # The TableConstraints defines the primary key and foreign key. # Optional. Tables Primary Key and Foreign Key information
"foreignKeys": [ # Optional. Present only if the table has a foreign key. The foreign key is not enforced.
{ # Represents a foreign key constraint on a table's columns.
"columnReferences": [ # Required. The columns that compose the foreign key.
{ # The pair of the foreign key column and primary key column.
"referencedColumn": "A String", # Required. The column in the primary key that are referenced by the referencing_column.
"referencingColumn": "A String", # Required. The column that composes the foreign key.
},
],
"name": "A String", # Optional. Set only if the foreign key constraint is named.
"referencedTable": {
"datasetId": "A String",
"projectId": "A String",
"tableId": "A String",
},
},
],
"primaryKey": { # Represents the primary key constraint on a table's columns.
"columns": [ # Required. The columns that are composed of the primary key constraint.
"A String",
],
},
},
"tableReference": { # Required. Reference describing the ID of this table.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"tableReplicationInfo": { # Replication info of a table created using `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv` # Optional. Table replication info for table created `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv`
"replicatedSourceLastRefreshTime": "A String", # Optional. Output only. If source is a materialized view, this field signifies the last refresh time of the source.
"replicationError": { # Error details. # Optional. Output only. Replication error that will permanently stopped table replication.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"replicationIntervalMs": "A String", # Optional. Specifies the interval at which the source table is polled for updates. It's Optional. If not specified, default replication interval would be applied.
"replicationStatus": "A String", # Optional. Output only. Replication status of configured replication.
"sourceTable": { # Required. Source table reference that is replicated.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
},
"timePartitioning": { # If specified, configures time-based partitioning for this table.
"expirationMs": "A String", # Optional. Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.
"field": "A String", # Optional. If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.
"requirePartitionFilter": false, # If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. This field is deprecated; please set the field with the same name on the table itself instead. This field needs a wrapper because we want to output the default value, false, if the user explicitly set it.
"type": "A String", # Required. The supported types are DAY, HOUR, MONTH, and YEAR, which will generate one partition per day, hour, month, and year, respectively.
},
"type": "A String", # Output only. Describes the table type. The following values are supported: * `TABLE`: A normal BigQuery table. * `VIEW`: A virtual table defined by a SQL query. * `EXTERNAL`: A table that references data stored in an external storage system, such as Google Cloud Storage. * `MATERIALIZED_VIEW`: A precomputed view defined by a SQL query. * `SNAPSHOT`: An immutable BigQuery table that preserves the contents of a base table at a particular time. See additional information on [table snapshots](https://cloud.google.com/bigquery/docs/table-snapshots-intro). The default value is `TABLE`.
"view": { # Describes the definition of a logical view. # Optional. The view definition.
"foreignDefinitions": [ # Optional. Foreign view representations.
{ # A view can be represented in multiple ways. Each representation has its own dialect. This message stores the metadata required for these representations.
"dialect": "A String", # Optional. Represents the dialect of the query.
"query": "A String", # Required. The query that defines the view.
},
],
"privacyPolicy": { # Represents privacy policy that contains the privacy requirements specified by the data owner. Currently, this is only supported on views. # Optional. Specifices the privacy policy for the view.
"aggregationThresholdPolicy": { # Represents privacy policy associated with "aggregation threshold" method. # Optional. Policy used for aggregation thresholds.
"privacyUnitColumns": [ # Optional. The privacy unit column(s) associated with this policy. For now, only one column per data source object (table, view) is allowed as a privacy unit column. Representing as a repeated field in metadata for extensibility to multiple columns in future. Duplicates and Repeated struct fields are not allowed. For nested fields, use dot notation ("outer.inner")
"A String",
],
"threshold": "A String", # Optional. The threshold for the "aggregation threshold" policy.
},
"differentialPrivacyPolicy": { # Represents privacy policy associated with "differential privacy" method. # Optional. Policy used for differential privacy.
"deltaBudget": 3.14, # Optional. The total delta budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of delta that is pre-defined by the contributor through the privacy policy delta_per_query field. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"deltaBudgetRemaining": 3.14, # Output only. The delta budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"deltaPerQuery": 3.14, # Optional. The delta value that is used per query. Delta represents the probability that any row will fail to be epsilon differentially private. Indicates the risk associated with exposing aggregate rows in the result of a query.
"epsilonBudget": 3.14, # Optional. The total epsilon budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of epsilon they request in their query. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"epsilonBudgetRemaining": 3.14, # Output only. The epsilon budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"maxEpsilonPerQuery": 3.14, # Optional. The maximum epsilon value that a query can consume. If the subscriber specifies epsilon as a parameter in a SELECT query, it must be less than or equal to this value. The epsilon parameter controls the amount of noise that is added to the groups — a higher epsilon means less noise.
"maxGroupsContributed": "A String", # Optional. The maximum groups contributed value that is used per query. Represents the maximum number of groups to which each protected entity can contribute. Changing this value does not improve or worsen privacy. The best value for accuracy and utility depends on the query and data.
"privacyUnitColumn": "A String", # Optional. The privacy unit column associated with this policy. Differential privacy policies can only have one privacy unit column per data source object (table, view).
},
"joinRestrictionPolicy": { # Represents privacy policy associated with "join restrictions". Join restriction gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view. # Optional. Join restriction policy is outside of the one of policies, since this policy can be set along with other policies. This policy gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view.
"joinAllowedColumns": [ # Optional. The only columns that joins are allowed on. This field is must be specified for join_conditions JOIN_ANY and JOIN_ALL and it cannot be set for JOIN_BLOCKED.
"A String",
],
"joinCondition": "A String", # Optional. Specifies if a join is required or not on queries for the view. Default is JOIN_CONDITION_UNSPECIFIED.
},
},
"query": "A String", # Required. A query that BigQuery executes when the view is referenced.
"useExplicitColumnNames": True or False, # True if the column names are explicitly specified. For example by using the 'CREATE VIEW v(c1, c2) AS ...' syntax. Can only be set for GoogleSQL views.
"useLegacySql": True or False, # Specifies whether to use BigQuery's legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery's GoogleSQL: https://cloud.google.com/bigquery/sql-reference/ Queries and views that reference this view must use the same flag value. A wrapper is used here because the default value is True.
"userDefinedFunctionResources": [ # Describes user-defined function resources used in the query.
{ # This is used for defining User Defined Function (UDF) resources only when using legacy SQL. Users of GoogleSQL should leverage either DDL (e.g. CREATE [TEMPORARY] FUNCTION ... ) or the Routines API to define UDF resources. For additional information on migrating, see: https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql#differences_in_user-defined_javascript_functions
"inlineCode": "A String", # [Pick one] An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.
"resourceUri": "A String", # [Pick one] A code resource to load from a Google Cloud Storage URI (gs://bucket/path).
},
],
},
}
setIamPolicy(resource, body=None, x__xgafv=None)
Sets the access control policy on the specified resource. Replaces any existing policy. Can return `NOT_FOUND`, `INVALID_ARGUMENT`, and `PERMISSION_DENIED` errors.
Args:
resource: string, REQUIRED: The resource for which the policy is being specified. See [Resource names](https://cloud.google.com/apis/design/resource_names) for the appropriate value for this field. (required)
body: object, The request body.
The object takes the form of:
{ # Request message for `SetIamPolicy` method.
"policy": { # An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources. A `Policy` is a collection of `bindings`. A `binding` binds one or more `members`, or principals, to a single `role`. Principals can be user accounts, service accounts, Google groups, and domains (such as G Suite). A `role` is a named list of permissions; each `role` can be an IAM predefined role or a user-created custom role. For some types of Google Cloud resources, a `binding` can also specify a `condition`, which is a logical expression that allows access to a resource only if the expression evaluates to `true`. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies). **JSON example:** ``` { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } ``` **YAML example:** ``` bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') etag: BwWWja0YfJA= version: 3 ``` For a description of IAM and its features, see the [IAM documentation](https://cloud.google.com/iam/docs/). # REQUIRED: The complete policy to be applied to the `resource`. The size of the policy is limited to a few 10s of KB. An empty policy is a valid policy but certain Google Cloud services (such as Projects) might reject them.
"auditConfigs": [ # Specifies cloud audit logging configuration for this policy.
{ # Specifies the audit configuration for a service. The configuration determines which permission types are logged, and what identities, if any, are exempted from logging. An AuditConfig must have one or more AuditLogConfigs. If there are AuditConfigs for both `allServices` and a specific service, the union of the two AuditConfigs is used for that service: the log_types specified in each AuditConfig are enabled, and the exempted_members in each AuditLogConfig are exempted. Example Policy with multiple AuditConfigs: { "audit_configs": [ { "service": "allServices", "audit_log_configs": [ { "log_type": "DATA_READ", "exempted_members": [ "user:jose@example.com" ] }, { "log_type": "DATA_WRITE" }, { "log_type": "ADMIN_READ" } ] }, { "service": "sampleservice.googleapis.com", "audit_log_configs": [ { "log_type": "DATA_READ" }, { "log_type": "DATA_WRITE", "exempted_members": [ "user:aliya@example.com" ] } ] } ] } For sampleservice, this policy enables DATA_READ, DATA_WRITE and ADMIN_READ logging. It also exempts `jose@example.com` from DATA_READ logging, and `aliya@example.com` from DATA_WRITE logging.
"auditLogConfigs": [ # The configuration for logging of each type of permission.
{ # Provides the configuration for logging a type of permissions. Example: { "audit_log_configs": [ { "log_type": "DATA_READ", "exempted_members": [ "user:jose@example.com" ] }, { "log_type": "DATA_WRITE" } ] } This enables 'DATA_READ' and 'DATA_WRITE' logging, while exempting jose@example.com from DATA_READ logging.
"exemptedMembers": [ # Specifies the identities that do not cause logging for this type of permission. Follows the same format of Binding.members.
"A String",
],
"logType": "A String", # The log type that this config enables.
},
],
"service": "A String", # Specifies a service that will be enabled for audit logging. For example, `storage.googleapis.com`, `cloudsql.googleapis.com`. `allServices` is a special value that covers all services.
},
],
"bindings": [ # Associates a list of `members`, or principals, with a `role`. Optionally, may specify a `condition` that determines how and when the `bindings` are applied. Each of the `bindings` must contain at least one principal. The `bindings` in a `Policy` can refer to up to 1,500 principals; up to 250 of these principals can be Google groups. Each occurrence of a principal counts towards these limits. For example, if the `bindings` grant 50 different roles to `user:alice@example.com`, and not to any other principal, then you can add another 1,450 principals to the `bindings` in the `Policy`.
{ # Associates `members`, or principals, with a `role`.
"condition": { # Represents a textual expression in the Common Expression Language (CEL) syntax. CEL is a C-like expression language. The syntax and semantics of CEL are documented at https://github.com/google/cel-spec. Example (Comparison): title: "Summary size limit" description: "Determines if a summary is less than 100 chars" expression: "document.summary.size() < 100" Example (Equality): title: "Requestor is owner" description: "Determines if requestor is the document owner" expression: "document.owner == request.auth.claims.email" Example (Logic): title: "Public documents" description: "Determine whether the document should be publicly visible" expression: "document.type != 'private' && document.type != 'internal'" Example (Data Manipulation): title: "Notification string" description: "Create a notification string with a timestamp." expression: "'New message received at ' + string(document.create_time)" The exact variables and functions that may be referenced within an expression are determined by the service that evaluates it. See the service documentation for additional information. # The condition that is associated with this binding. If the condition evaluates to `true`, then this binding applies to the current request. If the condition evaluates to `false`, then this binding does not apply to the current request. However, a different role binding might grant the same role to one or more of the principals in this binding. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).
"description": "A String", # Optional. Description of the expression. This is a longer text which describes the expression, e.g. when hovered over it in a UI.
"expression": "A String", # Textual representation of an expression in Common Expression Language syntax.
"location": "A String", # Optional. String indicating the location of the expression for error reporting, e.g. a file name and a position in the file.
"title": "A String", # Optional. Title for the expression, i.e. a short string describing its purpose. This can be used e.g. in UIs which allow to enter the expression.
},
"members": [ # Specifies the principals requesting access for a Google Cloud resource. `members` can have the following values: * `allUsers`: A special identifier that represents anyone who is on the internet; with or without a Google account. * `allAuthenticatedUsers`: A special identifier that represents anyone who is authenticated with a Google account or a service account. Does not include identities that come from external identity providers (IdPs) through identity federation. * `user:{emailid}`: An email address that represents a specific Google account. For example, `alice@example.com` . * `serviceAccount:{emailid}`: An email address that represents a Google service account. For example, `my-other-app@appspot.gserviceaccount.com`. * `serviceAccount:{projectid}.svc.id.goog[{namespace}/{kubernetes-sa}]`: An identifier for a [Kubernetes service account](https://cloud.google.com/kubernetes-engine/docs/how-to/kubernetes-service-accounts). For example, `my-project.svc.id.goog[my-namespace/my-kubernetes-sa]`. * `group:{emailid}`: An email address that represents a Google group. For example, `admins@example.com`. * `domain:{domain}`: The G Suite domain (primary) that represents all the users of that domain. For example, `google.com` or `example.com`. * `principal://iam.googleapis.com/locations/global/workforcePools/{pool_id}/subject/{subject_attribute_value}`: A single identity in a workforce identity pool. * `principalSet://iam.googleapis.com/locations/global/workforcePools/{pool_id}/group/{group_id}`: All workforce identities in a group. * `principalSet://iam.googleapis.com/locations/global/workforcePools/{pool_id}/attribute.{attribute_name}/{attribute_value}`: All workforce identities with a specific attribute value. * `principalSet://iam.googleapis.com/locations/global/workforcePools/{pool_id}/*`: All identities in a workforce identity pool. * `principal://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/subject/{subject_attribute_value}`: A single identity in a workload identity pool. * `principalSet://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/group/{group_id}`: A workload identity pool group. * `principalSet://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/attribute.{attribute_name}/{attribute_value}`: All identities in a workload identity pool with a certain attribute. * `principalSet://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/*`: All identities in a workload identity pool. * `deleted:user:{emailid}?uid={uniqueid}`: An email address (plus unique identifier) representing a user that has been recently deleted. For example, `alice@example.com?uid=123456789012345678901`. If the user is recovered, this value reverts to `user:{emailid}` and the recovered user retains the role in the binding. * `deleted:serviceAccount:{emailid}?uid={uniqueid}`: An email address (plus unique identifier) representing a service account that has been recently deleted. For example, `my-other-app@appspot.gserviceaccount.com?uid=123456789012345678901`. If the service account is undeleted, this value reverts to `serviceAccount:{emailid}` and the undeleted service account retains the role in the binding. * `deleted:group:{emailid}?uid={uniqueid}`: An email address (plus unique identifier) representing a Google group that has been recently deleted. For example, `admins@example.com?uid=123456789012345678901`. If the group is recovered, this value reverts to `group:{emailid}` and the recovered group retains the role in the binding. * `deleted:principal://iam.googleapis.com/locations/global/workforcePools/{pool_id}/subject/{subject_attribute_value}`: Deleted single identity in a workforce identity pool. For example, `deleted:principal://iam.googleapis.com/locations/global/workforcePools/my-pool-id/subject/my-subject-attribute-value`.
"A String",
],
"role": "A String", # Role that is assigned to the list of `members`, or principals. For example, `roles/viewer`, `roles/editor`, or `roles/owner`. For an overview of the IAM roles and permissions, see the [IAM documentation](https://cloud.google.com/iam/docs/roles-overview). For a list of the available pre-defined roles, see [here](https://cloud.google.com/iam/docs/understanding-roles).
},
],
"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help prevent simultaneous updates of a policy from overwriting each other. It is strongly suggested that systems make use of the `etag` in the read-modify-write cycle to perform policy updates in order to avoid race conditions: An `etag` is returned in the response to `getIamPolicy`, and systems are expected to put that etag in the request to `setIamPolicy` to ensure that their change will be applied to the same version of the policy. **Important:** If you use IAM Conditions, you must include the `etag` field whenever you call `setIamPolicy`. If you omit this field, then IAM allows you to overwrite a version `3` policy with a version `1` policy, and all of the conditions in the version `3` policy are lost.
"version": 42, # Specifies the format of the policy. Valid values are `0`, `1`, and `3`. Requests that specify an invalid value are rejected. Any operation that affects conditional role bindings must specify version `3`. This requirement applies to the following operations: * Getting a policy that includes a conditional role binding * Adding a conditional role binding to a policy * Changing a conditional role binding in a policy * Removing any role binding, with or without a condition, from a policy that includes conditions **Important:** If you use IAM Conditions, you must include the `etag` field whenever you call `setIamPolicy`. If you omit this field, then IAM allows you to overwrite a version `3` policy with a version `1` policy, and all of the conditions in the version `3` policy are lost. If a policy does not include any conditions, operations on that policy may specify any valid version or leave the field unset. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).
},
"updateMask": "A String", # OPTIONAL: A FieldMask specifying which fields of the policy to modify. Only the fields in the mask will be modified. If no mask is provided, the following default mask is used: `paths: "bindings, etag"`
}
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources. A `Policy` is a collection of `bindings`. A `binding` binds one or more `members`, or principals, to a single `role`. Principals can be user accounts, service accounts, Google groups, and domains (such as G Suite). A `role` is a named list of permissions; each `role` can be an IAM predefined role or a user-created custom role. For some types of Google Cloud resources, a `binding` can also specify a `condition`, which is a logical expression that allows access to a resource only if the expression evaluates to `true`. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies). **JSON example:** ``` { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } ``` **YAML example:** ``` bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') etag: BwWWja0YfJA= version: 3 ``` For a description of IAM and its features, see the [IAM documentation](https://cloud.google.com/iam/docs/).
"auditConfigs": [ # Specifies cloud audit logging configuration for this policy.
{ # Specifies the audit configuration for a service. The configuration determines which permission types are logged, and what identities, if any, are exempted from logging. An AuditConfig must have one or more AuditLogConfigs. If there are AuditConfigs for both `allServices` and a specific service, the union of the two AuditConfigs is used for that service: the log_types specified in each AuditConfig are enabled, and the exempted_members in each AuditLogConfig are exempted. Example Policy with multiple AuditConfigs: { "audit_configs": [ { "service": "allServices", "audit_log_configs": [ { "log_type": "DATA_READ", "exempted_members": [ "user:jose@example.com" ] }, { "log_type": "DATA_WRITE" }, { "log_type": "ADMIN_READ" } ] }, { "service": "sampleservice.googleapis.com", "audit_log_configs": [ { "log_type": "DATA_READ" }, { "log_type": "DATA_WRITE", "exempted_members": [ "user:aliya@example.com" ] } ] } ] } For sampleservice, this policy enables DATA_READ, DATA_WRITE and ADMIN_READ logging. It also exempts `jose@example.com` from DATA_READ logging, and `aliya@example.com` from DATA_WRITE logging.
"auditLogConfigs": [ # The configuration for logging of each type of permission.
{ # Provides the configuration for logging a type of permissions. Example: { "audit_log_configs": [ { "log_type": "DATA_READ", "exempted_members": [ "user:jose@example.com" ] }, { "log_type": "DATA_WRITE" } ] } This enables 'DATA_READ' and 'DATA_WRITE' logging, while exempting jose@example.com from DATA_READ logging.
"exemptedMembers": [ # Specifies the identities that do not cause logging for this type of permission. Follows the same format of Binding.members.
"A String",
],
"logType": "A String", # The log type that this config enables.
},
],
"service": "A String", # Specifies a service that will be enabled for audit logging. For example, `storage.googleapis.com`, `cloudsql.googleapis.com`. `allServices` is a special value that covers all services.
},
],
"bindings": [ # Associates a list of `members`, or principals, with a `role`. Optionally, may specify a `condition` that determines how and when the `bindings` are applied. Each of the `bindings` must contain at least one principal. The `bindings` in a `Policy` can refer to up to 1,500 principals; up to 250 of these principals can be Google groups. Each occurrence of a principal counts towards these limits. For example, if the `bindings` grant 50 different roles to `user:alice@example.com`, and not to any other principal, then you can add another 1,450 principals to the `bindings` in the `Policy`.
{ # Associates `members`, or principals, with a `role`.
"condition": { # Represents a textual expression in the Common Expression Language (CEL) syntax. CEL is a C-like expression language. The syntax and semantics of CEL are documented at https://github.com/google/cel-spec. Example (Comparison): title: "Summary size limit" description: "Determines if a summary is less than 100 chars" expression: "document.summary.size() < 100" Example (Equality): title: "Requestor is owner" description: "Determines if requestor is the document owner" expression: "document.owner == request.auth.claims.email" Example (Logic): title: "Public documents" description: "Determine whether the document should be publicly visible" expression: "document.type != 'private' && document.type != 'internal'" Example (Data Manipulation): title: "Notification string" description: "Create a notification string with a timestamp." expression: "'New message received at ' + string(document.create_time)" The exact variables and functions that may be referenced within an expression are determined by the service that evaluates it. See the service documentation for additional information. # The condition that is associated with this binding. If the condition evaluates to `true`, then this binding applies to the current request. If the condition evaluates to `false`, then this binding does not apply to the current request. However, a different role binding might grant the same role to one or more of the principals in this binding. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).
"description": "A String", # Optional. Description of the expression. This is a longer text which describes the expression, e.g. when hovered over it in a UI.
"expression": "A String", # Textual representation of an expression in Common Expression Language syntax.
"location": "A String", # Optional. String indicating the location of the expression for error reporting, e.g. a file name and a position in the file.
"title": "A String", # Optional. Title for the expression, i.e. a short string describing its purpose. This can be used e.g. in UIs which allow to enter the expression.
},
"members": [ # Specifies the principals requesting access for a Google Cloud resource. `members` can have the following values: * `allUsers`: A special identifier that represents anyone who is on the internet; with or without a Google account. * `allAuthenticatedUsers`: A special identifier that represents anyone who is authenticated with a Google account or a service account. Does not include identities that come from external identity providers (IdPs) through identity federation. * `user:{emailid}`: An email address that represents a specific Google account. For example, `alice@example.com` . * `serviceAccount:{emailid}`: An email address that represents a Google service account. For example, `my-other-app@appspot.gserviceaccount.com`. * `serviceAccount:{projectid}.svc.id.goog[{namespace}/{kubernetes-sa}]`: An identifier for a [Kubernetes service account](https://cloud.google.com/kubernetes-engine/docs/how-to/kubernetes-service-accounts). For example, `my-project.svc.id.goog[my-namespace/my-kubernetes-sa]`. * `group:{emailid}`: An email address that represents a Google group. For example, `admins@example.com`. * `domain:{domain}`: The G Suite domain (primary) that represents all the users of that domain. For example, `google.com` or `example.com`. * `principal://iam.googleapis.com/locations/global/workforcePools/{pool_id}/subject/{subject_attribute_value}`: A single identity in a workforce identity pool. * `principalSet://iam.googleapis.com/locations/global/workforcePools/{pool_id}/group/{group_id}`: All workforce identities in a group. * `principalSet://iam.googleapis.com/locations/global/workforcePools/{pool_id}/attribute.{attribute_name}/{attribute_value}`: All workforce identities with a specific attribute value. * `principalSet://iam.googleapis.com/locations/global/workforcePools/{pool_id}/*`: All identities in a workforce identity pool. * `principal://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/subject/{subject_attribute_value}`: A single identity in a workload identity pool. * `principalSet://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/group/{group_id}`: A workload identity pool group. * `principalSet://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/attribute.{attribute_name}/{attribute_value}`: All identities in a workload identity pool with a certain attribute. * `principalSet://iam.googleapis.com/projects/{project_number}/locations/global/workloadIdentityPools/{pool_id}/*`: All identities in a workload identity pool. * `deleted:user:{emailid}?uid={uniqueid}`: An email address (plus unique identifier) representing a user that has been recently deleted. For example, `alice@example.com?uid=123456789012345678901`. If the user is recovered, this value reverts to `user:{emailid}` and the recovered user retains the role in the binding. * `deleted:serviceAccount:{emailid}?uid={uniqueid}`: An email address (plus unique identifier) representing a service account that has been recently deleted. For example, `my-other-app@appspot.gserviceaccount.com?uid=123456789012345678901`. If the service account is undeleted, this value reverts to `serviceAccount:{emailid}` and the undeleted service account retains the role in the binding. * `deleted:group:{emailid}?uid={uniqueid}`: An email address (plus unique identifier) representing a Google group that has been recently deleted. For example, `admins@example.com?uid=123456789012345678901`. If the group is recovered, this value reverts to `group:{emailid}` and the recovered group retains the role in the binding. * `deleted:principal://iam.googleapis.com/locations/global/workforcePools/{pool_id}/subject/{subject_attribute_value}`: Deleted single identity in a workforce identity pool. For example, `deleted:principal://iam.googleapis.com/locations/global/workforcePools/my-pool-id/subject/my-subject-attribute-value`.
"A String",
],
"role": "A String", # Role that is assigned to the list of `members`, or principals. For example, `roles/viewer`, `roles/editor`, or `roles/owner`. For an overview of the IAM roles and permissions, see the [IAM documentation](https://cloud.google.com/iam/docs/roles-overview). For a list of the available pre-defined roles, see [here](https://cloud.google.com/iam/docs/understanding-roles).
},
],
"etag": "A String", # `etag` is used for optimistic concurrency control as a way to help prevent simultaneous updates of a policy from overwriting each other. It is strongly suggested that systems make use of the `etag` in the read-modify-write cycle to perform policy updates in order to avoid race conditions: An `etag` is returned in the response to `getIamPolicy`, and systems are expected to put that etag in the request to `setIamPolicy` to ensure that their change will be applied to the same version of the policy. **Important:** If you use IAM Conditions, you must include the `etag` field whenever you call `setIamPolicy`. If you omit this field, then IAM allows you to overwrite a version `3` policy with a version `1` policy, and all of the conditions in the version `3` policy are lost.
"version": 42, # Specifies the format of the policy. Valid values are `0`, `1`, and `3`. Requests that specify an invalid value are rejected. Any operation that affects conditional role bindings must specify version `3`. This requirement applies to the following operations: * Getting a policy that includes a conditional role binding * Adding a conditional role binding to a policy * Changing a conditional role binding in a policy * Removing any role binding, with or without a condition, from a policy that includes conditions **Important:** If you use IAM Conditions, you must include the `etag` field whenever you call `setIamPolicy`. If you omit this field, then IAM allows you to overwrite a version `3` policy with a version `1` policy, and all of the conditions in the version `3` policy are lost. If a policy does not include any conditions, operations on that policy may specify any valid version or leave the field unset. To learn which resources support conditions in their IAM policies, see the [IAM documentation](https://cloud.google.com/iam/help/conditions/resource-policies).
}
testIamPermissions(resource, body=None, x__xgafv=None)
Returns permissions that a caller has on the specified resource. If the resource does not exist, this will return an empty set of permissions, not a `NOT_FOUND` error. Note: This operation is designed to be used for building permission-aware UIs and command-line tools, not for authorization checking. This operation may "fail open" without warning.
Args:
resource: string, REQUIRED: The resource for which the policy detail is being requested. See [Resource names](https://cloud.google.com/apis/design/resource_names) for the appropriate value for this field. (required)
body: object, The request body.
The object takes the form of:
{ # Request message for `TestIamPermissions` method.
"permissions": [ # The set of permissions to check for the `resource`. Permissions with wildcards (such as `*` or `storage.*`) are not allowed. For more information see [IAM Overview](https://cloud.google.com/iam/docs/overview#permissions).
"A String",
],
}
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # Response message for `TestIamPermissions` method.
"permissions": [ # A subset of `TestPermissionsRequest.permissions` that the caller is allowed.
"A String",
],
}
update(projectId, datasetId, tableId, autodetect_schema=None, body=None, x__xgafv=None)
Updates information in an existing table. The update method replaces the entire Table resource, whereas the patch method only replaces fields that are provided in the submitted Table resource.
Args:
projectId: string, Required. Project ID of the table to update (required)
datasetId: string, Required. Dataset ID of the table to update (required)
tableId: string, Required. Table ID of the table to update (required)
body: object, The request body.
The object takes the form of:
{
"biglakeConfiguration": { # Configuration for BigLake managed tables. # Optional. Specifies the configuration of a BigLake managed table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read and write to external storage, such as Cloud Storage. The connection_id can have the form `{project}.{location}.{connection_id}` or `projects/{project}/locations/{location}/connections/{connection_id}".
"fileFormat": "A String", # Optional. The file format the table data is stored in.
"storageUri": "A String", # Optional. The fully qualified location prefix of the external folder where table data is stored. The '*' wildcard character is not allowed. The URI should be in the format `gs://bucket/path_to_table/`
"tableFormat": "A String", # Optional. The table format the metadata only snapshots are stored in.
},
"cloneDefinition": { # Information about base table and clone time of a table clone. # Output only. Contains information about the clone. This value is set via the clone operation.
"baseTableReference": { # Required. Reference describing the ID of the table that was cloned.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"cloneTime": "A String", # Required. The time at which the base table was cloned. This value is reported in the JSON response using RFC3339 format.
},
"clustering": { # Configures table clustering. # Clustering specification for the table. Must be specified with time-based partitioning, data in the table will be first partitioned and subsequently clustered.
"fields": [ # One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. The ordering of the clustering fields should be prioritized from most to least important for filtering purposes. Additional information on limitations can be found here: https://cloud.google.com/bigquery/docs/creating-clustered-tables#limitations
"A String",
],
},
"creationTime": "A String", # Output only. The time when this table was created, in milliseconds since the epoch.
"defaultCollation": "A String", # Optional. Defines the default collation specification of new STRING fields in the table. During table creation or update, if a STRING field is added to this table without explicit collation specified, then the table inherits the table default collation. A change to this field affects only fields added afterwards, and does not alter the existing fields. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"defaultRoundingMode": "A String", # Optional. Defines the default rounding mode specification of new decimal fields (NUMERIC OR BIGNUMERIC) in the table. During table creation or update, if a decimal field is added to this table without an explicit rounding mode specified, then the field inherits the table default rounding mode. Changing this field doesn't affect existing fields.
"description": "A String", # Optional. A user-friendly description of this table.
"encryptionConfiguration": { # Configuration for Cloud KMS encryption settings. # Custom encryption configuration (e.g., Cloud KMS keys).
"kmsKeyName": "A String", # Optional. Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.
},
"etag": "A String", # Output only. A hash of this resource.
"expirationTime": "A String", # Optional. The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed. The defaultTableExpirationMs property of the encapsulating dataset can be used to set a default expirationTime on newly created tables.
"externalCatalogTableOptions": { # Metadata about open source compatible table. The fields contained in these options correspond to hive metastore's table level properties. # Optional. Options defining open source compatible table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection is needed to read the open source table from BigQuery Engine. The connection_id can have the form `..` or `projects//locations//connections/`.
"parameters": { # Optional. A map of key value pairs defining the parameters and properties of the open source table. Corresponds with hive meta store table parameters. Maximum size of 4Mib.
"a_key": "A String",
},
"storageDescriptor": { # Contains information about how a table's data is stored and accessed by open source query engines. # Optional. A storage descriptor containing information about the physical storage of this table.
"inputFormat": "A String", # Optional. Specifies the fully qualified class name of the InputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"). The maximum length is 128 characters.
"locationUri": "A String", # Optional. The physical location of the table (e.g. `gs://spark-dataproc-data/pangea-data/case_sensitive/` or `gs://spark-dataproc-data/pangea-data/*`). The maximum length is 2056 bytes.
"outputFormat": "A String", # Optional. Specifies the fully qualified class name of the OutputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"). The maximum length is 128 characters.
"serdeInfo": { # Serializer and deserializer information. # Optional. Serializer and deserializer information.
"name": "A String", # Optional. Name of the SerDe. The maximum length is 256 characters.
"parameters": { # Optional. Key-value pairs that define the initialization parameters for the serialization library. Maximum size 10 Kib.
"a_key": "A String",
},
"serializationLibrary": "A String", # Required. Specifies a fully-qualified class name of the serialization library that is responsible for the translation of data between table representation and the underlying low-level input and output format structures. The maximum length is 256 characters.
},
},
},
"externalDataConfiguration": { # Optional. Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table.
"autodetect": True or False, # Try to detect schema and format options automatically. Any option specified explicitly will be honored.
"avroOptions": { # Options for external data sources. # Optional. Additional properties to set if sourceFormat is set to AVRO.
"useAvroLogicalTypes": True or False, # Optional. If sourceFormat is set to "AVRO", indicates whether to interpret logical types as the corresponding BigQuery data type (for example, TIMESTAMP), instead of using the raw type (for example, INTEGER).
},
"bigtableOptions": { # Options specific to Google Cloud Bigtable data sources. # Optional. Additional options if sourceFormat is set to BIGTABLE.
"columnFamilies": [ # Optional. List of column families to expose in the table schema along with their types. This list restricts the column families that can be referenced in queries and specifies their value types. You can use this list to do type conversions - see the 'type' field for more details. If you leave this list empty, all column families are present in the table schema and their values are read as BYTES. During a query only the column families referenced in that query are read from Bigtable.
{ # Information related to a Bigtable column family.
"columns": [ # Optional. Lists of columns that should be exposed as individual fields as opposed to a list of (column name, value) pairs. All columns whose qualifier matches a qualifier in this list can be accessed as `.`. Other columns can be accessed as a list through the `.Column` field.
{ # Information related to a Bigtable column.
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. 'encoding' can also be set at the column family level. However, the setting at this level takes precedence if 'encoding' is set at both levels.
"fieldName": "A String", # Optional. If the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as the column field name and is used as field name in queries.
"onlyReadLatest": True or False, # Optional. If this is set, only the latest version of value in this column are exposed. 'onlyReadLatest' can also be set at the column family level. However, the setting at this level takes precedence if 'onlyReadLatest' is set at both levels.
"qualifierEncoded": "A String", # [Required] Qualifier of the column. Columns in the parent column family that has this exact qualifier are exposed as `.` field. If the qualifier is valid UTF-8 string, it can be specified in the qualifier_string field. Otherwise, a base-64 encoded value must be set to qualifier_encoded. The column field name is the same as the column qualifier. However, if the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as field_name.
"qualifierString": "A String", # Qualifier string.
"type": "A String", # Optional. The type to convert the value in cells of this column. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. 'type' can also be set at the column family level. However, the setting at this level takes precedence if 'type' is set at both levels.
},
],
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. This can be overridden for a specific column by listing that column in 'columns' and specifying an encoding for it.
"familyId": "A String", # Identifier of the column family.
"onlyReadLatest": True or False, # Optional. If this is set only the latest version of value are exposed for all columns in this column family. This can be overridden for a specific column by listing that column in 'columns' and specifying a different setting for that column.
"type": "A String", # Optional. The type to convert the value in cells of this column family. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. This can be overridden for a specific column by listing that column in 'columns' and specifying a type for it.
},
],
"ignoreUnspecifiedColumnFamilies": True or False, # Optional. If field is true, then the column families that are not specified in columnFamilies list are not exposed in the table schema. Otherwise, they are read with BYTES type values. The default value is false.
"outputColumnFamiliesAsJson": True or False, # Optional. If field is true, then each column family will be read as a single JSON column. Otherwise they are read as a repeated cell structure containing timestamp/value tuples. The default value is false.
"readRowkeyAsString": True or False, # Optional. If field is true, then the rowkey column families will be read and converted to string. Otherwise they are read with BYTES type values and users need to manually cast them with CAST if necessary. The default value is false.
},
"compression": "A String", # Optional. The compression type of the data source. Possible values include GZIP and NONE. The default value is NONE. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats. An empty string is an invalid value.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id can have the form `{project_id}.{location_id};{connection_id}` or `projects/{project_id}/locations/{location_id}/connections/{connection_id}`.
"csvOptions": { # Information related to a CSV data source. # Optional. Additional properties to set if sourceFormat is set to CSV.
"allowJaggedRows": True or False, # Optional. Indicates if BigQuery should accept rows that are missing trailing optional columns. If true, BigQuery treats missing trailing columns as null values. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.
"allowQuotedNewlines": True or False, # Optional. Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, ISO-8859-1, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.
"fieldDelimiter": "A String", # Optional. The separator character for fields in a CSV file. The separator is interpreted as a single byte. For files encoded in ISO-8859-1, any single character can be used as a separator. For files encoded in UTF-8, characters represented in decimal range 1-127 (U+0001-U+007F) can be used without any modification. UTF-8 characters encoded with multiple bytes (i.e. U+0080 and above) will have only the first byte used for separating fields. The remaining bytes will be treated as a part of the field. BigQuery also supports the escape sequence "\t" (U+0009) to specify a tab separator. The default value is comma (",", U+002C).
"nullMarker": "A String", # Optional. Specifies a string that represents a null value in a CSV file. For example, if you specify "\N", BigQuery interprets "\N" as a null value when querying a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.
"preserveAsciiControlCharacters": True or False, # Optional. Indicates if the embedded ASCII control characters (the first 32 characters in the ASCII-table, from '\x00' to '\x1F') are preserved.
"quote": """, # Optional. The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote ("). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true. To include the specific quote character within a quoted value, precede it with an additional matching quote character. For example, if you want to escape the default character ' " ', use ' "" '.
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a CSV file that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"decimalTargetTypes": [ # Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown. Example: Suppose the value of this field is ["NUMERIC", "BIGNUMERIC"]. If (precision,scale) is: * (38,9) -> NUMERIC; * (39,9) -> BIGNUMERIC (NUMERIC cannot hold 30 integer digits); * (38,10) -> BIGNUMERIC (NUMERIC cannot hold 10 fractional digits); * (76,38) -> BIGNUMERIC; * (77,38) -> BIGNUMERIC (error if value exeeds supported range). This field cannot contain duplicate types. The order of the types in this field is ignored. For example, ["BIGNUMERIC", "NUMERIC"] is the same as ["NUMERIC", "BIGNUMERIC"] and NUMERIC always takes precedence over BIGNUMERIC. Defaults to ["NUMERIC", "STRING"] for ORC and ["NUMERIC"] for the other file formats.
"A String",
],
"fileSetSpecType": "A String", # Optional. Specifies how source URIs are interpreted for constructing the file set to load. By default source URIs are expanded against the underlying storage. Other options include specifying manifest files. Only applicable to object storage systems.
"googleSheetsOptions": { # Options specific to Google Sheets data sources. # Optional. Additional options if sourceFormat is set to GOOGLE_SHEETS.
"range": "A String", # Optional. Range of a sheet to query from. Only used when non-empty. Typical format: sheet_name!top_left_cell_id:bottom_right_cell_id For example: sheet1!A1:B20
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a sheet that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"hivePartitioningOptions": { # Options for configuring hive partitioning detect. # Optional. When set, configures hive partitioning support. Not all storage formats support hive partitioning -- requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.
"fields": [ # Output only. For permanent external tables, this field is populated with the hive partition keys in the order they were inferred. The types of the partition keys can be deduced by checking the table schema (which will include the partition keys). Not every API will populate this field in the output. For example, Tables.Get will populate it, but Tables.List will not contain this field.
"A String",
],
"mode": "A String", # Optional. When set, what mode of hive partitioning to use when reading data. The following modes are supported: * AUTO: automatically infer partition key name(s) and type(s). * STRINGS: automatically infer partition key name(s). All types are strings. * CUSTOM: partition key schema is encoded in the source URI prefix. Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. Note that this field should only be true when creating a permanent external table or querying a temporary external table. Hive-partitioned loads with require_partition_filter explicitly set to true will fail.
"sourceUriPrefix": "A String", # Optional. When hive partition detection is requested, a common prefix for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout: gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. CUSTOM detection requires encoding the partitioning schema immediately after the common prefix. For CUSTOM, any of * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:STRING}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:STRING} would all be valid source URI prefixes.
},
"ignoreUnknownValues": True or False, # Optional. Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don't match any column names Google Cloud Bigtable: This setting is ignored. Google Cloud Datastore backups: This setting is ignored. Avro: This setting is ignored. ORC: This setting is ignored. Parquet: This setting is ignored.
"jsonExtension": "A String", # Optional. Load option to be used together with source_format newline-delimited JSON to indicate that a variant of JSON is being loaded. To load newline-delimited GeoJSON, specify GEOJSON (and source_format must be set to NEWLINE_DELIMITED_JSON).
"jsonOptions": { # Json Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to JSON.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8.
},
"maxBadRecords": 42, # Optional. The maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats.
"metadataCacheMode": "A String", # Optional. Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source.
"objectMetadata": "A String", # Optional. ObjectMetadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the source_uris. If ObjectMetadata is set, source_format should be omitted. Currently SIMPLE is the only supported Object Metadata type.
"parquetOptions": { # Parquet Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to PARQUET.
"enableListInference": True or False, # Optional. Indicates whether to use schema inference specifically for Parquet LIST logical type.
"enumAsString": True or False, # Optional. Indicates whether to infer Parquet ENUM logical type as STRING instead of BYTES by default.
"mapTargetType": "A String", # Optional. Indicates how to represent a Parquet map if present.
},
"referenceFileSchemaUri": "A String", # Optional. When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.
"schema": { # Schema of a table # Optional. The schema for the data. Schema is required for CSV and JSON formats if autodetect is not on. Schema is disallowed for Google Cloud Bigtable, Cloud Datastore backups, Avro, ORC and Parquet formats.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"sourceFormat": "A String", # [Required] The data format. For CSV files, specify "CSV". For Google sheets, specify "GOOGLE_SHEETS". For newline-delimited JSON, specify "NEWLINE_DELIMITED_JSON". For Avro files, specify "AVRO". For Google Cloud Datastore backups, specify "DATASTORE_BACKUP". For Apache Iceberg tables, specify "ICEBERG". For ORC files, specify "ORC". For Parquet files, specify "PARQUET". [Beta] For Google Cloud Bigtable, specify "BIGTABLE".
"sourceUris": [ # [Required] The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one '*' wildcard character and it must come after the 'bucket' name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the '*' wildcard character is not allowed.
"A String",
],
},
"friendlyName": "A String", # Optional. A descriptive name for this table.
"id": "A String", # Output only. An opaque ID uniquely identifying the table.
"kind": "bigquery#table", # The type of resource ID.
"labels": { # The labels associated with this table. You can use these to organize and group your tables. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.
"a_key": "A String",
},
"lastModifiedTime": "A String", # Output only. The time when this table was last modified, in milliseconds since the epoch.
"location": "A String", # Output only. The geographic location where the table resides. This value is inherited from the dataset.
"materializedView": { # Definition and configuration of a materialized view. # Optional. The materialized view definition.
"allowNonIncrementalDefinition": True or False, # Optional. This option declares the intention to construct a materialized view that isn't refreshed incrementally.
"enableRefresh": True or False, # Optional. Enable automatic refresh of the materialized view when the base table is updated. The default value is "true".
"lastRefreshTime": "A String", # Output only. The time when this materialized view was last refreshed, in milliseconds since the epoch.
"maxStaleness": "A String", # [Optional] Max staleness of data that could be returned when materizlized view is queried (formatted as Google SQL Interval type).
"query": "A String", # Required. A query whose results are persisted.
"refreshIntervalMs": "A String", # Optional. The maximum frequency at which this materialized view will be refreshed. The default value is "1800000" (30 minutes).
},
"materializedViewStatus": { # Status of a materialized view. The last refresh timestamp status is omitted here, but is present in the MaterializedViewDefinition message. # Output only. The materialized view status.
"lastRefreshStatus": { # Error details. # Output only. Error result of the last automatic refresh. If present, indicates that the last automatic refresh was unsuccessful.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"refreshWatermark": "A String", # Output only. Refresh watermark of materialized view. The base tables' data were collected into the materialized view cache until this time.
},
"maxStaleness": "A String", # Optional. The maximum staleness of data that could be returned when the table (or stale MV) is queried. Staleness encoded as a string encoding of sql IntervalValue type.
"model": { # Deprecated.
"modelOptions": { # Deprecated.
"labels": [
"A String",
],
"lossType": "A String",
"modelType": "A String",
},
"trainingRuns": [ # Deprecated.
{
"iterationResults": [ # Deprecated.
{
"durationMs": "A String", # Deprecated.
"evalLoss": 3.14, # Deprecated.
"index": 42, # Deprecated.
"learnRate": 3.14, # Deprecated.
"trainingLoss": 3.14, # Deprecated.
},
],
"startTime": "A String", # Deprecated.
"state": "A String", # Deprecated.
"trainingOptions": { # Deprecated.
"earlyStop": True or False,
"l1Reg": 3.14,
"l2Reg": 3.14,
"learnRate": 3.14,
"learnRateStrategy": "A String",
"lineSearchInitLearnRate": 3.14,
"maxIteration": "A String",
"minRelProgress": 3.14,
"warmStart": True or False,
},
},
],
},
"numActiveLogicalBytes": "A String", # Output only. Number of logical bytes that are less than 90 days old.
"numActivePhysicalBytes": "A String", # Output only. Number of physical bytes less than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numBytes": "A String", # Output only. The size of this table in logical bytes, excluding any data in the streaming buffer.
"numCurrentPhysicalBytes": "A String", # Output only. Number of physical bytes used by current live data storage. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numLongTermBytes": "A String", # Output only. The number of logical bytes in the table that are considered "long-term storage".
"numLongTermLogicalBytes": "A String", # Output only. Number of logical bytes that are more than 90 days old.
"numLongTermPhysicalBytes": "A String", # Output only. Number of physical bytes more than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPartitions": "A String", # Output only. The number of partitions present in the table or materialized view. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This includes storage used for time travel.
"numRows": "A String", # Output only. The number of rows of data in this table, excluding any data in the streaming buffer.
"numTimeTravelPhysicalBytes": "A String", # Output only. Number of physical bytes used by time travel storage (deleted or changed data). This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numTotalLogicalBytes": "A String", # Output only. Total number of logical bytes in the table or materialized view.
"numTotalPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This also includes storage used for time travel. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"partitionDefinition": { # The partitioning information, which includes managed table, external table and metastore partitioned table partition information. # Optional. The partition information for all table formats, including managed partitioned tables, hive partitioned tables, iceberg partitioned, and metastore partitioned tables. This field is only populated for metastore partitioned tables. For other table formats, this is an output only field.
"partitionedColumn": [ # Optional. Details about each partitioning column. This field is output only for all partitioning types other than metastore partitioned tables. BigQuery native tables only support 1 partitioning column. Other table types may support 0, 1 or more partitioning columns. For metastore partitioned tables, the order must match the definition order in the Hive Metastore, where it must match the physical layout of the table. For example, CREATE TABLE a_table(id BIGINT, name STRING) PARTITIONED BY (city STRING, state STRING). In this case the values must be ['city', 'state'] in that order.
{ # The partitioning column information.
"field": "A String", # Required. The name of the partition column.
},
],
},
"rangePartitioning": { # If specified, configures range partitioning for this table.
"field": "A String", # Required. The name of the column to partition the table on. It must be a top-level, INT64 column whose mode is NULLABLE or REQUIRED.
"range": { # [Experimental] Defines the ranges for range partitioning.
"end": "A String", # [Experimental] The end of range partitioning, exclusive.
"interval": "A String", # [Experimental] The width of each interval.
"start": "A String", # [Experimental] The start of range partitioning, inclusive.
},
},
"replicas": [ # Optional. Output only. Table references of all replicas currently active on the table.
{
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
],
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.
"resourceTags": { # [Optional] The tags associated with this table. Tag keys are globally unique. See additional information on [tags](https://cloud.google.com/iam/docs/tags-access-control#definitions). An object containing a list of "key": value pairs. The key is the namespaced friendly name of the tag key, e.g. "12345/environment" where 12345 is parent id. The value is the friendly short name of the tag value, e.g. "production".
"a_key": "A String",
},
"restrictions": { # Optional. Output only. Restriction config for table. If set, restrict certain accesses on the table based on the config. See [Data egress](https://cloud.google.com/bigquery/docs/analytics-hub-introduction#data_egress) for more details.
"type": "A String", # Output only. Specifies the type of dataset/table restriction.
},
"schema": { # Schema of a table # Optional. Describes the schema of this table.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"selfLink": "A String", # Output only. A URL that can be used to access this resource again.
"snapshotDefinition": { # Information about base table and snapshot time of the snapshot. # Output only. Contains information about the snapshot. This value is set via snapshot creation.
"baseTableReference": { # Required. Reference describing the ID of the table that was snapshot.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"snapshotTime": "A String", # Required. The time at which the base table was snapshot. This value is reported in the JSON response using RFC3339 format.
},
"streamingBuffer": { # Output only. Contains information regarding this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
"estimatedBytes": "A String", # Output only. A lower-bound estimate of the number of bytes currently in the streaming buffer.
"estimatedRows": "A String", # Output only. A lower-bound estimate of the number of rows currently in the streaming buffer.
"oldestEntryTime": "A String", # Output only. Contains the timestamp of the oldest entry in the streaming buffer, in milliseconds since the epoch, if the streaming buffer is available.
},
"tableConstraints": { # The TableConstraints defines the primary key and foreign key. # Optional. Tables Primary Key and Foreign Key information
"foreignKeys": [ # Optional. Present only if the table has a foreign key. The foreign key is not enforced.
{ # Represents a foreign key constraint on a table's columns.
"columnReferences": [ # Required. The columns that compose the foreign key.
{ # The pair of the foreign key column and primary key column.
"referencedColumn": "A String", # Required. The column in the primary key that are referenced by the referencing_column.
"referencingColumn": "A String", # Required. The column that composes the foreign key.
},
],
"name": "A String", # Optional. Set only if the foreign key constraint is named.
"referencedTable": {
"datasetId": "A String",
"projectId": "A String",
"tableId": "A String",
},
},
],
"primaryKey": { # Represents the primary key constraint on a table's columns.
"columns": [ # Required. The columns that are composed of the primary key constraint.
"A String",
],
},
},
"tableReference": { # Required. Reference describing the ID of this table.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"tableReplicationInfo": { # Replication info of a table created using `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv` # Optional. Table replication info for table created `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv`
"replicatedSourceLastRefreshTime": "A String", # Optional. Output only. If source is a materialized view, this field signifies the last refresh time of the source.
"replicationError": { # Error details. # Optional. Output only. Replication error that will permanently stopped table replication.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"replicationIntervalMs": "A String", # Optional. Specifies the interval at which the source table is polled for updates. It's Optional. If not specified, default replication interval would be applied.
"replicationStatus": "A String", # Optional. Output only. Replication status of configured replication.
"sourceTable": { # Required. Source table reference that is replicated.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
},
"timePartitioning": { # If specified, configures time-based partitioning for this table.
"expirationMs": "A String", # Optional. Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.
"field": "A String", # Optional. If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.
"requirePartitionFilter": false, # If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. This field is deprecated; please set the field with the same name on the table itself instead. This field needs a wrapper because we want to output the default value, false, if the user explicitly set it.
"type": "A String", # Required. The supported types are DAY, HOUR, MONTH, and YEAR, which will generate one partition per day, hour, month, and year, respectively.
},
"type": "A String", # Output only. Describes the table type. The following values are supported: * `TABLE`: A normal BigQuery table. * `VIEW`: A virtual table defined by a SQL query. * `EXTERNAL`: A table that references data stored in an external storage system, such as Google Cloud Storage. * `MATERIALIZED_VIEW`: A precomputed view defined by a SQL query. * `SNAPSHOT`: An immutable BigQuery table that preserves the contents of a base table at a particular time. See additional information on [table snapshots](https://cloud.google.com/bigquery/docs/table-snapshots-intro). The default value is `TABLE`.
"view": { # Describes the definition of a logical view. # Optional. The view definition.
"foreignDefinitions": [ # Optional. Foreign view representations.
{ # A view can be represented in multiple ways. Each representation has its own dialect. This message stores the metadata required for these representations.
"dialect": "A String", # Optional. Represents the dialect of the query.
"query": "A String", # Required. The query that defines the view.
},
],
"privacyPolicy": { # Represents privacy policy that contains the privacy requirements specified by the data owner. Currently, this is only supported on views. # Optional. Specifices the privacy policy for the view.
"aggregationThresholdPolicy": { # Represents privacy policy associated with "aggregation threshold" method. # Optional. Policy used for aggregation thresholds.
"privacyUnitColumns": [ # Optional. The privacy unit column(s) associated with this policy. For now, only one column per data source object (table, view) is allowed as a privacy unit column. Representing as a repeated field in metadata for extensibility to multiple columns in future. Duplicates and Repeated struct fields are not allowed. For nested fields, use dot notation ("outer.inner")
"A String",
],
"threshold": "A String", # Optional. The threshold for the "aggregation threshold" policy.
},
"differentialPrivacyPolicy": { # Represents privacy policy associated with "differential privacy" method. # Optional. Policy used for differential privacy.
"deltaBudget": 3.14, # Optional. The total delta budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of delta that is pre-defined by the contributor through the privacy policy delta_per_query field. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"deltaBudgetRemaining": 3.14, # Output only. The delta budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"deltaPerQuery": 3.14, # Optional. The delta value that is used per query. Delta represents the probability that any row will fail to be epsilon differentially private. Indicates the risk associated with exposing aggregate rows in the result of a query.
"epsilonBudget": 3.14, # Optional. The total epsilon budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of epsilon they request in their query. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"epsilonBudgetRemaining": 3.14, # Output only. The epsilon budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"maxEpsilonPerQuery": 3.14, # Optional. The maximum epsilon value that a query can consume. If the subscriber specifies epsilon as a parameter in a SELECT query, it must be less than or equal to this value. The epsilon parameter controls the amount of noise that is added to the groups — a higher epsilon means less noise.
"maxGroupsContributed": "A String", # Optional. The maximum groups contributed value that is used per query. Represents the maximum number of groups to which each protected entity can contribute. Changing this value does not improve or worsen privacy. The best value for accuracy and utility depends on the query and data.
"privacyUnitColumn": "A String", # Optional. The privacy unit column associated with this policy. Differential privacy policies can only have one privacy unit column per data source object (table, view).
},
"joinRestrictionPolicy": { # Represents privacy policy associated with "join restrictions". Join restriction gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view. # Optional. Join restriction policy is outside of the one of policies, since this policy can be set along with other policies. This policy gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view.
"joinAllowedColumns": [ # Optional. The only columns that joins are allowed on. This field is must be specified for join_conditions JOIN_ANY and JOIN_ALL and it cannot be set for JOIN_BLOCKED.
"A String",
],
"joinCondition": "A String", # Optional. Specifies if a join is required or not on queries for the view. Default is JOIN_CONDITION_UNSPECIFIED.
},
},
"query": "A String", # Required. A query that BigQuery executes when the view is referenced.
"useExplicitColumnNames": True or False, # True if the column names are explicitly specified. For example by using the 'CREATE VIEW v(c1, c2) AS ...' syntax. Can only be set for GoogleSQL views.
"useLegacySql": True or False, # Specifies whether to use BigQuery's legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery's GoogleSQL: https://cloud.google.com/bigquery/sql-reference/ Queries and views that reference this view must use the same flag value. A wrapper is used here because the default value is True.
"userDefinedFunctionResources": [ # Describes user-defined function resources used in the query.
{ # This is used for defining User Defined Function (UDF) resources only when using legacy SQL. Users of GoogleSQL should leverage either DDL (e.g. CREATE [TEMPORARY] FUNCTION ... ) or the Routines API to define UDF resources. For additional information on migrating, see: https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql#differences_in_user-defined_javascript_functions
"inlineCode": "A String", # [Pick one] An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.
"resourceUri": "A String", # [Pick one] A code resource to load from a Google Cloud Storage URI (gs://bucket/path).
},
],
},
}
autodetect_schema: boolean, Optional. When true will autodetect schema, else will keep original schema
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{
"biglakeConfiguration": { # Configuration for BigLake managed tables. # Optional. Specifies the configuration of a BigLake managed table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read and write to external storage, such as Cloud Storage. The connection_id can have the form `{project}.{location}.{connection_id}` or `projects/{project}/locations/{location}/connections/{connection_id}".
"fileFormat": "A String", # Optional. The file format the table data is stored in.
"storageUri": "A String", # Optional. The fully qualified location prefix of the external folder where table data is stored. The '*' wildcard character is not allowed. The URI should be in the format `gs://bucket/path_to_table/`
"tableFormat": "A String", # Optional. The table format the metadata only snapshots are stored in.
},
"cloneDefinition": { # Information about base table and clone time of a table clone. # Output only. Contains information about the clone. This value is set via the clone operation.
"baseTableReference": { # Required. Reference describing the ID of the table that was cloned.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"cloneTime": "A String", # Required. The time at which the base table was cloned. This value is reported in the JSON response using RFC3339 format.
},
"clustering": { # Configures table clustering. # Clustering specification for the table. Must be specified with time-based partitioning, data in the table will be first partitioned and subsequently clustered.
"fields": [ # One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. The ordering of the clustering fields should be prioritized from most to least important for filtering purposes. Additional information on limitations can be found here: https://cloud.google.com/bigquery/docs/creating-clustered-tables#limitations
"A String",
],
},
"creationTime": "A String", # Output only. The time when this table was created, in milliseconds since the epoch.
"defaultCollation": "A String", # Optional. Defines the default collation specification of new STRING fields in the table. During table creation or update, if a STRING field is added to this table without explicit collation specified, then the table inherits the table default collation. A change to this field affects only fields added afterwards, and does not alter the existing fields. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"defaultRoundingMode": "A String", # Optional. Defines the default rounding mode specification of new decimal fields (NUMERIC OR BIGNUMERIC) in the table. During table creation or update, if a decimal field is added to this table without an explicit rounding mode specified, then the field inherits the table default rounding mode. Changing this field doesn't affect existing fields.
"description": "A String", # Optional. A user-friendly description of this table.
"encryptionConfiguration": { # Configuration for Cloud KMS encryption settings. # Custom encryption configuration (e.g., Cloud KMS keys).
"kmsKeyName": "A String", # Optional. Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.
},
"etag": "A String", # Output only. A hash of this resource.
"expirationTime": "A String", # Optional. The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed. The defaultTableExpirationMs property of the encapsulating dataset can be used to set a default expirationTime on newly created tables.
"externalCatalogTableOptions": { # Metadata about open source compatible table. The fields contained in these options correspond to hive metastore's table level properties. # Optional. Options defining open source compatible table.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection is needed to read the open source table from BigQuery Engine. The connection_id can have the form `..` or `projects//locations//connections/`.
"parameters": { # Optional. A map of key value pairs defining the parameters and properties of the open source table. Corresponds with hive meta store table parameters. Maximum size of 4Mib.
"a_key": "A String",
},
"storageDescriptor": { # Contains information about how a table's data is stored and accessed by open source query engines. # Optional. A storage descriptor containing information about the physical storage of this table.
"inputFormat": "A String", # Optional. Specifies the fully qualified class name of the InputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"). The maximum length is 128 characters.
"locationUri": "A String", # Optional. The physical location of the table (e.g. `gs://spark-dataproc-data/pangea-data/case_sensitive/` or `gs://spark-dataproc-data/pangea-data/*`). The maximum length is 2056 bytes.
"outputFormat": "A String", # Optional. Specifies the fully qualified class name of the OutputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"). The maximum length is 128 characters.
"serdeInfo": { # Serializer and deserializer information. # Optional. Serializer and deserializer information.
"name": "A String", # Optional. Name of the SerDe. The maximum length is 256 characters.
"parameters": { # Optional. Key-value pairs that define the initialization parameters for the serialization library. Maximum size 10 Kib.
"a_key": "A String",
},
"serializationLibrary": "A String", # Required. Specifies a fully-qualified class name of the serialization library that is responsible for the translation of data between table representation and the underlying low-level input and output format structures. The maximum length is 256 characters.
},
},
},
"externalDataConfiguration": { # Optional. Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table.
"autodetect": True or False, # Try to detect schema and format options automatically. Any option specified explicitly will be honored.
"avroOptions": { # Options for external data sources. # Optional. Additional properties to set if sourceFormat is set to AVRO.
"useAvroLogicalTypes": True or False, # Optional. If sourceFormat is set to "AVRO", indicates whether to interpret logical types as the corresponding BigQuery data type (for example, TIMESTAMP), instead of using the raw type (for example, INTEGER).
},
"bigtableOptions": { # Options specific to Google Cloud Bigtable data sources. # Optional. Additional options if sourceFormat is set to BIGTABLE.
"columnFamilies": [ # Optional. List of column families to expose in the table schema along with their types. This list restricts the column families that can be referenced in queries and specifies their value types. You can use this list to do type conversions - see the 'type' field for more details. If you leave this list empty, all column families are present in the table schema and their values are read as BYTES. During a query only the column families referenced in that query are read from Bigtable.
{ # Information related to a Bigtable column family.
"columns": [ # Optional. Lists of columns that should be exposed as individual fields as opposed to a list of (column name, value) pairs. All columns whose qualifier matches a qualifier in this list can be accessed as `.`. Other columns can be accessed as a list through the `.Column` field.
{ # Information related to a Bigtable column.
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. 'encoding' can also be set at the column family level. However, the setting at this level takes precedence if 'encoding' is set at both levels.
"fieldName": "A String", # Optional. If the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as the column field name and is used as field name in queries.
"onlyReadLatest": True or False, # Optional. If this is set, only the latest version of value in this column are exposed. 'onlyReadLatest' can also be set at the column family level. However, the setting at this level takes precedence if 'onlyReadLatest' is set at both levels.
"qualifierEncoded": "A String", # [Required] Qualifier of the column. Columns in the parent column family that has this exact qualifier are exposed as `.` field. If the qualifier is valid UTF-8 string, it can be specified in the qualifier_string field. Otherwise, a base-64 encoded value must be set to qualifier_encoded. The column field name is the same as the column qualifier. However, if the qualifier is not a valid BigQuery field identifier i.e. does not match a-zA-Z*, a valid identifier must be provided as field_name.
"qualifierString": "A String", # Qualifier string.
"type": "A String", # Optional. The type to convert the value in cells of this column. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. 'type' can also be set at the column family level. However, the setting at this level takes precedence if 'type' is set at both levels.
},
],
"encoding": "A String", # Optional. The encoding of the values when the type is not STRING. Acceptable encoding values are: TEXT - indicates values are alphanumeric text strings. BINARY - indicates values are encoded using HBase Bytes.toBytes family of functions. This can be overridden for a specific column by listing that column in 'columns' and specifying an encoding for it.
"familyId": "A String", # Identifier of the column family.
"onlyReadLatest": True or False, # Optional. If this is set only the latest version of value are exposed for all columns in this column family. This can be overridden for a specific column by listing that column in 'columns' and specifying a different setting for that column.
"type": "A String", # Optional. The type to convert the value in cells of this column family. The values are expected to be encoded using HBase Bytes.toBytes function when using the BINARY encoding value. Following BigQuery types are allowed (case-sensitive): * BYTES * STRING * INTEGER * FLOAT * BOOLEAN * JSON Default type is BYTES. This can be overridden for a specific column by listing that column in 'columns' and specifying a type for it.
},
],
"ignoreUnspecifiedColumnFamilies": True or False, # Optional. If field is true, then the column families that are not specified in columnFamilies list are not exposed in the table schema. Otherwise, they are read with BYTES type values. The default value is false.
"outputColumnFamiliesAsJson": True or False, # Optional. If field is true, then each column family will be read as a single JSON column. Otherwise they are read as a repeated cell structure containing timestamp/value tuples. The default value is false.
"readRowkeyAsString": True or False, # Optional. If field is true, then the rowkey column families will be read and converted to string. Otherwise they are read with BYTES type values and users need to manually cast them with CAST if necessary. The default value is false.
},
"compression": "A String", # Optional. The compression type of the data source. Possible values include GZIP and NONE. The default value is NONE. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats. An empty string is an invalid value.
"connectionId": "A String", # Optional. The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id can have the form `{project_id}.{location_id};{connection_id}` or `projects/{project_id}/locations/{location_id}/connections/{connection_id}`.
"csvOptions": { # Information related to a CSV data source. # Optional. Additional properties to set if sourceFormat is set to CSV.
"allowJaggedRows": True or False, # Optional. Indicates if BigQuery should accept rows that are missing trailing optional columns. If true, BigQuery treats missing trailing columns as null values. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.
"allowQuotedNewlines": True or False, # Optional. Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, ISO-8859-1, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.
"fieldDelimiter": "A String", # Optional. The separator character for fields in a CSV file. The separator is interpreted as a single byte. For files encoded in ISO-8859-1, any single character can be used as a separator. For files encoded in UTF-8, characters represented in decimal range 1-127 (U+0001-U+007F) can be used without any modification. UTF-8 characters encoded with multiple bytes (i.e. U+0080 and above) will have only the first byte used for separating fields. The remaining bytes will be treated as a part of the field. BigQuery also supports the escape sequence "\t" (U+0009) to specify a tab separator. The default value is comma (",", U+002C).
"nullMarker": "A String", # Optional. Specifies a string that represents a null value in a CSV file. For example, if you specify "\N", BigQuery interprets "\N" as a null value when querying a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.
"preserveAsciiControlCharacters": True or False, # Optional. Indicates if the embedded ASCII control characters (the first 32 characters in the ASCII-table, from '\x00' to '\x1F') are preserved.
"quote": """, # Optional. The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote ("). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true. To include the specific quote character within a quoted value, precede it with an additional matching quote character. For example, if you want to escape the default character ' " ', use ' "" '.
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a CSV file that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"decimalTargetTypes": [ # Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown. Example: Suppose the value of this field is ["NUMERIC", "BIGNUMERIC"]. If (precision,scale) is: * (38,9) -> NUMERIC; * (39,9) -> BIGNUMERIC (NUMERIC cannot hold 30 integer digits); * (38,10) -> BIGNUMERIC (NUMERIC cannot hold 10 fractional digits); * (76,38) -> BIGNUMERIC; * (77,38) -> BIGNUMERIC (error if value exeeds supported range). This field cannot contain duplicate types. The order of the types in this field is ignored. For example, ["BIGNUMERIC", "NUMERIC"] is the same as ["NUMERIC", "BIGNUMERIC"] and NUMERIC always takes precedence over BIGNUMERIC. Defaults to ["NUMERIC", "STRING"] for ORC and ["NUMERIC"] for the other file formats.
"A String",
],
"fileSetSpecType": "A String", # Optional. Specifies how source URIs are interpreted for constructing the file set to load. By default source URIs are expanded against the underlying storage. Other options include specifying manifest files. Only applicable to object storage systems.
"googleSheetsOptions": { # Options specific to Google Sheets data sources. # Optional. Additional options if sourceFormat is set to GOOGLE_SHEETS.
"range": "A String", # Optional. Range of a sheet to query from. Only used when non-empty. Typical format: sheet_name!top_left_cell_id:bottom_right_cell_id For example: sheet1!A1:B20
"skipLeadingRows": "A String", # Optional. The number of rows at the top of a sheet that BigQuery will skip when reading the data. The default value is 0. This property is useful if you have header rows that should be skipped. When autodetect is on, the behavior is the following: * skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. * skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. * skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.
},
"hivePartitioningOptions": { # Options for configuring hive partitioning detect. # Optional. When set, configures hive partitioning support. Not all storage formats support hive partitioning -- requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.
"fields": [ # Output only. For permanent external tables, this field is populated with the hive partition keys in the order they were inferred. The types of the partition keys can be deduced by checking the table schema (which will include the partition keys). Not every API will populate this field in the output. For example, Tables.Get will populate it, but Tables.List will not contain this field.
"A String",
],
"mode": "A String", # Optional. When set, what mode of hive partitioning to use when reading data. The following modes are supported: * AUTO: automatically infer partition key name(s) and type(s). * STRINGS: automatically infer partition key name(s). All types are strings. * CUSTOM: partition key schema is encoded in the source URI prefix. Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. Note that this field should only be true when creating a permanent external table or querying a temporary external table. Hive-partitioned loads with require_partition_filter explicitly set to true will fail.
"sourceUriPrefix": "A String", # Optional. When hive partition detection is requested, a common prefix for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout: gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. CUSTOM detection requires encoding the partitioning schema immediately after the common prefix. For CUSTOM, any of * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:STRING}/{country:STRING}/{id:INTEGER} * gs://bucket/path_to_table/{dt:DATE}/{country:STRING}/{id:STRING} would all be valid source URI prefixes.
},
"ignoreUnknownValues": True or False, # Optional. Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don't match any column names Google Cloud Bigtable: This setting is ignored. Google Cloud Datastore backups: This setting is ignored. Avro: This setting is ignored. ORC: This setting is ignored. Parquet: This setting is ignored.
"jsonExtension": "A String", # Optional. Load option to be used together with source_format newline-delimited JSON to indicate that a variant of JSON is being loaded. To load newline-delimited GeoJSON, specify GEOJSON (and source_format must be set to NEWLINE_DELIMITED_JSON).
"jsonOptions": { # Json Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to JSON.
"encoding": "A String", # Optional. The character encoding of the data. The supported values are UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE. The default value is UTF-8.
},
"maxBadRecords": 42, # Optional. The maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid. This setting is ignored for Google Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats.
"metadataCacheMode": "A String", # Optional. Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source.
"objectMetadata": "A String", # Optional. ObjectMetadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the source_uris. If ObjectMetadata is set, source_format should be omitted. Currently SIMPLE is the only supported Object Metadata type.
"parquetOptions": { # Parquet Options for load and make external tables. # Optional. Additional properties to set if sourceFormat is set to PARQUET.
"enableListInference": True or False, # Optional. Indicates whether to use schema inference specifically for Parquet LIST logical type.
"enumAsString": True or False, # Optional. Indicates whether to infer Parquet ENUM logical type as STRING instead of BYTES by default.
"mapTargetType": "A String", # Optional. Indicates how to represent a Parquet map if present.
},
"referenceFileSchemaUri": "A String", # Optional. When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.
"schema": { # Schema of a table # Optional. The schema for the data. Schema is required for CSV and JSON formats if autodetect is not on. Schema is disallowed for Google Cloud Bigtable, Cloud Datastore backups, Avro, ORC and Parquet formats.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"sourceFormat": "A String", # [Required] The data format. For CSV files, specify "CSV". For Google sheets, specify "GOOGLE_SHEETS". For newline-delimited JSON, specify "NEWLINE_DELIMITED_JSON". For Avro files, specify "AVRO". For Google Cloud Datastore backups, specify "DATASTORE_BACKUP". For Apache Iceberg tables, specify "ICEBERG". For ORC files, specify "ORC". For Parquet files, specify "PARQUET". [Beta] For Google Cloud Bigtable, specify "BIGTABLE".
"sourceUris": [ # [Required] The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one '*' wildcard character and it must come after the 'bucket' name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the '*' wildcard character is not allowed.
"A String",
],
},
"friendlyName": "A String", # Optional. A descriptive name for this table.
"id": "A String", # Output only. An opaque ID uniquely identifying the table.
"kind": "bigquery#table", # The type of resource ID.
"labels": { # The labels associated with this table. You can use these to organize and group your tables. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.
"a_key": "A String",
},
"lastModifiedTime": "A String", # Output only. The time when this table was last modified, in milliseconds since the epoch.
"location": "A String", # Output only. The geographic location where the table resides. This value is inherited from the dataset.
"materializedView": { # Definition and configuration of a materialized view. # Optional. The materialized view definition.
"allowNonIncrementalDefinition": True or False, # Optional. This option declares the intention to construct a materialized view that isn't refreshed incrementally.
"enableRefresh": True or False, # Optional. Enable automatic refresh of the materialized view when the base table is updated. The default value is "true".
"lastRefreshTime": "A String", # Output only. The time when this materialized view was last refreshed, in milliseconds since the epoch.
"maxStaleness": "A String", # [Optional] Max staleness of data that could be returned when materizlized view is queried (formatted as Google SQL Interval type).
"query": "A String", # Required. A query whose results are persisted.
"refreshIntervalMs": "A String", # Optional. The maximum frequency at which this materialized view will be refreshed. The default value is "1800000" (30 minutes).
},
"materializedViewStatus": { # Status of a materialized view. The last refresh timestamp status is omitted here, but is present in the MaterializedViewDefinition message. # Output only. The materialized view status.
"lastRefreshStatus": { # Error details. # Output only. Error result of the last automatic refresh. If present, indicates that the last automatic refresh was unsuccessful.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"refreshWatermark": "A String", # Output only. Refresh watermark of materialized view. The base tables' data were collected into the materialized view cache until this time.
},
"maxStaleness": "A String", # Optional. The maximum staleness of data that could be returned when the table (or stale MV) is queried. Staleness encoded as a string encoding of sql IntervalValue type.
"model": { # Deprecated.
"modelOptions": { # Deprecated.
"labels": [
"A String",
],
"lossType": "A String",
"modelType": "A String",
},
"trainingRuns": [ # Deprecated.
{
"iterationResults": [ # Deprecated.
{
"durationMs": "A String", # Deprecated.
"evalLoss": 3.14, # Deprecated.
"index": 42, # Deprecated.
"learnRate": 3.14, # Deprecated.
"trainingLoss": 3.14, # Deprecated.
},
],
"startTime": "A String", # Deprecated.
"state": "A String", # Deprecated.
"trainingOptions": { # Deprecated.
"earlyStop": True or False,
"l1Reg": 3.14,
"l2Reg": 3.14,
"learnRate": 3.14,
"learnRateStrategy": "A String",
"lineSearchInitLearnRate": 3.14,
"maxIteration": "A String",
"minRelProgress": 3.14,
"warmStart": True or False,
},
},
],
},
"numActiveLogicalBytes": "A String", # Output only. Number of logical bytes that are less than 90 days old.
"numActivePhysicalBytes": "A String", # Output only. Number of physical bytes less than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numBytes": "A String", # Output only. The size of this table in logical bytes, excluding any data in the streaming buffer.
"numCurrentPhysicalBytes": "A String", # Output only. Number of physical bytes used by current live data storage. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numLongTermBytes": "A String", # Output only. The number of logical bytes in the table that are considered "long-term storage".
"numLongTermLogicalBytes": "A String", # Output only. Number of logical bytes that are more than 90 days old.
"numLongTermPhysicalBytes": "A String", # Output only. Number of physical bytes more than 90 days old. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPartitions": "A String", # Output only. The number of partitions present in the table or materialized view. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This includes storage used for time travel.
"numRows": "A String", # Output only. The number of rows of data in this table, excluding any data in the streaming buffer.
"numTimeTravelPhysicalBytes": "A String", # Output only. Number of physical bytes used by time travel storage (deleted or changed data). This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"numTotalLogicalBytes": "A String", # Output only. Total number of logical bytes in the table or materialized view.
"numTotalPhysicalBytes": "A String", # Output only. The physical size of this table in bytes. This also includes storage used for time travel. This data is not kept in real time, and might be delayed by a few seconds to a few minutes.
"partitionDefinition": { # The partitioning information, which includes managed table, external table and metastore partitioned table partition information. # Optional. The partition information for all table formats, including managed partitioned tables, hive partitioned tables, iceberg partitioned, and metastore partitioned tables. This field is only populated for metastore partitioned tables. For other table formats, this is an output only field.
"partitionedColumn": [ # Optional. Details about each partitioning column. This field is output only for all partitioning types other than metastore partitioned tables. BigQuery native tables only support 1 partitioning column. Other table types may support 0, 1 or more partitioning columns. For metastore partitioned tables, the order must match the definition order in the Hive Metastore, where it must match the physical layout of the table. For example, CREATE TABLE a_table(id BIGINT, name STRING) PARTITIONED BY (city STRING, state STRING). In this case the values must be ['city', 'state'] in that order.
{ # The partitioning column information.
"field": "A String", # Required. The name of the partition column.
},
],
},
"rangePartitioning": { # If specified, configures range partitioning for this table.
"field": "A String", # Required. The name of the column to partition the table on. It must be a top-level, INT64 column whose mode is NULLABLE or REQUIRED.
"range": { # [Experimental] Defines the ranges for range partitioning.
"end": "A String", # [Experimental] The end of range partitioning, exclusive.
"interval": "A String", # [Experimental] The width of each interval.
"start": "A String", # [Experimental] The start of range partitioning, inclusive.
},
},
"replicas": [ # Optional. Output only. Table references of all replicas currently active on the table.
{
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
],
"requirePartitionFilter": false, # Optional. If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.
"resourceTags": { # [Optional] The tags associated with this table. Tag keys are globally unique. See additional information on [tags](https://cloud.google.com/iam/docs/tags-access-control#definitions). An object containing a list of "key": value pairs. The key is the namespaced friendly name of the tag key, e.g. "12345/environment" where 12345 is parent id. The value is the friendly short name of the tag value, e.g. "production".
"a_key": "A String",
},
"restrictions": { # Optional. Output only. Restriction config for table. If set, restrict certain accesses on the table based on the config. See [Data egress](https://cloud.google.com/bigquery/docs/analytics-hub-introduction#data_egress) for more details.
"type": "A String", # Output only. Specifies the type of dataset/table restriction.
},
"schema": { # Schema of a table # Optional. Describes the schema of this table.
"fields": [ # Describes the fields in a table.
{ # A field in TableSchema
"categories": { # Deprecated.
"names": [ # Deprecated.
"A String",
],
},
"collation": "A String", # Optional. Field collation can be set only when the type of field is STRING. The following values are supported: * 'und:ci': undetermined locale, case insensitive. * '': empty string. Default to case-sensitive behavior.
"dataPolicies": [ # Optional. Data policy options, will replace the data_policies.
{ # Data policy option proto, it currently supports name only, will support precedence later.
"name": "A String", # Data policy resource name in the form of projects/project_id/locations/location_id/dataPolicies/data_policy_id.
},
],
"defaultValueExpression": "A String", # Optional. A SQL expression to specify the [default value] (https://cloud.google.com/bigquery/docs/default-values) for this field.
"description": "A String", # Optional. The field description. The maximum length is 1,024 characters.
"fields": [ # Optional. Describes the nested schema fields if the type property is set to RECORD.
# Object with schema name: TableFieldSchema
],
"foreignTypeDefinition": "A String", # Optional. Definition of the foreign data type. Only valid for top-level schema fields (not nested fields). If the type is FOREIGN, this field is required.
"maxLength": "A String", # Optional. Maximum length of values of this field for STRINGS or BYTES. If max_length is not specified, no maximum length constraint is imposed on this field. If type = "STRING", then max_length represents the maximum UTF-8 length of strings in this field. If type = "BYTES", then max_length represents the maximum number of bytes in this field. It is invalid to set this field if type ≠ "STRING" and ≠ "BYTES".
"mode": "A String", # Optional. The field mode. Possible values include NULLABLE, REQUIRED and REPEATED. The default value is NULLABLE.
"name": "A String", # Required. The field name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 300 characters.
"policyTags": { # Optional. The policy tags attached to this field, used for field-level access control. If not set, defaults to empty policy_tags.
"names": [ # A list of policy tag resource names. For example, "projects/1/locations/eu/taxonomies/2/policyTags/3". At most 1 policy tag is currently allowed.
"A String",
],
},
"precision": "A String", # Optional. Precision (maximum number of total digits in base 10) and scale (maximum number of digits in the fractional part in base 10) constraints for values of this field for NUMERIC or BIGNUMERIC. It is invalid to set precision or scale if type ≠ "NUMERIC" and ≠ "BIGNUMERIC". If precision and scale are not specified, no value range constraint is imposed on this field insofar as values are permitted by the type. Values of this NUMERIC or BIGNUMERIC field must be in this range when: * Precision (P) and scale (S) are specified: [-10P-S + 10-S, 10P-S - 10-S] * Precision (P) is specified but not scale (and thus scale is interpreted to be equal to zero): [-10P + 1, 10P - 1]. Acceptable values for precision and scale if both are specified: * If type = "NUMERIC": 1 ≤ precision - scale ≤ 29 and 0 ≤ scale ≤ 9. * If type = "BIGNUMERIC": 1 ≤ precision - scale ≤ 38 and 0 ≤ scale ≤ 38. Acceptable values for precision if only precision is specified but not scale (and thus scale is interpreted to be equal to zero): * If type = "NUMERIC": 1 ≤ precision ≤ 29. * If type = "BIGNUMERIC": 1 ≤ precision ≤ 38. If scale is specified but not precision, then it is invalid.
"rangeElementType": { # Represents the type of a field element.
"type": "A String", # Required. The type of a field element. For more information, see TableFieldSchema.type.
},
"roundingMode": "A String", # Optional. Specifies the rounding mode to be used when storing values of NUMERIC and BIGNUMERIC type.
"scale": "A String", # Optional. See documentation for precision.
"type": "A String", # Required. The field data type. Possible values include: * STRING * BYTES * INTEGER (or INT64) * FLOAT (or FLOAT64) * BOOLEAN (or BOOL) * TIMESTAMP * DATE * TIME * DATETIME * GEOGRAPHY * NUMERIC * BIGNUMERIC * JSON * RECORD (or STRUCT) * RANGE Use of RECORD/STRUCT indicates that the field contains a nested schema.
},
],
"foreignTypeInfo": { # Metadata about the foreign data type definition such as the system in which the type is defined. # Optional. Specifies metadata of the foreign data type definition in field schema (TableFieldSchema.foreign_type_definition).
"typeSystem": "A String", # Required. Specifies the system which defines the foreign data type.
},
},
"selfLink": "A String", # Output only. A URL that can be used to access this resource again.
"snapshotDefinition": { # Information about base table and snapshot time of the snapshot. # Output only. Contains information about the snapshot. This value is set via snapshot creation.
"baseTableReference": { # Required. Reference describing the ID of the table that was snapshot.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"snapshotTime": "A String", # Required. The time at which the base table was snapshot. This value is reported in the JSON response using RFC3339 format.
},
"streamingBuffer": { # Output only. Contains information regarding this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
"estimatedBytes": "A String", # Output only. A lower-bound estimate of the number of bytes currently in the streaming buffer.
"estimatedRows": "A String", # Output only. A lower-bound estimate of the number of rows currently in the streaming buffer.
"oldestEntryTime": "A String", # Output only. Contains the timestamp of the oldest entry in the streaming buffer, in milliseconds since the epoch, if the streaming buffer is available.
},
"tableConstraints": { # The TableConstraints defines the primary key and foreign key. # Optional. Tables Primary Key and Foreign Key information
"foreignKeys": [ # Optional. Present only if the table has a foreign key. The foreign key is not enforced.
{ # Represents a foreign key constraint on a table's columns.
"columnReferences": [ # Required. The columns that compose the foreign key.
{ # The pair of the foreign key column and primary key column.
"referencedColumn": "A String", # Required. The column in the primary key that are referenced by the referencing_column.
"referencingColumn": "A String", # Required. The column that composes the foreign key.
},
],
"name": "A String", # Optional. Set only if the foreign key constraint is named.
"referencedTable": {
"datasetId": "A String",
"projectId": "A String",
"tableId": "A String",
},
},
],
"primaryKey": { # Represents the primary key constraint on a table's columns.
"columns": [ # Required. The columns that are composed of the primary key constraint.
"A String",
],
},
},
"tableReference": { # Required. Reference describing the ID of this table.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
"tableReplicationInfo": { # Replication info of a table created using `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv` # Optional. Table replication info for table created `AS REPLICA` DDL like: `CREATE MATERIALIZED VIEW mv1 AS REPLICA OF src_mv`
"replicatedSourceLastRefreshTime": "A String", # Optional. Output only. If source is a materialized view, this field signifies the last refresh time of the source.
"replicationError": { # Error details. # Optional. Output only. Replication error that will permanently stopped table replication.
"debugInfo": "A String", # Debugging information. This property is internal to Google and should not be used.
"location": "A String", # Specifies where the error occurred, if present.
"message": "A String", # A human-readable description of the error.
"reason": "A String", # A short error code that summarizes the error.
},
"replicationIntervalMs": "A String", # Optional. Specifies the interval at which the source table is polled for updates. It's Optional. If not specified, default replication interval would be applied.
"replicationStatus": "A String", # Optional. Output only. Replication status of configured replication.
"sourceTable": { # Required. Source table reference that is replicated.
"datasetId": "A String", # Required. The ID of the dataset containing this table.
"projectId": "A String", # Required. The ID of the project containing this table.
"tableId": "A String", # Required. The ID of the table. The ID can contain Unicode characters in category L (letter), M (mark), N (number), Pc (connector, including underscore), Pd (dash), and Zs (space). For more information, see [General Category](https://wikipedia.org/wiki/Unicode_character_property#General_Category). The maximum length is 1,024 characters. Certain operations allow suffixing of the table ID with a partition decorator, such as `sample_table$20190123`.
},
},
"timePartitioning": { # If specified, configures time-based partitioning for this table.
"expirationMs": "A String", # Optional. Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.
"field": "A String", # Optional. If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.
"requirePartitionFilter": false, # If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. This field is deprecated; please set the field with the same name on the table itself instead. This field needs a wrapper because we want to output the default value, false, if the user explicitly set it.
"type": "A String", # Required. The supported types are DAY, HOUR, MONTH, and YEAR, which will generate one partition per day, hour, month, and year, respectively.
},
"type": "A String", # Output only. Describes the table type. The following values are supported: * `TABLE`: A normal BigQuery table. * `VIEW`: A virtual table defined by a SQL query. * `EXTERNAL`: A table that references data stored in an external storage system, such as Google Cloud Storage. * `MATERIALIZED_VIEW`: A precomputed view defined by a SQL query. * `SNAPSHOT`: An immutable BigQuery table that preserves the contents of a base table at a particular time. See additional information on [table snapshots](https://cloud.google.com/bigquery/docs/table-snapshots-intro). The default value is `TABLE`.
"view": { # Describes the definition of a logical view. # Optional. The view definition.
"foreignDefinitions": [ # Optional. Foreign view representations.
{ # A view can be represented in multiple ways. Each representation has its own dialect. This message stores the metadata required for these representations.
"dialect": "A String", # Optional. Represents the dialect of the query.
"query": "A String", # Required. The query that defines the view.
},
],
"privacyPolicy": { # Represents privacy policy that contains the privacy requirements specified by the data owner. Currently, this is only supported on views. # Optional. Specifices the privacy policy for the view.
"aggregationThresholdPolicy": { # Represents privacy policy associated with "aggregation threshold" method. # Optional. Policy used for aggregation thresholds.
"privacyUnitColumns": [ # Optional. The privacy unit column(s) associated with this policy. For now, only one column per data source object (table, view) is allowed as a privacy unit column. Representing as a repeated field in metadata for extensibility to multiple columns in future. Duplicates and Repeated struct fields are not allowed. For nested fields, use dot notation ("outer.inner")
"A String",
],
"threshold": "A String", # Optional. The threshold for the "aggregation threshold" policy.
},
"differentialPrivacyPolicy": { # Represents privacy policy associated with "differential privacy" method. # Optional. Policy used for differential privacy.
"deltaBudget": 3.14, # Optional. The total delta budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of delta that is pre-defined by the contributor through the privacy policy delta_per_query field. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"deltaBudgetRemaining": 3.14, # Output only. The delta budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"deltaPerQuery": 3.14, # Optional. The delta value that is used per query. Delta represents the probability that any row will fail to be epsilon differentially private. Indicates the risk associated with exposing aggregate rows in the result of a query.
"epsilonBudget": 3.14, # Optional. The total epsilon budget for all queries against the privacy-protected view. Each subscriber query against this view charges the amount of epsilon they request in their query. If there is sufficient budget, then the subscriber query attempts to complete. It might still fail due to other reasons, in which case the charge is refunded. If there is insufficient budget the query is rejected. There might be multiple charge attempts if a single query references multiple views. In this case there must be sufficient budget for all charges or the query is rejected and charges are refunded in best effort. The budget does not have a refresh policy and can only be updated via ALTER VIEW or circumvented by creating a new view that can be queried with a fresh budget.
"epsilonBudgetRemaining": 3.14, # Output only. The epsilon budget remaining. If budget is exhausted, no more queries are allowed. Note that the budget for queries that are in progress is deducted before the query executes. If the query fails or is cancelled then the budget is refunded. In this case the amount of budget remaining can increase.
"maxEpsilonPerQuery": 3.14, # Optional. The maximum epsilon value that a query can consume. If the subscriber specifies epsilon as a parameter in a SELECT query, it must be less than or equal to this value. The epsilon parameter controls the amount of noise that is added to the groups — a higher epsilon means less noise.
"maxGroupsContributed": "A String", # Optional. The maximum groups contributed value that is used per query. Represents the maximum number of groups to which each protected entity can contribute. Changing this value does not improve or worsen privacy. The best value for accuracy and utility depends on the query and data.
"privacyUnitColumn": "A String", # Optional. The privacy unit column associated with this policy. Differential privacy policies can only have one privacy unit column per data source object (table, view).
},
"joinRestrictionPolicy": { # Represents privacy policy associated with "join restrictions". Join restriction gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view. # Optional. Join restriction policy is outside of the one of policies, since this policy can be set along with other policies. This policy gives data providers the ability to enforce joins on the 'join_allowed_columns' when data is queried from a privacy protected view.
"joinAllowedColumns": [ # Optional. The only columns that joins are allowed on. This field is must be specified for join_conditions JOIN_ANY and JOIN_ALL and it cannot be set for JOIN_BLOCKED.
"A String",
],
"joinCondition": "A String", # Optional. Specifies if a join is required or not on queries for the view. Default is JOIN_CONDITION_UNSPECIFIED.
},
},
"query": "A String", # Required. A query that BigQuery executes when the view is referenced.
"useExplicitColumnNames": True or False, # True if the column names are explicitly specified. For example by using the 'CREATE VIEW v(c1, c2) AS ...' syntax. Can only be set for GoogleSQL views.
"useLegacySql": True or False, # Specifies whether to use BigQuery's legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery's GoogleSQL: https://cloud.google.com/bigquery/sql-reference/ Queries and views that reference this view must use the same flag value. A wrapper is used here because the default value is True.
"userDefinedFunctionResources": [ # Describes user-defined function resources used in the query.
{ # This is used for defining User Defined Function (UDF) resources only when using legacy SQL. Users of GoogleSQL should leverage either DDL (e.g. CREATE [TEMPORARY] FUNCTION ... ) or the Routines API to define UDF resources. For additional information on migrating, see: https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql#differences_in_user-defined_javascript_functions
"inlineCode": "A String", # [Pick one] An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.
"resourceUri": "A String", # [Pick one] A code resource to load from a Google Cloud Storage URI (gs://bucket/path).
},
],
},
}