Returns the partitions Resource.
Close httplib2 connections.
create(parent, body=None, validateOnly=None, x__xgafv=None)
Create a metadata entity.
delete(name, etag=None, x__xgafv=None)
Delete a metadata entity.
get(name, view=None, x__xgafv=None)
Get a metadata entity.
list(parent, filter=None, pageSize=None, pageToken=None, view=None, x__xgafv=None)
List metadata entities in a zone.
Retrieves the next page of results.
update(name, body=None, validateOnly=None, x__xgafv=None)
Update a metadata entity. Only supports full resource update.
close()
Close httplib2 connections.
create(parent, body=None, validateOnly=None, x__xgafv=None)
Create a metadata entity.
Args:
parent: string, Required. The resource name of the parent zone: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}. (required)
body: object, The request body.
The object takes the form of:
{ # Represents tables and fileset metadata contained within a zone.
"access": { # Describes the access mechanism of the data within its storage location. # Output only. Identifies the access mechanism to the entity. Not user settable.
"read": "A String", # Output only. Describes the read access mechanism of the data. Not user settable.
},
"asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset.
"catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry.
"compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with.
"bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
"hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
},
"createTime": "A String", # Output only. The time when the entity was created.
"dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id.
"dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv.
"description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters.
"displayName": "A String", # Optional. Display name must be shorter than or equal to 256 characters.
"etag": "A String", # Optional. The etag associated with the entity, which can be retrieved with a GetEntity request. Required for update and delete requests.
"format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery.
"compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed.
"csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data.
"delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified.
"headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0.
"quote": "A String", # Optional. The character used to quote column values. Accepts '"' (double quotation mark) or ''' (single quotation mark). Defaults to '"' (double quotation mark) if unspecified.
},
"format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type.
"iceberg": { # Describes Iceberg data format. # Optional. Additional information about iceberg tables.
"metadataLocation": "A String", # Optional. The location of where the iceberg metadata is present, must be within the table path
},
"json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified.
},
"mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: application/x-parquet application/x-avro application/x-orc application/x-tfrecord application/x-parquet+iceberg application/x-avro+iceberg application/x-orc+iceberg application/json application/{subtypes} text/csv text/ image/{image subtype} video/{video subtype} audio/{audio subtype}
},
"id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores, and consist of 256 or fewer characters.
"name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}.
"schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response.
"fields": [ # Optional. The sequence of fields describing data in table entities. Note: BigQuery SchemaFields are immutable.
{ # Represents a column field within a table schema.
"description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters.
"fields": [ # Optional. Any nested field for complex types.
# Object with schema name: GoogleCloudDataplexV1SchemaSchemaField
],
"mode": "A String", # Required. Additional field semantics.
"name": "A String", # Required. The name of the field. Must contain only letters, numbers and underscores, with a maximum length of 767 characters, and must begin with a letter or underscore.
"type": "A String", # Required. The type of field.
},
],
"partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data.
{ # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. Note: Partition fields are immutable.
"name": "A String", # Required. Partition field name must consist of letters, numbers, and underscores only, with a maximum of length of 256 characters, and must begin with a letter or underscore..
"type": "A String", # Required. Immutable. The type of field.
},
],
"partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity.
"userManaged": True or False, # Required. Set to true if user-managed or false if managed by Dataplex. The default is false (managed by Dataplex). Set to falseto enable Dataplex discovery to update the schema. including new data discovery, schema inference, and schema evolution. Users retain the ability to input and edit the schema. Dataplex treats schema input by the user as though produced by a previous Dataplex discovery operation, and it will evolve the schema and take action based on that treatment. Set to true to fully manage the entity schema. This setting guarantees that Dataplex will not change schema fields.
},
"system": "A String", # Required. Immutable. Identifies the storage system of the entity data.
"type": "A String", # Required. Immutable. The type of entity.
"uid": "A String", # Output only. System generated unique ID for the Entity. This ID will be different if the Entity is deleted and re-created with the same name.
"updateTime": "A String", # Output only. The time when the entity was last updated.
}
validateOnly: boolean, Optional. Only validate the request, but do not perform mutations. The default is false.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # Represents tables and fileset metadata contained within a zone.
"access": { # Describes the access mechanism of the data within its storage location. # Output only. Identifies the access mechanism to the entity. Not user settable.
"read": "A String", # Output only. Describes the read access mechanism of the data. Not user settable.
},
"asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset.
"catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry.
"compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with.
"bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
"hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
},
"createTime": "A String", # Output only. The time when the entity was created.
"dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id.
"dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv.
"description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters.
"displayName": "A String", # Optional. Display name must be shorter than or equal to 256 characters.
"etag": "A String", # Optional. The etag associated with the entity, which can be retrieved with a GetEntity request. Required for update and delete requests.
"format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery.
"compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed.
"csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data.
"delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified.
"headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0.
"quote": "A String", # Optional. The character used to quote column values. Accepts '"' (double quotation mark) or ''' (single quotation mark). Defaults to '"' (double quotation mark) if unspecified.
},
"format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type.
"iceberg": { # Describes Iceberg data format. # Optional. Additional information about iceberg tables.
"metadataLocation": "A String", # Optional. The location of where the iceberg metadata is present, must be within the table path
},
"json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified.
},
"mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: application/x-parquet application/x-avro application/x-orc application/x-tfrecord application/x-parquet+iceberg application/x-avro+iceberg application/x-orc+iceberg application/json application/{subtypes} text/csv text/ image/{image subtype} video/{video subtype} audio/{audio subtype}
},
"id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores, and consist of 256 or fewer characters.
"name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}.
"schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response.
"fields": [ # Optional. The sequence of fields describing data in table entities. Note: BigQuery SchemaFields are immutable.
{ # Represents a column field within a table schema.
"description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters.
"fields": [ # Optional. Any nested field for complex types.
# Object with schema name: GoogleCloudDataplexV1SchemaSchemaField
],
"mode": "A String", # Required. Additional field semantics.
"name": "A String", # Required. The name of the field. Must contain only letters, numbers and underscores, with a maximum length of 767 characters, and must begin with a letter or underscore.
"type": "A String", # Required. The type of field.
},
],
"partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data.
{ # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. Note: Partition fields are immutable.
"name": "A String", # Required. Partition field name must consist of letters, numbers, and underscores only, with a maximum of length of 256 characters, and must begin with a letter or underscore..
"type": "A String", # Required. Immutable. The type of field.
},
],
"partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity.
"userManaged": True or False, # Required. Set to true if user-managed or false if managed by Dataplex. The default is false (managed by Dataplex). Set to falseto enable Dataplex discovery to update the schema. including new data discovery, schema inference, and schema evolution. Users retain the ability to input and edit the schema. Dataplex treats schema input by the user as though produced by a previous Dataplex discovery operation, and it will evolve the schema and take action based on that treatment. Set to true to fully manage the entity schema. This setting guarantees that Dataplex will not change schema fields.
},
"system": "A String", # Required. Immutable. Identifies the storage system of the entity data.
"type": "A String", # Required. Immutable. The type of entity.
"uid": "A String", # Output only. System generated unique ID for the Entity. This ID will be different if the Entity is deleted and re-created with the same name.
"updateTime": "A String", # Output only. The time when the entity was last updated.
}
delete(name, etag=None, x__xgafv=None)
Delete a metadata entity.
Args:
name: string, Required. The resource name of the entity: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{entity_id}. (required)
etag: string, Required. The etag associated with the entity, which can be retrieved with a GetEntity request.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); }
}
get(name, view=None, x__xgafv=None)
Get a metadata entity.
Args:
name: string, Required. The resource name of the entity: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{entity_id}. (required)
view: string, Optional. Used to select the subset of entity information to return. Defaults to BASIC.
Allowed values
ENTITY_VIEW_UNSPECIFIED - The API will default to the BASIC view.
BASIC - Minimal view that does not include the schema.
SCHEMA - Include basic information and schema.
FULL - Include everything. Currently, this is the same as the SCHEMA view.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # Represents tables and fileset metadata contained within a zone.
"access": { # Describes the access mechanism of the data within its storage location. # Output only. Identifies the access mechanism to the entity. Not user settable.
"read": "A String", # Output only. Describes the read access mechanism of the data. Not user settable.
},
"asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset.
"catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry.
"compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with.
"bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
"hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
},
"createTime": "A String", # Output only. The time when the entity was created.
"dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id.
"dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv.
"description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters.
"displayName": "A String", # Optional. Display name must be shorter than or equal to 256 characters.
"etag": "A String", # Optional. The etag associated with the entity, which can be retrieved with a GetEntity request. Required for update and delete requests.
"format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery.
"compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed.
"csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data.
"delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified.
"headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0.
"quote": "A String", # Optional. The character used to quote column values. Accepts '"' (double quotation mark) or ''' (single quotation mark). Defaults to '"' (double quotation mark) if unspecified.
},
"format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type.
"iceberg": { # Describes Iceberg data format. # Optional. Additional information about iceberg tables.
"metadataLocation": "A String", # Optional. The location of where the iceberg metadata is present, must be within the table path
},
"json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified.
},
"mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: application/x-parquet application/x-avro application/x-orc application/x-tfrecord application/x-parquet+iceberg application/x-avro+iceberg application/x-orc+iceberg application/json application/{subtypes} text/csv text/ image/{image subtype} video/{video subtype} audio/{audio subtype}
},
"id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores, and consist of 256 or fewer characters.
"name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}.
"schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response.
"fields": [ # Optional. The sequence of fields describing data in table entities. Note: BigQuery SchemaFields are immutable.
{ # Represents a column field within a table schema.
"description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters.
"fields": [ # Optional. Any nested field for complex types.
# Object with schema name: GoogleCloudDataplexV1SchemaSchemaField
],
"mode": "A String", # Required. Additional field semantics.
"name": "A String", # Required. The name of the field. Must contain only letters, numbers and underscores, with a maximum length of 767 characters, and must begin with a letter or underscore.
"type": "A String", # Required. The type of field.
},
],
"partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data.
{ # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. Note: Partition fields are immutable.
"name": "A String", # Required. Partition field name must consist of letters, numbers, and underscores only, with a maximum of length of 256 characters, and must begin with a letter or underscore..
"type": "A String", # Required. Immutable. The type of field.
},
],
"partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity.
"userManaged": True or False, # Required. Set to true if user-managed or false if managed by Dataplex. The default is false (managed by Dataplex). Set to falseto enable Dataplex discovery to update the schema. including new data discovery, schema inference, and schema evolution. Users retain the ability to input and edit the schema. Dataplex treats schema input by the user as though produced by a previous Dataplex discovery operation, and it will evolve the schema and take action based on that treatment. Set to true to fully manage the entity schema. This setting guarantees that Dataplex will not change schema fields.
},
"system": "A String", # Required. Immutable. Identifies the storage system of the entity data.
"type": "A String", # Required. Immutable. The type of entity.
"uid": "A String", # Output only. System generated unique ID for the Entity. This ID will be different if the Entity is deleted and re-created with the same name.
"updateTime": "A String", # Output only. The time when the entity was last updated.
}
list(parent, filter=None, pageSize=None, pageToken=None, view=None, x__xgafv=None)
List metadata entities in a zone.
Args:
parent: string, Required. The resource name of the parent zone: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}. (required)
filter: string, Optional. The following filter parameters can be added to the URL to limit the entities returned by the API: Entity ID: ?filter="id=entityID" Asset ID: ?filter="asset=assetID" Data path ?filter="data_path=gs://my-bucket" Is HIVE compatible: ?filter="hive_compatible=true" Is BigQuery compatible: ?filter="bigquery_compatible=true"
pageSize: integer, Optional. Maximum number of entities to return. The service may return fewer than this value. If unspecified, 100 entities will be returned by default. The maximum value is 500; larger values will will be truncated to 500.
pageToken: string, Optional. Page token received from a previous ListEntities call. Provide this to retrieve the subsequent page. When paginating, all other parameters provided to ListEntities must match the call that provided the page token.
view: string, Required. Specify the entity view to make a partial list request.
Allowed values
ENTITY_VIEW_UNSPECIFIED - The default unset value. Return both table and fileset entities if unspecified.
TABLES - Only list table entities.
FILESETS - Only list fileset entities.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # List metadata entities response.
"entities": [ # Entities in the specified parent zone.
{ # Represents tables and fileset metadata contained within a zone.
"access": { # Describes the access mechanism of the data within its storage location. # Output only. Identifies the access mechanism to the entity. Not user settable.
"read": "A String", # Output only. Describes the read access mechanism of the data. Not user settable.
},
"asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset.
"catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry.
"compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with.
"bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
"hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
},
"createTime": "A String", # Output only. The time when the entity was created.
"dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id.
"dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv.
"description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters.
"displayName": "A String", # Optional. Display name must be shorter than or equal to 256 characters.
"etag": "A String", # Optional. The etag associated with the entity, which can be retrieved with a GetEntity request. Required for update and delete requests.
"format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery.
"compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed.
"csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data.
"delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified.
"headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0.
"quote": "A String", # Optional. The character used to quote column values. Accepts '"' (double quotation mark) or ''' (single quotation mark). Defaults to '"' (double quotation mark) if unspecified.
},
"format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type.
"iceberg": { # Describes Iceberg data format. # Optional. Additional information about iceberg tables.
"metadataLocation": "A String", # Optional. The location of where the iceberg metadata is present, must be within the table path
},
"json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified.
},
"mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: application/x-parquet application/x-avro application/x-orc application/x-tfrecord application/x-parquet+iceberg application/x-avro+iceberg application/x-orc+iceberg application/json application/{subtypes} text/csv text/ image/{image subtype} video/{video subtype} audio/{audio subtype}
},
"id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores, and consist of 256 or fewer characters.
"name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}.
"schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response.
"fields": [ # Optional. The sequence of fields describing data in table entities. Note: BigQuery SchemaFields are immutable.
{ # Represents a column field within a table schema.
"description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters.
"fields": [ # Optional. Any nested field for complex types.
# Object with schema name: GoogleCloudDataplexV1SchemaSchemaField
],
"mode": "A String", # Required. Additional field semantics.
"name": "A String", # Required. The name of the field. Must contain only letters, numbers and underscores, with a maximum length of 767 characters, and must begin with a letter or underscore.
"type": "A String", # Required. The type of field.
},
],
"partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data.
{ # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. Note: Partition fields are immutable.
"name": "A String", # Required. Partition field name must consist of letters, numbers, and underscores only, with a maximum of length of 256 characters, and must begin with a letter or underscore..
"type": "A String", # Required. Immutable. The type of field.
},
],
"partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity.
"userManaged": True or False, # Required. Set to true if user-managed or false if managed by Dataplex. The default is false (managed by Dataplex). Set to falseto enable Dataplex discovery to update the schema. including new data discovery, schema inference, and schema evolution. Users retain the ability to input and edit the schema. Dataplex treats schema input by the user as though produced by a previous Dataplex discovery operation, and it will evolve the schema and take action based on that treatment. Set to true to fully manage the entity schema. This setting guarantees that Dataplex will not change schema fields.
},
"system": "A String", # Required. Immutable. Identifies the storage system of the entity data.
"type": "A String", # Required. Immutable. The type of entity.
"uid": "A String", # Output only. System generated unique ID for the Entity. This ID will be different if the Entity is deleted and re-created with the same name.
"updateTime": "A String", # Output only. The time when the entity was last updated.
},
],
"nextPageToken": "A String", # Token to retrieve the next page of results, or empty if there are no remaining results in the list.
}
list_next()
Retrieves the next page of results.
Args:
previous_request: The request for the previous page. (required)
previous_response: The response from the request for the previous page. (required)
Returns:
A request object that you can call 'execute()' on to request the next
page. Returns None if there are no more items in the collection.
update(name, body=None, validateOnly=None, x__xgafv=None)
Update a metadata entity. Only supports full resource update.
Args:
name: string, Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}. (required)
body: object, The request body.
The object takes the form of:
{ # Represents tables and fileset metadata contained within a zone.
"access": { # Describes the access mechanism of the data within its storage location. # Output only. Identifies the access mechanism to the entity. Not user settable.
"read": "A String", # Output only. Describes the read access mechanism of the data. Not user settable.
},
"asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset.
"catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry.
"compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with.
"bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
"hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
},
"createTime": "A String", # Output only. The time when the entity was created.
"dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id.
"dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv.
"description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters.
"displayName": "A String", # Optional. Display name must be shorter than or equal to 256 characters.
"etag": "A String", # Optional. The etag associated with the entity, which can be retrieved with a GetEntity request. Required for update and delete requests.
"format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery.
"compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed.
"csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data.
"delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified.
"headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0.
"quote": "A String", # Optional. The character used to quote column values. Accepts '"' (double quotation mark) or ''' (single quotation mark). Defaults to '"' (double quotation mark) if unspecified.
},
"format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type.
"iceberg": { # Describes Iceberg data format. # Optional. Additional information about iceberg tables.
"metadataLocation": "A String", # Optional. The location of where the iceberg metadata is present, must be within the table path
},
"json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified.
},
"mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: application/x-parquet application/x-avro application/x-orc application/x-tfrecord application/x-parquet+iceberg application/x-avro+iceberg application/x-orc+iceberg application/json application/{subtypes} text/csv text/ image/{image subtype} video/{video subtype} audio/{audio subtype}
},
"id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores, and consist of 256 or fewer characters.
"name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}.
"schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response.
"fields": [ # Optional. The sequence of fields describing data in table entities. Note: BigQuery SchemaFields are immutable.
{ # Represents a column field within a table schema.
"description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters.
"fields": [ # Optional. Any nested field for complex types.
# Object with schema name: GoogleCloudDataplexV1SchemaSchemaField
],
"mode": "A String", # Required. Additional field semantics.
"name": "A String", # Required. The name of the field. Must contain only letters, numbers and underscores, with a maximum length of 767 characters, and must begin with a letter or underscore.
"type": "A String", # Required. The type of field.
},
],
"partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data.
{ # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. Note: Partition fields are immutable.
"name": "A String", # Required. Partition field name must consist of letters, numbers, and underscores only, with a maximum of length of 256 characters, and must begin with a letter or underscore..
"type": "A String", # Required. Immutable. The type of field.
},
],
"partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity.
"userManaged": True or False, # Required. Set to true if user-managed or false if managed by Dataplex. The default is false (managed by Dataplex). Set to falseto enable Dataplex discovery to update the schema. including new data discovery, schema inference, and schema evolution. Users retain the ability to input and edit the schema. Dataplex treats schema input by the user as though produced by a previous Dataplex discovery operation, and it will evolve the schema and take action based on that treatment. Set to true to fully manage the entity schema. This setting guarantees that Dataplex will not change schema fields.
},
"system": "A String", # Required. Immutable. Identifies the storage system of the entity data.
"type": "A String", # Required. Immutable. The type of entity.
"uid": "A String", # Output only. System generated unique ID for the Entity. This ID will be different if the Entity is deleted and re-created with the same name.
"updateTime": "A String", # Output only. The time when the entity was last updated.
}
validateOnly: boolean, Optional. Only validate the request, but do not perform mutations. The default is false.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # Represents tables and fileset metadata contained within a zone.
"access": { # Describes the access mechanism of the data within its storage location. # Output only. Identifies the access mechanism to the entity. Not user settable.
"read": "A String", # Output only. Describes the read access mechanism of the data. Not user settable.
},
"asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset.
"catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry.
"compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with.
"bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
"hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore.
"compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store.
"reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store.
},
},
"createTime": "A String", # Output only. The time when the entity was created.
"dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id.
"dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv.
"description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters.
"displayName": "A String", # Optional. Display name must be shorter than or equal to 256 characters.
"etag": "A String", # Optional. The etag associated with the entity, which can be retrieved with a GetEntity request. Required for update and delete requests.
"format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery.
"compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed.
"csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data.
"delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified.
"headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0.
"quote": "A String", # Optional. The character used to quote column values. Accepts '"' (double quotation mark) or ''' (single quotation mark). Defaults to '"' (double quotation mark) if unspecified.
},
"format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type.
"iceberg": { # Describes Iceberg data format. # Optional. Additional information about iceberg tables.
"metadataLocation": "A String", # Optional. The location of where the iceberg metadata is present, must be within the table path
},
"json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data.
"encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified.
},
"mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: application/x-parquet application/x-avro application/x-orc application/x-tfrecord application/x-parquet+iceberg application/x-avro+iceberg application/x-orc+iceberg application/json application/{subtypes} text/csv text/ image/{image subtype} video/{video subtype} audio/{audio subtype}
},
"id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores, and consist of 256 or fewer characters.
"name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}.
"schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response.
"fields": [ # Optional. The sequence of fields describing data in table entities. Note: BigQuery SchemaFields are immutable.
{ # Represents a column field within a table schema.
"description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters.
"fields": [ # Optional. Any nested field for complex types.
# Object with schema name: GoogleCloudDataplexV1SchemaSchemaField
],
"mode": "A String", # Required. Additional field semantics.
"name": "A String", # Required. The name of the field. Must contain only letters, numbers and underscores, with a maximum length of 767 characters, and must begin with a letter or underscore.
"type": "A String", # Required. The type of field.
},
],
"partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data.
{ # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. Note: Partition fields are immutable.
"name": "A String", # Required. Partition field name must consist of letters, numbers, and underscores only, with a maximum of length of 256 characters, and must begin with a letter or underscore..
"type": "A String", # Required. Immutable. The type of field.
},
],
"partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity.
"userManaged": True or False, # Required. Set to true if user-managed or false if managed by Dataplex. The default is false (managed by Dataplex). Set to falseto enable Dataplex discovery to update the schema. including new data discovery, schema inference, and schema evolution. Users retain the ability to input and edit the schema. Dataplex treats schema input by the user as though produced by a previous Dataplex discovery operation, and it will evolve the schema and take action based on that treatment. Set to true to fully manage the entity schema. This setting guarantees that Dataplex will not change schema fields.
},
"system": "A String", # Required. Immutable. Identifies the storage system of the entity data.
"type": "A String", # Required. Immutable. The type of entity.
"uid": "A String", # Output only. System generated unique ID for the Entity. This ID will be different if the Entity is deleted and re-created with the same name.
"updateTime": "A String", # Output only. The time when the entity was last updated.
}