Close httplib2 connections.
Delete a TableDataProfile. Will not prevent the profile from being regenerated if the table is still included in a discovery configuration.
Gets a table data profile.
list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
Lists table data profiles for an organization.
Retrieves the next page of results.
close()
Close httplib2 connections.
delete(name, x__xgafv=None)
Delete a TableDataProfile. Will not prevent the profile from being regenerated if the table is still included in a discovery configuration.
Args:
name: string, Required. Resource name of the table data profile. (required)
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); }
}
get(name, x__xgafv=None)
Gets a table data profile.
Args:
name: string, Required. Resource name, for example `organizations/12345/locations/us/tableDataProfiles/53234423`. (required)
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # The profile for a scanned table.
"configSnapshot": { # Snapshot of the configurations used to generate the profile. # The snapshot of the configurations used to generate the profile.
"dataProfileJob": { # Configuration for setting up a job to scan resources for profile generation. Only one data profile configuration may exist per organization, folder, or project. The generated data profiles are retained according to the [data retention policy] (https://cloud.google.com/sensitive-data-protection/docs/data-profiles#retention). # A copy of the configuration used to generate this profile. This is deprecated, and the DiscoveryConfig field is preferred moving forward. DataProfileJobConfig will still be written here for Discovery in BigQuery for backwards compatibility, but will not be updated with new fields, while DiscoveryConfig will.
"dataProfileActions": [ # Actions to execute at the completion of the job.
{ # A task to execute when a data profile has been generated.
"exportData": { # If set, the detailed data profiles will be persisted to the location of your choice whenever updated. # Export data profiles into a provided location.
"profileTable": { # Message defining the location of a BigQuery table. A table is uniquely identified by its project_id, dataset_id, and table_name. Within a query a table is often referenced with a string in the format of: `:.` or `..`. # Store all table and column profiles in an existing table or a new table in an existing dataset. Each re-generation will result in new rows in BigQuery. Data is inserted using [streaming insert](https://cloud.google.com/blog/products/bigquery/life-of-a-bigquery-streaming-insert) and so data may be in the buffer for a period of time after the profile has finished. The Pub/Sub notification is sent before the streaming buffer is guaranteed to be written, so data may not be instantly visible to queries by the time your topic receives the Pub/Sub notification.
"datasetId": "A String", # Dataset ID of the table.
"projectId": "A String", # The Google Cloud Platform project ID of the project containing the table. If omitted, project ID is inferred from the API call.
"tableId": "A String", # Name of the table.
},
},
"pubSubNotification": { # Send a Pub/Sub message into the given Pub/Sub topic to connect other systems to data profile generation. The message payload data will be the byte serialization of `DataProfilePubSubMessage`. # Publish a message into the Pub/Sub topic.
"detailOfMessage": "A String", # How much data to include in the Pub/Sub message. If the user wishes to limit the size of the message, they can use resource_name and fetch the profile fields they wish to. Per table profile (not per column).
"event": "A String", # The type of event that triggers a Pub/Sub. At most one `PubSubNotification` per EventType is permitted.
"pubsubCondition": { # A condition for determining whether a Pub/Sub should be triggered. # Conditions (e.g., data risk or sensitivity level) for triggering a Pub/Sub.
"expressions": { # An expression, consisting of an operator and conditions. # An expression.
"conditions": [ # Conditions to apply to the expression.
{ # A condition consisting of a value.
"minimumRiskScore": "A String", # The minimum data risk score that triggers the condition.
"minimumSensitivityScore": "A String", # The minimum sensitivity level that triggers the condition.
},
],
"logicalOperator": "A String", # The operator to apply to the collection of conditions.
},
},
"topic": "A String", # Cloud Pub/Sub topic to send notifications to. Format is projects/{project}/topics/{topic}.
},
},
],
"inspectTemplates": [ # Detection logic for profile generation. Not all template features are used by profiles. FindingLimits, include_quote and exclude_info_types have no impact on data profiling. Multiple templates may be provided if there is data in multiple regions. At most one template must be specified per-region (including "global"). Each region is scanned using the applicable template. If no region-specific template is specified, but a "global" template is specified, it will be copied to that region and used instead. If no global or region-specific template is provided for a region with data, that region's data will not be scanned. For more information, see https://cloud.google.com/sensitive-data-protection/docs/data-profiles#data-residency.
"A String",
],
"location": { # The data that will be profiled. # The data to scan.
"folderId": "A String", # The ID of the Folder within an organization to scan.
"organizationId": "A String", # The ID of an organization to scan.
},
"projectId": "A String", # The project that will run the scan. The DLP service account that exists within this project must have access to all resources that are profiled, and the Cloud DLP API must be enabled.
},
"discoveryConfig": { # Configuration for discovery to scan resources for profile generation. Only one discovery configuration may exist per organization, folder, or project. The generated data profiles are retained according to the [data retention policy] (https://cloud.google.com/sensitive-data-protection/docs/data-profiles#retention). # A copy of the configuration used to generate this profile.
"actions": [ # Actions to execute at the completion of scanning.
{ # A task to execute when a data profile has been generated.
"exportData": { # If set, the detailed data profiles will be persisted to the location of your choice whenever updated. # Export data profiles into a provided location.
"profileTable": { # Message defining the location of a BigQuery table. A table is uniquely identified by its project_id, dataset_id, and table_name. Within a query a table is often referenced with a string in the format of: `:.` or `..`. # Store all table and column profiles in an existing table or a new table in an existing dataset. Each re-generation will result in new rows in BigQuery. Data is inserted using [streaming insert](https://cloud.google.com/blog/products/bigquery/life-of-a-bigquery-streaming-insert) and so data may be in the buffer for a period of time after the profile has finished. The Pub/Sub notification is sent before the streaming buffer is guaranteed to be written, so data may not be instantly visible to queries by the time your topic receives the Pub/Sub notification.
"datasetId": "A String", # Dataset ID of the table.
"projectId": "A String", # The Google Cloud Platform project ID of the project containing the table. If omitted, project ID is inferred from the API call.
"tableId": "A String", # Name of the table.
},
},
"pubSubNotification": { # Send a Pub/Sub message into the given Pub/Sub topic to connect other systems to data profile generation. The message payload data will be the byte serialization of `DataProfilePubSubMessage`. # Publish a message into the Pub/Sub topic.
"detailOfMessage": "A String", # How much data to include in the Pub/Sub message. If the user wishes to limit the size of the message, they can use resource_name and fetch the profile fields they wish to. Per table profile (not per column).
"event": "A String", # The type of event that triggers a Pub/Sub. At most one `PubSubNotification` per EventType is permitted.
"pubsubCondition": { # A condition for determining whether a Pub/Sub should be triggered. # Conditions (e.g., data risk or sensitivity level) for triggering a Pub/Sub.
"expressions": { # An expression, consisting of an operator and conditions. # An expression.
"conditions": [ # Conditions to apply to the expression.
{ # A condition consisting of a value.
"minimumRiskScore": "A String", # The minimum data risk score that triggers the condition.
"minimumSensitivityScore": "A String", # The minimum sensitivity level that triggers the condition.
},
],
"logicalOperator": "A String", # The operator to apply to the collection of conditions.
},
},
"topic": "A String", # Cloud Pub/Sub topic to send notifications to. Format is projects/{project}/topics/{topic}.
},
},
],
"createTime": "A String", # Output only. The creation timestamp of a DiscoveryConfig.
"displayName": "A String", # Display name (max 100 chars)
"errors": [ # Output only. A stream of errors encountered when the config was activated. Repeated errors may result in the config automatically being paused. Output only field. Will return the last 100 errors. Whenever the config is modified this list will be cleared.
{ # Details information about an error encountered during job execution or the results of an unsuccessful activation of the JobTrigger.
"details": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Detailed error codes and messages.
"code": 42, # The status code, which should be an enum value of google.rpc.Code.
"details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
{
"a_key": "", # Properties of the object. Contains field @type with type URL.
},
],
"message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
},
"timestamps": [ # The times the error occurred. List includes the oldest timestamp and the last 9 timestamps.
"A String",
],
},
],
"inspectTemplates": [ # Detection logic for profile generation. Not all template features are used by Discovery. FindingLimits, include_quote and exclude_info_types have no impact on Discovery. Multiple templates may be provided if there is data in multiple regions. At most one template must be specified per-region (including "global"). Each region is scanned using the applicable template. If no region-specific template is specified, but a "global" template is specified, it will be copied to that region and used instead. If no global or region-specific template is provided for a region with data, that region's data will not be scanned. For more information, see https://cloud.google.com/sensitive-data-protection/docs/data-profiles#data-residency.
"A String",
],
"lastRunTime": "A String", # Output only. The timestamp of the last time this config was executed.
"name": "A String", # Unique resource name for the DiscoveryConfig, assigned by the service when the DiscoveryConfig is created, for example `projects/dlp-test-project/locations/global/discoveryConfigs/53234423`.
"orgConfig": { # Project and scan location information. Only set when the parent is an org. # Only set when the parent is an org.
"location": { # The location to begin a discovery scan. Denotes an organization ID or folder ID within an organization. # The data to scan: folder, org, or project
"folderId": "A String", # The ID of the Folder within an organization to scan.
"organizationId": "A String", # The ID of an organization to scan.
},
"projectId": "A String", # The project that will run the scan. The DLP service account that exists within this project must have access to all resources that are profiled, and the Cloud DLP API must be enabled.
},
"status": "A String", # Required. A status for this configuration.
"targets": [ # Target to match against for determining what to scan and how frequently.
{ # Target used to match against for Discovery.
"bigQueryTarget": { # Target used to match against for discovery with BigQuery tables # BigQuery target for Discovery. The first target to match a table will be the one applied.
"cadence": { # What must take place for a profile to be updated and how frequently it should occur. New tables are scanned as quickly as possible depending on system capacity. # How often and when to update profiles. New tables that match both the filter and conditions are scanned as quickly as possible depending on system capacity.
"schemaModifiedCadence": { # The cadence at which to update data profiles when a schema is modified. # Governs when to update data profiles when a schema is modified.
"frequency": "A String", # How frequently profiles may be updated when schemas are modified. Defaults to monthly.
"types": [ # The type of events to consider when deciding if the table's schema has been modified and should have the profile updated. Defaults to NEW_COLUMNS.
"A String",
],
},
"tableModifiedCadence": { # The cadence at which to update data profiles when a table is modified. # Governs when to update data profiles when a table is modified.
"frequency": "A String", # How frequently data profiles can be updated when tables are modified. Defaults to never.
"types": [ # The type of events to consider when deciding if the table has been modified and should have the profile updated. Defaults to MODIFIED_TIMESTAMP.
"A String",
],
},
},
"conditions": { # Requirements that must be true before a table is scanned in discovery for the first time. There is an AND relationship between the top-level attributes. Additionally, minimum conditions with an OR relationship that must be met before Cloud DLP scans a table can be set (like a minimum row count or a minimum table age). # In addition to matching the filter, these conditions must be true before a profile is generated.
"createdAfter": "A String", # BigQuery table must have been created after this date. Used to avoid backfilling.
"orConditions": { # There is an OR relationship between these attributes. They are used to determine if a table should be scanned or not in Discovery. # At least one of the conditions must be true for a table to be scanned.
"minAge": "A String", # Minimum age a table must have before Cloud DLP can profile it. Value must be 1 hour or greater.
"minRowCount": 42, # Minimum number of rows that should be present before Cloud DLP profiles a table
},
"typeCollection": "A String", # Restrict discovery to categories of table types.
"types": { # The types of BigQuery tables supported by Cloud DLP. # Restrict discovery to specific table types.
"types": [ # A set of BigQuery table types.
"A String",
],
},
},
"disabled": { # Do not profile the tables. # Tables that match this filter will not have profiles created.
},
"filter": { # Determines what tables will have profiles generated within an organization or project. Includes the ability to filter by regular expression patterns on project ID, dataset ID, and table ID. # Required. The tables the discovery cadence applies to. The first target with a matching filter will be the one to apply to a table.
"otherTables": { # Catch-all for all other tables not specified by other filters. Should always be last, except for single-table configurations, which will only have a TableReference target. # Catch-all. This should always be the last filter in the list because anything above it will apply first. Should only appear once in a configuration. If none is specified, a default one will be added automatically.
},
"tables": { # Specifies a collection of BigQuery tables. Used for Discovery. # A specific set of tables for this filter to apply to. A table collection must be specified in only one filter per config. If a table id or dataset is empty, Cloud DLP assumes all tables in that collection must be profiled. Must specify a project ID.
"includeRegexes": { # A collection of regular expressions to determine what tables to match against. # A collection of regular expressions to match a BigQuery table against.
"patterns": [ # A single BigQuery regular expression pattern to match against one or more tables, datasets, or projects that contain BigQuery tables.
{ # A pattern to match against one or more tables, datasets, or projects that contain BigQuery tables. At least one pattern must be specified. Regular expressions use RE2 [syntax](https://github.com/google/re2/wiki/Syntax); a guide can be found under the google/re2 repository on GitHub.
"datasetIdRegex": "A String", # If unset, this property matches all datasets.
"projectIdRegex": "A String", # For organizations, if unset, will match all projects. Has no effect for data profile configurations created within a project.
"tableIdRegex": "A String", # If unset, this property matches all tables.
},
],
},
},
},
},
"cloudSqlTarget": { # Target used to match against for discovery with Cloud SQL tables. # Cloud SQL target for Discovery. The first target to match a table will be the one applied.
"conditions": { # Requirements that must be true before a table is profiled for the first time. # In addition to matching the filter, these conditions must be true before a profile is generated.
"databaseEngines": [ # Optional. Database engines that should be profiled. Optional. Defaults to ALL_SUPPORTED_DATABASE_ENGINES if unspecified.
"A String",
],
"types": [ # Data profiles will only be generated for the database resource types specified in this field. If not specified, defaults to [DATABASE_RESOURCE_TYPE_ALL_SUPPORTED_TYPES].
"A String",
],
},
"disabled": { # Do not profile the tables. # Disable profiling for database resources that match this filter.
},
"filter": { # Determines what tables will have profiles generated within an organization or project. Includes the ability to filter by regular expression patterns on project ID, location, instance, database, and database resource name. # Required. The tables the discovery cadence applies to. The first target with a matching filter will be the one to apply to a table.
"collection": { # Match database resources using regex filters. Examples of database resources are tables, views, and stored procedures. # A specific set of database resources for this filter to apply to.
"includeRegexes": { # A collection of regular expressions to determine what database resources to match against. # A collection of regular expressions to match a database resource against.
"patterns": [ # A group of regular expression patterns to match against one or more database resources. Maximum of 100 entries. The sum of all regular expression's length can't exceed 10 KiB.
{ # A pattern to match against one or more database resources. At least one pattern must be specified. Regular expressions use RE2 [syntax](https://github.com/google/re2/wiki/Syntax); a guide can be found under the google/re2 repository on GitHub.
"databaseRegex": "A String", # Regex to test the database name against. If empty, all databases match.
"databaseResourceNameRegex": "A String", # Regex to test the database resource's name against. An example of a database resource name is a table's name. Other database resource names like view names could be included in the future. If empty, all database resources match.
"instanceRegex": "A String", # Regex to test the instance name against. If empty, all instances match.
"projectIdRegex": "A String", # For organizations, if unset, will match all projects. Has no effect for Data Profile configurations created within a project.
},
],
},
},
"databaseResourceReference": { # Identifies a single database resource, like a table within a database. # The database resource to scan. Targets including this can only include one target (the target with this database resource reference).
"instance": "A String", # Required. The instance where this resource is located. For example: Cloud SQL's instance id.
"projectId": "A String", # Required. If within a project-level config, then this must match the config's project id.
},
"others": { # Match database resources not covered by any other filter. # Catch-all. This should always be the last target in the list because anything above it will apply first. Should only appear once in a configuration. If none is specified, a default one will be added automatically.
},
},
"generationCadence": { # How often existing tables should have their profiles refreshed. New tables are scanned as quickly as possible depending on system capacity. # How often and when to update profiles. New tables that match both the filter and conditions are scanned as quickly as possible depending on system capacity.
"refreshFrequency": "A String", # Data changes (non-schema changes) in Cloud SQL tables can't trigger reprofiling. If you set this field, profiles are refreshed at this frequency regardless of whether the underlying tables have changes. Defaults to never.
"schemaModifiedCadence": { # How frequency to modify the profile when the table's schema is modified. # When to reprofile if the schema has changed.
"frequency": "A String", # Frequency to regenerate data profiles when the schema is modified. Defaults to monthly.
"types": [ # The types of schema modifications to consider. Defaults to NEW_COLUMNS.
"A String",
],
},
},
},
},
],
"updateTime": "A String", # Output only. The last update timestamp of a DiscoveryConfig.
},
"inspectConfig": { # Configuration description of the scanning process. When used with redactContent only info_types and min_likelihood are currently used. # A copy of the inspection config used to generate this profile. This is a copy of the inspect_template specified in `DataProfileJobConfig`.
"contentOptions": [ # Deprecated and unused.
"A String",
],
"customInfoTypes": [ # CustomInfoTypes provided by the user. See https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes to learn more.
{ # Custom information type provided by the user. Used to find domain-specific sensitive information configurable to the data in question.
"detectionRules": [ # Set of detection rules to apply to all findings of this CustomInfoType. Rules are applied in order that they are specified. Not supported for the `surrogate_type` CustomInfoType.
{ # Deprecated; use `InspectionRuleSet` instead. Rule for modifying a `CustomInfoType` to alter behavior under certain circumstances, depending on the specific details of the rule. Not supported for the `surrogate_type` custom infoType.
"hotwordRule": { # The rule that adjusts the likelihood of findings within a certain proximity of hotwords. # Hotword-based detection rule.
"hotwordRegex": { # Message defining a custom regular expression. # Regular expression pattern defining what qualifies as a hotword.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
"likelihoodAdjustment": { # Message for specifying an adjustment to the likelihood of a finding as part of a detection rule. # Likelihood adjustment to apply to all matching findings.
"fixedLikelihood": "A String", # Set the likelihood of a finding to a fixed value.
"relativeLikelihood": 42, # Increase or decrease the likelihood by the specified number of levels. For example, if a finding would be `POSSIBLE` without the detection rule and `relative_likelihood` is 1, then it is upgraded to `LIKELY`, while a value of -1 would downgrade it to `UNLIKELY`. Likelihood may never drop below `VERY_UNLIKELY` or exceed `VERY_LIKELY`, so applying an adjustment of 1 followed by an adjustment of -1 when base likelihood is `VERY_LIKELY` will result in a final likelihood of `LIKELY`.
},
"proximity": { # Message for specifying a window around a finding to apply a detection rule. # Range of characters within which the entire hotword must reside. The total length of the window cannot exceed 1000 characters. The finding itself will be included in the window, so that hotwords can be used to match substrings of the finding itself. Suppose you want Cloud DLP to promote the likelihood of the phone number regex "\(\d{3}\) \d{3}-\d{4}" if the area code is known to be the area code of a company's office. In this case, use the hotword regex "\(xxx\)", where "xxx" is the area code in question. For tabular data, if you want to modify the likelihood of an entire column of findngs, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
"windowAfter": 42, # Number of characters after the finding to consider.
"windowBefore": 42, # Number of characters before the finding to consider. For tabular data, if you want to modify the likelihood of an entire column of findngs, set this to 1. For more information, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
},
},
},
],
"dictionary": { # Custom information type based on a dictionary of words or phrases. This can be used to match sensitive information specific to the data, such as a list of employee IDs or job titles. Dictionary words are case-insensitive and all characters other than letters and digits in the unicode [Basic Multilingual Plane](https://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane) will be replaced with whitespace when scanning for matches, so the dictionary phrase "Sam Johnson" will match all three phrases "sam johnson", "Sam, Johnson", and "Sam (Johnson)". Additionally, the characters surrounding any match must be of a different type than the adjacent characters within the word, so letters must be next to non-letters and digits next to non-digits. For example, the dictionary word "jen" will match the first three letters of the text "jen123" but will return no matches for "jennifer". Dictionary words containing a large number of characters that are not letters or digits may result in unexpected findings because such characters are treated as whitespace. The [limits](https://cloud.google.com/sensitive-data-protection/limits) page contains details about the size limits of dictionaries. For dictionaries that do not fit within these constraints, consider using `LargeCustomDictionaryConfig` in the `StoredInfoType` API. # A list of phrases to detect as a CustomInfoType.
"cloudStoragePath": { # Message representing a single file or path in Cloud Storage. # Newline-delimited file of words in Cloud Storage. Only a single file is accepted.
"path": "A String", # A URL representing a file or path (no wildcards) in Cloud Storage. Example: `gs://[BUCKET_NAME]/dictionary.txt`
},
"wordList": { # Message defining a list of words or phrases to search for in the data. # List of words or phrases to search for.
"words": [ # Words or phrases defining the dictionary. The dictionary must contain at least one phrase and every phrase must contain at least 2 characters that are letters or digits. [required]
"A String",
],
},
},
"exclusionType": "A String", # If set to EXCLUSION_TYPE_EXCLUDE this infoType will not cause a finding to be returned. It still can be used for rules matching.
"infoType": { # Type of information detected by the API. # CustomInfoType can either be a new infoType, or an extension of built-in infoType, when the name matches one of existing infoTypes and that infoType is specified in `InspectContent.info_types` field. Specifying the latter adds findings to the one detected by the system. If built-in info type is not specified in `InspectContent.info_types` list then the name is treated as a custom info type.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
"likelihood": "A String", # Likelihood to return for this CustomInfoType. This base value can be altered by a detection rule if the finding meets the criteria specified by the rule. Defaults to `VERY_LIKELY` if not specified.
"regex": { # Message defining a custom regular expression. # Regular expression based CustomInfoType.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Sensitivity for this CustomInfoType. If this CustomInfoType extends an existing InfoType, the sensitivity here will take precedence over that of the original InfoType. If unset for a CustomInfoType, it will default to HIGH. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"storedType": { # A reference to a StoredInfoType to use with scanning. # Load an existing `StoredInfoType` resource for use in `InspectDataSource`. Not currently supported in `InspectContent`.
"createTime": "A String", # Timestamp indicating when the version of the `StoredInfoType` used for inspection was created. Output-only field, populated by the system.
"name": "A String", # Resource name of the requested `StoredInfoType`, for example `organizations/433245324/storedInfoTypes/432452342` or `projects/project-id/storedInfoTypes/432452342`.
},
"surrogateType": { # Message for detecting output from deidentification transformations such as [`CryptoReplaceFfxFpeConfig`](https://cloud.google.com/sensitive-data-protection/docs/reference/rest/v2/organizations.deidentifyTemplates#cryptoreplaceffxfpeconfig). These types of transformations are those that perform pseudonymization, thereby producing a "surrogate" as output. This should be used in conjunction with a field on the transformation such as `surrogate_info_type`. This CustomInfoType does not support the use of `detection_rules`. # Message for detecting output from deidentification transformations that support reversing.
},
},
],
"excludeInfoTypes": True or False, # When true, excludes type information of the findings. This is not used for data profiling.
"includeQuote": True or False, # When true, a contextual quote from the data that triggered a finding is included in the response; see Finding.quote. This is not used for data profiling.
"infoTypes": [ # Restricts what info_types to look for. The values must correspond to InfoType values returned by ListInfoTypes or listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference. When no InfoTypes or CustomInfoTypes are specified in a request, the system may automatically choose a default list of detectors to run, which may change over time. If you need precise control and predictability as to what detectors are run you should specify specific InfoTypes listed in the reference, otherwise a default list will be used, which may change over time.
{ # Type of information detected by the API.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
],
"limits": { # Configuration to control the number of findings returned for inspection. This is not used for de-identification or data profiling. When redacting sensitive data from images, finding limits don't apply. They can cause unexpected or inconsistent results, where only some data is redacted. Don't include finding limits in RedactImage requests. Otherwise, Cloud DLP returns an error. # Configuration to control the number of findings returned. This is not used for data profiling. When redacting sensitive data from images, finding limits don't apply. They can cause unexpected or inconsistent results, where only some data is redacted. Don't include finding limits in RedactImage requests. Otherwise, Cloud DLP returns an error. When set within an InspectJobConfig, the specified maximum values aren't hard limits. If an inspection job reaches these limits, the job ends gradually, not abruptly. Therefore, the actual number of findings that Cloud DLP returns can be multiple times higher than these maximum values.
"maxFindingsPerInfoType": [ # Configuration of findings limit given for specified infoTypes.
{ # Max findings configuration per infoType, per content item or long running DlpJob.
"infoType": { # Type of information detected by the API. # Type of information the findings limit applies to. Only one limit per info_type should be provided. If InfoTypeLimit does not have an info_type, the DLP API applies the limit against all info_types that are found but not specified in another InfoTypeLimit.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
"maxFindings": 42, # Max findings limit for the given infoType.
},
],
"maxFindingsPerItem": 42, # Max number of findings that are returned for each item scanned. When set within an InspectContentRequest, this field is ignored. This value isn't a hard limit. If the number of findings for an item reaches this limit, the inspection of that item ends gradually, not abruptly. Therefore, the actual number of findings that Cloud DLP returns for the item can be multiple times higher than this value.
"maxFindingsPerRequest": 42, # Max number of findings that are returned per request or job. If you set this field in an InspectContentRequest, the resulting maximum value is the value that you set or 3,000, whichever is lower. This value isn't a hard limit. If an inspection reaches this limit, the inspection ends gradually, not abruptly. Therefore, the actual number of findings that Cloud DLP returns can be multiple times higher than this value.
},
"minLikelihood": "A String", # Only returns findings equal to or above this threshold. The default is POSSIBLE. In general, the highest likelihood setting yields the fewest findings in results and the lowest chance of a false positive. For more information, see [Match likelihood](https://cloud.google.com/sensitive-data-protection/docs/likelihood).
"minLikelihoodPerInfoType": [ # Minimum likelihood per infotype. For each infotype, a user can specify a minimum likelihood. The system only returns a finding if its likelihood is above this threshold. If this field is not set, the system uses the InspectConfig min_likelihood.
{ # Configuration for setting a minimum likelihood per infotype. Used to customize the minimum likelihood level for specific infotypes in the request. For example, use this if you want to lower the precision for PERSON_NAME without lowering the precision for the other infotypes in the request.
"infoType": { # Type of information detected by the API. # Type of information the likelihood threshold applies to. Only one likelihood per info_type should be provided. If InfoTypeLikelihood does not have an info_type, the configuration fails.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
"minLikelihood": "A String", # Only returns findings equal to or above this threshold. This field is required or else the configuration fails.
},
],
"ruleSet": [ # Set of rules to apply to the findings for this InspectConfig. Exclusion rules, contained in the set are executed in the end, other rules are executed in the order they are specified for each info type.
{ # Rule set for modifying a set of infoTypes to alter behavior under certain circumstances, depending on the specific details of the rules within the set.
"infoTypes": [ # List of infoTypes this rule set is applied to.
{ # Type of information detected by the API.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
],
"rules": [ # Set of rules to be applied to infoTypes. The rules are applied in order.
{ # A single inspection rule to be applied to infoTypes, specified in `InspectionRuleSet`.
"exclusionRule": { # The rule that specifies conditions when findings of infoTypes specified in `InspectionRuleSet` are removed from results. # Exclusion rule.
"dictionary": { # Custom information type based on a dictionary of words or phrases. This can be used to match sensitive information specific to the data, such as a list of employee IDs or job titles. Dictionary words are case-insensitive and all characters other than letters and digits in the unicode [Basic Multilingual Plane](https://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane) will be replaced with whitespace when scanning for matches, so the dictionary phrase "Sam Johnson" will match all three phrases "sam johnson", "Sam, Johnson", and "Sam (Johnson)". Additionally, the characters surrounding any match must be of a different type than the adjacent characters within the word, so letters must be next to non-letters and digits next to non-digits. For example, the dictionary word "jen" will match the first three letters of the text "jen123" but will return no matches for "jennifer". Dictionary words containing a large number of characters that are not letters or digits may result in unexpected findings because such characters are treated as whitespace. The [limits](https://cloud.google.com/sensitive-data-protection/limits) page contains details about the size limits of dictionaries. For dictionaries that do not fit within these constraints, consider using `LargeCustomDictionaryConfig` in the `StoredInfoType` API. # Dictionary which defines the rule.
"cloudStoragePath": { # Message representing a single file or path in Cloud Storage. # Newline-delimited file of words in Cloud Storage. Only a single file is accepted.
"path": "A String", # A URL representing a file or path (no wildcards) in Cloud Storage. Example: `gs://[BUCKET_NAME]/dictionary.txt`
},
"wordList": { # Message defining a list of words or phrases to search for in the data. # List of words or phrases to search for.
"words": [ # Words or phrases defining the dictionary. The dictionary must contain at least one phrase and every phrase must contain at least 2 characters that are letters or digits. [required]
"A String",
],
},
},
"excludeByHotword": { # The rule to exclude findings based on a hotword. For record inspection of tables, column names are considered hotwords. An example of this is to exclude a finding if it belongs to a BigQuery column that matches a specific pattern. # Drop if the hotword rule is contained in the proximate context. For tabular data, the context includes the column name.
"hotwordRegex": { # Message defining a custom regular expression. # Regular expression pattern defining what qualifies as a hotword.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
"proximity": { # Message for specifying a window around a finding to apply a detection rule. # Range of characters within which the entire hotword must reside. The total length of the window cannot exceed 1000 characters. The windowBefore property in proximity should be set to 1 if the hotword needs to be included in a column header.
"windowAfter": 42, # Number of characters after the finding to consider.
"windowBefore": 42, # Number of characters before the finding to consider. For tabular data, if you want to modify the likelihood of an entire column of findngs, set this to 1. For more information, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
},
},
"excludeInfoTypes": { # List of excluded infoTypes. # Set of infoTypes for which findings would affect this rule.
"infoTypes": [ # InfoType list in ExclusionRule rule drops a finding when it overlaps or contained within with a finding of an infoType from this list. For example, for `InspectionRuleSet.info_types` containing "PHONE_NUMBER"` and `exclusion_rule` containing `exclude_info_types.info_types` with "EMAIL_ADDRESS" the phone number findings are dropped if they overlap with EMAIL_ADDRESS finding. That leads to "555-222-2222@example.org" to generate only a single finding, namely email address.
{ # Type of information detected by the API.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
],
},
"matchingType": "A String", # How the rule is applied, see MatchingType documentation for details.
"regex": { # Message defining a custom regular expression. # Regular expression which defines the rule.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
},
"hotwordRule": { # The rule that adjusts the likelihood of findings within a certain proximity of hotwords. # Hotword-based detection rule.
"hotwordRegex": { # Message defining a custom regular expression. # Regular expression pattern defining what qualifies as a hotword.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
"likelihoodAdjustment": { # Message for specifying an adjustment to the likelihood of a finding as part of a detection rule. # Likelihood adjustment to apply to all matching findings.
"fixedLikelihood": "A String", # Set the likelihood of a finding to a fixed value.
"relativeLikelihood": 42, # Increase or decrease the likelihood by the specified number of levels. For example, if a finding would be `POSSIBLE` without the detection rule and `relative_likelihood` is 1, then it is upgraded to `LIKELY`, while a value of -1 would downgrade it to `UNLIKELY`. Likelihood may never drop below `VERY_UNLIKELY` or exceed `VERY_LIKELY`, so applying an adjustment of 1 followed by an adjustment of -1 when base likelihood is `VERY_LIKELY` will result in a final likelihood of `LIKELY`.
},
"proximity": { # Message for specifying a window around a finding to apply a detection rule. # Range of characters within which the entire hotword must reside. The total length of the window cannot exceed 1000 characters. The finding itself will be included in the window, so that hotwords can be used to match substrings of the finding itself. Suppose you want Cloud DLP to promote the likelihood of the phone number regex "\(\d{3}\) \d{3}-\d{4}" if the area code is known to be the area code of a company's office. In this case, use the hotword regex "\(xxx\)", where "xxx" is the area code in question. For tabular data, if you want to modify the likelihood of an entire column of findngs, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
"windowAfter": 42, # Number of characters after the finding to consider.
"windowBefore": 42, # Number of characters before the finding to consider. For tabular data, if you want to modify the likelihood of an entire column of findngs, set this to 1. For more information, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
},
},
},
],
},
],
},
"inspectTemplateModifiedTime": "A String", # Timestamp when the template was modified
"inspectTemplateName": "A String", # Name of the inspection template used to generate this profile
},
"createTime": "A String", # The time at which the table was created.
"dataRiskLevel": { # Score is a summary of all elements in the data profile. A higher number means more risk. # The data risk level of this table.
"score": "A String", # The score applied to the resource.
},
"dataSourceType": { # Message used to identify the type of resource being profiled. # The resource type that was profiled.
"dataSource": "A String", # Output only. An identifying string to the type of resource being profiled. Current values: google/bigquery/table, google/project
},
"datasetId": "A String", # If the resource is BigQuery, the dataset ID.
"datasetLocation": "A String", # If supported, the location where the dataset's data is stored. See https://cloud.google.com/bigquery/docs/locations for supported locations.
"datasetProjectId": "A String", # The Google Cloud project ID that owns the resource.
"encryptionStatus": "A String", # How the table is encrypted.
"expirationTime": "A String", # Optional. The time when this table expires.
"failedColumnCount": "A String", # The number of columns skipped in the table because of an error.
"fullResource": "A String", # The resource name of the resource profiled. https://cloud.google.com/apis/design/resource_names#full_resource_name
"lastModifiedTime": "A String", # The time when this table was last modified
"name": "A String", # The name of the profile.
"otherInfoTypes": [ # Other infoTypes found in this table's data.
{ # Infotype details for other infoTypes found within a column.
"estimatedPrevalence": 42, # Approximate percentage of non-null rows that contained data detected by this infotype.
"excludedFromAnalysis": True or False, # Whether this infoType was excluded from sensitivity and risk analysis due to factors such as low prevalence (subject to change).
"infoType": { # Type of information detected by the API. # The other infoType.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
},
],
"predictedInfoTypes": [ # The infoTypes predicted from this table's data.
{ # The infoType details for this column.
"estimatedPrevalence": 42, # Not populated for predicted infotypes.
"infoType": { # Type of information detected by the API. # The infoType.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
},
],
"profileLastGenerated": "A String", # The last time the profile was generated.
"profileStatus": { # Success or errors for the profile generation. # Success or error status from the most recent profile generation attempt. May be empty if the profile is still being generated.
"status": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Profiling status code and optional message. The `status.code` value is 0 (default value) for OK.
"code": 42, # The status code, which should be an enum value of google.rpc.Code.
"details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
{
"a_key": "", # Properties of the object. Contains field @type with type URL.
},
],
"message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
},
"timestamp": "A String", # Time when the profile generation status was updated
},
"projectDataProfile": "A String", # The resource name to the project data profile for this table.
"resourceLabels": { # The labels applied to the resource at the time the profile was generated.
"a_key": "A String",
},
"resourceVisibility": "A String", # How broadly a resource has been shared.
"rowCount": "A String", # Number of rows in the table when the profile was generated. This will not be populated for BigLake tables.
"scannedColumnCount": "A String", # The number of columns profiled in the table.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # The sensitivity score of this table.
"score": "A String", # The sensitivity score applied to the resource.
},
"state": "A String", # State of a profile.
"tableId": "A String", # If the resource is BigQuery, the BigQuery table ID.
"tableSizeBytes": "A String", # The size of the table when the profile was generated.
}
list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)
Lists table data profiles for an organization.
Args:
parent: string, Required. Resource name of the organization or project, for example `organizations/433245324/locations/europe` or `projects/project-id/locations/asia`. (required)
filter: string, Allows filtering. Supported syntax: * Filter expressions are made up of one or more restrictions. * Restrictions can be combined by `AND` or `OR` logical operators. A sequence of restrictions implicitly uses `AND`. * A restriction has the form of `{field} {operator} {value}`. * Supported fields/values: - `project_id` - The Google Cloud project ID. - `dataset_id` - The BigQuery dataset ID. - `table_id` - The ID of the BigQuery table. - `sensitivity_level` - HIGH|MODERATE|LOW - `data_risk_level` - HIGH|MODERATE|LOW - `resource_visibility`: PUBLIC|RESTRICTED - `status_code` - an RPC status code as defined in https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto * The operator must be `=` or `!=`. Examples: * `project_id = 12345 AND status_code = 1` * `project_id = 12345 AND sensitivity_level = HIGH` * `project_id = 12345 AND resource_visibility = PUBLIC` The length of this field should be no more than 500 characters.
orderBy: string, Comma separated list of fields to order by, followed by `asc` or `desc` postfix. This list is case insensitive. The default sorting order is ascending. Redundant space characters are insignificant. Only one order field at a time is allowed. Examples: * `project_id asc` * `table_id` * `sensitivity_level desc` Supported fields are: - `project_id`: The Google Cloud project ID. - `dataset_id`: The ID of a BigQuery dataset. - `table_id`: The ID of a BigQuery table. - `sensitivity_level`: How sensitive the data in a table is, at most. - `data_risk_level`: How much risk is associated with this data. - `profile_last_generated`: When the profile was last updated in epoch seconds. - `last_modified`: The last time the resource was modified. - `resource_visibility`: Visibility restriction for this resource. - `row_count`: Number of rows in this resource.
pageSize: integer, Size of the page. This value can be limited by the server. If zero, server returns a page of max size 100.
pageToken: string, Page token to continue retrieval.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # List of profiles generated for a given organization or project.
"nextPageToken": "A String", # The next page token.
"tableDataProfiles": [ # List of data profiles.
{ # The profile for a scanned table.
"configSnapshot": { # Snapshot of the configurations used to generate the profile. # The snapshot of the configurations used to generate the profile.
"dataProfileJob": { # Configuration for setting up a job to scan resources for profile generation. Only one data profile configuration may exist per organization, folder, or project. The generated data profiles are retained according to the [data retention policy] (https://cloud.google.com/sensitive-data-protection/docs/data-profiles#retention). # A copy of the configuration used to generate this profile. This is deprecated, and the DiscoveryConfig field is preferred moving forward. DataProfileJobConfig will still be written here for Discovery in BigQuery for backwards compatibility, but will not be updated with new fields, while DiscoveryConfig will.
"dataProfileActions": [ # Actions to execute at the completion of the job.
{ # A task to execute when a data profile has been generated.
"exportData": { # If set, the detailed data profiles will be persisted to the location of your choice whenever updated. # Export data profiles into a provided location.
"profileTable": { # Message defining the location of a BigQuery table. A table is uniquely identified by its project_id, dataset_id, and table_name. Within a query a table is often referenced with a string in the format of: `:.` or `..`. # Store all table and column profiles in an existing table or a new table in an existing dataset. Each re-generation will result in new rows in BigQuery. Data is inserted using [streaming insert](https://cloud.google.com/blog/products/bigquery/life-of-a-bigquery-streaming-insert) and so data may be in the buffer for a period of time after the profile has finished. The Pub/Sub notification is sent before the streaming buffer is guaranteed to be written, so data may not be instantly visible to queries by the time your topic receives the Pub/Sub notification.
"datasetId": "A String", # Dataset ID of the table.
"projectId": "A String", # The Google Cloud Platform project ID of the project containing the table. If omitted, project ID is inferred from the API call.
"tableId": "A String", # Name of the table.
},
},
"pubSubNotification": { # Send a Pub/Sub message into the given Pub/Sub topic to connect other systems to data profile generation. The message payload data will be the byte serialization of `DataProfilePubSubMessage`. # Publish a message into the Pub/Sub topic.
"detailOfMessage": "A String", # How much data to include in the Pub/Sub message. If the user wishes to limit the size of the message, they can use resource_name and fetch the profile fields they wish to. Per table profile (not per column).
"event": "A String", # The type of event that triggers a Pub/Sub. At most one `PubSubNotification` per EventType is permitted.
"pubsubCondition": { # A condition for determining whether a Pub/Sub should be triggered. # Conditions (e.g., data risk or sensitivity level) for triggering a Pub/Sub.
"expressions": { # An expression, consisting of an operator and conditions. # An expression.
"conditions": [ # Conditions to apply to the expression.
{ # A condition consisting of a value.
"minimumRiskScore": "A String", # The minimum data risk score that triggers the condition.
"minimumSensitivityScore": "A String", # The minimum sensitivity level that triggers the condition.
},
],
"logicalOperator": "A String", # The operator to apply to the collection of conditions.
},
},
"topic": "A String", # Cloud Pub/Sub topic to send notifications to. Format is projects/{project}/topics/{topic}.
},
},
],
"inspectTemplates": [ # Detection logic for profile generation. Not all template features are used by profiles. FindingLimits, include_quote and exclude_info_types have no impact on data profiling. Multiple templates may be provided if there is data in multiple regions. At most one template must be specified per-region (including "global"). Each region is scanned using the applicable template. If no region-specific template is specified, but a "global" template is specified, it will be copied to that region and used instead. If no global or region-specific template is provided for a region with data, that region's data will not be scanned. For more information, see https://cloud.google.com/sensitive-data-protection/docs/data-profiles#data-residency.
"A String",
],
"location": { # The data that will be profiled. # The data to scan.
"folderId": "A String", # The ID of the Folder within an organization to scan.
"organizationId": "A String", # The ID of an organization to scan.
},
"projectId": "A String", # The project that will run the scan. The DLP service account that exists within this project must have access to all resources that are profiled, and the Cloud DLP API must be enabled.
},
"discoveryConfig": { # Configuration for discovery to scan resources for profile generation. Only one discovery configuration may exist per organization, folder, or project. The generated data profiles are retained according to the [data retention policy] (https://cloud.google.com/sensitive-data-protection/docs/data-profiles#retention). # A copy of the configuration used to generate this profile.
"actions": [ # Actions to execute at the completion of scanning.
{ # A task to execute when a data profile has been generated.
"exportData": { # If set, the detailed data profiles will be persisted to the location of your choice whenever updated. # Export data profiles into a provided location.
"profileTable": { # Message defining the location of a BigQuery table. A table is uniquely identified by its project_id, dataset_id, and table_name. Within a query a table is often referenced with a string in the format of: `:.` or `..`. # Store all table and column profiles in an existing table or a new table in an existing dataset. Each re-generation will result in new rows in BigQuery. Data is inserted using [streaming insert](https://cloud.google.com/blog/products/bigquery/life-of-a-bigquery-streaming-insert) and so data may be in the buffer for a period of time after the profile has finished. The Pub/Sub notification is sent before the streaming buffer is guaranteed to be written, so data may not be instantly visible to queries by the time your topic receives the Pub/Sub notification.
"datasetId": "A String", # Dataset ID of the table.
"projectId": "A String", # The Google Cloud Platform project ID of the project containing the table. If omitted, project ID is inferred from the API call.
"tableId": "A String", # Name of the table.
},
},
"pubSubNotification": { # Send a Pub/Sub message into the given Pub/Sub topic to connect other systems to data profile generation. The message payload data will be the byte serialization of `DataProfilePubSubMessage`. # Publish a message into the Pub/Sub topic.
"detailOfMessage": "A String", # How much data to include in the Pub/Sub message. If the user wishes to limit the size of the message, they can use resource_name and fetch the profile fields they wish to. Per table profile (not per column).
"event": "A String", # The type of event that triggers a Pub/Sub. At most one `PubSubNotification` per EventType is permitted.
"pubsubCondition": { # A condition for determining whether a Pub/Sub should be triggered. # Conditions (e.g., data risk or sensitivity level) for triggering a Pub/Sub.
"expressions": { # An expression, consisting of an operator and conditions. # An expression.
"conditions": [ # Conditions to apply to the expression.
{ # A condition consisting of a value.
"minimumRiskScore": "A String", # The minimum data risk score that triggers the condition.
"minimumSensitivityScore": "A String", # The minimum sensitivity level that triggers the condition.
},
],
"logicalOperator": "A String", # The operator to apply to the collection of conditions.
},
},
"topic": "A String", # Cloud Pub/Sub topic to send notifications to. Format is projects/{project}/topics/{topic}.
},
},
],
"createTime": "A String", # Output only. The creation timestamp of a DiscoveryConfig.
"displayName": "A String", # Display name (max 100 chars)
"errors": [ # Output only. A stream of errors encountered when the config was activated. Repeated errors may result in the config automatically being paused. Output only field. Will return the last 100 errors. Whenever the config is modified this list will be cleared.
{ # Details information about an error encountered during job execution or the results of an unsuccessful activation of the JobTrigger.
"details": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Detailed error codes and messages.
"code": 42, # The status code, which should be an enum value of google.rpc.Code.
"details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
{
"a_key": "", # Properties of the object. Contains field @type with type URL.
},
],
"message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
},
"timestamps": [ # The times the error occurred. List includes the oldest timestamp and the last 9 timestamps.
"A String",
],
},
],
"inspectTemplates": [ # Detection logic for profile generation. Not all template features are used by Discovery. FindingLimits, include_quote and exclude_info_types have no impact on Discovery. Multiple templates may be provided if there is data in multiple regions. At most one template must be specified per-region (including "global"). Each region is scanned using the applicable template. If no region-specific template is specified, but a "global" template is specified, it will be copied to that region and used instead. If no global or region-specific template is provided for a region with data, that region's data will not be scanned. For more information, see https://cloud.google.com/sensitive-data-protection/docs/data-profiles#data-residency.
"A String",
],
"lastRunTime": "A String", # Output only. The timestamp of the last time this config was executed.
"name": "A String", # Unique resource name for the DiscoveryConfig, assigned by the service when the DiscoveryConfig is created, for example `projects/dlp-test-project/locations/global/discoveryConfigs/53234423`.
"orgConfig": { # Project and scan location information. Only set when the parent is an org. # Only set when the parent is an org.
"location": { # The location to begin a discovery scan. Denotes an organization ID or folder ID within an organization. # The data to scan: folder, org, or project
"folderId": "A String", # The ID of the Folder within an organization to scan.
"organizationId": "A String", # The ID of an organization to scan.
},
"projectId": "A String", # The project that will run the scan. The DLP service account that exists within this project must have access to all resources that are profiled, and the Cloud DLP API must be enabled.
},
"status": "A String", # Required. A status for this configuration.
"targets": [ # Target to match against for determining what to scan and how frequently.
{ # Target used to match against for Discovery.
"bigQueryTarget": { # Target used to match against for discovery with BigQuery tables # BigQuery target for Discovery. The first target to match a table will be the one applied.
"cadence": { # What must take place for a profile to be updated and how frequently it should occur. New tables are scanned as quickly as possible depending on system capacity. # How often and when to update profiles. New tables that match both the filter and conditions are scanned as quickly as possible depending on system capacity.
"schemaModifiedCadence": { # The cadence at which to update data profiles when a schema is modified. # Governs when to update data profiles when a schema is modified.
"frequency": "A String", # How frequently profiles may be updated when schemas are modified. Defaults to monthly.
"types": [ # The type of events to consider when deciding if the table's schema has been modified and should have the profile updated. Defaults to NEW_COLUMNS.
"A String",
],
},
"tableModifiedCadence": { # The cadence at which to update data profiles when a table is modified. # Governs when to update data profiles when a table is modified.
"frequency": "A String", # How frequently data profiles can be updated when tables are modified. Defaults to never.
"types": [ # The type of events to consider when deciding if the table has been modified and should have the profile updated. Defaults to MODIFIED_TIMESTAMP.
"A String",
],
},
},
"conditions": { # Requirements that must be true before a table is scanned in discovery for the first time. There is an AND relationship between the top-level attributes. Additionally, minimum conditions with an OR relationship that must be met before Cloud DLP scans a table can be set (like a minimum row count or a minimum table age). # In addition to matching the filter, these conditions must be true before a profile is generated.
"createdAfter": "A String", # BigQuery table must have been created after this date. Used to avoid backfilling.
"orConditions": { # There is an OR relationship between these attributes. They are used to determine if a table should be scanned or not in Discovery. # At least one of the conditions must be true for a table to be scanned.
"minAge": "A String", # Minimum age a table must have before Cloud DLP can profile it. Value must be 1 hour or greater.
"minRowCount": 42, # Minimum number of rows that should be present before Cloud DLP profiles a table
},
"typeCollection": "A String", # Restrict discovery to categories of table types.
"types": { # The types of BigQuery tables supported by Cloud DLP. # Restrict discovery to specific table types.
"types": [ # A set of BigQuery table types.
"A String",
],
},
},
"disabled": { # Do not profile the tables. # Tables that match this filter will not have profiles created.
},
"filter": { # Determines what tables will have profiles generated within an organization or project. Includes the ability to filter by regular expression patterns on project ID, dataset ID, and table ID. # Required. The tables the discovery cadence applies to. The first target with a matching filter will be the one to apply to a table.
"otherTables": { # Catch-all for all other tables not specified by other filters. Should always be last, except for single-table configurations, which will only have a TableReference target. # Catch-all. This should always be the last filter in the list because anything above it will apply first. Should only appear once in a configuration. If none is specified, a default one will be added automatically.
},
"tables": { # Specifies a collection of BigQuery tables. Used for Discovery. # A specific set of tables for this filter to apply to. A table collection must be specified in only one filter per config. If a table id or dataset is empty, Cloud DLP assumes all tables in that collection must be profiled. Must specify a project ID.
"includeRegexes": { # A collection of regular expressions to determine what tables to match against. # A collection of regular expressions to match a BigQuery table against.
"patterns": [ # A single BigQuery regular expression pattern to match against one or more tables, datasets, or projects that contain BigQuery tables.
{ # A pattern to match against one or more tables, datasets, or projects that contain BigQuery tables. At least one pattern must be specified. Regular expressions use RE2 [syntax](https://github.com/google/re2/wiki/Syntax); a guide can be found under the google/re2 repository on GitHub.
"datasetIdRegex": "A String", # If unset, this property matches all datasets.
"projectIdRegex": "A String", # For organizations, if unset, will match all projects. Has no effect for data profile configurations created within a project.
"tableIdRegex": "A String", # If unset, this property matches all tables.
},
],
},
},
},
},
"cloudSqlTarget": { # Target used to match against for discovery with Cloud SQL tables. # Cloud SQL target for Discovery. The first target to match a table will be the one applied.
"conditions": { # Requirements that must be true before a table is profiled for the first time. # In addition to matching the filter, these conditions must be true before a profile is generated.
"databaseEngines": [ # Optional. Database engines that should be profiled. Optional. Defaults to ALL_SUPPORTED_DATABASE_ENGINES if unspecified.
"A String",
],
"types": [ # Data profiles will only be generated for the database resource types specified in this field. If not specified, defaults to [DATABASE_RESOURCE_TYPE_ALL_SUPPORTED_TYPES].
"A String",
],
},
"disabled": { # Do not profile the tables. # Disable profiling for database resources that match this filter.
},
"filter": { # Determines what tables will have profiles generated within an organization or project. Includes the ability to filter by regular expression patterns on project ID, location, instance, database, and database resource name. # Required. The tables the discovery cadence applies to. The first target with a matching filter will be the one to apply to a table.
"collection": { # Match database resources using regex filters. Examples of database resources are tables, views, and stored procedures. # A specific set of database resources for this filter to apply to.
"includeRegexes": { # A collection of regular expressions to determine what database resources to match against. # A collection of regular expressions to match a database resource against.
"patterns": [ # A group of regular expression patterns to match against one or more database resources. Maximum of 100 entries. The sum of all regular expression's length can't exceed 10 KiB.
{ # A pattern to match against one or more database resources. At least one pattern must be specified. Regular expressions use RE2 [syntax](https://github.com/google/re2/wiki/Syntax); a guide can be found under the google/re2 repository on GitHub.
"databaseRegex": "A String", # Regex to test the database name against. If empty, all databases match.
"databaseResourceNameRegex": "A String", # Regex to test the database resource's name against. An example of a database resource name is a table's name. Other database resource names like view names could be included in the future. If empty, all database resources match.
"instanceRegex": "A String", # Regex to test the instance name against. If empty, all instances match.
"projectIdRegex": "A String", # For organizations, if unset, will match all projects. Has no effect for Data Profile configurations created within a project.
},
],
},
},
"databaseResourceReference": { # Identifies a single database resource, like a table within a database. # The database resource to scan. Targets including this can only include one target (the target with this database resource reference).
"instance": "A String", # Required. The instance where this resource is located. For example: Cloud SQL's instance id.
"projectId": "A String", # Required. If within a project-level config, then this must match the config's project id.
},
"others": { # Match database resources not covered by any other filter. # Catch-all. This should always be the last target in the list because anything above it will apply first. Should only appear once in a configuration. If none is specified, a default one will be added automatically.
},
},
"generationCadence": { # How often existing tables should have their profiles refreshed. New tables are scanned as quickly as possible depending on system capacity. # How often and when to update profiles. New tables that match both the filter and conditions are scanned as quickly as possible depending on system capacity.
"refreshFrequency": "A String", # Data changes (non-schema changes) in Cloud SQL tables can't trigger reprofiling. If you set this field, profiles are refreshed at this frequency regardless of whether the underlying tables have changes. Defaults to never.
"schemaModifiedCadence": { # How frequency to modify the profile when the table's schema is modified. # When to reprofile if the schema has changed.
"frequency": "A String", # Frequency to regenerate data profiles when the schema is modified. Defaults to monthly.
"types": [ # The types of schema modifications to consider. Defaults to NEW_COLUMNS.
"A String",
],
},
},
},
},
],
"updateTime": "A String", # Output only. The last update timestamp of a DiscoveryConfig.
},
"inspectConfig": { # Configuration description of the scanning process. When used with redactContent only info_types and min_likelihood are currently used. # A copy of the inspection config used to generate this profile. This is a copy of the inspect_template specified in `DataProfileJobConfig`.
"contentOptions": [ # Deprecated and unused.
"A String",
],
"customInfoTypes": [ # CustomInfoTypes provided by the user. See https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes to learn more.
{ # Custom information type provided by the user. Used to find domain-specific sensitive information configurable to the data in question.
"detectionRules": [ # Set of detection rules to apply to all findings of this CustomInfoType. Rules are applied in order that they are specified. Not supported for the `surrogate_type` CustomInfoType.
{ # Deprecated; use `InspectionRuleSet` instead. Rule for modifying a `CustomInfoType` to alter behavior under certain circumstances, depending on the specific details of the rule. Not supported for the `surrogate_type` custom infoType.
"hotwordRule": { # The rule that adjusts the likelihood of findings within a certain proximity of hotwords. # Hotword-based detection rule.
"hotwordRegex": { # Message defining a custom regular expression. # Regular expression pattern defining what qualifies as a hotword.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
"likelihoodAdjustment": { # Message for specifying an adjustment to the likelihood of a finding as part of a detection rule. # Likelihood adjustment to apply to all matching findings.
"fixedLikelihood": "A String", # Set the likelihood of a finding to a fixed value.
"relativeLikelihood": 42, # Increase or decrease the likelihood by the specified number of levels. For example, if a finding would be `POSSIBLE` without the detection rule and `relative_likelihood` is 1, then it is upgraded to `LIKELY`, while a value of -1 would downgrade it to `UNLIKELY`. Likelihood may never drop below `VERY_UNLIKELY` or exceed `VERY_LIKELY`, so applying an adjustment of 1 followed by an adjustment of -1 when base likelihood is `VERY_LIKELY` will result in a final likelihood of `LIKELY`.
},
"proximity": { # Message for specifying a window around a finding to apply a detection rule. # Range of characters within which the entire hotword must reside. The total length of the window cannot exceed 1000 characters. The finding itself will be included in the window, so that hotwords can be used to match substrings of the finding itself. Suppose you want Cloud DLP to promote the likelihood of the phone number regex "\(\d{3}\) \d{3}-\d{4}" if the area code is known to be the area code of a company's office. In this case, use the hotword regex "\(xxx\)", where "xxx" is the area code in question. For tabular data, if you want to modify the likelihood of an entire column of findngs, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
"windowAfter": 42, # Number of characters after the finding to consider.
"windowBefore": 42, # Number of characters before the finding to consider. For tabular data, if you want to modify the likelihood of an entire column of findngs, set this to 1. For more information, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
},
},
},
],
"dictionary": { # Custom information type based on a dictionary of words or phrases. This can be used to match sensitive information specific to the data, such as a list of employee IDs or job titles. Dictionary words are case-insensitive and all characters other than letters and digits in the unicode [Basic Multilingual Plane](https://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane) will be replaced with whitespace when scanning for matches, so the dictionary phrase "Sam Johnson" will match all three phrases "sam johnson", "Sam, Johnson", and "Sam (Johnson)". Additionally, the characters surrounding any match must be of a different type than the adjacent characters within the word, so letters must be next to non-letters and digits next to non-digits. For example, the dictionary word "jen" will match the first three letters of the text "jen123" but will return no matches for "jennifer". Dictionary words containing a large number of characters that are not letters or digits may result in unexpected findings because such characters are treated as whitespace. The [limits](https://cloud.google.com/sensitive-data-protection/limits) page contains details about the size limits of dictionaries. For dictionaries that do not fit within these constraints, consider using `LargeCustomDictionaryConfig` in the `StoredInfoType` API. # A list of phrases to detect as a CustomInfoType.
"cloudStoragePath": { # Message representing a single file or path in Cloud Storage. # Newline-delimited file of words in Cloud Storage. Only a single file is accepted.
"path": "A String", # A URL representing a file or path (no wildcards) in Cloud Storage. Example: `gs://[BUCKET_NAME]/dictionary.txt`
},
"wordList": { # Message defining a list of words or phrases to search for in the data. # List of words or phrases to search for.
"words": [ # Words or phrases defining the dictionary. The dictionary must contain at least one phrase and every phrase must contain at least 2 characters that are letters or digits. [required]
"A String",
],
},
},
"exclusionType": "A String", # If set to EXCLUSION_TYPE_EXCLUDE this infoType will not cause a finding to be returned. It still can be used for rules matching.
"infoType": { # Type of information detected by the API. # CustomInfoType can either be a new infoType, or an extension of built-in infoType, when the name matches one of existing infoTypes and that infoType is specified in `InspectContent.info_types` field. Specifying the latter adds findings to the one detected by the system. If built-in info type is not specified in `InspectContent.info_types` list then the name is treated as a custom info type.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
"likelihood": "A String", # Likelihood to return for this CustomInfoType. This base value can be altered by a detection rule if the finding meets the criteria specified by the rule. Defaults to `VERY_LIKELY` if not specified.
"regex": { # Message defining a custom regular expression. # Regular expression based CustomInfoType.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Sensitivity for this CustomInfoType. If this CustomInfoType extends an existing InfoType, the sensitivity here will take precedence over that of the original InfoType. If unset for a CustomInfoType, it will default to HIGH. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"storedType": { # A reference to a StoredInfoType to use with scanning. # Load an existing `StoredInfoType` resource for use in `InspectDataSource`. Not currently supported in `InspectContent`.
"createTime": "A String", # Timestamp indicating when the version of the `StoredInfoType` used for inspection was created. Output-only field, populated by the system.
"name": "A String", # Resource name of the requested `StoredInfoType`, for example `organizations/433245324/storedInfoTypes/432452342` or `projects/project-id/storedInfoTypes/432452342`.
},
"surrogateType": { # Message for detecting output from deidentification transformations such as [`CryptoReplaceFfxFpeConfig`](https://cloud.google.com/sensitive-data-protection/docs/reference/rest/v2/organizations.deidentifyTemplates#cryptoreplaceffxfpeconfig). These types of transformations are those that perform pseudonymization, thereby producing a "surrogate" as output. This should be used in conjunction with a field on the transformation such as `surrogate_info_type`. This CustomInfoType does not support the use of `detection_rules`. # Message for detecting output from deidentification transformations that support reversing.
},
},
],
"excludeInfoTypes": True or False, # When true, excludes type information of the findings. This is not used for data profiling.
"includeQuote": True or False, # When true, a contextual quote from the data that triggered a finding is included in the response; see Finding.quote. This is not used for data profiling.
"infoTypes": [ # Restricts what info_types to look for. The values must correspond to InfoType values returned by ListInfoTypes or listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference. When no InfoTypes or CustomInfoTypes are specified in a request, the system may automatically choose a default list of detectors to run, which may change over time. If you need precise control and predictability as to what detectors are run you should specify specific InfoTypes listed in the reference, otherwise a default list will be used, which may change over time.
{ # Type of information detected by the API.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
],
"limits": { # Configuration to control the number of findings returned for inspection. This is not used for de-identification or data profiling. When redacting sensitive data from images, finding limits don't apply. They can cause unexpected or inconsistent results, where only some data is redacted. Don't include finding limits in RedactImage requests. Otherwise, Cloud DLP returns an error. # Configuration to control the number of findings returned. This is not used for data profiling. When redacting sensitive data from images, finding limits don't apply. They can cause unexpected or inconsistent results, where only some data is redacted. Don't include finding limits in RedactImage requests. Otherwise, Cloud DLP returns an error. When set within an InspectJobConfig, the specified maximum values aren't hard limits. If an inspection job reaches these limits, the job ends gradually, not abruptly. Therefore, the actual number of findings that Cloud DLP returns can be multiple times higher than these maximum values.
"maxFindingsPerInfoType": [ # Configuration of findings limit given for specified infoTypes.
{ # Max findings configuration per infoType, per content item or long running DlpJob.
"infoType": { # Type of information detected by the API. # Type of information the findings limit applies to. Only one limit per info_type should be provided. If InfoTypeLimit does not have an info_type, the DLP API applies the limit against all info_types that are found but not specified in another InfoTypeLimit.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
"maxFindings": 42, # Max findings limit for the given infoType.
},
],
"maxFindingsPerItem": 42, # Max number of findings that are returned for each item scanned. When set within an InspectContentRequest, this field is ignored. This value isn't a hard limit. If the number of findings for an item reaches this limit, the inspection of that item ends gradually, not abruptly. Therefore, the actual number of findings that Cloud DLP returns for the item can be multiple times higher than this value.
"maxFindingsPerRequest": 42, # Max number of findings that are returned per request or job. If you set this field in an InspectContentRequest, the resulting maximum value is the value that you set or 3,000, whichever is lower. This value isn't a hard limit. If an inspection reaches this limit, the inspection ends gradually, not abruptly. Therefore, the actual number of findings that Cloud DLP returns can be multiple times higher than this value.
},
"minLikelihood": "A String", # Only returns findings equal to or above this threshold. The default is POSSIBLE. In general, the highest likelihood setting yields the fewest findings in results and the lowest chance of a false positive. For more information, see [Match likelihood](https://cloud.google.com/sensitive-data-protection/docs/likelihood).
"minLikelihoodPerInfoType": [ # Minimum likelihood per infotype. For each infotype, a user can specify a minimum likelihood. The system only returns a finding if its likelihood is above this threshold. If this field is not set, the system uses the InspectConfig min_likelihood.
{ # Configuration for setting a minimum likelihood per infotype. Used to customize the minimum likelihood level for specific infotypes in the request. For example, use this if you want to lower the precision for PERSON_NAME without lowering the precision for the other infotypes in the request.
"infoType": { # Type of information detected by the API. # Type of information the likelihood threshold applies to. Only one likelihood per info_type should be provided. If InfoTypeLikelihood does not have an info_type, the configuration fails.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
"minLikelihood": "A String", # Only returns findings equal to or above this threshold. This field is required or else the configuration fails.
},
],
"ruleSet": [ # Set of rules to apply to the findings for this InspectConfig. Exclusion rules, contained in the set are executed in the end, other rules are executed in the order they are specified for each info type.
{ # Rule set for modifying a set of infoTypes to alter behavior under certain circumstances, depending on the specific details of the rules within the set.
"infoTypes": [ # List of infoTypes this rule set is applied to.
{ # Type of information detected by the API.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
],
"rules": [ # Set of rules to be applied to infoTypes. The rules are applied in order.
{ # A single inspection rule to be applied to infoTypes, specified in `InspectionRuleSet`.
"exclusionRule": { # The rule that specifies conditions when findings of infoTypes specified in `InspectionRuleSet` are removed from results. # Exclusion rule.
"dictionary": { # Custom information type based on a dictionary of words or phrases. This can be used to match sensitive information specific to the data, such as a list of employee IDs or job titles. Dictionary words are case-insensitive and all characters other than letters and digits in the unicode [Basic Multilingual Plane](https://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane) will be replaced with whitespace when scanning for matches, so the dictionary phrase "Sam Johnson" will match all three phrases "sam johnson", "Sam, Johnson", and "Sam (Johnson)". Additionally, the characters surrounding any match must be of a different type than the adjacent characters within the word, so letters must be next to non-letters and digits next to non-digits. For example, the dictionary word "jen" will match the first three letters of the text "jen123" but will return no matches for "jennifer". Dictionary words containing a large number of characters that are not letters or digits may result in unexpected findings because such characters are treated as whitespace. The [limits](https://cloud.google.com/sensitive-data-protection/limits) page contains details about the size limits of dictionaries. For dictionaries that do not fit within these constraints, consider using `LargeCustomDictionaryConfig` in the `StoredInfoType` API. # Dictionary which defines the rule.
"cloudStoragePath": { # Message representing a single file or path in Cloud Storage. # Newline-delimited file of words in Cloud Storage. Only a single file is accepted.
"path": "A String", # A URL representing a file or path (no wildcards) in Cloud Storage. Example: `gs://[BUCKET_NAME]/dictionary.txt`
},
"wordList": { # Message defining a list of words or phrases to search for in the data. # List of words or phrases to search for.
"words": [ # Words or phrases defining the dictionary. The dictionary must contain at least one phrase and every phrase must contain at least 2 characters that are letters or digits. [required]
"A String",
],
},
},
"excludeByHotword": { # The rule to exclude findings based on a hotword. For record inspection of tables, column names are considered hotwords. An example of this is to exclude a finding if it belongs to a BigQuery column that matches a specific pattern. # Drop if the hotword rule is contained in the proximate context. For tabular data, the context includes the column name.
"hotwordRegex": { # Message defining a custom regular expression. # Regular expression pattern defining what qualifies as a hotword.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
"proximity": { # Message for specifying a window around a finding to apply a detection rule. # Range of characters within which the entire hotword must reside. The total length of the window cannot exceed 1000 characters. The windowBefore property in proximity should be set to 1 if the hotword needs to be included in a column header.
"windowAfter": 42, # Number of characters after the finding to consider.
"windowBefore": 42, # Number of characters before the finding to consider. For tabular data, if you want to modify the likelihood of an entire column of findngs, set this to 1. For more information, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
},
},
"excludeInfoTypes": { # List of excluded infoTypes. # Set of infoTypes for which findings would affect this rule.
"infoTypes": [ # InfoType list in ExclusionRule rule drops a finding when it overlaps or contained within with a finding of an infoType from this list. For example, for `InspectionRuleSet.info_types` containing "PHONE_NUMBER"` and `exclusion_rule` containing `exclude_info_types.info_types` with "EMAIL_ADDRESS" the phone number findings are dropped if they overlap with EMAIL_ADDRESS finding. That leads to "555-222-2222@example.org" to generate only a single finding, namely email address.
{ # Type of information detected by the API.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
],
},
"matchingType": "A String", # How the rule is applied, see MatchingType documentation for details.
"regex": { # Message defining a custom regular expression. # Regular expression which defines the rule.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
},
"hotwordRule": { # The rule that adjusts the likelihood of findings within a certain proximity of hotwords. # Hotword-based detection rule.
"hotwordRegex": { # Message defining a custom regular expression. # Regular expression pattern defining what qualifies as a hotword.
"groupIndexes": [ # The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included.
42,
],
"pattern": "A String", # Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub.
},
"likelihoodAdjustment": { # Message for specifying an adjustment to the likelihood of a finding as part of a detection rule. # Likelihood adjustment to apply to all matching findings.
"fixedLikelihood": "A String", # Set the likelihood of a finding to a fixed value.
"relativeLikelihood": 42, # Increase or decrease the likelihood by the specified number of levels. For example, if a finding would be `POSSIBLE` without the detection rule and `relative_likelihood` is 1, then it is upgraded to `LIKELY`, while a value of -1 would downgrade it to `UNLIKELY`. Likelihood may never drop below `VERY_UNLIKELY` or exceed `VERY_LIKELY`, so applying an adjustment of 1 followed by an adjustment of -1 when base likelihood is `VERY_LIKELY` will result in a final likelihood of `LIKELY`.
},
"proximity": { # Message for specifying a window around a finding to apply a detection rule. # Range of characters within which the entire hotword must reside. The total length of the window cannot exceed 1000 characters. The finding itself will be included in the window, so that hotwords can be used to match substrings of the finding itself. Suppose you want Cloud DLP to promote the likelihood of the phone number regex "\(\d{3}\) \d{3}-\d{4}" if the area code is known to be the area code of a company's office. In this case, use the hotword regex "\(xxx\)", where "xxx" is the area code in question. For tabular data, if you want to modify the likelihood of an entire column of findngs, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
"windowAfter": 42, # Number of characters after the finding to consider.
"windowBefore": 42, # Number of characters before the finding to consider. For tabular data, if you want to modify the likelihood of an entire column of findngs, set this to 1. For more information, see [Hotword example: Set the match likelihood of a table column] (https://cloud.google.com/sensitive-data-protection/docs/creating-custom-infotypes-likelihood#match-column-values).
},
},
},
],
},
],
},
"inspectTemplateModifiedTime": "A String", # Timestamp when the template was modified
"inspectTemplateName": "A String", # Name of the inspection template used to generate this profile
},
"createTime": "A String", # The time at which the table was created.
"dataRiskLevel": { # Score is a summary of all elements in the data profile. A higher number means more risk. # The data risk level of this table.
"score": "A String", # The score applied to the resource.
},
"dataSourceType": { # Message used to identify the type of resource being profiled. # The resource type that was profiled.
"dataSource": "A String", # Output only. An identifying string to the type of resource being profiled. Current values: google/bigquery/table, google/project
},
"datasetId": "A String", # If the resource is BigQuery, the dataset ID.
"datasetLocation": "A String", # If supported, the location where the dataset's data is stored. See https://cloud.google.com/bigquery/docs/locations for supported locations.
"datasetProjectId": "A String", # The Google Cloud project ID that owns the resource.
"encryptionStatus": "A String", # How the table is encrypted.
"expirationTime": "A String", # Optional. The time when this table expires.
"failedColumnCount": "A String", # The number of columns skipped in the table because of an error.
"fullResource": "A String", # The resource name of the resource profiled. https://cloud.google.com/apis/design/resource_names#full_resource_name
"lastModifiedTime": "A String", # The time when this table was last modified
"name": "A String", # The name of the profile.
"otherInfoTypes": [ # Other infoTypes found in this table's data.
{ # Infotype details for other infoTypes found within a column.
"estimatedPrevalence": 42, # Approximate percentage of non-null rows that contained data detected by this infotype.
"excludedFromAnalysis": True or False, # Whether this infoType was excluded from sensitivity and risk analysis due to factors such as low prevalence (subject to change).
"infoType": { # Type of information detected by the API. # The other infoType.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
},
],
"predictedInfoTypes": [ # The infoTypes predicted from this table's data.
{ # The infoType details for this column.
"estimatedPrevalence": 42, # Not populated for predicted infotypes.
"infoType": { # Type of information detected by the API. # The infoType.
"name": "A String", # Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/sensitive-data-protection/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$_-]{1,64}`.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # Optional custom sensitivity for this InfoType. This only applies to data profiling.
"score": "A String", # The sensitivity score applied to the resource.
},
"version": "A String", # Optional version name for this InfoType.
},
},
],
"profileLastGenerated": "A String", # The last time the profile was generated.
"profileStatus": { # Success or errors for the profile generation. # Success or error status from the most recent profile generation attempt. May be empty if the profile is still being generated.
"status": { # The `Status` type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by [gRPC](https://github.com/grpc). Each `Status` message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the [API Design Guide](https://cloud.google.com/apis/design/errors). # Profiling status code and optional message. The `status.code` value is 0 (default value) for OK.
"code": 42, # The status code, which should be an enum value of google.rpc.Code.
"details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
{
"a_key": "", # Properties of the object. Contains field @type with type URL.
},
],
"message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
},
"timestamp": "A String", # Time when the profile generation status was updated
},
"projectDataProfile": "A String", # The resource name to the project data profile for this table.
"resourceLabels": { # The labels applied to the resource at the time the profile was generated.
"a_key": "A String",
},
"resourceVisibility": "A String", # How broadly a resource has been shared.
"rowCount": "A String", # Number of rows in the table when the profile was generated. This will not be populated for BigLake tables.
"scannedColumnCount": "A String", # The number of columns profiled in the table.
"sensitivityScore": { # Score is calculated from of all elements in the data profile. A higher level means the data is more sensitive. # The sensitivity score of this table.
"score": "A String", # The sensitivity score applied to the resource.
},
"state": "A String", # State of a profile.
"tableId": "A String", # If the resource is BigQuery, the BigQuery table ID.
"tableSizeBytes": "A String", # The size of the table when the profile was generated.
},
],
}
list_next()
Retrieves the next page of results.
Args:
previous_request: The request for the previous page. (required)
previous_response: The response from the request for the previous page. (required)
Returns:
A request object that you can call 'execute()' on to request the next
page. Returns None if there are no more items in the collection.