Dataproc Clusters Source
Google Cloud Dataproc Clusters lets you provision and manage Apache Spark and Hadoop clusters.
less than a minute
About
The Dataproc Clusters source allows Toolbox to interact with Dataproc Clusters hosted on Google Cloud.
Available Tools
Dataproc Clusters Source Tools
| Tool Name | Description |
|---|---|
| dataproc-get-cluster | A "dataproc-get-cluster" tool retrieves a specific Dataproc cluster from the source. |
| dataproc-get-job | A "dataproc-get-job" tool retrieves a specific Dataproc job from the source. |
| dataproc-list-clusters | A "dataproc-list-clusters" tool returns a list of Dataproc clusters from the source. |
| dataproc-list-jobs | A "dataproc-list-jobs" tool returns a list of Dataproc jobs from the source. |
Requirements
IAM Permissions
Dataproc uses Identity and Access Management (IAM) to control user and group access to Dataproc resources.
Toolbox will use your Application Default Credentials
(ADC) to authorize and
authenticate when interacting with Dataproc. When using this method, you need to
ensure the IAM identity associated with your ADC has the correct
permissions
for the actions you intend to perform. Common roles include
roles/dataproc.editor or roles/dataproc.viewer. Follow this
guide to
set up your ADC.
Example
kind: source
name: my-dataproc-source
type: dataproc
project: my-project
region: us-central1
Reference
| field | type | required | description |
|---|---|---|---|
| type | string | true | Must be “dataproc”. |
| project | string | true | ID of the GCP project with Dataproc resources. |
| region | string | true | Region containing Dataproc resources. |
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.