Cloud SQL for PostgreSQL Source

Cloud SQL for PostgreSQL is a fully-managed database service for Postgres.

About

Cloud SQL for PostgreSQL is a fully-managed database service that helps you set up, maintain, manage, and administer your PostgreSQL relational databases on Google Cloud Platform.

If you are new to Cloud SQL for PostgreSQL, you can try creating and connecting to a database by following these instructions.

Available Tools

Cloud SQL for PostgreSQL Source maintains full compatibility with PostgreSQL Source, allowing you to use the following tools with your Cloud SQL for PostgreSQL Source source:

Tool NameDescription
postgres-database-overview ToolThe "postgres-database-overview" fetches the current state of the PostgreSQL server.
postgres-execute-sql ToolA "postgres-execute-sql" tool executes a SQL statement against a Postgres database.
postgres-get-column-cardinality ToolThe "postgres-get-column-cardinality" tool estimates the number of unique values in one or all columns of a Postgres database table.
postgres-list-active-queries ToolThe "postgres-list-active-queries" tool lists currently active queries in a Postgres database.
postgres-list-available-extensions ToolThe "postgres-list-available-extensions" tool retrieves all PostgreSQL extensions available for installation on a Postgres database.
postgres-list-database-stats ToolThe "postgres-list-database-stats" tool lists lists key performance and activity statistics of PostgreSQL databases.
postgres-list-indexes ToolThe "postgres-list-indexes" tool lists indexes in a Postgres database.
postgres-list-installed-extensions ToolThe "postgres-list-installed-extensions" tool retrieves all PostgreSQL extensions installed on a Postgres database.
postgres-list-locks ToolThe "postgres-list-locks" tool lists active locks in the database, including the associated process, lock type, relation, mode, and the query holding or waiting on the lock.
postgres-list-pg-settings ToolThe "postgres-list-pg-settings" tool lists PostgreSQL run-time configuration settings.
postgres-list-publication-tables ToolThe "postgres-list-publication-tables" tool lists publication tables in a Postgres database.
postgres-list-query-stats ToolThe "postgres-list-query-stats" tool lists query statistics from a Postgres database.
postgres-list-roles ToolThe "postgres-list-roles" tool lists user-created roles in a Postgres database.
postgres-list-schemas ToolThe "postgres-list-schemas" tool lists user-defined schemas in a database.
postgres-list-sequences ToolThe "postgres-list-sequences" tool lists sequences in a Postgres database.
postgres-list-stored-procedure ToolThe "postgres-list-stored-procedure" tool retrieves metadata for stored procedures in PostgreSQL, including procedure definitions, owners, languages, and descriptions.
postgres-list-table-stats ToolThe "postgres-list-table-stats" tool reports table statistics including size, scan metrics, and bloat indicators for PostgreSQL tables.
postgres-list-tables ToolThe "postgres-list-tables" tool lists schema information for all or specified tables in a Postgres database.
postgres-list-tablespaces ToolThe "postgres-list-tablespaces" tool lists tablespaces in a Postgres database.
postgres-list-triggers ToolThe "postgres-list-triggers" tool lists triggers in a Postgres database.
postgres-list-views ToolThe "postgres-list-views" tool lists views in a Postgres database, with a default limit of 50 rows.
postgres-long-running-transactions ToolThe postgres-long-running-transactions tool Identifies and lists database transactions that exceed a specified time limit. For each of the long running transactions, the output contains the process id, database name, user name, application name, client address, state, connection age, transaction age, query age, last activity age, wait event type, wait event, and query string.
postgres-replication-stats ToolThe "postgres-replication-stats" tool reports replication-related metrics for WAL streaming replicas, including lag sizes presented in human-readable form.
postgres-sql ToolA "postgres-sql" tool executes a pre-defined SQL statement against a Postgres database.

Pre-built Configurations

Requirements

IAM Permissions

By default, this source uses the Cloud SQL Go Connector to authorize and establish mTLS connections to your Cloud SQL instance. The Go connector uses your Application Default Credentials (ADC) to authorize your connection to Cloud SQL.

In addition to setting the ADC for your server, you need to ensure the IAM identity has been given the following IAM roles (or corresponding permissions):

  • roles/cloudsql.client

Tip

If you are connecting from Compute Engine, make sure your VM also has the proper scope to connect using the Cloud SQL Admin API.

Networking

Cloud SQL supports connecting over both from external networks via the internet (public IP), and internal networks (private IP). For more information on choosing between the two options, see the Cloud SQL page Connection overview.

You can configure the ipType parameter in your source configuration to public or private to match your cluster’s configuration. Regardless of which you choose, all connections use IAM-based authorization and are encrypted with mTLS.

Authentication

This source supports both password-based authentication and IAM authentication (using your Application Default Credentials).

Standard Authentication

To connect using user/password, create a PostgreSQL user and input your credentials in the user and password fields.

user: ${USER_NAME}
password: ${PASSWORD}

IAM Authentication

To connect using IAM authentication:

  1. Prepare your database instance and user following this guide.

  2. You could choose one of the two ways to log in:

    • Specify your IAM email as the user.
    • Leave your user field blank. Toolbox will fetch the ADC automatically and log in using the email associated with it.
  3. Leave the password field blank.

Example

kind: sources
name: my-cloud-sql-pg-source
type: cloud-sql-postgres
project: my-project-id
region: us-central1
instance: my-instance
database: my_db
user: ${USER_NAME}
password: ${PASSWORD}
# ipType: "private"

Tip

Use environment variable replacement with the format ${ENV_NAME} instead of hardcoding your secrets into the configuration file.

Managed Connection Pooling

Toolbox automatically supports Managed Connection Pooling. If your Cloud SQL for PostgreSQL instance has Managed Connection Pooling enabled, the connection will immediately benefit from increased throughput and reduced latency.

The interface is identical, so there’s no additional configuration required on the client. For more information on configuring your instance, see the Cloud SQL Managed Connection Pooling documentation.

Reference

fieldtyperequireddescription
typestringtrueMust be “cloud-sql-postgres”.
projectstringtrueId of the GCP project that the cluster was created in (e.g. “my-project-id”).
regionstringtrueName of the GCP region that the cluster was created in (e.g. “us-central1”).
instancestringtrueName of the Cloud SQL instance within the cluster (e.g. “my-instance”).
databasestringtrueName of the Postgres database to connect to (e.g. “my_db”).
userstringfalseName of the Postgres user to connect as (e.g. “my-pg-user”). Defaults to IAM auth using ADC email if unspecified.
passwordstringfalsePassword of the Postgres user (e.g. “my-password”). Defaults to attempting IAM authentication if unspecified.
ipTypestringfalseIP Type of the Cloud SQL instance; must be one of public, private, or psc. Default: public.