Deploy to Cloud Run

Before you begin

1. Install the Google Cloud CLI.

2. Set the PROJECT_ID environment variable:

   export PROJECT_ID="my-project-id"
   

3. Initialize gcloud CLI:

   gcloud init
   gcloud config set project $PROJECT_ID
   

4. Make sure you’ve set up and initialized your database.

5. You must have the following APIs enabled:

   gcloud services enable run.googleapis.com \
                          cloudbuild.googleapis.com \
                          artifactregistry.googleapis.com \
                          iam.googleapis.com \
                          secretmanager.googleapis.com 
   

6. To create an IAM account, you must have the following IAM permissions (or roles):

   • Create Service Account role (roles/iam.serviceAccountCreator)

7. To create a secret, you must have the following roles:

   • Secret Manager Admin role (roles/secretmanager.admin)

8. To deploy to Cloud Run, you must have the following set of roles:

   • Cloud Run Developer (roles/run.developer)
   • Service Account User role (roles/iam.serviceAccountUser)

Create a service account

1. Create a backend service account if you don’t already have one:

   gcloud iam service-accounts create toolbox-identity
   

2. Grant permissions to use secret manager:

   gcloud projects add-iam-policy-binding $PROJECT_ID \
       --member serviceAccount:toolbox-identity@$PROJECT_ID.iam.gserviceaccount.com \
       --role roles/secretmanager.secretAccessor
   

3. Grant additional permissions to the service account that are specific to the source, e.g.:

   • AlloyDB for PostgreSQL
   • Cloud SQL for PostgreSQL

Configure tools.yaml file

Create a tools.yaml file that contains your configuration for Toolbox. For details, see the configuration section.

Deploy to Cloud Run

1. Upload tools.yaml as a secret:

   gcloud secrets create tools --data-file=tools.yaml
   

   If you already have a secret and want to update the secret version, execute the following:

   gcloud secrets versions add tools --data-file=tools.yaml
   

2. Set an environment variable to the container image that you want to use for cloud run:

   export IMAGE=us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:latest
   

3. Deploy Toolbox to Cloud Run using the following command:

   gcloud run deploy toolbox \
       --image $IMAGE \
       --service-account toolbox-identity \
       --region us-central1 \
       --set-secrets "/app/tools.yaml=tools:latest" \
       --args="--tools_file=/app/tools.yaml","--address=0.0.0.0","--port=8080"
       # --allow-unauthenticated # https://cloud.google.com/run/docs/authenticating/public#gcloud
   

   If you are using a VPC network, use the command below:

   gcloud run deploy toolbox \
       --image $IMAGE \
       --service-account toolbox-identity \
       --region us-central1 \
       --set-secrets "/app/tools.yaml=tools:latest" \
       --args="--tools_file=/app/tools.yaml","--address=0.0.0.0","--port=8080" \
       # TODO(dev): update the following to match your VPC if necessary 
       --network default \
       --subnet default
       # --allow-unauthenticated # https://cloud.google.com/run/docs/authenticating/public#gcloud
   

Connecting to Cloud Run

Next, we will use gcloud to authenticate requests to our Cloud Run instance:

1. Run the run services proxy to proxy connections to Cloud Run:

   gcloud run services proxy toolbox --port=8080 --region=us-central1
   

   If you are prompted to install the proxy, reply Y to install.

2. Finally, use curl to verify the endpoint works:

   curl http://127.0.0.1:8080
   

Connecting with Toolbox Client SDK

Next, we will use Toolbox with client SDK:

1. Run the following to retrieve a non-deterministic URL for the cloud run service:

   gcloud run services describe toolbox --format 'value(status.url)'
   

2. Import and initialize the toolbox client with the URL retrieved above:

   • LangChain
   • Llamaindex

   from toolbox_langchain import ToolboxClient
   
   # Replace with the cloud run service URL generated above
   toolbox = ToolboxClient("http://$YOUR_URL")

   from toolbox_llamaindex import ToolboxClient
   
   # Replace with the cloud run service URL generated above
   toolbox = ToolboxClient("http://$YOUR_URL")