Introduction

The Google Cloud Client Libraries for Rust is a collection of Rust crates to interact with Google Cloud Services.

This guide is organized as a series of small tutorials showing how to perform specific actions with the client libraries. Most Google Cloud services follow a series of guidelines, collectively known as the AIPs. This makes the client libraries more consistent from one service to the next. The functions to delete or list resources almost always have the same interface.

Audience

This guide is intended for Rust developers who are familiar with the language and the Rust ecosystem. We will assume you know how to use Rust and its supporting toolchain.

At the risk of being repetitive, most of the guides do not assume you have used any Google Service or client library before (in Rust or other language). However, the guides will refer you to service specific tutorials to initialize their projects and services.

Service specific documentation

These guides are not intended as tutorials for each service or as extended guides on how to design Rust applications to work on Google Cloud. They are starting points to get you productive with the client libraries for Rust.

We recommend you read the service documentation at https://cloud.google.com to learn more about each service. If you need guidance on how to design your application for Google Cloud, the Cloud Architecture Center may have what you are looking for.

Reporting bugs

We welcome bugs about the client libraries or their documentation. Please use GitHub Issues.

License

The client libraries source and their documentation are release under the Apache License, Version 2.0.

Setting up your development environment

Prepare your environment for Rust app development and deployment on Google Cloud by installing the following tools.

Install Rust

1. To install Rust, see Getting Started.

2. Confirm that you have the most recent version of Rust installed:

   cargo --version
   

Install an editor

The Getting Started guide links popular editor plugins and IDEs, which provide the following features:

• Fully integrated debugging capabilities
• Syntax highlighting
• Code completion

Install the Google Cloud CLI

The Google Cloud CLI is a set of tools for Google Cloud. It contains the gcloud and bq command-line tools used to access Compute Engine, Cloud Storage, BigQuery, and other services from the command line. You can run these tools interactively or in your automated scripts.

To install the gcloud CLI, see Installing the gcloud CLI.

Install the Cloud Client Libraries for Rust in a new project

The Cloud Client Libraries for Rust is the idiomatic way for Rust developers to integrate with Google Cloud services, such as Secret Manager and Workflows.

For example, to use the package for an individual API, such as the Secret Manager API, do the following:

1. Create a new Rust project:

   cargo new my-project
   

2. Change your directory to the new project:

   cd my-project
   

3. Add the Secret Manager client library to the new project:

   cargo add google-cloud-secretmanager-v1
   

   If you haven't already enabled the Secret Manager API, enable it in APIs and services or by running the following command:

   gcloud services enable secretmanager.googleapis.com
   

4. Add the google-cloud-gax crate to the new project:

   cargo add google-cloud-gax
   

5. Add the tokio crate to the new project:

   cargo add tokio --features macros
   

6. Edit src/main.rs in your project to use the Secret Manager client library:

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    use google_cloud_gax::paginator::ItemPaginator as _;
    use google_cloud_secretmanager_v1::client::SecretManagerService;
    let project_id = std::env::args().nth(1).unwrap();
    let client = SecretManagerService::builder().build().await?;

    let mut items = client
        .list_secrets()
        .set_parent(format!("projects/{project_id}"))
        .by_item();
    while let Some(item) = items.next().await {
        println!("{}", item?.name);
    }
    Ok(())
}

6. Build your program:

   cargo build
   

   The program should build without errors.

Note: The source of the Cloud Client Libraries for Rust is on GitHub.

Running the program

1. To use the Cloud Client Libraries in a local development environment, set up Application Default Credentials.

   gcloud auth application-default login
   

   For more information, see Authenticate for using client libraries.

2. Run your program, supplying your Google Cloud Platform project's ID:

   PROJECT_ID=$(gcloud config get project)
   cargo run ${PROJECT_ID}
   

   The program will print the secrets associated with your project ID. If you don't see any secrets, you might not have any in Secret Manager. You can create a secret and rerun the program, and you should see the secret printed in the output.

What's next

• Explore authentication methods at Google.
• Browse the documentation for Google Cloud products.

Setting up Rust on Cloud Shell

Cloud Shell is a great environment to run small examples and tests. This guide shows you how to configure Rust and install one of the Cloud Client Libraries in Cloud Shell.

Start up Cloud Shell

1. In the Google Cloud console project selector, select a project.

2. Open https://shell.cloud.google.com to start a new shell. You might be prompted to authorize Cloud Shell to use your credentials for Google Cloud API calls.

Configure Rust

1. Cloud Shell comes with rustup pre-installed. You can use it to install and configure the default version of Rust:

   rustup default stable
   

2. Confirm that you have the most recent version of Rust installed:

   cargo --version
   

Install Rust client libraries in Cloud Shell

1. Create a new Rust project:

   cargo new my-project
   

2. Change your directory to the new project:

   cd my-project
   

3. Add the Secret Manager client library to the new project:

   cargo add google-cloud-secretmanager-v1
   

4. Add the google-cloud-gax crate to the new project:

   cargo add google-cloud-gax
   

5. Add the tokio crate to the new project:

   cargo add tokio --features macros
   

6. Edit src/main.rs in your project to use the Secret Manager client library:

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    use google_cloud_gax::paginator::ItemPaginator as _;
    use google_cloud_secretmanager_v1::client::SecretManagerService;
    let project_id = std::env::args().nth(1).unwrap();
    let client = SecretManagerService::builder().build().await?;

    let mut items = client
        .list_secrets()
        .set_parent(format!("projects/{project_id}"))
        .by_item();
    while let Some(item) = items.next().await {
        println!("{}", item?.name);
    }
    Ok(())
}

1. Run your program, supplying your Google Cloud Platform project's ID:

   PROJECT_ID=$(gcloud config get project)
   cargo run ${PROJECT_ID}
   

   The program will print the secrets associated with your project ID. If you don't see any secrets, you might not have any in Secret Manager. You can create a secret and rerun the program, and you should see the secret printed in the output.

How to initialize a client

The Google Cloud Client Libraries for Rust use clients as the main abstraction to interface with specific services. Clients are implemented as Rust structs, with methods corresponding to each RPC offered by the service. To use a Google Cloud service using the Rust client libraries, you need to first initialize a client.

Prerequisites

In this guide you'll initialize a client and then use the client to make a simple RPC. For this tutorial, you'll use the Secret Manager API. The same structure applies to any other service in Google Cloud.

We recommend that you follow one of the Secret Manager getting started guides, such as how to Create a secret, before attempting to use the client library. These guides cover service specific concepts in more detail, and provide detailed guidance on project prerequisites.

We also recommend that you follow the instructions in the Authenticate for using client libraries guide. This guide will show you how to log in to configure the Application Default Credentials used in this guide.

Dependencies

As usual with Rust, you must declare the dependency in your Cargo.toml file:

cargo add google-cloud-secretmanager-v1

To initialize a client, you first call Client::builder() to obtain an appropriate ClientBuilder and then call build() on that builder to create a client.

The following creates a client with the default configuration, which is designed to meet requirements for most use cases.

    let client = SecretManagerService::builder().build().await?;

Once the client is successfully initialized, you can use it to make RPCs:

    use google_cloud_gax::paginator::Paginator as _;
    let mut items = client
        .list_locations()
        .set_name(format!("projects/{project_id}"))
        .by_page();
    while let Some(page) = items.next().await {
        let page = page?;
        for location in page.locations {
            println!("{}", location.name);
        }
    }

This example shows a call to list_locations, which returns information about the supported locations for the service (in this case, Secret Manager). The output of the example should look something like this:

projects/123456789012/locations/europe-west8
projects/123456789012/locations/europe-west9
projects/123456789012/locations/us-east5
...

Full program

Putting all this code together into a full program looks as follows:

pub type Result = std::result::Result<(), Box<dyn std::error::Error>>;

pub async fn initialize_client(project_id: &str) -> Result {
    use google_cloud_secretmanager_v1::client::SecretManagerService;

    // Initialize a client with the default configuration. This is an
    // asynchronous operation that may fail, as it requires acquiring an an
    // access token.
    let client = SecretManagerService::builder().build().await?;

    // Once initialized, use the client to make requests.
    use google_cloud_gax::paginator::Paginator as _;
    let mut items = client
        .list_locations()
        .set_name(format!("projects/{project_id}"))
        .by_page();
    while let Some(page) = items.next().await {
        let page = page?;
        for location in page.locations {
            println!("{}", location.name);
        }
    }

    Ok(())
}

What's next

This guide showed you how to initialize a client using the Google Cloud Client Libraries for Rust. For a more complex example of working with a service, check out Generate text using the Vertex AI Gemini API.

Generate text using the Vertex AI Gemini API

In this guide, you send a text prompt request, and then a multimodal prompt and image request to the Vertex AI Gemini API and view the responses.

Prerequisites

To complete this guide, you must have a Google Cloud project with the Vertex AI API enabled. You can use the Vertex AI setup guide to complete these steps.

Add the Vertex AI client library as a dependency

The Vertex AI client library includes many features. Compiling all of them is relatively slow. To speed up compilation, you can just enable the features you need:

cargo add google-cloud-aiplatform-v1 --no-default-features --features prediction-service

Send a prompt to the Vertex AI Gemini API

First, initialize the client using the default settings:

    use google_cloud_aiplatform_v1 as vertexai;
    let client = vertexai::client::PredictionService::builder()
        .build()
        .await?;

Then build the model name. For simplicity, this example receives the project ID as an argument and uses a fixed location (global) and model id (gemini-2.0-flash-001).

    const MODEL: &str = "gemini-2.0-flash-001";
    let model = format!("projects/{project_id}/locations/global/publishers/google/models/{MODEL}");

If you want to run this function in your own code, use the project id (without any projects/ prefix) of the project you selected while going through the prerequisites.

With the client initialized you can send the request:

    let response = client
        .generate_content().set_model(&model)
        .set_contents([vertexai::model::Content::new().set_role("user").set_parts([
            vertexai::model::Part::new().set_text("What's a good name for a flower shop that specializes in selling bouquets of dried flowers?"),
        ])])
        .send()
        .await;

And then print the response. You can use the :#? format specifier to prettify the nested response objects:

    println!("RESPONSE = {response:#?}");

See below for the complete code.

Send a prompt and an image to the Vertex AI Gemini API

As in the previous example, initialize the client using the default settings:

    use google_cloud_aiplatform_v1 as vertexai;
    let client = vertexai::client::PredictionService::builder()
        .build()
        .await?;

And then build the model name:

    const MODEL: &str = "gemini-2.0-flash-001";
    let model = format!("projects/{project_id}/locations/global/publishers/google/models/{MODEL}");

The new request includes an image part:

                vertexai::model::Part::new().set_file_data(
                    vertexai::model::FileData::new()
                        .set_mime_type("image/jpeg")
                        .set_file_uri("gs://generativeai-downloads/images/scones.jpg"),
                ),

And the prompt part:

                vertexai::model::Part::new().set_text("Describe this picture."),

Send the full request:

    let response = client
        .generate_content()
        .set_model(&model)
        .set_contents(
            [vertexai::model::Content::new().set_role("user").set_parts([
                vertexai::model::Part::new().set_file_data(
                    vertexai::model::FileData::new()
                        .set_mime_type("image/jpeg")
                        .set_file_uri("gs://generativeai-downloads/images/scones.jpg"),
                ),
                vertexai::model::Part::new().set_text("Describe this picture."),
            ])],
        )
        .send()
        .await;

As in the previous example, print the full response:

    println!("RESPONSE = {response:#?}");

See below for the complete code.

Text prompt: complete code

pub async fn text_prompt(project_id: &str) -> crate::Result<()> {
    use google_cloud_aiplatform_v1 as vertexai;
    let client = vertexai::client::PredictionService::builder()
        .build()
        .await?;

    const MODEL: &str = "gemini-2.0-flash-001";
    let model = format!("projects/{project_id}/locations/global/publishers/google/models/{MODEL}");

    let response = client
        .generate_content().set_model(&model)
        .set_contents([vertexai::model::Content::new().set_role("user").set_parts([
            vertexai::model::Part::new().set_text("What's a good name for a flower shop that specializes in selling bouquets of dried flowers?"),
        ])])
        .send()
        .await;
    println!("RESPONSE = {response:#?}");

    Ok(())
}

Prompt and image: complete code

pub async fn prompt_and_image(project_id: &str) -> crate::Result<()> {
    use google_cloud_aiplatform_v1 as vertexai;
    let client = vertexai::client::PredictionService::builder()
        .build()
        .await?;

    const MODEL: &str = "gemini-2.0-flash-001";
    let model = format!("projects/{project_id}/locations/global/publishers/google/models/{MODEL}");

    let response = client
        .generate_content()
        .set_model(&model)
        .set_contents(
            [vertexai::model::Content::new().set_role("user").set_parts([
                vertexai::model::Part::new().set_file_data(
                    vertexai::model::FileData::new()
                        .set_mime_type("image/jpeg")
                        .set_file_uri("gs://generativeai-downloads/images/scones.jpg"),
                ),
                vertexai::model::Part::new().set_text("Describe this picture."),
            ])],
        )
        .send()
        .await;
    println!("RESPONSE = {response:#?}");

    Ok(())
}

Using Google Cloud Storage

Google Cloud Storage is a managed service for storing unstructured data.

The Rust client library provides an idiomatic API to access this service. The client library resumes interrupted downloads and uploads, and automatically performs integrity checks on the data. For metadata operations, the client library can retry failed requests, and automatically poll long-running operations.

Quickstart

This guide will show you how to create a Cloud Storage bucket, upload an object to this bucket, and then read the object back.

Prerequisites

The guide assumes you have an existing Google Cloud project with billing enabled.

Add the client library as a dependency

cargo add google-cloud-storage

Create a storage bucket

The client to perform operations on buckets and object metadata is called StorageControl:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

pub async fn quickstart(project_id: &str, bucket_id: &str) -> anyhow::Result<()> {
    use google_cloud_storage as gcs;
    use google_cloud_storage::client::StorageControl;
    let control = StorageControl::builder().build().await?;
    let bucket = control
        .create_bucket()
        .set_parent("projects/_")
        .set_bucket_id(bucket_id)
        .set_bucket(
            gcs::model::Bucket::new()
                .set_project(format!("projects/{project_id}"))
                .set_iam_config(
                    gcs::model::bucket::IamConfig::new().set_uniform_bucket_level_access(
                        gcs::model::bucket::iam_config::UniformBucketLevelAccess::new()
                            .set_enabled(true),
                    ),
                ),
        )
        .send()
        .await?;
    println!("bucket successfully created {bucket:?}");

    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let object = client
        .write_object(&bucket.name, "hello.txt", "Hello World!")
        .send_buffered()
        .await?;
    println!("object successfully uploaded {object:?}");

    let mut reader = client.read_object(&bucket.name, "hello.txt").send().await?;
    let mut contents = Vec::new();
    while let Some(chunk) = reader.next().await.transpose()? {
        contents.extend_from_slice(&chunk);
    }
    println!(
        "object contents successfully downloaded {:?}",
        bytes::Bytes::from_owner(contents)
    );

    control
        .delete_object()
        .set_bucket(&bucket.name)
        .set_object(&object.name)
        .set_generation(object.generation)
        .send()
        .await?;
    control
        .delete_bucket()
        .set_name(&bucket.name)
        .send()
        .await?;

    Ok(())
}

To create a bucket you must provide the project name and the desired bucket id:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

pub async fn quickstart(project_id: &str, bucket_id: &str) -> anyhow::Result<()> {
    use google_cloud_storage as gcs;
    use google_cloud_storage::client::StorageControl;
    let control = StorageControl::builder().build().await?;
    let bucket = control
        .create_bucket()
        .set_parent("projects/_")
        .set_bucket_id(bucket_id)
        .set_bucket(
            gcs::model::Bucket::new()
                .set_project(format!("projects/{project_id}"))
                .set_iam_config(
                    gcs::model::bucket::IamConfig::new().set_uniform_bucket_level_access(
                        gcs::model::bucket::iam_config::UniformBucketLevelAccess::new()
                            .set_enabled(true),
                    ),
                ),
        )
        .send()
        .await?;
    println!("bucket successfully created {bucket:?}");

    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let object = client
        .write_object(&bucket.name, "hello.txt", "Hello World!")
        .send_buffered()
        .await?;
    println!("object successfully uploaded {object:?}");

    let mut reader = client.read_object(&bucket.name, "hello.txt").send().await?;
    let mut contents = Vec::new();
    while let Some(chunk) = reader.next().await.transpose()? {
        contents.extend_from_slice(&chunk);
    }
    println!(
        "object contents successfully downloaded {:?}",
        bytes::Bytes::from_owner(contents)
    );

    control
        .delete_object()
        .set_bucket(&bucket.name)
        .set_object(&object.name)
        .set_generation(object.generation)
        .send()
        .await?;
    control
        .delete_bucket()
        .set_name(&bucket.name)
        .send()
        .await?;

    Ok(())
}

You can also provide other attributes for the bucket. For example, if you want all objects in the bucket to use the same permissions, you can enable Uniform bucket-level access:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

pub async fn quickstart(project_id: &str, bucket_id: &str) -> anyhow::Result<()> {
    use google_cloud_storage as gcs;
    use google_cloud_storage::client::StorageControl;
    let control = StorageControl::builder().build().await?;
    let bucket = control
        .create_bucket()
        .set_parent("projects/_")
        .set_bucket_id(bucket_id)
        .set_bucket(
            gcs::model::Bucket::new()
                .set_project(format!("projects/{project_id}"))
                .set_iam_config(
                    gcs::model::bucket::IamConfig::new().set_uniform_bucket_level_access(
                        gcs::model::bucket::iam_config::UniformBucketLevelAccess::new()
                            .set_enabled(true),
                    ),
                ),
        )
        .send()
        .await?;
    println!("bucket successfully created {bucket:?}");

    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let object = client
        .write_object(&bucket.name, "hello.txt", "Hello World!")
        .send_buffered()
        .await?;
    println!("object successfully uploaded {object:?}");

    let mut reader = client.read_object(&bucket.name, "hello.txt").send().await?;
    let mut contents = Vec::new();
    while let Some(chunk) = reader.next().await.transpose()? {
        contents.extend_from_slice(&chunk);
    }
    println!(
        "object contents successfully downloaded {:?}",
        bytes::Bytes::from_owner(contents)
    );

    control
        .delete_object()
        .set_bucket(&bucket.name)
        .set_object(&object.name)
        .set_generation(object.generation)
        .send()
        .await?;
    control
        .delete_bucket()
        .set_name(&bucket.name)
        .send()
        .await?;

    Ok(())
}

Then send this request and wait for the response:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

pub async fn quickstart(project_id: &str, bucket_id: &str) -> anyhow::Result<()> {
    use google_cloud_storage as gcs;
    use google_cloud_storage::client::StorageControl;
    let control = StorageControl::builder().build().await?;
    let bucket = control
        .create_bucket()
        .set_parent("projects/_")
        .set_bucket_id(bucket_id)
        .set_bucket(
            gcs::model::Bucket::new()
                .set_project(format!("projects/{project_id}"))
                .set_iam_config(
                    gcs::model::bucket::IamConfig::new().set_uniform_bucket_level_access(
                        gcs::model::bucket::iam_config::UniformBucketLevelAccess::new()
                            .set_enabled(true),
                    ),
                ),
        )
        .send()
        .await?;
    println!("bucket successfully created {bucket:?}");

    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let object = client
        .write_object(&bucket.name, "hello.txt", "Hello World!")
        .send_buffered()
        .await?;
    println!("object successfully uploaded {object:?}");

    let mut reader = client.read_object(&bucket.name, "hello.txt").send().await?;
    let mut contents = Vec::new();
    while let Some(chunk) = reader.next().await.transpose()? {
        contents.extend_from_slice(&chunk);
    }
    println!(
        "object contents successfully downloaded {:?}",
        bytes::Bytes::from_owner(contents)
    );

    control
        .delete_object()
        .set_bucket(&bucket.name)
        .set_object(&object.name)
        .set_generation(object.generation)
        .send()
        .await?;
    control
        .delete_bucket()
        .set_name(&bucket.name)
        .send()
        .await?;

    Ok(())
}

Upload an object

The client to perform operations on object data is called Storage:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

pub async fn quickstart(project_id: &str, bucket_id: &str) -> anyhow::Result<()> {
    use google_cloud_storage as gcs;
    use google_cloud_storage::client::StorageControl;
    let control = StorageControl::builder().build().await?;
    let bucket = control
        .create_bucket()
        .set_parent("projects/_")
        .set_bucket_id(bucket_id)
        .set_bucket(
            gcs::model::Bucket::new()
                .set_project(format!("projects/{project_id}"))
                .set_iam_config(
                    gcs::model::bucket::IamConfig::new().set_uniform_bucket_level_access(
                        gcs::model::bucket::iam_config::UniformBucketLevelAccess::new()
                            .set_enabled(true),
                    ),
                ),
        )
        .send()
        .await?;
    println!("bucket successfully created {bucket:?}");

    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let object = client
        .write_object(&bucket.name, "hello.txt", "Hello World!")
        .send_buffered()
        .await?;
    println!("object successfully uploaded {object:?}");

    let mut reader = client.read_object(&bucket.name, "hello.txt").send().await?;
    let mut contents = Vec::new();
    while let Some(chunk) = reader.next().await.transpose()? {
        contents.extend_from_slice(&chunk);
    }
    println!(
        "object contents successfully downloaded {:?}",
        bytes::Bytes::from_owner(contents)
    );

    control
        .delete_object()
        .set_bucket(&bucket.name)
        .set_object(&object.name)
        .set_generation(object.generation)
        .send()
        .await?;
    control
        .delete_bucket()
        .set_name(&bucket.name)
        .send()
        .await?;

    Ok(())
}

In this case we will create an object called hello.txt, with the traditional greeting for a programming tutorial:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

pub async fn quickstart(project_id: &str, bucket_id: &str) -> anyhow::Result<()> {
    use google_cloud_storage as gcs;
    use google_cloud_storage::client::StorageControl;
    let control = StorageControl::builder().build().await?;
    let bucket = control
        .create_bucket()
        .set_parent("projects/_")
        .set_bucket_id(bucket_id)
        .set_bucket(
            gcs::model::Bucket::new()
                .set_project(format!("projects/{project_id}"))
                .set_iam_config(
                    gcs::model::bucket::IamConfig::new().set_uniform_bucket_level_access(
                        gcs::model::bucket::iam_config::UniformBucketLevelAccess::new()
                            .set_enabled(true),
                    ),
                ),
        )
        .send()
        .await?;
    println!("bucket successfully created {bucket:?}");

    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let object = client
        .write_object(&bucket.name, "hello.txt", "Hello World!")
        .send_buffered()
        .await?;
    println!("object successfully uploaded {object:?}");

    let mut reader = client.read_object(&bucket.name, "hello.txt").send().await?;
    let mut contents = Vec::new();
    while let Some(chunk) = reader.next().await.transpose()? {
        contents.extend_from_slice(&chunk);
    }
    println!(
        "object contents successfully downloaded {:?}",
        bytes::Bytes::from_owner(contents)
    );

    control
        .delete_object()
        .set_bucket(&bucket.name)
        .set_object(&object.name)
        .set_generation(object.generation)
        .send()
        .await?;
    control
        .delete_bucket()
        .set_name(&bucket.name)
        .send()
        .await?;

    Ok(())
}

Download an object

To download the contents of an object use read_object():

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

pub async fn quickstart(project_id: &str, bucket_id: &str) -> anyhow::Result<()> {
    use google_cloud_storage as gcs;
    use google_cloud_storage::client::StorageControl;
    let control = StorageControl::builder().build().await?;
    let bucket = control
        .create_bucket()
        .set_parent("projects/_")
        .set_bucket_id(bucket_id)
        .set_bucket(
            gcs::model::Bucket::new()
                .set_project(format!("projects/{project_id}"))
                .set_iam_config(
                    gcs::model::bucket::IamConfig::new().set_uniform_bucket_level_access(
                        gcs::model::bucket::iam_config::UniformBucketLevelAccess::new()
                            .set_enabled(true),
                    ),
                ),
        )
        .send()
        .await?;
    println!("bucket successfully created {bucket:?}");

    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let object = client
        .write_object(&bucket.name, "hello.txt", "Hello World!")
        .send_buffered()
        .await?;
    println!("object successfully uploaded {object:?}");

    let mut reader = client.read_object(&bucket.name, "hello.txt").send().await?;
    let mut contents = Vec::new();
    while let Some(chunk) = reader.next().await.transpose()? {
        contents.extend_from_slice(&chunk);
    }
    println!(
        "object contents successfully downloaded {:?}",
        bytes::Bytes::from_owner(contents)
    );

    control
        .delete_object()
        .set_bucket(&bucket.name)
        .set_object(&object.name)
        .set_generation(object.generation)
        .send()
        .await?;
    control
        .delete_bucket()
        .set_name(&bucket.name)
        .send()
        .await?;

    Ok(())
}

Cleanup

Finally we remove the object and bucket to cleanup all the resources used in this guide:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

pub async fn quickstart(project_id: &str, bucket_id: &str) -> anyhow::Result<()> {
    use google_cloud_storage as gcs;
    use google_cloud_storage::client::StorageControl;
    let control = StorageControl::builder().build().await?;
    let bucket = control
        .create_bucket()
        .set_parent("projects/_")
        .set_bucket_id(bucket_id)
        .set_bucket(
            gcs::model::Bucket::new()
                .set_project(format!("projects/{project_id}"))
                .set_iam_config(
                    gcs::model::bucket::IamConfig::new().set_uniform_bucket_level_access(
                        gcs::model::bucket::iam_config::UniformBucketLevelAccess::new()
                            .set_enabled(true),
                    ),
                ),
        )
        .send()
        .await?;
    println!("bucket successfully created {bucket:?}");

    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let object = client
        .write_object(&bucket.name, "hello.txt", "Hello World!")
        .send_buffered()
        .await?;
    println!("object successfully uploaded {object:?}");

    let mut reader = client.read_object(&bucket.name, "hello.txt").send().await?;
    let mut contents = Vec::new();
    while let Some(chunk) = reader.next().await.transpose()? {
        contents.extend_from_slice(&chunk);
    }
    println!(
        "object contents successfully downloaded {:?}",
        bytes::Bytes::from_owner(contents)
    );

    control
        .delete_object()
        .set_bucket(&bucket.name)
        .set_object(&object.name)
        .set_generation(object.generation)
        .send()
        .await?;
    control
        .delete_bucket()
        .set_name(&bucket.name)
        .send()
        .await?;

    Ok(())
}

Next Steps

• Push data on object uploads
• Rewriting objects
• Speed up large object downloads

Full program

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

pub async fn quickstart(project_id: &str, bucket_id: &str) -> anyhow::Result<()> {
    use google_cloud_storage as gcs;
    use google_cloud_storage::client::StorageControl;
    let control = StorageControl::builder().build().await?;
    let bucket = control
        .create_bucket()
        .set_parent("projects/_")
        .set_bucket_id(bucket_id)
        .set_bucket(
            gcs::model::Bucket::new()
                .set_project(format!("projects/{project_id}"))
                .set_iam_config(
                    gcs::model::bucket::IamConfig::new().set_uniform_bucket_level_access(
                        gcs::model::bucket::iam_config::UniformBucketLevelAccess::new()
                            .set_enabled(true),
                    ),
                ),
        )
        .send()
        .await?;
    println!("bucket successfully created {bucket:?}");

    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let object = client
        .write_object(&bucket.name, "hello.txt", "Hello World!")
        .send_buffered()
        .await?;
    println!("object successfully uploaded {object:?}");

    let mut reader = client.read_object(&bucket.name, "hello.txt").send().await?;
    let mut contents = Vec::new();
    while let Some(chunk) = reader.next().await.transpose()? {
        contents.extend_from_slice(&chunk);
    }
    println!(
        "object contents successfully downloaded {:?}",
        bytes::Bytes::from_owner(contents)
    );

    control
        .delete_object()
        .set_bucket(&bucket.name)
        .set_object(&object.name)
        .set_generation(object.generation)
        .send()
        .await?;
    control
        .delete_bucket()
        .set_name(&bucket.name)
        .send()
        .await?;

    Ok(())
}

Push data on object writes

The client API to write Cloud Storage objects pulls the payload from a type provided by the application. Some applications generate the payload in a thread and would rather "push" the object payload to the service.

This guide shows you how to write an object to Cloud Storage using a push data source.

Prerequisites

The guide assumes you have an existing Google Cloud project with billing enabled, and a Cloud Storage bucket in that project.

Add the client library as a dependency

cargo add google-cloud-storage

Convert a queue to a StreamingSource

The key idea is to use a queue to separate the task pushing new data from the task pulling the payload. This tutorial uses a Tokio mpsc queue, but you can use any queue that integrates with Tokio's async runtime.

First wrap the receiver in our own type:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

Then implement the trait required by the Google Cloud client library:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

In this tutorial you write the rest of the code in a function that accepts the bucket and object name as parameters:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}
    // ... code goes here ...
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

Initialize a client:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

Create a queue, obtaining the receiver and sender:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

Use the client to write an object with the data received from this queue. Note that we do not await the future created in the write_object() method.

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

Create a task to process the queue and write the data in the background:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

In the main task, send some data to write:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

Once you have finished sending the data, drop the sender to close the sending side of the queue:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

Now you can wait for the task to finish and extract the result:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

Full program

Putting all these steps together you get:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use google_cloud_storage::streaming_source::StreamingSource;
use tokio::sync::mpsc::{self, Receiver};
#[derive(Debug)]
struct QueueSource(Receiver<bytes::Bytes>);
impl StreamingSource for QueueSource {
    type Error = std::convert::Infallible;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0.recv().await.map(Ok)
    }
}

pub async fn queue(bucket_name: &str, object_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;

    let (sender, receiver) = mpsc::channel::<bytes::Bytes>(32);
    let upload = client
        .write_object(bucket_name, object_name, QueueSource(receiver))
        .send_buffered();
    let task = tokio::spawn(upload);

    for _ in 0..1000 {
        let line = "I will not write funny examples in class\n";
        sender
            .send(bytes::Bytes::from_static(line.as_bytes()))
            .await?;
    }
    drop(sender);
    let object = task.await??;
    println!("object successfully uploaded {object:?}");

    Ok(())
}

Rewriting objects

Rewriting a Cloud Storage object can require multiple client requests, depending on the details of the operation. In such cases, the service will return a response representing its progress, along with a rewrite_token which the client must use to continue the operation.

This guide will show you how to fully execute the rewrite loop for a Cloud Storage object.

Prerequisites

The guide assumes you have an existing Google Cloud project with billing enabled, and a Cloud Storage bucket in that project.

Add the client library as a dependency

cargo add google-cloud-storage

Rewriting an object

Prepare client

First, create a client.

The service recommends an overall timeout of at least 30 seconds. In this example, we use a RetryPolicy that does not set any timeout on the operation.

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use gcs::Result;
use gcs::builder::storage_control::RewriteObject;
use gcs::client::StorageControl;
use gcs::model::Object;
use gcs::retry_policy::RetryableErrors;
use google_cloud_gax::retry_policy::RetryPolicyExt as _;
use google_cloud_storage as gcs;

pub async fn rewrite_object(bucket_name: &str) -> anyhow::Result<()> {
    let source_object = upload(bucket_name).await?;

    let control = StorageControl::builder()
        .with_retry_policy(RetryableErrors.with_attempt_limit(5))
        .build()
        .await?;

    let mut builder = control
        .rewrite_object()
        .set_source_bucket(bucket_name)
        .set_source_object(&source_object.name)
        .set_destination_bucket(bucket_name)
        .set_destination_name("rewrite-object-clone");

    // Optionally limit the max bytes written per request.
    builder = builder.set_max_bytes_rewritten_per_call(1024 * 1024);

    // Optionally change the storage class to force GCS to copy bytes
    builder = builder.set_destination(Object::new().set_storage_class("NEARLINE"));

    let dest_object = loop {
        let progress = make_one_request(builder.clone()).await?;
        match progress {
            RewriteProgress::Incomplete(rewrite_token) => {
                builder = builder.set_rewrite_token(rewrite_token);
            }
            RewriteProgress::Done(object) => break object,
        };
    };
    println!("dest_object={dest_object:?}");

    cleanup(control, bucket_name, &source_object.name, &dest_object.name).await;
    Ok(())
}

enum RewriteProgress {
    // This holds the rewrite token
    Incomplete(String),
    Done(Box<Object>),
}

async fn make_one_request(builder: RewriteObject) -> Result<RewriteProgress> {
    let resp = builder.send().await?;
    if resp.done {
        println!(
            "DONE:     total_bytes_rewritten={}; object_size={}",
            resp.total_bytes_rewritten, resp.object_size
        );
        return Ok(RewriteProgress::Done(Box::new(
            resp.resource
                .expect("A `done` response must have an object."),
        )));
    }
    println!(
        "PROGRESS: total_bytes_rewritten={}; object_size={}",
        resp.total_bytes_rewritten, resp.object_size
    );
    Ok(RewriteProgress::Incomplete(resp.rewrite_token))
}

// Upload an object to rewrite
async fn upload(bucket_name: &str) -> anyhow::Result<Object> {
    let storage = gcs::client::Storage::builder().build().await?;
    // We need the size to exceed 1MiB to exercise the rewrite token logic.
    let payload = bytes::Bytes::from(vec![65_u8; 3 * 1024 * 1024]);
    let object = storage
        .write_object(bucket_name, "rewrite-object-source", payload)
        .send_unbuffered()
        .await?;
    Ok(object)
}

// Clean up the resources created in this sample
async fn cleanup(control: StorageControl, bucket_name: &str, o1: &str, o2: &str) {
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o1)
        .send()
        .await;
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o2)
        .send()
        .await;
    let _ = control.delete_bucket().set_name(bucket_name).send().await;
}

Prepare builder

Next we prepare the request builder.

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use gcs::Result;
use gcs::builder::storage_control::RewriteObject;
use gcs::client::StorageControl;
use gcs::model::Object;
use gcs::retry_policy::RetryableErrors;
use google_cloud_gax::retry_policy::RetryPolicyExt as _;
use google_cloud_storage as gcs;

pub async fn rewrite_object(bucket_name: &str) -> anyhow::Result<()> {
    let source_object = upload(bucket_name).await?;

    let control = StorageControl::builder()
        .with_retry_policy(RetryableErrors.with_attempt_limit(5))
        .build()
        .await?;

    let mut builder = control
        .rewrite_object()
        .set_source_bucket(bucket_name)
        .set_source_object(&source_object.name)
        .set_destination_bucket(bucket_name)
        .set_destination_name("rewrite-object-clone");

    // Optionally limit the max bytes written per request.
    builder = builder.set_max_bytes_rewritten_per_call(1024 * 1024);

    // Optionally change the storage class to force GCS to copy bytes
    builder = builder.set_destination(Object::new().set_storage_class("NEARLINE"));

    let dest_object = loop {
        let progress = make_one_request(builder.clone()).await?;
        match progress {
            RewriteProgress::Incomplete(rewrite_token) => {
                builder = builder.set_rewrite_token(rewrite_token);
            }
            RewriteProgress::Done(object) => break object,
        };
    };
    println!("dest_object={dest_object:?}");

    cleanup(control, bucket_name, &source_object.name, &dest_object.name).await;
    Ok(())
}

enum RewriteProgress {
    // This holds the rewrite token
    Incomplete(String),
    Done(Box<Object>),
}

async fn make_one_request(builder: RewriteObject) -> Result<RewriteProgress> {
    let resp = builder.send().await?;
    if resp.done {
        println!(
            "DONE:     total_bytes_rewritten={}; object_size={}",
            resp.total_bytes_rewritten, resp.object_size
        );
        return Ok(RewriteProgress::Done(Box::new(
            resp.resource
                .expect("A `done` response must have an object."),
        )));
    }
    println!(
        "PROGRESS: total_bytes_rewritten={}; object_size={}",
        resp.total_bytes_rewritten, resp.object_size
    );
    Ok(RewriteProgress::Incomplete(resp.rewrite_token))
}

// Upload an object to rewrite
async fn upload(bucket_name: &str) -> anyhow::Result<Object> {
    let storage = gcs::client::Storage::builder().build().await?;
    // We need the size to exceed 1MiB to exercise the rewrite token logic.
    let payload = bytes::Bytes::from(vec![65_u8; 3 * 1024 * 1024]);
    let object = storage
        .write_object(bucket_name, "rewrite-object-source", payload)
        .send_unbuffered()
        .await?;
    Ok(object)
}

// Clean up the resources created in this sample
async fn cleanup(control: StorageControl, bucket_name: &str, o1: &str, o2: &str) {
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o1)
        .send()
        .await;
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o2)
        .send()
        .await;
    let _ = control.delete_bucket().set_name(bucket_name).send().await;
}

Optionally, we can limit the maximum amount of bytes written per call, before the service responds with a progress report. Setting this option is an alternative to increasing the attempt timeout.

Note that the value used in this example is intentionally small to force the rewrite loop to take multiple iterations. In practice, you would likely use a larger value.

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use gcs::Result;
use gcs::builder::storage_control::RewriteObject;
use gcs::client::StorageControl;
use gcs::model::Object;
use gcs::retry_policy::RetryableErrors;
use google_cloud_gax::retry_policy::RetryPolicyExt as _;
use google_cloud_storage as gcs;

pub async fn rewrite_object(bucket_name: &str) -> anyhow::Result<()> {
    let source_object = upload(bucket_name).await?;

    let control = StorageControl::builder()
        .with_retry_policy(RetryableErrors.with_attempt_limit(5))
        .build()
        .await?;

    let mut builder = control
        .rewrite_object()
        .set_source_bucket(bucket_name)
        .set_source_object(&source_object.name)
        .set_destination_bucket(bucket_name)
        .set_destination_name("rewrite-object-clone");

    // Optionally limit the max bytes written per request.
    builder = builder.set_max_bytes_rewritten_per_call(1024 * 1024);

    // Optionally change the storage class to force GCS to copy bytes
    builder = builder.set_destination(Object::new().set_storage_class("NEARLINE"));

    let dest_object = loop {
        let progress = make_one_request(builder.clone()).await?;
        match progress {
            RewriteProgress::Incomplete(rewrite_token) => {
                builder = builder.set_rewrite_token(rewrite_token);
            }
            RewriteProgress::Done(object) => break object,
        };
    };
    println!("dest_object={dest_object:?}");

    cleanup(control, bucket_name, &source_object.name, &dest_object.name).await;
    Ok(())
}

enum RewriteProgress {
    // This holds the rewrite token
    Incomplete(String),
    Done(Box<Object>),
}

async fn make_one_request(builder: RewriteObject) -> Result<RewriteProgress> {
    let resp = builder.send().await?;
    if resp.done {
        println!(
            "DONE:     total_bytes_rewritten={}; object_size={}",
            resp.total_bytes_rewritten, resp.object_size
        );
        return Ok(RewriteProgress::Done(Box::new(
            resp.resource
                .expect("A `done` response must have an object."),
        )));
    }
    println!(
        "PROGRESS: total_bytes_rewritten={}; object_size={}",
        resp.total_bytes_rewritten, resp.object_size
    );
    Ok(RewriteProgress::Incomplete(resp.rewrite_token))
}

// Upload an object to rewrite
async fn upload(bucket_name: &str) -> anyhow::Result<Object> {
    let storage = gcs::client::Storage::builder().build().await?;
    // We need the size to exceed 1MiB to exercise the rewrite token logic.
    let payload = bytes::Bytes::from(vec![65_u8; 3 * 1024 * 1024]);
    let object = storage
        .write_object(bucket_name, "rewrite-object-source", payload)
        .send_unbuffered()
        .await?;
    Ok(object)
}

// Clean up the resources created in this sample
async fn cleanup(control: StorageControl, bucket_name: &str, o1: &str, o2: &str) {
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o1)
        .send()
        .await;
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o2)
        .send()
        .await;
    let _ = control.delete_bucket().set_name(bucket_name).send().await;
}

Rewriting an object allows you to copy its data to a different bucket, copy its data to a different object in the same bucket, change its encryption key, and/or change its storage class. The rewrite loop is identical for all these transformations. We will change the storage class to illustrate the code.

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use gcs::Result;
use gcs::builder::storage_control::RewriteObject;
use gcs::client::StorageControl;
use gcs::model::Object;
use gcs::retry_policy::RetryableErrors;
use google_cloud_gax::retry_policy::RetryPolicyExt as _;
use google_cloud_storage as gcs;

pub async fn rewrite_object(bucket_name: &str) -> anyhow::Result<()> {
    let source_object = upload(bucket_name).await?;

    let control = StorageControl::builder()
        .with_retry_policy(RetryableErrors.with_attempt_limit(5))
        .build()
        .await?;

    let mut builder = control
        .rewrite_object()
        .set_source_bucket(bucket_name)
        .set_source_object(&source_object.name)
        .set_destination_bucket(bucket_name)
        .set_destination_name("rewrite-object-clone");

    // Optionally limit the max bytes written per request.
    builder = builder.set_max_bytes_rewritten_per_call(1024 * 1024);

    // Optionally change the storage class to force GCS to copy bytes
    builder = builder.set_destination(Object::new().set_storage_class("NEARLINE"));

    let dest_object = loop {
        let progress = make_one_request(builder.clone()).await?;
        match progress {
            RewriteProgress::Incomplete(rewrite_token) => {
                builder = builder.set_rewrite_token(rewrite_token);
            }
            RewriteProgress::Done(object) => break object,
        };
    };
    println!("dest_object={dest_object:?}");

    cleanup(control, bucket_name, &source_object.name, &dest_object.name).await;
    Ok(())
}

enum RewriteProgress {
    // This holds the rewrite token
    Incomplete(String),
    Done(Box<Object>),
}

async fn make_one_request(builder: RewriteObject) -> Result<RewriteProgress> {
    let resp = builder.send().await?;
    if resp.done {
        println!(
            "DONE:     total_bytes_rewritten={}; object_size={}",
            resp.total_bytes_rewritten, resp.object_size
        );
        return Ok(RewriteProgress::Done(Box::new(
            resp.resource
                .expect("A `done` response must have an object."),
        )));
    }
    println!(
        "PROGRESS: total_bytes_rewritten={}; object_size={}",
        resp.total_bytes_rewritten, resp.object_size
    );
    Ok(RewriteProgress::Incomplete(resp.rewrite_token))
}

// Upload an object to rewrite
async fn upload(bucket_name: &str) -> anyhow::Result<Object> {
    let storage = gcs::client::Storage::builder().build().await?;
    // We need the size to exceed 1MiB to exercise the rewrite token logic.
    let payload = bytes::Bytes::from(vec![65_u8; 3 * 1024 * 1024]);
    let object = storage
        .write_object(bucket_name, "rewrite-object-source", payload)
        .send_unbuffered()
        .await?;
    Ok(object)
}

// Clean up the resources created in this sample
async fn cleanup(control: StorageControl, bucket_name: &str, o1: &str, o2: &str) {
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o1)
        .send()
        .await;
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o2)
        .send()
        .await;
    let _ = control.delete_bucket().set_name(bucket_name).send().await;
}

Note that there is a minimum storage duration associated with the new storage class. While the object used in this example (3 MiB) incurs less than $0.001 of cost, the billing may be noticeable for larger objects.

Introduce rewrite loop helpers

Next, we introduce a helper function to perform one iteration of the rewrite loop.

We send the request and process the response. We log the progress made.

If the operation is done, we return the object metadata, otherwise we return the rewrite token.

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use gcs::Result;
use gcs::builder::storage_control::RewriteObject;
use gcs::client::StorageControl;
use gcs::model::Object;
use gcs::retry_policy::RetryableErrors;
use google_cloud_gax::retry_policy::RetryPolicyExt as _;
use google_cloud_storage as gcs;

pub async fn rewrite_object(bucket_name: &str) -> anyhow::Result<()> {
    let source_object = upload(bucket_name).await?;

    let control = StorageControl::builder()
        .with_retry_policy(RetryableErrors.with_attempt_limit(5))
        .build()
        .await?;

    let mut builder = control
        .rewrite_object()
        .set_source_bucket(bucket_name)
        .set_source_object(&source_object.name)
        .set_destination_bucket(bucket_name)
        .set_destination_name("rewrite-object-clone");

    // Optionally limit the max bytes written per request.
    builder = builder.set_max_bytes_rewritten_per_call(1024 * 1024);

    // Optionally change the storage class to force GCS to copy bytes
    builder = builder.set_destination(Object::new().set_storage_class("NEARLINE"));

    let dest_object = loop {
        let progress = make_one_request(builder.clone()).await?;
        match progress {
            RewriteProgress::Incomplete(rewrite_token) => {
                builder = builder.set_rewrite_token(rewrite_token);
            }
            RewriteProgress::Done(object) => break object,
        };
    };
    println!("dest_object={dest_object:?}");

    cleanup(control, bucket_name, &source_object.name, &dest_object.name).await;
    Ok(())
}

enum RewriteProgress {
    // This holds the rewrite token
    Incomplete(String),
    Done(Box<Object>),
}

async fn make_one_request(builder: RewriteObject) -> Result<RewriteProgress> {
    let resp = builder.send().await?;
    if resp.done {
        println!(
            "DONE:     total_bytes_rewritten={}; object_size={}",
            resp.total_bytes_rewritten, resp.object_size
        );
        return Ok(RewriteProgress::Done(Box::new(
            resp.resource
                .expect("A `done` response must have an object."),
        )));
    }
    println!(
        "PROGRESS: total_bytes_rewritten={}; object_size={}",
        resp.total_bytes_rewritten, resp.object_size
    );
    Ok(RewriteProgress::Incomplete(resp.rewrite_token))
}

// Upload an object to rewrite
async fn upload(bucket_name: &str) -> anyhow::Result<Object> {
    let storage = gcs::client::Storage::builder().build().await?;
    // We need the size to exceed 1MiB to exercise the rewrite token logic.
    let payload = bytes::Bytes::from(vec![65_u8; 3 * 1024 * 1024]);
    let object = storage
        .write_object(bucket_name, "rewrite-object-source", payload)
        .send_unbuffered()
        .await?;
    Ok(object)
}

// Clean up the resources created in this sample
async fn cleanup(control: StorageControl, bucket_name: &str, o1: &str, o2: &str) {
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o1)
        .send()
        .await;
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o2)
        .send()
        .await;
    let _ = control.delete_bucket().set_name(bucket_name).send().await;
}

Execute rewrite loop

Now we are ready to perform the rewrite loop until the operation is done.

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use gcs::Result;
use gcs::builder::storage_control::RewriteObject;
use gcs::client::StorageControl;
use gcs::model::Object;
use gcs::retry_policy::RetryableErrors;
use google_cloud_gax::retry_policy::RetryPolicyExt as _;
use google_cloud_storage as gcs;

pub async fn rewrite_object(bucket_name: &str) -> anyhow::Result<()> {
    let source_object = upload(bucket_name).await?;

    let control = StorageControl::builder()
        .with_retry_policy(RetryableErrors.with_attempt_limit(5))
        .build()
        .await?;

    let mut builder = control
        .rewrite_object()
        .set_source_bucket(bucket_name)
        .set_source_object(&source_object.name)
        .set_destination_bucket(bucket_name)
        .set_destination_name("rewrite-object-clone");

    // Optionally limit the max bytes written per request.
    builder = builder.set_max_bytes_rewritten_per_call(1024 * 1024);

    // Optionally change the storage class to force GCS to copy bytes
    builder = builder.set_destination(Object::new().set_storage_class("NEARLINE"));

    let dest_object = loop {
        let progress = make_one_request(builder.clone()).await?;
        match progress {
            RewriteProgress::Incomplete(rewrite_token) => {
                builder = builder.set_rewrite_token(rewrite_token);
            }
            RewriteProgress::Done(object) => break object,
        };
    };
    println!("dest_object={dest_object:?}");

    cleanup(control, bucket_name, &source_object.name, &dest_object.name).await;
    Ok(())
}

enum RewriteProgress {
    // This holds the rewrite token
    Incomplete(String),
    Done(Box<Object>),
}

async fn make_one_request(builder: RewriteObject) -> Result<RewriteProgress> {
    let resp = builder.send().await?;
    if resp.done {
        println!(
            "DONE:     total_bytes_rewritten={}; object_size={}",
            resp.total_bytes_rewritten, resp.object_size
        );
        return Ok(RewriteProgress::Done(Box::new(
            resp.resource
                .expect("A `done` response must have an object."),
        )));
    }
    println!(
        "PROGRESS: total_bytes_rewritten={}; object_size={}",
        resp.total_bytes_rewritten, resp.object_size
    );
    Ok(RewriteProgress::Incomplete(resp.rewrite_token))
}

// Upload an object to rewrite
async fn upload(bucket_name: &str) -> anyhow::Result<Object> {
    let storage = gcs::client::Storage::builder().build().await?;
    // We need the size to exceed 1MiB to exercise the rewrite token logic.
    let payload = bytes::Bytes::from(vec![65_u8; 3 * 1024 * 1024]);
    let object = storage
        .write_object(bucket_name, "rewrite-object-source", payload)
        .send_unbuffered()
        .await?;
    Ok(object)
}

// Clean up the resources created in this sample
async fn cleanup(control: StorageControl, bucket_name: &str, o1: &str, o2: &str) {
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o1)
        .send()
        .await;
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o2)
        .send()
        .await;
    let _ = control.delete_bucket().set_name(bucket_name).send().await;
}

Note how if the operation is incomplete, we supply the rewrite token returned by the server to the next request.

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use gcs::Result;
use gcs::builder::storage_control::RewriteObject;
use gcs::client::StorageControl;
use gcs::model::Object;
use gcs::retry_policy::RetryableErrors;
use google_cloud_gax::retry_policy::RetryPolicyExt as _;
use google_cloud_storage as gcs;

pub async fn rewrite_object(bucket_name: &str) -> anyhow::Result<()> {
    let source_object = upload(bucket_name).await?;

    let control = StorageControl::builder()
        .with_retry_policy(RetryableErrors.with_attempt_limit(5))
        .build()
        .await?;

    let mut builder = control
        .rewrite_object()
        .set_source_bucket(bucket_name)
        .set_source_object(&source_object.name)
        .set_destination_bucket(bucket_name)
        .set_destination_name("rewrite-object-clone");

    // Optionally limit the max bytes written per request.
    builder = builder.set_max_bytes_rewritten_per_call(1024 * 1024);

    // Optionally change the storage class to force GCS to copy bytes
    builder = builder.set_destination(Object::new().set_storage_class("NEARLINE"));

    let dest_object = loop {
        let progress = make_one_request(builder.clone()).await?;
        match progress {
            RewriteProgress::Incomplete(rewrite_token) => {
                builder = builder.set_rewrite_token(rewrite_token);
            }
            RewriteProgress::Done(object) => break object,
        };
    };
    println!("dest_object={dest_object:?}");

    cleanup(control, bucket_name, &source_object.name, &dest_object.name).await;
    Ok(())
}

enum RewriteProgress {
    // This holds the rewrite token
    Incomplete(String),
    Done(Box<Object>),
}

async fn make_one_request(builder: RewriteObject) -> Result<RewriteProgress> {
    let resp = builder.send().await?;
    if resp.done {
        println!(
            "DONE:     total_bytes_rewritten={}; object_size={}",
            resp.total_bytes_rewritten, resp.object_size
        );
        return Ok(RewriteProgress::Done(Box::new(
            resp.resource
                .expect("A `done` response must have an object."),
        )));
    }
    println!(
        "PROGRESS: total_bytes_rewritten={}; object_size={}",
        resp.total_bytes_rewritten, resp.object_size
    );
    Ok(RewriteProgress::Incomplete(resp.rewrite_token))
}

// Upload an object to rewrite
async fn upload(bucket_name: &str) -> anyhow::Result<Object> {
    let storage = gcs::client::Storage::builder().build().await?;
    // We need the size to exceed 1MiB to exercise the rewrite token logic.
    let payload = bytes::Bytes::from(vec![65_u8; 3 * 1024 * 1024]);
    let object = storage
        .write_object(bucket_name, "rewrite-object-source", payload)
        .send_unbuffered()
        .await?;
    Ok(object)
}

// Clean up the resources created in this sample
async fn cleanup(control: StorageControl, bucket_name: &str, o1: &str, o2: &str) {
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o1)
        .send()
        .await;
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o2)
        .send()
        .await;
    let _ = control.delete_bucket().set_name(bucket_name).send().await;
}

Also note that the rewrite token can be used to continue the operation from another process. Rewrite tokens are valid for up to one week.

Full program

Putting all these steps together you get:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use gcs::Result;
use gcs::builder::storage_control::RewriteObject;
use gcs::client::StorageControl;
use gcs::model::Object;
use gcs::retry_policy::RetryableErrors;
use google_cloud_gax::retry_policy::RetryPolicyExt as _;
use google_cloud_storage as gcs;

pub async fn rewrite_object(bucket_name: &str) -> anyhow::Result<()> {
    let source_object = upload(bucket_name).await?;

    let control = StorageControl::builder()
        .with_retry_policy(RetryableErrors.with_attempt_limit(5))
        .build()
        .await?;

    let mut builder = control
        .rewrite_object()
        .set_source_bucket(bucket_name)
        .set_source_object(&source_object.name)
        .set_destination_bucket(bucket_name)
        .set_destination_name("rewrite-object-clone");

    // Optionally limit the max bytes written per request.
    builder = builder.set_max_bytes_rewritten_per_call(1024 * 1024);

    // Optionally change the storage class to force GCS to copy bytes
    builder = builder.set_destination(Object::new().set_storage_class("NEARLINE"));

    let dest_object = loop {
        let progress = make_one_request(builder.clone()).await?;
        match progress {
            RewriteProgress::Incomplete(rewrite_token) => {
                builder = builder.set_rewrite_token(rewrite_token);
            }
            RewriteProgress::Done(object) => break object,
        };
    };
    println!("dest_object={dest_object:?}");

    cleanup(control, bucket_name, &source_object.name, &dest_object.name).await;
    Ok(())
}

enum RewriteProgress {
    // This holds the rewrite token
    Incomplete(String),
    Done(Box<Object>),
}

async fn make_one_request(builder: RewriteObject) -> Result<RewriteProgress> {
    let resp = builder.send().await?;
    if resp.done {
        println!(
            "DONE:     total_bytes_rewritten={}; object_size={}",
            resp.total_bytes_rewritten, resp.object_size
        );
        return Ok(RewriteProgress::Done(Box::new(
            resp.resource
                .expect("A `done` response must have an object."),
        )));
    }
    println!(
        "PROGRESS: total_bytes_rewritten={}; object_size={}",
        resp.total_bytes_rewritten, resp.object_size
    );
    Ok(RewriteProgress::Incomplete(resp.rewrite_token))
}

// Upload an object to rewrite
async fn upload(bucket_name: &str) -> anyhow::Result<Object> {
    let storage = gcs::client::Storage::builder().build().await?;
    // We need the size to exceed 1MiB to exercise the rewrite token logic.
    let payload = bytes::Bytes::from(vec![65_u8; 3 * 1024 * 1024]);
    let object = storage
        .write_object(bucket_name, "rewrite-object-source", payload)
        .send_unbuffered()
        .await?;
    Ok(object)
}

// Clean up the resources created in this sample
async fn cleanup(control: StorageControl, bucket_name: &str, o1: &str, o2: &str) {
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o1)
        .send()
        .await;
    let _ = control
        .delete_object()
        .set_bucket(bucket_name)
        .set_object(o2)
        .send()
        .await;
    let _ = control.delete_bucket().set_name(bucket_name).send().await;
}

We should see output similar to:

PROGRESS: total_bytes_rewritten=1048576; object_size=3145728
PROGRESS: total_bytes_rewritten=2097152; object_size=3145728
DONE:     total_bytes_rewritten=3145728; object_size=3145728
dest_object=Object { name: "rewrite-object-clone", ... }

Speed up large object downloads

In this tutorial you will learn how to use striped downloads to speed up downloads of large Cloud Storage objects.

Prerequisites

The guide assumes you have an existing Google Cloud project with billing enabled, and a Cloud Storage bucket in that project.

You will create some large objects during this tutorial, remember to clean up any resources to avoid excessive billing.

The tutorial assumes you are familiar with the basics of using the client library. If not, read the quickstart guide.

Add the client library as a dependency

cargo add google-cloud-storage

Create source data

To run this tutorial you will need some large objects in Cloud Storage. You can create such objects by seeding a smaller object and then repeatedly composing it to create objects of the desired size.

You can put all the code for seeding the data in its own function. This function will receive the storage and storage control clients as parameters. For information on how to create these clients, consult the quickstart guide:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}
    // ... details omitted ...
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

As usual, the function starts with some use declarations to simplify the code:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

Using the storage client, you create a 1MiB object:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

Then you use the storage control client to concatenate 32 copies of this object into a larger object. This operation does not require transferring any object data to the client, it is performed by the service:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

You can repeat the operation to create larger and larger objects:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

Striped downloads

Again, write a function to perform the striped download:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}
    // ... details below ...
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

Use the storage control client to query the object metadata. This metadata includes the object size and the current generation:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

We will split the download of each stripe to a separate function. You will see the details of this function in a moment, for now just note that it is async, so it returns a Future:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}
    // ... details below ...
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

You can compute the size of each stripe and then call write_stripe() to download each of these stripes. Then collect the results into a vector:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

You can use the standard Rust facilities to concurrently await all these futures:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

Once they complete, the file is downloaded.

Now you should complete writing the write_stripe() function. First, duplicate the write object and prepare to write starting at the desired offset:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

Start a download from Cloud Storage:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

To restrict the download to the desired stripe, use .with_read_offset() and .with_read_limit():

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

You may also want to restrict the download to the right object generation. This will avoid race conditions where another process writes over the object and you get inconsistent reads:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

Then you read the data and write it to the local file:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

Next steps

• Consider optimizing the case where the last stripe only has a few bytes

Expected performance

The performance of these downloads depends on:

• The I/O subsystem: if your local storage is not fast enough the downloads will be throttled by the writes to disk.
• The configuration of your VM: if you do not have enough CPUs, the downloads will be throttled on trying to decrypt the on data, as Cloud Storage and the client library always encrypt the data in transit.
• The location of the bucket and the particular object: the bucket may store all of the objects (or some objects) in a region different from your VM's location. In this case, you may be throttled by the wide-area network capacity.

With a large enough VM, using SSD for disk, and with a bucket in the same region as the VM you should get close to 1,000 MiB/s of effective throughput.

Full program

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


use google_cloud_storage::client::Storage;
use google_cloud_storage::client::StorageControl;
use google_cloud_storage::model::Object;

async fn seed(client: Storage, control: StorageControl, bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::model::compose_object_request::SourceObject;

    let buffer = String::from_iter(('a'..='z').cycle().take(1024 * 1024));
    let seed = client
        .write_object(bucket_name, "1MiB.txt", bytes::Bytes::from_owner(buffer))
        .send_unbuffered()
        .await?;
    println!(
        "Uploaded object {}, size={}KiB",
        seed.name,
        seed.size / 1024
    );

    let seed_32 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("32MiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed.name)
                .set_generation(seed.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    let seed_1024 = control
        .compose_object()
        .set_destination(Object::new().set_bucket(bucket_name).set_name("1GiB.txt"))
        .set_source_objects((0..32).map(|_| {
            SourceObject::new()
                .set_name(&seed_32.name)
                .set_generation(seed_32.generation)
        }))
        .send()
        .await?;
    println!(
        "Created object {}, size={}MiB",
        seed.name,
        seed.size / (1024 * 1024)
    );

    for s in [2, 4, 8, 16, 32] {
        let name = format!("{s}GiB.txt");
        let target = control
            .compose_object()
            .set_destination(Object::new().set_bucket(bucket_name).set_name(&name))
            .set_source_objects((0..s).map(|_| {
                SourceObject::new()
                    .set_name(&seed_1024.name)
                    .set_generation(seed_1024.generation)
            }))
            .send()
            .await?;
        println!(
            "Created object {} size={} MiB",
            target.name,
            target.size / (1024 * 1024)
        );
    }

    Ok(())
}

async fn download(
    client: Storage,
    control: StorageControl,
    bucket_name: &str,
    object_name: &str,
    stripe_size: usize,
    destination: &str,
) -> anyhow::Result<()> {
    let metadata = control
        .get_object()
        .set_bucket(bucket_name)
        .set_object(object_name)
        .send()
        .await?;

    let file = tokio::fs::File::create(destination).await?;
    let start = std::time::Instant::now();

    let size = metadata.size as u64;
    let limit = stripe_size as u64;
    let count = size / limit;
    let mut stripes = (0..count)
        .map(|i| write_stripe(client.clone(), &file, i * limit, limit, &metadata))
        .collect::<Vec<_>>();
    if size % limit != 0 {
        stripes.push(write_stripe(
            client.clone(),
            &file,
            count * limit,
            limit,
            &metadata,
        ))
    }

    futures::future::join_all(stripes)
        .await
        .into_iter()
        .collect::<anyhow::Result<Vec<_>>>()?;

    let elapsed = std::time::Instant::now() - start;
    let mib = metadata.size as f64 / (1024.0 * 1024.0);
    let bw = mib / elapsed.as_secs_f64();
    println!(
        "Completed {mib:.2} MiB download in {elapsed:?}, using {count} stripes, effective bandwidth = {bw:.2} MiB/s"
    );

    Ok(())
}

async fn write_stripe(
    client: Storage,
    file: &tokio::fs::File,
    offset: u64,
    limit: u64,
    metadata: &Object,
) -> anyhow::Result<()> {
    use google_cloud_storage::model_ext::ReadRange;
    use tokio::io::AsyncSeekExt;
    let mut writer = file.try_clone().await?;
    writer.seek(std::io::SeekFrom::Start(offset)).await?;
    let mut reader = client
        .read_object(&metadata.bucket, &metadata.name)
        .set_generation(metadata.generation)
        .set_read_range(ReadRange::segment(offset, limit))
        .send()
        .await?;
    while let Some(b) = reader.next().await.transpose()? {
        use tokio::io::AsyncWriteExt;
        writer.write_all(&b).await?;
    }
    Ok(())
}

pub async fn test(bucket_name: &str, destination: &str) -> anyhow::Result<()> {
    const MB: usize = 1024 * 1024;
    let client = Storage::builder().build().await?;
    let control = StorageControl::builder().build().await?;
    seed(client.clone(), control.clone(), bucket_name).await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        32 * MB,
        destination,
    )
    .await?;
    download(
        client.clone(),
        control.clone(),
        bucket_name,
        "32MiB.txt",
        MB,
        destination,
    )
    .await?;
    #[cfg(feature = "run-large-downloads")]
    {
        let destination = "/dev/shm/output";
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "32GiB.txt",
            256 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            1024 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "1GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "4GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "8GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
        download(
            client.clone(),
            control.clone(),
            bucket_name,
            "16GiB.txt",
            512 * MB,
            destination,
        )
        .await?;
    }
    Ok(())
}

Use errors to terminate object writes

In this guide you will learn how to use errors and custom data sources to terminate an object write before it is finalized. This is useful when applications want to stop the client library from finalizing the object creation if there is some error condition.

Prerequisites

The guide assumes you have an existing Google Cloud project with billing enabled, and a Cloud Storage bucket in that project.

The tutorial assumes you are familiar with the basics of using the client library. If not, you may want to read the quickstart guide first.

Add the client library as a dependency

cargo add google-cloud-storage

Overview

The client library creates objects from any type implementing the StreamingSource trait. The client library pulls data from implementations of the trait. The library terminates the object write on the first error.

In this guide you will build a custom implementation of StreamingSource that returns some data and then stops on an error. You will verify that an object write using this custom data source returns an error.

Create a custom error type

To terminate an object write without finalizing it, your StreamingSource must return an error. In this example you will create a simple error type, in your application code you can use any existing error type:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

The client library requires that your custom error type implements the standard Error trait:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

As you may recall, that requires implementing Display too:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

Create a custom StreamingSource

Create a type that generates the data for your object. In this example you will use synthetic data using a counter:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

Implement the streaming source trait for your type:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}
    // ... more details below ...
}

Define the error type:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

And implement the main function in this trait. Note how this function will (eventually) return the error type you defined above:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

Create the object

You will need a client to interact with Cloud Storage:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

Use the custom type to create the object:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

As expected, this object write fails. You can inspect the error details:

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

Next Steps

• Push data on object writes

Full Program

// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#[derive(Debug)]
pub enum MyError {
    ExpectedProblem,
    OhNoes,
}

impl std::error::Error for MyError {}

impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::ExpectedProblem => write!(f, "this kind of thing happens"),
            Self::OhNoes => write!(f, "oh noes! something terrible happened"),
        }
    }
}

#[derive(Debug, Default)]
struct MySource(u32);

impl google_cloud_storage::streaming_source::StreamingSource for MySource {
    type Error = MyError;
    async fn next(&mut self) -> Option<Result<bytes::Bytes, Self::Error>> {
        self.0 += 1;
        match self.0 {
            42 => Some(Err(MyError::ExpectedProblem)),
            n if n > 42 => None,
            n => Some(Ok(bytes::Bytes::from_owner(format!(
                "test data for the example {n}\n"
            )))),
        }
    }
}

pub async fn attempt_upload(bucket_name: &str) -> anyhow::Result<()> {
    use google_cloud_storage::client::Storage;
    let client = Storage::builder().build().await?;
    let upload = client
        .write_object(bucket_name, "expect-error", MySource::default())
        .send_buffered()
        .await;
    println!("Upload result {upload:?}");
    let err = upload.expect_err("the source is supposed to terminate the upload");
    assert!(err.is_serialization(), "{err:?}");
    use std::error::Error as _;
    assert!(err.source().is_some_and(|e| e.is::<MyError>()), "{err:?}");
    Ok(())
}

Update a resource using a field mask

This guide shows you how to update a resource using a field mask, so that you can control which fields on the resource will be updated. The guide uses a secret from Secret Manager as a resource, but the concepts apply to other resources and services as well.

Prerequisites

To complete this tutorial, you need a Rust development environment with the following dependencies installed:

• The Secret Manager client library
• Tokio

To get set up, follow the steps in Setting up your development environment.

Install well known types

The google_cloud_wkt crate contains well known types for Google Cloud APIs. These types typically have custom JSON encoding, and may provide conversion functions to and from native or commonly used Rust types. google_cloud_wkt contains the field mask type, FieldMask, so you'll need to add the crate as a dependency:

cargo add google-cloud-wkt

FieldMask

A FieldMask represents a set of symbolic field paths. Field masks are used to specify a subset of fields that should be returned by a get operation or modified by an update operation.

A field mask in an update operation specifies which fields of the targeted resource should be updated. The API is required to change only the values of the fields specified in the mask and leave the others untouched. If a resource is passed in to describe the updated values, the API ignores the values of all fields not covered by the mask. If a field mask is not present on update, the operation applies to all fields (as if a field mask of all fields had been specified).

In order to reset a field to the default value, you must include the field in the mask and set the default value in the provided resource. Thus, in order to reset all fields of a resource, provide a default instance of the resource and set all fields in the mask, or don't provide a mask.

Update fields on a resource

First, initialize a Secret Manager client and create a secret:

    let client = SecretManagerService::builder().build().await?;

    let secret = client
        .create_secret()
        .set_parent(format!("projects/{project_id}"))
        .set_secret_id("your-secret")
        .set_secret(model::Secret::new().set_replication(
            model::Replication::new().set_automatic(model::replication::Automatic::new()),
        ))
        .send()
        .await?;
    println!("CREATE = {secret:?}");

If you examine the output from the create operation, you'll see that both the labels and annotations fields are empty.

The following code updates the labels and annotations fields:

    let tag = |mut labels: HashMap<_, _>, msg: &str| {
        labels.insert("updated".to_string(), msg.to_string());
        labels
    };

    let update = client
        .update_secret()
        .set_secret(
            model::Secret::new()
                .set_name(&secret.name)
                .set_etag(secret.etag)
                .set_labels(tag(secret.labels, "your-label"))
                .set_annotations(tag(secret.annotations, "your-annotations")),
        )
        .set_update_mask(
            google_cloud_wkt::FieldMask::default().set_paths(["annotations", "labels"]),
        )
        .send()
        .await?;
    println!("UPDATE = {update:?}");

The set_etag method lets you set an etag on the secret, which prevents overwriting concurrent updates.

Having set labels and annotations on the updated secret, you pass a field mask to set_update_mask specifying the field paths to be updated:

        .set_update_mask(
            google_cloud_wkt::FieldMask::default().set_paths(["annotations", "labels"]),
        )

In the output from the update operation, you can see that the fields have been updated:

labels: {"updated": "your-label"},
...
annotations: {"updated": "your-annotations"},

See below for the complete code.

What's next

In this guide, you updated a resource using a field mask. The sample code uses the Secret Manager API, but you can use field masks with other clients too. Try one of the other Cloud Client Libraries for Rust:

• Generate text using the Vertex AI Gemini API
• Using Google Cloud Storage: Push data on object uploads

Update field: complete code

pub async fn update_field(project_id: &str) -> anyhow::Result<()> {
    use google_cloud_secretmanager_v1::client::SecretManagerService;
    use std::collections::HashMap;

    let client = SecretManagerService::builder().build().await?;

    let secret = client
        .create_secret()
        .set_parent(format!("projects/{project_id}"))
        .set_secret_id("your-secret")
        .set_secret(model::Secret::new().set_replication(
            model::Replication::new().set_automatic(model::replication::Automatic::new()),
        ))
        .send()
        .await?;
    println!("CREATE = {secret:?}");

    let tag = |mut labels: HashMap<_, _>, msg: &str| {
        labels.insert("updated".to_string(), msg.to_string());
        labels
    };

    let update = client
        .update_secret()
        .set_secret(
            model::Secret::new()
                .set_name(&secret.name)
                .set_etag(secret.etag)
                .set_labels(tag(secret.labels, "your-label"))
                .set_annotations(tag(secret.annotations, "your-annotations")),
        )
        .set_update_mask(
            google_cloud_wkt::FieldMask::default().set_paths(["annotations", "labels"]),
        )
        .send()
        .await?;
    println!("UPDATE = {update:?}");

    Ok(())
}

Configuring Retry Policies

The Google Cloud client libraries for Rust can automatically retry operations that fail due to transient errors.

This guide shows you how to customize the retry loop. First you'll learn how to enable a common retry policy for all requests in a client, and then how to override this default for a specific request.

Prerequisites

The guide uses the Secret Manager service. That makes the examples more concrete and therefore easier to follow. With that said, the same ideas work for any other service.

You may want to follow the service quickstart. This guide will walk you through the steps necessary to enable the service, ensure you have logged in, and that your account has the necessary permissions.

Dependencies

As usual with Rust, you must declare dependencies in your Cargo.toml file:

cargo add google-cloud-secretmanager-v1

Configuring the default retry policy

This example uses the Aip194Strict policy. This policy is based on the guidelines in AIP-194, which documents the conditions under which a Google API client should automatically retry a request. The policy is fairly conservative, and will not retry any error that indicates the request may have reached the service, unless the request is idempotent. As such, the policy is safe to use as a default. The only downside may be additional requests to the service, consuming some quota and billing.

To make this the default policy for the service, set the policy during the client initialization:

    let client = secret_manager::client::SecretManagerService::builder()
        .with_retry_policy(Aip194Strict)
        .build()
        .await?;

Then use the service as usual:

    let mut list = client
        .list_secrets()
        .set_parent(format!("projects/{project_id}"))
        .by_item();
    while let Some(secret) = list.next().await {
        let secret = secret?;
        println!("  secret={}", secret.name);
    }

See below for the complete code.

Configuring the default retry policy with limits

The Aip194Strict policy does not limit the number of retry attempts or the time spent retrying requests. However, it can be decorated to set such limits. For example, you can limit both the number of attempts and the time spent in the retry loop using:

    let client = secret_manager::client::SecretManagerService::builder()
        .with_retry_policy(
            Aip194Strict
                .with_attempt_limit(5)
                .with_time_limit(Duration::from_secs(15)),
        )
        .build()
        .await?;

Requests work as usual too:

    let mut list = client
        .list_secrets()
        .set_parent(format!("projects/{project_id}"))
        .by_item();
    while let Some(secret) = list.next().await {
        let secret = secret?;
        println!("  secret={}", secret.name);
    }

See below for the complete code.

Override the retry policy for one request

Sometimes applications need to override the retry policy for a specific request. For example, the application developer may know specific details of the service or application and determine it is safe to tolerate more errors.

For example, deleting a secret is idempotent, because it can only succeed once. But the client library assumes all delete operations are unsafe. The application can override the policy for one request:

    client
        .delete_secret()
        .set_name(format!("projects/{project_id}/secrets/{secret_id}"))
        .with_retry_policy(
            AlwaysRetry
                .with_attempt_limit(5)
                .with_time_limit(Duration::from_secs(15)),
        )
        .send()
        .await?;

See below for the complete code.

Configuring the default retry policy: complete code

pub async fn client_retry(project_id: &str) -> crate::Result<()> {
    use google_cloud_gax::paginator::ItemPaginator as _;
    use google_cloud_gax::retry_policy::Aip194Strict;
    use google_cloud_secretmanager_v1 as secret_manager;

    let client = secret_manager::client::SecretManagerService::builder()
        .with_retry_policy(Aip194Strict)
        .build()
        .await?;

    let mut list = client
        .list_secrets()
        .set_parent(format!("projects/{project_id}"))
        .by_item();
    while let Some(secret) = list.next().await {
        let secret = secret?;
        println!("  secret={}", secret.name);
    }

    Ok(())
}

Configuring the default retry policy with limits: complete code

pub async fn client_retry_full(project_id: &str) -> crate::Result<()> {
    use google_cloud_gax::paginator::ItemPaginator as _;
    use google_cloud_gax::retry_policy::Aip194Strict;
    use google_cloud_gax::retry_policy::RetryPolicyExt;
    use google_cloud_secretmanager_v1 as secret_manager;
    use std::time::Duration;

    let client = secret_manager::client::SecretManagerService::builder()
        .with_retry_policy(
            Aip194Strict
                .with_attempt_limit(5)
                .with_time_limit(Duration::from_secs(15)),
        )
        .build()
        .await?;

    let mut list = client
        .list_secrets()
        .set_parent(format!("projects/{project_id}"))
        .by_item();
    while let Some(secret) = list.next().await {
        let secret = secret?;
        println!("  secret={}", secret.name);
    }

    Ok(())
}

Override the retry policy for one request: complete code

use google_cloud_secretmanager_v1 as secret_manager;
pub async fn request_retry(
    client: &secret_manager::client::SecretManagerService,
    project_id: &str,
    secret_id: &str,
) -> crate::Result<()> {
    use google_cloud_gax::options::RequestOptionsBuilder;
    use google_cloud_gax::retry_policy::AlwaysRetry;
    use google_cloud_gax::retry_policy::RetryPolicyExt;
    use std::time::Duration;

    client
        .delete_secret()
        .set_name(format!("projects/{project_id}/secrets/{secret_id}"))
        .with_retry_policy(
            AlwaysRetry
                .with_attempt_limit(5)
                .with_time_limit(Duration::from_secs(15)),
        )
        .send()
        .await?;

    Ok(())
}

Error handling

Sometimes applications need to branch based on the type and details of the error returned by the client library. This guide shows you how to write code to handle such errors.

Retryable errors: One of the most common reasons to handle errors in distributed systems is to retry requests that fail due to transient errors. The Google Cloud Client Libraries for Rust implement a policy-based retry loop. You only need to configure the policies to enable the retry loop, and the libraries implement common retry policies. Consult the Configuring Retry Policies section before implementing your own retry loop.

Prerequisites

The guide uses the Secret Manager service to demonstrate error handling, but the concepts apply to other services as well.

You may want to follow the service quickstart, which shows you how to enable the service and ensure that you've logged in and that your account has the necessary permissions.

For complete setup instructions for the Rust libraries, see Setting up your development environment.

Dependencies

Add the Secret Manager library to your Cargo.toml file:

cargo add google-cloud-secretmanager-v1

In addition, this guide uses crc32c to calculate the checksum:

cargo add crc32c

Motivation

In this guide you'll create a new secret version. Secret versions are contained in secrets. You must create the secret before adding secret versions. A common pattern in cloud services is to use a resource as if the container for it existed, and only create the container if there is an error. If the container exists most of the time, such an approach is more efficient than checking if the container exists before making the request. Checking if the container exists consumes more quota, results in more RPC charges, and is slower when the container already exists.

Handling the error

This section walks you through a function to update a secret. For the full code sample, see update_secret.

First make an attempt to create a new secret version:

    match update_attempt(&client, project_id, secret_id, data.clone()).await {

If update_attempt succeeds, you can just print the successful result and return:

        Ok(version) => {
            println!("new version is {}", version.name);
            Ok(version)
        }

The request may have failed for many reasons: because the connection dropped before the request was fully sent, or the connection dropped before the response was received, or because it was impossible to create the authentication tokens.

The retry policies can deal with most of these errors. Here we are interested only in errors returned by the service:

        Err(e) => {
            if let Some(status) = e.status() {

and then only in errors that correspond to a missing secret:

                use gax::error::rpc::Code;
                if status.code == Code::NotFound {

If this is a "not found" error, you can try to create the secret. This simply returns on failure:

                    let _ = create_secret(&client, project_id, secret_id).await?;

Assuming create_secret is successful, you can try to add the secret version again, this time just returning an error if anything fails:

                    let version = update_attempt(&client, project_id, secret_id, data).await?;
                    println!("new version is {}", version.name);
                    return Ok(version);

What's next

Learn more about error handling:

• Examine error details
• Handling binding errors

Code samples

update_secret