Google Cloud Storage C++ Client  1.3.0
A C++ Client Library for Google Cloud Storage
Google Cloud Storage C++ Client Library

The Google Cloud Storage (GCS) C++ Client library offers types and functions access GCS from C++11 applications. It offers full access to the GCS API, including operations to list, read, write, and delete GCS objects and buckets. The library also provides functions to modify the IAM permissions on buckets, read and modify the metadata associated with objects and buckets, configure encryption keys, configure notifications via Cloud Pub/Sub, and change the access control list of object or buckets.

This library requires a C++11 compiler, it is supported (and tested) on multiple Linux distributions, as well as Windows and macOS. The top-level README on GitHub provides detailed instructions to install the necessary dependencies, as well as how to compile the client library.

Quickstart

The following instructions show you how to perform basic tasks in Cloud Storage using the C++ client library.

Costs that you incur in Cloud Storage are based on the resources you use. This quickstart typically uses less than $0.01USD worth of Cloud Storage resources.

If you have never used Cloud Storage we recommend that you first follow one of the language neutral quickstart guides before you follow this guide.

Before you begin

  1. Select or create a Google Cloud Platform (GCP) project using the manage resource page. Make a note of the project id, you will need to use it later.
  2. Make sure that billing is enabled for your project.
  3. Learn about key terms and concepts for Cloud Storage.
  4. Setup the authentication for the examples:

Downloading and Compiling the C++ Client Library

The source code for the Cloud Storage C++ Client Library can be found on GitHub. Download or clone this repository as usual:

git clone https://github.com/googleapis/google-cloud-cpp.git

The top-level README file in this repository includes detailed instructions on how to compile the library. The examples used in this guide should be automatically compiled when you follow said instructions.

Configuring authentication for the C++ Client Library

This library uses the GOOGLE_APPLICATION_CREDENTIALS environment variable to find the credentials file. For example:

Shell Command
Bash/zsh/ksh/etc. export GOOGLE_APPLICATION_CREDENTIALS=[PATH]
sh GOOGLE_APPLICATION_CREDENTIALS=[PATH]; export GOOGLE_APPLICATION_CREDENTIALS
csh/tsch setenv GOOGLE_APPLICATION_CREDENTIALS [PATH]
Windows Powershell $env:GOOGLE_APPLICATION_CREDENTIALS=[PATH]
Windows cmd.exe set GOOGLE_APPLICATION_CREDENTIALS=[PATH]

Setting this environment variable is the recommended way to configure the authentication preferences, though if the environment variable is not set, the library searches for a credentials file in the same location as the Cloud SDK.

Create a bucket

This is a short example to create a GCS bucket. This example assumes you have configured the authentication using GOOGLE_APPLICATION_CREDENTIALS:

#include <iostream>
int main(int argc, char* argv[]) {
if (argc != 3) {
std::cerr << "Missing project id and/or bucket name.\n";
std::cerr << "Usage: storage_quickstart <bucket-name> <project-id>\n";
return 1;
}
std::string bucket_name = argv[1];
std::string project_id = argv[2];
// Create aliases to make the code easier to read.
// Create a client to communicate with Google Cloud Storage. This client
// uses the default configuration for authentication and project id.
google::cloud::StatusOr<gcs::Client> client =
gcs::Client::CreateDefaultClient();
if (!client) {
std::cerr << "Failed to create Storage Client, status=" << client.status()
<< "\n";
return 1;
}
google::cloud::StatusOr<gcs::BucketMetadata> bucket_metadata =
client->CreateBucketForProject(bucket_name, project_id,
gcs::BucketMetadata());
if (!bucket_metadata) {
std::cerr << "Error creating bucket " << bucket_name
<< ", status=" << bucket_metadata.status() << "\n";
return 1;
}
std::cout << "Created bucket " << bucket_metadata->name() << "\n";
return 0;
}

This quickstart will always create a regional bucket in the us-east1 region. You must provide the project id and bucket name in the command-line when you run the quickstart program. Assuming you followed the build instructions referenced above this would be:

./cmake-out/google/cloud/storage/examples/storage_quickstart [BUCKET_NAME] [PROJECT_ID]

Using GOOGLE_CLOUD_PROJECT to set the default project

Some of the GCS APIs need a project as a parameter. For such APIs the application can (a) call an overload that takes the project id as a parameter, (b) define the project explicitly when the Client object is constructed, or (c) set the GOOGLE_CLOUD_PROJECT environment variable to the desired project id. If you have set this environment variable the code can be even simpler:

using ::google::cloud::StatusOr;
[](gcs::Client client, std::string bucket_name) {
StatusOr<gcs::BucketMetadata> bucket_metadata =
client.CreateBucket(bucket_name, gcs::BucketMetadata());
if (!bucket_metadata) {
throw std::runtime_error(bucket_metadata.status().message());
}
std::cout << "Bucket " << bucket_metadata->name() << " created."
<< "\nFull Metadata: " << *bucket_metadata << "\n";
}

Using the library in your own projects

Our continuous integration builds compile and test the code using both Bazel, and CMake. Integrating the GCS C++ Client library should be easy if you use either.

Integrating with Bazel

Select the google-cloud-cpp release you want to use. The release notes should include the expected SHA256 checksum of the tarball (and zip) files, make a note of them.

In your WORKSPACE file add a dependency to download and install the library, for example:

# Change the version and SHA256 hash as needed.
http_archive(
name = "com_github_googleapis_google_cloud_cpp",
url = "http://github.com/googleapis/google-cloud-cpp/archive/v0.9.0.tar.gz",
strip_prefix = "google-cloud-cpp-0.9.0",
sha256 = "a072103546cfa041ad8bfc599fe5a20c58e005a1a0ee18e94b2554dc3d485604",
)

Then load the dependencies of the library:

load("@com_github_googleapis_google_cloud_cpp//bazel:google_cloud_cpp_deps.bzl", "google_cloud_cpp_deps")
google_cloud_cpp_deps()
# Configure @com_google_googleapis to only compile C++ and gRPC:
load("@com_google_googleapis//:repository_rules.bzl", "switched_rules_by_language")
switched_rules_by_language(
name = "com_google_googleapis_imports",
cc = True, # C++ support is only "Partially implemented", roll our own.
grpc = True,
)
# Have to manually call the corresponding function for gRPC:
# https://github.com/bazelbuild/bazel/issues/1550
load("@com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
grpc_deps()

You can now use the library as a dependency in your BUILD files, for example:

cc_binary(
name = "my_program",
srcs = [
"my_program.cc",
],
deps = [
"@com_github_googleapis_google_cloud_cpp//google/cloud/storage:storage_client",
],
)

Integrating with CMake

Follow the instructions on the top-level INSTALL file on GitHub for details on how to compile and install the library for your platform and distribution. Once the library is installed, you can use it in your CMakeLists.txt file like any other package:

cmake_minimum_required(VERSION 3.5)
find_package(storage_client REQUIRED)
add_executable(my_program my_program.cc)
target_link_libraries(my_program storage_client)

Integrating with Make

The installation instructions on the top-level INSTALL file on GitHub also create pkg-config support files for application developers that prefer to use Make as their build system. Once the library is installed, you can use it in your Makefile like any other pkg-config module:

# Configuration variables to compile and link against the Google Cloud Storage
# C++ client library.
GCS_DEPS := storage_client
GCS_CXXFLAGS := $(shell pkg-config $(GCS_DEPS) --cflags)
GCS_CXXLDFLAGS := $(shell pkg-config $(GCS_DEPS) --libs-only-L)
GCS_LIBS := $(shell pkg-config $(GCS_DEPS) --libs-only-l)
# A target using the Google Cloud Storage C++ client library.
storage_install_test: storage_install_test.cc
$(CXX) $(CXXFLAGS) $(GCS_CXXFLAGS) $(GCS_CXXLDFLAGS) -o $@ $^ $(GCS_LIBS)

Error Handling

This library never throws exceptions to signal error. In general, the library returns a StatusOr<T> if an error is possible. Some functions return objects that already have an existing error handling mechanism, such as types derived from std::ostream where the application can check the state flags to determine if there was an error. In these cases no StatusOr wrapper is used.

Error Handling Example (without exceptions):

Applications that do not use exceptions to signal errors should check if the StatusOr<T> contains a value before using it. If the StatusOr<T> does contain a value then the StatusOr<T> can be used as a smart pointer to T. That is, operator->() and operator*() work as you would expect. If the StatusOr<T> does not contain a value then the error details are available using the .status() member function (and trying to access the value produces undefined behavior).

[](gcs::Client client) {
google::cloud::StatusOr<gcs::BucketMetadata> metadata =
client.GetBucketMetadata("my-bucket");
if (!metadata) {
std::cerr << "GetBucketMetadata: " << metadata.status() << "\n";
return;
}
// use `metadata` as a smart pointer to `BucketMetadata`
std::cout << "The metadata for bucket " << metadata->name()
<< " is " << *metadata << "\n";
}
Error Handling Example (with exceptions):

Applications that use exceptions to signal errors can simply call .value() on the StatusOr<T> object. This will return a T if the StatusOr<T> object contains a value, and will otherwise throw an exception.

[](gcs::Client client) {
gcs::BucketMetadata metadata = client.GetBucketMetadata(
"my-bucket").value(); // throws on error
std::cout << "The metadata for bucket " << metadata.name()
<< " is " << metadata << "\n";
}

Retry, Backoff, and Idempotency Policies.

The library automatically retries requests that fail with transient errors, and follows the recommended practice to backoff between retries. Application developers can override the default retry and backoff policies:

using ::google::cloud::StatusOr;
[](std::string bucket_name, std::string object_name, std::string contents) {
// Create a client that only gives up on the third error. The default policy
// is to retry for several minutes.
StatusOr<gcs::ClientOptions> options =
gcs::ClientOptions::CreateDefaultClientOptions();
if (!options) {
throw std::runtime_error(options.status().message());
}
gcs::Client client{*options, gcs::LimitedErrorCountRetryPolicy(3)};
StatusOr<gcs::ObjectMetadata> object_metadata =
client.InsertObject(bucket_name, object_name, std::move(contents),
gcs::IfGenerationMatch(0));
if (!object_metadata) {
throw std::runtime_error(object_metadata.status().message());
}
std::cout << "The object " << object_metadata->name()
<< " was created in bucket " << object_metadata->bucket()
<< "\nFull metadata: " << *object_metadata << "\n";
}

The default policies are to continue retrying for up to 15 minutes, and to use truncated (at 5 minutes) exponential backoff, doubling the maximum backoff period between retries.

By default the library retries all operations, even those that are not idempotent. Application developers can override the idempotency policy to follow a more conservative approach:

using ::google::cloud::StatusOr;
[](std::string bucket_name, std::string object_name, std::string contents) {
// Create a client that only retries idempotent operations, the default is
// to retry all operations.
StatusOr<gcs::ClientOptions> options =
gcs::ClientOptions::CreateDefaultClientOptions();
if (!options) {
throw std::runtime_error(options.status().message());
}
gcs::Client client{*options, gcs::StrictIdempotencyPolicy()};
StatusOr<gcs::ObjectMetadata> object_metadata =
client.InsertObject(bucket_name, object_name, std::move(contents),
gcs::IfGenerationMatch(0));
if (!object_metadata) {
throw std::runtime_error(object_metadata.status().message());
}
std::cout << "The object " << object_metadata->name()
<< " was created in bucket " << object_metadata->bucket()
<< "\nFull metadata: " << *object_metadata << "\n";
}
See also
LimitedTimeRetryPolicy and LimitedErrorCountRetryPolicy for alternative retry policies.
ExponentialBackoffPolicy to configure different parameters for the exponential backoff policy.
AlwaysRetryIdempotencyPolicy and StrictIdempotencyPolicy for alternative idempotency policies.

Next Steps

The documentation for each member function in the Client class includes short snippets on how to call the function. The snippets for ReadObject(), WriteObject(), ListObjects(), and DeleteObject() are good places to continue learning about the library.

Example for mocking GCS C++ Client