How to work with Optimistic Concurrency Control (OCC)

Optimistic Concurrency Control (OCC) is a strategy used to manage shared resources and prevent “lost updates” or race conditions when multiple users or processes attempt to modify the same resource simultaneously.

As an example, consider systems like Google Cloud IAM, where the shared resource is an IAM Policy applied to a resource (like a Project, Bucket, or Service). To implement OCC, systems typically use a version number or an etag (entity tag) field on the resource struct.

Introduction to OCC

Imagine two processes, A and B, try to update a shared resource at the same time:

1. Process A reads the current state of the resource.

2. Process B reads the same current state.

3. Process A modifies its copy and writes it back to the server.

4. Process B modifies its copy and writes it back to the server.

Because Process B overwrites the resource without knowing that Process A already changed it, Process A’s updates are lost.

OCC solves this by introducing a unique fingerprint which changes every time an entity is modified. In many systems (like IAM), this is done using an etag. The server checks this tag on every write:

1. When you read the resource, the server returns an etag (a unique fingerprint).

2. When you send the modified resource back, you must include the original etag.

3. If the server finds that the stored etag does not match the etag you sent (meaning someone else modified the resource since you read it), the write operation fails with an ABORTED or FAILED_PRECONDITION error.

This failure forces the client to retry the entire process—re-read the new state, re-apply the changes, and try the write again with the new etag.

Implementing the OCC loop

The core of the OCC implementation is a loop that handles the retry logic. You should set a reasonable maximum number of retries to prevent infinite loops in cases of high contention.

Steps of the loop:

The following code provides an example of how to implement the OCC loop using an IAM policy on a Secret resource as the target.

Note: This example assumes the use of the Secret Manager client, but the same OCC pattern applies to any service or database that implements versioned updates.

Example

As usual with Rust, you must declare the dependency in your Cargo.toml file:

cargo add google-cloud-secretmanager-v1

    const ROLE: &str = "roles/secretmanager.secretAccessor";
    const ATTEMPTS: u32 = 5;

    let secret_name = format!("projects/{project_id}/secrets/{secret_id}");
    let client = SecretManagerService::builder().build().await?;
    for _attempt in 0..ATTEMPTS {
        let mut current = client
            .get_iam_policy()
            .set_resource(&secret_name)
            .send()
            .await?;

        match current.bindings.iter_mut().find(|b| b.role == ROLE) {
            None => current
                .bindings
                .push(Binding::new().set_role(ROLE).set_members([member])),
            Some(b) => {
                if b.members.iter().any(|m| m == member) {
                    return Ok(current);
                }
                b.members.push(member.to_string());
            }
        };
        let updated = client
            .set_iam_policy()
            .set_resource(&secret_name)
            .set_policy(current)
            .set_update_mask(FieldMask::default().set_paths(["bindings"]))
            .send()
            .await;
        match updated {
            Ok(p) => return Ok(p),
            Err(e)
                if e.status().is_some_and(|s| {
                    s.code == Code::Aborted || s.code == Code::FailedPrecondition
                }) =>
            {
                continue;
            }
            Err(e) => return Err(e.into()),
        }
    }
    anyhow::bail!("could not set IAM policy after {ATTEMPTS} attempts")