LlamaIndex

Overview

The toolbox-llamaindex package provides a Python interface to the MCP Toolbox service, enabling you to load and invoke tools from your own applications.

Installation

pip install toolbox-llamaindex

Quickstart

Here’s a minimal example to get you started using LlamaIndex:

import asyncio

from llama_index.llms.google_genai import GoogleGenAI
from llama_index.core.agent.workflow import AgentWorkflow

from toolbox_llamaindex import ToolboxClient

async def run_agent():
  async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
    tools = toolbox.load_toolset()

    vertex_model = GoogleGenAI(
        model="gemini-3-flash-preview",
        vertexai_config={"project": "project-id", "location": "us-central1"},
    )
    agent = AgentWorkflow.from_tools_or_functions(
        tools,
        llm=vertex_model,
        system_prompt="You are a helpful assistant.",
    )
    response = await agent.run(user_msg="Get some response from the agent.")
    print(response)

asyncio.run(run_agent())

Usage

Import and initialize the toolbox client.

from toolbox_llamaindex import ToolboxClient

# Replace with your Toolbox service's URL
async with ToolboxClient("http://127.0.0.1:5000") as toolbox:

Transport Protocols

The SDK supports multiple transport protocols for communicating with the Toolbox server. By default, the client uses the latest supported version of the Model Context Protocol (MCP).

You can explicitly select a protocol using the protocol option during client initialization. This is useful if you need to use the native Toolbox HTTP protocol or pin the client to a specific legacy version of MCP.

Supported Protocols

Example

If you wish to use the native Toolbox protocol:

from toolbox_llamaindex import ToolboxClient
from toolbox_core.protocol import Protocol

async with ToolboxClient("http://127.0.0.1:5000", protocol=Protocol.TOOLBOX) as toolbox:
    # Use client
    pass

If you want to pin the MCP Version 2025-03-26:

from toolbox_llamaindex import ToolboxClient
from toolbox_core.protocol import Protocol

async with ToolboxClient("http://127.0.0.1:5000", protocol=Protocol.MCP_v20250326) as toolbox:
    # Use client
    pass

Loading Tools

Load a toolset

A toolset is a collection of related tools. You can load all tools in a toolset or a specific one:

# Load all tools
tools = toolbox.load_toolset()

# Load a specific toolset
tools = toolbox.load_toolset("my-toolset")

Load a single tool

tool = toolbox.load_tool("my-tool")

Loading individual tools gives you finer-grained control over which tools are available to your LLM agent.

Use with LlamaIndex

LlamaIndex’s agents can dynamically choose and execute tools based on the user input. Include tools loaded from the Toolbox SDK in the agent’s toolkit:

from llama_index.llms.google_genai import GoogleGenAI
from llama_index.core.agent.workflow import AgentWorkflow

vertex_model = GoogleGenAI(
    model="gemini-3-flash-preview",
    vertexai_config={"project": "project-id", "location": "us-central1"},
)

# Initialize agent with tools
agent = AgentWorkflow.from_tools_or_functions(
    tools,
    llm=vertex_model,
    system_prompt="You are a helpful assistant.",
)

# Query the agent
response = await agent.run(user_msg="Get some response from the agent.")
print(response)

Maintain state

To maintain state for the agent, add context as follows:

from llama_index.core.agent.workflow import AgentWorkflow
from llama_index.core.workflow import Context
from llama_index.llms.google_genai import GoogleGenAI

vertex_model = GoogleGenAI(
    model="gemini-3-flash-preview",
    vertexai_config={"project": "project-id", "location": "us-central1"},
)
agent = AgentWorkflow.from_tools_or_functions(
    tools,
    llm=vertex_model,
    system_prompt="You are a helpful assistant.",
)

# Save memory in agent context
ctx = Context(agent)
response = await agent.run(user_msg="Give me some response.", ctx=ctx)
print(response)

Manual usage

Execute a tool manually using the call method:

result = tools[0].call(name="Alice", age=30)

This is useful for testing tools or when you need precise control over tool execution outside of an agent framework.

Client to Server Authentication

This section describes how to authenticate the ToolboxClient itself when connecting to a Toolbox server instance that requires authentication. This is crucial for securing your Toolbox server endpoint, especially when deployed on platforms like Cloud Run, GKE, or any environment where unauthenticated access is restricted.

This client-to-server authentication ensures that the Toolbox server can verify the identity of the client making the request before any tool is loaded or called. It is different from Authenticating Tools, which deals with providing credentials for specific tools within an already connected Toolbox session.

When is Client-to-Server Authentication Needed?

You’ll need this type of authentication if your Toolbox server is configured to deny unauthenticated requests. For example:

• Your Toolbox server is deployed on Cloud Run and configured to “Require authentication.”
• Your server is behind an Identity-Aware Proxy (IAP) or a similar authentication layer.
• You have custom authentication middleware on your self-hosted Toolbox server.

Without proper client authentication in these scenarios, attempts to connect or make calls (like load_tool) will likely fail with Unauthorized errors.

How it works

The ToolboxClient allows you to specify functions (or coroutines for the async client) that dynamically generate HTTP headers for every request sent to the Toolbox server. The most common use case is to add an Authorization header with a bearer token (e.g., a Google ID token).

These header-generating functions are called just before each request, ensuring that fresh credentials or header values can be used.

Configuration

You can configure these dynamic headers as follows:

from toolbox_llamaindex import ToolboxClient

async with ToolboxClient(
    "toolbox-url", 
    client_headers={"header1": header1_getter, "header2": header2_getter},
) as client:

Authenticating with Google Cloud Servers

For Toolbox servers hosted on Google Cloud (e.g., Cloud Run) and requiring Google ID token authentication, the helper module auth_methods provides utility functions.

Step by Step Guide for Cloud Run

1. Configure Permissions: Grant the roles/run.invoker IAM role on the Cloud Run service to the principal. This could be your user account email or a service account.

2. Configure Credentials

   • Local Development: Set up ADC.
   • Google Cloud Environments: When running within Google Cloud (e.g., Compute Engine, GKE, another Cloud Run service, Cloud Functions), ADC is typically configured automatically, using the environment’s default service account.

3. Connect to the Toolbox Server

   from toolbox_llamaindex import ToolboxClient
   from toolbox_core import auth_methods
   
   auth_token_provider = auth_methods.aget_google_id_token(URL)
   async with ToolboxClient(
       URL,
       client_headers={"Authorization": auth_token_provider},
   ) as client:
       tools = await client.aload_toolset()
   
       # Now, you can use the client as usual.
   

Authenticating Tools

Some tools require user authentication to access sensitive data.

Supported Authentication Mechanisms

Toolbox currently supports authentication using the OIDC protocol with ID tokens (not access tokens) for Google OAuth 2.0.

Configure Tools

Refer to these instructions on configuring tools for authenticated parameters.

Configure SDK

You need a method to retrieve an ID token from your authentication service:

async def get_auth_token():
    # ... Logic to retrieve ID token (e.g., from local storage, OAuth flow)
    # This example just returns a placeholder. Replace with your actual token retrieval.
    return "YOUR_ID_TOKEN" # Placeholder

Add Authentication to a Tool

async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
    tools = toolbox.load_toolset()

    auth_tool = tools[0].add_auth_token_getter("my_auth", get_auth_token) # Single token

    multi_auth_tool = tools[0].add_auth_token_getters({"auth_1": get_auth_1}, {"auth_2": get_auth_2}) # Multiple tokens

    # OR

    auth_tools = [tool.add_auth_token_getter("my_auth", get_auth_token) for tool in tools]

Add Authentication While Loading

auth_tool = toolbox.load_tool(auth_token_getters={"my_auth": get_auth_token})

auth_tools = toolbox.load_toolset(auth_token_getters={"my_auth": get_auth_token})

Complete Example

import asyncio
from toolbox_llamaindex import ToolboxClient

async def get_auth_token():
    # ... Logic to retrieve ID token (e.g., from local storage, OAuth flow)
    # This example just returns a placeholder. Replace with your actual token retrieval.
    return "YOUR_ID_TOKEN" # Placeholder

async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
    tool = toolbox.load_tool("my-tool")

    auth_tool = tool.add_auth_token_getter("my_auth", get_auth_token)
    result = auth_tool.call(input="some input")
    print(result)

Parameter Binding

Predetermine values for tool parameters using the SDK. These values won’t be modified by the LLM. This is useful for:

• Protecting sensitive information: API keys, secrets, etc.
• Enforcing consistency: Ensuring specific values for certain parameters.
• Pre-filling known data: Providing defaults or context.

Binding Parameters to a Tool

async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
    tools = toolbox.load_toolset()

    bound_tool = tool[0].bind_param("param", "value") # Single param

    multi_bound_tool = tools[0].bind_params({"param1": "value1", "param2": "value2"}) # Multiple params

    # OR

    bound_tools = [tool.bind_param("param", "value") for tool in tools]

Binding Parameters While Loading

bound_tool = toolbox.load_tool("my-tool", bound_params={"param": "value"})

bound_tools = toolbox.load_toolset(bound_params={"param": "value"})

Binding Dynamic Values

Use a function to bind dynamic values:

def get_dynamic_value():
  # Logic to determine the value
  return "dynamic_value"

dynamic_bound_tool = tool.bind_param("param", get_dynamic_value)

Asynchronous Usage

For better performance through cooperative multitasking, you can use the asynchronous interfaces of the ToolboxClient.

import asyncio
from toolbox_llamaindex import ToolboxClient

async def main():
    async with ToolboxClient("http://127.0.0.1:5000") as toolbox:
        tool = await client.aload_tool("my-tool")
        tools = await client.aload_toolset()
        response = await tool.ainvoke()

if __name__ == "__main__":
    asyncio.run(main())