> ## Documentation Index
> Fetch the complete documentation index at: https://grandcentral.backbase.io/llms.txt
> Use this file to discover all available pages before exploring further.

# AI gateway

> OpenAI-compatible client for accessing AI models through the Backbase AI Platform

## Why AI gateway?

Instead of managing API keys per-model and per-environment, the AI Gateway provides a **centralized, authenticated entry point** for all LLM interactions. It handles authentication, agent ID validation, and policy enforcement - while maintaining full OpenAI SDK compatibility.

<Info>
  The gateway wraps the OpenAI SDK - **any framework that works with OpenAI works with AI Gateway**. No code changes required.
</Info>

<Note>
  **Prerequisite**: Ensure you've [installed the SDK](/agentic-ai/bb-ai-sdk/installation) and configured your environment before proceeding.
</Note>

## Quick start

### 1. Set up environment variables

Before using the gateway, configure your credentials:

```bash .env theme={"system"}
# Required for AI gateway
AI_GATEWAY_API_KEY=your-api-key
AI_GATEWAY_ENDPOINT=your-ai-gateway-endpoint
```

<Warning>
  Never commit API keys to version control. Add `.env` to your `.gitignore`.
</Warning>

### 2. Make your first call

```python theme={"system"}
from bb_ai_sdk.ai_gateway import AIGateway

gateway = AIGateway.create(
    model_id="gpt-4o",
    agent_id="550e8400-e29b-41d4-a716-446655440000"
)

response = gateway.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

<Check>
  That's it - you're making LLM calls through the Backbase AI Platform.
</Check>

<Tip>
  Don't have credentials yet? Refer to our [Onboarding guide](/agentic-ai/getting-started/onboarding).
</Tip>

## Tracing with observability

Call **`configure_observability()`** before creating the gateway ([integration steps](/agentic-ai/bb-ai-sdk/observability#set-up-the-sdk)):

```python theme={"system"}
from bb_ai_sdk.observability import configure_observability
from bb_ai_sdk.ai_gateway import AIGateway

configure_observability(service_name="my-agent")

gateway = AIGateway.create(model_id="gpt-4o", agent_id="...")
response = gateway.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

<Tip>
  FastAPI apps: pass `fastapi_app=app`. Agno/LangChain/LangGraph: set `framework=` and install the matching `[instrument-*]` extra.
</Tip>

## Sync vs Async

Choose based on your application architecture:

<Tabs>
  <Tab title="Sync">
    Use `AIGateway` for synchronous applications (scripts, simple APIs):

    ```python theme={"system"}
    from bb_ai_sdk.ai_gateway import AIGateway

    gateway = AIGateway.create(
        model_id="gpt-4o",
        agent_id="550e8400-e29b-41d4-a716-446655440000"
    )

    response = gateway.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    ```
  </Tab>

  <Tab title="Async">
    Use `AsyncAIGateway` for async applications (FastAPI, LangGraph):

    ```python theme={"system"}
    from bb_ai_sdk.ai_gateway import AsyncAIGateway

    gateway = AsyncAIGateway.create(
        model_id="gpt-4o",
        agent_id="550e8400-e29b-41d4-a716-446655440000"
    )

    response = await gateway.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    ```
  </Tab>
</Tabs>

## Common use cases

### Streaming responses

For real-time responses (chatbots, interactive UIs), enable streaming:

<Tabs>
  <Tab title="Sync streaming">
    ```python theme={"system"}
    stream = gateway.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Write a story"}],
        stream=True
    )

    for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="")
    ```
  </Tab>

  <Tab title="Async streaming">
    ```python theme={"system"}
    stream = await gateway.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Write a story"}],
        stream=True
    )

    async for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="")
    ```
  </Tab>
</Tabs>

## Framework adapters

The AI Gateway is OpenAI-compatible out of the box, but if you're using **LangChain**, **LangGraph**, or **Agno**, adapters convert the gateway into framework-native objects - no manual configuration required.

### LangChain

```python theme={"system"}
from bb_ai_sdk.ai_gateway import AIGateway
from bb_ai_sdk.ai_gateway.adapters.langchain import to_langchain

gateway = AIGateway.create(model_id="gpt-4o", agent_id="...")
model = to_langchain(gateway)  # Returns a ChatOpenAI-compatible model

# Use with LangChain components
from langchain.schema.output_parser import StrOutputParser

chain = model | StrOutputParser()
response = chain.invoke("Tell me a joke")
```

### LangGraph

```python theme={"system"}
from bb_ai_sdk.ai_gateway import AsyncAIGateway
from bb_ai_sdk.ai_gateway.adapters.langchain import to_langchain_async

gateway = AsyncAIGateway.create(model_id="gpt-4o", agent_id="...")
model = to_langchain_async(gateway)  # Returns async-compatible model

# Use in LangGraph nodes
async def generate(state):
    response = await model.ainvoke(state["messages"])
    return {"messages": [response]}
```

### Agno

```python theme={"system"}
from bb_ai_sdk.ai_gateway import AIGateway
from bb_ai_sdk.ai_gateway.adapters.agno import to_agno
from agno import Agent

gateway = AIGateway.create(model_id="gpt-4o", agent_id="...")
model = to_agno(gateway)  # Returns Agno-compatible model

agent = Agent(
    name="Assistant",
    model=model,
    instructions="You are helpful."
)
response = agent.run("Hello!")
```

## Common patterns

### Token usage tracking

Extract token consumption from responses:

```python theme={"system"}
from bb_ai_sdk.ai_gateway import get_token_usage

response = gateway.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

usage = get_token_usage(response)
if usage:
    print(f"Prompt tokens: {usage.prompt_tokens}")
    print(f"Completion tokens: {usage.completion_tokens}")
    print(f"Total tokens: {usage.total_tokens}")
```

## Error handling

Handle errors gracefully using the SDK's specific exception types for different failure scenarios:

```python theme={"system"}
from bb_ai_sdk.ai_gateway import (
    AIGateway,
    InvalidAgentIdError,
    ConfigurationError,
    AuthenticationError,
    RateLimitError,
)

# Handle creation errors
try:
    gateway = AIGateway.create(
        model_id="gpt-4o",
        agent_id="invalid"
    )
except InvalidAgentIdError:
    print("Invalid agent ID format - must be UUID v4")
except ConfigurationError:
    print("Missing API key or gateway URL")

# Handle request errors
try:
    response = gateway.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
except AuthenticationError:
    print("Invalid API key")
except RateLimitError:
    print("Rate limit exceeded - implement backoff")
```

### Error types reference

| Error                 | HTTP Code | Description                                 |
| --------------------- | --------- | ------------------------------------------- |
| `InvalidAgentIdError` | -         | Agent ID not in UUID v4 format              |
| `ConfigurationError`  | -         | Missing API key or invalid gateway URL      |
| `AuthenticationError` | 401       | Invalid or expired API key                  |
| `AuthorizationError`  | 403       | Insufficient permissions for this operation |
| `RateLimitError`      | 429       | Rate limit exceeded                         |
| `ValidationError`     | 400       | Invalid request parameters                  |
| `ModelNotFoundError`  | 404       | Requested model not available               |
| `ServiceError`        | 500+      | Server-side error                           |
| `NetworkError`        | -         | Connection failed                           |

## Configuration

### Environment variables

Configure credentials via environment variables (recommended):

```ini .env theme={"system"}
# Required
AI_GATEWAY_API_KEY=your-api-key
AI_GATEWAY_ENDPOINT=your-ai-gateway-endpoint
```

<Warning>
  Never commit API keys to version control. Add `.env` to your `.gitignore`.
</Warning>

### Create parameters

<ParamField path="model_id" type="string" required>
  Model identifier (e.g., `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`).
</ParamField>

<ParamField path="agent_id" type="string" required>
  Your agent's unique identifier in UUID v4 format. Obtained from the platform when you register your agent.
</ParamField>

<ParamField path="api_key" type="string">
  API key for authentication. Falls back to `AI_GATEWAY_API_KEY`, then `AZURE_OPENAI_API_KEY`, if not provided.
</ParamField>

<ParamField path="base_url" type="string">
  Gateway URL. Falls back to `AI_GATEWAY_ENDPOINT` environment variable or platform default.
</ParamField>

<ParamField path="api_version" type="string" default="2024-10-21">
  API version for the gateway.
</ParamField>

## Advanced: Accessing the underlying client

Access the underlying OpenAI client or raw configuration for advanced use cases:

### Get\_client()

Get the underlying OpenAI client for direct SDK access:

```python theme={"system"}
client = gateway.get_client()
# Returns OpenAI (Sync) or asyncopenai (Async) instance
```

### Get\_config()

Get configuration dictionary for manual framework setup:

```python theme={"system"}
config = gateway.get_config()
# Returns:
# {
# "api_key": "...",
# "base_url": "...",
# "default_headers": {"x-agent-id": "...", "api-key": "..."},
# "default_query": {"api-version": "2024-10-21"},
# "model": "gpt-4o"
# }
```

## API reference

### Aigateway

| Property/Method | Returns        | Description                        |
| --------------- | -------------- | ---------------------------------- |
| `chat`          | Chat interface | OpenAI-compatible chat completions |
| `model_id`      | `str`          | Configured model ID                |
| `agent_id`      | `str`          | Validated agent ID                 |
| `get_client()`  | `OpenAI`       | Underlying OpenAI client           |
| `get_config()`  | `dict`         | Configuration dictionary           |
| `create()`      | `AIGateway`    | Factory method (class method)      |

### Asyncaigateway

Same interface as `AIGateway` but returns `AsyncOpenAI` client and supports async operations.

## Next steps

<CardGroup cols={2}>
  <Card title="Observability" icon="chart-line" href="/agentic-ai/bb-ai-sdk/observability">
    Add tracing and monitoring to your agents
  </Card>

  <Card title="Starter kits" icon="rocket" href="/agentic-ai/starter-kits/overview">
    See AI Gateway integrated in production templates
  </Card>

  <Card title="Get started" icon="play" href="/agentic-ai/bb-ai-sdk/get-started">
    Build your first agent end-to-end
  </Card>

  <Card title="Examples" icon="code" href="https://github.com/backbase/bb-ai-sdk/tree/main/examples">
    View complete working examples
  </Card>
</CardGroup>