> ## Documentation Index
> Fetch the complete documentation index at: https://grandcentral.backbase.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Observability

> OpenTelemetry tracing and export to Langfuse, Grafana, or an OTLP endpoint

The **`bb_ai_sdk.observability`** module instruments your agent application with OpenTelemetry. When you call **`configure_observability()`** at startup, the SDK registers a tracer, attaches framework and HTTP instrumentors, and exports spans to Langfuse, Grafana, or an OTLP endpoint you configure. When you call **`bb_ai_sdk.logging.init()`**, the SDK adds trace and span IDs to log lines and applies the same redaction rules to logs and span attributes.

On each run, instrumentors record model calls, tool invocations, and request boundaries. You can stamp runs with **`scope()`** and **`bb.*`** attributes for filtering. The same setup works for local development, evaluation runs, and deployed services—you change export settings with environment variables, not application code.

<Info>
  The SDK builds on OpenTelemetry. You can switch export backends without rewriting agent code.
</Info>

## What the SDK provides

The observability module delivers three capabilities:

<CardGroup cols={3}>
  <Card title="Instrumentation" icon="wave-square">
    The SDK hooks into HTTP (FastAPI), agent frameworks (Agno, LangChain, LangGraph), LLM clients, outbound HTTP, and threaded work—so spans cover the full run without hand-written instrumentation for each call.
  </Card>

  <Card title="Context" icon="tags">
    The SDK defines **`bb.*`** attributes (`bb.project`, `bb.trace.name`, `bb.agent.name`, and others) and **`scope()`** so you can filter traces by deployable, team, environment, and tenant.
  </Card>

  <Card title="Export and correlation" icon="paper-plane">
    The SDK batches and exports spans via OTLP. With **`logging.init()`**, log lines carry **`trace_id`** and **`span_id`**, and one redaction engine covers logs and span attributes.
  </Card>
</CardGroup>

Use exported traces to monitor latency and token usage, review runs for audit, debug failures, and feed evaluation experiments—not only for incident response.

## What you call at startup

The SDK exposes two **`init()`** functions in different modules. Only one observability entry point belongs in your application code.

| Module                        | Function                        | Call from your app?                                                                                                                                                                                                                                                                     |
| ----------------------------- | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`bb_ai_sdk.logging`**       | **`init()`**                    | **Yes** — after **`logging.basicConfig`**, for redaction and log trace correlation                                                                                                                                                                                                      |
| **`bb_ai_sdk.observability`** | **`configure_observability()`** | **Yes** — once at startup, for tracing, instrumentors, and (optionally) FastAPI middleware                                                                                                                                                                                              |
| **`bb_ai_sdk.observability`** | **`init()`**                    | **Only for advanced export settings** — call **`init()`** first with **`backend`**, **`otlp_*`**, org fields, or batch tuning, then **`configure_observability(..., init_observability=False)`**. Do not call **`init()`** after **`configure_observability()`** with default settings. |

<Warning>
  Do **not** call **`observability.init()`** after **`configure_observability()`** with default settings — that double-initializes the tracer. For Langfuse or custom OTLP, **`configure_observability()`** alone is enough (it calls **`init(service_name=..., environment=...)`** internally). For Grafana presets, batch tuning, or org fields on the Resource, call **`init(...)`** first, then **`configure_observability(..., init_observability=False)`** — see [Advanced export settings](#advanced-export-settings).
</Warning>

Typical FastAPI + Agno startup (load **`.env`** before SDK imports when you use a local env file):

```python theme={"system"}
import logging
from dotenv import load_dotenv
from fastapi import FastAPI

load_dotenv()

from bb_ai_sdk.logging import DEFAULT_DATEFMT, STANDARD_FORMAT, init as init_logging
from bb_ai_sdk.observability import configure_observability

app = FastAPI(title="my-agent-api")

logging.basicConfig(
    level=logging.INFO,
    format=STANDARD_FORMAT,
    datefmt=DEFAULT_DATEFMT,
)
init_logging(capture_warnings=True)

configure_observability(
    fastapi_app=app,
    framework="agno",          # match your [instrument-*] extras
    service_name="my-agent-api",
    environment="development",
)
```

Create **`AIGateway`** and agent instances **after** **`configure_observability()`**. Set export destination with environment variables—see [Where traces go](#where-traces-go).

The following diagram shows the observability pipeline from installation to trace export:

```mermaid theme={"system"}
%%{init: {
  'theme': 'base',
  'themeVariables': {
    'primaryColor': '#ffffff',
    'primaryBorderColor': '#295eff',
    'primaryTextColor': '#091c35',
    'lineColor': '#091c35',
    'secondaryColor': '#f3f6f9',
    'tertiaryColor': '#ebf0f5',
    'fontFamily': 'Libre Franklin, sans-serif'
  }
}}%%
flowchart LR
    A[Install SDK extras] --> B[logging.init + configure_observability]
    B --> C[Agent runs]
    C --> D[SDK creates spans]
    D --> E[Redaction on logs and attributes]
    E --> F[OTLP export]
    F --> G[Langfuse, Grafana, or OTLP]
```

| Phase       | What you do                                                                                                          | What the SDK does                                                  |
| ----------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ |
| **Wire**    | Install `[instrument-*]` extras; set env vars; call `logging.init()` and `configure_observability()` once at startup | Registers TracerProvider, instrumentors, middleware, and redaction |
| **Run**     | Serve traffic; optional `scope(...)` per request; use `await team.arun(...)` in multi-agent HTTP apps                | Creates a span tree; applies `bb.*` at span start                  |
| **Capture** | Optional `@trace` or `trace_context` for custom steps                                                                | Records LLM, tool, and framework spans; tags guardrail blocks      |
| **Export**  | Configure proxy/`NO_PROXY` when needed                                                                               | Batches spans and sends them to your configured backend            |
| **Use**     | Open Langfuse, Grafana, or your OTLP UI; run evals; filter by `service.name` and `bb.*`                              | Same wiring in app and eval processes when `service_name` matches  |

<Warning>
  Never commit API keys to version control. Add `.env` to your `.gitignore`.
</Warning>

## Choose your path

Pick the setup that matches your application. Each path uses **`configure_observability()`** only—not **`observability.init()`**.

<CardGroup cols={2}>
  <Card title="FastAPI + Agno" icon="server" href="#set-up-the-sdk">
    HTTP agent with Agno teams or single agents. Install **`[instrument-fastapi,instrument-agno]`**.
  </Card>

  <Card title="Script or eval runner" icon="file-code" href="#scripts-and-evals">
    No FastAPI—local scripts, batch jobs, or **`bb-ai-sdk evals`**. Omit **`fastapi_app`**; set **`framework=`** to match your extras.
  </Card>

  <Card title="LangChain or LangGraph" icon="link" href="#langchain-and-langgraph">
    Pass **`framework="langchain"`** or **`framework="langgraph"`** to **`configure_observability()`**.
  </Card>

  <Card title="Gateway-only script" icon="plug" href="#scripts-and-evals">
    **`configure_observability(service_name=...)`** only—no **`framework=`** if you have no agent framework. See [Get started](/agentic-ai/bb-ai-sdk/get-started).
  </Card>
</CardGroup>

## Set up the SDK

Follow these steps to wire tracing for most agent applications.

<Steps>
  <Step title="Install instrumentation extras">
    Install **`bb-ai-sdk` 0.1.9** with the extras that match your app. See [Installation — observability extras](/agentic-ai/bb-ai-sdk/installation#observability-extras-recommended-for-agents).

    | `framework=` value | Extra                    |
    | ------------------ | ------------------------ |
    | `"agno"`           | `[instrument-agno]`      |
    | `"langchain"`      | `[instrument-langchain]` |
    | `"langgraph"`      | `[instrument-langgraph]` |
    | FastAPI HTTP spans | `[instrument-fastapi]`   |

    Typical FastAPI + Agno agent:

    ```bash theme={"system"}
    uv add "bb-ai-sdk[instrument-fastapi,instrument-agno]==0.1.9" --index backbase
    ```
  </Step>

  <Step title="Configure export destination">
    The SDK reads export settings from environment variables. Choose **Langfuse**, **Grafana**, or a **custom OTLP** endpoint—one export path per deployment unless your operations team instructs you otherwise.

    See [Where traces go](#where-traces-go) for the full variable list and path-specific notes.
  </Step>

  <Step title="Initialize logging and observability">
    Use the [startup sequence](#what-you-call-at-startup): **`logging.basicConfig`** → **`bb_ai_sdk.logging.init()`** → **`configure_observability(...)`**.

    For logging format presets, redaction patterns, and coverage details, see **[Logging and redaction](/agentic-ai/bb-ai-sdk/logging)**.
  </Step>

  <Step title="Run your agent">
    Create gateway and agent instances **after** `configure_observability()`. Instrumentors capture LLM and tool spans automatically.

    For multi-agent HTTP services, wrap each team run in **`scope(...)`** and use **`await team.arun(...)`**—not sync **`team.run(...)`**. See [Multi-agent HTTP services](#multi-agent-http-services).
  </Step>

  <Step title="Verify traces">
    1. Send one request or run one script invocation.
    2. Open Langfuse, Grafana, or your OTLP backend (traces may take up to **5 seconds** to appear while the SDK batches export).
    3. Confirm you see a request boundary span, framework spans, and LLM spans in one trace.

    <Tip>
      Don't have Langfuse credentials? See the [Onboarding guide](/agentic-ai/get-started/onboarding).
    </Tip>
  </Step>
</Steps>

### Scripts and evals

For eval runners or scripts without FastAPI, omit **`fastapi_app`**:

```python theme={"system"}
configure_observability(
    service_name="my-agent-api",
    environment="development",
    framework="agno",
)
```

Use the same **`service_name`** in **`evals/__init__.py`** as in your app so eval and production traces align. See **[Evaluate your agent](/agentic-ai/bb-ai-sdk/evaluation-framework)**.

### LangChain and LangGraph

Install the matching extra, then pass **`framework="langchain"`** or **`framework="langgraph"`** to **`configure_observability()`**—same startup sequence as Agno.

For per-chain callback control only, see [LangChain and LangGraph manual callbacks](#langchain-and-langgraph-manual-callbacks).

## Where traces go

The SDK exports spans over OTLP. You choose the destination—Langfuse, Grafana, or a custom OTLP endpoint—with environment variables. Application code stays the same.

### Export paths

| Path            | When to use                                                   | Required env vars                                                                                                          | Common optional vars                                                                                                                               |
| --------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Langfuse**    | Local development, evals, or when the app holds Langfuse keys | `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`                                                                               | `LANGFUSE_HOST`, `OBSERVABILITY_ENABLED`                                                                                                           |
| **Grafana**     | Teams using Grafana Cloud for traces                          | `GRAFANA_BEARER_TOKEN`, `OTEL_EXPORTER_OTLP_ENDPOINT` (or pass **`otlp_endpoint=`** to **`init(backend="grafana", ...)`**) | `OBSERVABILITY_ENABLED`                                                                                                                            |
| **Custom OTLP** | Deployed runtimes where operations owns trace export          | `OTEL_EXPORTER_OTLP_ENDPOINT` (or `OTLP_ENDPOINT`)                                                                         | `OTEL_EXPORTER_OTLP_HEADERS`, `OTEL_RESOURCE_ATTRIBUTES` (often `project=<your-project-identifier>`), `OTEL_SERVICE_NAME`, `OBSERVABILITY_ENABLED` |

On the **custom OTLP** path, your operations team may require **`OTEL_RESOURCE_ATTRIBUTES=project=<your-project-identifier>`** so traces route to the right project in the collector UI. The SDK merges this env var into the Resource—it does not require a specific `project=` value. Confirm the identifier with your operations team.

Do **not** mix **`LANGFUSE_*`**, **`GRAFANA_*`**, and **`OTEL_EXPORTER_OTLP_*`** unless your operations team instructs you to.

### Canonical environment block

```bash .env theme={"system"}
# Option A: Langfuse (typical for local dev or direct export)
LANGFUSE_PUBLIC_KEY=pk-lf-xxx
LANGFUSE_SECRET_KEY=sk-lf-xxx
# Optional: LANGFUSE_HOST=https://cloud.langfuse.com

# Option B: Grafana Cloud (omit LANGFUSE_*)
# GRAFANA_BEARER_TOKEN=your-grafana-token
# OTEL_EXPORTER_OTLP_ENDPOINT=https://your-grafana.com/otlp
# Or pass backend="grafana" and otlp_endpoint= to init() — see Grafana tab below

# Option C: Custom OTLP (typical on deployed runtimes — omit LANGFUSE_*)
# OTEL_EXPORTER_OTLP_ENDPOINT=https://your-otlp-endpoint
# OTLP_ENDPOINT=https://your-otlp-endpoint
# OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer your-token
# OTEL_RESOURCE_ATTRIBUTES=project=<your-project-identifier>
# OTEL_SERVICE_NAME=my-agent-api

# Tracing is on by default. Set OBSERVABILITY_ENABLED=false to disable.
# Only true or false are accepted (case-insensitive).
```

<Tabs>
  <Tab title="Langfuse">
    Without **`OTEL_EXPORTER_OTLP_ENDPOINT`**, the SDK targets Langfuse Cloud (honoring **`LANGFUSE_HOST`** when set). Set **`LANGFUSE_PUBLIC_KEY`** and **`LANGFUSE_SECRET_KEY`** so export is authenticated—not to choose the endpoint URL.

    In Langfuse you can inspect LLM and tool spans, token usage, latency, and session groupings. Langfuse features include cost tracking, trace hierarchies, and experiment integration via **`bb-ai-sdk evals`**.

    ```python theme={"system"}
    from bb_ai_sdk.observability import configure_observability

    configure_observability(service_name="my-agent", framework="agno")
    ```

    <Note>
      If Langfuse keys are missing, **`init()`** still succeeds: the endpoint defaults to Langfuse Cloud, auth headers are unset, spans are created locally, and export is rejected by the backend. The agent does not crash—the SDK fails safe on observability errors.
    </Note>
  </Tab>

  <Tab title="Grafana">
    Export to Grafana Cloud via OTLP. **`configure_observability()`** does not accept **`backend=`** or **`otlp_endpoint=`** — those parameters belong on **`init()`**. You must supply a Grafana bearer token (**`GRAFANA_BEARER_TOKEN`** in **`.env`** or **`grafana_bearer_token=`** on **`init()`**); without it, **`init(backend="grafana", ...)`** raises at configuration time.

    Call **`init()`** first, then wire instrumentors with **`init_observability=False`**:

    ```bash .env theme={"system"}
    GRAFANA_BEARER_TOKEN=your-grafana-token
    ```

    ```python theme={"system"}
    from bb_ai_sdk.observability import configure_observability, init

    init(
        service_name="my-agent",
        backend="grafana",
        otlp_endpoint="https://your-grafana.com/otlp",
        # grafana_bearer_token="..."  # optional if GRAFANA_BEARER_TOKEN is in .env
    )
    configure_observability(
        service_name="my-agent",
        framework="agno",
        init_observability=False,
    )
    ```

    Alternatively, set **`OTEL_EXPORTER_OTLP_ENDPOINT`** and **`OTEL_EXPORTER_OTLP_HEADERS`** (Bearer token) in **`.env`** and use **`configure_observability()`** only — same as the custom OTLP path.
  </Tab>

  <Tab title="Custom OTLP">
    Use a custom OTLP endpoint when operations exports traces for you instead of you setting Langfuse keys in the app.

    ```bash theme={"system"}
    OTEL_EXPORTER_OTLP_ENDPOINT=https://your-otlp-endpoint
    OTEL_RESOURCE_ATTRIBUTES=project=<your-project-identifier>
    ```

    Use the same **`configure_observability(...)`** call as the Langfuse path.
  </Tab>
</Tabs>

### Proxy configuration

On the corporate network, configure proxy settings so OTLP export reaches your backend:

```bash .env theme={"system"}
HTTP_PROXY=http://webproxy.infra.backbase.cloud:8888
HTTPS_PROXY=http://webproxy.infra.backbase.cloud:8888
NO_PROXY=localhost,127.0.0.1,cloud.langfuse.com,*.langfuse.com,langfuse
```

<Warning>
  **`NO_PROXY` must include your Langfuse, Grafana, or OTLP host.** The corporate web proxy can block trace export if this is missing.
</Warning>

## Customize trace context

After basic tracing works, add SDK context so you can filter and compare runs.

### `bb.*` attribute keys

The SDK defines vendor-neutral names in **`bb_ai_sdk.observability.attributes`**. Use the constants—not string literals—so attributes stay consistent across backends:

```python theme={"system"}
from bb_ai_sdk.observability import scope
from bb_ai_sdk.observability.attributes import (
    BB_PROJECT,
    BB_TRACE_NAME,
    BB_TRACE_INPUT,
    BB_AGENT_NAME,
)
```

| Key                                         | Where it appears      | How you set it                                                                                                                         |
| ------------------------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `service.name`                              | Resource (OTel)       | **`configure_observability(..., service_name=...)`** — one per deployable                                                              |
| `deployment.environment` / `bb.environment` | Resource              | **`environment=`** on **`configure_observability`**                                                                                    |
| `bb.agent.id`                               | Resource and/or span  | **`scope(agent_id=...)`** per request or team                                                                                          |
| `bb.agent.name`                             | Resource and/or span  | **`scope(agent_name=...)`** per run when multiple agents share one service                                                             |
| `bb.organization.id`                        | Resource              | **`init(..., organization_id=...)`** before **`configure_observability(..., init_observability=False)`**, or **`scope()`** when needed |
| `bb.organization.name`                      | Resource              | **`init(..., organization_name=...)`** before **`configure_observability(..., init_observability=False)`**                             |
| `bb.project`                                | Every span in context | Header **`X-Observability-Project`** or **`scope(project=...)`**                                                                       |
| `bb.trace.name`                             | Every span in context | **`scope(trace_name=...)`**                                                                                                            |
| `bb.trace.input`                            | Every span in context | **`scope(trace_input=...)`** — optional input preview                                                                                  |
| `guardrails.blocked`                        | Span                  | Set by the SDK when NeMo Guardrails blocks a call                                                                                      |

The SDK installs **`ScopeAttributeSpanProcessor`** when you call **`configure_observability()`**. It reads **`scope()`** context and applies **`bb.project`**, **`bb.trace.*`**, and optional **`bb.agent.*`** on span start.

<Tip>
  Do **not** set vendor-specific names (for example **`langfuse.trace.name`**) in application code. Use **`scope(trace_name=..., trace_input=...)`**; your backend maps **`bb.*`** keys as needed.
</Tip>

Optional routing header: send **`X-Observability-Project`** on API requests. The SDK middleware maps it to **`bb.project`** on nested spans.

### Multi-agent HTTP services

For one deployable that hosts multiple Agno teams, see also the **[Multi agent starter](/agentic-ai/starter-kits/multi-agent)**.

* Call **`configure_observability(fastapi_app=app, framework="agno", service_name="...", environment=...)`** once at import time.
* Install **`[instrument-fastapi,instrument-agno]`** so the SDK wires FastAPI, Agno, outbound HTTPX, and threading instrumentation.
* Use **`await team.arun(...)`**, not sync **`team.run(...)`**. Sync **`run`** can create a second root trace disconnected from the FastAPI span.
* Use one **`service_name`** per HTTP service; stamp each run with **`scope(agent_name=..., trace_name=..., trace_input=...)`**.

```python theme={"system"}
with scope(
    agent_name=team_name,
    trace_name=team_name,
    trace_input=query,
):
    result = await team.arun(query)
```

<Warning>
  **Symptom:** two trace IDs for one API call, or tool spans with no parent. **Check:** **`arun`** vs **`run`**, both instrument extras installed, and **`NO_PROXY`** includes your export host.
</Warning>

## Custom spans

Instrumentors capture LLM and framework steps. Add your own spans when you need visibility into business logic.

### The `@trace` decorator

```python theme={"system"}
from bb_ai_sdk.observability import trace

@trace()
def process_user_request(user_input: str) -> str:
    return "processed result"

@trace(name="validate-user-input")
def validate_input(data: dict) -> bool:
    return True
```

### Custom attributes

```python theme={"system"}
@trace(attributes={
    "prompt.version": "v1.2.3",
    "prompt.name": "customer-support-prompt",
    "user.id": "user-123",
})
def run_experiment():
    pass
```

### The `trace_context` context manager

```python theme={"system"}
from bb_ai_sdk.observability import trace_context

def complex_operation():
    with trace_context("multi-step-operation") as span:
        span.add_event("Step 1: Validating input")
        validate_input()
        span.set_attribute("result.count", len(result))
        return result
```

### Context utilities

```python theme={"system"}
from bb_ai_sdk.observability import (
    get_current_trace_id,
    get_current_span,
    get_tracer_provider,
)

trace_id = get_current_trace_id()
span = get_current_span()
provider = get_tracer_provider()
```

Use **`get_current_trace_id()`** with standard loggers after **`bb_ai_sdk.logging.init()`**—log format presets already include **`trace_id`** and **`span_id`**.

## Manual assembly

Use this path when you assemble **`OpenAIInstrumentor`**, LangChain callback handlers, or **`instrument()`** yourself instead of **`configure_observability()`**. For Grafana presets or batch tuning while still using **`configure_observability()`** for instrumentors, see [Advanced export settings](#advanced-export-settings) instead.

<Info>
  **`configure_observability()`** already calls **`init()`**, registers framework instrumentors, and adds FastAPI middleware. Prefer it for all new agents.
</Info>

### Gateway-only script

If you use **`configure_observability(service_name=...)`** without a **`framework=`**, the SDK wires HTTPX and threading instrumentors and exports spans when env vars are set—the same pattern as [Get started](/agentic-ai/bb-ai-sdk/get-started). The sync **`AIGateway`** client does not auto-attach **`OpenAIInstrumentor`** in the SDK today; use **`AsyncAIGateway`**, or attach **`OpenAIInstrumentor`** manually as below. Manual assembly without **`configure_observability()`**:

```python theme={"system"}
from dotenv import load_dotenv

load_dotenv()

from bb_ai_sdk.observability import init, get_tracer_provider
from openinference.instrumentation.openai import OpenAIInstrumentor
from bb_ai_sdk.ai_gateway import AIGateway

init(service_name="my-agent")
OpenAIInstrumentor().instrument(tracer_provider=get_tracer_provider())

gateway = AIGateway.create(
    model_id="gpt-4o",
    agent_id="550e8400-e29b-41d4-a716-446655440000",
)
```

<Info>
  **`OpenAIInstrumentor`** monkey-patches the OpenAI client so LLM calls through AIGateway appear as spans.
</Info>

### LangChain and LangGraph manual callbacks

When you use callback handlers **instead of** **`configure_observability(framework=...)`**, call **`init()`** once, then attach handlers per invocation:

<Tabs>
  <Tab title="LangChain">
    ```python theme={"system"}
    from bb_ai_sdk.observability import init, LangChainOpenTelemetryCallbackHandler
    from bb_ai_sdk.ai_gateway import AIGateway
    from bb_ai_sdk.ai_gateway.adapters.langchain import to_langchain
    from langchain.prompts import ChatPromptTemplate
    from langchain.schema.output_parser import StrOutputParser

    init(service_name="langchain-agent")
    callback = LangChainOpenTelemetryCallbackHandler()

    gateway = AIGateway.create(model_id="gpt-4o", agent_id="...")
    model = to_langchain(gateway)
    chain = ChatPromptTemplate.from_template("Tell me about {topic}") | model | StrOutputParser()

    result = chain.invoke(
        {"topic": "AI observability"},
        config={"callbacks": [callback]},
    )
    ```
  </Tab>

  <Tab title="LangGraph">
    ```python theme={"system"}
    from bb_ai_sdk.observability import init, LangGraphOpenTelemetryCallbackHandler
    from langgraph.graph import StateGraph, END
    from langchain_core.runnables import RunnableConfig

    init(service_name="langgraph-agent")
    callback = LangGraphOpenTelemetryCallbackHandler(graph_name="my-workflow")

    graph = StateGraph(AgentState)
    # add nodes and edges
    app = graph.compile()

    result = app.invoke(
        {"messages": [...]},
        config=RunnableConfig(callbacks=[callback]),
    )
    ```
  </Tab>
</Tabs>

## Advanced export settings

**`configure_observability()`** forwards only **`service_name`** and **`environment`** to **`init()`**. Export backend, OTLP tuning, organization fields, and custom Resource attributes are set on **`init()`** — then call **`configure_observability(..., init_observability=False)`** so instrumentors and FastAPI middleware still wire up without a second **`init()`**.

### Batch export

The SDK batches spans before export. By default, traces may take up to **5 seconds** to appear. Pass tuning parameters to **`init()`**:

```python theme={"system"}
from bb_ai_sdk.observability import configure_observability, init

init(
    service_name="my-agent",
    environment="development",
    otlp_batch_size=512,
    otlp_batch_timeout=5.0,
    otlp_max_queue_size=10000,
)
configure_observability(
    service_name="my-agent",
    framework="agno",
    init_observability=False,
)
```

<Tip>
  For faster feedback during development, set **`otlp_batch_timeout=1.0`**. Avoid low values in production—they increase network overhead.
</Tip>

### Custom resource attributes and tenancy

```python theme={"system"}
init(
    service_name="my-agent",
    environment="production",
    organization_id="org-123",
    organization_name="Acme Corp",
    resource_attributes={
        "service.version": "1.2.3",
        "deployment.region": "us-east-1",
    },
)
configure_observability(
    service_name="my-agent",
    framework="agno",
    init_observability=False,
)
```

You can also merge Resource attributes with **`OTEL_RESOURCE_ATTRIBUTES`** in **`.env`**.

### Disable tracing

Set **`OBSERVABILITY_ENABLED=false`**, or pass **`enabled=False`** to **`init()`**. Load environment variables before SDK imports when you use a **`.env`** file.

## Best practices

### Initialize once at startup

Call **`configure_observability()`** once when the process starts—before routes or agents handle traffic. Call **`bb_ai_sdk.logging.init()`** once after **`logging.basicConfig`**.

<Warning>
  With default settings, do **not** call **`observability.init()`** after **`configure_observability()`** — the SDK already invoked **`init()`** internally. For advanced export settings, call **`init()`** **first**, then **`configure_observability(..., init_observability=False)`**.
</Warning>

### Set environment and tenancy early

Use **`environment=`** on **`configure_observability`** (or on **`init()`** when you use the advanced path) to separate dev, staging, and production traces. Pass **`organization_id`** and **`organization_name`** on **`init()`** for multi-tenant cost attribution on the Resource.

### Use meaningful span names

Prefer **`@trace(name="validate-user-input")`** over generic names like **`process`** or **`step1`**.

### Keep credentials in environment variables

Do not hardcode Langfuse keys, Grafana tokens, or OTLP credentials in source code.

## Debug trace export

To troubleshoot export issues, enable OpenTelemetry debug logging **after** **`bb_ai_sdk.logging.init()`** so output stays redacted:

```python theme={"system"}
import logging

logging.getLogger("opentelemetry").setLevel(logging.DEBUG)
logging.getLogger("opentelemetry.sdk").setLevel(logging.DEBUG)
logging.getLogger("opentelemetry.exporter").setLevel(logging.DEBUG)
```

Disable verbose OTel logging in production. For log format, redaction coverage, and operator patterns, see **[Logging and redaction](/agentic-ai/bb-ai-sdk/logging)**.

## Troubleshooting

<AccordionGroup>
  <Accordion title="Traces not appearing in the backend">
    **Cause:** Missing or invalid export credentials, or wrong export path (Langfuse, Grafana, or OTLP).

    **Solution:** Verify environment variables for your chosen path. See [Export paths](#export-paths). Confirm traces in the Langfuse, Grafana, or OTLP UI.
  </Accordion>

  <Accordion title="Init succeeds but Langfuse shows no traces">
    **Cause:** **`LANGFUSE_PUBLIC_KEY`** or **`LANGFUSE_SECRET_KEY`** is missing or invalid. The SDK still initializes: the OTLP endpoint defaults to Langfuse Cloud, auth headers are unset, spans are buffered locally, and export is rejected. The agent keeps running.

    **Solution:** Set both **`LANGFUSE_*`** keys in **`.env`** (load before SDK imports). Enable OpenTelemetry debug logging—see [Debug trace export](#debug-trace-export)—to confirm export errors.
  </Accordion>

  <Accordion title="No LLM or tool spans in the trace">
    **Cause:** Missing instrumentation extras, **`configure_observability()`** called after gateway or agent creation, or sync **`AIGateway`** without **`OpenAIInstrumentor`**.

    **Solution:** Install the matching **`[instrument-*]`** extra. Call **`configure_observability()`** before creating **`AIGateway`** or agent instances. For gateway-only sync scripts, use **`AsyncAIGateway`** (auto-instruments when the tracer is up) or attach **`OpenAIInstrumentor`** as in [Gateway-only script](#gateway-only-script).
  </Accordion>

  <Accordion title="Two trace IDs for one HTTP request">
    **Cause:** Sync **`team.run()`** in a FastAPI handler, or missing FastAPI/Agno extras.

    **Solution:** Use **`await team.arun(...)`**. Install **`[instrument-fastapi,instrument-agno]`**. See [Multi-agent HTTP services](#multi-agent-http-services).
  </Accordion>

  <Accordion title="Traces not reaching the backend (proxy)">
    **Cause:** Corporate web proxy intercepting OTLP export.

    **Solution:** Set **`NO_PROXY`** to include your Langfuse, Grafana, or OTLP host. See [Proxy configuration](#proxy-configuration).
  </Accordion>

  <Accordion title="High memory usage">
    **Cause:** Span queue growing when export fails.

    **Solution:** Fix network connectivity to the export endpoint. Lower **`otlp_max_queue_size`** if needed.
  </Accordion>

  <Accordion title="OBSERVABILITY_ENABLED has no effect">
    **Cause:** Environment variables loaded after SDK import.

    **Solution:**

    ```python theme={"system"}
    from dotenv import load_dotenv

    load_dotenv()
    from bb_ai_sdk.observability import configure_observability
    ```
  </Accordion>
</AccordionGroup>

## API reference

### Configure\_observability()

<ResponseField name="configure_observability" type="function">
  **Application entry point for instrumentation.** When the tracer is not initialized yet and **`init_observability=True`** (default), the SDK calls **`init(service_name=..., environment=...)`**, then **`instrument()`** for HTTPX, threading, framework OTel, and (when **`fastapi_app`** is set) FastAPI route spans and **`X-Observability-Project`** middleware.

  Export backend, OTLP batch tuning, organization fields, and custom Resource attributes are **not** parameters on this function — set them on **`init()`** and pass **`init_observability=False`**. See [Advanced export settings](#advanced-export-settings).

  <Expandable title="Parameters">
    <ResponseField name="service_name" type="str | None">
      Identifies the deployable in traces as **`service.name`**. When **`fastapi_app`** is set and this is omitted, defaults to **`fastapi_app.title`**, then **`"bb-ai-agent-app-service"`**. Required for non-HTTP processes when **`init_observability=True`** and the tracer is not yet up.
    </ResponseField>

    <ResponseField name="framework" type="str | None">
      Framework to instrument: **`"agno"`**, **`"langchain"`**, or **`"langgraph"`**. Requires the matching **`[instrument-*]`** extra. Omit for gateway-only scripts with no agent framework.
    </ResponseField>

    <ResponseField name="fastapi_app" type="FastAPI | None">
      When set, the SDK instruments HTTP routes and installs observability middleware on this app. Repeating **`configure_observability()`** on the same app is a no-op after the first successful run.
    </ResponseField>

    <ResponseField name="environment" type="str | None">
      Deployment environment forwarded to **`init()`** when the tracer is initialized by this call—for example **`development`**, **`staging`**, or **`production`**.
    </ResponseField>

    <ResponseField name="project_headers" type="Iterable[str]" default="x-observability-project">
      HTTP headers read for **`bb.project`** when **`fastapi_app`** is set.
    </ResponseField>

    <ResponseField name="init_observability" type="bool" default="true">
      When **`True`** and the tracer is not initialized, calls **`init()`** with **`service_name`** and **`environment`** only. Set **`False`** when you already called **`init()`** with advanced kwargs (**`backend`**, **`otlp_*`**, org fields, batch tuning).
    </ResponseField>
  </Expandable>
</ResponseField>

### Scope()

<ResponseField name="scope" type="context manager">
  Sets **`bb.*`** context for the current async or sync block. The SDK applies values on span start via **`ScopeAttributeSpanProcessor`**.

  <Expandable title="Parameters">
    <ResponseField name="agent_name" type="str | None">
      Maps to **`bb.agent.name`** for this run.
    </ResponseField>

    <ResponseField name="agent_id" type="str | None">
      Maps to **`bb.agent.id`** for this run.
    </ResponseField>

    <ResponseField name="trace_name" type="str | None">
      Maps to **`bb.trace.name`**.
    </ResponseField>

    <ResponseField name="trace_input" type="str | None">
      Maps to **`bb.trace.input`**—optional preview of run input.
    </ResponseField>

    <ResponseField name="project" type="str | None">
      Maps to **`bb.project`** when not set by **`X-Observability-Project`** middleware.
    </ResponseField>
  </Expandable>
</ResponseField>

### Init() (observability)

<ResponseField name="init" type="function">
  Low-level TracerProvider and OTLP export setup. **`configure_observability()`** calls this with **`service_name`** and **`environment`** only when **`init_observability=True`** and the tracer is not yet initialized.

  Call **`init()`** yourself when you need parameters **`configure_observability()`** does not expose — then pass **`init_observability=False`** to **`configure_observability()`**. For full manual control without **`configure_observability()`**, call **`init()`** and **`instrument()`** — see [Manual assembly](#manual-assembly).

  <Expandable title="Parameters">
    <ResponseField name="service_name" type="str | None">
      Deployable service name (OpenTelemetry **`service.name`**). Use for FastAPI apps and multi-agent hosts. Does not set **`bb.agent.name`** on the Resource unless **`agent_name`** is also provided.
    </ResponseField>

    <ResponseField name="agent_name" type="str | None">
      Legacy single-process label. When **`service_name`** is omitted, sets **`service.name`** and mirrors **`bb.agent.name`** on the Resource. Prefer **`service_name`** plus **`scope(agent_name=...)`** for multi-agent services.
    </ResponseField>

    <ResponseField name="backend" type="str | None">
      Backend preset: **`langfuse`**, **`grafana`**, **`custom`**, or **`datadog`**. Default **`None`** — the legacy path still resolves a Langfuse Cloud endpoint when **`OTEL_EXPORTER_OTLP_ENDPOINT`** is unset; pass **`backend=`** to use preset auth and endpoint helpers.
    </ResponseField>

    <ResponseField name="agent_id" type="str | None">
      Stable agent UUID. Maps to **`bb.agent.id`** on the Resource. **`service.instance.id`** is always a per-process UUID generated by the SDK—not **`agent_id`**.
    </ResponseField>

    <ResponseField name="organization_id" type="str | None">
      Organization ID for multi-tenant context tracking.
    </ResponseField>

    <ResponseField name="organization_name" type="str | None">
      Human-readable organization name for filtering traces.
    </ResponseField>

    <ResponseField name="environment" type="str" default="development">
      Environment name: **`development`**, **`staging`**, or **`production`**.
    </ResponseField>

    <ResponseField name="otlp_endpoint" type="str | None">
      Custom OTLP endpoint URL.
    </ResponseField>

    <ResponseField name="otlp_headers" type="dict | None">
      Custom OTLP headers.
    </ResponseField>

    <ResponseField name="otlp_batch_size" type="int" default="512">
      Maximum spans per export batch.
    </ResponseField>

    <ResponseField name="otlp_batch_timeout" type="float" default="5.0">
      Maximum wait time before exporting (in seconds).
    </ResponseField>

    <ResponseField name="otlp_max_queue_size" type="int" default="10000">
      Maximum buffered spans.
    </ResponseField>

    <ResponseField name="enabled" type="bool" default="true">
      When **`false`**, tracing becomes a no-op. Also controlled by **`OBSERVABILITY_ENABLED`**.
    </ResponseField>

    <ResponseField name="resource_attributes" type="dict | None">
      Custom OpenTelemetry resource attributes.
    </ResponseField>

    <ResponseField name="langfuse_public_key" type="str | None">
      Langfuse public key (overrides environment variable).
    </ResponseField>

    <ResponseField name="langfuse_secret_key" type="str | None">
      Langfuse secret key (overrides environment variable).
    </ResponseField>

    <ResponseField name="grafana_bearer_token" type="str | None">
      Grafana bearer token (overrides environment variable).
    </ResponseField>
  </Expandable>
</ResponseField>

### Trace()

<ResponseField name="trace" type="decorator">
  Decorator that creates OpenTelemetry spans for functions.

  <Expandable title="Parameters">
    <ResponseField name="name" type="str | None">
      Span name. Defaults to **`module.function_name`**.
    </ResponseField>

    <ResponseField name="attributes" type="dict | None">
      Dictionary of span attributes.
    </ResponseField>
  </Expandable>
</ResponseField>

### Trace\_context()

<ResponseField name="trace_context" type="context manager">
  Context manager for manual span control.

  <Expandable title="Parameters">
    <ResponseField name="name" type="str" required>
      Span name.
    </ResponseField>

    <ResponseField name="attributes" type="dict | None">
      Initial span attributes.
    </ResponseField>
  </Expandable>
</ResponseField>

### Get\_tracer\_provider()

<ResponseField name="get_tracer_provider" type="function">
  Returns the **`TracerProvider`** the SDK created—use with manual instrumentors such as **`OpenAIInstrumentor`**.
</ResponseField>

### Callback handlers

<ResponseField name="LangChainOpenTelemetryCallbackHandler" type="class">
  Callback handler for LangChain operations when not using **`configure_observability(framework="langchain")`**.
</ResponseField>

<ResponseField name="LangGraphOpenTelemetryCallbackHandler" type="class">
  Callback handler for LangGraph operations. Optional **`graph_name`** constructor argument.
</ResponseField>

## Next steps

<CardGroup cols={2}>
  <Card title="Logging and redaction" icon="shield-halved" href="/agentic-ai/bb-ai-sdk/logging">
    Log formats, redaction patterns, and correlation with traces
  </Card>

  <Card title="Evaluate your agent" icon="flask" href="/agentic-ai/bb-ai-sdk/evaluation-framework">
    Run evals with the same **`service_name`** and export setup
  </Card>

  <Card title="Multi agent starter" icon="users" href="/agentic-ai/starter-kits/multi-agent">
    Full FastAPI + Agno + observability reference implementation
  </Card>

  <Card title="Get started" icon="play" href="/agentic-ai/bb-ai-sdk/get-started">
    Minimal gateway + tracing example
  </Card>
</CardGroup>