> ## Documentation Index
> Fetch the complete documentation index at: https://grandcentral.backbase.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent red teaming

> Systematic security testing for AI agents using Promptfoo

Red teaming is a systematic approach for finding vulnerabilities in AI agents before deployment. The starter kits include built-in support for adversarial testing using **Promptfoo**, enabling you to test for prompt injections, jailbreaks, harmful content generation, and other security issues.

<Info>
  All starter kits include red teaming which runs automatically as part of the CI/CD pipelines.
</Info>

## Overview

Red team testing helps identify security vulnerabilities by simulating adversarial attacks against your agents. The implementation tests across multiple categories:

* **Prompt injections**: Attempts to manipulate agent behavior through crafted inputs
* **Jailbreaks**: Techniques to bypass safety measures and restrictions
* **Harmful content**: Testing for hate speech, violence, and other harmful outputs
* **PII leakage**: Attempts to extract personally identifiable information
* **Prompt extraction**: Attempts to reveal system prompts and internal instructions

## File structure

```text theme={"system"}
starter-agent/
├── redteam/
│   ├── configs/                        # Promptfoo source configurations
│   │   ├── <agent_name>_promptfooconfig.yaml
│   └── tests/                          # Generated adversarial test cases
│       ├── <agent_name>_redteam.yaml
```

## Running red team tests

### Prerequisites

* **Node.js 20+**: Required for Promptfoo CLI
* **Running agent server**: Your agent must be accessible at its endpoint

### Step 1: Start your agent

```bash theme={"system"}
source .env && uv run python -m src.main
```

### Step 2: Run tests

Run the red team evaluation for any of the available agents:

```bash theme={"system"}
npx promptfoo@latest redteam eval \
  -c redteam/tests/<agent_name>_redteam.yaml
```

### Step 3: View the report

After running tests, launch the interactive report viewer:

```bash theme={"system"}
npx promptfoo@latest redteam report
```

The report provides:

* **Vulnerability categories**: Types of issues found (injections, harmful content, etc.)
* **Severity levels**: Classification by potential impact
* **Detailed logs**: Specific inputs that triggered vulnerabilities
* **Suggested mitigations**: Recommendations for addressing issues

## Configuration

The configuration file defines the target agent endpoint and red team settings. The following example shows the configuration:

```yaml theme={"system"}
description: Instructions Agent Red Team Configuration

targets:
  - id: http
    label: instructions-agent
    config:
      url: http://localhost:8000/run/instructions_agent
      method: POST
      headers:
        Content-Type: application/json
      body: '{"query":"{{prompt}}"}'
      transformResponse: json.output || json.detail

prompts:
  - '{{prompt}}'

redteam:
  purpose: |-
    An AI assistant that follows user instructions precisely 
    and provides concise, relevant responses.
  
  plugins:
    - id: harmful:hate
      numTests: 5
    - id: harmful:insults
      numTests: 5
    - id: pii:direct
      numTests: 5
    - id: prompt-extraction
      numTests: 5
    - id: hijacking
      numTests: 5
    
  strategies:
    - id: basic
    - id: jailbreak:meta
  
  numTests: 5
```

<Info>
  For details about available plugins, attack strategies, and advanced configuration options, see the **[Promptfoo Red Team documentation](https://www.promptfoo.dev/docs/red-team/)**.
</Info>

## Generating new test cases

To generate fresh adversarial test cases from your configuration:

### Interactive setup

```bash theme={"system"}
# Web UI setup
npx promptfoo@latest redteam setup

# CLI-only setup
npx promptfoo@latest redteam init --no-gui
```

### Generate from existing config

```bash theme={"system"}
npx promptfoo@latest redteam generate \
  -c redteam/configs/<agent_name>_promptfooconfig.yaml \
  -o redteam/tests/<agent_name>_redteam.yaml
```

## Best practices

<CardGroup cols={2}>
  <Card title="Use guardrails" icon="lock">
    Combine red teaming with **[Guardrails](/agentic-ai/bb-ai-sdk/guardrails)** for defense-in-depth
  </Card>

  <Card title="Update test cases" icon="rotate">
    Regenerate test cases periodically to cover new attack vectors
  </Card>
</CardGroup>

## Next steps

* **[Guardrails](/agentic-ai/bb-ai-sdk/guardrails)**: Add runtime safety checks to your agents
* **[Evaluations](/agentic-ai/bb-ai-sdk/evaluation-framework)**: Set up comprehensive agent evaluation
* **[CI/CD Workflows](/agentic-ai/ci-cd-workflows/overview)**: Configure automated testing pipelines