Phase 1: Prepare APIs and data
Set up data sources, APIs, and MCP servers:- Configure MCP servers (banking domain or custom)
- Set up vector database for RAG (if needed)
- Document and expose APIs
Phase 2: Build agents
Implement your agent with framework, tools, and knowledge:- Choose framework (Agno, LangChain, etc.)
- Code agent logic and instructions
- Connect tools (MCP servers or APIs)
- Set up RAG if needed
- Create FastAPI endpoints
Phase 3: Evaluate agents
Test performance, safety, and quality before deployment:- Run unit tests
- Evaluate with LLM judges (Langfuse)
- Human review for complex cases
- Red teaming (security tests)
- Measure metrics (latency, cost, accuracy)
Phase 4: Run agents
Deploy to production and expose via APIM:- Deploy via CI/CD (auto on merge) or Helm
- Configure APIM endpoint
- Set up monitoring dashboards
- Configure alerts
Continuous: Monitor and govern
Ongoing monitoring, security, and compliance: Monitoring:- Track performance (latency, cost, errors)
- Review traces for issues
- Set up alerts (PagerDuty, Grafana)
- Enforce guardrails (AI Gateway, Nemo Guardrails)
- Run security scans regularly
- Periodic red teaming
- Compliance audits
Next steps
- Starter kits: Jumpstart build phase
- CI/CD workflows: Automate run phase
- Creating Your First Agent: End-to-end guide