> ## Documentation Index
> Fetch the complete documentation index at: https://docs.agentsentinel.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Framework Integrations

> Integrate Agent Sentinel with LangChain, CrewAI, AutoGen, and other agent frameworks.

## Overview

Agent Sentinel provides native integrations with popular agent frameworks, enabling automatic tracking of:

* Agent actions and tool calls
* LLM invocations
* Chain execution
* Token usage and costs
* Errors and retries

The "Big Three" frameworks are supported:

* **LangChain**: Callback-based integration
* **CrewAI**: Wrapper-based integration
* **AutoGen**: Hook-based integration (NEW!)

## AutoGen integration

### SentinelInspector

Microsoft's AutoGen is the leading framework in enterprise environments. Agent Sentinel provides seamless integration using AutoGen's built-in `register_reply` hook system.

```python theme={null}
from autogen import AssistantAgent, UserProxyAgent
from agent_sentinel.integrations.autogen import SentinelInspector

# Create the Sentinel inspector
sentinel = SentinelInspector(
    run_name="my_autogen_run",
    enforce_policies=True,  # Enable active blocking
    track_costs=True,       # Track LLM costs
    track_messages=True,    # Track message flow
)

# Create AutoGen agents as normal
assistant = AssistantAgent(
    name="assistant",
    system_message="You are a helpful AI assistant.",
    llm_config={
        "model": "gpt-4",
        "api_key": "your-api-key-here"
    }
)

user_proxy = UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=10,
)

# Secure the agents with one line each
sentinel.register(assistant)
sentinel.register(user_proxy)

# Run as normal - Sentinel is now monitoring everything
sentinel.start_run()

user_proxy.initiate_chat(
    assistant,
    message="What's the weather in San Francisco?"
)

sentinel.end_run()

# Get execution summary
summary = sentinel.get_run_summary()
print(f"Cost: ${summary['run_cost_usd']:.6f}")
print(f"Messages: {summary['message_count']}")
print(f"LLM Calls: {summary['llm_call_count']}")
```

### How it works

AutoGen's architecture is different from LangChain/CrewAI:

1. **Agents communicate via messages**: Agents send messages back and forth
2. **Reply chain system**: AutoGen uses `register_reply()` to inject hooks
3. **LLM config**: LLMs are configured through `llm_config` dict

Sentinel leverages this by:

1. **Hooking the reply chain** (position 0 = highest priority)
2. **Wrapping LLM generation** to track token costs
3. **Monitoring message flow** for audit trail

This is **simpler than CrewAI** because AutoGen has a built-in hook system designed exactly for this purpose!

### What's tracked

The inspector automatically tracks:

1. **Agent-to-agent messages**:
   * Sender and recipient
   * Message content (preview)
   * Message count per agent
   * Timestamp and sequence

2. **LLM calls**:
   * Model name
   * Token usage (prompt + completion)
   * Calculated cost
   * Duration

3. **Policy enforcement**:
   * Authorization checks before replies
   * Budget validation
   * Rate limiting
   * Intervention recording

### Run lifecycle

Mark run boundaries for accurate tracking:

```python theme={null}
sentinel = SentinelInspector(run_name="conversation_1")

# Register agents
sentinel.register(assistant)
sentinel.register(user_proxy)

# Mark start
sentinel.start_run()

try:
    # Run conversation
    user_proxy.initiate_chat(assistant, message="Hello")
    
    # Mark successful completion
    sentinel.end_run(outcome="completed")
    
except Exception as e:
    # Mark failed completion
    sentinel.end_run(outcome="failed")
    raise
```

### Run summary

Get detailed statistics after execution:

```python theme={null}
summary = sentinel.get_run_summary()

print(f"Run: {summary['run_name']}")
print(f"Duration: {summary['duration_seconds']:.2f}s")
print(f"Cost: ${summary['run_cost_usd']:.6f}")
print(f"Messages: {summary['message_count']}")
print(f"LLM Calls: {summary['llm_call_count']}")

# Per-agent breakdown
for agent_name, count in summary['agent_message_counts'].items():
    print(f"  {agent_name}: {count} messages")

# Cost breakdown
for action, cost in summary['action_costs'].items():
    print(f"  {action}: ${cost:.6f}")
```

### Policy enforcement

Sentinel blocks agents that violate policies:

```python theme={null}
from agent_sentinel.policy import PolicyEngine

# Set strict budget
PolicyEngine.configure(run_budget=0.10)  # $0.10 max

sentinel = SentinelInspector(
    run_name="limited_run",
    enforce_policies=True  # Active blocking enabled
)

sentinel.register(assistant)
sentinel.start_run()

try:
    # This will be blocked if cost exceeds $0.10
    user_proxy.initiate_chat(
        assistant,
        message="Write a very long essay..."
    )
except BudgetExceededError as e:
    print(f"BLOCKED: {e}")
    # The assistant was prevented from replying
```

When blocked, Sentinel:

1. Returns a blocking message to the agent
2. Records an intervention for dashboard visibility
3. Prevents the agent from generating a reply
4. Raises `BudgetExceededError` to stop execution

### Convenience function

Create and secure agents in one step:

```python theme={null}
from agent_sentinel.integrations.autogen import create_sentinel_agents
from autogen import AssistantAgent, UserProxyAgent

agents, sentinel = create_sentinel_agents(
    agent_configs=[
        {
            "agent_class": AssistantAgent,
            "name": "assistant",
            "llm_config": {"model": "gpt-4", "api_key": "..."}
        },
        {
            "agent_class": UserProxyAgent,
            "name": "user_proxy",
            "human_input_mode": "NEVER"
        }
    ],
    run_name="my_run"
)

# Agents are already secured
sentinel.start_run()
agents[1].initiate_chat(agents[0], message="Hello!")
sentinel.end_run()
```

## LangChain integration

### SentinelCallbackHandler

Use Agent Sentinel's callback handler to track all LangChain activity:

```python theme={null}
from agent_sentinel.integrations.langchain import SentinelCallbackHandler
from langchain_openai import ChatOpenAI
from langchain.agents import create_openai_functions_agent, AgentExecutor
from langchain.tools import tool

# Create callback handler
sentinel_handler = SentinelCallbackHandler(
    agent_id="langchain-agent",
    run_id="run-123"
)

# Define tools
@tool
def search(query: str) -> str:
    """Search for information"""
    return f"Results for: {query}"

@tool
def calculate(expression: str) -> str:
    """Calculate a math expression"""
    return str(eval(expression))

# Create agent with callback
llm = ChatOpenAI(model="gpt-4o", callbacks=[sentinel_handler])
tools = [search, calculate]

agent = create_openai_functions_agent(llm, tools, prompt)
agent_executor = AgentExecutor(
    agent=agent,
    tools=tools,
    callbacks=[sentinel_handler],
    verbose=True
)

# Run agent - all activity automatically tracked
result = agent_executor.invoke({
    "input": "What is 15 * 23?"
}, config={"callbacks": [sentinel_handler]})
```

### What's tracked

The callback handler automatically tracks:

**1. LLM calls**:

* Model name (normalized to pricing database)
* Prompt tokens
* Completion tokens
* Total tokens
* Calculated cost (using latest pricing data)
* Duration
* Run ID and parent run ID (for nested chains)

**2. Tool/function calls**:

* Tool name
* Input arguments (first 200 chars)
* Output/result (first 200 chars)
* Duration
* Success/failure status
* Error details (if failed)

**3. Agent actions**:

* Tool selection
* Agent reasoning
* Observations
* Action outcomes

**4. Chain execution**:

* Chain type/name
* Chain start/end
* Duration
* Nested chain relationships (parent/child)
* Error handling

**5. Policy enforcement** (when `enforce_policies=True`):

* Authorization checks before LLM calls
* Authorization checks before tool execution
* Budget validation
* Intervention recording when actions are blocked

### Policy enforcement

Sentinel actively blocks operations that violate policies:

```python theme={null}
from agent_sentinel import PolicyEngine, BudgetExceededError
from agent_sentinel.integrations.langchain import SentinelCallbackHandler
from langchain_openai import ChatOpenAI

# Set strict budget
PolicyEngine.configure(run_budget=0.10)  # $0.10 max

sentinel_handler = SentinelCallbackHandler(
    run_name="limited_run",
    enforce_policies=True  # Active blocking (default: True)
)

llm = ChatOpenAI(model="gpt-4", callbacks=[sentinel_handler])

try:
    # LLM call authorization checked BEFORE API call
    response = llm.invoke("Write a very long essay...")
    # If current spend + estimated cost > budget, blocked
except BudgetExceededError as e:
    print(f"BLOCKED: {e}")
    # LLM call was prevented
    # Intervention recorded for dashboard
```

When blocked:

1. `on_llm_start()` runs authorization check
2. `PolicyEngine.check_action()` validates budget
3. If violation, `BudgetExceededError` is raised
4. Intervention is recorded (type, reason, cost, inputs)
5. LLM call never reaches the API
6. Exception propagates to your code

The same authorization happens for tools via `on_tool_start()`.

### Run summary

Get a complete summary after execution:

```python theme={null}
# After agent completes
summary = sentinel_handler.get_run_summary()

print(f"Total cost: ${summary['total_cost_usd']:.4f}")
print(f"LLM calls: {summary['llm_calls']}")
print(f"Tool calls: {summary['tool_calls']}")
print(f"Total actions: {summary['total_actions']}")
print(f"Errors: {summary['errors']}")

# Cost breakdown by action type
for action_type, cost in summary['costs_by_action'].items():
    print(f"  {action_type}: ${cost:.4f}")
```

### Async LangChain

The callback handler supports async chains:

```python theme={null}
from langchain.chains import LLMChain

chain = LLMChain(llm=llm, prompt=prompt, callbacks=[sentinel_handler])

# Async execution
result = await chain.ainvoke({"input": "Hello"})
```

## LangGraph integration

### SentinelToolNode

`SentinelToolNode` wraps a LangGraph tool node so every tool invocation runs through the Sentinel policy engine — including evidence requirements, grounding rules, and structured self-repair feedback when a call is rejected.

```python theme={null}
from agent_sentinel import PolicyConfig, PolicyEngine
from agent_sentinel.integrations.langgraph import SentinelToolNode

PolicyEngine._config = PolicyConfig(
    evidence_requirements={"issue_refund": ["lookup_order"]},
    grounding_rules={
        "issue_refund": {
            "amount": {"source_action": "lookup_order", "source_field": "amount"},
        }
    },
    commit_actions=["issue_refund"],
    evidence_actions=["lookup_order", "validate_refund"],
)
PolicyEngine._initialized = True

tools = SentinelToolNode(
    tools={
        "lookup_order":   (lookup_order,   {"produces_evidence": True}),
        "validate_refund": (validate_refund, {"produces_evidence": True}),
        "issue_refund": (
            issue_refund,
            {
                "is_commit": True,
                "requires": ["lookup_order"],
                "grounding_rules": {
                    "amount": {"source_action": "lookup_order", "source_field": "amount"},
                },
            },
        ),
    }
)

graph_builder.add_node("tools", tools)
```

Each entry maps a tool name to `(callable, guard_kwargs)` where `guard_kwargs` accepts the same metadata as `@guarded_action` (`is_commit`, `requires`, `grounding_rules`, `produces_evidence`, `risk_level`, etc.).

### Self-repair on block

When the policy engine rejects a tool call, `SentinelToolNode` returns a `ToolMessage` whose content is a **structured remediation payload** the LLM can read on the next turn:

```json theme={null}
{
  "blocked": true,
  "reason_code": "MISSING_EVIDENCE",
  "reason": "issue_refund requires evidence from lookup_order",
  "retry_guidance": "Call lookup_order(order_id=...) first, then retry issue_refund."
}
```

Most frontier models will read the `retry_guidance` and self-correct without any prompt-engineering on your side.

### Working example

A complete runnable example lives at [`examples/langgraph_sentinel.py`](https://github.com/agent-sentinel/agent-sentinel-sdk/blob/main/examples/langgraph_sentinel.py) in the public SDK repo — a 3-tool refund workflow that demonstrates evidence requirements, grounding constraints, and self-repair when the agent attempts a commit before the prerequisite lookup.

```bash theme={null}
pip install langchain-core langgraph
python examples/langgraph_sentinel.py
```

## CrewAI integration

### SentinelCrew wrapper

`SentinelCrew` provides **automatic security injection** for CrewAI - transforming it from passive tracking to active "Visa-like" control:

```python theme={null}
from crewai import Agent, Task
from crewai_tools import SerperDevTool
from agent_sentinel.integrations.crewai import SentinelCrew

# Standard CrewAI setup - NO CHANGES NEEDED
search_tool = SerperDevTool()

researcher = Agent(
    role="Researcher",
    goal="Research topics thoroughly",
    backstory="Expert researcher with attention to detail",
    tools=[search_tool],  # Tools are auto-secured!
    verbose=True
)

writer = Agent(
    role="Writer",
    goal="Write compelling content based on research",
    backstory="Professional writer with 10 years experience",
    verbose=True
)

# Define tasks
research_task = Task(
    description="Research the latest trends in AI safety for 2025",
    agent=researcher,
    expected_output="Detailed research findings with citations"
)

writing_task = Task(
    description="Write a 500-word article based on the research",
    agent=writer,
    expected_output="Publication-ready article",
    context=[research_task]
)

# SentinelCrew automatically secures everything
crew = SentinelCrew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    run_name="ai_safety_article",
    enforce_policies=True,   # Active blocking (default: True)
    max_agent_steps=50,      # Prevent runaway agents
    detect_loops=True,       # Detect infinite loops
    verbose=True
)

# Fully secured execution
result = crew.kickoff()

# Get detailed summary
summary = crew.get_run_summary()
print(f"Duration: {summary['duration_seconds']:.2f}s")
print(f"Cost: ${summary['run_cost_usd']:.6f}")
print(f"Agents: {summary['num_agents']}")
print(f"Tasks: {summary['num_tasks']}")
```

### What's automatically secured

`SentinelCrew` injects security at three levels:

**1. Tool Injection (The "Chip Reader")**

* Wraps ALL agent tools automatically
* No manual decoration required
* Works with SerperDevTool, DuckDuckGoSearch, FileReadTool, etc.
* Authorization checks run BEFORE tool execution
* Failed authorization blocks the tool and records intervention

**2. LLM Monitoring (The "LLM Meter")**

* Attaches `SentinelCallbackHandler` to all agent LLMs
* Tracks token costs in real-time
* Enforces budget limits before expensive API calls
* Works with OpenAI, Anthropic, and other LangChain-compatible LLMs

**3. Step Monitoring (The "Safety Net")**

* Tracks agent step counts
* Detects runaway agents (infinite loops)
* Enforces `max_agent_steps` limit
* Identifies repetition patterns
* Records interventions when agents are stopped

### Run summary

Get comprehensive execution statistics:

```python theme={null}
summary = crew.get_run_summary()

# Output:
{
    "run_name": "ai_safety_article",
    "num_agents": 2,
    "num_tasks": 2,
    "duration_seconds": 45.7,
    "run_cost_usd": 0.234,
    "total_cost_usd": 0.234,
    "action_counts": {
        "tool:search_tool": 3,
        "llm_call:gpt-4o": 5
    },
    "action_costs": {
        "llm_call:gpt-4o": 0.234
    },
    "started_at": 1704390000.0,
    "completed_at": 1704390045.7
}
```

### Policy enforcement example

Prevent runaway costs with active blocking:

```python theme={null}
from agent_sentinel import PolicyEngine, BudgetExceededError
from agent_sentinel.integrations.crewai import SentinelCrew

# Set strict budget
PolicyEngine.configure(run_budget=0.50)  # $0.50 max

crew = SentinelCrew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    run_name="limited_run",
    enforce_policies=True  # Active blocking enabled
)

try:
    result = crew.kickoff()
except BudgetExceededError as e:
    print(f"BLOCKED: {e}")
    # Tool or LLM call was prevented from executing
    # Intervention recorded for dashboard visibility
```

When a tool or LLM is blocked:

1. Authorization fails before execution
2. Intervention is recorded (type, reason, risk level)
3. `BudgetExceededError` or `PolicyViolationError` is raised
4. Agent execution stops cleanly
5. Visible in Console → Interventions page

### Runaway agent protection

Prevent infinite loops and excessive iterations:

```python theme={null}
crew = SentinelCrew(
    agents=[researcher],
    tasks=[task],
    run_name="protected_run",
    max_agent_steps=50,      # Stop after 50 steps
    detect_loops=True,       # Detect repetition
)

# If agent exceeds 50 steps or repeats the same action 5 times:
# - PolicyViolationError is raised
# - Intervention is recorded
# - Execution stops
# - Dashboard shows the issue
```

**Loop detection**: If an agent repeats the same action 5+ times in a row, a warning is logged and an intervention is recorded.

**Step limit**: If an agent exceeds `max_agent_steps`, a critical intervention is recorded and execution is blocked.

### Wrapping existing crews

Retrofit existing CrewAI crews without rewriting code:

```python theme={null}
from agent_sentinel.integrations.crewai import wrap_existing_crew
from crewai import Crew, Agent, Task

# Existing crew code
agent = Agent(role="Researcher", goal="Research topics", backstory="Expert")
task = Task(description="Research AI", agent=agent)
crew = Crew(agents=[agent], tasks=[task])

# Add Sentinel tracking
wrapped_crew = wrap_existing_crew(
    crew=crew,
    agent_id="legacy-crew",
    run_id="run-789"
)

# Execute - now tracked
result = wrapped_crew.kickoff()
summary = wrapped_crew.get_run_summary()
```

### Individual action wrapping

For fine-grained control, wrap individual actions:

```python theme={null}
from agent_sentinel.integrations.crewai import wrap_crew_action
from crewai import Agent, Task

@wrap_crew_action(name="web_search", cost_usd=0.02, tags=["search", "external"])
def search_web(query: str) -> str:
    """Search the web for information."""
    # Your search logic
    results = perform_search(query)
    return results

@wrap_crew_action(name="analyze_data", cost_usd=0.01, tags=["analysis"])
def analyze_data(data: dict) -> dict:
    """Analyze data and extract insights."""
    # Your analysis logic
    insights = extract_insights(data)
    return insights

# Use wrapped tools with agents
analyst = Agent(
    role="Data Analyst",
    goal="Analyze data and provide insights",
    backstory="Expert data analyst",
    tools=[search_web, analyze_data]  # Both tools are tracked
)

# Each tool execution is logged with:
# - Action name (web_search, analyze_data)
# - Cost ($0.02, $0.01)
# - Duration
# - Tags (search, external, analysis)
# - Success/failure status
```

The `@wrap_crew_action` decorator is a thin wrapper around `@guarded_action` that adds CrewAI-specific tags and metadata.

## Custom framework integration

For frameworks not yet supported, use the low-level `@guarded_action` decorator:

```python theme={null}
from agent_sentinel import guarded_action
from your_framework import Agent, Task

class TrackedAgent(Agent):
    @guarded_action(name="agent_step", cost_usd=0.0)
    def step(self, task):
        # Your agent logic
        result = super().step(task)
        return result

    @guarded_action(name="tool_call", cost_usd=0.02)
    def call_tool(self, tool_name, args):
        result = super().call_tool(tool_name, args)
        return result
```

## Combining integrations

Use multiple integrations together:

```python theme={null}
from agent_sentinel.integrations import instrument_openai
from agent_sentinel.integrations.langchain import SentinelCallbackHandler
from langchain_openai import ChatOpenAI

# Instrument OpenAI globally
instrument_openai()

# Use LangChain callback for structured tracking
handler = SentinelCallbackHandler(agent_id="hybrid-agent")

llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])

# Both instrumentation layers work together:
# 1. OpenAI instrumentation tracks raw API calls + costs
# 2. LangChain handler tracks chain/agent structure
```

## Best practices

<Tip>
  **Use framework integrations when available**: Framework-specific integrations provide better structure and context than raw `@guarded_action` decorators.
</Tip>

<Tip>
  **Set agent\_id and run\_id**: Always provide identifiers for filtering and analysis in the web console.
</Tip>

<Tip>
  **Review run summaries**: Use `get_run_summary()` to understand cost breakdown and identify expensive operations.
</Tip>

<Warning>
  **Framework compatibility**: Test integrations when upgrading LangChain or CrewAI versions, as internal APIs may change.
</Warning>

## Troubleshooting

### "LangChain events not tracked"

Ensure callbacks are passed at all levels:

```python theme={null}
# ✅ Correct - callbacks at LLM, agent, and executor
llm = ChatOpenAI(callbacks=[handler])
agent = create_openai_functions_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, callbacks=[handler])
result = executor.invoke(input, config={"callbacks": [handler]})

# ❌ Wrong - missing callbacks
llm = ChatOpenAI()  # No callbacks!
executor = AgentExecutor(agent=agent, tools=tools)
result = executor.invoke(input)  # Not tracked
```

### "CrewAI costs not accurate"

CrewAI cost tracking depends on:

1. LLM instrumentation (e.g., `instrument_openai()`)
2. Tool cost annotations via `@wrap_crew_action`

Ensure both are configured for accurate cost tracking.

### "Duplicate events"

If using both LLM instrumentation and framework callbacks, you may see duplicate LLM call records. This is expected - one from the low-level instrumentation, one from the framework callback. The framework callback provides richer context (chain name, agent reasoning) while the low-level instrumentation provides precise token costs.

## Example: Full stack tracking

```python theme={null}
from agent_sentinel import enable_remote_sync, PolicyEngine
from agent_sentinel.integrations import instrument_openai
from agent_sentinel.integrations.langchain import SentinelCallbackHandler
from langchain_openai import ChatOpenAI
from langchain.agents import create_openai_functions_agent, AgentExecutor

# 1. Configure policies
PolicyEngine.configure(
    session_budget=5.0,
    run_budget=1.0,
    rate_limits={
        "openai_chat_completion": {
            "max_count": 50,
            "window_seconds": 60
        }
    }
)

# 2. Enable platform sync
enable_remote_sync(
    platform_url="https://platform.agentsentinel.dev",
    api_token="as_your_api_key_here",
    run_id="run-production-123"
)

# 3. Instrument LLM provider
instrument_openai()

# 4. Create LangChain agent with callback
handler = SentinelCallbackHandler(
    agent_id="production-agent",
    run_id="run-production-123"
)

llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])
agent = create_openai_functions_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, callbacks=[handler])

# 5. Run agent - fully tracked with policies enforced
try:
    result = executor.invoke({"input": user_query}, config={"callbacks": [handler]})
    summary = handler.get_run_summary()
    print(f"Success! Cost: ${summary['total_cost_usd']:.4f}")
except BudgetExceededError:
    print("Budget exceeded - agent stopped")
```

## See also

* [LLM Integrations](/sdk/python/llm-integrations) - Direct LLM provider instrumentation
* [Instrumentation](/sdk/python/instrumentation) - Manual action tracking
* [Policies](/sdk/python/policies) - Cost and rate limiting