Settings Page - Agent Sentinel

Overview

The Settings Page is your control center for configuring Agent Sentinel. Manage policies that control agent behavior, generate API keys for SDK integration, and set up notification channels.

Three main tabs

Policies - Budget limits, denied actions, rate limits, approval rules
API Keys - Generate and manage authentication tokens
Notifications - Configure alert channels (Slack, Email, Webhooks)

Policies tab

Policy list

View all configured policies with:

Name - Policy identifier
Description - What it does
Scope - Global, agent-specific, or run-specific
Enabled - Toggle switch (green=on, gray=off)
Last Updated - Timestamp of last modification

Quick actions:

Enable/Disable - Toggle switch (instant)
Edit - Modify policy configuration
Delete - Remove policy (with confirmation)
Test Alert - Send test notification

Create new policy

Click Create Policy to open policy builder:

Basic settings

Name (required) - Unique identifier
- Example: “Production Safety”, “Dev Budget”, “Rate Limits”
Description (optional) - What this policy does
- Example: “Strict limits for production agents”
Enabled - Start enabled or disabled
Scope - Who this applies to:
- Global - All agents
- Agent-specific - One agent (enter agent_id)
- Run-specific - One run (enter run_id)

Budget limits

Set cost constraints: Session budget (USD)

Limit for entire application session
Example: $50.00
Applies across all runs in a session

Run budget (USD)

Limit per individual run
Example: $5.00
Resets for each new run

Action-specific budgets (USD)

Limit per action type
Add multiple action budgets
Example:
- call_llm: $0.50
- search_web: $0.10
- transfer_funds: $0.00 (effectively blocks if > 0)

Behavior on violation:

SDK raises BudgetExceededError
Action is blocked before execution
Intervention is logged

Action control lists

Denied actions:

Actions that are completely blocked
Add action names one per line or comma-separated

Example:

delete_production_database
drop_table
rm_rf_root
send_all_customer_emails

Matched exactly (case-sensitive)

Allowed actions (allowlist mode):

If specified, only these actions are permitted
All other actions are blocked
Use for strict security
Example (read-only mode):
read_database query_api log_message
Leave empty to disable allowlist mode

Behavior on violation:

SDK raises PolicyViolationError
Action is blocked before execution
Intervention is logged with type HARD_BLOCK

Rate limiting

Limit action frequency to prevent runaway agents: Add rate limit:

Action name - Which action to limit
Max count - Maximum calls allowed
Window (seconds) - Time window for count

Examples:

Action: api_call
Max count: 100
Window: 60 seconds
→ Limit to 100 API calls per minute

Action: send_email
Max count: 10
Window: 3600 seconds
→ Limit to 10 emails per hour

Action: expensive_llm_call
Max count: 5
Window: 86400 seconds
→ Limit to 5 calls per day

Behavior on violation:

SDK raises PolicyViolationError
Action is blocked before execution
Intervention is logged with type RATE_LIMITED

Human approval settings

Configure when actions require human approval: Required approval actions:

List of actions that must be approved

Example:

transfer_funds
delete_user_data
modify_production_config
send_customer_communication

Cost threshold (USD):

Auto-require approval if action cost > threshold
Example: $100.00
Any action costing more than threshold requires approval

Timeout (seconds):

How long to wait for human decision
Example: 600 (10 minutes)
After timeout, action is blocked with TimeoutError

Default approvers (emails):

Who should be notified for approvals
Comma-separated email list

Example:

manager@company.com, security@company.com

Receives email notifications for approval requests

Behavior:

SDK creates approval request via platform
Blocks and waits for human decision
If approved, action executes
If rejected or timeout, raises error
Intervention is logged with type APPROVAL_REQUIRED

Alert configuration

Set up notifications when policy is triggered: Alert emails:

Email addresses to notify
Comma-separated
Example: ops@company.com, cto@company.com

Alert threshold (%):

Notify when budget reaches X% of limit
Example: 80
Sends email when 80% of budget is consumed

Alert on failure:

Checkbox to enable failure alerts
Sends email when action is blocked by this policy
Useful for monitoring security violations

Test alert button:

Send test email to verify configuration
Click to send immediately
Check inbox for delivery

Prose / YAML / JSON authoring

The policy editor accepts policy source in four formats: YAML, JSON, Markdown, or plain English prose. Paste your source into the editor and the panel calls POST /api/v1/policies/compile on every blur.

Format	Validation	Notes
YAML / JSON	Pydantic schema	No LLM calls; instant feedback
Markdown / Prose	Gemini → Pydantic	`gemini-2.5-flash` lowers prose to the policy IR; the same Pydantic validator then runs over the model output so a hallucinated field cannot create an invalid policy

Errors show as red squiggles at the offending key. Warnings show as yellow callouts (e.g., overlap between allowed_actions and denied_actions, approval_* fields set without require_approval=true, or “policy compiled from prose by Gemini — review every field”). Commit only when ok: true. Full reference: SDK → Prose policies.

Version history & rollback

Every successful policy update creates an immutable PolicyVersion snapshot. The version history panel on the right side of each policy shows:

Version number, status (active, archived, draft, in_review)
Author and reviewer (with avatar)
Change summary (“Updated: denied_actions, approval_threshold_usd”)
Relative timestamp (“5m ago”, “2h ago”)
Diff button — opens side-by-side YAML diff against the previous version
Rollback button — copies that version’s policy_data back onto the active policy and records the rollback as a new version

The active version is always at the top with a green badge. Rollback writes a new version (it doesn’t delete history), so an audit trail is preserved.

Evidence graph viewer

For policies with evidence_requirements, commit_actions, or grounding_rules, the policy editor renders an Evidence DAG using ReactFlow + dagre auto-layout:

Evidence nodes (blue) — actions that produce evidence
Commit nodes (red) — actions that require prior evidence and run with grounding constraints
Plain nodes (gray) — actions with no evidence relationship

Edges are typed:

Evidence edges — “X must run before Y”
Grounding edges (toggleable) — field-level constraints like “issue_refund.amount must equal lookup_order.amount”

Use the viewer to verify the prerequisite chain matches what you intended before activating the policy.

Policy precedence

When multiple policies apply, most restrictive wins:

Run-specific (highest priority)
Agent-specific
Global (lowest priority)

Example:

Global policy: $10 run budget
Agent-specific policy: $5 run budget
Result: $5 limit applies (most restrictive)

Policy sync to SDK

Policies sync to SDK automatically:

SDK calls PolicyEngine.enable_remote_sync()
Platform endpoint /api/v1/policies/sync returns enabled policies
SDK caches locally
Refreshes every 5 minutes (configurable)
Most restrictive rules are merged and applied

Viewing synced policies:

Check SDK logs for policy sync messages
See “Last Updated” in policy list
Monitor intervention logs for policy enforcement

API Keys tab

API key list

View all API keys for your organization:

Column	Description
Key Prefix	First 12 characters (e.g., `as_a1b2c3d4...`)
Name	Human-readable identifier (optional)
Created	When key was generated
Last Used	Most recent API call with this key
Status	Active (green) or Inactive (gray)

Actions per key:

Deactivate - Disable key without deleting
Activate - Re-enable deactivated key
Delete - Permanently remove key (with confirmation)

Generate new API key

Click Generate New Key:

Optional name - Identifier for this key
- Example: “Production Agent”, “Development”, “CI/CD Pipeline”
Click Generate
Copy immediately - Key shown only once
- Format: as_ + 32 hex characters
- Example: as_a1b2c3d4e5f6789012345678901234ab
Store securely in:
- Password manager
- Secrets vault (HashiCorp Vault, AWS Secrets Manager)
- Environment variables (never in code)

Keys are shown only once! If you lose it, you must generate a new key. There is no recovery option.

Copy confirmation:

Checkbox: “I have copied the key”
Must check to close modal
Prevents accidental loss

Using API keys

In SDK:

from agent_sentinel import enable_remote_sync

enable_remote_sync(
    platform_url="https://platform.agentsentinel.dev",
    api_token="as_a1b2c3d4e5f6789012345678901234ab",  # Your API key
    agent_id="my-agent",
    run_id="run-001"
)

In API requests:

curl -H "Authorization: Bearer as_a1b2c3d4e5f6789012345678901234ab" \
  https://platform.agentsentinel.dev/api/v1/runs

Security best practices

Rotate keys regularly: Generate new keys every 90 days and delete old ones.

Use separate keys per environment: Different keys for dev, staging, production.

Never commit keys to git: Use environment variables or secrets management.

Monitor “Last Used”: Keys not used in 90 days should be deleted.

Troubleshooting API keys

“API key invalid”

Check you copied full key including as_ prefix
Verify key is Active (not deactivated)
Confirm you’re in correct organization
Check key wasn’t deleted

“Key not working after creation”

May take up to 30 seconds to propagate
Verify you copied correctly (easy to truncate)

Test with simple API call:

curl -H "Authorization: Bearer $KEY" \
  https://platform.agentsentinel.dev/api/v1/users/me

Notifications tab

Channel status

View configured notification channels:

Channel	Status	Actions
Email	🟢 Enabled	Configure, Test, Disable
Slack	🔴 Not configured	Set up
Webhooks	🔴 Not configured	Set up
PagerDuty	🟡 Coming soon	-
Discord	🟡 Coming soon	-

Email notifications

Configuration:

Email addresses (comma-separated)
Notification triggers:
- ☑️ Critical interventions (risk=critical)
- ☑️ Budget threshold reached (80%)
- ☑️ Budget exceeded
- ☑️ Approval requests (critical/high priority)
- ☑️ Policy violations
- ☑️ Agent failures (3+ consecutive errors)

Test email:

Click “Send Test Email”
Check inbox for delivery
Verify formatting and content

Example email:

Subject: [Agent Sentinel] Critical Intervention

Agent "trading-bot" attempted dangerous action

Action: delete_production_database
Risk Level: CRITICAL
Intervention: HARD_BLOCK
Time: 2024-12-28 14:30:00 UTC

Blast Radius: Prevented deletion of 1M+ customer records

View Details: https://console.agentsentinel.dev/interventions/int_123

Slack notifications (coming soon)

Setup:

Click “Set up Slack”
Authorize Agent Sentinel app
Choose channel (e.g., #agent-alerts)
Configure triggers
Test notification

Notification format:

🚨 Critical Intervention

Agent: trading-bot
Action: delete_production_database
Status: BLOCKED
Risk: CRITICAL

[View Details]

Webhooks (coming soon)

Setup:

Enter webhook URL
Choose events to send:
- Interventions
- Approvals
- Policy violations
- Budget alerts
Configure secret for signature verification
Test webhook

Payload format:

{
  "event_type": "intervention_created",
  "timestamp": "2024-12-28T14:30:00Z",
  "data": {
    "intervention_id": "int_123",
    "type": "hard_block",
    "action_name": "dangerous_action",
    "risk_level": "critical",
    "agent_id": "my-agent"
  },
  "signature": "sha256=..."
}

Critical escalation notice

Critical interventions always notify: Regardless of settings, critical-risk interventions always trigger notifications to prevent disasters.

Common workflows

Create production safety policy

Go to Policies tab
Click Create Policy
Configure:
- Name: “Production Safety”
- Description: “Strict limits for production agents”
- Scope: Agent-specific → “production-agent”
- Session budget: $50
- Run budget: $5
- Denied actions:
  delete_database drop_table modify_production_config
- Rate limits:
  - api_call: 100 per 60s
  - database_write: 10 per 60s
Enable policy
Test with agent

Set up approval workflow

Go to Policies tab
Create or edit policy
Scroll to “Human Approval Settings”
Configure:
- Required approval actions:
  transfer_funds delete_user_data
- Cost threshold: $100.00
- Timeout: 600 seconds
- Default approvers:
  manager@company.com, security@company.com
Go to Notifications tab
Enable email for approval requests
Save and test

Rotate API key

Go to API Keys tab
Click Generate New Key
Name it with date: “Production Key 2024-12”
Copy key

Update SDK configuration:

enable_remote_sync(
    platform_url="https://platform.agentsentinel.dev",
    api_token="as_new_key_copied_from_settings",  # New key
    run_id="run-001"
)

Deploy updated configuration
Monitor “Last Used” on new key
Once new key is active (within 24h), delete old key

Configure budget alerts

Go to Policies tab
Edit budget policy
Scroll to “Alert Configuration”
Set:
- Alert emails: finance@company.com, engineering@company.com
- Alert threshold: 80%
- Alert on failure: ✓
Click Test Alert
Verify email received
Save policy

Best practices

Start with permissive policies: Begin with high budgets and few denied actions, then tighten based on observed behavior.

Use descriptive policy names: Include environment/purpose in name: “Prod Safety”, “Dev Budget”, “QA Rate Limits”.

Test policies before production: Create test agent and verify policy enforcement works as expected.

Separate keys per environment: Never use same API key for dev, staging, and production.

Monitor policy effectiveness: Review intervention logs to ensure policies catch issues without blocking legitimate actions.

​Overview

​Three main tabs

​Policies tab

​Policy list

​Create new policy

​Basic settings

​Budget limits

​Action control lists

​Rate limiting

​Human approval settings

​Alert configuration

​Prose / YAML / JSON authoring

​Version history & rollback

​Evidence graph viewer

​Policy precedence

​Policy sync to SDK

​API Keys tab

​API key list

​Generate new API key

​Using API keys

​Security best practices

​Troubleshooting API keys

​Notifications tab

​Channel status

​Email notifications

​Slack notifications (coming soon)

​Webhooks (coming soon)

​Critical escalation notice

​Common workflows

​Create production safety policy

​Set up approval workflow

​Rotate API key

​Configure budget alerts

​Best practices

​See also

Overview

Three main tabs

Policies tab

Policy list

Create new policy

Basic settings

Budget limits

Action control lists

Rate limiting

Human approval settings

Alert configuration

Prose / YAML / JSON authoring

Version history & rollback

Evidence graph viewer

Policy precedence

Policy sync to SDK

API Keys tab

API key list

Generate new API key

Using API keys

Security best practices

Troubleshooting API keys

Notifications tab

Channel status

Email notifications

Slack notifications (coming soon)

Webhooks (coming soon)

Critical escalation notice

Common workflows

Create production safety policy

Set up approval workflow

Rotate API key

Configure budget alerts

Best practices

See also