Docs
Core Concepts
Three Pillars

Three Pillars of LLMOps

AgenticAnts implements LLMOps (Large Language Model Operations) through three foundational pillars that work together to provide comprehensive AI operations management.

Overview

LLMOps is the overarching discipline that encompasses the entire lifecycle of LLM operations from development to production. Our three pillars approach ensures complete coverage of AI operational needs:

┌─────────────────────────────────────┐
│                                     │
│           LLMOps Framework          │
│     Large Language Model Operations │
│                                     │
├─────────────────────────────────────┤
│                                     │
│      FinOps                       │
│     Cost Management & Optimization  │
│                                     │
├─────────────────────────────────────┤
│                                     │
│      SRE                          │
│     Reliability & Performance       │
│                                     │
├─────────────────────────────────────┤
│                                     │
│      Security Posture                       │
│     Security & Compliance           │
│                                     │
└─────────────────────────────────────┘

LLMOps Framework

LLMOps provides the comprehensive framework for managing LLM operations:

  • Model Lifecycle Management - Selection, versioning, deployment, and retirement
  • Prompt Operations - Prompt engineering, versioning, and optimization
  • Performance Optimization - Latency, throughput, and cost optimization
  • Model Governance - Policies, compliance, and risk management
  • Versioning & Deployment - CI/CD pipelines and rollback strategies

Learn more about LLMOps →

Pillar 1: FinOps

AI Cost Optimization and Financial Management

What is AI FinOps?

FinOps for AI helps organizations understand, control, and optimize AI spending through:

  • Visibility: See where every dollar is spent
  • Attribution: Track costs by customer, team, or product
  • Optimization: Identify and eliminate waste
  • Forecasting: Predict future costs and budget accordingly

Key Capabilities

Token Usage Monitoring

Track every token consumed by your AI systems:

// Automatically tracked
const response = await openai.chat.completions.create({
  model: 'gpt-4',
  messages: [{ role: 'user', content: query }]
})
 
// AgenticAnts records:
// - Model used: gpt-4
// - Tokens: prompt=150, completion=200, total=350
// - Cost: $0.0105 (based on current pricing)

Cost Per Customer Query

Understand the economics of your AI operations:

# View cost breakdown
customer_costs = ants.metrics.get_customer_costs(
    start_date='2025-10-01',
    end_date='2025-10-31',
    group_by='customer'
)
 
# Results:
# customer_123: $45.50 (450 queries)
# customer_456: $89.20 (920 queries)
# customer_789: $12.30 (95 queries)

Budget Management

Set budgets and receive alerts:

await ants.budgets.create({
  name: 'Q4 AI Spending',
  amount: 10000,
  period: 'quarterly',
  alerts: [
    { threshold: 0.7, type: 'warning' },  // 70%
    { threshold: 0.9, type: 'critical' }  // 90%
  ]
})

ROI Analytics

Measure the business impact of AI investments:

const roi = await ants.analytics.calculateROI({
  costs: 5000,        // AI costs
  revenue: 25000,     // Revenue attributed to AI
  timePeriod: 'month'
})
 
// ROI: 400% (5x return on investment)
// Cost per conversion: $2.50
// Customer lifetime value: $500

FinOps Best Practices

  1. Tag Everything: Use consistent tagging for cost attribution
  2. Set Budgets: Define spending limits for teams and projects
  3. Monitor Regularly: Review costs weekly, not monthly
  4. Optimize Models: Use smaller models where appropriate
  5. Cache Responses: Reduce redundant LLM calls

Learn more about FinOps →

Pillar 2: SRE

AI Reliability Engineering and Performance

What is AI SRE?

Site Reliability Engineering adapted for AI systems ensures they are:

  • Reliable: High availability and fault tolerance
  • Performant: Low latency and high throughput
  • Observable: Complete visibility into system behavior
  • Scalable: Handle increasing load gracefully

Key Capabilities

End-to-End Tracing

Follow requests through your entire AI stack:

// Trace shows complete execution path
Trace: customer-support-query (2.3s)
├─ Span: input-validation (10ms)
├─ Span: retrieve-customer-context (150ms)
│  └─ Span: database-query (145ms)
├─ Span: vector-search (200ms)
│  ├─ Span: embedding-generation (50ms)
│  └─ Span: similarity-search (150ms)
├─ Span: llm-inference (1.8s)
│  ├─ Span: prompt-construction (5ms)
│  ├─ Span: api-call (1.78s)
│  └─ Span: response-parsing (15ms)
└─ Span: response-formatting (140ms)

Performance Monitoring

Track key performance metrics:

# View performance metrics
metrics = ants.metrics.get_performance({
    agent: 'customer-support',
    period: 'last_24h'
})
 
print(f"Latency p50: {metrics.latency.p50}ms")  # 1,200ms
print(f"Latency p95: {metrics.latency.p95}ms")  # 3,500ms
print(f"Latency p99: {metrics.latency.p99}ms")  # 5,200ms
print(f"Error rate: {metrics.error_rate}%")     # 0.5%
print(f"Throughput: {metrics.throughput}/s")    # 45 req/s

Automated Alerting

Get notified when things go wrong:

await ants.alerts.create({
  name: 'High Error Rate',
  condition: 'error_rate > 5%',
  window: '5m',
  channels: ['slack', 'pagerduty'],
  severity: 'critical'
})
 
await ants.alerts.create({
  name: 'Slow Response Time',
  condition: 'p95_latency > 5000ms',
  window: '10m',
  channels: ['email'],
  severity: 'warning'
})

Incident Response

Quickly diagnose and resolve issues:

# Get incident details
incident = ants.incidents.get('inc-123')
 
# View timeline
for event in incident.timeline:
    print(f"{event.time}: {event.description}")
 
# Identify root cause
root_cause = ants.incidents.analyze_root_cause('inc-123')
print(f"Root cause: {root_cause.description}")
 
# View similar incidents
similar = ants.incidents.find_similar('inc-123')

SRE Best Practices

  1. Set SLOs: Define Service Level Objectives
  2. Monitor Proactively: Don't wait for users to report issues
  3. Automate Responses: Auto-remediate common issues
  4. Learn from Incidents: Conduct post-mortems
  5. Test Resilience: Implement chaos engineering

Learn more about SRE →

Pillar 3: Security Posture

AI Security Control and Compliance

What is AI Security Posture?

Security Posture for AI protects your systems and data through:

  • Data Protection: Prevent sensitive data leaks
  • Access Control: Manage who can access what
  • Compliance: Meet regulatory requirements
  • Audit Trails: Complete logs for forensics

Key Capabilities

PII Detection & Protection

Automatically identify and protect sensitive data:

// AgenticAnts automatically detects PII
const trace = await ants.trace.create({
  name: 'customer-query',
  input: 'My SSN is 123-45-6789 and email is john@example.com'
})
 
// Dashboard shows:
// - PII detected: SSN, Email
// - Automatically redacted in storage
// - Alert sent to security team
// - Audit log created

Security Guardrails

Prevent harmful or policy-violating outputs:

# Configure guardrails
ants.guardrails.create({
    'name': 'content-policy',
    'rules': [
        {'type': 'no_pii', 'action': 'redact'},
        {'type': 'no_toxic_content', 'action': 'block'},
        {'type': 'no_financial_advice', 'action': 'warn'}
    ]
})
 
# Automatically enforced on all outputs
response = agent.run(query)  # Checked against guardrails

Compliance Reporting

Generate compliance reports automatically:

// Generate SOC2 compliance report
const report = await ants.compliance.generate({
  framework: 'SOC2',
  period: 'Q4-2025',
  controls: [
    'access-control',
    'data-encryption',
    'audit-logging',
    'incident-response'
  ]
})
 
// Download GDPR data export
const gdprExport = await ants.compliance.exportData({
  userId: 'user-123',
  format: 'json'
})

RBAC & Access Control

Fine-grained permissions management:

# Create role
ants.roles.create({
    'name': 'data-scientist',
    'permissions': [
        'traces.read',
        'metrics.read',
        'dashboards.read',
        'projects.list'
    ],
    'resources': ['project-123', 'project-456']
})
 
# Assign to user
ants.users.assign_role('user-789', 'data-scientist')

Audit Trails

Complete logging of all activities:

// Query audit logs
const logs = await ants.audit.query({
  action: 'data.export',
  startDate: '2025-10-01',
  endDate: '2025-10-31'
})
 
for (const log of logs) {
  console.log(`${log.timestamp}: ${log.user} ${log.action}`)
  console.log(`  Resource: ${log.resource}`)
  console.log(`  IP: ${log.ip}`)
  console.log(`  Status: ${log.status}`)
}

Security Posture Best Practices

  1. Principle of Least Privilege: Give minimum necessary access
  2. Regular Audits: Review access and activities regularly
  3. Encrypt Everything: Data at rest and in transit
  4. Monitor Anomalies: Detect unusual access patterns
  5. Incident Response Plan: Have a plan before incidents occur

Learn more about Security Posture →

Integration of the Three Pillars

The pillars work together to provide comprehensive coverage:

Example: Production Incident

1. SRE: Detects high error rate
   └─> Alert sent to on-call engineer

2. FinOps: Identifies cost spike
   └─> Related to failed retries

3. Security Posture: Reviews audit logs
   └─> No security breach detected

4. Resolution:
   - SRE fixes the bug
   - FinOps tracks cost impact
   - Security Posture documents for compliance

Example: Cost Optimization

1. FinOps: Identifies expensive queries
   └─> Customer X costs $500/day

2. SRE: Analyzes performance
   └─> Slow queries causing retries

3. Security Posture: Checks data access
   └─> Proper authorization confirmed

4. Optimization:
   - SRE optimizes query performance
   - FinOps validates cost reduction
   - Security Posture audits changes

Getting Started with Each Pillar

Understanding AI AgentsCredit System