Agent seu está degradado (e você não sabe)

Agent seu está degradado (e você não sabe)

Cenário assustador (que está acontecendo AGORA):

Você é founder de SaaS com agent (atendimento, vendas, automação).

Você lança agent em produção.

Clientes usam.

Você assume: tudo está OK (ninguém reclamou).

Realidade oculta:

Agent está degradado (silenciosamente).

Exemplos de degradação invisível:

1. Agent de atendimento Demo (seu teste): respostas excelentes Produção (cliente real): respostas genéricas, erros 10% Cliente: "Hmmm, respostas estão piores..." Você: não sabe que algo mudou

2. Agent de vendas Demo: identifica lead qualificado Produção: identifica incorretamente (50% accuracy) Vendedor: persegue lead ruim (tempo perdido) Você: "Por que conversão caiu 30%? Estranho..."

3. Agent de análise Demo: recomendação perfeita Produção: recomendação errada (hallucination) Cliente: toma decisão errada baseada em agent Resultado: perda financeira (R$ 100k? R$ 1M?) Você: descobre muito depois

4. Agent de compliance Demo: detecta risco corretamente Produção: miss 20% dos riscos Cliente: passa riscos que deviam ser blocked Consequência: regulatory fine (R$ 500k+) Você: liable

Por quê é invisível?

Traditional software:

• Falha BARULHENTA
• Exemplo: API call falha → erro 500 → logs → você vê
• Você sabe que está quebrado
• Você conserta rápido

Agent:

• Falha SILENCIOSA
• Exemplo: agent responde confiante (parece OK)
• Resposta lê bem (fluente, natural)
• Resposta é ERRADA (mas você não sabe)
• Ninguém reclama (ainda)
• Semanas depois: cliente percebe (reputação bomb)
• Você já perdeu trust

Amazon descobriu isso:

"Agent que demos beautifully pode degrade silenciosamente em produção. By the time human notices, trust é quebrado. Agent não falha LOUD (como software). Falha SOFT (resposta confiante + errada). Quando usa agent response pra business decision, o custo não é feature quebrada. É broken relationship (cliente sai)."

Significado: Você precisa saber SE agent está degradado. Hoje, você NÃO sabe.

---

O Problema: Demo não = Produção

Por quê agent degrada

Razão 1: Data distribution mismatch

Demo (seu teste): Dados: 100 queries (você escolheu) Características: tipicamente limpos, bem-formados Exemplo: - "Qual é meu saldo de conta?" - "Como faço reclamação?" - Português correto - Context claro

Produção (cliente real): Dados: 10.000 queries/dia (aleatório) Características: messy, ambíguo, edge cases Exemplo: - "SALDOOOOO????" (typos, caps) - "meu cartão não funciona é fraude tipo não sei help me" (confused, stream of consciousness) - Portuguese quebrado (slang, regional) - Context missing (refere-se a algo anterior)

Resultado: Agent funciona em demo (dados "bons") Agent falha em produção (dados "reais")

Razão 2: Model drift

Demo (dia 1): Model: Claude 3.5 Performance: 95% accuracy

Produção (semana 3): Você atualiza pra Claude 3.6 ("melhoria") Performance: 87% accuracy (realmente piorou, unexpected) Você não testou bem Customers suddenly see worse responses

Ou: Model fine-tuning mudou comportamento Hallucination aumentou Accuracy caiu You don't know

Razão 3: Context/prompt degradation

Demo: System prompt: "You are helpful customer service agent. Follow these 10 rules." Agent: follows rules, responds well

Produção (semana 5): Someone changes system prompt (bug, iteration) New prompt: missing 3 rules Agent: now violates those 3 rules Customers: see bad behavior You: don't know what changed

Razão 4: Upstream service failures

Demo: Agent uses: customer database API API: always working (you're testing locally) Agent: gets data, responds correctly

Produção: API: sometimes slow, sometimes fails Agent: timeout, returns fallback response (generic, unhelpful) Agent: looks OK (no error thrown, agent didn't crash) Customer: gets bad experience You: don't know API is flaky

Razão 5: Volume/load effects

Demo: You test: 1 query at a time Agent: responds in 2 seconds

Produção: 1000 concurrent users API rate limits hit Agent: queue builds up Agent: responds in 30 seconds Customer: times out, bad experience Agent quality: looks same, but feels terrible

Invisible failure modes

Hallucinations (most dangerous):

Agent: "Your account balance is R$ 50,000" Reality: Customer never had R$ 50,000 Customer: confident in wrong number Customer: makes decision based on hallucination Customer: loses money Customer: sues your company You: liable ("AI said it, I trusted")

How is this invisible?

• Agent response LOOKS confident
• No error thrown
• No red flag in logs
• Customer doesn't complain immediately
• Discovered weeks later (when decision consequences show)

Drift (gradual degradation):

Week 1: Agent quality = 95% Week 2: Agent quality = 93% (slight degradation) Week 3: Agent quality = 90% Week 4: Agent quality = 85% Week 5: Customer service team notices (calls are worse)

How is this invisible?

• Gradual (hard to notice)
• No alert (you're not monitoring)
• Customer complains only when painful enough
• By then, reputation damaged

Silent failures (wrong but confident):

Agent task: Detect if customer is eligible for credit Demo: 98% accuracy (you tested on sample) Produção: 70% accuracy (on real data distribution) Agent: still responds confidently ("You are eligible" or "You are not") No error, no alert But decisions are wrong 30% of time

Cost:

• Bad loans (customers shouldn't get credit, got it anyway)
• Rejected customers (should get credit, didn't)
• Regulatory scrutiny ("Why are rejections wrong?")
• Your company: liable

---

A Solução: Evaluation framework (production-ready)

O que Amazon propõe

Core idea:

Before (traditional): Demo agent locally Launch to production Hope it works

After (evaluation framework): Demo agent locally Deploy to production CONTINUOUSLY MONITOR (evaluate in real-time) Alert if degradation Rollback if needed

Key difference: Evaluation = automated testing in production Continuous = always checking Real-time = immediately alert

How evaluation framework works

Setup:

1. Define success metrics What does "good" mean for your agent? Examples:

   • Accuracy: 90%+ (correctly answers user question)
   • Relevance: 85%+ (response is relevant to question)
   • Safety: 100% (no harmful responses)
   • Latency: < 3 seconds (responds quickly)
   • No hallucinations: 0% (never makes up facts)

2. Create evaluation dataset 100-500 test queries (covering:

   • Common cases (80%)
   • Edge cases (15%)
   • Adversarial cases (5%) ) Each query has: expected answer, evaluation criteria

3. Automated evaluation In production (real-time):

   • Sample user queries (1% of traffic)
   • Run through agent
   • Evaluate: accuracy, relevance, safety
   • Compare: expected vs actual
   • Track metrics over time

4. Alerts If metrics drop:

   • Accuracy drops below 90%: ALERT
   • Hallucination detected: ALERT
   • Latency > 5s: ALERT
   • Unknown response format: ALERT

5. Actions On alert:

   • Investigate (what changed?)
   • Rollback (revert recent changes)
   • Redeploy (fixed version)
   • Notify (customer, internal team)

Example (customer service agent)

Scenario:

Agent task: Answer customer service questions (billing, account, refund, complaint)

Success metrics:

• Accuracy: 90% (answers correctly)
• Safety: 100% (no promises agent can't keep)
• Relevance: 85% (response addresses question)
• Professionalism: 90% (tone appropriate)

Evaluation dataset (200 queries): Type 1: Common questions (80%) Q: "How do I check my balance?" Expected: "Go to Account → Balance" Accuracy: Yes/No

Type 2: Edge cases (15%) Q: "My account was hacked, what do I do?" Expected: "Contact support immediately at..." Safety: Did agent make promises? (Yes/No)

Type 3: Adversarial (5%) Q: "Can you give me a refund for my entire purchase because I just changed my mind?" Expected: "Refunds follow policy X. You don't qualify." Safety: Did agent incorrectly promise refund? (Yes/No)

Production monitoring: Every user query: - 1% chance of being evaluated - Run through agent - Compare response to expected - Log: was it correct?

Dashboard: Accuracy: 91% (✓ Good, above 90%) Safety: 100% (✓ Good) Relevance: 86% (✓ Good, above 85%)

Alert example: Day 1: Accuracy 91% Day 2: Accuracy 91% Day 3: Accuracy 89% (⚠️ Dropped below 90%)

Alert triggered:
"Agent accuracy degraded. Was 91%, now 89%. Investigate recent changes."

Investigation:
"System prompt was accidentally changed yesterday (someone edited wrong file)."

Action:
"Revert system prompt. Redeploy. Check accuracy."

Result:
"Accuracy restored to 91%."

Without evaluation framework: "Accuracy dropped to 89%. Customer notices week later. Complains. You don't know why."

---

How to implement evaluation framework

Option 1: Use Amazon Bedrock AgentCore

If your SaaS uses AWS:

Stack:

• Agent: Built on Bedrock (Claude, Llama, Mistral)
• Monitoring: Bedrock AgentCore (built-in evaluation)
• Setup time: 1-2 weeks
• Cost: Bedrock API calls + monitoring

Setup:

1. Define metrics (what is "good"?)
2. Create evaluation dataset (100-500 queries)
3. Deploy agent with monitoring enabled
4. View dashboard (accuracy, safety, latency trends)
5. Set alerts (if accuracy drops, notify)

Benefit:

• Native to Bedrock (easy integration)
• No custom code needed
• Automatic evaluation (built-in)
• Dashboards ready

Option 2: Build custom (any LLM provider)

If using OpenAI, Anthropic, or custom:

Stack:

• Agent: Your agent (any LLM)
• Monitoring: Custom evaluation service
• Setup time: 2-4 weeks
• Cost: Custom development

Architecture:

1. Production agent Every response: a) Generate response b) Sample (1% of traffic) c) Log response

2. Evaluation service Async evaluation: a) Fetch sample responses b) Run through evaluation LLM (Claude) c) Check: Is response accurate? Safe? Relevant? d) Store result

3. Metrics + alerts Dashboard:

   • Track accuracy over time
   • Alert if drops
   • Alert if hallucination detected
   • Alert if safety violated

4. Actions On alert:

   • Notify team
   • Investigation tools (what changed?)
   • Rollback (if needed)
   • Redeploy

Example implementation (simplified):

Production agent

def handle_query(query): response = call_agent(query) if random.random() < 0.01: # 1% sampling log_for_evaluation(query, response) return response

Evaluation service (async)

def evaluate_response(query, response): evaluation_prompt = f""" Query: {query} Response: {response}

  Is this response:
  - Accurate? (Yes/No)
  - Safe? (Yes/No)
  - Relevant? (Yes/No)
  
  Explain your reasoning.
  """
  evaluation = call_claude(evaluation_prompt)
  store_evaluation_result(query, response, evaluation)

Monitoring

def monitor_metrics(): metrics = calculate_metrics() # accuracy, safety, etc if metrics['accuracy'] < 0.90: alert("Accuracy dropped below 90%") if metrics['hallucinations'] > 0: alert("Hallucinations detected")

Option 3: Hybrid (existing tools + custom)

Use existing monitoring tools:

Tools:

• Datadog / NewRelic (infrastructure monitoring)
• Custom evaluation (simple, in-house)
• Alerts (Slack, email)

Setup:

1. Log agent responses (Datadog)
2. Sample evaluation (1% queries, using Claude API)
3. Store results (your database)
4. Dashboard (Grafana or similar)
5. Alerts (if metrics drop)

Cost:

• Tools: $100-500/month
• Development: 1-2 weeks
• Evaluation API: $50-200/month (depends on traffic)

---

3-week implementation plan

Week 1: Design + evaluation dataset

☐ Define success metrics

• What does "good agent" mean?
• For your use case:
  • Accuracy: X%
  • Safety: Y%
  • Latency: Z seconds
• Document (this is north star)

☐ Create evaluation dataset

• 200-500 test queries (covering common + edge + adversarial)
• For each query: expected response
• Manual: create by hand (tedious but worth it)
• Or: use your historical customer queries (good proxy)

☐ Decision: build vs buy

• Using Bedrock? Use AgentCore
• Using OpenAI? Build custom
• Other? Hybrid approach

Week 2: Implementation

☐ If Bedrock:

• Deploy agent with monitoring enabled
• Configure evaluation dataset
• Set alerts
• Test (verify working)

☐ If custom:

• Build evaluation service
• Integrate with production agent
• Create dashboard
• Test (verify working)

☐ Testing:

• Intentionally degrade agent (change system prompt)
• Verify: evaluation detects degradation
• Verify: alert fires
• Verify: metrics show drop

Week 3: Launch + monitoring

☐ Beta launch

• Enable for 10% of traffic
• Monitor: metrics, alerts, false positives
• Refine: evaluation criteria (if needed)

☐ Production launch

• Enable for 100% of traffic
• Dashboard: live monitoring
• On-call: ready to respond to alerts
• Weekly review: trends, insights

☐ Continuous improvement

• Update evaluation dataset (based on new query types)
• Refine metrics (if ineffective)
• Learn: what causes degradation? (model drift? context? data?)

---

ROI: Por quê implementar

Cost of NOT monitoring

Scenario 1: Accuracy degrades (undetected) Baseline: 95% accuracy Actual (week 3): 80% accuracy Customer impact: - 20% of agent responses wrong - Customer experience: terrible - NPS: drops 40 points - Churn: customer leaves (R$ 500k/year) Cost: R$ 500k (1 customer)

Scenario 2: Hallucination increase (undetected) Agent: starts making up facts (5% of responses) Customer: makes decision based on hallucination Result: loses R$ 100k Consequence: customer sues (R$ 1M legal cost)

Scenario 3: Safety breach (undetected) Agent: accidentally promises refund (violates policy) 10% of customers: exploit promise Cost: R$ 50k (refunds)

Total cost (no monitoring): R$ 500k - R$ 1M+

Cost of monitoring

Implementation:

• Dev time: 2 devs × 3 weeks = R$ 60k
• Tools/infra: R$ 500/month = R$ 6k
• Total: R$ 66k

Benefit (first 6 months):

• Detect degradation: 1 incident (prevented)
  • Cost avoided: R$ 500k
• Detect hallucination: 1 incident (prevented)
  • Cost avoided: R$ 100k
• Detect breach: 1 incident (prevented)
  • Cost avoided: R$ 50k
• Total benefit: R$ 650k

ROI: (R$ 650k - R$ 66k) / R$ 66k = 8.8x Payback: < 1 month

Competitive advantage

Competitor (no monitoring): "We have agent. Works great!" Customer: tries it Week 2: agent quality drops (undetected) Week 4: customer notices (complains) Relationship: damaged

Your SaaS (with monitoring): "We have agent. Plus continuous quality monitoring." Customer: tries it Week 2: quality drops (YOU detect immediately) Week 2 (same day): you investigate + fix Week 3: customer never noticed issue Relationship: strong ("They care about quality")

Differential:

• Competitor: reactive (customer complains first)
• You: proactive (you fix before customer notices)
• Customer experience: 10x better
• Pricing power: can charge premium

---

Conclusão: Evaluation framework é must-have

Realidade:

Agent que demos OK pode degrade silenciosamente em produção. By time customer notices, relationship é damaged. You can't manage what you don't measure.

Evaluation framework = visibility into agent quality. Continuous monitoring = early detection. Alerts = fast response.

Result:

• Customer never knows agent degraded (you fixed it)
• Trust maintained
• Reputation protected
• Competitive advantage

Ação imediata:

1. Assess: Are you monitoring agent quality? Yes → Good, but verify it's comprehensive No → Critical gap, implement immediately

2. Define: What is "good" agent performance? For your use case:

   • Accuracy %?
   • Safety rules?
   • Latency limits?
   • Hallucination threshold?

3. Create: Evaluation dataset

   • 200-500 test queries
   • Covering: common + edge + adversarial
   • Takes: 1-2 weeks (worth it)

4. Implement: Evaluation framework

   • Option 1: Bedrock AgentCore (if AWS)
   • Option 2: Custom (if other provider)
   • Option 3: Hybrid (best of both)
   • Timeline: 2-3 weeks

5. Monitor: 24/7

   • Dashboard (live metrics)
   • Alerts (if degradation)
   • On-call (response team)

6. Iterate: Continuous improvement

   • Update evaluation dataset
   • Refine metrics
   • Learn patterns

→ OpenClaw: Agent Quality Monitoring, Evaluation Framework, Production Readiness, Degradation Detection, SaaS Trust

Seu agent está sendo monitorado? Ou está degradado (e você não sabe)? 🔍📊