Seu agent alucinando? Culpa da busca (não do LLM)

Seu agent alucinando? Culpa da busca (não do LLM)

What just happened (October 2026):

Cohere released Rerank 5:

"Retrieval reranker (not LLM)."

"Orders search results by relevance BEFORE reaching LLM."

"Feature 1: Context Focus (extracts relevant passages inside documents)."

"Feature 2: Calibrated scores (confidence levels on ranking)."

"Message: Send fewer, better tokens to LLM (quality > quantity)."

What this means for your company:

Current situation (your WhatsApp agent):

Architecture (typical RAG): User: "Qual é a política de reembolso?" Search: Retrieves 10 documents (database, website, help center) Results: [doc1, doc2, doc3, doc4, doc5, doc6, doc7, doc8, doc9, doc10]

Problem: Search retrieves by keyword match (not relevance)
doc1: "Política de reembolso 100% (RELEVANT)"
doc2: "Como solicitar reembolso via Stripe" (SOMEWHAT relevant)
doc3: "Histórico de 2024: 1000 clientes reembolsados" (TANGENTIAL)
doc4: "Integração com PayPal (payment)" (NOT relevant)
doc5: "Termos de serviço página 5 (mentions refund)" (BURIED)
doc6: "Blog post: customer success stories" (NOT relevant)
doc7: "Forum: user asking about refunds" (NOT relevant)
doc8: "Roadmap 2025" (NOT relevant)
doc9: "Competitor pricing page (irrelevant)" (GARBAGE)
doc10: "Old archived policy (outdated)" (WRONG)

LLM receives: All 10 docs (majority garbage)
LLM confused: Which source is right? Conflicting info
LLM hallucinates: "Política é 90 dias, mas talvez 60... ou não sei"

Agent response: "Podemos oferecer reembolso, mas as políticas variam..."
Customer: "Que resposta vaga!"
Reality: LLM got bad sources (not LLM's fault)

With Rerank 5 (reordering BEFORE LLM): User: "Qual é a política de reembolso?" Search: Retrieves 10 documents Reranker 5: Orders by relevance

Reranked order:
Position 1: "Política de reembolso 100% (MOST RELEVANT)"
Position 2: "Como solicitar reembolso via Stripe" (RELEVANT)
Position 3: "Integração com PayPal" (SOMEWHAT)
Position 4-10: Garbage (ranked last)

LLM receives: Top 3 docs (high quality)
LLM confident: "Política é 100% within 30 days"

Agent response: "100% reembolso dentro de 30 dias, sem perguntas"
Customer: "Perfeito!"
Reality: Reranker fixed retrieval (LLM got good sources)

Difference (in numbers): Without rerank: LLM accuracy 40% (garbage sources) With rerank 5: LLM accuracy 92% (good sources) Delta: 2.3x better (not from better LLM, from better retrieval)

---

O problema oculto: Sua busca está quebrada

Por que search retrieval falha (e ninguém fala)

Limitação 1: Keyword matching (not semantic understanding)

Seu database: Doc A: "Oferecemos reembolso de 100% nos primeiros 30 dias." Doc B: "Política de devolução: itens não usados aceitos dentro de 60 dias." Doc C: "Cancelamento de assinatura resulta em crédito proporcional."

Customer pergunta: "Posso devolver se não gostar?"

Keyword search (típico): Busca: "devolver" (matches Doc B = devolução) Result: Doc B (correto por acaso)

Mas customer pergunta: "Quanto tempo pra reembolso?" Busca: "reembolso" (matches Doc A + Doc C) Result: [Doc A, Doc C] Problem: Doc C (cancelamento de assinatura) é NOT o que customer perguntou LLM recebe: Mixing refund policies + cancelation policies Response: Confusing (LLM não sabe qual aplicar)

Semantic search (com Rerank 5): Busca: "devolver se não gostar" (entende intent: return policy) Candidate docs: [A, B, C] Reranker: "Doc B mais relevante (devolução específica)" Result: Doc B top ranked LLM recebe: Contexto certo Response: Clear

Limitação 2: Quantity over quality (sending too much context)

Típico RAG pipeline: Vector search: Retorna top 10 documents (configuração padrão) LLM context window: 4K-128K tokens Problem: Sending all 10 documents = 8K tokens context

LLM token budget: System prompt: 500 tokens User message: 500 tokens Retrieved context: 8,000 tokens Available for reasoning: 500 tokens (reste) Result: LLM constrained (can't reason, just copy-paste)

With Rerank 5 (filtering): Vector search: Returns 10 documents Reranker 5: Selects top 3 (most relevant) Context passed: 2,400 tokens

LLM token budget: System prompt: 500 tokens User message: 500 tokens Retrieved context: 2,400 tokens Available for reasoning: 4,600 tokens (plenty) Result: LLM can reason deeply (not constrained)

Business impact: Less context = Better reasoning = Better answers = Happy customer

Limitação 3: No relevance scoring (can't tell good from bad)

Without Rerank 5: Search returns: [doc1, doc2, doc3, doc4, doc5] LLM thinks: All equally valid (no scoring) Problem: If doc2 is garbage, LLM might use it anyway Result: Hallucination possible

With Cohere Rerank 5 (calibrated scores): Search returns: [doc1(0.92), doc2(0.45), doc3(0.88), doc4(0.15), doc5(0.72)] Scores: Confidence in relevance LLM can see: doc2 and doc4 are low confidence LLM can skip: Low-scoring docs Result: Only high-confidence sources used

Practical: "Only use sources > 0.70 confidence" Outcome: 95% accuracy (filters garbage automatically)

---

Por que RAG architecture é crítica pro agent

Agent = LLM + Retrieval (não só prompt engineering)

Myth: Better prompt = Better agent

Common assumption: "If agent is wrong, write better prompt." Reality: Garbage in, garbage out

If sources are bad → Better prompt won't help If sources are good → Decent prompt will work

Priority: Fix sources (not prompt)

Example: Agent prompt: "You are helpful, concise, accurate assistant..." Retrieved sources: 5 irrelevant documents Agent response: "I don't have clear information on..." CEO says: "Improve the prompt!" Reality: Prompt is fine (sources are problem)

Fix: Add reranking (not rewrite prompt) Result: Agent now accurate

Why Cohere Rerank 5 changes the game:

Old RAG stack (typical):

1. User query
2. Vector search (returns 10 docs)
3. LLM (processes all 10)
4. Response

Problem: Step 2 (search) is weakest link Solution: Optimize prompt (step 3) - doesn't help much Real fix: Optimize search (step 2) - game changer

New RAG stack (with Rerank 5):

1. User query
2. Vector search (returns 10 docs)
3. Reranker 5 (orders by relevance, filters top 3)
4. LLM (processes only 3 best docs)
5. Response

Improvement: Step 3 (reranking) fixes weak link Result: 2-3x quality improvement (same LLM, same prompt) Cost: Minimal (reranking is fast, cheap)

---

Como reranking melhora agent quality (em prática)

Real-world scenario: Customer service agent

Sem Rerank 5:

Customer: "Minha cobrança foi duplicada. Como resolvo?"

Search results (10 docs):

1. "Billing FAQ" (general, not specific)
2. "How to cancel subscription" (not relevant)
3. "Invoice explanation" (somewhat relevant)
4. "Payment methods" (not relevant)
5. "Duplicate charge incident from 2024" (RELEVANT but old)
6. "Customer success story" (not relevant)
7. "Pricing page" (not relevant)
8. "Terms of service" (legal, not helpful)
9. "Roadmap" (not relevant)
10. "Blog post on billing best practices" (not relevant)

LLM sees: Mix of relevant (doc 3, 5) + irrelevant (6 docs) LLM confusion: What's the actual policy? Agent response: "Você pode contatar suporte e enviar comprovante. Talvez possamos reverter. Deixa eu escalona." Customer: Frustrated (should be solved) Company: Escalation cost (wasted time)

With Rerank 5: Same 10 docs Reranker: "Top 3: doc5 (0.94), doc3 (0.87), doc1 (0.72)" LLM sees: Only high-relevance docs LLM confident: "Duplicate charges reversed within 24 hours. Já processando seu crédito." Customer: Happy (problem solved) Company: No escalation (automatic resolution)

Impacto empresarial:

Sem Rerank 5: Escalation rate: 40% (agent can't resolve confidently) Avg handling time: 8 minutes (customer + agent confusion) CSAT: 6.2/10 (vague responses) Cost/resolution: R$50 (includes escalation)

Com Rerank 5: Escalation rate: 5% (agent resolves confidently) Avg handling time: 2 minutes (agent knows right answer) CSAT: 9.1/10 (clear, accurate responses) Cost/resolution: R$8 (automated, no human escalation)

Delta: Escalation reduction: 87% fewer Speed improvement: 4x faster Satisfaction: +47% higher Cost reduction: 84% cheaper

For 1,000 customer interactions/day:
  Escalation savings: 350 tickets (freed up support team)
  Time savings: 6,000 minutes (100 hours/day)
  Cost savings: R$42k/day × 22 days = R$924k/month

---

Cohere Rerank 5 vs outras soluções

Por que Rerank 5 é game-changer (não é só incremental)

Antes (RAG básico):

Search: "top 10 documents" LLM: "Process all 10" Result: Noisy (garbage included)

Limitações:

• Você paga LLM tokens pra processar lixo
• LLM fica confuso (contradictory sources)
• Accuracy não melhora (bad input = bad output)
• Escalação necessária (quando agent não tem certeza)

Agora (com Rerank 5):

Search: "top 10 documents" Rerank 5: "Sort by relevance, pick top 3" LLM: "Process only 3 best" Result: Clean (garbage filtered)

Benefício: ✓ You save LLM tokens (process less, pay less) ✓ LLM gets clear signal (high-confidence sources) ✓ Accuracy jumps (good input = good output) ✓ No escalation needed (agent confident) ✓ Calibrated scores (quantified confidence)

Cohere Rerank 5 specific features:

Feature 1: Context Focus Traditional reranking: Returns "doc3 is most relevant" Rerank 5: Returns "doc3 is relevant + here's the exact passage"

Benefit: LLM doesn't waste tokens reading whole doc Example: Doc3: "Our refund policy is 30-day guarantee, with exceptions... [1000 words of legal text]... Exceptions: Digital products, opened software, etc."

Old reranker: "Use doc3" (LLM reads full 1000 words)
Rerank 5: "Use doc3, passages: [30-day guarantee], 
          [Exceptions: Digital products]" (LLM reads 50 words)

Result: 20x token savings, same quality

Feature 2: Calibrated Scores Old: "doc1, doc2, doc3" (equal trust) Rerank 5: "doc1(0.94), doc2(0.72), doc3(0.45)" (confidence shown)

Benefit: Agent can threshold (only use > 0.80 confidence) Practical: If customer asks about rare edge case Search returns docs with 0.55 confidence Agent sees low confidence → "Let me escalate to specialist" vs. Old: Agent didn't know confidence → Guessed (hallucinated)

---

Implementação prática: Como aplicar Rerank 5 no seu SaaS

Step-by-step (não é complexo)

Passo 1: Diagnose seu problema (semanas 1-2)

Perguntas: [ ] Agent escalona frequentemente? (Sim = retrieval problem) [ ] Customers reclamam de respostas vagas? (Sim = sources ruins) [ ] Agent cita informação contradictória? (Sim = reranking needed) [ ] LLM accuracy lower than expected? (Sim = garbage in)

Metricas baseline: Escalation rate: ___% (target: < 10%) Avg handling time: ___ minutos (target: < 5) CSAT: ___/10 (target: > 8.5) Accuracy on retrieval: ___% (target: > 90%)

If any bad: Rerank 5 will help

Passo 2: Integrate Rerank 5 (semanas 3-4)

Stack típico:

1. Vector DB (Pinecone, Weaviate, Milvus)
2. LLM API (OpenAI, Claude, Cohere)
3. Agent framework (LangChain, LlamaIndex, CrewAI)

Adição: Between steps 1-2: Add Cohere Rerank 5

Code example (pseudocode):

# Before (no reranking)
results = vector_db.search(query, top_k=10)
llm_response = llm(results, query)

# After (with Rerank 5)
results = vector_db.search(query, top_k=10)
reranked = cohere_rerank(results, query)  # <-- NEW
top_3 = reranked[:3]  # <-- Filter
llm_response = llm(top_3, query)

Effort: 2 days (API integration, testing) Cost: +$50-200/month (Cohere Rerank 5 pricing)

Passo 3: Measure improvement (semanas 5-6)

Compare (before vs after Rerank 5):

Escalation rate: Before: 40% After: 8% Improvement: 80% reduction

Handling time: Before: 8 minutes After: 2 minutes Improvement: 75% faster

CSAT: Before: 6.2/10 After: 9.1/10 Improvement: +47%

Token usage (cost): Before: 8K tokens/query (all 10 docs) After: 2.4K tokens/query (top 3 docs) Improvement: 70% cheaper

Result: 1-2 week ROI (savings immediately visible)

Passo 4: Optimize (ongoing)

Tuning: Threshold: "Only use sources > 0.80 confidence" Top-K: "Use top 3 docs instead of 10" Context window: "Extract passages only (not full docs)"

Monitoring: [ ] Track escalation rate (target: < 5%) [ ] Monitor token usage (verify savings) [ ] Check CSAT (verify customer impact) [ ] Review low-confidence cases (manual fix)

Iterate monthly (rerank model improves over time)

---

Bottom line: Reranking is the missing piece

Your RAG pipeline right now:

Search (weak) → LLM (strong but overwhelmed) → Bad output

You optimize: LLM prompt (doesn't help) You ignore: Search quality (the bottleneck)

Result: Agent still bad (garbage in, garbage out)

RAG pipeline with Cohere Rerank 5:

Search (weak) → Rerank (fixes weak) → LLM (strong + focused) → Great output

You optimize: Retrieval quality (actual bottleneck) You ignore: Oversized LLM prompts (no longer needed)

Result: Agent now great (good in, great out)

Decision logic:

If agent escalating > 20% → Rerank 5 will help 80%+ of the time If CSAT < 8.0 → Retrieval quality is likely the issue If token usage > 5K per query → Too much garbage context

All three true? Rerank 5 is not optional, it's critical. Implementation cost: R$200-500/month ROI timeline: 1-2 weeks (cost savings + escalation reduction)

Don't optimize prompt. Fix retrieval. Deploy Rerank 5.

→ OpenClaw: RAG Architecture, Retrieval Quality, Reranking Strategy, Agent Accuracy, Context Optimization, Vector Search Tuning

Seu agent alucinando? Não é o LLM. É a busca mandando lixo. Rerank 5 filtra retrieval ANTES do LLM. 2-3x melhor accuracy, 70% menos tokens, sem code rewrite. Comece agora. 🎯📊✨