Seu agente é caótico porque usa 1 LLM (deveria ser 2)

Seu agente é caótico porque usa 1 LLM (deveria ser 2).

Você é founder de SaaS.

Seu agente roda no WhatsApp.

Fluxo simples:

1. Customer: "Qual é meu saldo?"
2. Agent (GPT-4): Busca saldo, responde
3. Customer: "Muito caro! Quer desconto?"
4. Agent (GPT-4): Oferece desconto, closea deal

Parece simples.

Mas tem problema:

Pergunta para seu agent: "Se customer pede desconto de 90%, você da?"

Agent (GPT-4): "Hmmm, customer pediu 90%, vou oferecer 20% como compromise... ou talvez 15%... ou talvez nego totalmente... não tenho certeza."

Resultado:

• Às vezes agent da 90% (perde R$10k)
• Às vezes agent da 5% (customer nega)
• Às vezes agent tenta negociar (customer fica frustrado)
• Às vezes agent rota pra human (subutiliza automation)

Nenhuma é previsível.

Por quê?

Porque você está usando 1 LLM pra 2 jobs diferentes:

1. Decisão crítica (dar desconto ou não): Precisa ser DETERMINÍSTICA (previsível, confiável)
2. Interação com customer (tone, naturalness, empathy): Precisa ser NÃO-DETERMINÍSTICA (flexível, criativa)

Misturar os dois em 1 LLM = CAOS.

Ontem, artigo fez notícia:

"Deterministic Core, Non-Deterministic Shell"

Arquitetura que separa os dois:

• Core: Lógica de negócio (determinística, regras claras)
• Shell: Interação com user (não-determinística, natural language)

Vamos explorar.

---

O problema: 1 LLM não consegue ser determinístico E criativo simultaneamente

Por que misturar os dois é fundamentalmente errado

=== THE PROBLEM ===

Current architecture (1 LLM):

Customer message: "Quero desconto de 50%" ↓ [GPT-4 System Prompt] ├─ "You are a helpful sales agent" ├─ "You can offer discounts up to 20%" ├─ "Be friendly and persuasive" ├─ "If customer asks for >30%, escalate to human" ├─ "Try to close the deal" └─ "Be conversational and natural" ↓ [GPT-4 Response]

Problem 1: Conflicting goals ├─ "Be friendly" (agree with customer) ├─ "Only 20% max" (disagree with customer) ├─ Agent oscillates: "I'd love to give 50%, but..." (wishy-washy) └─ Result: Confusing, ineffective

Problem 2: Non-deterministic decision ├─ Run same prompt 10 times, get 10 different decisions ├─ Sometimes: "OK, I'll offer 25%" ├─ Sometimes: "I can only do 15%" ├─ Sometimes: "Let me check with my manager" ├─ Sometimes: "I'm sorry, I can't do that" └─ Result: Inconsistent customer experience

Problem 3: Token waste ├─ Prompt has: │ ├─ Discount rules (deterministic) │ ├─ Tone guidance (non-deterministic) │ ├─ Escalation rules (deterministic) │ ├─ Conversational tips (non-deterministic) │ ├─ Company policy (deterministic) │ └─ Empathy instructions (non-deterministic) │ ├─ Total: 1500+ tokens (bloated) ├─ Only 30% of prompt is actually used per request └─ Result: Expensive, wasteful

Problem 4: Debugging nightmare ├─ Agent did wrong thing (gave 60% discount) ├─ You ask: "Why did you do that?" ├─ Agent: "I thought customer was a high-value account..." ├─ You: "No, policy is max 20%" ├─ Agent: "I know, but customer was so persuasive, and..." ├─ You: "I can't control this. It's non-deterministic." └─ Result: Can't fix behavior (LLM is black box)

Problem 5: Scaling nightmare ├─ As volume grows, inconsistency compounds ├─ Some agents do 15%, others do 25%, others do 40% ├─ Customer experience is unpredictable ├─ Some customers get angry (unfair discount rates) ├─ Revenue leaks (can't control margin) └─ Result: Business risk

=== THE CORE ISSUE ===

LLMs are fundamentally non-deterministic: ├─ They use temperature=0.7 (for creativity) ├─ They use sampling (not always same output) ├─ They can't follow complex rules reliably ├─ They can't maintain state across turns └─ They CAN'T be deterministic for critical decisions

But business logic MUST be deterministic: ├─ "If discount > 20%, escalate" (MUST be enforced) ├─ "If payment fails, retry 3x" (MUST be enforced) ├─ "If customer is VIP, apply 15% auto" (MUST be enforced) └─ Can't leave to LLM whim

The solution: Split them.

---

A solução: Deterministic Core + Non-Deterministic Shell

Como separar lógica de negócio de interação

=== CORRECT ARCHITECTURE ===

Deterministic CORE (Business Logic): ├─ Rule: "If discount_requested > 20% → escalate_to_human" ├─ Rule: "If customer.vip == true → apply_15%_auto" ├─ Rule: "If revenue_this_month < R$50k → offer_10%" ├─ Rule: "If inventory < 10 units → don't_discount" ├─ Rule: "If payment_failed → retry_3x_then_escalate" │ ├─ How it works: │ ├─ Input: discount_requested = 50% │ ├─ Evaluate rules: IF 50% > 20% → ACTION = escalate_to_human │ ├─ Output: {action: "escalate", reason: "discount_too_high"} │ └─ No LLM involved (pure logic) │ ├─ Properties: │ ├─ Deterministic: Same input → same output (always) │ ├─ Explainable: You know exactly why decision was made │ ├─ Testable: Easy to unit test │ ├─ Fast: <10ms per decision │ ├─ Reliable: No randomness, no failures │ └─ Cost: Negligible (no API calls)

└─ Example code: python def evaluate_discount(discount_requested, customer): if discount_requested > 20: return {"action": "escalate", "reason": "discount_too_high"} elif customer.vip: return {"action": "approve", "discount": 15} elif revenue_this_month < 50000: return {"action": "approve", "discount": 10} else: return {"action": "approve", "discount": 5}

Non-Deterministic SHELL (User Interaction): ├─ Task: Take core decision + make it natural ├─ Input: {action: "escalate", reason: "discount_too_high"} ├─ LLM task: "Generate empathetic message explaining we need manager approval" │ ├─ How it works: │ ├─ Core decided: "escalate" │ ├─ Shell generates: "I'd love to give you 50%! That's amazing, but I need my manager's approval for discounts over 20%. Let me connect you now." │ ├─ Another run: "Wow, 50%! That's a big ask. My policy only allows up to 20%, but let me get my manager to see if we can make an exception." │ ├─ Another run: "50% off would be incredible, but I need to check with my boss first. Let me see what we can do." │ └─ All three are OK (core decision is same, just tone varies) │ ├─ Properties: │ ├─ Non-deterministic: Different wordings each time (natural) │ ├─ Flexible: Adapts to customer personality │ ├─ Creative: Natural language, empathetic │ ├─ Fast: <500ms per message (still responsive) │ ├─ Cost: R$0.01-0.05 per request (cheap) │ └─ Safe: Can't make wrong decisions (core is locked)

└─ Example prompt:

System: "You are friendly sales agent. Your job: make this message sound natural." Input: {action: "escalate", reason: "discount_too_high"} Prompt: "Rephrase this in a friendly way: We need manager approval for 50% discounts." Output: "I'd love to give you 50%! But I need my manager to sign off. Let me connect you now..."

=== ARCHITECTURE DIAGRAM ===

Customer message: "Can I get 50% off?" ↓ [Extract Intent] ├─ discount_requested = 50% ├─ customer_id = 12345 └─ message_tone = friendly ↓ [DETERMINISTIC CORE] ├─ Rule 1: IF discount > 20% → escalate ├─ Rule 2: IF customer.vip → approve (15%) ├─ Rule 3: IF inventory < 10 → no discount ├─ ... ├─ Decision: {action: "escalate", reason: "discount_too_high"} └─ Time: <10ms, Cost: free ↓ [NON-DETERMINISTIC SHELL] ├─ Task: "Make escalation message feel natural" ├─ LLM: Generate empathetic response ├─ Output: "I'd love to give you 50%! But I need manager approval..." └─ Time: 500ms, Cost: R$0.01 ↓ Customer receives natural, empathetic response BUT core decision is guaranteed to be correct

=== BENEFITS ===

Business benefits: ├─ Consistent: Every customer gets same rules (fairness) ├─ Predictable: You know exactly how agent behaves ├─ Profitable: Margins are protected (rules enforce policy) ├─ Scalable: Rules scale infinitely (no LLM needed) ├─ Debuggable: "Why did X happen?" → Check rules └─ Auditable: Compliance team can audit all decisions

Customer experience benefits: ├─ Natural: Responses are conversational, empathetic ├─ Personalized: Tone adapts to customer ├─ Fast: Core decision instant, LLM only for wording └─ Trustworthy: Consistent treatment

Engineering benefits: ├─ Simple: Business logic is separate, easy to understand ├─ Testable: Core can be unit tested (100% coverage) ├─ Maintainable: Change rule in code, not prompt ├─ Cheap: Core has no cost (shell is <5% of total cost) ├─ Fast: Core is <10ms (shell is only bottleneck) └─ Reliable: No randomness, no surprises

=== EXAMPLES ===

Example 1: Discount request ├─ Customer: "Can I get 50% off?" ├─ Core decides: "escalate" (rule: >20% needs manager) ├─ Shell generates: "I'd love to! But I need my manager's approval..." ├─ Result: Consistent policy + natural conversation └─ Customer is happy (feels heard, but gets fair treatment)

Example 2: Refund request ├─ Customer: "Can I get a refund? I changed my mind." ├─ Core decides: "check return window" │ ├─ If <30 days: "approve" │ ├─ If >30 days: "deny" ├─ Shell generates: "Absolutely! I'll process that now..." OR "I understand, but our policy..." ├─ Result: Consistent policy + empathetic response └─ Customer gets fair treatment

Example 3: Urgent issue ├─ Customer: "My payment failed! I need help NOW!" ├─ Core decides: "escalate_to_priority_queue" (rule: payment_failure → priority) ├─ Shell generates: "I understand! Let me get you to our priority team right now..." ├─ Result: Guaranteed fast response + empathetic tone └─ Customer is satisfied (issue handled quickly)

Example 4: VIP customer ├─ Customer: "Can I get a discount?" ├─ Core decides: "approve 15%" (rule: customer.vip == true) ├─ Shell generates: "Of course! As a valued member, here's 15% off..." ├─ Result: Automatic VIP treatment + natural feel └─ Customer is delighted (feels special, gets reward)

---

Implementação: Como migrar pra arquitetura correta (2-3 semanas)

Roadmap passo-a-passo

=== PHASE 1: AUDIT (Week 1) ===

Step 1: Identify decisions your agent makes ├─ List all critical decisions (discount, refund, escalation, etc) ├─ For each: Is it deterministic or creative? ├─ Deterministic = should be in CORE (rules) ├─ Creative = should be in SHELL (LLM) │ ├─ Examples (most of these should be CORE): │ ├─ "Should I approve this discount?" → CORE (rules) │ ├─ "Should I escalate to human?" → CORE (rules) │ ├─ "Should I retry payment?" → CORE (rules) │ ├─ "How to phrase this nicely?" → SHELL (LLM) │ ├─ "What tone should I use?" → SHELL (LLM) │ └─ "How to empathize with customer?" → SHELL (LLM)

Step 2: Extract business rules ├─ For each CORE decision, write the rule ├─ "IF discount_requested > 20% THEN escalate" ├─ "IF customer.vip THEN apply_15%_auto" ├─ "IF payment_failed THEN retry_3x" ├─ "IF inventory < 10 THEN no_discount" ├─ Total: Should have 20-50 rules

Step 3: Audit current agent ├─ How many times per day does wrong decision happen? ├─ How many times inconsistent? ├─ What is the cost (margin loss, customer churn, etc)? ├─ Estimate ROI of splitting (should be 20-50% margin improvement)

Step 4: Plan migration ├─ Start with most critical decision (biggest cost/risk) ├─ Extract rules → implement in code ├─ Keep LLM only for response generation ├─ Measure improvement ├─ Repeat for next decision

=== PHASE 2: DESIGN (Week 1) ===

Step 1: Design core logic ├─ For each CORE decision: │ ├─ List all rules │ ├─ Define input (what data needed?) │ ├─ Define output (what decision?) │ └─ Define precedence (which rule wins if multiple match?) │ ├─ Example (discount): │ Input: {discount_requested, customer.vip, revenue_this_month, inventory} │ Rules: │ 1. IF discount_requested > 20% → escalate │ 2. IF customer.vip → approve_15% │ 3. IF revenue_this_month < 50k → approve_10% │ 4. ELSE → approve_5% │ Output: {action, discount%, reason}

Step 2: Design shell prompts ├─ For each core output, design a prompt ├─ "Given core decision is [action], rephrase as natural message" ├─ Keep prompts SHORT (only ask for wording, not decision) ├─ Example prompt: │ "Core decided: escalate. Rephrase in friendly way: 'I need manager approval for >20% discounts.'" │ └─ Result: Prompt is <100 tokens (cheap, fast)

Step 3: Design integration ├─ How does core connect to shell? ├─ Step 1: Extract intent from customer message (small LLM or rules) ├─ Step 2: Evaluate core logic (deterministic) ├─ Step 3: Generate shell response (LLM) ├─ Step 4: Return to customer │ └─ Total flow: <1 second end-to-end

=== PHASE 3: IMPLEMENT (Week 2) ===

Step 1: Implement core logic (most critical decision) ├─ Write code for decision rule ├─ Example (Python): python def evaluate_discount(request): discount_requested = request.discount_requested customer = request.customer

   # Rule 1: Max 20% without escalation
   if discount_requested > 20:
       return {"action": "escalate", "reason": "discount_too_high"}
   
   # Rule 2: VIP automatic 15%
   if customer.vip:
       return {"action": "approve", "discount": 15}
   
   # Rule 3: Low revenue threshold
   if revenue_this_month < 50000:
       return {"action": "approve", "discount": 10}
   
   # Default
   return {"action": "approve", "discount": 5}

Step 2: Implement shell LLM ├─ Write shell prompt (short, only about wording) ├─ Example prompt:

System: "You are a friendly sales agent. Your job: make messages sound natural."

Core Decision: {action: "escalate", reason: "discount_too_high"}

Task: "Rephrase this decision as a friendly message: 'I need manager approval for discounts over 20%.'"

Requirements:

• Be empathetic
• Acknowledge customer's request
• Explain why (policy)
• Show willingness to help (escalate)
• Keep it <50 words

Step 3: Integrate ├─ Step 1: Extract intent from customer message │ ├─ Customer: "Can I get 50% off?" │ ├─ Extract: {discount_requested: 50%, request_type: "discount"} │ ├─ Step 2: Call core logic │ ├─ evaluate_discount({discount_requested: 50%, customer: {...}}) │ ├─ Returns: {action: "escalate", reason: "discount_too_high"} │ ├─ Step 3: Call shell LLM │ ├─ generate_response({action: "escalate", reason: "discount_too_high"}) │ ├─ Returns: "I'd love to give you 50%! But I need my manager's approval..." │ └─ Step 4: Send to customer └─ "I'd love to give you 50%! But I need my manager's approval..."

Step 4: Test locally ├─ Test core logic (unit tests) ├─ Test: discount=50% → escalate ✓ ├─ Test: discount=15% + vip=true → approve_15% ✓ ├─ Test: discount=10% + revenue<50k → approve_10% ✓ └─ All tests should pass (deterministic) ├─ Test shell (manual) ├─ Run 10 times, responses should vary in wording but decision is same ✓ ├─ All responses should be natural and empathetic ✓

=== PHASE 4: PILOT (Week 2-3) ===

Step 1: Parallel testing ├─ Route 10% traffic → New architecture (deterministic core + shell) ├─ Route 90% traffic → Old architecture (single LLM) ├─ Measure: │ ├─ Decision consistency (should be 100% new, ~60% old) │ ├─ Margin (should improve with new) │ ├─ Customer satisfaction (should improve or same) │ ├─ Latency (should be similar) │ ├─ Cost (should decrease ~30% new) │ └─ Error rate (should decrease new)

Step 2: Analyze results ├─ Is new architecture more consistent? (target: 95%+ vs 60% old) ├─ Is margin protected? (target: discount average 5-10%, not 15-20%) ├─ Is customer satisfaction same or better? ├─ Any unexpected issues? └─ Decision: Ready to roll out?

Step 3: Iterate ├─ If issues found, adjust rules and retry ├─ If successful, proceed to gradual rollout

=== PHASE 5: ROLLOUT (Week 3) ===

Gradual migration: ├─ Day 1-2: 10% traffic → new (10% old) ├─ Day 3-4: 30% traffic → new (70% old) ├─ Day 5-6: 50% traffic → new (50% old) ├─ Day 7: 100% traffic → new (0% old) │ └─ At each step: Monitor metrics, rollback if issues

=== FINANCIAL IMPACT ===

Before (single LLM): ├─ Inconsistent discounts: Average 12% (policy says 5-10%) ├─ Cost margin loss: ~3% (R$1.5k/month on R$50k MRR) ├─ LLM cost: R$200/month (token usage) ├─ Total monthly cost: R$1.7k └─ Annual: R$20.4k

After (deterministic core + shell): ├─ Consistent discounts: Average 7% (policy enforced) ├─ Cost margin loss: ~0.5% (R$250/month) ├─ LLM cost: R$100/month (cheaper shell prompts) ├─ Total monthly cost: R$350 └─ Annual: R$4.2k

Savings: ├─ Monthly: R$1.35k ├─ Annual: R$16.2k ├─ Implementation: ~80 hours engineering (R$24k) ├─ Payback: 1.5 months └─ ROI: 8x first year

---

Conclusão

Simple verdade:

1 LLM não consegue ser determinístico E criativo simultaneamente.

Você precisa SEPARAR:

• Deterministic Core: Business logic (rules, decisions) - NO LLM
• Non-Deterministic Shell: User interaction (tone, naturalness) - LLM only

Benefícios:

• Consistent decisions: Same rules for all customers (fair, profitable)
• Natural responses: LLM makes them sound friendly (good UX)
• Fast: Core is <10ms, shell is only bottleneck
• Cheap: Core costs nothing, shell costs 50% less than monolithic LLM
• Reliable: No randomness, no failures
• Debuggable: Explainable decisions (audit trail)

Cost:

• Implementation: 80 hours (~R$24k)
• Timeline: 2-3 weeks
• Payback: 1.5 months
• ROI: 8x first year (R$16.2k annual savings)

Recomendação: Split your LLM architecture THIS WEEK.

Your single-LLM agent is costing you money and frustrating customers.

---

Próximos passos

Na OpenClaw, ajudamos SaaS builders arquitetar agents com deterministic core + non-deterministic shell:

• Architecture Assessment: É sua arquitetura single-LLM ou split? (diagnosis)
• Rule Extraction: Como identificar suas business rules? (planning)
• Core Logic Design: Como implementar deterministic core? (architecture)
• Shell Prompt Optimization: Como fazer shell LLM eficiente? (performance)
• Decision Logging: Como auditar todas as decisões? (compliance)
• A/B Testing: Como validar nova arquitetura? (safety)
• Gradual Migration: Como migrar sem quebrar produção? (rollout)
• Rule Maintenance: Como versionar e atualizar rules? (scalability)
• Cost Optimization: Como reduzir 50% do LLM cost? (efficiency)
• Customer Experience: Como manter naturalness com determinism? (balance)

Deterministic Core + Non-Deterministic Shell | Agent Architecture | Business Rules Automation →