Seu agente gera texto? (decision models são 10x mais baratos)

Seu agente gera texto? (decision models são 10x mais baratos)

Notícia: Amazon publicou tutorial: Deploy decision models no SageMaker AI (com llama.cpp). O quê é: Decision models = scored classification (agent escolhe entre opções fixas, não gera texto livre). Aplicação: Routing (encaminhar customer), policy outcomes (aplicar política), agent actions (selecionar ação). Problema resolvido: Agentes gerando texto livre (caro, lento, impreciso). Decision models = scored output (barato, rápido, preciso). Implicação: Seu agente WhatsApp (que gera responses em free-form) tá gastando 10x mais tokens + sendo mais lento do que deveria. Decision models = architectural shift (split agent em dois: decisor rápido + gerador quando necessário). Você precisa implementar AGORA ou tá jogando dinheiro fora.

Realidade: **

DECISION MODELS vs FREE-FORM GENERATION:

CURRENT AGENT (FREE-FORM TEXT):

Customer: "Quero saber status do meu pedido" └─ Agent prompt: "Analise essa request e responda" └─ Agent thinks: (internal reasoning) └─ Agent generates: ├─ "Para ajudá-lo com o status do pedido..." ├─ "Primeiro preciso confirmar seu ID de pedido..." ├─ "Qual é o seu número de pedido? Pode estar em..." └─ Total: 50-100 tokens (POR CADA REQUEST) └─ Cost: 50-100 tokens * $0.003/1K = $0.00015 PER REQUEST └─ Speed: 0.5-2 seconds (token generation time) └─ Problem: Expensive + slow + can hallucinate

WITH DECISION MODELS (SCORED CLASSIFICATION):

Customer: "Quero saber status do meu pedido" └─ Agent classification: "Which intent?" └─ Decision model input: Customer message (just text) └─ Decision model output: ├─ Intent 1: "Order status" → score 0.95 ✓ (chosen) ├─ Intent 2: "Refund inquiry" → score 0.03 ├─ Intent 3: "Product question" → score 0.02 └─ Total: 5 tokens (fixed, not variable!) └─ Then (if needed): ├─ Route to: Database lookup (not LLM) ├─ Response: Template (not generated) └─ Cost: Fixed (5 tokens + template) └─ Total cost: 5 tokens * $0.003/1K = $0.000015 PER REQUEST └─ Speed: <100ms (deterministic, not generation) └─ Accuracy: 99%+ (trained on labels, not hallucinating)

THE DIFFERENCE:

Free-form generation: ├─ Cost per request: $0.00015 (variable, can spike) ├─ Tokens per request: 50-100 (can grow) ├─ Speed: 0.5-2 seconds (depends on token count) ├─ Quality: 90-95% (sometimes hallucinate, make up info) ├─ Reliability: Medium (can give wrong answer) └─ Use case: Open-ended, creative responses

Decision models: ├─ Cost per request: $0.000015 (10x cheaper, fixed) ├─ Tokens per request: 5 (fixed, minimal) ├─ Speed: <100ms (instant, deterministic) ├─ Quality: 99%+ (trained on data, no hallucination) ├─ Reliability: High (always picks from valid options) └─ Use case: Routing, classification, policy selection

WHEN TO USE EACH:

Use FREE-FORM (LLM generation) for: ├─ Open-ended questions (customer asks custom question) ├─ Creative responses (personalized, narrative) ├─ Context-dependent answers (needs reasoning) └─ Example: "Tell me about your product features" (needs explanation)

Use DECISION MODELS (scored classification) for: ├─ Intent routing (what does customer want?) ├─ Policy application (which rule applies?) ├─ Action selection (what should agent do next?) ├─ Sentiment classification (is customer happy?) ├─ Priority routing (how urgent is this?) └─ Example: "Route this support ticket to right team"

RESULT: HYBRID AGENT

Customer: "Quero saber status do meu pedido" ├─ Step 1: Decision model │ ├─ Input: Customer message │ ├─ Output: Intent = "Order status" (scored) │ ├─ Cost: 5 tokens │ ├─ Speed: <100ms │ └─ Confidence: 0.95 (high) │ ├─ Step 2: Route (deterministic) │ ├─ Action: Look up database (not LLM!) │ ├─ Fetch: Order info (from DB) │ ├─ Cost: 0 (database is free) │ ├─ Speed: <50ms (database query) │ └─ Reliability: 100% (exact data) │ ├─ Step 3: Format response │ ├─ Use template: "Your order #123 status is..." │ ├─ Fill values: From database │ ├─ Cost: 0 (template, no LLM) │ ├─ Speed: <10ms │ └─ Quality: Perfect (data from source) │ └─ Total: ├─ Cost: $0.000015 (vs $0.00015 free-form = 10x cheaper) ├─ Speed: <200ms (vs 1-2s free-form = 10x faster) ├─ Quality: Perfect (vs 90% free-form = more reliable) └─ Result: Win-win-win (cheaper, faster, better)

---

How Amazon SageMaker makes decision models easy

Deploy scored classifiers in minutes (not days)

AMAZON SAGEMAKER + LLAMA.CPP (HOW IT WORKS):

Traditional ML deployment: ├─ Problem: Complex (need GPU, containerization, scaling) ├─ Setup: Weeks (data prep, training, testing) ├─ Cost: High (full ML infrastructure) ├─ Maintenance: Difficult (monitoring, updating) └─ Result: Only big companies do this

SageMaker with llama.cpp: ├─ Idea: Pre-trained LLM (Llama, Qwen, etc.) + prompt trick │ ├─ Trick: Use LLM to score (not generate) │ ├─ Prompt: "Classify this into: [Option A, B, C]. Return ONLY the letter." │ ├─ Output: "A" (single token, deterministic) │ └─ Benefit: No fine-tuning needed (few-shot works) │ ├─ Quantization (llama.cpp): │ ├─ What: Compress model (float32 → int8) │ ├─ Benefit: 4x smaller, 4x faster, same accuracy │ ├─ Cost: $0.001 vs $0.01 per inference │ └─ Speed: <100ms vs 1-2s │ ├─ SageMaker endpoint: │ ├─ Setup: API endpoint (managed by AWS) │ ├─ Scaling: Automatic (handles spikes) │ ├─ Cost: Pay per request ($0.000015 per inference) │ └─ Deploy: 5 minutes (click, don't code) │ └─ Result: Decision model in production (same day)

ARCHITECTURE:

Your agent: ├─ Receives: Customer message ├─ Calls: SageMaker endpoint (decision model) │ ├─ Input: "Quero status do meu pedido" │ ├─ Model: Llama2 (quantized, 7B params) │ ├─ Prompt: "Classify intent: [Status, Refund, Product, Other]" │ ├─ Output: "Status" (single token) │ └─ Latency: <100ms │ ├─ Receives: Classification result ├─ Routes: To appropriate handler │ ├─ Status → Database lookup │ ├─ Refund → Refund policy │ ├─ Product → Product DB │ └─ Other → Escalate to human │ ├─ Returns: Response (template or direct) └─ Cost: Minimal (5 tokens vs 50-100)

IMPLEMENTATION (STEP-BY-STEP):

Step 1: Prepare dataset ├─ Collect: 100-500 examples │ ├─ Example 1: "Quero status do meu pedido" → Intent: Status │ ├─ Example 2: "Quero cancelar meu pedido" → Intent: Refund │ ├─ Example 3: "Qual é o melhor produto?" → Intent: Product │ └─ Example N: ... │ ├─ Split: 80% train, 20% test (validate) └─ Format: JSON lines (SageMaker format)

Step 2: Create model (minimal code) ├─ Method 1: Fine-tune Llama2 (optional, helps) │ ├─ Data: Your 100-500 examples │ ├─ Time: 30 minutes (on SageMaker notebook) │ ├─ Cost: $5-20 (training is cheap) │ └─ Result: Model optimized for your intents │ ├─ Method 2: Few-shot prompt (no fine-tune needed) │ ├─ Prompt: 3-5 examples in context │ ├─ Time: 5 minutes (just write prompt) │ ├─ Cost: $0 (use pre-trained model) │ └─ Result: Works for simple tasks │ └─ Reality: Method 2 is often enough (start here)

Step 3: Containerize with llama.cpp ├─ Container: Docker image (provided by AWS) ├─ Setup: Add your model + prompt ├─ Build: 5 minutes (docker build) └─ Push: To ECR (AWS container registry)

Step 4: Deploy on SageMaker ├─ Endpoint: Create real-time endpoint ├─ Config: 1x ml.g4dn.xlarge GPU (small) ├─ Time: 5 minutes (SageMaker handles it) ├─ Cost: $0.50/hour (running) └─ Auto-scaling: Set to scale on traffic

Step 5: Call from agent ├─ Code: Simple API call │ python │ import boto3 │ client = boto3.client('sagemaker-runtime') │ response = client.invoke_endpoint( │ EndpointName='my-decision-model', │ Body='Customer message', │ ) │ intent = response['Body'].read() # "Status" │
│ ├─ Latency: <100ms ├─ Cost: $0.000015 per call └─ Scale: Handles 1000s of requests/second

Total time: 1-2 hours (not weeks!) Total cost: $20-50 (model + infrastructure) Payoff: Immediately (10x cheaper than free-form)

---

Real-world example: Support routing agent

How decision models cut costs while improving quality

SCENARIO: E-commerce support (1000 tickets/day)

CURRENT SYSTEM (FREE-FORM AGENT):

Flow: ├─ Customer submits: "Meu pedido não chegou" ├─ Agent (Claude) reads and thinks: │ ├─ Prompt: "What is the customer's issue? Respond helpfully." │ ├─ Generation: 80 tokens (reasoning + response) │ ├─ Cost: 80 * $0.003/1K = $0.00024 │ ├─ Time: 1.5 seconds │ └─ Output: "I understand you didn't receive your order. Let me help." │ ├─ Then agent tries to route (more tokens): │ ├─ Prompt: "What team should handle this?" │ ├─ Generation: 50 more tokens │ ├─ Cost: 50 * $0.003/1K = $0.00015 │ ├─ Total per ticket: $0.00024 + $0.00015 = $0.00039 │ └─ Daily cost (1000 tickets): $0.39 │ └─ Monthly cost: $11.70 (seems small but...) │ └─ Problem: Many errors ├─ Agent sometimes misclassifies (50% error rate) ├─ Tickets routed to wrong team (escalation) ├─ Customers unhappy (not resolved first time) ├─ Manual rework: 20% of tickets = 200/day └─ Real cost: $11.70 + human rework = $1000+/day

Annual cost: $11.70 * 30 * 12 = $4,200 (just tokens) + Rework (humans): $1000/day * 250 days = $250,000 + Lost customers (bad experience): ??? = MASSIVE COST

NEW SYSTEM (DECISION MODELS):

Setup (one-time): ├─ Training data: 200 ticket examples (labeled) ├─ Decision model: Llama2 7B quantized ├─ SageMaker deployment: 5 minutes ├─ Cost: $20 (setup) + $0.50/hour (running) └─ Time: 2 hours

Flow: ├─ Customer submits: "Meu pedido não chegou" ├─ Decision model (SageMaker): │ ├─ Input: Customer message │ ├─ Prompt: "Classify intent: [Not received, Defective, Late, Refund, Other]" │ ├─ Inference: 5 tokens (fixed) │ ├─ Cost: 5 * $0.003/1K = $0.000015 │ ├─ Time: <100ms │ ├─ Output: "Not received" (score: 0.98) │ └─ Confidence: High (will route correctly) │ ├─ Route (deterministic, no LLM): │ ├─ Action: Send to "Not received" team │ ├─ Template response: "We'll investigate your missing package..." │ ├─ Cost: $0 (template, no LLM) │ ├─ Time: <50ms │ └─ Result: Customer routed correctly (first time) │ ├─ Accuracy: 99%+ (vs 50% before) ├─ Cost per ticket: $0.000015 (vs $0.00039 = 26x cheaper!) ├─ Daily cost: $0.000015 * 1000 = $0.015 └─ Monthly cost: $0.015 * 30 = $0.45

SageMaker running cost: $0.50/hour * 24 * 30 = $360/month Total cost: $360 (infrastructure) + $0.45 (inference) = $360.45/month

Savings: $4,200 (old tokens) + $250,000 (rework) = $254,200/month! ROI: Payoff in 1 hour (literally, investment returns instantly)

Plus: ├─ Accuracy: 99% (vs 50%) ├─ Speed: <200ms (vs 2-3 seconds) ├─ Reliability: Deterministic (no hallucination) ├─ Customer satisfaction: Up (faster, correct routing) └─ Team productivity: Up (no rework)

REAL NUMBERS (YOUR CASE):

If you have: ├─ 1000 support tickets/day = $0.39/day free-form ├─ 10,000 support tickets/day = $3.90/day free-form ├─ 100,000 support tickets/day = $39/day free-form │ └─ With decision models: ├─ 1000/day = $0.015/day (0.004% cost) ├─ 10,000/day = $0.15/day (0.004% cost) ├─ 100,000/day = $1.50/day (0.004% cost) │ └─ Plus SageMaker: $360/month (fixed) ├─ 1000/day: Saves $11,700/month (26x) ├─ 10,000/day: Saves $117,000/month (26x) └─ 100,000/day: Saves $1,170,000/month (26x)

---

When to use decision models vs free-form agents

Decision matrix: classification vs generation

USE DECISION MODELS (SCORED CLASSIFICATION) FOR:

✓ Intent routing: "What does customer want?" └─ Example: Status, Refund, Product question, Complaint

✓ Sentiment classification: "Is customer happy?" └─ Example: Happy, Neutral, Angry, Confused

✓ Priority routing: "How urgent?" └─ Example: Critical, High, Medium, Low

✓ Category classification: "Which product category?" └─ Example: Electronics, Clothing, Home, Food

✓ Policy selection: "Which rule applies?" └─ Example: Refund eligible, Exchange eligible, No return

✓ Action routing: "What should agent do next?" └─ Example: Escalate, Process refund, Send info, Close ticket

✓ Quality check: "Is response good?" └─ Example: Good, Needs improvement, Bad

✓ Language detection: "What language?" └─ Example: Portuguese, English, Spanish, French

DO NOT USE DECISION MODELS FOR:

✗ Creative writing: "Write marketing copy" └─ Reason: Creative = open-ended (needs generation)

✗ Explanation: "Explain how to fix" └─ Reason: Needs reasoning + text generation

✗ Open-ended Q&A: "What's your opinion?" └─ Reason: No fixed answer set

✗ Personalization: "Tailor to customer" └─ Reason: Context-dependent (needs generation)

HYBRID APPROACH (BEST):

Step 1: Use decision model (classify intent) = 5 tokens Step 2: Use template or database (no LLM) = 0 tokens Step 3: Use free-form LLM ONLY if needed (edge cases) = 50 tokens

Result: ├─ 90% of requests: 5 tokens (decision model) ├─ 10% of requests: 50 tokens (free-form) ├─ Average cost: (0.9 * 5) + (0.1 * 50) = 4.5 + 5 = 9.5 tokens/request ├─ Total cost: 90% cheaper than pure free-form (50 tokens) └─ Quality: 99% (decision model) + creative (when needed)

---

Implementação passo-a-passo

Do zero até decision model em production

WEEK 1: PREPARE

Day 1: ├─ Collect data: 200-500 examples from your tickets ├─ Label manually: 2-3 hours of work ├─ Format: CSV with [customer_message, intent] └─ Split: 80% train, 20% test

Day 2-3: ├─ Choose model: Llama2 7B (quantized) ├─ Choose platform: Amazon SageMaker ├─ Setup AWS account (if not already) └─ Create SageMaker notebook

Day 4-5: ├─ Fine-tune (optional): 30 minutes on SageMaker │ └─ Or: Skip and use few-shot prompt ├─ Validate: Test on 20% holdout data ├─ Accuracy: Aim for >90% (iterate if needed) └─ Cost: $5-20

WEEK 2: DEPLOY

Day 1-2: ├─ Containerize: Docker image with llama.cpp ├─ Push: To AWS ECR ├─ Create SageMaker endpoint ├─ Config: 1x ml.g4dn.xlarge (small, $0.50/hr) └─ Deploy: 10 minutes

Day 3: ├─ Integrate: Add API calls to your agent ├─ Test: With real customer messages ├─ Validate: Routing works correctly └─ Monitor: Latency, accuracy, cost

Day 4-5: ├─ Scale: Enable auto-scaling ├─ Monitor: Cost, latency, accuracy ├─ Optimize: If needed (usually not) └─ Launch: To production

TOTAL TIME: 10 days TOTAL COST: $20-50 (setup) + $360/month (running) ROI: Payback in days (if you have >100 tickets/day)

---

Conclusão: Decision models = agent optimization 101

Fatos:

✓ Decision models = scored classification (fixed output) ✗ Free-form generation = variable tokens (expensive) ✓ Decision models = 10x cheaper ($0.000015 vs $0.00015) ✗ Free-form = slow (<100ms vs 1-2 seconds) ✓ Decision models = 99%+ accurate (trained on data) ✗ Free-form = can hallucinate (50% error) ✓ Amazon SageMaker makes it easy (deploy in 2 hours) ✗ No excuse to not implement (simple + proven)

YOUR NEXT STEP:

If you have: ├─ Support tickets: Use decision model for routing ✓ ├─ Sales pipeline: Use for lead classification ✓ ├─ Customer inquiries: Use for intent detection ✓ ├─ Any repeated task: Use for classification ✓ └─ Don't use for: Creative writing, open-ended Q&A

EXPECTED OUTCOMES:

After implementing decision models: ├─ Cost: 10x cheaper (tokens) ├─ Speed: 10x faster (latency) ├─ Quality: 99%+ accurate (vs 50% error) ├─ Reliability: Deterministic (no hallucination) ├─ Scalability: Handles 1000s/second └─ Time to implement: 2 weeks

BOTTOM LINE:

Decision models = agent optimization you haven't done yet. Savings = 10x cost reduction (immediate). Implementation = 2 weeks (simple). ROI = Payback in days (if >100 requests/day). Excuse = none (Amazon made it easy). Next step = start this week (not next month).

→ OpenClaw: Implementar Decision Models SageMaker (Agente Routing Eficiente)

Decision models = scored classification (não free-form text). Seu agente gerando 50-100 tokens POR REQUEST? Decision models fazem em 5 tokens (10x mais barato). Amazon SageMaker + llama.cpp = easy deploy (2 horas, não 2 meses). Setup: 20 examples, 2 horas, $20-50. Payoff: Imediato (10x cost savings). Hybrid agent = 90% decision models (cheap) + 10% free-form (creative quando necessário). You: 10 dias pra implementar vs competitors ainda gastando 10x mais. Decisão: AGORA (antes que competitor otimize) vs NUNCA (ficar caro pra sempre). 🚀💰⚡