Seu agente tá preso em texto (multimodal é agora)

Seu agente tá preso em texto (multimodal é agora)

Notícia: vLLM Semantic Router lançou Decision 3.0 (open-source, Apache 2.0). O quê: Models que leem texto + imagem + JSON em 1 call. Tamanhos: 0.85B até 26B (5 options). Implicação: Agentes que usam texto-only tão OBSOLETOS agora. Multimodal é commodity, não luxo. Seu agente? Precisa evoluir pra texto+imagem ou vai ficar pra trás.

Problema: Seu agente WhatsApp, hoje:

CUSTOMER: ├─ "Oi, meu produto veio quebrado" (texto) ├─ [envia foto do produto quebrado] (imagem) │ └─ AGENTE RESPONSE (texto-only): ├─ "Lamento. Pode descrever o problema?" ├─ Customer: "Foto já é descrição!" ├─ Problema: Agente não consegue VER foto ├─ Agente: "Preciso de texto, foto não ajuda" ├─ Customer: "Que agente burro" ├─ Experience: Péssima (agente é inútil pra foto) └─ Outcome: Customer abandona chat, vai pra competitor

COM MULTIMODAL (Decision 3.0): ├─ "Oi, meu produto veio quebrado" (texto) ├─ [envia foto do produto quebrado] (imagem) │ └─ AGENTE RESPONSE (multimodal): ├─ Agente VÊ foto: "Vejo o problema (canto esquerdo tá rachado)" ├─ Agente ANALISA: "É dano de transporte (comum)" ├─ Agente OFERECE: "Vou processar troca imediata. Aqui está RMA." ├─ Experience: Excelente (agente ENTENDE foto) ├─ Outcome: Problema resolvido em 1 msg (vs 5 msgs com texto-only) └─ Customer: "Wow, agente entendeu tudo de uma vez"

DIFERENÇA: ├─ Texto-only: 5 mensagens, frustração, abandono ├─ Multimodal: 1 mensagem, satisfação, loyalty ├─ Cost: Mesmo (1 agente call vs 5) ├─ Quality: Multimodal ganha 10x └─ Competitive: Multimodal = diferencial ENORME

Implicação: Você tá deixando dinheiro na mesa (NPS ruim, churn alto) porque agente não consegue ver imagens. Decision 3.0 = solução (fácil, open-source, free).

---

Por que multimodal muda TUDO

1. Clientes querem mandar fotos (não descrever)

REALIDADE: ├─ Customer: "Meu produto tá com problema" ├─ Natural: Tira foto, manda (picture says 1000 words) ├─ Antinatural: Escreve descrição em texto (effort, ambíguo) │ ├─ Com texto-only agente: │ ├─ Customer manda foto (agente ignora) │ ├─ Customer escreve descrição (frusante) │ ├─ Agente processa texto (ineficiente, pode misunderstand) │ ├─ Customer: "Por que agente não vê foto?" │ ├─ Result: Experience é ruim (agente seems stupid) │ └─ NPS impact: -2 pts (customer expected multimodal) │ └─ Com multimodal agente: ├─ Customer manda foto (agente processa) ├─ Agente VÊ problema (instantly) ├─ Agente responde (com confiança) ├─ Customer: "Agente entendeu de primeira" ├─ Result: Experience é excelente (agente seems smart) └─ NPS impact: +2 pts (customer impressed)

TOTAL DELTA: +4 pts NPS (just from multimodal)

2. Multimodal = routing mais preciso (menos erros)

EXEMPLO: Support ticket classification

TEXTO-ONLY: ├─ Customer: "Preciso de ajuda" ├─ Agente tenta classificar (texto vago) ├─ Agente: "É isso bug ou question?" ├─ Error rate: 30% (texto ambíguo) │ └─ Resultado: Ticket vai pra queue errada ├─ Bug → Support (certo) ├─ Bug → Sales (errado, routing fail) ├─ Sales tenta ajudar (não é expertise) ├─ Customer waits 2 horas (wrong queue) ├─ Escalate → Support (finally) ├─ Total time: 3 horas (vs 15 min com routing certo) └─ Cost: R$ 50 labor wasted

MULTIMODAL: ├─ Customer: "Preciso de ajuda" + [screenshot error] ├─ Agente VÊ screenshot (contexto claro) ├─ Agente: "Ah, é bug na página [X]" ├─ Agente classifica: Bug (confidence 95%) ├─ Error rate: 5% (imagem disambiguates) │ └─ Resultado: Ticket vai pra queue certa ├─ Bug → Support (certo, 1st time) ├─ Support tem contexto (screenshot) ├─ Resolve em 15 min ├─ Total time: 15 min (vs 3 horas) ├─ Cost: R$ 5 labor (vs R$ 50) └─ Saving: R$ 45 per ticket (profit 10x)

AT SCALE: ├─ 100 tickets/day ├─ Saving: R$ 4,500/day (multimodal routing) ├─ Annual: R$ 1.35M (just from routing accuracy) └─ ROI: Negative (multimodal is free) → 100% positive

3. Decision 3.0 = open, small, fast (não precisa GPT-4V)

ANTES (texto-only routing): ├─ Option 1: Custom rules │ ├─ "If [keyword], route to [queue]" │ ├─ Brittle (breaks on variations) │ ├─ Manual (need person to write rules) │ ├─ Slow (update rules → redeploy) │ └─ Cost: R$ 0 (but low quality) │ ├─ Option 2: LLM API (GPT-4, Claude) │ ├─ Cost: R$ 0.05 per call (expensive at scale) │ ├─ Latency: 2-5 seconds (slow for chat) │ ├─ Complexity: API calls (auth, retry, error handling) │ ├─ Multimodal? GPT-4V costs more (R$ 0.10+) │ ├─ Privacy: Data goes to external API (risk) │ └─ Quality: Good (but overkill for routing) │ └─ Option 3: Local small model (was unavailable) ├─ Wanted: Fast, cheap, local routing model ├─ Problem: No good open multimodal routing model ├─ Workaround: Use text-only (worse quality) └─ Result: Stuck with trade-off (speed vs quality)

AGORA (Decision 3.0): ├─ Available: 5 open multimodal models (0.85B - 26B) ├─ Cost: R$ 0 (open-source, self-hosted) ├─ Latency: <100ms (local, on-device) ├─ Complexity: One API call (local, no auth) ├─ Multimodal: Built-in (text+image+JSON) ├─ Privacy: Data stays local (no external API) ├─ Quality: Excellent (trained for routing) ├─ License: Apache 2.0 (free, commercial) └─ Result: Sweet spot (speed + quality + cost)

EXAMPLE: WhatsApp agente routing ├─ Customer message: text + image (photo of issue) ├─ Agente call Decision 3.0 (local, <100ms) ├─ Decision output: {category: "bug", confidence: 0.92, queue: "support"} ├─ Agente routes (to correct queue) ├─ Cost: ~R$ 0.0001 (negligible, local compute) ├─ Latency: 100ms (vs 2-5s with API) ├─ Quality: 92% confidence (vs 30% with rules) └─ Result: Perfect (fast + cheap + good)

---

Como usar Decision 3.0 no seu agente (3 estratégias)

Strategy 1: Image understanding (agente vê problemas visuais)

USE CASE: Support agente (customer sends product image)

BEFORE (texto-only): ├─ Customer: "Meu notebook tá lento" [sends laptop photo] ├─ Agente: "Que tipo de lentidão? Aplicativo específico?" ├─ Customer: "Tá lento em TUDO" ├─ Agente: "Que sistema operacional?" ├─ Customer: "Windows" ├─ Agente: "Pode reiniciar?" ├─ [5 mensagens depois...] ├─ Agente: "Ah, vejo foto agora que enviou. Ah, HD tá lotado!" ├─ Timeline: 10 minutos └─ Experience: Terrível (agente cego até fim)

AFTER (Decision 3.0 + image understanding): ├─ Customer: "Meu notebook tá lento" [sends laptop photo] ├─ Agente ANALISA FOTO: │ ├─ Vê: Windows 11 (sistema operacional) │ ├─ Vê: Taskbar com muitos apps abertos │ ├─ Vê: Task manager mostrando 92% CPU │ ├─ Interpreta: "Sistema overloaded (apps + CPU uso)" │ └─ Diagnóstico instant (sem 5 perguntas) │ ├─ Agente responde: │ ├─ "Vejo problema: muitos apps abertos (Task Manager mostra 92% CPU)" │ ├─ "Feche Chrome, Spotify, Discord (tá consumindo CPU)" │ ├─ "Se notebook continua lento, verificamos disk space" │ └─ Solução clara (from 1 message) │ ├─ Timeline: 1 minuto (1 exchange) └─ Experience: Excelente (agente ENXERGOU problema)

IMPLEMENTATION: ├─ Step 1: Customer sends image ├─ Step 2: Agente extracts image (base64) ├─ Step 3: Agente calls Decision 3.0 (query: "What's the technical problem in this image?") ├─ Step 4: Decision 3.0 returns: {problem: "high CPU usage", confidence: 0.95} ├─ Step 5: Agente generates response (based on Decision output) ├─ Step 6: Send to customer (1 message, solved) └─ Code example: python from vllm import LLM llm = LLM("deci-ai/decision-3-0-flash-8b") image_data = customer.image_base64 query = f"What technical problem do you see in this image? Return: problem, severity, solution." result = llm.generate(prompt=query, image=image_data) response = generate_response(result) send_to_customer(response)

BENEFITS: ├─ Instant diagnosis (no back-and-forth) ├─ High accuracy (image > text description) ├─ Better NPS (customer impressed) ├─ Lower CSAT time (5x faster) └─ Cost: Negligible (local, 0.85B-8B models fine)

Strategy 2: Document classification (agente lê receipts, invoices, etc)

USE CASE: Billing agente (customer disputes charge)

BEFORE (texto): ├─ Customer: "I was charged twice" [sends receipt photo] ├─ Agente: "Can you paste the receipt content?" ├─ Customer: Types out receipt manually (error-prone) ├─ Agente: "I see invoice #123. Let me check..." ├─ Agente: "Actually that's from March. Your charge was September." ├─ Customer: Frustrated (had to type, wrong receipt analyzed) └─ Result: Bad experience, escalation to human

AFTER (Decision 3.0): ├─ Customer: "I was charged twice" [sends receipt photo] ├─ Agente ANALYZES: │ ├─ Decision 3.0 extracts: invoice number, date, amount, items │ ├─ Decision output: {invoice: "INV-2026-0945", date: "2026-10-15", amount: 250.00} │ ├─ Agente checks billing system (matches) │ ├─ Agente detects: Two identical charges (same date, same amount) │ └─ Agente offers: "I see the duplicate. Processing refund now." │ ├─ Timeline: 30 seconds (instant resolution) └─ Result: Great experience, no escalation

IMPLEMENTATION: ├─ Decision 3.0 query: "Extract from invoice: invoice_number, date, total_amount, items" ├─ Model processes image → outputs JSON ├─ Agente parses JSON (typed, not text parsing) ├─ Agente validates against billing DB ├─ Agente auto-refunds (if duplicate confirmed) └─ Cost: ~R$ 0.0001 per document (negligible)

BENEFITS: ├─ No manual entry (OCR automatic) ├─ High accuracy (Decision 3.0 is routing model, great for OCR routing) ├─ Instant processing (vs manual review) ├─ Structured output (JSON, not text parsing) ├─ Scalable (handle 1000s documents/day) └─ NPS: +3 pts (customer appreciates speed)

Strategy 3: Semantic routing (agente direciona pra melhor handler)

USE CASE: Multi-department agente (sales, support, billing)

BEFORE (rules or text-only): ├─ Rules: "If contains 'refund', route to billing" ├─ Customer: "Can you refund my account?" ├─ Rule triggers: Routes to billing (correct) │ ├─ Customer: "I don't want to proceed, can I get a refund?" ├─ Rule triggers: Routes to billing (wrong, should be sales) ├─ Billing: "This is a sales question, not a refund" ├─ Escalates: Back to sales (2x routing) ├─ Delay: 5 minutes (wrong queue, then escalation) └─ Result: Bad experience (routing confusion)

AFTER (Decision 3.0 multimodal routing): ├─ Customer message: "I don't want to proceed, can I get a refund?" + [product screenshot] ├─ Decision 3.0 analyzes: {intent: "churn_prevention", queue: "sales", confidence: 0.94} ├─ Agente interprets: "Customer wants to cancel (sales issue, not billing refund)" ├─ Routes to: Sales (counter-offer, discount negotiation) ├─ Sales: "Wait, I can give you 20% discount?" ├─ Customer: "Actually, okay let's keep it" ├─ Timeline: 2 minutes (correct queue) └─ Result: Upsell prevented churn (sales win)

KEY DIFFERENCE: ├─ Text-only: "Refund" keyword → Simple rule → Often wrong ├─ Multimodal: Context (image, intent, nuance) → Semantic routing → Correct ├─ Example: Same word "refund" can mean: │ ├─ "Refund my money" (billing issue) │ ├─ "Can I get a refund if I change my mind?" (sales question) │ └─ "I deserve a refund for this broken product" (support issue) │ └─ Decision 3.0 distinguishes (multimodal context) │ └─ Cost: R$ 0.0001 per route (vs lost customer if routed wrong)

IMPLEMENTATION: python from vllm import LLM llm = LLM("deci-ai/decision-3-0-flash-8b")

def route_message(text: str, image: Optional[bytes] = None) -> str: query = f"Intent: is this a [sales|support|billing|other] issue? Confidence?" result = llm.generate(prompt=query, text=text, image=image) routing_decision = parse_json(result) # {queue, confidence} return routing_decision["queue"]

queue = route_message(customer_message, customer_image) send_to_queue(queue, customer_message)

BENEFITS: ├─ Semantic understanding (vs keyword rules) ├─ Multi-modal context (image + text) ├─ Higher accuracy (Decision 3.0 trained for routing) ├─ Fewer escalations (correct queue 1st time) ├─ Scale: Handle complex routing rules (no code changes) └─ ROI: Faster resolution = higher NPS = retention

---

Comparação: Decision 3.0 vs alternatives

┌─────────────────┬──────────────────┬──────────────────┬──────────────────┐ │ Métrica │ Custom Rules │ GPT-4V (API) │ Decision 3.0 │ ├─────────────────┼──────────────────┼──────────────────┼──────────────────┤ │ Cost │ R$ 0 (dev time) │ R$ 0.10/call │ R$ 0 (open) │ │ Latency │ <10ms │ 2-5s │ <100ms │ │ Multimodal │ No (text only) │ Yes (expensive) │ Yes (free) │ │ Accuracy │ ~50% (rules) │ ~95% │ ~90% │ │ Privacy │ Local │ External API │ Local │ │ Scalability │ Brittle (rules) │ API rate limits │ Linear (self-hosted) │ │ Setup effort │ High (code) │ Medium (API key) │ Low (model load) │ │ Context length │ N/A │ 128K tokens │ Not disclosed* │ │ Models available│ 1 (yours) │ 1 (GPT-4V) │ 5 sizes (0.85B-26B) │ │ License │ Yours │ Proprietary │ Apache 2.0 │ └─────────────────┴──────────────────┴──────────────────┴──────────────────┘

• Decision 3.0 context length not publicly disclosed yet, but Semantic Router has handled large contexts before

BEST CHOICE: ├─ If: Simple routing (1-2 categories), text-only → Custom rules ├─ If: Complex understanding, max quality, cost no issue → GPT-4V ├─ If: Balanced (speed + quality + cost), multimodal needed → Decision 3.0 ✓ BEST └─ Most B2B SaaS: Decision 3.0 (sweet spot)

---

Conclusão: Multimodal agora é obrigatório

Fatos:

✓ Texto-only agentes tão obsoletos (customers querem mandar imagens) ✓ Multimodal = 10x melhor UX (1 message vs 5 messages) ✓ Decision 3.0 = multimodal acessível agora (open, free, fast) ✓ Não precisa GPT-4V (Decision 3.0 melhor pra routing) ✓ Local deployment = privacy + speed (vs API) ✓ 5 tamanhos = escolhe modelo pra seu budget (0.85B até 26B) ✓ ROI: Positivo (melhor routing = menos escalações = profit) ✓ Timeline: Implementar em 2 semanas (simple) ✓ Cost: Negligible (local compute, open-source) ✓ Message: "Multimodal is now table stakes"

ACTION ITEMS (THIS MONTH):

1. TODAY: Audit agente (tá processando imagens?)
2. THIS WEEK: Test Decision 3.0 (spin up local model)
3. THIS WEEK: Identify use cases (routing, image understanding, document classification)
4. NEXT WEEK: Pick strategy (image understanding? routing? both?)
5. NEXT WEEK: Implement 1 strategy (start with highest ROI)
6. WEEK 3: Test with real messages (customer images)
7. WEEK 3: Measure impact (NPS, CSAT, resolution time)
8. WEEK 4: Scale to other use cases
9. WEEK 4: Monitor performance (accuracy, latency)
10. MONTH 2: Optimize model size (pick best model for your scale)

Problema resolvido quando: └─ Agente processa imagens (não ignore) └─ Multimodal routing (texto + imagem + JSON) └─ Instant understanding (sem back-and-forth) └─ NPS impact (customers impressed) └─ Cost negligible (local, fast) └─ Privacy maintained (no external API) └─ Scalable (self-hosted, no API limits) └─ Open-source (Apache 2.0, forever free) └─ Result: Agente 10x mais capaz + 0 cost

→ OpenClaw: Agente Multimodal (Decision 3.0 Integration)

Seu agente? Preso em texto. Decision 3.0? Multimodal (open, free, fast). Implementar agora = 10x melhor UX + NPS boost + zero cost. Multimodal é agora obrigatório. 📸✨