Seu agente de IA é lento? Precisa de 'compute scratch pad'

Seu agente de IA é lento? Precisa de 'compute scratch pad'

Você é founder de SaaS.

Seu produto:

• Agente de IA (WhatsApp, email, Slack)
• Processa conversas de clientes (suporte, vendas, automação)
• Roda 24/7, milhões de mensagens/dia

Seu problema:

• Agente é LENTO (demora 5-10 segundos por resposta)
• Customer: "Por que está demorando tanto?"
• Seu time: "LLM é assim, é lento"
• Verdade: Não é o LLM. É você não usando "scratch pad"
• Custo: Agente roda por 10 segundos (caro). Competitor roda 2 segundos (barato)
• Math: 5x mais lento = 5x mais caro = 5x menos margem
• Your customer: "Competitor é mais rápido e mais barato. Saio."
• You: Lost deal (e nem sabia por quê)

A notícia que explica:

Abnormal AI (behavioral security, protege 25% do Fortune 500) descobriu que agentes em produção (rodando BILHÕES de operações/dia) todos fazem a mesma coisa: precisam de "compute scratch pad".

O que é? Espaço temporário onde agente PENSA (calcula, valida, raciocina) sem chamar LLM (que é caro e lento).

Exemplo:

• Agente recebe: "Aprova essa transação de R$50K?"
• Sem scratch pad: Agente pergunta LLM, LLM pensa, demora 3 segundos, custa R$0.05
• Com scratch pad: Agente usa código/regex/logic local (não LLM), responde em 50ms, custa R$0.000001

Resultado: 60x mais rápido, 50x mais barato.

Se você não tem scratch pad, seu agente está deixando dinheiro na mesa.

---

O que é "compute scratch pad" (tradução real: memória de trabalho)

Agente pensa sem chamar LLM (economia real).

=== THE PROBLEM: WITHOUT SCRATCH PAD ===

User asks agent: "Does this email look like phishing?"

Agent workflow (SLOW WAY): ├─ Step 1: Agent calls LLM │ └─ LLM: "Is this phishing?" │ └─ Analyzes email (1 second) │ └─ Returns: "Yes, probably phishing" ├─ Step 2: Agent receives answer ├─ Step 3: Agent wants to verify (check sender domain) │ └─ Agent calls LLM again (for domain verification) │ └─ LLM: "Is example.com legitimate?" │ └─ Analyzes domain (1 second) │ └─ Returns: "No, it's spoofed" ├─ Step 4: Agent wants to cross-check with database │ └─ Agent calls LLM again (for database lookup) │ └─ LLM: "Is this sender in our blocklist?" │ └─ Thinks (1 second) │ └─ Returns: "Yes, it's blocked" ├─ Step 5: Agent aggregates all evidence │ └─ Agent calls LLM AGAIN (to summarize) │ └─ LLM: "Based on all evidence..." │ └─ Thinks (1 second) │ └─ Returns final answer └─ Total time: 4-5 seconds Total cost: 4-5 LLM calls × R$0.01 = R$0.04-0.05 per email Processing 1M emails/day = R$40K-50K/day = R$1.2M-1.5M/month

=== THE SOLUTION: WITH SCRATCH PAD ===

Agent workflow (FAST WAY): ├─ Step 1: Agent calls LLM (ONCE) │ └─ LLM: "Analyze this email for phishing" │ └─ LLM thinks deeply (but only ONCE) │ └─ Returns: Detailed analysis + evidence ├─ Step 2: Agent uses scratch pad (local compute) │ ├─ Verify sender domain (regex, local database lookup) - 10ms │ ├─ Check blocklist (local database) - 5ms │ ├─ Aggregate evidence (code logic) - 10ms │ ├─ Cross-reference patterns (regex) - 10ms │ └─ Format final answer - 5ms ├─ Total local compute: 40ms ├─ Total time: 1 second (LLM) + 0.04 seconds (scratch pad) = 1.04 seconds ├─ Total cost: 1 LLM call × R$0.01 = R$0.01 per email └─ Processing 1M emails/day = R$10K/day = R$300K/month (75% cheaper)

=== THE REAL METRIC ===

Without scratch pad (competitor): ├─ Speed: 5 seconds/email ├─ Cost: R$0.05/email ├─ Throughput: 200 emails/second ├─ Monthly for 1B emails: R$50M └─ Result: Expensive, slow (customer waits)

With scratch pad (you): ├─ Speed: 1 second/email ├─ Cost: R$0.01/email ├─ Throughput: 1000 emails/second ├─ Monthly for 1B emails: R$10M └─ Result: Cheap, fast (customer happy)

=== ABNORMAL AI EXAMPLE (REAL SCALE) ===

Abnormal AI protects Fortune 500 (billions of emails/day). Without optimization: Would cost $500M+/month to run. With scratch pad optimization: Costs $10M/month.

Difference: Being smart about when to use LLM (expensive) vs local logic (cheap).

---

Quando usar scratch pad vs LLM (decision matrix)

Nem tudo precisa de IA. Código é mais rápido (e mais barato).

=== DECISION TREE ===

Agent needs to do something: ├─ Is it logic/rule-based? │ ├─ Example: "If sender == blocklist, reject" │ ├─ Answer: YES → Use scratch pad (code), NOT LLM │ ├─ Why: Code is 1000x faster, free │ └─ Implementation: if statement in your agent │ ├─ Does it need semantic understanding? │ ├─ Example: "Is this email tone suspicious?" │ ├─ Answer: YES → Use LLM (needs AI) │ ├─ Why: Can't write rule for tone │ └─ Implementation: Call LLM once, cache result │ ├─ Is it database lookup? │ ├─ Example: "Is this sender in our database?" │ ├─ Answer: YES → Use scratch pad (database query), NOT LLM │ ├─ Why: Database is instant, LLM is hallucinating │ └─ Implementation: SQL query, 10ms max │ ├─ Is it aggregation/summarization? │ ├─ Example: "Combine all phishing signals into one score" │ ├─ Answer: Use scratch pad IF possible (weights + math) │ ├─ Why: Math is instant, LLM is slow │ └─ Implementation: sum([weights]) in code │ ├─ Is it truly ambiguous/semantic? │ ├─ Example: "Is this business email or spam?" │ ├─ Answer: YES → Use LLM (semantic task) │ ├─ Why: Only LLM can understand context │ └─ Implementation: Call LLM, don't repeat │ └─ Is it creative/generative? ├─ Example: "Write response to customer email" ├─ Answer: YES → Use LLM (generative task) ├─ Why: Only LLM can generate text └─ Implementation: Call LLM once, format in scratch pad

=== REAL EXAMPLES (YOUR SAAS) ===

1. Email security agent: ├─ Phishing detection: LLM (semantic) ├─ Sender validation: Scratch pad (regex + database) ├─ Blocklist check: Scratch pad (database) ├─ Evidence aggregation: Scratch pad (math) ├─ Generate report: Scratch pad (formatting) └─ Result: 1 LLM call (cheap), rest is code (fast)

2. Customer support agent: ├─ Intent detection: LLM (semantic) ├─ FAQ lookup: Scratch pad (database) ├─ Ticket creation: Scratch pad (API call) ├─ Response generation: LLM (generative) ├─ Format message: Scratch pad (template) └─ Result: 2 LLM calls (semantic + generative), rest is code

3. Sales qualification agent: ├─ Lead intent: LLM (semantic) ├─ Company lookup: Scratch pad (database) ├─ Revenue check: Scratch pad (database) ├─ Deal score: Scratch pad (math/weights) ├─ Email generation: LLM (generative) └─ Result: 2 LLM calls, rest is code

=== COST COMPARISON (REAL NUMBERS) ===

Old way (call LLM for everything): ├─ 1 customer = 10 emails/day ├─ 10 LLM calls/email (re-calling for each step) ├─ Cost: 10 × 10 × R$0.01 = R$1/day ├─ Monthly: 10 customers × 30 days × R$1 = R$300 ├─ Annual: R$3,600 └─ Per customer per year: R$360

New way (scratch pad + selective LLM): ├─ 1 customer = 10 emails/day ├─ 2 LLM calls/email (semantic + generative) ├─ Cost: 10 × 2 × R$0.01 = R$0.20/day ├─ Monthly: 10 customers × 30 days × R$0.20 = R$60 ├─ Annual: R$720 └─ Per customer per year: R$72

Savings: 80% cost reduction (R$360 → R$72) Extra benefit: 5x faster (customer sees response in 1 sec vs 5 sec)

=== ABNORMAL AI PATTERN (WHAT THEY LEARNED) ===

Running billions of emails/day, they discovered: ├─ 60% of tasks can use scratch pad (logic, database, aggregation) ├─ 30% need LLM (semantic, intent detection) ├─ 10% need deep reasoning (complex analysis) │ ├─ Strategy: │ ├─ Do 60% with code (instant, free) │ ├─ Do 30% with LLM once (semantic pass) │ ├─ Do 10% with LLM thinking (let it reason) │ └─ Result: Minimal LLM calls, maximum performance │ └─ Architecture: Amazon Bedrock AgentCore Code Interpreter ├─ Lets agent execute code (scratch pad) ├─ Sandboxed (safe, no security risk) ├─ Fast (instant execution) └─ Cheap (no LLM cost)

---

Como implementar scratch pad no seu agente (arquitetura)

3 camadas: LLM thinking, scratch pad compute, final action.

=== ARCHITECTURE (3-LAYER MODEL) ===

┌─────────────────────────────────────────┐ │ Layer 1: Input Processing (Scratch Pad) │ ├─────────────────────────────────────────┤ │ • Parse message │ │ • Check syntax/format │ │ • Validate input │ │ • Extract structured data │ │ → Time: 10-50ms (no LLM) │ └─────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────┐ │ Layer 2: Semantic Understanding (LLM) │ ├─────────────────────────────────────────┤ │ • Call LLM (ONE TIME) │ │ • Get intent, key info, reasoning │ │ • Cache result (reuse) │ │ → Time: 1-3 seconds (LLM call) │ └─────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────┐ │ Layer 3: Action/Reasoning (Scratch Pad) │ ├─────────────────────────────────────────┤ │ • Lookup data (database) │ │ • Validate against rules (code) │ │ • Aggregate evidence (math) │ │ • Format response (template) │ │ → Time: 10-100ms (no LLM) │ └─────────────────────────────────────────┘ ↓ Final Response Total: 1-3.2 seconds (vs 5+ seconds without optimization)

=== CODE EXAMPLE (PSEUDO) ===

Layer 1: Scratch pad (input)

def parse_email(email): sender = email.from_header # Fast subject = email.subject # Fast body = email.body # Fast return {sender, subject, body}

Layer 2: LLM (semantic)

def analyze_with_llm(parsed): response = llm.call( "Is this phishing? Analyze: " + parsed ) return response # 1-3 seconds

Layer 3: Scratch pad (action)

def validate_and_decide(llm_response, parsed): # Check blocklist (database) if parsed['sender'] in blocklist: return "REJECT (blocklisted)"

# Check domain reputation (database)
domain_score = database.lookup(parsed['sender'].domain)
if domain_score < 0.3:
    return "REJECT (low domain reputation)"

# Aggregate evidence
llm_confidence = llm_response['confidence']
combined_score = (domain_score * 0.3) + (llm_confidence * 0.7)

if combined_score > 0.7:
    return "REJECT (phishing likely)"
else:
    return "ALLOW (seems legitimate)"

Total time: 50ms + 2000ms + 50ms = ~2.1 seconds

Total cost: Only 1 LLM call

=== BEDROCK AGENTCORE CODE INTERPRETER (AWS TOOL) ===

What it does: ├─ Agent can execute code (scratch pad) ├─ Code runs in sandbox (secure) ├─ Agent can use Python (data analysis) ├─ Agent can use SQL (database queries) ├─ Agent can use regex (text parsing) └─ All without calling LLM

Why use it: ├─ Reduces LLM calls (only use when needed) ├─ Faster execution (code is instant) ├─ Lower cost (code is free) ├─ More reliable (logic is deterministic, LLM is probabilistic) └─ Better control (you define rules, not LLM)

When to use: ├─ High-volume processing (millions of operations) ├─ Cost-sensitive work (you care about margins) ├─ Performance-critical (customer sees response time) ├─ Deterministic logic (if/then rules) └─ Data aggregation (combining sources)

---

Seu agente é lento? Checklist de otimização

5 perguntas para saber se você precisa de scratch pad.

=== OPTIMIZATION CHECKLIST ===

❓ QUESTION 1: Você chama LLM múltiplas vezes por request? ├─ Yes: "Primeiro LLM para entender intent, depois LLM para validar, depois LLM para gerar resposta" ├─ Problem: 3+ LLM calls = 3+ segundos + R$0.03+ por request ├─ Solution: Combine em 1 LLM call, use scratch pad para validar/gerar └─ Impact: 3x faster, 3x cheaper

❓ QUESTION 2: Seu agente faz database lookups ou validações? ├─ Yes: "Verifica se usuário existe, se está ativo, se tem permissão" ├─ Problem: Você provavelmente chama LLM para isso (WRONG) ├─ Solution: Use scratch pad (database query = 10ms) └─ Impact: 100x faster, 100% cheaper (zero LLM cost)

❓ QUESTION 3: Seu agente processa dados estruturados? ├─ Yes: "Transações, emails, tickets com campos específicos" ├─ Problem: LLM é ruim em dados estruturados (usa tokens, é lento) ├─ Solution: Use scratch pad (regex, parsing, aggregation) └─ Impact: 10x faster, 10x cheaper

❓ QUESTION 4: Seu agente precisa de "reasoning steps"? ├─ Yes: "Agente pensa passo a passo, gerando múltiplas análises" ├─ Problem: Cada passo é 1 LLM call (slow) ├─ Solution: Let LLM think once (with thinking mode), then use scratch pad for verification └─ Impact: 2x faster, 2x cheaper

❓ QUESTION 5: Você está perdendo para competitors em performance? ├─ Yes: "Competitor é mais rápido, mais barato" ├─ Problem: They probably use scratch pad, you don't ├─ Solution: Audit your agent, remove unnecessary LLM calls └─ Impact: Win back speed/cost advantage

=== YOUR ACTION PLAN (THIS WEEK) ===

[ ] Step 1: Audit current agent ├─ Log every LLM call ├─ Count: How many per request? (should be 1-2, not 10+) ├─ Measure: Response time per request (baseline) └─ Owner: You + engineering

[ ] Step 2: Identify scratch pad opportunities ├─ Which steps are database lookups? (move to scratch pad) ├─ Which steps are data validation? (move to scratch pad) ├─ Which steps are formatting? (move to scratch pad) ├─ Which steps NEED LLM? (keep only these) └─ Owner: Product + engineering

[ ] Step 3: Implement scratch pad ├─ Extract logic from LLM prompts ├─ Write code (regex, SQL, rules) ├─ Test locally (make sure it works) ├─ Deploy to production └─ Owner: Engineering

[ ] Step 4: Measure impact ├─ New response time (should be 2-3x faster) ├─ New cost per request (should be 2-3x cheaper) ├─ User satisfaction (faster responses = happier users) └─ Owner: You + product

=== TIMELINE & INVESTMENT ===

Total effort: 2-4 weeks (depending on agent complexity) Team: 1 engineer (50%), you (10%) Cost: ~R$20K-40K (engineering time) Benefit: 2-3x faster, 2-3x cheaper (immediately) ROI: Payback in 1 month (margin improvement)

=== REAL EXAMPLE: SCALING FROM 1M TO 10B OPERATIONS/DAY ===

Without scratch pad: ├─ 1M ops/day: R$10K/day (1 LLM call each, R$0.01/op) ├─ 10M ops/day: R$100K/day (same ratio) ├─ 100M ops/day: R$1M/day ├─ 1B ops/day: R$10M/day = R$300M/month ├─ 10B ops/day: R$100M/day = R$3B/month (NOT VIABLE) └─ Business dies (costs exceed revenue)

With scratch pad (1 LLM call + scratch pad): ├─ 1M ops/day: R$2K/day (80% cheaper) ├─ 10M ops/day: R$20K/day (80% cheaper) ├─ 100M ops/day: R$200K/day (80% cheaper) ├─ 1B ops/day: R$2M/day = R$60M/month (VIABLE) ├─ 10B ops/day: R$20M/day = R$600M/month (VIABLE, scales) └─ Business survives (costs scale linearly, not exponentially)

Conclusion: Without scratch pad, you can't scale beyond 1B ops/day. With scratch pad, you can handle 10B+ ops/day profitably.

---

Conclusão: Scratch pad é seu segredo competitivo (ou sua desvantagem)

A realidade (2025-2026):

• Agentes de IA estão em produção em escala (bilhões de operações/dia)
• Winners implementaram scratch pad (2-3x mais rápido, 2-3x mais barato)
• Losers ainda chamam LLM para tudo (lento, caro, não escala)
• Market está dividindo (optimized players vs non-optimized)
• Customers percebem diferença (velocidade, custo, reliability)

Seu cenário (escolha agora):

┌────────────────────────────────────────────┐ │ OPÇÃO A: Keep calling LLM for everything │ ├────────────────────────────────────────────┤ │ Speed: Slow (5+ seconds per request) │ │ Cost: Expensive (R$0.05+ per request) │ │ Scalability: Poor (breaks at 10M ops/day) │ │ Competitor: 2-3x faster, cheaper │ │ Outcome: Lose to optimized competitors │ └────────────────────────────────────────────┘

┌────────────────────────────────────────────┐ │ OPÇÃO B: Implement scratch pad NOW ✓ │ ├────────────────────────────────────────────┤ │ Speed: Fast (1-2 seconds per request) │ │ Cost: Cheap (R$0.01 per request) │ │ Scalability: Excellent (scales to 10B ops) │ │ Competitor: Can't compete on cost/speed │ │ Outcome: Win market share, grow margins │ └────────────────────────────────────────────┘

Na OpenClaw:

Ajudamos SaaS otimizar agentes com scratch pad:

• Performance audit: Quanto tempo seu agente leva? Onde gasta tempo?
• LLM call analysis: Quantas chamadas por request? Pode reduzir?
• Scratch pad design: Quais tarefas rodam em código vs LLM?
• Architecture optimization: Implementamos 3-layer model (input → LLM → action)
• Code Interpreter setup: Bedrock AgentCore ou equivalent
• Measurement & monitoring: Track speed, cost, quality (não vai piorar)
• Ongoing tuning: Continuous optimization (sempre melhorando)

Você quer seu agente 3x mais rápido e 3x mais barato sem perder qualidade?

Performance Audit | Scratch Pad Architecture | LLM Optimization →