Seu agente custa demais (tool calling é caro)

Seu agente custa demais (tool calling é caro)

Notícia: Desenvolvedor (Ted Haley) publicou insight: Local indexing reduz tool calling + token usage MASSIVAMENTE. Significado: Agentes que chamam tools continuamente (pesquisa, cálculo, validação) tão desperdiçando tokens (e dinheiro). Local indexing = estratégia pra reduzir chamadas (cache local, não API).

Problema: Seu agente, hoje:

CUSTOMER QUERY: "Qual é o melhor plano pra empresa com 50 pessoas?"

SEM LOCAL INDEXING (WASTEFUL): ├─ Agente recebe query ├─ Tool call 1: Fetch pricing plans (API → LLM process) ├─ Tool call 2: Fetch company size limits (API → LLM process) ├─ Tool call 3: Fetch feature comparison (API → LLM process) ├─ Tool call 4: Fetch customer reviews (API → LLM process) ├─ Tool call 5: Validate recommendation (LLM re-check) ├─ Total: 5 API calls + 5 LLM processes ├─ Tokens used: ~2,000 tokens (each call adds context) ├─ Cost: ~R$ 0.30 (R$ 0.06 per 1K tokens) ├─ Time: 3-5 seconds (API latency) └─ Problem: Expensive, slow, wasteful

COM LOCAL INDEXING (EFFICIENT): ├─ Setup: Index pricing plans locally (1-time, 10 min) ├─ Setup: Index features locally (1-time, 10 min) ├─ Setup: Index company limits locally (1-time, 10 min) ├─ Agente recebe query ├─ Local lookup: Search indexed plans (0 API calls, <1ms) ├─ Local lookup: Search indexed features (0 API calls, <1ms) ├─ Local lookup: Search indexed limits (0 API calls, <1ms) ├─ LLM once: Process results + recommend (1 LLM call, not 5) ├─ Total: 0 API calls (vs 5) + 1 LLM call (vs 5) ├─ Tokens used: ~200 tokens (no redundant context) ├─ Cost: ~R$ 0.03 (10x cheaper) ├─ Time: <500ms (local lookup is instant) └─ Problem: SOLVED (efficient, fast, cheap)

COST COMPARISON: ├─ Scenario: 1,000 queries per day ├─ Without indexing: R$ 0.30 × 1,000 = R$ 300/dia = R$ 9,000/mês ├─ With indexing: R$ 0.03 × 1,000 = R$ 30/dia = R$ 900/mês ├─ Monthly savings: R$ 8,100 (90% reduction) ├─ Annual savings: R$ 97,200 (!!) └─ Investment: 30 min setup + maintenance

Implicação: Você tá gastando 10x mais em API calls do que precisa. Local indexing = solução simples (2 horas setup, forever savings).

---

Por que agentes chamam tools demais

1. Default behavior é chamar tool (não pensar localmente)

AGENT DESIGN LOGIC (NAIVE): ├─ When question asks for data: │ ├─ "What's the customer's balance?" │ ├─ Agent thinks: "I don't know balance → call tool" │ ├─ Tool call: API to billing system (R$ 0.01) │ ├─ Process result (tokens) │ ├─ Answer: "Balance is R$ 5,000" │ └─ Cost: R$ 0.01 + tokens │ ├─ Problem: Agent calls tool EVERY TIME │ ├─ Same customer queries balance 10x = 10 tool calls │ ├─ Tool data doesn't change between calls (unnecessary) │ ├─ Cost multiplies (10 × R$ 0.01 = R$ 0.10 per customer) │ └─ Scale 10,000 customers → R$ 1,000/day wasted │ └─ Better approach: Use indexed LOCAL data ├─ Index customer balances locally (one-time) ├─ Query local index (free, instant) ├─ Only update index periodically (e.g., hourly) ├─ Cost: 1 API call/hour (vs 100 calls/hour) └─ Savings: 99% reduction

REALITY: ├─ Most agents are designed: "Don't have data? Call API" ├─ Problem: No consideration for "Do I REALLY need fresh data?" ├─ Answer: Usually NO (data is stable) ├─ Result: Waste on unnecessary API calls

2. LLM context is expensive (each tool call adds tokens)

TOKEN EXPLOSION WITH TOOL CALLING: ├─ Each tool call adds context window: │ ├─ Tool request: "Fetch customer data" (50 tokens) │ ├─ Tool response: Customer data (500 tokens) │ ├─ LLM process: Analyze data (200 tokens) │ ├─ Total per call: 750 tokens │ └─ 5 tool calls = 3,750 tokens (expensive) │ ├─ With local indexing: │ ├─ No tool requests (0 tokens) │ ├─ No tool responses (0 tokens) │ ├─ Just LLM processing: Analyze indexed data (200 tokens) │ ├─ Total: 200 tokens (5x cheaper) │ └─ Cost: 5x reduction from fewer tool calls │ ├─ Cost scaling: │ ├─ Claude (example): R$ 0.003 per 1K input tokens │ ├─ Without indexing: 3,750 tokens × R$ 0.003/1K = R$ 0.01125 │ ├─ With indexing: 200 tokens × R$ 0.003/1K = R$ 0.0006 │ ├─ Savings per query: R$ 0.0105 (10x cheaper) │ └─ Scale 10,000 queries: R$ 105 saved │ └─ Hidden cost: Context bloat ├─ More tool calls = bigger context ├─ Bigger context = slower LLM (latency increases) ├─ Slower LLM = poor user experience (multi-second waits) ├─ Local indexing = minimal context = fast response └─ Speed is feature (customers happy)

3. Tool calling creates a dependency loop

DEPENDENCY LOOP (WASTEFUL): ├─ Agent needs data ├─ Agent calls tool ├─ Tool returns data ├─ Agent thinks: "Is this right?" (uncertainty) ├─ Agent calls tool again (validation) ├─ Tool confirms: "Yes, same data" ├─ Agent is now confident ├─ But: 2 tool calls for 1 query (wasteful) │ └─ Multiply this: ├─ 50% of agent queries have doubt → 2 tool calls ├─ 1,000 queries/day ├─ 500 queries × 2 calls = 1,000 tool calls (vs needed 500) ├─ 500 extra calls = wasted money └─ Solution: Local indexing removes doubt (data is indexed, validated)

WHY LOCAL INDEXING FIXES IT: ├─ Indexed data = pre-validated (source of truth) ├─ Local lookup = instant confidence (no API call needed) ├─ No doubt = no re-validation = 1 call instead of 2 ├─ Cost: 50% reduction (from eliminating validation calls) └─ Result: Faster, cheaper, more confident

---

Como implementar local indexing (3 estratégias)

Strategy 1: Static data indexing (pricing, features, limits)

USE CASE: Agente de vendas (customer asks "Qual é melhor plano?")

BEFORE (sem indexing): ├─ Customer: "Qual plano pra empresa com 50 pessoas?" ├─ Agente: "Preciso consultar planos" ├─ Tool call: GET /api/pricing-plans (API → return 20 plans) ├─ Tool call: GET /api/plan-limits (API → return limits) ├─ Tool call: GET /api/features (API → return 100+ features) ├─ LLM: Analyze (context bloat, slow) ├─ Response: "Recomendo Plano Pro" (after 3 seconds) ├─ Cost: R$ 0.30 └─ Problem: Slow, expensive, unnecessary (data never changes)

AFTER (com indexing): ├─ SETUP (once, 30 minutes): │ ├─ Step 1: Fetch all pricing plans (1 API call) │ ├─ Step 2: Fetch all plan limits (1 API call) │ ├─ Step 3: Fetch all features (1 API call) │ ├─ Step 4: Index locally (create JSON file or vector DB) │ ├─ Step 5: Embed in agente (load on startup) │ └─ Total setup: 3 API calls (one-time) │ ├─ RUNTIME (every query): │ ├─ Customer: "Qual plano pra empresa com 50 pessoas?" │ ├─ Agente: Search local index (no API call) │ ├─ Agente: Find matching plans (instant, <10ms) │ ├─ Agente: Build context from index (minimal tokens) │ ├─ LLM: Analyze (fast, few tokens) │ ├─ Response: "Recomendo Plano Pro" (500ms) │ ├─ Cost: R$ 0.03 (10x cheaper) │ └─ Problem: SOLVED │ ├─ MAINTENANCE (weekly): │ ├─ When pricing changes: Re-index (5 min job) │ ├─ Automated: Schedule nightly re-index │ ├─ Cost: 3 API calls/night (negligible) │ └─ Benefit: Agente always has fresh data │ └─ CODE EXAMPLE: python # Setup (one-time) def setup_indexing(): pricing_plans = fetch_from_api("/api/pricing-plans") features = fetch_from_api("/api/features") limits = fetch_from_api("/api/plan-limits")

     # Create local index (JSON or vector DB)
     index = {
         "plans": pricing_plans,
         "features": features,
         "limits": limits
     }
     
     # Save to file
     save_to_file("pricing_index.json", index)
 
 # Runtime (every query)
 def handle_query(query, index):
     # Search local index (no API call)
     matching_plans = search_index(query, index)
     
     # Build context
     context = json.dumps(matching_plans)
     
     # Call LLM once (with minimal context)
     response = llm.generate(
         prompt=f"Query: {query}\nContext: {context}"
     )
     
     return response
 

BENEFITS: ├─ Cost: 90% reduction (3 setup calls vs 3 per query) ├─ Speed: 6x faster (local lookup vs API) ├─ Quality: Better (full context, no truncation) ├─ Maintenance: Easy (automated re-indexing) └─ Scalability: Unlimited (local lookup is O(1))

Strategy 2: Temporal indexing (cache recent data)

USE CASE: Agente de suporte (customer asks "Como tá meu ticket?")

BEFORE (sem indexing): ├─ Customer: "Como tá meu ticket #12345?" ├─ Agente: Call /api/tickets/12345 (API call every time) ├─ Same customer asks 5x per day → 5 API calls ├─ Cost: 5 × R$ 0.01 = R$ 0.05 (unnecessary) └─ Better: Use cached/indexed version

AFTER (temporal indexing): ├─ STRATEGY: Cache recent tickets (last 24 hours) │ ├─ Every 5 minutes: Fetch updated tickets (batch API call) │ ├─ Index locally: Store tickets in memory/cache │ ├─ Serve from cache: No API calls (unless data stale) │ ├─ Fallback: If >24h old, fetch fresh │ └─ Cost: 1 API call every 5 min (vs 1 call per query) │ ├─ MATH: │ ├─ Queries per day: 1,000 │ ├─ Batch refreshes: 288 (one every 5 min) │ ├─ Without indexing: 1,000 API calls │ ├─ With indexing: 288 API calls (70% reduction) │ ├─ Cost savings: R$ 7.12/day = R$ 214/month │ └─ ROI: 5 minute setup → R$ 214/month benefit │ └─ CODE EXAMPLE: python import time from datetime import datetime, timedelta

 class TicketCache:
     def __init__(self):
         self.cache = {}
         self.last_refresh = None
         self.refresh_interval = 300  # 5 minutes
     
     def get_ticket(self, ticket_id):
         # Refresh if stale
         if self.should_refresh():
             self.refresh_cache()
         
         # Serve from cache (no API call)
         if ticket_id in self.cache:
             return self.cache[ticket_id]
         
         # Fallback: fetch fresh (rare)
         return fetch_from_api(f"/api/tickets/{ticket_id}")
     
     def should_refresh(self):
         if self.last_refresh is None:
             return True
         return (time.time() - self.last_refresh) > self.refresh_interval
     
     def refresh_cache(self):
         # Batch fetch all recent tickets
         recent_tickets = fetch_from_api("/api/tickets?recent=24h")
         self.cache = {t['id']: t for t in recent_tickets}
         self.last_refresh = time.time()
 

BENEFITS: ├─ Cost: 70% reduction (batch > per-query calls) ├─ Speed: Instant (cache lookup vs API) ├─ Freshness: Control tradeoff (5 min cache = fresh enough) ├─ Scalability: Handle spikes (cache absorbs traffic) └─ Simplicity: Easy to implement (in-memory dict)

Strategy 3: Vector indexing (semantic search without API)

USE CASE: Agente de suporte (customer describes problem, agente finds solution)

BEFORE (sem indexing): ├─ Customer: "My payments keep failing when I use my debit card" ├─ Agente: "Let me search our knowledge base..." ├─ Tool call: POST /api/search {query, embedding model call inside} ├─ Tool: Generate embedding of customer query (LLM call) ├─ Tool: Search vector DB (API call) ├─ Tool: Return top 3 articles ├─ LLM: Analyze articles + generate response ├─ Total: 2-3 API calls + embedding LLM call ├─ Cost: R$ 0.15 ├─ Time: 2-3 seconds └─ Problem: Expensive, slow

AFTER (vector indexing): ├─ SETUP (one-time, 1 hour): │ ├─ Fetch all help articles (API) │ ├─ Generate embeddings (local, using open model like MiniLM) │ ├─ Index vectors locally (Chroma, Faiss, Weaviate) │ └─ Load index on agente startup │ ├─ RUNTIME (every query): │ ├─ Customer: "My payments keep failing..." │ ├─ Agente: Generate embedding (local, no API) │ ├─ Agente: Search local vector DB (instant, no API) │ ├─ Agente: Find 3 most relevant articles (0 API calls) │ ├─ LLM: Generate response │ ├─ Total: 0 API calls (vs 2-3) │ ├─ Cost: R$ 0.05 (3x cheaper) │ ├─ Time: <500ms (vs 2-3s) │ └─ Result: Better + cheaper + faster │ └─ CODE EXAMPLE: python from sentence_transformers import SentenceTransformer import chromadb

 # Setup (one-time)
 def setup_vector_index():
     # Load embedding model (local, free)
     model = SentenceTransformer('all-MiniLM-L6-v2')
     
     # Fetch articles
     articles = fetch_from_api("/api/articles")
     
     # Generate embeddings (local)
     embeddings = [model.encode(a['content']) for a in articles]
     
     # Create vector DB
     client = chromadb.Client()
     collection = client.create_collection("articles")
     
     for i, (article, embedding) in enumerate(zip(articles, embeddings)):
         collection.add(
             ids=[str(i)],
             embeddings=[embedding],
             metadatas=[{"title": article['title'], "url": article['url']}]
         )
     
     return collection
 
 # Runtime (every query)
 def search_articles(query, collection, model):
     # Generate embedding (local, no API)
     query_embedding = model.encode(query)
     
     # Search vector DB (local, instant)
     results = collection.query(
         query_embeddings=[query_embedding],
         n_results=3
     )
     
     return results
 

BENEFITS: ├─ Cost: 3x reduction (no API calls for search) ├─ Speed: 4x faster (local vs API) ├─ Quality: Better matching (semantic vs keyword) ├─ Privacy: Data stays local (no external API) ├─ Scale: Handle unlimited searches (local) └─ Model: Use free embedding model (no licensing)

---

Comparação: Local indexing vs always calling tools

┌──────────────────┬──────────────────┬──────────────────┐ │ Métrica │ Always Tool Call │ Local Indexing │ ├──────────────────┼──────────────────┼──────────────────┤ │ Cost per query │ R$ 0.30 │ R$ 0.03 │ │ Queries per day │ 1,000 │ 1,000 │ │ Daily cost │ R$ 300 │ R$ 30 │ │ Monthly cost │ R$ 9,000 │ R$ 900 │ │ Annual cost │ R$ 108,000 │ R$ 10,800 │ │ SAVINGS │ - │ R$ 97,200/year │ │ │ │ │ │ Latency per call │ 2-5 seconds │ <500ms │ │ Total daily time │ 2,000-5,000s │ <500s │ │ Speed gain │ - │ 4-10x faster │ │ │ │ │ │ Setup time │ 0 (built-in) │ 1-2 hours │ │ Maintenance │ None │ Periodic refresh │ │ Complexity │ Low │ Medium │ │ │ │ │ │ Scalability │ API-dependent │ Local (unlimited)│ │ Freshness │ Real-time │ Configurable │ │ Privacy │ API exposure │ Local data │ └──────────────────┴──────────────────┴──────────────────┘

BOTTOM LINE: ├─ Cost: 90% savings (10x reduction) ├─ Speed: 4-10x faster ├─ Quality: Better (more context, no truncation) ├─ Investment: 1-2 hours setup (breaks even in days) ├─ ROI: R$ 97,000/year (for 1,000 queries/day) └─ Question: Por que você não tá usando isso agora?

---

Conclusão: Local indexing é no-brainer

Fatos:

✓ Agentes chamam tools demais (waste) ✓ Tool calling adiciona tokens (exponential cost) ✓ Muitos dados são static (indexable) ✓ Local indexing reduz calls 5-10x (massive savings) ✓ Setup é 1-2 horas (quick ROI) ✓ Velocidade melhora 4-10x (better UX) ✓ Privacidade melhora (data local) ✓ Scalabilidade melhor (no API limits) ✓ Break-even: ~1 semana (savings > setup cost) ✓ Message: "Local indexing é free lunch"

ACTION ITEMS (THIS WEEK):

1. TODAY: Audit agente (quais tools chama?)
2. TODAY: Identify static data (pricing, features, limits)
3. THIS WEEK: Estimate cost (tool calls × R$ 0.01)
4. THIS WEEK: Calculate potential savings
5. THIS WEEK: Prototype indexing (30 min experiment)
6. NEXT WEEK: Implement 1 strategy (highest ROI first)
7. NEXT WEEK: Measure impact (latency, cost)
8. WEEK 3: Expand to other use cases
9. WEEK 3: Monitor + maintain
10. MONTH 2: Optimize indexing strategy

Problema resolvido quando: └─ Agente chama tools MENOS (not reflexively) └─ Static data é indexed (não refetched) └─ Costs reduzem 5-10x (visible savings) └─ Latency melhora (users happy) └─ Maintenance é automated (no overhead) └─ Result: Agente 10x mais eficiente + barato

→ OpenClaw: Agente Eficiente (Local Indexing Strategy)

Seu agente tá desperdiçando dinheiro (tool calling excessivo). Local indexing = solução simples (1-2h setup, 90% savings). Implementar agora = R$ 97k/ano economizado. 💰⚡