Seu agent é lento (mas você não sabe por quê)

Seu agent é lento (mas você não sabe por quê).

Você é founder de SaaS.

Você tem agent.

Agent funciona.

But customers complain:

Customer: "Your agent takes 5 seconds to respond" You: "That's acceptable (LLM latency)" Customer: "ChatGPT responds in 1 second" You: "Hmm... let me check" │ You check: ├─ Agent logic: Looks fine ├─ LLM call: Using Claude (should be fast) ├─ Database query: Looks fine ├─ API call: Looks fine ├─ Network: Looks fine │ You: "I don't know why it's slow. Everything looks fine." Customer: "Well, it's slow. I'm switching to ChatGPT." │ === THE PROBLEM === │ You have no idea where the 5 seconds go: ├─ 0.5 seconds: API request overhead (network round-trip) ├─ 1.0 seconds: Database query (fetch customer context) ├─ 2.5 seconds: LLM inference (Claude thinking) ├─ 1.0 seconds: Response formatting ├─ Total: 5.0 seconds │ But you don't know this breakdown (no measurement). So you can't optimize (you don't know what's slow). │ === THE SOLUTION === │ Yesterday, you read Claude blog: "Once you can measure something, you can make it faster." │ Key insight: Anthropic optimized Claude by measuring: ├─ Token throughput (tokens per second) ├─ Latency (time to first token) ├─ Inference time (total request time) ├─ Cache hit rate (reusing computed tokens) ├─ Memory usage (how much RAM does inference use?) │ Result: Claude got 2x faster (not by better algorithms, but by measurement + targeted optimization). │ Your lesson: You need to measure agent latency (same way Anthropic measures Claude). │

Yesterday, you read:

Claude blog: "Once Claude can measure something, it can make it faster."

Key insight: "Anthropic optimized Claude by measuring performance bottlenecks (token throughput, latency, inference time, cache efficiency). Each measurement revealed optimization opportunity. Example: 'Cache hit rate was 30%. By optimizing cache, we got to 60%. Result: 2x faster inference.'"

Translation: Measurement is prerequisite for optimization. If you can't measure agent latency, you can't optimize it.

What this means (for your agent business):

=== THE MEASUREMENT GAP === │ Claude (Anthropic's approach): ├─ Measure: Token throughput (tokens/second) ├─ Measure: Time to first token (latency) ├─ Measure: Total inference time ├─ Measure: Cache hit rate ├─ Result: Know exactly where time is spent ├─ Optimize: Target highest-impact bottleneck ├─ Measure again: Verify improvement │ Your agent (probably): ├─ No measurement ("Is it slow? I guess...") ├─ No breakdown ("Where does the time go? I don't know...") ├─ No optimization ("How do I make it faster? No idea...") ├─ Result: Customers complain about latency, you can't fix it │ === THE COST OF NOT MEASURING === │ Scenario: Customer complains agent is slow │ Your approach (no measurement): ├─ Check code: "Looks fine" ├─ Check logs: "No errors" ├─ Guess: "Must be LLM latency (not our problem)" ├─ Do nothing ├─ Customer: "Still slow. I'm leaving." ├─ Churn: 1 customer lost │ Claude's approach (with measurement): ├─ Measure: Agent takes 5 seconds ├─ Breakdown: 0.5s API overhead, 1s DB query, 2.5s LLM, 1s formatting ├─ Find: DB query is slow (1 second, not LLM) ├─ Optimize: Add index to database table ├─ Measure: Agent now takes 3.5 seconds (1.5 seconds saved) ├─ Customer: "Wow, much faster!" ├─ Retention: Customer stays, upgrades plan │ === THE OPPORTUNITY === │ You measure latency → find bottleneck (usually not LLM): ├─ API overhead: Add connection pooling (save 0.5s) ├─ Database: Add index or cache (save 1s) ├─ LLM: Use faster model or prompt caching (save 0.5s) ├─ Formatting: Optimize parsing (save 0.2s) ├─ Total: Save 2+ seconds (40% latency reduction) │ Result: ├─ Agent responds in 2.5 seconds (instead of 5) ├─ Customer satisfaction: Much higher ├─ Competitive position: Now beats ChatGPT (faster response) ├─ Churn: Reduced (customers stay) ├─ NPS: Increased (customers recommend you) │

---

How to measure agent performance (step by step)

Layer 1: Identify what to measure

=== THE FOUR METRICS === │ Metric 1: Latency (time from request to response) ├─ What: How long does agent take to respond? ├─ Measure: Time when customer sends query → time when agent responds ├─ Goal: < 2 seconds (competitive vs ChatGPT) ├─ Why: Customer impatience (if > 3 seconds, feels slow) │ Metric 2: Throughput (requests per second) ├─ What: How many customers can agent serve simultaneously? ├─ Measure: Requests per second (during peak hours) ├─ Goal: 100+ RPS (enterprise SaaS standard) ├─ Why: Scalability (can you handle growth?) │ Metric 3: Error rate (% of requests that fail) ├─ What: How often does agent fail (timeout, crash, wrong answer)? ├─ Measure: Failed requests / total requests ├─ Goal: < 0.1% (enterprise SaaS standard) ├─ Why: Reliability (customer trust) │ Metric 4: Cost per request (how much does one inference cost?) ├─ What: How much do you spend on LLM tokens per customer request? ├─ Measure: LLM cost / total requests ├─ Goal: < R$0.10 per request (profitable SaaS) ├─ Why: Unit economics (can you make money?) │ === THE BREAKDOWN METRICS === │ Once you have latency (5 seconds), measure where time goes: ├─ API overhead: Time from request received → LLM call sent │ ├─ Network round-trip: 0.1-0.5 seconds │ ├─ Request parsing: 0.05 seconds │ ├─ Auth check: 0.1 seconds │ ├─ Database lookup: 0.5-2 seconds (usually the culprit) │ ├─ Prompt construction: 0.1 seconds │ ├─ LLM inference: Time for Claude to think │ ├─ Token generation: 1-3 seconds (depends on response length) │ ├─ Cache hit/miss: 0.5-2 seconds difference (big impact!) │ ├─ Post-processing: Time after LLM returns │ ├─ Response parsing: 0.05-0.1 seconds │ ├─ Database save: 0.5-1 seconds (sometimes) │ ├─ Response formatting: 0.1 seconds │ ├─ Network response: 0.1-0.5 seconds │ === MEASUREMENT TOOLS === │ Tool 1: Custom logging (DIY) ├─ Add timestamp at each step ├─ Log latency for each phase ├─ Example: │ Start: 2026-09-24 14:00:00.000 │ After auth: 2026-09-24 14:00:00.050 (50ms) │ After DB query: 2026-09-24 14:00:01.500 (1.45s) │ After LLM: 2026-09-24 14:00:03.500 (2.0s) │ Response sent: 2026-09-24 14:00:03.600 (0.1s) │ Total: 3.6s │ Tool 2: APM (Application Performance Monitoring) ├─ Services: Datadog, New Relic, Elastic APM ├─ Benefit: Automatic breakdown, dashboards, alerts ├─ Cost: R$500-5000/month (depends on traffic) ├─ Recommendation: Use if you have 1000+ RPS │ Tool 3: Distributed tracing (OpenTelemetry) ├─ Concept: Trace request through all services ├─ Tools: Jaeger, Zipkin, AWS X-Ray ├─ Benefit: See exactly where time is spent (across services) ├─ Cost: DIY (open-source) or managed (R$100-1000/month) ├─ Recommendation: Use if your agent calls multiple APIs │ Tool 4: LLM-specific metrics ├─ Track token generation rate (tokens/second) ├─ Track cache hit rate (% of requests using cached tokens) ├─ Track prompt size (input tokens) vs response size (output tokens) ├─ Tool: Claude API returns these metrics in response ├─ Recommendation: Always log these │

Layer 2: Set up measurement infrastructure

=== STEP 1: ADD LOGGING === │ Pseudocode: ├─ start_time = now() ├─ auth_start = now() ├─ auth_result = check_api_key() ├─ auth_time = now() - auth_start │ ├─ db_start = now() ├─ customer_context = fetch_from_db(customer_id) ├─ db_time = now() - db_start │ ├─ llm_start = now() ├─ llm_response = call_claude(customer_context) ├─ llm_time = now() - llm_start │ ├─ format_start = now() ├─ final_response = format_response(llm_response) ├─ format_time = now() - format_start │ ├─ total_time = now() - start_time ├─ log("Latency breakdown: auth={auth_time}ms, db={db_time}ms, llm={llm_time}ms, format={format_time}ms, total={total_time}ms") │ === STEP 2: COLLECT METRICS === │ Each request, log: ├─ Timestamp ├─ Customer ID ├─ Request size (input tokens) ├─ Response size (output tokens) ├─ Latency (total time) ├─ Latency breakdown (auth, DB, LLM, format) ├─ Error (if any) ├─ Cost (LLM tokens used) ├─ Cache hit (if using prompt caching) │ === STEP 3: AGGREGATE METRICS === │ Daily report: ├─ Median latency (50th percentile) ├─ P95 latency (95th percentile, "slow" requests) ├─ P99 latency (99th percentile, "very slow" requests) ├─ Average latency (mean) ├─ Error rate ├─ Throughput (requests/second) ├─ Total cost (sum of LLM tokens) ├─ Cache hit rate │ === STEP 4: VISUALIZE METRICS === │ Create dashboard: ├─ Graph 1: Latency over time (should be flat, not increasing) ├─ Graph 2: Latency breakdown (pie chart: auth%, DB%, LLM%, format%) ├─ Graph 3: P95/P99 latency (are slowdowns happening?) ├─ Graph 4: Error rate (any spikes?) ├─ Graph 5: Throughput (scaling up or down?) ├─ Graph 6: Cost per request (profitable?) │ Update: Daily or weekly (not real-time, but frequent enough to catch issues) │

Layer 3: Optimize based on measurements

=== SCENARIO 1: DATABASE IS SLOW === │ Measurement shows: ├─ Total latency: 5 seconds ├─ DB query time: 2 seconds (40% of total) ├─ LLM time: 1.5 seconds (30% of total) ├─ Other: 1.5 seconds (30% of total) │ Optimization: ├─ Add database index (on customer_id, created_at) ├─ Result: DB query drops to 0.5 seconds ├─ New total latency: 3.5 seconds (30% improvement) │ Verify: ├─ Measure again after optimization ├─ Confirm DB query is now 0.5s (not 2s) ├─ Confirm total latency is 3.5s (not 5s) │ === SCENARIO 2: LLM IS SLOW === │ Measurement shows: ├─ Total latency: 5 seconds ├─ LLM time: 3 seconds (60% of total) ├─ Other: 2 seconds (40% of total) │ Optimizations: ├─ Option A: Use faster LLM model (Claude Haiku instead of Opus) │ ├─ Result: LLM time drops to 1 second (2 seconds saved) │ ├─ Trade-off: Lower quality responses │ ├─ Option B: Use prompt caching (cache system prompt + context) │ ├─ Result: Cache hit saves 1-2 seconds per request │ ├─ Trade-off: Setup required, only works for repeated queries │ ├─ Option C: Use streaming (send tokens as they arrive) │ ├─ Result: User sees first token in 0.5 seconds (feels faster) │ ├─ Trade-off: User experiences streaming (not instant answer) │ === SCENARIO 3: API OVERHEAD IS HIGH === │ Measurement shows: ├─ Total latency: 5 seconds ├─ Auth check: 0.5 seconds ├─ Request parsing: 0.5 seconds ├─ Database lookup: 1 second ├─ LLM: 2 seconds ├─ Response formatting: 0.5 seconds │ Optimizations: ├─ Move auth to cache (check auth once, cache for 5 minutes) │ ├─ Result: Auth drops to 0.05 seconds │ ├─ Saving: 0.45 seconds │ ├─ Batch database queries (fetch multiple records in one query) │ ├─ Result: DB drops to 0.3 seconds │ ├─ Saving: 0.7 seconds │ ├─ Use background jobs for response formatting (do it async) │ ├─ Result: Formatting drops to 0.05 seconds (user doesn't wait) │ ├─ Saving: 0.45 seconds │ ├─ Total savings: 1.6 seconds (32% improvement) │

---

The measurement mindset (what Claude teaches us)

Why measurement matters more than optimization

=== THE PRINCIPLE === │ Claude's lesson: "Once you can measure something, you can make it faster." │ This implies: ├─ If you can't measure it, you can't optimize it ├─ Optimization without measurement is guessing ├─ Measurement reveals the real bottleneck (usually not what you think) ├─ Once you measure, optimization is obvious (target the bottleneck) │ === EXAMPLE === │ Guess-based optimization: ├─ You think: "LLM is probably slow" ├─ You spend 2 weeks optimizing LLM calls ├─ You try faster model, prompt caching, etc ├─ Result: Agent is still 5 seconds (because DB was the real problem) ├─ Time wasted: 2 weeks of engineering │ Measurement-based optimization: ├─ You measure: 5 seconds total ├─ Breakdown: 2 seconds DB, 1.5 seconds LLM, 1.5 seconds other ├─ You identify: DB is bottleneck (40% of total time) ├─ You optimize: Add database index (1 day work) ├─ Result: Agent drops to 3.5 seconds (30% improvement) ├─ Time spent: 1 day (vs 2 weeks guessing) ├─ ROI: 14x better │ === THE MINDSET SHIFT === │ Before (no measurement): ├─ Problem: Agent is slow ├─ Action: Try random optimizations ├─ Result: Maybe it works, maybe not ├─ Feeling: Frustrated (why doesn't optimization work?) │ After (with measurement): ├─ Problem: Agent is slow ├─ Action: Measure to find bottleneck ├─ Result: Always works (you target the real problem) ├─ Feeling: Confident (you know what to fix) │

Real-world example (support agent)

=== YOUR SUPPORT AGENT (BEFORE MEASUREMENT) === │ Complaint: "Agent takes 3 seconds to respond" │ You try: ├─ Optimize LLM prompt (no improvement) ├─ Switch to faster model (slight improvement, but worse quality) ├─ Add caching (complicated, minimal improvement) ├─ Give up ("LLM latency is just slow, customer will deal with it") │ === YOUR SUPPORT AGENT (AFTER MEASUREMENT) === │ You measure (3 seconds total): ├─ Auth check: 0.1 seconds ├─ Fetch customer context from DB: 1.5 seconds (50% of total!) ├─ LLM inference: 1 second ├─ Format response: 0.4 seconds │ You identify: Database is the bottleneck (1.5 seconds) │ You optimize: ├─ Customer context table is missing index on customer_id ├─ Add index (takes 1 hour) ├─ Fetch drops from 1.5 seconds to 0.3 seconds │ Result: ├─ Total latency: 3 seconds → 1.8 seconds (40% improvement) ├─ Customer satisfaction: Much higher ├─ Cost: 1 hour of engineering │

---

Conclusão

Simple verdade:

Claude proved that measurement is prerequisite for optimization. If you can't measure agent latency, you can't optimize it. Most founders don't measure (they guess). You measure (you win).

3 facts:

1. Measurement reveals the real bottleneck (usually not what you think). You probably think your agent is slow because of LLM latency (you can't control). Measurement shows the real problem is database (you can control). Most optimization effort targets the wrong thing (because no measurement).
2. Optimization without measurement is waste (guessing takes 2 weeks, measurement takes 1 hour). Claude proved this: by measuring, they found the real bottleneck and optimized it. You're wasting engineering time guessing.
3. Once you measure, optimization is obvious (target highest-impact bottleneck first). Database is 50% of latency? Add index. API overhead is 30%? Add caching. LLM is 20%? Switch model. Measurement tells you exactly where to invest effort.

3 action items (this week):

1. Add latency logging to your agent (Every request, log: auth_time, db_time, llm_time, format_time, total_time. Takes 2-4 hours.)
2. Collect one week of data (Let data accumulate. See breakdown: where does time go? Usually: DB > LLM > API overhead > formatting.)
3. Optimize highest-impact bottleneck (If DB is 50% of latency, add index. If API overhead is 30%, add caching. If LLM is 20%, switch model. Target #1 first.)

The cost of waiting:

• Your agent is slow (3-5 seconds)
• You don't measure (you guess)
• You optimize wrong thing (LLM, not DB)
• Nothing improves (you waste 2 weeks)
• Customers complain (still slow)
• Customers leave (switch to ChatGPT)

The benefit of acting now:

• Your agent is slow (3-5 seconds)
• You measure (find DB is bottleneck in 1 hour)
• You optimize DB (add index, 1 hour work)
• Latency drops 30-40% (1.8-3.5 seconds)
• Customers happy (fast enough)
• Customers stay (competitive)

---

Próximos passos

Na OpenClaw, ajudamos SaaS builders measure and optimize agent performance:

• Latency Audit: Quanto tempo seu agent leva? (measurement)
• Bottleneck Analysis: Aonde o tempo vai? (breakdown)
• Database Optimization: DB é o culpado? (indexing, caching, queries)
• LLM Optimization: LLM é lento? (model selection, prompt caching, streaming)
• API Overhead: Autenticação, parsing, networking? (caching, batching)
• Monitoring Setup: Como medir continuamente? (logging, dashboards, alerts)
• Performance Baseline: Qual é latência aceitável? (benchmarking vs competitors)
• Optimization Roadmap: Qual otimização primeiro? (ROI prioritization)
• Observability Stack: Datadog, New Relic, ou DIY? (tool selection)
• Continuous Improvement: Como manter agent rápido? (ongoing measurement)

Agent Performance Optimization | Latency Measurement | Database Tuning | Claude Optimization Patterns →