Agent seu custa R$ 10/request (margin problem)

Agent seu custa R$ 10/request (margin problem)

Cenário assustador (que está destruindo SaaS):

Você é founder de SaaS com agent (atendimento, vendas, automação).

Modelo de negócio:

Customer pays: R$ 5/request Your cost: ?

Você não sabe.

Agent executa:

1. Takes user query
2. Calls LLM (OpenAI, Claude, etc)
3. Processes response
4. Returns result

Hidden cost: TOKENS

Tokens = words/subwords that LLM processes.

Example:

User query: "Compare prices for flights São Paulo to Rio." Tokens: ~15

LLM processing: Tokens: ~500

LLM response: Tokens: ~300

Total: ~815 tokens per request

Token cost (OpenAI pricing, Oct 2026):

Input: R$ 0.005 per 1000 tokens Output: R$ 0.015 per 1000 tokens

Your cost per request: Input (815 tokens): R$ 0.004 Output (300 tokens): R$ 0.0045 Total: ~R$ 0.009 per request

But wait: that's just the LLM. Add infrastructure:

• Server (compute): R$ 0.002
• Storage: R$ 0.001
• Database: R$ 0.001
• Bandwidth: R$ 0.001

Total cost per request: ~R$ 0.014 (almost R$ 0.01)

Your pricing:

Customer pays: R$ 5/request Your cost: R$ 0.014 Margin: R$ 4.986 (99.7%!)

Wait... that's amazing. What's the problem?

Problem: Your LLM is inefficient.

---

The real problem: token bloat

What's actually happening

Real-world SaaS agent (typical):

User query: "Get customer order history."

LLM processing:

1. Read context (customer info): 500 tokens
2. Analyze query: 200 tokens
3. Generate thoughts: 300 tokens
4. Make decision: 100 tokens
5. Call function: 50 tokens
6. Process response: 200 tokens
7. Format output: 100 tokens

Total: ~1450 tokens (not 815)

Cost per request: ~R$ 0.02 (not R$ 0.014)

Worse: chain-of-thought

If you use chain-of-thought (agent thinks step-by-step): LLM: "Let me think..." Tokens generated: 500+ (just for thinking) User sees: none (it's internal) Cost: R$ 0.01+ (wasted)

Example: Agent query: "Find best product for customer." Agent thinking: - "First, I'll analyze customer profile" (100 tokens) - "Then search inventory" (100 tokens) - "Then compare features" (100 tokens) - "Then make recommendation" (100 tokens) Total thinking: 400 tokens (wasted) Cost: R$ 0.006 (thrown away)

Scale: your margin disappears

Scenario: Your SaaS scales to 10k requests/day

Best case (efficient agent): Tokens/request: 815 Cost/request: R$ 0.009 Daily cost: 10k × R$ 0.009 = R$ 90 Daily revenue: 10k × R$ 5 = R$ 50k Margin: 99.8%

Reality (typical agent): Tokens/request: 2000 (bloated) Cost/request: R$ 0.02 Daily cost: 10k × R$ 0.02 = R$ 200 Daily revenue: 10k × R$ 5 = R$ 50k Margin: 99.6%

Worse case (chain-of-thought heavy): Tokens/request: 5000 (thinking!) Cost/request: R$ 0.05 Daily cost: 10k × R$ 0.05 = R$ 500 Daily revenue: 10k × R$ 5 = R$ 50k Margin: 99%

Worst case (multiple LLM calls): Tokens/request: 10000 (multiple calls, retries) Cost/request: R$ 0.10 Daily cost: 10k × R$ 0.10 = R$ 1000 Daily revenue: 10k × R$ 5 = R$ 50k Margin: 98%

Then: API price increase OpenAI: "Prices up 20%." Your cost/request: R$ 0.12 Daily cost: R$ 1200 Margin: 97.6%

Then: Scale to 100k requests/day (success!) Daily cost: R$ 12k Daily revenue: R$ 500k Margin: 97.6% (still OK)

But: if tokens are 2-3x worse (inefficient) Daily cost: R$ 24-36k Daily revenue: R$ 500k Margin: 92-93% (worse)

And: if you have even 5 agents running in parallel (retries, fallbacks) Daily cost: R$ 60k+ Margin: 88% (bad)

Then: customer churn (competitor cheaper) You cut price: R$ 5 → R$ 3 Your cost: R$ 0.10 (fixed) Margin: (R$ 3 - R$ 0.10) / R$ 3 = 97% (still OK?) But: revenue is halved Daily revenue: R$ 300k (was R$ 500k) You need 67% more volume to make same money Acquisition cost up: can't compete Result: company dies

The real problem:

Token efficiency determines:

1. Your margin (profit per request)
2. Your pricing (how low can you go)
3. Your scaling (can you afford 1M requests/day?)
4. Your competitiveness (can you undercut others?)
5. Your survival (is margin > 0?)

If competitors optimize tokens:

• They can undercut your price (same margin)
• You can't compete
• You lose customers
• You die

If you optimize tokens:

• You can lower price (still profitable)
• You win customers
• You scale
• You survive (and thrive)

---

Solution: Hardware harnesses (token efficiency)

What are "harnesses"?

Definition:

Harnesses = hardware-level optimizations that reduce tokens needed.

How?

Traditional LLM inference:

• Run on GPU (compute tokens one-by-one)
• Each token = full forward pass through model
• Result: lots of compute, lots of time, lots of energy

With harnesses:

• Optimize GPU memory layout
• Optimize batch processing
• Optimize attention computation
• Optimize quantization (lower precision)
• Optimize kernel fusion (combine operations)
• Result: fewer tokens needed, faster, cheaper

Example: attention optimization

Attention mechanism (transformer core): Attention = softmax(Q × K^T) × V

Naive implementation:

• Q: 1024×1024 matrix
• K^T: 1024×1024 matrix
• Multiply: 1024×1024×1024 = 1B operations
• Softmax: 1M operations
• V multiply: 1B operations
• Total: 2B+ operations per token

With harness optimization (Flash Attention, etc):

• Batch operations (use tensor cores)
• Fuse operations (reduce memory reads)
• Smart computation (only compute what's needed)
• Total: 500M operations per token (4x faster)

Result:

• Token generation: 4x faster
• Same output, less compute
• Less energy, less cost

Example: quantization

LLM model precision: Traditional: FP32 (32-bit floats) Example model: 70B parameters × 32 bits = 2.2 TB RAM Need: A100 GPU × 20 = R$ 500k hardware Cost: R$ 100+/request

With quantization harness: INT8 (8-bit integers) Same model: 70B × 8 bits = 560 GB RAM Need: A100 GPU × 5 = R$ 125k hardware Cost: R$ 25/request (4x cheaper)

Result:

• Smaller model
• Faster inference
• Lower cost
• Slightly lower quality (acceptable trade-off)

How much does it save?

Token efficiency gains:

Situation 1: Baseline (no optimization) Tokens/request: 2000 Token cost: R$ 0.02 Hardware cost: R$ 0.01 Total: R$ 0.03/request

Situation 2: With 2x harness optimization Tokens/request: 1000 (50% reduction) Token cost: R$ 0.01 Hardware cost: R$ 0.005 (faster inference = cheaper infra) Total: R$ 0.015/request (50% reduction)

Situation 3: With 4x harness optimization Tokens/request: 500 (75% reduction) Token cost: R$ 0.005 Hardware cost: R$ 0.002 (even cheaper infra) Total: R$ 0.007/request (77% reduction)

Situation 4: With 10x harness optimization Tokens/request: 200 Token cost: R$ 0.002 Hardware cost: R$ 0.001 Total: R$ 0.003/request (90% reduction!)

Impact on your SaaS:

Before (R$ 0.03/request): Customer price: R$ 5 Your cost: R$ 0.03 Margin: 99.4% Daily revenue (10k requests): R$ 50k Daily cost: R$ 300 Daily profit: R$ 49,700

After (R$ 0.003/request, 10x improvement): Customer price: R$ 5 (or R$ 2, undercut competitors) Your cost: R$ 0.003 Margin: 99.94% (or 60% if you drop price) Daily revenue (10k requests): R$ 50k (or R$ 20k at lower price, but 3x volume) Daily cost: R$ 30 Daily profit: R$ 49,970 (or R$ 20k but 3x customers)

Competitive result: You: R$ 2/request (profitable, margin 40%) Competitor: R$ 5/request (same old price) Customer: chooses you (3x cheaper) You: win market

---

Implementation: How to optimize tokens

Option 1: Use optimized inference library

What:

Libraries that include harnesses:

• vLLM (vectorized LLM inference)
• TensorRT-LLM (NVIDIA optimized)
• Text Generation WebUI (with optimizations)
• Ollama (built-in optimization)

Example (vLLM):

python

Before (naive)

import torch from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b") model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b")

outputs = model.generate(input_ids, max_new_tokens=100)

Slow: ~1 token/sec (lots of overhead)

After (with vLLM)

from vllm import LLM

llm = LLM("meta-llama/Llama-2-7b") outputs = llm.generate(prompts, max_tokens=100)

Fast: ~50 tokens/sec (10x improvement!)

Same output, way better performance

Cost:

Before (naive): ~5000 tokens/request After (vLLM): ~1000 tokens/request (5x reduction)

• Input: 500 tokens
• Output: 300 tokens
• Overhead: 200 tokens (vs 4500 before)

Cost before: R$ 0.05/request Cost after: R$ 0.01/request Savings: R$ 0.04/request (80% reduction)

For 10k requests/day: Savings: 10k × R$ 0.04 = R$ 400/day Annual savings: R$ 146k

Option 2: Implement custom harnesses

If you need more optimization:

python

Custom attention optimization

import torch from torch.nn.functional import scaled_dot_product_attention

class OptimizedAttention(nn.Module): def forward(self, Q, K, V): # Use Flash Attention (CUDA kernel optimized) return scaled_dot_product_attention(Q, K, V, dropout_p=0.0) # 4x faster than naive implementation

Custom quantization

from bitsandbytes.nn import Linear4bit

model = AutoModel.from_pretrained( "model_name", device_map="auto", quantization_config=BitsAndBytesConfig( load_in_4bit=True, # 4-bit quantization bnb_4bit_compute_dtype=torch.bfloat16, ) )

4x smaller, faster, still ~same quality

Cost:

Development: 2-4 weeks (optimization engineer) Cost: R$ 50-100k

Payback: Annual savings: R$ 146k+ (from vLLM alone) ROI: 1-2x payback within 1 year

Option 3: Hardware acceleration (GPU optimization)

Use specialized hardware:

Optional: Nvidia H100 (optimized for inference)

• Better tensor cores (faster attention)
• Better memory bandwidth (faster data movement)
• Built-in optimizations (TensorRT, cuDNN)

Result: 2-3x faster inference Cost: ~R$ 30k/month (vs R$ 10k/month for A100)

For agent SaaS: probably not worth it

• Most improvements come from software
• Hardware is secondary
• Software (vLLM, etc) is cheaper

---

2-week action plan

Week 1: Audit + baseline

☐ Measure current token usage

• Add logging: tokens per request
• Histogram: distribution of token counts
• Identify bottlenecks (what wastes tokens?)

☐ Calculate current cost

• Tokens/request: measure
• Cost/request: calculate (OpenAI pricing)
• Daily cost: tokens/request × daily volume × unit cost

☐ Set optimization target

• Baseline: (today's tokens)
• Target: (baseline / 2) or (baseline / 5)
• Goal: reduce cost by 30-70%

Example: Baseline: 2000 tokens/request Target: 1000 tokens/request (50% reduction) Daily volume: 10k requests Current daily cost: 2000 × 10k × R$ 0.000005 = R$ 100 Target daily cost: 1000 × 10k × R$ 0.000005 = R$ 50 Daily savings: R$ 50 Annual savings: R$ 18.25k

Week 2: Implement + deploy

☐ Day 1-2: Try vLLM (easiest)

• Install: pip install vllm
• Replace inference code
• Test: same output, measure latency improvement
• Expect: 5-10x faster

☐ Day 3-4: Reduce context window

• Remove unnecessary context
• Use retrieval (only load relevant context)
• Expect: 30-50% token reduction

☐ Day 5: Deploy + measure

• Deploy vLLM version
• Measure new token count
• Calculate new cost
• If target not met: add quantization (next iteration)

---

ROI: Why this matters

Financial impact

Scenario: You scale to 100k requests/day

Before optimization: Tokens/request: 2000 Cost/request: R$ 0.02 Daily cost: 100k × R$ 0.02 = R$ 2k Monthly cost: R$ 60k Annual cost: R$ 720k

Customer price: R$ 5/request Daily revenue: 100k × R$ 5 = R$ 500k Annual revenue: R$ 182.5M Margin: (R$ 500k - R$ 2k) / R$ 500k = 99.6%

After 50% optimization (vLLM): Tokens/request: 1000 Cost/request: R$ 0.01 Daily cost: 100k × R$ 0.01 = R$ 1k Monthly cost: R$ 30k Annual cost: R$ 360k

Savings: R$ 360k/year (or re-invest in product)

After 80% optimization (vLLM + quantization + context reduction): Tokens/request: 400 Cost/request: R$ 0.004 Daily cost: 100k × R$ 0.004 = R$ 400 Monthly cost: R$ 12k Annual cost: R$ 144k

Savings: R$ 576k/year Or: lower price to R$ 2/request, still 80% margin, win market

Competitive advantage

Competitor (no optimization): Cost: R$ 0.02/request Price: R$ 5/request Margin: 99.6%

You (50% optimized): Cost: R$ 0.01/request Price: R$ 3/request (undercut competitor) Margin: 99.67% (still very profitable) Advantage: 40% cheaper for customer Result: customer chooses you

You (80% optimized): Cost: R$ 0.004/request Price: R$ 1/request (crushing competitor) Margin: 99.6% (still profitable!) Advantage: 80% cheaper Result: market leader

---

Conclusão: Token efficiency = survival

Realidade:

Agent SaaS margin depends on token efficiency

• Inefficient tokens: high cost
• High cost: can't compete on price
• Can't compete: lose customers
• Lose customers: die

Token efficiency depends on harnesses

• Software: vLLM, TensorRT-LLM, etc (immediate)
• Hardware: optimized inference (secondary)
• Prompt engineering: reduce unnecessary tokens (easy)

Optimization is non-negotiable

• Competitors will optimize
• If you don't: they undercut you
• Margin disappears: company dies
• You must optimize NOW (not later)

Action immediate:

1. Measure: tokens/request (today)

2. Optimize: try vLLM (1 day)

3. Deploy: measure impact

4. Scale: add more optimizations (quantization, context reduction)

5. Iterate: continuous improvement (margin increases)

6. Win: undercut competitors, capture market

→ OpenClaw: Token Efficiency, LLM Cost Optimization, Agent Margin, Hardware Harnesses

Seu agent está otimizado? Ou está desperdiçando tokens (e dinheiro)? 💸