Deploy agente: 12 decisões críticas por modelo

Deploy agente: 12 decisões críticas por modelo.

Você é founder de SaaS.

Seu agente de IA:

• Rodando em SageMaker (AWS managed)
• Your assumption: "Pick model, deploy, done."
• Reality: "AWS warns: 12+ critical decisions per model deployment."
• Your blind spot: ├─ Decision 1: Qual container image? (serving framework) ├─ Decision 2: Qual image tag? (versão regional correcta) ├─ Decision 3: Qual instance type? (t3.medium vs g4dn.xlarge?) ├─ Decision 4: Quantas instâncias? (1 vs 3 vs 10?) ├─ Decision 5: Autoscaling enabled? (scale down quando idle?) ├─ Decision 6: Qual métrica pra escalar? (CPU, GPU, custom?) ├─ Decision 7: Min/max replicas? (cost vs availability) ├─ Decision 8: Health check timeout? (how long to wait for response?) ├─ Decision 9: Qual CloudWatch alarms? (catch failures before users?) ├─ Decision 10: Logging enabled? (debug production issues?) ├─ Decision 11: VPC config? (public vs private endpoint?) ├─ Decision 12: IAM roles? (what data can agent access?) ├─ Result: "12 decisions × 2-3 options each = 48+ possible configs" ├─ Current state: "Every engineer makes different choices (chaos)" ├─ Your agent A: Configured by John (slow, over-provisioned) ├─ Your agent B: Configured by Maria (fast, under-provisioned) ├─ Your agent C: Configured by new hire (broken, misconfigured) └─ Result: "No consistency = unpredictable performance/cost."

AWS just warned:

"Deploying Hugging Face models on SageMaker requires making 12+ critical infrastructure decisions. Most teams make these decisions individually per model (no standard). Result: Inconsistent deployments, unpredictable costs, silent failures, engineering time wasted on repetitive decisions. Solution: Establish deployment pattern (template) that standardizes these 12 decisions. Use pattern for every agent deployment. Result: Faster deployment, consistent quality, predictable costs."

Translation to your SaaS:

• Old approach: "Deploy model, figure out on the way"
• New approach: "Deploy model using standard template (12 decisions decided upfront)"
• Old result: "Chaos (each deployment different)"
• New result: "Consistency (all deployments follow pattern)"
• Old speed: "2-3 days per deployment (too many unknowns)"
• New speed: "2-3 hours per deployment (template handles decisions)"
• Old cost: "$5k+/month (over-provisioned, inefficient)"
• New cost: "$1-2k/month (optimized, consistent)"

---

12 decisões de deployment: o que cada uma significa

Decision 1-2: Container image + image tag

=== DECISION 1: Qual container image usar? ===

Options: ├─ HuggingFace official: huggingface/transformers-pytorch-gpu ├─ SageMaker official: 763104330519.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-tgi-inference:2.3.0-tgi1.4.0-gpu-py311 ├─ Custom image: your-registry/your-image:latest └─ Decision criteria: What framework? (PyTorch, TensorFlow, TGI?)

Implications: ├─ Official HF: Less optimized, smaller, maintained by community ├─ SageMaker official: Optimized for SageMaker, maintained by AWS, region-specific ├─ Custom: Most optimized (but requires building/maintaining) └─ Recommendation: SageMaker official (balance of optimization + support)

=== DECISION 2: Qual image tag (versão)? ===

Options: ├─ Specific version: 2.3.0-tgi1.4.0-gpu-py311 (pinned, predictable) ├─ Latest major: 2.3.0-gpu-py311 (updates within major, might break) ├─ Latest: latest (always updates, might break) └─ Decision criteria: How important is stability vs freshness?

Implications: ├─ Specific version: Reproducible, stable, but manual updates ├─ Latest major: Good compromise (patch updates auto, major stable) ├─ Latest: Auto-updates, but risk of breaking changes └─ Recommendation: Specific version (pin, update intentionally quarterly)

=== CRITICAL: IMAGE TAG MUST MATCH REGION ===

Regions available: ├─ us-east-1: 763104330519.dkr.ecr.us-east-1.amazonaws.com/... ├─ us-west-2: 763104330519.dkr.ecr.us-west-2.amazonaws.com/... ├─ eu-west-1: 763104330519.dkr.ecr.eu-west-1.amazonaws.com/... ├─ ap-southeast-1: 763104330519.dkr.ecr.ap-southeast-1.amazonaws.com/... └─ Note: Image URL must match your SageMaker region (wrong region = deployment fails)

Mistake: ├─ Deploy in us-east-1, use image from us-west-2 ├─ Result: "Image not found" (confusing error) ├─ Debug time: 30 minutes (why is image missing?) └─ Prevention: Parameterize region in deployment config

Decision 3-4: Instance type + number of instances

=== DECISION 3: Qual instance type? ===

Options (sorted by price/performance): ├─ t3.large ($0.1/hour): CPU only, slow, cheap (not for LLMs) ├─ t3.xlarge ($0.2/hour): CPU only, slow (not for LLMs) ├─ g4dn.xlarge ($1.1/hour): 1x T4 GPU, good (small models) ├─ g4dn.2xlarge ($1.6/hour): 1x T4 GPU, good (medium models) ├─ g4dn.12xlarge ($7/hour): 4x T4 GPUs (large models) ├─ p3.2xlarge ($3.06/hour): 1x V100 GPU (high performance) └─ p3.8xlarge ($12/hour): 4x V100 GPUs (very large models)

Decision criteria: ├─ Model size: How many params? (7B, 13B, 70B?) ├─ Latency requirement: How fast must response be? (<100ms, <500ms?) ├─ Throughput requirement: How many requests/sec? (1, 10, 100?) ├─ Budget: How much can you spend? ($1k, $5k, $20k/month?) └─ Expected growth: Will you scale? (if yes, buy bigger now)

Example calculations: ├─ 7B model (Mistral): Fits in g4dn.xlarge ($1.1/hour) │ ├─ Cost: $1.1 × 730 hours/month = $800/month │ ├─ Throughput: ~5 requests/sec (limited by single GPU) │ └─ Latency: ~200ms (acceptable for chat) ├─ 70B model (Llama 2): Needs g4dn.12xlarge or p3.2xlarge │ ├─ Cost: $7/hour × 730 = $5,100/month (g4dn) or $2,234/month (p3) │ ├─ Throughput: ~20-30 requests/sec (multiple GPUs) │ └─ Latency: ~100-200ms (good) └─ Recommendation: Start with smallest that fits, upgrade if latency bad

=== DECISION 4: Quantas instâncias (replicas)? ===

Options: ├─ 1 replica: Cheapest, but single point of failure ├─ 3 replicas: High availability, can handle 3x traffic, 3x cost ├─ 5+ replicas: Very high availability, but expensive └─ Decision criteria: How important is uptime? How much traffic?

Implications: ├─ 1 replica: If instance dies, agent is down (bad) ├─ 3 replicas: If 1 dies, other 2 handle traffic (good) ├─ Auto-scaling: Can add replicas dynamically (if traffic spikes) └─ Recommendation: Start with 1, scale to 3 when traffic > 70% capacity

Cost impact: ├─ 1x g4dn.xlarge: $800/month ├─ 3x g4dn.xlarge: $2,400/month (3x cost) ├─ 5x g4dn.xlarge: $4,000/month (5x cost) └─ Trade-off: Uptime vs cost (choose wisely)

Decision 5-7: Autoscaling configuration

=== DECISION 5-7: Autoscaling (scale up/down automatically) ===

Decision 5: Enable autoscaling? ├─ Option 1: Yes (scale based on load, save costs) ├─ Option 2: No (fixed replicas, predictable costs) └─ Recommendation: Yes (unless traffic is constant)

Decision 6: Which metric to scale on? ├─ Option 1: CPU utilization (>70% → scale up) ├─ Option 2: GPU utilization (>80% → scale up) ├─ Option 3: Custom metric (requests/sec, latency) └─ Recommendation: GPU utilization (most relevant for LLMs)

Decision 7: Min/max replicas? ├─ Min: 1 (don't go below this, always have capacity) ├─ Max: 10 (don't exceed this, control costs) └─ Example config: Min=1, Max=5, Scale up at 80% GPU, Scale down at 30% GPU

Autoscaling in action: ├─ Traffic is low: 1 replica running (cost: $800/month) ├─ Traffic spikes: Auto-scales to 3 replicas (cost: $2,400/month) ├─ Traffic spikes more: Auto-scales to 5 replicas (cost: $4,000/month) ├─ Traffic dies: Auto-scales back to 1 replica (cost: $800/month) └─ Result: Pay for what you use (max ROI)

Scaling delays: ├─ Scale-up takes 2-5 minutes (new instance boots) ├─ Scale-down takes 5-10 minutes (drains connections, then terminates) ├─ In-flight requests: Graceful shutdown (don't lose requests) └─ Trade-off: Faster scaling = higher latency (more instances always running)

Decision 8-12: Monitoring, networking, security

=== DECISION 8: Health check timeout? ===

Options: ├─ 30 seconds: Fast detection of dead instances, might be too aggressive ├─ 60 seconds: Balanced (recommended) ├─ 300 seconds: Lenient, slow detection of failures └─ Decision: Latency tolerance vs failure detection speed

Implication: ├─ If timeout too short: False positives (instance marked dead while still booting) ├─ If timeout too long: Slow failure detection (users wait while instance dead) └─ Recommendation: 60 seconds

=== DECISION 9: CloudWatch alarms? ===

What to alarm on: ├─ Model latency >500ms: Agent is slow (something wrong?) ├─ Error rate >5%: Requests failing (model or infra issue?) ├─ GPU memory >90%: Running out of space (add instance?) ├─ Number of endpoints down: Availability check (alerting) └─ Custom metrics: Your specific KPIs (e.g., inference cost per request)

Alarm actions: ├─ Send to Slack: Team gets notified in real-time ├─ Auto-scale: Trigger scale-up if latency high ├─ Page on-call: Wake up engineer if critical └─ Recommendation: At least 3 critical alarms (latency, error rate, availability)

=== DECISION 10: Logging enabled? ===

What to log: ├─ Input tokens: What did user ask? (for debugging) ├─ Output tokens: What did model generate? (for quality check) ├─ Latency: How long did inference take? (performance tracking) ├─ Errors: What went wrong? (debugging failures) └─ Cost: How much did this inference cost? (cost tracking)

Logging destination: ├─ CloudWatch Logs: Integrated, searchable, queryable ├─ S3: Long-term storage, cheaper for large volume ├─ Custom database: If you want custom analysis └─ Recommendation: CloudWatch for real-time, S3 for archive

Cost impact: ├─ No logging: Cheapest, but no visibility ├─ Minimal logging: Small cost, good visibility ├─ Full logging: Higher cost (storage, analysis) └─ Recommendation: Log errors + latency always, log inputs/outputs only if needed

=== DECISION 11: VPC configuration? ===

Options: ├─ Public endpoint: Anyone with URL can call (bad for security) ├─ Private endpoint: Only from within VPC (good security) ├─ VPC with NAT: Private but can call external services (balanced) └─ Decision: Security vs accessibility

Implication: ├─ Public: Easy to test, bad for production (security risk) ├─ Private: Secure, but harder to integrate (requires VPC setup) └─ Recommendation: Always use private for production

=== DECISION 12: IAM roles (what data can agent access)? ===

Options: ├─ Full S3 access: Agent can read/write any S3 bucket (too permissive) ├─ Specific bucket: Agent can only access designated bucket (good) ├─ Read-only: Agent can only read data (safe) ├─ Denied: Agent has no data access (safest) └─ Decision: What does agent actually need?

Implication: ├─ Too permissive: If agent is compromised, hacker has access ├─ Too restrictive: Agent can't do its job └─ Recommendation: Least privilege (only what agent needs)

---

Como simplificar: deployment template padrão

Step 1: Create standard config

yaml

deployment-config.yml (standard template)

model: name: "mistral-7b-instruct" # Decision 1-2: Container + image container_image: "763104330519.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-tgi-inference:2.3.0-tgi1.4.0-gpu-py311" region: "us-east-1"

instance: type: "g4dn.xlarge" # Decision 3: Instance type count: 1 # Decision 4: Number of instances

autoscaling: # Decision 5-7: Autoscaling enabled: true min_replicas: 1 max_replicas: 5 metric: "GPU_utilization" target_value: 80

health: # Decision 8: Health check timeout: 60 interval: 30

monitoring: # Decision 9-10: Alarms + logging alarms: - latency > 500ms - error_rate > 5% - gpu_memory > 90% logging: "cloudwatch" log_level: "INFO"

networking: # Decision 11: VPC config vpc_id: "vpc-12345" subnet_ids: ["subnet-123", "subnet-456"] public: false

security: # Decision 12: IAM role iam_role: "arn:aws:iam::123456789:role/agent-inference" permissions: - s3:GetObject - s3:ListBucket resources: - "arn:aws:s3:::my-data-bucket"

Step 2: Use template for all deployments

bash

deploy.sh (use standard config)

Deploy agent A

aws sagemaker create-endpoint
--endpoint-name "agent-support"
--config deployment-config.yml

Deploy agent B (same config)

aws sagemaker create-endpoint
--endpoint-name "agent-sales"
--config deployment-config.yml

Deploy agent C (same config)

aws sagemaker create-endpoint
--endpoint-name "agent-billing"
--config deployment-config.yml

Result: All 3 agents deployed identically

- Same performance

- Same cost

- Same monitoring

- Same reliability

Step 3: Maintain template as team evolves

Quarterly review: ├─ Is our standard config still optimal? (latency, cost) ├─ Have we learned new best practices? (update config) ├─ Are there new instance types? (evaluate cost savings) ├─ Do monitoring alarms catch issues? (refine alerts) └─ Result: Template improves over time

---

Benefit: Standardization kills chaos

Before: Ad-hoc deployments (chaos)

Agent A (deployed by John): ├─ Instance: t3.xlarge (CPU only, slow, cheap) ├─ Replicas: 1 (no redundancy) ├─ Autoscaling: Off (fixed cost) ├─ Monitoring: None (blind) ├─ Result: Agent is slow + fails silently

Agent B (deployed by Maria): ├─ Instance: g4dn.12xlarge (massive, expensive) ├─ Replicas: 5 (expensive redundancy) ├─ Autoscaling: On, scaling too aggressively (expensive) ├─ Monitoring: Excessive (expensive logging) ├─ Result: Agent is fast + very expensive

Agent C (deployed by intern): ├─ Instance: p3.8xlarge (very expensive) ├─ Replicas: 3 (not enough for traffic) ├─ Autoscaling: Misconfigured (doesn't scale when needed) ├─ Monitoring: Broken alarms (doesn't alert on failures) ├─ Result: Agent is expensive + unreliable

Team result: ├─ Agent A: Slow (customers complain) ├─ Agent B: Expensive (margin erodes) ├─ Agent C: Unreliable (credibility damaged) ├─ Cost: $8k/month (should be $2-3k) ├─ Performance: Inconsistent (confusing) └─ Engineering time: 20% wasted on debugging + reconfiguring

After: Standard template (consistency)

Agent A, B, C (all deployed using template): ├─ Instance: g4dn.xlarge (right-sized) ├─ Replicas: 1 (baseline) ├─ Autoscaling: On, config tuned (scale when needed) ├─ Monitoring: Standard alarms (catch issues) ├─ Result: All agents have same performance/cost

Team result: ├─ Agent A: Fast (meets SLA) ├─ Agent B: Cost-efficient (margin healthy) ├─ Agent C: Reliable (customers trust) ├─ Cost: $2.4k/month (optimized) ├─ Performance: Consistent (predictable) └─ Engineering time: Zero wasted on decision-making (use template)

Bonus: ├─ Onboarding: New engineer uses template (no guidance needed) ├─ Scaling: Add 10 more agents (all identical) ├─ Updates: Improve template, update all agents automatically ├─ Debugging: "All agents use same config" = easier debugging └─ Cost: Savings = ~$70k/year (can reinvest in product)

---

12 decisões: não é complexidade, é oportunidade

O que aconteceu:

1. AWS warned: Deploying Hugging Face = 12+ decisions (not 1-2)

   • Implicação: "Ad-hoc deployment = inconsistent results."
   • Action: "Create standard template."

2. Each decision has trade-offs (cost vs performance, uptime vs budget)

   • Implicação: "No single 'right answer' (depends on your needs)."
   • Action: "Decide once in template, use everywhere."

3. Most teams make 12 decisions per agent (wasteful, error-prone)

   • Implicação: "Recurring overhead = wasted engineering time."
   • Action: "Make 12 decisions once, reuse template."

4. Standard template = consistency + cost savings ($70k+/year)

   • Implicação: "Standardization has real ROI."
   • Action: "Invest 1 week in template, save months of engineering."

5. Template evolves (as you learn, update template)

   • Implicação: "Continuous optimization (not one-time decision)."
   • Action: "Review quarterly, improve template."

Your options:

• Ignore: Keep deploying ad-hoc = chaos continues
• React: Someone breaks deployment, you create template = reactive
• Proactive: Create template NOW = recommended (save time today)

Recommendation: IF YOU'RE DEPLOYING MULTIPLE AGENTS: Spend 1 week creating standard template (12 decisions documented, container image identified, autoscaling configured, alarms set up). Save 2-3 hours per agent deployment going forward. By agent #5, you've paid for template time. By agent #10, you've saved 20+ engineering hours. Create template before deploying next agent.

Na OpenClaw:

Ajudamos SaaS builders simplify agent deployment:

• Deployment audit: Quantas decisões você está fazendo per agent? (current state)
• Template creation: Standard config baseado em seu use case (design)
• Container optimization: Container image certo + region config (setup)
• Autoscaling tuning: Min/max replicas + metrics (optimization)
• Monitoring setup: CloudWatch alarms + logging (observability)
• Security hardening: IAM roles + VPC config (safety)
• Cost analysis: Instance right-sizing + autoscaling efficiency (budget)
• Team training: How to use template (adoption).

Those 12 decisions aren't complexity—they're tuning knobs. Decide once, tune forever. Create standard, deploy fast. Your competitors are still making 12 decisions per agent. You're reusing template. That's competitive advantage.

Create Agent Deployment Template | SageMaker Optimization | Cost Savings →