Agente IA gera código ruim (como evitar lixo em produção)

Agente IA gera código ruim (como evitar lixo em produção)

Você é founder/CTO de SaaS.

Seu SaaS: agente IA em produção (automação, código-gen, integração).

Seu cenário (muito real):

• Usando agente IA pra: Gerar código (Copilot, Claude, Custom)
• Or: Automação (RPA, workflow, integração)
• Or: SQL queries, API calls, data transformations
• Your hope: "Agente gera código bom (salva tempo)"
• Your reality: "Agente gera código ruim. Muito ruim. E parece bom (até cair em produção)."
• Your problem: "Código ruim queimou produção. Customers down. Reputação hit."
• Your nightmare: "Não há pior código (ruim é infinito). Como rejeitar ruim se há INFINITAS formas de ser ruim?"

The paradox:

• Agente IA fast: Gera 1000 linhas de código em 10 segundos
• Agente IA confident: Looks plausible (passes human eye test)
• Agente IA wrong: Silently fails in production (edge cases, corner cases, business logic gaps)
• Human review slow: Takes 30 min pra revisar 1000 linhas (bottleneck)
• Your dilemma: "Use agente (fast but risky) OR skip agente (slow but safe)?"
• The answer: "Neither. Use agente WITH quality gates (fast + safe)."

---

Código ruim é ilimitado (por que tão difícil evitar)

The infinite badness problem

Example: Simples function (delete customer) python

Agente generates:

def delete_customer(customer_id): db.query(f"DELETE FROM customers WHERE id = {customer_id}") return "done"

How many ways is this bad?

1. SQL Injection ├─ Input: customer_id = "1; DROP TABLE customers;--" ├─ Query: "DELETE FROM customers WHERE id = 1; DROP TABLE customers;--" └─ Result: Entire table deleted (DISASTER)

2. No error handling ├─ Issue: db.query fails (connection error) ├─ Result: Exception not caught (app crashes) └─ Customer sees: 500 error (bad UX)

3. No validation ├─ Input: customer_id = "invalid" OR NULL OR negative ├─ Result: Unexpected behavior (delete wrong customer?) └─ Impact: Data corruption

4. No audit trail ├─ Issue: Who deleted customer? When? Why? ├─ Problem: Can't trace who did it (compliance risk) └─ Consequence: LGPD violation (audit trail required)

5. No soft delete ├─ Issue: Permanent deletion (can't recover) ├─ Problem: Customer disputes, customer wants data back └─ Result: Can't fulfill request (litigation risk)

6. No permissions check ├─ Issue: Anyone can delete anyone ├─ Problem: Customer A deletes Customer B's account └─ Result: Security breach (privilege escalation)

7. No transaction ├─ Issue: Delete succeeds, but related data cleanup fails ├─ Problem: Customer deleted, but orders still exist (orphaned data) └─ Result: Data inconsistency (nightmare)

8. No logging ├─ Issue: No record of what happened ├─ Problem: Can't debug when things go wrong └─ Result: Takes 10x longer to fix (time + money waste)

9. Wrong error message ├─ Issue: Returns "done" even if deletion failed ├─ Problem: Caller thinks success, but data not deleted └─ Result: Silent failure (hardest bug to find)

10. No rate limiting ├─ Issue: Attacker can delete all customers (loop) ├─ Problem: DoS attack (business interruption) └─ Result: Service down (revenue loss)

The point: There's NO LIMIT to how bad code can be. For every good way to write code, there are infinite bad ways. Agente IA can produce ANY of these bad patterns (and more you haven't thought of).

Why agente produces bad code

Reason 1: Training data includes bad code

Agente trained on: ├─ Stack Overflow (includes wrong answers) ├─ GitHub (includes real-world bad code) ├─ Textbooks (simplified, unrealistic examples) └─ Result: Agente learns patterns (good AND bad)

When agente generates: ├─ Most probable patterns = often mediocre ├─ Agente doesn't know "bad" (no penalty in training) └─ Result: Produces code that looks good (but is bad)

Reason 2: Agente optimizes for speed, not correctness

Agente's optimization: ├─ Fast generation (minimize tokens) ├─ Code that compiles (avoid syntax errors) ├─ Looks reasonable (matches prompt) └─ Does NOT optimize for: Security, performance, edge cases, maintainability

Result: ├─ Agente produces code that runs (but is wrong) ├─ Bugs are subtle (not obvious) └─ Fail in production (under load, edge cases)

Reason 3: Agente doesn't test (no feedback)

Agente workflow: ├─ Read prompt ├─ Generate code ├─ Return result (DONE) ├─ No testing (doesn't run code) ├─ No feedback (doesn't see failures) └─ Result: Never learns what went wrong

Human workflow: ├─ Read prompt ├─ Generate code ├─ Test (run it, find bugs) ├─ Fix bugs ├─ Test again (iterate) └─ Result: Code is solid (battle-tested)

Reason 4: Edge cases don't exist in prompt

Prompt: "Write function to delete customer" ├─ Agente generates: Basic code (happy path) ├─ Missing: Error handling, validation, edge cases ├─ Why: Prompt didn't mention them └─ Result: Code breaks in production (edge cases happen in real world)

---

Seu agente IA (onde código ruim queima você)

Deployment scenario (how bad code enters production)

Scenario 1: Copilot in IDE (developer uses agente code directly)

Workflow: ├─ Developer types: "write function to fetch user" ├─ Copilot suggests: Function (looks good) ├─ Developer: Copy-pastes (saves time) ├─ Testing: None (assumes Copilot knows what it's doing) ├─ Commit: Code enters repo (no review) ├─ Deploy: Code in production (LIVE)

What happens: ├─ Week 1: Works fine (common paths only) ├─ Week 2: Performance degrades (N+1 queries, no caching) ├─ Week 3: Customers complain (slow feature) ├─ Week 4: Debug reveals: Bad code (inefficient queries) ├─ Cost: 2-4 weeks of lost productivity (+ customer churn)

Scenario 2: Custom agente (automation, API calls)

Setup: ├─ Custom agente: Generates SQL queries (based on user input) ├─ Agente output: "SELECT * FROM users WHERE email = '{input}'" ├─ No review: Assuming agente is trustworthy ├─ No testing: Assuming agente knows SQL ├─ Deploy: Query in production (LIVE)

What happens: ├─ Attacker inputs: "admin@test.com' OR '1'='1" ├─ Query becomes: "SELECT * FROM users WHERE email = 'admin@test.com' OR '1'='1'" ├─ Result: Returns all users (SQL injection) ├─ Attacker: Access to all customer data (BREACH) ├─ Cost: R$ 1M+ (breach investigation, LGPD fines, customer churn)

Scenario 3: RPA agente (automating workflows)

Setup: ├─ Agente: Automates invoice processing (reads PDF, extracts data) ├─ Agente logic: Naive parsing (string.split()) ├─ No edge cases: What if invoice format changes? ├─ No fallback: What if agente can't parse? ├─ Deploy: Agente processes 1000 invoices/day (LIVE)

What happens: ├─ Vendor A changes invoice format (slightly different) ├─ Agente fails silently (returns wrong data) ├─ Invoices processed incorrectly (payment amount wrong) ├─ Months later: Reconciliation reveals errors (thousands of invoices) ├─ Manual fix: Takes 3 weeks of work (+ customer disputes) ├─ Cost: R$ 50K+ (labor, customer disputes, lost revenue)

Impact analysis (how bad code hurts)

Business impact:

☐ Customer downtime ├─ Code bug causes outage (feature broken) ├─ Customers can't use service (revenue impact) ├─ Downtime: 4 hours (your SLA violated) └─ Cost: R$ 10K-100K (depends on business size)

☐ Security breach ├─ Bad code has vulnerability (SQL injection, auth bypass) ├─ Attacker exploits (steals data) ├─ Breach discovered (weeks or months later) └─ Cost: R$ 1M+ (LGPD fines, incident response, churn)

☐ Data corruption ├─ Bad code modifies data incorrectly (wrong amounts, wrong assignments) ├─ Discovered during reconciliation (weeks later) ├─ Manual fix required (thousands of records) └─ Cost: R$ 50K-500K (labor + customer disputes)

☐ Reputation damage ├─ Customer hears: "Your agente caused outage/breach" ├─ Customer loses trust (not your fault, but they blame you) ├─ Customer leaves (switches to competitor) └─ Cost: Lost revenue + negative reviews

Technical debt impact:

☐ Maintenance nightmare ├─ Bad code is hard to fix (spaghetti, no structure) ├─ Takes 3x longer to debug (bad patterns) └─ Team morale: Low (frustrated with code quality)

☐ Test burden ├─ Bad code needs 10x more tests (to cover edge cases) ├─ Tests are fragile (break on small changes) └─ Testing slows development (defeats original time-saving goal)

☐ Performance degradation ├─ Bad code is inefficient (N+1 queries, nested loops) ├─ App slows down (as more data added) ├─ Customers complain (app feels slow) └─ Cost: Lost users, reduced engagement

---

Quality gates (como evitar código ruim)

Gate 1: Automated testing (unit + integration tests)

Setup:

For each agente output: ├─ Run unit tests (does function do what it promises?) ├─ Run integration tests (does it work with other code?) ├─ Run security tests (any obvious vulnerabilities?) ├─ Run performance tests (is it efficient?) ├─ All tests must pass (otherwise REJECT agente code)

Example: ├─ Agente generates: delete_customer() function ├─ Unit test: Test delete (verify customer deleted) ├─ Integration test: Test with auth (only authorized users) ├─ Security test: Test SQL injection (validate input) ├─ Performance test: Test with 1M customers (no N+1 queries) ├─ Result: All pass = accept, Any fail = REJECT + ask agente to fix

Implementation: python def validate_agente_code(agente_output, test_suite): # Run automated tests results = test_suite.run(agente_output)

# Check results
if results.all_passed():
    return {"status": "approved", "code": agente_output}
else:
    return {
        "status": "rejected",
        "reason": f"Failed tests: {results.failed_tests}",
        "feedback": "Please fix and regenerate"
    }

Gate 2: Static analysis (linting, security scanning)

Setup:

For each agente output: ├─ Lint code (style, obvious bugs) ├─ Security scan (OWASP, SQL injection, etc) ├─ Complexity check (too many nested loops?) ├─ Dependency check (using outdated packages?) ├─ All checks must pass (otherwise REJECT)

Tools: ├─ SonarQube, CodeClimate (code quality) ├─ Bandit, Safety (security) ├─ pylint, flake8 (style)

Gate 3: Human code review (peer review + domain expert)

Setup:

For each agente output: ├─ Code goes to review queue (not directly to production) ├─ Human reviewer checks: │ ├─ Logic (does it solve the problem?) │ ├─ Edge cases (what if...?) │ ├─ Performance (is it efficient?) │ ├─ Security (any vulnerabilities?) │ ├─ Maintainability (is it understandable?) │ └─ Best practices (follows company standards?) ├─ Reviewer decision: │ ├─ Approve (code is good, merge to production) │ ├─ Request changes (fix this, regenerate) │ └─ Reject (start over, agente doesn't understand requirement) └─ Approved code only: Deployed to production

Review time: ├─ Simple code: 5-10 min ├─ Complex code: 30-60 min ├─ Bottleneck: Yes (slows deployment) └─ Worth it: YES (prevents disasters)

Optimization: Targeted review

Not all code needs full review: ├─ Low risk (refactoring, utility functions): Light review (5 min) ├─ Medium risk (business logic): Standard review (30 min) ├─ High risk (auth, payments, data access): Deep review (60+ min) ├─ Result: Average review time = 15 min (balanced)

Gate 4: Staged deployment (canary, feature flags)

Setup:

Even after approval, don't deploy to 100% of customers: ├─ Stage 1 (Canary): Deploy to 1-5% of customers │ ├─ Monitor: Errors, performance, customer complaints │ ├─ If good: Continue to Stage 2 (30 min monitoring) │ ├─ If bad: Rollback immediately (before more customers hit it) │ └─ Result: Catch bugs before they affect everyone ├─ Stage 2 (Gradual): Deploy to 5-50% (ramp up over hours) │ ├─ Monitor: Errors scale linearly (expected) │ ├─ If problems: Stop deployment, investigate │ └─ If good: Continue to Stage 3 ├─ Stage 3 (Full): Deploy to 100% │ ├─ Monitor: 24 hours (ensure stability) │ └─ If stable: Success (monitor for weeks) └─ Timeline: 12-24 hours total (before full deployment)

Benefit: ├─ Catch bugs early (before affecting all customers) ├─ Fast rollback (if things go wrong) ├─ Confidence (staged approach reduces risk)

Gate 5: Monitoring + alerting (detect failures in production)

Setup:

Even with all gates, production has surprises: ├─ Monitor: │ ├─ Error rates (is code failing?) │ ├─ Performance metrics (is it slow?) │ ├─ Business metrics (conversion, churn, revenue) │ ├─ Security alerts (any attack patterns?) │ └─ Customer complaints (reported via support) ├─ Alert on: │ ├─ 2x error rate spike (something's wrong) │ ├─ Performance degradation (response time +50%) │ ├─ Security event (suspicious activity) │ └─ Customer complaints (support team flags) ├─ Response: │ ├─ Investigate (what went wrong?) │ ├─ Rollback (revert bad code) │ ├─ Fix (update agente, add tests) │ └─ Deploy again (with improvements) └─ Timeline: Detection within minutes (not hours)

---

Implementação (seu checklist)

Week 1: Setup quality gates

☐ Define test suite (unit, integration, security tests) ├─ What tests must pass (non-negotiable) ├─ Tool: Jest, pytest, etc └─ Owner: Engineering lead

☐ Setup static analysis ├─ Linter: eslint, pylint ├─ Security: Bandit, OWASP ├─ Tool: SonarQube or CodeClimate └─ Owner: DevOps/Security

☐ Code review process ├─ Who reviews (domain experts) ├─ Review criteria (checklist) ├─ Approval workflow (GitHub, GitLab) └─ Owner: Engineering lead

☐ Staging environment ├─ Canary deployment (1-5% traffic) ├─ Monitoring dashboards (errors, performance) ├─ Rollback procedure (documented) └─ Owner: DevOps/SRE

Week 2-4: Train team + implement

☐ Agente configuration ├─ Configure agente (e.g., Copilot, Claude API) ├─ Add quality gate checks (automated) ├─ Integrate with CI/CD (tests run automatically) └─ Owner: Engineering

☐ Team training ├─ Show team the quality gates ├─ Explain: Why gates exist (protect from bad code) ├─ Demo: Gate rejecting bad code (confidence builder) └─ Owner: Tech lead

☐ Test coverage ├─ Write tests for common functions ├─ Agente code must pass tests (requirement) ├─ Coverage: Aim for 80%+ (realistic) └─ Owner: QA + Engineering

Ongoing: Monitor + improve

☐ Metrics dashboard ├─ Agente output quality (% passing tests) ├─ Code review time (is it slowing us down?) ├─ Production issues (are gates preventing them?) └─ Owner: Engineering lead

☐ Feedback loop ├─ When gates reject code: Why? ├─ Common failure patterns (improve gates) ├─ Agente prompt tuning (better inputs = better outputs) ├─ Team learnings (share knowledge) └─ Owner: Engineering lead

☐ Iterate ├─ Monthly review (what's working?) ├─ Adjust gates (too strict? too loose?) ├─ Update tests (new edge cases found) ├─ Train agente (feedback for better outputs) └─ Owner: Tech lead

---

Conclusão: Código ruim é infinito (gates são necessário)

Signal ("No Limit to How Bad Code Can Get"):

• Código ruim tem infinitas formas
• Agente IA gera código plausível (mas ruim)
• Ruim só aparece em produção (too late)
• Manual review é slow (but necessary)
• Automation + human review = solution

Your situation now:

• Agente IA em produção (gera código/automação)
• Sem quality gates (shipping code directly)
• Code issues appearing in production (expensive)
• Customers seeing bugs (trust erosion)
• You spending time fixing (instead of building)

Your options:

Option 1: No gates (fast, risky)

• Pros: Agente code deploys instantly (time saved)
• Cons: Production bugs (frequent), customer impact (high), fixing costs (high)
• Risk: Alto (inevitable production issues)
• Recommendation: NOT recommended

Option 2: Heavy manual review only (safe, slow)

• Pros: Code quality high (humans catch everything)
• Cons: Review is slow (defeats agente time-saving goal)
• Risk: Baixo (but defeats purpose of agente)
• Recommendation: Workable, but suboptimal

Option 3: Automated gates + targeted review (RECOMMENDED)

• Pros: Fast (gates run in minutes), safe (catches most issues), balanced (human review only where needed)
• Cons: Initial setup (1-2 weeks)
• Risk: Baixo (if gates well-designed)
• ROI: High (prevents production issues, saves time)
• Recommendation: Best practice

At OpenClaw, we help SaaS teams implement quality gates for agente IA:

• DEFINE: Test suite (what must pass)
• CONFIGURE: Automated checks (static analysis, security scanning)
• IMPLEMENT: Code review workflow (targeted review)
• DEPLOY: Staged rollout (canary, feature flags)
• MONITOR: Production metrics (catch failures early)

Result: Agente IA é rápido AND confiável. Code quality é consistente. Production issues são raros. Customers confiável.

Seu agente IA gera código (ou automação)?

Você tem automated tests (que rejeitam código ruim)?

Você tem code review process (antes de produção)?

Você está vendo production issues (agente código quebrou algo)?

Você sabe quantas formas código pode ser ruim (infinito)?

Se não sabe ou quer expert guidance (test suite design, static analysis setup, code review automation, staging deployment, production monitoring):

Implementar Quality Gates AGORA (testes automatizados, análise estática, review workflow, staged deployment, monitoramento—evite código ruim em produção + acelere agente com confiança) →