Seu agent cita errado (OpenAI prova)

Seu agent cita errado (OpenAI prova)

O que a pesquisa descobriu (recentemente):

Researchers tested:

"Can we use OpenAI's API instead of ChatGPT for customer-facing citations?"

Methodology:

Take ChatGPT response with citations Example: "According to [link], feature X costs $50"

Test OpenAI API: "Generate same response (with citations) using API"

Compare: Does API cite the SAME link? Does API cite the EXACT PAGE (not just domain)?

Result:

ChatGPT citation: "According to https://example.com/pricing-page, price is $50" OpenAI API citation: "According to https://example.com, price is $50"

Domain matches: ✓ Yes Exact page matches: ✗ No

Customer clicks link: Expects: Pricing page (with $50 price) Gets: Homepage (no price visible) Confusion: "API citation is wrong!"

Conclusion:

"OpenAI API citations often:

• Match the domain (right source)
• But NOT the exact page (wrong specific URL)
• Sometimes cite related pages (not the actual source)
• Sometimes cite nothing (hallucinate sources)

Accuracy: ~60-70% (not good enough for customer-facing)

Then ask yourself:

"My SaaS agent uses OpenAI API. Agent answers customer questions: 'According to our docs, feature X does Y.' 'According to pricing page, plan Z costs $W.' 'According to terms, policy A applies.'

Agent cites links. Customer clicks link. Link doesn't match what agent said.

Customer thinks: 'Agent is broken.' 'Company is lying.' 'I don't trust this SaaS.'

Result: Customer leaves. Review damage. Churn.

Who's liable? YOU ARE (agent is yours)."

---

The citation problem (what's really broken)

Why OpenAI API citations fail

Root cause 1: Training data mismatch

OpenAI trained on:

• Web pages (full text)
• Not metadata (URL, page title, structure)
• Model learns: "This content is about X"
• Model doesn't learn: "This content is on URL Y"

Result: When asked to cite: Model generates: "URL that matches domain" But: Not exact page (just guesses)

Example: Training data: "Pricing is on pricing.html" Model learns: "Pricing info exists" Model doesn't learn: "Location is /pricing.html"

When asked to cite: Model: "I know it's on example.com" Model: "I'll cite https://example.com" But: Actually on https://example.com/pricing.html Result: Citation is WRONG

Root cause 2: Hallucinations (model invents sources)

OpenAI's training includes:

• Real pages
• Made-up pages (in fiction, examples, etc)
• Model can't distinguish

When asked to cite: Model: "I learned this somewhere" Model: "I'll make up a URL that sounds right"

Example: Customer: "What's your refund policy?" Agent: "According to https://example.com/refund-policy" But: You don't have /refund-policy page Agent: HALLUCINATED the URL Customer: Clicks link, gets 404 Customer: "Your agent is broken!"

Root cause 3: URL structure ambiguity

Your actual site: https://example.com/docs/features/pricing (nested structure)

OpenAI training: Model saw: - /pricing - /features/pricing - /docs/pricing - /products/pricing - Multiple versions of "pricing page"

Model learned: "Pricing info exists somewhere" "Not sure which URL"

When asked to cite: Model: "I'll guess... /pricing" But: You don't have /pricing You have: /docs/features/pricing Result: Citation is WRONG

The accuracy gap (ChatGPT vs OpenAI API)

Why ChatGPT is better:

ChatGPT (web interface):

• Uses: Real-time search (Bing)
• Gets: Current links + exact URLs
• Cites: Real pages (verified)
• Accuracy: ~85-90% (good for consumer use)

OpenAI API:

• Uses: Training data (static, from 2024)
• Gets: Hallucinated/guessed URLs
• Cites: Often wrong (domain right, path wrong)
• Accuracy: ~60-70% (bad for production use)

Why the difference:

ChatGPT pipeline: Question → Search (current web) → Get URL → Cite URL → Show to user

OpenAI API pipeline: Question → Generate answer → Guess URL → Cite URL → Show to user

Key difference: Real-time search vs guessing

---

Your liability exposure (citation is liability)

Scenario 1: Wrong price cited

Your SaaS agent: "Our Enterprise plan costs $500/month. According to https://example.com/pricing"

Actual page: https://example.com/plans/enterprise (different URL) Customer clicks cited link: Gets 404 Customer goes to correct page: Sees price is $5,000/month (not $500)

Customer complaint: "Agent told me $500, but it's actually $5,000! Agent lied! I'm charging back / leaving."

Your liability:

• Refund customer? (maybe)
• Deal with chargeback? (yes, $15 fee + investigation)
• Reputation damage? (yes, 1-star review)
• Legal action? (possible, if pattern emerges)

Scenario 2: Hallucinated feature

Your SaaS agent: "We have native Slack integration. According to https://example.com/features/slack"

Actual: You don't have Slack integration Agent: HALLUCINATED the feature + URL Customer: Buys plan expecting Slack integration Customer: "Where's the Slack integration?" You: "We don't have it. Agent was wrong." Customer: Refund request + bad review

Your liability:

• Lost sale ($$$)
• Chargeback fee ($15)
• Reputation damage (review)
• Potential FTC investigation (false advertising)

Scenario 3: False compliance claim

Your SaaS agent: "We're GDPR compliant. According to https://example.com/compliance/gdpr"

Actual:

• You're not GDPR compliant
• Page doesn't exist
• Agent hallucinated

Customer (EU company): "You told me you're GDPR compliant! I signed contract based on this! Now GDPR fines are MY problem, not yours!"

You: Sued for:

• Contract breach ($$$)
• Indemnification of GDPR fine ($$$)
• Damages ($$$)

Your liability:

• GDPR fine: Up to €20M (or 4% revenue, whichever higher)
• Customer's indemnification: Up to €20M
• Legal fees: $500k+
• Total exposure: $50M+ (single customer lawsuit)

Scope of liability (all customer-facing claims)

If your agent cites sources to customers for: ☐ Pricing information → Liability if citation wrong ☐ Feature descriptions → Liability if citation wrong ☐ Support policies → Liability if citation wrong ☐ Terms of service → Liability if citation wrong ☐ Compliance claims → Liability if citation wrong ☐ SLA guarantees → Liability if citation wrong ☐ Security features → Liability if citation wrong

Base risk: Customer sees wrong citation → Doesn't trust Escalated risk: Customer acts on wrong info → Files lawsuit Max risk: Pattern of hallucinations → FTC investigation

---

The verification problem (cost of accuracy)

How to fix (verification layers)

Option 1: Don't cite (safest)

Agent response (no citations): "Our Enterprise plan includes 24/7 support."

Advantages:

• No hallucinations (can't cite wrong URL)
• No liability (no citation = no false citation)
• Simple

Disadvantages:

• Less trustworthy (customers want sources)
• Customer can't verify (might not believe)
• Competitive disadvantage (agents WITH citations seem better)

Option 2: Cite only static, verified URLs

Agent only cites:

• URLs you explicitly approved
• Stored in database
• Known to be accurate
• Updated when pages change

Example: Database: PRICING_URL: "https://example.com/plans" FEATURES_URL: "https://example.com/features" TERMS_URL: "https://example.com/legal/terms"

Agent: "For details, see [PRICING_URL]" Agent generates: "For details, see https://example.com/plans"

Advantages:

• Citations are always accurate (verified)
• No hallucinations (only approved URLs)
• Customers can trust links
• No liability (citations are correct)

Disadvantages:

• Cost: Maintain URL database ($5k-$20k setup)
• Maintenance: Update when URLs change ($2k/month)
• Limited: Can only cite approved URLs
• Rigid: Can't cite new pages (requires approval)

Option 3: Real-time verification (before showing)

Agent pipeline:

1. Agent generates response + cites URL
2. Verification system checks:
   • Does URL exist? (HTTP request)
   • Does page contain cited content? (text match)
   • Is citation accurate? (semantic match)
3. If verified:
   • Send to customer
4. If NOT verified:
   • Remove citation
   • Send response without link
   • Log for review

Advantages:

• Citations are always verified (accurate)
• No hallucinations (system checks before deploy)
• Customers trust links (real verification)
• No liability (verified citations only)

Disadvantages:

• Cost: Verification system ($20k-$50k build + $5k/month ops)
• Latency: Takes extra time (verification requests slow down responses)
• Complex: Requires hosting verification system
• False negatives: Sometimes rejects valid citations

Option 4: Hybrid (best balance)

Agent pipeline:

1. Agent cites URL
2. Check if URL is in approved database
   • If YES: Send (we trust this URL)
   • If NO: Go to step 3
3. Verify URL in real-time
   • If valid: Send
   • If invalid: Remove citation, send without link

Advantages:

• Speed: Approved URLs skip verification (fast)
• Accuracy: Unknown URLs verified (safe)
• Cost: Lower than full verification (approved URLs cached)
• Trust: Customers trust citations (either approved or verified)

Disadvantages:

• Moderate complexity
• Moderate cost ($10k-$30k setup, $3k/month ops)
• Requires maintenance (update approved database)

Cost comparison (verification options)

Option 1: No citations Setup: $0 Monthly: $0 Risk: High (customers don't trust) Liability: Low (no citations = no false citations) Total cost/year: $0 + reputational damage

Option 2: Static approved URLs Setup: $10k (build database + integrate) Monthly: $2k (maintain, update) Risk: Medium (limited to approved URLs) Liability: Very low (always accurate) Total cost/year: $10k + (12 × $2k) = $34k

Option 3: Real-time verification Setup: $40k (build verification system) Monthly: $5k (hosting, maintenance) Risk: Low (all citations verified) Liability: Very low (all citations verified) Total cost/year: $40k + (12 × $5k) = $100k

Option 4: Hybrid (recommended) Setup: $20k (build + integrate both systems) Monthly: $3k (maintenance, updates) Risk: Low (approved URLs cached, others verified) Liability: Very low (all citations safe) Total cost/year: $20k + (12 × $3k) = $56k

Comparison: No cost approach: $0/year but high reputational risk Budget approach: $34k/year (approved URLs only) Recommended: $56k/year (hybrid = best balance) Enterprise: $100k/year (full verification = maximum safety)

---

Your action plan (citation safety)

Week 1: Audit

☐ Check: Does your agent cite sources?

• Review 100 recent agent responses
• Document what's cited
• Check if citations are accurate
• Identify hallucinations

☐ Check: What happens when customer clicks citation?

• Click cited links
• Does page match what agent said?
• Are citations accurate?
• Are there broken links (404s)?

☐ Assess: Liability exposure

• What claims does agent make?
• Are citations for critical info (pricing, compliance, features)?
• What's worst-case scenario if citation is wrong?
• Do you have insurance that covers this?

Week 2: Decision

☐ Choose: What level of verification do you need? Option A: Stop citations (safest, lowest trust) Option B: Approved URLs only (balanced) Option C: Real-time verification (highest trust, highest cost) Option D: Hybrid (recommended)

☐ Calculate: What's your risk vs cost tradeoff?

• If citations are for non-critical info: Option A or B
• If citations are for pricing/features: Option B or D
• If citations are for compliance/legal: Option D or C

☐ Get approval: Legal/executive review

• Show options to lawyer
• Show cost analysis
• Get recommendation
• Get executive approval

Week 3: Implementation

If Option B (Approved URLs): ☐ Build database of approved URLs - List all pages customers might need - Verify each URL is accurate - Store in database - Write documentation

☐ Update agent: - Modify agent to only cite approved URLs - Add fallback (if URL not approved, don't cite) - Test extensively - Deploy

☐ Maintenance plan: - When content changes, update database - Monthly audit of URLs - Test citations quarterly

If Option D (Hybrid): ☐ Build approved URL database (same as above) ☐ Build real-time verification system - Check if URL exists (HTTP 200) - Check if content matches claim (text similarity) - Cache results (10-minute TTL) - Log all verifications

☐ Update agent: - Agent cites URL - Check approved database - If approved: Send (skip verification) - If not approved: Verify in real-time - If verified: Send - If not verified: Remove citation, send response without link

☐ Testing: - Test with known good URLs - Test with broken URLs - Test with hallucinated URLs - Test latency impact

Ongoing: Monitoring

☐ Weekly: Review agent citations

• Sample 20 agent responses
• Click each citation
• Verify accuracy
• Track any failures

☐ Monthly: Citation accuracy report

• How many citations total?
• How many are accurate?
• How many are broken (404)?
• How many are hallucinated?
• Accuracy rate: Should be 95%+

☐ Quarterly: Verification system audit

• Is verification system working?
• Are false positives/negatives acceptable?
• Cost analysis (is it worth it?)
• Improvement opportunities

---

Conclusion: OpenAI API citations aren't ready (fix now)

What the paper showed:

OpenAI API citations:

• Don't match exactly (domain matches, path wrong)
• Sometimes hallucinate (URLs don't exist)
• Unreliable for production use
• ~70% accurate at best

If you use in production:

• Customers see wrong citations
• Customers lose trust
• You get liability
• Reputation damage

What to do (immediately):

☐ Audit: Are your agent citations broken? ☐ Assess: What's your liability exposure? ☐ Choose: Verification approach (B, C, or D recommended) ☐ Build: Implement verification system ☐ Monitor: Ongoing citation accuracy

Cost: $20k-$100k/year (depending on approach) Benefit: Eliminate citation liability + customer trust ROI: Prevents lawsuit (could be $100k-$10M+)

Bottom line:

If your agent cites sources:

• Verify they're accurate
• Build verification system
• Monitor continuously
• Document everything (legal protection)

If you don't:

• Customer sees hallucinated citation
• Customer loses trust
• Customer leaves + bad review
• Competitor with verified citations wins

→ OpenClaw: Agent citation verification, accuracy checking, hallucination prevention, real-time verification system, approved URL database, hybrid verification architecture

Seu agent cita links errados? Implementa verificação ANTES que customer clique em 404 e sai 🚨