When AI goes wrong

It's all fun and games when the APIs are working. But what happens when they're not?

Outcome

Build error handling that keeps your app useful when AI fails, configure AI Gateway fallbacks, and understand your AI costs.

Fast track

1. Create components/ai-summary-fallback.tsx showing "Customer Reviews" with rating (no AI text)
2. Wrap summarizeReviews() call in try/catch, return <AISummaryFallback /> on error
3. Vercel → AI Gateway → Settings → Add fallback chain: Claude Sonnet → GPT-4 Turbo → Claude Haiku

Hands-on exercise 3.2

Make your AI features production-ready:

Requirements:

1. Create a fallback component for when AI summaries fail
2. Add error handling to the AI summary component
3. Configure model fallbacks in AI Gateway
4. Review your cost dashboard and understand the numbers

Implementation hints:

• React error boundaries catch render errors
• Fallback UI should still be useful (show reviews without summary)
• AI Gateway fallbacks are configured per API key
• Cost tracking helps you predict bills before they arrive

What can go wrong?

AI features fail in ways traditional features don't:

Your App → AI Gateway → Claude API
              ↓
     - Claude outage (503)
     - Rate limit hit (429)
     - Request timeout (>30s)
     - Invalid response
     - Context too long (400)

Common failure scenarios:

Without handling, users see blank screens or cryptic errors. With handling, they see your app—just without the AI parts.

Step 1: Create a fallback component

When AI fails, show something useful instead of an error. Create components/ai-summary-fallback.tsx:

import { Card, CardHeader, CardTitle, CardContent } from "@/components/ui/card";
import { Product } from "@/lib/types";
import { FiveStarRating } from "./five-star-rating";
 
export function AISummaryFallback({ product }: { product: Product }) {
  const averageRating =
    product.reviews.reduce((acc, review) => acc + review.stars, 0) /
    product.reviews.length;
 
  return (
    <Card className="w-full max-w-prose p-10 grid gap-10">
      <CardHeader className="items-center space-y-0 gap-4 p-0">
        <div className="grid gap-1 text-center">
          <CardTitle className="text-lg">Customer Reviews</CardTitle>
          <p className="text-xs text-muted-foreground">
            Based on {product.reviews.length} customer ratings
          </p>
        </div>
        <div className="bg-gray-100 px-3 rounded-full flex items-center py-2 dark:bg-gray-800">
          <FiveStarRating rating={Math.round(averageRating)} />
          <span className="text-sm ml-4 text-gray-500 dark:text-gray-400">
            {averageRating.toFixed(1)} out of 5
          </span>
        </div>
      </CardHeader>
      <CardContent className="p-0 grid gap-4">
        <p className="text-sm leading-loose text-gray-500 dark:text-gray-400">
          Read the reviews below to see what customers are saying about this product.
        </p>
      </CardContent>
    </Card>
  );
}

What changed from the AI version:

• Title says "Customer Reviews" instead of "AI Summary"
• No AI-generated text
• Guides users to read individual reviews
• Same layout, so no jarring UI shift

Step 2: Add error handling to the AI component

Update components/ai-review-summary.tsx to handle failures:

import { Card, CardHeader, CardTitle, CardContent } from "@/components/ui/card";
import { Product } from "@/lib/types";
import { summarizeReviews } from "@/lib/ai-summary";
import { FiveStarRating } from "./five-star-rating";
import { AISummaryFallback } from "./ai-summary-fallback";
 
export async function AIReviewSummary({ product }: { product: Product }) {
  const averageRating =
    product.reviews.reduce((acc, review) => acc + review.stars, 0) /
    product.reviews.length;
 
  let summary: string;
  try {
    summary = await summarizeReviews(product);
  } catch (error) {
    console.error("AI summary failed, showing fallback:", error);
    return <AISummaryFallback product={product} />;
  }
 
  return (
    <Card className="w-full max-w-prose p-10 grid gap-10">
      <CardHeader className="items-center space-y-0 gap-4 p-0">
        <div className="grid gap-1 text-center">
          <CardTitle className="text-lg">AI Summary</CardTitle>
          <p className="text-xs text-muted-foreground">
            Based on {product.reviews.length} customer ratings
          </p>
        </div>
        <div className="bg-gray-100 px-3 rounded-full flex items-center py-2 dark:bg-gray-800">
          <FiveStarRating rating={Math.round(averageRating)} />
          <span className="text-sm ml-4 text-gray-500 dark:text-gray-400">
            {averageRating.toFixed(1)} out of 5
          </span>
        </div>
      </CardHeader>
      <CardContent className="p-0 grid gap-4">
        <p className="text-sm leading-loose text-gray-500 dark:text-gray-400">
          {summary}
        </p>
      </CardContent>
    </Card>
  );
}

Now when AI fails:

1. Error is logged (for debugging)
2. Fallback component renders
3. User still sees useful content
4. No ugly error screens

Step 2b: Add error handling to ReviewInsights

Apply the same pattern to the insights component. Update components/review-insights.tsx:

import { Card, CardHeader, CardTitle, CardContent } from "@/components/ui/card";
import { Product } from "@/lib/types";
import { getReviewInsights } from "@/lib/ai-summary";
 
export async function ReviewInsights({ product }: { product: Product }) {
  let insights;
  try {
    insights = await getReviewInsights(product);
  } catch (error) {
    console.error("AI insights failed:", error);
    return null; // Silently fail - summary is more important
  }
 
  return (
    <Card className="w-full max-w-prose">
      <CardHeader>
        <CardTitle className="text-lg">Key Insights</CardTitle>
      </CardHeader>
      <CardContent className="space-y-6">
        {/* Pros and Cons Grid */}
        <div className="grid md:grid-cols-2 gap-6">
          {/* Pros */}
          <div>
            <h3 className="text-sm font-semibold mb-3 text-green-700 dark:text-green-400">
              Pros
            </h3>
            <ul className="space-y-2">
              {insights.pros.map((pro, i) => (
                <li key={i} className="text-sm flex items-start gap-2">
                  <span className="text-green-600 mt-0.5">✓</span>
                  <span className="text-muted-foreground">{pro}</span>
                </li>
              ))}
            </ul>
          </div>
 
          {/* Cons */}
          <div>
            <h3 className="text-sm font-semibold mb-3 text-red-700 dark:text-red-400">
              Cons
            </h3>
            <ul className="space-y-2">
              {insights.cons.map((con, i) => (
                <li key={i} className="text-sm flex items-start gap-2">
                  <span className="text-red-600 mt-0.5">✗</span>
                  <span className="text-muted-foreground">{con}</span>
                </li>
              ))}
            </ul>
          </div>
        </div>
 
        {/* Themes */}
        <div>
          <h3 className="text-sm font-semibold mb-3">Key Themes</h3>
          <div className="flex flex-wrap gap-2">
            {insights.themes.map((theme, i) => (
              <span
                key={i}
                className="px-3 py-1 bg-gray-100 dark:bg-gray-800 rounded-full text-xs"
              >
                {theme}
              </span>
            ))}
          </div>
        </div>
      </CardContent>
    </Card>
  );
}

Key difference from summary fallback:

• Returns null instead of a fallback component
• Insights are supplementary—if they fail, users still have the summary and reviews
• The page remains functional without showing an error

Step 3: Understand AI Gateway fallbacks

AI Gateway can automatically try a different model when your primary model fails. This happens at the infrastructure level—no code changes needed.

How fallbacks work:

Request → AI Gateway
              ↓
         Try Claude 4.5
              ↓
         [429 Rate Limit]
              ↓
         Try GPT-4 Turbo (fallback)
              ↓
         Success → Return response

Configure fallbacks in Vercel dashboard:

1. Go to AI Gateway → Settings
2. Find Model Fallbacks
3. Configure fallback chain:

Primary:    anthropic/claude-sonnet-4.5
Fallback 1: openai/gpt-4-turbo
Fallback 2: anthropic/claude-haiku-3.5

When fallbacks trigger:

• Primary model returns 429 (rate limit)
• Primary model returns 503 (service unavailable)
• Primary model times out (configurable)

When fallbacks don't trigger:

• 400 errors (bad request—your code needs fixing)
• 401 errors (auth failed—your key is wrong)
• Successful responses (even if the content is poor)

Step 4: Understand your costs

AI Gateway tracks every request. Before your app scales, understand the numbers.

Find your cost dashboard:

1. Vercel Dashboard → AI Gateway
2. Click Usage tab
3. See breakdown by model, day, and request type

What you'll see:

Last 30 days
─────────────────────────────────
Total requests:     1,247
Total tokens:       892,340
Estimated cost:     $2.68

By model:
├─ anthropic/claude-sonnet-4.5
│  ├─ Requests: 1,198 (96%)
│  ├─ Tokens: 856,200
│  └─ Cost: $2.57
│
└─ anthropic/claude-haiku-3.5
   ├─ Requests: 49 (4%)
   ├─ Tokens: 36,140
   └─ Cost: $0.11

Understanding token costs:

See current pricing at:

• Anthropic Pricing
• OpenAI Pricing

As a rough guide, Claude Sonnet is a mid-tier model (~$3/1M input tokens), Claude Haiku is budget-friendly (~10x cheaper), and GPT-4 Turbo is premium (~3x more expensive than Sonnet). Always check official pricing pages for current rates.

Your summary function costs:

• Input: ~600 tokens (prompt + reviews)
• Output: ~100 tokens (summary)
• Cost per summary: ~$0.0021 with Claude Sonnet

At scale:

• 1,000 summaries/month: ~$2.10
• 10,000 summaries/month: ~$21.00
• 100,000 summaries/month: ~$210.00

With caching (from lesson 3.1), most requests hit cache. Real costs are 90-97% lower.

Choosing the right model

Not every AI call needs the best model. Match model capability to task complexity.

When to use Claude Sonnet 4.5 (your current choice):

• Complex analysis (structured insights)
• Nuanced summarization
• When quality directly impacts user trust

When to use Claude Haiku 3.5 (10x cheaper):

• Simple summaries
• Classification tasks
• High-volume, lower-stakes operations

When to use GPT-4o Mini (cheapest):

• Very simple tasks
• Fallback when cost matters more than quality
• Testing and development

Experiment:

Try switching your summary function to Haiku:

const { text } = await generateText({
  model: "anthropic/claude-haiku-3.5", // Was claude-sonnet-4.5
  prompt,
});

Compare the output quality. For short review summaries, you might not notice a difference—at 10x lower cost.

Cost optimization checklist

Before going to production, verify:

• Caching enabled (lesson 3.1) — Most requests should hit cache
• Fallbacks configured — Don't let outages break your app
• Error handling — Graceful degradation when AI fails
• Model selection — Right model for the job, not always the fanciest
• Rate limits understood — Know your provider limits

Estimated monthly cost for a review site:

Caching is the biggest cost lever. Use it.

Try it

1. Test error handling:

   • Temporarily break your API key in .env.local
   • Visit a product page
   • Verify fallback component appears
   • Fix the API key

2. Check your dashboard:

   • Visit Vercel → AI Gateway → Usage
   • See your actual costs so far
   • Note which models you're using

3. Configure fallbacks:

   • AI Gateway → Settings → Model Fallbacks
   • Add at least one fallback model
   • (You won't see it trigger unless your primary fails)

4. Optional: Test a cheaper model:

   • Change model to anthropic/claude-haiku-3.5
   • Generate a few summaries
   • Compare quality vs cost savings

Commit

git add components/ai-summary-fallback.tsx components/ai-review-summary.tsx
git commit -m "feat(ai): add error handling and fallback UI"
git push

Done-when

• Fallback component created
• AI summary component handles errors gracefully
• ReviewInsights component handles errors gracefully
• Understand AI Gateway fallback configuration
• Reviewed cost dashboard
• Know approximate cost per summary
• Understand model tradeoffs (quality vs cost)

What's next

Your AI features handle failures gracefully. But how do you know when something's wrong in production? In the next lesson, you'll set up observability—logging, analytics, and alerts—so you know what's happening before users complain.

Sources:

• Vercel AI Gateway
• Anthropic Pricing
• OpenAI Pricing
• AI SDK Error Handling