The proxy
you don't have to write.

One wrangler secret put and one URL change. The Worker code from the direct-fetch pattern still works — the gateway forwards the request to Azure OpenAI with the api-key header you set, returns the body, and logs every request and response. Open the AI Gateway dashboard and there are graphs by model, deployment, latency, and cost.

// Before — direct fetch (no observability)
const url = `${env.AOAI_ENDPOINT}/openai/deployments/${env.AOAI_DEPLOYMENT}/chat/completions?api-version=2025-01-01-preview`;

// After — same shape, gateway in front
const url = `https://gateway.ai.cloudflare.com/v1/${env.CF_ACCOUNT_ID}/${env.GATEWAY_ID}/azure-openai/${env.AOAI_RESOURCE}/${env.AOAI_DEPLOYMENT}/chat/completions?api-version=2025-01-01-preview`;

// Body, headers, auth — completely unchanged
const res = await fetch(url, {
  method: 'POST',
  headers: { 'api-key': env.AOAI_KEY, 'Content-Type': 'application/json' },
  body: JSON.stringify({ messages, max_tokens: 800 }),
});

Idempotent prompts (RAG-grounded Q&A, classification, structured extraction) repeat. The gateway hashes the request body and serves a cached response when the next identical request lands. You opt in per-request via the cf-aig-cache-ttl header — set 0 for chat conversations where every response should be fresh, set 3600 for "what's the return policy" prompts where the answer hasn't changed in a year. No KV setup, no hash function, no sha256 on the hot path.

const res = await fetch(url, {
  method: 'POST',
  headers: {
    'api-key': env.AOAI_KEY,
    'Content-Type': 'application/json',
    'cf-aig-cache-ttl': '3600',            // 1 hour
    'cf-aig-cache-key': tenantId,             // scope cache per tenant
  },
  body: JSON.stringify({ messages, max_tokens: 800, temperature: 0 }),
});

// Inspect the headers to see if it was a hit
const hit = res.headers.get('cf-aig-cache-status');  // "HIT" | "MISS" | "BYPASS"

SA North's tighter quota means 429s are routine. With direct fetch you write the retry loop yourself (see concept 03 there). With AI Gateway, you set retry headers on the request and the gateway honours retry-after-ms from Azure, falls back to exponential backoff, and only surfaces the failure to your Worker if all attempts fail. The Worker stays simple; the resilience lives in the gateway.

const res = await fetch(url, {
  method: 'POST',
  headers: {
    'api-key': env.AOAI_KEY,
    'Content-Type': 'application/json',
    'cf-aig-max-attempts': '3',
    'cf-aig-retry-delay': '500',             // initial ms
    'cf-aig-backoff': 'exponential',
  },
  body: JSON.stringify({ messages }),
});

// Inspect what the gateway did
const attempts = res.headers.get('cf-aig-attempts');   // "1" | "2" | "3"

For a multi-tenant SaaS, the gateway's per-key rate limit is the cleanest abstraction. Set a daily token budget per tenant in the dashboard; the gateway enforces it without your Worker tracking counters. For resilience against an Azure OpenAI region outage, configure a fallback — the gateway tries SA North first, falls back to West Europe on 5xx, and surfaces a header telling the Worker which provider answered. Useful for non-regulated workloads where availability outranks strict residency.

// Tenant identity → cache key + rate limit bucket
const res = await fetch(url, {
  method: 'POST',
  headers: {
    'api-key': env.AOAI_KEY,
    'cf-aig-metadata': JSON.stringify({ tenantId, userId }),
    'cf-aig-cache-key': tenantId,
  },
  body: JSON.stringify({ messages }),
});

// Rate limit hit → 429 from the gateway, before AOAI is even called
if (res.status === 429 && res.headers.get('cf-aig-source') === 'rate-limit') {
  return new Response('Tenant quota exceeded', { status: 429 });
}

// Fallback: gateway tried SA North, failed, served from West Europe
const answeredBy = res.headers.get('cf-aig-provider');  // "azure-openai-sa" | "azure-openai-eu"