Azure OpenAI
from a Worker.

Azure OpenAI uses the same REST shape as OpenAI's own API. Different base URL, an api-version query parameter, and an api-key header instead of Authorization: Bearer. That's the entire delta. From a Worker, you don't even need an SDK — fetch() is enough, and the response body is already a ReadableStream ready to hand back.

// src/index.ts — the smallest useful Worker that calls SA-North gpt-4o
type Env = {
  AOAI_ENDPOINT: string;     // https://aoai-imbila-sa.openai.azure.com
  AOAI_DEPLOYMENT: string;   // gpt-4o-za
  AOAI_KEY: string;          // wrangler secret put AOAI_KEY
};

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    const { messages } = await req.json();
    const url = `${env.AOAI_ENDPOINT}/openai/deployments/${env.AOAI_DEPLOYMENT}/chat/completions?api-version=2025-01-01-preview`;

    const aoai = await fetch(url, {
      method: 'POST',
      headers: { 'api-key': env.AOAI_KEY, 'Content-Type': 'application/json' },
      body: JSON.stringify({ messages, max_tokens: 800, temperature: 0.3 }),
    });

    return new Response(aoai.body, { status: aoai.status, headers: aoai.headers });
  }
};

Chat UX needs first-token-fast — the user wants to see the response start within ~300 ms of pressing send, not wait for the whole completion. Set stream: true on the request, and Azure responds with text/event-stream chunks. A Worker doesn't need to parse them. The ReadableStream from fetch() can be returned directly as the response body — Workers does no buffering, so the user's browser sees tokens as fast as they leave Azure.

// Same handler, streaming variant
const aoai = await fetch(url, {
  method: 'POST',
  headers: { 'api-key': env.AOAI_KEY, 'Content-Type': 'application/json' },
  body: JSON.stringify({
    messages,
    stream: true,
    max_tokens: 800,
    stream_options: { include_usage: true },  // final chunk has token counts
  }),
});

// Pass the SSE stream straight through. No JSON parsing, no buffering.
return new Response(aoai.body, {
  headers: {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache, no-transform',
    'X-Accel-Buffering': 'no',
  },
});

SA North's default token-per-minute quota is roughly 30–50% of the equivalent quota in East US. Burst traffic that auto-scales fine in US regions hits 429s here. Azure OpenAI returns a retry-after-ms header on throttle responses — honour it, fall back to exponential backoff if the header is missing, and cap retries at three. For idempotent prompts, layer KV caching on top so repeats absorb at zero cost.

async function chatWithRetry(
  env: Env, messages: unknown, attempt = 0
): Promise<Response> {
  const url = `${env.AOAI_ENDPOINT}/openai/deployments/${env.AOAI_DEPLOYMENT}/chat/completions?api-version=2025-01-01-preview`;
  const res = await fetch(url, {
    method: 'POST',
    headers: { 'api-key': env.AOAI_KEY, 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages, max_tokens: 800 }),
  });

  if (res.status === 429 && attempt < 3) {
    const hdr = res.headers.get('retry-after-ms');
    const wait = hdr ? Number(hdr) : 500 * 2 ** attempt;  // 500, 1000, 2000
    await new Promise(r => setTimeout(r, wait));
    return chatWithRetry(env, messages, attempt + 1);
  }
  return res;
}

// Idempotent prompt? Hash + KV.
const key = `aoai:${await sha256(JSON.stringify(messages))}`;
const cached = await env.CACHE.get(key);
if (cached) return new Response(cached, { headers: { 'Content-Type': 'application/json' } });

const fresh = await chatWithRetry(env, messages);
const body = await fresh.text();
await env.CACHE.put(key, body, { expirationTtl: 3600 });
return new Response(body);

API keys are fine for a prototype. For production — especially SARB-strict workloads — the better pattern is OAuth client credentials with a service principal in Entra ID. The Worker fetches an access token from login.microsoftonline.com, caches it in KV until ~5 minutes before expiry, and presents it as Authorization: Bearer <token> on each AOAI call. Rotate the client secret in Key Vault without redeploying the Worker. Workers can't have an Azure managed identity directly — this is the closest equivalent.

async function getAzureToken(env: Env): Promise<string> {
  const cached = await env.CACHE.get('aoai:token');
  if (cached) return cached;

  const res = await fetch(
    `https://login.microsoftonline.com/${env.AZURE_TENANT_ID}/oauth2/v2.0/token`,
    {
      method: 'POST',
      headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
      body: new URLSearchParams({
        client_id: env.AZURE_CLIENT_ID,
        client_secret: env.AZURE_CLIENT_SECRET,
        scope: 'https://cognitiveservices.azure.com/.default',
        grant_type: 'client_credentials',
      }),
    }
  );
  const { access_token, expires_in } = await res.json() as any;

  // Cache 5 min short of expiry — Entra tokens are 60-min by default
  await env.CACHE.put('aoai:token', access_token, {
    expirationTtl: expires_in - 300,
  });
  return access_token;
}

// Use it on the AOAI call
const token = await getAzureToken(env);
const aoai = await fetch(url, {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ messages }),
});