What the New API Management AI Gateway Tier Changes for App Service-Hosted Agents

Azure API Management now has a dedicated AI Gateway tier in public preview. For agents hosted on Azure App Service, the application runtime does not move: App Service still owns the web app, agent orchestration, streaming, deployment lifecycle, scaling, managed identity, and application telemetry. The new tier becomes the governed boundary around models and MCP tools.

The runnable Python sample sends both model and MCP traffic from App Service through AI Gateway. It demonstrates a stable model alias, structured token policy, separate runtime and backend credentials, managed identity to Microsoft Foundry, Key Vault-backed secrets, gateway telemetry, and bounded retry behavior without adding a direct provider fallback.

The full article distinguishes the dedicated tier from AI capabilities in regular APIM tiers and Foundry-associated APIM, then walks through the model and tool paths, tests that prove the gateway is in the request path, the identity boundaries, and the infrastructure deployed with azd.

👉 Read the full article on Tech Community and explore the companion sample on GitHub.