Sre

Chapter 2: The Deterministic 429 Watchdog In the previous post I explained what MCP is and how an agent decides its next move from the tools available. Now for something you could actually ship over a weekend: an MCP server that watches token consumption on your Azure OpenAI or Foundry deployment and warns you on Slack or email before the 429 lands in production. Why this is subtler than it looks The first reaction from anyone who’s never been bitten by a 429 is “easy, just measure usage and compare it to the quota.” The problem is that TPM (tokens per minute) and RPM (requests per minute) on Azure OpenAI are evaluated over rolling windows, on short intervals, typically 1 to 10 seconds, not a smooth average across the minute. That means you can blow the limit even while staying “under quota” in aggregate, simply because requests arrived in a burst instead of spread out. That’s why teams report 429s “even within the documented limit”: the problem isn’t total volume, it’s distribution over time. ...