Azure-Openai

Twelfth post in the series. In the previous one, we operated Azure OpenAI with HA and correct retry patterns. Now: when things break (and they will break). This post is organized as real-world failure scenarios. Each follows: Symptoms → Diagnosis → Root Cause → Resolution → Prevention. Read it once for pattern recognition. Then bookmark it; you’ll be back. Scenario 1: NVIDIA driver crash after kernel update Symptoms Monday morning. The ML team reports that all GPU workloads failed over the weekend. Nobody deployed anything. You SSH in: ...

Eleventh post in the series. In the previous one, we built the self-service AI platform with multi-tenancy and scheduling. Now: the service everyone wants to consume, Azure OpenAI, and how to operate it without getting 429’d in the face. The 429 that changed everything Your team launched an internal GPT-4o chatbot on Monday. Day 1: smooth sailing, demos for leadership, Slack full of praise. Day 3: “the bot is slow.” Day 5: 30% of requests return HTTP 429. You open Azure Monitor and discover you’re hitting the 80K TPM ceiling. ...

Ninth post in the series. In the previous one, we hardened the platform against prompt injection and data leakage. Now: how not to go bankrupt in the process. The $127,000 Monday Monday morning. Coffee in hand, email from Finance in the subject line: “URGENT: Azure invoice $127,000, please explain.” Forecast was $42,000. Two ND96isr_H100_v5 VMs, provisioned three weeks ago for a “quick experiment,” never shut down. At ~$98/hour each, running 24/7 for three weeks: $33,000 in idle GPU compute. Nobody using them. Nobody remembered they existed. ...

Seventh post in the series. In the previous one, we put models into production with CI/CD pipelines. Now: how do you know they’re actually healthy? The silent failure Your Azure OpenAI endpoint returns 200 OK on every request. Latency is normal, P95 under 800ms. CPU and memory within thresholds. Kubernetes shows healthy pods, no restarts. By every infra metric you trust, the system is perfect. But the support tickets keep coming. Users report the chatbot “gives worse answers.” Fluent but factually incorrect responses. Hallucinations are up, summarizations miss key points, code suggestions introduce subtle bugs. ...

As I embark on my journey of learning about artificial intelligence (AI), I am discovering the fascinating world of large language models (LLMs) and their applications in various technologies. In this article, I aim to share my newfound knowledge and insights with others who are also beginning their journey in AI. We will explore OpenAI, one of the leading organizations in AI research and development, and compare its offerings with Microsoft’s Azure OpenAI service. ...