Ai

Seventh post in the series. In the previous one, we put models into production with CI/CD pipelines. Now: how do you know they’re actually healthy? tl;dr Infra health is not model health. Track GPU, token, application, and answer-quality signals together or you will miss regressions while every dashboard stays green. The silent failure Your Azure OpenAI endpoint returns 200 OK on every request. Latency is normal, P95 under 800ms. CPU and memory within thresholds. Kubernetes shows healthy pods, no restarts. By every infra metric you trust, the system is perfect. ...

Sixth post in the series. In the previous one, we automated GPU cluster provisioning. Next comes what happens after the hardware is ready: how a model goes from “works on my notebook” to “running in production with an SLA.” tl;dr Models need the same artifact, promotion, and rollback discipline as application builds. Use a real registry with metadata and controlled deployments. Prefer MLflow aliases over deprecated stages when describing promotions. The model with no birth certificate A data scientist drops a message in the team channel with a link to a shared drive: “Here’s the model. It’s a 15 GB PyTorch checkpoint. We need it in production by Friday.” ...

Fifth post in the series. In the previous one, we went inside the GPU. This time we automate everything around it. Understanding GPUs is useful. Provisioning them consistently and at scale is where infrastructure engineering actually meets AI. tl;dr IaC is the only sane way to provision expensive GPU infrastructure repeatedly. Validate SKU choices, remote state, and deployment guardrails before apply. If the pipeline uses OIDC, Terraform and GitHub Actions both need explicit OIDC settings. The $4,000 typo I started the week with a win. I manually provisioned a GPU cluster in East US 2 for an ML experiment: AKS with a Standard_NC6s_v3 node pool, accelerated networking, GPU drivers, correct taints. It took most of a day, but it worked. ...

Fourth post in the series. In the previous one, you learned which GPU VMs to provision and how to connect them. This time we look inside the GPU so you can troubleshoot better and talk to the ML team without guessing. tl;dr GPU memory is consumed by more than model weights. Gradients, optimizer states, and activations usually dominate training memory. Understanding memory hierarchy and topology makes troubleshooting much faster. The 2 AM ticket Slack fires at 2 AM. The ML team’s training job crashed again. The error is a single line: ...

Third post in the series where I translate AI into the language of people who live and breathe infrastructure. In the previous post, we talked about the storage bottleneck nobody notices until it hurts. This one is about compute. Spoiler: it is not enough to buy the most expensive GPU. You need the right GPU, connected the right way. tl;dr Pick hardware based on training vs inference, not on sticker price. GPU family, memory size, and interconnect matter more than vCPU count. For distributed jobs, quota, availability, and network fabric decide whether the cluster performs or stalls. The story you don’t want to live The ML team asks for “a GPU cluster for training.” You do what any infra engineer would do under time pressure: provision eight Standard_D64s_v5 VMs. Sixty-four vCPUs each, 256 GiB of RAM, Premium SSD. On paper, it looks respectable. ...

This is the second post in the series where I translate AI into the language of infrastructure engineers. In the first post, I showed that AI is just another workload and that your infra skills already prepare you more than you think. Now for the bottleneck everyone ignores: storage. It is the hidden villain behind performance problems in almost every AI project I’ve seen. tl;dr Storage is usually the first bottleneck in AI training. Keep durable data in Blob or Data Lake, but cache the active working set on local NVMe. Check GPU, disk, and network metrics together before blaming the model or the GPUs. The midnight call You did everything right. The ML team asked for a GPU cluster and you delivered: eight NVIDIA A100s across two nodes, high-bandwidth networking, CUDA drivers up to date. Clean deployment. The team kicked off their first training job Friday at 6 PM and you went home feeling good. ...

This is the first post in a series where I’ll translate the world of AI into the language that infrastructure engineers already speak. If you’re the kind of professional who configures VMs, builds CI/CD pipelines, and gets woken up at 2 AM when Nagios fires, this content is for you. The series is based on my open-source book AI for Infrastructure Professionals, adapted and expanded here on the blog. tl;dr AI is another workload with different performance, cost, and data patterns. Infrastructure engineers already own the hard parts: compute, networking, security, observability, and cost control. If you can run production infrastructure well, you already have the foundation to support AI systems. The Monday morning message It’s 8:47 AM on a Monday. You’re halfway through your coffee, reviewing a Terraform plan for a network redesign, when a Slack message lights up your screen. It’s from the data science team lead: ...

Over the past year, I’ve been working closely with some of the bigger and more visible AI customers in Microsoft’s ecosystem. Large platforms. Fast-moving teams. High expectations. High stakes. On paper, that kind of visibility sounds exciting. In reality, it comes with a weight that’s hard to explain unless you’ve been there. Because being close to impact also means being close to consequence. Visibility changes everything When you work with smaller teams or early-stage projects, mistakes are usually contained. You can recover. You can explain. You can iterate. ...

As I embark on my journey of learning about artificial intelligence (AI), I am discovering the fascinating world of large language models (LLMs) and their applications in various technologies. In this article, I aim to share my newfound knowledge and insights with others who are also beginning their journey in AI. We will explore OpenAI, one of the leading organizations in AI research and development, and compare its offerings with Microsoft’s Azure OpenAI service. ...

As we continue our journey into artificial intelligence (AI), it’s important to understand how AI is transforming different industries and the ethical and legal challenges associated with its widespread adoption. In this new post, we will explore AI’s real-world applications and the complexities of ethical and legal concerns in detail. AI in Healthcare AI is making significant advances in healthcare, improving patient care and medical research: Diagnostics: AI-powered algorithms can analyze medical images, such as X-rays, MRIs, and CT scans, to identify diseases like cancer or fractures with high accuracy. These systems can serve as a second opinion for radiologists, improving diagnostic accuracy and efficiency. Personalized Medicine: AI enables the development of personalized treatment plans based on a patient’s unique genetic makeup. This approach can lead to more effective and targeted therapies, improving patient outcomes. Drug Discovery: AI accelerates the process of discovering new drugs by analyzing vast amounts of data to identify potential compounds and predict their efficacy. This reduces the time and cost associated with bringing new drugs to market. Remote Monitoring: AI-powered wearable devices and remote monitoring tools enable healthcare providers to track patients’ health in real-time, offering proactive care and reducing hospital readmissions. Administrative Efficiency: AI streamlines administrative tasks such as scheduling, billing, and insurance claims processing, freeing up healthcare professionals to focus on patient care. AI in Finance AI is reshaping the finance industry by providing innovative solutions to complex problems: ...