Azure

Fifth post in the series. In the previous one, we went inside the GPU. This time we automate everything around it. Understanding GPUs is useful. Provisioning them consistently and at scale is where infrastructure engineering actually meets AI. tl;dr IaC is the only sane way to provision expensive GPU infrastructure repeatedly. Validate SKU choices, remote state, and deployment guardrails before apply. If the pipeline uses OIDC, Terraform and GitHub Actions both need explicit OIDC settings. The $4,000 typo I started the week with a win. I manually provisioned a GPU cluster in East US 2 for an ML experiment: AKS with a Standard_NC6s_v3 node pool, accelerated networking, GPU drivers, correct taints. It took most of a day, but it worked. ...

Fourth post in the series. In the previous one, you learned which GPU VMs to provision and how to connect them. This time we look inside the GPU so you can troubleshoot better and talk to the ML team without guessing. tl;dr GPU memory is consumed by more than model weights. Gradients, optimizer states, and activations usually dominate training memory. Understanding memory hierarchy and topology makes troubleshooting much faster. The 2 AM ticket Slack fires at 2 AM. The ML team’s training job crashed again. The error is a single line: ...

Third post in the series where I translate AI into the language of people who live and breathe infrastructure. In the previous post, we talked about the storage bottleneck nobody notices until it hurts. This one is about compute. Spoiler: it is not enough to buy the most expensive GPU. You need the right GPU, connected the right way. tl;dr Pick hardware based on training vs inference, not on sticker price. GPU family, memory size, and interconnect matter more than vCPU count. For distributed jobs, quota, availability, and network fabric decide whether the cluster performs or stalls. The story you don’t want to live The ML team asks for “a GPU cluster for training.” You do what any infra engineer would do under time pressure: provision eight Standard_D64s_v5 VMs. Sixty-four vCPUs each, 256 GiB of RAM, Premium SSD. On paper, it looks respectable. ...

This is the second post in the series where I translate AI into the language of infrastructure engineers. In the first post, I showed that AI is just another workload and that your infra skills already prepare you more than you think. Now for the bottleneck everyone ignores: storage. It is the hidden villain behind performance problems in almost every AI project I’ve seen. tl;dr Storage is usually the first bottleneck in AI training. Keep durable data in Blob or Data Lake, but cache the active working set on local NVMe. Check GPU, disk, and network metrics together before blaming the model or the GPUs. The midnight call You did everything right. The ML team asked for a GPU cluster and you delivered: eight NVIDIA A100s across two nodes, high-bandwidth networking, CUDA drivers up to date. Clean deployment. The team kicked off their first training job Friday at 6 PM and you went home feeling good. ...

This is the first post in a series where I’ll translate the world of AI into the language that infrastructure engineers already speak. If you’re the kind of professional who configures VMs, builds CI/CD pipelines, and gets woken up at 2 AM when Nagios fires, this content is for you. The series is based on my open-source book AI for Infrastructure Professionals, adapted and expanded here on the blog. tl;dr AI is another workload with different performance, cost, and data patterns. Infrastructure engineers already own the hard parts: compute, networking, security, observability, and cost control. If you can run production infrastructure well, you already have the foundation to support AI systems. The Monday morning message It’s 8:47 AM on a Monday. You’re halfway through your coffee, reviewing a Terraform plan for a network redesign, when a Slack message lights up your screen. It’s from the data science team lead: ...

This article was originally published at https://cloud.redhat.com/experts/aro/private-cluster/ A Quickstart guide to deploying a Private Azure Red Hat OpenShift cluster. Prerequisites Azure CLI Obviously you’ll need to have an Azure account to configure the CLI against. MacOS See Azure Docs for alternative install options. Install Azure CLI using homebrew brew update && brew install azure-cli Install sshuttle using homebrew brew install sshuttle Linux See Azure Docs for alternative install options. Import the Microsoft Keys sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc Add the Microsoft Yum Repository cat << EOF | sudo tee /etc/yum.repos.d/azure-cli.repo [azure-cli] name=Azure CLI baseurl=https://packages.microsoft.com/yumrepos/azure-cli enabled=1 gpgcheck=1 gpgkey=https://packages.microsoft.com/keys/microsoft.asc EOF Install Azure CLI sudo dnf install -y azure-cli sshuttle Prepare Azure Account for Azure OpenShift Log into the Azure CLI by running the following and then authorizing through your Web Browser az login Make sure you have enough Quota (change the location if you’re not using East US) az vm list-usage --location "East US" -o table See Addendum – Adding Quota to ARO account if you have less than 36 Quota left for Total Regional CPUs ...

When working with development or test environments in Azure, a common need is secure access to internal resources without exposing them directly to the internet. While VPN solutions are a robust way to achieve this, they can often be overkill for simple use cases, especially when you just want to access a few VMs or services for testing. A jump host combined with sshuttle offers a simple, VPN-like solution that can be quickly deployed and used to tunnel traffic to your Azure resources—without the overhead of setting up a full VPN. ...

This article was originally published at https://cloud.redhat.com/experts/aro/acm-odf-aro/ A guide to deploying Advanced Cluster Management (ACM) and OpenShift Data Foundation (ODF) for Azure Red Hat OpenShift (ARO) Disaster Recovery. Overview VolSync is not supported for ARO in ACM: https://access.redhat.com/articles/7006295 so if you run into issues and file a support ticket, you will receive the information that ARO is not supported. In today’s fast-paced and data-driven world, ensuring the resilience and availability of your applications and data has never been more critical. The unexpected can happen at any moment, and the ability to recover quickly and efficiently is paramount. That’s where OpenShift Advanced Cluster Management (ACM) and OpenShift Data Foundation (ODF) come into play. In this guide, we will explore the deployment of ACM and ODF for disaster recovery (DR) purposes, empowering you to safeguard your applications and data across multiple clusters. ...

This article was originally published at Configure ARO to use Microsoft Entra ID Group Claims | Red Hat Cloud Experts This guide demonstrates how to utilize the OpenID Connect group claim functionality implemented in OpenShift 4.10. This functionality allows an identity provider to provide a user’s group membership for use within OpenShift. This guide will walk through the creation of an Azure Active Directory (Azure AD) application, configure the necessary Azure AD groups, and configure Azure Red Hat OpenShift (ARO) to authenticate and manage authorization using Azure AD. ...

This article was originally published at ARO with Nvidia GPU Workloads | Red Hat Cloud Experts ARO guide to running Nvidia GPU workloads. Prerequisites oc cli Helm jq, moreutils, and gettext package An ARO 4.14 cluster Note: If you need to install an ARO cluster, please read our ARO Terraform Install Guide. Please be sure if you’re installing or using an existing ARO cluster that it is 4.14.x or higher. Note: Please ensure your ARO cluster was created with a valid pull secret (to verify make sure you can see the Operator Hub in the cluster’s console). If not, you can follow these instructions. ...