Compute for AI: choosing the right hardware (and connecting it properly)
Third post in the series where I translate AI into the language of those who live and breathe infrastructure. In the previous post, we talked about the hidden storage bottleneck. Today we’re going to what everyone thinks is the main topic of AI: compute. Spoiler: it’s not just about having the most expensive GPU. It’s about having the right GPU, connected the right way. The story you don’t want to live The ML team asks for “a GPU cluster for training.” You do what any infra engineer would: provision eight Standard_D16s_v5 VMs. Sixty-four vCPUs each, 128 GiB of RAM, premium SSD. On paper, plenty of power. ...