GPU deep dive: what happens inside the silicon
Fourth post in the series. In the previous one, you learned which GPU VMs to provision and how to connect them. This time we look inside the GPU so you can troubleshoot better and talk to the ML team without guessing. tl;dr GPU memory is consumed by more than model weights. Gradients, optimizer states, and activations usually dominate training memory. Understanding memory hierarchy and topology makes troubleshooting much faster. The 2 AM ticket Slack fires at 2 AM. The ML team’s training job crashed again. The error is a single line: ...