GPU Performance Engineer

Prepare for the interview that asks how fast your kernel should run.

GPU performance interviews are unlike the ones most engineers have practised for. You may be asked to write a kernel on a shared screen with no compiler, then say how fast it ought to go before anyone has measured it. You may be shown a profile and asked what you would do next. You may be asked why a model's decode speed barely changed when the team moved to a GPU with twice the FLOP rate.

Algorithm practice does not prepare you for this. Neither do CUDA tutorials, which teach you how to make a kernel work but rarely how to reason about why it is slow.

This book starts from the arithmetic: what a kernel can reach on a named GPU, and why. Then it builds each kernel from a correct naive version to a fast one, one justified step at a time, because those steps are what interviewers want to hear.

Inside you'll find:

• The performance model: roofline, arithmetic intensity, latency hiding, occupancy and the numbers to carry into the room, with worked estimation drills
• CUDA in depth: execution, coalescing, shared memory and bank conflicts, warp-level programming, atomics, streams, tensor cores, and what Hopper and Blackwell changed
• Triton: the programming model, how to make it fast, and when to switch to CUDA
• Version ladders for the kernels interviewers ask for: reduction, scan, transpose, matrix multiplication, softmax, LayerNorm, FlashAttention and more
• Profiling and debugging: a workflow for Nsight Systems and Nsight Compute, correctness bugs, and 20 worked debugging scenarios
• System-level performance: training and inference arithmetic, MFU, decode floors, and 8 open-ended design questions
• Interview practice: over 240 questions with answers, rubrics of strong signals and red flags, and preparation plans from three days to eight weeks
• Reference appendices: hardware tables, CUDA and Triton quick reference, a glossary and further reading

Who it's for: engineers preparing for GPU performance, kernel, inference and ML-compiler roles, including CUDA developers moving into AI, ML engineers moving down the stack, and systems programmers moving across to GPUs. You should be able to write C++ and Python. Book Two of The AI Infrastructure Library; it can be read on its own.