All posts in the Lectures
A CUDA kernel becomes a real PyTorch operator only when tensor contracts, runtime semantics, and integration details are handled correctly
A custom operator is not complete until its schema, dispatch behavior, and meta-level shape logic are defined clearly
Backward design is really a question about what to save, what to recompute, and how to preserve correct semantics
Fusion is valuable when it reduces memory traffic and intermediate materialization, not just when it reduces the number of visible ops
A production-quality custom operator has to behave correctly under mixed precision, not just benchmark well in isolation
The purpose of internals knowledge is to make a performance trace interpretable enough that you can actually change it
PyTorch is no longer only an eager framework; compiler paths are now an important part of its optimization story
Triton is not just a convenient kernel language; it is part of the modern PyTorch kernel and compilation story
DDP and FSDP are not external magic; they depend directly on autograd timing and tensor-state management inside the runtime