All posts in the Lectures
To reason well about performance, custom operators, and distributed runtime behavior, PyTorch has to be understood as a runtime, not just a Python library
If you think of a tensor only as an n-dimensional array, you will misunderstand views, layouts, and hidden copies
Layout affects both operator selection and performance, and sometimes the most expensive thing in a path is an invisible copy
A single operator name in PyTorch may map to many implementations, and the dispatcher is the runtime layer that decides which one runs
Autograd is not just automatic differentiation; it is a graph-construction and backward-execution runtime
Custom autograd functions are a practical place to define forward-backward contracts before dropping to lower-level extensions
PyTorch GPU memory behavior is shaped by a caching allocator, so observed memory usage is not just a story about current tensor objects
Many PyTorch CUDA operations are asynchronous, so timing, synchronization, and dependency need to be reasoned about explicitly
A C++ extension is the first practical bridge between user-defined logic and the PyTorch runtime