undefined min read
PyTorch Internals 18 - Where Autograd Meets Distributed Runtime
DDP and FSDP are not external magic; they depend directly on autograd timing and tensor-state management inside the runtime
DDP and FSDP are not external magic; they depend directly on autograd timing and tensor-state management inside the runtime