undefined min read
GPU Systems 10 - Tiled Matrix Multiplication and Shared Memory
Why tiled matrix multiplication and shared memory create such a big performance difference
Posts tagged "gpu"
undefined min read
GPU Systems 11 - Shared Memory Bank Conflicts
Why shared memory is not automatically fast and how bank conflicts appear
undefined min read
GPU Systems 12 - Warp Shuffle and Warp-Level Primitives
Why warp-level primitives matter for reductions and lighter-weight cooperation
undefined min read
GPU Systems 13 - Reduction Kernels in Depth
Using reduction kernels to connect shared memory, warp primitives, and synchronization
undefined min read
GPU Systems 14 - Why Softmax Is Such a Good Kernel Exercise
How softmax combines reductions, memory traffic, and numerical stability in one kernel
undefined min read
GPU Systems 15 - LayerNorm and RMSNorm Kernel Structure
Why normalization kernels are often memory-bound and structurally important
undefined min read
GPU Systems 16 - Vectorized Loads, Stores, and Alignment
How wider memory operations and alignment affect bandwidth utilization
undefined min read
GPU Systems 17 - Register Pressure and Spilling
Why using more registers can improve local efficiency but still reduce total throughput
undefined min read
GPU Systems 18 - Tensor Cores and Mixed Precision
How tensor cores change performance in compute-heavy kernels and why mixed precision matters