Attention kernel optimization
Technique · Architecture & Efficiency · 1 item since Apr 2026
Hand-tuned GPU kernels that compute attention faster on specific hardware.
Related
- Organizations
Items
- Mar 5
FlashAttention-4 for Blackwell GPUs
Princeton, Together AI, Meta, and NVIDIA redesign attention for Blackwell, where tensor-core throughput outpaces memory bandwidth, reaching 1,613 TFLOPs/s (71% utilization) on B200, 1.3x faster than cuDNN 9.13 and 2.7x faster than Triton.