Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
Published in International Conference on Learning Representations (ICLR), 2026
Tactic is a calibration-free sparse attention mechanism that dynamically selects tokens based on cumulative attention scores.
Recommended citation: Kan Zhu*, Tian Tang*, Qinyu Xu, Zhan Jin, Yile Gu, Zhichen Zeng, Rohan Kadekodi, Liangyu Zhao, Ang Li, Arvind Krishnamurthy, and Baris Kasikci. "Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs." International Conference on Learning Representations (ICLR), 2026.
*Equal contribution.
Download Paper