reviews: 
Comments: This paper addressed the flexibility of the sparse attention pattern to various input length and task currently applied in LLM. The authors split the attention heads into two types: the Query-Aware heads that capture variable sparse pattern upon query, and the Vertical-Slash heads that capture structured and invariant sparse patterns observed in pervious works. The authors decide the sparse pattern for each head by evaluating the discrepancy between the estimated and true attention distribution, and select the sparse pattern by summing up the largest attention scores to a threshold. The proposed method balances the computational efficiency with attention accuracy. The authors evaluate their approach on RULER and Infinite Bench and show that their method preserves high performance across multiple context length and tasks.
Strengths: - The paper propose a more dynamic and flexible approach for sparse attention calculation. Their method performs sparse pattern search and sparse index selection both in a dynamic, query-aware manner. The flexibility and dynamism distinct it from MInference, and contribute to the performance improvement.
- The method is based on solid observations. 
- The paper gives a concrete and simple algorithm to construct these sparse attention patterns.
- The "Cumulative-Attention Based Index Selection" method is convincing with the details discussed in Appendix A. The authors derive the algorithm from the original  multi-objective optimization problem in details, providing the rationale in summing up the largest attention scores.
Weaknesses: Methodology:

- **Insufficient Comparison with Baselines**: There are also other sparse attention method in recent works such as SampleAttention [1] and HiP-Attention [2]. The drawbacks of StreamingLLM in handling longer contexts is obvious in other papers, it would be better to compare to other more recent methods.
- **Insufficient evaluation of latency**: Although "Cumulative-Attention Based Index Selection" improves performance, it may result in unbalanced computation allocation between different attention heads. I think this could lead to an overall increase in latency, as MInference explicitly avoids this by setting a target FLOPs for each pattern. I think it is necessary to show to what extend the dynamic, head-distinctive computation allocation increases the end-to-end latency compared to the fixed budget setting within the same FLOPs.
- **Determination of attention patterns**: The algorithm select the vertical-slash pattern when the distribution of estimated and true attention is not that close. However, in my opinion, the distribution discrepancy only indicates that the result of block-wise sparse attention deviates the full attention, but doesn't lead to using the vertical-slash patterns. Is there any theoretical or empirical guarantee that vertical flash patterns are better than block-wise pattern in this scenario?

Presentation:

- "eacct" -> "exact" in Line 1,043.
- The caption of Table 5 should be above the table.
- In Figure 4, it would be more clear to show the values of $\gamma$ for each point, which will be helpful to provide an empirical selection of $\gamma$. 
- The authors states "accelerating the computational process" in Line 357 but Table 1 does not include any result concerning efficiency. It would be better to give a reference to figure 4 here. 

[1] Zhu, Q., Duan, J., Chen, C., Liu, S., Li, X., Feng, G., ... & Yang, C. (2024). Near-lossless acceleration of long context llm inference with adaptive structured sparse attention. *arXiv preprint arXiv:2406.15486*.

[2] Lee, H., Park, G., Lee, Y., Kim, J., Jeong, W., Jeon, M., & Hwang, S. J. (2024). HiP Attention: Sparse Sub-Quadratic Attention with Hierarchical Attention Pruning. *arXiv preprint arXiv:2406.09827*.
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3