PolarQuant: Leveraging Polar Transformation for Key Cache Quantization and Decoding Acceleration
Authors: Songhao Wu
Ang Lv
xiao feng
Yufei zhang
Xun Zhang
Guojun Yin
Wei Lin
Rui Yan
Venue: NeurIPS 2025 poster ()
Keywords: Quantization
KV Cache
LLM Inference
Primary area: deep_learning

Abstract: The increasing demand for long-context generation has made the KV cache in large language models a bottleneck in memory consumption. Quantizing the cache to lower bit widths is an effective way to reduce memory costs; however, previous methods struggle with key cache quantization due to outliers, resulting in suboptimal performance. We propose a novel quantization approach PolarQuant, which provides a new perspective for key cache quantization and efficiently addresses the outlier dilemma. We observe that the distribution of the key states reveals well-structured patterns under polar transformation. Outliers generally appear in only one of the two dimensions, which are rotated together by a specific angle when rotary position embeddings are applied. When represented as two-dimensional vectors, these dimensions exhibit well-organized patterns, with radii and angles smoothly distributed in polar space. This alleviates the channel-wise outliers, making them well-suited for key cache quantization. PolarQuant divides key vectors into groups of two-dimensional sub-vectors, encoding them as the quantized radius and the polar angle, rather than quantizing original key vectors directly. PolarQuant achieves the superior efficiency in KV cache quantization and accelerates the decoding process by turning the query-key inner product into a table lookup, all while maintaining the downstream performance of full-precision models.
Our code is available at https://github.com/ericshwu/PolarQuant.

PolarQuant Leveraging Polar Transformation for Key Cache Quantization and Decoding Acceleration
Songhao Wu Ang Lv
Xiao Feng²
Yufei Zhang²
Xun Zhang² Guojun Yin Wei Lin Rui Yan
Gaoling School of Artificial Intelligence, Remmin University of China
2 ShanghaiTech University 3 Mcituan
{songhaowu, anglv, ruiyan/@ruc. Craaaeduuuuuuuurrrr. edu.cn fengziao2023ehanghaif (zhangyufei08, zhangxun12, yinguojun02, 1inwei31/@meituan com
edu. cn
Abstract
The increasing demand for long-context generation has made the KV cache large language models bottleneck in memory consumption. Quantizing the cache lower bit widths is an effective way to reduce memory costs; however, previous methods struggle with key cache quantization due outliers, resulting in subopti- mal performance. We propose a novel quantization approach PolarQuant, which provides new perspective for key cache quantization and efficiently addresses the outlier dilemma. We observe that the distribution of the key states reveals well-structured patterns under polar transformation. Outliers generally appear only one of the two dimensions, which are rotated together by a specific angle when rotary position embeddings are applied. When represented vectors, these dimensions exhibit well-organized patterns, with radii and angles smoothly distributed in polar space. This alleviates the channel-wise outliers, mak- ing them well-suited for key cache quantization. PolarQuant divides key vectors into groups two-dimensional sub-vectors, encoding them as the quantized radius and the polar angle, rather than quantizing original key vectors directly. Polar- Quant achieves the superior efficiency in KV cache quantization and accelerates the decoding process by turning the query-key inner product into table lookup, all while maintaining the downstream performance of full prec-sori models. Our code is available at httts:::s/// //t:://gttttt com/ srricchw//ollaarhhaant
wo-diiensiionl
Introduction
Large language models (LLMs) have achieved remarkable success across wide range real- world applications. As these models evolve, the demand for enhanced long- mang-conextt capabilities also increases, especially in tasks like contextual retrieval in question answering [7 and reasoning chains generation for complex reflection and decision-making [21] 20). However, significant challenge in developing long-context L.MSs is the rising memory cost associated with increasing context lengths, which hinders both their practical deployment and further research.
The attention mechanism consumption, which increase rapidly with context length. To reduce this cost, Key-Value cache (KV cache) is common strategy, which stores and reuses keys and values for generation to avoid the
L a major source of computational overhead and memory
"Equal contribution. Work done during Songhao Wu's internship at Meituan. i Cooresponding authors: Rui Yan (riyannruc.eddu. en) and Guojun Yin (yinguo ianiguoouuO2@miituaa.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

PolarQuant
KR 22 1]
K.U]I.
(p.0)
(r.t)
001
011
Magnitude
L
256
128
Token
Channel
128
26
(+) 1aama-31--B--nstrue Layer Hlead Key Cache
(6) Radil Distribution with Structured Carcolar Pattern
(c) Quantiratonn Schema
Figure 1: (a) Illustration of the activation distribution for the key cache, exemplified by Llama 3.1- 8B-Instruct (Layer O, Head O). The key cache exhibits channel-win outliers, where the magnitudes of a few channels significantly larger than others across tokens. We observe that these outliers generally appear in only one of the two dimensions rotated together by RoPE. (b) When viewing these two dimensions in voo-dimensional plane, although the individual X- or y-axis may contain outliers, they collectively form stable circular patterns, making quantization of the original outliers easier. Each blue dot represents mapped wwo-dimensional vector, with transparency indicating frequency. (c) PolarQuant using r = 2 bits to quantize radii and = 3 bits to quantize polar angles. The colorful arrows indicate sub-vectors formed by pairs of dimensions in the keys; the quantized results are shown with colorful dashed arrows. The quantization error is represented by the grey dashed arrow.
redundant computation, Nevertheless, the context length increase, the memory required for KV cache can surpass that of the model weights, making it the dominant factor in overall memory usage.
Many solutions have been proposed to reduce the memory cost of KV cache. Some studies introduce memory-efficient attention modules, such as MQA 255 GQA (2) and MLA (9]. While promising, these module architectures require training LLMs from scratch, which limits their applicability. Another research line focuses on the reduction KV cache size in compatible manner with pre- trained LLMs. This includes techniques like KV cache eviction [7] [5] 36). which identifies and
drops unimportant tokens from the cache and KV cache quantization 12 19 37
This paper focuses on the key cache quantization, which converts the floating foating-point key cache into low- bit integers to reduce memory usage. In general, key cache quantization more challenging than value cache due to the presence of channel-wise distributed outliers. Prior studies [33 [9] have highlighted the widespread existence of such outliers key cache. As shown in Figure Ia), the key states exhibit larger activations along certain channel dimensions, making token-wiss wise quantization difficult. To address this issue, KIVI [19] proposes a channel-win quantization strategy that groups and quantizes key elements along the channel dimensions. Building upon this perspective, KVQuant [I3] further identifies RoPE as the primary source responsible of the outliers observed in the key cache. The rotation operations in RoPE disturb the magnitude consistency, making accurate quantization complicated. To mitigate this, KVQuant proposes quantizing the keys before applying RoPE, which is described pre-RoPE quantization. Promising it is, this approach requires on-the-fly RoPE computation, which consequently introduces potential computational overhead. In this work, we aim preserve the benefits of pre-Re Pre-RoPE quantization in reducing approximation errors while eliminating redundant computations at each generation step. We propose a polar transformation perspective on handling outliers in the key cache, and effectively address the dilemma in 2D polar coordinates.
KVQuant [33 observes clear and structured pattern in pre-RoPE key activations: channel-wise magnitude are highly consistent. Recall that RoPE operates rotation to every two-dimensional
'In this paper, we focus on decoder-only Transformer-bast 29 LLMs using rotary position embedding (RoPE, (26), which the predominant implementation of advanced LLMs.

---

sub-vector within the key using an orthogonal 2 x 2 rotary matrices. Since rotation is a magnitude- preserving transformation, these 2D sub-vectors inherit the magnitude characteristics seen in the pre-RoPF case. As shown in Figure [b), they form well-structured circular patterns when analyzed 2D polar coordinates. By encoding each sub-vector as its corresponding radius p and polar angle 0, the entire key vector can be represented as a collection of all radii and angles. This transformation effectively mitigates outliers, as both the radii and polar angles become smoothly distributed. Building on this, we propose novel quantization method, PolarQuant, which significantly simplifies the quantization of the key cache. PolarQuant reduces the problem of quantizing key vectors asymmetrically quantizing p and 0 into an r-bit and an t-bit integer. Intuitively. PolarQuant defines 2 distinct regions based on 2" angles and 2' radii. Each sub-vector is then encoded by the index of the region it belongs to. Figure T(e illustrates PolarQuant for
and t = 3.
Polar Quant achieves superior quantization effectiveness and efficiency over previous methods. On one hand, polar transformation enables smoother distributions of radii and angles, which alleviates the burden of channel-we quantization outliers. The superior performance on downstream tasks further demonstrates PolarQuant's superiority in quantization error reduction.
On the other hand, PolarQuant offers a brand new perspective on key cache quantization, which enables a novel decoding acceleration method. Unlike pre-RoPE quantization like [T3], PolarQuant eliminates the overhead of RoPE recomputation. In the attention mechanism, it replaces the standard query-key multiplication with inner products between two-dimensional query sub- vectors and a quantized polar coordinate representation of key sub-vectors, which have finite and deterministic states. This transforms matrix multiplication to table lookup, greatly speeding up attention computation. Our contributions are threefold:
(I) We introduce polar transformation for key cache quantization for the first time and derive PolarQuant, novel and efficient quantization approach;
(2) We propose a new decoding acceleration algorithm as a natural byproduct of PolarQuant. We implement custom Triton kernels to perform fused dequantization and query-ke multiplication, which achieves up to 3.18x x speedup in long-contex. generation;
(3) We conduct comprehensive experiments on tasks and models, which further demonstrate the superiority and robustness of our PolarQuant across wide range of model families and tasks.
2 Background
Consider specific Transformer layer where the input hidden states the attention block are denoted as X E RLLDD where L is the sequence length and D is the hidden state dimension. For any attention head, the d-dimensional query, key, and value vectors are obtained by applying three linear transformations to X. Specifically, for each head h, the corresponding computations are as follows:
Q=XWo K
Wv,
where each W. E R Dxd and the resulting variables have shapes of 4xd
The query and key vectors are then applied with RoPE (26) incorporate positional information. For query or key vector at position m E [1, LJ. the corresponding rotary matrix Rm, E Rdxd is defined as:
Tm, dn
0
cos(mmp)) sin(mas)] sin(m):) cos(m);)
Rm,
T' O
Tm.
(I)
O
Tm.daya
where O is a zero matrix, and each Tm.o, is a 2 x 2 orthogonal matrix. Here, Φ, is typically defined as Φ; = b 2i/d/ where b is the hyperparameter for RoPE base.
This formulation encodes the relative distance m between query position m and key at position n into their inner product, as shown by:
(Q.. Rm, (K,, Rr, Q. Rm, n KT
FFor readers unfamiliar with RoPE, please refer Section
3

---

the keys (after applying RoPE) and values, specifically K,, Rn. (which we abbreviate as K,,) and n,, are thus cached to avoid re-computation, known as the KV cache. The KV cache leads to increased memory usage when processing long-context inputs.
Quantization is simple but effective way reduce the KV cache size. To clarify the mechanism, provide brief overview of key-value quantization below. For the value at position we follow the token-wise paradigm
[9] and quantize nn n Ra into b-bit, denoted Q(Vn.).
For an arbitrary dimension S < we have:
(L
V Vn 2n
0.2% -0.2--1) -1).
QQ bi) Clamp
Sp
where on min( Vn (:)) is the zero-point, 8, (inax(VnB) min( (ann(VnBll (.) (2 -1 is the scaling factor. The colon here denotes iteration over all dimensions, following Python indexing syntax. The function Clamp(xr min, mar) restricts integers within the range (min, max)).
Outliers in key states make per-token quantization challenging, as we discussed earlier and illustrated in Figure [a). To address this, previous approaches 19 quantize key vectors channel-wise.
For arbitrary dimension quantized key Q(K,, position is given by:
Qk), = Clamp (L Kn - =
s1abi] [
20.2* - 1)
where the zero-point and scaling factor alternate as:
max
K [i]
min K sgli))
mnbi min ( ( 8j
2
Here, the colon in the subscript denotes iteration over all token positions.
3 Method
We begin by presenting the key findings of the activation patterns the key cache (Section 3.1. These insights serve as the foundation for our proposed quantization approach, PolarQuant (Section
3.1 Motivation
Channel-wise Magnitude Paired Dimension
As discussed earlier, outliers in key vectors pose dilemma for key cache quantization. Our solution to this challenge arises from key observation:
Magnitude
When mapping the paired dimen- sions with outliers polar coordinates, the result- ing 2D vectors show consistent magnitudes. The outliers present in channel compensated by the activations of the other, which significantly simplifies quantization.
Observation:
Quantiation
Range
Magnitude of Mapped Vectors on 2D Cartesian Coordinate
Magnitude
Recall that key vectors, elements in certain di-
Quaitiiation
mensions are jointly rotated by the same rotary sub-matrix Vn.o.- Our analysis shows that the most prominent outliers tend to occur in only one of these dimension pairs 5
Ranzz
Figure 2: This figure supplements the transfor- mation from Figure (a) to [ showing how PolarQuant better resolves the outliers.
Figure T (b) maps the paired dimensions from Fig- ure (a) onto a 2D Cartesian coordinate system, where the x-axis represents the first dimension and
the y-axis represents the second.
*For efficiency, rotary matrix typically applied in element- wise multiplication manner (66 To simplify implementation, dimensions i and + d/2 are often rotated together, rather than and + 1. This results in non-ad non-adjacent outiers in Figure [a), but it does not affect our analysis, which is based on the matrix multiplication formulation (Eq.

---

Despite large variations in individual x and values (which would indicate outliers in isolation), the mapped vectors form rell-structured pattern. In other words, when transformed into polar coordinates, the outliers are characterized by a smoothly distributed radial coordinate and polar angle This structure significantly alleviates the quantization challenges faced by key caches.
Figure provides a supplementary illustration of how this polar transformation improves quantization. The top figure illustrates pair of dimensions exhibiting outliers, each with a large individual value range. Quantizing this large range inevitably results in a loss of precision. In the bottom figure, by combining these dimensions into 2D vector, the norm (i.e., the polar radius) shows a significantly narrowed value range, facilitating quantization.
3.2 PolarQuant: Polar-coordinate-basee quantization of post-RoPE key eache
Building these insights, introduce novel key cache quantization approach based polar transformation. Since the benefits of adopting a polar-coordimate have been outlined in the previous subsection, here we focus the implementation details.
For an arbitrary 2-dimensional subvector K [2j]. (2, in the key cache at position n, where 0 S j < d/2, we interpret [2i], [2j + 11) as Cartesian coordinates in the zy-plane. This 2D vector is then converted to polar coordinates, where the radius pr.[]] is given by:
Pnli] = V k,l22) + Kn. + 1J²,
and the polar angle is:
en[p atan2 [2j + K,,,K, n, On E (0, 2m),
where atan2 (y, x) returns the angle between the positive x-axis and the point (x, y).
We perform asymmetric group-wise quantization on Pr and [i). using group size of with r-bit precision for pn [j] and t-bit precision for [n]r]]:
.0, -10 -1)
(pr(Di) Clamp
Sp(a] Li]
Qi0, Cl amp
(L
0n151 2 Li]
-122
2 1).
5ot: Li]
where >0B1bi). 201] are the zero-points and spt][]] sap [i] are the scaling factors:
$ $pgDi> max(pp]5))
min (pal)
>nn[1== max(p1] min(pp
2"
2"
( max(0 min (0)l)))
=mtlli]= max(0p(ii) min (0g
2
2
Intuitively, PolarQuant partitions the -wo-dimensional plane into 2 regions, spanned by 2" radii and 2 polar angles. Each 2D sub-vector of the key vector is then represented by the center of the region in which it resides. [ provides an illustration of the quantization process. The corresponding Cartesian coordinates in the key vector at dimensions 2 and 2j are then calculated for the quantized representation (Q(p.Li)), Q(OnLi)) which is formulated as:
K (27], K2 1]] [n cos (d.13i) PnlLi sin (d..ii))
where i. li] and [n]ii] are the dequantized radius and polar angle:
Pn (Q(p.l)) + ) -oali *ppli), 0.Li (Q(e, ) smglil + =0///5
5

---

3.3 Efficient decoding with PolarQuant
In this section, we present PolarQuant's design for query-key multiplication, which incorporates the idea of cost-multiplicationn dequantization to achieve acceleration of the decoding process. We begin by reviewing the conventional approach to dequantized generation. Specifically, during the generation phase, the cached keys must be dequantized before being multiplied by the current query Qm at position m. For each dimension 0 S j < d, we have:
K..li = [ $ [i] + =pbl).
where K,, denotes the dequantized key, and the inner product then computed as Qm Kt---
The standard dequantization-tinn demands extra computation. We argue that this overhead is redundant. At any dimension j. the dequantized outcomes come from a finite set of size 2*+ Since the cache size far exceeds this set, is more efficient to pre-compute and reuse the most-uultiplication intermediate results using a lookup table (LUT). This approach of leveraging LUTss has been explored in prior studies [13 35]. KVQuant adopts an LUT to dequantize the key cache and restores positional information via RoPE recomputation. PolarQuant, in contrast, avoids RoPE recomputation by directly constructing LUT for QK product on the fly. This the key insight behind how PolarQuant accelerates decoding.
operation introduces overhead for PolarQuant, as
Specifically, PolarQuant builds its lookup table within each channel by mapping quantized polar coordinates to Cartesian coordinates and computing the dot products with the query sub-vectors. We implement custom Triton kernels to perform fused dequantization and query-key multiplication for efficient GPU execution of PolarQuant. The breakdown time analysis presented in Section 4.2 further demonstrates the effectiveness of our PolarQuant implementation. More implementation details can be found in Appendix A and our released code.
Experiment
In this section, evaluate the performance of PolarQuant to highlight its effectiveness and efficiency. Our empirical results confirm that PolarQuant can be integrated with LLMs, while maintaining near- lossless performance of generative tasks (Section I. We also highlight the speedup achieved by PolarQuant to showcase the superiority of our decoding algorithm and implementation (Section 4.2)
4.1 Preserving performance in quantized language and reasoning models
General setup. To ensure fair comparisons, we retain the value cache in full precision to avoid potential bias from its quantization downstream tasks. We evaluate PolarQuant against several quantization baselines built on HuggingFace Transformers codebase (31]. For all group-wise quanti- zation methods, the group size is fixed at g = 128. Additional details about the baseline methods and experimental setup are provided in Appendix B We evaluate all baselines alongside our Polar on
a range of models from mainstream model families, including Llama
28 and Qwen (27
Quantizing language models. In real-world applications of language models, the key-value cache often becomes the primary memory bottleneck when processing long-context inputs. We evaluate PolarQuant and baseline methods on LongBench [5]. widely used benchmark for long-context evaluation. Table presents results on Qwen and Llama. There are two advantages of PolarQuant:
PolarQuant performs robustly across different model backbones. Quantization is especially challenging for Qwen models, which exhibit extreme channel-wise outliers in their key cache
6 When applied to these models, token-wise quantization methods - such as per- token Int. and ZipCache tend to collapse. In contrast, KIVI-4 and PolarQuant constrain the accuracy drop to within 10%. For average performance under 3-bit quantization, whereas all baseline methods, including KIVI, result in performance degradation.
PolarQuant even improves
2. Averaged across all evaluated settings-covering both model families and quantization precisions-Polane PolarQuant achieves the best overall performance preservation.
'Qwen2.5 language models configured with attention bias that can introduce outliers in specific channels.

---

Table 1: Performance comparison of quantization methods LongBench. Cell colors reflect the degree of performance degradation compared to the backbone model. QJL results for Qwen2.5 are excluded due to incompatibility with its official kernel. Parenthetical values in the last column denote the performance drop of the quantized model relative to its backbone.
Single Doc. QA
Multi Doc. QA
Code Completion
Quantization
Bits
NtrvQA
Qasper
MF-en
2Wiki
Hotpot
Musique
Lec
RepoBench
Avg.
Qwen-2.5-1.5--mme (128K)
Bf16
19.44
37.22
49.69
32.68
41.57
23.99
41.86
48.55
38.88
Int-4
4.25
10.60
11.99
5.90
32.82
24.65
22.57
15.30 (-23,58)
ZipCache,
4.25
4.35
8.52
12.07
14.14
17.04
7.57
23.68
21.48
13.61 (-25.27)
KIVI-4
4.25
19.89
36.52
49.83
32,18
41.51
22.89
40.69
48.72
36.53 (-2.35)
PolarQuant,
4.25
19.34
36.48
50.80
32.80
43.25
23.25
38.62
46.76
36.41 (-2.47)
Int-3
9.53
3.64
22.32
10.70 (-28.18)
ZipCachey
3.25
5.28
7.81
9.38
10.30
12.66
6.36
26.53
22.72
12.63 (-26.25)
QJL
3.13
N.A
KIVI-2
3.00
36.07
47.94
32.51
42.09
23.33
39.50
45.31
35.64
(-3.24)
PolarQuant
3.25
19.09
35.60
49.47
32.16
43.47
23.02
35.77
44.16
35.34
(-3.54)
Llama-2-7B-CHA (4K)
8f16
16
18.95
21.14
37,70
30.64
27.81
6.94
$8.30
52.17
31,71
Int-4
4.25
18.24
21.79
37.56
29.80
26.24
8.83
57.95
52.70
31.64
(-0.07)
TipCache,
36.35
26.35
(-0.16)
KIVI-4
4.25
18.38
21.16
37.19
31.67
26.90
7.85
58.32
$1.99
31.68
(-0,03)
PolarQuant,s
4.25
18.40
21.37
35.05
30.18
27.92
8.59
58.82
51.95
31.54
(-0.17)
Int-3
3,25
16.51
21.41
36.59
29.34
27.59
9.74
57.75
$1.42
31.29
(-0.42)
ZipCachez
3.25
18.73
20.11
34.32
28.50
26.53
8.39
57.51
51.42
30.69
(-1.02)
QIL
3.13
19.13
20.51
35,93
30.74
25,60
5.79
57.79
50.92
30,80
(-0.91)
KIVI-2
3.00
18.79
20.46
35.51
27.52
26.36
8.12
56.82
$0.26
30.48 (-1.23)
PolarQuant,a
3.25
19.75
35.47
31.15
26.60
7.68
58.26
52.41
31,20
(-0.51)
1.ama-3.1 1-8---118---ttreee (128K)
Bf16
16
31.38
46.65
56.81
49.46
57.85
32.63
62.88
56.43
49,26
Int-d
4.25
31.76
45.49
56.47
49.52
57.67
32.82
63.17
55.46
49.05
(-0.21)
TipCache,
4.25
32.26
45.97
56.77
49.50
58.67
33.03
63.41
55.97
49.45
(+0.19)
KIVI-4
4.25
31.23
47.15
49.14
58.05
32.67
63.05
56.45
49,36
(+0.10)
PolarQuant
4.25
31.36
46.78
56.72
49.47
58.54
32.23
63.28
56,73
49,39
(40.13)
Int-3
3.25
29.66
45.07
55.15
49.79
58.31
32.73
60.56
$4.80
48.26
(-1.00)
ZipCache,
3.25
31.98
55.61
49.16
58,33
31.53
61.93
54.19
(-0.83)
QHL.
3.13
32.41
44.75
56.18
48.50
57.34
32.07
61.66
$5.99
48.61
(-0.65)
KIVI-2
3.00
31.90
45.39
54.96
49.88
$8.08
32.25
62.03
$4.93
48.68
(-0.58)
PolarQuant
3.25
32.49
46.72
56.56
49.96
58.33
32.20
63.45
56.54
49.53
(+0.27)
Beyond long-context processing, we also apply PolarQuant on LLMs to examine their generative abilities with normal-length inputs. To explore how quantization affects the emergent capabilities of LLMs, such chain-of-thought reasoning (30) and in-context learning (6). we benchmark PolarQuant's performance on the 5-shot CoT GSMSK [8] The results, presented in Table show
that PolarQuant effectively supports both short- and long-context inputs, without compromising performance on reasoning or mnwwledge-inenniig tasks.
Table 2: Evaluations on 5-shot CoT GSMSK.
Llama-2
Quantization
Br16
Int-4
ZipCache-4
KIVI-4
PolarQuant,,
7B-Chut
Acc. (Bits)
20.92 (16)
19.79 (4.25)
23.12 (4.25)
21.61 (4.25)
22.61 (4.25)
Llama-3.1
Quantization
BE16
Int-4
ZlpCache-4
KIVI-4
PolarQuant,
8B-Instruct
Acc. (Bits)
78.85 (16)
76.35 (4.25)
78.70 (4.25)
78.32 (4.25)
78.77 (4.25)
Quantizing reasoning models. Large reasoning models (LRMs) exhibit remarkable capability solving complex problems by long chains of thought. Recent studies (18) have shown that
are of quantization errors. Given that LRMss rely on both long-context processing and the generation of
complex tasks-such as mathematics [16 8 and reasoning 23 are sensitive to the accumulation
lengthy outputs, quantization techniques are particularly critical for their application. Moreover, the
7

---

underlying mechanisms of LRMs differ from those of LLMs, making quantization more challenging and offering valuable insights into the effectiveness of different approaches.
We apply key quantization to the distillation-based reasoning models of DeepSeek-RI 20 The accuracy scores across different quantization methods and tasks are presented in Table The quantized reasoning models of PolarQuant achieve substantial improvements over the baselines, which provides strong supplementary evidence of PolarQuant's superiority.
Table 3: Overall performance comparison of quantized DeepSeek-RR 1-iistilll models across various reasoning benchmarks. Results for ZipCache Qwen2.5 are omitted due its performance collapse. Cell colors represent the degree of performance degradation compared to the backbone model. Parenthetical values in the last column denote the performance drop of the quantized model relative to its backbone. Best results are highlighted in bold.
AIME
Quantization
MATH
GPQA
Overall
AIME24
AIME25
AVG.
DeepSeek- RI- Dist--isll-eee Sestill-Owen--.5B
Bf16
36.67
23.33
30.00
85.20
39.90
51.70
ZipCache-4
N.A
KIVI-4
20.00
23.33
21.67
80.40
33.84
45.30 (-6 40)
PolarQuant
30.00
20.00
25.00
80,20
37.88
47.69 (-3.31)
DeepSeek-RR R Distill-lamm--al
Bf16
50,00
36.67
43.33
91.20
51.52
62.01
ZipCache-4
43.33
43.33
43.33
91.60
48.48
61.13 (-0.88)
KIVI-4
43.33
33.33
38.33
89.80
51.01
59.71 (-2.30)
PolarQuant
60.00
36.67
48.33
89.00
50.00
62.44 (+0.43)
4.2 Superior efficiency of PolarQuant
To evaluate the efficiency of the customized decoding algorithm, we provide a comprehensive time breakdown analysis our PolarQuant implementation. In experiments, we use the L.ama-3.1 Instruct model configuration, which includes 32 query heads of dimension 128, and 8 key/value heads (grouped-query attention (2).. We benchmark the latency of our tailored kernel implementation, as well as the end-to-end generation throughput.
Latency for query-key multiplication kernel. We make comparisons of the query-key multipli- cation implementations for LLM decoding. Specifically, we evaluate the wall-clock latency across different sequence lengths and batch size settings. We benchmark the runtime by summing across 10K iterations of the calculations, and report results for: Fp16, KIVI-4, KIVI-2, PolarQuant44 and PolarQuantg3. Figure 3 illlustrates the performance comparison among kernel implementations, while Table 4 reports the multppicatioon latency with batch size of 8. see that, PolarQuant achieves up to 2.7 x speedup compared with KIVI and 1.6x speedup compared with Fp16 Torch implementation.
Batch
Batch Size
Batch Size
KINE
PalarQuante4
1.2
3.0
Leeene mm)
Laneeey lime
0.8-
32k
64h
1208
3dk
322
641
120k
46
34kk
64k
128k
Contact Length
Contrctt Langth
Contest Lengthh
Figure 3: Latency comparisons of PolarQuant across varying batch sizes and context lengths

---

Throughput comparison. We incorporate our custom kernels into the generation pipeline and measure their end-to-end performance. We determine the maximum supported batch size with a sequence length of 32K tokens, with Hugging Face's implementation. We fix the input length at 256 tokens and measure throughput across different generation lengths. Table presents the results; entries marked with indicate the application of 2-bit value quantization. As is shown in Table PolarQuant achieves up to 3.18x throughput improvements, and demonstrates significant improvements over KIVI in both latency and throughput. Values in parentheses denote the speedup.
Table 4: Latency, throughput and memory usage comparisons.
Operation
Latency (8)
4K
8K
32K
128K
Fp16
0.22
0.42
1.58
6.22
KIVI-4
0.41 (0.54x)
0.82 (0.51x)
3.19 (0.50x)
PolarQuant
0.24 (0.92x)
0.42 (1.00x)
1.49 (1.06x)
5.79 (1.07x)
KIVI-2
0.32 (0.69 (0)6
0.61 (0.69x)
2.35 (0.67x)
N.A 3.97 (L.5x)
PolarQuant,3
0.18 (1.22x)
0.30 (1.40x)
1.03 (1.53x)
TP. (token/s) / Mem. (GB)
Configuration
4K
16K
32K
Fp16
20.78
84.5 25.21
53.6/34.11 34,07
51.77
KIVI-4
129.0 (1.09)) 19.03
96.9 (1.15x) 21.88
65.1 (1.21x) 27.59
39.0 (2.51x) 38,96
Po
138.8 (1.17x) 19.04
108.5 (1.28x) 21.90
75.9 (1.42x) 27.61
46.8 (3.05 x) 39.06
KIVI-2
133.2 (1.12x) 18.89
(8.899
99.9 (1.18x) 21.55
67.7 (1.26x) 26.91
40.5 (2.64x) 3 37.62
PolarQuantss
144.8 (1.22x) 19.04
111.1 (1.31x) /21.90
78.6 (1.46x) 27.61
48.7 (3.18x) 39.06
KIVI-4
129.0
(1.09x) 17.08
115.8 (1.37x) 17.99
90.8 (1.69x) 19.92
30.13 (1.97x) 23.60
PolarQuant
144.1 (1.21x)
128.0 (1.5) 17.97
(2.08x) 19.81
(3.06x)) 23.46
Discussions
5.1 Ablation studies of PolarQuant
Effect of group size g. PolarQuant applies group-wise quantization along the channel dimension. We conduct ablation studies to investigate the impact of the group size on model performance.
Table 5: Ablation study of group size on LongBench.
Group Size
32
64
128
256
LongBench (Bits)
KIVI-4
49.48 (5.00)
49.47 (4.50)
49.36
(4.25)
49.52 (4.13)
PolarQuant,4
49.50 (5.00)
49.33 (4.50)
49.39 (4.25)
49.58 (4.13)
Specifically, we benchmark PolarQuant and KIVI on LongBench using the Llamm-3.1 --8--nrrrttttttt model. The results, presented in Table 5 indicate that PolarQuant performs competitively with KIVI across all tested group sizes. It is noteworthy that the quantization parameters occupy 32/g bits; therefore, smaller values of g result in higher average bit widths. To strike optimal balance between performance and parameter overhead, we set g as 128 throughout this paper.
Impact of RoPE configuration. We conduct experiments to test the sensitivity of PolarQuant to different RoPE configurations and draw the following conclusions:
(1) PolarQuant exhibits consistent performance across LLMs with different RoPE base frequen- cies (see Table The experimental results for basic values of { {10000, 500000, 1000000), highlight PolarQuant's insensitivity to the choice of base frequency.
(2) PolarQuant is adaptable to different RoPE variants. We employ NTK RoPE scaling 24 to extend the I.M'' context and apply PolarQuant the key cache. Critically, mo significant performance drop is observed. We provide detailed experimental results and setups in Appendix C

---

Bitwidth allocation for radii and polar angles. In PolarQuant, we assign bitwidth asymmetrically between radii and angles. To determine which component requires higher precision, we conduct ablation studies on various bitwidth configurations, enabling more flexible allocation strategy.
Table 6: Ablation study on asymmetrical bitwidth allocation in PolarQuant.
Bits
(r5, t3)
(r4, t4)
(r3, (5)
Bits
(r4, t2)
(r3, t3)
(r2, t4)
LongBench
4.25
49.41
49.39
49.51
3.25
47.66
49.53
49.00
We take Llama-3. 1-8.1-88--snttte as backbone and benchmark PolarQuant on LongBench, to assess the impact of different bitwidth settings. From Table we have the following observations:
Angle quantization is more sensitive to bitwidth. Allocating fewer than 3 bits to angles often results a significant drop in performance.
Observation 1:
Observation 2: Despite the asymmetry in bitwidth allocation, a more balanced distribution between radii and angles can still achieve strong performance.
5.2 Compatibility with existing KV cache compression techniques
In this section, we explore the integration of existing KV compression techniques with PolarQuant, which allows for further reductions in KV cache memory occupation. We present the experimental results of Llama-3.1-80 -.-stut-tttt on LongBench. As stated in Section 4.11 we retain the value cache in full precision, key cache is more sensitive to low-precision quantization. Appendix D provides further analysis to support this. We combine value quantization with PolarQuant to verify its compatibility. standard eoken-wise quantization is applied to the value cache, consistent with KIVI [19]. Table the results. The introduction of value quantization results in only marginal performance degradation, even at 2-bit precision.
Table 7: LongBench results of PolarQuant with value cache quantization.
Quantization
Bits.
NtrvQA
Qasper
MF--nn
2Wiki
Hotpot
Musique
RepoBench
Avg.
16
31.36
46.78
56.72
49.47
58.54
32.23
63,28
56.73
49.39
PolarQuant,e
4
31.67
46.51
56.48
49.74
58.48
32.41
63.50
56.59
49.42 (+0.03)
2
31.26
57.59
47.88
58.51
32.98
63.24
55.95
49.26 (-0.13)
We further explore the integration of PolarQuant with token-eviction strategies [15] As shown in T PolarQuant does not exhibit significant performance degradation. We left for future work to combine PolarQuant with existing mixed-precs precciion quantization for further memory saving
10
33
Table 8: LongBench Evaluations of PolarQuant with SnapKV.
LLM
NtrvQA
Qasper
MF-en
2Wiki
Hotpot
Musique
Lcc
RepoBench
Avg.
Full KV
31.36
46.78
56.72
49.47
58.34
32,23
63.28
56.73
49,39
SnapKV:
31.31
46.51
57.03
49.71
57.99
32.79
62.90
55.75
49.25 (-0.14)
PolarQuant
31.21
46.38
56.45
49.38
$8.05
31.99
62.90
55.75
49.01 (-0.38)
SnapKV: 1024
31.51
43.45
56.14
49.61
57.78
32.01
62.45
56.00
48.62 (-0.77)
PolarQuant
31.27
61.54
54.57
(-1.59)
6 Conclusion
In this paper, we view the outliers in the key cache of LLMs from novel polar-coordinate. based perspective, which provides an efficient and effective solution, PolarQuant, to reduce the complexity and quantization costs in previous methods. PolarQuant well preserves downstream performance even in long-co long-context understanding and long chain-of -fto--gg-thhhhh generation, comparable to previous works under 4-bit precision while achieving superior efficiency. We hope the polar coordinate view can inspire the community to advance new low-bit precision guantization techniques.
10

---

Acknowledgments
This work is supported by Meituan through Agentic system Program. Songhao Wu is supported by "Qiushi ocademic-Dongliant
Project of Renmin University of China (No. RUC24QSDL015) We appreciate all reviewers for their valuable comments and suggestions, which are crucial for improving our work. We are also grateful to Xintong Qiu, Jixiang Hong, Tao Tan, Jia-Nan Li and Yuxuan Liu for their insightful suggestions on the manuscript.
References
[1] A Meta. Llama model card. 2024. URL httpp:///githui http://ggithuu...m//mmaataaa com/meta- 1ww.lalaaaaaaammaaammm aallammlllamaa / Blob/main/MMDER CARIOODDEELRRDD...OEEEEE md
[2] Joshua Ainslie, James Lee-Thorp Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. GQA: Training generalized multi-query transformer models from multi-head checkpoints. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4895-4901, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023 emnlp-main.22 URL
org/2023. 2023. emnlp 1p main. 298/
[3] Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlier-fre 4-bit inference in rotated Ilms, 2024. URL battts://aaaiitt www.orgg//aaaa abgg/4bb/2444..... 00445404444000666
[4] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate, 2016. URL bttpps://arxiiee org/ abs/1409.047
[5] Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. Longbench: A bilingual, multitask benchmark for long context understanding. arXiv preprint arXiv:2308. 14508, 2023.
[6] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan. Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Vosse Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL. httt:htts:///aeee httppps:///xxx.or.r...o wwwp://.orgg/a/////////// 14165
[7] Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, and Wen Xiao. Pyramidky: Dynamic kv cache compression based on pyramidal information funneling, 2024. URL httph::////et /ttp////xvvv.v.////////e///..a org/ wwrggabss/000 2406 02069
[8] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2 10. 14168, 2021.
[9] DeepSeek-AI, Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang. Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jin Chen, Jingyang Yuan, Junjie Qiu, Junxiao Song, Kai Dong, Kaige Gao, Kang Guan, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. Chen, R. L. Jin, Ruiqi Ge, Ruizhe Pan, Runxin Xu, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Size Zheng, T. Wang, Tian Pei, Tian Yuan, Tianyu Sun, W. L. Xiao, Wangding Zeng, Wei An
11

---

Wen Liu, Wenfeng Liang, Wenjun Gao, Wentao Zhang, X. Q. Li, Xiangyue Jin, Xianzu Wang. Xiao Bi, Xiaodong Liu, Xiaohan Wang, Xiaojin Shen, Xiaokang Chen, Xiaosha Chen, Xiaotao Nie, Xiaowen Sun, Xiaoxiang Wang. Xin Liu, Xin Xie, Xingkai Yu, Xinnan Song, Xinyi Zhou, Xinyu Yang. Xuan Lu, Xuecheng Su, Y. Wu, Y. K. Li, Y. Wei, Y. Zhu, Yanhong Xu, Yanping Huang, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Li, Yaohui Wang, Yi Zheng, Yichao Zhang, Yiliang Xiong, Yilong Zhao, Ying He, Ying Tang, Yishi Piao, Yixin Dong, Yixuan Tan, Yiyuan Liu, Yongji Wang, Yonggiang Guo. Yuchen Zhu, Yuduan Wang. Yuheng Zou, Yukun Zha, Yunxian Ma, Yuting Yan, Yuxiang You, Yuxuan Liu, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhen Huang. Zhen Zhang, Zhenda Xie, Zhewen Hao, Zhihong Shao, Zhiniu Wen, Zhipeng Xu, Zhongyu Zhang, Zhuoshu Li, Zihan Wang, Zihui Gu, Zilin Li, and Ziwei Xie. Deepseck-v2: A strong, economical, and efficient mixture-of-expee -xpertss language model, 2024. URL
abs//////////// awwp://bbs244..... 220.24445.00 04434
[10] Shichen Dong, Wen Cheng, Jiayu Qin, and Wei Wang. Qaq: Quality adaptive quantization for Ilm kv cache, 2024. URL https: http:/o////aaaaeeeeeeee 0442//243300443333
[11] Nathan Habib, Clémentine Fourrier, Hynek Kydliček, Thomas Wolf, and Lewis Tunstall. Lighteval: A lightweight framework for Ilm evaluation, 2023. BtI.bttps://// thgt thub. com/ Beigginnfaae/lighteee.
[12] Yefei He, Luoming Zhang, Weijia Wu, Jing Liu, Hong Zhou, and Bohan Zhuang, Zipcache: Accurate and efficient kv cache quantization with salient token identification, 2024. URL
https://arxiv org/ abs/24/5 14256
[13] Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. Kvquant: Towards 10 million context length Ilm inference with kv cache quantization, 2024. URL/ttpss://axii wwwpsiii.orr..ooooooe wwt.//g///////ooooeo/...o.e.eee/ abaaaaa//2220 2401. 18079
[14] Hao Kang, Qingru Zhang, Souvik Kundu, Geonhwa Jeong, Zaoxing Liu, Tushar Krishna, and Tuo Zhao. Gear: An efficient ky cache compression recipe for near-lossless generative inference of llm, 2024. https:/:: wwrriiivrrrg//// org/ awtpaa///...// 05527
[15] Yuhong Li, Yingbing Huang, Bowen Yang. Bharat Venkitesh, Acyr Locatelli, Hanchen Ye. Tianle Cai, Patrick Lewis, and Deming Chen. Snapkv: Llm knows what you are looking for before generation. arXiv preprint wrXhv:244.114699 2024.
[16] Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lec, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step, 2023. URL
httts::/aarxa: http:://axivvvvree
20050
[17] Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua. Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157-173, 2024. doi: 16.1 a 00008.8.
2024. 1.91-1.9/
[18) Ruikang Liu, Yuxuan Sun, Manyi Zhang, Haoli Bai, Xianzhi Yu, Tiezheng Yu, Chun Yuan, and Lu Hou. Quantization hurts reasoning? empirical study on quantized reasoning models, 2025. URL httllhhtps://// //apasv//////vv//aa orgo htee:/ba/255000 04823
[19] Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen (Henry) Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. Kivi: a tuning-free asymmetric 2bit quantization for kv cache. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2025.
[20] Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, and Abhinav Rastogi. Improve mathematical reasoning in language models by automated process supervision, 2024. URL //arxiv. org/ abs/
06592
[21] OpenAl. Learning to reason with 1lms. https://openai.com learning- to reason with-1lms/ 11ms/ 2024. [Accessed 19-09-2024).
12

---

[22] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library, 2019. URL htt.tttts///axii org/obgg/a.. 1912.01703
[23] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel Bowman. Gpqa: A graduate-leve google-proot q&a benchmark, 2023. URL https: /awxiv.xvvvvv/////v////////aaav/ org/ htrggabs/2330 2311. 12022
[24] Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong. Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usumier, Thomas Scialom, and Gabriel Synnaeve. Code llama: Open foundation models for code. 2024. BtLLhhtts://aaaee org ab////8.////2/./////////// 12950
[25] Noam Shazeer. Fast transformer decoding: One write-head is all you need, 2019. URL httgg//b////11./1111111
[26] Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding, 2023. URL https://arxiv org/ abs/2104. 09864
[27] Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL https:// qwenlm. www.lmm..itub... wtt.ll//bbllggoo hoollllg///ee.nng/ qwen2.5/ 5/
[28] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale. Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog. Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiao- qing Ellen Tan, Binh Tang. Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. URL/https:/ //aaxi/v org/ abs /bbs/2307 09288
[29] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, ed- itors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, mww.::edddiiinnnnnnnnseeoo.ueeee htcp/ppprr.llllspppr////////////
Inc., 2017. URL $
Paper pdf
[30] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Fd Chi, Quoe Ie, and Denny Zhou. -hain-of-thought prompting elicits reasoning in large language models, 2023. URL htttts::///atii www.aaavvvoo/ra///////aa
[31] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: off art natural language processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Languuge Processing: System Demonstrations, pages 38 45, Online, October 2020. Association for Computational Linguistics. URL httpt:tt///// / //www aclweb. www.ww..wgg/aae..e..eo.o.oo.ee.e anthology/
2020. emnlp-demos.6
13

---

[32]
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng. Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang. Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. https:////rrrxi org ab05 09388
[33] June Yong Yang, Byeongwook Kim, Jeongin Bae, Beomseok Kwon, Gunho Park, Eunho Yang, Se Jung Kwon, and Dongsoo Lee. No token left behind: Reliable kv cache compression via importance-aware mixed precision quantization, 2024. URL BtLLitttss//arxiioorr///// 18096
[34] Amir Zandieh, Majid Daliri, and Insu Han. Qjl: 1-bit quantized jl transform for kv cache quantization with zero overhead, 2024. URL https:///axxi:i /arxiv.. http://axxvvv...aaaaae. Terg/abs/2400 03482
[35] Hailin Zhang, Xiaodong Ji, Yilin Chen, Fangeheng Fu, Xupeng Miao, Xiaonan Nie, Weipeng Chen, and Bin Cui. Pqcache: Product quantization-hasee kvcache for long context Ilm inference. Proceedings of the ACM on Management of Data, 3(3):1-30, 2025.
[36] Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, and Beidi Chen. H2o: Heavy-hitter oracle for efficient generative inference of large language models, 2023. URL
https://arxiv org/ abs/ 2lbs/2330 2306. 14048
[37] Yilong Zhao, Chien- Yu Lin, Kan Zhu, Zihao Ye, Lequn Chen, Size Zheng, Luis Ceze, Arvind Krishnamurthy, Tianqi Chen, and Baris Kasikci. Atom: Low-bit quan- tization for efficient and accurate Ilm serving. In Gibbons, G. Pekhimenko, and C. De Sa, editors, Proceedings of Machine Learning and Systems, volume 6, pages 196- 209, 2024. URL/ttts::/prrcceddngg org oww.ssssrrrpppppppeppeeee laapp_rfllesssssllllllllllaleeee wwwls//pappe///2/2/ haabbr/cceetbbeeo/////rr01222222
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract, we claim that PolarQuant serve novel key cach quantization, while PolarQuant enhances the quantization performance of the downstream tasks.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We create separate "Limitations" section, i.e., Section Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
Our paper does not introduce theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have submited our code in the supplementary materials, enabling easy reproduction.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
We have submited our code in the supplementary materials, enabling easy reproduction.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (https
ssiicyolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Section Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
The statistical significance of the experiment has not been reported in this
work.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide detailed information on the computing resources required to reproduce the experiments in Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We have conformed to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: We have included discussion of the potential positive and negative societal impacts of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have correctly cited all the data, scripts, and models used.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have included README document with our code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
21

---

A Accelerated Query-Key Inner Product with Cached Keys in PolarQuant
For a quantized representation (((m.l))).Qee(lii) we first dequantize the polar coordinates back into Cartesian coordinates, which serves the dimensions 2j and 2j 1:
An cos (..51l
K [2j] Kr, 12 + +1]
Palil [ sin On Onlil)
where Pnli], 0,,[5] is the dequantization for (p.0) respectively. Pm[s] and (n(5) are formulated as: P..Di (Q(p)D)) + } *stalil =p|1(o] On(i] (Q(0, ) sols + =01: [+]-
(e(%, represents [2j]. K, [2j + 1]] state in the lookup table.
When computing the query-key inner product for the dimensions 2j and 2j + 1, the result is:
product Qn[2 K, [2j] Qn(2i [2j + 1],
and the final inner product is the sum:
product 2j.2+11
Figure presents naive PyTorch 22 implementation the aforementioned dequantization-an multiplication operation. We implement fused Triton kernel that reproduces the functionality of the PyTorch code for improved computational efficiency. More details can be found our code.
import torch
def attention how.:/oodddooddeettodddrdd...
9, # q's shape: (B, N, 1, 2, D)
r, rscale, rmn,
r's shape: (B, N, G, D)
t, tacale, ten, # t's shape: (B, N, G, D)
# rscale, rmn, tacale and tmn's shape (B, N, 1, 1, D) rbits: int 4, tbits: int 4,
):
phi: finite set for theta
phi = torch ( 2 ** tbits) [None, None, None, None] phi = (2 phi + 1) / 2 tscale + tmn
rho: finite set for rho
rho = torch. (0, 2 ** rbits) [None, None, None, None]
rho = (2 * rho + 1) / 2 rscale + rmn
phi torch cat ([phi cos (),, phi sin,sin(), dim=-2)
accumulator torch. surh.summ (q phi, dim=-2) # (B, N, tttbits, D) aather((ccummuus (accumulator, 2,
accumulator torch gather unsquezze .x1)aeeppand (accumulator))
accumulator torch. gather (rho squeeze (-2), 2,
r. unsqueeze (-1) eeppeexpaddaa_ (accumulator))
accumulator accumulator cuntttorruummmm1 (-1)
return accumulator
Figure 4: Pytorch implementation of the accelerated Query-Key Inner Product in PolarQuant.
22

---

B Details experiment setup
In this section, we provide additional details about the experimental setups.
B.1 Setup of baseline methods
This section provides an overview of the baseline methods. Following this, we outline the quantization configurations and variants for both the baselines and our proposed PolarQuant. We also explain how the average number of bits for the quantization parameters calculated.
Int-N applies token-wise N-bit quantization to the key states. This token-wise quantization incurs 32/d / bits quantization parameters (16 bits for zero-points and 16 bits for scaling factors per token), which amounts to 0.25 bits per token when d = 128.
ZipCache-N N [12 introduces a channel-separable token-wise scheme for key quantization, where N denotes the quantization bits. Each key channel is normalized by the square root of its maximum magnitude before quantization. Similar to Int-N, this method performs token-wise quantization, allocating 0.25 bits for zero-points and scales.
KIVI-N [9] employs asymmetric strategy for KV cache quantization, applying channel-wise quantization to the key cache and token-wise quantization to the value cache. For 4-bit quantization, we use KIVI-4 with group size of 128. For 3-bit quantization, we use KIV1-2 with group size of 32, as the official implementation does not support 3-bit quantization. This channel-wise quantization introduces (16 + 16)d bits of quantization parameters per group, which increases the average bitwidth by 32/g: bit for = 32 and 0.25 bits for = 128 respectively.
QJL 34 applies Johnson-L SaannsonnLindenstraan Lindenstrauss transformation to key states, removing the memory over- heads associated with storing quantization constants. For 3-bit quantization schema, this method achieves a key cache bitwidth of 3.13 bits.
PolarQuant,; assigns bits for radii quantization and bits for polar angles, resulting in (r f)/2 bitwidth for key states quantization. PolarQuant also employs channel-wise quantization with grouping, this group partitioning adds an overhead of 32/g bits.
Nearly all methods discussed here require a buffer size or residual length for applying quantization. We exclude the contribution of these residual key states to bit counts for comparisons.
B.2 Configurations of the open-sourced LLMs
Qwwen-2.5-1.B--Instrrue [27] is the instruction-tuned 1.5B Qwen2.5 model, which supports
context length of up to 131,072 tokens and features a base RoPE frequency of 1,000,000.
Llama-2-7B-Chat (28 has context length of 4096 and base RoPE frequency of 10,000.
[ is billion parameter language model, designed to handle context
length of up to 131,072 tokens, and has the base RoPE frequency set to 500,000.
B.3 Reasoning model evaluation
For fair comparison, the evaluation code is built on the Huggingface Lighteval framework 11 We the default generation configuration for datasets splitting, and the EM score reported in Table More implementation details can be found in our released code.
C NTK RoPE scaling experiment
We adopt NTK RoPE scaling 24 to extend the context window of the Saama-2-7B-Cie model from 4096 to 8192. Specifically, we implement dynamic RoPE updates based on the Hugging Face codebase. PolarQuantachieve an average performance of 32.44 on LongBench. Compared to the 32.15 score of the Bfl6 baseline, PolarQuant's performance remains competitive.
23

---

D Sensitivity Analysis of Key-Value Quantization
combine 4-bit key quantization PolarQuant44 with 2-bit value quantization and observe minimal performance degradation. We further evaluate value quantization (KIVI, group size 128) on LongBench. By retaining the key cache at full precision, KIVI result no performance drop. However, when the key is quantized to the same bitwidth while the value is kept at full precision, the performance drops significantly. Table 9 presents the results, the notation (Kn; V.) denotes key quantization to b-bit and value quantization to e-bit.
In Section 5.2,
Table 9: Impact of key and value quantization bitwidths on LongBench performance.
(K16, V16)
(K16, V4)
(K16, V2)
(K2, V16)
LongBench
49.26
49.54
49.30
47.73
E
Limitation
Although PolarQuant achieves promising results in reducing storage and computational resources, we also discuss the limitations of our current work. This work focuses exclusively on decoder-only Transformer-basee large language models (LLMs) that utilize rotary position embedding (RoPE) as the underlying position encoding mechanism. RoPE has become a prevalent choice in many state-of-the thrt open-source LLMs. However, the effectiveness of PolarQuant when applied to models with alternative position encoding methods or attention mechanisms remains an open question and warrants further investigation. Furthermore, exploring more recent LLM backbones [32 is necessary, but due to time and computational limitations, we leave this part of the work for future research.
24


Reviews:
[{"review_id": "v5jZNEQ4jb", "reviewer": "Reviewer_Jun3", "summary": "PolarQuant aims to accurately quantize the key cache to low precision by leveraging a polar representation for the keys. Their approach is designed to address the impact of the RoPE embedding (which rotates neighboring channels in key activations). They observe that when representing pairs of channels in polar form, the outlier channels in keys (which are typically coupled with a non-outlier channel) become easier to quantize since they have a consistent norm along the sequence length dimension. By quantizing keys in polar form, they achieve superior efficiency (in terms of improved accuracy for the same bit width) relative to existing methods. They also provide an efficient system implementation that exploits table lookups to accelerate inference, showing the efficiency gains of their approach through both kernel-level and end-to-end benchmarking.", "questions": "- What is the overhead of quantizing new keys online during inference using this method - is it negligible?\n- Do you have any analysis demonstrating the relative higher sensitivity of the key cache to low-precision quantization (compared with values)? Also, is the key cache still more sensitive than the value cache even after applying the polar quant method?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- They propose a novel polar transformation to tackle the effects of RoPE on the key distributions (which counteracts the effect of the rotation on widening the range of outlier channels).\n- They present convincing evaluation across multiple models (llama3.1 and qwen2.5) and datasets (LongBench + GSM8K) as well as comparisons with existing strong baselines like KIVI/QJL.\n- They present a lookup table mapping for efficient decoding as well as custom Triton kernels which show substantial latency gains with their method. Their speedup evaluation includes kernel-level and end-to-end evaluation.\n\nWeaknesses:\n- The insight that the key cache has outlier channels with consistent magnitude pre-rope and that the magnitude gets rotated with the neighboring channel (leading to varying magnitude along the sequence length) is not novel relative to prior work. Although they argue that their method is superior to applying RoPE during inference as the key cache is loaded (which is an alternative strategy to account for the rotation), they do not justify that the overhead of performing recomputation using RoPE is substantial (particularly since KV cache loading is memory bandwidth bound, this overhead may potentially be hidden / overlapped with memory loads).\n- The fast decoding method using lookup tables makes sense, but is not entirely novel in the context of KV cache quantization - for example, previous non-uniform quantization methods like KVQuant have leveraged lookup tables for fast dequantization. \n- Their method focuses mainly on efficient key quantization; however, even if we can push keys to low bit precision, we will still be limited by the precision with which we can represent values. A more detailed analysis of challenges with value quantization is not presented. Alternatively, some ablations demonstrating that the relative difficulty of value quantization relative to key quantization is low would be useful.", "was_revised": true, "final_justification": "The rebuttal clarified my concerns around the key cache vs value cache sensitivity and overhead of quantizing new keys.\n\nWith respect to the concerns around efficiency of fused-RoPE attention, I am not convinced (based on the speedups of more optimized fused-RoPE kernels) that there is a reason that this can't be efficiently implemented with low-bit quantization. However, I understand the author's feedback that there aren't existing kernel implementations for this and that it is difficult to prove definitively that this can't be implemented efficiently, and that they compared with the best available open-source method. As such, I will raise my score to a 4.", "rebuttal": "Thank you for valuable feedbacks! Hope our explanation helps address your questions.\n\n---\n\n**W1:** The authors do not justify that the overhead of RoPE recomputation in previous work is substantial, particularly since KV cache loading is memory bandwidth bound.\n\nThank you for the suggestion.\n\n> First of all, we contend that **the overhead of RoPE recomputation is substantial**. \n\n1. Perspective of Theoretical Analysis\n\nKVCache loading in full precision is memory-bound [1, 2], but low-bit KV cache loading with matrix multiplication dequantization does **NOT** exhibt the same on more advanced GPU architectures.\n\nKVQuant utilizes A6000 to analyze kernel efficiency, whereas our study uses the A800, which is a more advanced GPU and reflects a more practical scenario.\nWe monitor the Memory Throughput and Compute Throughput of the vecquant4matmul kernel of KVQuant on A800.\nThe NCU report indicates that such kernel implementations exhibit a computation-bound.\n\n2. Perspective of Experimental Results\n\nWe conduct further experiments to empirically confirm that the overhead of RoPE recomputation is substantial.\n\nOn A800, we benchmark the latency of the KVQuant kernel implementation, as well as its end-to-end throughput performance in Transformer.\nFor batch size, we identify the maximum supported batch size for a generation length of 4K, with other experimental settings following those outlined in Section 4.2.\n\n\nThe experimental results are as follows:\n\nLatency comparisons  of kernels across  context lengths\n\n| Method \\ Length |  4k  |   8k  |  16k |  32k  |\n|-|-|-|-|-|\n|\tFp16             | 0.35 |  0.66 | 1.30 |  2.56 |\n| KVQuant          | 1.23 |  2.22 | 4.29 |  8.44 |\n| KIVI4\t\t         | 0.69 |  1.38 | 2.75 |  5.47 |\n| PolarQuant44     | 0.33 |  0.61 | 1.16 |  2.26 |\n\nThroughput comparison\n|   Method             |  Fp16   |  KVQuant | KIVI4  | PolarQuant44 |\n|-|-|-|-|-|\n| Throughput (token/s) |  129.78 |  135.97  | 140.94 |    157.16    |\n\n\nThese two aspects will demonstrate that the overhead of RoPE recomputation is substantial.\n\n> We hope the reviewer can reconsider the novelty of PolarQuant, given that the overhead of RoPE recomputation is substantial.\n\n1. We highlight the limitations of KVQuant, where in real-world scenarios like on A800, **the recomputation of RoPE is substantial**, leading to efficiency issues.\n\n2. PolarQuant introduces a **novel polar transformation approach to key cache quantization** that avoids this recomputation. \n\nSimilar to KVQuant, our method utilizes pre-RoPE key properties. However we argue that the polar transformation is an innovative approach to preserve pre-RoPE benefits without the recomputation burden, leading to faster inference across a wider range of applications.\nWe bilieve this is one of our contributions.\n\nThank you for the insightful suggestion. We hope you will reconsider your evaluation of the novelty of our work.\nWe will include the necessary discussion in the revision to clarify any potential misunderstandings.\n\n**References:**\n\n[1] SqueezeLLM: Dense-and-Sparse Quantization\n\n[2] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization\n\n---\n\n**W2**: The novelty of PolarQuant's use of lookup table.\n\nWe highlight our novelty and contribution by distinguishing (1) the usage of lookup tables and (2) the effectiveness of lookup tables ultization between PolarQuant and KVQuant.\n\n> Differences in lookup table usage scenarios\n\n- KVQuant utilizes a **lookup table to dequantize the key cache** and restores the positional information of key values via RoPE recomputation.\n- PolarQuant avoids RoPE recomputation; instead, it directly constructs **a lookup table for QK product** on the fly to achieve acceleration.\n\n> Differences in effectiveness\n\nAs highlighted in our response to W1, the cost of RoPE recomputation has become non-negligible on advanced GPUs, emerging as a bottleneck that limits KVQuant’s efficiency.\nOn the other hand, PolarQuant offers improved acceleration.\n\n\n\nIn summary, our use of the lookup table is neither a follow-up nor an incremental improvement over KVQuant. While lookup tables are a common technique in quantization [1, 2, 3], many aspects of PolarQuant are novel:\n\n- The challenging problem we address.\n- The rationale behind our application of lookup tables.\n- The specific objects our tables target.\n\n**References:**\n\n[1] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization\n\n[2] PQCache: Product Quantization-based KVCache for Long Context LLM Inference\n\n[3] PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling\n\n--- \n\n**W3 & Q2**: Is Key cache more sensitive to low-precision quantization than Value cache?\n\nYes. In Section 5.2, we combine 4-bit key quantization (PolarQuant_44) with 2-bit value quantization and observe minimal performance degradation.\n\nAdditionally, we test value quantization (KIVI with group size 128) on LongBench while keeping the key cache at full precision and resulting in no performance drop. However, when quantize key in same bit and keep value in full precision, the performance greatly drops.\n| Method        | Key bits | Value bits | LongBench (Avg.) |\n|-|-|-|-|\n| BF16          | 16       | 16         | 49.26            |\n| KIVI          | 16       | 4.25       | 49.54            |\n| KIVI          | 16       | 2.25       | 49.30            |\n| KIVI          | 2.25     | 16         | 47.73            |\n\nThese results suggest that:\n\n1. Existing quantization methods are already sufficiently effective in value quantization.\n2. Key quantization is more sensitive to precision changes than value quantization in current methods.\nAs a result, key quantization presents more challenges than value quantization.\n\nThank you for the suggestion, we will include this discussion in our revised version.\n\n--- \n\n**Q1:** Overhead of quantizing new keys ?\n\nThe following is the detailed time breakdown for the PolarQuant_44 during the decoding phase (with a batch size of 1, input length of 256, and generation length of 4096):\n| Operation | Time (s) |\n|-|-|\n| Decode    |   91.07    |\n| Quant     |   0.55    |\n\nFrom this, we can conclude that the overhead of quantizing new keys is 0.6% compared to other operations. \nQuantization is performed every n steps, causing only a slight overhead during decoding, which explains why it is negligible.\n\n---"}, {"review_id": "eOBHrcihGY", "reviewer": "Reviewer_Mgry", "summary": "This paper introduces PolarQuant, a novel quantization method for the K-cache in large language models. The method leverages a polar coordinate transformation to address the challenge of outliers in key vectors, which traditionally hinder effective quantization. By encoding key vectors as quantized radii and angles, PolarQuant achieves both memory efficiency and decoding acceleration, while maintaining high performance across various tasks and model architectures.", "questions": "- Recent models like Qwen3 have adopted a pre-RoPE normalization layer (https://github.com/huggingface/transformers/blob/main/src/transformers/models/qwen3/modeling_qwen3.py#L185), which retains the polar characteristics of RoPE while reducing the quantization burden. Would PolarQuant still be applicable or effective for such models?\n\n- How was the value cache (V-cache) quantization method chosen? Was it selected because KIVI demonstrated the best performance among existing approaches\n\n- Given that PolarQuant relies on the structured circular patterns induced by RoPE, how sensitive is the method to variations in RoPE configurations (e.g., different base frequencies or dimensional pairings)? Could this affect its generalizability across diverse model architectures or custom RoPE implementations?", "limitations": "-", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strength:\n\n- The use of polar coordinates to mitigate outlier effects in key vectors is a novel and elegant idea. The authors show that outliers often appear in only one of the two dimensions rotated by RoPE, and that these pairs form smooth circular patterns in polar space, making them more amenable to quantization.\n- The authors provide detailed implementation and reproducibility information, which is commendable.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable comments and hope our responses address your concerns.\n\n---\n\n**Q1**: More diverse model architectures like Qwen3 ?\n\nIn our experiments, we focus on popular model architectures, such as Llama and Qwen, as we believe they have demonstrated the broad applicability of PolarQuant across different architectures.\n\nYour observation is very insightful, and we agree that the pre-RoPE normalization holds significant value.\nGiven the time constraints, we have decided to leave the exploration of integrating PolarQuant with Qwen3 for future work.\nThank you for your valuable suggestion.\n\n---\n\n**Q2**: How was the value cache quantization method chosen ?\n\nOur paper focuses on addressing the challenges of quantizing keys, and is orthogonal to specific value quantization algorithms, as demonstrated in the following aspects:\n\n1. In Tables 1, 2, 3, 5, we maintain the value cache in full precision. This effectively isolate the influence of value quantization algorithms and clearly demonstrate the effectiveness of our method in addressing the challenges of key quantization.\n2. After establishing the effectiveness of our method, in Table 6, we employ KIVI to quantize values. This step is taken to showcase the compatibility of PolarQuant with value quantization algorithm, thus making our paper more comprehensive. **We selected KIVI simply because it is a classic method and a frequently compared baseline in the field.**\n\nTaken together, these twofold experiments demonstrate that PolarQuant works well with both full-precision values to low-bit quantized  values using classic methods—without performance degradation.\n\n---\n\n**Q3**: How sensitive is PolarQuant to the RoPE configuration? Could this affect its generalizability across diverse model architectures or custom RoPE implementations?\n\nThank you for the insightful question. We have experimented with diverse model architectures, each with a unique RoPE setup, including Llama-2-7B-Chat (base frequency 10000), Llama-3.1-8B-Instruct (with rope base as 500000), and Qwen-2.5 (with rope base as 1000000).\n\nIn addition, we show that PolarQuant is adaptable to different RoPE variants, which addresses your concern about its generalization to custom RoPE implementations. Specifically, we perform experiments with NTK-aware Scaled RoPE on Llama-2-7B-Chat to demonstrate this.\n\n|Method|LongBench (avg.)|\n|-|-|\n|NTK| 32.15 |\n|NTK-Polar44| 32.44|\n\nThe experimental results indicate that PolarQuant can adapt well to the customized RoPE configuration, hope this can address your concern."}, {"review_id": "pttJFSDiv0", "reviewer": "Reviewer_bUkd", "summary": "This paper proposes to quantize KV cache through effectively addressing outlier challenges via polar transformation.\nSpecifically, the authors observe that although the individual axis may contain outliers, they collectively form stable circular patterns, making quantization of the original outliers easier.\nExperimental results include models like Llama and Qwen, showing competitive performance with 3-4 bit quantization and up to 3.18x speedup.", "questions": "1. Why does Table 4 show PolarQuant using less memory than KIVI despite lookup tables?\n2. How does group size $g$ (Sec 3.2) affect performance? With small $g$, there might not be outliers in each group. Will PolarQuant still maintain its superiority under that circumstance?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Polar transformation novelly exploits RoPE-induced circular patterns (Fig 1b) to mitigate channel-wise outliers.\n2. Polar transformation enables decoding acceleration through lookup tables, speeding up attention computation.\n3. Comprehensive evaluation on long-context (LongBench), normal context (GSM8K) and reasoning (DeepSeek-R1) across multiple model families.\n4. Clear methodology and abundant implementation details.\n\nWeaknesses:\n1. Figure 1a claims to show \"significantly larger activations\" but lacks visual evidence of outliers (no obvious outliers in channel dimension).\n2. Given that key value quantization aims to tackle memory bottleneck, the lookup table in PolarQuant incurs extra memory overhead, so it is unclear why memory usage of PolarQuant is smaller than KIVI quant (shown in Table 4).", "was_revised": true, "final_justification": "Most of my concerns have been addressed by authors' reply. Given its contributions, I choose to maintain my original score.", "rebuttal": "We sincerely appreciate your valuable comments! We hope our response will help address your concerns.\n\n--- \n\n**W1**: Figure 1 needs refinement in presentation.\n\nThank you for the valuable suggestion to improve the presentation of our paper. \nWe will choose more contrasting colors to better highlight the presence of outliers, thereby enhancing the overall presentation of the paper.\n\n---\n\n**W2 & Q1**：Why does Table 4 show PolarQuant using less memory than KIVI despite lookup tables?\n\n> First of all，here is a misunderstaning that, the lookup table in PolarQuant incurs extra memory overhead.\n\nWe want to clarify that the lookup table is actually not cached; instead, it is computed on the fly at block level during decoding (For more details on the implementation, please refer to Appendix A). \nAs a result, therotically there should be no obvious difference in memory overhead for PolarQuant and KIVI when the same bit budget is used.\n\n> Second, your observation is insightful that the setups in Table 4 show differences in memory usage, and there are some theoretical discrepancies.\n\n - For the second comparison group (KIVI-2 & Polar-33):\n\nPolar-33, has higher memory usage than KIVI2 (with more than 1GB at 128K), which is due to the fact that we use our customized 4-bit kernel for Polar-33 in our implementation, introducing 1 bit of redundant memory waste.\n\nThe 3-bit kernel implementation requires additional integer unpacking and is complicated to implement, so we use our 4-bit kernel as a replacement.\n\n - For the third comparison group (KIVI-4† & Polar-44†):\n\nWe assume that some hardware issues might be the cause and are currently investigating.\n\nIn Polar_44†, we use our kernel implementation for key and KIVI kernel for value.\nIn KIVI-4†, the dequantization and matmul operations for both the key and value share the same kernel.\nIf the intermediate results by repeated kernel calls are not released, GPU memory usage may accumulate continuously, causing peak memory usage to rise.\n\nWe are actively working on pinpointing the memory differences mentioned above. Thank you for your question!\n\n---\n\n**Q2**: Ablation studies on group size g ?\n\nThanks for your suggestion! We conduct an ablation study exploring different group size using Llama-3.1-8B-instruct. \n\nThe table below presents the performance of PolarQuant and KIVI on LongBench with a 4-bit setting (* indicates better preformance with same group size):\n\n| model | group size | bits | LongBench (Avg.) |\n|-|-|-|-|\n| KIVI  | 32         | 5.00 |     49.48        |\n| KIVI  | 64         | 4.50 |     49.47*       |\n| KIVI  | 128        | 4.25 |     49.36        |\n| KIVI  | 256        | 4.13 |     49.52        |\n| Polar | 32         | 5.00 |     49.50*       |\n| Polar | 64         | 4.50 |     49.33        |\n| Polar | 128        | 4.25 |     49.39*       |\n| Polar | 256        | 4.13 |     49.58*       |\n\nThis suggests that PolarQuant performs competitively with KIVI across different group sizes.\n\n> With small g, there might not be outliers in each group. Will PolarQuant still maintain its superiority under that circumstance?\n\nWe would like to make two points in response:\n\n(1) The quantization parameters are of size 32/g bits. It is clear that smaller group sizes introduce an additional cost.\n\n(2) Considering the overhead of quantization parameters caused by a smaller g and the sufficiently good performance with a relatively large g, we recommend using 128, which is the setting used in our paper.\n\n---"}, {"review_id": "f4gVYg4KFA", "reviewer": "Reviewer_FV6N", "summary": "The paper presents PolarQuant, an approach that applies a polar transform on the KV caches and quantizes the angle and radius.", "questions": "1. Can you please explain how the idea can be mixed with rotation-based quantization techniques like [1]? For example, can't we apply Hadamards of size 2 and fuse it into the weights of the previous layers (for W_k and W_v) and then quantize the KV?\n\n2. I found the figures a bit complex and not easy to understand. Can you please polish them and make them easier?\n\n3. Can you provide an ablation on the group sizes during the quantization?\n\n4. I would like to see a breakdown over the runtimes to see the overhead of getting polar coordinates (like atan2 operation) especially during the generation phase which is the main target of KV-cache quantization.\n\n5. Can you please provide more details about the GPU architecture you have yoused for the benchmarking experiments in the paper? \n\n6. Do you have any explanation about the \"Observation 1\" in line 252? Why the angles need more precision here? Do you have any intuition?\n\n[1] https://arxiv.org/abs/2404.00456", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- KV-cache compression is an important area for LLM efficiency (especially for long-context inference) and the intuition of using a polar transformation is very clever. I found the idea (and observations) interesting\n- The authors provide a real latency evaluation and compare them against the SoTA KV-cache quantization schemes\n- Paper is well-written\n\nWeaknesses\n\n- Figures are a bit complex and hard to understand", "was_revised": true, "final_justification": "I found the idea of quantizing the angles (instead of the whole vector) for KV-cache quantization quite interesting. This is something that we may be able to use in other quantization schemes as the angles between embedding vectors is the most important piece of information that the model has (and adds during the inference). However, I think my initial score is fair and didn't change it after discussion with the authors.", "rebuttal": "We sincerely thank you for your constructive suggestions and valuable comments! We hope our rebuttal helps address your concerns.\n\n---\n**W1 & Q2**: Figures need refinement in presentation.\n\nThank you for your helpful suggestion.\nWe will refine Figure 1 and Figure 2 to more effectively showcase our insights and pipeline. \n\n---\n\n**Q1**: Can PolarQuant combine with the rotation-based weight quantization?\n\nWe find your question interesting, but in this work we primarily lies in activation quantization (e.g., KV cache quantization) rather than weight quantization. There are differences between the two domains, both in their core focuses and in the inherent characteristics of weights versus activations. This means we cannot immediately speak to the feasibility of combining PolarQuant with rotation-based weight quantization.\n\nWe acknowledge the value of exploring such intersections and need some time to delve into the paper you referenced to gain deeper insights, which may inform and guide the development of our future work. Thank you again for your interest and for bringing this perspective to our attention.\n\n---\n\n**Q3**: Ablation on the group size.\n\nThank you for the insightful suggestion! We conduct an ablation study with different group sizes using Llama-3.1-8B-Instruct. The average results across datasets from LongBench are presented below:\n\n| method   | group size |  LongBench (Avg.) |\n|-|-|-|\n| Polar_44 |     32     |      49.50        |\n| Polar_44 |     64     |      49.33        |\n| Polar_44 | \t\t128     |      49.39        |\n| Polar_44 |    256     |      49.58        |\n| Polar_33 |     32     |      49.68        |\n| Polar_33 |     64     |      49.38        |\n| Polar_33 | \t\t128     |      49.39        |\n| Polar_33 |    256     |      49.09        |\n\nWe find that PolarQuant maintains consistent performance across different group size settings.\n\nThank you for the suggestion, we will include this discussion in our revised version.\n\n---\n\n**Q4**: Breakdown time analysis of key cache quantization in PolarQuant.\n\nThank you for your thoughtful question. Your input is valuable in refining our paper’s presentation.\n\nWe perform a time breakdown analysis of PolarQuant during the generation phase (with a batch size of 1, input length of 256, and generation length of 4096):\n| Operation |         |   Time (s) |\n|-|-|-|\n| Decode    |  All    |   91.07    |\n| Quant     |  Others |    0.45    |\n|           |  Tan2   |    0.10    |\n\nFrom this analysis, we can see that the overhead introduced by the new key quantization is only 0.6% in comparison to the overall generation cost.\n\nWe appreciate your suggestion and hope this can clarify your concerns.\n\n---\n\n**Q5**: Details about the GPU architecture.\n\nWe benchmark our experiments on NVIDIA A800-SXM4-80GB. Thank you for the reminder, I will detail the hardware we use in the revised paper.\n\n---\n\n**Q6**: Why the angles need more precision?\n\nWe notice a significant decline in downstream performance when fewer than 3 bits are assigned to the angles. \nAs indicated in Table 5, the (r4, t2) configuration shows a considerable performance drop compared to both the (r3, t3) and (r4, t3) settings.\n\nThe angle should have higher precision to ensure that the differences in rotation angles between token positions are as distinguishable as possible, because RoPE introduces positional information based on the differences in these rotation angles. \n\nSpecifically, for the lower dimensions of RoPE, after increasing the token position by a group size (g), they undergo multiple full cycles (i.e., the input is rotated by more than 2π). As a result, the quantization of these high-frequency signals in the lower dimensions of RoPE requires higher precision.\n\nOur experimental results in Table 5 also validate the above point.\n\n---"}]

Author rebuttal:

Meta-review: This paper introduces PolarQuant, a method for key-cache quantization using polar coordinates to address RoPE-induced outliers, combined with a lookup-table–based decoding acceleration.

The approach is well-motivated and empirically validated across multiple models and datasets, demonstrating notable speedups and robustness. Reviewers highlight the innovative use of polar transformation and its practical acceleration benefits.
Some clarifications remain regarding pre-RoPE normalization, sensitivity to RoPE configurations, recomputation with lookup tables, and ablations on group size and base frequencies, which should be included in the final version.

Overall,  the work presents a creative, technically sound, and practically impactful contribution.

Decision: Accept
Decision: Accept (poster)