CollectiveKV: Decoupling and Sharing Collaborative Information in Sequential Recommendation
Authors: Jingyu Li
Zhaocheng Du
Qianhui Zhu
kaiyuan Li
Zhicheng Zhang
Song-Li Wu
Chaolang Li
Pengwen Dai
Venue: ICLR 2026 Poster (2026)
Keywords: KV cache
Sequential recommendation
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: Sequential recommendation models are widely used in applications, yet they face stringent latency requirements. 
Mainstream models leverage the Transformer attention mechanism to improve performance, but its computational complexity grows with the sequence length, leading to a latency challenge for long sequences.
Consequently, KV cache technology has recently been explored in sequential recommendation systems to reduce inference latency.
However, KV cache introduces substantial storage overhead in sequential recommendation systems, which often have a large user base with potentially very long user history sequences.
In this work, we observe that KV sequences across different users exhibit significant similarities, indicating the existence of collaborative signals in KV. 
Furthermore, we analyze the KV using singular value decomposition (SVD) and find that the information in KV can be divided into two parts: the majority of the information is shareable across users, while a small portion is user-specific.
Motivated by this, we propose CollectiveKV, a cross-user KV sharing mechanism.
It captures the information shared across users through a learnable global KV pool. During inference, each user retrieves high-dimensional shared KV from the pool and concatenates them with low-dimensional user-specific KV to obtain the final KV.
Experiments on five sequential recommendation models and three datasets show that our method can compress the KV cache to only 0.8\% of its original size, while maintaining or even enhancing model performance.

Published as a conference paper at ICLR 2026
COLLECTIVERV: DECOUPLING AND SHARING COL- LABORATIVE INFORMATION IN SEQUENTIAL RECOM- MENDATION
Jingyu Zhaocheng Duu; Qianhui Zhu", kaiyuan Li Zhicheng Zhang', Song-Li Wu', Chaolang Li Pengwen Dai
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University Huawei Noah's 's Ark Lab
Shennzeen Key Laboratory of Adversarial Artificial Intelligence
"Tsinghua Shenzhen International Graduate School {1ijy768, 1i 1ichlang5] @mail2 sysu, edu. cn, duzhaochengl 9980 com (zhugh23, 1 1skaiyua23, da daipwemai 1. sysu. edu.cn
-cang ws ws124] is..iilseee ils.ssinghua sedingha.eed..... edu. cn
ABSTRACT
Sequential recommendation models are widely used in applications, yet they face stringent latency requirements. Mainstream models leverage the Transformer at- tention mechanism improve performance, but its computational complexity grows with the sequence length, leading to a latency challenge for long sequences. Consequently, KV cache technology has recently been explored in sequential rec- ommendation systems reduce inference latency. However, KV cache introduces substantial storage overhead in sequential recommendation systems, which often have large user base with potentially very long user history sequences. In this work, we observe that KV sequences across different users exhibit significant sim- ilarities, indicating the existence of collaborative signals in K.V. Furthermore, we analyze the KV using singular value decomposition (SVD) and find that the in- formation in KV can be divided into two parts: the majority of the information is shareable across users, while a small portion is user-specific. Motivated by this, we propose CollectiveKV, the information shared across users through a learnable global KV pool. Dur- ing inference, each user retrieves high-dimensional shared KV from the pool and concatenates them with low-dimensional user-specific KV to obtain the final KVV. Experiments on five sequential recommendation models and three datasets show that method can compress the KV cache to only 0.8% of its original size, while maintaining or even enhancing model performance.
cross-user KV sharing mechanism. It captures
INTRODUCTION
Sequential recommendation models widely used in Internet applications such as short video platforms, e-commerce, and real-time advertising, where ensuring a smooth user experience requires meeting stringent latency constraints. Existing methods usually focus on improving the accuracy of recommendations by introducing or improving the Transformer attention mechanism Vaswani et al. (2017). However, the computational cost of attention grows with the input sequence length, making it challenging to satisfy the strict latency requirements in practical deployment.
In the attention mechanism, projecting the input sequence into queries (Q), keys (K), and values (V) via linear transformations accounts for a significant portion of the computational cost. To mitigate inference latency, recent works such as HSTU Zhai al. (2024) and MARM Lv et al. (2024) intro- duce the KV cache technology into recommendation systems, which precomputes and stores keys
"These authors contributed equally to this work. Corresponning author

---

Published as a conference paper at ICLR 2026
and values for reuse during inference. However, sequential recommendation systems usually serve a massive user base, where each user may have a long behavior history Zhang et al. (026)); Ma et al. 2025 : Yu et al. (2026). When user history sequences projected into the KV cache, they will in- cur substantial storage overhead and introduce considerable communication latency. This motivates the need for efficient KV cache compression.
Recently, many works |haazeer (2019): Ainslie et al. 2023); : Singhania et al. (2024): Tang et al. (2024) Liu al. (2024) have attempted to reduce the KV cache size in LLMs, primarily by com- pressing each KV sequence through reducing its length or dimension. In contrast, sequential recom- mendation scenarios present unique property: user behaviors are not independent but interrelated, exhibiting strong collaborative signals. Motivated by this, we analyze the similarities of Keys and Values across users in multiple sequential recommendation models. As shown in Figure T different users indeed display a certain degree of K/V similarity, suggesting that collaborative signals are also embedded in the K/V representations. This observation inspires us to explore a new compression shaiing the KV cache across users.
To further quantify the shareable portion of K/V, we apply singular value decomposition (SVD) to decompose the keys and values into principal component subspace and a residual subspace, denoted as principal K/V and residual K/V, respectively. The principal K/V captures the majority of the information, while the residual K/V contains only small fraction. We then measure cross- user similarities separately on these two parts. As shown in Figure 2 principal K/V are more likely to exhibit positive negative correlations across users, whereas residual K/V similarities cluster around zero. This finding suggests that most information in the keys and values can be shared across users, while only a small portion remains user-specific.
Motivated by these results, we decompose the KV into two parts: the low-dimensional user-specific KV and the high-dimensional collective KV shared across users. To be specific, we utilize a learn- able global K/V pool to capture the information shared across users. In this pool, the global KVs are designed to be high-dimensional to provide greater information capacity. Each user retrieves the corresponding shared K/V-refernt on their own behavior sequence. On the other side, the user-specific KVs are represented as low- dimensional linear projections of each user's historical behavior sequence embeddings. Finally, the user-specific K/V and collective K/V are concatenated to formulate the final K/V. In this way, by introducing the sharing mechanism, the user-specific KV only needs to carry limited information, allowing it to be represented with just a few dimensions, thereby reducing the KV cache size. Ex- periments on both target attention and self-attention models demonstrate that our method achieves excellent compression while preserving or even improving model performance.
to as the collective fromrom the global K/V pool, based
BACKGROUND AND RELATED WORK
2.1 KV CACHE IN SEQUENTIAL RECOMMENDATION SYSTEMS
In sequential recommendation models, target attention and self-attention are two widely used atten- tion mechanisms. In target attention models pi al. (2000) Cao al. (022); Chen et al. (2022); Chang et al. (2023), the query Q E Rixd is projected from a target item, where d is the head di- mension. In self-attention models Kang McAuley 2018 Zhai et al. 2024), the query Q xxd is projected from the input sequence, where s is the sequence length. In both paradigms, the key K Rxd and value V E IR are projected from the input sequence, meaning that their shapes are identical and both can benefit from the KV cache mechanism.
Building on this property, recent works such as HSTU Zhai al. (2024) and MARM Lv et al. (024)) introduce precomputing and caching the keys and values before online inference, thereby reducing online inference latency. These studies demonstrate the effectiveness of KV cache in sequential rec- ommendation. However, unlike in LLM scenarios where the KV cache is maintained per sequence during a single inference session, recommendation systems must maintain the KV cache per user. Given the massive user base (often hundreds of millions) and the potentially long interaction history of each user, the aggregate KV cache size quickly exceeds GPU memory capacity. Consequently, offloading KV caches to CPU memory external storage becomes inevitable. Yet this offloading introduces significant transfer latency between the GPU and external storage. Moreover, the KV cache for all users requires substantial storage space, further challenging large-scale deployment.
2

---

Published as a conference paper at ICLR 2026
Gay
33
25-
Deaatty
03
-2.5 -0.3 0.00
030
075 1B
-24 -22
50
62
04
20-
-24 -5.2
65
32
58
38
Codre Smarty
Simirtyy
Cosne intly
(a) SIM, Microvideo
(b) SDIM, Microvideo
(c) TWIN, Microviden
Mr
Rey
Sy
2.3
Dupsite
1.3
10
Cosine Sinilatty
Cosine Siibaiiy
Cosie Simiarity
(d) SIM, Kuaivideo
(c) SDIM, Kuaivideo
(f) TWIN, Kuaivideo
Figure 1: Probability density (estimated via kernel density estimation) of cosine similarities of K/v
between different users on two datasets and three models.
These issues together highlight the necessity of developing KV cache compression techniques that can simultaneously reduce memory usage and inference latency.
2.2 KV CACHE COMPRESSION
Existing approaches KV cache compression be broadly categorized into types: methods that reduce the sequence length s of the KV cache, and methods that reduce its dimensionality d. In the first category, only the tokens deemed important are retained, while the KV caches of less informative tokens are discarded, along with the information they contain. For example, Loki Singhania et al. al. (2024) leverages PCA compute sparse attention scores and selects the top-k tokens accordingly. Similarly. Quest Tang et al.] (202)) selects past tokens based on the current token, and InfiniGen Lee et al. (2024) predicts important tokens the next layer based on the current layer's attention inputs. These methods are typically training-free but may result in performance degradation due to information loss.
The second category targets the KV dimensionality, often requiring architectural modifications and end-to-end training. A representative example is MLA al. (2024), which first projects the high- dimensional input into low-dimensional feature space and then applies separate linear mappings to obtain and V, thereby reducing storage requirements. While effective, these methods often introduce additional computational overhead.
Notably, these methods are primarily designed for LLMs, focusing on compressing the KV cache for individual sequences. When applied directly recommendation scenarios, they fail exploit the collaborative signals naturally present across users. In contrast, our work demonstrates that in sequential recommendation models, a substantial portion of KV information be shared across users. By leveraging this cross-user similarity, the KV cache can be further compressed without sacrificing model performance.
3 ANALYSIS OF KEYS AND VALUES
3.1 CROSS-USER SIMILARITY
We first investigate whether there exists high degree of similarity in keys and values across users, to validate the feasibility of sharing them. Specifically, taking the key as an example, we randomly sample 1,000 users from the dataset, where each user's keys represented as matrices of shape n x (with n denoting the length of the historical sequence, which may vary across users, and denoting the attention dimension, which is consistent across users). We then compute mean key
3

---

Published as a conference paper at ICLR 2026
Recitol
Reditoed
Prnopp Vdue
Precpu Nukuz
Prinppd Vilus
Nolbba Yalas
Pao cual Value
Rabud Valae
1.5
1.3
10
25
-10
2.5 Costa 5m Bertyy
-20
-5.
05
1.0
20
-10
-55
15
Coabte Simtaty
Cosne 5im larty
(a) SIM, MicroVideo
(h) SDIM, Micro Video
(c) TWIN, Micro Video
Marrpll Kry
Wrnpp By
Prrcpd Roy
key
Rad dual Uai
seddia Vil
Tha dual Wess
0.3
35
Codine 5 Smilukt
Cishte Siniiar by
Cosre Snemmmbiii ballly
(d) SIM, KuaiVideo
(e) SDIM, KuaiVides
(I) TWIN, KuaiVideo
Figure 2: Probability density (estimated via kernel density cstimation) of cosine similarity of prin-
cipal K/V and residual K/V across users.
vector K of size 1 x d for each user by averaging along the sequence dimension. Therefore, the cosine similarity of the mean keys between users can be formulated as:
(ui, Uz) cosine Kj), K
K..: E ER
Next, we randomly select the mean key vector of a user and compute its cosine similarities with the mean key vectors of all other users. Finally, we estimated the probability density of these similarities via kernel density estimation (KDH). As shown in Figure [] on three models and two datasets, the keys and values exhibit a high degree of positive cosine similarity across users, which suggests that sharing K/V representations among users feasible.
3.2 KV INFORMATION SHAREABILITY
To investigate the proportion of shareable information in the KV cache, we project the K/V into the principal component subspace and the residual subspace, and then compute their cross-user similarities. Specifically, taking the key vector K E ga (where the sequence length > d and the dimension = 256) as an example, we first perform singular value decomposition (SVD) on it,
K -EV," Ve dxd
(2)
Then we project the key into the top-k (A: 10 in experiments) principal components obtain the principal component key K, E Rink which preserves the majority the information (typically 90% when k 10). Subsequently, we project the key into the remaining residual dimensions to obtain the residual key K, E (d-RR, which carries only small fraction of the information (typically < 10% when 10),
K, KV A). K, = KV[,
Next, we compute the cross-user similarity based on the principal keys and residual keys,
Sk(u,, u,) cosine(K)) KT). ST(u,, uy) cosine K, KJ).
(4)
Finally, we estimate their probability distributions using the method in Sec. 3.1 As shown in Figure 2. the similarity distribution of principal K/V is relatively flat, spanning a wide range between (- I. 1). In contrast, the similarity distribution of residual K/V is highly concentrated around zero, with most values falling within the range (-0.5, 0.5). Consequently, principal K/V, which carry the majority of information, containing much less information, are more user-specific. This leads to the conclusion that most of the information contained in K/V potentially be shared across users.
more likely to exhibit cross-user correlations, whereas residual K/V,
4

---

Published as a conference paper at ICLR 2026
Prefill Stage
Decode Stage
Lincar Projection
Final K/V
User A
de
Atin.
User-speeific K/V
Query
Router
Gather
History Sequence Embedding
Shared Parameters
Global K/V
User A
Cache
User
History Sequence Embedding
g
Router
Gather
Global K/V Indices
Quxry
9n
de
Attn.
User-specific K/V
Prefill Stage
User B
Linear Projection
Decode Stage
K/V
Figure 3: Overall framework of CollectiveKV. All the network modules, including the router net- work, the linear projection layer, and the global K/V pool, have the same parameters across different users. During prefilling, user-specific K/V and global K/V indices are calculated and cached.
COLLECTIVEKV
4.1 DECOMPOSING KV
As analyzed in Sec 3 the information contained in the KV can be decoupled into two parts: user- specific information and cross-user shared information. Based on this analysis, we present Col- lectiveKV, item-level cross-user KV sharing strategy. In CollectiveKV, the KV is decomposed into two components: the user-specific KV and the collective KV.. The user-specific KV low- dimensional and encodes the personalized signals unique each user's behavior sequence. In con- trast, the collective KV is high-dimensional and shared across users, capturing the collaborative information that is common in recommendation scenarios.
4.2 FRAMEWORK
In real-time sequential recommendation systems, the computation of the attention module can be divided into two stages- the prefill stage and the decode stage- reduce inference latency. In the prefill stage, the user's historical sequence is projected into keys and values and stored as KV cache. When a user request arrives, the user enters the decode stage, where the cached KV retrieved to compute attention scores with the user's query.
As shown in Figure our method follows the practice of separating the prefill and decode stages.
In the prefill stage, given input embedding S {s1, 12, 5n) Raxd, (typically the user history sequence embedding) with length of n and dimension of de, we use a linear projection to reduce its dimensionality and get user-specific key K, RR or value V. Raxd, denoted as,
K, =W bk: V. SW, +b..
(5)
where Wk. We E Bdd xd, are learnable projection matrices and by. by E Rd are learnable bias.
Meanwhile, router network maps the sequence embedding into series of item-level global K/V indices I,, I, E Nax1, The details of the router are presented in the next section. After that, both the global K/V indices and the user-specific K/V will be cached.
In the decode stage, both of them will be retrieved from the cache. The indices serve to gather level keys values from learnable global K/V pool Pk, P, Rmxd, to form the collective
5

---

Published as a conference paper at ICLR 2026
Ke Ve Ruxds where is the pool size and dg is the global K/V dimension,
K Pk [ Ve = Pv [+]]
i ==
(6)
Then, we concatenate user-specific K/V and collective K/V to get the final K, V E Rexd (do du + dg) as.
=
K
concat(I Ke), V concat V Ve).
Finally, the final KV and the user's query serve to compute the attention output.
4.3 COLLECTIVERV ROUTER
The information across users is shared through the global K/V pool with a special router network, which end-to-end trained. As illustrated in Figure 3 the router maps the embedding into global K/V indices, which are then used to gather the collective K/V from the pool.
Specifically, taking the key as an example, the router first projects the embedding S gnxx into routing map M E Raxm, Then, for each item, it selects the index of the maximum element along second dimension to form global key indices I Nax1;
M SW, br: Ix = arg max Mij.
where W,,, b, are parameters to be learned. After that, the collective keys are gathered according to Eq. the next step, the model behaves slightly differently during inference and training.
During inference, it directly concatenates the collective keys and the user-specific keys as Eq. However, if the same operation is applied during training. the gradients cannot be propagated through the indices back to the router network. Therefore, during training, we multiply the collec- tive key by the corresponding sigmoid output of the router, through which gradients can be success- fully backpropagated to the router. So the Eq. G actually formulated as:
[o(Mi I(l) PR]I if training, [x]]]
K
I...., n
inference,
where a denotes the sigmoid activation function. However, we need to ensure consistency between training and inference by encouraging the sigmoid weights to be close 1. At this point, the reason we chose sigmoid activation becomes clear: simply encouraging the input weights to be large could guarantee the sigmoid outputs close 1. Consequently, we introduce peak loss to supervise the outputs of the router network:
n Spaak Σ
log (M(Mi, bx))
(10)
T
Meanwhile, ensure that each key in the global key pool learns meaningful information, we need to encourage every key to have equal probability of being selected. So we introduce load balance loss to supervise the outputs of the router as:
1 m
Chalance
Σ Py log
D3
(oftmax(M)as
(11)
where Pú denotes the average probability of selecting the j-th key across the sequence, and u the uniform probability. This loss is the KL divergence between the average selection probabilities of the keys and a uniform distribution.
4.4 LATENCY ANALYSIS
During the decode stage, the latency introduced by the original KV cache technique mainly stems from the loading delay, i.e., transferring K/V from external storage to GPU memory. This latency can be described as:
sda
Tload(sda) = Tectup +
(12)
B

---

Published as a conference paper at ICLR 2026
Table 1: The performance of our method on different models and datasets. The CR metric denotes the compression rate of the KV cache. Ebnerd Kruse et al. (2024) has not been adapted for HSTU in our work; thus, we do not include it for comparison in the table.
MicroVideo
Kuai KuaiVidoo
Ebnerd
GAUCT
AUC1
Logloss
CR]
GAUC1
AUC
Logloss
CRL
GAUC1
AUC1
Logloss
CR.
SIM
0.6954
0.6933
0.4282
/
0.6577
0.6798
0.4620
/
0.6975
0.7024
0.2754
/
SIM+ours
0.6973
0.7057
0.4203
0.016
0.6604
0.6900
0.4520
0.012
0.7003
0.7088
0.2749
0.027
SDIM
0.6857
0.6749
0,4389
0.6506
0.6766
0,4611
0
0.6890
0.6948
0,2760
SDIM-ours
0,6883
0.6871
0.4323
0.012
0.6345
0.6786
0,4583
0.016
0.6941
0.6956
0.2713
0.008
ETA 0.6960
0.6911
0.4325
0.6699
0,6886
0.4577
0.6931
0.6990
0.2717
ETA+ours
0.6984
0.7015
0.4295
0.012
0.6677
0.6889
0,4561
0,020
0.7020
0.7032
0,2686
0.027
TWIN
0.6984
0.7064
0.4233
0
0.6483
0.6756
0.4618
/
0.6969
0.7028
0.2710
TWIN+ous
0.6990
0.7074
0.4226
0.016
0.6558
0.6786
0.4627
0.020
0.7051
0.7081
0.2679
0.016
HSTU 0,6859
0,7381
0.4101
0.6663
0.7454
0,4388
/
HSTU+ours
0.6862
0.7385
0.4100
0.043
0.6668
0.7475
0.4398
0.012
where sd, is the KV cache size, sectup is the transfer setup latency, and B is the bus bandwidth.
In our approach, the loading latency is denoted as (sdu + s), where da < da and the external 8 represents the global KV indices. After this loading step, the model needs to gather the collective KV from the global K/V pool according to the global KV indices. Since the global K/V pool resides entirely in GPU memory, this operation amounts to a GPU memory indexing, whose latency is much lower compared to Toad- We denote the latency of this indexing operation Tidee (sdg), where sdy is the collective KV size.
In summary, the latency of our method is (soad(sd, +s) Tindez (sde). The latency comparison experiments with the KV cache are presented in Sec. 5.2
EXPERIMENTS
5.1 MODELS, DATASETS AND METRICS
Models. To comprehensively evaluate the effectiveness of our proposed KV sharing strategy, we consider both target-attention-bas and self-attention based recommendation models. Specifically, we select four representative models built upon the target attention paradigm: SIM et 20200) SDIM Cao et al. (2022), ETA Chen et al. (2022). and TWIN Chang et al. 2023). In addition, we reproduce HSTU Zhai et al. (2024), a self sete-attention- based model that has recently introduced the KV cache mechanism into recommendation. The implementations of the above baseline models and the corresponding experimental pipeline largely reuse the existing FuxiCTR codebase and experimental setups from prior work Zhu et al. (2022 2021).
Datasets. We evaluate our method on three seng-sequence CTR prediction datasets. Specifically, we use Micro Chen et al. (2018), widely adopted benchmark dataset for short-video rec- ommendation; Video, which originates from the Kuaishou competition at ChinaMM 2018 and contains rich user-video interaction sequences; and EBNeRD-Smin Kruse al. (2024). Danish news recommendation dataset. These datasets are characterized by long user behavior histories, which make them suitable for evaluating the efficiency and effectiveness of KV caching strategies. Moreover, the dataset splitting strategy and configuration settings follow the protocol introduced in LAIN Zhang et al. (0266).
Metrics. We adopt three widely used evaluation metrics: AUC, GAUC, and Logloss. In addition, introduce the Compression Rate (CR) assess the efficiency of KV cache compression, defined as the ratio between the reduced KV cache size and the original one. Note that the CollectiveK cache includes two parts: the global KV indices and the user-specific KV.
5.2 COMPARISON WITH BASELINE
The baseline setting is keeping the model unchanged, i.e., retaining the original target attention or self-attention structure. As shown in Table [
our proposed method significantly reduces the KV
7

---

Published as a conference paper at ICLR 2026
Table 2: Latency analysis in comparison with KV cache, conducted on an A100 GPU.
Batch Size
8
32
64
128
256
512
KV Cache Latency (ms)
0.099
1.030
1.808
3,418
6.679
15.216
32.991
CollectiveKV Latency (ms)
0.084
0.129
0.136
0.152
0.222
0.375
0.695
(a) GAUC
(b) AUC
(c) Logloss
Figure 4: The influence of the user-specific feature dimension on the model performance.
cache size, achieving a compression ratio as low as 0.008, while maintaining or even enhancing model performance compared to the baseline.
Specifically, two target attention models, SIM Pi al. (2200) and SDIM Cao al. (222)) our method not only substantially compresses the KV cache but also improves model performance. In the other two target attention models, ETA Chen et al. (2022) and TWIN Chang al. (023)), method also achieves better performance attention model HSTU et al. (2024), our method also surpasses the baseline on the Micro Video dataset. On the Kuaivideo dataset, method also achieves higher GAUC and AUC with compres- sion rate of 0.012. These satisfying results demonstrate that CollectiveKV can effectively capture collaborative information among users, thereby enhancing recommendation accuracy.
the Video and Ebnerd datasets. In the self-
Furthermore, we conduct experiments to measure and compare the actual latency between the stan- dard KV cache and our CollectiveKV cache. The experiments are conducted on the SIM model and the Kuaivideo dataset. For the standard KV cache, we measure the average time required to load a batch of keys from CPU memory to GPU memory as its latency. For CollectiveKV. we measure the total time in the decode stage before the attention calculation as its latency.
As shown in Table [ our method achieves significantly lower latency compared to the original KV cache. Moreover, as the batch size increases, the latency of the KV cache grows sharply. Specifically, when the batch size increases from 512, the latency of the KV cache rises from 0.099 ms to 32.991 ms; in contrast, the latency of our method only increases from 0.084 ms to 0,695 ms.
5.3 INFLUENCE OF THE
INFORMATION
As analyzed in Sec 3.2 user-specific information might constitute only a small fraction of the overall K/V representations, while the majority of the information could be shared across users. Here, investigate how varying the proportion of user-specific information affects the model performance.
Specifically, we evaluate model performance under different user-specific KV dimensions using the SIM model and the MicroVideo dataset, and the results with a quadratic polynomial. As shown in Figure 4 the model performance first decreases and then increases. This is consistent with the findings in Section 3.22 a smaller user-specific dimension better matches the distribution of collaborative and user-specific signals. However, the dimension continues to increase, the shared dimension decreases, and Co gradually degenerates into the original KV.
5.4 DISCUSSION OF GLOBAL K/V POOL SIZE
The size of the global K/V pool determines the upper bound of the capacity for learnable shared information. As the user population grows, the amount shared information might increase, ne- cessitating corresponding enlargement of the global K/V pool, To investigate how the pool size
8

---

Published as a conference paper at ICLR 2026
2.122-
8:00
0.02
(a) GAUC
(b) AUC
(c) Logloss
Figure 5: The influence of the global K/V pool size on the model performance.
Table Ablation experiments on the peak loss and the balance loss.
MicroVideo
KuaiVideo AUC
Fibnerd AUC
GAUC
AUC
Logloss
GAUC
Logloss
GAUC
Logloss
Baseline
0.6954
0.6933
0.4282
0.6577
0.6798
0.4620
0.6975
0.7024
0.2754
CollectiveKV
0.6971
0.7105
0.4339
0.6604
0.6900
0.4520
0.6976
0.7011
0.2727
+peak loss
0.6964
0.7058
0.4221
0.6599
0,6892
0.4531
0.6964
0.7017
0.2772
+balance loss
0.6941
0,7038
0.4999
0.6588
0,6856
0.4554
0.6967
0.6995
0.2729
+all losses
0.6973
0.7057
0.4203
0.6585
0.6890
0.4533
0.7003
0.7088
0.2749
influences model performance, we conduct a series of experiments with different pool sizes on the SIM model and the Micro Video dataset.
As shown in Figure 5 the polynomial fitting results reveal that GAUC and AUC metrics exhibit upward trends as the pool size increases to around 7000, after which both metrics begin to decline. The Logloss metric exhibits the opposite trend. Therefore, we conjecture that enlarging the pool beyond the capacity needed to capture shared information may instead expand the representation space unnecessarily, introducing noise and reducing the router's routing accuracy, which in turn can lead to performance degradation.
5.5 ABLATION STUDY
We conduct ablation experiments to verify the effectiveness of the key components in our method. First, we examine the impact of the proposed peak loss Cpeak and load balance loss Lhalance on train- ing the router network. As shown in Table 3 our method achieves the best performance on the VicroVideo and Ebnerd datasets when both losses are applied, which demonstrates their effective- ness. On the Vidi dataset, due to its substantial differences from the other two datasets such as longer average sequence lengths and more challenging prediction tasks-the two losses perform suboptimally under the same loss weight settings. Nonetheless, the results with both losses still outperform the baseline, which further demonstrates the advantage of our method.
Furthermore, we analyze the impact of sharing keys and values separately on the model perfor- mance, as shown in Table E On the KuaiVideo and Ebnerd datasets, the model achieves the best performance when both keys and values are shared. On the Micro Video dataset, sharing only the keys (Collectivek) yields the highest GAUC, while sharing only the values (CollectiveV) results in the best AUC and Logloss. Mcanwhile, sharing both K and simultancously achieves the second- best performance across all three metrics. These results suggest that the optimal sharing strategy may vary across datasets, indicating a trade-off between key and value sharing.
6 LIMITATIONS AND DISCUSSION
A key limitation of our method lies in determining the appropriate dimensionality for the user- specific KV. The choice of this dimension directly impacts the overall compression rate: if it is set too high, the KV cache will remain large and model performance may decrease; if set too low, the model may fail to capture sufficient user-specific information, potentially harming performance. Currently, based empirical observations, the user-specific KV dimension can be set very small,

---

Published as a conference paper at ICLR 2026
Table 4: Ablation experiments on sharing keys or values.
MicroVides
Kuai
Ebnerd
GAUC
AUC:
Logloss
GAUC
AUC
Logloss
GAUC
AUC
Logloss
Buseline
0.6954
0.6933
0.4282
0.6577
0.6798
0.4620
0.6975
0.7024
0.2754
CollectiveK
0.6977
0.6936
0.4281
0.6627
0.6878
0.4534
0.6947
0.6991
0.2820
Collective
0.6965
0.7073
0.4179
0.6591
0.6872
0.4538
0.6974
0.6998
0.2780
CollectiveKV
0.6973
0.7057
0.4203
0.6604
0.6900
0.4520
0.7003
0.7088
0.2749
typically within 4% of the attention head dimension, while still achieving optimal performance. Developing automated adaptive strategy determine the optimal user-specifie KV dimension remains an important direction for future work.
Overall, our work makes two main contributions. First, introduce compression by sharing KV information across users. This paradigm could potentially be extended other application scenarios beyond sequential recommendation. Second, we propose novel KV cache sharing mechanism that significantly compresses the KV cache and reduces inference latency, while simultancously maintaining or improving model performance.
paradigm for KV cache
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (62302532), Guang- dong Basic and Applied Basic Research Foundation (2025A151501 1224), and Shenzhen Science and Technology Program (CCTD2022100103555000 8, SYSRD20250529 13401002).
Furthermore, we would like to thank the contributors of the baseline codebase Zhu et al. (2022) 202T]: (Zhag et al. (2026) used in this work. The provided implementations and experiment setups offered important support for reproducing the bascline models and conducting the empirical studies in this paper.
REFERENCES
Joshua Ainslie, James Lee- Thorp. Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. GQA: Training generalized multi-query transformer models from mull check- points. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
Yue Cao, Xiaojiang Zhou, Jiaqi Feng, Peihao Huang, Yao Xiao, Dayao Chen, and Sheng Chen. Sampling is all you need on modeling long- aang-term user behaviors for ctr prediction. In Proceedings of the s1 ACM International Conference on Information & Knowledge Management, pp. 2974 2983, 2022.
Jianxin Chang, Chenbin Zhang, Zhiyi Fu, Xiaoxue Zang, Lin Guan, Jing Lu, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, and Kun Gai. Twin: Two-stage interest network for lifelong user behavior modeling in etr prediction kuaishou. In Proceedings of the 29th ACM SIGKDD Con- ference Knowledge Discovery and Data Mining, pp. 3785-3794. Association for Computing Machinery, 2023.
Qiwei Chen, Yue Xu, Changhua Pei, Shanshan Lv, Tao Zhuang, and Junfeng Ge. Efficient long sequential user data modeling for click-through rate prediction. arXiv preprint arXiv:2209 12212, 2022.
Xusong Chen, Dong Liu, Zheng-Jun Zha, Wengang Zhou, Zhiwei Xiong, and Yan Li. Temporal hierarchical attention at category and item-level for micro-video click-through prediction. In Proceedings of the 26th ACM International Conference on Multimedia, pp. 1146-115, 2018
Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining, pp. 197-206. IEEE, 2018
Johannes Kruse, Kasper Lindskow, Saikishore Kalloori, Marco Polignano, Claudio Pomo, Abhishek Srivastava, Anshuk Uppal, Michael Riis Andersen, and Jes Frellsen. Eb-nerd large-scale dataset
10

---

Published as a conference paper at ICLR 2026
for news recommendation. In Proceedings of the Recommender Systems Challenge 2024, Pp- 1-11. Association for Computing Machinery, 2024.
Wonbeom Lee, Jungi Lee, Junghwan Seo, and Jaewoong Sim. Infinigen: Efficient generative infer- ence of large language models with dynamic kv cache management. In 18th USENIX Symposium on Operating Systems Design and Implementation, pp. 153-172, 2024.
Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseck-v2: A strong, economical, and efficient mixture- of-experts language model. arXiv preprint arXiv:2405.00 04434, 2024.
Xiao Lv, Jiangxia Cao, Shijie Guan, Xiaoyou Zhou, Zhiguang Qi, Yaqiang Zang, Ming Li, Ben Wang, Kun Gai, and Guorui Zhou. Marm: Unlocking the future of recommendation systems through memory augmentation and scalable complexity. arXiv preprint arXiv:241 1.09425, 2024.
Mengyang Ma. Xiaopeng Li, Wanyu Wang, Zhaocheng Du, Jingtong Gao, Pengyue Jia, Yuyang Ye, Yiqi Wang. Yunpeng Weng, Weihong Luo, et al. Blossomrec: Block-level fused sparse attention mechanism for sequential recommendations. arXiv preprint 25..::25/2.1336666 2025.
Qi Pi, Guorui Zhou, Yujing Zhang, Zhe Wang, Lejian Ren, Ying Fan, Xiaoqiang Zhu, and Kun
Gai. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 2685-2692. Association for Computing Machinery, 2020.
Noam Shazeer. Fast transformer decoding: One write-head is all you need. arXiv preprint
arXiv:1911.02150, 2019.
Prajwal Singhania, Siddharth Singh, Shwai He, Soheil Feizi, and Abhinav Bhatele. Loki: Low- rank keys for efficient sparse attention. In Advances in Neural Information Processing Systems, volume 37, pp. 16692-16723, 16723. Curran Associates, Inc., 2024.
Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikei, and Song Han. QUEST: Query-aware sparsity for efficient long-context LLM inference. In Proceedings of the 41st In- ternational Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 199901-47911.. PMLR, 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances Neural Infor- mation Processing Systems, volume 30. Curran Associates, Inc., 2017.
Qihang Yu, Kairui Fu, Zhaocheng Du, Yuxuan Si, Kaiyuan Li, Weihao Zhao, Zhicheng Zhang, Jieming Zhu, Quanyu Dai, Zhenhua Dong, al. Malloc: Benchmarking the memory-aware long sequence compression for large sequential recommendation. arXiv preprint arXiv:2601.2023,, 2026.
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, Yinghai Lu, and Yu Shi. Actions speak louder than words: trillion- parameter sequential transducers for generative recommendations. In Proceedings of the 41st International Conference on Machine Learning. JMLR.org. 2024.
Zhicheng Zhang, Zhaocheng Du, Jieming Zhu, Jiwei Tang, Fengyuan Lu, Wang Jiaheng, Song-Li Wu, Qianhui Zhu, Jingyu Li, Hai-Tao Zheng, and Zhenhua Dong. Length-adaptive interest net- work for balancing long and short sequence modeling in etr prediction. Proceedings of the AAAI (001)//e22e
Conference Artificial htt.://se.......aaa....../aaaaaa 40094. URL
Mar. 2026. doi: v0.1609/
Jieming Zhu, Jinyang Liu, Shuai Yang, Qi Zhang, and Xiuqiang He. Open benchmarking for click- through rate prediction. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM '21, pp. 2759-2769, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 978 9781450384469.
Jieming Zhu, Quanyu Dai, Liangcai Su, Rong Ma, Jinyang Liu, Guohao Cai, Xi Xiao, and Rui Zhang. Bars: Towards open benchmarking for recommender systems, 2022. URL https: wawxiv.org/abs//222202
11

---

Published as a conference paper at ICLR 2026
A LLM USAGE STATEMENT
Large Language Models (LLMs) were solely employed for improving the writing clarity and lan- guage fluency of this paper, and were not involved in the idea generation, methodology design, experiments, or analysis.
B CODE AND OTHER SUPPLEMENTARY MATERIALS
We provide the code of our method in the repository: https: http:///tttttuu.commm/ee//////a/
let i vekV
C IMPLEMENTATION DETAILS
In our experiments, set the global K/V pool size to 10000 by default. The weights of the peak loss Lpaak and the load balance loss Lialance are 0.01 and 1.0, respectively. The target-attention-base models from the LongCTR benchmark: https: //gitttttbb com/ reczoo Taccoo//Loogee LongCTR The experiments are conducted on a single A A100 GPU, with Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz, and ITB RAM, based Pytorch v2.0.1.
D VISUALIZATION OF ROUTER ACTIVATIONS
To gain deeper insights into the behavior of the router network, provide visualization of its activation patterns. Specifically, we randomly select 100 users and collect the indices of the activated keys produced by the router outputs. We then plot the histogram of the activation indices with bin size of 50, as shown in Figure
From the histograms, we observe that the router activations are distributed across different indices rather than being concentrated on some special keys, which demonstrates the effectiveness of the peak loss and balance loss in guiding the router network. This balanced activation pattern indicates that the router is able to leverage diverse components of the global KV pool, thereby enhancing the model's s ability to capture shared information while avoiding expert collapse.
E ANALYSIS OF ROUTER ACTIVATIONS FOR HIGHLY SIMILAR USERS
To further investigate the relationship between user similarity and router activations. we select two users whose history sequence embeddings exhibit cosine similarity higher than 0.8. We then plot the histograms of their router activation indices, using bin size of 100, as shown Figure
From the histograms, it is evident that the two users share large number of activation indices, reflecting the high similarity in their behavior sequences. Nevertheless, there are also a few indices that are not shared, indicating user-specific differences captured by the router. Quantitatively, the overlap ratio of activated indices between the two users, whose similarity 0.9887, is 0.8571.
This analysis demonstrates that the router effectively balances shared and user-specific information: highly similar users tend to activate largely overlapping portions the global KV pool, while still maintaining small amount of unique activations to preserve individual preferences.
F DISCUSSION OF THE SHARING MECHANISM
We believe that method outperforms the baseline because it explicitly decouples the collaborative signals and the user-specific signals in the feature space. In the original K/V representations, these two types of signals share the same feature space of dimension d.. This inevitably leads certain degree of information entanglement, making it difficult for the model to distinguish and effectively utilize the components.
In contrast, CollectiveKV assigns the collaborative and user-specific signals to separate subspaces: the first d dimensions of the K/V vectors encode user-specific information, while the remaining
12

---

Published as a conference paper at ICLR 2026
Distribution of Top-k Activated Global Keys (K)
1000
Frequency
400
200
2000
10000
Global Pool Index
(a) TWIN, KuaiVideo
Distribution of Top-k Activated Global Keys (K)
400
350
300
Frequency
250
200
150
100
2000
4000
6000
8000
10000
Global
(b) SDIM, MicroVide
Figure 6: Visualization of Router Activations.
13

---

Published as a conference paper at ICLR 2026
Top-k router Activation Comparison
11961
User 11140
10.
8-
frequency
2000
4000
6000
80:00
Global Pool Index
(a) User similarity = 0.9887, overlap ratio 0.8571.
Top-k router K Activation Comparison
User 12516
User 4142
17.5
15.0
12.5
Frequency
10.0
7.5
5.0
2.5
20.00
4000
6000
8000
Global Pool
(b) User similarity = 0.9088, overlap ratio = 0.7143.
Top-k router K Activation Comparison
User 4142
17.5
User 1260
frequency
5.0
2.5
0.0
2000
4000
6000
8000
Global Pool Index
(c) User similarity 0.8315, overlap ratio 0.7959.
Figure 7: Router activation histograms for two users with high cosine similarity in their history sequence embeddings. Bin size is 100.
14

---

Published as a conference paper at ICLR 2026
Setuence sngin Netogam
Szowerice loregh
145
1.0
0.5
65-
300
29.23
335
Figure 8: Sequence length distribution of the MicroVides dataset.
piCE dEaeeoo
p0d aice-1000
A785-
Loglooa
0,70
4.47
9.6
Tongtatl STAC
longrall PoOA
Iangtatt JEAT 7708
(a) GAUC
(b) AUC
(c) Logloss
Figure 9: Model performance long-tail users with different global KV pool sizes.
d dimensions encode collaborative information, with d, + dd = = Under the constraint of a fixed total dimensionality, this explicit separation enables clearer decomposition of the two types of signals, making it easier for the model to identify and leverage them effectively.
A similar theoretical perspective comes from domain-invariant disentangled representation learn- ing: if each user is regarded as a domain, then we can decompose the K/V representations into nomain-invariant (shared) component and domain-specific (user-specific) residual. Another sim- ilar scenario is multi-domain recommendation, in which users are clustered into different domains according to some specific features. In our work, we can treat each user as a special domain, since each user has a unique behavior pattern. Considering different user group has within-group collabo- rative signal, these signals can be compressed into domain invariant parameters (shared pool). And their specific behaviors can be modeled as domain specific parameters.
G INFLUENCE OF THE GLOBAL KV POOL CAPACITY ON LONG-TAIL USERS
In recommender systems, long-tail refer the large group of users with very short behav- ior sequences, who typically account for high proportion of the entire user base. Because their sequences are short, usually difficult generate accurate recommendations for these users. Moreover, the collaborative signals contained in such short sequences are harder to capture.
Therefore, we aim investigate whether global KV pool with insufficient capacity may struggle to learn the behavioral patterns of long-tail users. First, we analyze the distribution of user behavior sequence lengths in the MicroVideo dataset. As shown in Figure g 50% of users have sequence lengths shorter than 1 122, 90% shorter than 253, and fewer than 1 % have lengths greater than 470. This indicates that the long-tail user issue also exists in this dataset.
We then partition the dataset based on user sequence lengths, isolating the long-tail users with the shortest sequences and evaluating the model specifically on them. In the experiments, we set a very small KV pool size of 100 ensure that the pool capacity is insufficient to capture all collaborative signals in the dataset. We conduct multiple experiments by varying the parameter longtail user rate, which determines the proportion of users included in the evaluation. For example, when longtail user rate = 0.1, we test on only 10% of users with the shortest behavior sequences; when longtail 1.0 1.0, all users tested.
15

---

Published as a conference paper at ICLR 2026
The experimental results are shown in Figure We observe that long-tail users exhibit lower GAUC and AUC, as well as higher logloss, no matter what pool size is chosen. These results indicates that long-tail tail users typically have weaker collaborative signals, making it harder to model their behavior pattern.
16


Reviews:
[{"review_id": "T3Ii3p0tUG", "reviewer": "Reviewer_tHC6", "summary": "This paper introduces CollectiveKV, a novel approach for compressing the KV cache in sequential recommendation systems by leveraging collaborative signals across users. The authors first analyze and identify significant similarities in KV sequences across different users, and through Singular Value Decomposition (SVD), confirm that KV information can be decoupled into largely shareable collaborative information and a small portion of user-specific information. Based on this, they design the CollectiveKV method, which centers on a learnable global KV pool to capture cross-user shared information and a router network for dynamic retrieval of shared KV. During inference, low-dimensional user-specific KV is concatenated with high-dimensional shared KV to drastically compress the cache size.", "questions": "- The global K/V pool size is fixed, which determines the upper limit of information capacity. Experiments show that the global K/V pool size has a relatively small impact on AUC. So, does there exist a lower bound for the pool size?\n- If a lower bound exists, how would recommendation performance be affected when long-tail user behaviors cannot be adequately covered?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The motivation is clear. Through data-driven analysis, the existence of collaborative signals in KV caches is demonstrated from the perspectives of similarity and information decomposition, thus leading to the core idea of \"decoupling and sharing\".\n- The paper pioneers the incorporation of cross-user collaborative signals into KV cache compression, distinguishing it from traditional methods focused on individual sequences in LLMs. By decoupling KV information into shared and user-specific components, it establishes a novel paradigm that could be extended to other recommendation scenarios.", "weaknesses": "-  The paper notes that the selection of the user-specific KV dimension lacks an automated strategy (as discussed in Section 6) and currently relies on manual setting (e.g., dimensions within 4% of the attention head dimension). This could necessitate iterative experimentation across different datasets or models.\n- As shown in Figure 4, increasing the user-specific KV dimension can degrade performance, but the paper does not deeply explore the redundancy threshold.  \n-  Ablation experiments reveal that the effectiveness of the sharing strategy varies by dataset (e.g., suboptimal results of loss functions on KuaiVideo), suggesting sensitivity to data characteristics like sequence length and task difficulty. This may limit robustness in diverse scenarios, requiring further cross-domain validation.\n- The method is effective on various models and datasets, but its performance is affected by data characteristics (such as sequence length), indicating the need for scenario adaptation. Testing on diverse datasets (such as Amazon product data or Spotify sequences) verifies the method's robustness to heterogeneous behavioral patterns.\n- Although the paper provides intuitive support through SVD and similarity analysis, the theoretical basis of the sharing mechanism (e.g., the integrity of shared information after decoupling) is not explored in depth. This may affect the trustworthiness of the method in more complex systems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zeSv4fuOys", "reviewer": "Reviewer_U1XG", "summary": "It is hard to satisfy the latency requirement for Transformer based sequential recommendation models. CollaborativeKV analyzes the collaborative signals in KV caches and divides it into inter-user and intra-user components by SVD. The proposed method achieves better effectiveness and efficiency than state-of-the-art baselines on benchmark video recommendation datasets.", "questions": "--Though the cache efficiency improvement is obvious, the effectiveness improvement of CollaborativeKV is relatively small.\n\n--According to the choice of benchmark datasets, it is better to change the title to clarify that the proposed method is designed for video recommendation.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "--CollaborativeKV improve the efficiency of KV cache and at the same time the recommendation accuracy without tradeoff.\n\n--Practically, it solves the urgent latency problem by low-rank factorization for Transformer based sequential recommendation models.\n\n--The massive reduction in storage overhead has been verified on real-world datasets in Table 1.", "weaknesses": "-The proposed method highly relies on the user similarities, which might not be effective for applications with non-overlapping user interests.\n\n-The recommendation accuracy in terms of GAUC is increased by 0.005 or so with the increase of the global KV pool size to 10000 according to Figure5(a). This performance improvement cannot be ignored compared with its improvement in Table 1. So how to derive the conclusion in the last passage of Page 8: …a relatively small global KV pool is SUFFICIENT to capture shared KV information….", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XAYDsrbpFW", "reviewer": "Reviewer_uWXd", "summary": "This paper, \"CollectiveKV,\" addresses the significant challenge of storage overhead and inference latency introduced by the Keys and Values (K/V) cache mechanism in sequential recommendation systems, particularly those leveraging Transformer attention. Sequential recommendation models, unlike LLMs, serve a massive user base with potentially long interaction histories, leading to KV cache sizes that exceed GPU capacity and necessitate costly offloading.\n\nThe core insight driving this work is the observation that K/V sequences across different users exhibit substantial similarities, suggesting the presence of collaborative signals embedded within the K/V representations. Through singular value decomposition (SVD) analysis, the authors confirm that the majority of K/V information is shareable (principal component subspace), while only a small fraction is user-specific (residual subspace).\n\nMotivated by this, CollectiveKV proposes a novel compression paradigm that decouples the K/V into a high-dimensional collective KV shared via a learnable global pool, and a low-dimensional user-specific KV. This decomposition and sharing mechanism, governed by a router network trained with specific losses (peak and load balance loss), allows for drastic compression. The experimental results demonstrate that CollectiveKV achieves remarkable compression and latency reduction while maintaining or improving recommendation accuracy across five models and three datasets.", "questions": "NA", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Groundbreaking Compression Rate and Performance Preservation: The method achieves an exceptional level of compression, reducing the KV cache size to as low as 0.8% of its original size (a compression ratio of 0.008). Crucially, this compression is achieved while simultaneously maintaining or even enhancing model performance across various target-attention and self-attention models (SIM, SDIM, ETA, TWIN, HSTU) and three long-sequence datasets.\n\n2. Significant Inference Latency Reduction: CollectiveKV effectively addresses the stringent latency requirements of real-time sequential recommendation. Comparative experiments show that the proposed method yields significantly lower latency than the standard KV cache, particularly for larger batch sizes. For instance, at a batch size of 512, the standard KV cache latency was 32.991 ms, whereas CollectiveKV achieved 0.695 ms. This result confirms the practical deployment benefit of the approach in mitigating transfer latency caused by offloading large caches.\n\n3. Novel and Well-Motivated Compression Paradigm: The paper introduces a new compression paradigm for K/V caches by sharing information across users, a unique approach that exploits the strong collaborative signals inherent in recommendation scenarios. This is a fundamental distinction from existing methods primarily designed for LLMs, which focus on compressing individual sequences", "weaknesses": "1. Empirical Determination of User-Specific KV Dimensionality: A primary limitation acknowledged by the authors is the difficulty in determining the optimal dimensionality ($d_u$) for the low-dimensional user-specific KV. The choice of this dimension critically affects both the overall compression rate and the model's ability to capture sufficient personalized signals; if set incorrectly, performance may degrade or compression benefits may be lost.\n\n2. Lack of Adaptive Strategy for $d_u$: Related to the above, the current optimal setting for the user-specific KV dimension relies on empirical observations (e.g., typically set within 4% of the attention head dimension). The paper highlights that developing an automated or adaptive strategy to determine this dimension remains a crucial direction for future work, indicating that the current method relies on a tuning process for optimal results.\n\n3. Inconsistent Robustness of Training Losses: While the peak loss and load balance loss are generally effective, the ablation study shows that they perform suboptimally on the KuaiVideo dataset under the standardized loss weight settings. This suggests that the hyperparameter weights for these auxiliary losses might need substantial dataset-specific tuning to maintain optimal performance across diverse recommendation environments.\n\n4. Dataset-Dependent Optimal Sharing Strategy: The ablation study demonstrates that the best sharing configuration (sharing keys only, values only, or both) varies across different datasets. For example, sharing only values (CollectiveV) yielded the best AUC/Logloss on MicroVideo, while sharing both (CollectiveKV) was best on KuaiVideo and Ebnerd. This suggests an inherent trade-off between key and value sharing that is not universally optimized by the current CollectiveKV framework, potentially requiring empirical selection per dataset", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sbDGbQQOVI", "reviewer": "Reviewer_peMb", "summary": "This paper proposes CollectiveKV, a cross-user KV cache sharing mechanism for sequential recommendation models. The key idea is to decouple the KV cache into a high-dimensional global KV shared across users and a low-dimensional user-specific KV. CollectiveKV employs a router network to retrieve shared KV items from a global pool and concatenates them with user-specific KV for attention computation. Experimental results on five sequential recommendation models and three datasets show large cache compression rates while reportedly maintaining or even improving accuracy metrics compared to baselines.", "questions": "- The novelty is effective but relatively minor. It is more like a balance between the existing group clustering-based attention and the ordinary attention.\n- Further exploration experiments and deeper analysis are needed to explain why and under what conditions the method can outperform the standard uncompressed KV cache.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Addresses a relevant efficiency problem in sequential recommendation — KV cache storage and latency — with a novel sharing-based compression approach.\n- The method design is conceptually simple yet elegant, with clear separation of shared and user-specific KV components.\n- Experimental evaluation spans multiple models and datasets, reporting consistent compression gains and performance improvements.\n- The work offers practical value for large-scale industrial recommendation systems where KV cache size is a bottleneck.", "weaknesses": "- While the work is innovative, the overall innovation point is relatively simple and may not be sufficient for a strong theoretical contribution. It is more like a balance between the existing group clustering-based attention and the ordinary attention.\n- There is a theoretical concern: since global modeling is used to compress the KV cache, the actual recommendation performance should, in principle, be at most the same as before compression. However, the reported experiments show that the proposed method is significantly superior to the baseline, which raises questions about the cause. Further exploration experiments and deeper analysis are needed to explain why and under what conditions the method can outperform the standard uncompressed KV cache.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)