P-Law: Predicting Quantitative Scaling Law with Entropy Guidance in Large Recommendation Models
Authors: Tingjia Shen
Hao Wang
Chuhan Wu
Jin Yao Chin
Wei Guo
Yong Liu
Huifeng Guo
Defu Lian
Ruiming Tang
Enhong Chen
Venue: NeurIPS 2025 poster ()
Keywords: Recommendation System
Sequential Recommendation
Scaling Law
Primary area: deep_learning

Abstract: With the growing size of data and models in Large Recommendation Models, the time required for debugging has become increasingly prohibitive, underscoring the urgent need for effective guidance in parameter configuration.
The Scaling Law (SL) offers analogous guidance in the Sequential Language domain, having achieved significant success by predicting model loss when scaling model size.
However, the existing guidance from SL for Sequential Recommendation (SR) remains qualitative, which is because quantitative analysis of SL on SR encounters challenges with quality measurement on redundant sequences along with loss-performance discrepancy.
In response, we introduce the Performance Law (P-Law) for SR models, which predicts model performance across various settings, intending to provide a quantitative framework for guiding the parameter optimization of future models.
Initially, Performance Law utilizes Real Entropy to measure data quality, 
aiming to remove the low-quality influence of low-entropy redundant sequences.
Subsequently, Performance Law investigates a fitting decay
term, which facilitated the prediction of the major loss-performance discrepancy phenomena of overfitting, ultimately achieving quantitative performance prediction. 
Extensive experiment on various datasets demonstrates the effectiveness of Performance Law by displaying exceptional quantitative prediction ability against the original and modified qualitative SL. 
Additional application experiments on optimal parameter prediction and model expansion potential prediction also demonstrated the broad applicability of the Performance Law.

P-Law: Predicting Quantitative Scaling Law with Entropy Guidance in Large Recommendation Models
Tingjia Shen', Hao Wang's Chuhan Wu?, Chin Jin Yao', Wei Guo?, Yong Liu², Huifeng Guo², Defu Lian", Ruiming Tang", Enhong Chen
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China Shennzen Huawei Technologies Lt.Ltd. cww/USTC StW/U//TC-SSaaaeaaa//ae
Source Code: https:///githu: http://tttu..coo/SSCCCOC
Abstract
With the growing size of data and models in Large Recommendation Models, the time required for debugging has become increasingly prohibitive, underscoring the urgent need for effective guidance in parameter configuration. The Scaling Law (SL) offers analogous guidance the Sequential Language domain, hav- ing achieved significant success by predicting model loss when scaling model size. However, the existing guidance from SL for Sequential Recommendation (SR) remains qualitative, which is because quantitative analysis of SL SR counters challenges with quality measurement on redundant sequences along with loss-performance discrepancy. In response, we introduce the Performance Law (P-Law) for SR models, which predicts model performance across various settings, intending to provide quantitative framework for guiding the parameter optimiza- tion of future models. Initially, Performance Law utilizes Real Entropy to measure data quality, aiming to remove the low-quality influence of low-entropy redundant sequences. Subsequently, Performance Law investigates a fitting decay term, which facilitated prediction of the major loss-performance discrepancy phenomena overfitting, ultimately achieving quantitative performance prediction. Extensive experiment on various datasets demonstrates the effectiveness of Performance Law by displaying exceptional quantitative prediction ability against the original and modified qualitative SL. Additional application experiments on optimal parameter prediction and model expansion potential prediction also demonstrated the broad applicability of the Performance Law.
Introduction
Recently, billions of data points are generated daily across various application platforms for large recommendation models(I). To effectively model this data, large-scale recommendation models have been introduced due to their potential to enhance performance(2155 3). As larger recommendation expensive parameters these models(4) leads to high costs
models are developed, optimizing and unpredictable performance during development. This challenge motivates researchers to explore the Scaling Law (SL) to efficiently plan time and GPU consumption across various model (5) The concept of SL was initially investigated within the realm of Large Language Models (LLMs) (6) The introduction of the Chinchilla model (7). which provides a quantitative framework for modeling the final pre- pee-training loss L(N, D) based on the number of model parameters N and the number of
tokens D, has led to the adoption of this principle in models such as LLaMA2 ( and Mistral 9
*Correspondence wanghao30ustc. edu.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

As SL has made progress with LLMs, researchers are interested in extending these laws to large-scale recommendation models, as they share similar Transformer architectures. Several large Sequential Recommendation (SR) models, including HSTU(()) and Wukong(1I), have developed qualitative recommendation SL. to assess model effectiveness under various scenarios. Research across several domains has similarly reported performance enhancements due to large-scale models, evident in Click- Through Rate (CTR) prediction (12), reranking(13), and retrieval (14). Nonetheless, current studies primarily remain qualitative, highlighting the need for advancing toward quantitative prediction of model performance.
Our work attempts to extend the Scaling Law to enable quantitative prediction and analysis of SR model performance. How- ever, as illustrated in Figure [] this ap- proach faces the following two challenges: (1) Quality Measure Deficiency. Cur- rently, the analysis of data in SL is mea- sured using token counts. While this is rel- atively reasonable in language models, SR often exhibits variable vocabulary sizes, re- dundant user sequences, and random item noise. This results in significant discrepan- cies in the amount of information contained within the same token count, necessitating the introduction of quality metrics refine the SL. (2) Loss-Performance Discrep- ancy. While SR systems prioritize model performance, SL models focus on analyzing and predicting training loss. However, as overtrained SR models frequently display overfitting [15) behaviors, training loss does not necessarily correspond to actual model performance. Thus, even with the enhancement of Quality Measures to quantitatively analyze model scaling loss, a reduction in model loss does not always translate into significant performance improvements. We term this dilemma the Loss-Performan
Coloborntion differecce between WMand Quality Measure Deficiency: Scaling Law ignores colleboralive and struiuurl properies.
Quality messur deficencyy
Real Entrop erhuncement
In
Scaing Law gaidance of mosel Expursion Dovetiting phenomenn
Poun Modiicattoo
Loss-Perforannce Discregancy: Scaling Law guides on excand indefiniell for reducing model loss. hindering prediction of overitting.
Two challenges when expanding the origin Scaling Law to quantitative predict SR performance: Quality Measure Deficiency and Loss-Performance Dis- crepancy, along with our approach: Real Entropy en- hancement and Decay Modification.
Figure 1:
Discrepancy challenge.
To address these challenges, introduce novel Performance Law that studies the SR system with an entropy-enhanced performance analysis formula, thereby quantitatively predicting the performance of SR models. Specifically, first collected model performance data across wide range of different model parameters for analysis. Subsequently, in response to challenge (1), we define the Data Scale as the minimum encoding length of datasets, aiming minimize encoding redundancy on specific datasets and thereby enable adaptive research across different encoding vocabularies. Next, we introduce the concept of Real Entropy (17) greal as novel measure of data quality. By calculating the distributional differences in user interactions using Greal we propose a correction for the removal of redundant low-entropy item sequences in SR. ultimately facilitating the transfer of SL from LLMs to SR models. Toward challenge (2), we investigated the consistency between testing loss and two metries: normalized discounted cumulative gain (NDCG) and hit rate (HR), aiming to address the inconsistency between loss and performance, facilitating quantitative modeling and prediction of model performance. Subsequently, we introduce decay term into the fitting formula for SL, altering the previously predicted trend of monotonically increasing performance with model size into a rise-then-fall pattern. This modification enables the quantitative prediction of model overfitting phenomena, ultimately establishing the Performance Law for quantitative guidance on model parameters and framework. Our contributions are summarized as follows:
We developed Performance Law, the first quantitative approach to predict model performance across various settings, intending to provide a more quantitative framework for anticipating the effects of how both data and model parameters change will guide the optimization of future models.
Performance Law innovatively utilizes minimum encoding length to measure data scale and Real Entropy Sreal to remove low-quality influence of low-entropy redundant sequences, enhancing the transfer of SL in SR. This approach enables quantitative analysis of data in SR.
Performance Law further analyzes the difference between training loss and performance, enabling quantitative analysis of model performance. This is accomplished through extending fitting decay term, which facilitates the quantitative prediction of overfitting.

---

Extensive experiment on various datasets demonstrates the effectiveness of Performance Law by displaying exceptional quantitative prediction ability against qualitative SL. Experiments on optimal model parameter prediction and model expansion potential further validate the applicability and promise of the Performance Law.
2 Related Work
2.1 Sequential Recommendation
The focus of recommendation systems has undergone significant transformations. Among these, sequential recommendation (I8) is technique that aims to delve into and understand users' interest patterns by analyzing their historical interactions (99 20). Initially, statistical analysis techniques such as Markov chains (2I) and collaborate filtering (22) were employed in sequential recommenda- tion (SR). However, with the emergence of neural networks, deep-learning approaches have been developed for sequential recommendation tasks. Convolutional Neural Networks(CNN)
(23) Graph Neural Networks (GNN) (24) from GCE-GNN (25) to SR-GNN (26). However, the compatibility of RNN (2)) with sequences has ultimately garnered more attention. Early work like GRU4Rec (28) and Caser (29) were introduced to improve recommendation accuracy. Another notable technique in sequential recommendation is the attention mechanism. SASRee (30), for instance, utilizes self- attention to independently learn the impact each interaction on the target behavior. On the other hand, BERT4Rec (31) incorporates bi-directional transformer layers after conducting pre-training tasks. Since LLaMA4Rec (32) and HSTU (IO) demonstrated improvements SR performance brought by large models and datasets, it is meaningful to study how the model performance would change as the model scales up.
2.2 Scaling Law on Large Sequential Models
Scaling laws were first explored in the context of Large Language Models (8). Specifically, since the introduction of the Chinchilla scaling law, which models the final pre-training loss L(N, D) as a function of the number of model parameters N and the number of training tokens D, models such LLaMA2 (8, Mistral (9), and Gemma (33) have applied this principle. Empirical evidence indicates that model performance consistently improves with increased model size and training data volume (6) 34 35 36) and in Sequential Recommendation [1 38 3). Extensive experiments have explored neural scaling laws under various conditions, including constraints on computational budget (39), data limitations (40), and regeneration (41), and instances of over-training (42). (43)
tailoring content to individual preferences towards expanding modules, while (44) further processing has been done refine SL's description of model precision However, increasing the model size does not necessarily lead to better performance. Some studies have observed decline in performance due to overfitting (55 46). Following theoretical analysis, (47) and (48) empirically validated this point, underscoring the necessity for an enhanced understanding of scaling laws. However, as mentioned in the introduction, these methods are still qualitative analyses, while quantitative analyses are facing challenges related to Quality Measure Deficiency and Loss-Performance Discrepancy, resulting in only superficial and non-quantitative analysis for Sequential Recommendation.
3 Preliminary and Definition
3.1 Problem Definition
Definitionl. (Scaling Law) Scaling Law indicates that model performance consistently improves with increased model size and training data volume. These analyses employ decomposition of
expected risk, resulting in the fit of L(N, D)
Ne
+ Dc
De
where L. denotes the model
N
the number tokens in the dataset in SL, Nc. De, ON. and a are parameters. In some works, the scaling law formula might be simplified to L'NN, E + No B with parameter 0 (a, 3) The goal of SL. is minimize İŻ. L'| with 0, where î. represents the actual model training loss, making a more accurate prediction.
Loss, the number of parameters,
Definition2. (Performance Law) The only distinction between the Performance Law and the SI. replacing the research target L(N, D) with P(N, D), where P represents the model's performance.
3

---

To delve further into model performance, we refine model parameter N into more granular compo- nents with the number of layers H, the hidden layer dimension h, and the embedding dimension demb- To ensure alignment between the input and output for model stacking, we set h = demb- Consequently, the formula we ultimately aim to derive is P(H, demb, D, ), where 0 denotes the parameter set fitting the Performance Law. In contrast, the objective of the Performance Law is to minimize |P P
P with 0*, where P represents the actual model performance.
3.2 Preliminary
Real Entropy We introduce the Real Entropy (17 (Sreal) factor to enhance scaling laws in SR further. Real entropy is a refined measure that captures user interaction distributions across patterns of varying lengths. It is computed as: greal Erer P(T") log logg|P(T'). (P(")), where T represents the complete sequence formed by concatenating all user interaction sequences in the dataset, P(T") represents the probability of each subset of transitions T". By calculating the entropy of distributional differences in user interactions, we make a correction for removing redundant low-entropy item sequences in SR, ultimately addressing the quality measure deficiency in transferring SL from LLM to SR. However, the complexity of calculating 5reat using the above definition is extremely high. To reduce this complexity, we utilize the following lemma:
Lemma 3.1. LZ compression (49). For user interaction sequence with length [Su). its Real Entropy
Sreal is estimated by: STeel = (s In [Sul,, where A, denotes the minimum length such that the subsequence starting from position i with length j does not appear as a continuous sub-sequence of Su [2,,12, .2,,.., -|s.1..-
From Lemma 3.1 it can be observed that Real Entropy Steal has positive correlation with the model's 's duplication rate, whereas conventional entropy tends to have a negative correlation with data duplication rate. Therefore, although Sreat is referred entropy, its trend is opposite traditional entropy. To avoid confusion, we use Streal = 1/Sreal as the final measure Real Entropy.
4 Methodology
4.1 Outline of Performance Law
Overall, we identify two main challenges in applying Scaling Laws: the deficiency of data quality evaluation and the loss-performance discrepancy of model overfitting. To clearly illustrate the entire process, we present the complete pipeline of the Performance Law in Figure 2 Specifically, as shown in Part A, we first collected model performance data across a wide range of different model parameters for analysis. Regarding challenge (1), we measure the data scale D' utilizing the Minimum Encoding Length Cmsnn with D' = |U|Cmin as it represents the minimal token pattern constraint sufficient to encode the entire dataset sequence, accommodating the analytical needs of different vocabularies across various datasets. We then offer a theoretical proof on the lower bound of data scale in Theorem A.2, establishing that the data scale D' is constrained by D' #T'okens- Streal which is shown on Lemma 3.1 and illustrated in Part B Figure 2 For challenge (2), we incorporate decay term + log(-) when the model layer H. embedding dimension demb and data scale D are scaled up, as demonstrated in Theorem A.4 and illustrated in Part C Figure [ Building upon the theorems outlined, we will complete the construction of the Performance Law.
4.2 Scaling Parameters
4.2.1 Scaling Model Parameters
We first construct the Scaling Transformer Framework to scale up the model-side parameters: the number of layers and the embedding dimension dent. which shown in Part (A) in Figure 2 Following the prior empirical study HSTU (I0). for all experiments, we adopt the decoder-only transformer models as the backbone. Specifically, for each user u, items in the user behavior sequence S, = [.,, .2.... ... s..]] are firstly encoded into embeddings, forming e,, = [e" e re.,...., e'iaul with embedding dimension |et| = demb. After the embedding layer, we stack multiple Transformer decoder blocks. At each layer ( E (1,2..... ... H). query Q, key K, and value V are projected from

---

(A)
Step Sealing Model Parameters
Slep2 Model Performance Prediction
Embedding Dimension
Deelermmne
[HR, NDCC)
Quantiiive Performanee Law Fitting
Model Depth
Verfomaane
Combimatton
Gate Network Attention
H
litting
Last
(B)
Scaling Data Parameters
Titting
Data Scale (0 F
Stale D
Performance
D.R.dee
logtO Ride
Messurment
Drigss Scaling
Lnngth Measurement
Minimum Encoding
Tokenss
Real Entropy
D.H.d
Overiitiiss
Figure 2: Illustration of Performance Law. Parts A, B, and C in the figure correspond to Section 4.2.1
Section 4.2.2 and Section respectively.
the same input hidden representation matrix Rg We modified the SiLU activation module and the Rab positional encoding module within the transformer block to ensure effectiveness. Specifically, the Spatial Aggregation Layer is defined as: Att = SiLU + Rabi) Vi. Upon deriving matrices Q. K, and V. the spatial aggregation layer utilizes an attention mechanism to adjust value V. This layer is distinguished by two key features: the SiLU activation function and a relative attention bias, enriching the model with positional and temporal information, thus enhancing its ability to discern contextual interrelations and capture dependencies within the data. Subsequently, the Pointwise Transformation Layer is defined as follows: ct+1 = Norm( (Attni(e)))) @ Gater (er)). Following the spatial aggregation layer, the pointwise transformation layer independently transforms each individual data point. Here, the gating weights Gater()) are combined with the normalized values N Norm Attni (- (-))) via Hadamard product, effectively gating the transformed representations and allowing the model to selectively emphasize relevant entries while weakening less significant ones. The results are then transformed by a single-layer MLP. Finally, we then calculate the similarity of this item e f+++ with those the entire item pool I, retrieving the most similar item and storing in the candidate set for each user I" The loss function for the retrieval task is defined as:
exp( y 4Ju! _ellsu) ))
Lg = - log
exp((e)) 21 (elel)))))))))))))))))))) + Ev eNegt exp((e"")Su) v-))
where ) denotes the product, which measures similarity and negative samples Neg, are randomly drawn from the item pool 7 {i...)). distinguishing the positive item from imelevant ones.
After constructing the Scaling Transformer model, as shown Part A. Figure 2 impact of varying model parameters on performance by adjusting the number of model layers, H, and the embedding dimension, demb. However, beyond model parameters, the scale and quality of the data are also crucial factors influencing performance. In the following sections, we will detail the construction methodology of the data parameter and its quantitative analysis.
we investigated the
4.2.2 Scaling Data Parameters
To fit the Performance Law formula mentioned in Section 3.1
we have collected performance data under different model parameters. However, we still face significant vocabulary differences across datasets and issues of quality measure deficiency. First, we evaluate the Data Scale D using minimum encoding length D' [m]Cmin with user count [U, aiming to minimize encoding redundancy on specific datasets, enabling adaptive research across different encoding vocabularies. We then assess data quality using the Real Entropy Sreal in Section 3.2 Through our analysis, these two factors exhibit the following relationship:
5

---

Theorem 4.1. Assuming that the user sequence can be modeled as a first-order aperiodic stationary Markov chuin (50). If the user sequence S = {S.. u E U}. then the relationship between the sum of minimum encoding length [U|mmin and Real Entropy S g/reat is given by:
D'D'= |U/Con 2 (Eu]Sul)- -S
The detailed proof Theorem 4.1 is elaborated in Appendix A. results by proving the following two key equations:
At high level, we establish our
2(1 - p(Sa))
Eu [Su (Eu [Sul +2)
10|Cmin
Σ
P(Sw) log2 p(Sw)
Inc
Σ
((s.)
p(Su) +
4 Eu|Sa
(3)
V8a
vSu
-1
Σ A,
In ESll 2
2
Su
5)
In En (Sa
S'real
Du SulSul
10| [S
5/2
(4)
EuuS (Enl + 2) 1.
In E., [Sul 1 (E. +
In Enlu.l
The product of Inequality (3) and Inequality derivations are provided in Appendix A
4 completes the proof of the theorem, while the detailed Inequality (19) and Inequality (17).
Overall, the theory allows
achieve adaptive adjustments for different vocabularies through minimum encoding length Cmin and to introduce Real Entropy S'real for analyzing the collabvrative distribution across different datasets. This, in turn, enables a more quantitative analysis of the data
scale = | [mmn = (E.]),.))= 5 real
SR models, which shown in Part (B) Figure 2 Ultimately, by integrating model depth H and embedding dimension demb from Section 4.2.
obtained model loss and performance across various scaled parameters. This integration allows for a
quantitative measurement of Scaling Law L(H, demb, D) and Performance Law P demb:: D).
4.3
Model Performance Prediction
After obtaining the model's loss and performance under different data and model parameters, we need to construct fitting function for the final model performance. Specifically, introduce the Squeeze Theorem of Performance Fitting to establish a function for understanding the relationship between model configurations and performance. According the definition of the Squeeze Theorem, we will identify a reasonable fitting function for model performance evaluation. The whole process is shown in Part (C) in Figure Through our analysis, model performance P P(H, demb; D) is squeezed with the functions below:
Theorem 4.2. Squeeze Theorem of Performance Fitting There exist 183, w/y, V0, w/ such that
log(d ( D
+ log HI 8 P(H, D)
Jula d Duds
(5)
S loggdd' Duls)
1
log H +
1
+ 5,
Dw
H
where 8 is small constant shift. The detailed proof of Theorem is elaborated in Appendix A.2
At a high level, we establish our results with Lemma A.3 and employing moss-metric consistency:
AF
1 < PIH, demb, D) log Ze
log H
4 [[IMetir)) ([ 6, (6)
Zr
H Vds
D V. drn (V
dia Zne
din
< Z, D T Varn
(V
exp(
dia 2Tre
2nee'
where Leest denotes testing loss, MMeur denotes the specific metric (NDCG, HR) loss. Var (r) nt 41 pdin (T(1 + diz) dr, = |.S]| max x demb the dimension of input sequence. Building upon the theorems outlined, the model's performance is encapsulated in the following equation:

---

Final Formula of Performance Law P(H, demb, D):
P(H, dembs D) wi (log H w8 + Hws
Pi
) + wy(log demb + P1
) d
log D
+
P2 Dws
(8)
= -ulss Real #Tokens s Real
Ultimately, we will collate the performance matrices obtained by altering various parameters in Section 4.2.1] These matrices will be used to fit the performance model using the specified formula in conjunction with the least squares method. Finally, we will compare the differences with the Scaling Law fitting approach by evaluating their correlation coefficient R² to determine the quantitative effectiveness and accuracy of our method.
Experimental Evaluation
Datasets. To demonstrate the performance of our proposed approach across various kinds of datasets, we conducted experiments on three publicly available datasets: MovieLens-IM (5T) (ML- 1M), Amazon Books (52) (AMZ-Books), KuaiRand-Pure (53) (KR-Pure) and one private dataset Industrial. The private industrial dataset is drawn from an online music platform that has impressions of more than 400 million users, challenging existing scaling laws. The specific details of the dataset are presented in Table 3 of Appendix A.3
Baselines. Our primary baseline for comparison is the fitting formula of the Scaling Law, which serves the research paradigm for the majority of SR extension laws. Only the Precision Scaling Law has introduced some modifications. The specific T baseline introduced as follows:
Scaling Law (SL) empirical SL for neural model performance, showing that cross-entropy loss scales with model size, dataset size, and compute, guiding optimal compute budget allocation. The majority of large recommendation models (10) 37 33 B) utilize this formula for research.
Precision Scaling Law (PcSL) (44): consider small shift on low precision towards SL, enabling predictions of loss and guiding compute-optimal strategies by accounting for precision effects on model parameters and degradation.
We divide the comparison with the baseline into two parts examine the effectiveness of our Quality Measure Extension Section 5.1.I and Performance Fitting in Section (5.1.2) Since both basclines measure data by token count, we do not differentiate them in the first part, and will then compare them individually in the second part.
Experiment Settings. We adopt the leave-one-out strategy for evaluation, following prior re- search (55 56 57). For each sequence, the most recent interaction is used for testing, the second for validation, and the rest for training. We assess we evaluate the Top-K recommendation performance using HR (58) and NDCG (59). The equipment, time, and specific parameter details used in our model are illustrated in the Appendix A.4
5.1 Overall Validation of Performance Law
5.1.1
Experiment Validation on Quality Measure Extension
To verify the rationale for using Real Entropy data quality metric, we need evaluate if provides a more accurate analysis of data compared to token count #Tokens alone, as proposed in the original Scaling Law. Specifically, we modified the data coefficient D, integrating it parameter into both the standard Scaling Law formula (with fitting results shown in Figure 3 Dosss) and our enhanced Performance Law formula (with fitting results shown Figure 3 D H and DNDCG). A higher correlation coefficient R?2 indicates a better predictive capability of the data modeling. From the Figure 3 we can draw the following conclusions: (1). Whether in the fitting of data parameters by the Scaling Law (left) or the Performance Law (right), Performance Law with # TTokens S'reai measurement (blue or green) consistently provides better fit than # Tokens in SL (red or orange). This indicates that our proposed Real Entropy successfully enhances data quality, effectively extending the Scaling Law to the more synergistic and structured task of recommendation.
7

---

Dataset Parameter Analysis Scaling Law
Dataset Parameter Analysis On Performance Law
10 it 5 =0999111
100 1.Dga
NDCG@10 (#Tokons 5 ).8 0.9906
100wm
HRg (4Tokens). R 0.9250
NDCG 10 ( (*Tozenv), 0.8160
Takens 5 0.9061
Tokens, RF 08776
1/8Tokons, MilTooonns srel)
1/8Tokonn, TToooss 5
The linear correlation between parameter and #Tokens S Real The left figure validates this relationship within the context of the SL Loss, while the right figure verifies it within the Performance Law Metric, evaluated with a correlation coefficient R².
Figure 3:
Baseline: Scaling Law in KuaiRand-Pure
Baseline: Scaling Law in Industrial
Lnc 30 CAID7
ReCHORR 100 NOTR
AcDeseeeeent 2014 -ULD
Pesieneee EXPT
Performance KuaiRand-Pure
Work: Performance Law Industrial
Figure The PL. and SL. function fitting between model performance (red points) and the prediction (surface) of different kinds of functions: PL and the most authoritative baseline: SL. The plot includes annotations of the correlation coefficients Ri2. All comparisons are statistically significant with p < 0.01. Situations are the same as all other datasets and metrics in Appendix A.6
(2). # #Tokens S'real offers quantitative foundation the data level for our subsequent analysis of the Performance Fitting, as both HR and NDCG have a fitting R² greater than 0.99, which is a relatively high value in the context of the fitting. Additionally, despite the significant differences between these two metrics, the curves obtained from fitting are remarkably similar. This further demonstrates that our enhancement is valid across different metrics of Performance Fitting. The specific numerical values presented in Figure 3 are detailed in Table 3 Appendix A.3
5.1.2 Experiment Validation on Performance Fitting
To verify the resolution of Performance Law on quantitative performance prediction, we fit the model's 's performance on HR and NDCG and compare among Performance Law, SL, and PcSL with the correlation coefficient R². The results of Scaling Law Fitting and Performance Law Fitting are depicted in Table [ To formally demonstrate the improvements of the Performance Law over the Scaling Law, we present the fitting surface of them in Figure
By comparing the results
of Table between the SL fitting and the Performance Law Fitting, we draw the following conclusions: (1). In smaller datasets (ML-IM, KR-Pure), the Performance Law sig- nificantly improves over the Scaling Law. This indicates that our pro- posed decay terms provide quantita- tive predictions for model overfitting. In ML-1M, when the maximum se- quence length is 200, the improve- ment can be as much as 345%. (2). In larger datasets (AMZ-Books and Industrial), the Performance Law still demonstrates its strong capability in
Table I:
The Performance Law (PL) against SL and PeSL function fitting between model performance with model and data parameters, evaluated with correlation coefficient R?, all results are statistically significant with p<0.01.
Metric
(HR@10 10)
(NDCG@ 10)
Dataset
S.
PL
SL
PeSL
PL
SL
PoSL
25
0.792
0.671
0.672
0.925
0.667
0.667
KR-Pure
50
0.939
0.465
0.466
0.928
0.480
0.481
100
0.649
0.172
0.182
0.657
0.142
0.142
100
0.898
0.551
0.699
0.851
0.543
0.647
ML-IM
150
0.916
0.764
0.756
0.882
0.740
0.735
200
0.892
0.189
0.200
0.907
0.174
0.172
25
0.892
0.803
0.837
0.904
0.855
0.879
AMZ-Books
50
0.851
0.751
0.830
0.866
0.809
0,836
Industrial
25
0.939
0.855
0.856
0.913
0.850
0.851
50
0.951
0.803
0.767
0.938
0.800
0.750
modeling model performance. This
8

---

Table 2: Verification of Performance Law's Optimal Parameter Search Capability. The best parameter marked as "Prediction" " in the last row of each table is computed using the Performance Law. All model performance is actual but not predicted. Results statistically significant with p<0.05.
Globul optimll solution
Global optimall solution
H
does
NDCG@10 10
NDCG@5 50
HR
HR@50
H
demb
NDCG@10
NDCG@50
HR 10
HR@50 50
Smallest
8
54
0,1831
0,2418
0.3265
0.5916
28
25
0.1732
0.2326
0.3101
0.5791
Dataset
54
0,1824
0.2409
0.5913
0.3311
0.5917
ML-IM
16
54
0.1853
0.2434
0.3286
0.5903
28
75
0.1810
0.2408
0.3203
0,5882
32
54
0,1810
0,2387
0.3216
0.5837
28
100
0.1226
0.2307
0.3102
0.5741
Prediction
28
34
0,1878
0,2443
0,3322
0.5924
28
54
0.1878
0,2443
0.3322
0,5924
Optimall spluation (with constraine 1f -64)
Lphmma solution (with
I7
512)
11
dens
NDCG 10
NDCG@5 50
11
IR
II
demb
NDCG@110
NDCG&
HR
HR
Largest
64
256
0,2019
0,2623
0.3481
0.6205
4
128
0.1758
0.2371
0.3111
0.5854
Datiset
64
370
0,2038
0,2633
0.6226
0.13
0.3118
0.5858
Industrial
64
512
0.2032
0,2636
0.3502
0.6226
10
51
0.1758
0.2365
0.3092
0.5840
64
1024
0.1981
0,2590
0.3448
0.6195
16
0.1704
0.2305
0.3007
0.5732
Prodiction
64
603
0,2040
0.2644
0,3512
0.6235
12
44
0.1777
0,2383
0.3121
0.5863
success is attributed to our direct analysis of model performance rather than model loss. In all larger datasets, the correlation coefficient R exceeds 0.85. Moreover, compared to the Scaling Law, it achieves at least 2.8% improvement. This also demonstrates that the Performance Law can effectively predict and guide parameter configuration for training as data scale increases. Beyond It², we also report results using two additional evaluation metries: Mean Absolute Error (MAB) and Root Mean Squared Error (RMSE) Appendix
5.2
Applications of Performance Law
5.2.1 Application 1: Global and Local Optimal Parameter Search
practical application of the Performance Law is search best parameter by predicting the per- formance gain from model expansion techniques. We divide the calculation of optimal parameters into two parts: (1) global optimal parameter prediction and (2) optimal parameter prediction under constraints. For the second part, we provide two constraints: one with constant model depth H and the other with a constant total number of parameters H demb (as calculated from (10). To determine the optimal parameters of the model, we compute the best configuration by fitting the Performance Law and compare with other parameter configurations. The actual performance in different parame- ter settings is presented in Table 2 with parameter configuration marked as "Prediction" in the last row of each table computed using the Performance Law. From the results, we draw the following conclusions: (1) The Performance Law exhibits high accuracy. This is evidenced by Performance Law predicted parameters outperforming others in global optimal parameter prediction. This ensures the high potential for application of the guidance searching optimal parameter of the Performance Law. (2) The Performance Law demonstrates robustness. This is evidenced by Performance Law predicted parameters continuing outperform others in local optimal parameter predictions under both conditions. This ensures that the Performance Law can yield meaningful guidance on parameter setting in a variety of practical application scenarios.
5.2.2 Application 2: Exploring Performance Law Potential Among Framework
Another application of the Performance Law is assess potential performance gains when scaling up the model. We conducted experiments and fitting analyses on three different frameworks (HSTU (00, LLaMA2 (8. and SASRec (15)) evaluated at different precisions: float32 and 66loat16. Experiments were performed on the smaller dataset (ML-IM) while larger values of w/ and W2 indicate a better scaling-up potential for the model with analysis in Appendix AA5]] The fitting results are presented in Table Appendix A.55 From the table, we conclude that the model's performance closely aligns with the magnitude trend of wi and w2, further underscoring the accuracy of our quantitative fitting. This also demonstrates that the Performance Law can effectively guide model structure configuration, thereby reducing memory and time when modifying frameworks.
5.3 Further Evaluation
After demonstrating the applicability of the Performance Law, we need to further validate our Performance Law across a broader range of models and qualities. We approach the expansion of Performance Law evaluation from the following two aspects

---

5.3.1 Model Extension and Generalization
To further validate the universality and scalability of the proposed Performance Law, we extend our experiments to several representative models and diverse recommendation scenarios:
LightGCN (60) is highly efficient graph convolutional network tailored for collaborative filtering. Mamba (1) is recent architecture for sequence modeling leveraging state-space models for high accuracy and scalability. Wukong (62) is stacked factorization machine architecture with synergistic upscaling strategy, specifically designed to realize scaling laws recommendation tasks. DiffuRec (63) is novel sequential recommendation framework that leverages diffusion models for item representation construction and uncertainty injection. For long-tail recommendation, we conduct analyses on the challenging Amazon Beauty dataset, where user-item interactions are sparse and item distributions are highly skewed. Complete experimental results and metric comparisons with Scaling Law and Precision Scaling Law baselines for these additional settings are presented in Appendix B Across all models and scenarios, Performance Law consistently yields higher R² and lower error metrics, evidencing its robust generalization and superior predictive precision under both dense and sparse, as well as multimodal recommendation paradigms.
5.3.2 Comparison of Data Quality Evaluation Functions
Beyond Real Entropy, we explored a variety of data quality evaluation functions, including Approxi- mate Entropy (64), Shannon Entropy (65), and Kolmogorov Complexity (66). Such alternatives have been widely adopted to measure system complexity and randomness in different domains. To provide thorough comparison, we conducted experiments by replacing Real Entropy with each candidate function in our framework. As shown in Table 6 in the Appendix C Real Entropy consistently demonstrates the superior fitting ability over the alternatives. This empirical finding further supports the efficacy of Real Entropy as the preferred choice for characterizing data quality in recommendation scaling law analysis.
6 Discussion
Limitation and Future Directions. Performance Law has currently been thoroughly tested in the SR domain, but with suitable metries, our theoretical framework remains applicable other recommendation tasks. For our future work, we aim to extend Performance Law to larger datasets and a broader range of Recommendation tasks, such as ranking and retrieval.
Conclusion. In this paper, we have investigated novel problem concerning a quantitative approach to predicting SR model performance across various settings. To tackle this problem, first introduced Performance Law, introducing minimum encoding length and Real Entropy to remove the low- quality influence of low-entropy redundant sequences, providing quantitative data analysis for SL. Subsequently, we further analyzed the difference between training loss and performance by including consistent metrics and fitting decay term. This facilitated the prediction of overfitting and provided quantitative analysis of model performance. Performance Law displayed exceptional quantitative prediction accuracy against the original qualitative Scaling Law, successful experiments on optimal model parameter prediction and model expansion potential prediction also demonstrated the broad applicability of the Performance Law.
Broader Impact. The Performance Law serves as a framework for quantitatively predicting and ana- lyzing SR performance. While current tasks are limited to recommending items, future applications might overlook fairness issues, leading to potential biases. Nevertheless, the Performance Law can offer effective guidance for the optimal parameter configuration of models.
7 Acknowledgement
This work was supported by the National Natural Science Foundation of China (Nos. 62441239, U23A20319, 62472394, and 62441227), well as the Anhui Province Science and Technology Inno- vation Project (Nos. 202423k09020010 and 202423k09020011), We also gratefully acknowledge the USTC Supercomputing Center for providing the computational resources essential this research.
hhtpss//snap.ssanffrdd. http:////waatooloooooooooooooooe
10

---

References
[1] H. Fang, D. Zhang, Y. Shu, and G. Guo, "Deep learning for sequential recommendation: Algorithms, influential factors, and evaluations," ACM Trans. Inf. Syst. vol. 39, no. pp. 10:1-10:42, 2020. [Online]. Available: http://doo.rr/11.1114/44340
[2] w. Guo, H. Wang, L. Zhang, J. Y. Chin, Z. Liu, K. Cheng, Q. Pan, Y. Q. Lee, W. Xue, T. Shen et al., "Scaling new frontiers: Insights into large recommendation models," arXiv preprint arXiv:2412.00714, 2024.
[3] K. Shin, H. Kwak, Y. Kim, M. N. Ramström, Jeong. J.-W. Ha, and K.-M. Kim, "Scaling law for recommendation models: Towards general-purpose user representations." in Proceedings of the AAAI conference on artificial intelligence, vol. 37, no. 4, 2023, pp. 4596-4604.
[4] Y. Ding, Z. Lai, P. Mok, and T.-S. Chua, "Computational technologies for fashion recommenda- tion: survey." ACM Computing Surveys, vol. 56, no. 5, pp. 1-45, 2023.
[5] Z. Lin, L. Feng, E. K. Ardestani, J. Lee, J. Lundell, C. Kim, A. Kejariwal, and J. D. Owens, "Building performance model for deep learning recommendation model training on gpus," in 2022 IEEE 29th International Conference on High Performance Computing, Data, and Analytics (IiiP)). IEEE, 2022, pp. 48-58.
[6] J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Rad- ford, J. Wu, and D. Amodei, "Scaling laws for neural language models," arXiv preprint arXiv:2001.0883
2020.
[7] J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark et al., "Training compute-optimal large language models," arXiv preprint arXiv: 200.:52203.,556, 2022.
[8] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaci, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., "Llama 2: Open foundation and fine-tuned chat models," arXiv preprint arXiv:2307.00288 2023.
[9] A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. I. Casas, F. Bressand, G. Lengyel, Lample, Saulnier al., "Mistral 7b," " arXiv preprint 2rXiv:2310.06825, 2023.
[10] J. Zhai, L. Liao, X. Liu, Y. Wang, R. Li, X. Cao, L. Gao, Z. Gong, F. Gu, M. He et al., "Actions speak louder than words: Trillion-paramee sequential transducers for generative recommendations, arXiv preprint 2rXiv:2402. 17152, 2024.
[11] J. Gu, X. Meng. G. Lu, L. Hou, N. Minzhe, X. Liang, L. Yao, R. Huang, W. Zhang, X. Jiang et al., "Wukong: A 100 million large-scale chinese cross-modal pre-training benchmark," Advances in Neural Information Processing Systems, vol. 35, pp. 26418-26431, 2022.
[12] N. Ardalani, C.-J. Wu, Z. Chen, B. Bhushanam, and A. Aziz, "Understanding scaling laws for recommendation models," arXiv preprint wwx.ix:2208.00480800 2022.
[13] Y. Xiong, Chen, X. Ye, H. Chen, Z. Lin, Lian, Z. Su, Niu, and Ding, "Temporal scaling law for large language models," arXiv preprint arXiv:2404. 17785, 2024.
[14] Y. Wang. Z. Yang, Z. Zhang, Z. Wang, Yang. S. Wen, Jiang, and K. Gai, "Scaling laws for online advertisement retrieval," arXiv preprint arXiv:2411.13322 2024.
[15] W.-C. Kang and J. McAuley, "Self-attentive sequential recommendation." in 2018 IEEE inter- national conference on data mining (ICDM). IEFF, 2018, pp. 197-206.
[16] M. Li, A. Vardasbi, A. Yates, and M. De Rijke, "Repetition and exploration in sequential recommendationn, in Proceedings of the 46th international ACM SIGIR Conference on Research and Development in Information Retrieval, 2023, pp. 2532-2541.
[17] C. Song, Z. Qu, N. Blumm, and A.. Barabási, "Limits of predictability in human mobility." Science, vol. 327, no. 5968, pp. 1018-1021, 2010.
2010.

---

[18] Z. He, W. Liu, W. Guo, J. Qin, Y. Zhang, Y. Hu, and R. Tang, "A survey on user behavior modeling recommender systems," arXiv preprint wrXiv:2302.11077 2023.
[19] J. Tong. M. Yin, H. Wang, Q. Pan, D. Lian, and E. Chen, "Mdap: A multi-viee disentangled and adaptive preference learning framework for cross-domain recommendation, arXiv preprint arXiv:2410.05877 2024.
[20] L. Wu, Z. Zheng, Z. Qiu, H. Wang, H. Gu, T. Shen, C. Qin, C. Zhu, H. Zhu, Q. Liu el al., "A survey on large language models for recommendation." World Wide Web, vol. 27, no. 5, p. 60, 2024.
[21] R. He and J. McAuley, "Fusing similarity models with markov chains for sparse sequential recommendation." in 2016 IEEE 16th international conference on data mining (ICDM). IEEE, 2016, pp. 191-200.
[22] K. Choi, D. Yoo, G. Kim, and Y. Suh, "A hybrid online-product recommendation system: Com- bining implicit rating-basee collaborative filtering and sequential pattern analysis," electronic commerce research and applications, vol. 11, no. 4, pp. 309-317, 2012.
[23] J. Tang and K. Wang, "Personalized top-n sequential recommendation via convolutional se- quence embedding." in Proceedings of the eleventh ACM international conference web search and data mining, 2018, pp. 565-573
[24] S. Wu, Y. Tang. Y. Zhu, L. Wang, X. Xie, and 'T. Tan, "Session-baseer recommendation with graph neural networks," in Proceedings of the AAAI conference on artificial intelligence, vol. 33, 01, 2019, pp. 346-353.
[25] Z. Wang, W. Wei, G. Cong, X.-L. Li, X.-L. Mao, and M. Qiu, "Global context enhanced graph neural networks for session-based recommendation," in Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR '20. New York, NY, USA: Association for Computing Machinery, 2020, p. 169-178. [Online].
Available:
[26] S. Wu, Y. Tang, Y. Zhu, L. Wang. X. Xie, and T. Tan, "Session- based recommendation with graph neural networks." CoRR, vol. 1.00055, 2018. [Online]. Available: http://arxiv.org/abs//// 1ttp://aaaaii1100....... 1.00855
[27] B. Hidasi, "Session-based recommendations with recurrent neural networks," arXiv preprint arXiv:1511.06939, 2015.
[28] B. Hidasi and A. Karatzoglou, "Recurrent neural networks with top-k gains for session- based recommendations." in Proceedings of the 27th ACM International Conference Information and Knowledge Management, ser. CIKM 18. New York, NY, USA: Association for Computing Machinery, 2018, p. 843 852. [Online]. Available: httpps//chhaaog////011////266222
[29] J. Tang and K. Wang, "Personalized top-n sequential recommendation via convolutional sequence embedding." in Proceedings of the Eleventh ACM International Conference Web Search and Data Mining, ser. WSDM '18. New York, NY, USA: Association for Computing Machinery, 2018, p. 565-573. [Online]. Available: https//c...g///00011111555353311
[30] W.-C. Kang and J. McAuley, "Self-attentive sequential recommendation. 2018 IEEE Inter- national Conference on Data Mining (ICDM), 2018, pp. 197-206.
[31] Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang, "Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer." in Proceedings of the 28th ACM International Conference on Information and Knowledge Management, ser. CIKM '19. New York, NY, USA: Association for Computing Machinery. 2019, 1441-1450. [Online]. Available: https://oo.r//000115333333333353
[32] S. Luo, Y. Yao, B. He, Y. Huang, A. Zhou, X. Zhang, Y. Xiao, M. Zhan, and L. Song, "Integrating large language models into recommendation via mutual augmentation and adaptive aggregation," arXiv preprint MXX1:2401.13870 2024.
12

---

[33] G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivière, M. S. Kale, J. Love et al., "Gemma: Open models based on gemini research and technology." arXiv preprini arXiv:2403.08295, 2024.
[34] U. Khandelwal. O. Levy, D. Jurafsky, L. Zettlemoyer, and M. Lewis, 'Generalization through memorization: Nearest neighbor language models," arXiv preprini arXiv: 19Xx00//11100072 2019.
[35] W. Rae, S. Borgeaud, T. Cai, K. Millican, Hoffmaann, F. Song, Aslanides, S. Henderson, R. Ring, S. Young et al., "Scaling language models: Methods, analysis & insights from training gopher," arXiv preprint arXiv:21 12. 1446, 2021.
[36] A. Chowdhery, S. Narang. J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., "Palm: Scaling language modeling with pathways." Journal of Machine Learning Research, vol. 24, no. 240, pp. 1-113, 2023.
[37] G. Zhang, Y. Hou, H. Lu, Y. Chen, W. X. Zhao, and J.-R. Wen, "Scaling law of large sequential recommendation models," in Proceedings of the 18th ACM Conference on Recommender Systems, 2024, pp. 453.
[38] P. Zivic, H. Vazquez, and J. Sánchez, "Scaling sequential recommendation models with transformers," in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR '24. New York, NY, USA: Association for Computing Machinery, 2024, p. 1567-1577. [Online]. Available: http://b..orr/0011114//766666..2
[39]
Hoffmann, S. Borgeaud, Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark et al., "An empirical analysis of compute-optimal large language model training." Advances in Neural Information Processing Systems, vol. 35, pp. 0001--00000 2022.
[40] N. Muennighoff, A. Rush, B. Barak, T. Le Scao, N. Tazi, A. Piktus, S. Pyysalo, T. Wolf, and C. A. Raffel, "Scaling data-constrained language models," Advances in Neural Information Processing Systems, vol. 36, pp. 50 358-55333300 376, 2023.
[41] M. Yin, H. Wang. W. Guo, Y. Liu, S. Zhang. S. Zhao, D. Lian, and E. Chen, "Dataset regenera- tion for sequential recommendation." in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 3954-3965.
[42] Y. Gadre, G. Smyrnis, Shankar, Gururangan, M. Wortsman, R. Shao, Mercat, A. Fang, J. Li, S. Keh et al., "Language models scale reliably with over-training and on downstream tasks," arXiv preprint arXiv:2403.08544 2024.
[43] P. Zivic, H. Vazquez, and J. Sánchez, "Scaling sequential recommendation models with trans- formers," " in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, pp. 1567-1577.
[44] Kumar, Z. Ankner, B. F. Spector, B. Bordelon, N. Muennighoff, M. Paul, C. Pehlevan, C. Ré, and A. Raghunathan, "Scaling laws for precision," arXiv preprint arXiv:2411.04330 2024.
[45] M. Belkin, D. Hsu, S. Ma, and S. Mandal, "Reconciling modern machine-leaming practice and the classical bias-variance trade-off," Proceedings of the National Acudemy of Sciences, vol. 116, no. 32, pp. 15 859-15 854, 2019.
[46] P. Nakkiran, G. Kaplun, Y. Bansal, T. Yang, B. Barak, and I. Sutskever, "Deep double descent: Where bigger models and more data hurt," Journal of Statistical Mechanics: Theory and Experiment, vol. 2021, no. 12, p. 124003, 2021.
[47] A. Power, Y. Burda, H. Edwards, I. Babuschkin, and Misra, "Grokking: Generalization beyond overfitting small algorithmic datasets." arXiv preprint arXiv:2201.02177, 2022.
[48] X. Niu, B. Bai, L. Deng, and W. Han, "Beyond scaling laws: Understanding transformer performance with associative memory," arXiv preprint arXiv:2405.08707 2024.
13

---

[49] J. Ziv and A. Lempel, "A universal algorithm for sequential data compression." IEEE Transac- tions on information theory, vol. 23, no. 3, pp. 337-343, 1977.
[50] R. Norris, Markov chains. Cambridge university press, 1998, no. 2.
[51] F. M. Harper and J. A. Konstan, "The movielens datasets: History and context," Acm transactions on interactive intelligent systems (tiis, vol. 5, no. 4, pp. 1-19, 2015.
[52] J. McAuley and J. Leskovee, "Hidden factors and hidden topics: understanding rating dimen- sions with review text," in Proceedings of the 7th ACM conference on Recommender systems, 2013, pp. 165-172.
[53] C. Gao, S. Li, Y. Zhang, J. Chen, B. Li, W. Lei, P. Jiang, and X. He, "Kuairand: an unbiased sequential recommendation dataset with randomly exposed videos," in Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2022, pp- 3953-3957.
[54] J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, and L. Sifre, "Traiming compute-optimal large language models," 2022. [Online]. Available: https//arrv..ogg/ab2200....55
[55] W. Kang and J. J. McAuley, "Self-attentive sequential recommendation." in IEEE International Conference on Data Mining, ICDM 2018, Singapore, November 17-20, 2018. IEEE Computer Society, 2018, pp. 197-206 [Online]. Available: httpp:://dii.rr/// 10. 10001/0000/0CCM00...0.00
[56] F. Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang. "Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer," in Proceedings of the 28th ACM International Conference
Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019, W. Zhu, D. Tao, X. Cheng, P. Cui, E. A. Rundensteiner, [Online]. Available:
D. Carmel, Q. He, and J. X. Yu, Eds. ACM, 2019, pp. 1441-1450 htps://doi.org/100 1110///11145335558433333335
[57] K. Zhou, H. Wang, W. X. Zhao, Y. Zhu, S. Wang, F. Zhang, Z. Wang, and J. Wen, "S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization," in CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020, M. d' Aquin, S. Dietze, ACM, 2020, pp. 1893-1902. [Online].
C. Hauff, E. Curry, and P. Cudré-Mauroux Eds.
Available: httpps://doi.org// 14/33444/3405311331
[58] S. Waters, "Hit ratios," " The Computer Journal, vol. 19, no. 1, pp. 21-24, 1976.
[59] K. Järvelin and J. Kekäläinen, "Cumulated gain-based evaluation of IR techniques," ACM Trans. Inf. Syst., vol. 20, no. 4, pp. 422 446, 2002. [Online]. Available:
http://doi.acm.com www.m.org/1..... 1155//..111///2215555555
[60] X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang, "Lightgen: Simplifying and powering graph convolution network for recommendation." in Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR '20. New York, NY, USA: Association for Computing Machinery, 2020, p. -39.648. [Online]. Available: https://ooo.rr///0111///3////3/9
[61] A. Gu and T. Dao, "Mamba: Linear-time sequence modeling with selective state spaces," in First Conference on Language Modeling. 2024. [Online]. Available: hwtpp//wwwiiii.ee//oommmmddaaaaa
[62] B. Zhang, L. Luo, Y. Chen, J. Nie, X. Liu, D. Guo, Y. Zhao, S. Li, Y. Hao, Y. Yao, G. Lakshminarayanan, E. D. Wen, J. Park, M. Naumov, and W. Chen, "Wukong: Towards scaling law for large-scale recommendationn" 2024. [Online]. Available: httpss//ariv..rr//ab//44.......
[63] Z. Li, A. Sun, and C. Li, "Diffuree: A diffusion model for sequential recomemmtttnn ACM Trans. Inf. Syst., vol. 42, no. 3, Dec. 2023. [Online]. Available: http:://bb..og//001143333616
14

---

[64] S. M. Pincus, "Approximate entropy as a measure of system complexity." Proceedings of the national academy of sciences, vol. 88, no. 6. pp. 2297-2301, 1991.
[65] C. E. Shannon, "A mathematical theory of communication." ACM SIGMOBILE Mob. Comput. Commun. Rev. vol. 5, no. pp. 3-55, 2001. [Online]. Available: https//doi.org/ httpo///0..111//000011..388
[66] M. and M. B. Vitányi, "Kolmogorov complexity and its applications." in Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity, J. van Leeuwen, Ed.
Elsevier and MIT Press, 1990. pp. 187-254.
[67] J. J. Ruel and M. P. Ayres, "Jensen's 's inequality predicts effects of environmental variation," Trends in Ecology Evolution, vol. 14, no. 9, pp. 361-366, 1999.
[68] L. G. Kraft, "A device for quantizing, grouping, and coding amplitude-modulated pulses," Ph.D. dissertation, Massachusetts Institute of Technology. 1949.
[69] Y. Li, K. Liu, R. Satapathy, S. Wang, and E. Cambria, "Recent developments in recommender systems: A survey," 2023. [Online]. Available: https://axiiv.or//as//25..... 12680
[70] P. Ravikumar, A. Tewari, and E. Yang, "On ndeg consistency of listwise ranking methods," in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, 2011, 1, pp. 618-626.
[71] S. Abnar, H. Shah, D. Busbridge, A. M. E. Ali, Susskind, and V. Thilak, "Parameters flops: Scaling laws for optimal sparsity for mixture-of-experts language models," " 2025. [Online]. Available: https///axiv..gg/ab22250..1.1
[72] W. Krichene and S. Rendle, "On sampled metrics for item recommendation," in Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 2020, pp. 1748-1757.
A Appendix supplemental material
A.1 Proof of Theorem
Entropy Inequality following entropy inequality:
To facilitate our analysis of the Real Entropy, we need to introduce the
Lemma A.1.
67 If Vis = l,,,, n. and 1i 12 20 Ui, then
4
1
xz log,
< Te log,
Z4
V/
Based this lemma, will construct proof of the inequality between the Data Scale D and Real Entropy S below:
Theorem A.2. Assuming that the sequence can be modeled first-order aperiodic stationary Markov chain (50). If the user sequence S = {Su, E U), then the relationship between the sum of minimumm encoding length [U] (0(mmm and Real Entropy S'real is given by:
D~ |U|Cmin 2 (Eu(Sul) Streal
(10)
Proof. From Lemma A.I follows that:
U Umim Σ P(E.S.)4 Σ p( 2-1.
V..
V..S..
(11)
> HT, = Σ Sa) p(E
Y.
15

---

where HS(S represents the entropy of the sequence in log-base 2. and Kraft's inequality 68 Ei=l 2- I 1, I is encoding length was utilized. Meanwhile, from Lemma 3.1 with Real Entropy S'real 1/5real it follows that:
EulSal
Ey Su
(12)
Sreal
Σj A3
In Sal
Ea iS
Combining Eq.(11
and Eq. 12
the Eq.
10
is equivalent to proving:
En [a
Σ VS.Su.
logg
(13)
p SE)
M. S.. Σ₁ b,) In E.I.
where it necessary ensure that interaction distribution p Sw),S.)) minimal, which is more easily satisfied in recommendations with large recommendation item vocabulary. Combining the above equations, need to demonstrate that:
Σ SE log2 Sw,
S. E.
In En 2 E.
(14)
Su
We decompose inequality
(4) into the following two inequalities
15 and 16 for our final proof:
1
In In Sul
Σas
As
In En
(15)
En Sa
(E Sul + 2)
Sul Sul + 2) 4 In Em|Su
Σ p()in
log2 ((En.S))
(16)
S.
For the first inequality derived, since j is the subsequence starting from after concatenating all sequences, the minimum value of g'real is achieved when all sequences in the interaction are identical. In this case, a is a double arithmetic sequence increasing from to |Sul/2 and then returning to 1 in each user IUI. It is formally expressed follows:
Proof of Inequality
15
Eu|Sul
ΣA/
Ay
EnEn/sal2
5)
In Enlal
Du S.
1
S.a
2
3/2
(17)
Ew (..,((En) (E |S,l + 2)
4|U In E.
In ESA.
4/U [2 1/ Sy
(2, Sall
Here, it is assumed that In( S.l) recommendation systems.
10 when Vu, |Sal 2 4. which is reasonable in the context of
x., 1S.112+2
Proof of Inequality (16) On the other hand, the generation probability distribution of most sequences tends to be relatively uniform and considering all permutations of all user sequences, we have:
1 Σ P(EuSu) log, P(E., P(E....) VE
S. ln2
P(Eu P(E.sS,)P(ESSuu loga S......)
In [7]
(18)
Due the differing distributions across datasets, deriving conclusions without considering ac- tual circumstances can be challenging. To address this, we introduce an empirical inequality: In |n E,, 2 1.S. + 2. The left side of this inequality increases as the dataset grows larger, whereas the right side remains a relatively small constant in typical recommendation tasks (maximum of 500 in our datasets) (69). In our smallest dataset, Kuairand, the left side evaluates to 581, still satisfying the inequality. Therefore, we can assert that this inequality will likely hold under the Scaling Data condition. Under this inequality, we have:
Su In |T 5|0 SS In / S..
Σ E.y, logz
p S(E.Sw))
In2
5 2 In Σ,, S.al
VE.S.
(19)
((U)() + 2),
(S,, S.I + 2)
Σn S. (E., +
5 2 U In Σn Sal
5 2 In E, Swl
4U In En Sall
16

---

Overall, Inequality (77) proves Inequality (1).. and Inequality (19 proves Inequality (16). Thus, we have completed the proof of Inequality (14) and, ultimately, the proof of the theorem.
A.2 Proof of Theorem 4.2
After obtaining the model's loss and performance under different data and model parameters, we need construct fitting function for the final model performance. However, nearly all sequential recommendation metrics (e.g., NDCG, HR, etc.) are discrete, making it difficult to establish their relationship with the model directly. Therefore, We first select appropriate metric by examining the differences between performance and testing loss, choosing one with minimal discrepancy. We then analyze the loss constraint inequality during testing. Due the guarantee of consistency, this inequality can be directly applied to Performance. Ultimately, we introduce the Squeeze Theorem of Performance Fitting to establish a rigorous foundation for understanding the relationship between model configurations and performance. According to the definition of the Squeeze Theorem, we will identify a reasonable fitting function for model performance evaluation. The whole process is shown in Part (C) in Figure 2 Since our analysis focuses on testing losses, our analytical approach still addresses the overfitting challenges faced by the original Scaling Law.
A.2.1 Metric-Loss Consistency
We then need to discuss the consistency between testing loss and Performance. We chose NDCG and HR as metrics in our experimental analysis measure the model's performance. Their consistency ensured by (70), respectively, and possesses the following properties:
AF
|DDC Theeel S
Vo(LNDOG) -[(ee)))-
(20)
Here, LNDCG represents an NDCG-like function, which is formalized as M Σラ F(wG)) By replacing weight distribution w(j) with the mean value, it becomes HR. 4 denotes a convex function, while Aş F and Ag are parameters. The upper bound of this difference ensures the appropriateness of our selected metrics, ensuring that in our subsequent analysis of model performance, the formulas for performance and testing loss allow for interchangeable use.
G(r))
A.2.2 Inequaly on Performance Decay
Subsequently, the testing loss, which can also be interpreted as performance due to consistency, is constrained follows:
Lemma A.3. (48) Suppose input sequence of each user [ee e²
ehisu]] E Rdin, then
with small offset we huve
< Trast ( H, demb, D) = =log 74 +
+ log H + H 6,
(21)
Z:
D Varn (v
dia 2nee
din
< 11 < D V (V
(22)
exp(v
2TE
Zire
where Lreat denotes testing loss, Vdd (r) dimension of input sequence.
din din T[ dim dra = man X domb the
we can utilize constraints provided by Lemma A.3 and Eq, (20). which can be formulated
1
P(H, demb, D) log Zz
Z, log H + ( (Lnest). Next, will introduce the V
5,
(23)
H
where 8 is small offset including Ar.
Squeeze Theorem Performance Fitting. Overall, we first introduce exponential parameters wy and W4 for the model depth H and the embedding dimension demb, respectively. We then use the properties of the Squeeze Theorem prove the existence of these parameters w3 and wa between lower bounds uig and u.g, and upper bounds w/ and w/. Specifically, we have:
17

---

Table The basic information for different datasets, where S. denotes the sequence length, along with the fitted data parameter D in different metrics (HR, NDCG, Loss). It's relationship versus
D becomes excessively large in Industrial affecting the fitting process, we uniformly take the reciprocal, 1/-, for the analysis.
#/# Tokens (S' R and / #T is illustrated in Figure 3
Dataset Details, # okenss
E..
Dataset Parameter Fitting * 1E-07)
Dataset
Sulmaz
#Tokens
S S'Real Real
D/uR
1/DNDCG
1/D
25
447.407
0.2998
0.9800
1.0531
0,0352
User: 27,285
KR-Pure
50
570,537
0,3153
0.8081
0.7080
0.0235
Item: 7,551
100
661,028
0.3544
0.5560
0,5669
0.0199
100
505,108
0.1864
1.3699
1,5003
0,0459
User: 6,040
ML-IM
150
802,493
0.1856
0.8972
0.8130
0.0263
Item: 3,706
200
1,058,511
0.1854
0.7839
0.7039
0.0218
User: 694,897
50
8,044,865
0.1130
0.2100
0.1693
0.0021
AMZ-Boo AMZ-Books
Item: 686,623
25
7,076,238
0.1129
0.2404
0.2093
0.0030
User: 19,252,028
50
$13,878,761
0.3769
0.0142
0.0175
0.0001
Industrial
Item: 234,488
25
327,509,107
0,4001
0.0197
0,0249
0.0001
Theorem A.4. Squeeze Theorem of Performance Fitting. There exist 183, w/y, w1, w's such that
log(dii
emb emb D +
+ log + H P(H, D)
det Duia
(24)
< (og( Duls)
log H +
+
dmnl emb
Dw
H
The lower bound of 4 is the utilize of the property of function ! which is formulated as:
Proof.
log D deme)
S log (+))
(25)
d emb
D
Z.
On the other side, the upper bound can be proven using the following incqualities:
din
Dv Ain
din 2mm
dia 2Te
D
74 S D Vn(1
k
r( d(d
+ +1)
Smardems
ge
(26)
log( Zz) +
7 < Oloog( DD - - +
D
.d -emb
Here, we apply Stirling's approximation [(2 + 1) V2TT (2) as the proof of the rightmost inequality. This requires Z, 1, which common in the overfitting phenomenon when the model studied in this paper increases rapidly, as evidenced by the u-shape images in the experiments. This approach enables mutual decay terms, optimizing the fit for performance. Similar to (7)) We factorize the product d=s Dws into dmds and D adding extra parameter w', W2 and ws as the final structure of Eq. 8 which represents the ultimate form of our performance fitting model.
decompose the loss into the form + log(-). where 1 and log(-) serve as
A.3 Details on Dataset and Data Parameter Fitting
We present the specific details of the dataset and the precise numerical values for the images shown in Section 5.1.1 in Table 3
A.4 Detailed Experiment Settings
Following previous works 55 56 57). we leverage the leave-one-out method to calculate the
recommendation performance. Besides, we adopt the whole item set as the candidate item set during evaluation to avoid the sampling bias of the candidate selection (72). Then, we evaluate the Top-K recommendation performance by Normalized Discounted Cumulative Gain (NDCG) (59) and Hit
18

---

Nocia UPEP CHAROTH
Figure 5: The PL function fitting between model performance (Z-axis, HR) and the number of layers (X-axis, H), well the embedding dimensions (y-axis, demb), the plot includes annotations of the correlation coefficients R².
Rate (HR) (58). To effectively demonstrate the performance of models with varying parameters across different datasets, we selected different parameters for fitting based on the size of each dataset. Regarding model configurations, for the MovieLens- 1M, KuaiRand -ure,, and Amazon Books datasets, we configured N E {4, 8, 12, 16, 24, 32] and demb E {25, 50, 75, 100). For the Industrial dataset, perspective, we selected the maximum sequence length for truncation based on the average length of each dataset. In the MovieLens- IM dataset, we selected according to the maximum sequence length Smaz E {100, 150, 200). In the KuaiRand-Pure dataset, we set the maximum sequence length Smar E [25, 50, 1003. Finally, for the Amazon Books and Industrial datasets, we configured the maximum sequence length Smaz E {25, 50}. The largest model we executed reached a model depth of = 64, an embedding dimension of demb 1024, and vocabulary size of [Z] 19, 252, 028. We utilized 48 industrial GPUs to run this experiment, with the largest experiment taking 24 hours. This truly allowed us study model performance extreme data and model scales.
set N {8, 16, 16,32, 64) and demb E {128, 256, 512, 1024). From a data
A.5 Application 2: Exploring Performance Law Potential Among Framework
Another application of the Performance Law is to assess potential performance gains when scaling up the model. We con- ducted experiments and fitting analyses on three different frameworks (HSTU (IO), LLaMA2 (8) and SASRec (15) evalu- ated at different precisions: float32 and 66loat16. Experiments were performed on the smaller dataset (ML-IM) to enable the models to more easily reach their op- timal upper bounds. In the expressions pl and (
Table 4: Comparison of Model Parameters and Perfor- mance Across Different Precisions in Movielens- 1M with NG denotes NDCG. All results are statistically significant with p<0.05.
Precision Model
Float32
B6aat
USTU
SASRec
LLaMAZ
SASRec
ws
0.033
0.036
0.007
0.003
0.015
-U.11A
wy
0.085
0.159
0.001
0.034
0.086
0.008
HR 10
0.332
0.346
0.302
0.352
0.336
0.293
HR 50
0.585
0,598
0.573
0.594
0.598
0.561
NG# 101
0.185
0.194
0.172
0.187
0.188
0.162
NG#501
0.242
0.252
0.231
0.247
0.249
0.221
wi (log H + Hu3)
21 ) in Eq.(8), the upper bound of log(H")) + 21 does not change with variations in w when pl
is fixed. Therefore, larger values of wi and wy indicate a better scaling-up potential for the model. We tested several different types of models (HSTU, LLaMA, and SASRec) and different precisions (Float32 and 6float16) to observe the relationship between optimal model performance and the fitted parameters wi and w2. The results are presented Table 4 From the table, we conclude that the model's 's performance closely aligns with the magnitude trend of wi and w2, further underscoring the accuracy of our quantitative fitting. This also demonstrates that the Performance Law can effectively guide model structure configuration, thereby reducing memory and time when modifying frameworks.
19

---

Nocia UPEP DHAROUE
Figure 6: The SL function fitting between model performance (Z-axis, HR) and the number of layers (X-axis, H), well the embedding dimensions (y-axis, demb), the plot includes annotations of the correlation coefficients R
Figure 7: The PI. function fitting between model performance (Z-axis, NDCG) and the number of layers (x-axis, H), as well as the embedding dimensions (y-axis, demb), the plot includes annotations of the correlation coefficients R².
A.6 Result Illusion Extension
B
Additional Model Extension Results
Detailed quantitative results for Performance Law, Scaling Law, and Precision Scaling Law on additional advanced models and the Amazon Beauty long-tail scenario are presented below.
As shown, the Performance Law consistently surpasses conventional Scaling Law and Precision Scaling Law in terms of both fitting quality (/12) and error measures (MAE, RMSE), demonstrating its extensibility and robustness across graph-based, sequence-based, multimodal, diffusion-based. and long-tail recommendation models.
C Experimental Results for Data Quality Evaluation Functions
We provide detailed comparison of different data quality evaluation functions: Real Entropy, Token Only, Approximate Entropy (64). Shannon Entropy, and Kolmogorov Complexity. The results across three metrics (NG, HR, L) are presented in Table where higher values indicate better fitting ability.
As shown above, Real Entropy achieves the best overall performance as data quality metric, outperforming Approximate Entropy (64), Shannon Entropy, and Kolmogorov Complexity.
20

---

Figure 8: The PL function fitting between model performance (Z-axis, NDCG) and the number of layers (x-axis, H), as well as the embedding dimensions (y-axis, demb). the plot includes annotations of the correlation coefficients R².
D Additional Evaluation Metrics: MAE and RMSE
W5: The evaluation metric is overly dependent on R2; additional metrics such MAE/RMSE or downstream utility of predicted parameters would strengthen the empirical support.
Answer:
Thank you for your valuable suggestion. To address this concern, have supplemented the empirical analysis with corresponding MAE and RMSE results for each dataset and model size, in addition to R Tables and below report the detailed MAE and RMSE comparisons. The results demonstrate that Performance Law consistently achieves lower MAE and RMSE values than both Scaling Law and Precision Scaling Law across all scenarios, supporting its robust predictive accuracy beyond the 1² metric.
21

---

Table Performance comparison on LightGCN, Mamba, Wukong, DiffuRec, and Amazon Beauty
(long-tail).
Model
Metric
Rº
MAE (xle-3)4
RMSE (xle-3))
LightGCN
HR@ 10
Performance Law
0.879
4.665
5,806
Scaling Law
0.825
5,456
6.978
Precision Scaling Law
0.866
4.918
6.116
NDCG@10
Performance Law
0.850
5.423
6.744
Scaling Law
0.800
6.120
7,802
Precision Scaling Law
0.806
5.797
7.223
Mamba
HR@ 10
Performance Law
0.818
1,941
2,512
Scaling Law
0.532
3.342
4.021
Precision Scaling Law
0.551
3.169
3.805
NDCG@10
Performance Law
0.810
0.398
0.530
Scaling Law Precision Scaling Law
0.663
0,553
0.705
0.663
0,532
0.701
Wukong
AUC
Performance Law
0.528
0.624
0.868
Scaling Law Precision Scaling Law
0.479
0.654
0,912
0.503
0.636
0.891
DiffuRec
HR@ 10
Performance Law
0.766
34.933
46.596
Scaling Law
0.608
50.277
60.358
Precision Scaling Law
0.652
46.882
56.908
NDCG@10
Performance Law
0.748
17.077
20.707
Scaling Law
0.617
21.006
25.505
Precision Scaling Law
0.666
19.339
23.818
Amazon Beauty (Ling-tail Rec.)
HR@ 10
Performance Law
0.868
1,857
2.271
Scaling Law Precision Scaling Law
0.335
4.265
5.092
0.334
4.267
5.094
NDCG@10
Performance Law
0.890
1.329
1.668
Scaling Law
0.513
2,956
3.509
Precision Scaling Law
0.513
2.957
3,510
Table 6: Comparison of data quality evaluation functions (R²).
Metrie
Real Entropy
Token Only
Approx. Entropy
Shannon Entropy
Kolmogorov Complexity
NDCG
0.9906
0.8160
0,9825
0.7982
0.8955
HR
0.9913
0.8259
0,9774
0,7998
0.8663
Loss
0.9881
0.8776
0.9471
0.8617
0.9374
22

---

Table MAE comparison for HR @ 10 and NDCG@ 10 across datasets and model sizes.
HR @ 10 (MAE x10
NDCG@ 10 (MAE x x10
Dataset
|
Perf. Law
Sealing Law
Precision Law
Perf. Law
Scaling Law
Precision Law
KR-Pure
2.111
2.793
2,800
0.768
1.655
1.658
50
1.027
3.269
3,273
0.619
1.921
1.922
100
1.871
2.587
2.588
0.958
1.458
1.459
ML-IM
100
2.990
6,064
6.086
1.928
3.927
3,934
150
2.013
2.716
2.725
2.013
2.716
2.725
200
1.645
4.844
4,846
1.645
4.844
4.846
AMZ-Books
25
2.704
3.495
3.462
1.633
2.271
2.252
50
2.719
2.725
2.720
1.662
1.715
1.712
Industrial
25
1.112
2.291
2.302
1.031
1.864
1,870
0.972
3.356
3.356
0.830
2.700
2.700
Table 8: RMSE comparison for HR 10 and NDCG@ 10 across datasets and model sizes.
HR@ 10 (RMSE x x10
NDCG@ 10 (RMSE x10-3)
Dataset
|Sal
Perf. Law
Scaling Law
Precision Law
Perf. Law
Scaling Law
Precision Law
KR-Pure
2.880
3.610
3.615
0.923
1.943
1.946
50
1.260
3.754
3.758
0.806
2.170
2.171
100
2.534
3,302
3,303
1.147
1.827
1.827
ML-IM
100
3,417
7.332
7.351
2.525
4.425
4.431
150
2.657
3,948
3,957
2.657
3,948
3.957
200
1.912
5.493
5.494
1.912
5,493
5.494
AMZ-Books
25
3,535
4.084
4,058
2.148
2.626
2.612
50
3.527
3.610
3.609
2.117
2.173
2.171
Industrial
25
1.294
2,676
2.693
1.186
2.223
2.232
1.341
3,857
3.865
1.140
3.108
3.111
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Justification: In the abstract and introduction section, the theoretical and technical contributions have been illustrated, which match the experimental results. The scope is clearly defined the field of sequential recommendation systems
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Limitations this work are discussed Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
24

---

Answer: [Yes]
Justification: The assumption and proof of the two proposed theory are provided in Section Appendix A.T. and Appendix A.2
4.2
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The code is provided at https://anonymous.oppoese aatppesscceneee/ffffommaeeeeeeee DDE3/ The implementation details to reproduce the experimental results are provided in
Appendix A.4
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
25

---

reproducible, they 5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification: The code is provided at http://aaaoanooooos......cuuccor DDE3/ along with sufficient instructions.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https::/nnis.cc
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/nipss wwtpcc/pcppp.pceeeeeeeooo gwwcc//gu/es//eee /wweeeeeeooobdbmiicccceca wwwninsssonPooo swiniioisiiiyccyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The implementation details to reproduce the experimental results are provided in Section[A.4
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: We have performed statistical significance analysis and provided the corre- sponding p-value in Table [ Table 2 and A.5
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
26

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The compute resources are detailed in Section A.4
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https::/neuripp: cttpouurpppscpppuliiieepeee
Answer: [Yes]
Justification: We have carefully checked the NeurIPS Code of Ethics and the conducted research conforms with it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Both potential positive societal impacts and negative societal impacts have been discussed in Section
Guidelines:
The answer NA means that there is no societal impact of the work performed.
27

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The proposed model does have high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The creators of used code and data are credited in Section 4.2.1 We adhere to licenses of use by including citations, version details, and relevant license information.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
28

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, batasets.papersstt paperswi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code repositories will provide comprehensive details for newly introduced assets, ensuring that other researchers can effectively utilize these resources.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
29

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
30


Reviews:
[{"review_id": "oH8GHNLaCv", "reviewer": "Reviewer_HQgL", "summary": "This paper introduces the Performance Law for sequential recommendation algorithms, aiming to predicts model performance across various settings, intending to provide a quantitative framework for guiding the parameter optimization of future models. \nIt first utilizes Real Entropy to measure data quality, aiming to remove the low-quality influence of low-entropy redundant sequences. It then investigates a fitting decay term to facilitate the prediction of the major loss-performance discrepancy phenomena of overfitting, ultimately achieving quantitative performance prediction. Extensive experiments on diverse datasets demonstrate quantitative improvements over established scaling law baselines in both performance prediction and parameter optimization tasks.", "questions": "1. Can you discuss about the feasibility of the proposed Performance Law in other other SR architectures / advanced generative techniques?\n2. It would be beneficial to include additional entropy-based data quality metrics for comparison.", "limitations": "Please refer to the weakness and questions. btw, I think this work is interesting, I would like to raise my score if the authors can solve my concerns.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The proposed Performance Law is the first approach to move scaling law analysis in SR from qualitative into quantitative domain to anticipate the effects of how both data and model parameters change will guide the optimization of future models.\n2. The proposed Performance laws are grounded in theoretical claims with explicit bounds and relationships.\n3.  Extensive experiments span several benchmarks and an industrial-scale dataset, demonstrating the effectiveness of Performance Law by displaying exceptional quantitative prediction ability against qualitative SL. \n4. The authors have publicly released the dataset and corresponding code.\n\nWeaknesses:\n1. The proposed Performance Law focus on scaling Transformer-based models. It is unclear whether the law applies equally well to other SR architectures (e.g., RNN- or GNN-based SR in Sec. 2) / advanced generative techniques (e.g., DM-based SR).\n[1] Diffurec: A diffusion model for sequential recommendation\n[2] Plug-in diffusion model for sequential recommendation\n[3] Generate what you prefer: Reshaping sequential recommendation via guided diffusion\n2. The definitions of S^{real} (Real Entropy) and S^{\\prime real} (the final measure of Real Entropy) are overly similar, and both are referred to as “Real Entropy” in Lines 213 and 215. The authors could use distinct symbols or notation to clearly differentiate between them.\n3. There are only the comparison between #Tokens·S^{\\prime real} and #Tokens in In Figure 3. Including additional entropy-based data quality metrics in the comparison could further support the validity of the proposed Performance Law.", "was_revised": true, "final_justification": "All of my concerns have been addressed by the authors, and they have also released all the code from the rebuttal period in their own anonymous GitHub repository for future researchers to study. Therefore, I have decided to raise my initial score.", "rebuttal": "We sincerely thank you for your valuable comments. We hope the following responses can address your concerns.\n>W1  It is unclear whether the law applies equally well to other SR architectures (e.g., RNN- or GNN-based SR in Sec. 2) / advanced generative techniques (e.g., DM-based SR). \n\nThanks for the suggestions on validation for different structures. We have provided a comparison of the predictive performances between the LightGCN[1] and DiffuRec[2]. Overall, our results still surpass those of the Scaling Law and Precision Scaling Law.\n\n[1]He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., & Wang, M. (2020). LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. SIGIR'20\n\n[2]Li Z, Sun A, Li C. Diffurec: A diffusion model for sequential recommendation[J]. ACM Transactions on Information Systems, 2023, 42(3): 1-28.\n\n| LightGCN                  |                       | $R^2\\uparrow$ | MAE (x1e-3)$\\downarrow$ | RMSE (x1e-3)$\\downarrow$ |\n| ------------------------- | --------------------- | ------------- | ------------------------ | ------------------------- |\n| [HR@10](mailto:HR@10)     | Performance Law       | **0.879**     | **4.665**                | **5.806**                 |\n|                           | Scaling Law           | 0.825         | 5.456                    | 6.978                     |\n|                           | Precision Scaling Law | 0.866         | 4.918                    | 6.116                     |\n| [NDCG@10](mailto:NDCG@10) | Performance Law       | **0.850**     | **5.423**                | **6.744**                 |\n|                           | Scaling Law           | 0.800         | 6.120                    | 7.802                     |\n|                           | Precision Scaling Law | 0.806         | 5.797                    | 7.223                     |\n\n| DiffuRec                  |                       | $R^2\\uparrow$ | MAE (x1e-3)$\\downarrow$ | RMSE (x1e-3)$\\downarrow$ |\n| ------------------------- | --------------------- | ------------- | ------------------------ | ------------------------- |\n| [HR@10](mailto:HR@10)     | Performance Law       | **0.766**     | **34.933**               | **46.596**                |\n|                           | Scaling Law           | 0.608         | 50.277                   | 60.358                    |\n|                           | Precision Scaling Law | 0.652         | 46.882                   | 56.908                    |\n| [NDCG@10](mailto:NDCG@10) | Performance Law       | **0.748**     | **17.077**               | **20.707**                |\n|                           | Scaling Law           | 0.617         | 21.006                   | 25.505                    |\n|                           | Precision Scaling Law | 0.666         | 19.339                   | 23.818                    |\n\n\n\n> W2 The definitions of S^{real} (Real Entropy) and S^{\\prime real} (the final measure of Real Entropy) are overly similar\n\n Thanks for pointing out the presentation issue of this paragraph. In subsequent revisions, we will rename $(S^{real}$ to \"Original Real Entropy\" and modify $S^{\\prime real}$ to $\\overline{S^{real}}$ to enhance their distinctiveness.\n\n\n> W3 There are only the comparison between #Tokens·S^{\\prime real} and #Tokens in In Figure 3. Including additional entropy-based data quality metrics in the comparison could further support the validity of the proposed Performance Law.\n\n  We have added experiments comparing the replacement of Real Entropy with Approximate Entropy [3], Shannon Entropy, and Kolmogorov Complexity (Implemented by compression rate)[4]. The results indicate that Real Entropy maintains the best fitting ability.\n\n|  $R^2$    | Real Entropy $S^{real}$| Token Only | Approximate Entropy | Shannon Entropy | Kolmogorov Complexity |\n| ---- | ------------ | ---------- | ------------------- | --------------- | --------------------- |\n| NG   | **0.9906**     | 0.8160     | 0.9825              | 0.7982          | 0.8955                |\n| HR   | **0.9913**     | 0.8259     | 0.9774              | 0.7998          | 0.8663                |\n| L    | **0.9881**   | 0.8776     | 0.9471              | 0.8617          | 0.9374                |\n\n\\[3\\]S.M. Pincus, Approximate entropy as a measure of system complexity., Proc. Natl. Acad. Sci. U.S.A. 88 (6) 2297-2301\n\n[4] Yin, M., Wu, C., Wang, Y., Wang, H., Guo, W., Wang, Y., Liu, Y., Tang, R., Lian, D., & Chen, E. (2024). Entropy Law: The Story Behind Data Compression and LLM\n\n\n >Q1 Can you discuss the feasibility of the proposed Performance Law in other SR architectures / advanced generative techniques?\n\n We appreciate your feedback! We have compared the fitting and predictive capabilities of the Performance Law against other baselines within the advanced generative frameworks of Mamba and DiffuRec (For DiffuRec, see reference W1 above). The Performance Law continues to demonstrate superiority. \n\n| Mamba                     |                       | $R^2\\uparrow$ | MAE (x1e-3)$\\downarrow$ | RMSE (x1e-3)$\\downarrow$ |\n| ------------------------- | --------------------- | ------------- | ------------------------ | ------------------------- |\n| [HR@10](mailto:HR@10)     | Performance Law       | **0.818**         | **1.941**                    | **2.512**                     |\n|                           | Scaling Law           | 0.532         | 3.342                    | 4.021                     |\n|                           | Precision Scaling Law | 0.551         | 3.169                    | 3.805                     |\n| [NDCG@10](mailto:NDCG@10) | Performance Law       | **0.810**         | **0.398**                    | **0.530**                     |\n|                           | Scaling Law           | 0.663         | 0.553                    | 0.705                     |\n|                           | Precision Scaling Law | 0.663         | 0.532                    | 0.701                     |\n\n\n\n >Q2 It would be beneficial to include additional entropy-based data quality metrics for comparison.\n\n  Thank you for your suggestions regarding entropy value comparisons. For detailed information, please refer to W3."}, {"review_id": "q7ee9u5JoT", "reviewer": "Reviewer_7fJJ", "summary": "This paper proposes the Performance Law, highlighting that a reduction in loss does not always lead to improvements in performance. The authors aim to address this issue by introducing Real Entropy as a corrective factor. Compared to existing scaling laws, the proposed approach demonstrates higher prediction accuracy and designs a new objective function capable of explaining overfitting phenomena.", "questions": "1. Since the experiments mainly focus on transformer-based models, I am curious whether the proposed formulation would also apply to other types of recommendation models (e.g., RNN, GNN, Mamba).\n2. Have you conducted any experiments on recommendation models based on large language models (LLMs)?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "- **Quality**: I highly appreciate the motivation to address the limitations of scaling laws, which do not translate well to sequential recommendation tasks.\n- **Clarity**: Overall, the paper is clearly written, but the heavy reliance on mathematical formulations makes it challenging to grasp the intuition behind the approach.\n- **Significance**: Given the increasing prevalence of large-scale models in recommendation systems, establishing a quantitative framework for performance prediction is likely to have a positive impact on the field.\n- **Originality**: I found the topic of quantitatively predicting the performance gains from scaling sequential recommendation models to be novel and interesting.", "was_revised": true, "final_justification": "The concern about the lack of non-Transformer results was well addressed in the rebuttal. I find the main novelty of this paper to be its focus on the underexplored relationship between model size and performance in recommender systems.", "rebuttal": "We are grateful for your kind remarks. We hope the following responses can address your remaining concerns.\n> Q1 Since the experiments mainly focus on transformer-based models, I am curious whether the proposed formulation would also apply to other types of recommendation models (e.g., RNN, GNN, Mamba)\n\nAnswer: We appreciate your feedback! We have added a comparison of Performance Law within the LightGCN and Mamba frameworks, where it continues to outperform other Scaling Laws.\n\n| LightGCN                  |                       | $R^2\\uparrow$ | MAE (x1e-3)$\\downarrow$ | RMSE (x10e-3)$\\downarrow$ |\n| ------------------------- | --------------------- | ------------- | ----------------------- | ------------------------- |\n| [HR@10](mailto:HR@10)     | Performance Law       | **0.879**     | **4.665**               | **5.806**                 |\n|                           | Scaling Law           | 0.825         | 5.456                   | 6.978                     |\n|                           | Precision Scaling Law | 0.866         | 4.918                   | 6.116                     |\n| [NDCG@10](mailto:NDCG@10) | Performance Law       | **0.850**     | **5.423**               | **6.744**                 |\n|                           | Scaling Law           | 0.800         | 6.120                   | 7.802                     |\n|                           | Precision Scaling Law | 0.806         | 5.797                   | 7.223                     |\n\n| Mamba                     |                       | $R^2\\uparrow$ | MAE (x10e-3)$\\downarrow$ | RMSE (x10e-3)$\\downarrow$ |\n| ------------------------- | --------------------- | ------------- | ------------------------ | ------------------------- |\n| [HR@10](mailto:HR@10)     | Performance Law       | **0.818**     | **1.941**                | **2.512**                 |\n|                           | Scaling Law           | 0.532         | 3.342                    | 4.021                     |\n|                           | Precision Scaling Law | 0.551         | 3.169                    | 3.805                     |\n| [NDCG@10](mailto:NDCG@10) | Performance Law       | **0.810**     | **0.398**                | **0.530**                 |\n|                           | Scaling Law           | 0.663         | 0.553                    | 0.705                     |\n|                           | Precision Scaling Law | 0.663         | 0.532                    | 0.701                     |\n\n>Q2 Have you conducted any experiments on recommendation models based on large language models (LLMs)?\n\nIn recommendation systems, LLMs are often utilized through fine-tuning, which differs from the large recommendation models studied in this paper. However, some studies [1] have mentioned that the input sequence length can also affect scaling outcomes. Therefore, we conducted a scaling analysis with sequences of similar user input lengths, as shown in the table below. The performance law still exhibits superior predictive capability compared to the scaling law.\n\n| LLM                       |                       | $R^2\\uparrow$ | MAE (x10e-3)$\\downarrow$ | RMSE (x10e-3)$\\downarrow$ |\n| ------------------------- | --------------------- | ------------- | ------------------------ | ------------------------- |\n| [HR@10](mailto:HR@10)     | Performance Law       | **0.862**     | **0.195**                | **0.253**                 |\n|                           | Scaling Law           | 0.841         | 0.221                    | 0.271                     |\n|                           | Precision Scaling Law | 0.847         | 0.217                    | 0.265                     |\n| [NDCG@10](mailto:NDCG@10) | Performance Law       | **0.875**     | **0.407**                | **0.477**                   |\n|                           | Scaling Law           | 0.555         | 0.844                    | 0.890                     |\n|                           | Precision Scaling Law | 0.555         | 0.849                    | 0.890                     |\n\n\\[1\\]Yue, Z., Zhuang, H., Bai, A., Hui, K., Jagerman, R., Zeng, H., Qin, Z., Wang, D., Wang, X., & Bendersky, M. (2025). Inference Scaling for Long-Context Retrieval Augmented Generation. ICLR 2025"}, {"review_id": "E39R1vlsDW", "reviewer": "Reviewer_UySy", "summary": "This paper introduces Performance Law, a novel framework for quantitatively modeling and predicting the performance of sequential recommendation (SR) models. It addresses key limitations of prior Scaling Laws (SLs) in SR by (1) introducing Real Entropy to evaluate data quality and eliminate redundancy, and (2) incorporating a decay term to resolve the loss-performance discrepancy caused by overfitting. Extensive experiments demonstrate superior predictive accuracy over SL baselines. The approach is also shown to be useful for optimal parameter search and model expansion analysis.", "questions": "see weakness", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths  \n- Proposes a new *Performance Law* that directly predicts model performance (e.g., HR, NDCG), rather than just training loss, which is more relevant to recommendation quality.  \n- Introduces Real Entropy, a well-justified and empirically effective data quality measure tailored for sequential recommendation data.  \n- Adds a decay term to the performance fitting function to model overfitting phenomena, which improves the predictive power.  \n- Provides strong theoretical analysis with formal definitions, theorems (e.g., Theorem 4.1, 4.2), and derivations.  \n- Offers extensive experiments on multiple datasets (including a massive industrial one) and consistent gains over SL and PcSL in R² performance.  \n- Demonstrates practical utility by using the proposed law for optimal parameter search under different constraints and model scaling strategies.  \n- Paper is well-organized, clearly written, and code is available.\n\n---\n\n### Weaknesses  \n- The claim that Performance Law is generalizable beyond SR is not empirically supported; all experiments are limited to sequential recommendation tasks.  \n- The derivation of Theorem 4.1 depends on a first-order stationary Markov assumption for user sequences, which may not hold in real-world applications.  \n- The baseline comparison is narrow, mainly focusing on SL variants; recent performance estimation techniques are not included.  \n- The efficiency (runtime, memory) of computing Real Entropy, especially on industrial datasets, is not discussed.  \n- The evaluation metric is overly dependent on R²; additional metrics such as MAE/RMSE or downstream utility of predicted parameters would strengthen the empirical support.", "was_revised": true, "final_justification": "I believe the authors have provided a strong rebuttal, effectively addressing my concerns. I think it’s a good paper, so I will maintain my positive evaluation and score.", "rebuttal": "We sincerely thank you for reviewing our manuscript. We hope the following responses can address your concerns.\n> W1 The claim that Performance Law is generalizable beyond SR is not empirically supported; all experiments are limited to sequential recommendation tasks.\n\nThank you for your advice! We evaluated the predictive performance of the Performance Law in graph learning (LightGCN) and content-based recommendation (Wukong), where it continues to demonstrate leading capabilities.\n\n| LightGCN                  |                       | R^2$\\uparrow$ | MAE (x1e-3)$\\downarrow$ | RMSE (x1e-3)$\\downarrow$ |\n| ------------------------- | --------------------- | ------------- | ----------------------- | ------------------------ |\n| [HR@10](mailto:HR@10)     | Performance Law       | **0.879**     | **4.665**               | **5.806**                |\n|                           | Scaling Law           | 0.825         | 5.456                   | 6.978                    |\n|                           | Precision Scaling Law | 0.866         | 4.918                   | 6.116                    |\n| [NDCG@10](mailto:NDCG@10) | Performance Law       | **0.850**     | **5.423**               | **6.744**                |\n|                           | Scaling Law           | 0.800         | 6.120                   | 7.802                    |\n|                           | Precision Scaling Law | 0.806         | 5.797                   | 7.223                    |\n\n| Wukong |                       | R^2$\\uparrow$ | MAE (x1e-3)$\\downarrow$ | RMSE (x1e-3)$\\downarrow$ |\n| ------ | --------------------- | ------------- | ----------------------- | ------------------------ |\n| AUC    | Performance Law       | **0.528**     | **0.624**               | **0.868**                |\n|        | Scaling Law           | 0.479         | 0.654                   | 0.912                    |\n|        | Precision Scaling Law | 0.503         | 0.636                   | 0.891                    |\n\n\n> W2 First-order stationary Markov assumption for user sequences may not hold in real-world applications.\n\nThe Markov assumption for user sequences is considered reasonable in the recommendation domain \\[1\\]\\[2\\], making its application in real-world scenarios appropriate as well. We introduce this condition mainly to ensure that the probability of a user's next item interaction can be calculated for all users. Since the Real Entropy for all datasets can be computed, this condition can effectively be overlooked.\n\n[1] Rendle S, Freudenthaler C, Schmidt-Thieme L. Factorizing personalized Markov chains for next-basket recommendation[C]//Proceedings of the 19th international conference on World Wide Web. 2010: 811-820.  \n[2] Yang Y, Jang H J, Kim B. A hybrid recommender system for sequential recommendation: combining similarity models with Markov chains[J]. IEEE Access, 2020, 8: 190136-190146.\n\n> W3 The baseline comparison is narrow, mainly focusing on SL variants; recent performance estimation techniques are not included？\n\nApologize for any misunderstanding. Recent research on large recommendation models has focused on model-based rather than theoretical approaches, with all enhanced framework recommendation models being guided by the Scaling Law. We have already discussed the comparison with the latest Precision Scaling Law [3], as shown in W5.\n\n[3]T. Kumar, Z. Ankner, B. F. Spector, B. Bordelon, N. Muennighoff, M. Paul, C. Pehlevan, C. Ré,494 and A. Raghunathan, “Scaling laws for precision,” arXiv preprint arXiv:2411.04330, 2024\n\n> W4 The efficiency (runtime, memory) of computing Real Entropy, especially on industrial datasets, is not discussed.\n\nWe thank the reviewer for pointing out the effectiveness trade-off. We extende our computation on averaging approach to expedite computations on industrial datasets by segmenting the original dataset into shards of 100,000 sequences each and calculating their average values. \n\nThis segmentation strategy allows for complete parallelization, resulting in a computation time of 1 hour and 32 minutes for the industrial dataset containing 513,878,761 sequences. This is relatively efficient compared to the training time of recommendation models of this scale. For the Amazon Books dataset, which contains 7,076,238 sequences, the computation takes 15 minutes. (Due to parallel computing, the time is not strictly proportional to the dataset size.) \n\nThis method also facilitates updates, as new sequences just need to be divided into segments of the same size. To demonstrate the stability of sharded averaging, we enumerate the relationship between the calculated values and shard size as follows:\n\n| Segment Length | 100    | 1000   | 10000  | 100000 | All    |\n| -------------- | ------ | ------ | ------ | ------ | ------ |\n| Real Entropy   | 0.2173 | 0.1464 | 0.1142 | 0.1132 | 0.1130 |\n\n It can be observed that when the slice count exceeds 10,000, the results are already quite close to the true value. This demonstrates the rationality of using fragmented approximation.\n\n\n\n> W5 The evaluation metric is overly dependent on R²; additional metrics such as MAE/RMSE or downstream utility of predicted parameters would strengthen the empirical support.\n\n Thank you for your valuable  suggestion. We have supplemented the paper with all the compared MAE/RMSE values, and we still outperform the remaining Scaling Law analyses.\n\n\n\n|            |      | MAE (HR@10) (x1e-3) |             |                       | MAE (NDCG@10) (x1e-3) |             |                       |\n| ---------- | ---- | -------------------- | ----------- | --------------------- | ---------------------- | ----------- | --------------------- |\n| Dataset    |      | Performance Law      | Scaling Law | Precision Scaling Law | Performance Law        | Scaling Law | Precision Scaling Law |\n| KR-Pure    | 25   | **2.111** | 2.793 | 2.800 | **0.768** | 1.655 | 1.658 |\n|            | 50   | **1.027** | 3.269 | 3.273 | **0.619** | 1.921 | 1.922 |\n|            | 100  | **1.871** | 2.587 | 2.588 | **0.958** | 1.458 | 1.459 |\n| ML-1M      | 100  | **2.990** | 6.064 | 6.086 | **1.928** | 3.927 | 3.934 |\n|            | 150  | **2.013** | 2.716 | 2.725 | **2.013** | 2.716 | 2.725 |\n|            | 200  | **1.645** | 4.844 | 4.846 | **1.645** | 4.844 | 4.846 |\n| AMZ-Books  | 25   | **2.704** | 3.495 | 3.462 | **1.633** | 2.271 | 2.252 |\n|            | 50   | **2.719** | 2.725 | 2.720 | **1.662** | 1.715 | 1.712 |\n| Industrial | 25   | **1.112** | 2.291 | 2.302 | **1.031** | 1.864 | 1.870 |\n|            | 50   | **0.972** | 3.356 | 3.356 | **0.830** | 2.700 | 2.700 |\n\n\n\n|            |      | RMSE (HR@10) (x1e-3) |             |                       | RMSE (NDCG@10) (x1e-3) |             |                       |\n| ---------- | ---- | -------------------- | ----------- | --------------------- | ---------------------- | ----------- | --------------------- |\n| Dataset    |      | Performance Law      | Scaling Law | Precision Scaling Law | Performance Law        | Scaling Law | Precision Scaling Law |\n| KR-Pure    | 25   | **2.880**            | 3.610       | 3.615                 | **0.923**              | 1.943       | 1.946                 |\n|            | 50   | **1.260**            | 3.754       | 3.758                 | **0.806**              | 2.170       | 2.171                 |\n|            | 100  | **2.534**            | 3.302       | 3.303                 | **1.147**              | 1.827       | 1.827                 |\n| ML-1M      | 100  | **3.417**            | 7.332       | 7.351                 | **2.525**              | 4.425       | 4.431                 |\n|            | 150  | **2.657**            | 3.948       | 3.957                 | **2.657**              | 3.948       | 3.957                 |\n|            | 200  | **1.912**            | 5.493       | 5.494                 | **1.912**              | 5.493       | 5.494                 |\n| AMZ-Books  | 25   | **3.535**            | 4.084       | 4.058                 | **2.148**              | 2.626       | 2.612                 |\n|            | 50   | **3.527**            | 3.610       | 3.609                 | **2.117**              | 2.173       | 2.171                 |\n| Industrial | 25   | **1.294**            | 2.676       | 2.693                 | **1.186**              | 2.223       | 2.232                 |\n|            | 50   | **1.341**            | 3.857       | 3.865                 | **1.140**              | 3.108       | 3.111                 |"}, {"review_id": "ntPzS2q3tn", "reviewer": "Reviewer_UaaA", "summary": "The paper introduces Performance Law (PL), a novel theoretical framework designed to address the performance prediction challenge in Sequential Recommendation Systems. PL integrates real entropy-based data quality measurement and decay-aware performance modeling to improve the optimization of sequential recommendation models. By leveraging LZ compression for entropy estimation and mathematical modeling of performance decay, the framework provides a principled approach for predicting model performance under different parameter configurations while avoiding the limitations of traditional scaling laws that rely solely on token counts or loss reduction. The proposed method demonstrates particular effectiveness in small-data scenarios and industrial-scale applications through rigorous experimentation.", "questions": "1. The experiments only include a comparison between SL and PcSL methods. It remains unclear why recommendation algorithms based on SL were not included in the evaluation. Could the authors clarify the distinctions between the proposed PL approach and other SL-based recommendation algorithms, such as Wukong? A detailed comparison would help elucidate the unique contributions and advantages of PL.\n2. The paper demonstrates the effectiveness of Real Entropy for filtering redundant sequences, but does not compare it with other information-theoretic measures (e.g., Shannon entropy or Kolmogorov complexity) that could similarly quantify sequence diversity. Why were these alternatives not explored? Additionally, how does the choice of LZ compression—instead of other complexity estimators—affect the robustness of Real Entropy in noisy or non-stationary user behavior scenarios?\n3. While the Performance Law shows empirical results on datasets like MovieLens and Industrial, its reliance on Real Entropy calculation via LZ compression may introduce computational bottlenecks. For real-time industrial systems processing billions of sequences, what optimizations (e.g., approximate algorithms or parallelization) are proposed to ensure scalability? Furthermore, how would the method handle dynamically evolving user interactions where Real Entropy needs incremental updates?", "limitations": "YES", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "1. This method introduces Real Entropy to measure the information content of user behavior sequences, addressing the issue of redundancy.\n2. Unlike traditional scaling laws that rely on fitting loss values, this approach fits performance metrics of recommendation systems, which is more applicable and rational in industrial scenarios.\n3. Experimental results demonstrate performance improvements, particularly on small datasets.\n\nWeaknesses:\n1. Although an approximate LZ compression algorithm is proposed, the computational cost may still be high in large-scale online settings, which is unfavorable for industrial applications.\n2. The proposed method involves a substantial number of hyperparameters, including those related to entropy estimation, model architecture, and performance metric fitting. The sensitivity of the method to these hyperparameters can pose challenges in practical deployment. Moreover, the optimal configuration is likely to vary significantly across different domains, datasets, and system constraints, necessitating extensive tuning efforts that may hinder scalability and generalizability.\n3. The experimental evaluation in this study lacks testing in several important special scenarios, such as graph-based recommendation and long-tail user segments. These scenarios often present unique challenges and are highly relevant in real-world applications. The absence of such evaluations makes it difficult to ascertain the effectiveness and generalizability of the proposed method in these complex or less typical settings.\n4. The experiments only compare the proposed method with general scaling law approaches, without comparing it to recommendation algorithms based on scaling laws. As such, the superiority of this method in recommendation systems remains unsubstantiated.", "was_revised": true, "final_justification": "Since most of the concerns have been discussed, I will improve the rating to 4 in the final output. For example, authors provide more experiments to respond to our question, which is a good sign.", "rebuttal": "We sincerely thank you for your valuable comments. We hope the following responses can address your concerns.\n\n>  W1 Although an approximate LZ compression algorithm is proposed, the computational cost may still be high in large-scale online settings, which is unfavorable for industrial applications\n\n We thank the reviewer for pointing out the effectiveness trade-off. We have extended the fragmented averaging strategy to accelerate computations on industrial datasets by segmenting the original dataset into fragments of 100,000 sequences each, and then calculating their averages. This slicing strategy allows for full parallelization. \n\nFor the industrial dataset we used, which contains 513,878,761 sequences, the computation time was 1 hour and 32 minutes. This is relatively insignificant compared to the training time of recommendation models of a similar scale. For the 7,076,238-sequence Amazon Books dataset, the computation time was 15 minutes. (Due to parallel computation, the time does not scale strictly proportionally with the size.) \n\nAdditionally, this method is advantageous for updates, as new sequences can be simply trimmed to the same fragment size. To demonstrate the stability of fragmented averaging, we present the relationship between the computed values and fragment sizes as follows:\n\n| Segment Length | 100    | 1000   | 10000  | 100000 | All    |\n| -------------- | ------ | ------ | ------ | ------ | ------ |\n| Real Entropy   | 0.2173 | 0.1464 | 0.1142 | 0.1132 | 0.1130 |\n\n It can be observed that when the slice count exceeds 10,000, the results are already quite close to the true value.  This demonstrates the rationality of using fragmented approximation.\n\n>  W2.1 The sensitivity of the method to these hyperparameters can pose challenges in practical deployment.\n\nThank you for your attention to parameter sensitivity. For each model parameter (such as the number of layers \\( N \\) and data \\( D \\)), our approach introduces only one additional hyperparameter compared to the Scaling Law: \\( p1 \\) for the model and \\( p2 \\) for the data. We fixed these additional parameters at 1 to align with the sensitivity of the Scaling Law and conducted the following comparison: the fitting capability differs from the original Performance Law by at most 0.5%, still outperforming other fitting approaches.\n\n| Dataset    | $\\|S_u\\|$ | Performance Law($R^2$) | Scaling Law($R^2$) | Precision Scaling Law($R^2$) |\n| ---------- | --------- | ---- | :-- | - |\n| KR-Pure    | 25        | **0.791**    | 0.672    | 0.672  |\n|            | 50        | **0.938**      | 0.466     | 0.466      |\n|            | 100       | **0.648**     | 0.172        | 0.182  |\n| ML-1M      | 100       | **0.898**     | 0.551              | 0.699 |\n|            | 150       | **0.916**  | 0.765              | 0.757   |\n|            | 200       | **0.890**     | 0.190              | 0.200 |\n| AMZ-Books  | 25        | **0.892**   | 0.803              | 0.837    |\n|            | 50        | **0.851**              | 0.751              | 0.830  |\n| Industrial | 25        | **0.939**              | 0.855              | 0.857  |\n|            | 50        | **0.948**  | 0.803        | 0.767    |\n\n> W2.2 Moreover, the optimal configuration is likely to vary significantly across different domains\n\n The Scaling Law also varies significantly across different domains; however, it has found widespread application. Our shared aim with the Scaling Law is to predict model performance in each domain using the Performance Law fitted from small-scale experiments. The key difference between the Performance Law and the Scaling Law is that, rather than relying on the loss, the Performance Law directly fits the model's effectiveness.\n\n> W3 The experimental evaluation in this study lacks testing in several important special scenarios, such as graph-based recommendation and long-tail user segments.\n\n Thank you for your suggestion on validation towards important special scenarios. We have included validation under LightGCN, as well as verification using the Amazon Beauty data, which is widely utilized in enhancing long-tail recommendations \\[1\\]\\[2\\]. Performance Law still outperforms other fitting approaches in these scenarios.\n\n\\[1\\]Liu Q, Wu X, Wang Y, et al. Llm-esr: Large language models enhancement for long-tailed sequential recommendation[J]. Advances in Neural Information Processing Systems, 2024, 37: 26701-26727.\n\n\\[2\\]Yang, K., Zheng, S., Li, T., Li, X., & Li, H. (2025). GENPLUGIN: A Plug-and-Play Framework for Long-Tail Generative Recommendation with Exposure Bias Mitigation. \n\n| LightGCN                  |                       |$R^2\\uparrow$ | MAE (x1e-3)$\\downarrow$ | RMSE (x1e-3)$\\downarrow$ |\n| ----- | ---- | --- | -- | --- |\n| [HR@10](mailto:HR@10)     | Performance Law       | **0.879**     | **4.665**                | **5.806**                 |\n|  | Scaling Law           | 0.825         | 5.456                    | 6.978                     |\n| | Precision Scaling Law | 0.866         | 4.918                    | 6.116                     |\n| [NDCG@10](mailto:NDCG@10) | Performance Law       | **0.850**     | **5.423**                | **6.744**                 |\n|| Scaling Law           | 0.800         | 6.120                    | 7.802                     |\n|  | Precision Scaling Law | 0.806         | 5.797                    | 7.223                     |\n\n\n| long-tail recommendation  |                       | $R^2\\uparrow$ | MAE (x1e-3)$\\downarrow$ | RMSE (x1e-3)$\\downarrow$ |\n| ---- | - | -------- | -- | ----- |\n| [HR@10](mailto:HR@10)     | Performance Law       | **0.868**         | **1.857**   | **2.271**    |\n|   | Scaling Law           | 0.335         | 4.265                    | 5.092                     |\n| | Precision Scaling Law | 0.334         | 4.267                    | 5.094                     |\n| [NDCG@10](mailto:NDCG@10) | Performance Law       | **0.890**         | **1.329**                    | **1.668**   |\n|                           | Scaling Law           | 0.513         | 2.956                    | 3.509                     |\n|                           | Precision Scaling Law | 0.513         | 2.957                    | 3.510                     |\n\n\n\n> W4 The experiments only compare the proposed method with general scaling law approaches, without comparing it to recommendation algorithms based on scaling laws.\n\nAnswer: Thank you for your reminder! Please see Q1.\n\n\n\n> Q1 It remains unclear why recommendation algorithms based on SL were not included in the evaluation. Could the authors clarify the distinctions between the proposed PL approach and other SL-based recommendation algorithms, such as Wukong? \n\nThe primary focus of Performance Law and Scaling Law is to provide theoretical proofs for scalability, setting them apart from frameworks like Wukong and HSTU, which primarily aim to enhance framework effect and stability.\n\nOur main goal is to improve the predictive capabilities of frameworks under varying parameters. Both Performance Law and Scaling Law can demonstrate their effectiveness on different frameworks. This is evidenced in the Wukong table below, which includes examples such as Wukong and LightGCN in W3, as well as the transformer from our original paper. Notably, on the Wukong framework, Performance Law still outperforms both variations of Scaling Laws.\n| Wukong |                       | $R^2\\uparrow$ | MAE (x1e-3)$\\downarrow$ | RMSE (x1e-3)$\\downarrow$ |\n| ------ | -----| ------------- | ---- | -- |\n| AUC    | Performance Law       | **0.528**         | **0.624**                    | **0.868**                     |\n|        | Scaling Law           | 0.479         | 0.654                    | 0.912                     |\n|        | Precision Scaling Law | 0.503         | 0.636                    | 0.891                     |\n\n\n\n> Q2.1 Why does not compare it with other information-theoretic measures (e.g., Shannon entropy or Kolmogorov complexity)\n\nApologies for the misunderstanding. This is because Shannon entropy and Kolmogorov complexity are used to compute the entropy of a set, not a sequence. To demonstrate the robustness of our model, we conducted additional experiments replacing Real Entropy with Approximate Entropy [3], Shannon Entropy, and Kolmogorov complexity. Real Entropy maintained optimal fitting capability.\n\n|      | Real Entropy | Token Only | Approximate Entropy | Shannon Entropy | Kolmogorov Complexity |\n| ---- | ------- | ---------- | -- | ---- | ------ |\n| NDCG   | **0.9906**       | 0.8160      | 0.9825  | 0.7982          | 0.8955                |\n| HR   | **0.9913**      | 0.8259     | 0.9774 | 0.7998          | 0.8663                |\n| Loss    | **0.9881**     | 0.8776     | 0.9471   | 0.8617          | 0.9374                |\n\n\\[3\\]S.M. Pincus, Approximate entropy as a measure of system complexity., Proc. Natl. Acad. Sci. U.S.A. 88 (6) 2297-2301, PNAS 1991.\n\n>Q2.2 Additionally, how does the choice of LZ compression—instead of other complexity estimators—affect the robustness of Real Entropy in noisy or non-stationary user behavior scenarios?\n\nThank you for pointing out any confusion regarding the concept of approximation. LZ compression is not an approximation; it is equivalent to the original computation. Therefore, it has no impact on robustness, noise, or non-stationary user behavior scenarios. We discussed our extension on its approximation towards computation cost in W1.\n\n> Q3.1 LZ compression may introduce computational bottlenecks. \n\nOur method's computational complexity is entirely adequate for adoption in industrial scenarios, as referenced in W1.\n\n> Q3.2 Furthermore, how would the method handle dynamically evolving user interactions where Real Entropy needs incremental updates?\n\n As stated at the end of W1, this method is also quite advantageous for updates, as new sequences only need to be trimmed to the same fragment size, or the user's corresponding fragment can be replaced."}]

Author rebuttal:

Meta-review: This paper introduces the Performance Law for sequential recommendation algorithms, enabling prediction of model performance for various settings. It also provides a quantitative framework for guiding the parameter optimization of future models. Extensive experiments show that the proposed approach has superior predictive accuracy compared to the tested baselines. It was found useful for optimization tasks as well.

The reviewers appreciated the use of Performance Law tailored to recommendation systems. While the experiments provided in the initial submission was found as lacking sufficient generality, the additional experiments along with other revisions provided in the rebuttal and discussions alleviated the concerns of the reviewers. Therefore, I feel that the paper can be a useful contribution at NeurIPS, and I encourage the authors to include the suggested revisions.
Decision: Accept (poster)