Enhancing User Intent Capture in Session-Based Recommendation with Attribute Patterns
Authors: Xin Liu
Zheng Li
Yifan Gao
Jingfeng Yang
Tianyu Cao
Zhengyang Wang
Bing Yin
Yangqiu Song
Venue: NeurIPS 2023 poster (2023)
Keywords: session-based recommendation
representation learning
pattern mining

Abstract: The goal of session-based recommendation in E-commerce is to predict the next item that an anonymous user will purchase based on the browsing and purchase history. However, constructing global or local transition graphs to supplement session data can lead to noisy correlations and user intent vanishing. In this work, we propose the Frequent Attribute Pattern Augmented Transformer (FAPAT) that characterizes user intents by building attribute transition graphs and matching attribute patterns. Specifically, the frequent and compact attribute patterns are served as memory to augment session representations, followed by a gate and a transformer block to fuse the whole session information. Through extensive experiments on two public benchmarks and 100 million industrial data in three domains, we demonstrate that FAPAT consistently outperforms state-of-the-art methods by an average of 4.5% across various evaluation metrics (Hits, NDCG, MRR). Besides evaluating the next-item prediction, we estimate the models' capabilities to capture user intents via predicting items' attributes and period-item recommendations.

Enhancing User Intent Capture in Session-Based Recommendation with Attribute Patterns
Xin Liu
Zheng Li Yifan Gao?
Jingfeng Yang²
Tianyu Cao²
Zhengyang Wang' Bing Yin Yangqiu Song'
Department of Computer Science and Engineering, HKUST "Amazon.com Inc {xliucr, ygsonyysonocooooo. ust. (amzzhe yifangao, jingfe, caoty, zhengywa, allebbii)@aaaaooeee com
Abstract
The goal of session-based recommendation in E-commerce is to predict the next item that an anonymous user will purchase based on the browsing and purchase history. However, constructing global or local transition graphs to supplement session data can lead to noisy correlations and user intent vanishing. In this work, propose the Frequent Attribute Pattern Augmented Transformer (FAPAT) that characterizes user intents by building attribute transition graphs and matching attribute patterns. Specifically, the frequent and compact attribute patterns are served as memory to augment session representations, followed by a gate and a transformer block fuse the whole session information. Through extensive experi- ments on two public benchmarks and 100 million industrial data in three domains, demonstrate that FAPAT consistently outperforms state-of-the-ard methods by an average 4.5% across various evaluation metrics (Hits, NDCG, MRR). Besides evaluating the next-item prediction, estimate the models' capabilities capture user intents via predicting items' attributes and period-ine rrriod-item recommendations.
Introduction
With the explosive demand for E-commerce services 42 23
43), numerous user behaviors are emerging. Understanding these historical action records is critical in comprehending users' interests and intent evolution, particularly in a cold-start regime that lacks sufficient context. This has spurred that capture user-side dynamics from a short-time period (namely session) using temporally historical information. Numerous SBR algorithms have been proposed, ranging from sequence-based methods [27] 19 31 28
research on session-based recommendations (SBR) g
35
37 1
g to 37 30 for learning dynamic user characterization. However, both lines of methods have their limitations. Specifically, sequence-based methods treat each user behaviors in a session as an action sequence and model the local dependencies inside. This only capture users' preference evolution via chronological order while failing to identify the complex non-adjacent item correlation, especially when the session length is insufficient to support temporal prediction 33). To address this issue, graph-based based methods adopt higher perspective by introducing global item transition graph, which aggregates local session graphs constructed from historical session sequences. Thus, newly-emeriing short session sequence benefit from the global topology (e.g., global co-occurrences) and representations (e.g.. item semantics) Unfortunately, existing session graph construction ignores temporal signals. Figure shows that two different sessions result in the same session graph, leading to vanishing of user intent variation during sequence-to-grape conversion. And such global graphs are fragile due noise from random clicks.
graph-base based methods [35 38
34
"Work was done during Xin's intershiip at Amazon. Corresponding author: Liu and Zheng Prof. Yangqiu Song is visiting academic scholar at Amazon.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

C
Gropts
hype
phace plase
ouegundy
Seskon gaap
Amtiaate Sequenees
Tiaxdne
Soulce
Timebie
(Iasgaral (nerraaii
Seedoon Sequeaen
Multiglee Seekin Graph
Figure Local session
(Theaporal
graph construction.
Figure 2: A session graph enriched by multiplex attribute graphs
Besides the global item transition graph, there are other possible solutions for item correlations. One feasible solution involves item-side knowledge. Items can be connected through shared attributes, such being manufactured by the same company (28)- We argue that the current use of item-side metadata provides little assistance in SBR models as user intent may change over time. However, such meta-data are still useful from the view of graphs. As illustrated in Figure 2 session attributes can be organized into attribute graphs and anchored to the local session graph to create multiplexes. Such multiplexes provide several intent clues, in addition item-side correlations. For example, the color pattern silver ++ silver ++ blue <+ blue reveals the user's color intent, while the brand pattern Apple ++ Apple ++ Samsung implies a potential change in intent. Different sessions can benefit from shared attribute topologies and representations, which can ultimately entail implicit high-order correlations. However, the session graph is essentially a general conditional random field, and optimization becomes intractable due to the large candidate size. Graph neural networks (GNNs)
also face challenges due to the possibility of over-smoothing and data noise 46
18
45
32
To alleviate the aforementioned issues, we propose a novel framework called Frequent Attribute Pattern Augmented Transformer (FAPAT) that considers highly frequent attribute patterss supple- mentary instead of directly learning multiplex graphs. In traditional graph learning, graphlets (such as triangles, triangular pyramids, etc.) have been proven to be useful features in graph classification and representation learning [29] 15]. Therefore, we employ frequent graph pattern mining algorithms to find consequential graphlets and view them as compact hyper-edges (c.g.. cell phone, tablet, notebook in Figure 2). We then use these attribute patterns as accessible memory to augment session sequence encoding. Before encoding patterns, we use Jaccard similarities to rank and retrieve the most highly correlated patterns, which significantly reduces the graph density and computational cost. It has been shown that GNNs can estimate the isomorphism and frequency of substructures 39 21 20 Thus, we leverage multi- head graph attention to learn pattern and local session graph representations in the aligned space (TO 22 To incorporate temporal signals and capture user intents, we distribute graph representations back to session sequences and use external pattern memory to augment sequence representations via memory attention with relative position bias. Finally, the sequence is fully fused by transformer block. In other words, graph information is used to aggregate attribute patterns, while temporal actions are used to encode items.
To validate the effectiveness of FAPAT, we conduct extensive experiments on two public benchmark datasets and three real-world large-scale industrial datasets with around 100 million clicks, and experimental results demonstrate significant improvement with an average boost of 4.5% across various evaluation metrics (Hits, NDCG, MRR). Compared with baselines, the attribute pattern density can significantly relieve over-smoothing. Besides, we also extend evaluation to attribute estimations and sequential recommendations to measure the model capability to capture user intents. Code and data are availiable at httpps:: // thub. com BKKKKKTTTKKKKCO//AAAAAAAA
2 Related Work
Neural Methods for Session-based Recommendation Systems. Table I presents a summary of the distinctions between current neural techniques and our novel FAPAT. The concept of Temporal Information implies Markov decision processes with previous histories. The technique History Attention employs attention for learning long-distance sequences. The approach of Local Session

---

Table Comparison with existing popular methods.
Methods Temporal Information History Attention Local Session Topology GGlohal Ttem Correlation Attribute Association
x Topology involves modeling session sequences from the view of session graphs. Lastly, Global Item Correlation and Attribute Association place emphasis on capturing item-side and attribute knowledge.
FPMC
GRUARec
NARM
STAMP
CSRM
S3-Roc
GNN
K
GC-SAN
X
S2-DHCN
GCE-GNN
x
MSGIESR
FAPAT
first-order Markov chain and matrix factorization identify sequential patterns of long-term dependencies. However, the Markov-based method usually has difficulty exploring complicated temporal patterns beyond first-order relationships. Recently, neural networks have shown power in exploiting sequential data in SBR tasks, such as GRU4Rec [8] extends GRUs with attention to emphasize the user's primary purchase purpose. Similarly, STAMP 19 uses an attention-based memory network to capture the user's current interest. These attention-based models separately deal with the user's last behaviors and the whole session history to detect the general and latest interests. But they mainly focus on the user's preference from temporal view but ignore the item correlations. Pre-trate vre-training techniques 47 and multi-tase learning [28] also demonstrate effectiveness in injecting item metadata to embeddings and predicting item attributes. Besides, some recent sequence- based approaches leverage generative pretrained language models to provide explicit explanations for recommendation systems 4
Sequence-bused Models. FPMC 27
NARM [6]
Graph-based Models. Graph neural networks (GNNs) have recently been explored in SBRs due to the substantial implications behind natural transition topologies. SR-GNN (35) adopts gate GNN to obtain item embeddings over the local session graph and predict the next item with weighted sum pooling, showing impressive results benchmark data. Some advanced variants have further boosted
performance, such as GC-SAN (38) with self-attention mechanism and FGNN 224 with weighted attention graph layers. To acquire further collaborative information, S2-DHCN graphs to capture correlations among neighbor sessions, and GCE-GNN B4) directly applies graph convolution over the global transitions aggregate more relevant items for local sessions. However, GNN-based methods still face challenges in capturing temporal signals, filtering noise, and leveraging implicit high-order collaborative information. Other methods, such LESSSR [ and MSGIFSR (7]. have also shown significant improvement by building multigraphs and shortcut graphs for session representation learning and user intents from different granularities, respectively.
31 constructs line
Pattern Mining for Recommendation Systems. Pattern mining is an important data mining technique with board applications. In recommendation systems, sequential pattern mining assists in analyzing customer purchase behaviors through frequent sequential patterns. Such mining focuses on item patterns with frequencies above threshold in all sessions, which reduces the diversity of the recommended items (2). Personalized sequential pattern mining 42 effectively learns user-specific sequence importance knowledge and improves the accuracy of recommendations for target users. be challenging to generalize to SBR systems when neural networks have already implicitly captured such behavior patterns. But attribute graph patterns still need to be explored in SBR.
3 Background and Motivations
3.1 Problem Definition
Gession-Based Recommendation (SBR) assumes that users" historical behaviors outside the current session are inaccessible, in contrast to general recommendations, For example, users not log in due to user privacy and security reasons. SBR predicts the next item that anonymous user is most

---

likely to click on or purchase based on historical behaviors within a short period. Despite the lack of personal profiles, this universal setting can better reflect the quality of item-side recommendations. Suppose that there are N unique items in the database, and each session is represented as a repeatable sequence of items = [01, v2,,... V. 1 E V (1 I i S L) represents the i-th behavioral item of the anonymous user within session S, where V is the item set collected from overall sessions, and L is the length of the session. Given a session S, the goal is to recommend the top-K items
(1 S K < N) that have the highest probabilities of being clicked by the anonymous user.
3.2 Session Graphs and Transition Graphs
Session sequence modeling is not always sufficient for SBR as it only reflects transitions from the user side. To account for item correlations, SR-GNN (35] converts session sequences to session graphs. Each session graph G5 = (Vs, Eg) is directed graph with node set V. C V, consisting of unique items in the session, and edge set Es. recording adjacent relations between two items in session S. Edge weights can be normalized by indegrees or outdegrees to model transition probabilities. GCE-GNN 34 extends this graph modeling by merging all session graphs as a global transition graph, which aggregates more item correlations but faces over-smoothing and data noise [8).
Instead of modeling global item transitions, we enrich session graphs with attributes and patterns. Assume there are M different kinds of attributes, and the m-th attribute type A(m)
(1 S m S M) consists of [mA possible values. Thus, each item v E Vs has attribute list [ag Qn
(1)
(2) a. [M)]
where de (mm) E A(m) denotes the m-th attribute value of v. Each session sequence S corresponds to dez (mm)
M attribute histories, with the m-th attribute sequence denoted as S(m) [o (o ai
Using the sequence-to-grapp transform, we convert ((m) to G whi is usually denser than the session graph Gs. Finally, the M attribute sequences are separately transformed into M attribute session graphs in different property-specific channels, anchored in items Vs. Finally, we add edges
for item 2 and attribute deg to construct multiplex session graph, preserving attribute values and transitions. We represent the session graph with attributes as the multiplex Gg (as Figure
3.3 Frequent Pattern Mining
Frequent pattern mining aims to extract inductive clues from data to comprehend data distributions, which includes two main categories: sequential pattern mining and graph pattern mining. The former is concerned with sequence databases composed of ordered elements, while the latter statistics the important graph structures. For two sequences S' = [v' 1, 2..... [ and S [U1, u2 ... Un]. we refer to S' as the pattern of S if S' is a subsequence of S. Similarly, graph Gs' (Vs., Es- subgraph of Gs (Vs, Eş) if Vsa S Vs and Es. C Es. Compared with sequence pattern mining, graph pattern mining is more general since it involves the structural topology and attribute information. For instance, as depicted in Figure 2 [cell phone, tablet, notebook) corresponds to a triangle. It may be challenging to discover the triangle from the temporal sequence, but it is a vital clue from the graph view. Thus, we stick on frequent graph patterns rather than sequence patterns.
Methodology
We present the Frequent Attribute Pattern Augmented Transformer (FAPAT), a novel framework that captures user intents and item correlations. Our method is built upon session sequences and corresponding attribute graphs. Initially, we mine frequent attribute patterns from the attribute graphs to explore coarse-grained item correlations. These patterns are then used as memory to enhance the session encoder, which consists of graph-nested transformer layers. Figure 3 illlustrates the overview.
4.1 Frequent Attribute Pattern Acquisition
In this subsection, we describe how to extract frequent patterns from training recommendation sessions. The aim of frequent pattern mining is to minimize the impact of random clicks in the global transition graph and avoid over-smoothing when learning multiplex attribute graphs. To achieve this, we design mining-filtering
filtering paradigm to ensure representativeness.

---

Training Aurbuuic
Granhs
Pattem Structures
PF & Add & Norm
Nome
Mall Hoad Sell Attertoon
Mining
Gate
(n
Menory Similin
Pihertog
[
10
Mulli Bostin
GraplaSecueee
Malle Grach À Herbon
210
Attribute Multiplex G5
Session Sequeace
Figure 3: An overview of FAPAT.
4.1.1 Graph Pattern Mining
Small graph patterns, also called motifs graphlets, are valuable features in graph learning and property prediction [29][5] exhibiting strong statistical correlations between graph structures and node semanties. In this study, we focus similar graphlet structures but extend to SBR scenarios. We collect patterns consisting of no more than four nodes (representing different attribute values), and further restrict them those containing either a circle triangle significantly reduce the number of candidates. We adopt gSpan [4 to acquire undirected patterns from attribute session graphs and keep patterns belonging to of twenty types shown in Figure
4.1.2 Loose Pattern Filtering
However, complex patterns may contain smaller ones. For instance, the first pattern in Figure 3 is subgraph the second. While each subgraph has an equal higher frequency than its supergraph. such loose patterns do not convey much information. To eliminate them, we employ VF2 (3 subgraph isomorphism algorithm to filter. pattern P' is subgraph of another pattern P, then p excluded from the pattern candidates. Finally, we retain compact and frequent patterns for session encoding.
4.2 Intent-aware Sequence Encoding
To recommend items of high interest to users, we use a GAT-based encoder to learn pattern represen- tations from the item side, which are then served memory to augment session encoding.
4.2.1 Relevant Graph Pattern Retrieval
The input item sequence is converted to a multiplex session graph representing the transitional information of different item attributes, as depicted Figure To improve graph representations by utilizing frequent attribute subgraphs, we retrieve relevant patterns from those mined in $4.1 For the mith attribute type ((m) (1 S m & M), we denote an arbitrary subgraph mined from the previous step as G(mm) (m)))) EP e(m)). Then retrieve at most I subgraph patterns that have the
most considerable Jaccard similarities to the transition graph g(m) as Eq. This can be done within
(()(()) x ( |), but this always linear because of (>(m) <
(mm)
(mm)
Jaccard (V(m) >((²))
(1)
(m))
1 m) - 1 (m)
0 (m))
+
5

---

4.2.2 Attribute Pattern Representation
After retrieving the relevant patterns for the corresponding multiplex session graph, we encode them using multi-head relational graph attention for further memory augmentation. For a pattern 9/x) for the m-th attribute, we compute the attention weight for two arbitrary nodes amm) and (m) by:
Leaky (m)
(m) o ee))))))
(m) softmax oij
(2)
Lea (m) (e oe (mm))))
(m)) "N(,"")
where ed(m) denotes the embedding of a." ( o indicates element- wise multiplication, .V(a.TT))
represents the neighbors of a: ((")) (including itself), and r.) corresponds to the relation-specific
vector. Then the representation of h (m) for node a; a(m) in the pattern aggregated by:
Σ
(m)
(m).
(nd) (a( (
After computing the representation of each node in pattern subgraph g/m²) pooling layer (c.g, average pooling) over all node representations is to aggregate the pattern presentation for g(m)
p (m) Pool (h cm oo E g
4.2.3 Attribute Memory Augmentation
Meanwhile, we also employ graph attention to obtain the native graph representations of the multiplex session graph ( We apply this each mith attribute transition graph _ computing node representations a (m) using Eq. (2) and Eq. (3) and denote the resulting node representation F:(m) To compute the aggregated attribute representation of each node o; in the original session graph Gs. we combine its different representations from all attribute transition graphs by:
(5)
M
where Cx2 is the representation of by the item embedding lookup.
Suppose all retrieved attribute patterns associated with (m) are p(m),, whose representations from }. To preserve temporal information when utilizing these patterns, we map graph representations (M (Tr"")))s |v; E Vs} to sequence representations (T( S i < L.) with length L. Next, we concatenate all pattern representations {p!m) E p(m)), all session
Eq. 4 are {p(m) (m(m) E p(m)).
representations (F( |1 < i L). and two special embeddings ects and eMasK to separate the two parts and indicate the item predict:
p (m))
|asL|
(m) h
|leMAKK
pmm) pp(m)
ISISL
Then, augment sequence encoding with memory, taking inspiration from TransformerXL. 5 and Memorizing Transformer [36]. As shown in Figure we introduce T5 relative position bias (25 to distinct short and long-term histories. Furthermore, on the user's current intent instead of global intent, which may become increasingly noisy over time.
utilize unidirectional attention focus solely
4.2.4 User Intent Aggregation
To aggregate memory- -ugmentd representations from M different attributes and pattern sets, employ gating mechanism:
M
h
()) h (m)
+ TT,,, mm = softmax W (m (m)
h + b (m)
(6)

---

Figure 4: Schema of pattern augmented attention: relative position bias is added, and one item can only access memory and previous histories.
ip (pom) gommg
eas +t., (8) + 9-1l < mm
where ((() Rdxd and ((m)) E Rd are trainable parameters (d is the dimension of hidden states), h ((m) is the
results, and s(m) controls the importance of attribute patterns. While h, combines attribute information, it disregards long-range histories as Fo! solely accesses one-hop neighbors. Thus, we incorporate transformer block to aggregate global session information:
hi [ ))
= Transformer(
hea.s]
IsiSL
where fuCLS and MASK directly taken from Eq. (6) since they do not belong to the original session but still observe the memory patterns and relative temporal signals.
4.3 Next-item Recommendation
Once the sequence representations are obtained, the next step is to predict the next item that may interest the user for clicking or purchasing. We adopt the approach used in previous work [3] 34 37 to concatenate additional reversed positional embeddings as follows:
z tanh [= [h, tt-+41] + b.).0 Sis
(8)
where h; is the i-th item representation H [i] from Eq. 7 and W₂ E R2dxd and by E RR are trainable parameters. This concatenation of positional embeddings is intended prioritize nearest intents over long-distance historical purposes. 2 demonstrates this idea through two cases: the male user is more likely to be interested in Android cell phones since his last click was in that category, while the female user sticks to Apple products as she reviews iPhone and iPad once again.
We utilize the representation of the MASK to compute soft attention and then represent the session and the user's latest intent through a weighted average:
u Σ hh s.t. 7:- =r Sigmoid(W, 22 + Zmask + by);
where W,, E RRd and by E Rdd are trainable parameters. We compute the prediction of the next item using ummilarity-based approach rather than linear layer. This approach is similar to the optimization of Bayesian Personalized Ranking (BPR) [26). Alternatively, we can train the model using cross-entropy minimization. Finally, the prediction is obtained by ğ argmax, e.).
5
Experiment
5.1 Setup
Datasets. We first evaluate our method on public benchmarks. diginetica contains the browser logs and anonymized transactions, Tmall collects anonymous users' shopping logs on the Tmall online website. We also acquire sessions from the browse and purchase logs from our E-commerce platform. We target at beauty, books, and electronics and gather 20-minute interactions within last successful purchases into one session after removing long-tail items. Appendix C provides more details.
Baselines. We compare method with seven sequence- based baselines (FPMC (27]. GRU4Rec (8). NARM [16]. STAMP 19) CSRM [3I] S3-Rec [47) and M2TRec (8 and six graph- based base-
lines (SR-GNN 35 GC-SAN 38 S2-DHCN 337, GCE-GNN 34 LESSR (I), and MSGIFSR (7).
Model details are given in Appendix D Each model aligned with the official code implementation.
Evaluation. We evaluate SBRs as ranking problem and employ Hits K, NDCG@ K, MRR@ as standard metrics. Hits@ K measures the percentage of ranks up to and including K, while NDCG @ K
7

---

Table 2: Performance evaluation for next-item prediction, where standard deviations are enclosed in brackets. The best and second-bes, results are respectively highlighted in bold and underlined.
Methods that use attributes are marked with $ and * indicates the p-value < 0.0001 in t-test.
diginetica
Tmail
Model
16
NDCG@11
MRR@10
Ilis@20
NDCG@20
MRR
[Eis@ 10
NDCG6 10
MRR@10 10
1l NDCG@20
MRR#20
FPMC
31.57
17,40
13.08
43.19
20.33
13,88
13.71
9.02
7.56
16.44
9.71
7.74
GRU4Rec
36.77
20.71
15.80*
49.68*
23.97
16,70*
18.82*
12.28
10.25
22.68
13.25
10.51*
NARM
35,98
20.18*
15.36*
48,89
21.44
16,26
22.74
13.19
26.73
16.47
STAMP
33.59*
18,89*
14.41*
45,87*
22.00*
15.26*
24.32*
16.55*
14.12*
28.40"
17.58*
14,41*
CSRM
33.97*
19.43*
14.98
45,83*
22.42*
15.80*
25.13*
18.56*
16.48*
27.94*
19.27*
16.68*
S3-Rec3
33,48
18.58*
14.04
45,97*
21.74
14,90"
18.24
12.30*
10.46*
22.31
13.324
10.74*
M2TRec
29.67*
16.30"
12.23
41.23*
19.22
13.02
11.42*
7.56
6.36
13.75
8.15
6.52
SR-GNN
35.21*
19.68
14.94
47,98%
22,90
15.82
18.21
10.20
21.34
12.91
10.42*
GC-SAN
35.25*
19.72
14.97*
47,87*
22.90*
15.85*
19.29
12.80*
10.78*
23.18*
13.78*
11.05*
S2-DHCN
30.76*
17.04*
12.86*
42.39*
19.98*
13.66*
22.00*
13,36*
10.68*
27.23*
14.69
11,05*
GCE OCE.GNN
36.32*
20.77*
16.02
48.67*
23.89*
16.87
28.33*
20.01*
17.32*
30,24*
20.50*
17.45*
LESSR
33.68*
18.71
14.14
46.23*
21.88*
15.01
20.99'
14.64*
12.13*
25.92
13.96
10.50*
MSGIFSR
34.74*
19,43
14.76*
46,23
21,88*
15.01
23.18*
15.19*
12.69*
27.78*
16.35*
13.01
FAPAT:
37,42
21.31
16.39
50.41
24.59
17.29
32.45
22.02
18.72
36.18
22.97
18,99
Improv
3.03%
2.60%
2.37%
1.46%
2.59%
2.49%
14.19%
10.04%
8,08%
19.64%
12.05%
8.83%
Table 3: Performance evaluation for next-item prediction on our 100 million industrial data.
Beauty
Books
Electronics
Model
Hinee 10 NDCGe11
Hisst20 NDCG#20
Hixee 10
NDCGe 10
Hinud:20 NDCG0-20 20
10
NDCG611 Hiss NDCG6800
CSRM
89,74
75.28
92.61
76.01
78.69
56.70
$2.88
57.77
62.28
44.35
67.47
45.67
S3-Roct
89.64
75.56
92.53
76.30
75.00
58.54
79.45
59.67
74.36
$6.03
79.63
$7.37
M2TRee5
80.13
65,97
83.66
66.87
32.56
22.58
35.39
25.70
57.32
44.84
61.70
45.95
SR GNN
88.69
70.42
91.74
71.20
66.55
47.55
69.77
48.37
74.86
$4.30
79.66
55.52
GCE-GNN
89.34
73.15
91.29
73.65
77.61
57.60
80.03
58.22
72.93
51.74
78.49
55,15
LESSR
89,95
71.29
92.98
72.06
73.72
53,86
82.31
54.77
72.91
50.46
78.78
51.96
MSGIFSR
90.18
73.62
V2.50
74.21
72.93
52.23
76.33
53.09
73.56
53.83
77.45
54.73
FAPAT:
92.72
76.29
94.10
76.87
81.62
61,08
85.12
61.97
78.36
56.81
82.81
57.94
Improv
2.82%
0.97%
1.20%
0.75%
3.72%
4.34%
2.70%
3.85%
4.68%
1.39%
3,95%
0.99%
assigns higher scores to hits at the top of the list. MRR @ K is the average of reciprocal ranks, with ranks above K assigned O. For public benchmarking, we report the average performance across seeds (2020, ... 2024), while for industrial data, we use a fixed seed (see Appendix EE for more details).
5.2 Next-item Prediction Evaluation
We first compare our models with selected baselines on two public datasets in Table Overall, there is huge difference between sequence and graph models when the session historical information is limited (e.g., in diginetica). However, we observe a performance boost with the history attention mechanism and session topology. FPMC that utilizes first-order Markov chains and matrix factorization is the worst. This reveals the difference between traditional recommendation and session-based recommendation. In contrast, RNN- based methods (GRU4Rec, NARM, STAMP, and CSRM) show better generalizability, along with the benefits of attention and memory. But we do not see further gains from pretraining and multi-task learning in S3-Rec and M2TRec. On the other hand, RNN-based algorithms for local sessions also achieve comparable results even with the absence of temporal signals. Moreover, the heterogeneous graphs of MSGIFSR beat the shorteut graphs from LESSR, indicating over-smoothing and potential noise. Although explicit global collaborative graphs enhance GCE-GNN, the implicit collaborative information from neurons and patterns in FAPAT easily surpasses other baselines significantly. At the same time, we evaluate algorithms on our 100 million industrial data. Table 3 demonstrates that most sequence models become unstable except S3-Rec. It employs incomplete data to predict masked items and attributes during pretraining and identifies contextualized collaborations by determining whether two incomplete sequences belong to the same session. But the multi-tasking learning for predicting attributes makes M2TRec hard to recommend items. While most GNN methods are competitive. S3-Rec and FAPAT are still better.
Experimental Results.
Over-smoothing Relief. We examine different graph topologies in baselines and FAPAT by compar- ing the graph density of different methods: the density of local session graphs (in SR-GNN), global collaborative filtering graphs(in GCE-GNN), shortcut graphs (in LESSR), heterogeneous graphs (in MSGIFSR), attribute patterns (in FAPAT) on E-commerce data are 3.658, 114.910, 25.447, 2.598, and 1.1 respectively. The density decrease significantly relieves the over-smoothing. Detailed analyses and expectations are provided in Appendix B
8

---

S38 METRR
GOR-GNN
NIGIFSR
85
SI Bue
ASCIIFRR
LEASS
PAPAI
GUE
EAPAT
$1.33
35.53
39.45
80
71-48
77.00
18.00
79.04
30
76-41
73.66
78.99
75.09
Hite
Hito@ito
75
15
25.5
31A2
X3.13
$2.14
39.40
3348
70-
30
VI.4
19.24
19.1
6199
6639
65
11.49
12.0
w/o Att.
w/ brand
w: e'tergury
Attss.
60
Atte
wi category
pablister
w: author
TV/ grare
TV Atrrt
(a) Tmall
(b) Books
Figure 5: Effects of different attribute settings.
Table 4: Results of encoder comparison, where indicates the p-value < 0.05 in I-test.
diginetica Hits@ MRR @ 10 Hits@
Tmall
Beauty
Books
Encoder
MRR @ 10
Hits@ 10
MRR Hits@
MRR
FAPAT w/o Attr.
36.82
16.29
31.53
18.86
88.70
67.11
77.18
49.66
GraphFormer
36.05*
16.17
30.05
18.58
88.48
65.65
77.03
47.86
Transformer
36.30*
16.02*
28.83*
18.30*
88.10
65.90
74.67
46.54
5.3 Ablation Study
Attribute Pattern Augmentation. To evaluate the effect of attribute patterns, we conduct exper- iments on variants with single attributes or without any attribute. The same soft attention strategy from Eq. (6) is employed to fuse attribute embeddings for competitive baselines. Results in Figure show that attribute pattern augmentation is more stable than attribute soft attention. FAPAT benefits from graph-nested attention, where the graph attention aligns the hidden space, and the memory attention captures item correlations and user intents. But attribute embeddings may have side effects optimization in baselines, especially in graph neural networks. We also discover that not all attributes have a positive impact. Comparing among attribute pattern numbers, attribute patterns with significant frequencies (slightly lower than similar to the item number) can have adverse effects.
Graph-nested Attention. The graph-nested attention is one of our contributions, distinguishing from GraphFormer [41]. Unlike GraphFormer, our graph attention is integrated inside the blocks, which allows for direct benefit from the broader attention in the following self-attention via back- propagation. To ensure fairness, replace the encoding module of FAPAT with GraphFormer and vanilla Transformer. Results in Table 4 demonstrate the advantages of our proposed graph-nested attention. Our experiments also show that state-of-the-as models by significant margin, indicating the importance emphasizing temporal information in SBRs and the appropriateness of attention for capturing long-distance dependencies.
simple vanilla Transformer
outperform previous
5.4 Intent Capture Inspection
Beyond item predictions, we also estimate the awareness of user intents from the product attribute side. We do not require models (except M2TRec) to predict attributes but to retrieve attributes from predicted items instead. We consider it successful estimation if the top-ranked items have the same attribute value the ground truth. M2TRec performs well on public data with multi-task learning but struggles on industrial E-commerce data with four attributes, as shown in T Pretraining S3-Rec is not helpful due to catastrophic forgetting. GNN-based models perform similarly, except on Tmall, where the data are too sparse so that global collaborative information assists. On contrary, FAPAT achieves robust predictions across four datasets. Even when session data sufficient, frequent patterns remain effective.
Attribute Estimation.

---

Table Attribute estimation evaluation.
diginetica
Tmall
Beauty
Books
Model
Hits@ 10
MRR @ 10
Hits@ 10
MRR@ @ 10
Hits 10
MRR@ 10
Hits@ 10
MRR @ 10
CSRM
89.87"
87.91*
48.03*
34.97*
94.82
83.57
88.93
73.31
M2TRect
94.10
89.76
59.44
39.72
95.93
84.95
83.80
71.22
S3-Rec$
89.66
88.42*
44.95*
32.60*
95.35
84.79
88.11
75.63
GCE-GNN
89.89*
87. 87.96*
55,88*
38.57*
95,44
84,16
89.78
75.68
LESSR
88.60*
86.24
49.43*
32.61*
95.02
81.56
89.57
73.73
MSGIESR
89.48*
87.19*
50.09*
33.95*
95.79
85.42
86.67
71.99
FAPAT
89.99
88.23*
59.49
40,56
95.94
86.87
90.77
76.83
w/o Attr.
89.35
87.50
58.22
40.08
95.07
81.65
89.29
74.12
35
70
70-
CSRM (Ro-012211)
25
60
60
20
40
40
30
30
10
20
-.. FAPST (NICG 211
5-
10
10
53Rec (NDCG810
-GE--GN (NBCO881))
Period Length
Period Length
Period Length
(a) Tmall
(b) Books
(c) Electronics
Figure 6: Period-item recommendation evaluation.
Period-item Recommendation. In addition to single-step evaluation, we revisit recommenders. A sound and robust SRB system must understand user intents in deep and foresee the possible consistency and the potential change. Autoregressive settings pose challenges for GNNs due to short click histories and error accumulation. Therefore, we evaluate period recommendations like search engines by comparing the top- 10 predicted items with the next 3/5/10 clicks. Figure demonstrates period-recommmendation performance. Sequence models offer steady results in Recall, indicating that temporal information one of the prerequisites to analyzing users' latest intents. However, pretraining may hinder models' ability to generalize over long periods, resulting in severe decline in NDCG. Our FAPAT achieves the best performance in all metrics, indicating the effectiveness of attribute graphlets in capturing deep user intents.
6 Conclusion
Our paper introduces FAPAT, a novel framework that leverages attribute graph patterns to augment anonymous sequence encoding for session-based recommendations. Compared to other GNN-based methods, frequent attribute graphlets can reduce noise and topology densities for enhancing user intent capture. Our sequence encoder can better preserve temporal signals and forecast the user's latest intents. Experimental results clearly illustrate the effectiveness, and extensive ablation studies and intent capture inspections provide additional support. We discuss limitations in Appendix A One of the future works is to improve ranking priority by combining pretraining and pattern augmentation.
Acknowledgments
The authors of this paper were supported by the NSFC Fund (U20B2053) from the NSFC of China, the RIF (R6020-19 and R6021-20 and the GRF (16211520 and 16205322) from RGC of Hong Kong. We also thank the support from the UGC Research Matching Grants (RMGS20EG01-D) RMGS20CRIL RMGS20CR12. RMGS20EG19, RMGS20EG21, RMGS23CR0S, RMGS23EG08).
10

---

References
[1] Tianwen Chen and Raymond Chi-Wing Wong. Handling information loss of graph neural networks for session- saason recommendation. In SIGKDD, pages 1172-1180, 2020.
[2] Yen- Yen-Liang Chen, Mi-Hao Kuo, Shin-yi Wu, and Kwei Tang. Discovering recency, frequency, and monetary (RFM) sequential patterns from customers' purchasing data. Electronic Commerce Research and Applications, 8(5):241-251, 2009.
[3] Luigi P. Cordella, Pasquale Foggia, Carlo Sansone, and Mario Vento. A (sub)graph isomorphism algorithm for matching large graphs. TPAMI,
2004.
[4] Zeyu Cui, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. M6-rec: Gener- ative pretrained language models are open-ended recommender systems. arXiv preprint arXiv:2205. 08084, 2022.
[5] Zihang Dai, Zhilin Yang, Yiming Yang. Jaime G. Carbonell, Quoc Viet Le, and Ruslan Salakhut- dinov. Transformer-xl: Attentive language models beyond a fixed-length context. In ACL, pages 2978-2988, 2019.
[6] Shijic Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. Recommendation language processing (RLP): A unified pretrain, personalized prompt & predict paradigm (P5). In RecSys, pages 299-315, 2022.
[7] Jiayan Guo. Yaming Yang. Xiangehen Song, Yuan Zhang, Yujing Wang, Jing Bai, and Yan Zhang. Learning multi gulliigranularity consecutive user intent unit for session-based recommenda- tion. In WSDM, pages 343-352, 2022.
[8] Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. In ICLR, 2016.
[9] Yupeng Hou, Binbin Hu, Zhiqiang Zhang, and Wayne Xin Zhao. CORE: simple and effective session-based recommendation within consistent representation space. In SIGIR, pages 1796- 1801. ACM, 2022.
[10] Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. Heterogeneous graph transformer. In wwW, pages 2704-2710, 2020.
[11] Jie Huang, Yifan Gao, Zheng Li, Jingfeng Yang. Yangqiu Song, Chao Zhang, Zining Zhu, Haoming Jiang, Kevin Chen-Chuse Chang, and Bing Yin. Cegen: Explainable complementary concept generation in e-commerce. arXiv preprint arXiv:2300. /480, 2023.
[12] Haoming Jiang, Tianyu Cao, Zheng Li, Chen Luo, Xianfeng Tang, Qingyu Yin, Danging Zhang, Rahul Goutam, and Bing Yin. Short text pre-training with extended token classification for e-commerce query understanding. arXiv preprint arXhv:2210003913 2022.
[13] Wei Jin, Haitao Mao, Zheng Li, Haoming Jiang, Chen Luo, Hongzhi Wen, Haoyu Han, Hanqing Lu, Zhengyang Wang. Ruirui Li, Zhen Li, Monica Xiao Cheng, Rahul Goutam, Haiyang Zhang, Karthik Subbian, Suhang Wang. Yizhou Sun, Jiliang Tang, Bing Yin, and Xianfeng Tang. Amazon-m2: A multilingual multi-locale shopping session dataset for recommendation and text generation. In NeurIPS, 2023.
[14] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR. 2015.
[15] Nils M. Kriege, Fredrik D. Johansson, and Christopher Morris. survey on graph kernels. Appl. Netw. Sci., 5(1):6, 2020.
[16] Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. Neural attentive session-based recommendation. In CIKM, pages 1419-1428, 2017.
[17] Zihan Lin, Changxin Tian, Yupeng Hou, and Wayne Xin Zhao. Improving graph collaborative filtering with neighborhood-enriele contrastive learning. In WWW, pages 2320-2329, 2022.

---

[18] Fan Liu, Zhiyong Cheng, Lei Zhu, Zan Gao, and Liqiang Nie. Interest-aware passing GCN for recommendation. WWW, pages 1296-1305, 2021.
[19] Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. STAMP: short-term attention/memorr priority model for session- based recommendation. In SIGKDD, pages 1831-1839 2018.
[20] Xin Liu and Yangqiu Song. Graph convolutional networks with dual message passing for subgraph isomorphism counting and matching. AAAI, pages 7594-7602, 2022.
[21] Xin Liu, Haojie Pan, Mutian He, Yangqiu Song, Xin Jiang, and Lifeng Shang. Neural subgraph isomorphism counting. In SIGKDD, pages 1959-1999999 1969. ACM, 2020.
[22] Xin Liu, Jiayang Cheng, Yangqiu Song, and Xin Jiang. Boosting graph structure learning with dummy nodes. In ICML, volume 162, pages 13704 13716. PMLR, 2022.
[23] Chen Luo, William Headden, Neela Avudaiappan, Haoming Jiang, Tianyu Cao, Qingyu Yin, Yifan Gao, Zheng Li, Rahul Goutam, Haiyang Zhang, and Bing Yin. Query attribute recom- mendation at amazon search. In RecSys, page 506-508 2022.
[24] Ruihong Qiu, Jingjing Li, Zi Huang. and Hongzhi Yin. Rethinking the item order in session- based recommendation with graph neural networks. In CIKM, pages 579-588, 2019.
[25] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21: 1:::::4:1114000 1-400667, 2020.
[26] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt- Thieme. BPR: bayesian personalized ranking from implicit feedback. In UAI, pages 452 0552466,1- 2009.
[27] Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme, Factorizing personalized markov chains for next-basket recommendation. In WWW, pages 81 8-1-220, 2010.
[28] Walid Shalaby, Sejoon Oh, Amir Afsharinejad, Srijan Kumar, and Xiquan Cui. M2tree: Metadata-aware multi-task transformer for large-scale and cold-start free session-based recom- mendations. In RecSys, pages 573-578, 2022.
[29] Nino Shervashidze, S. V. N. Vishwanathan, Tobias Petri, Kurt Mehlhorn, and Karsten M. Borgwardt. Efficient graphlet kernels for large graph comparison. In AISTATS, volume 5, pages 488-495, 2009.
[30] Zhongwei Wan, Xin Liu, Benyou Wang, Jiezhong Qiu, Boyu Li, Ting Guo, Guangyong Chen, and Yang Wang. Spatio-tempore contrastive learning enhanced gnns for session-based recommendation. Transactions on Information Systems, 2023.
[31] Meirui Wang, Pengjie Ren, Lei Mei, Zhumin Chen, Jun Ma, and Maarten de Rijke. A collabora- tive session- based recommendation approach with parallel memory modules. In SIGIR, pages 345-354, 2019.
[32] Ruijie Wang. Zheng Li, Dachun Sun, Shengzhong Liu, Jinning Li, Bing Yin, and Tarek Abdelzaher. Learning to sample and aggregate: Few-shot reasoning over temporal knowledge graphs. In NeurlPS, pages 16863-16876, 2022.
[33] Shoujin Wang, Qi Zhang, Liang Hu, Xiuzhen Zhang, Yan Wang. and Charu Aggarwal. Sequentiallsession--aaes recommendations: Challenges, approaches, applications and op- portunities. In SIGIR, pages 3425-3428, 2022.
[34] Ziyang Wang, Wei Wei, Gao Cong, Xiao-Li Li, Xianling Mao, and Minghui Qiu. Global context enhanced graph neural networks for session-based recommendation. In SIGIR, pages 169-178, 2020.
[35] Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. Session-based recommendation with graph neural networks. In AAAI, pages 346-353, 2019.
[36] Yuhuai Wu, Markus N. Rabe, DeLesley Hutchins, and Christian Szegedy. Memorizing trans- formers. In ICLR, 2022.
12

---

[37] Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang, Self- supervised hypergraph convolutional networks for ression-based recommendation. AAAI. pages 4503-4511 2021.
[38] Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Victor S. Sheng, Jiajie Xu, Fuzhen Zhuang, Junhua Fang. and Xiaofang Zhou. Graph contextualized self attenttio network for session- based recommendation. In IJCAI, pages 3940-3946, 2019.
[39] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.
[40] Xifeng Yan and Jiawei Han. gspan: Graph-based substructure pattern mining. In ICDM, pages 721-724, 2002.
[41] Junhan Yang, Zheng Liu, Shitao Xiao, Chaozhuo Li, Defu Lian, Sanjay Agrawal, Amit Singh, Guangzhong Sun, and Xing Xie. Graphformers: Gnn-nested transformers for representation learning on textual graph. In NeurlPS, pages
2021.
[42] Ghim-Eng Yap. Xiaoli Li, and Philip S. Yu. Effective next-items recommendation via personal- ized sequential pattern mining. In DASFAA, volume 7239, pages 48-64, 2012.
[43] Changlong Yu, Weiqi Wang, Xin Liu, Jiaxin Bai, Yangqiu Song, Zheng Li, Yifan Gao, Tianyu Cao, and Bing Yin. FolkScope: Intention knowledge graph construction for E-commerce commonsense discovery. In Finding of ACL. pages 1173-1191, 2023.
[44] Danqing Zhang, Zheng Li, Tianyu Cao, Chen Luo, Tony Wu, Hanqing Lu, Yiwei Song, Bing Yin, Tuo Zhao, and Qiang Yang. Queaco: Borrowing treasures from weakly-labeled behavior data for query attribute value extraction. CIKM, page 4362 4372, 2021.
[45] Peiyan Zhang, Jiayan Guo, Chaozhuo Li, Yueqi Xie, Jaeboum Kim, Yan Zhang, Xing Xie, Haohan Wang, and Sunghun Kim. Efficiently leveraging multi-level user intent for session-based recommendation via atten-mixer network. In WSDM, 2023.
[46] Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. In ICLR, 2020.
[47] Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In CIKM, pages 1893-1902, 2020.
13

---

St
Session Sequene
Local Session Graph Global Transition Graph
Shortut Graph
Heterogeneuus Graphs
Attribute Patterns
Figure 7: Different granularities of graphs derived from a session, where the violet edges, blue edges, and green edges correspond to the global collaborations, shorteuts, and higher-level heterogeneous connections, respectively.
A Limitations
There are three limitations to current proposed method and evaluation. First, our method separately processes and retrieves patterns for each attribute type. We do not merge all attributes in a candidate pool because aim for our method easily generalize real recommendation systems with hundreds attribute types and category hierarchies. The current implementation supports adding a new attribute type a model long as its embeddings align with the embeddings of other attributes. Second, we conducted experiments based on "clean" session data. Most E-commerce platforms do have truly clean data on product attributes, so attribute data, in general, is very sparse and full of invalid values. We performed human-centric attribute regularization to drop products without valid attribute values, which may create a gap compared to real industrial system. Third, the evaluation does not consider the same products with different identifiers. Therefore, evaluating results (especially MRR) cannot accurately reflect the performance. To better reflect the real performance with error tolerance, a larger K is suggested. The current comparison is still fair for all algorithms,, and we address this synonym problem in attribute estimation $ $5.4 where we merge attribute values based on semantics and syntax.
B Transition Graph Density
The graph structure is crucial for neural networks to capture explicit transitions and implicit connec- tions. A local session records the history of a user's clicks purchases, which is usually sparse. In contrast, the global collaborative graph could be extremely dense because each pair of items may have potential connection. From this perspective, the density indicates the explicit information provided from session data. On the other hand, different graph topologies and densities also present different focuses and challenges. The sparse local transition graph emphasizes current intents, while the global collaboration indicates broader interests and revenues. Graph neural networks excel at capturing local features, but large number of neighbors can overshadow important connections with less significant ones. Considering that previous methods have focused on different granularities individually, we summarize them Figure 7 and compare them in terms of optimization interpretations
Local session graphs correspond item transitions within session, where edges are created between two consecutively clicked/purchased items. The density is usually sparse (slightly greater than 1.0), allowing exploration and global collaboration be learned through model parame- ters instead of explicit connections. Therefore, generalizing to unseen click patterns becomes challenging.
Global transition graphs record all collaborations. In a real industrial system, the density is usually beyond one hundred or even one thousand. Ideally, any session can benefit from this global collaborative information, including multi-hop connections. However, optimizing graph neural
Graphs typically considered undirected in practical algorithms.
14

---

networks to learn such topologies (due to oversmoothing) and building a large model for real-time inference (due latency and streaming processing) pose challenges.
Shorteut graphs aim to avoid constructing global graphs and make session learning more efficient.
They proposed by LESSR ① to address information loss in graph convolutions. Specifically, they allow latent items to be aware of all previous clicks, resembling shortcuts for multi-hop neighbors in the directed local session graph. However, they lack the extensive exploration capabilities of the global transition graph and suffer from oversmoothing issues due to dense connections.
Heterogeneous graphs strike a balance between shortcuts and local adjacency. Nodes with different numbers of items are categorized into different groups, and transition edges capture varying levels of spatial continuity. From a high-level perspective, this graph is sparser than the local session graph, resulting in faster convergence for optimization. However, the propagation of high-order information introduces additional processing costs and the risk of overfitting.
Patterns, especially attribute patterns, should be the most efficient features for recommendations a large candidate item pool. Each pattern can be considered a higher-grained heterogeneous graph. However, pattern filtering significantly eliminate noise influence, not mention the benefits gained from offline indexing. Besides, the partial match patterns can provide the intent information from other sessions, making the learning and prediction more reliable and steady.
C Experimental Data
C.1 Public Benchmarks
We choose two public benchmarks for session-basse bbseed recommendation evaluation: diginetica is CIKM Cup 2016 that contains the browser logs and anonymized transactions; Tmall comes competition IJCAI-15 which collects anonymous users' shopping logs on the Tmall online website. We acquire attributes from the original data and drop items without attributes or with invalid values. Therefore, the performance of baselines may not exactly same as the reported numbers in original papers.
Tmaeeecomess from
C.2 E-commerce Data Collection
We collect E-commerce data from our log systems in two months. We follow the same procedure to clean and process session data beauty, books, and electronics domain
We focus on successful purchases so that we only keep sessions ending with "purchase" actions.
II To make sure previous clicks can reflect the purchase intent, drop actions 20 minutes ago.
III We filter out items with missing attributes (i.e., books without publishers, authors, genre, and electronics without colors and brands).
IV We adopt the 20-core setting to finalize the item sets, in which items appear on at least 20 different days.
V Only sessions whose length is no greater than 50 are preserved.
VI We retrieve item attributes in our attribute databases.
VII For GNN models that requires the global transition graph from training data, maintain 12 neighbors based on the co-occurrence, which consistent with GCE-GNN [34].
The sampled data scales and distributions different in real systems due of items filtering.
15

---

Table 6: Statistics of datasets based timestamps.
Public
Industrial (F-commerce)
diginctica
Tmall
Beauty
Books
Electronies
#User
57,623
7,576
2.6 M
3.2
10.2 M
fliem
43,074
39,768
39.2
94.8
244.7
#Click
993,163
438,3155
27.2
38.8
115.6
Len.
4.850
6,649
10.325
11,912
11.249
#Train
6300789
303,181
19.6 M
28.2 M
84.1 M
#Valid
78,708
33.735
2.4 M
10.5 M
#Test
78,907
35,481
2.5 M
3.8
10.6 M
category: 339
calegory: 18
type: 123
#Attribute
calegory: : 995
callgooy
hrand 4,304
brand 4,359
color:
publisher: 2.751
author: 27,651
callewory'
color: 2,099
size
genre 2,634
24,196
calogory: 970
eategory:
type: 9,289
calegory: 33_582
color: : 4,059
publisher 4,370
calegory 13,991
#aaliemn
callgoory: 1,866
254
author
color
size 1,091
genre: 12.535
hrand: : 14,043
Local: 0.856
Local: 1.249
Local: 4.510
Local: 3,554
Local: 2,910
Glohal: 11.329
Global: 10.222
Global: 70.504
Glohul: 99.389
Global: 128.041
Density
Shortcut: 2.512
Shestcut: 4,983
Shortcut: 29 827
Shortcut: 36.649
Shoricut: 19.865
Heterngeneuu:: Heteregenecus;
Heterngeneou::
Heterogeneous: Heterggneoox: 2.049
Pattern: 1,023
Patter:: 1,165
Patter: 1.095
Pattern: 1.085
Partem: 1.189
C.3
Data Split
We follow previous settings that split training/aliddation//ssngg data based on timestamps. For diginetica, we gather the last 8- 14 days validation, the last 7 days as testing, and remaining as training. For Tmall, we use the last 101-200 seconds as validation, the last 100 seconds as testing, and remaining as training. For our industrial E-commerce data (i.e., Beauty, Books, Electronics), select the last 6-10 days as validation, the last days as testing, and remaining as training.
C.4 Data Statistics
Table 6 summarizes the statistics of the experimental datasets based on timestamps. The density is calculated based on undirected graphs, which would be doubled during graph convolution in practice. Local densiry, used in SR-GNN and GC-SAN corresponds the average density of local session graphs in E-commerce sessions. On the other hand, global density, as used in GCE-GNN, refers to the density of the global collaborative graph obtained connecting all adjacent items appearing in all sessions. Shortcut density, as used in LESSR, is the density resulting from connecting all items in single session as a complete graph. Heterogeneous density, as used in MSGIFSR. refers to the average density of the heterogencous graphs obtained by regarding the consecutive adjacent anneggraneed intent unit. Lastly, pattern density,
two nodes of the acquired frequent and compact patterns. From Table 6 it is evident that leveraging patterns the most effective way of characterizing user intents because other graph topologies vary with data sources and scales, making it difficult to generalize and provide stable performance. Besides, patterns can be preprocessed as indicies aid recommendations, making them more practical in industrial scenarios. Moreover, it is casy to update attribute patterns dynamically, whereas other graph structures are more closely coupled with input sessions and are more sensitive to tiny variations.
used in FAPAT, is the density
D Baselines
We compare method with the following baselines:
Sequence-based methods
FPMC 27 learns the representation of session via Markov-chain based methods.
GRU4Rec is the first RNN-based approach that simulates the Markov Decision Process (MDP) but has a better generalization.
NARM [6 is a attention based RNN model to learn session embeddings.
STAMP 19 adopts attention mechanism between the last item previous histories represent users' short-term interests.
16

---

CSRM 31 proposes to engage an inner memory encoder and external memory network to capture
correlations between neighborhood sessions to enrich the collaborative representations.
S3-Rec 47 is the first pretrained SBR model that predicts items, attributes, and segments during
the pretraining stage.
M2TRec (8 is metadata-awaid multi-task Transformer model. the original paper, the authors ignore item embeddings. For a fair comparison, also regard the item ids as one of metadata.
Graph-based methods
SR-GNN 35 is the first GNN-based model for the SBR task, which transforms the session data
into a direct unweighted graph and learns the representation of the item-transitions graph.
GC-SAN 38 uses gated GNNs to extract local context information and then self-attention to obtain the global representation.
S2-DHCN [3
transforms the session data into hyper-graphs and line- tnnegraphs and encodes them via GCNs to enhance the session representations.
GCE-GNN 34 aggregates two levels of item embeddings from session graphs and global graphs with soft attention.
LESSR [ preserves the edge order and constructs shortcuts to encode sessions for GNNs.
captures the user intents from multiple granularities relieve the computational burden of long- wene-depnndenc... In experiments, we search the best model from the level-1, 1, level-2, and level-3 consecutive intent units.
MSGIFSR 7
E Experimental Settings
We fix all embeddings and hidden dimensions 100, and the batch size is searched among (100, 200, 500) for all methods. We also choose the number of layers/iterations (if applicable) from the validation performance (c.g., MRR @ 10). A learning scheduler with 10% linear warmup and 90% decay is associated with the Adam optimizer [14]. The initial learning rate is set as 1e-3, and the regularization weight is tuned among (le-4, le-5, 1e-6). We seek the dropout probability between two modules from (0.0, 0.2, 0.4). but fix the attention dropout rate as 0.2. The number of attention heads empirically set 4. We follow the setting of GCE-GNN that the maximum one-hop neighbor number in GAT is 12. In the interest of fairness, we also set the maximum selected pattern number as 12. Hyper-parameter tuning time costly on our industrial data that we use the best combinations obtained from one day transactions. We implement our methods and run experiments with Python and PyTorch over 8 A100 NVIDIA GPUs.
Experimental Results
Due to the space limit, we only report some results in the main content. More comprehensive comparisons are shown in Tables IT where standard deviations are enclosed brackets. The best and second-best results are respectively highlighted in bold and underlined. Methods that use attributes are marked with f. and indicates the p-value < 0.0001 t-test.
17

---

Table 7: Performance evaluation for next-item prediction on diginetica.
diginetica
Model
Hits 10
NDCG@ 10
MRR @ 10
Hits Hits@02
NDCG@20
MRR@20
FPMC
31.57(0.04)*
17. 17.40(0.01)*
13,08(0.02)*
43.19'0.05)*
20.33(0.03)*
13,88(0.03)*
GRU4Rec
20.71 36.77
15.8 "
49. 68(0.06)
23.97:0.03)*
16.70(0.03))
NARM
35.98(0.10 i0y*
180.80))
15.3610.06)
18.
23.44(0.06)*
16.26.0.06)
STAMP
13.
18.
(0.41
15
19. 12.
CSRM
33.97(0.08y*
19.43
14. 14.98(0.
(5.83(0.02)*
22.42(0.02)*
S3-Rect
33,48
18,58(0.09)*
14,04(0.10)*
45.97(0.08)*
21.74(0.09)*
14.90(o.10)
M2TRect
29.6700.43)*
16.30(0.24)
(2.23(0.18)*
41.23(0..3)*
19.22(0.29)*
(3.02(0.20))
SR-GNN
35.21
(9.68(0.04)*
14,94(0.04)
47.99(0.04)*
22.90(0.04)
15.82(0.04)*
GC-SAN
35.2510.09)*
19.72(0.04)*
14.97(0.03)*
47.87(0.09)*
22.90(0.04)*
(5.85(0.03)*
S2-DHIN
30.76(0.07)*
17.04(0.14)*
12.86(0.16)*
42.39((.07)*
19.98(0.13)
13.66(0.16)
GCE-GNN
36.3210.09)*
20.77(0.07)*
16,02(0.07)*
48. 48.67(..1))
23.89
(6.87(0.03)*
LESSR
33.68r0.05ye
(0.71
14.14
46.23(0.11)*
21.88(0.5))*
(5.01(0.03)*
MSGIFSR
34.74(0.09)
14.76(0.07) (9.43(,,06)*
46.23(
21.88(0.05)*
(5.01(0.03)*
FAPAT;
37.42(0.00)
21.31
16.39(0.04)
50.41
24.59(0.66
17.29
Improv.
3.03%
2.60%
2.31%
1.46%
2.59%
2.49%
Table 8: Performance evaluation for next-item prediction on Tmall.
Tmall
Model
Hits@ 10
NDCG@ 10
MRR@ 10
Hits@20
NDCG@20
MRR@20
FPMC
13.71
9.02(0.02)
7.56/0.03)
16.44(0.23)
9.71(0.04)
7.74(0.02)
GRU4Rec
18.82(0.17)*
12.28(0.11)*
10.25(0.09)*
22.68(0.21)*
10.51 13.25
NARM
22.74 15.4
13.19
26.73(0.26)*
16.47(0.13)*
13.47(0.10)
STAMP
24.32(0.311* 16.55(0.29)
14.12(0.29)*
2
(7.58
(0,41
CSRM
25.13(0.19) 18.5610.18)
16.4810.18)*
27.9 1
19.27(0.17)
16.68(0.18)*
S3-Rect
18.24(0.11) 12.30(0.07)*
10.46(0.06)*
(0.31
13.32(0.08)*
10.74(0.00)
M2TRect
11.42(0.21)* 7.56(0.06)*
6,36(0.11)*
13.75(0.35)*
8,
6,52(0.10)
SR-GNN
18.21(0.51)*
12.11
10.20(0.28)*
49.
(2.91(0.31)
10.42(0.28)*
GC-SAN
19.29(0.14)*
12.80r0.07)*
10.78r0.13)*
13.78(0.09*
11.05(0.12)*
S2-DHCN
22.00(0.36)* 13.36(0.21)*
10.68(0.17)*
27.23(0.33)*
14.69(0.20)
11.05(0.17)*
GCE-GNN
28.3310.13y* 17.32.0.13y 20.01
30.24(0.16) 16)*
17.45(0.13) 10.50(0.13)*
LESSR
20.99(0.26)*
12.13 14.64(0.18)*
25.92(0.23)*
13.96(022)
10.50k0.23)*
MSGIFSR
23.18:0.199 199*
12.69 19
10y*
27.78(0.25)*
16.35
13.01
FAPAT&
32,45(0.11)
22.02.0.15)
18.72(0.13)
36.18:o.21)
22.970.14)
18.99(0.13)
Improv.
14.19%
10.04%
8.08%
19.64%
12.05%
8.83%
Table 9: Performance evaluation for next-item prediction Beauty.
Beauty MRR@1
Model
Hits 10
NDCG@10
Hits @
NDCG@20
MRR@20
FPMC
72.00
57.20
52.42
75.91
58.19
52.70
GRU4Rec
73.95
58.19
53.13
78.54
59.36
53.45
NARM
88.09
70.44
64.68
91.50
71.31
64.93
STAMP
80.08
63.76
58.47
83.84
64.72
58.73
CSRM
89.74
75.28
70.56
92.61
76.01
70.77
S3-Rect
89.64
75.56
70.99
92.53
76.30
71.19
M2TRect
80.13
65.97
61.65
83.66
66.87
61.65
SR-GNN
88.69
70.42
64.44
91.74
71.20
64.65
GC-SAN
86.67
70.80
64.71
88.98
72.50
65.97
S2-DHCN
7.25
5.38
4,80
8.87
5.79
4.91
GCE-GNN
89.34
73.15
67.80
91.29
73.65
67.94
LESSR
89.95
71.29
65.18
92.98
72.06
65.40
MSGIFSR
90.18
73.62
65.18
92.50
74.21
65.65
FAPATS
92.72
76.29
71.09
94.10
76.87
71.24
Improv.
2.82%
0.97%
0.14%
1.20%
0.75%
0.07%
18

---

Table 10: Performance evaluation for next-item prediction on Books.
Books
Model
Hits 10
NDCG@ 10
MRR@ 10
Hits
NDCG@20
MRR@2
FPMC
36.51
24.32
20.49
41.90
25.69
20.87
GRU4Rec
47.21
31.86
27.02
53.55
33.47
27.46
NARM
76.09
54.22
47.13
80.83
55,43
47.36
STAMP
61.49
42.13
35.95
67.46
43.65
36.37
CSRM
78.69
56.70
49.54
82.88
$7.77
49.83
S3- S3-Rec
75.00
58.54
53.23
79.45
59.67
53.55
M2TRec
32.56
22.58
24.98
35.39
25.70
22.78
SR-GNN
66.55
47.55
41.32
69.77
48.37
41.55
GC-SAN
72.56
54.92
49.25
75.73
56.05
50.14
S2-DHCN
4.69
3.42
3.03
5.60
3.65
3.09
GCE-GNN
77.61
57.60
51.00
80.03
58.22
51.17
LESSR
73.72
53.86
47.36
82.31
54.77
47.61
MSGIFSR
72.93
52.23
45.66
76.33
53.09
45.66
FAPATę
81.62
61.08
54.39
85.12
61.97
54.64
Improw.
3.72%
4.34%
2.18%
2.70%
3.85%
2.04%
Table 11: Performance evaluation for next-item prediction on Electronics.
Electronics
Model
Hits 10
NDCG@10
MRR 10
Hits@20
NDCG@20
MRR@2I
FPMC
37.87
26.91
23.42
42.07
27.97
23.71
GRU4Rec
58.46
40.69
35.02
64.42
42.21
35.44
NARM
61.10
41.20
32.05
77.36
44.56
33.75
STAMP
59.30
42.04
36.53
67.94
45.07
36.97
CSRM
62.28
44.35
38.59
67.47
45.67
38.96
S3-Rect
74.36
56.03
50.16
79.63
57.37
50.53
M2TRcc
57.32
44.84
40.85
61.70
45.95
41.15
SR-GNN
74.86
54.30
47.66
79,66
55.52
48.00
GC-SAN
72.76
53.37
45.98
77.34
46.34
49.91
S2-DHCN
4.18
2.65
2.18
5.08
2.88
2.24
GCE-GNN
72.93
53.74
47.59
78.49
55.15
47.98
LESSR
72.91
50.46
43.26
78.78
51.96
43.67
MSGIFSR
73.56
$3.83
47.77
77.45
$4.73
48.02
FAPATS
78,36
56.81
49.80
82.81
57.94
50.12
Improw.
4.68%
1.39%
-0.07%
3.95%
0.99%
-0.81%
19


Reviews:
[{"review_id": "UYDGFopHZE", "reviewer": "Reviewer_jZGH", "summary": "The paper proposes a framework that effectively utilizes attribute graph patterns to enhance anonymous sequence encoding for session-based recommendations. Given the lack of personal information in session-based recommendation scenarios, making accurate item suggestions within the session is crucial. The authors performed an extensive experimental evaluation, and it seems there is an improvement. I would appreciate more info on methodology used for experimentation.\n\nMinor comments:\n\nThe citations are clickable, which is convenient for further reading.\nIt would be beneficial to add references to Table 1 for better context.\nProviding more explanation in lines 70-75 would enhance clarity and understanding.\nWhile I'm not an expert in the area, it seems that the related work section could benefit from including more recent papers. Currently, there is only one paper from 2023.\nFig 5-6 are clear and easy to read, which is commendable.", "questions": "Can you please list a number of the current practical applications that would benefit from the proposed method? \n\nCould you provide more information about the costs associated with the model? Additionally, I'm interested in knowing about the latency, or response time, of the model.", "limitations": "Yes", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Extensive experimentation\n\n", "weaknesses": "1. Not clearly demonstrated novelty of the method \n2. The paper is a bit hard to follow\n3. Some examples of the data can be useful in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. Unclear novelty**\n\nA1. The session-based recommendation (SBR) does not provide the user profile in E-commerce services, such as for new or private mode users. Solely relying on temporal data remains limited in prior research. Simultaneously, constructing graph topologies might introduce noise from random clicks, while graph neural networks can exacerbate over-smoothing. We thus suggest a transformer architecture for the temporal view, while aggregating collaborative data via meta-data and frequent attribute patterns in the spatial view. Comprehensive experiments underscore the proposal's novelty and effectiveness.\n\n**W2. Hard to follow**\n\nA2. We commence by defining the problem and subsequently introduce the session and transition graph construction procedure. The primary methodology comprises three parts: (1) acquiring frequent attribute patterns through data mining, (2) encoding sessions via relevant pattern retrieval and memory augmentation, (3) providing recommendations through attention. In the experimental section (Section 6), we initially assess next-item prediction by computing Recall and ranks (MRR and NDCG). We demonstrate the significance of attribute patterns and our proposed graph-nested attention. Furthermore, we conduct two experiments on intent capture by estimating attribute predictions and period-item recommendations. We welcome suggestions and comments to enhance clarity, if feasible.\n\n**W3. Example illustration**\n\nA3. In Figure 1 and Figure 2, we present two cases: silver ↔ silver ↔ blue ↔ blue, which illustrates the user's color intent; additionally, the brand pattern Apple ↔ Apple ↔ Samsung suggests a potential change in intent.\n\n**Q1. More practical applications**\n\nA1. Session-based recommendations (SBR) serve as a direct industrial use, while our method aids other graph-related applications, such as answering questions over graphs.\n\n**Q2. Latency and cost**\n\nA2. Computation involves offline and online parts. Offline includes pattern mining and retrieval, while online encodes session data with attribute pattern augmentation. Offline costs are negligible due to linearity with data size, especially compared to resulting structures—far fewer than original sessions. As seen in Table 6, attributes and patterns total several thousand, sessions possibly several million. Graph density for frequent patterns remains stable at around 1.0, in contrast to methods like global or shortcut, which are 10x or 100x. As evident from performance, our model runs as efficiently as transformers.\n"}, {"review_id": "S1cYb5BVuG", "reviewer": "Reviewer_2VLV", "summary": "This paper studies session-based recommendation in E-commerce. The authors propose to enhance user intent identification by constructing attribute transition graphs and using frequent attribute patterns as memory to augment session representations. The study employs frequent graph pattern mining algorithms to find consequential graphlets and uses these attribute patterns as accessible memory to augment session sequence encoding. It leverages multi-head graph attention to learn patterns and local session graph representations in the aligned space. The experiments are conducted on two public benchmark datasets and three large-scale industrial datasets, demonstrating notable improvement across various evaluation metrics.", "questions": "How about the computational cost of this work? Is there any analysis of the real training and inference time? ", "limitations": "The authors mainly discussed the limitations at the technique level, but are not aware of the potential negative societal impact. Some discussions on recommendation fairness and diversity can be added.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Session-based recommendation is an important research topic in data mining.\n2. The proposed method achieves better results.\n3. The authors provide sufficient comparisons with baseline methods.", "weaknesses": "1. The motivations for different model components are not clearly stated nor discussed. The model components are directly introduced, without detailed discussions on their intuitions and big pictures. The experimental analysis also does not well elaborate on the reasons and technical findings.\n\n2. The proposed method is complicated and its computational cost is high. Session-based recommendation has strict latency restrictions and it is unclear whether the proposed method meets the requirements. Further efficiency analysis is highly needed, which is important for supporting the main motivation.\n\n3. The evaluation is purely offline and no online experimental result is reported. It is not clear whether the proposed method can be deployed online with acceptable effort.\n\n4. The proposed framework is a combination of mature techniques. Thus, the technical contributions are not quite salient. More empirical analysis could be made.\n\n5. It would be helpful to compare the results on long-tailed/cold-start attributes and items, since the proposed method seems to be suitable for handling these cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. Unclear motivation**\n\nA1. In E-commerce services, like scenarios involving new users or those in private mode, session-based recommendation (SBR) is challenging as it doesn't incorporate user profiles. Solely relying on temporal data is a limitation in prior research. Simultaneously, constructing graph topologies might introduce noise due to random clicks; here, utilizing graph neural networks helps address the issue of over-smoothing. In Section 6, our experimental evaluation begins by assessing next-item prediction through metrics like Recall and ranks (MRR and NDCG). We underscore the importance of attribute patterns and our proposed graph-nested attention. Furthermore, we conduct two experiments focused on intent capture by estimating attribute predictions and period-item recommendations. We welcome suggestions and comments aimed at enhancing clarity, if feasible.\n\n**W2. Latency and cost**\n\nA2. Computation encompasses offline and online components. The offline facet involves pattern mining and retrieval, while the online facet entails encoding session data via attribute pattern augmentation. The offline costs can be disregarded, given their nearly linear correlation with data size, not to mention resulting data structures fewer than those in the original sessions. As depicted in Table 6, attributes and patterns number in the thousands, whereas sessions could reach several million. Moreover, graph density of resulting frequent patterns remains stable at around 1.0. In contrast, other methods (e.g., global and shortcut) might exhibit factors of 10x or even 100x. As evident from our runtime, our model operates as efficiently as transformers.\n\n**W3. Online evaluation**\n\nA3. Online assessment of recommendations entails corporate security considerations, necessitating approval from the company.\n\n**W4. Combination of mature techniques**\n\nA4. Regrettably, we differ from the reviewer's perspective. Contemporary data mining technologies find widespread utility across diverse fields. We employ these techniques to reliably extract refined patterns, utilizing them as a repository to enhance recommendations. Furthermore, we introduce graph-nested attention, showcasing its effectiveness by outperforming state-of-the-art approaches by an average of 4.5%. Our framework stands poised to enrich the SBR and graph-related retrieval research community.\n\n**W5. Long-tail and cold-start evaluation**\n\nA5. In Figure 6, we address the cold-start scenario. Short-period cases relate to brief sessions, where our model excels in stability.\n\n**Q1. Computational cost**\n\nA1. Please see W2 and A2 for the cost analysis. Due to the attention mechanism's parallelism, both training and inference can be as efficient as regular transformers. There's an additional memory attention cost, but it's capped at 12 patterns."}, {"review_id": "63VNHDgQQ0", "reviewer": "Reviewer_BdTQ", "summary": "This paper presents a novel framework for session-based recommendations. The code idea of the method is to extract highly frequent attribute patterns from graphs to augment session sequence encoding. Specifically, it first leverages frequent graph pattern mining for attribute pattern retrieval. Then it applies GAT-based encoders for item representations and use these representations as memory to faciliate session-based encoding. Finally, it converts item-side graph representations into sequences and aggregates them with session sequences for transformer-based model training. Empirical results compared with sequence-based and graph-based baselines show the advantages of the method.", "questions": "Being not familiar with graph-based transformers, I mainly have concerns about the design choice mentioned in its method.  \n- What is the reason of choosing the GAT-based encoder rather than GCN and GraphSAGE, for learning attribute graph representations?  \n- What is the insights and design intuition of using attribute representations as memory to augment sequence encoding? \n- Why using sequences as the final representations rather than aggregate information using graph representations?  \n", "limitations": "Design intuition and technical details can be explained more clearly.  \nIt would be better to provide discussion and comparision of more state-of-the-art baselines.  ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written wrt the high-level topic and what the high-level intuitions are.  \nGood motivation of mining attribute features for sequence encoding enhancement, which is a significant problem to the community.  \nExperiment is extensive and convincing.  Experimental results on two public datasets and one industrial dataset seem promising compared with sequence-based and graph-based baselines.  \nComprehensive analysis in the ablation studies and the appendix.  ", "weaknesses": "Some clarifications can be made more clear. For example, it mentions using gSpan to mine patterns and keep patterns in the twenty types as it shows. However, it is not clear why keeping patterns in these twenty types, like it is a common practice or it can faciliate training.   \nMore powerful sequence-based approaches can be discussed.  As far as I know, P5 [1] and M6-rec [2] are more powerful sequence-based baselines. It would be better to discuss these methods. \n\n[1] Geng, Shijie, et al. \"Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5).\" RecSys 2022.  \n[2] Cui et al. \"M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems.\" Arxiv 2022.  \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. Clarifications for pattern acquisition and baselines**\n\nA1. Upon the motifs containing three or four nodes, we exclusively consider those featuring a circle or a triangle. These two structural types play a crucial role in diminishing randomness and enhancing robustness. In response to the reviewer's suggestions, we will explore and elaborate on recent literature. However, these references lack specificity regarding session-based recommendations, which, in turn, present more formidable challenges due to truncated behavioral histories and the absence of user profiles.\n\n**Q1. Reasons to choose GAT rather than GCN**\n\nA1. We tested various message-passing methods and found GAT to have higher recall.\n\n**Q2. Insights of memory augmentation**\n\nA2. Relying solely on temporal historical data remains constrained in prior research. Simultaneously, forming graph topologies might introduce noise from random clicks, and using graph neural networks would help counteract over-smoothing. Hence, we suggest incorporating the transformer architecture from a temporal perspective, while combining collaborative details via metadata and frequent attribute patterns in the spatial view.\n\n**Q3. Reasons to choose sequence models**\n\nA3. Recommendations regarding session behaviors must consider temporal signals. Research is ongoing on constructing suitable graph topologies for training and inference. Furthermore, utilizing graph neural networks could help address over-smoothing concerns."}, {"review_id": "JygzZCdlKy", "reviewer": "Reviewer_WQyi", "summary": "This paper proposes FAPAT, which augments session-based recommendation models using item attributes. Specifically, FAPAT constructs attribute graphs and models user intent using pattern mining and an improved graph attention mechanism.", "questions": "Please refer to \"Weakness\".", "limitations": "N/A", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "1. The paper studies an important application task, i.e., session-based recommendation.\n2. Experiments are conducted on 2 public datasets and 3 large-scale industrial datasets.", "weaknesses": "1. Over-complicated method. Session-based recommendation methods that incorporate session graphs have been criticized for being overly complex. The significant increase in algorithm complexity generally yields limited improvements in performance. The method proposed in this paper, involving modules such as motif extraction and graph attention mechanisms, adds further complexity to this paradigm. However, based on the experiments in Table 2, even with such complexity, this method only outperforms the concise baseline GRU4Rec by a limited margin on the classic benchmark dataset diginetica. The results raise doubts about the necessity of such a complex approach. It could be possible to achieve comparable results by introducing attributes using a much simpler method.\n2. Missing baselines. Two important baselines are missed. SASRec [1] is a highly popular baseline method for sequential/session-based recommendation, based on Transformer modules. FDSA [2] is a straightforward method based on SASRec that introduces item attributes. Both of these methods are concise and can be regarded as a basic version of the proposed model. These two methods should be included in experiments and compared against the proposed method.\n3. Code is not available during the reviewing phase, making reproduction of the reported results difficult.\n\n[1] Kang et al. Self-Attentive Sequential Recommendation. ICDM 2018.\n\n[2] Zhang et al. Feature-level Deeper Self-Attention Network for Sequential Recommendation. IJCAI 2019.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. Over-complicated method**\n\nA1. Differing Perspective on GRU4Rec. I must respectfully disagree with the reviewer's perspective. The counter-example of GRU4Rec showcases remarkable performance on diginetica (mean length: 4.850). Nevertheless, its effectiveness significantly wanes on Tmall (mean length: 6.649) and our extensive industrial dataset (mean length exceeding 10). This results in substantial disparities (18.82 vs. 32.45, 73.95 vs. 92.72, 47.21 vs. 81.62, and 58.46 vs. 78.36). Such variations provide robust validation for the ingenuity and novelty driving our proposed approach.  \n\n**W2. Missing baselines**\n\nA2: We extend our gratitude to the reviewer for highlighting the two baselines. We will comprehensively discuss these baselines in the final version. Regarding SASRec [1], which employs stacked Transformer layers, it can be seen as the non-pretrained S3-Rec model. The performance comparison is presented below:\n| method | | diginetica | | | Tmall | | | Beauty | | | Books | | | Electronics | |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| | Recall@10 | NDCG@10 | MRR@10 | Recall@10 | NDCG@10 | MRR@10 | Recall@10 | NDCG@10 | MRR@10 | Recall@10 | NDCG@10 | MRR@10 | Recall@10 | NDCG@10 | MRR@10 |\nSASRec | 32.15 | 17.86 | 13.52 | 13.69 | 9.47 | 8.16 | 85.18 | 70.87 | 66.21 | 67.09 | 52.10 | 47.25 | 61.14 | 45.31 | 40.08 |\nS3Rec | 33.48 | 18.58 | 14.04 | 18.24 | 12.30 | 10.46 | 89.64 | 75.56 | 70.99 | 75.00 | 58.54 | 53.23 | 74.36 | 56.03 | 50.16 |\nFAPAT | 37.42 | 21.31 | 16.39 | 32.45 | 22.02 | 18.72 | 92.72 | 76.29 | 71.09 | 81.62 | 61.08 | 54.39 | 78.36 | 56.81 | 49.80 | FDSA [2] introduces separate channels for features, demonstrating the effectiveness of metadata. We are unable to present the results due to data security concerns. However, based on our experience, FDSA's performance lies between SASRec and S3-Rec. A comprehensive discussion of these two baselines will be included in the final version.\n\n**W3. Unavailable code**\n\nA3. Code can't be uploaded due to security concerns currently. It'll be released upon paper acceptance and security department's approval."}, {"review_id": "KAWVXUU08O", "reviewer": "Reviewer_LurS", "summary": "The paper presents a transformer-based method, namely Frequent Attribute Pattern Augmented Transformer (FAPAT), that considers attribute patterns as supplementary information. More specifically, FAPAT builds attribute transition graphs, mine frequent attribute patterns, and match attribute patterns to better captures user intents. Experiments conducted on two public benchmarks as well as three industrial datasets show that the proposed method deliver noticeable performance boosts (4.5%) averaged over all experiments. The code and datasets are promised to be released after acceptance.", "questions": "1. In introduction, can you elaborate more on “We argue that the current use of item-side metadata provides little assistance in SBR models as user intent may change over time.“?\n2. Typo in line 131 - \"whi\".\n3. I am not sure what is the purpose of Section 3 - Background and Motivations, especially 3.2 and 3.3, they read like they should belong to Related Work. ", "limitations": "Yes, the authors discuss limitations in Appendix A.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Introduction section is written very well to present the problem and the high-level picture of the proposed method.\n2. Table 1 in Related Work section is very straight-forward and efficient in comparing a large number of existing methods and demonstrating how they differ from the proposed method. And the categorical items (temporal info, history attention, etc.) are reasonably defined.\n3. Methodology section is clear and easy to follow.\n4. Experiments are thorough with a good number of baselines.", "weaknesses": "1. Some of the thought process is not explained, for example, in section 4.1.1, what are the thoughts process behind adopting gSpan and how were those twenty types of patterns chosen?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. Clarifications for pattern acquisition**\n\nA1. End-to-end retrieval becomes a viable option in the absence of resource constraints. However, given the multitude of substructures—numbering in the millions or even billions—within industrial sessions, we must ponder efficiency. Data mining methods, like gSpan, offer near-linear complexity, drastically trimming the pool of pattern candidates to a few thousand. This ensures efficiency and effectiveness in both training and inference. We exclusively consider motifs with three or four nodes, encompassing only those exhibiting circular or triangular structures. These two types of structures notably curtail randomness and enhance robustness."}]

Author rebuttal:

Meta-review: Overall somewhat borderline; recommending acceptance mainly due to calibration versus other papers on similar topics, though could go either way. Mostly borderline scores, though one slightly more positive review (though it's the shortest) creates something of a case for acceptance. The rebuttal was very detailed, though not very productive in terms of improving the scores.

Partly recommending acceptance since many of the issues are due to clarifications (though clarity does seem to be something of an issue), which could probably be fixed. Some more substantive issues around motivation/novelty, though these issues were raised by fewer reviewers.
Decision: Accept (poster)