SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 6: Statistics of datasets based on timestamps.
Public	Industrial (E-commerce)
diginetica	Tmall	Beauty	Books	Electronics
#User	57,623	7,576	2.6 M	3.2 M	10.2 M
#Item	43,074	39,768	39.2 K	94.8 K	244.7 K
Click	993,163	438,315	27.2 M	38.8 M	115.6 M
Avg. Len.	4.850	6.649	10.325	11.912	11.249
Train	630,789	303,181	19.6 M	28.2 M	84.1 M
Valid	78,708	33,735	2.4 M	3.5 M	10.5 M
Test	78,907	35,481	2.5 M	3.8 M	10.6 M
#Attribute	category: 995	category: 821 brand: 4,304	category: 359 color: 1,101 brand: 4,359 size: 1,883	category: 18 publisher: 2,751 author: 27,651 genre: 2,634	type: 123 category: 881 color: 2,096 brand: 24,196
#Pattern	category: 1,866	category: 33,582 brand: 2,497	category: 970 color: 4,059 brand: 254 size: 1,091	category: 24 publisher: 4,370 author: 1,399 genre: 12,535	type: 9,289 category: 13,991 color: 146,402 brand: 14,043
Density	Local: 0.886 Global: 11.329 Shortcut: 2.512 Heterogeneous: 0.543 Pattern: 1.023	Local: 1.249 Global: 10.222 Shortcut: 4.983 Heterogeneous: 0.707 Pattern: 1.165	Local: 4.510 Global: 70.504 Shortcut: 29.827 Heterogeneous: 3.412 Pattern: 1.095	Local: 3.554 Global: 99.389 Shortcut: 26.649 Heterogeneous: 2.333 Pattern: 1.085	Local: 2.910 Global: 128.041 Shortcut: 19.865 Heterogeneous: 2.049 Pattern: 1.189
C.3 Data Split
We follow previous settings that split training/validation/testing data based on timestamps. For diginetica, we gather the last 8-14 days as validation, the last 7 days as testing, and remaining as training. For Tmall, we use the last 101-200 seconds as validation, the last 100 seconds as testing, and remaining as training. For our industrial E-commerce data (i.e., Beauty, Books, Electronics), we select the last 6-10 days as validation, the last 5 days as testing, and remaining as training.
C.4 Data Statistics
Table 6 summarizes the statistics of the experimental datasets based on timestamps. The density is calculated based on undirected graphs, which would be doubled during graph convolution in practice. Local density, as used in SR-GNN and GC-SAN, corresponds to the average density of local session graphs in E-commerce sessions. On the other hand, global density, as used in GCE-GNN, refers to the density of the global collaborative graph obtained by connecting all adjacent items appearing in all sessions. Shortcut density, as used in LESSR, is the density resulting from connecting all items in a single session as a complete graph. Heterogeneous density, as used in MSGIFSR, refers to the average density of the heterogeneous graphs obtained by regarding the consecutive adjacent two nodes as a fine-grained intent unit. Lastly, pattern density, as used in FAPAT, is the density of the acquired frequent and compact patterns. From Table 6, it is evident that leveraging patterns is the most effective way of characterizing user intents because other graph topologies vary with data sources and scales, making it difficult to generalize and provide stable performance. Besides, patterns can be preprocessed as indices to aid recommendations, making them more practical in industrial scenarios. Moreover, it is easy to update attribute patterns dynamically, whereas other graph structures are more closely coupled with input sessions and are more sensitive to tiny variations.
D Baselines
We compare our method with the following baselines:
Sequence-based methods
- FPMC [27] learns the representation of session via Markov-chain based methods.
- GRU4Rec [8] is the first RNN-based approach that simulates the Markov Decision Process (MDP) but has a better generalization.
- NARM [16] is a attention-based RNN model to learn session embeddings.
- STAMP [19] adopts attention mechanism between the last item to previous histories to represent users' short-term interests.
- CSRM [31] proposes to engage an inner memory encoder and external memory network to capture correlations between neighborhood sessions to enrich the collaborative representations.
- S3-Rec [47] is the first pretrained SBR model that predicts items, attributes, and segments during the pretraining stage.
- M2TRec [28] is a metadata-aware multi-task Transformer model. In the original paper, the authors ignore item embeddings. For a fair comparison, we also regard the item ids as one of metadata.
Graph-based methods
- SR-GNN [35] is the first GNN-based model for the SBR task, which transforms the session data into a direct unweighted graph and learns the representation of the item-transitions graph.
- GC-SAN [38] uses gated GNNs to extract local context information and then self-attention to obtain the global representation.
- S2-DHCN [37] transforms the session data into hyper-graphs and line-graphs and encodes them via GCNs to enhance the session representations.
- GCE-GNN [34] aggregates two levels of item embeddings from session graphs and global graphs with soft attention.
- LESSR [1] preserves the edge order and constructs shortcuts to encode sessions for GNNs.
- MSGIFSR [7] captures the user intents from multiple granularities to relieve the computational burden of long-dependency. In experiments, we search the best model from the level-1, level-2, and level-3 consecutive intent units.
E Experimental Settings
We fix all embeddings and hidden dimensions as 100, and the batch size is searched among  $\{100, 200, 500\}$  for all methods. We also choose the number of layers/iterations (if applicable) from the validation performance (e.g., MRR@10). A learning scheduler with  $10\%$  linear warmup and  $90\%$  decay is associated with the Adam optimizer [14]. The initial learning rate is set as 1e-3, and the regularization weight is tuned among  $\{1e - 4, 1e - 5, 1e - 6\}$ . We seek the dropout probability between two modules from  $\{0.0, 0.2, 0.4\}$ , but fix the attention dropout rate as 0.2. The number of attention heads is empirically set as 4. We follow the setting of GCE-GNN that the maximum one-hop neighbor number in GAT is 12. In the interest of fairness, we also set the maximum selected pattern number as 12. Hyper-parameter tuning is time costly on our industrial data so that we use the best combinations obtained from one day transactions. We implement our methods and run experiments with Python and PyTorch over 8 x A100 NVIDIA GPUs.
F Experimental Results
Due to the space limit, we only report some results in the main content. More comprehensive comparisons are shown in Tables 7-11, where standard deviations are enclosed in brackets. The best and second-best results are respectively highlighted in bold and underlined. Methods that use attributes are marked with  $\ddagger$ , and * indicates the  $p$ -value  $< 0.0001$  in t-test.
Table 7: Performance evaluation for next-item prediction on diginetica.
Model	diginetica
Hits@10	NDCG@10	MRR@10	Hits@20	NDCG@20	MRR@20
FPMC	31.57(0.04)*	17.40(0.01)*	13.08(0.02)*	43.19(0.05)*	20.33(0.03)*	13.88(0.03)*
GRU4Rec	36.77(0.14)*	20.71(0.05)*	15.80(0.03)*	49.68(0.06)*	23.97(0.03)*	16.70(0.03)*
NARM	35.98(0.10)*	20.18(0.06)*	15.36(0.06)*	48.89(0.12)*	23.44(0.06)*	16.26(0.06)*
STAMP	33.59(0.15)*	18.89(0.18)*	14.41(0.19)*	45.87(0.15)*	22.00(0.18)*	15.26(0.19)*
CSRM	33.97(0.08)*	19.43(0.03)*	14.98(0.03)*	45.83(0.02)*	22.42(0.02)*	15.80(0.02)*
S3-Rec‡	33.48(0.13)*	18.58(0.09)*	14.04(0.10)*	45.97(0.08)*	21.74(0.09)*	14.90(0.10)*
M2TRec‡	29.67(0.43)*	16.30(0.24)*	12.23(0.18)*	41.23(0.63)*	19.22(0.29)*	13.02(0.20)*
SR-GNN	35.21(0.02)*	19.68(0.04)*	14.94(0.04)*	47.99(0.04)*	22.90(0.04)*	15.82(0.04)*
GC-SAN	35.25(0.09)*	19.72(0.04)*	14.97(0.03)*	47.87(0.09)*	22.90(0.04)*	15.85(0.03)*
S2-DHCN	30.76(0.07)*	17.04(0.14)*	12.86(0.16)*	42.39(0.07)*	19.98(0.13)*	13.66(0.16)*
GCE-GNN	36.32(0.09)*	20.77(0.07)*	16.02(0.07)*	48.67(1.12)*	23.89(0.23)*	16.87(0.03)*
LESSR	33.68(0.05)*	18.71(0.03)*	14.14(0.03)*	46.23(0.11)*	21.88(0.05)*	15.01(0.03)*
MSGIFSR	34.74(0.09)*	19.43(0.06)*	14.76(0.07)*	46.23(0.11)*	21.88(0.05)*	15.01(0.03)*
FAPAT‡	37.42(0.10)	21.31(0.03)	16.39(0.04)	50.41(0.15)	24.59(0.06)	17.29(0.04)
Improv.	3.03%	2.60%	2.31%	1.46%	2.59%	2.49%
Table 8: Performance evaluation for next-item prediction on Tmall.
Model	Tmall
Hits@10	NDCG@10	MRR@10	Hits@20	NDCG@20	MRR@20
FPMC	13.71(0.16)*	9.02(0.02)*	7.56(0.03)*	16.44(0.23)*	9.71(0.04)*	7.74(0.02)
GRU4Rec	18.82(0.17)*	12.28(0.11)*	10.25(0.09)*	22.68(0.21)*	13.25(0.12)*	10.51(0.10)*
NARM	22.74(0.20)*	15.46(0.12)*	13.19(0.10)*	26.73(0.26)*	16.47(0.13)*	13.47(0.10)*
STAMP	24.32(0.31)*	16.55(0.29)*	14.12(0.29)*	28.40(0.35)*	17.58(0.30)*	14.41(0.29)*
CSRM	25.13(0.19)*	18.56(0.18)*	16.48(0.18)*	27.94(0.15)*	19.27(0.17)*	16.68(0.18)*
S3-Rec‡	18.24(0.11)*	12.30(0.07)*	10.46(0.06)*	22.31(0.17)*	13.32(0.08)*	10.74(0.06)*
M2TRec‡	11.42(0.21)*	7.56(0.06)*	6.36(0.11)*	13.75(0.35)*	8.15(0.04)*	6.52(0.10)*
SR-GNN	18.21(0.51)*	12.11(0.32)*	10.20(0.28)*	21.34(0.49)*	12.91(0.31)*	10.42(0.28)*
GC-SAN	19.29(0.14)*	12.80(0.07)*	10.78(0.13)*	23.18(0.23)*	13.78(0.04)*	11.05(0.12)*
S2-DHCN	22.00(0.36)*	13.36(0.21)*	10.68(0.17)*	27.23(0.33)*	14.69(0.20)*	11.05(0.17)*
GCE-GNN	28.33(0.13)*	20.01(0.12)*	17.32(0.13)*	30.24(0.16)*	20.50(0.13)*	17.45(0.13)*
LESSR	20.99(0.26)*	14.64(0.18)*	12.13(0.19)*	25.92(0.23)*	13.96(0.22)*	10.50(0.23)*
MSGIFSR	23.18(0.19)*	15.19(0.11)*	12.69(0.10)*	27.78(0.25)*	16.35(0.11)*	13.01(0.09)*
FAPAT‡	32.45(0.21)	22.02(0.15)	18.72(0.13)	36.18(0.21)	22.97(0.14)	18.99(0.13)
Improv.	14.19%	10.04%	8.08%	19.64%	12.05%	8.83%
Table 9: Performance evaluation for next-item prediction on Beauty.
Model	Beauty
Hits@10	NDCG@10	MRR@10	Hits@20	NDCG@20	MRR@20
FPMC	72.00	57.20	52.42	75.91	58.19	52.70
GRU4Rec	73.95	58.19	53.13	78.54	59.36	53.45
NARM	88.09	70.44	64.68	91.50	71.31	64.93
STAMP	80.08	63.76	58.47	83.84	64.72	58.73
CSRM	89.74	75.28	70.56	92.61	76.01	70.77
S3-Rec‡	89.64	75.56	70.99	92.53	76.30	71.19
M2TRec‡	80.13	65.97	61.65	83.66	66.87	61.65
SR-GNN	88.69	70.42	64.44	91.74	71.20	64.65
GC-SAN	86.67	70.80	64.71	88.98	72.50	65.97
S2-DHCN	7.25	5.38	4.80	8.87	5.79	4.91
GCE-GNN	89.34	73.15	67.80	91.29	73.65	67.94
LESSR	89.95	71.29	65.18	92.98	72.06	65.40
MSGIFSR	90.18	73.62	65.18	92.50	74.21	65.65
FAPAT‡	92.72	76.29	71.09	94.10	76.87	71.24
Improv.	2.82%	0.97%	0.14%	1.20%	0.75%	0.07%
Table 10: Performance evaluation for next-item prediction on Books.
Model	Books
Hits@10	NDCG@10	MRR@10	Hits@20	NDCG@20	MRR@20
FPMC	36.51	24.32	20.49	41.90	25.69	20.87
GRU4Rec	47.21	31.86	27.02	53.55	33.47	27.46
NARM	76.09	54.22	47.13	80.83	55.43	47.36
STAMP	61.49	42.13	35.95	67.46	43.65	36.37
CSRM	78.69	56.70	49.54	82.88	57.77	49.83
S3-Rec‡	75.00	58.54	53.23	79.45	59.67	53.55
M2TRec‡	32.56	22.58	24.98	35.39	25.70	22.78
SR-GNN	66.55	47.55	41.32	69.77	48.37	41.55
GC-SAN	72.56	54.92	49.25	75.73	56.05	50.14
S2-DHCN	4.69	3.42	3.03	5.60	3.65	3.09
GCE-GNN	77.61	57.60	51.00	80.03	58.22	51.17
LESSR	73.72	53.86	47.36	82.31	54.77	47.61
MSGIFSR	72.93	52.23	45.66	76.33	53.09	45.66
FAPAT‡	81.62	61.08	54.39	85.12	61.97	54.64
Improv.	3.72%	4.34%	2.18%	2.70%	3.85%	2.04%
Table 11: Performance evaluation for next-item prediction on Electronics.
Model	Electronics
Hits@10	NDCG@10	MRR@10	Hits@20	NDCG@20	MRR@20
FPMC	37.87	26.91	23.42	42.07	27.97	23.71
GRU4Rec	58.46	40.69	35.02	64.42	42.21	35.44
NARM	61.10	41.20	32.05	77.36	44.56	33.75
STAMP	59.30	42.04	36.53	67.94	45.07	36.97
CSRM	62.28	44.35	38.59	67.47	45.67	38.96
S3-Rec‡	74.36	56.03	50.16	79.63	57.37	50.53
M2TRec‡	57.32	44.84	40.85	61.70	45.95	41.15
SR-GNN	74.86	54.30	47.66	79.66	55.52	48.00
GC-SAN	72.76	53.37	45.98	77.34	46.34	49.91
S2-DHCN	4.18	2.65	2.18	5.08	2.88	2.24
GCE-GNN	72.93	53.74	47.59	78.49	55.15	47.98
LESSR	72.91	50.46	43.26	78.78	51.96	43.67
MSGIFSR	73.56	53.83	47.77	77.45	54.73	48.02
FAPAT‡	78.36	56.81	49.80	82.81	57.94	50.12
Improv.	4.68%	1.39%	-0.07%	3.95%	0.99%	-0.81%