SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For instance, if the product is available in the US, users can access the product by using the following link: https://www.amazon.com/dp/ASIN_Number
product frequency and repeat pattern across the six locales, which is consistent with the observation that we made on the whole dataset in Section 3.
(a) DE
(b) ES
(c) FR
(d) IT
(e) JP
(f) UK
Figure 4: Session length w.r.t. locales where the x-axis corresponds to the session length (the number of items in a session), the y-axis indicates the number of sessions with the corresponding session length. A clear long-tail phenomenon can be found, where only a few sessions show a session length of more than 100.
B.3 License
The Amazon-M2 dataset can be freely downloaded at https://www.aicrowd.com/challenges/amazon-kdd-cup-23-multilingual-recommendation-challenge/problems/task-1- next-product-recommendation/dataset_files and used under the license of Apache 2.0. The authors agree to bear all responsibility in case of violation of rights, etc.
B.4 Extended Discussion
(f) UK
Figure 5: The number of repeat items w.r.t. locales where the x-axis corresponds to the number of repeat items, the y-axis indicates to the number of session with the corresponding number of repeat items. Notably, we exclude those sessions with no repeat patterns. A clear long-tail phenomenon can be found, where only a few sessions show many repeat items.
C More Experimental Results
C.1 Additional results on NDCG@100
We provide additional NDCG@100 results on task 1 and 2. Results are shown in Table 7 and ??, respectively. A similar observation can be found with Recall@100 metric.
C.2 Task 1. Next-product Recommendation
In this subsection, we provide the model performance comparison on the phase-1 test and phase-2 test in Table 8 and Table 9, respectively. We can have similar observations as we made in Section 4.1: the popularity heuristic generally outperforms the deep models with respect to both MRR and Recall, with the only exception that CORE achieves better performance on Recall. This suggests that the popularity heuristic is a strong baseline and the challenging Amazon-M2 dataset requires new recommendation strategies to handle. We believe that it is potentially helpful to design strategies that can effectively utilize the available product attributes.
C.3 Task 2. Next-product Recommendation with Domain Shifts
We report the mode performances on the phase-1 test and phase-2 test in Table 10 and Table 11, respectively. Note that we omit the supervised training results since we have already identified that finetuning can significantly improve it. From the tables, we arrive at a similar observation as presented in Section 4.2 that the finetuned deep models generally outperform the popularity heuristic in Recall but underperform it in MRR. This illustrates that the deep models have the capability to retrieve a substantial number of pertinent products, but they fall short in appropriately ranking them. As a result, there is a need to enhance these deep models further in order to optimize their ranking efficacy.
Table 7: NDCG@100 results on Task 1.
UK	DE	JP	Overall
Popularity	0.2065	0.1993	0.2515	0.2175
GRU4Rec++	0.2255	0.2184	0.2733	0.2374
NARM	0.2336	0.2248	0.2823	0.2452
STAMP	0.2262	0.2198	0.2726	0.2379
SRGNN	0.2325	0.2345	0.2715	0.2448
CORE	0.2698	0.2667	0.3204	0.2839
MGS	0.2314	0.2357	0.2782	0.2471
Table 8: Experimental results on Task 1 phase-1 test set.
MRR@100	Recall@100
UK	DE	JP	Overall	UK	DE	JP	Overall
Popularity	0.2723	0.2746	0.3196	0.2875	0.4940	0.5261	0.5652	0.5262
GRU4Rec++	0.2094	0.2082	0.2527	0.2222	0.4856	0.5192	0.5416	0.5137
NARM	0.2235	0.2233	0.2705	0.2378	0.5220	0.5594	0.5814	0.5524
STAMP	0.2398	0.2398	0.2888	0.2547	0.4265	0.4538	0.4867	0.4538
SRGNN	0.2240	0.2211	0.2670	0.2361	0.4986	0.5311	0.5540	0.5262
CORE	0.1777	0.1797	0.2103	0.1882	0.6513	0.6927	0.7009	0.6801
C.4 Task 3. Next-product Title Prediction
We expand Table 6 to include the results on phase-2 test and the full results are shown in Table 12. From the table, we have the same observations as we made in Section 4.3: (1) Extending the session history length  $(K)$  does not contribute to a performance boost, and (2) The simple heuristic of Last Product Title outperforms all other baselines. It calls for tailored designs of language models for this challenging task.
D Limitation & Broader Impact
The release of the Amazon-M2 dataset brings several potential broader impacts and research opportunities in the field of session-based recommendation and language modeling. It provides the potential for research in the session recommendation domain to access the rich semantic attributes and knowledge from multiple locales, enabling better recommendation systems for diverse user populations.
While the Amazon-M2 dataset offers significant research potential, it is crucial to consider the certain limitations associated with its use. Despite efforts that have been made to include diverse user behaviors and preferences with multiple locales and languages, it may not capture the full linguistic and cultural diversity of all regions. Moreover, the dataset can be only collected within the Amazon platform, which may not fully capture the diversity of user behaviors in other domains or platforms, leading to a potentially biased conclusion and may not hold true in different contexts.
We also carefully consider the broader impact from various perspectives such as fairness, security, and harm to people. There may be one potential fairness issue, Amazon-2M may inadvertently reinforce biases if specific demographics are disproportionately represented or underrepresented in different regions. Such issue may happen as we promote diversity with sessions from more locales. While there might be fairness concerns, we assert that the Amazon-M2 dataset underscores the need for progress in fair recommendation development and serves as an effective platform for assessing fair recommendation approaches.
Table 9: Experimental results on Task 1 phase-2 test set.
MRR@100	Recall@100
UK	DE	JP	Overall	UK	DE	JP	Overall
Popularity	0.2711	0.2754	0.3205	0.2875	0.4937	0.5283	0.5660	0.5271
GRU4Rec++	0.2081	0.2097	0.2533	0.2224	0.4843	0.5220	0.5420	0.5143
NARM	0.2219	0.2235	0.2720	0.2377	0.5209	0.5624	0.5786	0.5522
STAMP	0.2387	0.2402	0.2894	0.2546	0.4234	0.4585	0.4864	0.4541
SRGNN	0.2224	0.2224	0.2695	0.2367	0.4974	0.5336	0.5529	0.5262
CORE	0.1755	0.1807	0.2111	0.1880	0.6518	0.6966	0.7002	0.6813
Table 10: Experimental results on Task 2 phase-1 test.
MRR@100	Recall@100
Methods	ES	FR	IT	Overall	ES	FR	IT	Overall
Heuristic	Popularity	0.2934	0.2968	0.2887	0.2927	0.5725	0.5825	0.5861	0.5816
Pretraining & finetuning	GRU4Rec++	0.2665	0.2829	0.2527	0.2669	0.6467	0.6612	0.6600	0.6573
NARM	0.2707	0.2890	0.2608	0.2733	0.6556	0.6612	0.6685	0.6629
STAMP	0.2757	0.2860	0.2653	0.2753	0.5254	0.5377	0.5371	0.5346
SRGNN	0.2853	0.2979	0.2706	0.2840	0.6263	0.6505	0.6453	0.6427
CORE	0.2058	0.2091	0.1984	0.2040	0.7457	0.7384	0.7545	0.7466
Table 11: Experimental results on Task 2 phase-2 test.
MRR@100	Recall@100
Methods	ES	FR	IT	Overall	ES	FR	IT	Overall
Heuristic	Popularity	0.3017	0.3068	0.2902	0.2989	0.5818	0.5934	0.5826	0.5863
Pretraining & finetuning	GRU4Rec++	0.2648	0.2867	0.2569	0.2695	0.6473	0.6619	0.6600	0.6577
NARM	0.2742	0.2938	0.2658	0.2779	0.6617	0.6624	0.6742	0.6670
STAMP	0.2809	0.2922	0.2653	0.2787	0.5340	0.5400	0.5387	0.5381
SRGNN	0.2878	0.3035	0.2701	0.2863	0.6359	0.6582	0.6491	0.6493
CORE	0.2016	0.2138	0.1967	0.2040	0.7530	0.7387	0.7572	0.7495
Table 12: Full results of BLEU scores in Task 3.
Validation	Phase-1 Test	Phase-2 Test
mT5-small, K = 1	0.2499	0.2265	0.2245
mT5-small, K = 2	0.2401	0.2176	0.2166
mT5-small, K = 3	0.2366	0.2142	0.2098
mT5-base, K = 1	0.2477	0.2251	0.2190
Last Product Title	0.2500	0.2677	0.2655