Leveraging Distribution Alignment via Stein Path for Cross-Domain Cold-Start Recommendation
Authors: Weiming Liu
Jiajie Su
Chaochao Chen
Xiaolin Zheng
Venue: NeurIPS 2021 Poster (2021)
Keywords: Recommendation
Domain Adaptation
Stein Path

Abstract: Cross-Domain Recommendation (CDR) has been popularly studied to utilize different domain knowledge to solve the cold-start problem in recommender systems. In this paper, we focus on the Cross-Domain Cold-Start Recommendation (CDCSR) problem. That is, how to leverage the information from a source domain, where items are 'warm', to improve the recommendation performance of a target domain, where items are 'cold'. Unfortunately, previous approaches on cold-start and CDR cannot reduce the latent embedding discrepancy across domains efficiently and lead to model degradation. To address this issue, we propose DisAlign, a cross-domain recommendation framework for the CDCSR problem, which utilizes both rating and auxiliary representations from the source domain to improve the recommendation performance of the target domain. Specifically, we first propose Stein path alignment for aligning the latent embedding distributions across domains, and then further propose its improved version, i.e., proxy Stein path, which can reduce the operation consumption and improve efficiency. Our empirical study on Douban and Amazon datasets demonstrate that DisAlign significantly outperforms the state-of-the-art models under the CDCSR setting.


Leveraging Distribution Alignment via Stein Path for Cross-Domain Cold-Start Recommendation
Weiming Liu, Jiajie Su, Chaochao Chen, and Xiaolin Zheng"
Zhejiang University, Hangzhou, China 221831010,sujaaiie zjucec, xlzheng)@zju.co edu, en
Abstract
Cross-Domand Recommendation (CDR) has been popularly studied to utilize differ- ent domain knowledge to solve the cold-start problem in recommender systems. In this paper, we focus on the Cross-Domuin Cold-Start Recommenuaaion (CDCSR) problem. That is. how to leverage the information from source domain, where items are 'warm', to improve the recommendation performance of target domain, where items are 'cold'. Unfortumately, previous approaches cold-start and CDR cannot reduce the latent embedding discrepancy across domains efficiently and lead model degradation. To address this issue, propose DisAlign, cross-domain recommendation framework for the CDCSR problem, which utilizes both rating and auxiliary representations from the source domain to improve the recommen- dation performance of the target domain. Specifically, we first propose Stein path alignment for aligning the latent embedding distributions across domains, and then further propose its improved version, i.e., proxy Stein path, which can reduce the operation consumption and improve efficiency. Our empirical study on Douban and Amazon datasets demonstrates that DisAlign significantly outperforms the oftte----th-arr models under the CDCSR setting.
Introduction
Data sparsity and cold-start long-standing problems in recommender systems [15 19.. development of internet techniques, most users always participant in many platforms or domains for different purposes. Therefore, Cross-Domain Recommendation (CDR) has emerged utilise the relatively richer information from a source domain to improve the recommendation accuracy in a target domain (52 (53). Most existing CDR models tackle the data sparsity problem in the target domain by assuming the existence of overlapped users or items with similar tastes or attributions across domains [5].
With the
Instead of focusing on solving the data sparsity problem, we consider cold-start item recommendation under the CDR setting. Specifically, concentrate the tion (CDCSR) problem, that is, two domains share the same user set but different items, and both domains have auxiliary representations such item profiles descriptions. The prime challenge is how to leverage the information from the source domain, where the items are 'warm' to improve the recommendation performance of the target domain, where the items are 'cold', The CDCSR problem popularly exists in practice, for instance, a movie marketing platform newly launches a book renting service where there is no user-book interaction yet, as shown in Figure [[
Cold-Start Recommenda-
Existing researches on cold-start recommendation and CDR cannot solve the above problem well. On the hand, existing cold-start recommendation models assume that the distributions of cold items
should consistent with the warm ones as they are homogeneous 54
18 12 On the other hand,
CCoresspodiigg Author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

existing CDR models assume that both source and target domains have user-item interactions for learning the mapping functions (25) Since the cold and warm items are heterogeneous with different latent embedding distributions in practice, and there is no user-item interaction in the target domain, conventional cold-start and CDR models cannot properly suitable to the CDCSR problem.
Similar to the transfer learning task, the key to the CDCSR problem is reduce the discrepancy between the latent embed- ding distributions across domains. However, both the warm and cold item representations are scattered and complicated due to the fact that the latent embeddings may represent diverse information. Thus, existing transfer learning based domain
Warm Dem
Culé Deen
Auxiliery Reporsentation
User
adaptation approaches [23 45 37 cannot achieve good align- ment results, which limits their performances.
To address the aforementioned issue, in this paper, we propose DisAlign, cross-domain recommendation framework for the CDCSR problem. In order to better align the complicated latent embedding distributions and make high quality of rating predictions, we utilize two modules in DisAlign, i.e., rating prediction module and embedding distribution alignment module, as will be shown in Figure
Sourc
Terrpt
(Mosie)
(Hooll
Figure 1: The CDCSR Problem.
2 The rating prediction module aims to capture user and item collaborative preferences in the source domain, and we propose metric-basee contrastive learning for modelling. The goal of distribution alignment module is to properly match the latent embedding distributions across domains, and we propose two techniques for it, i.e., Stein path aligument and improved version called proxy Stein path alignment. Specifically, inspired by the particle-bet based inference algorithm Stein Variational Gradient Descent (SVGD) 22 13 20), we first propose Stein path alignment to minimize the domain discrepancy through the particle-moving process, which can take both the source probability and target intra-domain structure into account. Although Stein path can obtain satisfying performance, it has to involve all the target samples during the training process, which is time consuming when data size is large. Thus, we further propose proxy Stein path alignment which only needs to exploit typical samples to represent the target data distribution, and thus can accelerate the operation speed. The comparison and visualization results in experiments will show the reliability and efficiency of DisAlign.
We summarize the main contributions of this paper as follows: (1) We propose novel framework, i.e., DisAlign, for the CDCSR problem, which can utilize both rating and auxiliary representations from the source domain to improve the recommendation performance of the target domain. (2) To our best knowledge, this the first attempt in literature to propose Stein path alignment for aligning the latent embedding distributions across domains, and we also propose its improved version, i.e., proxy Stein path, for higher efficiency. (3) Empirical studies on Douban and Amazon datasets demonstrate that DisAlign significantly improves the state-of- offthe-art art models under the CDCSR setting.
2 The proposed model
2.1 Framework of DisAlign
First, we describe notations. We assume there two domains, i.e., a source domain S and a target domain T. We assume both domains S and
User UserOze Hot ID
x
Fir
Rating Prodiction
Lax
T have Ny users, S has Ng warm items, and T has Ny cold items. Let R E No xNs be the
Warm Ttam One Hot ID
x²
Fi
lax
Rating Prediction Module
Thrm Clustering
warm rating matrix in S and R NuNTN be the cold rating matrix in T. In CDCSR setting, RD is absence during training and will be only used for test, since items are cold in T. We also assume that the warm items and the cold items have auxiliary representations X E R N and X E N2 respectively, with Z denoting the dimension of auxiliary representations. The auxiliary representations usually include useful side- information, e.g., themes, reviews, profiles in movie domain. Our purpose is to predict the absent R in T by leveraging R$ in S and the auxiliary representations in both S and T.
L2 Constraint
Wamn Irom
Auciiirry Represebition
xw
Gv
W
Embrdding Distribution Aignment Module
Shared
Distrtbution A Aignment
Colé
Ausiliory
x
Ge
Represettation
Figure 2: The Framework of DisAlign.

---

Then, we introduce the overview of our proposed DisAlign framework, as is illustrated in Figure DisAlign model mainly has two modules, i.c., rating prediction module and embedding distribution alignment module. To avoid the error superimposition problem [54), the rating prediction module in the source domain mainly provides end-to-end joint training of modelling user and item collaborative embeddings and matching the item collaborative embeddings with item auxiliary embeddings. The embedding distribution alignment module aligns the distributions between the warm and cold items across domains, minimizing the discrepancy between auxiliary latent feature embeddings in the source and target domains sufficiently. We will introduce these two modules in details later.
2.2 Rating prediction module
We first introduce the rating prediction module of DisAlign. For the i-th user and the j-th item in the source (warm) domain, we define their corresponding one-hot ID vectors as X!¹ and X respectively. For the j-th warm item, we also define its auxiliary representation as X The rating prediction module mainly has three purposes, including (1) exploiting user and item collaborative embeddings based on ratings, which is the prime purpose; (2) matching the item collaborative embeddings with item auxiliary embeddings; and (3) obtaining more discriminative item collaborative embeddings using unsupervised clustering method.
First, user and item collaborative embeddings should accurately represent the corresponding rating interactions. We obtain the user and item collaborative embeddings by Fu (xx) = U E R N
and Fv (x = V R Nx respectively. Here, Fú and Fv denote the user and item encoding networks respectively, N is batch size, and D is the dimension of collaborative embeddings. After that, we use pairwise ranking loss LBE based on metric-hased contrastive learning 10
exp(U Vi) Σ log exp(U, Vz) ex,(U, V.)
min LBF
(WEDD
where D :== V, RAIR > } denotes the original preference pairs 32 and (..-) denotes the inner product. The loss function LB can pull the positive items close and push the negative items away for a certain user according to his/her preference.
Second, item's 's collaborative embedding should be similar to its auxiliary embeddings avoid the error superimposition problem [54]. We utilize network Gy to translate the auxiliary representations into auxiliary embeddings Gv(X
[ W) W ERND, After it, the item embedding matching loss VI
is given by LBM W
Third, similar item collaborative embeddings should be clustered in order to obtain more discrimina- tive latent features. We adopt deep unsupervised K-Means clustering approach [24] 48) for this, and the corresponding loss is min =I LBK [r is the cluster indicator matrix and K denotes the cluster number.
Tr(FVV " FT)), where F e k
In summary, the loss of the rating prediction module is combination of the three losses, that is:
min LB LBE + NLBM + CLBK, FFT=I
(2)
where Ç and 1) represent the balance hyper- ppper-parameters The optimization procedure is given as below: (1) Fixing the other variables except F, we update F through min Tr Tr(FVVT (FVTFF FT)) with singular value decomposition algorithm; (2) Fixing F, we update other variables through gradient descent methods for several iterations then go back to step (1) until it convergences. For the sake of stability, practice, we update F every 15 iterations.
2.3
Embedding distribution alignment module
2.3.1
Overview
We then introduce the embedding distribution alignment module of DisAlign. We Gv C E NxD to denote the auxiliary embeddings of the cold items in the target domain. Specifically, Gr(-) is a two-stream siamese network with shared weights for encoding both warm item auxiliary representation X and cold item auxiliary representation xc We denote pw and pc as the warm and cold item auxiliary embedding probability distributions, respectively, and denote pv the warm
3

---

item collaborative embedding probability distribution. In CDCSR setting. pw # PC and pv # Pc- because the embeddings generated from the source (warm) domain and the target (cold) domain are heterogeneous, which leads to the domain discrepancy problem. Let us consider case where the source domain has user-book interactions while the target domain has user-movie interactions. Although books and movies share some similar characteristics, the auxiliary representations of the Book domain usually include authors and writing styles, while the auxiliary representations of the Movie domain include directors and actors, which brings discrepancy. Without embedding distribution alignment, recommender system may recommend horror books instead of history books to user who likes history movies rather than horror movies due to domain discrepancy, illustrated in the left of Figure 3 After alignment, the history movies/books and horror movies/books are aligned, as is shown in the right of Figure 3 and thus the recommender system can provide more reliable results. In order to reduce the distribution discrepancy between the source and target domains, introduce two approaches, i.e., Stein path alignment and Proxy stein path alignment.
2.3.2 Stein path alignment
As mentioned Section I since the latent em- beddings from two domains in CDCSR are scat- tered and complicated due to the fact that they may represent diverse information, previous do- main adaptation methods cannot be effectively utilized to solve the distribution discrepancy problem. Therefore, we propose a new distri- bution alignment approach, named Stein path alignment. Stein path alignment can prompt the target samples move the source domain through proper paths, according to the target intra-domain structure and the probability distribution of the source domain. Stein path alignment relies on Stein Variational Gradient Descent (SVGD) (21(24 variational inference method that starts from of initial particles and iteratively updates them with approximate steepest direction, whose main iteration process is:
(Warm timi Cold Ttem
History Movie
BomeMonn
History Book
Homee Book
User
Like
User
Like
Dislike
Dellkn
Before Aligmment
After Alimmmaat
Figure 3:
Demonstration of the necessity of em- bedding distribution alignment.
N
1 = 21,1-1 + c0p(Zzu-1), pp(z) = N Σ [12 23 V= logp(z) + V [=k(2,2))1
(3)
where Ii, denotes the i-th original target sample the (-th iteration, p(z) denotes the source probability distribution, denotes the step size, and k(æ, y) exp (-( c y //)²²) is the Gaussian kernel function with a denoting the bandwidth. Existing researches (2 7 have proved
that mean field theory can guarantee the rigorous theoretical convergence of SVGD, that is. the
gradient dynamics at particle level will approach to zero: lim (1/N Ei (z(.)) 0.
Stein path distance. We denote z the Stein mirror point of 21,0 when SVGD convergences the 1-th iteration. We propose Stein path distance as below:
N
N
Pr-s (
= N
-
||
Σ 2=0
+ c0s(Z z
(4)
N
1m0
Stein path distance quantifies the discrepancy between the source domain S and the target domain T by taking the average length of all paths from T to S through the t-th iteration. Stein path considers the source probability distribution and intra-domain structures, and thus can avoid negative transfer arisen from coarsely pairwise matching by traditional methods. Meanwhile Stein path is also explainable. Let 2g,0 denote the auxiliary embedding of i-th book in the Book domain, 2,,t could be taken similar movie in the Movie domain, e.g., the movie is based on the story of the book. The calculation of Stein path distance mainly has three steps. First, adopting kernel density estimation 30 [29] 39 with radial basis function kernel to estimate the probabilities of W and V. Second, finding the Stein mirror point of the cold item auxiliary embeddings through SVGD by Equation (3). Third, calculating the Stein path distance using Equation (4). The calculation details will be given in Appendix A.
Stein path loss. summary, the better the source and target domains are aligned, the smaller the Stein path distance. Therefore, we innovatively propose Stein path loss to align the cold item auxiliary

---

embedding C' with warm item collaborative preference V and auxiliary embedding W as below: min Lyp Pc-w (C) + Pc-v + PP Pc-v(C))l2
(5)
where the first two terms denote the Stein path distances from C' to W and C' to V, respectively, and the third term reinforces that these two distances should be similar.
2.3.3 Proxy Stein path alignment
Although Stein path alignment achieves satisfying performance, has scalability problem when facing large dataset. Because all the cold items in each batch need to be used for calculating Stein path distance. Therefore, is urgent to reduce the computation cost to accelerate the optimization process. To do this, propose proxy Stein path approach which only needs to choose the most typical cold item proxies to represent the global properties in order to speed up the aligmment process through SVGD. We now describe the main steps for finding the proxies and the optimization procedure.
Souroe Pribiiiity Didriebbo
Tagee Ieen
Paucy Targe Cold
Pracy Stai Matur
-.... Sten Pub
Fieding Pooces
Multiple-proxier algorithm. We first introduce highly efficient mudriple-proxies algorithm, which aims to find typical proxy ples in the target domain. Suppose there exists M E "x typical proxies for the cold item auxiliary embeddings C, where H denotes the number of proxies. Let V E TRNH be the simi- larity matrix between the cold items in the target domain and the proxies. Inspired by [ 28). we formulate the multiple-proxies optimization problem
Builing Putl
Fand Aligsnaan
Figure 4: The main procedures of proxy Stein path alignment.
N
H
N
H
min M. M.
Uig c. m,
Jhe log Vigs
(6)
i=1j=1
where c; denotes the i-th cold item auxiliary embedding and m, denotes the j-th corresponding proxy. The entropy norm regularization term Σ/ -i=1 Σラ= tiy log thy is set to avoid trivial solution with a denoting the regularization strength. Compared with the square norm || ll2, the entropy norm can not only obtain a nonnegative and nonlinearly representational similarity matrix but also reduce the computational cost []]. In summary, the multiple-proxies optimization algorithm has two main steps, i.e., (1) updating V, which has closed-form solution, and (2) updating M as m. E // Ex Wiy- The optimization could be done by repeating (1) and (2) until and M convergence. We will present the optimization details in Appendix A.2.
Proxy Stein path distance. After finding the typical proxies M and setting L as constant, propose proxy Stein path distance according to the original Stein path as below:
H
P7 (M)
1
Σ m H
1
m0,ol2
1 H
m + cōs(mi,.-)) m
where m denotes the i-th proxy m. the f-th iteration. Notably, each batch, proxy Stein path only needs to move the number of proxy samples (ff) in the target domain rather than the total number of samples (NV. Since H N, proxy Stein path can reduce the time consumption on calculating the Stein path distance.
Proxy Stein path loss. Similarly to the Stein path loss, the proxy Stein path loss is given by:
(8) The optimization of proxy Stein path alignment mainly has four steps. The first step is adopting the multiple-proxies algorithm to figure out the typical proxies M the target domain. The following three steps are similar as Stein path alignment mentioned in the Section 2.3.2 except that are moving proxies M rather than C. We will present the optimization details in Appendix A.3.
min Lesr PC--w (M) + Pa-v IPP Po-v PC-v(M1
Time complexity analysis. The time complexity of Stein path alignment is O(N31,), where i is the iteration number. The time complexities of the multiple-proxier algorithm and proxy Stein path
5

---

Table Experimental results on Douban and Amazon datasets.
(Douhan) Mavie- BBook
(Douhan)
(Donhan) Book->Mavis
(Amazon) Mavie - Music
HR
Recall
NDCG
IRR
Recall
NDCG
HI
Recall
NDCG
IR
Recall
NDCG
DropoutNet
28666
.1528
.0959
.893
.1896
.1134
2248
.0976
.053
.2911
.1463
.0786
LLAE
2914
.1744
.1778
.3055
.2039
.1788
.2511
.1104
0618
2243
.1504
0850
Heater
2983
.1816
.1135
.3233
.2104
.1310
.1263
.707
WCF
.028
.1920
.1266
.3376
.2177
.1385
.2704
.1385
07%
.2677
.1744
.1103
.3146
.1491
DARec
3139
.2149
.1356
.3350
.1166
.1899
.2749
.1407
0824
.2911
1910
1229
DisALiga-Base
2991
.1846
.1189
.258
.1088
.1324
.6177
.1766
0733
.2399
.1766
.054
DisAliga-SPUU
.375
.2333
.1466
36500
.2485
.1587
.3023
.1622
.979
.3022
.0066
.1303
DisAlign.SB SP
.1508
.2506
.0088
DiaAlign-PS
3401
2405
1482
.3795
.2528
.6233
.3022
.1711
.1076
.3881
.2276
.1395
(Douban) Book Music
(Douban) Music Movic
(Douban) Music Mook
(Amazon) Music Movic
NDCG
NDCG
DropoutNet
2584
.1215
.0686
2595
.3310
.0767
2632
.1196
0663
.2622
.1743
.1005
LLAE
2685
.1268
.0710
2635
.1424
.0819
.1245
06588
.2753
Heater
2724
.1289
.0742
.2701
.1422
.8333
.2834
.1342
.0333
28488
.1876
.1115
WCF
.2110
.8664
.1295
ESAM
2837
.1398
.8003
.8766
.1709
0935
.8688
.1486
0847
.3733
.2204
1415
DARee
2866
.1410
.0839
.2918
.663
.9166
.2917
.1409
.0811
.3113
.2933
AasAlign-Buse
2712
.1303
.0745
.2644
.1490
.US58
.2746
.1305
0697
.2133
1928
.1811
DisAlign SP()
.1107
SS
2983
.1581
.905
.3107
.1988
.1156
.3073
.1692
.1015
.3428
.2505
1609
DisAlign-PSP .3188
.1933
.0944
.3121
.1990
.1194
.3005
1644
0988
.3485
.2542
.1644
alignment are ONN Ht2) and Nt,) respectively, where t2 is the inner-loop iteration number. Therefore, the total time complexity of proxy Stein path aligmment is O(N Ht2) + O(H2Nt) O(NH12 + Nt,) Since < N, proxy Stein path alignment much cheaper than Stein path alignment. Empirically, we set H
2.4 Putting together
The total loss of DisAlign could be obtained by combining the losses of the rating prediction module and the embedding distribution alignment module. That is, the losses of DisAlign with Stein Path (DisAlign-SP) and Proxy Stein Path (DisAlign-PSP) are:
min DistlignuSP LB + AspLep, min Loiedlign-PSP LB + ApspLesps
where ASp and APSP are predict the missing rating in the target domain by taking the inner product of user embeddings U and cold item embeddings C.
to balance the two types of losses. In testing phase, one
Experiments
3.1 Experimental setup
Datasets and Tasks. We conduct extensive experiments on two popuarly used real-world datasets, i... Douban and Amazon. First, the Douban dataset (50) [T] has three domains, i.e., Book, Music, and Movie, which contains ratings, reviews, tags, and item details. There are six CDCSR tasks on Douban by randomly choosing two domains source domain and target domain respectively. Second. the Amazon dataset 44 27 has two domains, i.e., Movies and TV (Movie), and CDs and Vinyl (Music). There are two tasks on Amazon, i.e., Amazon Movie - Amazon Music and Amazon Music - Amazon Movie. For both datasets, we binarize the ratings to and 0. Specifically, we take the ratings higher or equal to 4 as positive and others as negative. We also filter the users and items with less than interactions, following existing research [4]] We list the detailed information on these datasets and tasks in Section B.1 of the supplementary material.
We compare our proposed DisAlign with the following state-of-the-an cold-start and CDR models. (1) DropoutNet (41 inputs both auxiliary representations and collaborative filtering representations and randomly dropouts pre-trained collaborative filtering representations for training.
Baselines.

---

(a) Heater
(b) ESAM
(c) DARec
(d) DisAlign-SP
Figure 5: The t-SNE visualization of Douban Movie-DDobba Book, where Douban Movie is the source domain with red dots and Douban Book is the target domain with blue dots.
(2) LLAE [8] introduces semantic auto-encoder using the idea of zero-shot learning to solve the cold-start recommendation problem. (3) Heater [54] the latest cold-start recommendation model which combines separate-training and joint-training framework to overcome the error superimposition issue. (4) WCF (26] is the first attempt apply Wasserstein distance optimal transport for item cold-start recommendation. (5) ESAM [4] adopts attribute correlation alignment to improve long- tail recommendation performance by suppressing inconsistent distribution between displayed and non-displayed items. (6) DARec [46] adopts adversarial training strategy to extract and transfer knowledge patterns for shared users across domains and achieves the state-of-the
performance
in CDR. For DropoutNet, LLAE, Heater, and WCF, we use the same setting as reported in their original papers. For DARec and ESAM, since they cannot be directly applied to cold-start tasks, we adopt the same rating prediction module as DisAlign. Note that, for fair comparison, all the models use the same types of data and pre-processing methods during experiments.
Implemented details. We provide the implemented details of our proposed model and baselines. The auxiliary representations for X W and X across domains include genres, themes, reviews, item profiles, etc. We split auxiliary representations into each word and adopt directional skip-gram [33] on Douban for Chinese words and apply Glove [31] on Amazon for English words to obtain the average feature representations with dimension Z 200. We use all the user-item rating interactions in the source domain, and all the items auxiliary representations in both the source domain and the target domain for training the model, following standard evaluation for unsupervised adaptation [1] For all the experiments, we perform five random experiments and report the average results. We choose Adam [6] as optimizer, and adopt Hit Rate@20 @2)), Recall and NDCG@20 43 as the ranking evaluation metrics.
Hyper-parameter settings. We set batch size N 256 for both the source and target domains. The latent embedding dimension is set to D = 128. For the rating prediction module, we set the balance parrmeerraetters 17 0.01 and S 0.01, and number of cluster K 5 for item unsupervised clustering. For the stein path alignment module, we set the moving step size as c = 0.01 and the kernel bandwidth as 0.5. For the proxy stein path alignment module, set a 0.1 and H = 64 according to Section 2.3.3 Finally, for the balance parameters, ASp and Apsp are first selected according accuracy Douban Movie + Douban Book and then fixed as the best values, i.e., Äsp XPSP = 0.5. Although there are many hyper-parameters, we first optimize the hyper-parametters of the rating prediction module, and then optimize the other ppper-paraeeters.
3.2 Recommendation performance
Results and discussion. The comparison results on Douban and Amazon datasets are shown in Table ! From it, we can find that (1) Although Heater can get better results on conventional cold-start problem, it cannot achieve satisfying solutions on CDCSR problem since it cannot reduce the discrepancy across domains. (2) WCF obtains better performance than Heater in some tasks, but optimal transport with Wasserstein distance is easily affected by noisy samples, resulting in the over-adaptation errors in boundaries and limiting the transportation results. (3) ESAM and ReC provide
alignment and adversarial training to match source and target domains, while such coarsely matching methods lead to limited prediction enhancement. (4) DisAligm. SP or DisAlign-PSI consistently achieves the best performance, which proves that Stein path alignment strategy can significantly improve the prediction accuracy. (5) MasAligg-PSS outperforms DisAlign-SP on several tasks, e.g., Music and Movie domains on both datasets, which demostrates
7

---

Source Itams (Movie)
DisAlign- Base on Target Items (Book)
DIsA DisA lign-SP SP on Target Items (Book)
THC
Source Tiems (Book)
Disk lignn Base on Target Items (Movie)
DisA LignSS on Target Tlems (Movie)
Case study on the recommendation task of Douban Movie Douban Book and Douban Movie. The left part are the user preferences in the source domain. The middle and right parts are the recommendation results of DisAlign-Ban and DisAlign-SP, respectively.
Figure 7:
that typical proxies can filter out the outliers and improve model robustness. Besides, we further investigate the time consumption of each model on different tasks, and report the results in Figure 6 From it, we find that DisAlign-SP is the slowest, because it has to transport the whole batchsize of samples from the target domain to the source domain. In contrast, DisAlignt ign-PSP is much faster than SisASi and also faster than WCF and DARec, since it only needs to transport typical target proxies.
Visualization. To show the feature transferability, we visual- ize the t-SNE embeddings [17] of the source item auxiliary embeddings (W) and the target item auxiliary embeddings (C). The results of Douban Movie->Doube
W.7 ESAM yo @Dalige-57 gesi
400
Book are shown in Figure [ From it, we can see that (1) Heater does not has the ability bridge the gap across different domains, and thus the embeddings are separated in source and target do- mains, shown in Figure a): (2) ESAM and DA Rec have the tendency to draw the source and target embeddings closer, while they still have certain distance, as shown in Figure 5)b) and Figure This indicates that they can only align the marginal probability distribution; (3) DisAlign-SP in Figure depicts that the embeddings trained through Stein path alignment achieves more closer gap between the source and target domains. The visualization on Amazon dataset shows similar result, and we present in Section B.2 of the supplementary material.
200
130
102
Doussan
Doutan
Aprispo
Amszzs
Figure 6: Comparison of running time on four tasks.
3.3 Analysis
Ablation study. To study how does each module of DisAlign contribute the final performance, we compare DisAlign ig with its several variants, including DisAlign-Base and DisAl i ign-SP(I). (1) DisAlign-Bir -Bse only consists of the rating prediction module with collaborative embeddings clustering. (2) Disslign-SSPP only aligns the warm item auxiliary embedding W and the cold item auxiliary embedding C. The comparison results shown in Table From it, we can observe that (1) DisAligg -n-gBase without the Stein path distribution alignment module cannot transfer knowledge from the source domain to the target domain, resulting in poor performance, (2) DisAlign-SPD achieves better performance than DisAlign MaAligg-BBse,, where we only align the distributions between W and C, and (3) By extra aligning C' with V. DisAl i SP can further promote the performance of DisAlign-SP(I, Overall, the above ablation study demonstrates that our proposed embedding distribution alignment module is effective in solving the CDCSR problem.
Case study. In order to illustrate the domain discrepancy problem mentioned in the Section 2.3 (Figure we visualize the cases on Douban Movie-Doubar Book and Douban Book-Doubal Movie. F ssowss the recommendation results. The left part is the user-item interactions in the source domain where the blue and red frames indicate users like or dislike the items respectively. The middle part denotes the top-5 recommendation results for the corresponding users based on DisAlign-
where we can see that these users will probably dislike the recommended items in the target domain due to the lack of embedding distribution alignment. After applying Stein path alignment on the right part, the recommender system can effectively improve the results. The results indirectly demonstrate that Stein path can properly translate the items across different domains according to the latent probability distribution.
8

---

Parameter sensitivity. We finally study the effects of hyper-paramene paraamtaessss on model performance. First, we vary Aşp and Äpsp in {0.1, 0.3, 0.5, 0.7, 1.3, 5, 10) and report the results in Section B.3 of the supplementary materials. From it, we see that, the embedding distribution alignment module cannot play a central role in the training process when Asr; Ärsp - O, bringing the discrepancy between the source and target domains. When Asp and APSP become too large, the embedding distribution alignment module will suppress the rating prediction module, which also decreases the recommendation results. The above results indicate that choosing the proper typer-parameters to balance the embedding distribution alignment loss and rating prediction loss can effectively improve the performance of DisAlign. Then, study the effect of embedding dimension (D) on DisAlign, and report the results in Section B.4 of the supplementary materials. We find that the recommendation accuracy of Dis Align-SP and DisAlign-Pa increase with D, indicating that larger embedding can represent user and item preferences more precisely.
4 Related work
Cold-start recommendation. Existing approaches on this are mainly of two types, i.e., separate- training and joint-training. The former models, e.g., LinMap [9] and DeepMusic [40], separate the learning of the collaborative filtering embeddings and auxiliary embeddings, and thus always lead to error superimposition [54]. The later jointly minimizes the recommendation error on the user-item interaction and transformation function, including DropoutNet [41], LLAE [18] and Heater [54]. Among them, Heater [54] is the state-of-the-ar model which integrates randomized training mechanism with mixture-of-experte to provide better performance. However, the above cold-start approaches cannot adjust to the CDCSR problem due to the discrepancy across different domains.
Cross domain recommendation (CDR), According to [2], existing CDR models have three main types, i.e., transfer-bul based methods, clustered-based methods, and multitask-basel methods. Transfer- learn a linear or nonlinear mapping function across domains. Some recent method [46] even adopts adversarial learning strategy obtain more reliable knowledge across domains with shared users. Clustered-basee methods 44 adopt co-clustering approach to learn cross-domain comprehensive embeddings by collectively leveraging single-domain and cross-domain information within a unified framework. Multi-task based methods [1] 50, 51 (3) enable dual knowledge transfer across domains by introducing shared connection modules in neural networks. Nevertheless, conventional CDR approaches cannot solve the CSCDR problem where user-item interaction data in absent in the target domain.
based methods 49 25
Domain adaptation. Existing works on this are mainly of three types, i.e., discrepaney-based methods, adversarial-base methods, and sample-bate -ased matching methods [55] 36). Discrepancy- based methods learn the -nnariaat embeddings by the adaptation layer for moment matching, e.g., Maximum Mean Discrepancy (MMD) [3], Correlation Alignment (CORAL) B4 35), and Center
Moment Discrepancy (CMD) 47 Recently, ESAM 4 extends CORAL by utilizing attribution alignment for domain adaptation in long-tail item recommendation.
Adversarial-basee methods
integrate domain discriminator for adversarial training, e.g., Domain Adversarial Neural Network (DANN) [ and Adversarial Discriminative Domain Adaptation (ADDA) [8]. Sample-based matching methods are mainly based on the optimal transport 6 45). which have the ability of encoding class-structure in distributions for minimizing the global transportation cost. However, the above approaches cannot work well in CDCSR setting, because the latent representations are always more scattered, complicated, and diverse.
Conclusion
In this paper, we propose Distribution Aligmment (DisAlign), which includes the rating prediction module and the embedding distribution alignment module, for solving the cross-domain cold-start recommendation problem. We innovatively propose Stein path alignment and proxy Stein path alignment for embedding alignment across domains. We also conducted extensive experiments to demonstrate the superior performance of our DisAlign model. the future, we plan to extend DisAlign to multi-domain cold-start recommendation tasks and conduct more comprehensive experiments on new datasets. We will also modify our model to adapt to the situation where there are also user-item interactions in the target domain.

---

6 Acknowledgments
This work was supported in part by the National Key R&D Program of China NN..2018YFB1400000 and sponsored by the CCF-AFSG Research Fund (No.RF20200005).
References
[1] Liang Bai and Jiye Liang. Sparse subspace clustering with entropy-norm. In International Conference on Machine Learning, pages 561-568. PMLR, 2020.
[2] G. Baxter. Book review: Probability and related topics in physical sciences. Bulletin of the American Mathematical Sociery, 66(6):472-44 476, 1960.
[3] Karsten M Borgwardt, Arthur Gretton, Malte J Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex J Smola. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics, 22(14)ee4--e5 -57, 2006.
[4] Zhihong Chen, Rong Xiao, Chenliang Li, Gangfeng Ye, Haochuan Sun, and Hongbo Deng. Esam: Discriminative domain adaptation with non-displayed items to improve long-tail perfor- mance. In SIGIR, pages 579-588, 2020.
[5] Maurizio Ferrari Dacrema, Iván Cantador, Ignacio Fernández- Tobias, Shlomo Berkovsky, and Paolo Cremonesi. Design and evaluation of cross-domain recommender systems.
[6] Bharath Bhushan Damodaran, Benjamin Kellenberger, Rémi Flamary, Devis Tuia, and Nicolas Courty. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 447-463, 2018.
[7] Pierre Del Moral. Mean field simulation for Monte Carlo integration. CRC press, 2013.
[8] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Nommin-adversarrinn training of neural networks. The journal of machine learning research, (7()):209--2030 2016.
[9] Zeno Gantner, Lucas Drumond, Christoph Freudenthaler, Steffen Rendle, and Lars Schmidt- Thieme. Learning caribute-tt-feattee mappings for cold-start recommendations. In 2010 IEEE International Conference Data Mining, pages 176 185, IEEE, 2010.
[10] Jacob Goldherger, Geoffrey E Hinton, Sam Roweis, and Russ R Salakhutdinov. Neighbourhood components analysis. Advances in neural information processing systems, 17:513-221 2004.
[11] Boging Gong, Kristen Grauman, and Fei Sha. Connecting the dots with landmarks: Diserimina- tively learning domain-invariant features for unsupervised domain adaptation. In International Conference on Machine Learning, pages 222-230. PMLR, 2013.
[12] Jyotirmoy Gope and Sanjay Kumar Jain. A survey on solving cold start problem in recommender systems. In 2017 International Conference on Computing, Communication and Automation (ICCCA), pages 133-138. IEEE, 2017.
[13] Jun Han and Qiang Liu. Stein variational gradient descent without gradient. In International Conference on Machine Learning, pages 1900-1908. PMLR, 2018.
[14] Cheng-Kang Hsich, Longqi Yang, Yin Cui, Tsung Yi Lin, Serge Belongie, and Deborah Estrin.
Collaborative metric learning. In WWW, pages 193-201, 2017.
[15] Guangneng Hu, Yu Zhang, and Qiang Yang. Conet: Collaborative cross networks for cross- domain recommendation. In Proceedings of the 27th ACM international conference on informa- tion and knowledge management, pages 667-676, 2018.
[16] D. Kingma and J. Ba. Adam: A method for stochastic optimization. Computer Science, 2014.
[17] Van Der Maaten Laurens and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(2605):257 2605. 2008
[18] Jingjing Li, Mengmeng Jing, Ke Lu, Lei Zhu, Yang Yang, and Zi Huang. From zero-shot learning to cold-start recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4189-4196, 2019.
[19] Pan Li and Alexander Tuzhilin. Ddtedr: Deep dual transfer eross domain recommendation. In WSDM, pages 331-339. 2020.
[20] Qiang Liu. Stein variational gradient descent as gradient flow. arXiv preprint a7Xi:: 2017.
[21] Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. arXiv preprint
2016.
10

---

[22] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International conference on machine learning, pages 97-105. PMLR, 2015.
[23] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. In Advances in Neural Information Processing Systems, pages 1647-1657, 2018.
[24] Qianli Ma, Jiawei Zheng, Sen Li, and Gary W Cottrell. Learning representations for time series clustering. Advances in neural information processing systems, 32:3781-3791, 2019.
[25] Tong Man. Huawei Shen, Xiaolong Jin, and Xueqi Cheng. Cross-domain recommendation: An embedding and mapping approach. In IJCAI. pages 2464-24700 2017.
[26] Yitong Meng. Xiao Yan, Weiwen Liu, Huanhuan Wu, and James Cheng. Wasserstein collabora- tive filtering for item cold-start recommendation. In Proceedings of the 28th ACM Conference
on User Modeling, Adaptation and Personalization, pages 318-322, 2020.
[27] Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly- labeled reviews and fine-grained aspects. In EMMN.P-JCCNIP,, pages 188-197, 2019.
[28] Feiping Nie, Xiaoqian Wang, and Heng Huang. Clustering and projected clustering with adaptive neighbors. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 14, page 977-986, 2014.
[29] Travis A O'Brien, William D Collins, Sara Rauscher, and Todd D Ringler. Reducing the computational cost of the eef using a nufft: A fast and objective probability density estimation method. Computational Statistics & Data Analysis, 79:222-253 234, 2014.
[30] Travis A O Brien, Karthik Kashinath, Nicholas R Cavanaugh, William D Collins, and John O'Brien. A fast and objective multidimensional kernel density estimation method: fastkde. Computational Statistics & Data Analysis, 101:148-160, 2016.
[31] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. volume 14, pages 1532-1543, 01 2014.
[32] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thiemn Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint /205:2//05.26111 2012. [33] Yan Song, Shuming Shi, Jing Li, and Haisong Zhang. Directional skip-gram: Explicitly distin- guishing left and right context for word embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume (Short Papers), pages 175-180, 2018.
[34] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European conference on computer vision, pages 443-450. Springer, 2016.
[35] B. Sun and K. Saenko. Deep coral: Correlation alignment for deep domain adaptation. Springer International Publishing, 2016.
[36] Chuanqi Tan, Fuchun Sun, Tao Kong, Wenchang Zhang, Chao Yang, and Chunfang Liu, A survey on deep transfer learning. In International conference on artificial neural networks, pages 270-279. Springer, 2018.
[37] Eric Tzeng. Judy Hoffman, Trevor Darrell, and Kate Saenko. Simultaneous deep transfer actoss domains and tasks. In Proceedings of the IEEE international conference on computer vision, pages 4068 4076, 2015.
[38] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In CVPR, pages 7167-7176, 2017.
[39] Evgeniya Ustinova and Victor Lempitsky. Learning deep embeddings with histogram loss. arXiv preprint arXiv: weXivv1611.000 1.1100022 2016.
[40] Aäron Van Den Oord, Sander Dieleman, and Benjamin Schrauwen. Deep content- based music recommendation. In NIPS, volume 26. Neural Information Processing Systems Foundation (NIPS), 2013.
[41] Maksims Volkovs, Guang Wei Yu, and Tomi poutanen. Dropoutnet: Addressing cold start in recommender systems. In NIPS, pages 4957-4966, 2017.
[42] D. Wang, Z. Tang, C. Bajaj, and Q. Liu. Stein variational gradient descent with matrix-valued kernels. Advances in neural information processing systems, 32:7834-7888 7844, 2019.
[43] Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pages 950-958, 2019.
11

---

[44] Yaqing Wang, Chumyan Feng, Caili Guo, Yunfei Chu, and Jenq-Neng Hwang. Solving the sparsity problem recommendations via cross-domain item embedding based on co-clustering. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pages 717-725, 2019.
[45] Renjun Xu, Pelen Liu, Liyan Wang, Chao Chen, and Jindong Wang. Reliable weighted optimal transport for unsupervised domain adaptation. In CVPR, pages 4394-4403, 2020.
[46] Feng Yuan, Lina Yao, and Boualem Benatallah. Darec: Deep domain adaptation for cross- domain recommendation via transferring rating patterns. arXiv preprint arXiv:/905, 10760, 2019.
[47] Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Platz Central moment discrepancy (emd) for domain-invarian representation leam- ing. arXiv preprint arXiv: / 77770002/118/// 1, 2017.
[48] Hongyuan Zha, Xiaofeng He, Chris Ding, Ming Gu, and Horst Simon. Spectral relaxation for k-means clustering. In NIPS, pages 1057-1064, 2001.
[49] Cheng Zhao, Chenliang Li, Rong Xiao, Hongbo Deng, and Aixin Sun. Catn: Cross-domain
recommendation for cold-start users via aspect transfer network. pages 229-238, 07 2020.
[50] Feng Zhu, Chaochao Chen, Yan Wang, Guanfeng Liu, and Xiaolin Zheng. Diedr: A framework for dual-target cross-domain recommendation. In CIKM, pages 1533-1542, 2019.
[51] Feng Zhu, Yan Wang, Chaochao Chen, Guanfeng Liu, and Xiaolin Zheng. A graphical and attentional framework for dual-target cross-domain recommendation. In IJCAI, pages 3001- 3008, 2020.
[52] Feng Zhu, Yan Wang, Chaochao Chen, Jun Zhou, Longfei Li, and Guanfeng Liu. Cross-domain recommendation: Challenges, progress, and prospects. In IJCAI, pages 4721-4728. ijcai.org, 2021.
[53] Feng Zhu, Yan Wang. Jun Zhou, Chaochao Chen, Longfei Li, and Guanfeng Liu. A uni- fied framework for cross-domale and cross-systen recommendations. IEEE Transactions on Knowledge and Data Engineering. 2021.
[54] Ziwei Zhu, Shahin Sefati, Parsa Saadatpanah, and James Caverlee. Recommendation for new users and new items via randomized training and mixture-of-experts transformation. In SIGIR, pages 1121-1130. 2020.
[55] Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongho Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong. and Qing He. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1):43-76, 2021.
12


Reviews:
[{"review_id": "rSFBT4OaflD", "reviewer": "Reviewer_NZWF", "summary": "Summary:\n\nThe paper addresses the cold-start item recommendation with auxiliary information. To attack the heterogeneity/ domain discrepancy across the source domain and target domain, Stein path is adopted to align the domain distributions. In specific, the distribution of target item auxiliary embedding is to be aligned with both source item auxiliary embedding and source item collaborative embedding. A proxy Stein path is proposed to reduce the cubic time complexity. Good experiments are achieved on real-world datasets by comparing with many SOTA methods.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strong points:\n\nS1: Achieving good experimental results, outperforming many SOTA methods.\n\n\nWeak points:\n\nW1: The technical contribution is limited. It might be novel in applying Stein path to the cold-start item recommendation with auxiliary information.\n\nW2: The motivation is not strong. What is the physical meaning of feeding cold item auxiliary embedding   C with warm item collaborative preference V and auxiliary embedding W to the Stein path?  \n\n\nOther comments:\n\nC1: The examples in Figure 1 and Figure 2 show that the auxiliary information seems to be Visual while it is Text in evaluation experiments.\n\nC2: In terms of Fig 1 and Fig 3, the source domain is Movie while the target domain is Book, while it is not consistent with the body text in Line 138. \n\nC3: in Fig 2, the L_BM should correspond to the L2 Constraint while the L_BK to Item Clustering. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "n/a", "review_text": "", "extra_scores": {"ethical_concerns": "no", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "afCIj_Jyrtl", "reviewer": "Reviewer_Dtrp", "summary": "In this paper, the authors study a relatively new problem called cross-domain cold-start recommendation (CDCSR), where the goal is to transfer knowledge from a warm source domain with user-item ratings and item descriptions to a cold target domain with item descriptions only (i.e., without any user-item interactions). Notice that the users in two domains are aligned. In particular, the authors propose a novel recommendation framework, i.e., DisAlign, which contains two main modules, i.e., a rating prediction module and an embedding distribution alignment module. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1 Originality: The studied problem, i.e., cross-domain cold-start recommendation (CDCSR), is interesting and challenging. The proposed method, i.e., DisAlign, is new since no previous work addresses the CDCSR problem by aligning the latent distributions using Stein path alignment or its improvement version proxy Stein path.\n\n2 Quality: The proposed model is technically sound, which is also supported by the empirical studies.\n\n3 Clarity: Overall, the paper is well presented and is easy to follow. The problem is clearly defined and the techniques are presented with sufficient details (some are in the appendix). Moreover, some illustraitons are included for better understanding, e.g., Figure 1 for the studied problem, Figure 2 fort the motivation of distribution alignment and Figure 3 of the proxy Stein path alignment.\n\n4 Significance: \nThe authors have conducted extensive empirical studies, including 8 cross-domain settings and 6 baseline methods, as well as ablation studies and case studies.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "TvrKTfA4pgm", "reviewer": "Reviewer_6Rcq", "summary": "This paper focuses on solving the cold-start issue in Cross-Domain Recommendation (CDR) setting. For this purpose, the authors propose a DisAlign framework, which mainly has two modules, i.e., rating prediction module and embedding distribution alignment module. The main novelty relies in the embedding distribution alignment module. In it, the authors propose Stein path alignment and proxy Stein path alignment. The experiments on benchmark datasets, compared with six baselines, show the superior of the proposed model. The experimental analysis also shows the effectiveness of the model ability. ", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- Originality: \nData sparsity is a long-standing problem in recommender systems. Solving this problem in CDR setting by proposing distribution alignment is novel to me. The proposed Stein path alignment and proxy Stein path alignment are novel to me, and the accuracy improvement brought by the proposed models is quite promising.\n\n- Quality: \n  This work is technically written with detailed algorithms. Most of the arguments presented in this work are supported by experiment results. \n\n- Clarity: \n  This paper is well-organized, and it clearly states the research problem in the introduction. The proposed model is well-motivated and clearly presented. \n\n- Significance: \n  To my knowledge, this paper is the first work solving the cold-start problem in CDR setting. The experiments show remarkable improvements (in terms of HR, Recall, and NDCG) over the prior models.  \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. This paper focuses on the setting where all the items are cold in the target domain, but most recommender systems consider some items as warm items and the remaining as cold items. Is this setting also suitable for the proposed method?\n2. What specific data pre-processing methods are used during experiments?\n3. Why DisAlign-SP outperforms DisAlign-PSP in some cases? \n", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7 hours"}, "extra_text": {}}, {"review_id": "QdL_bK290EK", "reviewer": "Reviewer_RKBg", "summary": "This paper proposes a framework for cross-domain item cold-start recommendation. The main idea follows the algorithm of Stein Variational Gradient Descent, by iteratively transporting the auxiliary embeddings of cold items in the target domain to match the embedding distribution of warm items in the source domain. The authors further propose to reduce computational time costs by selecting typical proxies of cold items.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength\n1. This paper is well-organized and mostly clearly written.\n2. It is technically sound to apply SVGD to transport the auxiliary embeddings of cold items to match the embedding distribution of warm items. \n3. It is novel to reduce the time cost of SVGD by finding typical proxy samples in the target domain.\n4. The experimental results are promising compared with existing methods.\n---\nWeakness\n1. It is not clear to me why the auxiliary features of items in different domains could be encoded with a shared network. For example, an item in the movie domain may have three feature fields: director, actor, category while an item in the book domain may have two different feature fields: author, book name. In this way, it doesn't make sense to use a shared network to encode their features. This may limit the applicability of the proposed method in real-world cross-domain recommendation systems. Besides, the so-called two-stream siamese network is not defined, which should be better demonstrated.\n2. In the Proxy Stein path alignment section, it is not shown how the embeddings of cold items are updated. Although I am able to understand the general procedure after reading the supplemental materials, it is still suggested to make it clearer in the paper.\n3. It is not shown why Equation (2) has to be solved through alternating optimization. There is no proof of the convergence of the alternating optimization procedure. Also, it is not reasonable to say updating F every 15 iterations could guarantee stability.\n4. Similar to 3, there is no proof of the convergence of the alternating optimization of multiple proxies in Equation (6).\n5. Although the proposed proxy approach can reduce the time cost of the original SVGD, the time complexity is still $O(N^3)$, since $H^*=[\\frac{N}{2}]$. Thus the proposed method may still hardly scale to industrial scenarios. Besides, it would be helpful to see how would the size of proxies affect the model performance.\n---\nAFTER REBUTTAL: The authors' responses addressed most of my concerns, and there are still some improvements to be made in the paper for better clarification. Thus I would like to increase my rating to above the threshold.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviews are quite mixed, and admittedly the one most positive review is somewhat of an outlier, though there's enough here to recommend acceptance. Still somewhat borderline given the two marginal scores (2x5), though with two strong voices for acceptance I'd lean positive.

In spite of the mixed scores, and the detailed rebuttal, opinions didn't really move during the discussion phase. So mostly my judgement is based on the initial scores, as well as what appears to be a reasonable rebuttal (even if it didn't move the needle on scores).

Reviewers were not totally aligned in their opinions. Some praised the originality and significance of the paper (especially the most positive review), while others found the technical contribution more limited. Other issues centered around lack of clarity w.r.t. various components. Ultimately the negative points seemed not to be deal-breakers and could be fixed easily.

Again, enough here to recommend acceptance, and nothing that seems like a red flag, though still somewhat borderline in terms of scores / distribution of scores.
Decision: Accept (Poster)