From Intervention to Domain Transportation: A Novel Perspective to Optimize Recommendation

1 INTRODUCTION . For information retrieval ( IR ) systems , the users ’ willingness to interact is often intervened by the content we show . Recommendations not only impact the potential response of the users , but also change the nature of the data collected for training machine learning models ( Bottou et al. , 2013 ) . The interventional nature of recommendation , as well as the fact that many questions in IR are counterfactual , e.g. , `` what the response would have been if we recommended something else '' , makes it natural to rephrase recommendation in the context of causal inference or missing data problems ( Rubin , 2005 ; Little et al. , 2002 ) . Among the two disciplines , technical tools of importance weighting ( IW ) and domain adaptation ( DA ) have been commonly applied . Recently , there has been widespread interest in adapting those tools for learning the best recommendation ( Schnabel et al. , 2016 ; Bonner and Vasile , 2018 ) , but we find a critical discussion missing from most existing literature in this direction . In many scientific disciplines , the effectiveness of IW and DA relies heavily on overlapping between source and target domain ( Austin and Stuart , 2015 ; David et al. , 2010 ) . The underlying reason also happens to be the motivation of exploration-exploitation techniques including bandits and reinforcement learning ( Auer et al. , 2002 ; Sutton and Barto , 1998 ) : in the less-explored regions where few observations are collected , the confidence of selecting any hypothesis is discounted . Unfortunately , modern IR systems are often cautious about exposing users to less relevant content for the sake of immediate satisfaction and revenue , even with the help of bandits or RL ( Bonner and Vasile , 2018 ) . In particular , many real-world recommenders are deterministic , e.g. , an item is shown with probability either zero or one , which further limits the exploration and thus the coverage of collected data . In Section 3 , we rigorously prove for both IW- and DA-based learning the hardness or impossibility results caused by insufficient overlap . Intuitively , when a less-explored instance is passed to a candidate hypothesis , even if DA or IW can improve the accuracy of the prediction , they inevitably increase the uncertainty of that prediction as a consequence of the bias-variance tradeoff . To a certain point , the increased uncertainty from having too many instances in the weakly overlapped regions will fundamentally limit what any learning procedure can achieve . Further , if the instance lies in the non-overlapped region , we can barely exceed regular extrapolation1 ( Xu et al. , 2021b ) . We resolve the fundamental limitation of insufficient overlapping in IR with the key insight that the interventional impact of recommendation is not exclusive to observed data , but also interferes with the target domain of interest — the instances that will be affected by the recommendations we are about to make . For instance , the placement of a recommendation on the webpage proposed to the user directly influences the occurrence of clicks . Therefore , we alternatively consider optimizing recommendation as searching for an intervention that best transports the patterns it learns from the source domain to its intervention domain . The procedure is best described by transport instead of transfer because it will interfere with how the learnt patterns are carried to the target domain . This change of view describes a complex but realistic working mechanism that involves learning , intervention , and transportation , as depicted in Figure 1 . On the one hand , we wish to learn patterns that can be transported to the target domain of interest . On the other hand , the patterns we identify lead to the intervention ( in the form of recommendation ) that generates the target domain . Following this working mechanism , we impose constraints on the domain transportation between the ( reweighted ) source domain and the future intervention domain . We point out that reweighting is essential for learning from interventional feedback data ( Rosenbaum and Rubin , 1983 ) , especially for the working mechanism we identify here because the relationship between the source and target domain is more involved . Towards this end , we propose a novel solution via transportation-constraint risk minimization . We rigorously investigate for our objective : • the consistency , generalization , and excessive risk bounds for the theoretical insights and guarantees ; • equivalence to a two-player minimax game efficiently solved by gradient descent-ascent algorithm ; • convergence to the standard IW-based solution for the ideal fully-overlapped bandit feedback ; We demonstrate the empirical performance of our method via comprehensive experiments via simulation , real-data analysis , as well as online experimentation and testing in a real-world IR system . 2 RELATED WORK . Bottou et al . ( 2013 ) first reveals the importance of characterizing the working mechanism of IR systems . However , they study specifically the advertising systems where the underlying causal structures are assumed given . Our work significantly extends their discussion by moving beyond causal mechanisms . Issues caused by ignoring the interventional nature of recommendation have been pointed out for numerous scenarios ( Chen et al. , 2020 ) . Steck ( 2010 ) and Liang et al . ( 2016 ) address optimizing recommendation in the context of missing data and causal inference , and the majority of recent follow-up work investigate particularly inverse propensity weighting ( IPW ) and learning domain invariant representation methods ( Chen et al. , 2020 ; Saito , 2020 ; Yang et al. , 2018 ; Joachims et al. , 2017 ; Bonner and Vasile , 2018 ; Agarwal et al. , 2019 ; Xu et al. , 2020 ) . As mentioned in Section 1 , they largely ignore the overlapping issue even when the recommendation is deterministic . Our theoretical results on the interplay between overlapping and learning performance are novel and add to the current venue of revealing the tradeoff for IW- and DA-based learning ( Byrd and Lipton , 2019 ; Johansson et al. , 2019 ; Ben-David and Urner , 2012 ) . See Appendix D for further discussions . Existing solutions for insufficient overlapping focus primarily on reducing or calibrating the variance , e.g. , as discussed in Swaminathan and Joachims ( 2015b ) ; Wang et al . ( 2020 ) ; Johansson et al . 1The presumption for this argument is that the underlying causal mechanism is unknown , which applies to the settings we discuss in the paper . ( 2020 ) . They do not account for the unique working mechanism of IR where the domain discrepancy can be actively controlled . We devise the domain-transportation constraint using the Wasserstein distance originated from the optimal transport theory ( Courty et al. , 2016 ; Redko et al. , 2017 ) . The idea of converting a Wasserstein-constraint objective to two-player minimax game is similar to that of the generative adversarial network ( Gulrajani et al. , 2017 ) , however , we do not make any generative assumption on the observed data . Finally , learning from bandit feedback and the associated counterfactual risk minimization can be viewed as a special case of our framework ( Swaminathan and Joachims , 2015a ) . We will use it as a test bed to examine the guarantees of the proposed approach . 3 PRELIMINARIES . We use upper-case letters to denote random variables and measures , and bold-font letters for vectors and matrices . We denote by ∥ · ∥2 , ∥ · ∥∞ and ∥ · ∥L the ℓ2 , ℓ∞ norm and the Lipschitz constant of a function . For clarity , we borrow the item recommendation setting to illustrate the IR system we discuss next . The set of all users and items are given by U and I . To be concise , we denote the features ( or pre-trained embeddings ) for users and items by xu for u ∈ U and xi for i ∈ I . We study primarily the practical implicit-feedback setting with Yui ∈ { 0 , 1 } , e.g. , users express preferences implicitly via clicks . Let D be the population of all possible ( xu , xi ) or ( u , i ) depending on the context . The embeddings can also be free parameters under the representation mapping f from X or U ∪ I to Rd . We let P be any base probability measure supported on the subsets of D , and let Pn be the empirical versions according to the collected feedback of Dn where n is the sample size . We assume the weight of a user-item is positive and bounded by a constant M . Equipped with either individual weights wui ∈ [ 0 , M ] or weighting function w : U × I ( or X × X ) → [ 0 , M ] , the reweighted measures are given by such as : Pw ( u , i ) ∝ w ( u , i ) · P ( u , i ) . The candidate recommendation algorithm f ∈ F outputs a score or probability for each user-item pair . Top-k recommender often ranks the output and then renders the top-k items to the user . We denote this procedure by the function : reco ( u , i ; f ) ∈ { 0 , 1 } . Define the new measure onD induced by deploying f on D as : Pf ( u , i ) ∝ reco ( u , i ; f ) · P ( u , i ) . If we choose the base measure P as uniform , it is clear that Pf is a uniform measure on the ( u , i ) pairs that will be exposed : { ( u , i ) ∈ D : reco ( u , i ; f ) = 1 } . Finally , given a loss function ℓ for f , we define the corresponding risk under P : EPR ( f ) = E ( u , i ) ∼P ℓ ( f ( u , i ) , yui ) , the weighted risk under Pw : EPwR ( f ) = E ( u , i ) ∼P ℓ ( f ( u , i ) , yui ) · w ( u , i ) . Our setting covers a wide range of recommendation problems from the classical collaborative filtering to sequential recommendation ( when user embedding is a function of the item sequence ) , as well as those using bandits for exploration where f ( u , i ) becomes the exploration probability , and the base measure is given by the inverse propensities . We defer the detail of that setting to Appendix A.1 . Consequence of Insufficient Overlapping . We now rigorously justify our previous intuition that insufficient overlapping causes uncertainty that fundamentally limits what IW- and DA-based learning can achieve . While the results we show for IW are universal , i.e. , it applies to any hypothesis class , we discuss particularly for DA the skip-gram negative sampling ( SGNS ) algorithm ( Mikolov et al. , 2013 ) . The reason is that SGNS is a widespread technique for learning user or item embedding in IR systems ( Xu et al. , 2021a ) , and its more straightforward formulation will allow us to present concise theoretical results to the readers . As a gentle introduction , IW-based learning uses the ratio Q ( u , i ) /P ( u , i ) – where P and Q represent the source and target domain – to reweigh the loss of each instance in order to correct the distribution shift . On the other hand , DA-based learning assumes there exists some hypothesis that makes the optimal joint risk of inff∈F [ EQR ( f ) + EPR ( f ) ] small . For IW , we show the hardness of learning , i.e. , the best target risk inff EQR ( f ) can be uncontrolled for any F due to insufficient overlapping . For DA , we show the impossibility of learning , i.e. , there does not exist a solution of SGNS that can make the optimal joint risk small when P and Q are not aligned . Proposition 1 . Let P and Q be the training and target distribution supported on SP , SQ ⊆ D , and w ( u , i ) = Q ( u , i ) P ( u , i ) , for ( u , i ) ∈ SP ∩ SQ . Dn consists of training instances sampled i.i.d from P . Hardness of learning with IW : if SP = SQ , for any f ∈ F , with probability at least 1− δ , it holds : EQR ( f ) ≲ EPn , wR ( f ) + M ( log 1δ + logN∞ ( 1 n , F ) n + √ Md1 ( P∥Q ) ( log 1δ + logN∞ ( 1 n , F ) n , and P ( ∣∣EQR ( f ) − EPn , wR ( f ) ∣∣ ≳ √ ( d1 ( P∥Q ) − 1 ) /n ) > 0 under mild condition , where M : = max { w ( u , i ) } , N∞ ( 1n , F ) : = N ( 1n , F , ℓ 2n ∞ ) is the 1 n -covering number for F in ∥ · ∥∞ based on 2n i.i.d samples from P , and d1 ( P∥Q ) = ∫ SQ ( dP/dQ ) dP is a divergence measure . Further , if SP ̸= SQ , for any prior and posterior distributions π and τ on F , for risks associated with the 0-1 loss , it holds with probability at least 1− δ : Ef∼τ , QR ( f ) ≲ Ef∼τ , Pn , wR ( f ) + dKL ( π∥τ ) + log d1 ( P∥Q ) δ n + Ef∼τ , SQ\SPR ( f ) +O ( Dn ( τ ) ) , where dKL is the Kullback-Leibler divergence , d1 ( P∥Q ) is the same as above but defined only on the overlapped part , and Dn ( τ ) consists of disagreement terms detailed in Appendix A.2 . Impossibility of learning for DA with SGNS : let R ( f ) be the associated risk for the SGNS algorithm , and Cu , i ( P ) , Cu , i ( Q ) be the co-occurrence statistics under P , Q , as mentioned in Xu et al . ( 2021a ) . Even if SP = SQ , it holds that : inf f [ EPR ( f ) + EQR ( f ) ] ≥ 1 |SP | ∑ ( u , i ) ∈SP dKL ( Cu , i ( P ) ∥Cu , i ( Q ) ) + c , where c takes some positive value and equals to inff EPR ( f ) + inff EQR ( f ) when P = Q a.s. We defer all the proofs to Appendix A.2 . For IW , we reveal the generalization bounds for both the frequentist setting ( the first set of bounds ) and the PAC-Bayesian setting ( the second bound ) . While they emphasize conceptually on different learning guarantees ( Germain et al. , 2009 ) , and while the latter can handle the existence of non-overlap ( SP ̸= SQ ) , we find them both showing insufficient overlapping ( characterized by the discrepancy terms d1 and dKL ) can cause arbitrarily large gaps leading to poor generalizations to the target domain . For DA with SGNS , our lower bound reveals the impact of insufficient overlapping indirectly via a notion of co-occurrence statistics defined in Xu et al . ( 2021a ) . In particular , if the source and target domains are well-aligned in terms of the co-occurrence statistics , the increased divergence term can make it impossible to find a hypothesis that simultaneously performs well in both domains . Although this result holds only for SGNS , it nevertheless suggests the limitations of DA caused by insufficient overlapping in IR .

Summary: In this paper, the authors first point out that the requirement of efficiency of IW- and DA-based methods is the overlapping between the source domain and target domain. Meanwhile, theoretical analysis also shows that insufficient overlapping problems can cause the hardness or impossibility of IW- and DA-based learning methods. In light of this, this paper aims to propose a novel method to optimize recommendation results by solving such insufficient overlapping problems. Specifically, a principled transportation-constraint risk minimization objective function is devised to optimize the recommendation results, which is able to transport the learned patterns from the source domain to the intervention domain better. Then a two-layer adversarial model (GDA) is deployed to optimize the transportation-constraint risk minimization objective function, and sufficient analysis of the GDA is provided. Extensive experiments were conducted on both real datasets and semi-synthetic datasets to show the superiority of the proposed method.