IEDR: A Context-aware Intrinsic and Extrinsic Disentangled Recommender System
Authors: Yixin Su
Wei Jiang
Yunxiang Zhao
Fangquan Lin
Cheng Yang
Sarah Monazam Erfani
Junhao Gan
Venue: Submitted to ICLR 2023 (2023)
Keywords: Recommender Systems
Intrinsic and Extrinsic Factors
Contrastive Learning
Disentangled Representation
Mutual Information

Abstract: Intrinsic and extrinsic factors jointly affect users' decisions in item selection (e.g., click, purchase). Intrinsic factors reveal users' real interests and are invariant in different contexts (e.g., time, weather), whereas extrinsic factors can change w.r.t. different contexts. Analyzing these two factors is an essential yet challenging task in recommender systems. However, in existing studies, factor analysis is either largely neglected, or designed for a specific context (e.g., the time context in sequential recommendation), which limits the applicability of such models. In this paper, we propose a generic model, IEDR, to learn intrinsic and extrinsic factors from various contexts for recommendation. IEDR contains two key components: a contrastive learning component, and a disentangling component. The two components collaboratively enable our model to learn context-invariant intrinsic factors and context-based extrinsic factors from all available contexts. Experimental results on real-world datasets demonstrate the effectiveness of our model in factor learning and impart a significant improvement in recommendation accuracy over the state-of-the-art methods.

Under review as a conference paper at ICLR 2023
IEDR: A CONTEXT-AWARE INTRINSIC AND EXTRIN- SIC DISENTANGLED RECOMMENDER SYSTEM
Anonymous authors
Paper under double-blind review
ABSTRACT
Intrinsic and extrinsic factors jointly affect users' decisions in item selection (e.g.. click, purchase). Intrinsic factors reveal users' real interests and are invariant in different contexts (e.g., time, weather), whereas extrinsic factors change w.r.t. different contexts. Analyzing these two factors is an essential yet challenging task in recommender systems. However, in existing studies, factor analysis is either largely neglected, or designed for specific context (c.g., the time context in se- quential recommendation), which limits the applicability of such models. In this paper, we propose generic model, IEDR, to learn intrinsic and extrinsic factors from various contexts for recommendation. IEDR contains two key components: a contrastive learning component, and disentangling component. The two com- ponents collaboratively enable our model to learn context-invariant intrinsic fac- tors and context-based extrinsic factors from all available contexts. Experimental results on world datasets demonstrate the effectiveness of our model in factor learning and impart significant improvement in recommendation accuracy over the state-of- state-of-the-art methods.
INTRODUCTION
Recommender systems aim to predict the probability of a user selecting a given item (e.g., click, purchase). This is a challenging prediction as each decision is jointly affected by multiple factors (Ma et al. (211)). Psychological research has revealed that users' decision making is mainly influ- enced by two factors: intrinsic and extrinsic factors (Bénabou & Tirole 2003 Vallerand| 1997). An intrinsic factor is an internal motivation for inherent satisfaction, which is often stable for an individ- ual. contrast, an extrinsic factor is a contextual motivation triggered by the environment (external stimulation), and it often varies among different contexts (e.g., weather, time) (Ryan & Deci 2000). For example, on a day with heavy rain, a user decides to take an Uber (a taxi calling app) to work. In this case, the choice of Uber over other taxi calling apps is because the user is more comfortable with this app's 's user interface (intrinsic factor), while the choice of taking a ride to work is motivated by the weather condition (extrinsic factor).
Although the importance of capturing these factors in recommender systems has been recognized, their full potential has not been explored by the existing works. (1) Some studies neglect the intrinsic and extrinsic factor disentangling, and the final prediction mainly relies on learning entangled repre-
sentations (Barkan & Koenigstein 2016 Covington et al., 2016 Wu et al. 2019). With the intrinsic and extrinsic factors ontangled behind each decision, the real factors that derive the decision may be incorrectly inferred, resulting in a suboptimal recommendation Wang et al. (2020). (2) Some studies leam intrinsic and extrinsic factors, but just under a specific context. For example, some sequential recommendation models leverage the time context (order sequence) to learn intrinsic and extrinsic factors (they call them long- and short-term interests) (Hidasi et al. 20166) Yu et al. 2019b); some point-of-interes recommendation models leverage the spatial context (geometric distance) to learn the two factors (Li et al.) 2017 Wu et al. 2020). In such models, the factor learning approaches are dommin-specific, so it would be difficult to generalize them to other contexts. Meanwhile, the factors may be influenced by multiple contexts. Hence, focusing in inferior factor learning. Therefore, is still an open question of how to effectively incorporate various context information for learning intrinsic and extrinsic factors in recommender systems.
single context may result

---

Under review as a conference paper at ICLR 2023
Focusing this question, propose generic recommendation framework that can learn intrin- sic and extrinsic factors from various contexts. We first formally define context-agnostic intrin- sic and extrinsic factors for recommendation tasks. Following these definitions, propose an Intrinsic-Extrinss Disentangled Recommendation (IEDR) model, which contains two modules: a recommendation prediction (RP) module, and contrastive intrinsic-extrinsic disentangling (CIED) module. For each user-item interaction, the PR module constructs all the contexts as a graph (con- text graph), and the context representation is obtained via learning the graph. The same procedures are done to obtain the user and item representations from their attributes (e.g., user gender, item cat- egory), respectively. Then, the intrinsic and extrinsic factors are learned from these representations for the user and the item perspectives. Meanwhile, the CIED module contains two components: contrastive learning component that learns a context-invariant intrinsic factor, and disentangling component that disentangles the intrinsic and extrinsic factors via mutual information minimization. The two components jointly ensure IEDR to learn intrinsic and extrinsic factors.
In this paper, we made the following contributions:
To better analyze the factors influencing users' decisions, we formalize the context-agnostic intrin- sic and extrinsic factors for recommender systems. Following these definitions, we propose IEDR to learn disentangled intrinsic and extrinsic factors from various contexts for recommendation.
IEDR comprises a context-ing inntrriannvvriant contrastive learning component, and a mutual information minimization-bart disentangling component to effectively disentangle the learned factors.
Extensive experiments on real-world datasets show that (1) IEDR significantly outperforms the state-of-the-ard baselines when various contexts are available; (2) our proposed CIED module can successfully learn intrinsic and extrinsic factors.
2 RELATED WORK
This section summarizes the current research progress recommender systems and contrastive learning related to our work.
Feature interaction modeling Many recommender systems leverage feature interactions to im- prove prediction accuracy. One of the most common techniques is the factorization machine (FM) (Rendle (2100), which models feature interactions through dot product and achieves great success. Recent studies extend FM with deep neural networks for more powerful feature interaction mod- eling (Xiao al. 2017: He & Chua 2017: Yu et al. 2019a). The Wide & Deep model (WDL.) (Cheng et al. (016) proposes framework that combines shallow and deep modeling of features for recommendation. Guo et al. (2017) combine FM and WDL by replacing the shallow part of WDL with an FM model. Su et al. (202T leverage the relation reasoning power of graph neural networks for feature interaction modeling. However, these models do not incorporate context information for better factor analysis, and we overcome this issue by leveraging this information to disentangle and learn intrinsic and extrinsic factors for recommendation.
Factor disentanglement Intrinsic and extrinsic factors considered two basic factors for individual decision making in psychological research (Ryan & Deci 2000 Bénabou & Tirole 2003) Vallerand 1997). Recent recommender systems have borrowed the idea of capturing these two factors to achieve more accurate recommendation. For example, in sequential recommendation, Hidasi et al. (0016) are the first leverage the recurrent neural networks to capture users' long- and short-term (LS-term) interests from their interacted item sequences. Yu et al. (20195) propose a time-aware controller to capture the differences between LS-term interests for more accurate interest learning. Zheng et al.] (2022) further emphasize the disentanglement between the LS-term interests different time scales to differentiate the I.S-term interests. In point-of-interest recommendation, studies are leveraging spatial context to capture the intrinsic and extrinsic factors (Li et al.) 2017] Wu et al., 2020). However, all of the above studies focus on specific contexts. As result, their factor learning approaches are hard to apply other recommendation domains, which may result in a suboptimal solution if other contexts jointly influence these factors. Some studies learn users' multiple factors without knowing the meaning of each factor (i.e., implicit factor). They first define the number factors (e.g., 4) to be learned, and then disentangle the representations of each pair of factors (Ma al. 2019 Wang et al. 2020). These models only ensure that the learned factor
2

---

Under review as a conference paper at ICLR 2023
representations disentangled, but cannot guarantee whether they really refer to important factors. Our IEDR model incorporates various contexts for explicit intrinsic and extrinsic factor learning.
Contrastive learning Contrastive learning has achieved great success in computer vision CChen
et al., 2020; Chuang et al.] 2020 Khosla et al. 2020] Tian et al., 2020) Chen & Hej [021] and
neural language processing (Oord al. 2018 Yang al., 2019 Gao et al., 202 Gunel et al.
2021
Recently, contrastive learning has attracted attention in recommender systems. Yao et al.
(202
conduct contrastive learning users and items respectively on a two-tower framework to
learn robust user and item representations. In addition, Wu et al. (2021) propose contrastive learn- ing framework on user-item bipartite graph to capture robust high-degree relationships between users and items. Lin et al. (202T) and Jiang et al.] (202T) leverage contrastive learning to eliminate popularity bias. We propose the method that learns intrinsic factor representations that are invariant to context through contrastive learning approach.
3 PROBLEM STATEMENT AND DEFINITIONS
Let U, V, and C. denote the user set, item set, and context set, respectively. Each user u E U consists set of user features u {=f, zi ... 2p3 (e.g., user ID, gender). Similarly, each item v V is represented by a of item features t = {=ī, z8; (e.g., branch, color). A context E C is set of context features {=f, 2,,..., }. denoting the context state when (e.g., weather, daytime). Let D be a dataset containing N instances (i.e., data samples) of (u, U, c), with an corresponding label y E {1, 0} indicating whether not the user selects the item under the context c. The recommendation task can be formulated as predicting the selection probability y' p(u, v, c). our proposed IEDR model, the intrinsic factor Oin and the extrinsic factor Oaz are explicitly inferred for both users and items, and jointly leveraged to perform the prediction.
selects an item
Next, we formally define intrinsic and extrinsic factors. We believe these two factors exist from both users' and items' perspectives. This is reasonable since a user selecting an item not only relates to the factors (motivations) of users, e.g., preferring a ride (intrinsic factor) over walking work on rainy day (extrinsic factor), but also relates to the factors (attractiveness) of items, e.g., Taxi calling App with comfortable user interface (intrinsic factor) and has a discount (extrinsic factor). In the following, we define intrinsic and extrinsic factors from the users' perspective only, they are similar from the items' perspective.
Definition 1. (Intrinsic Factor and Extrinsic Factor) Consider a user and a set of contexts C; an intrinsic factor of the user u is factor that is invariant to the contexts in C, Le., fin(u,c) c) Sin fin(u, e²), where I'm is a function learning intrinsic factor representations, and and d' are two arbitrary contexts in C. On the other hand, an extrinsic factor of the user u is a factor that changes w.r.t the context, i.e. there exist contexts and c' in C such that fea u, c) fea d'), where fex learns extrinsic factor representations.
4 INTRINSIC-EXTRINE
DISENTANGLED RECOMMENDATION MODEL
The overview of our model is visualized in Figure I More specifically, our proposed IEDR model consists of the following two modules, which will be detailed in the next subsections:
A recommendation prediction (RP) module that takes a user and an item as input, and combines them with set of contexts, generate intrinsic and extrinsic factor representations for both the user and the item. The predicted probability y' is then jointly learned from these representations, contrastive intrinsic-extrinsic disentangled (CTED) module is applied to both the user and the item sides to support the intrinsic and extrinsic factor learning. The module contains a context- invariant contrastive learning component and a disentangling component, to ensure the learned factors satisfy Definition
4.1 THE RECOMMENDATION PREDICTION MODULE
The recommendation prediction (RP) module is a symmetric structure that generates user intrinsic and extrinsic factor representations (o), o² from the user side, and generates item intrinsic and
3

---

Under review as a conference paper at ICLR 2023
CIED (user side)
CIED (item side)
Disentunglement
Disentanglement
CgP
Liv
0.x
Di
Contrastive eamning
Contrnstive Leaming
Ceici"
Uj
Side
An Overview of IEDR. It is a symmetric structure on the user side and the item side. The middle part (the black arrows) represents the recommendation prediction (RP) module (Section [.T). It generates the intrinsic and extrinsic factor representations (0i,. and O...) for producing the recommendation prediction y'- The side parts are two contrastive intrinsic-extrinsi disentanglement (CIED) modules. Each CIED includes a context-invariant contrastive learning component (the red arrows, Section 4.2.1). and disentangling compo- nent (the blue arrows, Section 4.2.2 to ensure the success of the factor learning. The losses generated through these modules
Figure 1:
api DDss) will be optimized two-step multi-task training (Section 4.3)
extrinsic factor representations (o oz.) from the item side. On the user side, we first generate a user representation and context representation based on user features and context features, respec- tively. Here we use the SIGN model (Su et al. 2021) to generate the representations. SIGN has been proven effective in user/item/context representation learning through modeling feature interactions via graph neural networks. Appendix A..] provides a detailed description of SIGN. More formally, let fu(u): : Roxd IRd be the function for SIGN-based feature modeling. fu(u) first maps each user feature 2/ E into a dddimensional feature embedding z"" Then, it models these feature embeddings to output the user representation u. Similarly, SIGN learns context representation c through fe- Next, factor generation function (u, c) 22xd + R2 (c.g.. neural network) takes the user representation and the context representation as input, and simultancously generates user intrinsic representation o,, and user extrinsic representations o Here, the output is 2d- dimensional vector, with the first d-dimensional terms as o and the rest as o On the item side, a similar module structure is adopted. We use different SIGN-based function for the item represen-
tation learning v fu(v), while using the same context representation as that on the user side. A factor generating function fi (v, e) is applied to obtain the item intrinsic factor representation 0% and extrinsic factor representation o....
Finally, we jointly learn the prediction V/ fpred 0 o )zz)). We linearly combine the intrinsic and extrinsic representations and use the dot product as the prediction function: Fpred (0% o , 0²2z) (o%, + ==z) )T=) To", o²z). A cross-entropy loss function is adopted to minimize the prediction error: LRP( u, = log('') + + y) log(1 - v').
4.2 THE CONTRASTIVE INTRINSIC-EXTRINES DISENTANGLING MODULE
While the RP module can generate factor representations, solely using this module cannot correctly distinguish intrinsic representations from extrinsic ones. To address this, we propose a contrastive marinsic-exttinsiti disentangled (CIED) module and apply it to both the user and the item sides. In the following, we only describe the CIED on the user side, as the module on the item side has the same structure. CIED consists of a component. The contrastive learning component learns intrinsic representations that are invariant in different contexts, while the disentangling component leverages a mutual information minimization task to disentangle the intrinsic and extrinsic representations. the following, we describe these two components in detail.
contrustive learning componen/ and disentangling
4.2. THE INVARIANT CONTRASTIVE LEARNING COMPONENT
We propose inntext-invariant contrastive learning component to learn the intrinsic representations. More specifically, we maximize the agreement between the intrinsic representation pairs generated from the same user under different contexts (positive pairs), and minimize the agreement between the
4

---

Under review as a conference paper at ICLR 2023
intrinsic representation pairs generated from the same context with different users (negative pairs) the same time. More formally, we represent the intrinsic representations with the subscript (o ):y if it generated through user u, (from i-th data sample) and context cj (from j-th data sample), i.e., (om)) = ft (u; e;). For the i-th data sample (u4, 04, Gi) E 'D we calculate the objective function based on InfoNCE (Oord et al. 2018):
(u, ci) == log exp (sim)(((()) )i, (15))))
Σn,er exp (( (0% (+))))
where oj is an arbitrary context, sim(-) is the cosine similarity, and T is a temperature value.
Implementation. To optimize Equation ( we need to generate ej (c, + oj). We adopt simple method to perform random sampling from the contexts within the same batch. Also, inspired by (Gao et al. 202T). where vector with large dropout rate can be considered as new vector, propose a dropout-based method to generate new context representations (e.g.. with the dropout integrate the two methods to generate different contexts oj
rate larger than 50%). In practice, for each c; (we empirically show in Appendix G.4 that the integrated generating method results in better prediction accuracies than either methods). Meanwhile, we need to iterate over all the users to generate (o ): (the intrinsic representations generated from an arbitrary user the and context c,), which is prohibitive when the users' number is large. Here, we randomly sample L users from the same batch to generate the negative intrinsic representations (o" )as where { 112.... ....... L. We use the categorical cross-entropy to optimize Equation (T) following (Oord al. 2018).
4.2.2 THE DISENTANGLING COMPONENT
We then perform an intrinsic-extrinst factor disentangling via minimizing the mutual information between the on and 0 generated from f (u, c). Inspired by vCLUB (Cheng et al. 2020), we minimize their mutual information by estimating vCLUB- upper bound. However, the asym- metric property of the original vCLUB may lead to a less robust and inferior intrinsic-extrinsee disen- tangling (further discussions on this drawback of vCLUB can be found in Appendix [D). Therefore, we propose a simple yet effective bidirectional extension of vCLUB for symmetry, which is more robust and achieves better disentanglement In the bidirectional vCLUB, two variational distribu- |=
tions (e.g., approximated via neural networks) of' (o" 0 0T) and 9 (o%,,o" lo" 0%) are proposed with parameters 0r and 0% to predict the two types of factors, respectively. Then a bidirectional vCLUB-based mutual information upper bound can be obtained as
TAwCCLUB CoALuu((oo (O )²ee) = [Epio oz [log oi (o. Epio Inton [log qi' (o oo..)]
+ E Epto oy [log 0² (o )o Epcog, v [log z (o lonlo"))))
Through minimizing the bidirectional upper bound IR-VCLUB (o% %% as above, we minimize the mutual information between oh, and o Experimental results in Section 5.2.2 show that vCLUB more robust and achieves better factor learning.
The optimization of the disentangling component is conducted in two steps iter- atively. In the first step, we estimate the upper bound by training 0% and 03 to minimize the loss (a
Implementation.
function Cia- c)) == - log 인ी' ((l")))l((")))
+ log q ( ((o )a )ot. Following
Cheng et al. 2020p, we use the mean squared error to optimize of and q/- In the second step, we freeze 0f and 0%. and minimize the mutual information of 0%, and 0% by training other parameters to minimize the upper bound Lbs (u,, G3) VCLUB ((o)) )ae Dasi (0%))))
4.3 MULTI-TASK TRAINING
We perform two-step multi-task training to minimize the empirical risk of IEDR. The two steps run alternatively until convergence. Appendix provides the pseudo-code of the training procedure. In the first step, freeze all the parameters except for 0% 0% 0% and 0% where 0; 0 the parameters of (o"
2 lo 8 and q (o", lo..: 0%) in the disentangling component on the item side.
'IX-CCLUD (0 oz2) is the average of two vCLUB based upper bounds of different directions. Therefore, it obvious that CLUD (o o% is still an upper bound of I(o", or.).
5

---

Under review as a conference paper at ICLR 2023
oppr (U,, ci)) In the second step, we freeze O Ozz 0r, and 02 and minimize the following function:
07 We then minimize R(0 00......... 87, 07,05) Ei= appr (u, ci) + Cpi
arR min R(w)
CRP(W, o.. An (FCICL (erCiicCee,nn Ccicu (et, v.) (CB CDi (vu.ee)))
where bii-appr CCILL and Lis are the losses the item side, X and X2 the weight factors, and w are all the trainable parameters except for 0f. 02 0i and 0z.
4.4 THRORETICAL ANALYSIS: CONTEXT-NVARIANT CONTRASTIVE LEARNING IN INFORMATION THEORY
In this section, we reason the context-invariant contrastive learning from the perspective of informa- tion theory. As formally defined in Theorem [ optimizing Equation (T is equivalent to maximizing the mutual information between the intrinsic representations and user representations, and simul- taneously minimizing the mutual information between the intrinsic representations and the context representations. The theorem the item side can be derived in the same fashion. The proof of this equivalence can be found in Appendix
Theorem (Equivalence of contrastive loss Lere.). Optimizing the contrastive loss is equivalent to solving:
Cicel (u;, = arg max (I(Oī u) II(o e))
arg min
(4)
EXPERIMENTS
We conduct extensive experiments to demonstrate the effectiveness of our model. this section, we focus on 1) the recommendation performance of IEDR compared the state-of-the-an methods; 2) the effectiveness of each component in IEDR; and 3) the ability to disentangle intrinsic and extrinsic factors of IEDR. We discuss the datasets, baselines, implementations, and parameter settings in detail in Appendix F
5.1 OVERALL PERFORMANCE
We evaluate the recommendation performance of model, by comparing with various baselines in two scenarios. the first scenario, we learn intrinsic and extrinsic factors from various contexts. In the second scenario, learn the factors from a specific (time) context and compare with sequen- tial recommendation baselines that also learn intrinsic and extrinsic factors (i.e., long and short term interests). We use three common evaluation metries for recommender systems: AUC, NDCG@A k, and HR @ le with k being 5 and 10.
5.1.1 FACTOR LEARNING FROM VARIOUS CONTEXTS
We run our model and feature interaction- based baselines on the datasets that contain various con- texts: the Frappe dataset [Baltrunas et al., 2015) for app recommendation, and the Yelp dataset (Wul
al. 2022) for restaurant recommendation. As our baselines,
feature interaction-based ap-
proaches (Xiao et al. 2017 He & Chua 2017 Song et al. 2019 Guo et al., 2017 Cheng et al., 2016 Wang et al. 2021] I Yu et al. 2019a Su et al. (021) that capture these contexts neglect the
factor learning. We also compare with implicit factor disentanglement methods DisRec (Ma et al.) 2019) and DGCF (Wang al. 2020). The performance results are reported in Table Fach result the average of 10 times run. For each metric, the best results are in bold, and the best baseline results are underlined. The rows Improv (standing for Improvements) and p-value show the improvement and statistical significance (via the Wilcoxon signed-rank test (Wilcoxon 1992)) of IEDR over the best baseline results, respectively.
From Table I observe that model significantly outperforms all baselines under all evalua- tion metrics, and the p-values are all less than the 5% threshold, indicating the significance of the improvements. IEDR outperforms feature interaction-based baselines because IEDR captures the

---

Under review as a conference paper at ICLR 2023
Table 1: Comparing the prediction performance (in percentage) with the baselines. The Improv and p-value rows show the relative improvements and the statistical significance of IEDR over the best performed baselines, respectively. N@k refers to NDCG@k and H@k refers to HR@k.
Frappe
Yelp
AUC
N@5
N@ 10
H@S
H@ 10
AUC
N@5
N@ 10
H@S
H@ 10
AFM
93.18
63.52
67.44
77.84
84.71
91.96
42.79
47.17
58.69
72.21
NFM
95.86
68.30
70.73
83.00
90.40
93.32
45,99
50.33
61.90
75.27
AutoInt
95.83
69.45
71.41
84.04
90.10
93.82
46.61
50.80
63.72
76.55
DeepPM
96.09
69.20
71.28
82.70
89.50
93.26
44.20
48.50
60.26
73.55
WDL
95.96
68.02
70.33
81.70
88.90
93.41
45.47
49.71
61.90
74.89
DCNv2
95.25
68.15
70.34
82.15
89.91
93.66
43,41
48.26
60.97
74,88
IFM
95.32
66.91
69.13
80.90
87.60
93.83
46.74
50.86
63.04
75,69
SIGN
95,92
69.38
71.49
83,91
90.37
93.67
46.80
50,94
63.68
76.41
DisRec
85,51
56.81
60.07
67.42
76.29
84.01
34.82
37.90
48,29
63.17
DGCF
86.13
58.40
61.44
69.05
77.53
85.29
36.35
39.06
50.05
64.62
IEDR
96.34
72.40
74.11
85.94
91,25
94.22
48.68
53.05
65.23
78.29
Improv 0.26%
4.24%
3.66%
2.26%
0.94%
0.42%
4.01%
4.14%
2.38%
2.28%
p-value 3.72%
0.25%
0.25%
0.25%
0.83%
2.34%
0.25%
0.25%
0.25%
0.25%
intrinsic and extrinsic factors, while these baselines neglect the factor learning. Meanwhile, the im- plicit factor disentanglement methods (DisRec and DGCF) also perform inferior to our model. One reason is that implicit factor disentanglement not the best way to infer these factors. Section 5.2 we empirically verify that replacing the disentanglement module in IEDR with an implicit ap- proach (as in DisRec and DGCF) leads to a decrease in recommendation accuracy. Another reason that the factor disentanglement of DisRec and DGCF purely based on user-item interactions, and do not consider context information. This may lead to critical information loss for recommen- dation. Our model explores context information learn disentangled intrinsic and extrinsic factors for recommendation, and hence achieves better prediction accuracy.
5.1.2 FACTOR LEARNING FROM SPECIFIC CONTEXT We then evaluate IEDR two Amazon datasets (Movies and CDs) (McAuley et al., 2015) that contain only the time context. TEDR takes the (bucketed) time context as features to learn in- trinsic and extrinsic factors. We compare with the state-of-the-art sequential recommendation baselines (Hidasi et al., 2016 Yu et al. 2019b; Zheng et al. 2022) that learn LS-term interests from the item sequences ordered by the time. The experimental results are reported in Table 2, where our model achieves competitive accu- racy compared the baselines. This proves the ability of our model to achieve state-of-the-art recommendation accuracy in the context-specific scenario, even compared with the models designed for the context. Moreover, our IEDR is more versatile and can be applied to various contexts.
Table 2: Comparing the performance IEDR and the baselines on time context-specific scenario.
Movies
CDs
AUC
N@ 10
AUC
N@ 10
GRU4Rec
77.11
25.18
78.86
19.41
SLI-Rec
78.69
26.85
79.37
20.27
CLSR
80.02
26.98
80.42
21.07
IEDR
80.14
26.68
80.34
20.95
5.2 EFFECTIVENESS OF OUR MODEL'S COMPONENTS
This section evaluates the components in IEDR in detail. We demonstrate the results in NDCG@ 10 due to the space limit. Other metrics show similar trends.
5.2.1
ABLATION STUDY CONTRASTIVE CHRRNSSI-EXTTRISIC DISENTANGLING MODULE
The contrastive entrinsic-extrinsic disentangling (CIED) module contains a context-invariant trastive learning component and a disentangling component. In this section, we conduct an ablation study show the impact of these components. We model in three variations: 1) without the

---

Under review as a conference paper at ICLR 2023
Frappe
Yelp
Frappe
Yelp
74
53
72
52
52
51
nxCTED
IEDR
noC1. noCTED
VCLUB
VCLUB Disenlangling method
Components
Components
Disentangling method
(a)
(b)
Figure 2: (a) Ablation studies with different component(s) removed. (b) The performance and variance statis- ties of vCLUB and BiDis. The vertical axis NDCG@ 10.
contrastive learning component (noCL); 2) without the disentangling component (noDis); 3) with- the contrastive learning and disentangling components (noCIED), i.e., the CIED module is not applied. Figure 2a compares our IEDR model with these three variants. The inferior performance of noCIED compared with our full model IEDR demonstrates the importance of learning intrinsic and extrinsic factors for accurate recommendation prediction. noCI. can be regarded as performing implicit factor learning. The inferior performance of noC1. indicates that explicit intrinsic and ex- trinsic factor learning superior to implicit factor learning methods (as in the factor disentangling baselines) inferring the real reason for users' decisions. noDis learns intrinsic factors but does not guarantee the extrinsic factor learning. Therefore, it also obtains worse results than CIED. Em- ploying both components achieves better performance than learning with only one. This is because either component cannot individually learn intrinsic and extrinsic factors successfully, highlighting the importance of learning the two factors for an accurate recommendation.
5.2.2 DISENTANGLING COMPONENT EVALUATION
We propose a bidirectional -CLUB-based disentangling method (BiDis) that disentangles the intrin- sic and extrinsic factors. In this section, we compare our BiDis method with the original vCLUB method (vCLUB). From Figure 2b) we observe that our BiDis achieves a better performance than vCLUB, and its variance is much smaller than vCLUB. This indicates that BiDis generates more robust predictions, which is consistent with our analysis in Section 4.2.2 In addition, we visualize the intrinsic and extrinsic representations learned by the disentanglement methods with t-SNE in Appendix G.7, and observe that using BiDis results in better intrinsic-extrinsic disentanglement. This is the reason why BiDis delivers a better performance than vCI.UB.
5.3 DISENTANGLEMENT VERIFICATION
This section verifies the intrinsic and extrinsic factor disentangling ability IEDR, including visualization of the learned intrinsic and extrinsic representations and case study to show the differences of these factors in users' maaksi.n-making.
5.3.1 VISUALIZATION OF INTRINSIC AND EXTRINSIC REPRESENTATION.
In Figure we visualize intrinsic and extrinsic factor representations learned from our model. We see that when the model is equipped with the CIED module (IEDR). the factors are well disentan- gled. However, when we not use CIED module (noCIE),, the intrinsic and extrinsic factor representations are mixed together. This indicates that these factors cannot be well learned and dis- entangled without our CIED module. We provide more visualizations and analysis for our model with different component combinations in Appendix G.7
5.3.2 CASE STUDY
We conducted a case study to analyze the differences between the learned intrinsic and extrinsic factors. We randomly choose a user from the Frappe dataset and generate the intrinsic matching scores (the dot product of the user's intrinsic representation and the items' (apps) intrinsic repre- sentations) two different contexts (Weekday and Weekend). The same for the extrinsic matching scores. We sort the matching scores for the intrinsic and extrinsic factors, respectively, and list the
8

---

Under review as a conference paper at ICLR 2023
(a) User IEDR.
(b) Item IEDR.
(c) User noCIED.
(d) Item noCIED
Figure 3: Visualization of the learned intrinsic and extrinsic representations with t-SNE for the Frappe dataset. The blue dots are the intrinsic representations, and the red dots are the extrinsic representations.
Table Items (in category) of the highest intrinsic and extrinsic scores for different users Weekday.
20
.00
80
70-
Userl
User2
60--
Rank
Inrrinsic
Extrinsic
Intrinsic
Extrinsic
500-
40
lantial Wedldday
Photography
Tool
Cards&
Communication
30:
Sports
Communication
Productivity
Tool
20
Exemric Wicluder
Healthh&
Mediadi
Cards&Casino
News&Maguznne
Items
Taols
Personalization
Sports Games
Tool
Health&
Communication
Brain&Puzzle
Communication
Figure 4: A user's top 100 intrin- sic and extrinsic scores in different contexts (Weekend vs. Weekday).
Porsonalization
Casual
Commanication
News&Magazines
Personalization
Music&Audio
Tools
Personalization
Communication
News& NaguinMg
Sports
Media& Video
Personalization
Communication
Arcade&Action
Tool
Healthh&Fitees
Toavel&Local
Tools
Communication
top 100 items. The results demonstrated in Figure 4 Note that the top 100 items for intrinsic and extrinsic factors are different. According to Figure from weekday to weekend, the extrinsic scores vary lot, while the intrinsic scores remain invariant. These observations demonstrate that, in different contexts, the user has different intrinsic factors, as well as consistent intrinsic factors.
Then, illustrate how intrinsic and extrinsic factors may have different impacts on users' choices. Table lists the categories of the items with the 10 highest intrinsic/extrinsic scores for two users, respectively. observe that users have individual intrinsic interests that show their real hobbits in personal time, e.g., Userl prefers sports and fitness apps while User2 prefers gaming apps. On the other hand, extrinsic factors give higher rank the items based on the contexts (Workday). e.g., Tool (Google Search) and Communication (Gmail) rank highest in Userl's 's extrinsic scores.
Remark. In addition to the above experiments, further evaluate our model by 1) running our model on other context/feature modeling methods (Appendix G.I). 2) comparing the proposed model with naive baseline learning intrinsic factor representations (Appendix and 3) study- ing how the hyperparameter settings influence the performance (Appendix G.6
6 CONCLUSION
Capturing accurate intrinsic and extrinsic factors from contexts is an essential research topic in recommender systems. Focusing on the problem of existing studies that either neglect the fac- tor learning, or learn the factors from only one specific context, we propose the intrinsic-extrinsic disentangled recommendation (IEDR) model. This generic model effectively learns intrinsic and extrinsic factors from various contexts for a more accurate recommendation. IEDR comprises context-invariant contrastive learning component, and mutual information minimization based dis- entangling component to ensure the success of the factor learning. Extensive experiments prove our model's 's ability to learn intrinsic and extrinsic factors and leverage the learned factors for more ac- curate recommendation prediction. Following this work, may discover other types of factors that can be considered besides the intrinsic and extrinsic ones, and learn more fine- ine-grained intrinsic and extrinsic factors (e.g., multiple intrinsic factors).

---

Under review as a conference paper at ICLR 2023
REFERENCES
Linas Baltrunas, Karen Church, Alexandros Karatzoglou, and Nuria Oliver. Frappe: Under- standing the usage and perception of mobile app recommendations in-the-wild. arXiv preprint arXiv:1505.0301 03004, 2015.
Oren Barkan and Noam Koenigstein, Item2vee: Neural item embedding for collaborative filtering. In MLSP, pp. 1-6, 2016.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In ICML, pp. 531-540, 2018.
Roland Bénabou and Jean Tirole. Intrinsic and extrinsic motivation. The Review of Economic Studies, pp. 489-520, 2003,
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, pp. 1597-1607, 2020.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In CVPR, pp- 15750-15758, 2021.
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recom- mender systems. In Recsys, pp. 7-10, 2016.
Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. Club: A contrastive log-ratio upper bound of mutual information. In ICML, pp. 1779-1788. PMLR, 2020.
Ching-Yas Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba, and Stefanic Jegelka. De- biased contrastive learning. In NeurIPS, pp. 8765-8775, 2020.
Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommendations.
In RecSys, pp. 191-198, 2016.
Tianyu Gao, Xingeheng Yao, and Danqi Chen. Simese: Simple contrastive learning of sentence embeddings. In EMNIP, pp. 6894-6910, 2021.
Beliz Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov. Supervised contrastive learning for pre-trained language model fine-tuning. In ICLR, pp. 1-13, 2021.
Huifeng Guo, Ruiming Tang. Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: factorization- machine based neural network for etr prediction. In ISCAI, pp. 1725-1731, 2017.
Xiangnan He and Tat-Seng Chua. Neural factorization machines for sparse predictive analytics. In SIGIR, pp. 355-364, 2017.
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based rec- ommendations with recurrent neural networks. In ICLR, pp. 1-10, 2016.
Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildos Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In ICLR, pp. 1-14, 2019.
Wei Jiang, Fangquan Lin, Jihai Zhang, Cheng Yang, Hanwei Zhang, and Ziqiang Cui. Dynamic sequential recommendation: Decoupling user intent from temporal context. In ICDMW, pp. 18- 26, 2021.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In NewrlPS, pp. 1-13, 2020.
Huayu Li, Yong Ge, Defu Lian, and Hao Liu. Learning user's 's intrinsic and extrinsic interests for ciitt-of-itterst recommendation: A unified approach. In IJCAI, pp. 2117-2123, 2017.
10

---

Under review as a conference paper at ICLR 2023
Fangquan Lin, Wei Jiang, Jihai Zhang, and Cheng Yang. Dynamic popularity-aware contrastive learning for recommendation. ACML, pp. 964-968, 2021.
Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. Learning disentangled repre- sentations for recommendation. In NeurIPS, pp. 5712-5723, 2019.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-based rec- ommendations on styles and substitutes. In SIGIR, pp. 43-52, 2015.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.00 807003748, 2018.
Steffen Rendle. Factorization machines. In ICDM, pp. 995-1000, 2010.
Richard M Ryan and Edward L Deci. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, pp. 54-67, 2000.
Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. Autoint: Automatic feature interaction learning via self-attentive neural networks. CIKM, pp. 1161-1170, 2019.
Daria Sorokina, Rich Caruana, Mirek Riedewald, and Daniel Fink. Detecting statistical interactions with additive groves of trees. In ICML, pp. 1000-1007, 2008.
Yixin Su, Rui Zhang, Sarah Erfani, and Zhenghua Xu. Detecting beneficial feature interactions for recommender systems. AAAI, pp.
2021.
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? In NeurlPS, pp. 6827-6839, 2020.
Michael Tsang, Hanpeng Liu, Sanjay Purushotham, Pavankumar Murali, and Yan Liu. Neural in- teraction transparency (nit): Disentangling learned interactions for improved interpretability. In NeurlPS, pp. 5804-5813, 2018.
Robert Vallerand. Toward a hierarchical model of intrinsic and extrinsic motivation. Advances in Experimental Social Psychology, pp. 271-360, 1997.
Ruoxi Wang. Rakesh Shivanna. Derek Cheng. Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. Den v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. In WWW, pp. 1785-1797, 2021.
Xiang Wang, Hongye Jin, An Zhang, Xiangnan He, Tong Xu, and Tat-Seng Chua. Disentangled graph collaborative filtering. In SIGIR, pp. 1001-1010, 2020.
Frank Wilcoxon. Individual comparisons by ranking methods. Breakthroughs in Statistics, pp- 996-202, 1992.
Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. Self- supervised graph learning for recommendation. In SIGIR, pp. 726-735, 2021.
Jiancan Wu, Xiangnan He, Xiang Wang, Qifan Wang, Weijian Chen, Jianxun Lian, and Xing Xie. Graph convolution machine for context-aware recommender system. Frontiers of Computer Sci- ence, pp. 1-12, 2022.
Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. Session-based recom- mendation with graph neural networks. In AAAI, pp- 346-353, 2019.
Yuxia Wu, Ke Li, Guoshuai Zhao, and QIAN Xueming. Personalized long-and and short-term preference learning for next poi recommendation. IEEE Transactions on Knowledge and Data Engineering, 2020.
Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. Attentional factor- ization machines: Learning the weight of feature interactions via attention networks. IJCAI, pp. 3119-3125, 2017. 2017.

---

Under review as a conference paper at ICLR 2023
Zonghan Yang, Yong Cheng, Yang Liu, and Maosong Sun. Reducing word omission errors in neural machine translation: A contrastive learning approach. In ACL, pp. 6191-6196, 2019.
Tiansheng Yao, Xinyang Yi, Derek Zhiyuan Cheng, Felix Yu, Ting Chen, Aditya Menon, Lichan Hong, Ed H Chi, Steve Tjoa, Jieqi Kang, al. Self-supervised learning for large-scale item recommendations. In CIKM, pp. 4321-4330, 2021.
Yantao Yu, Zhen Wang, and Bo Yuan. An input-aware factorization machine for sparse prediction. In IJCAI, pp. 1466-1472, 2019a.
Zeping Yu, Jianxun Lian, Ahmad Mahmoody, Gongshen Liu, and Xing Xie. Adaptive user modeling with long and short-term preferences for personalized recommendation. In IJCAI, pp. 4213-4219, 2019b.
Yu Zheng, Chen Gao, Jianxin Chang, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. Disentan- gling long and short-term interests for recommendation. In WWW, pp. 2256-2267, 2022.
12

---

Under review as a conference paper at ICLR 2023
A PRELIMINARY
A.1 STATISTICAL INTERACTION GRAPH NETWORK (SIGN)
The statistical interaction graph network (SIGN) (Su et al. 2021) explicitly models feature in- teractions through a graph neural network. IEDR, learns user representation u (as well as a context representation, and an item representation) using the SIGN-based model fu(u), where {2]', 22. ................................ 2% fu regards the user u as user graph E). In this representtation, V = the node set that each feature 2 node, and is the edge set containing all the combina- tions of pairwise feature interactions, with each feature interaction ( being edge linking to corresponding nodes. Accordingly, the user representation learning becomes graph learning problem.
In SIGN, first, each feature 2," is mapped into a feature embedding "," E E of d dimensions as the node embedding. The embeddings are first randomly initialized and are updated through training. Then, SIGN learns the user graph using the function fu:
fu(G) = ((v(tee 2 bevllee,,,,
(5)
where & and sbr are aggregation functions (e.g., element-wise mean), h(.) R2ddd -> Rd is an MLP that models each feature interaction, eij {0, is the edge indicator (since we use all pairwise feature interactions, Eij = for all edges).
Ja outputs the modeled user representation u E IRd of d dimensions.
A.2 VARIATIONAL CONTRASTIVE LOG-RATIO UPPER BOUND (vCLUB) OF MUTUAL INFORMATION
Suppose we aim to learn an intrinsic representation o and an extrinsic representation o where their mutual information is minimized. In the vCLUB method (Cheng al.] 2020). a variational distribution qf (o Jo ( of parameter 0Fr (e.g., an MIP) is proposed to predict the extrinsic factor given the intrinsic factor. Then, the vCL.UB-based mutual information upper bound can be derived as:
ECLUB(O oex)) = Boto,, og. [log oi (o [o Bpto Inton, [log qi (o... o.h,],
where p(o ) is the joint distribution and p(o", )p(0))) are is the marginal distribution.
vCLUB performs mutual information estimation and minimization in two steps iteratively. In the first step, to ensure Equation (6) holds as the upper bound, 0f is trained make the likelikelihood function Lappr (u, c) ;= 3/ E log ((o")) (of)))) maximized (Theorem 3.2 of (Cheng al. 2) In the second step, 0 is frozen, and other parameters (e.g., the parameters to generate % and o%)) are trained to minimize I.CLBB( (o% 0%) so that the mutual information is minimized.
B PROOF OF THEOREM
Proof. Since the mutual information is not explicitly intractable. we approximate the right side of Equation (4) with a lower bound (i.e., MINE (Belghazi et al., 2018) and an upper bound (i.e., CLUB (Cheng et al. 2020)) of mutual information, respectively. More formally.
I(o u) 2 IMINE( o u) := Ep uu p(o,, u)] log Eptor, In(a) Ip(o,. w)];
(7)
I(o c) S (o c) == Epior .) [log p(o |c)] - Exro poce [og lc)]. (8)
With the approximated terms above, proving Equation. {4) turns to verify:
N
arg min Ecic((4, c.) = arg max IMINE u) (o e))
By minimizing LOIEL, aim to make (0%, )a similar to (o%) iy.. This procedure be interpreted in probability as: increasing the probability of f [u;, cj) to predict (o"; )a. Therefore, maximiz- ing the (xp )a, (o% )i)/r) in Equation (1 is equivalent to maximizing p((o%, in )ad Jalui, Cÿ)
13

---

Under review as a conference paper at ICLR 2023
(exp(.) is monotone increasing that does not influence the conclusion). Similarly, minimizing exp(sim( (o inni (()))) is equivalent to minimizing p((o), /ue c,). Therefore, we have
(u,
exp(sim((O (o (0))))))7))
log
Enyeu exp(sim((0))))) (o.)) {/)
N
(ot))))) Σ logl Σ
exp(sim((O (o (=7))
ej)l
logl Σ
p((oin)sluu,es))
My Ei
Equation only samples context Cj for each data point. However, during the training, all contexts in C are expected to be sampled. If we count all contexts, we have
Σ logl Σ
p((omm)iluee c.)
MyEl
N
N
(10)
Σ logi Σ p((oin)iv ur, us,
1=1 EyEc
Mott Poggpo /u, c]] Eptor, log Bpcu (p(o [u,
Equation 100 is the probability form of the objective function of the mnteex-innvarian counteractive
learning component (Equation (I)). Equation (10) maximizes the likelihood p(o u,, c) given the joint distribution of users and intrinsic factors, with the marginal distribution of contexts. Mean- while, it minimizes the likelihood p(o /u, c) given the joint distribution of contexts and intrinsic factors, with the marginal distribution of user
From Equation
10 we further have:
Eplor, wote u, c)] Eptor e) log Ep(w)lp(o.. c]]
( Eptoy, loggee c)] Eptor chot poogs//oooooo c)]
Eptor, wogee c)] Epior cl (lgg u c)] (Bpta) [(w)) ppco(loo
Pplor, Dogp( c)p( (1)) Pooo che Poog [u,
- [l))(())
logp m/c)] Pooo ( pog u c)] F (og
Extoy, mppto) llggpoo.. u/c]] Bpoo ( llo c)] Ex (og
Dtppegg. /u, c)] Ep hoop
Eptor, ()p(o) llggp u/c)]
wug.ooo......llp.o..o....o....ll B (lgg((u))
Eo loog ju, c)] bwwprr c)]
uhite (ogp u/c)]
Po bumpt pogle m/c]]
en c)] Ep bon (oots
= -Ep(c) wo u/c)] - Ex Inw llog u/c))
Bppa)
c)] De bio log
(11)
Note that only if fiz (u, c) is mone-to-one (or onc-to-one) mapping then Equation 10 and Equation will be equivalent. Otherwise, given sample pair (u, c). f.r (u, c) may have different o outputs (i.c., one- to- fo-many). In this situation, the first term of Equation (1O) cannot guarantee that the same user with different context will have the same intrinsic factor (i.e., they may have various intrinsic factor representations while still meet the objective of the first term Equation (10). We use an MLP as f" fu, c). which a many-to-one mapping function. Therefore, we can ensure the equivalence hetween Equation 10 and Equation (T)
14

---

Under review as a conference paper at ICLR 2023
In the second term, pushing the log inside the expectation dose not change the minimizer.
Comparing Equation 7 and the first term of Equation (T). they both act like classifiers whose objectives maximize the expected log-ratio of the joint distribution over the product of marginal dis- tributions (Hjelm et al,, 2019). Therefore, maximizing this term in Equation (IT) will have the same effect to maximizing Equation 7 We can interpret the first term of Equation (11) maximizing the mutual information between users and the corresponding intrinsic factor, conditioned on a given context. Similarly, maximizing the negative of the second
of Equation (IT) will have the same effect of minimizing Equation ( which can be interpreted as minimizing the mutual information between contexts and the corresponding intrinsic factors, conditioned on given users.
Therefore, we can conclude that:
arg min (WG,PL.c.)ED
Σ
Luccl( ci) arg max [o u) ICLUB(O c).
C PREVENTING THE TRIVIAL SOLUTION OF CIED
The two components in the CIED module, the contrastive learning component and the disentangling component, jointly ensure the success the intrinsic and extrinsic factor representation learning- However, CIED may fall into trivial solution: fi (u, c) maps to of without considering c, and maps to of, without considering u. Although this trivial solution minimizes CCiCL(u, e) and D (u, c), o (resp. 0% is not the intrinsic (resp. extrinsic) factor, but just mapping of the information (resp. context information). We prove that this trivial solution can be avoided by setting fie (u, c) as a non-linear function, leading u and e statistically interacted.
Statistical Interaction We start with introducing the statistical interaction (or non-additive inter- action), which ensures a joint influence of several variables on an output variable is not additive (Tsagg et al. 2018). Based on Sorokina et al. (2008), F(X) shows statistical interaction between variables T; and x if f fúg F(X) cannot be expressed as:
F(X) + f1(21..... F411............................ xn) + (2 F. Ind)
(12)
More generally, if using u; E Rd describe the i-th variable with d-dimension vector (Rendle) 2010
Su et al. (02I). e.g., variable embedding, each variable can be described in a vector form D; Then, we define the pairwise statistical interaction in vector form by changing the
Equation (12 into:
F(X) f,(u1.....
un) + fyg
Uj Uj+t
un)
(13)
Preventing the Trivial Solution Based on the definition of statistical interaction, we can express the trivial solution as that fi (u, c) learns no statistical interaction between u and
fie c) = (u) + >2fa(c),
(14)
where fi outputs o f2 outputs 0 and are weight scalars.
To prevent the trivial solution, we need to ensure that function f (t, c) cannot be modeled in the form of Equation (14). Therefore, if u and e are modeled as a statistical interaction in l' (u, c), the trivial solution can be prevented. Since f% (u, c) only takes and as inputs, we just need fül to be a non-additive model. That is, f (u, c) should contain a third term fa(u, c):
f (u, = (1) + >efe(c) + Aa,
(15)
where fa is a non-additive model and A3 + 0.
Therefore, in the optimized situation, o >1 (u) learns part of the information from users that do not interact with context information. 0t = )2)2(())+ + Aufs(u, c) leams the context information (fz(c)) and the information that changes given different contexts (fa(u.c).
In Appendix G.2, we empirically analyze how the trivial solution will influence the prediction per- formance.
15

---

Under review as a conference paper at ICLR 2023
Training Direction
4 (fixed)
0tH in
Oex
Training Direction
Figure 5: An illustrative example demonstrating the potential problem asymmetric learning vCLUB. The blue circles are intrinsic representations, and the red circles are extrinsic representations. The dotted arrows are the directions that vCLUB will push o and o to move toward in their space.
D POTENTIAL PROBLEMS OF THE ASYMMETRIC VCLUB METHOD
The vCLUB-based mutual information minimization method proposed in (Cheng al. 2020 is an asymmetric method. Appendix A.2 gives an introduction about how vCLUB method performs mutual information minimization. this section we explain the possible reason that vCLUB less robust and perform worse than our proposed bidirectional vCLUB method (BiDis).
If we directly apply vCLUB to our disentangling component, the parameter 0r of variational distribution ( jo ( will be trained to approach the vCL.B- upper bound in Equation (6) (Step 1). Then, 0f is frozen, and 0% of, are trained to minimize I(o o% via minimizing the upper bound IcCUB((), o%22) (Step 2). However, this way of minimizing mutual information may result in an unexpected outcome: the mutual information may be minimized via making o contain as less information possible. To better illustrate the possible outcome,
design of linear function and well trained Step to ensure Equation 6 an upper bound of [oo, o",.). F shows how the unexpected result may occur. In Step o "h, will be trained minimize Equation (6). To achieve this goal, it ensures of cannot predict o given the corresponding o from the joint distribution (the first term of Equation (6)) and the same time ensures the output of 9 is similar to the other o 's from the marginal distribution (the second term of Equation (6)).
From o. perspective (blue circles), the goal can be achieved by pushing the of to move from its original position (optimizing the first term of Equation (6)). and move towards the mean of the other oN 's (optimizing the second term of Equation (6). From O perspective (red circles), the goal can be achieved by pushing the o away from its original position (optimizing the first term of Equation (6) and the the mean of the other ot (optimizing the second term of Equation (6).
This clusters all the ",, 's together, making o contain less information, while all the o 's try to split away from each other, making 0% contain more information. The mutual information minimization procedure is like "transfering" the information from o 's to 0% 's, which is not what we expect. BiDis, however, a symmetric disentangling method on o and 0% so that will not result in this issue. This may be the reason that vCLUB performs worse and less robust than our proposed symmetrical disentangling component.
E TIME COMPLEXITY ANALYSIS
Briefly speaking, the time complexity of the whole model is comparable to feature interaction- recommender systems (c.g., AutoInt, SIGN). The overhead of the alternative optimizing procedure for the disentanglement component is marginal in the whole optimizing procedure.
The most time-consuming computations are the feature interaction learning to get user, item, and context representations, which need to conduct interaction modeling every pair of feature inter- actions. This procedure has also been done on other feature interaction-basee models, therefore, the time complexity of the proposed module is comparable with those methods.
Our model takes additional computations on the context-invariam contrastive learning (CICL) com- ponent and the disentangling component:
16

---

Under review as a conference paper at ICLR 2023
For the CICL component, do not need perform the feature interaction modeling again, but reuse the generated userfitem/context representations, which saves the majority of the overhead. We only need to perform fice (L+1) times, where L is the number of negative samples. Since fic is one-hidden layer (with 128 hidden units) MIP, the overhead is marginal.
For the disentangling component, also reuse the generated user/item/contex: representations. As we discussed in the paper, we use two-step learning policy to train our model. Regarding to the reviewer's main concern, the first step in the two-step learning actually takes very little overhead. This is because this step only tries to optimize the parameters of the functions qi and 42 (Eq.2), which are two MLPs with one hidden layer. For each data sample, we only run 91 and d one time using On and Oez-
In summary, since all of the computations above do need to perform feature interaction modeling (the most time-consuming procedure in all feature interaction-base models), the small imposed overhead is acceptable considering the effectiveness of our model in capturing accurate intrinsic and extrinsic factors.
F EXPERIMENTAL SETTING
Datasets We evaluate our models in two scenarios with various contexts: mobile app recom- mendation and a restaurant recommendation. In the mobile app recommendation, we use the Frappe (Baltrunas et al. 2015) dataset that records mobile app usage logs. Each data sample logs users' app usage in certain context (e.g., weather, time, location). In addition, some relevant properties of the apps are also captured (c.g., category, developer). In the restaurant recommendation, we use the Yelp dataset (Wu et al. 2022). It records users' reviews local restaurants. Due the fact that user usually goes to restaurants in the same city, geographic isolation appears in the dataset. There- fore, select the records in New York City. We regard each record data sample that the user has been to the restaurant. We leverage the user/item features and context features (e.g., day of the week) predict whether user will go a given restaurant in a specific context. We also evaluate our model in two Amazon datasets (Movies and CDs) (McAuley et al.) 2015) that have been used in sequential recommendation tasks et et al. 2019b). The datasets contain user-item interactions with timestamps. For the sequential recommendation, we use the same IEDR model structure as that for the Frappe and Yelp datasets, but modify the data input to our model. More specifically, we do not directly learn behavior sequences, but consider each behavior as a data sample with time context in- formation. That is, we consider the bucketed timestamp of each user behavior as time context (we categorized time context). Therefore, behaviors in the same time interval have the same time context, indicating that these behaviors share some similar short-term (extrinsic) interests (e.g., item popularity).
consider one month
For each dataset, only users that have more than 5 records (Frappe and Yelp) and more than 20 records (Movies and CDs) are chosen. We choose the last record of each user for testing, and the second last record of each user for validation. The rest of the records are used for training. Each of these data sample is considered as a positive sample (y = 1). In addition, for each positive data sample in the training set, we randomly choose 2 items (but keep the user and contexts) as negative samples (y 0), meaning the user did select the 2 items that context. For each test/validation data sample, we randomly choose 99 items as negative samples to ensure more robust evaluation. The statistics of the datasets are shown in Table 4
Table 4: Dataset statistics. "Count" refers to the number of users/items, "Features" represent the number of different features (for User and Item, the number of features excludes the user/item ids).
Datasets
Data Samples
User
Item
Context
Train
Valid
Test
Count
Features
Count
Features
Features
Frappe
282,426
69,500
69,500
695
4,082
2,892
318
Yelp
518,208
633,600
633,600
6,336
24
12,902
66
13,034
Movies
2,305,362
39,663
1,322,100
13,221
49,189
161
193
CDs
879,030
16,392
546,400
5,464
16,184
209
195
17

---

Under review as a conference paper at ICLR 2023
Baseline methods IEDR models the feature interactions of users, items, and contexts. Therefore, we compare our model with competitive feature interaction- recommendation methods. The methods include attentional factorization machine (AFM) (Xiao al.] 2017). neural factorization machine (NFM) (He & Chua 2017), self-attention-basedd feature interaction model (AutoInt) (Song
et al.] 2019). deep factorization machine (DeepFM) (Guo al. 2017), wide & deep model (WDL) [Cheng et al. 2016), improved deep & cross network (DCNv2) [Wang et al. (2021). and input-aware factorization machine (IFM) (Yu et al., 2019a). We implement these methods using the DeepCTR package. The statistical interaction graph neural network (SIGN) (Su et al., 202T) is applied based on the released code. The above methods model all the factors in unified representation with- out considering the factors that affect users' decisions. Meanwhile, we compare IEDR with the methods that learn implicit factors. They are disentangled variational auto-encoder for recommen- dation (DisRec) (Ma et al. 2019) and disentangled graph collaborative filtering (DGCF) (Wang) ict al. 2020). We implement these methods based on their released code. Note that since DisRec and DGCF models do not consider any feature, their task is to simply predict whether user will select an item. IERD and other baseline models, however, consider the user-item interactions in specific contexts (a user's decision to select item may be different in different contexts). For DisRec and DGCF, to prevent the test data samples from appearing in the training set, we remove the data samples from the training set that appear in the test (with different contexts in other models). For fair comparison, we set the factor number to 4 for DisRec and DGCF. For sequential recom- mendation baselines, we compare our model with the models that consider LS-term interests. They are session-based recommender system with recurrent neural networks (GRU4Rec) (Hidasi et al.) 2016), Short-term and Long-term preference Integrated Recommender system (SLI-Recc (Yu et al,, 2019b), and Contrastive learning framework of Long and Short-term interests for Recommendation (CLSR) (Zheng et al. 2022). We use the same MLP structure for feature interaction modeling and the same embedding size for features our IEDR model.
Implementation details In IEDR, all the MLPs have the same hidden structure: one hidden layer of 128 dimensions and ReLU activation after that. The input and output sizes of MLPs varies based on their needs. We set the embedding dimension to 32 for all the features. fie an MLP that outputs a 64-dimension vector, with the first 32 dimensions are the intrinsic factor representation and the last 32 dimensions are the extrinsic factor representation. For the second dtropout-basedd negative context generating method in the context-invarian contrastive learning component, the dropout rate set to 0.5. The number of negative pairs for contrastive learning is 40 for each data sample (note that the actual negative pairs will be doubled since both (o), )a and (o%, Jay will generate 40 negative pairs). The temperature T is set to 0.5. In the disentangling component, Qi and Ø2 are MLPs that output vectors that have the same dimension of intrinsic/extrinsic factor representations. The number of negative samples of the bidirectional vCLUB based method is 5 for each direction. We set X to 0.1 for the Frappe dataset and 0.01 for the Yelp dataset, and set A to 0.1 for both datasets. The X and 2o are both 0.01 for the Movies and the CDs datasets. We run all the experiments on a machine equipped with CPU: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz, and GPU: Nvidia Tesla v100 GPU.
The model structure of IEDR and its variations used in the experiments are detailed in Table 5 and Table 6 Note that the component structures of variations are the same as the IEDR if not specified.
G ADDITIONAL EXPERIMENTAL RESULTS
In this section, we provide further experimental results that are not included in the main paper.
G.1 OTHER FEATURE MODELING METHODS
In the RP module, although we use a SIGN- method (Su et al. 2021) to learn user, item, and context features, the module can use any feature modeling method. Here, we use other methods to evaluate whether our model still performs well. Specifically, we run our model with the other three variations using different feature modeling methods: 1) averaging feature embeddings (MEAN): 2) adding MLP on top of the averaged feature embedding (MLP); and 3) modeling and aggregating feature interactions through a Bi-interaction layer proposed in (He & Chua 2017) (BI). The results shown in Figure 6 We report the results of each variation with and without the CIED module.
18

---

Under review as a conference paper at ICLR 2023
Table 5: Implementation details of different variations on the recommendation prediction module.
"_" represent the operation is the same as our original IEDR setting.
Variation
Recommendation Prediction Module
feature model!
fae
TEDR
((M(MMTP((= @ zj))ieu))ee
MTP c)
(o oiar
AVG
((2"") leu + MLP( ((Eu)) (z 2j 2j),,en
MLP
BI
Linear Nonlinear
W c) + lo o W [u, c)) [o Ocel
IEDR, sp
M MLP (u) - o... MLP3
Ozz
'Here we use user representation learning as an example. The item and context learning have the same structure. Ø, z are both clement averaging functions and O is the clement-wise product.
'Here we user factor learning example. is flexible operation combine vector, i.c. glement-wiss product for the Frappe dataset, and element wise summation the Yelp dataset. [; the concatenation operation. W linear transformation matrix, a a ReLUU activation.
Table 6: Implementation details of different variations of the contrastive intrinsic-extrre disentan- glement module. represents the operation is the same as our original IEDR setting. x represents the variation does not contain the component.
Variation
Contrastive Barinsic-Extrinsis Disentanglement Module
Contrastive Learning Component"
Disentangling Component
IEDR
positive sample: fie [Us, Cj) (oti
MLPa, (o% (o"r) (qi)
negative sample: fic (the, Ct) -> (o..))
MLPo, (o + (o%,) (g))
für (Mee C5) + (oj randChooiee (NegGen
(eegGeeen NegGen2)
noDis
noCL
noCIED
NegGenl
is generated from NegGenl ey is generated from NegGen2
NegGen2
NegGenl
vCLUB
MLPb, (o...) (o (91)
BiDis
IEDR,p
positive sample:
(ropout((O".)) )) (ot,.)"
negative sample: dropout((O).) + (o..)"
*For IEDR ap, the positive samples (o%, are generated through a dropout of the intrinsic representation of the user, and the negative samples (o",)) )" are generated through dropout of intrinsic representations of random users.
From this figure, we can see that when equipped with the CIED module, all feature modeling meth- ods perform better than those without the module. It shows that our proposed CIED module can learn intrinsic and extrinsic factors for more accurate recommendation when different feature mod- eling methods are applied. Meanwhile, we can see that the feature modeling methods can impact the performance. MEAN is just a linear aggregation of features, resulting in the worst performance. Both MLP and B/ have better feature modeling ability and hence have better performance than MEAN. The SIGN-based feature modeling (SIGN) is the state-of of-the-art feature interaction model- ing method and performs the best.
19

---

Under review as a conference paper at ICLR 2023
Frappe
Yelp
64
74
w CIBD
CIED
72
wil
52
70
68
60
66
SIGN
SIGN
ML.P
method
Feature
Figure 6: Model performance when equipped with different feature modeling methods.
Intrinsic
Extrinsic
Intrinsic
Extrinsic
Frappe
User
Nonlinear
Context
Context
Yelp
Nonlincar
Lincar
Linear
Nonlineur
47
48
49
50
Figure 7: Visualization of fie weights for the Linear and Nonlinear models.
Figure 8: Comparing the performance the Linear and Nonlinear models on different datasets.
G.2 FALLING INTO TRIVIAL SOLUTIONS
As discussed in Section C our model may fall into trivial solution if (w, c) is linear mapping method. To evaluate how the trivial solution influences our model in learning the factors, we run our model with Sie being linear. Specifically, we concatenate and and feed them into an MLP without hidden layer or activation (a linear mapping), making easy to fall into the trivial solution. We call this variation Linear. Then, avoid this by simply adding nonlinear activation function (RcLU) activation after the linear mapping. We call this variation Nonlinear. Figure 7 shows the weight values of fie of the two variations. The color shows the weights mapping from user/context representations to intrinsic/extrinsic representations. The darker the color, the larger the weight (more information of user/context is mapped into intrinsic/extrinsic representations). The figure shows that in the Linear variation, user information is largely mapped into intrinsic representation (user-intrinsic block) but extrinsic representation (user-extrinsic block). Context information is largely mapped into extrinsic representation (context-extrinsic block) but not intrinsic representa- tion (context-intrinsic block). This means that the Linear variation falls into the trivial solution. On the contrary, in the Nonlinear variation, user information is mapped into extrinsic representation (user-extrinsic block), showing that the extrinsic representation contains both user and context in- formation. Figure S shows the performance of the two variations. We can see that Linear model performs worse than Nonlinear model. It proves that learning intrinsic and extrinsic factors results in better performance than simply mapping user and context information into two representations, respectively (the trivial solution).
G.3 COMPARING THE IMPACT OF DIFFERENT CONTRASTIVE LEARNING VARIATIONS
To learn the intrinsic factor, we propose a context- -nvarritt contrastive learning method. However, directly generating intrinsic factor representations through only user information seems to more direct way, i.e., o f (u). We argue that the intrinsic representations learned this way could not guarantee that the representations are intrinsic factors. This is because the information in the learned intrinsic factor representations can vary with different contexts, since these factors have never been modeled w.r.t. the contexts.
In this section, we empirically show that this approach to learning intrinsic factors is inferior to our context-invarian contrastive learning method in producing accurate recommendations, To do so, design variation (IEDR,p) by splitting the intrinsic-extrrnsie factor generation into two functions: 0f = f (u), and o" = f (u, c). Both f. and for have the same structure as fie, with the output dimension being half to ensure the consistency of the factor representation
20

---

Under review as a conference paper at ICLR 2023
Table 7: Comparing the performance 1EDRap with different dropout rates (for NegGen2).
Table 8: Comparing the performance of IEDR using different negative con- text generating methods (for the con- trastive learning component).
Frappe
Yelp
IEDR, sp. p=0.1
70.68
52.03
IEDR, sp. p=0.5
68.25
51.49
Frappe
Yelp
IEDR p=0.1, noDis
70.56
52.02
IEDR sp, p=0.1, noCL
70.31
51.62
NegGenl
73.01
52.49
IEDR, p=0.1, noCIED
70.16
51.10
NegGen2
71.50
51.82
NegGenl
74.11
53.05
IEDR
74.11
53.05
dimension. The contrastive learning component does not consider context information but uses standard InfoNCE based contrastive learning for learning robust user/item representations following Yao et al. 2021 Table 7 illustrates the results of 1EDRsp compared to model with sEDRsp using different dropout rates (p 0.1 and p = 0.5) in the contrastive learning component, and different component combinations (noDis, noCL. noCIED). We the variation in recommendation accuracy. It proves that IEDR cannot ensure a successful intrinsic factor learning and hence incur worse recommendation accuracy. Unlike IEDR, IEDR,, gains better performance with a lower dropout rate. This is because, in IEDR.p. the dropout generates views representing the same user instead of different users, which consistent with the conclusion in (Gao et al. 2021).
that our model outperforms
G.4 DIFFERENT NEGATIVE CONTEXT GENERATION METHODS
We propose two negative context generating methods in the contrastive learning component: 1) sample other contexts; 2)
large dropout rate the original context. We evaluate the two methods in this section. Table [ shows the results of our model when using only NegGensl, only NegGens2, and NegGen/&2. &2. We can that NegGen/ results in better performance than ing NegGen2. This is because NegGen/ uses true context representations. which are consistent with what may appear in the test samples. Meanwhile, we that NegGen &2 results in the best performance, This is because NegGen2 provides more unseen (randomly generated) context repre- sentations, which strengthens the generalization ability of our model. Next, we evaluate Neg with different dropout rates in Figure 9 The best performance can be achieved when the dropout rates range from 0.5 to 0.7. This consistent with our claim in Section 4.2.1 The reason that small dropout rate (e.g., 0.1) pushes the generated context representation too close to the origi- nal one; hence it cannot be considered a different context. However, a relatively large dropout rate (e.g., 0.9) loses too much information; hence, it is no longer valid context representation. In addi- tion, for NegGenl &2 of the dropout rates, the results consistently outperform those that only use NegGen2.
G.5 EMPIRICAL ANALYSIS OF TIME COMPLEXITY
We summarize the overall time consumption of IEDR and several feature interaction-bart baseline models in Table 9 The results are recorded by running the models for one batch (batch size 1024) on the Frappe dataset on a machine with CPU:12th Gen Intel(R) Core(TM) 19-12K, RAM: 32GB, GPU: NVIDIA GeForce RTX 3090.
We can see that our model's 's overall time consumption is slightly higher than other baselines. Next, we summarize the time cost of critical procedures IEDR in Table The first four rows are model forwarding procedures, and the last two
model (alternative) optimizing procedures.
Table 10 shows the feature interaction modeling procedure takes most of the time, which is consis- tent with analysis above. Both CICL and disentangling forward procedures (row 2-4) do not pose much overhead since they reuse the feature interaction modeling results. Optimization (stepl) 1) is the alternative training step that updates the parameters of the models disentangling component (g) and 92). The alternative optimizing procedure produces little overhead (2.21 ms), which is negligible in the whole procedure.
21

---

Under review as a conference paper at ICLR 2023
Table 9: The overall time consumption of different models in one batch training.
Table 10: The time consumption of critical procedures in IEDR one batch training.
Model
Time (ms)
Procedure
Time (ms)
DCNv2
34.40
Graph Learning (Feature Interaction Modeling)
14.16
AutoInt
37.53
CICL
8.05
SIGN
40.41
Disentangling (step 1)
0.16
IEDR
44.61
Disentangling (step 2)
1.93
Optimization (step 1)
2.21
Optimization (step 2)
8.52
Frappe
Yclp
Frappe
Yelp
NegGen2
56
NegGen2
74
2ogGen1&2
Ne
52
70
50
68
50
66
0.1
D.3
D.5
0.7
0.9
D.1
0.3
0.5
0.7
0.9
8
32
64
128
256
32
64
128
256
Dropout Rate
Dropout Rate
Embedliing dimension
Embediing dimension
Figure 9: The performance of different dropout rates for method 2 (NegGen2).
Figure 10: Hyperparameter study: different embed- ding dimensions
G.6 EFFECTIVENESS OF MODEL HYPER PARAMETERS
We evaluate our model with different hyperparameter settings, including embedding dimensions, number negative samples, and loss weight values. Below, we summarics our observations.
G.6.1 EMBEDDING DIMENSION
We model with different feature embedding dimensions. We show the results of rumning model using different embeddings in Figure [0] Choosing the embedding dimension is a trade off between the expression ability and efficiency. From the figure, can that larger dimensions result better prediction accuracy. However, the improvement is not significant when the dimension is larger than 32. A larger dimension may reduce the performance due the overfitting problem (e.g., dimension 256 for the Frappe dataset).
G.6.2 THE NUMBER OF NEGATIVE SAMPLE AND Loss WEIGHT
The contrastive learning and disentangling components are both contrastive-based methods that quire negative sampling. This section evaluates how the number of negative samples influences performance. We also compare the influence of different loss weights of the two components. We run our model with different negative sample numbers and loss weights for the two components, respectively. From Figure []] we can see that a large loss weight, or a large number of negative samples does not necessarily result in better performance. There is the best combination of the loss weight and the number of negative samples for both components. A large or small loss weight may make the multi-task training unbalanced, harming the final performance. For the number of negative samples, small number will make the training insufficient, while large number may cause overfitting problem.
G.7 MORE VISUALIZATIONS OF INTRINSIC AND EXTRINSIC REPRESENTATIONS
This section provides complete intrinsic and extrinsic representation visualizations in three varia- tions: 1) the contrastive learning component is removed (noCL); 2) the disentangling component removed (noDis); and 3) the asymmetric disentanglement method (vCLUB) is used. Figure 12 compares these results. We include main observations below:
The intrinsic and extrinsic factors perfectly disentangled with our CIED module (IEDR).
22

---

Under review as a conference paper at ICLR 2023
0.001
0.01
0.1
10
0.001
0.01
0.1
10
66677
68.16
71.98
66.42
58.91
71.81
72.55
73.35
72.25
57.56
Negativve Samples
10
69.76
73.59
67.02
59.42
72.85
72.06
67.28
40
69.58
70.1
76.11
69-12
60.71
10
70.08
72.54
73.43
72.19
67.82
80
80-66849
69.29
73.97
68.29
60.35
72.19
72.85
73.96
72.87
67.76
160-68.05
06.03
68.89
73.92
68307
61.05
80
60
Aa
Figure 11: The performance of different numbers of negative samples and the loss weights in the risk min- imization function for the contrastive learning component (left) and the disentangling component (right), re- spectively.
Without the disentangling component (noDis), the intrinsic and extrinsic disentangling procedure may not succeed. This is because there is restriction extrinsic representations. Therefore, the extrinsic representations can contain any information, including the information of the intrinsic factor.
noCL has worse disentangling performance than IEDR, either. This is because the factors disen- tangled in noCL. are implicit. The implicit factors only ensure the disentanglement between the factors of the same data sample, but not between the factors of other data samples. For example, some context information may be stored in the intrinsic representation in data sample but be stored in the extrinsic representation in data sample 2.
noCIED performs worst among all variations, which is reasonable since does not distinguish the intrinsic and extrinsic representations.
vCLUB performs disentanglement, but is not very stable in some situations. This consistent with our analysis in Section 4.2.2
H ALGORITHM
This section provides the training process of our IEDR model in Algorithm In each epoch, we use the batch stochastic gradient descent method.
23

---

end for Under review as a conference paper at ICLR 2023
Algorithm Batch stochastic gradient descent training of IEDR.
1: Input: D {(u,, f=) with the corresponding true label y; for each data sample.
2: Hyperparameters: B: batch size; negative sample number for the context-invariant con- trastive learning component; Ldis: negative sample number for the disentangling component.
3: Parameters: 8 8%z 8 0²: parameters for 9¹ 92 92, 92 respectively; w: parameters of IEDR except for Of, 0% Or, 05.
4: function CONTRASTOO ATTIAIIVEEELLLRNNNGSSRRRRRR ((;, i..B)
5:
for -- 1,... B do
6: (o + (u,,
7:
ContertGinn t BanddnChhoiee Cj ContertGen(())
8:
9:
(0 )ij f% (o) C2) for = ,,.,, do
Generate positive samples.
10:
Generate negative samples.
ur Choncce (us))icce( (0 )e (ue C.)
t randomChoices (u, (o )nn) = fic (1g2) Cj) 12: end for
14:
EcICI(u;, c:) + Equation (4 based on the above positive and negative samples
16: return
17: end function
18: function CONTRASTIVELLAARIIGG ITEM() (TEM({ c=))B..))
Symmetric to CONTRASTIVELEARNINS 20: end function
19:
21: function DISENTANGLEMENT USER({(u)) Ci))i=1:B)
22:
for i
1,... B do (0 (0% t (u,, ei)
23:
24:
(o qo, ((o)) (o%)) pored +- go, (0,((00))
Generate positive samples. pred)
Opos MSE((O")) (o spred) apos MSE((O), l (
25:
26:
aneg
t- 0, anng +- 0
for j .... Lidis do randomChoiee (( )i, (ox))); Jim..))
Generate negative samples.
28:
(0) ( )r (0
29:
(o pred 40, ((0%) )r), (oh)) )rred qo, ((o
30:
Oneg f- aneg + (MSE((", oo )is (ot) predd
anog aneg +MSE((O, )sc, ( prredy
32:
end for
aprr)) { + a
34:
(La)))) + end for
ancy anee (apos + apos))
Nes
35:
36: return average(( (Coos (wo.eaaa.....ooooeeecceeeeeaeeee
37: end function
38: function DISENTANGLEMENT ITEM (vv i=)):=
39:
Symmetric to DISENTANGLEMENT USER.
40: end function
41:
24

---

Under review as a conference paper at ICLR 2023
Algorithm Batch stochastic gradient descent training of IEDR (continued).
42:
shuf fle(( Us: ())r=1.)) for each batch {(u,, Us. 1-1:n do
43:
44:
for i !,.... B do
Line 45-47 are the recommendation prediction module.
45:
u. + fa;)) v2 + fo(U)) t- f
46:
(o (ot lit fir (ui, ci), (0 (0z)) c.),
47:
V/ 5 fprea( (o )is, (o )i, (o Ja, (ozz)it
(LRp); t CrossEntropy Vi) end for
49:
50:
CRP + average (( ( i-
51:
LCICE +- CONTRASTIVELEARNTIA USER Cu)
52:
CCICE CONTRASTIEELEARRING.IIII (;
53:
Cirappr Cis DISENTIANGLEMENT.SSIRII (.) Cii-appr: Civr DISENTANGLEMENT ITEMEEN [TEM(((;;,4))=1..))
54:
55:
Freeze w, update 0i. 02 Oi 0% through minimizing R(00, 0%, 07, 03)
Step Step
56:
Freeze 0r, 022 07. 0% update w through minimizing R(w)
end for
25

---

Under review as a conference paper at ICLR 2023
Frappe IEDR
Frappe noDis
User
Item
User
Item
Frappe noCL
Frappe noCIED
User
Item
User
Item
Yelp IEDR
Yelp noDis
User
Item
User
Item
Yclp noCL
Yelp noCIED
User
Item
User
Item
Frappe vCLUB
Yelp vCLUB
User
Item
User
Item
Figure 12: The complete intrinsic-extrinsic disentanglement visualizations in L-SNE, The blue dots are intrin- sic representations, and the red dots extrinsic representations.
26


Reviews:
[{"review_id": "uFm33sFGKQO", "reviewer": "Reviewer_TrQ3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an intrinsic-extrinsic disentangled model named IEDR for recommendation. The idea is that user-related (or item-related) features are intrinsic factors that don't change with time, while contextual features are extrinsic factors that change with time.\n\nThus they aim to get the intrinsic factors by maximizing the agreement between the encoded representation of the same user under different contexts, and minimizing the agreement between the encoded representation of the same context with different users. This is done by optimizing an InfoNCE loss in the contrastive learning module.\n\nBesides, they also try to disentangle extrinsic factors by minimizing the mutual information between the extrinsic factors and the above-generated intrinsic factors.\n\nExperiments are conducted on two public datasets to conclude the superior performance of IEDR in contrast to other CTR prediction models.", "summary_of_the_review": "Due to the weakness in motivation and evaluation, I would vote for rejection.\n\n---\nAFTER THE AUTHOR RESPONSE:\n\nThe author's response does help to solve my confusion about the motivation of the proposed method. \n\nI'm still a bit worried about the convincing of the experimental results.  The authors explain that Taobao and Avazu datasets are not suitable for the scenario because they only span less than two weeks. But there are some other available benchmark datasets like Movielens-20M with timestamp features that span decades. My suggestion is that at least the main comparison result should also be provided for the two Amazon datasets in addition to Frappe and Yelp.\n\nDue to the above points, I would like to raise my score to marginally above the acceptance threshold.", "strength_and_weaknesses": "S1. Overall, the paper is well-organized and well-written. The main idea is clearly illustrated and the figures are quite helpful for understanding the proposed IEDR method.\n\nS2. They conduct extensive experiments on two public datasets, including performance comparison, ablation study, visualization, and case study. The main comparison results are promising and the other results meet their expectations.\n\n---\nW1. The main concern is about the motivation of the proposed method. The paper claims that intrinsic factors are user-related (or item-related) factors that don't change with context, while extrinsic factors are contextual features that don't change with user or item. \n\nThen it is straightforward to just use user-related and item-related features as intrinsic factors, and use contextual features as extrinsic factors. It doesn't bring any benefit to getting the intrinsic factors by extracting context-free factors from the input of both contextual features and user (item) features. Thus the method seems to be superfluous as it is trying to seek an inferior answer with the best answer in mind.\n\nI have noticed the clarification of preventing the trivial solution in Appendix C. But I don't think it is a trivial solution to map user features to intrinsic factors and to map contextual features to extrinsic factors, because they are just the right answer according to both the motivation and the proposed method.\n\nW2.  Although the paper conducted experiments on two public datasets, the used datasets are both very small datasets with less than one million samples, which hurts the convincing of the comparison results. It is strongly suggested to add results on benchmark CTR prediction datasets like Avazu or Taobao.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "mXMmLlgNWsN", "reviewer": "Reviewer_vJvU", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method (IEDR) for recommendation systems which explicitly models the intrinsic representations and the extrinsic representations. Intrinsic representations and extrinsic representations are invariant and variant to context features, respectively; a proposed contractive learning component and a mutual information minimization-based disentangling component are collaboratively utilized. Experiments on public datasets demonstrate IEDR’s effectiveness over the state-of-the-art methods.", "summary_of_the_review": "See “Strength And Weaknesses” and “Clarity, Quality, Novelty And Reproducibility”.", "strength_and_weaknesses": "This paper is well written and clearly presented. \n\nThe idea of splitting up context-invariant intrinsic factors and context-based extrinsic factors are interesting and inspiring.\n\nThe mutual information minimization-based disentangling component looks over-designed. This component directly introduces the alternative optimization procedure. As such, a natural question is not addressed: does IEDR suffer from more computational overhead? This issue should be carefully discussed and also empirically validated in the experiment section.\n\nThe implementation of IEDR in Section 5.1.2 is not sufficiently clear. Is the user behavior sequence not used as a context for IEDR? If so, the experimental results of this section is surprising in some sense.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "dUQaNH1oXS", "reviewer": "Reviewer_ywxS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors discussed intrinsic and extrinsic factors that jointly affect users’ decisions in items selection. This problem is interesting and important. This paper provides a new factor learning method for more contexts that existing studies have not mainly considered.\nThe contributions are:\nC1. Proposing a context-agnostic intrinsic and extrinsic factors learning framework that can tackle different contexts.\nC2. Exploiting a context-invariant contrastive learning and a mutual information minimization-based disentangling to disentangle the factors, and providing an explainable results about the different factors.\nC3. Providing a theorem analysis about the proposed contrastive loss.\nC4. Experiments show the effectiveness of the proposed framework.\n", "summary_of_the_review": "This paper tries to solve a novel problem in recommender systems. The motivation and method design are clearly presented. However, one main concern is the flexibility of the method since there are many tasks and hyper-parameters need to be optimized, the tuning process might be very complicated.", "strength_and_weaknesses": "Strength:\nS1. Interesting problem and novel idea in recsys. \nS2. Addressing the issue of factors learning in complex contexts and disentangling the factors into intrinsic and extrinsic parts for better recommendation.\nS3. Providing some theorem analysis about the optimization problem.\n\nWeaknesses: \nW1. Some claims are not clear. For example, in 4.2.1, why choosing the negative pairs from the same context with different users? Are there any assumptions? \nW2. The proposed method seems to be complicated. Should give a specific analysis of the time complexity.\nW3. Please provide more analysis and experiments on other specific context except for the time context in the section 5.1.2.\nW4. There is no comparison with other methods in the section 5.3.2. \nW5. The results on Movies and CDs datasets are missing in Table 1.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Mzi5CDhGOnK", "reviewer": "Reviewer_N1Ar", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a generic model to learn intrinsic and extrinsic factors from various contexts using a contrastive learning component, and a disentangling component. Experimental results on real-world datasets demonstrate the effectiveness of the method. Additionally, theoretical proofs are provided for the refined vCLUB.", "summary_of_the_review": "The research problem is well-motivated. The authors give completed theoretical proofs and conduct rich experiments. But the technic is lack innovation.", "strength_and_weaknesses": "Strength: \n1.\tThe paper proposes a good perspective to learn disentangled representations using contrastive learning.\n2.\tIt is interesting to use psychological theory to divide users' purchasing decisions into intrinsic and extrinsic reasons\n3.\tThe rich experiment results prove the effectiveness of the method.\n\nWeakness: \n1.\tThe proposed method learns two types of disentangled representations. I’m curious about how authors can distinguish whether they are intrinsic or extrinsic factors. \n2.\tSome technics are based on previous work. Despite some improvement, it is still not innovative enough, e.g., existing SIGN, vCLUB, using contrastive learning to learn disentangled representations.\n3.\tAbout the experiment part, when comparing the sequential recommending methods, there is no detailed description of the settings of the proposed method, like how they process.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject