SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zhongzhou Liu
School of Computing and Information Systems
Singapore, 178902
zzliu.2020@phdcs.smu.edu.sg
Yuan Fang*
yfang@smu.edu.sg
Min Wu
Singapore, 138632
wumin@i2r.a-star.edu.sg
Causality-based recommendation systems focus on the causal effects of user-item interactions resulting from item exposure (i.e., which items are recommended or exposed to the user), as opposed to conventional correlation-based recommendation. They are gaining popularity due to their multi-sided benefits to users, sellers and platforms alike. However, existing causality-based recommendation methods require additional input in the form of exposure data and/or propensity scores (i.e., the probability of exposure) for training. Such data, crucial for modeling causality in recommendation, are often not available in real-world situations due to technical or privacy constraints. In this paper, we bridge the gap by proposing a new framework, called Propensity Estimation for Causality-based Recommendation (PROPcARE). It can estimate the propensity and exposure from a more practical setup, where only interaction data are available without any ground truth on exposure or propensity in training and inference. We demonstrate that, by relating the pairwise characteristics between propensity and item popularity, PROPcARE enables competitive causality-based recommendation given only the conventional interaction data. We further present a theoretical analysis on the bias of the causal effect under our model estimation. Finally, we empirically evaluate PROPcARE through both quantitative and qualitative experiments.
Recommendation systems have been widely deployed in many real-world applications, such as streaming services [34, 5], online shopping [17] and job searching [19]. The primary aim of recommendation systems, such as boosting sales and user engagement [10], depends heavily on user interactions, such as clicking on or purchasing items. Hence, a classical paradigm is to predict user-item interactions, and accordingly, recommend items with the highest probability of being interacted (e.g., clicked or purchased) to users [8, 22, 33, 35]. This paradigm ignores the causal impact behind recommendation [31]: If an item already has a high probability of being interacted by a user without being recommended, is there really a need to recommend the item to this user?
(a) Previous framework where exposure is observable
Figure 1: Causal diagrams under different frameworks.  $\mathrm{pop}_i$  is the popularity (prior) of item  $i$ .  $Y_{u,i}$  indicates if user  $u$  interacts with item  $i$ .  $Z_{u,i}$  indicates if item  $i$  is exposed to user  $u$ .
(b) Our framework PropCare where exposure is unobservable and popularity is incorporated as prior
Unobserved variable
Observed variable
Recently, a few studies [38, 30, 28, 37] have shifted the focus to this question. They aim to recommend an item based on the uplift, also called the causal effect, in the user's behavior (e.g., clicks or purchases) caused by different treatments (i.e., recommending/exposing the item or not) [9]. Such causality-based recommendation systems posit that recommending items with a higher causal effect carries greater merit than those with a higher interaction probability. Typical approaches involve quantifying the causal effect in the user's behavior, based on the observed data and the counterfactual treatment [37]. Existing works assume that the exposure data (i.e., whether an item has been recommended to a user or not), or the propensity scores [25] (i.e., the probability of recommending/exposing an item to a user), are observable at least during the training stage. However, in real-world scenarios, those data are often unavailable. For instance, while it is feasible to log each user who purchased an item in a e-commerce platform, it may be difficult to distinguish between purchases made with or without exposure due to technical and privacy constraints in determining if a user has been exposed to the item a priori. Without the exposure data and/or propensity scores provided during training, existing causality-based recommenders cannot be deployed.
Toward practical causality-based recommendation, we consider a more relaxed and realistic setup where exposure and propensity scores are not observable. Although some previous works [38, 42, 1, 14, 21] have attempted to estimate propensity scores in a different context (e.g., addressing biases in recommendation), they further suffer two key limitations. First, most state-of-the-art methods still require exposure data to train the propensity estimator [38, 1, 14]. Second, they fail to integrate prior knowledge into the propensity estimator, resulting in less robust estimation. To address these challenges and bridge the data gap in many recommendation scenarios and benchmarks, we propose a novel framework of Propensity Estimation for Causality-based Recommendation (PROPcARE), to estimate the propensity score and exposure of each item for each user. Specifically, we observe a pairwise characteristic that relates propensity scores and item popularity when the probability of user-item interaction is well controlled. (The observation is formalized as Assumption 1 and empirically validated in Sect. 4.2.) Based on the observation, we incorporate item popularity as prior knowledge to guide our propensity estimation. Furthermore, we present a theoretical analysis on the bias of the estimated causal effect. The analysis enables us to investigate the factors that influence our estimation and subsequently guide our model and experiment design.
In summary, we compare previous propensity estimation and PROPCARE in Fig. 1, highlighting our key advantages: PROPCARE does not need propensity or exposure data at all, and incorporates prior information for robust estimation. The contributions of this paper include the following. (1) Our proposed framework bridges the gap in existing causality-based recommendation systems, where the propensity score and/or exposure data are often unavailable but required for model training or inference. (2) We incorporate the pairwise relationship between propensity and item popularity as prior knowledge for more robust propensity estimation. We present a further analysis on the factors that influence our model. (3) We conduct extensive experiments to validate the effectiveness of PROPCARE through both quantitative and qualitative results.
Causal effect estimation in recommendation While typical recommendation systems consider positive feedback or interactions like clicks and purchases as successful, it may be more beneficial to optimize the uplift in interactions, also called the causal effect, solely caused by recommendations [18]. However, obtaining the causal effect in real-world scenarios is challenging because of its
counterfactual nature [9]. Conducting online A/B tests to compare exposure strategies may be feasible but expensive and susceptible to selection bias [27]. To address these issues, several causal effect estimators have been proposed. The naive estimator [30] assumes random exposure assignment to all user-item pairs, which is inconsistent with most recommendation scenarios. The inverse propensity score (IPS) estimator [30] incorporates the propensity score, defined as the probability of exposure [25], to overcome this limitation. Direct model estimators like CausCF [38] directly predict outcomes using parametric models based on different exposure statuses. A recently proposed doubly robust estimator [37] integrates a parametric model with the non-parametric IPS estimator for reduced bias and variance. However, these estimators require access to input data containing propensity scores and/or exposure data, at least during the training stage, which are often unavailable due to technical and privacy limitations.
Propensity estimation in recommendation Existing causal effect estimation approaches require exposure data and/or propensity scores at least in training, which are frequently unavailable or subject to the missing-not-at-random (MNAR) issue [32]. Hence, we have to rely on their estimations. Some methods estimate propensity in a heuristic way, such as using item popularity [30] or other side information (e.g., items participating in promotional campaigns) [28]. However, these estimations lack personalization and may result in noisy results. Other approaches utilize interaction models (also called click models) [24, 3, 42, 21] to relate propensity scores, relevance and interactions. However, without additional constraints, the interaction model alone can be difficult to optimize as we will elaborate in Sect. 4.1. Besides, matrix factorization [16, 38], linear regression [28], dual learning [21] and doubly robust learning [14] can also learn propensity scores, but they assume exposure data as training labels or known variables, which is incompatible with our setup without any observable propensity or exposure data.
Data notations Consider a typical recommendation dataset that contains only interactions between users and items, such as purchases or clicks. Let  $Y_{u,i} \in \{0,1\}$  denote the observed interaction between user  $u \in \{1,2,\dots,U\}$  and item  $i \in \{1,2,\dots,I\}$ .  $D = \{(Y_{u,i})\}$  denotes the collection of observed training user-item interaction data. Note that our framework does not assume the availability of any additional data except the interaction data. Moreover, let  $Z_{u,i} \in \{0,1\}$  denote an unobservable indicator variable for exposure, i.e.,  $Z_{u,i} = 1$  iff item  $i$  is exposed/recommended to user  $u$ . We use  $p_{u,i}$  to represent the propensity score, which is defined as the probability of exposure, i.e.,  $p_{u,i} = P(Z_{u,i} = 1)$ .
Causal effect modelling Let  $Y_{u,i}^{0}$  and  $Y_{u,i}^{1} \in \{0,1\}$  be the potential outcomes for different exposure statuses. Specifically,  $Y_{u,i}^{1}$  is defined as the interaction between user  $u$  and item  $i$  when  $i$  has been exposed to  $u$ . Accordingly,  $Y_{u,i}^{0}$  is the interaction when  $i$  has not been exposed to  $u$ . This setup assumes a counterfactual model: In the real world only one of the scenarios can happen, but not both. Subsequently, the causal effect  $\tau_{u,i} \in \{-1,0,1\}$  is defined as the difference between the two potential outcomes [26], i.e.,  $\tau_{u,i} = Y_{u,i}^{1} - Y_{u,i}^{0}$ . In other words,  $\tau_{u,i} = 1$  means recommending item  $i$  to user  $u$  will increase the interaction between  $u$  and  $i$  and  $\tau_{u,i} = -1$  means the opposite.  $\tau_{u,i} = 0$  means recommending or not will not change the user's interaction behavior. Naturally, users, sellers and platforms could all benefit from recommendations that result in positive causal effects.
Causal effect estimation The causal effect cannot be directly computed based on observed data due to its counterfactual nature. Among the various estimators introduced in Sect. 2, direct parametric models [38, 37] are sensitive to the prediction error of potential outcomes [37]. Hence, high-quality labeled exposure data are required in parametric models, which is not the setup of this work. To avoid this issue, we adopt a nonparametric approach, known as the inverse propensity score (IPS) estimator [30], for causal effect estimation as follows.
$$
\hat {\tau} _ {u, i} = \frac {Z _ {u , i} Y _ {u , i}}{p _ {u , i}} - \frac {(1 - Z _ {u , i}) Y _ {u , i}}{1 - p _ {u , i}}. \tag {1}
$$
Interaction model In line with prior works [21, 42, 39], we adopt an interaction model  ${}^{1}$  [24, 3] that assumes the following relationship between interactions, propensity and relevance:
$$
y _ {u, i} = p _ {u, i} r _ {u, i}, \tag {2}
$$
where  $y_{u,i} = P(Y_{u,i} = 1)$  is the probability of interaction between user  $u$  and item  $i$ , and  $r_{u,i}$  represents the probability that item  $i$  is relevant to user  $u$ .
4 Proposed Approach: PROPCARE
In this section, we introduce our propensity estimation approach PROPCARE. We start with a naive approach, followed by our observation on prior knowledge, before presenting the overall loss for propensity learning and how the learned propensity can be used for causality-based recommendation. We end the section by discussing a theoretical property of our estimation.
4.1 Naive propensity estimator
The overall objective is to estimate propensity scores and exposure from a more practical setup where only interaction data are observable. Since the propensity score  $p_{u,i}$  is the probability of exposure  $P(Z_{u,i} = 1)$ , we focus on the estimation of propensity scores, whereas the corresponding exposure can be readily sampled based on the propensity. The interaction model in Eq. (2) intuitively leads us to the naive loss function below.
$$
\mathcal {L} _ {\text {n a i v e}} = - Y _ {u, i} \log f _ {p} \left(\mathbf {x} _ {u, i}; \Theta_ {p}\right) f _ {r} \left(\mathbf {x} _ {u, i}; \Theta_ {r}\right) - \left(1 - Y _ {u, i}\right) \log \left(1 - f _ {p} \left(\mathbf {x} _ {u, i}; \Theta_ {p}\right) f _ {r} \left(\mathbf {x} _ {u, i}; \Theta_ {r}\right)\right), \tag {3}
$$
where  $\mathbf{x}_{u,i} = f_{e}(u,i;\Theta_{e})$  is a joint user-item embedding output by a learnable embedding function  $f_{e};f_{p}$  and  $f_{r}$  are learnable propensity and relevance functions to produce the estimated propensity score  $\hat{p}_{u,i}$  and relevance probability  $\hat{r}_{u,i}$ , respectively. Note that each learnable function  $f_{*}$  is parameterized by  $\Theta_{*}$ , and we implement each as a multi-layer perceptron (MLP).
However, through the naïve loss we cannot learn meaningful propensity and relevance functions  $(f_{p}$  and  $f_{r})$ , since they are always coupled in a product and can be collapsed into one function. It is equivalent to learning a single interaction function, instead of learning each individual factor.
4.2 Incorporating prior knowledge
To avoid the above issue, one solution is to introduce prior knowledge to further constrain the propensity or relevance function. In particular, it has been observed that more popular items will have a higher chance to be exposed [41]. The popularity of item  $i$ ,  $\mathrm{pop}_i$ , is defined based on the total number of observed interactions in the dataset, i.e.,  $\mathrm{pop}_i = \sum_{u=1}^{U} Y_{u,i} / \sum_{j=1}^{I} \sum_{u=1}^{U} Y_{u,j}$ . However, this observation [41], while intuitive, is not adequate in explaining the relationship between popularity and exposure. In particular, items with a higher interaction probability also tend to have a higher chance to be exposed, especially when prior exposure was decided by recommenders in the classical paradigm. To incorporate popularity as a prior toward propensity/exposure estimation, we propose to introduce a control on the interaction probability, and formulate the following assumption.
Assumption 1 (Pairwise Relationship on Popularity and Propensity) Consider a user  $u$  and a pair of items  $(i, j)$ . Suppose the popularity of item  $i$  is greater than that of  $j$ , and their interaction probabilities with user  $u$  are similar. Then it follows that item  $i$  is more likely to be exposed to user  $u$  than item  $j$  is.
The intuition is that, when a user's interaction probabilities are similar toward two items  $i$  and  $j$ , but item  $i$  is more likely to be exposed to the user, the reason could be item  $i$  is more popular than  $j$ . Our assumption essentially places a control on the interaction probability to eliminate its influence on the exposure, and simultaneously isolate the effect of popularity on the exposure.
Empirical validation of Assumption 1 In the following, we examine our assumption by calculating the fraction of item pairs that satisfy this assumption in three datasets, namely, DH ORIGINAL,
Also called the "click" model when the interaction refers to click in some literature.
Figure 2: Histogram of item pairs  $(i,j)$  that satisfy Assumption 1. The bins are based on the inverse similarity in interaction probabilities,  $|y_{u,j} - y_{u,i}|$ , divided by  $\{0,0.01,\dots,0.09,0.1,0.2,\dots,0.5\}$ . That is, the first 10 bins have an equal width of 0.01 and the last 4 bins have an equal width of 0.1.
DH和个人ized and ML (see Sect. 5.1 for dataset descriptions). Specifically, we first estimate the probability  $y_{u,i}$  of each interaction  $Y_{u,i}$  using logistic matrix factorization [11]. We also obtain the propensity score  $p_{u,i}$  from ground truth values provided by the datasets (note that we only use the ground truth for evaluation purposes, not in model training or inference). Then, for each user  $u$ , we place an item pair  $(i,j)$ , where a randomly sampled  $i$  is paired with each of the remaining items, into several bins based on  $i$  and  $j$ 's similarity in their interaction probability with  $u$ . More specifically, each bin  $b$  contains  $(i,j)$  pairs such that  $|y_{u,j} - y_{u,i}|$  falls into  $b$ 's boundaries. Finally, we compute the ratio of  $(i,j)$  pairs consistent with Assumption 1 to the total pairs in each bin  $b$ , as follows.
$$
\operatorname {r a t i o} _ {b} = \frac {1}{U} \sum_ {u = 1} ^ {U} \frac {\# \text {i t e m p a i r s} (i , j) \text {f o r u s e r} u \text {i n b i n} b \text {s . t .} \left(p _ {u , j} - p _ {u , i}\right) \left(\operatorname {p o p} _ {j} - \operatorname {p o p} _ {i}\right) > 0}{\# \text {i t e m p a i r s} (i , j) \text {s a m p l e d f o r u s e r} u \text {i n b i n} b}. \tag {4}
$$
We report the ratios in Fig. 2. It can be observed that when  $|y_{u,j} - y_{u,i}|$  is smaller (i.e.,  $i$  and  $j$ 's interaction probabilities with  $u$  are more similar), a higher fraction of items pairs in the bin satisfy our assumption. In contrast, when  $|y_{u,j} - y_{u,i}|$  grows larger (i.e., the interaction probabilities are not well controlled and become less similar), the validity of the original observation [41] becomes weaker. In summary, the results on the three datasets demonstrate the validity of Assumption 1.
Integrating prior knowledge Based on Assumption 1, we utilize item popularity to inject prior knowledge on the probability of exposure (i.e., propensity score) through the following loss.
$$
- \log \left[ \sigma \left(f _ {p} \left(\mathbf {x} _ {u, i}\right) - f _ {p} \left(\mathbf {x} _ {u, j}\right)\right) \right] \text {s . t .} \operatorname {p o p} _ {i} > \operatorname {p o p} _ {j}, y _ {u, i} \approx y _ {u, j}, \tag {5}
$$
where  $\sigma$  is the sigmoid activation and  $y_{u,i}$  is computed as  $f_{p}(\mathbf{x}_{u,i})f_{r}(\mathbf{x}_{u,i})$ . While Eq. (3) models propensity in a point-wise manner, Eq. (5) incorporates popularity as prior knowledge in a pairwise manner. The advantage is twofold. First, it decouples the propensity and relevance functions, using only item popularity which can be readily computed from the interaction data shown earlier without the need for external information. Second, by separating the estimated propensity of less popular items and more popular items, it prevents all predicted values from clustering in a narrow range near 1 or 0. This is beneficial in mitigating the issue of high variance caused by extreme values [37].
To materialize the control  $y_{u,i} \approx y_{u,j}$  on the interaction probabilities in Eq. (5), we adopt the following loss that involves a soft version of  $y_{u,i} \approx y_{u,j}$ .
$$
\mathcal {L} _ {\mathrm {p o p}} = - \kappa_ {u, i, j} \log \left[ \sigma \left(\operatorname {s g n} _ {i, j} \cdot \left(f _ {p} \left(\mathbf {x} _ {u, i}\right) - f _ {p} \left(\mathbf {x} _ {u, j}\right)\right)\right) + \sigma \left(\operatorname {s g n} _ {i, j} \cdot \left(f _ {r} \left(\mathbf {x} _ {u, j}\right) - f _ {r} \left(\mathbf {x} _ {u, i}\right)\right)\right) \right], \tag {6}
$$
where  $\operatorname{sgn}_{i,j} \in \{1, -1\}$  is the sign of  $(\mathrm{pop}_i - \mathrm{pop}_j)$  and  $\kappa_{u,i,j}$  is a weighting function such that it will assign a higher weight if  $y_{u,i}$  and  $y_{u,j}$  are closer. Specifically, we choose  $\kappa_{u,i,j} = e^{\eta (y_{u,i} - y_{u,j})^2}$ , where  $\eta < 0$  is a learnable parameter. Moreover, according to the interaction model in Eq. (2), for a fixed  $y_{u,i}$ , a higher  $p_{u,i}$  implies a lower  $r_{u,i}$ . This explains the additional constraint on the relevance function  $f_r$  in Eq. (6), which will further improve model training.
4.3 Propensity learning
Based on the discussions in Sect. 4.1-4.2, the naive loss essentially optimizes the interaction model, whereas the pairwise loss utilizes popularity as prior information for propensity learning. For more robust learning, we further take a global view on the distribution of propensity scores, which usually follow a long-tailed distribution [40, 42]. In particular, we employ a beta distribution to regularize the
propensity scores, as has been done in literature in modeling propensity or other long-tailed quantities [4, 15]. Overall, we minimize the following loss toward propensity learning:
$$
\min  _ {\Theta} \mathcal {L} = \sum_ {u, i, j} \left(\mathcal {L} _ {\text {n a i v e}} + \lambda \mathcal {L} _ {\text {p o p}}\right) + \mu \mathrm {K L} (Q \| \operatorname {B e t a} (\alpha , \beta)). \tag {7}
$$
Here  $Q$  is the empirical distribution of all estimated propensity scores  $\hat{p}_{u,i}$ .  $\mathrm{Beta}(\alpha ,\beta)$  is a reference beta distribution with parameters  $\alpha$  and  $\beta$  which are selected to simulate a long-tailed shape.  $\mathrm{KL}(\cdot \| \cdot)$  computes the Kullback-Leibler divergence between two distributions.  $\lambda$  and  $\mu$  are trade-off hyperparameters to balance different terms.
Finally, we use the estimated propensity score  $\hat{p}_{u,i}$  to predict the exposure variable  $Z_{u,i}$ :  $\hat{Z}_{u,i} = 1$  if  $\mathrm{Norm}(\hat{p}_{u,i})\geq \epsilon$ , and 0 otherwise, where  $\epsilon$  is a threshold hyperparameter and Norm is a normalization function such as  $Z$ -score normalization. The overall training steps are sketched in Algorithm 1 in Appendix A.
4.4 Causality-based recommendation
We resort to DLCE [30], a state-of-the-art causality-based recommender equipped with an IPS estimator. It takes interaction  $Y_{u,i}$ , exposure  $Z_{u,i}$  and propensity  $p_{u,i}$  as input, and outputs a ranking score  $\hat{s}_{u,i}$  for each user-item pair. Given a triplet  $(u,i,j)$  such that  $u$  is a user and  $i \neq j$  are randomly sampled from the item set, the loss of DLCE is defined as follows [30].
$$
\frac {Z _ {u , i} Y _ {u , i}}{\max  \left(p _ {u , i} , \chi^ {1}\right)} \log \left(1 + e ^ {- \omega \left(\hat {s} _ {u, i} - \hat {s} _ {u, j}\right)}\right) + \frac {\left(1 - Z _ {u , i}\right) Y _ {u , i}}{\max  \left(1 - p _ {u , i} , \chi^ {0}\right)} \log \left(1 + e ^ {\omega \left(\hat {s} _ {u, i} - \hat {s} _ {u, j}\right)}\right), \tag {8}
$$
where  $\chi^1, \chi^0$  and  $\omega$  are hyperparameters. We follow the standard training procedure of DLCE, except that we substitute the ground-truth exposure and propensity score with our estimated values  $\hat{Z}_{u,i}$  and  $\hat{p}_{u,i}$ , respectively, in the above loss. Hence, the entire training process for our propensity learning and DLCE do not require any ground-truth exposure or propensity data. After DLCE is trained, for each user  $u$ , we generate a ranked list of all items based on the optimized  $\hat{s}_{u,i}$ .
4.5 Theoretical property
The performance of causality-based recommendation depends on how accurate we can model the causal effect in the user-item interactions. Although it has been established elsewhere [30] that the IPS estimator defined in Eq. (1) is unbiased as long as exposure  $Z_{u,i}$  and propensity score  $p_{u,i}$  are correctly assigned, in our setup only estimated propensity scores and exposure are available. Thus, we characterize the bias of the IPS estimator when estimations are used instead.
Proposition 1 Suppose we replace the ground truth values of  $Z_{u,i}$  and  $p_{u,i}$  with the estimated  $\hat{Z}_{u,i}$  and  $\hat{p}_{u,i}$  in Eq. (1), respectively. Then, the bias of the estimated causal effect  $\hat{\tau}_{u,i}$  is