LBD: Decouple Relevance and Observation for Individual-Level Unbiased Learning to Rank
Abstract
Using Unbiased Learning to Rank (ULTR) to train the ranking model with biased click logs has attracted increased research interest.
The key idea is to explicitly model the user’s observation behavior when building the ranker with a large number of click logs.
Considering the simplicity, recent efforts are mainly based on the position bias hypothesis, in which the observation only depends on the position.
However, this hypothesis does not hold in many scenarios due to the neglect of the distinct characteristics of individuals in the same position.
On the other hand, directly modeling observation bias for each individual is quite challenging, since the effects of each individual’s features on relevance and observation are entangled.
It is difficult to ravel out this coupled effect and thus obtain a correct relevance model from click data.
To address this issue, we first present the concept of coupling effect for individual-level ULTR.
Then, we develop the novel Lipschitz and Bernoulli Decoupling (LBD) model to decouple the effects on relevance and observation at the individual level.
We prove theoretically that our proposed method could recover the correct relevance order for the ranking objective.
Empirical results on two LTR benchmark datasets show that the proposed model outperforms the state-of-the-art baselines and verify its effectiveness in debiasing data. Our codes are available at https://github.com/Keytoyze/Lipschitz-Bernoulli-Decoupling .
Introduction
Learning to Rank (LTR) has been widely used in modern information retrieval systems, which aims to learn a ranking model that sorts documents with their relevance.
Recently, using implicit feedback (e.g. clicks) instead of relevance labels (typically obtained by human annotation) to train the ranking model has become popular since implicit feedback is an attractive proxy of relevance, which is cheap and relatively easy to obtain on a large scale [38].
However, they inherently contain a lot of bias from user behavior [28].
Using Unbiased Learning to Rank (ULTR) to remove these biases has attracted increasing research interest [1, 29].
The key is to factorize the clicks into relevance probabilities and observation probabilities and model a user’s observation probability on an item in a ranking list.
By reweighing the click signals based on the reciprocal of observation probabilities, ULTR provides an unbiased estimate of the ranking objective.
Most of the existing ULTR methods are based on the position bias hypothesis [29, 4, 43, 17, 3, 14], which assumes that the observation only depends on the position.
We refer to this method as group-level ULTR method, in which various heterogeneous features (typically encoded with the [Table] Equal [Table]
information of query, document, and user) share the same observation probability according to some property.
However, the observation bias inherently varies across individuals.
For example, the exact match in result titles and abstracts (which is usually shown in a different presentation style) affects users’ attention and judgment, known as attractive bias [45, 7].
Except for attractive bias, there are lots of factors that can influence a user’s observation [10, 47, 37, 7, 24, 31, 32].
These factors are encoded in (or can be inferred from) the features, which leads to heterogeneous observation.
To be in line with these real phenomena, we argue there is an urgent need to develop an individuallevel ULTR, which can capture the effect of heterogeneous features for observation behavior.
That is to say, the observation model should be a function related to documents.
[Figure]
Unfortunately, this problem is very challenging.
As shown in Figure 1, there exists a coupling effect from features to observation and relevance.
There are infinite solutions to factorize the click rate into the relevance function and observation function (all of them are functions of features), but few of them imply a correct ranking model.
In the extreme case, a naive ranking model which outputs identical relevance probability still has a chance to perfectly fit the click data with a properly designed association between features and observation [41].
A straightforward solution to avoid coupling is to divide the features manually into two separate parts: one related to relevance and the other related to observation.
But it’s intractable in practice because some of these features affect both.
For example, the exact match in result titles and abstracts affects observation (attractive bias), but on the other hand, it is naturally an important feature encoding how relevant the document is.
Besides, result domain [7] and users’ search intent [37] are also common factors affecting both.
To address the coupling effect problem, we find that it is unnecessary to correctly estimate the true relevance probability (named hard decoupling), yet only need to recover the correct relevance order from click data (named soft decoupling) because of the ranking objective.
Given that we don’t know the true relevance, this goal is difficult to achieve directly on the ranking model.
We instead modify the observation model, which helps to achieve soft decoupling when the click probability estimation is unbiased.
We propose two special observation models, the Lipschitz Observation Model (LOM), which has a Lipschitz constraint β on the observation function, and the Bernoulli Observation Model (BOM) which randomly cancels debiasing operation with a probability of t on the observation function output.
The two models act on different mechanisms, thus they are complementary and can be combined.
We provide a theoretical guarantee that when β and t are bounded, these two observation models can achieve soft decoupling whenever estimating unbiased click probability with a ranking model.
Given this, we propose a principled novel individual-based ULTR framework, called Lipschitz and Bernoulli Decoupling (LBD) model.
We conducted comprehensive experiments on two
LTR benchmark datasets, which shows that the proposed LBD outperforms the baseline methods and verifies its effectiveness in debiasing data.
We further studied the different application areas of the two decoupling techniques and found that the performance can be better when they are combined.
To the best of our knowledge, we are the first to study the coupling effect for ULTR and propose a feasible solution to decouple the effects.
The main contributions of this work are:
We present the concept of coupling effect for individual-level ULTR, as well as two targets to achieve decoupling.
We propose a novel Lipschitz and Bernoulli Decoupling model, which could help to decouple the effects on relevance and observation.
We provide theoretical proof to guarantee the decoupling ability of our proposed techniques.
We conducted comprehensive experiments on two LTR benchmark datasets in the coupling effect scene, which shows that the two decoupling techniques have different application areas, and combining them can outperform the baseline methods.
Related Work
Debiasing Click Data for LTR .
Debiasing click data is an important research direction in the information retrieval field.
We review this line of related work in § A.
Individual-Level Observation .
There is much work finding that the observation bias can vary from instance to instance.
The user’s observation can be influenced by result type [32], result quality [10], search intent [37], result domain [7, 24], exact match [45], and even the relevance itself [31].
These factors can be inferred from document features.
Particularly, [44] found that in mobile search, the user may be directly satisfied with the result without clicking any links on certain kinds of documents.
[37] argued that users with different search intent might have different examination behavior.
All these discoveries suggest the necessity to estimate the bias at the individual level.
Recently, researchers attempted to find solutions in ULTR fields to deal with individual-level observation bias.
[23] proposed to manually impose a stronger regularization on the position bias estimated by Unbiased LambdaMART, to alleviate the influence of user’s different click behaviors, while it’s heuristic and has no theoretical proof.
[30] proposed wLambdaMART that estimates the confidence of click data with a few labeled data to correct bias, without the need to design a specific bias type.
But this method requires extra labeled data.
[39] employed heterogeneous treatment effect estimation techniques to estimate the bias between the position k and the position 1 .
However, this method is based on the assumption that the click rates at the first position reveal their accurate relevance.
This is only true when the observation probability at the first position is constant.
Preliminaries
In this work, we use bold-faced letters to denote vectors, upper-case letters to denote random variables, and the corresponding lower-case letters to denote values.
P ( · ) denotes the distribution of a random variable.
Generally, the core of LTR is to learn a ranking model f , which assigns a relevance score to a document with its query-document features X ∈ R l .
Then, the documents for a query can be sorted in descending order of their scores.
In traditional LTR, the ranking model is directly learned with the true relevance score R to make the ranking order of the query list close to the optimal order [29].
In the click setting, users’ click logs are used as substitutes of relevance due to their abundance and cheapness.
While click signals are naturally biased from the true relevance [27], much prior work follows the examination hypothesis to address it, which can be formulated as:
[Equation]
where r ( x ) =
Pr ( R = 1 | X = x ) , o p ( x ) =
Pr ( O = 1 | X = x , P = p ) and c p ( x )
=
Pr ( C = 1 | X = x , P = p ) denote the probabilities of relevance, observation and click, and P denotes the position.
By explicitly modeling the bias effect via observation probability, it is able to attain an unbiased estimate of the ranking objective.
Our method
In this section, we first introduce the coupling effect in § 4.1, and propose two decoupling methods in § 4.2 and § 4.3.
4.1 Coupling Effect
Let ˆ r ( x ; θ 1 ) denote the estimation of ranking model, and ˆ o ( x ; θ 2 ) denote the estimation of observation model.
We use clicks to train the two models using Eq.(1), to let their production close to the click.
What we care about is if the relevance model is correct when the model converges.
To simplify the later analysis, we suppose that the model has a universal approximation capacity [22] to predict an unbiased click probability, and make the following assumption.
Assumption 1 (Unbiased Click Prediction) .
There exists a parameter space Θ ∗ which consists of pairs of ranking model parameter and observation model parameter, such that for all ( θ ∗ 1 , θ ∗ 2 )
∈
Θ ∗ , the predictive click probability is unbiased:
[Equation]
From Assumption 1 we can find that the factorization of relevance probability ˆ r ( x ; θ ∗ 1 ) and observation probability ˆ o p ( x ; θ ∗ 2 ) is unidentifiable given the click probability c p ( x ) , which makes it challenging to uncover the correct ranking model.
This is because the features X have an effect on both relevance R and observation O .
It’s difficult to distinguish which parts of the features affect relevance and observation, respectively.
In the extreme case, the ranking model will degenerate into a trivial case that always predicts the same scores for all documents [41].
We refer to this problem as the coupling effect .
Suppose the correct factorization is c p ( x )
=
r ( x ) · o p ( x ) , where r ( x ) is the true relevance and o p ( x ) is the true observation.
To achieve a good decoupling of the features’ effect on relevance and observation, we aim to make ˆ r ( x ; θ 1 ) close to r ( x ) whenever predicting an accurate click probability.
Based on this, we introduce the definition of hard decoupling as follows.
Definition 1 (Hard Decoupling) .
For any features x with the true relevance r ( x ) , a parameter space Θ ′ ⊆
Θ ∗ can do hard decoupling, if for all ( θ ∗ 1 , θ ∗ 2 )
∈ Θ ′ , we have r ( x ) =
ˆ r ( x ; θ ∗ 1 ) .
Θ ∗ is defined in Assumption 1.
Considering the pairwise comparison of the ranking objective, we further relax the hard decoupling constraint and refer to this as soft decoupling as follows.
Definition 2 (Soft Decoupling) .
For any two features x 1 and x 2 , without loss of generality we suppose r ( x 1 ) ≥ r ( x 2 ) .
A parameter space Θ ′ ⊆
Θ ∗ can do soft decoupling, if for all ( θ ∗ 1 , θ ∗ 2 )
∈ Θ ′ , we have ˆ r ( x 1 ; θ ∗ 1 ) ≥ ˆ r ( x 2 ; θ ∗ 1 ) .
Θ ∗ is defined in Assumption 1.
Soft decoupling requires identifying the relevance, which is challenging when relevance is unobserved.
In this subsection, we propose two mechanisms (Lipschitz Decoupling and Bernoulli Decoupling) for soft decoupling with theoretical guarantees.
From this point forward, we assume that all of the parameters θ take the value from Θ ∗ , and omit θ of ˆ r ( · ; θ ) and ˆ o p ( · ; θ ) for ease of notation.
4.2 Lipschitz Decoupling
Usually, the rate of change between the observation and features will not be substantial in real scenarios.
Therefore, we propose modifying the observation model’s smoothness to make it match the true observation function, which helps to achieve soft decoupling.
We start with defining the Lipschitz continuity for a function.
Let ||·|| be a norm on R l .
In this work, we employ the Euclidean norm (L-2) when not mentioned otherwise:
|| x
|| =
|| x
|| 2 = (cid:113)(cid:80) li
=1 x 2 i .
Definition 3 (Lipschitz Continuity) .
A function f ( x ) : R l → R is α -Lipschitz to the input x if
[Equation]
where α is referred to as the Lipschitz constant.
Particularly, if an α -Lipschitz function f is differentiable, we have ||∇ x f ( x ) || ≤ α .
We suppose r ( x ) , o p ( x ) , ˆ r ( x ) and ˆ o p ( x ) are all differentiable.
Assumption 2 (True Observation Lipschitz) .
For each p ∈ [ n ] , the true observation o p ( x ) is αLipschitz.
It assumes that there is an upper bound on the rate at which the true observation changes with the features.
This is a reasonable extension for well-known position-based model [29], which assumes that α = 0 .
Similarly, we assume the observation estimation is also Lipschitz.
We refer to this kind of observation model as Lipschitz Observation Model (LOM).
Definition 4 (Lipschitz Observation Model) .
An observation model is β − Lipschitz observation model ( β -LOM), if its estimation ˆ o p ( x ) is β -Lipschitz for all p ∈ [ n ] .
We claim that soft decoupling can be achieved by selecting a lower β , as the following theorem.
Theorem 1 (Lipschitz Decoupling) .
For any two features x 1 and x 2 , without loss of generality we suppose r ( x 1 ) ≥ r ( x 2 ) .
For a β -LOM, if the following condition is satisfied,
[Equation]
where Γ( x 1 , x 2 ) denotes the collection of all curves from point x 1 to point x 2 .
Then we have: ˆ r ( x 1 ) ≥ ˆ r ( x 2 ) .
Remark [-
-]1.
If the gap between r ( x 1 ) and r ( x 2 ) is large (i.e., r ( x 1 ) r ( x 2 ) is large) and α is small enough, there exists a β such that the condition can be satisfied.
Namely, when the true observation doesn’t change too fast with the features, we can select a lower β to achieve soft decoupling.
At a high level, β -LOM controls the capacity of the function space Θ ∗ by restricting the Lipschitz constant β .
While a low β benefits the decoupling, it also suffers from the risk of underfitting.
For example, if we adopt a 0 -LOM (known as the Position-based Model) in the setting that the true observation has a large Lipschitz constant, we may not find a ˆ r ( x ) and a ˆ o p ( x ) such that their product is equal to a given click rate c p ( x ) .
In other words, β has a lower bound.
This can be shown formally in the following theorem.
Theorem 2 (Lower Bound of β ) . Let c ( x ) denote a vector containing click rates at each position:
[Equation]
For any two different features x 1 and x 2 , we have
[Equation]
Remark [-
-]2.
We consider a special situation where the true observation is 0-Lipschitz ( α = 0 ).
In this case, o is no longer a function of x but a constant, thus we can find m, n such that m c ( x 1 ) − n c ( x 2 ) is a zero vector, and the lower bound of β can be zero.
When the true observation has a large Lipschitz number, m c ( x 1 ) − n c ( x 2 ) can no longer be a zero vector and β should be larger.
It means the models should change quickly with the input feature to fit clicks.
Theorem 1 and Theorem 2 give the upper bound and lower bound of β , respectively.
The proofs of these two theorems are provided in Appendix § B.
Since the bounds focus on the pairwise comparison of documents and it’s difficult to find the best parameter for the global dataset, in practice we can tune the value of β to achieve the best global effect for soft decoupling.
We refer to this method as Lipschitz Decoupling .
4.3 Bernoulli Decoupling
Unfortunately, when α is large in a given dataset, it’s difficult to select a proper β such that the upper bound and the lower bound are satisfied at the same time.
Under this circumstance, it may perform even worse than that without any debiasing operation (see § 6.3 for discussion).
As a complementary, we propose another decoupling technique, named Bernoulli Decoupling.
Unlike
Lipschitz Decoupling, which makes constraints on the observation model’s estimation ˆ o p ( x ; θ 2 ) , we explicitly add an additional sampling step to transform its output, as follows:
[Equation]
That is to say, we randomly cancel the debiasing operation with a certain probability t (i.e., force the observation model to predict 1 as an observation score and let the ranking model learn directly from clicks).
Since the features are generally collected for the ranking task, the effects from features to relevance are usually stronger than the ones from features to observation.
Thus, we hope the relevance model could learn more effects from features to clicks than the observation model.
The
Bernoulli sampling prevents the relevance model from over-relying on a wrong observation model and encourages the relevance model to relearn what was learned by the observation model.
The probability t can be seen as a global parameter to control the strength of relevance effects.
Similar to our method, Zhao et al.[46] also uses a dropout technique to prevent the model over-relying on the position.
We define the observation model based on this new sampling process as t - Bernoulli Observation Model ( t -BOM).
For convenience, we denote the expectation E (ˆ o ′ p
x ; θ 2 , t ))
as the estimation of the observation model after transformation, and Assumption 1 holds based on this expectation.
We can prove that by choosing a sufficiently large t , BOM can achieve soft decoupling:
Theorem 3 (Bernoulli Decoupling) .
For any two different features x 1 and x 2 , without loss of generality we suppose r ( x 2 ) ≥ r ( x 1 ) .
For a t − BOM, if the following condition is satisfied,
[Equation]
then we have: ˆ r ( x 2 ) ≥ ˆ r ( x 1 ) .
Remark [-
-]3.
If the true observation o p ( x ) is 0-Lipschitz (i.e., α = 0 ), there must exist a t ≤ 1 making the above error condition satisfied to achieve soft decoupling.
For a larger Lipschitz number, we can increase the value of t appropriately.
Similar to LOM, BOM is another method to control the capacity of Θ ∗ , and it also suffers from the risk of underfitting.
In order to make it fit the click rates, t has an upper bound.
Theorem 4 (Upper Bound of t ) .
For any feature x and a t − BOM, we have
[Equation]
Theorem 3 and Theorem 4 give the lower bound and the upper bound of t respectively.
The proofs of the theorems are provided in Appendix § B. Like LOM, we take t as a tuning parameter in practice.
We refer to this method as Bernoulli Decoupling .
Bernoulli Decoupling is a complement to Lipschitz Decoupling, and they can be used simultaneously to improve the efficiency of decoupling.
This is because the range in which the two hyperparameters take effect is an ”or” relationship: Even if β is outside the scope of Theorem 1 and
Theorem 2, as long as t is within the scope of Theorem 3 and Theorem 4, the combined model can still achieve soft decoupling.
This shows that it is beneficial to combine the two models together.
Finally, we propose our ( β, t ) -Lipschitz and Bernoulli Decoupling (LBD) model, which combines the β -LOM and t -BOM.
We treat LBD as our complete solution for individual-based ULTR.
Model Implementation
[Figure] Up to this point, we have shown that a ( β, t ) -LBD can effectively decouple clicks on a relatively lowLipschitz observation dataset.
In this section, we first introduce the implementation of the proposed
LBD, then present the optimization objective of the proposed model.
The overall model architecture is illustrated in Figure 2. We first use a neural network f as the ranking model (take features as input) and a neural network g as the observation model (take features and position as inputs), and output the relevance and the observation:
ˆ r = f
( x ) , ˆ o p =
g p ( x ) .
Lipschitz Decoupling Based on Theorem 1 and 2, we need to enforce a proper Lipschitz constraint on the observation model to employ Lipschitz Decoupling.
Since we use a neural network to implement the observation model, it’s difficult to directly control the Lipschitz constant without adjusting the network structure too much.
As an easy alternative, [19] proposed to add a penalty on the gradient norm.
Similar to them, we employ a gradient penalty loss function: L gp ( x ) = λ (cid:80) ni
=1 ||∇ x
(ˆ o i )
||
This is based on the fact that a differentiable function is k -Lipschtiz if and only if it has gradients with the norm at most k everywhere.
The gradient penalty loss encourages the model to have a low Lipschitz constant, and we can adjust λ to control the penalty level and obtain models with different Lipschitz constants.
Thus, λ can be a substitute for β .
Bernoulli Decoupling Based on Theorems 3 and 4, in order to implement the Bernoulli Decoupling technique and avoid degradation of the debiasing process, we sample a γ from Bernoulli distribution parameterized by 1 − t .
Our final logarithmic observation probability is generated by log
ˆ o ′ p
= γ log ˆ o p , where γ ∼ Bernoulli (1 − t ) .
The final predictive click score is: log ˆ c p = log ˆ r + log ˆ o ′ p .
Objective Function and Training Process Similar to [4], we adopt a list-wise loss based on softmax cross entropy.
Given a query q with the document ranking list π q = { x 1 , x 2 , · · · , x n } , we generate click ˆ c p for feature x p at each position p .
Suppose the real click signals for each document in π q are c 1 , c 2 , · · · , c n , we use the following supervise loss to train our models:
[Equation]
Finally, the objective function based on a query q can be written as L ( q ) =
L sv ( q )+ (cid:80) ni
=1 L gp ( x i ) .
We jointly update the ranking model and the observation model with the derivatives of L ( q ) and repeat the process until the algorithm converges.  Experiments
In this section, we describe our experimental setup and show the empirical results.
6.1 Experimental Setup
Dataset We followed the standard setup in CLTR [5, 4, 15, 39] and conducted semi-synthetic experiments on two widely used benchmark datasets: Yahoo!
LETOR ‡ [12] and Istella-S § [33].
We provide further details for these datasets in Appendix § C.1.
We followed the given data split of training, validation and testing.
To generate initial ranking lists for click simulation, we followed the standard process [29, 4, 15] to train a Ranking SVM model [26] with 1% of the training data with relevance labels, and sort the documents. Based on these initial ranking lists, we sampled clicks according to the examination hypothesis. Following the steps proposed by [13], we set the relevance probability to be: [Equation]
where y x ∈ [0 , y max ] is the relevance level of x by human annotation, and y max = 4 in both of the two datasets. ϵ is the click noise level and we set ϵ = 0 . 1 as the default setting.
To the best of our knowledge, we are the first to study the coupling effects, thus there is no precedent for the observation simulation.
In order to simulate the observation, We first select some crux features as the factors that are related to observation, such as result type, result quality or search intent, etc.
Because the exact meaning of the features given by the datasets is unknown, we follow the method in previous work [39, 15] to select some features as substitutes: (1) use normalized features and relevances in the total dataset to train ExtRa Trees [18]; (2) based on this model, select and combine the top-10 important features as crux features.
We adopted the methodology in [39] to model the feature’s influence on observation.
The observation probability is set to be:
[Equation]
where v p is the position-based examination probability at position p by eye-tracking studies [27].
w is a 10-dimensional vector uniformly drawn from [ − η, η ] , where η is a hyperparameter to control the dependency between the observation and crux features.
x ′ denotes the crux features of x .
Note that η can be explained as the coupling level between relevance and observation.
Baselines We combined the state-of-the-art ULTR methods and two LTR models for comparison.
Debiasing methods include Vectorization [14], RegressionEM [43], DLA [4], PairDebias [23], HTE [39], Labeled Data (uses human-annotated relevance labels to train the ranker directly) and
Click Data (uses the raw click data to train the ranker directly).
Ranking models include DNN and Linear .
We adopted the codes in ULTRA framework [5, 6] to implement RegressionEM, DLA, PairDebias and the ranking models, and kept the same hyperparameters.
Note that Regression-EM,
DLA and PairDebias are all group-level ULTR methods.
HTE is an individual-level ULTR, but it’s not a jointly learning algorithm, so we compared it in another scene.
Our method is referred to as LBD .
We also use three degenerate versions for ablation study: (1) set t = 0 to remove the Bernoulli sampling step (named as LBD Lips ), (2) set λ = 0 to remove the gradient penalty loss function (named as LBD Ber ), and (3) remove both (named as Unlimited ).
Training details can be found in § C.2.
6.2 Performance on different rankers and datasets
[Table]
We first study the performance of jointly learning-based ULTR models.
Table 1 summarizes the
NDCG@10 results about the performance of different rankers on different datasets.
We see that our proposed LBD significantly outperforms almost all the other baseline methods since we properly handle the coupling effect.
The results of other baselines are compatible with those reported in [6].
Particularly, we have the following findings:
LBD works better than LBD Lips and LBD Ber , which indicates that using Lipschitz Decoupling and Bernoulli Decoupling at the same time is better than using them alone.
The reason seems to be that the two decoupling methods have different conditions (see Theorem 1 and Theorem 3).
When using them together, the scope of decoupling can be expanded.
Unlimited works worst than almost all the best baselines, indicating the existence of the coupling effect.
If we train the models unlimitedly, the performance would be quite bad.
Additionally, the coupling effect is more serious when using the Linear ranker.
Lipschitz Decoupling performs better than Bernoulli Decoupling on Yahoo!
dataset, while the opposite is true on the Istella-S dataset.
It shows that the application of two decoupling methods is tightly related to
the data distribution.
6.3 Performance on different coupling levels
In this experiment, we investigate the performance under different degrees of coupling level on the
Yahoo! dataset, where the ranker is DNN.
We vary η from 0 to 0.6 when generating clicks. [Figure] [Figure]
[Figure] [Figure] [Figure] [Figure]
[Figure] Baselines [Figure] [Figure]
n D C [Figure] Decoupling Techniques
[Figure]
Figure 3(a) shows the influence of coupling levels on different offline ULTR methods.
We can observe that our model achieves the best performance under all degrees of coupling level.
When the observation is 0-Lipschitz (i.e., η = 0 ), our model and DLA perform similarly, since both of them employ a similar listwise loss function.
The debiasing performance of other baselines diminishes with the increased coupling level.
For a large coupling level ( η ≥ 0 . 3 ), all baselines perform worse than the raw click data.
In contrast, our proposed method always performs better than click data, which shows the strong decoupling ability of LBD.
Besides, our method exhibits a downward trend with the increased coupling level, which demonstrates that it’s more difficult to decouple the effects when the true observation has a large Lipschitz constant.
Figure 3(b) shows the performance of different decoupling methods we propose in this work.
One can see that for a low degree of coupling level ( η < 0 . 2 ), LBD Lips performs better than LBD Ber , which implies that LOM can decouple the effects under a low Lipschitz observation dataset.
For a large coupling level, LOM is worse than BOM in turn.
One explanation is that BOM can avoid the deterioration of ULTR when the relevance and observation effects are difficult to decouple since it randomly drops out the debiasing operation.
Besides, LBD always performs better than LBD Lips and LBD Ber , once again showing that using them together could expand the scope of decoupling.
The previous study focuses most on joint-learning ULTR methods.
To compare the Heterogeneous
Treatment Effect (HTE), we conduct another experiment based on intervention data.
Following [39], we simulate 2,560,000 queries for the intervention experiment before training the ranking model (this number is equal to the number of total queries we used to train models).
For each query, we randomly chose a k ∈ [ n ] and swapped the document at the first position with the one at the k -th position.
We collected clicks generated on these ranking lists and calculated the HTE estimator ˆ τ k ( x ) .
Based on ˆ τ k ( x ) , we debiased the click data and use it to train a ranking model directly.
We also used an accurate HTE estimator τ k ( x ) from the click model we used, which can be regarded as the upper bound of the HTE method.
Based on the clicks corrected with τ k ( x ) , we train a ranking model directly (named HTE-oracle).
We also use these clicks to train our model (named LBD-HTE-oracle).
We present the results in Figure 3(c).
We can observe that our LBD (trained without any intervention data) performs even better than HTE and HTE-oracle in a low degree of coupling level because HTE is based on an assumption that the click rates at the first position can reveal the correct relevance, which is biased when the observation probability is not constant.
For a high degree of coupling level, HTE performs better than our method.
One possible reason is that intervention can augment the dataset (since each document has an opportunity to display at the top position), which is helpful when the effects are difficult to decouple.
Besides, when our model is trained on the clicks corrected with the oracle HTE, it can always outperform the vanilla HTE method.
This demonstrates once again that HTE is still biased and verifies our decoupling effectiveness.
6.4 Hyper Parameter Analysis
From the previous discussion, we know that the hyperparameters should have an upper bound and lower bound depending on the data.
To investigate it, we study the performance with different hyperparameters. Figure 4 shows the results when changing hyper parameters. We can see that there is a focal point on each heat map, and the grid points around the focal point have a lower performance, [Figure]
which verifies our discussion Besides, the focal point gradually shifts from the upper right corner to the lower-left corner as an increase. This shows again that Lipschitz Decoupling takes a better effect on a low Lipschitz observation dataset, while Bernoulli Decoupling performs better on a large Lipschitz observation dataset
[Figure]  Conclusion In this work, we take the first step to studying the coupling effect in individual-level ULTR.
We propose two goals: hard decoupling, where the true relevance probability should be estimated ac- curately and soft decoupling, where the correct relevance order can be recovered.
We modify the observation model to achieve soft decoupling when the click probability estimation is unbiased
We propose two special observation models, Lipschitz Observation Model (LOM) and Bernoulli Observation Model (BOM).
We provide a theoretical guarantee that when their hyperparameters are bounded , soft decoupling can be achieved by these methods.
We conduct comprehensive experiments on two LTR benchmark datasets, which shows that the proposed LBD outperforms the baseline methods and verifies its effectiveness in debiasing data
Limitations.
(1) For simplicity, our theoretical results are based on Assumption 1, which requires the models to fit clicks perfectly.
It would be interesting to investigate how the decoupling effect will be affected if the click estimation is not so accurate.
(2) Our bounds in the theorems contain a value α from Assumption 2 which is related to the real data generating process.
In some cases such as sparse, high dimensional, or high noise cases, the value of alpha may be too large and invalidate the derived bounds.
How to decouple data in these cases is an open question.
Societal Impact
To the best of our knowledge, the approaches in this paper raise no major ethical concerns or societal consequences.
Researchers and practitioners from the ULTR domain may benefit from our research since debiasing implicit feedback is a significant challenge in real-world applications.
The worst possible outcome when the proposed approach fails is that it reduces to the standard position-based observation estimation and stops making the desired impact.
Finally, the proposed approach aims at solving the coupling effects of the data, the extent of which depends on the properties of the data. Acknowledgments
Thanks to Yusu Hong for the discussion on the theorems, and the reviewers for their valuable comments and suggestions.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[No] This work is not resourceintensive.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[N/A] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A] Appendix A Extended Related Work
There are two groups of approaches to debias click data for ranking. The first is to model user’s behavior to infer relevance from biased click signals, known as click models [16, 21, 20, 9, 8]. However, most click models focus on predicting clicks, rather than optimizing the ranking performance [29, 4]. They are usually separated from the LTR frameworks, and the relevance inference is an afterthought [4].
The second group tries to directly learn unbiased ranking models from biased clicks, known as unbiased learning to rank (ULTR).
Joachims et al.
[29] proposed the inverse propensity scoring (IPS) method to reweigh the click signals based on the reciprocal of observation probabilities (called propensity scores) and provide an unbiased estimate of the ranking objective.
The propensity scores are estimated by randomized experiments [29, 42], which hurt users’ experience, unfortunately.
To address it, Agarwal et al.
[3] and Fang et al.
[17] proposed to do intervention harvest by exploiting click logs with multiple ranking models.
Nevertheless, they have a relatively narrow scope of application due to the strict assumption to construct interventional sets.
Recently, some researchers proposed to jointly estimate relevance and bias [43, 4, 23, 25, 14].
Similar to them, our proposed method could jointly train the ranking model and observation model without intervention.
On the other side, researchers developed models to deal with all kinds of click bias, based on ULTR framework.
According to types of bias, they can be divided into several categories: (1) position bias, where the bias only depends on position [43, 4, 23, 11, 14]; (2) selection bias, where some documents have a zero probability of being observed since it is ranked below a certain cutoff [34, 35, 36]; (3) trust bias, where users are more likely to click incorrectly on higher-ranked items [2, 41]; (4) contextual bias, where the observation bias varies from query to query [17, 39]; (5) interactional bias, where the observation is influenced by interactions among clicks in the same ranking list [15, 40].
However, these types of work mainly model the observation bias at a group level and ignore the characteristics of each document.
B Proofs of the Theorems B.1 Proof of Theorem
Let σ p ( x ) =
ˆ o p ( x ) o p ( x ) , for each p ∈ [ n ] we have:
[Equation]
This implies the following bounds:
[Equation]
Now by using the condition in Theorem 1, we obtain:
[Equation]
In the above deviation, the second inequality uses the upper bound of ||∇ x ( σ p ( x ))
|| , and the third inequality uses the condition in Theorem 1.
Now by using Assumption 1, we have:
[Equation]
where the inequality uses the above conclusion, and the first and the last equality uses Assumption 1.
This implies the desired bound.
B.2 Proof of Theorem
Given that this model is a β -LOM, by using Assumption 1 we have
[Equation]
[Equation]
∀ p ∈ [ n ] , where c p ( x ) =
o p ( x ) r ( x ) .
Now by taking the lower bound on the right side over p , we obtain: ∃ ˆ r ( x 1 ) , ˆ r ( x 2 ) ∈ [0 , 1] , s.t.,
[Equation]
|| x 1 − x 2 || .
By taking m = 1 ˆ r
x 1 ) and n = 1 ˆ r ( x 1 ) , we can obtain the desired result.
B.3 Proof of Theorem
From the condition, we have:
[Equation]
Let ˆ o ′ p
( x ; t ) denote the estimation of a t -BOM.
From the definition, we have:
[Equation]
Note that 0 ≤ ˆ o p ( x ) ≤ 1 , we obtain t ≤ ˆ o ′ p
( x ; t ) ≤ 1 .
By using Assumption 1, we have
[Equation]
This shows that t can reduce the upper bound of ˆ r ( x ) .
Therefore, we obtain:
[Equation]
where the first and the last inequality uses the Eq.(5), and the second inequality uses the Eq.(4).
B.4 Proof of Theorem
From Eq.(5), we obtain:
[Equation]
Therefore we have: r ( x ) sup p ∈ [ n ] o p ( x ) ≤ 1 t r ( x ) inf p ∈ [ n ] o p ( x ) , which implies the desired bound.
C Experiments C.1 Further details of datasets
Table 2 shows the characteristics of the two datasets we used, Yahoo!
and Istella-S.
[Table]
C.2 Training details
[Table]
[Table]
[Table]
[Table]
[Table]
Linear
[Table]
[Table]
Istella-S
[Table]
[Table]
Linear 0.
We combined the baselines with the same ranking models: DNN and Linear, where the implementation and hyperparameters are the same as ULTRA framework [5, 6].
To make fair comparisons, all the baselines and our models shared the same number of the hidden layer.
The only difference between our model and baselines was the output dimensions.
We trained these methods with a batch size of 256.
We used
SGD to train the Linear Ranker, and
AdaGrad to train the DNN Ranker.
For
LBD, we selected the hyper-parameter λ from { 1 , 100 , 10000 } and t from { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 } .
For convenience, we expanded the output dimension of the ranking model, and separate them into the ranking model (with 2 outputs representing the mean and the variance) and observation models (with 2 n outputs representing each observation distribution on each positions) as our implementation, since they have the same input data.
We used nDCG@ k ( k = 1 , 3 , 5 , 10) and ARP as the performance metrics.
Each model was trained for 10,000 epochs, and we adopted the hyperparameters with the best results based on nDCG@tested on the validation set.
We run each experiment 5 times and reported the average results testing on the test set.
Final hyper-parameters used in all experiment settings are listed in Table 3, and the full experimental results are listed in Table 4.
[Table]
Unlimited 1.611 −− 0.
0.687 −−
LBD Lips 1.631 −− 0.640 − 0.
0.631 − 0.686 −−
LBD Ber 1.464 + 0.645 ++ 0.618 ++ 0.643 ++ 0.707 ++
LBD 1.
0.651 ++ 0.623 ++ 0.648 +
+ 0.709 ++
Linear
Labeled Data 1.
DLA 1.
Vectorization 1.
PairDebias 1.
RegressionEM 2.
Click Data 1.
Unlimited 2.121 − 0.
0.514 − 0.536 − 0.592 −
LBD Lips 1.761 −− 0.619 + 0.583 − 0.605 −− 0.659 −−
LBD Ber 1.586 −− 0.613 + 0.
LBD 1.581 −− 0.616 ++ 0.591 ++ 0.618 ++ 0.