OpenRaiser/Intern-Atlas huggingface dataset:
ADL: Adaptive Distribution Learning Framework for Multi-Scenario CTR Prediction
Authors: Jinyun Li, Huiwen Zheng, Yuanlin Liu, Minfang Lu, Lixia Wu, Haoyuan Hu
Venue: Conference SIGIR 2023
Year: 2023

Large-scale commercial platforms usually involve numerous business scenarios for diverse business strategies. To provide click-through rate (CTR) predictions for multiple scenarios simultaneously, existing promising multi-scenario models explicitly construct scenario-specific networks by manually grouping scenarios based on particular business strategies. Nonetheless, this pre-defined data partitioning process heavily relies on prior knowledge, and it may neglect the underlying data distribution of each scenario, hence limiting the model's representation capability. Regarding the above issues, we propose Adaptive Distribution Learning (ADL): an end-to-end optimization distribution framework which is composed of a clustering process and classification process. Specifically, we design a distribution adaptation module with a customized dynamic routing mechanism. Instead of introducing prior knowledge for pre-defined data allocation, this routing algorithm adaptively provides a distribution coefficient for each sample to determine which cluster it belongs to. Each cluster corresponds to a particular distribution so that the model can sufficiently capture the commonalities and distinctions between these distinct clusters. Our results on both public and large-scale industrial datasets show the effectiveness and efficiency of ADL: the model yields impressive prediction accuracy with more than 50% reduction in time cost during the training phase when compared to other methods.
CFGL-LCR: A Counterfactual Graph Learning Framework for Legal Case Retrieval
Authors: Kun Zhang 0041, Chong Chen 0001, Yuanzhuo Wang, Qi Tian 0001, Long Bai 0002
Venue: Conference KDD 2023
Year: 2023

Legal case retrieval, which aims to find relevant cases based on a short case description, serves as an important part of modern legal systems. Despite the success of existing retrieval methods based on Pretrained Language Models, there are still two issues in legal case retrieval that have not been well considered before. First, existing methods underestimate the semantics associations among legal elements, e.g., law articles and crimes, which played an essential role in legal case retrieval. These methods only adopt the pre-training language model to encode the whole legal case, instead of distinguishing different legal elements in the legal case. They randomly split a legal case into different segments, which may break the completeness of each legal element. Second, due to the difficulty in annotating the relevant labels of similar cases, legal case retrieval inevitably faces the problem of lacking training data. In this paper, we propose a counterfactual graph learning framework for legal case retrieval. Concretely, to overcome the above challenges, we transform the legal case document into a graph and model the semantics of the legal elements through a graph neural network. To alleviate the low resource and learn the causal relationship between the semantics of legal elements and relevance, a counterfactual data generator is designed to augment counterfactual data and enhance legal case representation. Extensive experiments based on two publicly available legal benchmarks demonstrate that our CFGL-LCR can significantly outperform previous state-of-the-art methods in legal case retrieval.
Seven pitfalls to avoid when running controlled experiments on the web
Authors: Thomas Crook, Brian Frasca, Ron Kohavi, Roger Longbotham
Venue: Conference KDD 2009
Year: 2009

Controlled experiments, also called randomized experiments and A/B tests, have had a profound influence on multiple fields, including medicine, agriculture, manufacturing, and advertising. While the theoretical aspects of offline controlled experiments have been well studied and documented, the practical aspects of running them in online settings, such as web sites and services, are still being developed. As the usage of controlled experiments grows in these online settings, it is becoming more important to understand the opportunities and pitfalls one might face when using them in practice. A survey of online controlled experiments and lessons learned were previously documented in Controlled Experiments on the Web: Survey and Practical Guide (Kohavi, et al., 2009). In this follow-on paper, we focus on pitfalls we have seen after running numerous experiments at Microsoft. The pitfalls include a wide range of topics, such as assuming that common statistical formulas used to calculate standard deviation and statistical power can be applied and ignoring robots in analysis (a problem unique to online settings). Online experiments allow for techniques like gradual ramp-up of treatments to avoid the possibility of exposing many customers to a bad (e.g., buggy) Treatment. With that ability, we discovered that it's easy to incorrectly identify the winning Treatment because of Simpson's paradox.
Recommendations for streaming data
Authors: Karthik Subbian, Charu C. Aggarwal, Kshiteesh Hegde
Venue: Conference CIKM 2016
Year: 2016

Recommender systems have become increasingly popular in recent years because of the broader popularity of many web-enabled electronic commerce applications. However, most recommender systems today are designed in the context of an offline setting. The online setting is, however, much more challenging because the existing methods do not work very effectively for very large-scale systems. In many applications, it is desirable to provide real-time recommendations in large-scale scenarios. The main problem in applying streaming algorithms for recommendations is that the in-core storage space for memory-resident operations is quite limited. In this paper, we present a probabilistic neighborhood-based algorithm for performing recommendations in real-time. We present experimental results, which show the effectiveness of our approach in comparison to state-of-the-art methods.
Beyond the baseline: Establishing the value in mobile phone based poverty estimates
Authors: Chris Smith-Clarke, Licia Capra
Venue: Conference WWW 2016
Year: 2016

Within the remit of `Data for Development' there have been a number of promising recent works that investigate the use of mobile phone Call Detail Records (CDRs) to estimate the spatial distribution of poverty or socio-economic status. The methods being developed have the potential to offer immense value to organisations and agencies who currently struggle to identify the poorest parts of a country, due to the lack of reliable and up to date survey data in certain parts of the world. However, the results of this research have thus far only been presented in isolation rather than in comparison to any alternative approach or benchmark. Consequently, the true practical value of these methods remains unknown. Here, we seek to allay this shortcoming, by proposing two baseline poverty estimators grounded on concrete usage scenarios: one that exploits correlation with population density only, to be used when no poverty data exists at all; and one that also exploits spatial autocorrelation, to be used when poverty data has been collected for a few regions within a country. We then compare the predictive performance of these baseline models with models that also include features derived from CDRs, so to establish their real added value. We present extensive analysis of the performance of all these models on data acquired for two developing countries -- Senegal and Ivory Coast. Our results reveal that CDR-based models do provide more accurate estimates in most cases; however, the improvement is modest and more significant when estimating (extreme) poverty intensity rates rather than mean wealth.
Searching microblogs: coping with sparsity and document quality
Authors: Nasir Naveed, Thomas Gottron, Jérôme Kunegis, Arifah Che Alhadi
Venue: Conference CIKM 2011
Year: 2011

Two of the main challenges in retrieval on microblogs are the inherent sparsity of the documents and difficulties in assessing their quality. The feature sparsity is immanent to the restriction of the medium to short texts. Quality assessment is necessary as the microblog documents range from spam over trivia and personal chatter to news broadcasts, information dissemination and reports of current hot topics. In this paper we analyze how these challenges can influence standard retrieval models and propose methods to overcome the problems they pose. We consider the sparsity's effect on document length normalization and introduce "interestingness" as static quality measure. Our results show that deliberately ignoring length normalization yields better retrieval results in general and that interestingness improves retrieval for underspecified queries.
HLBPR: A Hybrid Local Bayesian Personal Ranking Method
Authors: Xu Chen, Pengfei Wang, Zheng Qin, Yongfeng Zhang
Venue: Conference WWW 2016
Year: 2016

Bayesian Personal Ranking(BPR) method is a well-known model due to its high performance in the task of item recommendation. However, this method fail to distinguish user preference among the non-interacted items. In this paper, to enhance traditional BPR's performance, we introduce and analyse a hybrid method, namely Hybrid Local Bayesian Personal Ranking method(HLBPR for short). Our main idea is to construct additional item preference pairs among the products which haven't been purchased, and then utilize the extened pairs to optimize the ranking object. Experiments on two real-world transaction datasets demonstrated the effectiveness of our approach as compared with the state-of-the-art methods.
Localized Matrix Factorization for Recommendation based on Matrix Block Diagonal Forms
Authors: Yongfeng Zhang, Min Zhang, Yiqun Liu, Shaoping Ma, Shi Feng
Venue: Conference WWW 2013
Year: 2013

Matrix factorization on user-item rating matrices has achieved significant success in collaborative filtering based recommendation tasks. However, it also encounters the problems of data sparsity and scalability when applied in real-world recommender systems. In this paper, we present the Localized Matrix Factorization (LMF) framework, which attempts to meet the challenges of sparsity and scalability by factorizing Block Diagonal Form (BDF) matrices. In the LMF framework, a large sparse matrix is first transformed into Recursive Bordered Block Diagonal Form (RBBDF), which is an intuitionally interpretable structure for user-item rating matrices. Smaller and denser submatrices are then extracted from this RBBDF matrix to construct a BDF matrix for more effective collaborative prediction. We show formally that the LMF framework is suitable for matrix factorization and that any decomposable matrix factorization algorithm can be integrated into this framework. It has the potential to improve prediction accuracy by factorizing smaller and denser submatrices independently, which is also suitable for parallelization and contributes to system scalability at the same time. Experimental results based on a number of real-world public-access benchmarks show the effectiveness and efficiency of the proposed LMF framework.
The Death and Life of Great Italian Cities: A Mobile Phone Data Perspective
Authors: Marco De Nadai, Jacopo Staiano, Roberto Larcher, Nicu Sebe, Daniele Quercia, Bruno Lepri
Venue: Conference WWW 2016
Year: 2016

The Death and Life of Great American Cities was written in 1961 and is now one of the most influential book in city planning. In it, Jane Jacobs proposed four conditions that promote life in a city. However, these conditions have not been empirically tested until recently. This is mainly because it is hard to collect data about "city life". The city of Seoul recently collected pedestrian activity through surveys at an unprecedented scale, with an effort spanning more than a decade, allowing researchers to conduct the first study successfully testing Jacobs's conditions. In this paper, we identify a valuable alternative to the lengthy and costly collection of activity survey data: mobile phone data. We extract human activity from such data, collect land use and socio-demographic information from the Italian Census and Open Street Map, and test the four conditions in six Italian cities. Although these cities are very different from the places for which Jacobs's conditions were spelled out (i.e., great American cities) and from the places in which they were recently tested (i.e., the Asian city of Seoul), we find those conditions to be indeed associated with urban life in Italy as well. Our methodology promises to have a great impact on urban studies, not least because, if replicated, it will make it possible to test Jacobs's theories at scale.
A Comparison of General vs Personalised Affective Models for the Prediction of Topical Relevance
Authors: Ioannis Arapakis, Konstantinos Athanasakos, Joemon M. Jose
Venue: Conference SIGIR 2010
Year: 2010

Information retrieval systems face a number of challenges, originating mainly from the semantic gap problem. Implicit feedback techniques have been employed in the past to address many of these issues. Although this was a step towards the right direction, a need to personalise and tailor the search experience to the user-specific needs has become evident. In this study we examine ways of personalising affective models trained on facial expression data. Using personalised data we adapt these models to individual users and compare their performance to a general model. The main goal is to determine whether the behavioural differences of users have an impact on the models' ability to determine topical relevance and if, by personalising them, we can improve their accuracy. For modelling relevance we extract a set of features from the facial expression data and classify them using Support Vector Machines. Our initial evaluation indicates that accounting for individual differences and applying personalisation introduces, in most cases, a noticeable improvement in the models' performance.
A Study on the Effects of Personalization and Task Information on Implicit Feedback Performance
Authors: Ryen W. White, Diane Kelly
Venue: Conference CIKM 2006
Year: 2006

While Implicit Relevance Feedback (IRF) algorithms exploit users' interactions with information to customize support offered to users of search systems, it is unclear how individual and task differences impact the effectiveness of such algorithms. In this paper we describe a study on the effect on retrieval performance of using additional information about the user and their search tasks when developing IRF algorithms. We tested four algorithms that use document display time to estimate relevance, and tailored the threshold times (i.e., the time distinguishing relevance from non-relevance) to the task, the user, a combination of both, or neither. Interaction logs gathered during a longitudinal naturalistic study of online information-seeking behavior are used as stimuli for the algorithms. The findings show that tailoring display time thresholds based on task information improves IRF algorithm performance, but doing so based on user information worsens performance. This has implications for the development of effective IRF algorithms.
The million song dataset challenge
Authors: Brian McFee, Thierry Bertin-Mahieux, Daniel P. W. Ellis, Gert R. G. Lanckriet
Venue: Conference WWW 2012
Year: 2012

We introduce the Million Song Dataset Challenge: a large-scale, personalized music recommendation challenge, where the goal is to predict the songs that a user will listen to, given both the user's listening history and full information (including meta-data and content analysis) for all songs. We explain the taste profile data, our goals and design choices in creating the challenge, and present baseline results using simple, off-the-shelf recommendation algorithms.
CCCF: Improving Collaborative Filtering via Scalable User-Item Co-Clustering
Authors: Yao Wu, Xudong Liu, Min Xie, Martin Ester, Qing Yang
Venue: Conference WSDM 2016
Year: 2016

Collaborative Filtering (CF) is the most popular method for recommender systems. The principal idea of CF is that users might be interested in items that are favorited by similar users, and most of the existing CF methods measure users' preferences by their behaviours over all the items. However, users might have different interests over different topics, thus might share similar preferences with different groups of users over different sets of items. In this paper, we propose a novel and scalable method CCCF which improves the performance of CF methods via user-item co-clustering. CCCF first clusters users and items into several subgroups, where each subgroup includes a set of like-minded users and a set of items in which these users share their interests. Then, traditional CF methods can be easily applied to each subgroup, and the recommendation results from all the subgroups can be easily aggregated. Compared with previous works, CCCF has several advantages including scalability, flexibility, interpretability and extensibility. Experimental results on four real world data sets demonstrate that the proposed method significantly improves the performance of several state-of-the-art recommendation algorithms.
Spotting misbehaviors in location-based social networks using tensors
Authors: Evangelos E. Papalexakis, Konstantinos Pelechrinis, Christos Faloutsos
Venue: Conference WWW 2014
Year: 2014

The proliferation of mobile devices that are capable of estimating their position, has lead to the emergence of a new class of social networks, namely location-based social networks (LBSNs for short). The main interaction between users in an LBSN is location sharing. While the latter can be realized through continuous tracking of a user's whereabouts from the service provider, the majority of LBSNs allow users to voluntarily share their location, through check-ins. LBSNs provide incentives to users to perform check-ins. However, these incentives can also lead to people faking their location, thus, generating false information. In this work, we propose the use of tensor decomposition for spotting anomalies in the check-in behavior of users. To the best of our knowledge, this is the first attempt to model this problem using tensor analysis.
Spam double-funnel: Connecting web spammers with advertisers
Authors: Yi-Min Wang, Ming Ma, Yuan Niu, Hao Chen
Venue: Conference WWW 2007
Year: 2007

Spammers use questionable search engine optimization (SEO) techniques to promote their spam links into top search results. In this paper, we focus on one prevalent type of spam - redirection spam - where one can identify spam pages by the third-party domains that these pages redirect traffic to. We propose a five-layer, double-funnel model for describing end-to-end redirection spam, present a methodology for analyzing the layers, and identify prominent domains on each layer using two sets of commercial keywords. one targeting spammers and the other targeting advertisers. The methodology and findings are useful for search engines to strengthen their ranking algorithms against spam, for legitimate website owners to locate and remove spam doorway pages, and for legitimate advertisers to identify unscrupulous syndicators who serve ads on spam pages.
Capturing feature-level irregularity in disease progression modeling
Authors: Kaiping Zheng, Wei Wang, Jinyang Gao, Kee Yuan Ngiam, Beng Chin Ooi, James Wei Luen Yip
Venue: Conference CIKM 2017
Year: 2017

Disease progression modeling (DPM) analyzes patients' electronic medical records (EMR) to predict the health state of patients, which facilitates accurate prognosis, early detection and treatment of chronic diseases. However, EMR are irregular because patients visit hospital irregularly based on the need of treatment. For each visit, they are typically given different diagnoses, prescribed various medications and lab tests. Consequently, EMR exhibit irregularity at the feature level. To handle this issue, we propose a model based on the Gated Recurrent Unit by decaying the effect of previous records using fine-grained feature-level time span information, and learn the decaying parameters for different features to take into account their different behaviours like decaying speeds under irregularity. Extensive experimental results in both an Alzheimer's disease dataset and a chronic kidney disease dataset demonstrate that our proposed model of capturing feature-level irregularity can effectively improve the accuracy of DPM.
Context-Aware Intent Identification in Email Conversations
Authors: Wei Wang, Saghar Hosseini, Ahmed Hassan Awadallah, Paul N. Bennett, Chris Quirk
Venue: Conference SIGIR 2019
Year: 2019

Email continues to be one of the most important means of online communication. People spend a significant amount of time sending, reading, searching and responding to email in order to manage tasks, exchange information, etc. In this paper, we study intent identification in workplace email. We use a large scale publicly available email dataset to characterize intents in enterprise email and propose methods for improving intent identification in email conversations. Previous work focused on classifying email messages into broad topical categories or detecting sentences that contain action items or follow certain speech acts. In this work, we focus on sentence-level intent identification and study how incorporating more context (such as the full message body and other metadata) could improve the performance of the intent identification models. We experiment with several models for leveraging context including both classical machine learning and deep learning approaches. We show that modeling the interaction between sentence and context can significantly improve the performance.
Learning to model relatedness for news recommendation
Authors: Yuanhua Lv, Taesup Moon, Pranam Kolari, Zhaohui Zheng, Xuanhui Wang, Yi Chang
Venue: Conference WWW 2011
Year: 2011

With the explosive growth of online news readership, recommending interesting news articles to users has become extremely important. While existing Web services such as Yahoo! and Digg attract users' initial clicks by leveraging various kinds of signals, how to engage such users algorithmically after their initial visit is largely under-explored. In this paper, we study the problem of post-click news recommendation. Given that a user has perused a current news article, our idea is to automatically identify "related" news articles which the user would like to read afterwards. Specifically, we propose to characterize relatedness between news articles across four aspects: relevance, novelty, connection clarity, and transition smoothness. Motivated by this understanding, we define a set of features to capture each of these aspects and put forward a learning approach to model relatedness. In order to quantitatively evaluate our proposed measures and learn a unified relatedness function, we construct a large test collection based on a four-month commercial news corpus with editorial judgments. The experimental results show that the proposed heuristics can indeed capture relatedness, and that the learned unified relatedness function works quite effectively.