OpenRaiser/Intern-Atlas huggingface dataset:
Predicting Audio Advertisement Quality
Authors: Samaneh Ebrahimi, Hossein Vahabi, Matthew Prockup, Oriol Nieto
Venue: WSDM
Year: 2018

Online audio advertising is a particular form of advertising used abundantly in online music streaming services. In these platforms, which tend to host tens of thousands of unique audio advertisements (ads), providing high quality ads ensures a better user experience and results in longer user engagement. Therefore, the automatic assessment of these ads is an important step toward audio ads ranking and better audio ads creation. In this paper we propose one way to measure the quality of the audio ads using a proxy metric called Long Click Rate (LCR), which is defined by the amount of time a user engages with the follow-up display ad (that is shown while the audio ad is playing) divided by the impressions. We later focus on predicting the audio ad quality using only acoustic features such as harmony, rhythm, and timbre of the audio, extracted from the raw waveform. We discuss how the characteristics of the sound can be connected to concepts such as the clarity of the audio ad message, its trustworthiness, etc. Finally, we propose a new deep learning model for audio ad quality prediction, which outperforms the other discussed models trained on hand-crafted features. To the best of our knowledge, this is the first large-scale audio ad quality prediction study.
HeteroNAM: International Workshop on Heterogeneous Networks Analysis and Mining
Authors: Shobeir Fakhraei, Yanen Li, Yizhou Sun, Tim Weninger
Venue: WSDM
Year: 2018

The first International Workshop on Heterogeneous Networks Analysis and Mining is held in Los Angeles, California, USA on February 9th, 2018 and is co-located with the 11th ACM International Conference on Web Search and Data Mining. The goal of this workshop is to bring together computing researchers and practitioners to address challenges in the mining and analysis of real-world heterogeneous networks. This workshop has an exciting program that spans a number of subareas including: graph mining, learning from structured data, statistical relational learning, and network science in general. The program includes six invited speakers, lively discussion on emerging topics, and presentations of several original research papers.
Unsubscription: A Simple Way to Ease Overload in Email
Authors: Iftah Gamzu, Liane Lewin-Eytan, Natalia Silberstein
Venue: WSDM
Year: 2018

The constant growth of machine-generated mail, which today consists of more than 90% of non-spam mail traffic, is a major contributor toinformation overload in email, where users become overwhelmed with a flood of messages from commercial entities. A large part of this traffic is often junk mail that the user would prefer not to receive. Surprisingly, nearly 95% of this traffic is in fact solicited by the users themselves in the form of subscriptions to mailing services. These subscriptions are many times unintentional. Although unsubscription option from such services is enforced by commercial laws, it is hardly actually used by users. We perform a large scale study ofunsubscribable traffic, namely, messages that provide unsubscription option to users. We consider users behavior over such traffic in Yahoo Web mail service, and demonstrate a significant gap between users low interest in this traffic, and their lack of active behavior in decreasing its load. We conjecture that the cause of this gap is the lack of an efficient and easily accessible mechanism that would help users to unsubscribe. We validate our conjecture with an online large scale experiment, where we provide users with a novel mail feature for managing unsubscribable traffic, based on personalized recommendations. The experiment demonstrates the imminent need that exists for such a mechanism.
Offline A/B Testing for Recommender Systems
Authors: Alexandre Gilotte, Clément Calauzènes, Thomas Nedelec, Alexandre Abraham, Simon Dollé
Venue: WSDM
Year: 2018

Online A/B testing evaluates the impact of a new technology by running it in a real production environment and testing its performance on a subset of the users of the platform. It is a well-known practice to run a preliminary offline evaluation on historical data to iterate faster on new ideas, and to detect poor policies in order to avoid losing money or breaking the system. For such offline evaluations, we are interested in methods that can compute offline an estimate of the potential uplift of performance generated by a new technology. Offline performance can be measured using estimators known as counterfactual or off-policy estimators. Traditional counterfactual estimators, such as capped importance sampling or normalised importance sampling, exhibit unsatisfying bias-variance compromises when experimenting on personalized product recommendation systems. To overcome this issue, we model the bias incurred by these estimators rather than bound it in the worst case, which leads us to propose a new counterfactual estimator. We provide a benchmark of the different estimators showing their correlation with business metrics observed by running online A/B tests on a large-scale commercial recommender system.
Workshop on Two-sided Marketplace Optimization: Search, Pricing, Matching & Growth
Authors: Mihajlo Grbovic, Thanasis Noulas
Venue: WSDM
Year: 2018

The 1st International Workshop on Two-sided Marketplace Optimization: Search, Pricing, Matching & Growth(TSMO) will be held in Los Angeles, California, USA on February 9th, 2018, co-located with the 11th ACM International Conference on Web Search and Data Mining(WSDM). The main objective of the workshop is to address the challenges of two-sided marketplace optimization in web-scale settings. The workshop brings together interdisciplinary researchers in information retrieval, recommender systems, personalization, and related areas, to share, exchange, learn, and develop preliminary results, new concepts, ideas, principles, and methodologies on applying data mining technologies to marketplace optimization. We have constructed an exciting program papers and invited talks that will help us better understand the future of two-sided marketplaces
Identifying Informational vs. Conversational Questions on Community Question Answering Archives
Authors: Ido Guy, Victor Makarenkov, Niva Hazon, Lior Rokach, Bracha Shapira
Venue: WSDM
Year: 2018

Questions on community question answering websites usually reflect one of two intents: learning information or starting a conversation. In this paper, we revisit this fundamental classification task of informational versus conversational questions, which was originally introduced and studied in 2009. We use a substantially larger dataset of archived questions from Yahoo Answers, which includes the question»s title, description, answers, and votes. We replicate the original experiments over this dataset, point out the common and different from the original results, and present a broad set of characteristics that distinguish the two question types. We also develop new classifiers that make use of additional data types, advanced machine learning, and a large dataset of unlabeled data, which achieve enhanced performance.
Care to Share?: Learning to Rank Personal Photos for Public Sharing
Authors: Ido Guy, Alexander Nus, Dan Pelleg, Idan Szpektor
Venue: WSDM
Year: 2018

With mobile devices, users are taking ever-growing numbers of photos every day. These photos are uploaded to social sites such as Facebook and Flickr, often automatically. Yet, the portion of these uploaded photos being publicly shared is low, and on a constant decline. Deciding which photo to share takes considerable time and attention, and many users would rather forfeit the social interaction and engagement than sift through their piles of uploaded photos. In this paper, we introduce a novel task of recommending socially-engaging photos to their creators for public sharing. This will turn a tedious manual chore into a quick, software-assisted process. We provide extensive analysis over a large-scale dataset from the Flickr photo sharing website, which reveals some of the traits of photo sharing in such sites. Additionally, we present a ranking algorithm for the task that comprises three steps:(a) grouping of near-duplicate photos;(b) ranking the photos in each group by their "shareability"; and(c) ranking the groups by their likelihood to contain a shareable photo. A large-scale experiment allows us to evaluate our algorithm and show its benefits compared to competitive baselines and algorithmic alternatives.
Automatic Ranking of Information Retrieval Systems
Authors: Maram Hasanain
Venue: WSDM
Year: 2018

Typical information retrieval system evaluation requires expensive manually-collected relevance judgments of documents, which are used to rank retrieval systems. Due to the high cost associated with collecting relevance judgments and the ever-growing scale of data to be searched in practice, ranking of retrieval systems using manual judgments is becoming less feasible. Methods to automatically rank systems in absence of judgments have been proposed to tackle this challenge. However, current techniques are still far from reaching the ranking achieved using manual judgments. I propose to advance research on automatic system ranking using supervised and unsupervised techniques.
Robust Transfer Learning for Cross-domain Collaborative Filtering Using Multiple Rating Patterns Approximation
Authors: Ming He, Jiuling Zhang, Peng Yang, Kaisheng Yao
Venue: WSDM
Year: 2018

Collaborative filtering techniques are a common approach for building recommendations, and have been widely applied in real recommender systems. However, collaborative filtering usually suffers from limited performance due to the sparsity of user-item interaction. To address this issue, auxiliary information is usually used to improve the performance. Transfer learning provides the key idea of using knowledge from auxiliary domains. An assumption of transfer learning in collaborative filtering is that the source domain is a full rating matrix, which may not hold in many real-world applications. In this paper, we investigate how to leverage rating patterns from multiple incomplete source domains to improve the quality of recommender systems. First, by exploiting the transferred learning, we compress the knowledge from the source domain into a cluster-level rating matrix. The rating patterns in the low-level matrix can be transferred to the target domain. Specifically, we design a knowledge extraction method to enrich rating patterns by relaxing the full rating restriction on the source domain. Finally, we propose a robust multiple-rating-pattern transfer learning model for cross-domain collaborative filtering, which is called MINDTL, to accurately predict missing values in the target domain. Extensive experiments on real-world datasets demonstrate that our proposed approach is effective and outperforms several alternative methods.
Collaborative Filtering via Additive Ordinal Regression
Authors: Jun Hu, Ping Li
Venue: WSDM
Year: 2018

Accurately predicting user preferences/ratings over items are crucial for many Internet applications, e.g., recommender systems, online advertising. In current main-stream algorithms regarding the rating prediction problem, discrete rating scores are often viewed as either numerical values or(nominal) categorical labels. Practically, viewing user rating scores as numerical values or categorical labels cannot precisely reflect the exact degree of user preferences. It is expected that for each user, the quantitative distance/scale between any pair of adjacent rating scores could be different.
Exploring Expert Cognition for Attributed Network Embedding
Authors: Xiao Huang, Qingquan Song, Jundong Li, Xia Hu
Venue: WSDM
Year: 2018

Attributed network embedding has been widely used in modeling real-world systems. The obtained low-dimensional vector representations of nodes preserve their proximity in terms of both network topology and node attributes, upon which different analysis algorithms can be applied. Recent advances in explanation-based learning and human-in-the-loop models show that by involving experts, the performance of many learning tasks can be enhanced. It is because experts have a better cognition in the latent information such as domain knowledge, conventions, and hidden relations. It motivates us to employ experts to transform their meaningful cognition into concrete data to advance network embedding. However, learning and incorporating the expert cognition into the embedding remains a challenging task. Because expert cognition does not have a concrete form, and is difficult to be measured and laborious to obtain. Also, in a real-world network, there are various types of expert cognition such as the comprehension of word meaning and the discernment of similar nodes. It is nontrivial to identify the types that could lead to a significant improvement in the embedding. In this paper, we study a novel problem of exploring expert cognition for attributed network embedding and propose a principled framework NEEC. We formulate the process of learning expert cognition as a task of asking experts a number of concise and general queries. Guided by the exemplar theory and prototype theory in cognitive science, the queries are systematically selected and can be generalized to various real-world networks. The returned answers from the experts contain their valuable cognition. We model them as new edges and directly add into the attributed network, upon which different embedding methods can be applied towards a more informative embedding representation. Experiments on real-world datasets verify the effectiveness and efficiency of NEEC.
Recommendation in Heterogeneous Information Networks Based on Generalized Random Walk Model and Bayesian Personalized Ranking
Authors: Zhengshen Jiang, Hongzhi Liu, Bin Fu, Zhonghai Wu, Tao Zhang
Venue: WSDM
Year: 2018

Recommendation based on heterogeneous information network(HIN) is attracting more and more attention due to its ability to emulate collaborative filtering, content-based filtering, context-aware recommendation and combinations of any of these recommendation semantics. Random walk based methods are usually used to mine the paths, weigh the paths, and compute the closeness or relevance between two nodes in a HIN. A key for the success of these methods is how to properly set the weights of links in a HIN. In existing methods, the weights of links are mostly set heuristically. In this paper, we propose a Bayesian Personalized Ranking(BPR) based machine learning method, called HeteLearn, to learn the weights of links in a HIN. In order to model user preferences for personalized recommendation, we also propose a generalized random walk with restart model on HINs. We evaluate the proposed method in a personalized recommendation task and a tag recommendation task. Experimental results show that our method performs significantly better than both the traditional collaborative filtering and the state-of-the-art HIN-based recommendation methods.
Topic Chronicle Forest for Topic Discovery and Tracking
Authors: Noriaki Kawamae
Venue: WSDM
Year: 2018

To ease comprehension of given time-stamped corpora, we extend topic models to handle both the specificity and temporality of topics; this is a significant advance over previous models which fail to provide both views simultaneously. Our proposed model consists of the Topic Chronicle Forest(TCF) and Thematic Dirichlet Processes(TDP). TCF is a set of Topic Chronicle Trees, where each tree is a hierarchy of topics that becomes more specialized toward the leaves. Only one tree is defined in each time interval, a region, and is used for TDP to generate a document. The advantage of our approach lies in providing more compact topic organization, while preserving both the semantic of a given corpus and the thematic of each document. Experiments show that TCF is a useful extension for longitudinal topic discovery and tracking, and helps us to organize and digest data sets.
Neural Networks for Information Retrieval
Authors: Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Venue: WSDM
Year: 2018

Machine learning plays a role in many aspects of modern IR systems, and deep learning is applied in all of them. The fast pace of modern-day research has given rise to many approaches to many IR problems. The amount of information available can be overwhelming both for junior students and for experienced researchers looking for new research topics and directions. The aim of this full- day tutorial is to give a clear overview of current tried-and-trusted neural methods in IR and how they benefit IR.
Deep Neural Architecture for Multi-Modal Retrieval based on Joint Embedding Space for Text and Images
Authors: Saeid Balaneshin Kordan, Alexander Kotov
Venue: WSDM
Year: 2018

Recent advances in deep learning and distributed representations of images and text have resulted in the emergence of several neural architectures for cross-modal retrieval tasks, such as searching collections of images in response to textual queries and assigning textual descriptions to images. However, the multi-modal retrieval scenario, when a query can be either a text or an image and the goal is to retrieve both a textual fragment and an image, which should be considered as an atomic unit, has been significantly less studied. In this paper, we propose a gated neural architecture to project image and keyword queries as well as multi-modal retrieval units into the same low-dimensional embedding space and perform semantic matching in this space. The proposed architecture is trained to minimize structured hinge loss and can be applied to both cross- and multi-modal retrieval. Experimental results for six different cross- and multi-modal retrieval tasks obtained on publicly available datasets indicate superior retrieval accuracy of the proposed architecture in comparison to the state-of-art baselines.
MIS2: Misinformation and Misbehavior Mining on the Web
Authors: Srijan Kumar, Meng Jiang, Taeho Jung, Roger Jie Luo, Jure Leskovec
Venue: WSDM
Year: 2018

Misinformation and misbehavior mining on the web(MIS2) workshop is held in Los Angeles, California, USA on February 9, 2018, and co-located with the 11th ACM International Conference on Web Search and Data Mining(WSDM 2018). Web is a dynamic ecosystem that enables malicious users to create and spread deceptive information to a wide audience in a matter of minutes. These malicious actors work on a wide variety of platforms, such as social media, e-commerce, and more. The main object of MIS2 is to discuss new and upcoming research on modeling, discovery, detection, and mitigation methods of misbehavior and misinformation on the web. MIS2 is an interdisciplinary venue for leading researchers and practitioners from the areas of data mining, social network analysis, cybersecurity, communications, human-computer interaction, and natural language processing. The topics addressed in MIS2 are extremely timely and the research presented by refereed papers and invited keynote speakers will participants a full dose of emerging research.
Joint Non-negative Matrix Factorization for Learning Ideological Leaning on Twitter
Authors: Preethi Lahoti, Kiran Garimella, Aristides Gionis
Venue: WSDM
Year: 2018

People are shifting from traditional news sources to online news at an incredibly fast rate. However, the technology behind online news consumption promotes content that confirms the users» existing point of view. This phenomenon has led to polarization of opinions and intolerance towards opposing views. Thus, a key problem is to model information filter bubbles on social media and design methods to eliminate them. In this paper, we use a machine-learning approach to learn a liberal-conservative ideology space on Twitter, and show how we can use the learned latent space to tackle the filter bubble problem. We model the problem of learning the liberal-conservative ideology space of social media users and media sources as a constrained non-negative matrix-factorization problem. Our model incorporates the social-network structure and content-consumption information in a joint factorization problem with shared latent factors. We validate our model and solution on a real-world Twitter dataset consisting of controversial topics, and show that we are able to separate users by ideology with over 90% purity. When applied to media sources, our approach estimates ideology scores that are highly correlated(Pearson correlation 0.9) with ground-truth ideology scores. Finally, we demonstrate the utility of our model in real-world scenarios, by illustrating how the learned ideology latent space can be used to develop exploratory and interactive interfaces that can help users in diffusing their information filter bubble.
Tutorial on Metrics of User Engagement: Applications to News, Search and E-Commerce
Authors: Mounia Lalmas, Liangjie Hong
Venue: WSDM
Year: 2018

User engagement plays a central role in companies operating online services, such as search engines, news portals, e-commerce sites, and social networks. A main challenge is to leverage collected knowledge about the daily online behavior of millions of users to understand what engage them short-term and more importantly long-term. The most common way that engagement is measured is through various online metrics, acting as proxy measures of user engagement. This tutorial will review these metrics, their advantages and drawbacks, and their appropriateness to various types of online services. As case studies, we will focus on three types of services, news, search and e-commerce. We will also briefly discuss how to develop better machine learning models to optimize online metrics, and design experiments to test these models.
Supporting Large-scale Geographical Visualization in a Multi-granularity Way
Authors: Mingzhao Li, Zhifeng Bao, Farhana Murtaza Choudhury, Timos Sellis
Venue: WSDM
Year: 2018

Urban data (e.g., real estate data, crime data) often have multiple attributes which are highly geography-related. With the scale of data increases, directly visualizing millions of individual data points on top of a map would overwhelm users' perceptual and cognitive capacity and lead to high latency when users interact with the data. In this demo, we present ConvexCubes, a system that supports interactive visualization of large-scale multidimensional urban data in a multi-granularity way. Comparing to state-of-the-art visualization-driven data structures, it exploits real-world geographic semantics (e.g., country, state, city) rather than using grid-based aggregation. Instead of calculating everything on demand, ConvexCubes utilizes existing visualization results to efficiently support different kinds of user interactions, such as zooming & panning, filtering and granularity control. Our system can be accessed at http://115.146.89.158/ConvexCubes/.