OpenRaiser/Intern-Atlas huggingface dataset:
Objective Bayesian Two Sample Hypothesis Testing for Online Controlled Experiments
Authors: Alex Deng
Venue: WWW
Year: 2015

As A/B testing gains wider adoption in the industry, more people begin to realize the limitations of the traditional frequentist null hypothesis statistical testing (NHST). The large number of search results for the query ``Bayesian A/B testing'' shows just how much the interest in the Bayesian perspective is growing. In recent years there are also voices arguing that Bayesian A/B testing should replace frequentist NHST and is strictly superior in all aspects. Our goal here is to clarify the myth by looking at both advantages and issues of Bayesian methods. In particular, we propose an objective Bayesian A/B testing framework for which we hope to bring the best from Bayesian and frequentist methods together. Unlike traditional methods, this method requires the existence of historical A/B test data to objectively learn a prior. We have successfully applied this method to Bing, using thousands of experiments to establish the priors.
Linked Ethnographic Data: From Theory to Practice
Authors: Dominic DiFranzo, Marie Joan Kristine Gloria, James A. Hendler
Venue: WWW
Year: 2015

As Web Science continues to mix methods from the many disciplines that study the web, we must begin to seriously look at mixing and linking data across the Qualitative and Quantitative divide. A large difficulty in this is in modeling and archiving Qualitative data. In this paper, we outline what these difficulties are in detail with a focus on the data practices of Ethnography. We describe how linked data technologies can address these issues. We demonstrate this with a case study in modeling data from audio interviews that were taken in an ethnographic study conducted in our lab. We conclude with a discussion on future work that needs to be done to better equip researchers with these tools and methods.
The Dynamics of Micro-Task Crowdsourcing: The Case of Amazon MTurk
Authors: Djellel Eddine Difallah, Michele Catasta, Gianluca Demartini, Panagiotis G. Ipeirotis, Philippe Cudré-Mauroux
Venue: WWW
Year: 2015

Micro-task crowdsourcing is rapidly gaining popularity among research communities and businesses as a means to leverage Human Computation in their daily operations. Unlike any other service, a crowdsourcing platform is in fact a marketplace subject to human factors that affect its performance, both in terms of speed and quality. Indeed, such factors shape the dynamics of the crowdsourcing market. For example, a known behavior of such markets is that increasing the reward of a set of tasks would lead to faster results. However, it is still unclear how different dimensions interact with each other: reward, task type, market competition, requester reputation, etc. In this paper, we adopt a data-driven approach to (A) perform a long-term analysis of a popular micro-task crowdsourcing platform and understand the evolution of its main actors (workers, requesters, and platform). (B) We leverage the main findings of our five year log analysis to propose features used in a predictive model aiming at determining the expected performance of any batch at a specific point in time. We show that the number of tasks left in a batch and how recent the batch is are two key features of the prediction. (C) Finally, we conduct an analysis of the demand (new tasks posted by the requesters) and supply (number of tasks completed by the workforce) and show how they affect task prices on the marketplace.
The Dynamics of Micro-Task Crowdsourcing: The Case of Amazon MTurk
Authors: Djellel Eddine Difallah, Michele Catasta, Gianluca Demartini, Panagiotis G. Ipeirotis, Philippe Cudré-Mauroux
Venue: WWW
Year: 2015

Micro-task crowdsourcing is rapidly gaining popularity among research communities and businesses as a means to leverage Human Computation in their daily operations. Unlike any other service, a crowdsourcing platform is in fact a marketplace subject to human factors that affect its performance, both in terms of speed and quality. Indeed, such factors shape the \emph{dynamics} of the crowdsourcing market. For example, a known behavior of such markets is that increasing the reward of a set of tasks would lead to faster results. However, it is still unclear how different dimensions interact with each other: reward, task type, market competition, requester reputation, etc.
Hate Speech Detection with Comment Embeddings
Authors: Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, Narayan Bhamidipati
Venue: WWW
Year: 2015

We address the problem of hate speech detection in online user comments. Hate speech, defined as an "abusive speech targeting specific group characteristics, such as ethnicity, religion, or gender", is an important problem plaguing websites that allow users to leave feedback, having a negative impact on their online business and overall user experience. We propose to learn distributed low-dimensional representations of comments using recently proposed neural language models, that can then be fed as inputs to a classification algorithm. Our approach addresses issues of high-dimensionality and sparsity that impact the current state-of-the-art, resulting in highly efficient and effective hate speech detectors.
Future User Engagement Prediction and Its Application to Improve the Sensitivity of Online Experiments
Authors: Alexey Drutsa, Gleb Gusev, Pavel Serdyukov
Venue: WWW
Year: 2015

Modern Internet companies improve their services by means of data-driven decisions that are based on online controlled experiments (also known as A/B tests). To run more online controlled experiments and to get statistically significant results faster are the emerging needs for these companies. The main way to achieve these goals is to improve the sensitivity of A/B experiments. We propose a novel approach to improve the sensitivity of user engagement metrics (that are widely used in A/B tests) by utilizing prediction of the future behavior of an individual user. This problem of prediction of the exact value of a user engagement metric is also novel and is studied in our work. We demonstrate the effectiveness of our sensitivity improvement approach on several real online experiments run at Yandex. Especially, we show how it can be used to detect the treatment effect of an A/B test faster with the same level of statistical significance.
Using Context to Get Novel Recommendation in Internet Message Streams
Authors: Doina Alexandra Dumitrescu, Simone Santini
Venue: WWW
Year: 2015

Novelty detection algorithms usually employ similarity measures with the previous seen and relevant documents to decide if a document is of user's interest. The problem that arises by using this approach is that the system might recommend redundant documents. Thus, it has become extremely important to be able to distinguish between "redundant" and "novel" information. To address this limitation, we apply a contextual and semantic approach by building the user profile using self-organizing maps that have the advantage to easily follow the changes in the users interests.
Social Networking by Proxy: Analysis of Dogster, Catster and Hamsterster
Authors: Daniel Dünker, Jérôme Kunegis
Venue: WWW
Year: 2015

Online pet social networks provide a unique opportunity to study an online social network in which a single user manages multiple user profiles, i.e. one for each pet they own. These types of multi-profile networks allow us to investigate two questions: (1) What is the relationship between the pet-level and human-level network, and (2) what is the relationship between friendship links and family ties? Concretely, we study the online pet social networks Catster, Dogster and Hamsterster, and show how the networks on the two levels interact, and perform experiments to find out whether knowledge about friendships on a profile-level alone can be used to predict which users are behind which profile.
Scalable Preference Learning from Data Streams
Authors: Fabon Dzogang, Thomas Lansdall-Welfare, Saatviga Sudhahar, Nello Cristianini
Venue: WWW
Year: 2015

We study the task of learning the preferences of online readers of news, based on their past choices. Previous work has shown that it is possible to model this situation as a competition between articles, where the most appealing articles of the day are those selected by the most users. The appeal of an article can be computed from its textual content, and the evaluation function can be learned from training data. In this paper, we show how this task can benefit from an efficient algorithm, based on hashing representations, which enables it to be deployed on high intensity data streams. We demonstrate the effectiveness of this approach on four real world news streams, compare it with standard approaches, and describe a new online demonstration based on this technology.
Collaborative Exchange of Systematic Literature Review Results: The Case of Empirical Software Engineering
Authors: Fajar J. Ekaputra, Marta Sabou, Estefanía Serral, Stefan Biffl
Venue: WWW
Year: 2015

Complementary to managing bibliographic information as done by digital libraries, the management of concrete research objects (e.g., experimental workflows, design patterns) is a pre-requisite to foster collaboration and reuse of research results. In this paper we describe the case of the Empirical Software Engineering domain, where researchers use systematic literature reviews (SLRs) to conduct and report on literature studies. Given their structured nature, the outputs of such SLR processes are a special and complex type of research object. Since performing SLRs is a time consuming process, it is highly desirable to enable sharing and reuse of the complex knowledge structures produced through SLRs. This would enable, for example, conducting new studies that build on the findings of previous studies. To support collaborative features necessary for multiple research groups to share and reuse each other's work, we hereby propose a solution approach that is inspired by software engineering best-practices and is implemented using Semantic Web technologies.
A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems
Authors: Ali Mamdouh Elkahky, Yang Song, Xiaodong He
Venue: WWW
Year: 2015

Recent online services rely heavily on automatic personalization to recommend relevant content to a large number of users. This requires systems to scale promptly to accommodate the stream of new users visiting the online services for the first time. In this work, we propose a content-based recommendation system to address both the recommendation quality and the system scalability. We propose to use a rich feature set to represent users, according to their web browsing history and search queries. We use a Deep Learning approach to map users and items to a latent space where the similarity between users and their preferred items is maximized. We extend the model to jointly learn from features of items from different domains and user features by introducing a multi-view Deep Learning model. We show how to make this rich-feature based user representation scalable by reducing the dimension of the inputs and the amount of training data. The rich user feature representation allows the model to learn relevant user behavior patterns and give useful recommendations for users who do not have any interaction with the service, given that they have adequate search and browsing history. The combination of different domains into a single model for learning helps improve the recommendation quality across all the domains, as well as having a more compact and a semantically richer user latent feature vector. We experiment with our approach on three real-world recommendation systems acquired from different sources of Microsoft products: Windows Apps recommendation, News recommendation, and Movie/TV recommendation. Results indicate that our approach is significantly better than the state-of-the-art algorithms (up to 49% enhancement on existing users and 115% enhancement on new users). In addition, experiments on a publicly open data set also indicate the superiority of our method in comparison with transitional generative topic models, for modeling cross-domain recommender systems. Scalability analysis show that our multi-view DNN model can easily scale to encompass millions of users and billions of item entries. Experimental results also confirm that combining features from all domains produces much better performance than building separate models for each domain.
What's Hot in The Theme: Query Dependent Emerging Topic Extraction from Social Streams
Authors: Yuki Endo, Hiroyuki Toda, Yoshimasa Koike
Venue: WWW
Year: 2015

Analyzing emerging topics from social media enables users to overview social movement and several web services to adopt current trends. Although existing studies mainly focus on extracting global emerging topics, efficient extraction of local ones related to a specific theme is still a challenging and unavoidable problem in social media analysis. We focus on extracting emerging social topics related to user-specified query words, and propose an extraction framework that uses a non-negative matrix factorization (NMF) modified for detecting temporal concentration and reducing noises. We conduct preliminary experiments for verifying our method using a Twitter dataset.
A Practical Framework for Privacy-Preserving Data Analytics
Authors: Liyue Fan, Hongxia Jin
Venue: WWW
Year: 2015

The availability of an increasing amount of user generated data is transformative to our society. We enjoy the benefits of analyzing big data for public interest, such as disease outbreak detection and traffic control, as well as for commercial interests, such as smart grid and product recommendation. However, the large collection of user generated data contains unique patterns and can be used to re-identify individuals, which has been exemplified by the AOL search log release incident. In this paper, we propose a practical framework for data analytics, while providing differential privacy guarantees to individual data contributors. Our framework generates differentially private aggregates which can be used to perform data mining and recommendation tasks. To alleviate the high perturbation errors introduced by the differential privacy mechanism, we present two methods with different sampling techniques to draw a subset of individual data for analysis. Empirical studies with real-world data sets show that our solutions enable accurate data analytics on a small fraction of the input data, reducing user privacy risk and data storage requirement without compromising the analysis results.
Co-evolutionary Dynamics of Information Diffusion and Network Structure
Authors: Mehrdad Farajtabar, Manuel Gomez-Rodriguez, Yichen Wang, Shuang Li, Hongyuan Zha, Le Song
Venue: WWW
Year: 2015

Information diffusion in online social networks is obviously affected by the underlying network topology, but it also has the power to change that topology. Online users are constantly creating new links when exposed to new information sources, and in turn these links are alternating the route of information spread. However, these two highly intertwined stochastic processes, information diffusion and network evolution, have been predominantly studied separately, ignoring their co-evolutionary dynamics. In this project, we propose a probabilistic generative model, COEVOLVE, for the joint dynamics of these two processes, allowing the intensity of one process to be modulated by that of the other. This model allows us to efficiently simulate diffusion and network events from the co-evolutionary dynamics, and generate traces obeying common diffusion and network patterns observed in real-world networks. Furthermore, we also develop a convex optimization framework to learn the parameters of the model from historical diffusion and network evolution traces. We experimented with both synthetic data and data gathered from Twitter, and show that our model provides a good fit to the data as well as more accurate predictions than alternatives.
Supply and Demand: Propagation and Absorption of News
Authors: Anastassia Fedyk
Venue: WWW
Year: 2015

The importance of the media for individual and market behavior cannot be overstated. For example, a front-page article in the New York Times that mostly reprints information from six months prior can cause a company's stock price to jump by over 300%. To better understand the channels through which the media affects markets and the resulting implications for news production, we study how individuals process information in news. Do readers display a preference for news with a positive slant? Are consumers of news segregated based on the media outlets they favor? Do individuals recognize which news is novel, and which simply reprints old information? While these questions are grounded in fundamental human psychology, they are also inextricably linked to the rapidly changing technology of news production. With over a million stories a day passing through the Bloomberg terminal alone, the volume of data -- both on the content of news and the behavior of readers -- has skyrocketed. As a result, analysis of media production and consumption requires ever more sophisticated techniques for identifying the informational value of news and the behavioral patterns of its modern readers.
BrowserCloud: A Personal Cloud for Browser Session Migration and Management
Authors: Junjie Feng, Aaron Harwood
Venue: WWW
Year: 2015

Web browsers are de facto clients for an ever-increasing range of web applications. At the same time, web users are accessing these applications from a wide range of devices. This paper presents a solution for runtime browser session migration and management, called BrowserCloud, which allows a user to securely manage multiple browsers from a personal or third-party Cloud service, migrate snapshots of active browser sessions between browsers over different devices, using a robust security module. The design of BrowserCloud is based on browser extensions/plugins that can preserve and restore browser session state, and a PHP server that stores browser sessions securely. We have tested our implementation over a range of increasingly complex web applications, including WebRTC and HTML5 video. To the best of our knowledge, our implementation is the most robust and secure approach to runtime browser session management to date.
Fast Search for Distance Dependent Chinese Restaurant Processes
Authors: Weiwei Feng, Peng Wang, Chuan Zhou, Li Guo, Peng Zhang
Venue: WWW
Year: 2015

The distance dependent Chinese Restaurant Processes(dd-CRP), a nonparametric Bayesian model, can model distance sensitive data. Existing inference algorithms for dd-CRP, such as Markov Chain Monte Carlo (MCMC) and variational algorithms, are inefficient and unable to handle massive online data, because posterior distributions of dd-CRP are not marginal invariant. To solve this problem, we present a fast inference algorithm for dd-CRP based on the A-star search. Experimental results show that the new search algorithm is faster than existing dd-CRP inference algorithms with comparable results.
Question Classification by Approximating Semantics
Authors: Guangyu Feng, Kun Xiong, Yang Tang, Anqi Cui, Jing Bai, Hang Li, Qiang Yang, Ming Li
Venue: WWW
Year: 2015

A central task of computational linguistics is to decide if two pieces of texts have similar meanings. Ideally, this depends on an intuitive notion of semantic distance. While this semantic distance is most likely undefinable and uncomputable, in practice it is approximated heuristically, consciously or unconsciously. In this paper, we present a theory, and its implementation, to approximate the elusive semantic distance by the well-defined information distance. It is mathematically proven that any computable approximation of the intuitive concept of semantic distance is "covered" by our theory. We have implemented our theory to question answering (QA) and performed experiments based on data extracted from over 35 million question-answer pairs. Experiments demonstrate that our initial implementation of the theory produces convincingly fewer errors in classification compared to other academic models and commercial systems.
MICO: Towards Contextual Media Analysis
Authors: Sergio Fernández, Sebastian Schaffert, Thomas Kurz
Venue: WWW
Year: 2015

With the tremendous increase in multimedia content on the Web and in corporate intranets, discovering hidden meaning in raw multimedia is becoming one of the biggest challenges. Analysing multimedia content is still in its infancy, requires expert knowledge, and the few available products are associated with excessive price tags, while still not delivering sufficient quality for many tasks. This makes it hard, especially for small and medium-size enterprises, to make use of this technology. In addition analysis components typically operate in isolation and do not consider the context (e.g. embedding text) of a media resource. This paper presents how MICO tries to address these problems by providing an open source service platform, that allows to analyse media in context and includes various analysis engines for video, images, audio, text, link structure and metadata.
Compressed Indexes for String Searching in Labeled Graphs
Authors: Paolo Ferragina, Francesco Piccinno, Rossano Venturini
Venue: WWW
Year: 2015

Storing and searching large labeled graphs is indeed becoming a key issue in the design of space/time efficient online platforms indexing modern social networks or knowledge graphs. But, as far as we know, all these results are limited to design compressed graph indexes which support basic access operations onto the link structure of the input graph, such as: given a node u, return the adjacency list of u. This paper takes inspiration from the Facebook Unicorn's platform and proposes some compressed-indexing schemes for large graphs whose nodes are labeled with strings of variable length - i.e., node's attributes such as user's (nick-)name - that support sophisticated search operations which involve both the linked structure of the graph and the string content of its nodes.
Model-driven Testing of RESTful APIs
Authors: Tobias Fertig, Peter Braun
Venue: WWW
Year: 2015

In contrast to the increasing popularity of REpresentational State Transfer (REST), systematic testing of RESTful Application Programming Interfaces (API) has not attracted much attention so far. This paper describes different aspects of automated testing of RESTful APIs. Later, we focus on functional and security tests, for which we apply a technique called model-based software development. Based on an abstract model of the RESTful API that comprises resources, states and transitions a software generator not only creates the source code of the RESTful API but also creates a large number of test cases that can be immediately used to test the implementation. This paper describes the process of developing a software generator for test cases using state-of-the-art tools and provides an example to show the feasibility of our approach.
whoVIS: Visualizing Editor Interactions and Dynamics in Collaborative Writing Over Time
Authors: Fabian Flöck, Maribel Acosta
Venue: WWW
Year: 2015

The visualization of editor interaction dynamics and provenance of content in revisioned, collaboratively written documents has the potential to allow for more transparency and intuitive understanding of the intricate mechanisms inherent to collective content production. Although approaches exist to build editor interactions from individual word changes in Wikipedia articles, they do not allow to inquire into individual interactions, and have yet to be implemented as usable end-user tools.
Interpreting News Recommendation Models
Authors: Blaz Fortuna, Pat Moore, Marko Grobelnik
Venue: WWW
Year: 2015

This paper presents an approach for recommending news articles on a large news portal. Focus is given to interpretability of the developed models, analysis of their performance, and deriving understanding of short and long-term user behavior on a news portal.