OpenRaiser/Intern-Atlas huggingface dataset:
Quantifying controversy in social media
Authors: Kiran Garimella, Gianmarco De Francisci Morales, Aristides Gionis, Michael Mathioudakis
Venue: Conference WSDM 2016
Year: 2016

Which topics spark the most heated debates in social media? Identifying these topics is a first step towards creating systems which pierce echo chambers. In this paper, we perform a systematic methodological study of controversy detection using social media network structure and content. Unlike previous work, rather than identifying controversy in a single hand-picked topic and use domain-specific knowledge, we focus on comparing topics in any domain. Our approach to quantifying controversy is a graph-based three-stage pipeline, which involves (i) building a conversation graph about a topic, which represents alignment of opinion among users; (ii) partitioning the conversation graph to identify potential sides of the controversy; and (iii)measuring the amount of controversy from characteristics of the~graph. We perform an extensive comparison of controversy measures, as well as graph building approaches and data sources. We use both controversial and non-controversial topics on Twitter, as well as other external datasets. We find that our new random-walk-based measure outperforms existing ones in capturing the intuitive notion of controversy, and show that content features are vastly less helpful in this task.
Will this #hashtag be popular tomorrow?
Authors: Zongyang Ma, Aixin Sun, Gao Cong
Venue: Conference SIGIR 2012
Year: 2012

Hashtags are widely used in Twitter to define a shared context for events or topics. In this paper, we aim to predict hashtag popularity in near future (i.e., next day). Given a hashtag that has the potential to be popular in the next day, we construct a hashtag profile using the tweets containing the hashtag, and extract both content and context features for hashtag popularity prediction. We model this prediction problem as a classification problem and evaluate the effectiveness of the extracted features and classification models.
Adding semantics to microblog posts
Authors: Edgar Meij, Wouter Weerkamp, Maarten de Rijke
Venue: Conference WSDM 2012
Year: 2012

Microblogs have become an important source of information for the purpose of marketing, intelligence, and reputation management. Streams of microblogs are of great value because of their direct and real-time nature. Determining what an individual microblog post is about, however, can be non-trivial because of creative language usage, the highly contextualized and informal nature of microblog posts, and the limited length of this form of communication. We propose a solution to the problem of determining what a microblog post is about through semantic linking: we add semantics to posts by automatically identifying concepts that are semantically related to it and generating links to the corresponding Wikipedia articles. The identified concepts can subsequently be used for, e.g., social media mining, thereby reducing the need for manual inspection and selection. Using a purpose-built test collection of tweets, we show that recently proposed approaches for semantic linking do not perform well, mainly due to the idiosyncratic nature of microblog posts. We propose a novel method based on machine learning with a set of innovative features and show that it is able to achieve significant improvements over all other methods, especially in terms of precision.
Meaning as Collective Use: Predicting Semantic Hashtag Categories on Twitter
Authors: Lisa Posch, Claudia Wagner, Philipp Singer, Markus Strohmaier
Venue: Conference WWW 2013
Year: 2013

This paper sets out to explore whether data about the usage of hashtags on Twitter contains information about their semantics. Towards that end, we perform initial statistical hypothesis tests to quantify the association between usage patterns and semantics of hashtags. To assess the utility of pragmatic features - which describe how a hashtag is used over time - for semantic analysis of hashtags, we conduct various hashtag stream classification experiments and compare their utility with the utility of lexical features. Our results indicate that pragmatic features indeed contain valuable information for classifying hashtags into semantic categories. Although pragmatic features do not outperform lexical features in our experiments, we argue that pragmatic features are important and relevant for settings in which textual information might be sparse or absent (e.g., in social video streams).
Modelling Information Needs in Collaborative Search Conversations
Authors: Sosuke Shiga, Hideo Joho, Roi Blanco, Johanne R. Trippas, Mark Sanderson
Venue: Conference SIGIR 2017
Year: 2017

The increase of voice-based interaction has changed the way people seek information, making search more conversational. Development of effective conversational approaches to search requires better understanding of how people express information needs in dialogue. This paper describes the creation and examination of over 32K spoken utterances collected during 34 hours of collaborative search tasks. The contribution of this work is three-fold. First, we propose a model of conversational information needs (CINs) based on a synthesis of relevant theories in Information Seeking and Retrieval. Second, we show several behavioural patterns of CINs based on the proposed model. Third, we identify effective feature groups that may be useful for detecting CINs categories from conversations. This paper concludes with a discussion of how these findings can facilitate advance of conversational search applications.
Chat More: Deepening and Widening the Chatting Topic via A Deep Model
Authors: Wenjie Wang, Minlie Huang, Xin-Shun Xu, Fumin Shen, Liqiang Nie
Venue: Conference SIGIR 2018
Year: 2018

The past decade has witnessed the boom of human-machine interactions, particularly via dialog systems. In this paper, we study the task of response generation in open-domain multi-turn dialog systems. Many research efforts have been dedicated to building intelligent dialog systems, yet few shed light on deepening or widening the chatting topics in a conversational session, which would attract users to talk more. To this end, this paper presents a novel deep scheme consisting of three channels, namely global, wide, and deep ones. The global channel encodes the complete historical information within the given context, the wide one employs an attention-based recurrent neural network model to predict the keywords that may not appear in the historical context, and the deep one trains a Multi-layer Perceptron model to select some keywords for an in-depth discussion. Thereafter, our scheme integrates the outputs of these three channels to generate desired responses. To justify our model, we conducted extensive experiments to compare our model with several state-of-the-art baselines on two datasets: one is constructed by ourselves and the other is a public benchmark dataset. Experimental results demonstrate that our model yields promising performance by widening or deepening the topics of interest.
Approximating the spectrum of a graph
Authors: David Cohen-Steiner, Weihao Kong, Christian Sohler, Gregory Valiant
Venue: Conference KDD 2018
Year: 2018

The spectrum of a network or graph $G=(V,E)$ with adjacency matrix A , consists of the eigenvalues of the normalized Laplacian $L= I - D^-1/2 A D^-1/2 $. This set of eigenvalues encapsulates many aspects of the structure of the graph, including the extent to which the graph posses community structures at multiple scales. We study the problem of approximating the spectrum, $łambda = (łambda_1,\dots,łambda_|V| )$, of G in the regime where the graph is too large to explicitly calculate the spectrum. We present a sublinear time algorithm that, given the ability to query a random node in the graph and select a random neighbor of a given node, computes a succinct representation of an approximation $\widetilde łambda = (\widetilde łambda_1,\dots,\widetilde łambda_|V| )$, such that $\|\widetilde łambda - łambda\|_1 łe ε |V|$. Our algorithm has query complexity and running time $exp(O(1/\eps))$, which is independent of the size of the graph, $|V|$. We demonstrate the practical viability of our algorithm on synthetically generated graphs, and on 15 different real-world graphs from the Stanford Large Network Dataset Collection, including social networks, academic collaboration graphs, and road networks. For the smallest of these graphs, we are able to validate the accuracy of our algorithm by explicitly calculating the true spectrum; for the larger graphs, such a calculation is computationally prohibitive. The spectra of these real-world networks reveal insights into the structural similarities and differences between them, illustrating the potential value of our algorithm for efficiently approximating the spectrum of large large networks.
Characterizing email search using large-scale behavioral logs and surveys
Authors: Qingyao Ai, Susan T. Dumais, Nick Craswell, Daniel J. Liebling
Venue: Conference WWW 2017
Year: 2017

As the number of email users and messages continues to grow, search is becoming more important for finding information in personal archives. In spite of its importance, email search is much less studied than web search, particularly using large-scale behavioral log analysis. In this paper we report the results of a large-scale log analysis of email search and complement this with a survey to better understand email search intent and success. We characterize email search behaviors and highlight differences from web search. When searching for email, people know many attributes about what they are looking for; they often look for specific known items; their queries are shorter and they click on fewer items than in web search. Although repeat queries are common in both email and web search, repeat visits to the same search result are much less common in email search suggesting that the same query is used for different search intents over time. We consider search intent from multiple angles. In email search logs, we find that people use email search not just to find information but also to perform tasks such as cleanup or organization, and that the distribution of actions they perform depends on the type of query. In our survey, people reported that they looked for specific information in both email search and web search, but they were much less likely to search for general information on a topic in email. The differences in overall behavior, re-finding patterns and search intents we observed between email and web search have important implications for the design of email search algorithms and interfaces.
Do social explanations work?: studying and modeling the effects of social explanations in recommender systems
Authors: Amit Sharma, Dan Cosley
Venue: Conference WWW 2013
Year: 2013

Recommender systems associated with social networks often use social explanations (e.g. "X, Y and 2 friends like this") to support the recommendations. We present a study of the effects of these social explanations in a music recommendation context. We start with an experiment with 237 users, in which we show explanations with varying levels of social information and analyze their effect on users' decisions. We distinguish between two key decisions: the likelihood of checking out the recommended artist, and the actual rating of the artist based on listening to several songs. We find that while the explanations do have some influence on the likelihood, there is little correlation between the likelihood and actual (listening) rating for the same artist. Based on these insights, we present a generative probabilistic model that explains the interplay between explanations and background information on music preferences, and how that leads to a final likelihood rating for an artist. Acknowledging the impact of explanations, we discuss a general recommendation framework that models external informational elements in the recommendation interface, in addition to inherent preferences of users.
Why We Read Wikipedia
Authors: Philipp Singer, Florian Lemmerich, Robert West, Leila Zia, Ellery Wulczyn, Markus Strohmaier, Jure Leskovec
Venue: Conference WWW 2017
Year: 2017

Wikipedia is one of the most popular sites on the Web, with millions of users relying on it to satisfy a broad range of information needs every day. Although it is crucial to understand what exactly these needs are in order to be able to meet them, little is currently known about why users visit Wikipedia. The goal of this paper is to fill this gap by combining a survey of Wikipedia readers with a log-based analysis of user activity. Based on an initial series of user surveys, we build a taxonomy of Wikipedia use cases along several dimensions, capturing users' motivations to visit Wikipedia, the depth of knowledge they are seeking, and their knowledge of the topic of interest prior to visiting Wikipedia. Then, we quantify the prevalence of these use cases via a large-scale user survey conducted on live Wikipedia with almost 30,000 responses. Our analyses highlight the variety of factors driving users to Wikipedia, such as current events, media coverage of a topic, personal curiosity, work or school assignments, or boredom. Finally, we match survey responses to the respondents' digital traces in Wikipedia's server logs, enabling the discovery of behavioral patterns associated with specific use cases. For instance, we observe long and fast-paced page sequences across topics for users who are bored or exploring randomly, whereas those using Wikipedia for work or school spend more time on individual articles focused on topics such as science. Our findings advance our understanding of reader motivations and behavior on Wikipedia and can have implications for developers aiming to improve Wikipedia's user experience, editors striving to cater to their readers' needs, third-party services (such as search engines) providing access to Wikipedia content, and researchers aiming to build tools such as recommendation engines.
A Music Recommendation System Based on Music Data Grouping and User Interests
Authors: Hung-Chen Chen, Arbee L. P. Chen
Venue: Conference CIKM 2001
Year: 2001

With the growth of the World Wide Web, a large amount of music data is available on the Internet. In addition to searching expected music objects for users, it becomes necessary to develop a recommendation service. In this paper, we design the Music Recommendation System (MRS) to provide a personalized service of music recommendation. The music objects of MIDI format are first analyzed. For each polyphonic music object, the representative track is first determined, and then six features are extracted from this track. According to the features, the music objects are properly grouped. For users, the access histories are analyzed to derive user interests. The content-based, collaborative and statistics-based recommendation methods are proposed, which are based on the favorite degrees of the users to the music groups. A series of experiments are carried out to show that our approach is feasible.
Sub-corpora Impact on System Effectiveness
Authors: Nicola Ferro, Mark Sanderson
Venue: Conference SIGIR 2017
Year: 2017

Understanding the factors comprising IR system effectiveness is of primary importance to compare different IR systems. Effectiveness is traditionally broken down, using ANOVA, into a topic and a system effect but this leaves out a key component of our evaluation paradigm: the collections of documents. We break down effectiveness into topic, system and sub-corpus effects and compare it to the traditional break down, considering what happens when different evaluation measures come into play. We found that sub-corpora are a significant effect. The consideration of which allows us to be more accurate in estimating what systems are significantly different. We also found that the sub-corpora affect different evaluation measures in different ways and this may impact on what systems are considered significantly different.
A General Linear Mixed Models Approach to Study System Component Effects
Authors: Nicola Ferro, Gianmaria Silvello
Venue: Conference SIGIR 2016
Year: 2016

Topic variance has a greater effect on performances than system variance but it cannot be controlled by system developers who can only try to cope with it. On the other hand, system variance is important on its own, since it is what system developers may affect directly by changing system components and it determines the differences among systems. In this paper, we face the problem of studying system variance in order to better understand how much system components contribute to overall performances. To this end, we propose a methodology based on General Linear Mixed Model (GLMM) to develop statistical models able to isolate system variance, component effects as well as their interaction by relying on a Grid of Points (GoP) containing all the combinations of analysed components. We apply the proposed methodology to the analysis of TREC Ad-hoc data in order to show how it works and discuss some interesting outcomes of this new kind of analysis. Finally, we extend the analysis to different evaluation measures, showing how they impact on the sources of variance.
On Per-topic Variance in IR Evaluation
Authors: Stephen E. Robertson, Evangelos Kanoulas
Venue: Conference SIGIR 2012
Year: 2012

We explore the notion, put forward by Cormack & Lynam and Robertson, that we should consider a document collection used for Cranfield-style experiments as a sample from some larger population of documents. In this view, any per-topic metric (such as average precision) should be regarded as an estimate of that metric's true value for that topic in the full population, and therefore as carrying its own per-topic variance or estimate precision or noise. As in the two mentioned papers, we explore this notion by simulating other samples from the same large population. We investigate different ways of performing this simulation. One use of this analysis is to refine the notion of statistical significance of a difference between two systems (in most such analyses, each per-topic measurement is treated as equally precise). We propose a mixed-effects model method to measure significance, and compare it experimentally with the traditional t-test.
On Topic Difficulty in IR Evaluation: The Effect of Systems, Corpora, and System Components
Authors: Fabio Zampieri, Kevin Roitero, J. Shane Culpepper, Oren Kurland, Stefano Mizzaro
Venue: Conference SIGIR 2019
Year: 2019

In a test collection setting, topic difficulty can be defined as the average effectiveness of a set of systems for a topic. In this paper we study the effects on the topic difficulty of: (i) the set of retrieval systems; (ii) the underlying document corpus; and (iii) the system components. By generalizing methods recently proposed to study system component factor analysis, we perform a comprehensive analysis on topic difficulty and the relative effects of systems, corpora, and component interactions. Our findings show that corpora have the most significant effect on topic difficulty.
Efficient Bayesian Hierarchical User Modeling for Recommendation System
Authors: Yi Zhang, Jonathan Koren
Venue: Conference SIGIR 2007
Year: 2007

A content-based personalized recommendation system learns user specific profiles from user feedback so that it can deliver information tailored to each individual user's interest. A system serving millions of users can learn a better user profile for a new user, or a user with little feedback, by borrowing information from other users through the use of a Bayesian hierarchical model. Learning the model parameters to optimize the joint data likelihood from millions of users is very computationally expensive. The commonly used EM algorithm converges very slowly due to the sparseness of the data in IR applications. This paper proposes a new fast learning technique to learn a large number of individual user profiles. The efficacy and efficiency of the proposed algorithm are justified by theory and demonstrated on actual user data from Netflix and MovieLens.
Revisiting mobile advertising threats with madlife
Authors: Gong Chen, Wei Meng, John A. Copeland
Venue: Conference WWW 2019
Year: 2019

Online advertising is one of the primary funding sources for various of content, services, and applications on both web and mobile platforms. Mobile in-app advertising reuses many existing web technologies under the same ad-serving model (i.e., users - publishers - ad networks - advertisers). Nevertheless, mobile in-app advertising is different from the traditional web advertising in many aspects. For example, malicious app developers can generate fraudulent ad clicks in an automated fashion, but malicious web publishers have to launch click fraud with bots. In spite of using the same underlying web infrastructure, advertising threats behave differently on the two platforms. Existing works have studied separately click fraud and malvertising in the mobile setting. However, it is unknown if there exists a relationship between these two dominant threats. In this paper, we present an ad collection framework – MAdLife – on Android to capture all the in-app ad traffic generated during an ad's entire lifespan. MAdLife allows us to revisit both threats in a fine-grained manner and study the relationship between them. It further enables the exploration of other threats related to ad landing pages. We analyzed 5.7K Android apps crawled from the Google Play Store, and collected 83K ads and their landing pages using MAdLife. Similar to traditional web ads, 58K ads landed on web pages. We discovered 37 click-fraud apps, and found that 1.49% of the 58K ads were malicious. We also revealed a strong correlation between fraudulent apps and malicious ads. Specifically, 15.44% of malicious ads originated from the fraudulent apps. Conversely, 18.36% of the ads served in the fraudulent apps were malicious, while only 1.28% were malicious in the rest apps. This suggests that users of fraudulent apps are much more (14x) likely to encounter malicious ads. Additionally, we discovered that 243 popular JavaScript snippets embedded by over 10% of the landing pages were malicious. Finally, we conducted the first analysis on inappropriate mobile in-app ads.
Detection and analysis of drive-by-download attacks and malicious JavaScript code
Authors: Marco Cova, Christopher Krügel, Giovanni Vigna
Venue: Conference WWW 2010
Year: 2010

JavaScript is a browser scripting language that allows developers to create sophisticated client-side interfaces for web applications. However, JavaScript code is also used to carry out attacks against the user's browser and its extensions. These attacks usually result in the download of additional malware that takes complete control of the victim's platform, and are, therefore, called "drive-by downloads." Unfortunately, the dynamic nature of the JavaScript language and its tight integration with the browser make it difficult to detect and block malicious JavaScript code.
Field-aware factorization machines in a real-world online advertising system
Authors: Yuchin Juan, Damien Lefortier, Olivier Chapelle
Venue: Conference WWW 2017
Year: 2017

Predicting user response is one of the core machine learning tasks in computational advertising. Field-aware Factorization Machines (FFM) have recently been established as a state-of-the-art method for that problem and in particular won two Kaggle challenges. This paper presents some results from implementing this method in a production system that predicts click-through and conversion rates for display advertising and shows that this method it is not only effective to win challenges but is also valuable in a real-world prediction system. We also discuss some specific challenges and solutions to reduce the training time, namely the use of an innovative seeding algorithm and a distributed learning mechanism.