OpenRaiser/Intern-Atlas huggingface dataset:
Utilizing Online Social Media for Disaster Relief: Practical Challenges in Retrieval
Authors: Moumita Basu
Venue: SIGIR
Year: 2017

In recent years, several disaster events (e.g., earthquakes in Nepal-India and Italy, terror attacks in Paris and Brussels) have proven the crucial role of Online Social Media (OSM) in providing actionable situational information. However, in such media, the crucial information is typically obscured by a lot of insignificant information (e.g., personal opinions, prayers for victims). Moreover, when time is critical, owing to the rapid speed and huge volume of microblogs, it is infeasible for human subjects to go through all the tweets posted. Hence, automated IR methods are needed to extract the relevant information from the deluge of posts.
Accounting for the Correspondence in Commented Data
Authors: Renqin Cai, Chi Wang, Hongning Wang
Venue: SIGIR
Year: 2017

One important way for people to make their voice heard is to comment on the articles they have read online, such as news reports and each other's posts. The user-generated comments together with the commented documents form a unique correspondence structure. Properly modeling the dependency in such data is thus vital for one to obtain accurate insight of people's opinions and attention.
Stacking Bagged and Boosted Forests for Effective Automated Classification
Authors: Raphael R. Campos, Sérgio D. Canuto, Thiago Salles, Clebson C. A. de Sá, Marcos André Gonçalves
Venue: SIGIR
Year: 2017

Random Forest (RF) is one of the most successful strategies for automated classification tasks. Motivated by the RF success, recently proposed RF-based classification approaches leverage the central RF idea of aggregating a large number of low-correlated trees, which are inherently parallelizable and provide exceptional generalization capabilities. In this context, this work brings several new contributions to this line of research. First, we propose a new RF-based strategy (BERT) that applies the boosting technique in bags of extremely randomized trees. Second, we empirically demonstrate that this new strategy, as well as the recently proposed BROOF and LazyNN_RF classifiers do complement each other, motivating us to stack them to produce an even more effective classifier. Up to our knowledge, this is the first strategy to effectively combine the three main ensemble strategies: stacking, bagging (the cornerstone of RFs) and boosting. Finally, we exploit the efficient and unbiased stacking strategy based on out-of-bag (OOB) samples to considerably speedup the very costly training process of the stacking procedure. Our experiments in several datasets covering two high-dimensional and noisy domains of topic and sentiment classification provide strong evidence in favor of the benefits of our RF-based solutions. We show that BERT is among the top performers in the vast majority of analyzed cases, while retaining the unique benefits of RF classifiers (explainability, parallelization, easiness of parameterization). We also show that stacking only the recently proposed RF-based classifiers and BERT using our OOB-based strategy is not only significantly faster than recently proposed stacking strategies (up to six times) but also much more effective, with gains up to 21% and 17% on MacroF1 and MicroF1, respectively, over the best base method, and of 5% and 6% over a stacking of traditional methods, performing no worse than a complete stacking of methods at a much lower computational effort.
A Probabilistic Reformulation of Memory-Based Collaborative Filtering: Implications on Popularity Biases
Authors: Rocío Cañamares, Pablo Castells
Venue: SIGIR
Year: 2017

We develop a probabilistic formulation giving rise to a formal version of heuristic k nearest-neighbor (kNN) collaborative filtering. Different independence assumptions in our scheme lead to user-based, item-based, normalized and non-normalized variants that match in structure the traditional formulations, while showing equivalent empirical effectiveness. The probabilistic formulation provides a principled explanation why kNN is an effective recommendation strategy, and identifies a key condition for this to be the case. Moreover, a natural explanation arises for the bias of kNN towards recommending popular items. Thereupon the kNN variants are shown to fall into two groups with similar trends in behavior, corresponding to two different notions of item popularity. We show experiments where the comparative performance of the two groups of algorithms changes substantially, which suggests that the performance measurements and comparison may heavily depend on statistical properties of the input data sample.
What Are You Known For?: Learning User Topical Profiles with Implicit and Explicit Footprints
Authors: Cheng Cao, Hancheng Ge, Haokai Lu, Xia Hu, James Caverlee
Venue: SIGIR
Year: 2017

User interests and expertise are valuable but often hidden resources on social media. For example, Twitter Lists and LinkedIn's Skill Tags provide a partial perspective on what users are known for (by aggregating crowd tagging knowledge), but the vast majority of users are untagged; their interests and expertise are essentially hidden from important applications such as personalized recommendation, community detection, and expert mining. A natural approach to overcome these limitations is to intelligently learn user topical profiles by exploiting information from multiple, heterogeneous footprints: for instance, Twitter users who post similar hashtags may have similar interests, and YouTube users who upvote the same videos may have similar preferences. And yet identifying "similar" users by exploiting similarity in such a footprint space often provides conflicting evidence, leading to poor-quality user profiles. In this paper, we propose a unified model for learning user topical profiles that simultaneously considers multiple footprints. We show how these footprints can be embedded in a generalized optimization framework that takes into account pairwise relations among all footprints for robustly learning user profiles. Through extensive experiments, we find the proposed model is capable of learning high-quality user topical profiles, and leads to a 10-15% improvement in precision and mean average error versus a cross-triadic factorization state-of-the-art baseline.
But Is It Statistically Significant?: Statistical Significance in IR Research, 1995-2014
Authors: Ben Carterette
Venue: SIGIR
Year: 2017

We analyze 5,792 IR conference papers published over 20 years to investigate how researchers have used and are using statistical significance testing in their experiments
Statistical Significance Testing in Information Retrieval: Theory and Practice
Authors: Ben Carterette
Venue: SIGIR
Year: 2017

The past 20 years have seen a great improvement in the rigor of information retrieval experimentation, due primarily to two factors: high-quality, public, portable test collections such as those produced by TREC (the Text REtrieval Conference), and the increased practice of statistical hypothesis testing to determine whether measured improvements can be ascribed to something other than random chance. Together these create a very useful standard for reviewers, program committees, and journal editors; work in information retrieval (IR) increasingly cannot be published unless it has been evaluated using a well-constructed test collection and shown to produce a statistically significant improvement over a good baseline. But, as the saying goes, any tool sharp enough to be useful is also sharp enough to be dangerous. Statistical tests of significance are widely misunderstood. Most researchers and developers treat them as a "black box": evaluation results go in and a p-value comes out. But because significance is such an important factor in determining what research directions to explore and what is published, using p-values obtained without thought can have consequences for everyone doing research in IR. Ioannidis has argued that the main consequence in the biomedical sciences is that most published research findings are false; could that be the case in IR as well?
Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL 2017)
Authors: Muthu Kumar Chandrasekaran, Kokil Jaidka, Philipp Mayr
Venue: SIGIR
Year: 2017

The large scale of scholarly publications poses a challenge for scholars in information seeking and sensemaking. Bibliometrics, information retrieval (IR), text mining and NLP techniques could help in these search and look-up activities, but are not yet widely used. This workshop is intended to stimulate IR researchers and digital library professionals to elaborate on new approaches in natural language processing, information retrieval, scientometrics, text mining and recommendation techniques that can advance the state-of-the-art in scholarly document understanding, analysis, and retrieval at scale. The BIRNDL workshop at SIGIR 2017 will incorporate an invited talk, paper sessions and the third edition of the Computational Linguistics (CL) Scientific Summarization Shared Task.
Optimizing Trade-offs Among Stakeholders in Real-Time Bidding by Incorporating Multimedia Metrics
Authors: Xiang Chen, Bowei Chen, Mohan S. Kankanhalli
Venue: SIGIR
Year: 2017

Displaying banner advertisements (in short, ads) on webpages has usually been discussed as an Internet economics topic where a publisher uses auction models to sell an online user's page view to advertisers and the one with the highest bid can have her ad displayed to the user. This is also called real-time bidding (RTB) and the ad displaying process ensures that the publisher's benefit is maximized or there is an equilibrium in ad auctions. However, the benefits of the other two stakeholders - the advertiser and the user - have been rarely discussed. In this paper, we propose a two-stage computational framework that selects a banner ad based on the optimized trade-offs among all stakeholders. The first stage is still auction based and the second stage re-ranks ads by considering the benefits of all stakeholders. Our metric variables are: the publisher's revenue, the advertiser's utility, the ad memorability, the ad click-through rate (CTR), the contextual relevance, and the visual saliency. To the best of our knowledge, this is the first work that optimizes trade-offs among all stakeholders in RTB by incorporating multimedia metrics. An algorithm is also proposed to determine the optimal weights of the metric variables. We use both ad auction datasets and multimedia datasets to validate the proposed framework. Our experimental results show that the publisher can significantly improve the other stakeholders' benefits by slightly reducing her revenue in the short-term. In the long run, advertisers and users will be more engaged, the increased demand of advertising and the increased supply of page views can then boost the publisher's revenue.
On the Benefit of Incorporating External Features in a Neural Architecture for Answer Sentence Selection
Authors: Ruey-Cheng Chen, Evi Yulianti, Mark Sanderson, W. Bruce Croft
Venue: SIGIR
Year: 2017

Incorporating conventional, unsupervised features into a neural architecture has the potential to improve modeling effectiveness, but this aspect is often overlooked in the research of deep learning models for information retrieval. We investigate this incorporation in the context of answer sentence selection, and show that combining a set of query matching, readability, and query focus features into a simple convolutional neural network can lead to markedly increased effectiveness. Our results on two standard question-answering datasets show the effectiveness of the combined model.
Personalized Key Frame Recommendation
Authors: Xu Chen, Yongfeng Zhang, Qingyao Ai, Hongteng Xu, Junchi Yan, Zheng Qin
Venue: SIGIR
Year: 2017

Key frames are playing a very important role for many video applications, such as on-line movie preview and video information retrieval. Although a number of key frame selection methods have been proposed in the past, existing technologies mainly focus on how to precisely summarize the video content, but seldom take the user preferences into consideration. However, in real scenarios, people may cast diverse interests on the contents even for the same video, and thus they may be attracted by quite different key frames, which makes the selection of key frames an inherently personalized process. In this paper, we propose and investigate the problem of personalized key frame recommendation to bridge the above gap. To do so, we make use of video images and user time-synchronized comments to design a novel key frame recommender that can simultaneously model visual and textual features in a unified framework. By user personalization based on her/his previously reviewed frames and posted comments, we are able to encode different user interests in a unified multi-modal space, and can thus select key frames in a personalized manner, which, to the best of our knowledge, is the first time in the research field of video content analysis. Experimental results show that our method performs better than its competitors on various measures.
Meta-evaluation of Online and Offline Web Search Evaluation Metrics
Authors: Ye Chen, Ke Zhou, Yiqun Liu, Min Zhang, Shaoping Ma
Venue: SIGIR
Year: 2017

As in most information retrieval (IR) studies, evaluation plays an essential part in Web search research. Both offline and online evaluation metrics are adopted in measuring the performance of search engines. Offline metrics are usually based on relevance judgments of query-document pairs from assessors while online metrics exploit the user behavior data, such as clicks, collected from search engines to compare search algorithms. Although both types of IR evaluation metrics have achieved success, to what extent can they predict user satisfaction still remains under-investigated. To shed light on this research question, we meta-evaluate a series of existing online and offline metrics to study how well they infer actual search user satisfaction in different search scenarios. We find that both types of evaluation metrics significantly correlate with user satisfaction while they reflect satisfaction from different perspectives for different search tasks. Offline metrics better align with user satisfaction in homogeneous search (i.e. ten blue links) whereas online metrics outperform when vertical results are federated. Finally, we also propose to incorporate mouse hover information into existing online evaluation metrics, and empirically show that they better align with search user satisfaction than click-based online metrics.
Top-N Recommendation with High-Dimensional Side Information via Locality Preserving Projection
Authors: Yifan Chen, Xiang Zhao, Maarten de Rijke
Venue: SIGIR
Year: 2017

In this paper,we leverage high-dimensional side information to enhance top-N recommendations. To reduce the impact of the curse of high dimensionality, we incorporate a dimensionality reduction method, Locality Preserving Projection (LPP), into the recommendation model. A joint learning model is proposed to achieve the task of dimensionality reduction and recommendation simultaneously and iteratively. Specifically, item similarities generated by the recommendation model are used as the weights of the adjacency graph for LPP while the projections are used to bias the learning of item similarity. Employing LPP for recommendation not only preserves locality but also improves item similarity. Our experimental results illustrate that the proposed method is superiorover state-of-The-Art methods.
A Hierarchical Multimodal Attention-based Neural Network for Image Captioning
Authors: Yong Cheng, Fei Huang, Lian Zhou, Cheng Jin, Yuejie Zhang, Tao Zhang
Venue: SIGIR
Year: 2017

A novel hierarchical multimodal attention-based model is developed in this paper to generate more accurate and descriptive captions for images. Our model is an "end-to-end" neural network which contains three related sub-networks: a deep convolutional neural network to encode image contents, a recurrent neural network to identify the objects in images sequentially, and a multimodal attention-based recurrent neural network to generate image captions. The main contribution of our work is that the hierarchical structure and multimodal attention mechanism is both applied, thus each caption word can be generated with the multimodal attention on the intermediate semantic objects and the global visual content. Our experiments on two benchmark datasets have obtained very positive results.
Exploring User-Specific Information in Music Retrieval
Authors: Zhiyong Cheng, Jialie Shen, Liqiang Nie, Tat-Seng Chua, Mohan S. Kankanhalli
Venue: SIGIR
Year: 2017

With the advancement of mobile computing technology and cloud-based streaming music service, user-centered music retrieval has become increasingly important. User-specific information has a fundamental impact on personal music preferences and interests. However, existing research pays little attention to the modeling and integration of user-specific information in music retrieval algorithms/models to facilitate music search. In this paper, we propose a novel model, named User-Information-Aware Music Interest Topic (UIA-MIT) model. The model is able to effectively capture the influence of user-specific information on music preferences, and further associate users' music preferences and search terms under the same latent space. Based on this model, a user information aware retrieval system is developed, which can search and re-rank the results based on age- and/or gender-specific music preferences. A comprehensive experimental study demonstrates that our methods can significantly improve the search accuracy over existing text-based music retrieval methods.
On the Power Laws of Language: Word Frequency Distributions
Authors: Flavio Chierichetti, Ravi Kumar, Bo Pang
Venue: SIGIR
Year: 2017

About eight decades ago, Zipf postulated that the word frequency distribution of languages is a power law, i.e., it is a straight line on a log-log plot. Over the years, this phenomenon has been documented and studied extensively. For many corpora, however, the empirical distribution barely resembles a power law: when plotted on a log-log scale, the distribution is concave and appears to be composed of two differently sloped straight lines joined by a smooth curve. A simple generative model is proposed to capture this phenomenon. The word frequency distributions produced by this model are shown to match the observations both analytically and empirically.
TOTEM: Personal Tweets Summarization on Mobile Devices
Authors: Jin Yao Chin, Sourav S. Bhowmick, Adam Jatowt
Venue: SIGIR
Year: 2017

Tweets summarization aims to find a group of representative tweets for a specific topic. In recent times, there have been several research efforts toward devising a variety of techniques to summarize tweets in Twitter. However, these techniques are either not personal (i.e., consider only tweets in the timeline of a specific user) or are too expensive to be realized on a mobile device. Given that 80% of active Twitter users access the site on mobile devices, in this demonstration we present a lightweight, personalized, on-demand, topic modeling-based tweets summarization engine called TOTEM, designed for such devices. Specifically, TOTEM summarizes most recent tweets on a user's timeline and enables her to visualize and navigate representative topics and associated tweets in a user-friendly tap-and-swipe manner.
SIGIR 2017 Workshop on Open Knowledge Base and Question Answering (OKBQA2017)
Authors: Key-Sun Choi, Teruko Mitamura, Piek Vossen, Jin-Dong Kim, Axel-Cyrille Ngonga Ngomo
Venue: SIGIR
Year: 2017

Over the past years, several challenges and calls for research projects have pointed out the dire need for pushing natural language interfaces. In this context, the importance of Semantic Web data as a premier knowledge source is rapidly increasing. But we are still far from having accurate natural language interfaces that allow handling complex information needs in a user-centric and highly performant manner. The development of such interfaces requires collaboration of a range of different fields, including natural language processing, information extraction, knowledge base construction and population, reasoning, and question answering. With the goal to join forces in the collaborative development of natural language QA systems, the second OKBQA workshop is organized within the 40th SIGIR conference.
Making Ads More Relevant Innovations in Digital Advertising
Authors: Sudong Chung
Venue: SIGIR
Year: 2017

In the past decades, we found ourselves spending more and more time on the Internet. As we surf the Internet longer, we are exposed to digital advertisement more. Some of us got used to it and even take it for granted. And some of us sought for a remedy and use ad-blocking software. This could be an end of story, especially if we were not computer scientists Who are making those advertisements and why? As we know that the television networks make money by broadcasting tv commercial to their viewers. They use the money to create contents and make profit. Therefore, it is obvious that we watch commercial to watch their shows. Of course, some ad-free television networks make funds from monthly subscription fee. But why do we feel that we see more ads online than before? Are we right about it? The answer is Yes. We are seeing more ads than before and it is not only because we spend more time online but also because more marketing money has shifted from offline to online marketing. The increase in digital marketing is due to several reasons; 1) increase of time spent online 2) descent of publishers' offline business 3) innovations in digital advertising technology and creatives. In this talk, I will talk about the history of digital advertising and the recent innovations in advertising technologies to make ads more relevant to individual viewer. Also I will explain why the advanced machine learning and massive user data are behind the innovations.