OpenRaiser/Intern-Atlas huggingface dataset:
K-plet Recurrent Neural Networks for Sequential Recommendation
Authors: Xiang Lin, Shuzi Niu, Yiqiao Wang, Yucheng Li
Venue: SIGIR
Year: 2018

Recurrent Neural Networks have been successful in learning meaningful representations from sequence data, such as text and speech. However, recurrent neural networks attempt to model only the overall structure of each sequence independently, which is unsuitable for recommendations. In recommendation system, an optimal model should not only capture the global structure, but also the localized relationships. This poses a great challenge in the application of recurrent neural networks to the sequence prediction problem. To tackle this challenge, we incorporate the neighbor sequences into recurrent neural networks to help detect local relationships. Thus we propose a K -plet R ecurrent Neural Network (Kr Network for short) to accommodate multiple sequences jointly, and then introduce two ways to model their interactions between sequences. Experimental results on benchmark datasets show that our proposed architecture Kr Network outperforms state-of-the-art baseline methods in terms of generalization, short-term and long term prediction accuracy.
Who is the Mr. Right for Your Brand?: - Discovering Brand Key Assets via Multi-modal Asset-aware Projection
Authors: Yang Liu, Tobey H. Ko, Zhonglei Gu
Venue: SIGIR
Year: 2018

Following the rising prominence of online social networks, we observe an emerging trend for brands to adopt influencer marketing, embracing key opinion leaders (KOLs) to reach potential customers (PCs) online. Owing to the growing strategic importance of these brand key assets, this paper presents a novel feature extraction method named Multi-modal Asset-aware Projection (M2A2P) to learn a discriminative subspace from the high-dimensional multi-modal social media data for effective brand key asset discovery. By formulating a new asset-aware discriminative information preserving criterion, M2A2P differentiates with the existing multi-model feature extraction algorithms in two pivotal aspects: 1) We consider brand's highly imbalanced class interest steering towards the KOLs and PCs over the irrelevant users; 2) We consider a common observation that a user is not exclusive to a single class (e.g. a KOL can also be a PC). Experiments on a real-world apparel brand key asset dataset validate the effectiveness of the proposed method.
SIGIR 2018 Workshop on Intelligent Transportation Informatics
Authors: Yan Liu, Zhenhui Li, Wei Ai, Lingyu Zhang
Venue: SIGIR
Year: 2018

We propose a half-day workshop at SIGIR 2018 for the professionals, researchers, and practitioners who are interested in mining and understanding big and heterogeneous data generated in transportation to improve the transportation system. We plan to have both paper presentations and invited talks.
Towards Designing Better Session Search Evaluation Metrics
Authors: Mengyang Liu, Yiqun Liu, Jiaxin Mao, Cheng Luo, Shaoping Ma
Venue: SIGIR
Year: 2018

User satisfaction has been paid much attention to in recent Web search evaluation studies and regarded as the ground truth for designing better evaluation metrics. However, most existing studies are focused on the relationship between satisfaction and evaluation metrics at query-level. However, while search request becomes more and more complex, there are many scenarios in which multiple queries and multi-round search interactions are needed (e.g. exploratory search). In those cases, the relationship between session-level search satisfaction and session search evaluation metrics remain uninvestigated. In this paper, we analyze how users' perceptions of satisfaction accord with a series of session-level evaluation metrics. We conduct a laboratory study in which users are required to finish some complex search tasks and provide usefulness judgments of documents as well as session-level and query level satisfaction feedbacks. We test a number of popular session search evaluation metrics as well as different weighting functions. Experiment results show that query-level satisfaction is mainly decided by the clicked document that they think the most useful (maximum effect). While session-level satisfaction is highly correlated with the most recently issued queries (recency effect). We further propose a number of criteria for designing better session search evaluation metrics.
A Flexible Forecasting Framework for Hierarchical Time Series with Seasonal Patterns: A Case Study of Web Traffic
Authors: Zitao Liu, Yan Yan, Milos Hauskrecht
Venue: SIGIR
Year: 2018

In this work, we focus on models and analysis of multivariate time series data that are organized in hierarchies. Such time series are referred to as hierarchical time series (HTS) and they are very common in business, management, energy consumption, social networks, or web traffic modeling and analysis. We propose a new flexible hierarchical forecasting framework, that takes advantage of the hierarchical relational structure to predict individual time series. Our new forecasting framework is able to (1) handle HTS modeling and forecasting problems; (2) make accurate forecasting for HTS with seasonal patterns; (3) incorporate various individual forecasting models and combine heuristics based on the HTS datasets' own characterization. The proposed framework is evaluated on a real-world web traffic data set. The results demonstrate that our approach is superior when applied to hierarchical web traffic prediction problems, and it outperforms alternative time series prediction models in terms of accuracy
Index Compression for BitFunnel Query Processing
Authors: Xinyu Liu, Zhaohua Zhang, Boran Hou, Rebecca J. Stones, Gang Wang, Xiaoguang Liu
Venue: SIGIR
Year: 2018

Large-scale search engines utilize inverted indexes which store ordered lists of document identifies (docIDs) relevant to query terms, which can be queried thousands of times per second. In order to reduce storage requirements, we propose a dictionary-based compression approach for the recently proposed bitwise data-structure BitFunnel, which makes use of a Bloom filter. Compression is achieved through storing frequently occurring blocks in a dictionary. Infrequently occurring blocks (those which are not represented in the dictionary) are instead referenced using similar blocks that are in the dictionary, introducing additional false positive errors. We further introduce a docID reordering strategy to improve compression. Experimental results indicate an improvement in compression by 27% to 30%, at the expense of increasing the query processing time by 16% to 48% and increasing the false positive rate by around 7.6 to 10.7 percentage points.
Learning Geo-Social User Topical Profiles with Bayesian Hierarchical User Factorization
Authors: Haokai Lu, Wei Niu, James Caverlee
Venue: SIGIR
Year: 2018

Understanding user interests and expertise is a vital component toward creating rich user models for information personalization in social media, recommender systems and web search. To capture the pair-wise interactions between geo-location and user's topical profile in social-spatial systems, we propose the modeling of fine-grained and multi-dimensional user geo-topic profiles. We then propose a two-layered Bayesian hierarchical user factorization generative framework to overcome user heterogeneity and another enhanced model integrated with user's contextual information to alleviate multi-dimensional sparsity. Through extensive experiments, we find the proposed model leads to a 5\textasciitilde13% improvement in precision and recall over the alternative baselines and an additional 6\textasciitilde11% improvement with the integration of user's contexts.
Selective Gradient Boosting for Effective Learning to Rank
Authors: Claudio Lucchese, Franco Maria Nardini, Raffaele Perego, Salvatore Orlando, Salvatore Trani
Venue: SIGIR
Year: 2018

Learning an effective ranking function from a large number of query-document examples is a challenging task. Indeed, training sets where queries are associated with a few relevant documents and a large number of irrelevant ones are required to model real scenarios of Web search production systems, where a query can possibly retrieve thousands of matching documents, but only a few of them are actually relevant. In this paper, we propose Selective Gradient Boosting (SelGB), an algorithm addressing the Learning-to-Rank task by focusing on those irrelevant documents that are most likely to be mis-ranked, thus severely hindering the quality of the learned model. SelGB exploits a novel technique minimizing the mis-ranking risk, i.e., the probability that two randomly drawn instances are ranked incorrectly, within a gradient boosting process that iteratively generates an additive ensemble of decision trees. Specifically, at every iteration and on a per query basis, SelGB selectively chooses among the training instances a small sample of negative examples enhancing the discriminative power of the learned model. Reproducible and comprehensive experiments conducted on a publicly available dataset show that SelGB exploits the diversity and variety of the negative examples selected to train tree ensembles that outperform models generated by state-of-the-art algorithms by achieving improvements of NDCG@10 up to 3.2%.
Mention Recommendation for Multimodal Microblog with Cross-attention Memory Network
Authors: Renfeng Ma, Qi Zhang, Jiawen Wang, Lizhen Cui, Xuanjing Huang
Venue: SIGIR
Year: 2018

The users of Twitter-like social media normally use the "@'' sign to select a suitable person to mention. It is a significant role in promoting the user experience and information propagation. To help users easily find the usernames they want to mention, the mention recommendation task has received considerable attention in recent years. Previous methods only incorporated textual information when performing this task. However, many users not only post texts on social media but also the corresponding images. These images can provide additional information that is not included in the text, which could be helpful in improving the accuracy of a mention recommendation. To make full use of textual and visual information, we propose a novel cross-attention memory network to perform the mention recommendation task for multimodal tweets. We incorporate the interests of users with external memory and use the cross-attention mechanism to extract both textual and visual information. Experimental results on a dataset collected from Twitter demonstrated that the proposed method can achieve better performance than state-of-the-art methods that use textual information only.
Online Job Search: Study of Users' Search Behavior using Search Engine Query Logs
Authors: Behrooz Mansouri, Mohammad Sadegh Zahedi, Ricardo Campos, Mojgan Farhoodi
Venue: SIGIR
Year: 2018

Over the last few years, an increasing number of user's and enterprises on the internet has generated a global marketplace for both employers and job seekers. Despite the fact that online job search is now more preferable than traditional methods - leading to better matches between the job seekers and the employer's intents - there is still little insight into how online job searches are different from general web searches. In this paper, we explore the different characteristics of online job search and their differences with general searches, by leveraging search engine query logs. Our experimental results show that job searches have specific attributes which can be used by search engines to increase the quality of the search results.
Deep Domain Adaptation Based on Multi-layer Joint Kernelized Distance
Authors: Sitong Mao, Xiao Shen, Fu-Lai Chung
Venue: SIGIR
Year: 2018

Domain adaptation refers to the learning scenario where a model learned from the source data is applied on the target data which have the same categories but different distributions. In information retrieval, there exist application scenarios like cross domain recommendation characterized similarly. In this paper, by utilizing deep features extracted from the deep networks, we proposed to compute the multi-layer joint kernelized mean distance between the k th target data predicted as the i th category and all the source data of the j th category $d_ij ^k$. Then, target data $T_m$ that are most likely to belong to the i th category can be found by calculating the relative distance $d_ii ^k/\sum_j d_ij ^k$. By iteratively adding $T_m$ to the training data, the finetuned deep model can adapt on the target data progressively. Our results demonstrate that the proposed method can achieve a better performance compared to a number of state-of-the-art methods.
Case-Based Retrieval Using Document-Level Semantic Networks
Authors: Stefano Marchesin
Venue: SIGIR
Year: 2018

We propose a research that aims at improving the effectiveness of case-based retrieval systems through the use of automatically created document-level semantic networks. The proposed research leverages the recent advancements in information extraction and relational learning to revisit and advance the core ideas of concept-centered hypertext models. The automatic extraction of semantic relations from documents --- and their centrality in the creation and exploitation of the documents' semantic networks --- represents our attempt to go one step further than previous approaches.
A/B Testing with APONE
Authors: Mónica Marrero, Claudia Hauff
Venue: SIGIR
Year: 2018

In order to improve long-term retention, ad conversion rates, and so on, A/B testing has become the norm within Web portals, enabling efficient large-scale experimentation. While A/B testing is also increasingly used by academic researchers (with crowd-working platforms offering a large pool of artificial users), few platforms are freely available to this end. Academic researchers usually develop adhoc solutions, leading to many duplicated efforts and time spent on work not directly related to one's research. As an alternative, we have developed and open sourced APONE, an A cademic P latform for ON line Experiments. APONE uses PlanOut, a framework and high-level language, to specify online experiments, and offers Web services and a Web GUI to easily create, manage and monitor them. By building a user friendly Web application, we enable not only experts to conduct valid A/B experiments. In particular as a secondary use case, we envision large classrooms to also benefit from the deployment of APONE, a vision we put into practice in a graduate Information Retrieval course. We open-source APONE at https://marrerom.github.io/APONE. A demo version is running at http://ireplatform.ewi.tudelft.nl:8080/APONE.
On Link Prediction in Knowledge Bases: Max-K Criterion and Prediction Protocols
Authors: Jiajie Mei, Richong Zhang, Yongyi Mao, Ting Deng
Venue: SIGIR
Year: 2018

Building knowledge base embedding models for link prediction has achieved great success. We however argue that the conventional top-k criterion used for evaluating the model performance is inappropriate. This paper introduces a new criterion, referred to as max-k. Through theoretical analysis and experimental study, we show that the top-k criterion is fundamentally inferior to max-k. We also introduce two prediction protocols for the max-k criterion. These protocols are strongly justified theoretically. Various insights concerning the max-k criterion and the two protocols are obtained through extensive experiments.
How Much is Too Much?: Whole Session vs. First Query Behaviors in Task Type Prediction
Authors: Matthew Mitsui, Jiqun Liu, Chirag Shah
Venue: SIGIR
Year: 2018

One of the emerging and important problems in Interactive Information Retrieval research is predicting search tasks. Given a searcher's behavior during a search session, can the searcher's task be predicted? Which aspects of the task can be predicted, and how quickly and how accurately can the prediction be made? Much past literature has examined relationships between browsing behavior and task type at a statistical level, and recent work is moving towards prediction. While one may think whole session measures are useful for prediction, recent findings on common measures have suggested the contrary. Can less of the session still be useful? We examine the opposite end: the first query. Using multiple data sets for comparison, our results suggest that first query measures can be at least as good as -- and sometimes better than -- whole session measures for certain task type predictions.
Theoretical Analysis of Interdependent Constraints in Pseudo-Relevance Feedback
Authors: Ali Montazeralghaem, Hamed Zamani, Azadeh Shakery
Venue: SIGIR
Year: 2018

Axiomatic analysis is a well-defined theoretical framework for analytical evaluation of information retrieval models. The current studies in axiomatic analysis implicitly assume that the constraints (axioms) are independent. In this paper, we revisit this assumption and hypothesize that there might be interdependence relationships between the existing constraints. As a preliminary study, we focus on the pseudo-relevance feedback (PRF) models that have been theoretically studied using the axiomatic analysis approach. In this paper, we introduce two novel interdependent PRF constraints which emphasize on the effect of existing constraints on each other. We further modify two state-of-the-art PRF models, log-logistic and relevance models, in order to satisfy the proposed constraints. Experiments on three TREC newswire and web collections demonstrate that the proposed modifications significantly outperform the baselines, in all cases.
Transparent Tree Ensembles
Authors: Alexander Moore, Vanessa Murdock, Yaxiong Cai, Kristine Jones
Venue: SIGIR
Year: 2018

Every day more technologies and services are backed by complex machine-learned models, consuming large amounts of data to provide a myriad of useful services. While users are willing to provide personal data to enable these services, their trust in and engagement with the systems could be improved by providing insight into how the machine learned decisions were made. Complex ML systems are highly effective but many of them are black boxes and give no insight into how they make the choices they make. Moreover, those that do often do so at the model-level rather than the instance-level. In this work we present a method for deriving explanations for instance-level decisions in tree ensembles. As this family of models accounts for a large portion of industrial machine learning, this work opens up the possibility for transparent models at scale.
Computational Surprise in Information Retrieval
Authors: Xi Niu, Wlodek Zadrozny, Kazjon Grace, Weimao Ke
Venue: SIGIR
Year: 2018

The concept of surprise is central to human learning and development. However, compared to accuracy, surprise has received little attention in the IR community, yet it is an essential component of the information seeking process. This workshop brings together researchers and practitioners of IR to discuss the topic of computational surprise, to set a research agenda, and to examine how to build datasets for research into this fascinating topic. The themes in this workshop include discussion of what can be learned from some well-known surprise models in other fields, such as Bayesian surprise; how to evaluate surprise based on user experience; and how computational surprise is related to the newly emerging areas, such as fake news detection, computational contradiction, clickbait detection, etc.
Locality-adapted Kernel Densities for Tweet Localization
Authors: Ozer Ozdikis, Heri Ramampiaro, Kjetil Nørvåg
Venue: SIGIR
Year: 2018

We propose a location prediction method for tweets based on the geographical probability distribution of their terms over a region. In our method, the probabilities are calculated using Kernel Density Estimation (KDE), where the bandwidth of the kernel function for each term is determined separately according to the location indicativeness of the term. Prediction for a new tweet is performed by combining the probability distributions of its terms weighted by their information gain ratio. The method we propose relies on statistical approaches without requiring any parameter tuning. Experiments conducted on three tweet sets from different regions of the world indicate significant improvement in prediction accuracy compared to the state-of-the-art methods.
CoNEREL: Collective Information Extraction in News Articles
Authors: Minh C. Phan, Aixin Sun
Venue: SIGIR
Year: 2018

We present CoNEREL, a system for collective named entity recognition and entity linking focusing on news articles and readers' comments. Different from other systems, CoNEREL processes articles and comments in batch mode, to make the best use of the shared contexts of multiple news stories and their comments. Particularly, a news article provides context for all its comments. To improve named entity recognition, CoNEREL utilizes co-reference of mentions to refine their class labels ( e.g. , person, location). To link the recognized entities to Wikipedia, our system implements Pair-Linking, a state-of-the-art entity linking algorithm. Furthermore, CoNEREL provides an interactive visualization of the Pair-Linking process. From the visualization, one can understand how Pair-Linking achieves decent linking performance through iterative evidence building, while being extremely fast and efficient. The graph formed by the Pair-Linking process naturally becomes a good summary of entity relations, making CoNEREL a useful tool to study the relationships between the entities mentioned in an article, as well as the ones that are discussed in its comments.
Machine Learning @ Amazon
Authors: Rajeev Rastogi
Venue: SIGIR
Year: 2018

In this talk, I will first provide an overview of key problem areas where we are applying Machine Learning techniques within Amazon such as product demand forecasting, product search, and information extraction from reviews, and associated technical challenges. I will then talk about two specific applications where we use a variety of methods to learn semantically rich representations of data: question answering where we use deep learning techniques and product size recommendations where we use probabilistic models. Rajeev Rastogi is a Director of Machine Learning at Amazon where he is developing ML platforms and applications for the e-commerce domain. Previously, he was Vice President of Yahoo! Labs Bangalore and the founding Director of the Bell Labs Research Center in Bangalore, India. Rajeev is an ACM Fellow and a Bell Labs Fellow. He is active in the fields of databases, data mining, and networking, and has served on the program committees of several conferences in these areas. He currently serves on the editorial boards of the CACM, VLDB Journal and ACM Computing Surveys, and has been an Associate editor for IEEE Transactions on Knowledge and Data Engineering in the past. He has published over 125 papers, and holds over 50 patents. Rajeev received his B. Tech degree from IIT Bombay, and a PhD degree in Computer Science from the University of Texas, Austin.
A Dataset and an Examination of Identifying Passages for Due Diligence
Authors: Adam Roegiest, Alexander K. Hudek, Anne McNulty
Venue: SIGIR
Year: 2018

We present and formalize the due diligence problem, where lawyers extract data from legal documents to assess risk in a potential merger or acquisition, as an information retrieval task. Furthermore, we describe the creation and annotation of a document collection for the due diligence problem that will foster research in this area. This dataset comprises 50 topics over 4,412 documents and ~15 million sentences and is a subset of our own internal training data. Using this dataset, we present what we have found to be the state of the art for information extraction in the due diligence problem. In particular, we find that when treating documents as sequences of labelled and unlabelled sentences, Conditional Random Fields significantly and substantially outperform other techniques for sequence-based (Hidden Markov Models) and non-sequence based machine learning (logistic regression). Included in this is an analysis of what we perceive to be the major failure cases when extraction is performed based upon sentence labels.