OpenRaiser/Intern-Atlas huggingface dataset:
Exploiting web search engines to search structured databases
Authors: Sanjay Agrawal, Kaushik Chakrabarti, Surajit Chaudhuri, Venkatesh Ganti, Arnd Christian König, Dong Xin
Venue: Conference WWW 2009
Year: 2009

Web search engines often federate many user queries to relevant structured databases. For example, a product related query might be federated to a product database containing their descriptions and specifications. The relevant structured data items are then returned to the user along with web search results. However, each structured database is searched in isolation. Hence, the search often produces empty or incomplete results as the database may not contain the required information to answer the query. In this paper, we propose a novel integrated search architecture. We establish and exploit the relationships between web search results and the items in structured databases to identify the relevant structured data items for a much wider range of queries.Our architecture leverages existing search engine components to implement this functionality at very low overhead. We demonstrate the quality and efficiency of our techniques through an extensive experimental study.
Result enrichment in commerce search using browse trails
Authors: Debmalya Panigrahi, Sreenivas Gollapudi
Venue: Conference WSDM 2011
Year: 2011

Commerce search engines have become popular in recent years, as users increasingly search for (and buy) products on the web. In response to an user query, they surface links to products in their catalog (or index) that match the requirements specified in the query. Often, few or no product in the catalog matches the user query exactly, and the search engine is forced to return a set of products that partially match the query. This paper considers the problem of choosing a set of products in response to an user query, so as to ensure maximum user satisfaction. We call this the result enrichment problem in commerce search.
Just-in-time contextual advertising
Authors: Aris Anagnostopoulos, Andrei Z. Broder, Evgeniy Gabrilovich, Vanja Josifovski, Lance Riedel
Venue: Conference CIKM 2007
Year: 2007

Contextual Advertising is a type of Web advertising, which, given the URL of a Web page, aims to embed into the page (typically via JavaScript) the most relevant textual ads available. For static pages that are displayed repeatedly, the matching of ads can be based on prior analysis of their entire content; however, ads need to be matched also to new or dynamically created pages that cannot be processed ahead of time. Analyzing the entire body of such pages on-the-fly entails prohibitive communication and latency costs. To solve the three-horned dilemma of either low-relevance or high-latency or high-load, we propose to use text summarization techniques paired with external knowledge (exogenous to the page) to craft short page summaries in real time. Empirical evaluation proves that matching ads on the basis of such summaries does not sacrifice relevance, and is competitive with matching based on the entire page content. Specifically, we found that analyzing a carefully selected 5% fraction of the page text sacrifices only 1%-3% in ad relevance. Furthermore, our summaries are fully compatible with the standard JavaScript mechanisms used for ad placement: they can be produced at ad-display time by simple additions to the usual script, and they only add 500-600 bytes to the usual request.
Retrieval models for audience selection in display advertising
Authors: Sarah K. Tyler, Sandeep Pandey, Evgeniy Gabrilovich, Vanja Josifovski
Venue: Conference CIKM 2011
Year: 2011

Web applications often rely on user profiles of observed user actions, such as queries issued, page views, etc. In audience selection for display advertising, the audience that is likely to be responsive to a given ad campaign is identified via such profiles. We formalize the audience selection problem as a ranked retrieval task over an index of known users. We focus on the common case of audience selection where a small seed set of users who have previously responded positively to the campaign is used to identify a broader target audience. The actions of the users in the seed set are aggregated to construct a query, the query is then executed against an index of other user profiles to retrieve the highest scoring profiles. We validate our approach on a real-world dataset, demonstrating the trade-offs of different user and query models and that our approach is particularly robust for small campaigns. The proposed user modeling framework is applicable to many other applications requiring user profiles such as content suggestion and personalization.
Leveraging wikipedia concept and category information to enhance contextual advertising
Authors: Zongda Wu, Guandong Xu, Rong Pan, Yanchun Zhang, Zhiwen Hu, Jianfeng Lu
Venue: Conference CIKM 2011
Year: 2011

As a prevalent type of Web advertising, contextual advertising refers to the placement of the most relevant ads into a Web page, so as to increase the number of ad-clicks. However, some problems of homonymy and polysemy, low intersection of keywords etc., can lead to the selection of irrelevant ads for a page. In this paper, we present a new contextual advertising approach to overcome the problems, which uses Wikipedia concept and category information to enrich the content representation of an ad (or a page). First, we map each ad and page into a keyword vector, a concept vector and a category vector. Next, we select the relevant ads for a given page based on a similarity metric that combines the above three feature vectors together. Last, we evaluate our approach by using real ads, pages, as well as a great number of concepts and categories of Wikipedia. Experimental results show that our approach can improve the precision of ads-selection effectively.
PEQ: An Explainable, Specification-based, Aspect-oriented Product Comparator for E-commerce
Authors: Abhishek Sikchi, Pawan Goyal, Samik Datta
Venue: Conference CIKM 2016
Year: 2016

While purchasing a product, consumers often rely on specifications as well as online reviews of the product for decision-making. While comparing, one often has in mind a specific aspect or a set of aspects which are of interest to them. Previous work has used comparative sentences, where two entities are compared directly in a single sentence by the review author, towards the comparison task. In this paper, we extend the existing model by incorporating the feature specifications of the products, which are easily available, and learn the importance to be associated with each of them. To test the validity of these product ranking measures, we comprehensively test it on a digital camera dataset from Amazon.com and the results show good empirical outperformance over the state-of-the-art baselines.
Characterising Dataset Search Queries
Authors: Emilia Kacprzak, Laura Koesten, Jeni Tennison, Elena Simperl
Venue: Conference WWW 2018
Year: 2018

The amount of data generated and published on the web is increasing rapidly, but search for structured data on the web still presents challenges. In this paper we explore dataset search by analysing queries specifically generated for this work through a crowdsourcing experiment and comparing them to a search log analysis of queries on data portals. The change in search environment together with the task we gave people altered the generated queries. We found that queries issued in our experiment were much longer than search queries for datasets on data portals. They further contained seven times more mentions of geospatial and of temporal information and are more likely to be structured as questions. These insights can be used to tailor search functionalities to the particular information needs and characteristics of dataset search.
A study on the Interpretability of Neural Retrieval Models using DeepSHAP
Authors: Zeon Trevor Fernando, Jaspreet Singh, Avishek Anand
Venue: Conference SIGIR 2019
Year: 2019

A recent trend in IR has been the usage of neural networks to learn retrieval models for text based adhoc search. While various approaches and architectures have yielded significantly better performance than traditional retrieval models such as BM25, it is still difficult to understand exactly why a document is relevant to a query. In the ML community several approaches for explaining decisions made by deep neural networks have been proposed -- including DeepSHAP which modifies the DeepLift algorithm to estimate the relative importance (shapley values) of input features for a given decision by comparing the activations in the network for a given image against the activations caused by a reference input. In image classification, the reference input tends to be a plain black image. While DeepSHAP has been well studied for image classification tasks, it remains to be seen how we can adapt it to explain the output of Neural Retrieval Models (NRMs). In particular, what is a good "black" image in the context of IR? In this paper we explored various reference input document construction techniques. Additionally, we compared the explanations generated by DeepSHAP to LIME (a model agnostic approach) and found that the explanations differ considerably. Our study raises concerns regarding the robustness and accuracy of explanations produced for NRMs. With this paper we aim to shed light on interesting problems surrounding interpretability in NRMs and highlight areas of future work.
Medical question answering for clinical decision support
Authors: Travis R. Goodwin, Sanda M. Harabagiu
Venue: Conference CIKM 2016
Year: 2016

The goal of modern Clinical Decision Support (CDS) systems is to provide physicians with information relevant to their management of patient care. When faced with a medical case, a physician asks questions about the diagnosis, the tests, or treatments that should be administered. Recently, the TREC-CDS track has addressed this challenge by evaluating results of retrieving relevant scientific articles where the answers of medical questions in support of CDS can be found. Although retrieving relevant medical articles instead of identifying the answers was believed to be an easier task, state-of-the-art results are not yet sufficiently promising. In this paper, we present a novel framework for answering medical questions in the spirit of TREC-CDS by first discovering the answer and then selecting and ranking scientific articles that contain the answer. Answer discovery is the result of probabilistic inference which operates on a probabilistic knowledge graph, automatically generated by processing the medical language of large collections of electronic medical records (EMRs). The probabilistic inference of answers combines knowledge from medical practice (EMRs) with knowledge from medical research (scientific articles). It also takes into account the medical knowledge automatically discerned from the medical case description. We show that this novel form of medical question answering (Q/A) produces very promising results in (a) identifying accurately the answers and (b) it improves medical article ranking by 40%.
Uncovering the spatio-temporal dynamics of memes in the presence of incomplete information
Authors: Hancheng Ge, James Caverlee, Nan Zhang, Anna Cinzia Squicciarini
Venue: Conference CIKM 2016
Year: 2016

Modeling, understanding, and predicting the spatio-temporal dynamics of online memes are important tasks, with ramifications on location-based services, social media search, targeted advertising and content delivery networks. However, the raw data revealing these dynamics are often incomplete and error-prone; for example, API limitations and data sampling policies can lead to an incomplete (and often biased) perspective on these dynamics. Hence, in this paper, we investigate new methods for uncovering the full (underlying) distribution through a novel spatio-temporal dynamics recovery framework which models the latent relationships among locations, memes, and times. By integrating these hidden relationships into a tensor-based recovery framework -- called AirCP -- we find that high-quality models of meme spread can be built with access to only a fraction of the full data. Experimental results on both synthetic and real-world Twitter hashtag data demonstrate the promising performance of the proposed framework: an average improvement of over 27% in recovering the spatio-temporal dynamics of hashtags versus five state-of-the-art alternatives.
Interpretable nonnegative matrix decompositions
Authors: Saara Hyvönen, Pauli Miettinen, Evimaria Terzi
Venue: Conference KDD 2008
Year: 2008

A matrix decomposition expresses a matrix as a product of at least two factor matrices. Equivalently, it expresses each column of the input matrix as a linear combination of the columns in the first factor matrix. The interpretability of the decompositions is a key issue in many data-analysis tasks. We propose two new matrix-decomposition problems: the nonnegative CX and nonnegative CUR problems, that give naturally interpretable factors. They extend the recently-proposed column and column-row based decompositions, and are aimed to be used with nonnegative matrices. Our decompositions represent the input matrix as a nonnegative linear combination of a subset of its columns (or columns and rows).
Query- Task Mapping
Authors: Michael Völske, Ehsan Fatehifar, Benno Stein, Matthias Hagen
Venue: Conference SIGIR 2019
Year: 2019

Several recent task-based search studies aim at splitting query logs into sets of queries for the same task or information need. We address the natural next step: mapping a currently submitted query to an appropriate task in an already task-split log. This query-task mapping can, for instance, enhance query suggestions---rendering efficiency of the mapping, besides accuracy, a key objective. Our main contributions are three large benchmark datasets and preliminary experiments with four query-task mapping approaches: (1) a Trie-based approach, (2) MinHash~LSH, (3) word movers distance in a Word2Vec setup, and (4) an inverted index-based approach. The experiments show that the fast and accurate inverted index-based method forms a strong baseline.
Searchbots: Using Chatbots in Collaborative Information- Seeking Tasks
Authors: Sandeep Avula
Venue: Conference SIGIR 2017
Year: 2017

The popularity of messaging platforms such as Slack has given rise to thousands of different chatbots that users can engage with individually or as a group. The proposed dissertation research will investigate the use of searchbots (i.e., chatbots that perform specific search operations) during collaborative information-seeking tasks. Specifically, we will address the following research goals. RG1: Our first research goal will be to investigate the use of searchbots in a collaborative search scenario. The goal of collaborative search is to develop systems that help two or more people collaborate synchronously or asynchronously on information-seeking tasks. Collaborative search systems such as SearchTogether~\cite{Morris2007}, Coagmento~\cite{shah2010coagmento}, CollabSearch~\cite{Yue2012}, and ResultsSpace~\cite{Capra2012} allow users to share information, communicate asynchronously or in real-time, and provide interactive visualizations that raise awareness of each user's search activities, allowing users to learn from each other's search strategies and avoid duplicating work. Prior research shows that while people often search in pairs and in larger groups, they do so without the use of specialized search tools and instead coordinate via "out-of-channel" communication tools such as email, text messaging, phone, and social media~\cite{morris2008survey,Morris2013}. Our first goal will be to investigate the use of searchbots during real-time collaborative search tasks. Our interest in the use of searchbots for collaborative search echoes a suggestion made by Morris~\cite{Morris2013} to develop lightweight collaborative search tools over existing communication platforms.
cwl_eval: An Evaluation Tool for Information Retrieval
Authors: Leif Azzopardi, Paul Thomas, Alistair Moffat
Venue: Conference SIGIR 2019
Year: 2019

We present a tool ("cwl_eval") which unifies many metrics typically used to evaluate information retrieval systems using test collections. In the CWL framework metrics are specified via a single function which can be used to derive a number of related measurements: Expected Utility per item, Expected Total Utility, Expected Cost per item, Expected Total Cost, and Expected Depth. The CWL framework brings together several independent approaches for measuring the quality of a ranked list, and provides a coherent user model-based framework for developing measures based on utility (gain) and cost. Here we outline the CWL measurement framework; describe the cwl_eval architecture; and provide examples of how to use it. We provide implementations of a number of recent metrics, including Time Biased Gain, U-Measure, Bejewelled Measure, and the Information Foraging Based Measure, as well as previous metrics such as Precision, Average Precision, Discounted Cumulative Gain, Rank-Biased Precision, and INST. By providing state-of-the-art and traditional metrics within the same framework, we promote a standardised approach to evaluating search effectiveness.
Users versus Models: What Observation Tells us about Effectiveness Metrics
Authors: Alistair Moffat, Paul Thomas, Falk Scholer
Venue: Conference CIKM 2013
Year: 2013

Retrieval system effectiveness can be measured in two quite different ways: by monitoring the behavior of users and gathering data about the ease and accuracy with which they accomplish certain specified information-seeking tasks; or by using numeric effectiveness metrics to score system runs in reference to a set of relevance judgments. In the second approach, the effectiveness metric is chosen in the belief that user task performance, if it were to be measured by the first approach, should be linked to the score provided by the metric.
Interleaved Sequence RNNs for Fraud Detection
Authors: Bernardo Branco, Pedro Abreu, Ana Sofia Gomes, Mariana S. C. Almeida, João Tiago Ascensão, Pedro Bizarro
Venue: Conference KDD 2020
Year: 2020

Payment card fraud causes multibillion dollar losses for banks and merchants worldwide, often fueling complex criminal activities. To address this, many real-time fraud detection systems use tree-based models, demanding complex feature engineering systems to efficiently enrich transactions with historical data while complying with millisecond-level latencies. In this work, we do not require those expensive features by using recurrent neural networks and treating payments as an interleaved sequence, where the history of each card is an unbounded, irregular sub-sequence. We present a complete RNN framework to detect fraud in real-time, proposing an efficient ML pipeline from preprocessing to deployment. We show that these feature-free, multi-sequence RNNs outperform state-of-the-art models saving millions of dollars in fraud detection and using fewer computational resources.
Learning in a Pairwise Term-Term Proximity Framework for Information Retrieval
Authors: Ronan Cummins, Colm O&apos, Riordan
Venue: Conference SIGIR 2009
Year: 2009

Traditional ad hoc retrieval models do not take into account the closeness or proximity of terms. Document scores in these models are primarily based on the occurrences or nonoccurrences of query-terms considered independently of each other. Intuitively, documents in which query-terms occur closer together should be ranked higher than documents in which the query-terms appear far apart. \nThis paper outlines several term-term proximity measures and develops an intuitive framework in which they can be used to fully model the proximity of all query-terms for a particular topic. As useful proximity functions may be constructed from many proximity measures, we use a learning approach to combine proximity measures to develop a useful proximity function in the framework. An evaluation of the best proximity functions show that there is a significant improvement over the baseline ad hoc retrieval model and over other more recent methods that employ the use of single proximity measures.
Toward an interactive patent retrieval framework based on distributed representations
Authors: Walid Shalaby, Wlodek Zadrozny
Venue: Conference SIGIR 2018
Year: 2018

We present a novel interactive framework for patent retrieval leveraging distributed representations of concepts and entities extracted from the patents text. We propose a simple and practical interactive relevance feedback mechanism where the user is asked to annotate relevant/irrelevant results from the top n hits. We then utilize this feedback for query reformulation and term weighting where weights are assigned based on how good each term is at discriminating the relevant vs. irrelevant candidates. First, we demonstrate the efficacy of the distributed representations on the CLEF-IP 2010 dataset where we achieve significant improvement of 4.6% in recall over the keyword search baseline. Second, we simulate interactivity to demonstrate the efficacy of our interactive term weighting mechanism. Simulation results show that we can achieve significant improvement in recall from one interaction iteration outperforming previous semantic and interactive patent retrieval methods.
Learning to Persist: Exploring the Tradeoff Between Model Optimization and Experience Consistency
Authors: Dmitri Goldenberg, Guy Tsype, Igor Spivak, Javier Albert, Amir Tzur
Venue: Conference WWW 2021
Year: 2021

Machine learning models and recommender systems play a crucial role in web applications, providing personalized experiences to each customer. Recurring visits of the same customer raise a nontrivial question about the persistence of the experience. Given a changing user context, alongside online algorithms that update over time, the optimal treatment might differ from past model decisions. However, changing customer experience may create inconsistency and harm customer satisfaction and business process completion. This paper discusses the tradeoff between providing the user with a consistent experience and suggesting an up-to-date optimal treatment. We offer preliminary approaches to tackle the persistence problem and explore the tradeoffs in a simulated study.
Advances in recommender systems: From multi-stakeholder marketplaces to automated recsys
Authors: Rishabh Mehrotra, Ben Carterette, Yong Li 0008, Quanming Yao, Chen Gao 0001, James T. Kwok, Qiang Yang 0001, Isabelle Guyon
Venue: Conference KDD 2020
Year: 2020

The tutorial focuses on two major themes of recent advances in recommender systems: Part A: Recommendations in a Marketplace: Multi-sided marketplaces are steadily emerging as valuable ecosystems in many applications (e.g. Amazon, AirBnb, Uber), wherein the platforms have customers not only on the demand side (e.g. users), but also on the supply side (e.g. retailer). This tutorial focuses on designing search & recommendation frameworks that power such multi-stakeholder platforms. We discuss multi-objective ranking/recommendation techniques, discuss different ways in which stakeholders specify their objectives, highlight user specific characteristics (e.g. user receptivity) which could be leveraged when developing joint optimization modules and finally present a number of real world case-studies of such multi-stakeholder platforms.