OpenRaiser/Intern-Atlas huggingface dataset:
Towards context-aware search with right click
Authors: Aixin Sun, Chii-Hian Lou
Venue: SIGIR
Year: 2014

Many queries are submitted to search engines by right-clicking the marked text (i.e., the query) in Web browsers. Because the document being read by the searcher often provides sufficient contextual information for the query, search engine could provide much more relevant search results if the query is augmented by the contextual information captured from the source document. How to extract the right contextual information from the source document is the main focus of this study. To this end, we evaluate 7 text component extraction schemes, and 5 feature extraction schemes. The former determines from which text component (e.g., title, meta-data, or paragraphs containing the selected query) to extract contextual information; the latter determines which words or phrases to extract. In total 35 combinations are evaluated and our evaluation results show that noun phrases extracted from all paragraphs that contain the query word is the best option.
Load balancing for partition-based similarity search
Authors: Xun Tang, Maha Alabduljalil, Xin Jin, Tao Yang
Venue: SIGIR
Year: 2014

All pairs similarity search, used in many data mining and information retrieval applications, is a time consuming process. Although a partition-based approach accelerates this process by simplifying parallelism management and avoiding unnecessary I/O and comparison, it is still challenging to balance the computation load among parallel machines with a distributed architecture. This is mainly due to the variation in partition sizes and irregular dissimilarity relationship in large datasets. This paper presents a two-stage heuristic algorithm to improve the load balance and shorten the overall processing time. We analyze the optimality and competitiveness of the proposed algorithm and demonstrates its effectiveness using several datasets. We also describe a static partitioning algorithm to even out the partition sizes while detecting more dissimilar pairs. The evaluation results show that the proposed scheme outperforms a previously developed solution by up to 41% in the tested cases.
Cache-conscious runtime optimization for ranking ensembles
Authors: Xun Tang, Xin Jin, Tao Yang
Venue: SIGIR
Year: 2014

Multi-tree ensemble models have been proven to be effective for document ranking. Using a large number of trees can improve accuracy, but it takes time to calculate ranking scores of matched documents. This paper investigates data traversal methods for fast score calculation with a large ensemble. We propose a 2D blocking scheme for better cache utilization with simpler code structure compared to previous work. The experiments with several benchmarks show significant acceleration in score calculation without loss of ranking accuracy.
Compositional data analysis (CoDA) approaches to distance in information retrieval
Authors: Paul Thomas, David Lovell
Venue: SIGIR
Year: 2014

Many techniques in information retrieval produce counts from a sample, and it is common to analyse these counts as proportions of the whole---term frequencies are a familiar example. Proportions carry only relative information and are not free to vary independently of one another: for the proportion of one term to increase, one or more others must decrease. These constraints are hallmarks of compositional data. While there has long been discussion in other fields of how such data should be analysed, to our knowledge, Compositional Data Analysis (CoDA) has not been considered in IR.
Principled dictionary pruning for low-memory corpus compression
Authors: Jiancong Tong, Anthony Wirth, Justin Zobel
Venue: SIGIR
Year: 2014

Compression of collections, such as text databases, can both reduce space consumption and increase retrieval efficiency, through better caching and better exploitation of the memory hierarchy. A promising technique is relative Lempel-Ziv coding, in which a sample of material from the collection serves as a static dictionary; in previous work, this method demonstrated extremely fast decoding and good compression ratios, while allowing random access to individual items. However, there is a trade-off between dictionary size and compression ratio, motivating the search for a compact, yet similarly effective, dictionary. In previous work it was observed that, since the dictionary is generated by sampling, some of it (selected substrings) may be discarded with little loss in compression. Unfortunately, simple dictionary pruning approaches are ineffective. We develop a formal model of our approach, based on generating an optimal dictionary for a given collection within a memory bound. We generate measures for identification of low-value substrings in the dictionary, and show on a variety of sizes of text collection that halving the dictionary size leads to only marginal loss in compression ratio. This is a dramatic improvement on previous approaches.
An event extraction model based on timeline and user analysis in Latent Dirichlet allocation
Authors: Bayar Tsolmon, Kyung-Soon Lee
Venue: SIGIR
Year: 2014

Social media such as Twitter has come to reflect the reaction of the general public to major events. Since posts are short and noisy, it is hard to extract reliable events based on word frequency. Even though an event term appears in a particularly low frequency, as long as at least one reliable user mentions the term, it should be extracted. This paper proposes an event extraction method which combines user reliability and timeline analysis. The Latent Dirichlet Allocation (LDA) topic model is adapted with the weights of event terms on timeline and reliable users to extract social events. The reliable users are detected on Twitter according to their tweeting behaviors: socially well-known users and active users. Reliable and low-frequency events can be detected based on reliable users In order to see the effectiveness of the proposed method, experiments are conducted on a Korean tweet collection; the proposed model achieved 72% in precision. This shows that the LDA with timeline and reliable users is effective for extracting events on the Twitter test collection.
How k-12 students search for learning?: analysis of an educational search engine log
Authors: Arif Usta, Ismail Sengör Altingövde, Ibrahim Bahattin Vidinli, Rifat Ozcan, Özgür Ulusoy
Venue: SIGIR
Year: 2014

In this study, we analyze an educational search engine log for shedding light on K-12 students' search behavior in a learning environment. We specially focus on query, session, user and click characteristics and compare the trends to the findings in the literature for general web search engines. Our analysis helps understanding how students search with the purpose of learning in an educational vertical, and reveals new directions to improve the search performance in the education domain.
Novelty and diversity enhancement and evaluation in recommender systems and information retrieval
Authors: Saúl Vargas
Venue: SIGIR
Year: 2014

The development and evaluation of Information Retrieval and Recommender Systems has traditionally focused on the relevance and accuracy of retrieved documents and recommendations, respectively. However, there is an increasing realization that accuracy alone might be a sub-optimal strategy for a successful user experience. Properties such as novelty and diversity have been explored in both fields for assessing and enhancing the usefulness of search results and recommendations. In this doctoral research we study the assessment and enhancement of both properties in the confluence of Information Retrieval and Recommender Systems.
Assessing the reliability and reusability of an E-discovery privilege test collection
Authors: Jyothi Keshavan Vinjumur, Douglas W. Oard, Jiaul H. Paik
Venue: SIGIR
Year: 2014

In some jurisdictions, parties to a lawsuit can request documents from each other, but documents subject to a claim of privilege may be withheld. The TREC 2010 Legal Track developed what is presently the only public test collection for evaluating privilege classification. This paper examines the reliability and reusability of that collection. For reliability, the key question is the extent to which privilege judgments correctly reflect the opinion of the senior litigator whose judgment is authoritative. For reusability, the key question is the degree to which systems whose results contributed to creation of the test collection can be fairly compared with other systems that use those privilege judgments in the future. These correspond to measurement error and sampling error, respectively. The results indicate that measurement error is the larger problem.
The effect of sampling strategy on inferred measures
Authors: Ellen M. Voorhees
Venue: SIGIR
Year: 2014

Using the inferred measures framework is a popular choice for constructing test collections when the target document set is too large for pooling to be a viable option. Within the framework, different amounts of assessing effort is placed on different regions of the ranked lists as defined by a sampling strategy. The sampling strategy is critically important to the quality of the resultant collection, but there is little published guidance as to the important factors. This paper addresses this gap by examining the effect on collection quality of different sampling strategies within the inferred measures framework. The quality of a collection is measured by how accurately it distinguishes the set of significantly different system pairs. Top-K pooling is competitive, though not the best strategy because it cannot distinguish topics with large relevant set sizes. Incorporating a deep, very sparsely sampled stratum is a poor choice. Strategies that include a top-10 pool create better collections than those that do not, as well as allow Precision(10) scores to be directly computed.
On run diversity in Evaluation as a Service
Authors: Ellen M. Voorhees, Jimmy Lin, Miles Efron
Venue: SIGIR
Year: 2014

"Evaluation as a service" (EaaS) is a new methodology that enables community-wide evaluations and the construction of test collections on documents that cannot be distributed. The basic idea is that evaluation organizers provide a service API through which the evaluation task can be completed. However, this concept violates some of the premises of traditional pool-based collection building and thus calls into question the quality of the resulting test collection. In particular, the service API might restrict the diversity of runs that contribute to the pool: this might hamper innovation by researchers and lead to incomplete judgment pools that affect the reusability of the collection. This paper shows that the distinctiveness of the retrieval runs used to construct the first test collection built using EaaS, the TREC 2013 Microblog collection, is not substantially different from that of the TREC-8 ad hoc collection, a high-quality collection built using traditional pooling. Further analysis using the `leave out uniques' test suggests that pools from the Microblog 2013 collection are less complete than those from TREC-8, although both collections benefit from the presence of distinctive and effective manual runs. Although we cannot yet generalize to all EaaS implementations, our analyses reveal no obvious flaws in the test collection built using the methodology in the TREC 2013 Microblog track.
Collaborative personalized Twitter search with topic-language models
Authors: Jan Vosecky, Kenneth Wai-Ting Leung, Wilfred Ng
Venue: SIGIR
Year: 2014

The vast amount of real-time and social content in microblogs results in an information overload for users when searching microblog data. Given the user's search query, delivering content that is relevant to her interests is a challenging problem. Traditional methods for personalized Web search are insufficient in the microblog domain, because of the diversity of topics, sparseness of user data and the highly social nature. In particular, social interactions between users need to be considered, in order to accurately model user's interests, alleviate data sparseness and tackle the cold-start problem. In this paper, we therefore propose a novel framework for Collaborative Personalized Twitter Search. At its core, we develop a collaborative user model, which exploits the user's social connections in order to obtain a comprehensive account of her preferences. We then propose a novel user model structure to manage the topical diversity in Twitter and to enable semantic-aware query disambiguation. Our framework integrates a variety of information about the user's preferences in a principled manner. A thorough evaluation is conducted using two personalized Twitter search query logs, demonstrating a superior ranking performance of our framework compared with state-of-the-art baselines.
Learning to bridge colloquial and formal language applied to linking and search of E-Commerce data
Authors: Ivan Vulic, Susana Zoghbi, Marie-Francine Moens
Venue: SIGIR
Year: 2014

We study the problem of linking information between different idiomatic usages of the same language, for example, colloquial and formal language. We propose a novel probabilistic topic model called multi-idiomatic LDA (MiLDA). Its modeling principles follow the intuition that certain words are shared between two idioms of the same language, while other words are non-shared, that is, idiom-specific. We demonstrate the ability of our model to learn relations between cross-idiomatic topics in a dataset containing product descriptions and reviews. We intrinsically evaluate our model by the perplexity measure. Following that, as an extrinsic evaluation, we present the utility of the new MiLDA topic model in a recently proposed IR task of linking Pinterest pins (given in colloquial English on the users' side) to online webshops (given in formal English on the retailers' side). We show that our multi-idiomatic model outperforms the standard monolingual LDA model and the pure bilingual LDA model both in terms of perplexity and MAP scores in the IR task.
Modeling action-level satisfaction for search task satisfaction prediction
Authors: Hongning Wang, Yang Song, Ming-Wei Chang, Xiaodong He, Ahmed Hassan Awadallah, Ryen W. White
Venue: SIGIR
Year: 2014

Search satisfaction is a property of a user's search process. Understanding it is critical for search providers to evaluate the performance and improve the effectiveness of search engines. Existing methods model search satisfaction holistically at the search-task level, ignoring important dependencies between action-level satisfaction and overall task satisfaction. We hypothesize that searchers' latent action-level satisfaction (i.e., whether they believe they were satisfied with the results of a query or click) influences their observed search behaviors and contributes to overall search satisfaction. We conjecture that by modeling search satisfaction at the action level, we can build more complete and more accurate predictors of search-task satisfaction. To do this, we develop a latent structural learning method, whereby rich structured features and dependency relations unique to search satisfaction prediction are explored. Using in-situ search satisfaction judgments provided by searchers, we show that there is significant value in modeling action-level satisfaction in search-task satisfaction prediction. In addition, experimental results on large-scale logs from Bing.com demonstrate clear benefit from using inferred action satisfaction labels for other applications such as document relevance estimation and query suggestion.
Influential nodes selection: a data reconstruction perspective
Authors: Zhefeng Wang, Hao Wang, Qi Liu, Enhong Chen
Venue: SIGIR
Year: 2014

Influence maximization is the problem of finding a set of seed nodes in social network for maximizing the spread of influence. Traditionally, researchers view influence propagation as a stochastic process and formulate the influence maximization problem as a discrete optimization problem. Thus, most previous works focus on finding efficient and effective heuristic algorithms within the greedy framework. In this paper, we view the influence maximization problem from the perspective of data reconstruction and propose a novel framework named \textsl{Data Reconstruction for Influence Maximization}(DRIM). In our framework, we first construct an influence matrix, each row of which is the influence of a node to other nodes. Then, we select $k$ most informative rows to reconstruct the matrix and the corresponding nodes are the seed nodes which could maximize the influence spread. Finally, we evaluate our framework on two real-world data sets, and the results show that DRIM is at least as effective as the traditional greedy algorithm.
Efficiently identify local frequent keyword co-occurrence patterns in geo-tagged Twitter stream
Authors: Xiaoyang Wang, Ying Zhang, Wenjie Zhang, Xuemin Lin
Venue: SIGIR
Year: 2014

With the prevalence of the geo-position enabled devices and services, a rapidly growing amount of tweets are associated with geo-tags. Consequently, the real time search on geo-tagged Twitter streams has attracted great attentions.In this paper, we advocate the significance of the co-occurrence of keywords for the geo-tagged tweets data analytics, which is overlooked by existing studies. Particularly, we formally introduce the problem of identifying local frequent keyword co-occurrence patterns over the geo-tagged Twitter streams, namely LFP\xspace query. To accommodate the high volume and the rapid updates of the Twitter stream, we develop an inverted KMV sketch (IK\xspace sketch for short) structure to capture the co-occurrence of keywords in limited space. Then efficient algorithms are developed based on IK\xspace sketch to support LFP\xspace queries as well as its variant. The extensive empirical study on real Twitter dataset confirms the effectiveness and efficiency of our approaches.
Discovering real-world use cases for a multimodal math search interface
Authors: Keita Del Valle Wangari, Richard Zanibbi, Anurag Agarwal
Venue: SIGIR
Year: 2014

To use math expressions in search, current search engines require knowing expression names or using a structure editor or string encoding (e.g., LaTeX). For mathematical non-experts, this can lead to an "intention gap" between the query they wish to express and what the interface will allow them to express. min is a search interface that supports drawing expressions on a canvas using mouse/touch, keyboard and images. We present a user study examining whether min changes search behavior for mathematical non-experts, and to identify real-world usage scenarios for multimodal math search interfaces. Participants found query-by-expression using hand-drawn input useful, and identified scenarios in which they would like to use systems like min such as for locating, editing and sharing complex expressions (e.g., with many Greek letters), and working on complex math problems.
The data revolution: how companies are transforming with big data
Authors: Hugh E. Williams
Venue: SIGIR
Year: 2014

Spelling correction in the 1990s was all about algorithms and small dictionaries. This century, it is about mining vast data sets of past user behaviors, simple algorithms, and using those to correct mistakes. The large Internet giants are data-driven enterprises that use data to transform and continually improve user experiences. In this talk, Hugh Williams shares stories about data and how it is used to build Internet products, and explains why he believes data will transform businesses as we know them. Every major company is becoming a data-driven company, and Hugh shares examples of transformations occurring in health, aviation, farming, and telecommunications. He recently joined Pivotal, a company that is assembling the toolkit that exists in only a few consumer Internet companies, and making that toolkit open and available to every industry, including big data platforms, development frameworks, and an open, cloud-independent Platform-as-a-Service. He will conclude by sharing details about Pivotal, the Pivotal vision, and roadmap.
FitYou: integrating health profiles to real-time contextual suggestion
Authors: Christopher Wing, Hui Yang
Venue: SIGIR
Year: 2014

Obesity and its associated health consequences such as high blood pressure and cardiac disease affect a significant proportion of the world's population. At the same time, the popularity of location-based services (LBS) and recommender systems is continually increasing with improvements in mobile technology. We observe that the health domain lacks a suggestion system that focuses on healthy lifestyle choices. We introduce the mobile application FitYou, which dynamically generates recommendations according to the user's current location and health condition as a real-time LBS. It utilizes preferences determined from user history and health information from a biometric profile. The system was developed upon a top performing contextual suggestion system in both TREC 2012 and 2013 Contextual Suggestion Tracks.
Search result diversification via data fusion
Authors: Shengli Wu, Chunlan Huang
Venue: SIGIR
Year: 2014

In recent years, researchers have investigated search result diversification through a variety of approaches. In such situations, information retrieval systems need to consider both aspects of relevance and diversity for those retrieved documents. On the other hand, previous research has demonstrated that data fusion is useful for improving performance when we are only concerned with relevance. However, it is not clear if it helps when both relevance and diversity are both taken into consideration. In this short paper, we propose a few data fusion methods to try to improve performance when both relevance and diversity are concerned. Experiments are carried out with 3 groups of top-ranked results submitted to the TREC web diversity task. We find that data fusion is still a useful approach to performance improvement for diversity as for relevance previously.
Learning to personalize trending image search suggestion
Authors: Chun-Che Wu, Tao Mei, Winston H. Hsu, Yong Rui
Venue: SIGIR
Year: 2014

Trending search suggestion is leading a new paradigm of image search, where user's exploratory search experience is facilitated with the automatic suggestion of trending queries. Existing image search engines, however, only provide general suggestions and hence cannot capture user's personal interest. In this paper, we move one step forward to investigate personalized suggestion of trending image searches according to users' search behaviors. To this end, we propose a learning-based framework including two novel components. The first component, i.e., trending-aware weight-regularized matrix factorization (TA-WRMF), is able to suggest personalized trending search queries by learning user preference from many users as well as auxiliary common searches. The second component associates the most representative and trending image with each suggested query. The personalized suggestion of image search consists of a trending textual query and its associated trending image. The combined textual-visual queries not only are trending (bursty) and personalized to user's search preference, but also provide the compelling visual aspect of these queries. We evaluate our proposed learning-based framework on a large-scale search logs with 21 million users and 41 million queries in two weeks from a commercial image search engine. The evaluations demonstrate that our system achieve about 50% gain compared with state-of-the-art in terms of query prediction accuracy.