OpenRaiser/Intern-Atlas huggingface dataset:
Mining Tags Using Social Endorsement Networks
Authors: Theodoros Lappas, Kunal Punera, Tamás Sarlós
Venue: Conference SIGIR 2011
Year: 2011

Entities on social systems, such as users on Twitter, and images on Flickr, are at the core of many interesting applications: they can be ranked in search results, recommended to users, or used in contextual advertising. Such applications assume knowledge of an entity's nature and characteristic attributes. An effective way to encode such knowledge is in the form of tags. An untagged entity is practically inaccessible, since it is hard to retrieve or interact with. To address this, some platforms allow users to manually tag entities. However,while such tags can be informative, they can oftentimes be inadequate, trivial, ambiguous, or even plain false. Numerous automated tagging methods have been proposed to address these issues. However,most of them require pre-existing high-quality tags or descriptive texts for every entity that needs to be tagged. In our work, we propose a method based on social endorsements that is free from such constraints.
Modeling temporal effects of human mobile behavior on location-based social networks
Authors: Huiji Gao, Jiliang Tang, Xia Hu, Huan Liu
Venue: Conference CIKM 2013
Year: 2013

The rapid growth of location-based social networks (LBSNs) invigorates an increasing number of LBSN users, providing an unprecedented opportunity to study human mobile behavior from spatial, temporal, and social aspects. Among these aspects, temporal effects offer an essential contextual cue for inferring a user's movement. Strong temporal cyclic patterns have been observed in user movement in LBSNs with their correlated spatial and social effects (i.e., temporal correlations). It is a propitious time to model these temporal effects (patterns and correlations) on a user's mobile behavior. In this paper, we present the first comprehensive study of temporal effects on LBSNs. We propose a general framework to exploit and model temporal cyclic patterns and their relationships with spatial and social data. The experimental results on two real-world LBSN datasets validate the power of temporal effects in capturing user mobile behavior, and demonstrate the ability of our framework to select the most effective location prediction algorithm under various combinations of prediction models.
Partitioned Elias-Fano Indexes
Authors: Giuseppe Ottaviano, Rossano Venturini
Venue: Conference SIGIR 2014
Year: 2014

The Elias-Fano representation of monotone sequences has been recently applied to the compression of inverted indexes, showing excellent query performance thanks to its efficient random access and search operations. While its space occupancy is competitive with some state-of-the-art methods such as gamma-delta-Golomb codes and PForDelta, it fails to exploit the local clustering that inverted lists usually exhibit, namely the presence of long subsequences of close identifiers. In this paper we describe a new representation based on partitioning the list into chunks and encoding both the chunks and their endpoints with Elias-Fano, hence forming a two-level data structure. This partitioning enables the encoding to better adapt to the local statistics of the chunk, thus exploiting clustering and improving compression. We present two partition strategies, respectively with fixed and variable-length chunks. For the latter case we introduce a linear-time optimization algorithm which identifies the minimum-space partition up to an arbitrarily small approximation factor.
Using term location information to enhance probabilistic information retrieval
Authors: Baiyan Liu, Xiangdong An, Jimmy Xiangji Huang
Venue: Conference SIGIR 2015
Year: 2015

Nouns are more important than other parts of speech in information retrieval and are more often found near the beginning or the end of sentences. In this paper, we investigate the effects of rewarding terms based on their location in sentences on information retrieval. Particularly, we propose a novel Term Location (TEL) retrieval model based on BM25 to enhance probabilistic information retrieval, where a kernel-based method is used to capture term placement patterns. Experiments on five TREC datasets of varied size and content indicate the proposed model significantly outperforms the optimized BM25 and DirichletLM in MAP over all datasets with all kernel functions, and excels the optimized BM25 and DirichletLM over most of the datasets in [email protected] and [email protected] with different kernel functions.
Do bots impact twitter activity?
Authors: Zafar Gilani, Reza Farahbakhsh, Jon Crowcroft
Venue: Conference WWW 2017
Year: 2017

The WWW has seen massive growth in population of automated programs (bots) for a variety of exploits on online social networks (OSNs). In this paper we extend on our previous work to study the affects of bots on Twitter. By setting up a bot account on Twitter and conducting analysis on a click logs dataset from our web server, we show that despite bots being in smaller numbers, they exercise a profound impact on content popularity and activity on Twitter.
Label distribution augmented maximum likelihood estimation for reading comprehension
Authors: Lixin Su, Jiafeng Guo, Yixing Fan, Yanyan Lan, Xueqi Cheng
Venue: Conference WSDM 2020
Year: 2020

Reading comprehension (RC) aims to locate a text span from a context passage to answer the given question. Despite the effectiveness of modern neural RC models, most existing work relies on maximum likelihood estimation (MLE) and ignores the structure of the output space. That is during training, one treats all the text spans do not match the ground truth as equally poor, leading to overconfident predictions on ground truth labels and reduced generalization ability in test. One way to bridge the gap between training and test is to take into account the task reward of alternative outputs using the reinforcement learning (RL) algorithms, which is often deficient in optimization as compared with MLE. In this paper, we propose a new learning criterion for the RC task which combines the merits of both MLE and RL-based methods. Specifically, we show that we are able to derive the distribution of the outputs, i.e., label distribution, using their corresponding task rewards based on the decomposition property of the RC problem. We then optimize the RC model by directly learning towards the auxiliary label distribution, instead of the ground truth label, using the MLE framework. In this way, we can make use of the structure of the output space for better generalization (as RL) via efficient optimization (as MLE). We name our approach as Label Distribution augmented MLE (LD-MLE), which is a general learning criterion that could be adopted by almost all the existing RC models. Experiments on three representative benchmark datasets demonstrate that RC models learned with the LD-MLE criterion can achieve consistently improved results over those based on the traditional MLE and RL-based criteria.
Characterizing and predicting search engine switching behavior
Authors: Ryen W. White, Susan T. Dumais
Venue: Conference CIKM 2009
Year: 2009

Search engine switching describes the voluntarily transition from one Web search engine to another. In this paper we present a study of search engine switching behavior that combines large-scale log-based analysis and survey data. We characterize aspects of switching behavior, and develop and evaluate predictive models of switching behavior using features of the active query, the current session, and user search history. Our findings provide insight into the decision-making processes of search engine users and demonstrate the relationship between switching and factors such as dissatisfaction with the quality of the results, the desire for broader topic coverage or verification of encountered information, and user preferences. The findings also reveal sufficient consistency in users' search behavior prior to engine switching to afford accurate prediction of switching events. Predictive models may be useful for search engines who may want to modify the search experience if they can accurately anticipate a switch.
Learning novelty-aware ranking of answers to complex questions
Authors: Shahar Harel, Sefi Albo, Eugene Agichtein, Kira Radinsky
Venue: Conference WWW 2019
Year: 2019

Result ranking diversification has become an important issue for web search, summarization, and question answering. For more complex questions with multiple aspects, such as those in community-based question answering (CQA) sites, a retrieval system should provide a diversified set of relevant results, addressing the different aspects of the query, while minimizing redundancy or repetition. We present a new method, DRN , which learns novelty-related features from unlabeled data with minimal social signals, to emphasize diversity in ranking. Specifically, DRN parameterizes question-answer interactions via an LSTM representation, coupled with an extension of neural tensor network, which in turn is combined with a novelty-driven sampling approach to automatically generate training data. DRN provides a novel and general approach to complex question answering diversification and suggests promising directions for search improvements.
Novelty based Ranking of Human Answers for Community Questions
Authors: Adi Omari, David Carmel, Oleg Rokhlenko, Idan Szpektor
Venue: Conference SIGIR 2016
Year: 2016

Questions and their corresponding answers within a community based question answering (CQA) site are frequently presented as top search results forWeb search queries and viewed by millions of searchers daily. The number of answers for CQA questions ranges from a handful to dozens, and a searcher would be typically interested in the different suggestions presented in various answers for a question. Yet, especially when many answers are provided, the viewer may not want to sift through all answers but to read only the top ones. Prior work on answer ranking in CQA considered the qualitative notion of each answer separately, mainly whether it should be marked as best answer. We propose to promote CQA answers not only by their relevance to the question but also by the diversification and novelty qualities they hold compared to other answers. Specifically, we aim at ranking answers by the amount of new aspects they introduce with respect to higher ranked answers (novelty), on top of their relevance estimation. This approach is common in Web search and information retrieval, yet it was not addressed within the CQA settings before, which is quite different from classic document retrieval. We propose a novel answer ranking algorithm that borrows ideas from aspect ranking and multi-document summarization, but adapts them to our scenario. Answers are ranked in a greedy manner, taking into account their relevance to the question as well as their novelty compared to higher ranked answers and their coverage of important aspects. An experiment over a collection of Health questions, using a manually annotated gold-standard dataset, shows that considering novelty for answer ranking improves the quality of the ranked answer list.
Validating models for disease detection using Twitter
Authors: Todd J. Bodnar, Marcel Salathé
Venue: Conference WWW 2013
Year: 2013

Data mining social media has become a valuable resource for infectious disease surveillance. However, there are considerable risks associated with incorrectly predicting an epidemic. The large amount of social media data combined with the small amount of ground truth data and the general dynamics of infectious diseases present unique challenges when evaluating model performance. In this paper, we look at several methods that have been used to assess influenza prevalence using Twitter. We then validate them with tests that are designed to avoid and illustrate issues with the standard k-fold cross validation method. We also find that small modifications to the way that data are partitioned can have major effects on a model's reported performance.
Does BERT pay attention to cyberbullying?
Authors: Fatma Elsafoury, Stamos Katsigiannis, Steven R. Wilson 0001, Naeem Ramzan
Venue: Conference SIGIR 2021
Year: 2021

Social media have brought threats like cyberbullying, which can lead to stress, anxiety, depression, and in some severe cases, suicide attempts. Detecting cyberbullying can help to warn/ block bullies and provide support to victims. However, very few studies have used self-attention-based language models like BERT for cyberbullying detection and they typically only report BERT's performance without examining in depth the reasons for its performance. In this work, we examine the use of BERT for cyberbullying detection on various datasets and attempt to explain its performance by analyzing its attention weights and gradient-based feature importance scores for textual and linguistic features. Our results show that attention weights do not correlate with feature importance scores and thus do not explain the model's performance. Additionally, they suggest that BERT relies on syntactical biases in the datasets to assign feature importance scores to class-related wordsrather than cyberbullying-related linguistic features.
Scattered or connected? an optimized parameter-efficient tuning approach for information retrieval
Authors: Xinyu Ma, Jiafeng Guo, Ruqing Zhang, Yixing Fan, Xueqi Cheng
Venue: Conference CIKM 2022
Year: 2022

Pre-training and fine-tuning have achieved significant advances in the information retrieval (IR). A typical approach is to fine-tune all the parameters of large-scale pre-trained models (PTMs) on downstream tasks. As the model size and the number of tasks increase greatly, such approach becomes less feasible and prohibitively expensive. Recently, a variety of parameter-efficient tuning methods have been proposed in natural language processing (NLP) that only fine-tune a small number of parameters while still attaining strong performance. Yet there has been little effort to explore parameter-efficient tuning for IR. In this work, we first conduct a comprehensive study of existing parameter-efficient tuning methods at both the retrieval and re-ranking stages. Unlike the promising results in NLP, we find that these methods cannot achieve comparable performance to full fine-tuning at both stages when updating less than 1% of the original model parameters. More importantly, we find that the existing methods are just parameter-efficient, but not learning-efficient as they suffer from unstable training and slow convergence. To analyze the underlying reason, we conduct a theoretical analysis and show that the separation of the inserted trainable modules makes the optimization difficult. To alleviate this issue, we propose to inject additional modules alongside the pre-trained models (PTMs) to make the original scattered modules connected. In this way, all the trainable modules can form a pathway to smooth the loss surface and thus help stabilize the training process. Experiments at both retrieval and re-ranking stages show that our method outperforms existing parameter-efficient methods significantly, and achieves comparable or even better performance over full fine-tuning.
Meta-learning for neural relation classification with distant supervision
Authors: Zhenzhen Li, Jian-Yun Nie, Benyou Wang, Pan Du, Yuhan Zhang, Lixin Zou, Dongsheng Li
Venue: Conference CIKM 2020
Year: 2020

Distant supervision provides a means to create a large number of weakly labeled data at low cost for relation classification. However, the resulting labeled instances are very noisy, containing data with wrong labels. Many approaches have been proposed to select a subset of reliable instances for neural model training, but they still suffer from noisy labeling problem or underutilization of the weakly-labeled data. To better select more reliable training instances, we introduce a small amount of manually labeled data as reference to guide the selection process. In this paper, we propose a meta-learning based approach, which learns to reweight noisy training data under the guidance of reference data. As the clean reference data is usually very small, we propose to augment it by dynamically distilling the most reliable elite instances from the noisy data. Experiments on several datasets demonstrate that the reference data can effectively guide the selection of training data, and our augmented approach consistently improves the performance of relation classification comparing to the existing state-of-the-art methods.
Comparative web search questions
Authors: Alexander Bondarenko, Pavel Braslavski, Michael Völske, Rami Aly, Maik Fröbe, Alexander Panchenko, Chris Biemann, Benno Stein, Matthias Hagen
Venue: Conference WSDM 2020
Year: 2020

\beginabstract We analyze comparative questions, i.e., questions asking to compare different items, that were submitted to Yandex in 2012. Responses to such questions might be quite different from the simple "ten blue links'' and could, for example, aggregate pros and cons of the different options as direct answers. However, changing the result presentation is an intricate decision such that the classification of comparative questions forms a highly precision-oriented task. From a year-long Yandex log, we annotate a random sample of 50,000~questions; 2.8%~of which are comparative. For these annotated questions, we develop a precision-oriented classifier by combining carefully hand-crafted lexico-syntactic rules with feature-based and neural approaches---achieving a recall of~0.6 at a perfect precision of~1.0. After running the classifier on the full year log (on average, there is at least one comparative question per second), we analyze 6,250~comparative questions using more fine-grained subclasses (e.g., should the answer be a "simple'' fact or rather a more verbose argument) for which individual classifiers are trained. An important insight is that more than 65%~of the comparative questions demand argumentation and opinions, i.e., reliable direct answers to comparative questions require more than the facts from a search engine's knowledge graph. In addition, we present a qualitative analysis of the underlying comparative information needs (separated into 14~categories likeconsumer electronics orhealth ), their seasonal dynamics, and possible answers from community question answering platforms. \endabstract
What users ask a search engine: Analyzing one billion russian question queries
Authors: Michael Völske, Pavel Braslavski, Matthias Hagen, Galina Lezina, Benno Stein
Venue: Conference CIKM 2015
Year: 2015

We analyze the question queries submitted to a large commercial web search engine to get insights about what people ask, and to better tailor the search results to the users' needs. Based on a dataset of about one billion question queries submitted during the year 2012, we investigate askers' querying behavior with the support of automatic query categorization. While the importance of question queries is likely to increase, at present they only make up 3-4% of the total search traffic.
Translating representations of knowledge graphs with neighbors
Authors: Chun-Chih Wang, Pu-Jen Cheng
Venue: Conference SIGIR 2018
Year: 2018

Knowledge graph completion is a critical issue because many applications benefit from their structural and rich resources. In this paper, we propose a method named TransN, which consid- ers the dependencies between triples and incorporates neighbor information dynamically. In experiments, we evaluate our model by link prediction and also conduct several qualitative analyses to prove effectiveness. Experimental results show that our model could integrate neighbor information effectively and outperform state-of-the-art models.
Operation diagnosis on procedure graph: The task and dataset
Authors: Ruipu Luo, Qi Zhu, Qin Chen, Siyuan Wang, Zhongyu Wei, Weijian Sun, Shuang Tang
Venue: Conference CIKM 2021
Year: 2021

Users usually consult the manufacturers or the internet when they encounter operation questions with an electronics product. In this paper, we explore to represent an operation question as a procedure graph and formulate the problem of operation diagnosis as two sub-tasks, namely error node detection, and correction, on top of the graph. We construct the first benchmark for this task and propose a transformer-based model to integrate external knowledge and context information to enhance the performance. Experimental results show the effectiveness of our proposed model.
Multilingual and multimodal hate speech analysis in twitter
Authors: Gretel Liz De la Peña Sarracén
Venue: Conference WSDM 2021
Year: 2021

Automatic hate speech detection has become a crucial task nowadays, due the increase of hate on the Internet and its negative consequences. Therefore, in our PhD we propose the design and implementation of methods for the automatic processing of hate messages. The study is focused on the hate messages on Twitter. The hypothesis on which the research is based is that the prediction of hate speech, considering textual content, can be improved by the combination of features such as the activity and communities of users, as well as the images that can be shared with the tweets. In this way, we intend to develop strategies for the automatic detection of hate with multimodal and also multilingual (both in English and Spanish) approaches. Furthermore, our research includes the study of counter-narrative as an alternative to mitigate the effects of hate speech. To address the problem, we employ deep learning techniques, deepening the study of approaches based on representation with graphs.
Revisiting code search in a two-stage paradigm
Authors: Fan Hu, Yanlin Wang, Lun Du, Xirong Li, Hongyu Zhang, Shi Han, Dongmei Zhang
Venue: Conference WSDM 2023
Year: 2023

With a good code search engine, developers can reuse existing code snippets and accelerate software development process. Current code search methods can be divided into two categories: traditional information retrieval (IR) based and deep learning (DL) based approaches. DL-based approaches include the cross-encoder paradigm and the bi-encoder paradigm. However, both approaches have certain limitations. The inference of IR-based and bi-encoder models are fast; however, they are not accurate enough; while cross-encoder models can achieve higher search accuracy but consume more time. In this work, we propose TOSS, a two-stage fusion code search framework that can combine the advantages of different code search methods. TOSS first uses IR-based and bi-encoder models to efficiently recall a small number of top-K code candidates, and then uses fine-grained cross-encoders for finer ranking. Furthermore, we conduct extensive experiments on different code candidate volumes and multiple programming languages to verify the effectiveness of TOSS. We also compare TOSS with six data fusion methods. Experimental results show that TOSS is not only efficient, but also achieves state-of-the-art accuracy with an overall mean reciprocal ranking (MRR) score of 0.763, compared to the best baseline result on the CodeSearchNet benchmark of 0.713.