OpenRaiser/Intern-Atlas huggingface dataset:
Shard Ranking and Cutoff Estimation for Topically Partitioned Collections
Authors: Anagha Kulkarni, Almer S. Tigelaar, Djoerd Hiemstra, Jamie Callan
Venue: Conference CIKM 2012
Year: 2012

Large document collections can be partitioned into 'topical shards' to facilitate distributed search. In a low-resource search environment only a few of the shards can be searched in parallel. Such a search environment faces two intertwined challenges. First, determining which shards to consult for a given query: shard ranking. Second, how many shards to consult from the ranking: cutoff estimation. In this paper we present a family of three algorithms that address both of these problems. As a basis we employ a commonly used data structure, the central sample index (CSI), to represent the shard contents. Running a query against the CSI yields a flat document ranking that each of our algorithms transforms into a tree structure. A bottom up traversal of the tree is used to infer a ranking of shards and also to estimate a stopping point in this ranking that yields cost-effective selective distributed search. As compared to a state-of-the-art shard ranking approach the proposed algorithms provide substantially higher search efficiency while providing comparable search effectiveness.
To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers
Authors: Hang Li, Shuai Wang, Shengyao Zhuang, Ahmed Mourad, Xueguang Ma, Jimmy Lin, Guido Zuccon
Venue: Conference SIGIR 2022
Year: 2022

Current pre-trained language model approaches to information retrieval can be broadly divided into two categories: sparse retrievers (to which belong also non-neural approaches such as bag-of-words methods, e.g., BM25) and dense retrievers. Each of these categories appears to capture different characteristics of relevance. Previous work has investigated how relevance signals from sparse retrievers could be combined with those from dense retrievers via interpolation. Such interpolation would generally lead to higher retrieval effectiveness. In this paper we consider the problem of combining the relevance signals from sparse and dense retrievers in the context of Pseudo Relevance Feedback (PRF). This context poses two key challenges: (1) When should interpolation occur: before, after, or both before and after the PRF process? (2) Which sparse representation should be considered: a zero-shot bag-of-words model (BM25), or a learned sparse representation? To answer these questions we perform a thorough empirical evaluation considering an effective and scalable neural PRF approach (Vector-PRF), three effective dense retrievers (ANCE, TCTv2, DistillBERT), and one state-of-the-art learned sparse retriever (uniCOIL). The empirical findings from our experiments suggest that, regardless of sparse representation and dense retriever, interpolation both before and after PRF achieves the highest effectiveness across most datasets and metrics.
Relevant Document Distribution Estimation Method for Resource Selection
Authors: Luo Si, James P. Callan
Venue: Conference SIGIR 2003
Year: 2003

Prior research under a variety of conditions has shown the CORI algorithm to be one of the most effective resource selection algorithms, but the range of database sizes studied was not large. This paper shows that the CORI algorithm does not do well in environments with a mix of "small" and "very large" databases. A new resource selection algorithm is proposed that uses information about database sizes as well as database contents. We also show how to acquire database size estimates in uncooperative environments as an extension of the query-based sampling used to acquire resource descriptions. Experiments demonstrate that the database size estimates are more accurate for large databases than estimates produced by a competing method; the new resource ranking algorithm is always at least as effective as the CORI algorithm; and the new algorithm results in better document rankings than the CORI algorithm.
SUSHI: Scoring Scaled Samples for Server Selection
Authors: Paul Thomas, Milad Shokouhi
Venue: Conference SIGIR 2009
Year: 2009

Modern techniques for distributed information retrieval use a set of documents sampled from each server, but these samples have been underutilised in server selection. We describe a new server selection algorithm, SUSHI, which unlike earlier algorithms can make full use of the text of each sampled document and which does not need training data. SUSHI can directly optimise for many common cases, including high precision retrieval, and by including a simple stopping condition can do so while reducing network traffic.
A context-based framework for resource citation classification in scientific literatures
Authors: He Zhao, Zhunchen Luo, Chong Feng, Yuming Ye
Venue: Conference SIGIR 2019
Year: 2019

In this paper, we introduce the task of resource citation classification for scientific literature using a context-based framework. This task is to analyze the purpose of citing an on-line resource in scientific text by modeling the role and function of each resource citation. It can be incorporated into resource indexing and recommendation systems to help better understand and classify on-line resources in scientific literature. We propose a new annotation scheme for this task and develop a dataset of 3,088 manually annotated resource citations. We adopt a neural-based model to build the classifiers and apply them on the large ARC dataset to examine the revolution of scientific resources from trends in their function over time.
Multi-level connection enhanced representation learning for script event prediction
Authors: Lihong Wang, Juwei Yue, Shu Guo, Jiawei Sheng, Qianren Mao, Zhenyu Chen, Shenghai Zhong, Chen Li
Venue: Conference WWW 2021
Year: 2021

Script event prediction (SEP) aims to choose a correct subsequent event from a candidate list, given a chain of ordered context events. Event representation learning has been proposed and successfully applied to this task. Most previous methods learning representations mainly focus on coarse-grained connections at event or chain level, while ignoring more fine-grained connections between events. Here we propose a novel framework which can enhance the representation learning of events by mining their connections at multiple granularity levels, including argument level, event level and chain level. In our method, we first employ a masked self-attention mechanism to model the relations between the components of events (i.e. arguments). Then, a directed graph convolutional network is further utilized to model the temporal or causal relations between events in the chain. Finally, we introduce an attention module to the context event chain, so as to dynamically aggregate context events with respect to the current candidate event. By fusing threefold connections in a unified framework, our approach can learn more accurate argument/event/chain representations, and thus leads to better prediction performance. Comprehensive experiment results on public New York Times corpus demonstrate that our model outperforms other state-of-the-art baselines. Our code is available in https://github.com/YueAWu/MCer.
Learning discriminative neural representations for event detection
Authors: Jinzhi Liao, Xiang Zhao 0002, Xinyi Li 0001, Lingling Zhang 0005, Jiuyang Tang
Venue: Conference SIGIR 2021
Year: 2021

Retrieving event instances from texts is pivotal to various natural language processing applications (e.g., automatic question answering and dialogue systems), and the first task to perform is event detection. There are two related sub-tasks therein-trigger identification and type classification, and the former is considered to play a dominant role. Nevertheless, it is notoriously challenging to predict event triggers right. To handle the task, existing work has made tremendous progress by incorporating manual features, data augmentation and neural networks, etc. Due to the scarcity of data and insufficient representation of trigger words, however, they still fail to precisely determine the spans of triggers (coined as trigger span detection problem). To address the challenge, we propose to learn discriminative neural representations (DNR) from texts. Specifically, our DNR model tackles the trigger span detection problem by exploiting two novel techniques: 1) a contrastive learning strategy, which enlarges the discrepancy between representations of words inside and outside triggers; and 2) a Mixspan strategy, which better trains the model to differentiate words nearby triggers' span boundaries. Extensive experiments on benchmarks-ACE2005 and TAC2015-demonstrate the superiority of our DNR model, leading to state-of-the-art performance.
EasyAug: An Automatic Textual Data Augmentation Platform for Classification Tasks
Authors: Siyuan Qiu, Binxia Xu, Jie Zhang, Yafang Wang, Xiaoyu Shen, Gerard de Melo, Chong Long, Xiaolong Li
Venue: Conference WWW 2020
Year: 2020

Imbalanced data is a perennial problem that impedes the learning abilities of current machine learning-based classification models. One approach to address it is to leverage data augmentation to expand the training set. For image data, there are a number of suitable augmentation techniques that have proven effective in previous work. For textual data, however, due to the discrete units inherent in natural language, techniques that randomly perturb the signal may be ineffective. Additionally, due to the substantial discrepancy between different textual datasets (e.g., different domains), an augmentation approach that facilitates the classification on one dataset may be detrimental on another dataset. For practitioners, comparing different data augmentation techniques is non-trivial, as the corresponding methods might need to be incorporated into different system architectures, and the implementation of some approaches, such as generative models, is laborious. To address these challenges, we develop EasyAug, a data augmentation platform that provides several augmentation approaches. Users can conveniently compare the classification results and can easily choose the most suitable one for their own dataset. In addition, the system is extensible and can incorporate further augmentation approaches, such that with minimal effort a new method can comprehensively be compared with the baselines.
Extracting situational information from microblogs during disaster events: A classification-summarization approach
Authors: Koustav Rudra, Subham Ghosh, Niloy Ganguly, Pawan Goyal, Saptarshi Ghosh
Venue: Conference CIKM 2015
Year: 2015

Microblogging sites like Twitter have become important sources of real-time information during disaster events. A significant amount of valuable situational information is available in these sites; however, this information is immersed among hundreds of thousands of tweets, mostly containing sentiments and opinion of the masses, that are posted during such events. To effectively utilize microblogging sites during disaster events, it is necessary to (i) extract the situational information from among the large amounts of sentiment and opinion, and (ii) summarize the situational information, to help decision-making processes when time is critical. In this paper, we develop a novel framework which first classifies tweets to extract situational information, and then summarizes the information. The proposed framework takes into consideration the typicalities pertaining to disaster events where (i) the same tweet often contains a mixture of situational and non-situational information, and (ii) certain numerical information, such as number of casualties, vary rapidly with time, and thus achieves superior performance compared to state-of-the-art tweet summarization approaches.
Improving (dis) agreement detection with inductive social relation information from comment-reply interactions
Authors: Yun Luo, Zihan Liu, Stan Z. Li, Yue Zhang
Venue: Conference WWW 2023
Year: 2023

(Dis)agreement detection aims to identify the authors' attitudes or positions (\textit{agree, disagree, neutral}) towards a specific text. It is limited for existing methods merely using textual information for identifying (dis)agreements, especially for cross-domain settings. Social relation information can play an assistant role in the (dis)agreement task besides textual information. We propose a novel method to extract such relation information from (dis)agreement data into an inductive social relation graph, merely using the comment-reply pairs without any additional platform-specific information. The inductive social relation globally considers the historical discussion and the relation between authors. Textual information based on a pre-trained language model and social relation information encoded by pre-trained RGCN are jointly considered for (dis)agreement detection. Experimental results show that our model achieves state-of-the-art performance for both the in-domain and cross-domain tasks on the benchmark -- DEBAGREEMENT. We find social relations can boost the performance of the (dis)agreement detection model, especially for the long-token comment-reply pairs, demonstrating the effectiveness of the social relation graph. We also explore the effect of the knowledge graph embedding methods, the information fusing method, and the time interval in constructing the social relation graph, which shows the effectiveness of our model.
Stance inference in Twitter through graph convolutional collaborative filtering networks with minimal supervision
Authors: Zhiwei Zhou, Erick Elejalde
Venue: Conference WWW 2023
Year: 2023

Social Media (SM) has become a stage for people to share thoughts, emotions, opinions, and almost every other aspect of their daily lives. This abundance of human interaction makes SM particularly attractive for social sensing. Especially during polarizing events such as political elections or referendums, users post information and encourage others to support their side, using symbols such as hashtags to represent their attitudes. However, many users choose not to attach hashtags to their messages, use a different language, or show their position only indirectly. Thus, automatically identifying their opinions becomes a more challenging task. To uncover these implicit perspectives, we propose a collaborative filtering model based on Graph Convolutional Networks that exploits the textual content in messages and the rich connections between users and topics. Moreover, our approach only requires a small annotation effort compared to state-of-the-art solutions. Nevertheless, the proposed model achieves competitive performance in predicting individuals’ stances. We analyze users’ attitudes ahead of two constitutional referendums in Chile in 2020 and 2022. Using two large Twitter datasets, our model achieves improvements of 3.4% in recall and 3.6% in accuracy over the baselines.
Copate: A novel contrastive learning framework for patent embeddings
Authors: Huahang Li, Shuangyin Li, Yuncheng Jiang, Gansen Zhao
Venue: Conference CIKM 2022
Year: 2022

Patents are legal rights issued to inventors to protect their inventions for a certain period and play an important role in today's artificial innovation. With the ever-increasing number of patents each year, an effective and efficient patent management and search system is indispensable for determining how different an invention is from prior works from the vast amount of patent data. However, the chnologists are using now is still based on the strategy of traditional keyword-based Boolean, which requires complex bool expressions. This type of strategy leads to poor performance and costs too much labor power to filter in post-processing. To address these issues, we proposed CoPatE: a novel Contrastive Learning Framework for Patent Embeddings to capture the high-level semantics of the large-scale patents, where a patent semantic compression module learns the informative claims to reduce the computational complexity, and a tags auxiliary learning module is to enhance the semantics of a patent from the structure to learn the high-quality patent embeddings. The CoPatE is trained with the patents from USPTO from 2013 to 2020 and tested by the patents from 2021 with the CPC scheme. The experimental results demonstrate that our model achieves a 17.7% increase at Recall@100 compared to the second-best method on the patent retrieval task and achieves 64.5% at Micro-F1 in the patent classification task.
RETA: A schema-aware, end-to-end solution for instance completion in knowledge graphs
Authors: Paolo Rosso, Dingqi Yang, Natalia Ostapuk, Philippe Cudré-Mauroux
Venue: Conference WWW 2021
Year: 2021

Knowledge Graph (KG) completion has been widely studied to tackle the incompleteness issue (i.e., missing facts) in modern KGs. A fact in a KG is represented as a triplet (h, r, t) linking two entities h and t via a relation r. Existing work mostly consider link prediction to solve this problem, i.e., given two elements of a triplet predicting the missing one, such as (h, r, ?). This task has, however, a strong assumption on the two given elements in a triplet, which have to be correlated, resulting otherwise in meaningless predictions, such as (Marie Curie, headquarters location, ?). In addition, the KG completion problem has also been formulated as a relation prediction task, i.e., when predicting relations r for a given entity h. Without predicting t, this task is however a step away from the ultimate goal of KG completion. Against this background, this paper studies an instance completion task suggesting r-t pairs for a given h, i.e., (h, ?, ?). We propose an end-to-end solution called RETA (as it suggests the Relation and Tail for a given head entity) consisting of two components: a RETA-Filter and RETA-Grader. More precisely, our RETA-Filter first generates candidate r-t pairs for a given h by extracting and leveraging the schema of a KG; our RETA-Grader then evaluates and ranks the candidate r-t pairs considering the plausibility of both the candidate triplet and its corresponding schema using a newly-designed KG embedding model. We evaluate our methods against a sizable collection of state-of-the-art techniques on three real-world KG datasets. Results show that our RETA-Filter generates of high-quality candidate r-t pairs, outperforming the best baseline techniques while reducing by 10.61%-84.75% the candidate size under the same candidate quality guarantees. Moreover, our RETA-Grader also significantly outperforms state-of-the-art link prediction techniques on the instance completion task by 16.25%-65.92% across different datasets.
Half-day tutorial on combating online hate speech: The role of content, networks, psychology, user behavior, etc
Authors: Sarah Masud, Pinkesh Pinkesh, Amitava Das, Manish Gupta, Preslav Nakov, Tanmoy Chakraborty
Venue: Conference WSDM 2022
Year: 2022

While the rise in popularity of social media is seen as a hugely positive development, it is also accompanied by a proliferation of hate speech, which has recently become a major concern. On the one hand, hateful content creates an unsafe environment for certain members of society. On the other hand, manual moderation causes distress to content moderators, and the volume of harmful content is far beyond what human moderators can manually flag and react to. Thus, researchers in machine learning, social computing, and other areas have worked on developing tools to help automate the process. While initially studied as a text classification problem, over time, researchers realized that hate speech is multi-faceted and requires analysis of the role of linguistic expressions, context, and network structure, while using inspiration from psychology and user behavior, among others. With this in mind, we provide a holistic view of what the research community has explored so far, and what we believe are promising future research directions.
A simple yet effective framework for few-shot aspect-based sentiment analysis
Authors: Zengzhi Wang, Qiming Xie, Rui Xia
Venue: Conference SIGIR 2023
Year: 2023

The pre-training and fine-tuning paradigm has become the main-stream framework in the field of Aspect-Based Sentiment Analysis (ABSA). Although it has achieved sound performance in the domains containing enough fine-grained aspect-sentiment annotations, it is still challenging to conduct few-shot ABSA in domains where manual annotations are scarce. In this work, we argue that two kinds of gaps, i.e., domain gap and objective gap, hinder the transfer of knowledge from pre-training language models (PLMs) to ABSA tasks. To address this issue, we introduce a simple yet effective framework called FS-ABSA, which involves domain-adaptive pre-training and text-infilling fine-tuning. We approach the End-to-End ABSA task as a text-infilling problem and perform domain-adaptive pre-training with the text-infilling objective, narrowing the two gaps and consequently facilitating the knowledge transfer. Experiments show that the resulting model achieves more compelling performance than baselines under the few-shot setting while driving the state-of-the-art performance to a new level across datasets under the fully-supervised setting. Moreover, we apply our framework to two non-English low-resource languages to demonstrate its generality and effectiveness.
GNEM: a generic one-to-set neural entity matching framework
Authors: Runjin Chen, Yanyan Shen, Dongxiang Zhang
Venue: Conference WWW 2021
Year: 2021

Entity Matching is a classic research problem in any data analytics pipeline, aiming to identify records referring to the same real-world entity. It plays an important role in data cleansing and integration. Advanced entity matching techniques focus on extracting syntactic or semantic features from record pairs via complex neural architectures or pre-trained language models. However, the performances always suffer from noisy or missing attribute values in the records. We observe that comparing one record with several relevant records in a collective manner allows each pairwise matching decision to be made by borrowing valuable insights from other pairs, which is beneficial to the overall matching performance. In this paper, we propose a generic one-to-set neural framework named GNEM for entity matching. GNEM predicts matching labels between one record and a set of relevant records simultaneously. It constructs a record pair graph with weighted edges and adopts the graph neural network to propagate information among pairs. We further show that GNEM can be interpreted as an extension and generalization of the existing pairwise matching techniques. Extensive experiments on real-world data sets demonstrate that GNEM consistently outperforms the existing pairwise entity matching techniques and achieves up to 8.4% improvement on F1-Score compared with the state-of-the-art neural methods.