Reviewerly/exHarmony huggingface dataset:
GCL-KGE: Graph Contrastive Learning for Knowledge Graph Embedding

Knowledge graph embedding models characterize entities and relations in structured knowledge graphs as vectors, which is essential for many downstream tasks. Some studies show that knowledge graph embedding models based on graph neural networks can exploit higher-order neighborhood information and generate meaningful representations. However, most models suffer from interference from distant neighborhood noise information. To address the challenge, we propose a graph contrastive learning knowledge graph embedding (GCL-KGE)model to enhance the representation of entities. Specifically, we use the graph attention network to aggregate multi-order neighbor information optimizing the pre-trained entity representation. To avoid the inclusion of redundant information in the graph attention network, we combine contrastive learning to provide auxiliary supervised signals. A new method of constructing positive instances in contrastive learning makes the entity representation in the hidden layer produce a marked effect in this paper. We use a triple scoring function to evaluate representation on link prediction. The experimental results on four datasets show that our model can alleviate the interactive noise and achieve better results than baseline models.
MSK-Net: Multi-source Knowledge Base Enhanced Networks for Script Event Prediction

Script event prediction (SEP) aims to choose a correct subsequent event from a candidate list, according to a chain of ordered context events. It is easy for human but difficult for machine to perform such event reasoning. The reason is that human have relevant commonsense knowledge. If we supplement this knowledge from external knowledge bases, machine may be able to improve the reasoning ability. To this end, we introduce a novel approach, named MSK-Net, which consists of Question Encoder, Knowledge Searcher, Knowledge Encoder and Result Predictor. As far as we know, this is the first model utilizing multi-source knowledge to solve SEP problem. Specifically, first we use Question Encoder to encode the question, including candidate event to be judged and context events, focusing on intra-event contextualization and inter-event order information modelling. Then, we use Knowledge Searcher to retrieve relevant knowledge from multi-source knowledge bases (such as ASER and ATOMIC). Third, Knowledge Encoder is used to encode the knowledge retrieved in the second step. Last, Result Predictor gives the final prediction. Experiments on the widely-used multiple choice narrative cloze (MCNC) task demonstrate our approach achieves state-of-the-art performance compared to other methods. Also, it is worth noting that MSK-Net without external knowledge is still very competitive.
Efficient Anomaly Detection in Property Graphs

Property graphs are becoming increasingly popular for modeling entities, their relationships, and properties. Due to the computational complexity, users are seldom to build complex user-defined integrity constraints; worse, the systems often do not have the capabilities of defining complex integrity constraints. For these reasons, violation of the implicit integrity constraints widely exists and leads to various data quality issues in property graphs. In this paper, we aim to automatically extract abnormal graph patterns and efficiently mine all matches in large property graphs to the abnormal patterns that are taken as anomalies. For this purpose, we first propose a new concept namely CGPs(Conditional Graph Patterns). CGPs have the capability of modeling anomalies in the property graph by capturing both abnormal graph patterns and the attribute (i.e., property) constraints. All matches to any abnormal CGP are taken as anomalies. To mine abnormal CGPs and their matches automatically and efficiently, we then propose an efficient parallel approach called ACGPMiner (Abnormal Conditional Graph Pattern Miner). ACGPMiner follows the generation-and-validation paradigm and does the anomaly detection level by level. At each level i, we generate CGPs with i edges, validate whether CGPs are abnormal, and mine all matches to any abnormal CGPs. Further, we propose two optimizations, pre-search pruning to reduce the search space of match enumerations and a two-stage strategy for balancing the workload in distributed computing settings. Using real-life graphs, we experimentally show that our approach is feasible for anomaly detection in large property graphs.
Video Retrieval with Tree-Based Video Segmentation

Text-to-video retrieval aims to find relevant videos from text queries. The recently introduced Contrastive Language Image Pretraining (CLIP), a pretrained vision-language model trained on large-scale image and caption pairs, has been extensively used in the literature. Existing studies have focused on directly applying CLIP to learn the temporal dependency. While leveraging the dynamics of the video intuitively sounds reasonable, learning temporal dynamics has demonstrated no advantage or only small improvements. When temporal dynamics are not incorporated, most studies focus on constructing representative images from a video. However, we found these images tend to be noisy, degrading the performance of text-to-video task. This observation is the intuition for designing the proposed model, we introduce a novel tree-based frame division method to focus on the most relevant image for learning.
Wukong-CMNER: A Large-Scale Chinese Multimodal NER Dataset with Images Modality

So far, Multimodal Named Entity Recognition (MNER) has been performed almost exclusively on English corpora. Chinese phrases are not naturally segmented, making Chinese NER more challenging; nonetheless, Chinese MNER needs to be paid more attention. Thus, we first construct Wukong-CMNER, a multimodal NER dataset for the Chinese corpus that includes images and text. There are 55,423 annotated image-text pairs in our corpus. Based on this dataset, we propose a lexicon-based prompting visual clue extraction (LPE) module to capture certain entity-related visual clues from the image. We further introduce a novel cross-modal alignment (CA) module to make the representations of the two modalities more consistent through contrastive learning. Through extensive experiments, we observe that: (1) Discernible performance boosts as we move from unimodal to multimodal, verifying the necessity of integrating visual clues into Chinese NER. (2) Cross-modal alignment module further improves the performance of the model. (3) Our two modules decouple from the subsequent predicting process, which enables a plug-and-play framework to enhance Chinese NER models for Chinese MNER task. LPE and CA achieve state-of-the-art (SOTA) results on Wukong-CMNER when combined with W2NER [11], demonstrating its effectiveness.
HAEP: Heterogeneous Environment Aware Edge Partitioning for Power-Law Graphs

Graph partitioning is an important preprocessing step for distributed processing of large-scale graph data. By balancing workloads and reducing communication costs among nodes, graph partitioning methods improve the efficiency of homogeneous clusters for processing power-law graphs. However, a real cluster usually consists of heterogeneous nodes, each with different computing and communication ability. Nodes handle the same workload with different time cost, and the slowest node is the bottleneck. Therefore, a Heterogeneous environment Aware Edge Partitioning method (HAEP) is proposed to balance graph processing time by skewing the workload. HAEP can adapt to the challenge of unbalanced performance among nodes. First, a k-time balanced graph partitioning problem is defined to balance the expected time cost of graph processing in heterogeneous environments. Then, a neighborhood heuristic expansion is performed according to the node performance, minimizing the communication time among nodes and assigning an appropriate workload for each node. Further, a distributed method of HAEP, DHAEP, is proposed to improve the efficiency of graph partitioning. The performance evaluation shows that HAEP and DHAEP can improve graph processing efficiency by up to 41 $$\%$$ compared to state-of-the-art partitioning methods, and the graph partition time of DHAEP is 15 $$\%$$ of HAEP.
GRMI: Graph Representation Learning of Multimodal Data with Incompleteness

Multimodal data can provide supplementary information of the subjects, which is of great potential for exploring the data-driven insights in various application scenarios. A large amount of researches focus on modal fusion to deriving quality representations of multimodal data. However, missing modality is a common issue, i.e. a sample may not contain full modalities, bringing difficulties to apply existing modal fusion methods on the incomplete multimodal data. In this paper, we present GRMI, a graph-based framework for representation learning of multimodal data with incompleteness. GRMI constructs a bipartite graph for multimodal data, where samples and modalities are viewed as two types of nodes, and the observed modality values as edges. GRMI leverages Graph Neural Network (GNN) to derive edge embeddings and sample node embeddings on the graph, which can be respectively used for missing modality imputation and modal fusion. A self-supervised strategy is utilized to pretrain the GNN by fully exploiting the multimodal data. Extensive experiment results show the superiority of the proposed framework over existing state-ofthe-art methods for both modality imputation task and modal fusion task.(The source code has been anonymously uploaded to https://github.com/GRMI2022/GRMI ).
A Unified Visual Prompt Tuning Framework with Mixture-of-Experts for Multimodal Information Extraction

Recently, multimodal information extraction has gained increasing attention in social media understanding, as it helps to accomplish the task of information extraction by adding images as auxiliary information to solve the ambiguity problem caused by insufficient semantic information in short texts. Despite their success, current methods do not take full advantage of the information provided by the diverse representations of images. To address this problem, we propose a novel unified visual prompt tuning framework with Mixture-of-Experts to fuse different types of image representations for multimodal information extraction. Extensive experiments conducted on two different multimodal information extraction tasks demonstrate the effectiveness of our method. The source code can be found at https://github.com/xubodhu/VisualPT-MoE .
Mining Top-k Frequent Patterns over Streaming Graphs

Mining top-k frequent patterns is an important operation on graphs, which is defined as finding k interesting subgraphs with the highest frequency. Most existing work assumes a static graph. However, graphs are dynamic in nature, which is described as streaming graphs. Mining top-k frequent patterns in streaming graphs is challenging due to the streaming nature of the input and the exponential time complexity of the problem. A naive solution is to calculate approximations of the frequent patterns in the streaming graph and then find the top-k answers, which is a memory- and time-consuming method. In this paper, we design a novel auxiliary data structure, FPC, to detect valid subgraph patterns and their frequency in real-time. We first convert each newly produced subgraph into a sequence and then map it into corresponding tracks in FPC based on hash functions. We theoretically prove that FPC can provide unbiased estimation and then give an error bound of our algorithm. In addition, we propose a vertical hashing and candidate buckets sampling technique to further improve FPC with higher space utilization and higher accuracy. Extensive experiments confirm that our approach generates high-quality results compared to the baseline method.
Optimizing Empathetic Response by Generating and Integrating Emotion Feedback and Topic Discussion

Expressing empathy is a trait in human daily conversation, in which people are willing to give responses containing appropriate emotions and topics on the basis of understanding the interlocutor's situation. However, empathetic dialogue models trained by data-driven training method tend to generate general responses, which are usually monotonous and difficult to infuse emotions and topics concurrently. To solve this issue, we propose a novel model that generates two sub-responses, namely, emotion feedback and topic discussion, then integrates them to optimize empathetic responses. Specifically, in the sub-response generation stage, we introduce emotion lexicon and commonsense knowledge to make sub-responses focus on emotional words and topic-related words respectively, which drives the sub-responses to be contextually related from different perspectives. Afterward, we utilize cross attention to integrate the global information to optimize the final response. Our model is trained on the pre-trained language model BART. Experimental results show that our method can generate responses involving emotion and topic well, and compared with existing methods, empathy and relevance are improved. Our code is available at https://github.com/outsider-lj/edsgi_bart .
Semi-supervised Learning for Fine-Grained Entity Typing with Mixed Label Smoothing and Pseudo Labeling

Distant supervision (DS) has been proposed to automatically annotate data and achieved significant success in fine-grained entity typing(FET). Despite its efficiency, distant supervision often suffers from the noisy labeling problem. To solve the noisy labeling problem, existing approaches assume the existence of "clean" and "noisy" sets in the training data and use different types of methods to utilize them. However, they still suffer from the confirmation bias problem in the "noisy" set and the false positive problem in the "clean" set. To address these issues, we propose a novel semi-supervised learning method with mixed label smoothing and pseudo labeling for distantly supervised fine-grained entity typing. Specifically, to solve the false positive problem on the "clean" set, we propose a mixed label smoothing method to smooth the labels of the "clean" set to train the FET model. To solve the confirmation bias problem on the "noisy" set, we do not consider the labels in the "noisy" set and use a pseudo labeling technique to deal with the "noisy" set. Extensive experiments conducted on three widely used FET datasets show the effectiveness of our proposed approach. The source code is publicly available at https://github.com/xubodhu/NFETC-SSL .
FairHELP: Fairness-Aware Heterogeneous Information Network Embedding for Link Prediction

Heterogeneous information networks (HINs) are ubiquitous in real-world social systems. To effectively learn representations of HINs, Graph Neural Networks (GNNs) have been widely studied as a powerful tool. Nevertheless, there is growing concern that GNNs are prone to make biased predictions in critical decision-making scenarios such as link prediction and social recommendation. Despite recent progress on fair graph learning, few attempts have been made toward promoting fairness in HIN embedding models. In this paper, we study the problem of mitigating link prediction bias in HINs. First, we formalize the definition of fairness in link prediction in HINs, and design fairness measures for the link prediction task. Second, we propose a flexible and model-agnostic debiasing framework named FairHELP for learning fair embeddings in HINs. Third, we conduct extensive experiments on three real-world datasets. The results validate the effectiveness of the proposed fairness measures and the FairHELP framework in achieving fair and accurate link prediction results.
A Prompt-Based Representation Individual Enhancement Method for Chinese Idiom Reading Comprehension

Chinese idiom is a distinctive language phenomenon, which usually consists of four Chinese characters and expresses a non-compositional and metaphorical meaning. Therefore, Chinese idioms pose unique challenges for Chinese machine reading comprehension. To address this issue, researchers proposed a Chinese idiom cloze task and a large-scale Chinese idioms dataset ChID. Existing methods have proposed a number of models and achieved reasonable performance on ChID. However, they fall short of fully exploring the precise representations and distinctions of the meanings of idioms, especially idioms with similar meanings. In this paper, we propose a prompt-based representation individual enhancement method (PRIEM). This method fuses the context-specific representation and the generic definition representation of the idioms, and uses the prompt method to guide the model in learning the metaphorical meanings of idioms purposefully. To further improve the distinction representations of idioms with similar meanings, PRIEM adopts a method of idiom representation mapping and decomposing based on orthogonal projection to obtain the common and individual representations of idioms respectively. Experimental results on ChID show that our model outperforms state-of-the-art models.
An Adaptive Video Clip Sampling Approach for Enhancing Query-Based Moment Retrieval in Videos

Query-based moment retrieval aims to localize the most relevant moment in an untrimmed video according to the given natural language query. Existing retrieval models require the same length for easy training and use. Therefore, videos with different lengths are pre-processed using the fixed sampling method. As a result, the longer the video, the more video clips are lost, thus affecting the accuracy of retrieval. We observed the fixed sampling method causes two accuracy issues, including missing clips and sparse clips. In this paper, we propose an adaptive video clip sampling method including resampling missing clips and enhancing sparse sampled clips to increase the retrieval accuracy. Resampling missing clips is used to address situations in which annotated clips are completely lost during fixed sampling. Enhancing sparse sampled clips aims to prevent the clips containing the same semantics from being too sparse. Our approach first obtains multiple video features through the adaptive sampling methods based on the backbone networks. Then we propose a consistency loss maintenance method to learn the semantics of adaptive sampled features. The extensive experiments on three real datasets demonstrate the effectiveness of our proposed method, especially for long videos.
Meta-Path Based Social Relation Reasoning in a Deep and Robust Way

Social relation reasoning in heterogeneous social networks (HSNs) should not only infer whether people are connected, but also why they know each other and what their relationship is. This task is challenging because the heterogeneous social network is non-trivial to leverage and the collected data is always noisy and incomplete, which leads that the existing rule-based and representation methods cannot efficiently and accurately reason social relations. To reason social relations in noisy and incomplete HSNs, we shift from simply inferring meta-path based social relations by rule-based methods to reasoning by social relation learning. Then, we present an inductive model named Heterogeneous Graph Variational AutoEncoders (HGVAE) towards robust social relation learning on noisy and incomplete HSN data. In HGVAE, the well-designed heterogeneous graph encoder, multi-signal decoder, and variational inference mechanism bring prominent robustness and significant performance improvement. Extensive experiments are conducted to compare our methods against a set of the most representative baseline methods. The efficiency analysis, robust analysis and ablation studies confirm the value of our proposed approach for modelling meta-path based social relations buried in various HSNs.
A Two-Stage Label Rectification Framework for Noisy Event Extraction

Event extraction aims to identify event triggers and corresponding arguments. Existing methods are mainly devoted to well-designed deep neural networks based on the assumption that training data are high-quality. However, noisy labels are unavoidable in real-world scenarios, which is challenging for current event extraction applications. This paper proposes a novel Two-stage label rectification framework (Tolar) to tackle this problem from explicit and latent aspects. In the first stage, an event schema mapping module is designed to rectify the explicit label inconsistency. Then a self-adaptive iteration module addresses the latent semantic noise in the second stage. In order to cope with extremely noisy labels, we further design a Cooperative Global Pointer Network (CoGPN) to train two global pointer networks concurrently and let them filter possibly noisy labels mutually. Extensive experiments on ACE 2005 and MAVEN with synthetic noise demonstrate that our framework Tolar effectively enhances event extraction methods, and our CoGPN achieves state-of-the-art performance in extremely noisy settings.
CoDE: Contrastive Learning Method for Document-Level Event Factuality Identification

Document-level event factuality identification (DEFI) aims to assess the veracity degree to which an event mentioned in a document has happened, which is crucial and fundamental for many downstream tasks of Natural Language Processing (NLP). Thus far, studies on DEFI typically regard it as a supervised classification task relying on annotated information, suffering from data scarcity since there is only one existing corpus (DLEF) consisting of two annotated sub-corpora based on news reports. The uneven distribution and data scarcity of DLEF and the existing annotation-relied methods limit the research progress on the DEFI task. To tackle this issue, we introduce a two-stage data augmentation strategy from text to graph via contrastive learning for DEFI (CoDE), which modifies a document at lexical, sentence, and document levels differently. Experiments on two widely used datasets show that our proposed model outperforms the state-of-the-art model, which demonstrates the effectiveness of our method for DEFI.
Subgraph Reconstruction via Reversible Subgraph Embedding

Reconstructing a subgraph through an embedding is very useful for many subgraph-level tasks, e.g., subgraph matching and minimum Steiner tree problem. To support subgraph reconstruction, a naive approach is materializing subgraph embeddings for all possible candidate subgraphs in advance, which is impractical since the subgraphs are exponential to the size of the input graph. Therefore, it is desired to devise a subgraph embedding based on which the subgraph can be reconstructed. To the end, we develop a novel reversible subgraph embedding in this paper. By importing the compressed sensing theory into learning node embeddings, we design a reversible read-out operation such that the aggregation vector can be recovered according to the subgraph embedding, where the aggregation vector acts as a bridge between the adjacency matrix and subgraph embedding. To reconstruct the structure of the subgraph from the decoded aggregation vector, we present a bijective rule by applying a simple transformation between binary number and decimal number with a scale operation. We conduct extensive experiments over real graphs to evaluate the proposed subgraph embedding. Experimental results demonstrate that our proposed method greatly and consistently outperforms the baselines in three tasks.