Reviewerly/exHarmony huggingface dataset:
CCAE: A Corpus of Chinese-Based Asian Englishes

Language models have been foundations in various scenarios of NLP applications, but it has not been well applied in language variety studies, even for the most popular language like English. This paper represents one of the few initial efforts to utilize the NLP technology in the paradigm of World Englishes, specifically in creating a multi-variety corpus for studying Asian Englishes. We present an overview of the CCAE — Corpus of Chinese-based Asian English, a suite of corpora comprising six Chinese-based Asian English varieties. It is based on 340 million tokens in 448 thousand web documents from six regions. The ontology of data would make the corpus a helpful resource with enormous research potential for Asian Englishes (especially for Chinese Englishes for which there has not been a publicly accessible corpus yet so far) and an ideal source for variety-specific language modeling and downstream tasks, thus setting the stage for NLP-based World Englishes studies. And preliminary experiments on this corpus reveal the practical value of CCAE. Finally, we make CCAE available at https://huggingface.co/datasets/CCAE/CCAE-Corpus .
Cross and Self Attention Based Graph Convolutional Network for Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis aims to recognize the sentiment polarity of an aspect in reviews. In general, to analyze the sentiment of an aspect in a sentence, it is essential to capture the dependencies between aspects and the corresponding contexts. Recently, graph neural networks over global dependency structures like dependency trees or self-attention score matrices have been explored for this task. However, these models rely heavily on the quality of information extracted from the global dependency structures. In the meantime, the pairwise correlations between aspects and contexts provide an equally important perspective for sentiment analysis, which is usually ignored in previous works. Motivated by this, we propose a novel approach for aspect-based sentiment analysis by integrating the information extracted from global dependency structures as well as pairwise correlations. To capture the aspect-to-text correlations, we design a CAGCN module based on the cross-attention mechanism. Meanwhile, to effectively exploit the syntactic graph, we design an SAGCN module with the self-attention mechanism to build the overall text-to-text connections. Experimental results on five benchmarks show the effectiveness of our proposed model, producing significantly better results than the baselines.
Neural Knowledge Bank for Pretrained Transformers

The ability of pretrained Transformers to remember factual knowledge is essential but still limited for existing models. Inspired by existing work that regards Feed-Forward Networks (FFNs) in Transformers as key-value memories, we design a Neural Knowledge Bank (NKB) and a knowledge injection strategy to introduce extra factual knowledge for pretrained Transformers. The NKB is in the form of additional knowledgeable memory slots to the FFN and the memory-like architecture makes it highly interpretable and flexible. When injecting extra knowledge with the Salient Span Masking (SSM) pretraining objective, we fix the original pretrained model and train only the NKB. This training strategy makes sure the general language modeling ability of the original pretrained model is not influenced. By mounting the NKB onto T5, we verify its strong ability to store extra factual knowledge based on three closed-book question answering datasets. Also, we prove that mounting the NKB will not degrade the general language modeling ability of T5 through two representative tasks, summarization and machine translation. Further, we analyze the interpretability of the NKB and reveal the meaning of its keys and values in a human-readable way. Finally, we show the flexibility of the NKB by directly modifying its value vectors to update the factual knowledge stored in it. The code is available at https://github.com/Hunter-DDM/nkb .
A Novel Semantic-Enhanced Time-Aware Model for Temporal Knowledge Graph Completion

Researchers have investigated various graph embedding methods to complete Knowledge Graphs (KGs), most of which merely focus on Static KGs (SKGs) without emphasizing the time dependence of triple-formed facts. However, in reality, KGs are dynamic and definitely there is correlations between facts with different timestamps. Due to the sparsity of Temporal KGs (TKGs), SKG’s embedding methods cannot be directly applied to TKGs, which triggers the current discussions about TKG Completion (TKGC) task. And existing TKGC methods universally suffer from two issues: (i) The modeling procedure for temporal information in encoder is usually disjointed or conflict with that in decoder. (ii) Current methods are overwhelmingly dependent on temporal signals for measuring the probability of candidate entity, while ignoring other signals (such as entity’s semantics, etc.,). To overcome these problems, this paper proposes a novel semantic-driven time-aware relational graph neural network model for TKGC task, which consists of a semantic-enhanced encoder and a convolution-based decoder.
FedEAE: Federated Learning Based Privacy-Preserving Event Argument Extraction

Benefiting from Pre-trained Language Model (PLM), Event Argument Extraction (EAE) methods have achieved SOTA performance in general scenarios of Event Extraction (EE). However, with increasing concerns and regulations on data privacy, aggregating distributed data among different institutions in some privacy-sensitive territories (e.g., medical record analysis, financial statement analysis, etc.) becomes very difficult, and it’s hard to train an accurate EAE model with limited local data. Federated Learning (FL) provides promising methods for a large number of clients to collaboratively learn a shared global model without the need to exchange privacy-sensitive data. Therefore, we propose a privacy-preserving EAE method named FedEAE based on FL to solve the current difficulties. To better adapt to federated scenarios, we design a dataset named FedACE generated from the ACE2005 dataset under IID and Non-IID for our experiments. Extensive experiments show that FedEAE achieves promising performance compared to existing baselines, thus validates the effectiveness of our method. To the best of our knowledge, FedEAE is the first to apply FL in the EAE task.
Enhancing Semantic Consistency in Linguistic Steganography via Denosing Auto-Encoder and Semantic-Constrained Huffman Coding

Linguistic steganography is a useful technique to hide secret messages within a normal cover text, playing a crucial role in the field of data protection. Compared to the data encryption techniques, steganography can make the security data transmission process more imperceptible because the outputted stego (steganography) texts are not garbled codes but look like normal texts. Consequently, the essential of linguistic steganography is to improve the imperceptibility of the outputted stego texts. Although prior works can already generate fluent stego texts, how to ensure the semantics of stego text be natural and reasonable in human cognition is still a challenging problem. To alleviate this issue, this work proposes a novel Semantic-Preserved Linguistic Steganography Auto-Encoder (SPLS-AutoEncoder) to improve imperceptibility by enhancing semantic consistency. SPLS-AutoEncoder first minimizes the possible distortion when embedding the secret message into the cover text by using the denoising auto-encoder BART as the backbone model. Then, we propose a novel Dynamic Semantic-Constrained Huffman Coding, which uses a dynamic context information embedding and a global topic embedding to ensure semantic consistency between the cover text and stego text. Experimental results on two Chinese datasets show that our method has excellent performance compared with the previous methods. The datasets and code are released at https://github.com/Y-NLP/LinguisticSteganography/tree/main/NLPCC2023_SPLS-AutoEncoder .
OnMKD: An Online Mutual Knowledge Distillation Framework for Passage Retrieval

Dense passage retriever recalls a set of relevant passages from a large corpus according to a natural language question. The dual-encoder architecture is prevalent in dense passage retrievers, which is based on large-scale pre-trained language models (PLMs). However, existing PLMs usually have thick structures and bulky parameters, resulting in large memory and time consumption. To overcome the limitation of PLMs, in this paper we apply online distillation to passage retrieval and propose an Online Mutual Knowledge Distillation framework (OnMKD). Specifically, we obtain a lightweight retriever by simultaneously updating two peer networks with the same dual-encoder structure and different initial parameters, named Online Mutual Knowledge Refinement. To further interact with the latent knowledge of intermediate layers, we utilize a novel cross-wise contrastive loss to alternate the representation of questions and passages. Experimental results indicate that our framework outperforms other small baselines with the same number of layers on multiple QA benchmarks. Compared to the heavy PLMs, OnMKD significantly accelerates the inference process and reduces storage requirements with only a slight sacrifice in performance.
Faster and More Robust Low-Resource Nearest Neighbor Machine Translation

Transformer-based neural machine translation (NMT) models have achieved performance close to human-level on some languages, but still suffer from poor interpretability and scalability of the models. Many advanced studies enhance the model’s translation ability by building external memory modules and utilizing retrieval operations, however, it suffers from poor robustness and low decoding efficiency while improving the model performance, especially for low-resource translation tasks. In this paper, we propose a confidence-based gating mechanism to optimize the decoding efficiency by building a sub-network to determine the confidence of the model’s own translation capability and then decide whether the current translation needs to be retrieved from the memory module. By reducing the number of retrievals to improve the model’s translation speed without degrading the translation quality as much as possible. In addition, we use a nonparametric dynamic Monte Carlo-based algorithm to fuse retrieval probabilities and model predictions to improve the generalization and robustness of the model. Extensive experiments on different datasets demonstrate the effectiveness of our method.
Dual-Prompting Interaction with Entity Representation Enhancement for Event Argument Extraction

Event argument extraction (EAE) aims to recognize arguments that are entities involved in events and their roles. Previous prompt-based methods focus on designing appropriate prompt template for events, while neglecting to explicitly model entities in the input, resulting in models that are not well aware of arguments. In this paper, we propose a novel Dual-Prompting Interaction approach with Entity Representation Enhancement (DPIERE) to explicitly model entities for both sentence-level and document-level EAE. 1) We design appropriate event and input prompt templates to model argument roles and entities in the input respectively so that pre-trained language model (PLM) can better capture the interaction between them. 2) We introduce a simple but effective entity representation enhancement method with adaptive selection to infuse entity knowledge into PLM. Finally, Argument span prediction is employed to predict the start and end word among the input for each role. In addition, DPIERE devises position marks in event prompt template to distinguish multiple occurrences of the same argument role. Comprehensive experimental results on three benchmarks show the effectiveness of our proposed approach.
ECA-TFUnet: A U-shaped CNN-Transformer network with efficient channel attention for organ segmentation in anatomical sectional images of canines

Automated organ segmentation in anatomical sectional images of canines is crucial for clinical applications and the study of sectional anatomy. The manual delineation of organ boundaries by experts is a time-consuming and laborious task. However, semi-automatic segmentation methods have shown low segmentation accuracy. Deep learning-based CNN models lack the ability to establish long-range dependencies, leading to limited segmentation performance. Although Transformer-based models excel at establishing long-range dependencies, they face a limitation in capturing local detail information. To address these challenges, we propose a novel ECA-TFUnet model for organ segmentation in anatomical sectional images of canines. ECA-TFUnet model is a U-shaped CNN-Transformer network with Efficient Channel Attention, which fully combines the strengths of the Unet network and Transformer block. Specifically, The U-Net network is excellent at capturing detailed local information. The Transformer block is equipped in the first skip connection layer of the Unet network to effectively learn the global dependencies of different regions, which improves the representation ability of the model. Additionally, the Efficient Channel Attention Block is introduced to the Unet network to focus on more important channel information, further improving the robustness of the model. Furthermore, the mixed loss strategy is incorporated to alleviate the problem of class imbalance. Experimental results showed that the ECA-TFUnet model yielded 92.63% IoU, outperforming 11 state-of-the-art methods. To comprehensively evaluate the model performance, we also conducted experiments on a public dataset, which achieved 87.93% IoU, still superior to 11 state-of-the-art methods. Finally, we explored the use of a transfer learning strategy to provide good initialization parameters for the ECA-TFUnet model. We demonstrated that the ECA-TFUnet model exhibits superior segmentation performance on anatomical sectional images of canines, which has the potential for application in medical clinical diagnosis.
FGCS: A Fine-Grained Scientific Information Extraction Dataset in Computer Science Domain

As scientific communities grow and evolve, more and more papers are published, especially in computer science field (CS). It is important to organize scientific information into structured knowledge bases extracted from a large corpus of CS papers, which usually requires Information Extraction (IE) about scientific entities and their relationships. In order to construct high-quality structured scientific knowledge bases by supervised learning way, as far as we know, in computer science field, there have been several handcrafted annotated entity-relation datasets like SciERC and SciREX, which are used to train supervised extracted algorithms. However, almost all these datasets ignore the annotation of following fine-grained named entities: nested entities, discontinuous entities and minimal independent semantics entities. To solve this problem, this paper will present a novel Fine-Grained entity-relation Extraction dataset in Computer Science field (FGCS), which contains rich fine-grained entities and their relationships. The proposed dataset includes 1,948 sentences of 6 entity types with up to 7 layers of nesting and 5 relation types. Extensive experiments show that the proposed dataset is a good benchmark for measuring an information extraction model’s ability of recognizing fine-grained entities and their relations. Our dataset is publicly available at https://github.com/broken-dream/FGCS .
A Novel POS-Guided Data Augmentation Method for Sign Language Gloss Translation

Due to its profound significance for the hearing-impaired community, there has been an abundance of recent research focused on Sign Language Translation (SLT), which is often decomposed into video-to-gloss recognition (S2G) and gloss-to-text (G2T) translation. Here, a gloss represents a sequence of transcribed spoken language words arranged in the order they are signed. In this paper, our emphasis lies in G2T, a crucial aspect of sign language translation. However, G2T encounters a scarcity of data, leading us to approach it as a low-resource neural machine translation (NMT) problem. Nevertheless, in contrast to traditional low-resource NMT, gloss-text pairs exhibit a greater lexical overlap but lower syntactic overlap. Hence, leveraging this characteristic, we utilize part-of-speech (POS) distributions obtained from numerous monolingual spoken language text to generate high-quality pseudo glosses. By simultaneously training numerous pseudo gloss-text sentence pairs alongside authentic data, we have significantly improved the translation performance of Chinese Sign Language (+3.10 BLEU), German Sign Language (+1.10 BLEU), and American Sign Language (+5.06 BLEU) in G2T translation, respectively.
Dual Hierarchical Contrastive Learning for Multi-level Implicit Discourse Relation Recognition

Implicit discourse relation recognition (IDRR) is a challenging but vital task in discourse analysis, which aims at classifying logical relations between arguments. Previous work infuses ACCL takes external knowledge or label semantics to alleviate data scarcity, which either brings noise or underutilizes semantic information contained in label embedding. Meanwhile, it is difficult to model label hierarchy. In this paper, we make full use of label embedding as positives and negatives for our dual hierarchical contrastive learning framework, which contains two parts: 1) hierarchical label contrastive loss (HLCL), which promotes fine-grained labels to be more similar to correlative medium-grained labels than related coarse-grained labels. 2) arguments and connectives contrastive loss (ACCL), which makes arguments aggregate around correlative fine-grained labels. The two modules interact with each other, making the similarity between arguments and correlative fine-grained labels are higher than that with related coarse-grained labels. In this process, the multi-level label semantics are integrated to arguments, which provides guidance for classification. Experimental results show that our method achieves competitive performance against state-of-the-art system.
Improving Event Representation for Script Event Prediction via Data Augmentation and Integration

Script event prediction aims to predict the most likely following events, given the historical events in the script. This task requires the capability to learn more information between events. Most previous methods mainly focused on the current local information, while ignoring more inner semantic features of events. In this work, we propose a novel framework, called ECer, which can obtain more comprehensive event information by utilizing data augmentation and information integration. We first employ rectified linear attention to connect the initial event representation at the argument level. Then, to learn richer semantic information, data augmentation is further applied to expand data and introduce external knowledge. Furthermore, the initial representation and the features of augmented data were mixed by Mixup. Finally, an attention module is applied to the context event chain to integrate context events concerning the current candidate event dynamically. The experimental results on the widely-used New York Times corpus demonstrate the effectiveness and superiority of the proposed model.
Towards Effective Ancient Chinese Translation: Dataset, Model, and Evaluation

Interpreting ancient Chinese has been the key to comprehending vast Chinese literature, tradition, and civilization. In this paper, we propose Erya for ancient Chinese translation. From a dataset perspective, we collect, clean, and classify ancient Chinese materials from various sources, forming the most extensive ancient Chinese resource to date. From a model perspective, we devise Erya training method oriented towards ancient Chinese. We design two jointly-working tasks: disyllabic aligned substitution (DAS) and dual masked language model (DMLM). From an evaluation perspective, we build a benchmark to judge ancient Chinese translation quality in different scenarios and evaluate the ancient Chinese translation capacities of various existing models. Our model exhibits remarkable zero-shot performance across five domains, with over +12.0 BLEU against GPT-3.5 models and better human evaluation results than ERNIE Bot. Subsequent fine-tuning further shows the superior transfer capability of Erya model with +6.2 BLEU gain.
Neural News Recommendation with Interactive News Encoding and Mixed User Encoding

Personalized news recommendation is of great significance in helping users find news of interest. In the task of news recommendation, the main point is the representation learning of news and users. Current news encoding methods tend to overlook the interactive information mining that occurs among the title, content, and entities, ultimately resulting in a weakened ability to accurately model news. In former user modeling methods, the approach often focused on linear modeling of user features, which lacked structural and serialization feature mining for users, ultimately resulting in insufficient user encoding capabilities. To improve the performance of news recommendation, we propose a neural news recommendation model with Interactive News Encoding (INE) and Mixed User Encoding (MUE). In the INE, the cross-encoding module is utilized to encode the interactions among the title, content and entity, which enables a deeper semantic analysis of their interactive information. In the MUE, graph convolutional networks (GCN) is used to learn the structural features of user preferences, and the multi-head self-attention networks are used to capture serialization features of user browsing records. We carried out comparative experiments and compatibility experiments on the MIND dataset. According to the results, the INE and MUE outperform baseline models, indicating that the INE and MUE are effective and universal.
Extract Then Adjust: A Two-Stage Approach for Automatic Term Extraction

Automatic Term Extraction (ATE) is a fundamental natural language processing task that extracts relevant terms from domain-specific texts. Existing transformer-based approaches have indeed achieved impressive improvement. However, we observe that even state-of-the-art (SOTA) extractors suffer from boundary errors, which are distinguished by incorrect start or end positions of a candidate term. The minor differences between candidate terms and ground-truth leads to a noticeable performance decline. To alleviate the boundary errors, we propose a two-stage extraction approach. First, we design a span-based extractor to provide high-quality candidate terms. Subsequently, we adjust the boundaries of these candidate terms to enhance performance. Experiment results show that our approach effectively identifies and corrects boundary errors in candidate terms, thereby exceeding the performance of previous state-of-the-art models.