Reviewerly/exHarmony huggingface dataset:
How Effective and Robust is Sentence-Level Data Augmentation for Named Entity Recognition?

Data augmentation is a simple but effective way to improve the effectiveness and the robustness of pre-trained models. However, they are difficult to adapt to token-level tasks such as named entity recognition (NER) because of the different semantic granularity and more fine-grained labels. Inspired by some mixup augmentations in computer vision, we proposed three sentence-level data augmentations including CMix, CombiMix, TextMosaic, and adapted them to the NER task. Through empirical experiments on three authoritative datasets (OntoNotes4, CoNLL-03, OntoNotes5), we found that these methods will improve the effectiveness of the models if controlling the number of augmented samples. Strikingly, the results show our approaches can greatly improve the robustness of the pre-trained model even over strong baselines and token-level data augmentations. We achieved state-of-the-art (SOTA) in the robustness evaluation of the CCIR CUP 2021. The code is available at https://github.com/jrmjrm01/SenDA4NER-NLPCC2022 .
Two-Layer Context-Enhanced Representation for Better Chinese Discourse Parsing

As a fundamental task of Natural Language Processing (NLP), discourse parsing has attracted more and more attention in the past decade. Previous studies mainly focus on tree construction, while for the EDU representation, most researchers just conduct a simple flat word-level representation. Structural information within EDU and relationships between EDUs, especially between non-adjacent EDUs, are largely ignored. In this paper, we propose a two-layer enhanced representation approach to better model the context of EDUs. For the bottom layer (i.e., intra-EDU), we use Graph Convolutional Network (GCN) to continuously update the representation of words according to existing dependency paths from the root to the leaves. For the upper layer (i.e., inter-EDU), we use Star-Transformer to connect non-adjacent EDUs by the delay node and thus incorporate global information. Experimental results on the CDTB corpus show that the proposed two-layer context-enhanced representation can contribute much to Chinese discourse parsing in neural architecture.
Short Message Service Spam Detection Using BERT

It is undeniable, fraud on smartphone users is still very widespread today. One of the scams that is quite common and is detrimental to smartphone users is fraud that is spread through information media such as short message services. Many users still lack knowledge which causes them to suffer losses by performing actions they do not understand. Therefore, in this study a method is proposed to detect messages sent via short message services are messages that are not dangerous or vice versa. The dataset used is 1,525 messages consisting of normal, scam, and advertisement messages. The method used in this research is classification using the BERT approach. Based on the simulation results for the classification, the average accuracy obtained is 91.13% with the highest accuracy reaching 91.33%. It can be concluded that this approach provides good value to be used as a reference in detecting incoming messages, as well as convincing smartphone users to make decisions before taking action on information.
An Enhanced New Word Identification Approach Using Bilingual Alignment

Traditional new word detection focused on finding the positional distribution of new words on Chinese text, but rarely on other languages. It was also difficult to obtain semantic information or translations of these new words. This paper proposed NEWBA, an enhanced new word identification algorithm by using bilingual corpus alignment. It indicated that NEWBA performs better than the traditional unsupervised method. In addition, it can obtain bilingual word pairs, which was able to provide us with translations beyond detection. NEWBA can expand the scope of traditional new word detection and therefore obtain more valuable information from bilingual aligned corpora.
Users’ attitude on perceived security of mobile cloud computing: empirical evidence from SME users in China

Purpose The purpose of this paper is to rank the users’ attitude on major components of mobile cloud computing (MCC) security and investigate the degree of impact of these components on MCC security as a whole. Design/methodology/approach Hypotheses were evolved and tested by data collected through an online survey-questionnaire. The survey was administered to 363 users from Chinese organizations. Statistical analysis was carried out and structural equation model was built to validate the interactions. Findings The eight components of MCC security in the order of importance are as follows: mobile device related, user identity related, deployment model related, application-level security issues, data related, virtualization related, network related and service delivery related. The empirical analysis validates that these security issues are having significant impact on perceived security of MCC. Practical implications Constant vigilance on these eight issues and improving the level of user awareness on these issues enhance the overall security. Social implications These issues can be used for designing and developing secured MCC system. Originality/value While several previous research has studied various security factors in the MCC security domain, a consolidated understanding on the different components of MCC security is missing. This empirical research has identified and ranked the major components of MCC security. The degree of impact of each of these components on overall MCC security is identified. This provides a different perspective for managing MCC security by explaining what components are most important.
QuatSE: Spherical Linear Interpolation of Quaternion for Knowledge Graph Embeddings

Knowledge graph embedding aims to learn representations of entities and relations in a knowledge graph. Recently, QuatE has introduced the graph embeddings into the quaternion space. However, there are still challenges in dealing with complex patterns, including 1-N, N-1, and multiple-relations between two entities. Since the learned entity embeddings tend to overlap with each other in the first two cases, and the learned relation embeddings tend to overlap with each other in the last case. To deal with these issues, we propose QuatSE, a novel knowledge embedding model that adjusts graph embeddings via spherical linear interpolation (Slerp) of entities and relations. For a triple (head entity, relation, tail entity), QuatSE calculates Slerp between each entity and its relation, and adds the normalized interpolation to the corresponding entity. The operation avoids the problem of embedding overlap and ensures the information of original entity is not missed. We further compare the effect of interpolation using different normalization strategies ( $$L_1$$ or $$L_2$$ ) for Slerp. Several experiments suggest that QuatSE works well in 1-N, N-1 and multiple-relations pattern. QuatSE outperforms the existing quaternion-based models.
Adversarial Transfer Learning for Named Entity Recognition Based on Multi-Head Attention Mechanism and Feature Fusion

Named entity recognition (NER) is an important task of natural language processing. In most scenarios, there are not sufficient annotated data for NER. The existing methods can only solve the problem of a certain type of the lack of labeled data. To tackle this problem, we present a unified model named Multi-head-attention-Feature-fusion-Adversarial-Transfer-NER (MFAT-NER) to solve the cross-task, cross-lingual and cross-domain NER tasks. The proposed model has the advantages of obtaining better feature representations by word-level and character-level feature fusion and of learning relevant information in different representational subspaces of a sentence by multi-head attention mechanism that can obtain more semantic interaction information. To make the proposed model more generalization, the MFAT-NER model exploits the shared BiLSTM component for extracting the shared feature representations and introduces the adversarial transfer learning method. For the cross-lingual NER, cross-domain NER and cross-task NER, the experimental results show that the proposed unified method outperforms other state-of-the-art methods.
Online Self-boost Learning for Chinese Grammatical Error Correction

Grammatical error correction (GEC) aims to automatically detect and correct grammatical errors in sentences. With the development of deep learning, neural machine translation-based approach becomes the mainstream approach for this task. Recently, Chinese GEC attracts a certain amount of attention. However, Chinese GEC has two main problems that limit model learning: (1) insufficient data; (2) flexible error forms. In this paper, we attempt to address these limitations by proposing a method called online self-boost learning for Chinese GEC. Online self-boost learning enables the model to generate multiple instances with different errors for model’s weaknesses from each original sample within each batch and to learn the new data in time without additional I/O. And taking advantage of the features of the new data, a consistency loss is introduced to drive the model to produce similar distributions for different inputs with the same target. Our method is capable of fully exploiting the potential knowledge of the annotated data. Meanwhile, it allows for the use of unlabeled data to extend to a semi-supervised method. Sufficient experiments and analyses show the effectiveness of our method. Besides, our method achieves a state-of-the-art result on the Chinese benchmark.
TEMPLATE: TempRel Classification Model Trained with Embedded Temporal Relation Knowledge

In recent years, the mainstream Temporal Relation (TempRel) classification methods may not take advantage of the large amount of semantic information contained in golden TempRel labels, which is lost by the traditional discrete one-hot labels. To solve this problem, we propose a new approach that can make full use of golden TempRel label information and make the model perform better. Firstly we build a TempRel Classification (TC) model, which consists of a RoBERTa and a Classifier. Secondly, we establish fine-grained templates to automatically generate sentences to enrich golden TempRel label information and build an Enhanced Data-set. Thirdly we use the Enhanced Data-set to train the Knowledge Encoder, which has the same structure as the TC model, and get embedded knowledge. Finally, we get a TC model Trained with EMbedded temPoral reLATion knowldgE (TEMPLATE) using our designed Cosine balanced MSE loss function. Extensive experimental results show that our approach achieves new state-of-the-art results on TB-Dense and MATRES and outperforms the TC model trained with only traditional cross entropy loss function with up to 5.51% $$F_1$$ on TB-Dense and 2.02% $$F_1$$ on MATRES.
Label Semantic Extension for Chinese Event Extraction

Event extraction (EE) is an essential yet challenging information extraction task, which aims at extracting event structures from unstructured text. Recent work on Chinese event extraction has achieved state-of-the-art performance by modeling events using the pre-trained model. However, the event type information has not been well utilized in existing event extraction methods. To address the issue, we propose the label semantic extension, which selects extension words according to the semantics of event type labels and adds them to the input sequence. Moreover, we propose $$p-n\ ETF$$ values to measure the relationship between words and event types. Experiments on the ACE 2005 corpus show that our proposed method can significantly improve the performance of event extraction.
Adversarial Fine-Grained Fact Graph for Factuality-Oriented Abstractive Summarization

Although neural abstractive summarization models can generate text with significant fluency and coherence, controlling the correctness of facts in generation remains a challenge. Recent work simply classifies factuality into two categories: factual or non-factual, which is coarse-grained and lacks a deep understanding of the linguistic typology of factual errors. In this paper, we present insights into two main and common factual errors (Semantic Frame Errors and Coreference Error) and propose a LASum model to enhance the factuality of abstractive summarization by adversarial fine-grained fact graph. Specifically, we first construct the fine-grained fact graph for the source document and summary to represent facts, respectively, consisting of a dependency channel and coreference channel to address the above two factual errors. And then we enhance the seq2seq model to grasp the internal facts in the fact graph from an intuitive perspective by adversarial learning (GAN). The experimental results show that our model can substantially improve the factuality compared to several strong baselines.
Teaching Text Classification Models Some Common Sense via Q &amp;A Statistics: A Light and Transplantable Approach

Sociolinguistics believes that common sense is the key factor for people to ensure the robustness of language understanding and text classification in complex social environments. However, commonsense facts are often vague and implicit, and hard to be formalized or combined with current text classification techniques. As an opportunity for us, the Q &A community is a socialized platform for people to communicate knowledge, and implicit common sense can be more easily reflected by the distribution of keywords contained in Q &A data. To this end, our key insight in this paper is to obtain implicit common sense by innovatively exploiting statistical keyword distributions from annotated Q &A corpora to present a Statistics-based Label Interactive Model (SLIM) as an enhancement framework for current deep learning text classification models. Specifically, we first draw inspirations from the sociality shared by Q &A community and commonsense knowledge, and propose a light pre-classification strategy. Then, to exploit the valuable semantics of Q &A domain labels, we devise a commonsense attention module to enhance the textual representation. Afterwards, we design a label interactive attention module to make our methods transplantable to any other deep learning models and text classification datasets. Finally, extensive experiments with several deep learning text classification models on three datasets exactly verify the rationality and effectiveness of our methods.
FuncSA: Function Words-Guided Sentiment-Aware Attention for Chinese Sentiment Analysis

Sentiment analysis is an important natural language processing application that empowers many other technologies, including product review analysis and recommendation systems. Knowledge has been proven crucial for providing supervision information and improving performance. However, Chinese function words’ knowledge, especially for degree adverbs, negative adverbs, and conjunctions, which may play an essential role in describing the sentiment polarity, is not well investigated in current Chinese sentiment analysis approaches. In this paper, we propose a Function words-guided Sentiment-aware Attention model (FuncSA) for Chinese sentiment analysis to leverage function words’ knowledge. Specifically, we integrate discrete sentiment lexical information using degree adverbs, negative adverbs, and conjunctions in the Chinese Function Word Usage Knowledge Base(CFKB), and improve self-attention to integrate function words’ knowledge into the model. We implement our approach on several open datasets and show that function words are essential in guiding sentiment identification.
LoCSGN: Logic-Contrast Semantic Graph Network for Machine Reading Comprehension

Logical reasoning plays a very important role in Natural Language Understanding. In recent years, research on logical reasoning has shown a booming trend. Previous works either construct graphs to perform implicit reasoning or augment labeled training data with consistent symbol logical rules. In this paper, we combine the advantages of both graph based implicit reasoning and symbolic logic based consistent transformation in a deep learning framework. In order to make full use of the semantic and logical structures in text, we exploit Abstract Meaning Representation (AMR) to help logical reasoning, which can explicitly provide core semantic knowledge and logical structures. Based on AMR graph extracted, we design two tasks: 1) Construct joint graph and strengthen the interaction between context and option subgraph to predict right choice. 2) Leverage symbolic rules to construct logical consistent and inconsistent graphs to let model identify and differentiate logical structures in different graphs by contrastive learning. Experiments are conducted on two logical reasoning datasets: Reclor and LogiQA. And our method has a significant improvement over the baselines and most previous methods.
Employing Internal and External Knowledge to Factuality-Oriented Abstractive Summarization

Abstractive summarization models based on neural network have successfully generated human-readable and fluent summaries. However, the generated summary often has factual errors: it is inconsistent with the facts included in the source document (internal factual error) or commonsense knowledge (external factual error). To alleviate these two factual errors, we propose a novel Knowledge Aware Summarization model (KASum) that enhances the factuality of the summary by integrating internal and external knowledge simultaneously. First, KASum obtains external knowledge by utilizing the pre-trained model ERNIE combined with Knowledge Graph (KG) to reduce external factual errors. Besides, KASum obtains internal knowledge by extracting the source document’s Semantic Role Information (SRI) to improve internal factuality. Finally, KASum captures the interaction of internal and external knowledge by an interactive attention module to avoid internal and external factual errors further. Experimental results on CNN/DM and XSUM show that KASum significantly improves the factuality of the generated summary compared with strong baseline models.
Blockchain Deployment in the Retail Supply Chain

The research aims to explore blockchain adoption for retail supply chain applications using two theoretical perspectives: supply chain capabilities and business value-added. Blockchain deployment in the retail supply chain helps in resolving some of the supply chain problems such as time delays, risks mitigation, information sharing, and product counterfeiting. In this study, various frameworks are conceptualized to understand, synthesize, and determine the value creation process. The research emphasizes that the benefits of blockchain deployment in the retail supply chain are broad-based and include enhanced trust and transparency, cost reduction, automation, increased processing speed, streamlined process, and disintermediation. The research also provides various illustrations to explain how retailers have successfully deployed blockchain technology in retail supply chains and improved their overall performance. Research also discusses the barriers to blockchain deployment in the retail supply chain and provides practical insights for smooth implementation.
Multi-view Document Clustering with Joint Contrastive Learning

Multi-view document clustering, which learns common representations from multiple views to achieve consistent partition, has emerged lots of increasing work. Though promising performance has been demonstrated in various applications, their view representations are learned with no consideration of achieving a consistent clustering partition. In this paper, we propose a Multi-view document Clustering model with Joint Contrastive learning (MCJC) to address the aforementioned issue. Our model learns the view representations with a joint contrastive learning module by introducing a task-specific objective so that it can effectively achieve consistency both in cluster-wise and feature-wise hidden spaces. Meanwhile, in the clustering module, we collect the view-level cluster agreement and document-level clustering partition to refine the contrastive learning and obtain document assignments. As a result, the proposed model can use a joint contrastive module to learn clustering-friendly representation and through multi-level clustering to achieve better clustering performance. Extensive experiments on real datasets demonstrate that our model achieves state-of-the-art clustering effectiveness.
Automatic Academic Paper Rating Based on Modularized Hierarchical Attention Network

Automatic academic paper rating (AAPR) remains a difficult but useful task to automatically predict whether to accept or reject a paper. Having found more task-specific structure features of academic papers, we present a modularized hierarchical attention network (MHAN) to predict paper quality. MHAN uses a three-level hierarchical attention network to shorten the sequence for each level. In the network, the modularized parameter distinguishes the semantics of functional chapters. And a label-smoothing mechanism is used as a loss function to avoid inappropriate labeling. Compared with MHCNN and plain HAN on an AAPR dataset, MHAN achieves a state-of-the-art accuracy of 65.33%. Ablation experiments show that the proposed methods are effective.
Training Two-Stage Knowledge-Grounded Dialogues with Attention Feedback

Knowledge-grounded retrieval-based dialogue systems have attracted more and more attention. Among them, the two-stage dialogue models which separate the training stage into knowledge retrieving (via a retriever) and response ranking (via a ranker) are proved powerful. However, these approaches require knowledge-grounded dialogues with corresponding hand-annotated knowledge labels. Therefore, in this paper, we propose training two-stage knowledge-grounded dialogues with knowledge attention feedback from the ranker to the retriever. In each training iteration, the ranker provides knowledge attention scores as pseudo supervised feedback for the optimization of retriever. We conduct experiments on two public data sets. The experimental results demonstrate that our proposed method is superior to the existing baselines.
Address Cybersecurity Risks Due to COVID-19 in Small Business Environments

COVID-19 affects people and companies. They are experiencing dramatic changes in work processes, communication processes, and customer management. This whole new paradigm has brought new challenges to organizations, particularly to small and medium enterprises, which have limited budgets and resources to ensure their technological infrastructure. This study aims to explore how these companies have adapted to the challenges posed by COVID-19 in their compliance practices with general data protection regulation (GDPR), auditing, and forensic security analysis. Furthermore, this study analyzes the risks that have been amplified in this period and identifies new vulnerabilities that have arisen from increased dependence on information technology in a period when most organizations have their employees in teleworking.