Reviewerly/exHarmony huggingface dataset:
A New Adaptive Indexing for Real-Time Web Search

Adaptive indexing is an alternative to the self-tuning methods. It is especially useful in the scenario of unpredictable workload, and there is no idle time to invest in index creation. The authors present their ongoing work on a new realistic adaptive indexing that transforms the previous data crawling offline approach to a data-driven online approach. The proposed approach consists of three tasks: topic prediction, resource selection, and results combination and ranking. They work simultaneously to retrieve highly relevant results to the user's query in real time. To make the index highly refreshed and up-to-date, they collected data from highly prominent resources (e.g., Facebook, Twitter, Wikipedia, etc.). The empirical results showed that the proposed model is better than the traditional models that work offline and spend hours or days for building the index in different periods. In addition, the experiments showed that the training results are highly relevant for adhoc and diversity tasks.
Topic-Features for Dialogue Summarization

Texts such as news reports and academic papers come from one single speaker and are well-structured. However, dialogues often come from two or more speakers exchanging information. In this case, the topic or intention may change in a dialogue, and the key information is often scattered in utterances of different speakers, which brings challenges to the abstractive dialogue summarization. It is difficult to apply the traditional topic modeling approaches because of too much noise and the inherent characteristics of dialogue. In order to effectively model the entire dialogue and capture various topic information, this paper proposes a topic-feature approach based on neural topic model, including word-level embedding and dialogue-level representation. Experimental results on the largest dialogue summarization corpus SAMSum show that the proposed approach can significantly improve over competitive baselines. In addition, we also conduct experiments on other datasets from different domains to verify the effectiveness and generality of our proposed approach.
PGBERT: Phonology and Glyph Enhanced Pre-training for Chinese Spelling Correction

Chinese Spelling Correction (CSC) is a challenging task that requires the ability to model the language and capture the implicit pattern of spelling error generation. In this paper, we propose PGBERT as Phonology and Glyph Enhanced Pre-training for CSC. For phonology, PGBERT uses Bi-GRU to encode single Chinese character’s Pinyin sequence as phonology embedding. For glyph, we introduce Ideographic Description Sequence (IDS) to decompose Chinese character into binary tree of basic strokes, and then an encoder based on gated units is utilized to encode the glyph tree structure recursively. At each layer of original model, PGBERT extends extra channels for phonology and glyph encoding respectively, then performs a multi-channel fusion function and a residual connection to yield an output for each channel. Empirical analysis shows PGBERT is a powerful method for CSC and achieves state-of-the-art performance on widely-used benchmarks.
Contrastive Learning for Robust Neural Machine Translation with ASR Errors

Neural machine translation (NMT) models have recently achieved excellent performance when translating non-noised sentences. Unfortunately, they are also very brittle and easily falter when fed with noisy sentences, i.e., from automatic speech recognition (ASR) output. Due to the lack of Chinese-to-English translation test set with natural Chinese-side ASR output, related studies artificially add noise into Chinese sentences to evaluation translation performance. In this paper we eliminate such strong limitation and manually construct natural ASR output on popular Chinese-to-English NIST translation test dataset, NISTasr which consists of 680 documents with 7,688 sentences. To train an NMT model being robust to ASR output, we take contrastive learning framework to close the gap among representations of original input and its perturbed counterpart. Experimental results on NIST Chinese-to-English translation show that our approach significantly improves the translation performance of ASR output while having negligible effect on the translation performance of non-noised sentences. It also shows that the translation performance gap is quite big (e.g., 44.28 in BLEU v.s. 35.20) when translating non-noised sentences and sentences of ASR output.
KGAT: An Enhanced Graph-Based Model for Text Classification

As a fundamental task in natural language processing, text classification, which is to predict the class label of a given text, has been intensively studied. Consequently, a host of techniques have been developed, among which techniques that are based on graph neural network and its variant $$e.g., $$ graph attention network ( $$\textsc {GAT}$$ ) achieved impressive performances, as they show superiority in dealing with complex graph-structured data. Despite effectiveness, most of these techniques suffer from several limitations, $$e.g., $$ incapability in well-capturing correlation among words in a text. In light of these, we propose a comprehensive approach $$\textsc {KGAT}$$ which incorporates multi-head $$\textsc {GAT}$$ with enhanced attention and customized ReadOut operation for text classification. (1) Our approach constructs a text graph $$G_T$$ with edge weights from a text such that both semantic and structural information (with correlation degree) can be well captured. (2) On text graph $$G_T$$ , a novel attention mechanism is incorporated in a multi-head $$\textsc {GAT}$$ for representation learning. (3) Our approach customizes ReadOut operation such that the representation of a text is refined by using a set of influential nodes of $$G_T$$ . Intensive experimental studies on both typical benchmark datasets and a newly created one ( $${\textsf{Sensitive}}$$ ) show that our approach substantially outperforms other baseline methods and yields a promising technique for text classification.
Generative Text Steganography via Multiple Social Network Channels Based on Transformers

Generative text steganography uses the conditional probability to encode the candidate words when generating tokens by language model, and then selects the corresponding word to output according to the secret message to be embedded, so as to generate stego text. The complex and open characteristics of social network provide a good camouflage environment for the transmission of stego texts, but also bring challenges: transmitting stego text through a single channel is easy to cause the destruction and loss of secret message; the speech of each social account needs to be combined with its background knowledge, so it has different language features. The existing text steganography schemes cannot solve these problems well. This paper proposes a multi-channel generative text steganography scheme in the context of social network, which hides secret message into multiple semantically natural texts, even if only a part of which can reconstruct secret message. Combined with the characteristics of social network, the bag-of-words models are used to control the topics of the stego texts in the process of text generation by language model. Two goal programming models are proposed to optimize the topic relevance and text quality of stego text. The experiment verifies the effectiveness of this scheme.
A Multi-step Attention and Multi-level Structure Network for Multimodal Sentiment Analysis

Multimodal sentiment analysis aims to predict sentiment polarity from several modalities, which is an essential task for widespread applications. The core part of this task is to design a suitable fusion schema to integrate the heterogeneous information from different modalities. However, previous methods usually adopted simple interaction strategies, such as gate or attention mechanisms, which may lead to extracted features containing redundant information. In addition, most of them only focus on the interaction information between single modality, ignoring the modality pair’s interaction information. In this paper, we propose a Multi-step Attention and Multi-level Structure network (MAMS) to address the above problems. Specifically, the multi-step attention mechanism extracts the critical information multiple times during the fusion process, which can reduce the interference of redundant information. Furthermore, the multi-level structure can capture both single modality’s and modality pair’s interaction information. Experimental results on two datasets (CMU-MOSI and CMU-MOSEI) demonstrate the superiority and effectiveness of our proposed MAMS model.
Random Concatenation: A Simple Data Augmentation Method for Neural Machine Translation

Neural machine translation system heavily depends on large-scale parallel corpus, which is not available for some low-resource languages, resulting in poor translation quality. To alleviate such data hungry problem, we present a high quality data augmentation method which merely utilize the given parallel corpus. Specifically, we propose to augment the low-resource parallel corpus with a language-mixed bitext, which is simply built by concatenating two sentences in different languages. Furthermore, our approach which only takes advantage of parallel corpus is supplementary to existing data manipulation strategies, i.e. back-translation, self-training and knowledge distillation. Experiments on several low-resource datasets show that our approach achieves significant improvement over a strong baseline, despite its simplicity.
Hierarchical Planning of Topic-Comment Structure for Paper Abstract Writing

Paper abstract writing aims to generate medium-long content with the given paper title. It is challenging to generate coherent and accurate paper abstract. Previous work utilize graph data for assistance, while they cannot perform well in long-term coherence due to the lack of efficient text planning, or simply utilize the knowledge graph without distinguishing, thus introducing redundant information which may impair the quality of generated abstract. To this end, we propose a hierarchical planning model based on topic-comment structure for paper abstract writing. Specifically, first we acquire a sequence of topics from graph data in discourse-level planning about what each sentence to do, and then plan comments in sentence-level planning about how to do with the corresponding topic. Topic and comment are the core of abstract. Furthermore, in the second-stage planning, we propose a novel planning mechanism that can separate the knowledge graph into comment sub-graphs. It only preserves useful vertices that reduces the candidate set. Experiments prove our model’s effectiveness in generating coherent and accurate paper abstracts, and its overall performance is better than the baseline models in both automatic and human evaluation.
KBRTE: A Deep Learning Model for Chinese Textual Entailment Recognition Based on Synonym Expansion and Sememe Enhancement

The mainstream textual entailment recognition models ignore the existing language knowledge, so the inference knowledge from the training data is limited, and the generalization ability is not strong. Therefore, this paper proposes a model KBRTE (fusing Knowledge Base in RTE) that combines an attention mechanism and a pre-trained model and uses word vectors based on sememe representation in the HowNet. We use the enhanced CNLI and XNLI datasets as the model’s training set. On the basis of these datasets, monosemous and polysemous in the CiLin are integrated to further enhance the knowledge. Experimental results show that this method could bring significant gains.
Retrieval, Selection and Writing: A Three-Stage Knowledge Grounded Storytelling Model

Storytelling is a knowledge-driven task, which requires the model to associate relevant information given a context and organize them into a reasonable story. Some knowledge-enhanced storytelling models are proposed. However, there exist certain drawbacks in them, including needing better retrieval and selection strategy. Target on these, we propose a three-stage knowledge grounded storytelling model, which combines semantic knowledge retrieval, a knowledge selection and a story generation module. We build in-domain and open-domain knowledge, which is suitable for story generation. The knowledge selection and story generation modules are built in the readily available transformer-based framework. We devise two approaches to train the knowledge selection module. One is to leverage constructed pseudo labels. Another is to jointly train the knowledge selection and story generation modules, so as to leverage the supervision of the ground-truth story. We conduct experiments on the public ROCStories dataset, and the automatic and human evaluation demonstrates the effectiveness of our method.
Entity Difference Modeling Based Entity Linking for Question Answering over Knowledge Graphs

Entity linking plays a vital role in Question Answering over Knowledge Graphs (KGQA), and the representation of entities is a fundamental component of entity linking for user questions. In order to alleviate the problem of entity descriptions that unrelated texts obfuscate similar entities, we present a new entity linking framework, which refines the encodings of entity descriptions based on entity difference modeling, so that entity linking’s ability to distinguish among similar entities is improved. The entity differences are modeled in a two-stage approach: the initial differences are first computed among similar entity candidates by comparing their descriptions, and then interactions between the initial differences and questions are performed to extract key differences, which identify critical information for entity linking. On the basis of the key differences, subsequent entity description encodings are refined, and entity linking is then performed using the refined entity representations. Experimental results on end-to-end benchmark datasets demonstrate that our approach achieves state-of-the-art precision, recall and F1-score.
Blockchain Application for Cybersecurity in Cyber-Physical Systems

Current research in the fields of control architectures, cyber-physical systems (CPS), and the IoT must be brought together in order to ensure the viability of Industry 4.0 (IoT). Industrial systems are increasingly vulnerable, necessitating new approaches to fight and mitigate these risks. As an example of this innovation, blockchain offers an antidote to the cybersecurity concerns of Industry 4,0. This paper examines and discusses blockchain-based cybersecurity for the cyber-physical systems in more detail.
Federated Offline Reinforcement Learning for Autonomous Systems

Offline reinforcement learning (RL) provides a safe learning method that can be applied to real-world applications through a data-driven learning process. In general, this process attributes to learning on large datasets, which is similar to conventional supervised learning techniques. In this paper, we propose a federated offline RL framework where policy parameters learned locally on separate datasets of RL agents are aggregated in communication rounds to achieve a high-performance global policy. The framework can solve the data insufficiency issue by performing offline RL among multiple agents without data sharing and integration. We show that using the framework, agents achieve a stable-performance control policy in a 2D navigation environment, even when each has only 1% of the required training trajectory amount (e.g., 10 trajectories).
Exploring the Characteristics and Security Risks of Emerging Emoji Domain Names

Emoji domains, such as (xn–i-7iq.ws), are distinctive and attractive to registrants due to their eye-catching visuals. Despite its long history (over 20 years), little has been done to understand its development status and security issues. In this paper, we identify 54,403 emoji domains from 1,366 TLD zone files and a large-scale passive DNS dataset. And then, we correlate them with auxiliary data sources like domain WHOIS records. It allowed us to conduct by far the most systematic study to characterize the ecosystem, and retrieve multiple valuable insights. On one hand, the scale of emoji domains is constantly expanding in the wild, with dozens of ccTLD registries actively promoting registering domains with emoji characters and domain owners configuring emoji characters in sub-level domains. And emoji domains may act as promotional portals, as web requests are usually redirected to other websites. Besides, emoji domains are also leveraged to provide disposable email services, pornography or gambling pages, and even the distribution of malware. On the other hand, the concern is that the community still lacks best security practices in supporting and parsing emoji domains. Through empirical studies, we demonstrate that inconsistencies in rendering emoji characters can be exploited to launch visual phishing domain scams. Meanwhile, mainstream implementations may incorrectly parse or trans-code emoji domains, resulting in the security threat of traffic hijacking. Our study calls for standardization and best security practices for applications to handle emoji domains securely.
Language Identification Using Combination of Machine Learning Algorithms and Vectorization Techniques

Language Identification refers to the process of ascertaining and discerning the language found in a particular text or document. In this work, approaches for language identification, using Machine Learning Algorithms and Vectorization methods have been compared and contrasted. Three machine learning algorithms, along with two vectorization techniques have been used. The ML Algorithms used are Na√Øve bayes, Logistic Regression, and SVM (Support Vector Machine), and the vectorization techniques used are Term Frequency-Inverse Document Frequency (TFIDF), and Count Vectorizer (Bag of Words (BoW)). This research put forwards the contrast and comparison of the above-mentioned classification algorithms and vectorization methods. It is also a web development-based work.
MaleficNet: Hiding Malware into Deep Neural Networks Using Spread-Spectrum Channel Coding

The training and development of good deep learning models is often a challenging task, thus leading individuals (developers, researchers, and practitioners alike) to use third-party models residing in public repositories, fine-tuning these models to their needs usually with little-to-no effort. Despite its undeniable benefits, this practice can lead to new attack vectors. In this paper, we demonstrate the feasibility and effectiveness of one such attack, namely malware embedding in deep learning models. We push the boundaries of current state-of-the-art by introducing MaleficNet, a technique that combines spread-spectrum channel coding with error correction techniques, injecting malicious payloads in the parameters of deep neural networks, all while causing no degradation to the model’s performance and successfully bypassing state-of-the-art detection and removal mechanisms. We believe this work will raise awareness against these new, dangerous, camouflaged threats, assist the research community and practitioners in evaluating the capabilities of modern machine learning architectures, and pave the way to research targeting the detection and mitigation of such threats.
Resilience Analysis of Deep Q-Learning Algorithms in Driving Simulations Against Cyberattacks

Deep reinforcement learning (DRL) has attracted attentions by researchers to complete complex tasks in engineering, such as autonomous driving, that are typically very difficult to achieve using traditional model-based approaches. With the safety being critical in self-driving vehicles and the increased reliance on vehicle connectivity, the resilience to cyberattacks has to be systematically studied. In this paper, we train a deep Q-learning based agent to drive autonomously in the CARLA simulator under various scenarios that the agent may experience during a cyberattack. Specifically, we observe the agent’s behavior and performance in the presence of denial-of-service and deception attacks. The results reflect an inbuilt level of resilience to cyberattacks with the DRL methods. Comparing with conventional driving agents, deep Q-learning agents can learn to deal with uncertainty and missing information without explicitly modeling such behavior
Applying Reward Design Based on Payment Mechanism to Shaped-Reward DQN for Beer Game

We focus on the application of multiagent reinforcement learning for supply chain management. The beer game is an example of a problem in supply chain management and has been studied as a cooperation problem in multiagent systems. In the previous study, a method SRDQN that is based on deep reinforcement learning and reward shaping has been applied as a solution to the beer game. In the previous study of SRDQN, a single agent in a game performs reinforcement learning considering other agents to reduce the global cost for inventories of beers. However, it is possible to employ other reward shaping techniques to improve learning stability. It can also be effective in the systems consisting of multiple agents that perform reinforcement learning. We apply a reward shaping technique based on mechanism design to SRDQN to improve the cooperative policies, and then we empirically evaluate the effectiveness of the proposed approach.
Verification of textual entailment in Japanese Using Sigmoid and ReLU Function

Many items must be especially considered in recognizing the textual entailment of Japanese compared with the multi-lingual. In this paper, we will create several machine learning models that take as input two sentences to be checked for textual entailment and their correct labels and examine which methods are effective for inference of textual entailment.
Development and evaluation of car training system using VR and eye tracking technology

When practicing car driving in a real world, there are risks like traffic accidents. With the development of virtual reality technology, we can create environments in which people can practice without being exposed to certain dangers like traffic accidents that potentially occur in real environments. So far, many researches regarding car driving in VR have been developed, but little attention has been paid to analyzing and visualizing eye gaze data collected in VR education systems. Analyzing and visualizing the collected eye gaze data is potential to grasp information of whether learners overlooked objects such as traffic signals and signs. In this study, we developed a driving simulator in VR and collected eye tracking data from five university students while they were practicing driving using eye tracking technology. Also, we have developed a system that allows learners to reflect on their own driving using the collected their eye gaze data. We compared the changes in gaze between those who received feedback to improve their driving skills and those who did not. The results showed that the feedback was effective because it reduced the number of oversights while driving.