Reviewerly/exHarmony huggingface dataset:
Are Training Samples Correlated? Learning to Generate Dialogue Responses with Multiple References

Due to its potential applications, open-domain dialogue generation has become popular and achieved remarkable progress in recent years, but sometimes suffers from generic responses.Previous models are generally trained based on 1-to-1 mapping from an input query to its response, which actually ignores the nature of 1-to-n mapping in dialogue that there may exist multiple valid responses corresponding to the same query.In this paper, we propose to utilize the multiple references by considering the correlation of different valid responses and modeling the 1-to-n mapping with a novel two-step generation architecture.The first generation phase extracts the common features of different responses which, combined with distinctive features obtained in the second phase, can generate multiple diverse and appropriate responses.Experimental results show that our proposed model can effectively improve the quality of response and outperform existing neural dialogue models on both automatic and human evaluations.
Numeracy-600K: Learning Numeracy for Detecting Exaggerated Information in Market Comments

In this paper, we attempt to answer the question of whether neural network models can learn numeracy, which is the ability to predict the magnitude of a numeral at some specific position in a text description.A large benchmark dataset, called Numeracy-600K, is provided for the novel task.We explore several neural network models including CNN, GRU, BiGRU, CRNN, CNN-capsule, GRU-capsule, and BiGRU-capsule in the experiments.The results show that the BiGRU model gets the best micro-averaged F1 score of 80.16%, and the GRU-capsule model gets the best macroaveraged F1 score of 64.71%.Besides discussing the challenges through comprehensive experiments, we also present an important application scenario, i.e., detecting exaggerated information, for the task.
Self-Attentional Models for Lattice Inputs

Lattices are an efficient and effective method to encode ambiguity of upstream systems in natural language processing tasks, for example to compactly capture multiple speech recognition hypotheses, or to represent multiple linguistic analyses.Previous work has extended recurrent neural networks to model lattice inputs and achieved improvements in various tasks, but these models suffer from very slow computation speeds.This paper extends the recently proposed paradigm of self-attention to handle lattice inputs.Self-attention is a sequence modeling technique that relates inputs to one another by computing pairwise similarities and has gained popularity for both its strong results and its computational efficiency.To extend such models to handle lattices, we introduce probabilistic reachability masks that incorporate lattice structure into the model and support lattice scores if available.We also propose a method for adapting positional embeddings to lattice structures.We apply the proposed model to a speech translation task and find that it outperforms all examined baselines while being much faster to compute than previous neural lattice models during both training and inference.
Side-Channel Analysis of Chaos-Based Substitution Box Structures

There is a theoretically strong relationship between chaos and cryptology. In practice, one of the most successful applications of this relationship is chaos-based s-box structures. However, the performance metrics of chaos-based s-box designs are worse compared to those for s-box structure based on algebraic techniques used in modern encryption algorithms, such as the AES algorithm. What is the reason for using chaos-based s-box structures, even though they have worse cryptographic characteristics? Researchers claim that they may be an alternative defense against implementation attacks, especially side-channel analyses. However, no studies have supported or refuted this claim so far. In this paper, side-channel analyses have been performed for two different chaos-based s-box structures. These two s-box structures have been selected for having with the best and worst performance measurements for s-box structures previously proposed in the literature. The results were compared with the AES s-box structure. Analysis of the results showed that chaos-based s-box structures are more resistant against side-channel attacks. Therefore, chaos-based designs may be an alternative defense against implementation attacks, as alleged. However, both algebraic and chaos-based s-box designs have been observed to be insecure if the attacker has more than 30 plaintexts in the side channel analyses. These results show that implementation analysis studies are required in the chaos-based cryptology literature.
Multi-Attention Object Detection Model in Remote Sensing Images Based on Multi-Scale

Ground object detection, based on remote sensing satellite imagery, provides the groundwork of numerous applications, so the detection accuracy is of vital importance. The background of remote sensing images is complex, the object size is various, and there are many small objects. In view of the above problems, a multi-attention object detection method (MA-FPN) based on multi-scale is proposed in this paper, which can effectively make the network pay attention to the location of the object and reduce the loss of small object information. According to feature pyramid network (FPN), we firstly put forward a global spatial attention module, which extracts spatial location-related information from shallow features and fuses it with deep features to enhance the position expression ability of deep features. Besides, the paper provides a pixel feature attention module: the multi-scale convolution kernel is employed to generate the feature map of the same size as the input, as well as channel attention is used to assign weights to each layer of feature maps to obtain pixel-level attention maps with good details. Experiments on NWPU, RSOD, and DOTA datasets show that the proposed algorithm outperforms state of the art methods.
Comparing heuristics for graph edit distance computation

Because of its flexibility, intuitiveness, and expressivity, the graph edit distance (GED) is one of the most widely used distance measures for labeled graphs. Since exactly computing GED is NP-hard, over the past years, various heuristics have been proposed. They use techniques such as transformations to the linear sum assignment problem with error correction, local search, and linear programming to approximate GED via upper or lower bounds. In this paper, we provide a systematic overview of the most important heuristics. Moreover, we empirically evaluate all compared heuristics within an integrated implementation.
I/O Characterization and Performance Evaluation of BeeGFS for Deep Learning

Parallel File Systems (PFSs) are frequently deployed on leadership High Performance Computing (HPC) systems to ensure efficient I/O, persistent storage and scalable performance. Emerging Deep Learning (DL) applications incur new I/O and storage requirements to HPC systems with batched input of small random files. This mandates PFSs to have commensurate features that can meet the needs of DL applications. BeeGFS is a recently emerging PFS that has grabbed the attention of the research and industry world because of its performance, scalability and ease of use. While emphasizing a systematic performance analysis of BeeGFS, in this paper, we present the architectural and system features of BeeGFS, and perform an experimental evaluation using cutting-edge I/O, Metadata and DL application benchmarks. Particularly, we have utilized AlexNet and ResNet-50 models for the classification of ImageNet dataset using the Livermore Big Artificial Neural Network Toolkit (LBANN), and ImageNet data reader pipeline atop TensorFlow and Horovod. Through extensive performance characterization of BeeGFS, our study provides a useful documentation on how to leverage BeeGFS for the emerging DL applications.
Angular Triplet-Center Loss for Multi-View 3D Shape Retrieval

How to obtain the desirable representation of a 3D shape, which is discriminative across categories and polymerized within classes, is a significant challenge in 3D shape retrieval. Most existing 3D shape retrieval methods focus on capturing strong discriminative shape representation with softmax loss for the classification task, while the shape feature learning with metric loss is neglected for 3D shape retrieval. In this paper, we address this problem based on the intuition that the cosine distance of shape embeddings should be close enough within the same class and far away across categories. Since most of 3D shape retrieval tasks use cosine distance of shape features for measuring shape similarity, we propose a novel metric loss named angular triplet-center loss, which directly optimizes the cosine distances between the features. It inherits the triplet-center loss property to achieve larger inter-class distance and smaller intra-class distance simultaneously. Unlike previous metric loss utilized in 3D shape retrieval methods, where Euclidean distance is adopted and the margin design is difficult, the proposed method is more convenient to train feature embeddings and more suitable for 3D shape retrieval. Moreover, the angle margin is adopted to replace the cosine margin in order to provide more explicit discriminative constraints on an embedding space. Extensive experimental results on two popular 3D object retrieval benchmarks, ModelNet40 and ShapeNetCore 55, demonstrate the effectiveness of our proposed loss, and our method has achieved state-ofthe-art results on various 3D shape datasets.
A hierarchical contextual attention-based network for sequential recommendation

The sequential recommendation is one of the most fundamental tasks for Web applications. Recently, recurrent neural network (RNN) based methods become popular and show effectiveness in many sequential recommendation tasks, such as next-basket recommendation and location prediction. The last hidden state of RNN is usually applied as the sequence’s representation to make recommendations. RNN can capture the long-term interest with the help of gated activations or regularizers but has difficulty in acquiring the short-term interest due to the ordered modeling. In this work, we aim to strengthen the short-term interest, because it is beneficial to generate responsive recommendation according to recent behaviors. Accordingly, we propose a Hierarchical Contextual Attention-based (HCA) network. First, RNN is extended to model several adjacent factors at each time step. Such kind of multiple factors can be considered as a context where the short-term interest comes from. Then, within the context, the attention mechanism is used to find the important items that contribute to the short-term interest. This contextual attention-based technique is conducted on the input and hidden state of RNN respectively. In this way, we can relieve the limitation of ordered modeling of RNN, model the complicated correlations among recent factors, and strengthen the short-term interest. Experiments on two real-world datasets show that HCA can effectively generate the personalized ranking list and achieve considerable improvements.
Anomaly Detection of CAN Bus Messages Using A Deep Neural Network for Autonomous Vehicles

The in-vehicle controller area network (CAN) bus is one of the essential components for autonomous vehicles, and its safety will be one of the greatest challenges in the field of intelligent vehicles in the future. In this paper, we propose a novel system that uses a deep neural network (DNN) to detect anomalous CAN bus messages. We treat anomaly detection as a cross-domain modelling problem, in which three CAN bus data packets as a group are directly imported into the DNN architecture for parallel training with shared weights. After that, three data packets are represented as three independent feature vectors, which corresponds to three different types of data sequences, namely anchor, positive and negative. The proposed DNN architecture is an embedded triplet loss network that optimizes the distance between the anchor example and the positive example, makes it smaller than the distance between the anchor example and the negative example, and realizes the similarity calculation of samples, which were originally used in face detection. Compared to traditional anomaly detection methods, the proposed method to learn the parameters with shared-weight could improve detection efficiency and detection accuracy. The whole detection system is composed of the front-end and the back-end, which correspond to deep network and triplet loss network, respectively, and are trainable in an end-to-end fashion. Experimental results demonstrate that the proposed technology can make real-time responses to anomalies and attacks to the CAN bus, and significantly improve the detection ratio. To the best of our knowledge, the proposed method is the first used for anomaly detection in the in-vehicle CAN bus.
Medical Concept Embedding with Multiple Ontological Representations

Learning representations of medical concepts from the Electronic Health Records (EHR) has been shown effective for predictive analytics in healthcare. Incorporation of medical ontologies has also been explored to further enhance the accuracy and to ensure better alignment with the known medical knowledge. Most of the existing work assumes that medical concepts under the same ontological category should share similar representations, which however does not always hold. In particular, the categorizations in medical ontologies were established with various factors being considered. Medical concepts even under the same ontological category may not follow similar occurrence patterns in the EHR data, leading to contradicting objectives for the representation learning. In this paper, we propose a deep learning model called MMORE which alleviates this conflicting objective issue by allowing multiple representations to be inferred for each ontological category via an attention mechanism. We apply MMORE to diagnosis prediction and our experimental results show that the representations obtained by MMORE can achieve better predictive accuracy and result in clinically meaningful sub-categorization of the existing ontological categories.
Sentence-Level Content Planning and Style Specification for Neural Text Generation

Building effective text generation systems requires three critical components: content selection, text planning, and surface realization, and traditionally they are tackled as separate problems.Recent all-in-one style neural generation models have made impressive progress, yet they often produce outputs that are incoherent and unfaithful to the input.To address these issues, we present an end-toend trained two-step generation model, where a sentence-level content planner first decides on the keyphrases to cover as well as a desired language style, followed by a surface realization decoder that generates relevant and coherent text.For experiments, we consider three tasks from domains with diverse topics and varying language styles: persuasive argument construction from Reddit, paragraph generation for normal and simple versions of Wikipedia, and abstract generation for scientific articles.Automatic evaluation shows that our system can significantly outperform competitive comparisons.Human judges further rate our system generated text as more fluent and correct, compared to the generations by its variants that do not consider language style.
UER: An Open-Source Toolkit for Pre-training Models

Existing works, including ELMO and BERT, have revealed the importance of pre-training for NLP tasks.While there does not exist a single pre-training model that works best in all cases, it is of necessity to develop a framework that is able to deploy various pre-training models efficiently.For this purpose, we propose an assemble-on-demand pre-training toolkit, namely Universal Encoder Representations (UER).UER is loosely coupled, and encapsulated with rich modules.By assembling modules on demand, users can either reproduce a state-of-the-art pre-training model or develop a pre-training model that remains unexplored.With UER, we have built a model zoo, which contains pre-trained models based on different corpora, encoders, and targets (objectives).With proper pre-trained models, we could achieve new state-of-the-art results on a range of downstream datasets.
Real-Time Surveillance Through Face Recognition Using HOG and Feedforward Neural Networks

Real-time security requirements continue to increase due to the occurrence of various suspicious activities in open and closed environments. Day-to-day security threats may seriously affect everyone lives. Many techniques have been introduced in this regard, but still some issues remain unaddressed. The work presented in this paper provides video surveillance with improved accuracy and less computational complexity. The most significant part of the system consists of face localization, detection and recognition. The system obtains underlined facial data through a video dataset or from the real-time environment. Subsequently, face/foreground and background keyframes are extracted from at hand captured video data. Finally, extracted facial image data is compared with the facial images in the database. In case no match is found with the existing data, a security alarm or signal is generated, alerting security personnel to take action. The proposed system is more accurate, has better performance, and is low cost compared with existing systems.
An Information-Theoretic Approach to Transferability in Task Transfer Learning

Task transfer learning is a popular technique in image processing applications that uses pre-trained models to reduce the supervision cost of related tasks. An important question is to determine task transferability, i.e. given a common input domain, estimating to what extent representations learned from a source task can help in learning a target task. Typically, transferability is either measured experimentally or inferred through task relatedness, which is often defined without a clear operational meaning. In this paper, we present a novel metric, H-score, an easily-computable evaluation function that estimates the performance of transferred representations from one task to another in classification problems using statistical and information theoretic principles. Experiments on real image data show that our metric is not only consistent with the empirical transferability measurement, but also useful to practitioners in applications such as source model selection and task transfer curriculum learning.
Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog

Dialog policy decides what and how a taskoriented dialog system will respond, and plays a vital role in delivering effective conversations.Many studies apply Reinforcement Learning to learn a dialog policy with the reward function which requires elaborate design and pre-specified user goals.With the growing needs to handle complex goals across multiple domains, such manually designed reward functions are not affordable to deal with the complexity of real-world tasks.To this end, we propose Guided Dialog Policy Learning, a novel algorithm based on Adversarial Inverse Reinforcement Learning for joint reward estimation and policy optimization in multi-domain task-oriented dialog.The proposed approach estimates the reward signal and infers the user goal in the dialog sessions.The reward estimator evaluates the state-action pairs so that it can guide the dialog policy at each dialog turn.Extensive experiments on a multi-domain dialog dataset show that the dialog policy guided by the learned reward function achieves remarkably higher task success than state-of-the-art baselines.
Findings of the WMT 2019 Shared Task on Automatic Post-Editing

We present the results from the 5 th round of the WMT task on MT Automatic Post-Editing.The task consists in automatically correcting the output of a "black-box" machine translation system by learning from human corrections.Keeping the same general evaluation setting of the previous four rounds, this year we focused on two language pairs (English-German and English-Russian) and on domain-specific data (Information Technology).For both the language directions, MT outputs were produced by neural systems unknown to participants.Seven teams participated in the English-German task, with a total of 18 submitted runs.The evaluation, which was performed on the same test set used for the 2018 round, shows further progress in APE technology: 4 teams achieved better results than last year's winning system, with improvements up to -0.78 TER and +1.23 BLEU points over the baseline.Two teams participated in the English-Russian task submitting 2 runs each.On this new language direction, characterized by a higher quality of the original translations, the task proved to be particularly challenging.Indeed, none of the submitted runs improved the very high results of the strong system used to produce the initial translations (16.16 TER, 76.20 BLEU).
Towards Understanding Neural Machine Translation with Word Importance

Although neural machine translation (NMT) has advanced the state-of-the-art on various language pairs, the interpretability of NMT remains unsatisfactory.In this work, we propose to address this gap by focusing on understanding the input-output behavior of NMT models.Specifically, we measure the word importance by attributing the NMT output to every input word through a gradient-based method.We validate the approach on a couple of perturbation operations, language pairs, and model architectures, demonstrating its superiority on identifying input words with higher influence on translation performance.Encouragingly, the calculated importance can serve as indicators of input words that are under-translated by NMT models.Furthermore, our analysis reveals that words of certain syntactic categories have higher importance while the categories vary across language pairs, which can inspire better design principles of NMT architectures for multi-lingual translation.
Assessing the digital competence of educators in social studies: An analysis in initial teacher training using the TPACK-21 model

Fostering the digital competence of educators is a key aspect that can be addressed in initial teacher training. The TPACK model (Mishra &amp; Koehler, 2006) can be complemented with an approach that takes the Digital Competence of Educators framework (Redecker &amp; Punie, 2017) and 21st century competences into account. This study analysed the practical utility of this conceptual model, and the effectiveness of a teaching intervention in a university setting, during two academic years, with social studies secondary education prospective teachers. Using a quantitative approach and the TPACK-21 questionnaire (Valtonen et al., 2017), the study examined the starting point and evolution of the participants. Results indicate the adequacy of the framework and instrument used and favourable progress towards competence after the assessment of seven factors. Although pre-service teachers showed a low degree of confidence regarding their capabilities of integrating technological with pedagogical and content knowledge, especially in three of the components, this obstacle was overcome after the intervention. It is possible to conclude that the pedagogical and conceptual orientation of the teaching proposal has shown a positive effect, evidencing the effectiveness of a comprehensive approach capable of adapting to the specificity and challenges of social studies education.