Reviewerly/exHarmony huggingface dataset:
Group-wise Contrastive Learning for Neural Dialogue Generation

Neural dialogue response generation has gained much popularity in recent years.Maximum Likelihood Estimation (MLE) objective is widely adopted in existing dialogue model learning.However, models trained with MLE objective function are plagued by the lowdiversity issue when it comes to the opendomain conversational setting.Inspired by the observation that humans not only learn from the positive signals but also benefit from correcting behaviors of undesirable actions, in this work, we introduce contrastive learning into dialogue generation, where the model explicitly perceives the difference between the well-chosen positive and negative utterances.Specifically, we employ a pretrained baseline model as a reference.During contrastive learning, the target dialogue model is trained to give higher conditional probabilities for the positive samples, and lower conditional probabilities for those negative samples, compared to the reference model.To manage the multimapping relations prevalent in human conversation, we augment contrastive dialogue learning with group-wise dual sampling.Extensive experimental results show that the proposed group-wise contrastive learning framework is suited for training a wide range of neural dialogue generation models with very favorable performance over the baseline training approaches.
Inferring the dynamics of oscillatory systems using recurrent neural networks

We investigate the predictive power of recurrent neural networks for oscillatory systems not only on the attractor, but in its vicinity as well. For this we consider systems perturbed by an external force. This allows us to not merely predict the time evolution of the system, but also study its dynamical properties, such as bifurcations, dynamical response curves, characteristic exponents etc. It is shown that they can be effectively estimated even in some regions of the state space where no input data were given. We consider several different oscillatory examples, including self-sustained, excitatory, time-delay and chaotic systems. Furthermore, with a statistical analysis we assess the amount of training data required for effective inference for two common recurrent neural network cells, the long short-term memory and the gated recurrent unit.
Claim Check-Worthiness Detection as Positive Unlabelled Learning

As the first step of automatic fact checking, claim check-worthiness detection is a critical component of fact checking systems.There are multiple lines of research which study this problem: check-worthiness ranking from political speeches and debates, rumour detection on Twitter, and citation needed detection from Wikipedia.To date, there has been no structured comparison of these various tasks to understand their relatedness, and no investigation into whether or not a unified approach to all of them is achievable.In this work, we illuminate a central challenge in claim check-worthiness detection underlying all of these tasks, being that they hinge upon detecting both how factual a sentence is, as well as how likely a sentence is to be believed without verification.As such, annotators only mark those instances they judge to be clear-cut check-worthy.Our best performing method is a unified approach which automatically corrects for this using a variant of positive unlabelled learning that finds instances which were incorrectly labelled as not check-worthy.In applying this, we outperform the state of the art in two of the three tasks studied for claim check-worthiness detection in English.
INS/UWB-Based Quadrotor Localization Under Colored Measurement Noise

Data fusion in the quadrotor integrated navigation system combining information from the inertial navigation system (INS) and ultra wide band (UWB)-based system is provided in the presence of the UWB-induced colored measurement noise (CMN). To improve the localization performance under the CMN, we modify the fusion Kalman filter (KF) to be cKF and the unbiased finite impulse response (UFIR) filter to be cUFIR filter. The cKF and cUFIR filter are derived using measurement differencing and by de-correlating noise vectors. The original and modified filters are tested in the best and worst cases of tuning parameters. Based on test measurements without the ground truth, it is shown that the cKF improves the performance in the best case, but loses to the KF in the worst case due to a higher sensitivity to errors in the noise covariances. It is also shown that the cUFIR filter is more robust and thus more suitable than the cKF and KF for quadrotor localization under the industrial conditions.
Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition

Past progress on neural models has proven that named entity recognition is no longer a problem if we have enough labeled data.However, collecting enough data and annotating them are labor-intensive, time-consuming, and expensive.In this paper, we decompose the sentence into two parts: entity and context, and rethink the relationship between them and model performance from a causal perspective.Based on this, we propose the Counterfactual Generator, which generates counterfactual examples by the interventions on the existing observational examples to enhance the original dataset.Experiments across three datasets show that our method improves the generalization ability of models under limited observational examples.Besides, we provide a theoretical foundation by using a structural causal model to explore the spurious correlations between input features and output labels.We investigate the causal effects of entity or context on model performance under both conditions: the non-augmented and the augmented.Interestingly, we find that the non-spurious correlations are more located in entity representation rather than context representation.As a result, our method eliminates part of the spurious correlations between context representation and output labels.
Pre-Defined Sparse Neural Networks With Hardware Acceleration

Neural networks have proven to be extremely powerful tools for modern artificial intelligence applications, but computational and storage complexity remain limiting factors.This paper presents two compatible contributions towards reducing the time, energy, computational, and storage complexities associated with multilayer perceptrons.Pre-defined sparsity is proposed to reduce the complexity during both training and inference, regardless of the implementation platform.Our results show that storage and computational complexity can be reduced by factors greater than 5X without significant performance loss.The second contribution is an architecture for hardware acceleration that is compatible with pre-defined sparsity.This architecture supports both training and inference modes and is flexible in the sense that it is not tied to a specific number of neurons.For example, this flexibility implies that various sized neural networks can be supported on various sized Field Programmable Gate Array (FPGA)s.
Continuous-time quantum walks on dynamical percolation graphs

We address continuous-time quantum walks on graphs in the presence of time- and space-dependent noise. Noise is modeled as generalized dynamical percolation, i.e., classical time-dependent fluctuations affecting the tunneling amplitudes of the walker. In order to illustrate the general features of the model, we review recent results on two paradigmatic examples: the dynamics of quantum walks on the line and the effects of noise on the performances of quantum spatial search on the complete and the star graph. We also discuss future perspectives, including extension to many-particle quantum walk, to noise model for on-site energies and to the analysis of different noise spectra. Finally, we address the use of quantum walks as a quantum probe to characterize defects and perturbations occurring in complex, classical and quantum, networks.
QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines

Discourse relations describe how two propositions relate to one another, and identifying them automatically is an integral part of natural language understanding.However, annotating discourse relations typically requires expert annotators.Recently, different semantic aspects of a sentence have been represented and crowd-sourced via question-and-answer (QA) pairs.This paper proposes a novel representation of discourse relations as QA pairs, which in turn allows us to crowd-source widecoverage data annotated with discourse relations, via an intuitively appealing interface for composing such questions and answers.Based on our proposed representation, we collect a novel and wide-coverage QADiscourse dataset, and present baseline algorithms for predicting QADiscourse relations.
Learning from past mistakes: improving automatic speech recognition output via noisy-clean phrase context modeling

Automatic speech recognition (ASR) systems often make unrecoverable errors due to subsystem pruning (acoustic, language and pronunciation models); for example pruning words due to acoustics using short-term context, prior to rescoring with long-term context based on linguistics. In this work we model ASR as a phrase-based noisy transformation channel and propose an error correction system that can learn from the aggregate errors of all the independent modules constituting the ASR and attempt to invert those. The proposed system can exploit long-term context using a neural network language model and can better choose between existing ASR output possibilities as well as re-introduce previously pruned or unseen (out-of-vocabulary) phrases. It provides corrections under poorly performing ASR conditions without degrading any accurate transcriptions; such corrections are greater on top of out-of-domain and mismatched data ASR. Our system consistently provides improvements over the baseline ASR, even when baseline is further optimized through recurrent neural network language model rescoring. This demonstrates that any ASR improvements can be exploited independently and that our proposed system can potentially still provide benefits on highly optimized ASR. Finally, we present an extensive analysis of the type of errors corrected by our system.
GANHopper: Multi-hop GAN for Unsupervised Image-to-Image Translation

We introduce GANHopper, an unsupervised image-to-image translation network that transforms images gradually between two domains, through multiple hops. Instead of executing translation directly, we steer the translation by requiring the network to produce in-between images that resemble weighted hybrids between images from the input domains. Our network is trained on unpaired images from the two domains only, without any in-between images. All hops are produced using a single generator along each direction. In addition to the standard cycle-consistency and adversarial losses, we introduce a new hybrid discriminator, which is trained to classify the intermediate images produced by the generator as weighted hybrids, with weights based on a predetermined hop count. We also add a smoothness term to constrain the magnitude of each hop, further regularizing the translation. Compared to previous methods, GANHopper excels at image translations involving domain-specific image features and geometric variations while also preserving non-domain-specific features such as general color schemes.
DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances

Recent advances in pre-trained language models have significantly improved neural response generation. However, existing methods usually view the dialogue context as a linear sequence of tokens and learn to generate the next word through token-level self-attention. Such token-level encoding hinders the exploration of discourse-level coherence among utterances. This paper presents DialogBERT, a novel conversational response generation model that enhances previous PLM-based dialogue models. DialogBERT employs a hierarchical Transformer architecture. To efficiently capture the discourse-level coherence among utterances, we propose two training objectives, including masked utterance regression and distributed utterance order ranking in analogy to the original BERT training. Experiments on three multi-turn conversation datasets show that our approach remarkably outperforms three baselines, such as BART and DialoGPT, in terms of quantitative evaluation. The human evaluation suggests that DialogBERT generates more coherent, informative, and human-like responses than the baselines with significant margins.
Rethinking the Value of Transformer Components

Transformer becomes the state-of-the-art translation model, while it is not well studied how each intermediate component contributes to the model performance, which poses significant challenges for designing optimal architectures.In this work, we bridge this gap by evaluating the impact of individual component (sub-layer) in trained Transformer models from different perspectives.Experimental results across language pairs, training strategies, and model capacities show that certain components are consistently more important than the others.We also report a number of interesting findings that might help humans better analyze, understand and improve Transformer models.Based on these observations, we further propose a new training strategy that can improves translation performance by distinguishing the unimportant components in training.
Deconstruct to Reconstruct a Configurable Evaluation Metric for Open-Domain Dialogue Systems

Many automatic evaluation metrics have been proposed to score the overall quality of a response in open-domain dialogue.Generally, the overall quality is comprised of various aspects, such as relevancy, specificity, and empathy, and the importance of each aspect differs according to the task.For instance, specificity is mandatory in a food-ordering dialogue task, whereas fluency is preferred in a language-teaching dialogue system.However, existing metrics are not designed to cope with such flexibility.For example, BLEU score fundamentally relies only on word overlapping, whereas BERTScore relies on semantic similarity between reference and candidate response.Thus, they are not guaranteed to capture the required aspects, i.e., specificity.To design a metric that is flexible to a task, we first propose making these qualities manageable by grouping them into three groups: understandability, sensibleness, and likability, where likability is a combination of qualities that are essential for a task.We also propose a simple method to composite metrics of each aspect to obtain a single metric called USL-H, which stands for Understandability, Sensibleness, and Likability in Hierarchy 1 .We demonstrated that USL-H score achieves good correlations with human judgment and maintains its configurability towards different aspects and metrics.
Elderly Care Based on Hand Gestures Using Kinect Sensor

Technological advances have allowed hand gestures to become an important research field especially in applications such as health care and assisting applications for elderly people, providing a natural interaction with the assisting system through a camera by making specific gestures. In this study, we proposed three different scenarios using a Microsoft Kinect V2 depth sensor then evaluated the effectiveness of the outcomes. The first scenario used joint tracking combined with a depth threshold to enhance hand segmentation and efficiently recognise the number of fingers extended. The second scenario utilised the metadata parameters provided by the Kinect V2 depth sensor, which provided 11 parameters related to the tracked body and gave information about three gestures for each hand. The third scenario used a simple convolutional neural network with joint tracking by depth metadata to recognise and classify five hand gesture categories. In this study, deaf-mute elderly people performed five different hand gestures, each related to a specific request, such as needing water, meal, toilet, help and medicine. Next, the request was sent via the global system for mobile communication (GSM) as a text message to the care provider’s smartphone because the elderly subjects could not execute any activity independently.
Short-Term Photovoltaic Power Prediction Based on Similar Days and Improved SOA-DBN Model

Existing methods in predicting short-term photovoltaic (PV) power have low accuracy and cannot satisfy actual demand. Thus, a prediction model based on similar days and seagull optimization algorithm (SOA) is proposed to optimize a deep belief network (DBN). Fast correlation-based filter (FCBF) method is used to select a meteorological feature set with the best correlation with PV output and avoid redundancy among meteorological factors affecting PV output. In addition, a comprehensive similarity index combining European distance and gray correlation degree is proposed to select the similar day. Then, SOA is used to optimize the number of neurons and the learning rate parameters in DBN. Based on the nonuniform mutation and opposition-based learning method, an improved seagull optimization algorithm (ISOA) with higher optimization accuracy is proposed. Finally, the ISOA-DBN prediction model is established, and the experimental analysis is conducted using the actual data of PV power stations in Australia. Results show that compared with DBN, support vector machine (SVM), extreme learning machine (ELM), radial basis function (RBF), Elman, and back propagation (BP), the mean absolute percentage error indicator of ISOA-DBN is only 1.512% on a sunny day, 5.975 on a rainy day, 3.359 on a cloudy to sunny day, and 1.911% on a sunny to cloudy day. Therefore, the good accuracy of the proposed model is verified.
Neural Machine Translation Models with Back-Translation for the Extremely Low-Resource Indigenous Language Bribri

This paper presents a neural machine translation model and dataset for the Chibchan language Bribri, with an average performance of BLEU 16.9±1.7.This was trained on an extremely small dataset (5923 Bribri-Spanish pairs), providing evidence for the applicability of NMT in extremely low-resource environments.We discuss the challenges entailed in managing training input from languages without standard orthographies, we provide evidence of successful learning of Bribri grammar, and also examine the translations of structures that are infrequent in major Indo-European languages, such as positional verbs, ergative markers, numerical classifiers and complex demonstrative systems.In addition to this, we perform an experiment of augmenting the dataset through iterative back-translation (Sennrich et al., 2016a;Hoang et al., 2018), by using Spanish sentences to create synthetic Bribri sentences.This improves the score by an average of 1.0 BLEU, but only when the new Spanish sentences belong to the same domain as the other Spanish examples.This contributes to the small but growing body of research on Chibchan NLP.
Traffic Sign Detection and Recognition Using Multi-Scale Fusion and Prime Sample Attention

Traffic sign detection, though one of the key technologies in intelligent transportation, still has bottleneck in accuracy due to the small size and diversity of traffic signs. To solve this problem, we proposed a two-stage CNN object detection algorithm based on multi-scale feature fusion and prime sample attention. We improved the original Faster R-cnn model in terms of feature extraction and sampling strategy. For feature extraction, to elevate the ability of neural networks to detect small objects, we adopted HRNet as the feature extractor. There are four stages in HRNet - a series of high resolution subnets as the starting point with repeated adding parallel high to low resolution subnets to form other stages. In the whole process, the information in the parallel multi-resolution sub-network is repeatedly exchanged to perform repeated multi-scale fusion. For sampling strategy, we adopted a simple and effective sampling and learning strategy called Prime Sample Attention (PISA), consisting of Importance-based Sample Reweighting (ISR) and Classification Aware Regression Loss (CARL). PISA proposed the concepts of IoU Hierarchical Partial Sorting (IoU-HLR) and Hierarchical Partial Score Sorting (Score-HLR), which sort the importance of positive samples and negative samples in mini-batch respectively. With the proposed method, the training process is focusing on prime samples rather than evenly treat all ones. The algorithm complexity of our method is lower than that of other state-of-the-art. After experiments by TT100K dataset, our method can attain a comparable or even better detection accuracy and robustness.
Can cryptocurrencies be a safe haven during the novel COVID-19 pandemic? Evidence from the Tunisian Stock Market

In this paper, we discuss the behavior of stock market returns in Tunisia during the COVID-19 outbreak. Using the OLS regression, we find that Bitcoin act as a hedge and Ethereum as a diversifier for Tunisia’s stock market before the COVID-19 outbreak; however, Bitcoin and Ethereum cannot generate benefits from portfolio diversification and hedging strategies for financial investors during the COVID-19. Moreover, Dash, Monero, and Ripple act as hedges before the COVID-19 outbreak and as diversifiers during this pandemic. Our results reveal that gold acts as a hedge and diversifier before the pandemic, but it's neither hedge nor a haven during the COVID-19 pandemic. Besides, the results indicated that the expected volatility of the US stock market has an impact on the Tunisian stock market. Finally, our results indicate that the growth rate of the COVID-19 confirmed cases and deaths harms Tunisia's stock market.
Compressible Latent-Space Invertible Networks for Generative Model-Constrained Image Reconstruction

There remains an important need for the development of image reconstruction methods that can produce diagnostically useful images from undersampled measurements. In magnetic resonance imaging (MRI), for example, such methods can facilitate reductions in data-acquisition times. Deep learning-based methods hold potential for learning object priors or constraints that can serve to mitigate the effects of data-incompleteness on image reconstruction. One line of emerging research involves formulating an optimization-based reconstruction method in the latent space of a generative deep neural network. However, when generative adversarial networks (GANs) are employed, such methods can result in image reconstruction errors if the sought-after solution does not reside within the range of the GAN. To circumvent this problem, in this work, a framework for reconstructing images from incomplete measurements is proposed that is formulated in the latent space of invertible neural network-based generative models. A novel regularization strategy is introduced that takes advantage of the multiscale architecture of certain invertible neural networks, which can result in improved reconstruction performance over classical methods in terms of traditional metrics. The proposed method is investigated for reconstructing images from undersampled MRI data. The method is shown to achieve comparable performance to a state-of-the-art generative model-based reconstruction method while benefiting from a deterministic reconstruction procedure and easier control over regularization parameters.
The ABCs of Review Articles

Recently, our editorial team had a regular meeting, and the slate of pre-submission (presub) inquiries was next on the agenda. For some reason, the cohort of submissions was overwhelmed by review article proposals. We tend to solicit and commission most of our reviews and perspectives, but we are always open to unsolicited ideas (feel free to e-mail them to matter@cell.com; be creative!), so such presubs are not out of the norm. The sheer number of review article proposals (as opposed to research), however, was surprising.