Reviewerly/exHarmony huggingface dataset:
Reinforcement Learning with Perturbed Rewards

Recent studies have shown that reinforcement learning (RL) models are vulnerable in various noisy scenarios. For instance, the observed reward channel is often subject to noise in practice (e.g., when rewards are collected through sensors), and is therefore not credible. In addition, for applications such as robotics, a deep reinforcement learning (DRL) algorithm can be manipulated to produce arbitrary errors by receiving corrupted rewards. In this paper, we consider noisy RL problems with perturbed rewards, which can be approximated with a confusion matrix. We develop a robust RL framework that enables agents to learn in noisy environments where only perturbed rewards are observed. Our solution framework builds on existing RL/DRL algorithms and firstly addresses the biased noisy reward setting without any assumptions on the true distribution (e.g., zero-mean Gaussian noise as made in previous works). The core ideas of our solution include estimating a reward confusion matrix and defining a set of unbiased surrogate rewards. We prove the convergence and sample complexity of our approach. Extensive experiments on different DRL platforms show that trained policies based on our estimated surrogate reward can achieve higher expected rewards, and converge faster than existing baselines. For instance, the state-of-the-art PPO algorithm is able to obtain 84.6% and 80.8% improvements on average score for five Atari games, with error rates as 10% and 30% respectively.
Automatic labeling of cortical sulci using patch- or CNN-based segmentation techniques combined with bottom-up geometric constraints

The extreme variability of the folding pattern of the human cortex makes the recognition of cortical sulci, both automatic and manual, particularly challenging. Reliable identification of the human cortical sulci in its entirety, is extremely difficult and is practiced by only a few experts. Moreover, these sulci correspond to more than a hundred different structures, which makes manual labeling long and fastidious and therefore limits access to large labeled databases to train machine learning. Here, we seek to improve the current model proposed in the Morphologist toolbox, a widely used sulcus recognition toolbox included in the BrainVISA package. Two novel approaches are proposed: patch-based multi-atlas segmentation (MAS) techniques and convolutional neural network (CNN)-based approaches. Both are currently applied for anatomical segmentations because they embed much better representations of inter-subject variability than approaches based on a single template atlas. However, these methods typically focus on voxel-wise labeling, disregarding certain geometrical and topological properties of interest for sulcus morphometry. Therefore, we propose to refine these approaches with domain specific bottom-up geometric constraints provided by the Morphologist toolbox. These constraints are utilized to provide a single sulcus label to each topologically elementary fold, the building blocks of the pattern recognition problem. To eliminate the shortcomings associated with the Morphologist's pre-segmentation into elementary folds, we complement this regularization scheme using a top-down perspective which triggers an additional cleavage of the elementary folds when required. All the newly proposed models outperform the current Morphologist model, the most efficient being a CNN U-Net-based approach which carries out sulcus recognition within a few seconds.
Fast state tomography with optimal error bounds

Abstract Projected least squares is an intuitive and numerically cheap technique for quantum state tomography: compute the least-squares estimator and project it onto the space of states. The main result of this paper equips this point estimator with rigorous, non-asymptotic convergence guarantees expressed in terms of the trace distance. The estimator’s sample complexity is comparable to the strongest convergence guarantees available in the literature and—in the case of the uniform POVM—saturates fundamental lower bounds. Numerical simulations support these competitive features.
Recent Advances of Image Steganography With Generative Adversarial Networks

In the past few years, the Generative Adversarial Network (GAN), which proposed in 2014, has achieved great success. There have been increasing research achievements based on GAN in the field of computer vision and natural language processing. Image steganography is an information security technique aiming at hiding secret messages in common digital images for covert communication. Recently, research on image steganography has demonstrated great potential by introducing GAN and other neural network techniques. In this paper, we review the art of steganography with GANs according to the different strategies in data hiding, which are cover modification, cover selection, and cover synthesis. We discuss the characteristics of the three strategies of GAN-based steganography and analyze their evaluation metrics. Finally, some existing problems of image steganography with GAN are summarized and discussed. Potential future research topics are also forecasted.
STA-CNN: Convolutional Spatial-Temporal Attention Learning for Action Recognition

Convolutional Neural Networks have achieved excellent successes for object recognition in still images. However, the improvement of Convolutional Neural Networks over the traditional methods for recognizing actions in videos is not so significant, because the raw videos usually have much more redundant or irrelevant information than still images. In this paper, we propose a Spatial-Temporal Attentive Convolutional Neural Network (STA-CNN) which selects the discriminative temporal segments and focuses on the informative spatial regions automatically. The STA-CNN model incorporates a Temporal Attention Mechanism and a Spatial Attention Mechanism into a unified convolutional network to recognize actions in videos. The novel Temporal Attention Mechanism automatically mines the discriminative temporal segments from long and noisy videos. The Spatial Attention Mechanism firstly exploits the instantaneous motion information in optical flow features to locate the motion salient regions and it is then trained by an auxiliary classification loss with a Global Average Pooling layer to focus on the discriminative non-motion regions in the video frame. The STA-CNN model achieves the state-of-the-art performance on two of the most challenging datasets, UCF-101 (95.8%) and HMDB-51 (71.5%).
Cross-modality paired-images generation and augmentation for RGB-infrared person re-identification

RGB-Infrared (IR) person re-identification is very challenging due to the large cross-modality variations between RGB and IR images. Considering no correspondence labels between every pair of RGB and IR images, most methods try to alleviate the variations with set-level alignment by reducing marginal distribution divergence between the entire RGB and IR sets. However, this set-level alignment strategy may lead to misalignment of some instances, which limit the performance for RGB-IR Re-ID. Different from existing methods, in this paper, we propose to generate cross-modality paired-images and perform both global set-level and fine-grained instance-level alignments. Our proposed method enjoys several merits. First, our method can perform set-level alignment by disentangling modality-specific and modality-invariant features. Compared with conventional methods, ours can explicitly remove the modality-specific features and the modality variation can be better reduced. Second, given cross-modality unpaired-images of a person, our method can generate cross-modality paired images from exchanged features. With them, we can directly perform instance-level alignment by minimizing distances of every pair of images. Third, our method learns a latent manifold space. In the space, we can random sample and generate lots of images of unseen classes. Training with those images, the learned identity feature space is more smooth can generalize better when test. Finally, extensive experimental results on two standard benchmarks demonstrate that the proposed model favorably against state-of-the-art methods.
When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition With Limited Data

We present a new deep dictionary learning and coding network (DDLCN) for image-recognition tasks with limited data. The proposed DDLCN has most of the standard deep learning layers (e.g., input/output, pooling, and fully connected), but the fundamental convolutional layers are replaced by our proposed compound dictionary learning and coding layers. The dictionary learning learns an overcomplete dictionary for input training data. At the deep coding layer, a locality constraint is added to guarantee that the activated dictionary bases are close to each other. Then, the activated dictionary atoms are assembled and passed to the compound dictionary learning and coding layers. In this way, the activated atoms in the first layer can be represented by the deeper atoms in the second dictionary. Intuitively, the second dictionary is designed to learn the fine-grained components shared among the input dictionary atoms; thus, a more informative and discriminative low-level representation of the dictionary atoms can be obtained. We empirically compare DDLCN with several leading dictionary learning methods and deep learning models. Experimental results on five popular data sets show that DDLCN achieves competitive results compared with state-of-the-art methods when the training data are limited. Code is available at https://github.com/Ha0Tang/DDLCN.
SimPropNet: Improved Similarity Propagation for Few-shot Image Segmentation

Few-shot segmentation (FSS) methods perform image segmentation for a particular object class in a target (query) image, using a small set of (support) image-mask pairs. Recent deep neural network based FSS methods leverage high-dimensional feature similarity between the foreground features of the support images and the query image features. In this work, we demonstrate gaps in the utilization of this similarity information in existing methods, and present a framework - SimPropNet, to bridge those gaps. We propose to jointly predict the support and query masks to force the support features to share characteristics with the query features. We also propose to utilize similarities in the background regions of the query and support images using a novel foreground-background attentive fusion mechanism. Our method achieves state-of-the-art results for one-shot and five-shot segmentation on the PASCAL-5i dataset. The paper includes detailed analysis and ablation studies for the proposed improvements and quantitative comparisons with contemporary methods.
DACE: Distribution-Aware Counterfactual Explanation by Mixed-Integer Linear Optimization

Counterfactual Explanation (CE) is one of the post-hoc explanation methods that provides a perturbation vector so as to alter the prediction result obtained from a classifier. Users can directly interpret the perturbation as an "action" for obtaining their desired decision results. However, an action extracted by existing methods often becomes unrealistic for users because they do not adequately care about the characteristics corresponding to the empirical data distribution such as feature-correlations and outlier risk. To suggest an executable action for users, we propose a new framework of CE for extracting an action by evaluating its reality on the empirical data distribution. The key idea of our proposed method is to define a new cost function based on the Mahalanobis' distance and the local outlier factor. Then, we propose a mixed-integer linear optimization approach to extracting an optimal action by minimizing our cost function. By experiments on real datasets, we confirm the effectiveness of our method in comparison with existing methods for CE.
Paraphrase Augmented Task-Oriented Dialog Generation

Neural generative models have achieved promising performance on dialog generation tasks if given a huge data set.However, the lack of high-quality dialog data and the expensive data annotation process greatly limit their application in real-world settings.We propose a paraphrase augmented response generation (PARG) framework that jointly trains a paraphrase model and a response generation model to improve the dialog generation performance.We also design a method to automatically construct paraphrase training data set based on dialog state and dialog act labels.PARG is applicable to various dialog generation models, such as TSCP (Lei et al., 2018) and DAMD (Zhang et al., 2019).Experimental results show that the proposed framework improves these state-of-the-art dialog models further on CamRest676 and MultiWOZ.PARG also significantly outperforms other data augmentation methods in dialog generation tasks, especially under low resource settings. 1 2
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models

Recent models for unsupervised representation learning of text have employed a number of techniques to improve contextual word representations but have put little focus on discourse-level representations.We propose CONPONO 1 , an inter-sentence objective for pretraining language models that models discourse coherence and the distance between sentences.Given an anchor sentence, our model is trained to predict the text k sentences away using a sampled-softmax objective where the candidates consist of neighboring sentences and sentences randomly sampled from the corpus.On the discourse representation benchmark DiscoEval, our model improves over the previous state-of-the-art by up to 13% and on average 4% absolute across 7 tasks.Our model is the same size as BERT-Base, but outperforms the much larger BERT-Large model and other more recent approaches that incorporate discourse.We also show that CONPONO yields gains of 2%-6% absolute even for tasks that do not explicitly evaluate discourse: textual entailment (RTE), common sense reasoning (COPA) and reading comprehension (ReCoRD).
Path planning of multiple UAVs with online changing tasks by an ORPFOA algorithm

The unmanned aerial vehicle (UAV) is a new type oilfield inspection tool which is characterized by high flexibility, low cost and high efficiency. In the UAV based oilfield inspection technology, the path planning is an indispensable element which finds an optimal flight path for UAV to finish the inspection jobs successfully. In comparison with the other researches, our study focuses on two challenging issues: path planning of multiple UAVs by traversing a certain amount of task points in the three-dimensional environment within the required completion time, and optimizing solving for the best flight path with online changing tasks. In the research, a novel task assignment method including the initial task assignment and the task assignment with changing tasks is proposed to determine the initial task sequences of each UAV and rapidly replan task sequences after tasks change. An improved fruit fly optimization algorithm (named ORPFOA) is proposed to solve the path planning problem in both initial task sequences and new task sequences after tasks change, in which the optimal reference point and a distance cost matrix are used to reach both faster solving and higher optimizing precision for the optimal flight path. In ORPFOA, two cost functions are defined to evaluate the optimizing results in the initial phase and the new phase after task changes, respectively. A simulation model of the three-dimensional oilfield environment is established to verify the effectiveness of the proposed method in comparison with other six algorithms.
A holistic representation guided attention network for scene text recognition

Reading irregular scene text of arbitrary shape in natural images is still a challenging problem, despite the progress made recently. Many existing approaches incorporate sophisticated network structures to handle various shapes, use extra annotations for stronger supervision, or employ hard-to-train recurrent neural networks for sequence modeling. In this work, we propose a simple yet strong approach for scene text recognition. With no need to convert input images to sequence representations, we directly connect two-dimensional CNN features to an attention-based sequence decoder which guided by holistic representation. The holistic representation can guide the attention-based decoder focus on more accurate area. As no recurrent module is adopted, our model can be trained in parallel. It achieves 1.5× to 9.4× acceleration to backward pass and 1.3× to 7.9× acceleration to forward pass, compared with the RNN counterparts. The proposed model is trained with only word-level annotations. With this simple design, our method achieves state-of-the-art or competitive recognition performance on the evaluated regular and irregular scene text benchmark datasets.
A consensus-based global optimization method for high dimensional machine learning problems

We improve recently introduced consensus-based optimization method, proposed in [R. Pinnau, C. Totzeck, O. Tse, S. Martin, Math. Models Methods Appl. Sci. 27 (2017) 183–204], which is a gradient-free optimization method for general non-convex functions. We first replace the isotropic geometric Brownian motion by the component-wise one, thus removing the dimensionality dependence of the drift rate, making the method more competitive for high dimensional optimization problems. Secondly, we utilize the random mini-batch ideas to reduce the computational cost of calculating the weighted average which the individual particles tend to relax toward. For its mean-field limit – a nonlinear Fokker-Planck equation – we prove, in both time continuous and semi-discrete settings, that the convergence of the method, which is exponential in time, is guaranteed with parameter constraints independent of the dimensionality. We also conduct numerical tests to high dimensional problems to check the success rate of the method.
Automatic Hate Speech Detection using Machine Learning: A Comparative Study

The increasing use of social media and information sharing has given major benefits to humanity.However, this has also given rise to a variety of challenges including the spreading and sharing of hate speech messages.Thus, to solve this emerging issue in social media sites, recent studies employed a variety of feature engineering techniques and machine learning algorithms to automatically detect the hate speech messages on different datasets.However, to the best of our knowledge, there is no study to compare the variety of feature engineering techniques and machine learning algorithms to evaluate which feature engineering technique and machine learning algorithm outperform on a standard publicly available dataset.Hence, the aim of this paper is to compare the performance of three feature engineering techniques and eight machine learning algorithms to evaluate their performance on a publicly available dataset having three distinct classes.The experimental results showed that the bigram features when used with the support vector machine algorithm best performed with 79% off overall accuracy.Our study holds practical implication and can be used as a baseline study in the area of detecting automatic hate speech messages.Moreover, the output of different comparisons will be used as state-of-art techniques to compare future researches for existing automated text classification techniques.
Engineering Relaxation Pathways in Building Blocks of Artificial Spin Ice for Computation

Nanomagnetic logic is promising for low-power computing, and for integration of data processing and memory in the same architecture. In this work the thermal relaxation paths associated with logic operations in artificial spin ice are considered, which involve switching of individual nanomagnets toward a low-energy state. Both monotonic and intermittent paths have been identified, which lend themselves to deterministic and probabilistic computing respectively. Furthermore, the balance of competing relaxation paths can be shifted, which is essential for implementing probabilistic computation---for example, in artificial neural networks where the outcome can be tuned via a feedback loop.
FedED: Federated Learning via Ensemble Distillation for Medical Relation Extraction

Unlike other domains, medical texts are inevitably accompanied by private information, so sharing or copying these texts is strictly restricted.However, training a medical relation extraction model requires collecting these privacy-sensitive texts and storing them on one machine, which comes in conflict with privacy protection.In this paper, we propose a privacypreserving medical relation extraction model based on federated learning, which enables training a central model with no single piece of private local data being shared or exchanged.Though federated learning has distinct advantages in privacy protection, it suffers from the communication bottleneck, which is mainly caused by the need to upload cumbersome local parameters.To overcome this bottleneck, we leverage a strategy based on knowledge distillation.Such a strategy uses the uploaded predictions of ensemble local models to train the central model without requiring uploading local parameters.Experiments on three publicly available medical relation extraction datasets demonstrate the effectiveness of our method.
Automatic Image and Video Caption Generation With Deep Learning: A Concise Review and Algorithmic Overlap

Methodologies that utilize Deep Learning offer great potential for applications that automatically attempt to generate captions or descriptions about images and video frames. Image and video captioning are considered to be intellectually challenging problems in imaging science. The application domains include automatic caption (or description) generation for images and videos for people who suffer from various degrees of visual impairment; the automatic creation of metadata for images and videos (indexing) for use by search engines; general-purpose robot vision systems; and many others. Each of these application domains can positively and significantly impact many other task-specific applications. This article is not meant to be a comprehensive review of image captioning; rather, it is a concise review of both image captioning and video captioning methodologies based on deep learning. This study treats both image and video captioning by emphasizing the algorithmic overlap between the two.
Efficient Policy-Hiding and Large Universe Attribute-Based Encryption With Public Traceability for Internet of Medical Things

Modern day medical systems are closely integrated and interconnected with other systems, such as those comprising Internet-of-Medical Things (IoMT) devices that facilitate remote healthcare services, say during pandemics (e.g., COVID-19). Attribute-based encryption (ABE) is a promising cryptographic primitive to support fine-grained access control in the ciphertext environment; in other words, ABE can potentially be used to ensure data confidentiality and user privacy in the IoMT ecosystem. In this article, we propose an efficient partially-policy-hidden and large universe ABE scheme with public traceability to construct a practical IoMT system (hereafter referred to as PTIoMT). The system is designed to achieve the following features: 1) the access policy is partially hidden: only nonsensitive attribute labels/names are displayed, while sensitive attribute values are hidden in the encrypted electronic health records (EHRs); 2) the number of the attributes is independent of the public parameters and, thus, can be arbitrarily large; 3) any user who discloses the decryption key can be efficiently tracked; and 4) fewer bilinear pairing operations are required during the decryption process. The security analysis and performance evaluation demonstrate the security and efficiency of PTIoMT.