Reviewerly/exHarmony huggingface dataset:
Neural Trust Region/Proximal Policy Optimization Attains Globally Optimal Policy

Proximal policy optimization and trust region policy optimization (PPO and TRPO) with actor and critic parametrized by neural networks achieve significant empirical success in deep reinforcement learning. However, due to nonconvexity, the global convergence of PPO and TRPO remains less understood, which separates theory from practice. In this paper, we prove that a variant of PPO and TRPO equipped with overparametrized neural networks converges to the globally optimal policy at a sublinear rate. The key to our analysis is the global convergence of infinite-dimensional mirror descent under a notion of one-point monotonicity, where the gradient and iterate are instantiated by neural networks. In particular, the desirable representation power and optimization geometry induced by the overparametrization of such neural networks allow them to accurately approximate the infinite-dimensional gradient and iterate.
Significance-Based Estimation-of-Distribution Algorithms

Estimation-of-distribution algorithms (EDAs) are randomized search heuristics that create a probabilistic model of the solution space, which is updated iteratively, based on the quality of the solutions sampled according to the model. As previous works show, this iteration-based perspective can lead to erratic updates of the model, in particular, to bit-frequencies approaching a random boundary value. In order to overcome this problem, we propose a new EDA based on the classic compact genetic algorithm (cGA) that takes into account a longer history of samples and updates its model only with respect to information which it classifies as statistically significant. We prove that this significance-based cGA (sig-cGA) optimizes the commonly regarded benchmark functions OneMax (OM), LeadingOnes, and BinVal all in quasilinear time, a result shown for no other EDA or evolutionary algorithm so far. For the recently proposed stable compact genetic algorithm - an EDA that tries to prevent erratic model updates by imposing a bias to the uniformly distributed model - we prove that it optimizes OM only in a time exponential in its hypothetical population size. Similarly, we show that the convex search algorithm cannot optimize OM in polynomial time.
Cross-Domain Transferability of Adversarial Perturbations

Adversarial examples reveal the blind spots of deep neural networks (DNNs) and represent a major concern for security-critical applications. The transferability of adversarial examples makes real-world attacks possible in black-box settings, where the attacker is forbidden to access the internal parameters of the model. The underlying assumption in most adversary generation methods, whether learning an instance-specific or an instance-agnostic perturbation, is the direct or indirect reliance on the original domain-specific data distribution. In this work, for the first time, we demonstrate the existence of domain-invariant adversaries, thereby showing common adversarial space among different datasets and models. To this end, we propose a framework capable of launching highly transferable attacks that crafts adversarial patterns to mislead networks trained on wholly different domains. For instance, an adversarial function learned on Paintings, Cartoons or Medical images can successfully perturb ImageNet samples to fool the classifier, with success rates as high as $\sim$99\% ($\ell_{\infty} \le 10$). The core of our proposed adversarial function is a generative network that is trained using a relativistic supervisory signal that enables domain-invariant perturbations. Our approach sets the new state-of-the-art for fooling rates, both under the white-box and black-box scenarios. Furthermore, despite being an instance-agnostic perturbation function, our attack outperforms the conventionally much stronger instance-specific attack methods.
Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks

High-performance Deep Neural Networks (DNNs) are increasingly deployed in many real-world applications e.g., cloud prediction APIs. Recent advances in model functionality stealing attacks via black-box access (i.e., inputs in, predictions out) threaten the business model of such applications, which require a lot of time, money, and effort to develop. Existing defenses take a passive role against stealing attacks, such as by truncating predicted information. We find such passive defenses ineffective against DNN stealing attacks. In this paper, we propose the first defense which actively perturbs predictions targeted at poisoning the training objective of the attacker. We find our defense effective across a wide range of challenging datasets and DNN model stealing attacks, and additionally outperforms existing defenses. Our defense is the first that can withstand highly accurate model stealing attacks for tens of thousands of queries, amplifying the attacker's error rate up to a factor of 85$\times$ with minimal impact on the utility for benign users.
SCALOR: Generative World Models with Scalable Object Representations

Scalability in terms of object density in a scene is a primary challenge in unsupervised sequential object-oriented representation learning. Most of the previous models have been shown to work only on scenes with a few objects. In this paper, we propose SCALOR, a probabilistic generative world model for learning SCALable Object-oriented Representation of a video. With the proposed spatially parallel attention and proposal-rejection mechanisms, SCALOR can deal with orders of magnitude larger numbers of objects compared to the previous state-of-the-art models. Additionally, we introduce a background module that allows SCALOR to model complex dynamic backgrounds as well as many foreground objects in the scene. We demonstrate that SCALOR can deal with crowded scenes containing up to a hundred objects while jointly modeling complex dynamic backgrounds. Importantly, SCALOR is the ﬁrst unsupervised object representation model shown to work for natural scenes containing several tens of moving objects.
A Survey of Similarity Measures for Collaborative Filtering-Based Recommender System

Recommendation system is one of the most valuable approaches to provide personalized services for users. It helps in finding the relevant information as per user’s interest from enormous amount of data. To achieve this, a similarity measure is used that computes the similarity between two users or items. There are multifarious methods to compute the similarity between users/items, but each method has some limitations. In this paper, we calculate the similarity between users through Pearson Correlation Coefficient (PCC), Cosine Correlation, Constrained Pearson Correlation Coefficient (CPCC), Sigmoid function-based Pearson Correlation Coefficient (SPCC), Jaccard similarity, and Minkowski distance measures (Euclidean distance and City block distance). The results show that Minkowski metric gives better result than other similarity calculation measures as it is less affected by size and sparsity of data set.
Sequential classification system for recognition of malaria infection using peripheral blood cell images

Aims Morphological recognition of red blood cells infected with malaria parasites is an important task in the laboratory practice. Nowadays, there is a lack of specific automated systems able to differentiate malaria with respect to other red blood cell inclusions. This study aims to develop a machine learning approach able to discriminate parasitised erythrocytes not only from normal, but also from other erythrocyte inclusions, such as Howell-Jolly and Pappenheimer bodies, basophilic stippling as well as platelets overlying red blood cells. Methods A total of 15 660 erythrocyte images from 87 smears were segmented using histogram thresholding and watershed techniques, which allowed the extraction of 2852 colour and texture features. Dataset was split into a training and assessment sets. Training set was used to develop the whole system, in which several classification approaches were compared with obtain the most accurate recognition. Afterwards, the recognition system was evaluated with the assessment set, performing two steps: (1) classifying each individual cell image to assess the system’s recognition ability and (2) analysing whole smears to obtain a malaria infection diagnosis. Results The selection of the best classification approach resulted in a final sequential system with an accuracy of 97.7% for the six groups of red blood cell inclusions. The ability of the system to detect patients infected with malaria showed a sensitivity and specificity of 100% and 90%, respectively. Conclusions The proposed method achieves a high diagnostic performance in the recognition of red blood cell infected with malaria, along with other frequent erythrocyte inclusions.
Extracting medical entities from social media

Accurately extracting medical entities from social media is challenging because people use informal language with different expressions for the same concept, and they also make spelling mistakes. Previous work either focused on specific diseases (e.g., depression) or drugs (e.g., opioids) or, if working with a wide-set of medical entities, only tackled individual and small-scale benchmark datasets (e.g., AskaPatient). In this work, we first demonstrated how to accurately extract a wide variety of medical entities such as symptoms, diseases, and drug names on three benchmark datasets from varied social media sources, and then also validated this approach on a large-scale Reddit dataset.
Collision Avoidance of Hexacopter UAV Based on LiDAR Data in Dynamic Environment

A reactive three-dimensional maneuver strategy for a multirotor Unmanned Aerial Vehicle (UAV) is proposed based on the collision cone approach to avoid potential collision with a single moving obstacle detected by an onboard sensor. A Light Detection And Ranging (LiDAR) system is assumed to be mounted on a hexacopter to obtain the obstacle information from the collected point clouds. The collision cone approach is enhanced to appropriately deal with the moving obstacle with the help of a Kalman filter. The filter estimates the position, velocity, and acceleration of the obstacle by using the LiDAR data as the associated measurement. The obstacle state estimate is utilized to predict the future trajectories of the moving obstacle. The collision detection and obstacle avoidance maneuver decisions are made considering the predicted trajectory of the obstacle. Numerical simulations, including a Monte Carlo campaign, are conducted to verify the performance of the proposed collision avoidance algorithm.
Performance Optimization for Blockchain-Enabled Distributed Network Function Virtualization Management and Orchestration

Distributed network function virtualization management and orchestration (NFV-MANO) offers a flexible way to manage and orchestrate diversified network services in large-scale Internet of vehicles (IoV). However, it is challenging to manage different services and resources in distributed NFV due to the difficulties of reliable message synchronization among multiple MANO systems. Recently, blockchain technology has emerged to solve the trust and security problems for the interconnections of multiple MANO systems. Moreover, multi-access edge computing (MEC) has become a prospective paradigm shift from the centralized cloud due to its advantages of completing tasks near users. In this work, we propose a blockchain-enabled distributed NFV framework to reach consensus among multiple MANO systems where the computation tasks of the blockchain are processed with MEC. The consensus procedures of MANO systems and blockchain nodes are explained in detail and the representation of the blockchain throughput is given. The blockchain throughput is the number of transactions a blockchain system can handle per second, which is an important evaluation indicator for the performance of a blockchain system. We make decisions for the primary node selection, the MANO system selection and the edge server selection for reaching consensus. Moreover, the blockchain throughput, the processing delay of computation tasks of blockchain and operational costs are jointly considered in the problem formulation. A dueling deep reinforcement learning approach is applied to solve this problem. Simulation results show the effectiveness of the proposed scheme.
ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation

We propose to train a non-autoregressive machine translation model to minimize the energy defined by a pretrained autoregressive model. In particular, we view our non-autoregressive translation system as an inference network (Tu and Gimpel, 2018) trained to minimize the autoregressive teacher energy. This contrasts with the popular approach of training a non-autoregressive model on a distilled corpus consisting of the beam-searched outputs of such a teacher model. Our approach, which we call ENGINE (ENerGy-based Inference NEtworks), achieves state-of-the-art non-autoregressive results on the IWSLT 2014 DE-EN and WMT 2016 RO-EN datasets, approaching the performance of autoregressive models.
Path Planning with Improved Artificial Potential Field Method Based on Decision Tree

Path planning is one of the key research directions in the field of mobile robots. It ensures that moving objects can reach the target point safely and without collision in a complex obstacle environment. The path planning is to search an optimal path from the starting point to the target point for the mobile robot in an environment with obstacles, according to certain evaluation criteria (such as the time, the best path, the minimum energy consumption, etc.). The path planning based on artificial potential field method has been paid more and more attention because of its advantages such as convenient calculation, simple implementation of hardware and outstanding real-time performance. However, the artificial potential field method has some limitations, such as the local minimum, the oscillation of moving objects among obstacles and so on. To solve these problems, we can introduce the idea of decision tree into the artificial potential field method for improvement. In machine learning, decision tree is usually used for classification. It is a prediction model, which represents a mapping relationship between object attributes and object values. By utilizing the advantages of decision tree in rule expression and extraction, an improved artificial potential field path planning model based on decision tree is constructed, which can realize real-time and accurate identification of current behavior and fast decision-making of next time behavior in path planning. Aiming at the dynamic path planning problem of mobile robots in indoor complex environment, based on the traditional artificial potential field method, this paper introduces the distance term into the potential field function, and proposes an improved artificial potential field method based on the idea of decision tree, to solve the local minimum, the oscillation between obstacles and concave obstacle problems. According to repulsion coefficient, deflection angle of resultant force and velocity, a reasonable classification decision is made to meet the needs of different obstacle distribution scenarios, and the effectiveness of the proposed method is verified by simulation experiments. Simulation results show that, compared with the traditional artificial potential field method, the planning time of improved algorithm is reduced by 50%, and the smoothness of path planning by the improved algorithm is increased by 43.3%.
Action Quality Assessment Using Siamese Network-Based Deep Metric Learning

Automated vision-based score estimation models can be used to provide an alternate opinion to avoid judgment bias. Existing works have learned score estimation models by regressing the video representation to ground truth score provided by judges. However, such regression-based solutions lack interpretability in terms of giving reasons for the awarded score. One solution to make the scores more explicable is to compare the given action video with a reference video, which would capture the temporal variations vis-á-vis the reference video and map those variations to the final score. In this work, we propose a new action scoring system termed as Reference Guided Regression (RGR), which comprises (1) a Deep Metric Learning Module that learns similarity between any two action videos based on their ground truth scores given by the judges, and (2) a Score Estimation Module that uses the first module to find the resemblance of a video with a reference video to give the assessment score. The proposed scoring model is tested for Olympics Diving and Gymnastic vaults and the model outperforms the existing state-of-the-art scoring models.
A Convolutional Neural Network to Perform Object Detection and Identification in Visual Large-Scale Data

In recent years, big data became a hard challenge. Analyzing big data needs a lot of speed precision combination. In this article, we describe a deep learning-based method to deal with big data with a focus on precision and speed. In our case, the data are images that are the hardest type of data to manipulate because of their complex structure that needs a lot of computation power. Besides, we will solve a hard task on images, which is object detection and identification. Thus, every object in the image will be localized and classified according to the range of classes provided by the training data set. To solve this challenge, we propose an approach based on a deep convolutional neural network (CNN). Moreover, CNN is the most used deep learning model in computer vision tasks such as image classification and object recognition because of its power in self-features extraction and provides useful techniques in the prediction of decision-making. Our approach outperforms state-of-the-art models such as R-CNN, Fast R-CNN, Faster R-CNN, and YOLO (you only look once), with 77% of mean average precision on the Pascal_voc 2007 testing data set and a speed of 16.54 FPS using an Nvidia Geforce GTX 960 GPGPU.
TrojDRL: Evaluation of Backdoor Attacks on Deep Reinforcement Learning

We present TrojDRL, a tool for exploring and evaluating backdoor attacks on deep reinforcement learning agents. TrojDRL exploits the sequential nature of deep reinforcement learning (DRL) and considers different gradations of threat models. We show that untargeted attacks on state-of-the-art actor-critic algorithms can circumvent existing defenses built on the assumption of backdoors being targeted. We evaluated TrojDRL on a broad set of DRL benchmarks and showed that the attacks require only poisoning as little as 0.025% of the training data. Compared with existing works of backdoor attacks on classification models, TrojDRL provides a first step towards understanding the vulnerability of DRL agents.
A Systematic Review on Implicit and Explicit Aspect Extraction in Sentiment Analysis

Aspect-based sentiment analysis (ABSA) is currently among the most vigorous areas in natural language processing (NLP). Individuals, private and government institutions are increasingly using media sources for decision making. In the last decade, aspect extraction has been the most essential phase of sentiment analysis (SA) to conduct an abridged sentiment classification. However, previous studies on sentiment analysis mostly focused on explicit aspects extraction with limited work on implicit aspects. To the best of our knowledge, this is the first systematic review that covers implicit, explicit, and the combination of both implicit and explicit aspect extractions. Therefore, this systematic review has been conducted to, 1) identify techniques used for extracting implicit, explicit, or both implicit and explicit aspects; 2) analyze the various evaluation metrics, data domains, and languages involved in the implicit and explicit aspect extraction in sentiment analysis from years 2008 to 2019; 3) identify the key challenges associated with the techniques based on the result of a comprehensive comparative analysis; and finally, 4) highlight the feasible opportunities for future research directions. This review can be used to assist novice and prominent researchers to understand the concept of both implicit and explicit aspect extractions in aspect-based sentiment analysis domain.
ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search

Users of Web search engines reveal their information needs through queries and clicks, making click logs a useful asset for information retrieval. However, click logs have not been publicly released for academic use, because they can be too revealing of personally or commercially sensitive information. This paper describes a click data release related to the TREC Deep Learning Track document corpus. After aggregation and filtering, including a k -anonymity requirement, we find 1.4 million of the TREC DL URLs have 18 million connections to 10 million distinct queries. Our dataset of these queries and connections to TREC documents is of similar size to proprietary datasets used in previous papers on query mining and ranking. We perform some preliminary experiments using the click data to augment the TREC DL training data, offering by comparison: 28x more queries, with 49x more connections to 4.4x more URLs in the corpus. We present a description of the dataset's generation process, characteristics, use in ranking and other potential uses.
Set Coherence: Basis-Independent Quantification of Quantum Coherence

The coherence of an individual quantum state can be meaningfully discussed only when referring to a preferred basis. This arbitrariness can, however, be lifted when considering sets of quantum states. Here we introduce the concept of set coherence for characterizing the coherence of a set of quantum systems in a basis-independent way. We construct measures for quantifying set coherence of sets of quantum states as well as quantum measurements. These measures feature an operational meaning in terms of discrimination games and capture precisely the advantage offered by a given set over incoherent ones. Along the way, we also connect the notion of set coherence to various resource-theoretic approaches recently developed for quantum systems.
Understanding and Improving Fast Adversarial Training

A recent line of work focused on making adversarial training computationally efficient for deep learning models. In particular, Wong et al. (2020) showed that $\ell_\infty$-adversarial training with fast gradient sign method (FGSM) can fail due to a phenomenon called when the model quickly loses its robustness over a single epoch of training. We show that adding a random step to FGSM, as proposed in Wong et al. (2020), does not prevent catastrophic overfitting, and that randomness is not important per se -- its main role being simply to reduce the magnitude of the perturbation. Moreover, we show that catastrophic overfitting is not inherent to deep and overparametrized networks, but can occur in a single-layer convolutional network with a few filters. In an extreme case, even a single filter can make the network highly non-linear locally, which is the main reason why FGSM training fails. Based on this observation, we propose a new regularization method, GradAlign, that prevents catastrophic overfitting by explicitly maximizing the gradient alignment inside the perturbation set and improves the quality of the FGSM solution. As a result, GradAlign allows to successfully apply FGSM training also for larger $\ell_\infty$-perturbations and reduce the gap to multi-step adversarial training. The code of our experiments is available at this https URL.