Reviewerly/exHarmony huggingface dataset:
Three-Photon Discrete-Energy-Entangled <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:mi>W</mml:mi></mml:mrow></mml:math> State in an Optical Fiber

We experimentally demonstrate the generation of a three-photon discrete-energy-entangled $W$ state using multiphoton-pair generation by spontaneous four-wave mixing in an optical fiber. We show that, by making use of prior information on the photon source, we can verify the state produced by this source without resorting to frequency conversion.
Enhanced List-Based Simulated Annealing Algorithm for Large-Scale Traveling Salesman Problem

List-based simulated annealing (LBSA) algorithm is a novel simulated annealing algorithm where list-based cooling scheme is used to control the change of parameter temperature. Aiming to improve the efficiency of the LBSA algorithm for large-scale optimization problems, this paper proposes an enhanced LBSA (ELBSA) algorithm for solving large-scale traveling salesman problem (TSP). The ELBSA algorithm can drive more sampling at more suitable temperatures and from more promising neighborhoods. Specifically, heuristic augmented sampling strategy is used to ensure that more neighbors are from promising neighborhoods, systematic selection strategy is proposed to guarantee that each component of the current solution has a chance to be improved, and variable Markov chain length (VMCL), based on arithmetic sequence, is used to sample more neighbors at more suitable temperatures. Extensive experiments were performed to show the contribution of the heuristic augmented sampling strategy, and to verify the advantage of using systematic selection and VMCL. Comparative experiments, which were conducted on a wide range of large-scale TSP instances, show that the ELBSA algorithm is better than or competitive with most other state-of-the-art metaheuristics.
Incremental Learning to Personalize Human Activity Recognition Models: The Importance of Human AI Collaboration

This study presents incremental learning based methods to personalize human activity recognition models. Initially, a user-independent model is used in the recognition process. When a new user starts to use the human activity recognition application, personal streaming data can be gathered. Of course, this data does not have labels. However, there are three different ways to obtain this data: non-supervised, semi-supervised, and supervised. The non-supervised approach relies purely on predicted labels, the supervised approach uses only human intelligence to label the data, and the proposed method for semi-supervised learning is a combination of these two: It uses artificial intelligence (AI) in most cases to label the data but in uncertain cases it relies on human intelligence. After labels are obtained, the personalization process continues by using the streaming data and these labels to update the incremental learning based model, which in this case is Learn++. Learn++ is an ensemble method that can use any classifier as a base classifier, and this study compares three base classifiers: linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and classification and regression tree (CART). Moreover, three datasets are used in the experiment to show how well the presented method generalizes on different datasets. The results show that personalized models are much more accurate than user-independent models. On average, the recognition rates are: 87.0% using the user-independent model, 89.1% using the non-supervised personalization approach, 94.0% using the semi-supervised personalization approach, and 96.5% using the supervised personalization approach. This means that by relying on predicted labels with high confidence, and asking the user to label only uncertain observations (6.6% of the observations when using LDA, 7.7% when using QDA, and 18.3% using CART), almost as low error rates can be achieved as by using the supervised approach, in which labeling is fully based on human intelligence.
A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case

A key element of understanding the efficacy of overparameterized neural networks is characterizing how they represent functions as the number of weights in the network approaches infinity. In this paper, we characterize the norm required to realize a function $f:\mathbb{R}^d\rightarrow\mathbb{R}$ as a single hidden-layer ReLU network with an unbounded number of units (infinite width), but where the Euclidean norm of the weights is bounded, including precisely characterizing which functions can be realized with finite norm. This was settled for univariate univariate functions in Savarese et al. (2019), where it was shown that the required norm is determined by the L1-norm of the second derivative of the function. We extend the characterization to multivariate functions (i.e., networks with d input units), relating the required norm to the L1-norm of the Radon transform of a (d+1)/2-power Laplacian of the function. This characterization allows us to show that all functions in Sobolev spaces $W^{s,1}(\mathbb{R})$, $s\geq d+1$, can be represented with bounded norm, to calculate the required norm for several specific functions, and to obtain a depth separation result. These results have important implications for understanding generalization performance and the distinction between neural networks and more traditional kernel learning.
A New Hybrid Short-Term Interval Forecasting of PV Output Power Based on EEMD-SE-RVM

The main characteristics of the photovoltaic (PV) output power are the randomness and uncertainty, such features make it not easy to establish an accurate forecasting method. The accurate short-term forecasting of PV output power has great significance for the stability, safe operation and economic dispatch of the power grid. The deterministic point forecast method ignores the randomness and volatility of PV output power. Aiming at overcoming those defects, this paper proposes a novel hybrid model for short-term PV output power interval forecasting based on ensemble empirical mode decomposition (EEMD) as well as relevance vector machine (RVM). Firstly, the EEMD is used to decompose the PV output power sequences into several intrinsic mode functions (IMFs) and residual (RES) components. After that, based on the decomposed components, the sample entropy (SE) algorithm is utilized to reconstruct those components where three new components with typical characteristics are obtained. Then, by implementing RVM, the forecasting model for every component is developed. Finally, the forecasting results of every new component are superimposed in order to achieve the overall forecasting results with certain confidence level. Simulation results demonstrate, by comparing them with some previous methods, that the hybrid method based on EEMD-SE-RVM has relatively higher forecasting accuracy, more reliable forecasting interval and high engineering application value.
Understanding Knowledge Distillation in Non-autoregressive Machine Translation

Non-autoregressive machine translation (NAT) systems predict a sequence of output tokens in parallel, achieving substantial improvements in generation speed compared to autoregressive models. Existing NAT models usually rely on the technique of knowledge distillation, which creates the training data from a pretrained autoregressive model for better performance. Knowledge distillation is empirically useful, leading to large gains in accuracy for NAT models, but the reason for this success has, as of yet, been unclear. In this paper, we first design systematic experiments to investigate why knowledge distillation is crucial to NAT training. We find that knowledge distillation can reduce the complexity of data sets and help NAT to model the variations in the output data. Furthermore, a strong correlation is observed between the capacity of an NAT model and the optimal complexity of the distilled data for the best translation quality. Based on these findings, we further propose several approaches that can alter the complexity of data sets to improve the performance of NAT models. We achieve the state-of-the-art performance for the NAT-based models, and close the gap with the autoregressive baseline on WMT14 En-De benchmark.
Gradient-Based Neural DAG Learning

We propose a novel score-based approach to learning a directed acyclic graph (DAG) from observational data. We adapt a recently proposed continuous constrained optimization formulation to allow for nonlinear relationships between variables using neural networks. This extension allows to model complex interactions while avoiding the combinatorial nature of the problem. In addition to comparing our method to existing continuous optimization methods, we provide missing empirical comparisons to nonlinear greedy search methods. On both synthetic and real-world data sets, this new method outperforms current continuous methods on most tasks, while being competitive with existing greedy search methods on important metrics for causal inference.
Cloud Manufacturing Architecture Based on Public Blockchain Technology

With Industry 4.0, IT infrastructure has started to be used more effectively in the manufacturing sector. Cyber physical systems, IoT, cloud manufacturing, big data are some of the technologies that make up the concept of Industry 4.0. These technologies have solved many problems in the manufacturing sector. One of these technologies, cloud manufacturing technology, has emerged with the idea of pay as you go. This technology has enabled manufacturing resources to be leased and shared on a global scale. However, it has problems arising from its central structure and the need for a reliable 3rd party. Reliability, security, continuity, scalability, data lock-in, single point failure, data manipulation are some of the main problems. Blockchain (BC) is a decentralized and distributed technology. The data stored on the BC network cannot be altered in any way. With these features, we believe that BC-supported cloud manufacturing systems can overcome the aforementioned problems and eliminates the need for a reliable 3rd party. Based on this belief, in this study the agreements and communication between the resource provider and the customer, which is one of the basic functions of cloud manufacturing platforms, are realized with a decentralized application using BC-based smart contracts (SCs). The designed application is called the decentralized cloud manufacturing application (DCMApp). DCMApp does not operate on a fully public BC network, it has a hybrid structure and uses the Ethereum network as a public BC network. These features make DCMApp different from other BC-based cloud manufacturing applications. DCMApp's hybrid structure has enabled more transparent, economic and safe manufacturing agreements. It is also possible to store agreements on the BC network at a low cost without installing any server infrastructure. The use of Ethereum network makes it almost impossible to manipulate agreements.
Analysis of watermarking framework for color image through a neural network-based approach

Abstract In the research presented here, the general idea of watermarking framework is analyzed to deal with color image under a set of attacks through a neural network-based approach. It is realized in the area of transformation, especially with a focus on contourlet transform to address the proposed technique, as long as the bands of the suitable coefficients are accurately chosen. In summary, there is the logo information that is embedded in the edge of color image, while the Zenzo edge detector is correspondingly realized to handle the approach. In fact, the edge of the second subband is acquired, and subsequently, the capability of the above-referenced edge is calculated. A number of techniques are discussed to cope with the above-captioned watermarking framework through the new integration of contourlet transform in association with the multilayer perceptron to extract the logo information, appropriately. The approaches of the embedding and the de-embedding in case of learning algorithm of the aforementioned neural network through individual training data set are considered in the present research to carry out a series of experiments with different scenarios for the purpose of verifying the effectiveness of the proposed approach, obviously.
A Survey on Stylometric Text Features

Ways of individual style expression in a natural language include amongst other things stylometric features. These can be automatically detected with the use of computational linguistics methods. In this survey we systematize the recent studies devoted to extraction and application of stylometric features in solving natural language processing tasks: authorship attribution, authorship verification, style change detection, authorship profiling, and text classification by genre and sentiment. For that purpose we define stylometric feature categories that provide for the most effective solutions, discuss reasons for their successful application, touch upon the limitations of approaches based on their application, and make suggestions for future research.
Challenges in Scaling-up the Control Interface of a Quantum Computer

Challenges at the quantum-classical interface are examined with the goal of architecting a scaled-up quantum computer comprising many thousands of qubits. Separating the distinct sub-systems of the interface that perform readout and control, general arguments are given for why distributing the components of these sub-systems over significant distances and across large temperature gradients presents a major challenge to scaling-up the technology. Largely addressing these issues, an architecture for the interface that leverages cryo-CMOS circuits proximal to the quantum plane is motivated, leveraging protocols that enable massively-parallel readout of qubits via frequency multiplexing.
Learning with Differentiable Perturbed Optimizers

Machine learning pipelines often rely on optimization procedures to make discrete decisions (e.g., sorting, picking closest neighbors, or shortest paths). Although these discrete decisions are easily computed, they break the back-propagation of computational graphs. In order to expand the scope of learning problems that can be solved in an end-to-end fashion, we propose a systematic method to transform optimizers into operations that are differentiable and never locally constant. Our approach relies on stochastically perturbed optimizers, and can be used readily together with existing solvers. Their derivatives can be evaluated efficiently, and smoothness tuned via the chosen noise amplitude. We also show how this framework can be connected to a family of losses developed in structured prediction, and give theoretical guarantees for their use in learning tasks. We demonstrate experimentally the performance of our approach on various tasks.
Automatic generation of high-performance quantized machine learning kernels

Quantization optimizes machine learning inference for resource constrained environments by reducing the precision of its computation. In the extreme, even single-bit computations can produce acceptable results at dramatically lower cost. But this ultra-low-precision quantization is difficult to exploit because extracting optimal performance requires hand-tuning both high-level scheduling decisions and low-level implementations. As a result, practitioners settle for a few predefined quantized kernels, sacrificing optimality and restricting their ability to adapt to new hardware.
Sim-to-Real Quadrotor Landing via Sequential Deep Q-Networks and Domain Randomization

The autonomous landing of an Unmanned Aerial Vehicle (UAV) on a marker is one of the most challenging problems in robotics. Many solutions have been proposed, with the best results achieved via customized geometric features and external sensors. This paper discusses for the first time the use of deep reinforcement learning as an end-to-end learning paradigm to find a policy for UAVs autonomous landing. Our method is based on a divide-and-conquer paradigm that splits a task into sequential sub-tasks, each one assigned to a Deep Q-Network (DQN), hence the name Sequential Deep Q-Network (SDQN). Each DQN in an SDQN is activated by an internal trigger, and it represents a component of a high-level control policy, which can navigate the UAV towards the marker. Different technical solutions have been implemented, for example combining vanilla and double DQNs, and the introduction of a partitioned buffer replay to address the problem of sample efficiency. One of the main contributions of this work consists in showing how an SDQN trained in a simulator via domain randomization, can effectively generalize to real-world scenarios of increasing complexity. The performance of SDQNs is comparable with a state-of-the-art algorithm and human pilots while being quantitatively better in noisy conditions.
Enhancing time-delay suppression in a semiconductor laser with chaotic optical injection via parameter mismatch

Time-delay signature (TDS) suppression of an external-cavity semiconductor laser (ECSL) is important for chaos-based applications and has been widely studied in the literature. In this paper, the chaotic output of an ECSL is injected into a semiconductor laser and TDS suppression in the regenerated time series is revisited. The focus of the current work is the influence of parameter mismatch on the TDS evolution, which is investigated experimentally and compared systematically to simulations. The experimental results demonstrate that it is much easier to achieve desired TDS suppression in the configuration composed of mismatched laser pairs. Numerical simulations confirm the validity of the experimental results. In the experiments and simulations, the influence of the injection parameters on TDS suppression is also studied and good agreement is obtained.
Real-Time Recognition of Facial Expressions Using Facial Electromyograms Recorded Around the Eyes for Social Virtual Reality Applications

Recent developments of social virtual reality (VR) services using avatars have increased the need for facial expression recognition (FER) technology. FER systems are generally implemented using optical cameras; however, the performance of these systems can be limited when users are wearing head-mounted displays (HMDs) as users' faces are largely covered by the HMDs. Facial electromyograms (fEMGs) that can be recorded around users' eyes can be potentially used for implementing FER systems for VR applications. However, this technology lacks practicality owing to the need for large-scale training datasets; furthermore, it is hampered by a relatively low performance. In this study, we proposed an fEMG-based FER system based on the Riemannian manifold-based approach to reduce the number of training datasets needed and enhance FER performance. Our experiments with 42 participants showed an average classification accuracy as high as 85.01% for recognizing 11 facial expressions with only a single training dataset for each expression. We further developed an online FER system that could animate a virtual avatar's expression reflecting a user's facial expression in real time, thus demonstrating that our FER system can be potentially used for practical interactive VR applications, such as social VR networks, smart education, and virtual training.
Training Restricted Boltzmann Machines With a D-Wave Quantum Annealer

Restricted Boltzmann Machine (RBM) is an energy based, undirected graphical model. It is commonly used for unsupervised and supervised machine learning. Typically, RBM is trained using contrastive divergence (CD). However, training with CD is slow and does not estimate exact gradient of log-likelihood cost function. In this work, the model expectation of gradient learning for RBM has been calculated using a quantum annealer (D-Wave 2000Q), which is much faster than Markov chain Monte Carlo (MCMC) used in CD. Training and classification results are compared with CD. The classification accuracy results indicate similar performance of both methods. Image reconstruction as well as log-likelihood calculations are used to compare the performance of quantum and classical algorithms for RBM training. It is shown that the samples obtained from quantum annealer can be used to train a RBM on a 64-bit `bars and stripes' data set with classification performance similar to a RBM trained with CD. Though training based on CD showed improved learning performance, training using a quantum annealer eliminates computationally expensive MCMC steps of CD.
Contextual Sentiment Neural Network for Document Sentiment Analysis

Abstract Although deep neural networks are excellent for text sentiment analysis, their applications in real-world practice are occasionally limited owing to their black-box property. In this study, we propose a novel neural network model called contextual sentiment neural network (CSNN) model that can explain the process of its sentiment analysis prediction in a way that humans find natural and agreeable and can catch up the summary of the contents. The CSNN has the following interpretable layers: the word-level original sentiment layer, word-level sentiment shift layer, word-level global importance layer, word-level contextual sentiment layer, and concept-level contextual sentiment layer. Because of these layers, this network can explain the process of its document-level sentiment analysis results in a human-like way using these layers. Realizing the interpretability of each layer in the CSNN is a crucial problem in the development of this CSNN because the general back-propagation method cannot realize such interpretability. To realize this interpretability, we propose a novel learning strategy called initialization propagation (IP) learning. Using real textual datasets, we experimentally demonstrate that the proposed IP learning is effective for improving the interpretability of each layer in CSNN. We then experimentally demonstrate that the CSNN has both the high predictability and high explanation ability.
AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos

Fig. 1: Left: Human instructions for each stage (top) are translated at the pixel level into robot instructions (bottom) via CycleGAN.Right: The robot attempts the task stage-wise, automatically resetting and retrying until the instruction classifier signals success, prompting the human to confirm via key press.Algorithmic details are provided in Algorithm 1.Abstract-Robotic reinforcement learning (RL) holds the promise of enabling robots to learn complex behaviors through experience.However, realizing this promise for long-horizon tasks in the real world requires mechanisms to reduce human burden in terms of defining the task and scaffolding the learning process.In this paper, we study how these challenges can be alleviated with an automated robotic learning framework, in which multi-stage tasks are defined simply by providing videos of a human demonstrator and then learned autonomously by the robot from raw image observations.A central challenge in imitating human videos is the difference in appearance between the human and robot, which typically requires manual correspondence.We instead take an automated approach and perform pixel-level image translation via CycleGAN to convert the human demonstration into a video of a robot, which can then be used to construct a reward function for a model-based RL algorithm.The robot then learns the task one stage at a time, automatically learning how to reset each stage to retry it multiple times without human-provided resets.This makes the learning process largely automatic, from intuitive task specification via a video to automated training with minimal human intervention.We demonstrate that our approach is capable of learning complex tasks, such as operating a coffee machine, directly from raw image observations, requiring only 20 minutes to provide human demonstrations and about 180 minutes of robot interaction.