Reviewerly/exHarmony huggingface dataset:
SigNet: An Advanced Deep Learning Framework for Radio Signal Classification

Deep learning methods achieve great success in many areas due to their powerful feature extraction capabilities and end-to-end training mechanism, and recently they are also introduced for radio signal modulation classification. In this paper, we propose a novel deep learning framework called SigNet, where a signal-to-matrix (S2M) operator is adopted to convert the original signal into a square matrix first and is co-trained with a follow-up CNN architecture for classification. This model is further accelerated by integrating 1D convolution operators, leading to the upgraded model SigNet2.0. The experiments on two signal datasets show that both SigNet and SigNet2.0 outperform a number of well-known baselines, achieving the state-of-the-art performance. Notably, they obtain significantly higher accuracy than 1D-ResNet and 2D-CNN (at most increasing 70.5\%), while much faster than LSTM (at most saving 88.0\% training time). More interestingly, our proposed models behave extremely well in few-shot learning when a small training data set is provided. They can achieve a relatively high accuracy even when 1\% training data are kept, while other baseline models may lose their effectiveness much more quickly as the datasets get smaller. Such result suggests that SigNet/SigNet2.0 could be extremely useful in the situations where labeled signal data are difficult to obtain.
Gaussian Gated Linear Networks

We propose the Gaussian Gated Linear Network (G-GLN), an extension to the recently proposed GLN family of deep neural networks. Instead of using backpropagation to learn features, GLNs have a distributed and local credit assignment mechanism based on optimizing a convex objective. This gives rise to many desirable properties including universality, data-efficient online learning, trivial interpretability and robustness to catastrophic forgetting. We extend the GLN framework from classification to multiple regression and density modelling by generalizing geometric mixing to a product of Gaussian densities. The G-GLN achieves competitive or state-of-the-art performance on several univariate and multivariate regression benchmarks, and we demonstrate its applicability to practical tasks including online contextual bandits and density estimation via denoising.
Multi-constraint UAV Fast Path Planning Based on Improved A* Algorithm

Abstract Path planning is an important part of UAV intelligent control technology. For the current UAV path planning, the A* algorithm has a large memory overhead during the planning process, and the search speed is slow. It cannot meet the path planning in a complex three-dimensional environment. The immediacy requirement and considering the constraints are less, this paper made the following improvements to the A* algorithm. The first, combining an anytime repair search framework with a weighted A* algorithm, it is possible to quickly find a feasible path during the search process. The second, aiming at the defect of relying on low heuristic function weights in the algorithm, a double ranking criterion is proposed to improve the efficiency of the algorithm approaching the optimal path in the iterative process. And third, to reduce the number of node expansions in the planning process, increase list storage constraints have been improved. Finally, the results of simulation experiments show that the improved algorithm proposed in this paper can quickly generate feasible paths, and can continuously optimize the approach to the optimal path within a specified time, which is far superior to the traditional A* algorithm in terms of planning efficiency and immediacy.
Information-theoretic Task Selection for Meta-Reinforcement Learning

In Meta-Reinforcement Learning (meta-RL) an agent is trained on a set of tasks to prepare for and learn faster in new, unseen, but related tasks. The training tasks are usually hand-crafted to be representative of the expected distribution of test tasks and hence all used in training. We show that given a set of training tasks, learning can be both faster and more effective (leading to better performance in the test tasks), if the training tasks are appropriately selected. We propose a task selection algorithm, Information-Theoretic Task Selection (ITTS), based on information theory, which optimizes the set of tasks used for training in meta-RL, irrespectively of how they are generated. The algorithm establishes which training tasks are both sufficiently relevant for the test tasks, and different enough from one another. We reproduce different meta-RL experiments from the literature and show that ITTS improves the final performance in all of them.
Adaptive Importance Sampling for Finite-Sum Optimization and Sampling with Decreasing Step-Sizes.

Reducing the variance of the gradient estimator is known to improve the convergence rate of stochastic gradient-based optimization and sampling algorithms. One way of achieving variance reduction is to design importance sampling strategies. Recently, the problem of designing such schemes was formulated as an online learning problem with bandit feedback, and algorithms with sub-linear static regret were designed. In this work, we build on this framework and propose Avare, a simple and efficient algorithm for adaptive importance sampling for finite-sum optimization and sampling with decreasing step-sizes. Under standard technical conditions, we show that Avare achieves $\mathcal{O}(T^{2/3})$ and $\mathcal{O}(T^{5/6})$ dynamic regret for SGD and SGLD respectively when run with $\mathcal{O}(1/t)$ step sizes. We achieve this dynamic regret bound by leveraging our knowledge of the dynamics defined by the algorithm, and combining ideas from online learning and variance-reduced stochastic optimization. We validate empirically the performance of our algorithm and identify settings in which it leads to significant improvements.
Rational neural networks

We consider neural networks with rational activation functions. The choice of the nonlinear activation function in deep learning architectures is crucial and heavily impacts the performance of a neural network. We establish optimal bounds in terms of network complexity and prove that rational neural networks approximate smooth functions more efficiently than ReLU networks with exponentially smaller depth. The flexibility and smoothness of rational activation functions make them an attractive alternative to ReLU, as we demonstrate with numerical experiments.
Audeo: Audio Generation for a Silent Performance Video

We present a novel system that gets as an input video frames of a musician playing the piano and generates the music for that video. Generation of music from visual cues is a challenging problem and it is not clear whether it is an attainable goal at all. Our main aim in this work is to explore the plausibility of such a transformation and to identify cues and components able to carry the association of sounds with visual events. To achieve the transformation we built a full pipeline named `\textit{Audeo}' containing three components. We first translate the video frames of the keyboard and the musician hand movements into raw mechanical musical symbolic representation Piano-Roll (Roll) for each video frame which represents the keys pressed at each time step. We then adapt the Roll to be amenable for audio synthesis by including temporal correlations. This step turns out to be critical for meaningful audio generation. As a last step, we implement Midi synthesizers to generate realistic music. \textit{Audeo} converts video to audio smoothly and clearly with only a few setup constraints. We evaluate \textit{Audeo} on `in the wild' piano performance videos and obtain that their generated music is of reasonable audio quality and can be successfully recognized with high precision by popular music identification software.
Building a Lung and Ovarian Cancer Data Warehouse

Despite the collection of vast amounts of data by the healthcare sector, effective decision-making in medical practice is still challenging. Data warehousing technology can be applied for the collection and management of clinical data from various sources to provide meaningful insights for physicians and administrators. Cancer data are extremely complicated and massive; hence, a clinical data warehouse system can provide insights into prevention, diagnosis and treatment processes through the use of online analytical processing tools for the analysis of multi-dimensional data at different granularity levels.In this study, a clinical data warehouse was developed for lung cancer data, which were kindly provided by the United States National Cancer Institute. Lung and ovarian cancer data were imported in specific formats and cleaned to remove errors and redundancies. SQL server integration services (SSIS) were used for the extract-transform-load (ETL) process.The design of the clinical data warehouse responds efficiently to all types of queries by adopting the fact constellation schema model. Various online analytical processing queries can be expressed using the proposed approach.This model succeeded in responding to complex queries, and the analysis of data is facilitated by using online analytical processing cubes and viewing multilevel data details.
Derivative of the disturbance with respect to information from quantum measurements

To study the trade-off between information and disturbance, we obtain the first and second derivatives of the disturbance with respect to information for a fundamental class of quantum measurements. We focus on measurements lying on the boundaries of the physically allowed regions in four information--disturbance planes, using the derivatives to investigate the slopes and curvatures of these boundaries and hence clarify the shapes of the allowed regions.
A detailed study about CDW-PSO, BWO and GM-CPSO methods on continuous function optimization

Gauss map based chaotic particle swarm optimization (GM-CPSO) is a state-of-the-art method involving the necessary chaotic map for PSO and has proved itself in global optimization, hybrid classifier design, etc. GM-CPSO can outperform recent techniques such as chaotic dynamic weight PSO (CDW-PSO), outclassing 20 optimization methods. However, the behavior of GM-CPSO on continuous characterized functions is unknown. As the main aim, this paper comprehensively determines the performance of GM-CPSO specifically on continuous function optimization.Black widow optimization (BWO) includes a non-stable population size that cannot be fixed. In BWO, the population can increase without any intervention during iterations, which prevents an objective comparison of the method with other methods. Thus, as the second aim, a new viewpoint on BWO population size selection is suggested for an objective comparison of the method.In various disciplines, stochastic optimization is inevitable to efficiently perform function optimization. Here, the necessary question concerns with which method the best convergence and performance can be achieved. As the third aim, we evaluate three state-of-the-art optimization methods to answer this question.To realize all of these aims, GM-CPSO is compared with CDW-PSO and BWO methods by using 10 continuous benchmark functions to perform a detailed comparison and reveal which one can achieve reliable scores on low-, middle-, and high-dimensional problems. Fitness-based comparisons, computation time analysis, and convergence-based evaluations are presented to determine the robustness of algorithms. As a result, GM-CPSO arises as the most remarkable method, especially for the middle-and high-dimensional continuous functions.
CoADNet: Collaborative Aggregation-and-Distribution Networks for Co-Salient Object Detection

Co-Salient Object Detection (CoSOD) aims at discovering salient objects that repeatedly appear in a given query group containing two or more relevant images. One challenging issue is how to effectively capture co-saliency cues by modeling and exploiting inter-image relationships. In this paper, we present an end-to-end collaborative aggregation-and-distribution network (CoADNet) to capture both salient and repetitive visual patterns from multiple images. First, we integrate saliency priors into the backbone features to suppress the redundant background information through an online intra-saliency guidance structure. After that, we design a two-stage aggregate-and-distribute architecture to explore group-wise semantic interactions and produce the co-saliency features. In the first stage, we propose a group-attentional semantic aggregation module that models inter-image relationships to generate the group-wise semantic representations. In the second stage, we propose a gated group distribution module that adaptively distributes the learned group semantics to different individuals in a dynamic gating mechanism. Finally, we develop a group consistency preserving decoder tailored for the CoSOD task, which maintains group constraints during feature decoding to predict more consistent full-resolution co-saliency maps. The proposed CoADNet is evaluated on four prevailing CoSOD benchmark datasets, which demonstrates the remarkable performance improvement over ten state-of-the-art competitors.
A Boolean Task Algebra for Reinforcement Learning

The ability to compose learned skills to solve new tasks is an important property of lifelong-learning agents. In this work, we formalise the logical composition of tasks as a Boolean algebra. This allows us to formulate new tasks in terms of the negation, disjunction and conjunction of a set of base tasks. We then show that by learning goal-oriented value functions and restricting the transition dynamics of the tasks, an agent can solve these new tasks with no further learning. We prove that by composing these value functions in specific ways, we immediately recover the optimal policies for all tasks expressible under the Boolean algebra. We verify our approach in two domains---including a high-dimensional video game environment requiring function approximation---where an agent first learns a set of base skills, and then composes them to solve a super-exponential number of new tasks.
All macroscopic quantum states are fragile and hard to prepare

We study the effect of local decoherence on arbitrary quantum states. Adapting techniques developed in quantum metrology, we show that the action of generic local noise processes --though arbitrarily small-- always yields a state whose Quantum Fisher Information (QFI) with respect to local observables is linear in system size <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>N</mml:mi></mml:math>, independent of the initial state. This implies that all macroscopic quantum states, which are characterized by a QFI that is quadratic in <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>N</mml:mi></mml:math>, are fragile under decoherence, and cannot be maintained if the system is not perfectly isolated. We also provide analytical bounds on the effective system size, and show that the effective system size scales as the inverse of the noise parameter <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>p</mml:mi></mml:math> for small <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>p</mml:mi></mml:math> for all the noise channels considered, making it increasingly difficult to generate macroscopic or even mesoscopic quantum states. In turn, we also show that the preparation of a macroscopic quantum state, with respect to a conserved quantity, requires a device whose QFI is already at least as large as the one of the desired state. Given that the preparation device itself is classical and not a perfectly isolated macroscopic quantum state, the preparation device needs to be quadratically bigger than the macroscopic target state.
Walsh-Hadamard Variational Inference for Bayesian Deep Learning

Over-parameterized models, such as DeepNets and ConvNets, form a class of models that are routinely adopted in a wide variety of applications, and for which Bayesian inference is desirable but extremely challenging. Variational inference offers the tools to tackle this challenge in a scalable way and with some degree of flexibility on the approximation, but for over-parameterized models this is challenging due to the over-regularization property of the variational objective. Inspired by the literature on kernel methods, and in particular on structured approximations of distributions of random matrices, this paper proposes Walsh-Hadamard Variational Inference (WHVI), which uses Walsh-Hadamard-based factorization strategies to reduce the parameterization and accelerate computations, thus avoiding over-regularization issues with the variational objective. Extensive theoretical and empirical analyses demonstrate that WHVI yields considerable speedups and model reductions compared to other techniques to carry out approximate inference for over-parameterized models, and ultimately show how advances in kernel methods can be translated into advances in approximate Bayesian inference.
Neural Path Features and Neural Path Kernel : Understanding the role of gates in deep learning

Rectified linear unit (ReLU) activations can also be thought of as \emph{gates}, which, either pass or stop their pre-activation input when they are \emph{on} (when the pre-activation input is positive) or \emph{off} (when the pre-activation input is negative) respectively. A deep neural network (DNN) with ReLU activations has many gates, and the {on/off} status of each gate changes across input examples as well as network weights. For a given input example, only a subset of gates are \emph{active}, i.e., on, and the sub-network of weights connected to these active gates is responsible for producing the output. At randomised initialisation, the active sub-network corresponding to a given input example is random. During training, as the weights are learnt, the active sub-networks are also learnt, and potentially hold very valuable information. In this paper, we analytically characterise the role of active sub-networks in deep learning. To this end, we encode the {on/off} state of the gates of a given input in a novel \emph{neural path feature} (NPF), and the weights of the DNN are encoded in a novel \emph{neural path value} (NPV). Further, we show that the output of network is indeed the inner product of NPF and NPV. The main result of the paper shows that the \emph{neural path kernel} associated with the NPF is a fundamental quantity that characterises the information stored in the gates of a DNN. We show via experiments (on MNIST and CIFAR-10) that in standard DNNs with ReLU activations NPFs are learnt during training and such learning is key for generalisation. Furthermore, NPFs and NPVs can be learnt in two separate networks and such learning also generalises well in experiments.
Knowing Right from Wrong: Should We Use More Complex Models for Automatic Short-Answer Scoring in Bahasa Indonesia?

We compare three solutions to UKARA 1.0 challenge on automated short-answer scoring: single classical, ensemble classical, and deep learning. The task is to classify given answers to two questions, whether they are right or wrong. While recent development shows increasing model complexity to push the benchmark performances, they tend to be resource-demanding with mundane improvement. For the UKARA task, we found that bag-of-words and classical machine learning approaches can compete with ensemble models and Bi-LSTM model with pre-trained word2vec embedding from 200 million words. In this case, the single classical machine learning achieved less than 2% difference in F1 compared to the deep learning approach with 1/18 time for model training.
Bandit Samplers for Training Graph Neural Networks.

Several sampling algorithms with variance reduction have been proposed for accelerating the training of Graph Convolution Networks (GCNs). However, due to the intractable computation of optimal sampling distribution, these sampling algorithms are suboptimal for GCNs and are not applicable to more general graph neural networks (GNNs) where the message aggregator contains learned weights rather than fixed weights, such as Graph Attention Networks (GAT). The fundamental reason is that the embeddings of the neighbors or learned weights involved in the optimal sampling distribution are changing during the training and not known a priori, but only partially observed when sampled, thus making the derivation of an optimal variance reduced samplers non-trivial. In this paper, we formulate the optimization of the sampling variance as an adversary bandit problem, where the rewards are related to the node embeddings and learned weights, and can vary constantly. Thus a good sampler needs to acquire variance information about more neighbors (exploration) while at the same time optimizing the immediate sampling variance (exploit). We theoretically show that our algorithm asymptotically approaches the optimal variance within a factor of 3. We show the efficiency and effectiveness of our approach on multiple datasets.
ECLAT Algorithm for Frequent Item Set Generation with Association Rule Mining Algorithm

Eclat is a program for frequent item set mining, a data mining method that was originally developed for market basket analysis. Frequent item set mining aims at finding regularities in the shopping behavior of the customers of supermarkets, mail-order companies and online shops. In particular, it tries to identify sets of products that are frequently bought together. Once identified, such sets of associated products may be used to optimize the organization of the offered products on the shelves of a supermarket or the pages of a mail-order catalog or web shop, may give hints which products may conveniently be bundled, or may allow suggesting other products to customers. However, frequent item set mining may be used for a much wider variety of tasks, which share that one is interested in finding regularities between (nominal) variables in a given data set. For an overview of frequent item set mining in general and several specific algorithms (including Eclat)
Grey Wolf Optimizer with Crossover and Opposition-Based Learning

Singh, Shitu Chand Bansal, JagdishGrey wolf optimizer (GWO) is a relatively new optimizer in the field of swarm intelligence. It is based on the leadership hierarchy and hunting behavior of grey wolves in nature. Due to less number of parameters and ease of implementation, it has gained significant interest among the researchers of different fields. However, in some cases, the insufficient diversity of wolves prone it toward the local optima and degrades the performance of GWO. To resolve this issue, a new Grey wolf optimizer with crossover and opposition-based Learning (GWO-XOBL) is proposed. In the GWO-XOBL, two strategies have been introduced to get a proper balance between exploration and exploitation. First, the crossover is employed to increase the population diversity of GWO. Then, opposition-based learning is applied to improve the convergence rate and to avoid the stagnation. To investigate the effectiveness of the proposed algorithm, it is tested on 13 well-known standard benchmark problems. Statistical tests of the numerical results demonstrate that the performance of the proposed GWO-XOBL is significantly better than GWO and other compared nature-inspired algorithms.