Reviewerly/exHarmony huggingface dataset:
Continual Learning Using Bayesian Neural Networks

Continual learning models allow them to learn and adapt to new changes and tasks over time. However, in continual and sequential learning scenarios, in which the models are trained using different data with various distributions, neural networks (NNs) tend to forget the previously learned knowledge. This phenomenon is often referred to as catastrophic forgetting. The catastrophic forgetting is an inevitable problem in continual learning models for dynamic environments. To address this issue, we propose a method, called continual Bayesian learning networks (CBLNs), which enables the networks to allocate additional resources to adapt to new tasks without forgetting the previously learned tasks. Using a Bayesian NN, CBLN maintains a mixture of Gaussian posterior distributions that are associated with different tasks. The proposed method tries to optimize the number of resources that are needed to learn each task and avoids an exponential increase in the number of resources that are involved in learning multiple tasks. The proposed method does not need to access the past training data and can choose suitable weights to classify the data points during the test time automatically based on an uncertainty criterion. We have evaluated the method on the MNIST and UCR time-series data sets. The evaluation results show that the method can address the catastrophic forgetting problem at a promising rate compared to the state-of-the-art models.
Understanding adversarial robustness via critical attacking route

Deep neural networks (DNNs) are vulnerable to adversarial examples which are generated by inputs with imperceptible perturbations. Understanding adversarial robustness of DNNs has become an important issue, which would for certain result in better practical deep learning applications. To address this issue, we try to explain adversarial robustness for deep models from a new perspective of critical attacking route, which is computed by a gradient-based influence propagation strategy. Similar to rumor spreading in social networks, we believe that adversarial noises are amplified and propagated through the critical attacking route. By exploiting neurons’ influences layer by layer, we compose the critical attacking route with neurons that make the highest contributions towards model decision. In this paper, we first draw the close connection between adversarial robustness and critical attacking route, as the route makes the most non-trivial contributions to model predictions in the adversarial setting. By constraining the propagation process and node behaviors on this route, we could weaken the noise propagation and improve model robustness. Also, we find that critical attacking neurons are useful to evaluate sample adversarial hardness that images with higher stimulus are easier to be perturbed into adversarial examples.
Solving the Multi-Functional Heterogeneous UAV Cooperative Mission Planning Problem Using Multi-Swarm Fruit Fly Optimization Algorithm

The complexity of unmanned aerial vehicle (UAV) missions is increasing with the rapid development of UAV technology. Multiple UAVs usually cooperate in the form of teams to improve the efficiency of mission execution. The UAVs are equipped with multiple sensors with complementary functions to adapt to the complex mission constraints. Reasonable task assignment, task scheduling, and UAV trajectory planning are the prerequisites for efficient cooperation of multi-functional heterogeneous UAVs. In this paper, a multi-swarm fruit fly optimization algorithm (MFOA) with dual strategy switching is proposed to solve the multi-functional heterogeneous UAV cooperative mission planning problem with the criterion of simultaneously minimizing the makespan and the total mission time. First, the multi-swarm mechanism is introduced to enhance the global search capability of the fruit fly optimization algorithm. Second, in the smell-based search phase, the local search strategies and large-scale search strategies are designed to drive multiple fruit fly swarms, and the dual strategy switching method is presented. Third, in the vision-based search stage, the greedy selection strategy is adopted. Finally, numerical simulation experiments are designed. The simulation results show that the MFOA algorithm is more effective and stable for solving the multi-functional heterogeneous UAV cooperative mission planning problem compared with other algorithms.
Frame-Wise Detection of Double HEVC Compression by Learning Deep Spatio-Temporal Representations in Compression Domain

Detection of double compression is regarded as one primary step in analyzing the integrity of digital videos, which is of prominent importance in video forensics. However, current methods are vulnerable with the severe lossy quantization in the recompression process such that it is challenging to obtain reliable frame-wise detection results, especially for the high efficiency video coding (HEVC) standard. In view of these issues, in this paper, a hybrid neural network is proposed to reveal abnormal frames in HEVC videos with double compression by learning robust spatio-temporal representations from coding information in the compression domain. Based on the statistical analysis of Coding Units (CUs), it is interesting to find that HEVC video streams contain “rich” coding information that could be leveraged to identify abnormal traces caused by double compression. Two types of coding information maps, including CU Size Map (CSM) and CU Prediction mode Map (CPM), are exploited. In contrast with the conventional paradigm relying on pixel-level representations of decoded frames, CSMs and CPMs of a short-time video clip are treated as the input, aiming to achieve high robustness against recompression of low quality. In our hybrid neural network, an attention-based two-stream residual network is proposed to learn hierarchical representations from CSM and CPM, which are then jointly optimized by the attention-based fusion module. Finally, the temporal variation is modeled by Long Short-Term Memory (LSTM) to obtain frame-wise detection results. We have conducted extensive experiments considering various video content and coding parameters, such as bitrates and sizes of Group of Picture. Experimental results show that our approach can obtain state-of-the-art performance compared with conventional methods, especially when videos are recompressed in the low bitrate coding scenarios.
A Privacy-Preserving-Oriented DNN Pruning and Mobile Acceleration Framework

Weight pruning of deep neural networks (DNNs) has been proposed to satisfy the limited storage and computing capability of mobile edge devices. However, previous pruning methods mainly focus on reducing the model size and/or improving performance without considering the privacy of user data. To mitigate this concern, we propose a privacy-preserving-oriented pruning and mobile acceleration framework that does not require the private training dataset. At the algorithm level of the proposed framework, a systematic weight pruning technique based on the alternating direction method of multipliers (ADMM) is designed to iteratively solve the pattern-based pruning problem for each layer with randomly generated synthetic data. In addition, corresponding optimizations at the compiler level are leveraged for inference accelerations on devices. With the proposed framework, users could avoid the time-consuming pruning process for non-experts and directly benefit from compressed models. Experimental results show that the proposed framework outperforms three state-of-art end-to-end DNN frameworks, i.e., TensorFlow-Lite, TVM, and MNN, with speedup up to 4.2×, 2.5×, and 2.0×, respectively, with almost no accuracy loss, while preserving data privacy.
Usage of Deep Learning and Blockchain in Compilation and Copyright Protection of Digital Music

In order to explore the application of deep learning algorithms in arrangement and composition, and the role of blockchain in the protection of digital music copyright, a monophonic melody composition model based on the deep generative adversarial networks (DCGANs) is constructed firstly, and the composition performance of the model is analyzed using hymn as input sample in this study. Later, the multi-instrument co-arrangement (MICA) model based on the multi-task learning is proposed, and the composition performance is analyzed by taking the actual music as an input sample. Finally, the improved practical byzantine fault tolerance (IPBFT) algorithm is proposed, and a digital music copyright protection system is designed based on the blockchain in this study. The results indicate that the accuracies constructed DCGANs model in predicting the Soprano and Alto voice melody are higher than those of the DeepBatch model by 2.29% and 3.32%, respectively. The performance on the harmony score, note accuracy, Levenshtein similarity (LS), notes distribution mean square error, and empty as well as the convergence speed of the constructed MICA model are better than those of other models. The average transaction per second (TPS) value of the proposed IPBFT algorithm in the real digital music copyright protection system is 3469, which is superior to other blockchain technologies. Finally, the digital music copyright protection system is achieved, the error rate of completing the request is 0% in the state of many users operating concurrently, and a high TPS value can be guaranteed. In short, the DCGANs and MICA models pointed out in this study can be used in the composition of monophonic melodies and complex melodies, and the digital music copyright protection system based on the blockchain has excellent performance in practical applications.
Novel hybrid algorithm for Team Orienteering Problem with Time Windows for rescue applications

Robots for rescue operations after a disaster are an interesting and challenging research problem that has the potential to save lives and reduce economic losses after a disaster. We developed TOPTWR, an extension of the popular TOPTW model, to model the issues in task allocation for teams of rescue robots. Our hybrid algorithm is based on a team of heterogeneous humanoid robots trying to optimize five objectives (task rewards, task completion time, total energy, maximum energy consumption for a single robot, and missed deadline penalties). A common approach to solve these kinds of problems are multi-objective evolutionary algorithms (MOEAs), but their major disadvantage is that they cannot deal with dynamic environments easily. This paper presents an efficient solution for TOPTWR by combining MOEAs with learning algorithms. A novel Extended Multi-Start Simulated Annealing Iterated Local Search (EMSAILS) operator using a modern state-of-the-art NSGA-III algorithm is proposed. In addition, we applied Q-Learning to learn the likely changes in the environment and how to react to them. This algorithm, HMO-TOPTWR-NSGA-III (HMO-N-L), uses an artificial neural network (ANN) as a function approximator to make the huge state and action spaces tractable. This paper includes a thorough empirical evaluation demonstrating the effectiveness of the multi-objective algorithm in both static and dynamic environments. The evaluation shows that the proposed algorithm reduces the error by up to 42% against three state-of-the-art approaches to TOPTW (HMO-N, MSA, and IPI).
Revealing Every Story of Data in Blockchain Systems

The success of Bitcoin and other cryptocurrencies bring enormous interest to blockchains. A blockchain system implements a tamper-evident ledger for recording transactions that modify some global states. The system captures the entire evolution history of the states. The management of that history, also known as data provenance or lineage, has been studied extensively in database systems. However, querying data history in existing blockchains can only be done by replaying all transactions. This approach is feasible for large-scale, offline analysis, but is not suitable for online transaction processing. We present LineageChain, a fine-grained, secure, and efficient provenance system for blockchains. LineageChain exposes provenance information to smart contracts via simple interfaces, thereby enabling a new class of blockchain applications whose execution logics depend on provenance information at runtime. LineageChain captures provenance during contract execution and stores it in a Merkle tree. LineageChain provides a novel skip list index that supports efficient provenance queries. We have implemented LineageChain on top of Hyperledger Fabric and a blockchainoptimized storage system called ForkBase. We conduct extensive evaluation, demonstrating benefits of LineageChain, its efficient querying, and its small storage overhead.
Finger Gesture Tracking for Interactive Applications

This paper presents FinGTrAC, a system that shows the feasibility of fine grained finger gesture tracking using low intrusive wearable sensor platform (smart-ring worn on the index finger and a smart-watch worn on the wrist). The key contribution is in scaling up gesture recognition to hundreds of gestures while using only a sparse wearable sensor set where prior works have been able to only detect tens of hand gestures. Such sparse sensors are convenient to wear but cannot track all fingers and hence provide under-constrained information. However application specific context can fill the gap in sparse sensing and improve the accuracy of gesture classification. Rich context exists in a number of applications such as user-interfaces, sports analytics, medical rehabilitation, sign language translation etc. This paper shows the feasibility of exploiting such context in an application of American Sign Language (ASL) translation. Noisy sensor data, variations in gesture performance across users and the inability to capture data from all fingers introduce non-trivial challenges. FinGTrAC exploits a number of opportunities in data preprocessing, filtering, pattern matching, context of an ASL sentence to systematically fuse the available sensory information into a Bayesian filtering framework. Culminating into the design of a Hidden Markov Model, a Viterbi decoding scheme is designed to detect finger gestures and the corresponding ASL sentences in real time. Extensive evaluation on 10 users shows a recognition accuracy of 94.2% for 100 most frequently used ASL finger gestures over different sentences. When the size of the dictionary is extended to 200 words, the accuracy is degrades gracefully to 90% thus indicating the robustness and scalability of the multi-stage optimization framework.
A Novel Change Detection Method for Natural Disaster Detection and Segmentation from Video Sequence

Change detection (CD) is critical for natural disaster detection, monitoring and evaluation. Video satellites, new types of satellites being launched recently, are able to record the motion change during natural disasters. This raises a new problem for traditional CD methods, as they can only detect areas with highly changed radiometric and geometric information. Optical flow-based methods are able to detect the pixel-based motion tracking at fast speed; however, they are difficult to determine an optimal threshold for separating the changed from the unchanged part for CD problems. To overcome the above problems, this paper proposed a novel automatic change detection framework: OFATS (optical flow-based adaptive thresholding segmentation). Combining the characteristics of optical flow data, a new objective function based on the ratio of maximum between-class variance and minimum within-class variance has been constructed and two key steps are motion detection based on optical flow estimation using deep learning (DL) method and changed area segmentation based on an adaptive threshold selection. Experiments are carried out using two groups of video sequences, which demonstrated that the proposed method is able to achieve high accuracy with F1 value of 0.98 and 0.94, respectively.
A Fast Clustering Algorithm for Modularization of Large-Scale Software Systems

A software system evolves over time in order to meet the needs of users. Understanding a program is the most important step to apply new requirements. Clustering techniques through dividing a program into small and meaningful parts make it possible to understand the program. In general, clustering algorithms are classified into two categories: hierarchical and non-hierarchical algorithms (such as search-based approaches). While clustering problems generally tend to be NP-hard, search-based algorithms produce acceptable clustering and have time and space constraints and hence they are inefficient in large-scale software systems. Most algorithms which currently used in software clustering fields do not scale well when applied to large and very large applications. In this paper, we present a new and fast clustering algorithm, FCA, that can overcome space and time constraints of existing algorithms by performing operations on the dependency matrix and extracting other matrices based on a set of features. The experimental results on ten small-sized applications, ten folders with different functionalities from Mozilla Firefox, a large-sized application (namely ITK), and a very large-sized application (namely Chromium) demonstrate that the proposed algorithm achieves higher quality modularization compared with hierarchical algorithms. It can also compete with search-based algorithms and a clustering algorithm based on subsystem patterns. But the running time of the proposed algorithm is much shorter than that of the hierarchical and non-hierarchical algorithms. The source code of the proposed algorithm can be accessed at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/SoftwareMaintenanceLab</uri> .
Realizing Deep High-Order TSK Fuzzy Classifier by Ensembling Interpretable Zero-Order TSK Fuzzy Subclassifiers

Although high-order Takagi–Sugeno–Kang (TSK) fuzzy systems have demonstrated their computational advantages and simultaneously circumvent the weakness that the number of rules with the number of input variables and membership functions grows exponentially in both zero-order and first-order TSK fuzzy systems for complex modeling tasks, they still face two serious issues: incapability for a changing environment and no interpretability of the coefficients in high-order polynomial used in the consequent part of each fuzzy rule. In order to circumvent these two challenges, a novel stacked architecture of an interpretable deep higher order TSK fuzzy classifier called DHO-TSK and its deep learning method are proposed by proving the equivalence between a high-order TSK fuzzy classifier and a deep ensemble of interpretable zero-order TSK fuzzy classifiers in this article. DHO-TSK can be built by assembling interpretable zero-order TSK fuzzy classifiers in a special stacked way. Each zero-order TSK fuzzy classifier can be learnt by randomly selecting input features, randomly assigning an antecedent fuzzy subset from a fixed fuzzy partition to each of the selected input features, and then multiplying the output of each TSK fuzzy classifier by a randomly selected feature. Except for the abovementioned solid theoretical equivalence, DHO-TSK is featured in the following aspects: first, the consequent part of each fuzzy rule in DHO-TSK becomes interpretable and the output expression of each layer in DHO-TSK becomes comprehensible due to the adopted stacked ensemble; second, its enhanced classification performance can be achieved in a stacked deep learning way; third, DHO-TSK has its adoptability for changing environments owing to random selection of both features and fuzzy membership functions. Our experimental results on the benchmarking UCI and KEEL datasets and a real dataset indicate the effectiveness of DHO-TSK and its learning method in the sense of both classification performance and interpretability.
Analysis of entrepreneurship education in colleges and based on improved decision tree algorithm and fuzzy mathematics

In order to improve the performance of entrepreneurship and innovation education in colleges and universities, this study attempts to build an evaluation system and model of innovation and entrepreneurship in colleges and universities to provide a complete and practical tool for government education authorities and universities to evaluate the implementation of innovation and entrepreneurship education. In this research, decision tree and fuzzy mathematics are used as the basis of the model algorithm, and the algorithm is improved based on the analysis of traditional algorithms. Moreover, based on the improved decision tree algorithm, an evaluation index system for university innovation and entrepreneurship education is constructed. After determining the evaluation indicators of innovation and entrepreneurship education in colleges and universities, this study uses several universities as examples to analyze and define the definitions of various indicators. In addition, this study statistically analyzes the results of entrepreneurship and innovation education in colleges and universities through simulation. The research shows that the model proposed in this paper has a certain practical effect, and based on the simulation results, this study makes several suggestions.
Simulation of English speech emotion recognition based on transfer learning and CNN neural network

The difference between English and Chinese expressions is that English emphasizes the stress of syllables, so the recognition of English speech emotions plays an important role in learning English. This study uses transfer learning as the technical support to study English speech emotion recognition. The acoustic model based on weight transfer has two different training strategies: single-stage training and two-stage training strategy. By comparing the performance of the English speech emotion recognition model based on CNN neural network and the model proposed in this paper, the statistical comparison data is drawn into a statistical graph. The research results show that transfer learning has certain advantages over other algorithms in English speech emotion recognition. In the subsequent teaching and real-time translation equipment research, transfer learning can be applied to English models.
Improving Image Autoencoder Embeddings with Perceptual Loss

Autoencoders are commonly trained using element-wise loss. However, element-wise loss disregards high-level structures in the image which can lead to embeddings that disregard them as well. A recent improvement to autoencoders that helps alleviate this problem is the use of perceptual loss. This work investigates perceptual loss from the perspective of encoder embeddings themselves. Autoencoders are trained to embed images from three different computer vision datasets using perceptual loss based on a pretrained model as well as pixel-wise loss. A host of different predictors are trained to perform object positioning and classification on the datasets given the embedded images as input. The two kinds of losses are evaluated by comparing how the predictors performed with embeddings from the differently trained autoencoders. The results show that, in the image domain, the embeddings generated by autoencoders trained with perceptual loss enable more accurate predictions than those trained with element-wise loss. Furthermore, the results show that, on the task of object positioning of a small-scale feature, perceptual loss can improve the results by a factor 10. The experimental setup is available online <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .