Reviewerly/exHarmony huggingface dataset:
Reweighted Block Sparsity Regularization for Remote Sensing Images Destriping

Destriping has attracted a lot of interest in the field of remote sensing images processing. Recent research works consider the characteristics of stripes, among which low-rankness and group sparsity regularizations are widely used. However, for real scenarios, these regularizations sometimes fail to work when the underlying assumptions are violated. Motivated by this observation, we propose a novel and flexible regularization, named as the reweighted block sparsity (RBS), for complex stripes (e.g., the partial stripes). RBS divides the stripe layer into several blocks along the stripe-perpendicular direction and boosts the group sparsity of each block, where the sparsity level is adaptively controlled by updating weights. Based on RBS, we further propose a destriping model by integrating RBS and unidirectional total variation regularization, which can better detect and remove complex stripes. Moreover, we develop an alternating direction method of multipliers algorithm to solve the proposed model. Experimental results demonstrate that the proposed method outperforms the state-of-the-art competitors qualitatively and quantitatively.
A Combined Forecasting Model for Satellite Network Self-Similar Traffic

Since satellite network traffic is self-similar and long-range-dependent (LRD), after analyzing current network traffic forecasting models, a satellite network traffic combined forecasting model that is based on the decomposition fruit fly optimization algorithm - extreme learning machine (FOA-ELM) is proposed. This forecasting model decomposes LRD network traffic into multiple short-range dependent (SRD) components via empirical mode decomposition (EMD), applies the FOA-ELM forecasting model to the decomposed high-frequency components, and applies the ELM forecasting model to low-frequency components. The simulation results show that the forecasting model can improve the forecasting accuracy and forecasting speed, reduce the complexity, and achieve effective and efficient forecasting of satellite network traffic.
On Correlation of Features Extracted by Deep Neural Networks

Redundancy in deep neural network (DNN) models has always been one of their most intriguing and important properties. DNNs have been shown to overparameterize, or extract a lot of redundant features. In this work, we explore the impact of size (both width and depth), activation function, and weight initialization on the susceptibility of deep neural network models to extract redundant features. To estimate the number of redundant features in each layer, all the features of a given layer are hierarchically clustered according to their relative cosine distances in feature space and a set threshold. It is shown that both network size and activation function are the two most important components that foster the tendency of DNNs to extract redundant features. The concept is illustrated using deep multilayer perceptron and convolutional neural networks on MNIST digits recognition and CIFAR-10 dataset, respectively.
Automatic Configuration of Deep Neural Networks with Parallel Efficient Global Optimization

Designing the architecture for an artificial neural network is a cumbersome task because of the numerous parameters to configure, including activation functions, layer types, and hyper-parameters. With the large number of parameters for most networks nowadays, it is intractable to find a good configuration for a given task by hand. In this paper the Mixed Integer Parallel Efficient Global Optimization (MIP-EGO) algorithm is proposed to automatically configure convolutional neural network architectures. It is shown that on several image classification tasks this approach is able to find competitive network architectures in terms of prediction accuracy, compared to the best hand-crafted ones in literature, when using only a fraction of the number of training epochs. Moreover, instead of the standard sequential evaluation in EGO, several candidate architectures are proposed and evaluated in parallel, which reduces the execution overhead significantly and leads to an efficient automation for deep neural network design.
Application Inference using Machine Learning based Side Channel Analysis

The proliferation of ubiquitous computing requires energy-efficient as well as secure operation of modern processors. Side channel attacks are becoming a critical threat to security and privacy of devices embedded in modern computing infrastructures. Unintended information leakage via physical signatures such as power consumption, electromagnetic emission (EM) and execution time have emerged as a key security consideration for SoCs. Also, information published on purpose at user privilege level accessible through software interfaces results in software-only attacks. In this paper, we used a supervised learning based approach for inferring applications executing on android platform based on features extracted from EM side-channel emissions and software exposed dynamic voltage frequency scaling (DVFS) states. We highlight the importance of machine learning based approach in utilizing these multi-dimensional features on a complex SoC, against profiling-based approaches. We also show that learning the instantaneous frequency states polled from on-board frequency driver (cpufreq) is adequate to identify a known application and flag potentially malicious unknown application. The experimental results on benchmarking applications running on ARMv8 processor in Snapdragon 820 board demonstrates early detection of these apps in 700 ms, and atleast 85% accuracy in detecting unknown applications. Overall, the highlight is to utilize a low-complexity path to application inference attacks through learning instantaneous frequency states pattern of CPU core.
An Initial Study on the Relationship Between Meta Features of Dataset and the Initialization of NNRW

The initialization of neural networks with random weights (NNRW) has a significant impact on model performance. However, there is no suitable way to solve this problem so far. In this paper, the relationship between meta features of a dataset and the initialization of NNRW is studied. Specifically, we construct seven regression datasets with known attributes' distributions, then initialize NNRW with different distributions and trained them based on the datasets to get the corresponding models respectively. The relationship between the attributes' distributions of the datasets and the initialization of NNRW is analyzed by the performance of the models. Several interesting phenomena are observed: firstly, initializing NNRW with the Gaussian distribution can help the model to have a faster convergence rate than ones with the Gamma and Uniform distribution. Secondly, if one or more attributes in a dataset that follow the Gamma distribution, using Gamma distribution to initialize NNRW may result in a slower convergence rate and easy overfitting. Thirdly, initializing NNRW with a specific distribution with smaller variances can always achieve faster convergence rate and better generalization performance than the one with larger variances. The above experimental results are not sensitive to the activation function and the type of NNRW. Some theoretical analyses about the above observations are also given in the study.
Netwulf: Interactive visualization of networks in Python

Network visualization is an effective way to illustrate properties of a complex system.It is an important tool for exploring and finding patterns, and is used by researchers and practitioners across many fields and industries.Currently, there exist a number of tools for visualizing networks.Networkx (Hagberg, Schult, & Swart, 2008) is a popular Python package for network analysis which provides limited functionality for computing layouts and plotting networks statically.Layout computations are done in Python or using the php-based software Graphviz (Ellson, Gansner, Koutsofios, North, & Woodhull, 2002), which is slow.Another Python package for network analysis and visualization is graph-tool (Peixoto, 2014), which relies on a high number of external C++-libraries for installation/compilation which can be overwhelming for beginners.Furthermore, its visualization functions have limited interactive features.Gephi (Bastian, Heymann, & Jacomy, 2009) and Cytoscape (Shannon et al., 2003) are dedicated interactive visualization and analysis software programs.They are both Javabased and run desktop clients with a GUI where users save and load networks as separate files.Webweb (Wapman & Larremore, 2019) enables interactive visualization for Python and Matlab networks using the d3.js (Bostock, Ogievetsky, & Heer, 2011) force layout.Its main purpose, however, is exploration of network features and exporting one-time visualizations as SVG or HTML.
Hyperspectral Image Denoising Using Global Weighted Tensor Norm Minimum and Nonlocal Low-Rank Approximation

A hyperspectral image (HSI) contains abundant spatial and spectral information, but it is always corrupted by various noises, especially Gaussian noise. Global correlation (GC) across spectral domain and nonlocal self-similarity (NSS) across spatial domain are two important characteristics for an HSI. To keep the integrity of the global structure and improve the details of the restored HSI, we propose a global and nonlocal weighted tensor norm minimum denoising method which jointly utilizes GC and NSS. The weighted multilinear rank is utilized to depict the GC information. To preserve structural information with NSS, a patch-group-based low-rank-tensor-approximation (LRTA) model is designed. The LRTA makes use of Tucker decompositions of 4D patches, which are composed of a similar 3D patch group of HSI. The alternating direction method of multipliers (ADMM) is adapted to solve the proposed models. Experimental results show that the proposed algorithm can preserve the structural information and outperforms several state-of-the-art denoising methods.
Two-Stream Convolution Neural Network with Video-stream for Action Recognition

Recently, as the application of the convolutional neural network in artificial intelligence is becoming increasingly diversified, a growing number of neural network methods are put forward. For example, 3D convolution and two-stream convolution method based on RGB and optical stream are applied to the neural network. Convolutional neural network with 3D convolutional core is able to extract spatio-temporal features directly from a set of video sequences, used for action recognition. Although the 3D convolutional neural network can obtain partial spatio-temporal information, a new ConvNet architecture called CVDN(Combined Video-stream Deep Network) is proposed to extract more spatio-temporal features from video fragments so as to effectively utilize the temporal information in the dataset. We evaluate our method on the UCF-101 dataset and obtain a good result. The following is some details about our method:First, we use pre-trained ResNets models on Kinetics dataset to initialize our training models, training and extracting the video stream features from UCF-101 dataset. Then, optical flow graphs obtained from the UCF-101 dataset, which are the input of the optical stream, are used to extract the optical features. At length, two-stream features are combined and the results are obtained after Softmax layer. When the linear fusion ratio of video stream features and optical stream features is 5:4, CVDN obtains good results. And the accuracy of our method with Resnet-101 achieves 92.2%.
Parallel Transfer Learning in Multi-Agent Systems: What, when and how to transfer?

Multi-agent Reinforcement Learning (RL) is frequently used in large-scale autonomous systems to learn the behaviours that best suit the system's operating environment. Learning can take a significant amount of time during which an RL system's performance is necessarily suboptimal. Transfer learning (TL), a method of reusing knowledge which has been gained in one task to improve the performance in another, has been used to speed up learning in single RL agent systems. TL requires learning on a source task to complete before transferring it to a target task, i.e., transfer is done offline. Parallel Transfer Learning (PTL), a technique which enables the source and target tasks to run concurrently, has been proposed to enable online transfers. However, the online selection of knowledge to be transferred, as well as online ways of integration of that knowledge on the receiving agents remain open issues. This paper proposes methods for selecting the knowledge to be transferred in PTL, frequency and size of transfers, and methods for knowledge integration into the target task. We evaluate the proposed approaches in two canonical RL examples: Mountain Car and Co-operative Predator Prey Pursuit. We show that PTL, similarly to RL, is highly sensitive to parameter selection and that suitable parameters differ per scenario.
Curriculum goal masking for continuous deep reinforcement learning

Deep reinforcement learning has recently gained a focus on problems where policy or value functions are based on universal value function approximators (UVFAs) which renders them independent of goals. Evidence exists that the sampling of goals has a strong effect on the learning performance, and the problem of optimizing the goal sampling is frequently tackled with intrinsic motivation methods. However, there is a lack of general mechanisms that focus on goal sampling in the context of deep reinforcement learning based on UVFAs. In this work, we introduce goal masking as a method to estimate a goal's difficulty level and to exploit this estimation to realize curriculum learning. Our results indicate that focusing on goals with a medium difficulty level is appropriate for deep deterministic policy gradient (DDPG) methods, while an “aim for the stars and reach the moon-strategy”, where difficult goals are sampled much more often than simple goals, leads to the best learning performance in cases where DDPG is combined with hindsight experience replay (HER).
On the Performance of Differential Evolution for Hyperparameter Tuning

Automated hyperparameter tuning aspires to facilitate the application of machine learning for non-experts. In the literature, different optimization approaches are applied for that purpose. This paper investigates the performance of Differential Evolution for tuning hyperparameters of supervised learning algorithms for classification tasks. This empirical study involves a range of different machine learning algorithms and datasets with various characteristics to compare the performance of Differential Evolution with Sequential Model-based Algorithm Configuration (SMAC), a reference Bayesian Optimization approach. The results indicate that Differential Evolution outperforms SMAC for most datasets when tuning a given machine learning algorithm - particularly when breaking ties in a first-to-report fashion. Only for the tightest of computational budgets SMAC performs better. On small datasets, Differential Evolution outperforms SMAC by 19% (37% after tie-breaking). In a second experiment across a range of representative datasets taken from the literature, Differential Evolution scores 15% (23% after tie-breaking) more wins than SMAC.
Assessing blockchain potential for improving the cybersecurity of financial transactions

Purpose Blockchain holds promise as a potential solution to the problem of cybersecurity in financial transactions. However, difficulty exists for both the industry and organizations in assessing this potential solution. Hence, it is important to understand how organizations in the financial sector can address these concerns by exploring blockchain implementation for financial transactions in the context of cybersecurity. To do this, the problem question is threefold: first, what objectives are important based on the strategic values of an organization for evaluating cybersecurity to improve the security of financial transactions? Second, how can they be used to ensure the cybersecurity of financial transactions in a financial organization? Third, how can these objectives be used to evaluate blockchain as a potential solution for enhancing the cybersecurity of organizations in the financial sector relative to existing cybersecurity methods? The paper aims to discuss this issue. Design/methodology/approach To accomplish this goal we utilize Keeney’s (1992) multi-objective decision analytics technique, termed value-focused thinking (VFT), to demonstrate how organizations can assess a blockchain solution’s value to maximize value-add within financial organization. Findings The presented model clearly demonstrates the viability of using Keeney’s (1992) VFT technique as a multi-criteria decision analysis tool for assessing blockchain technology. Further, a clear explanation of how this model can be extended and adapted for individual organizational use is provided. Originality/value This paper engages both the academic literature as well as an expert panel to develop an assessment model for blockchain technology related to financial transactions by providing a useful method for structuring the decision-making process of organizations around blockchain technology.
Autonomous Reinforcement Learning of Multiple Interrelated Tasks

Autonomous multiple tasks learning is a fundamental capability to develop versatile artificial agents that can act in complex environments. In real-world scenarios, tasks may be interrelated (or “hierarchical”) so that a robot has to first learn to achieve some of them to set the preconditions for learning other ones. Even though different strategies have been used in robotics to tackle the acquisition of interrelated tasks, in particular within the developmental robotics framework, autonomous learning in this kind of scenarios is still an open question. Building on previous research in the framework of intrinsically motivated open-ended learning, in this work we describe how this question can be addressed working on the level of task selection, in particular considering the multiple interrelated tasks scenario as an MDP where the system is trying to maximise its competence over all the tasks.
Webly-supervised zero-shot learning for artwork instance recognition

This paper describes experiments on supervised approaches to webly-labeled artwork instance recognition and zero-shot learning for unseen artwork instance recognition. We build on our earlier work on webly-supervised learning using the NoisyArtdataset. The dataset consists of more than 90,000 images and in more than 3,000 webly-supervised classes, and a subset of 200 classes with verified test images. Document embeddings are provided for short descriptions of all artworks. NoisyArt is designed to support research on webly-supervised artwork instance recognition, zero-shot learning, and other approaches to visual recognition of cultural heritage objects. We report results of experiments on artwork instance recognition using the NoisyArt dataset of webly-labeled images as well as on the CMU-Oxford Sculptures dataset. In addition, we perform extensive experiments on zero-shot learning using webly-labeled training images for unseen artwork recognition. Our results demonstrate the benefits and limitations of zero-shot learning for instance recognition over webly-supervised data.
Multilayer Perceptron for Sparse Functional Data

In this paper, we propose a novel algorithm for generalizing Multilayer Perceptron (MLP) to handle sparse functional data, wherein for a given subject there are multiple observations available over time and these observations are sparsely and irregularly distributed within the considered time range. The algorithm uses pooled observations across all the subjects to estimate a set of basis functions for the underlying correlation between time steps and then use these basis functions to build a sparse functional neuron that extracts features for each subject. We justify the validity of our algorithm through theoretical arguments. We also demonstrate the use of the proposed algorithm in solving three data challenges: the classification of synthetic curves, the prediction of patient's long-term survival, and the estimation of the remaining time to critical failures for turbofan engines. To show the superiority of our algorithm under sparse functional data scenarios, we compare the performance of our model with two alternative common practices, and demonstrate that our method outperforms the baseline methods in all numerical studies.
Visual Abstraction

In this chapter, we revisit the concept of abstraction as it is used in visualization and put it on a solid formal footing. While the term “abstraction” is utilized in many scientific disciplines, arts, as well as everyday life, visualization inherits the notion of data abstraction or class abstraction from computer science, topological abstraction from mathematics, and visual abstraction from arts. All these notions have a lot in common, yet there is a major discrepancy in the terminology and basic understanding about visual abstraction in the context of visualization. We thus root the notion of abstraction in the philosophy of science, clarify the basic terminology, and provide crisp definitions of visual abstraction as a process. Furthermore, we clarify how it relates to similar terms often used interchangeably in the field of visualization. Visual abstraction is characterized by a conceptual space where this process exists, by the purpose it should serve, and by the perceptual and cognitive qualities of the beholder. These characteristics can be used to control the process of visual abstraction to produce effective and informative visual representations.
Conceptualizing and operationalizing the student relationship management strategy: Towards a more sustainable-based platform

Universities are at the forefront of developing future human capital, but require new strategic approaches to succeed. The present paper discusses the issues concerning the student relationship management strategy, which appeared under the system of customer relationship management. As a new line of research, there are opportunities for innovative studies. This investigation comprehensively contributes to the concept of this strategy as well as a structural model based on the perception of 382 university students and experts using integrated quantitative and qualitative modeling. Upon confirmation of the proposed model, the results revealed a six-factor structure, which was assessed by knowledge acquisition and application, student relationship management technology, knowledge diffusion, student orientation, student relationship management results, and employee’s involvement. The analyses indicated that ‘knowledge management’ is an infrastructure with high driving power. The research findings underlined the significant role of this strategic approach in embedding the student satisfaction-retention-loyalty chain to develop sustainable relationships with students. A more sustainable-based platform under the theme of “sustainable student relationship management (Sustainable SRM)” is also constituted and discussed for future research.