Reviewerly/exHarmony huggingface dataset:
Reliable solar irradiance prediction using ensemble learning-based models: A comparative study

Accurately predicting solar irradiance is important in designing and efficiently managing photovoltaic systems. This paper aims to provide a reliable short-term prediction of solar irradiance based on various meteorological factors using ensemble learning-based models that take into account the time-dependent nature of the solar irradiance data. The use of ensemble learning models is motivated by their desirable characteristics in combining several weak regressors to achieve an improved prediction quality relative to conventional single learners. Furthermore, they reduce the overall prediction error and have the ability to combine different models. In this paper, we first investigate the prediction performance of the well-known ensemble methods, Boosted Trees, Bagged Trees, Random Forest, and Generalized Random Forest in short-term prediction of solar irradiance. The performance of these ensemble methods has been compared to two commonly known prediction methods namely Gaussian process regression, and Support Vector Regression. Typical Meteorological Year data are used to verify the prediction performance of the considered models. Results showed that ensemble methods offer superior prediction performance compared to the individual regressors. Furthermore, the results showed that the ensemble models have a consistent and reliable prediction when applied to data from different locations. Lastly, variables contribution assessment showed that the lagged solar irradiance variables contribute significantly to the ensemble models, which help in designing more parsimonious models.
A Comprehensive Survey on Geometric Deep Learning

Deep learning methods have achieved great success in analyzing traditional data such as texts, sounds, images and videos. More and more research works are carrying out to extend standard deep learning technologies to geometric data such as point cloud or voxel grid of 3D objects, real life networks such as social and citation network. Many methods have been proposed in the research area. In this work, we aim to provide a comprehensive survey of geometric deep learning and related methods. First, we introduce the relevant knowledge and history of geometric deep learning field as well as the theoretical background. In the method part, we review different graph network models for graphs and manifold data. Besides, practical applications of these methods, datasets currently available in different research area and the problems and challenges are also summarized.
How do visual explanations foster end users' appropriate trust in machine learning?

We investigated the effects of example-based explanations for a machine learning classifier on end users' appropriate trust. We explored the effects of spatial layout and visual representation in an in-person user study with 33 participants. We measured participants' appropriate trust in the classifier, quantified the effects of different spatial layouts and visual representations, and observed changes in users' trust over time. The results show that each explanation improved users' trust in the classifier, and the combination of explanation, human, and classification algorithm yielded much better decisions than the human and classification algorithm separately. Yet these visual explanations lead to different levels of trust and may cause inappropriate trust if an explanation is difficult to understand. Visual representation and performance feedback strongly affect users' trust, and spatial layout shows a moderate effect. Our results do not support that individual differences (e.g., propensity to trust) affect users' trust in the classifier. This work advances the state-of-the-art in trust-able machine learning and informs the design and appropriate use of automated systems.
Automated breast cancer detection in digital mammograms: A moth flame optimization based ELM approach

Early detection of breast cancer based on a digital mammogram is an important research domain in the field of medical image analysis. An improved CAD model is proposed in this paper for the classification of breast masses into the normal or abnormal and benign or malignant category. The proposed model utilizes lifting wavelet transform (LWT) to extract the features from the region of interest mammogram images. The dimension of the feature vectors is then reduced by using a fusion of PCA and LDA methods. Finally, the classification is performed using a combination of an extreme learning machine and moth flame optimization technique (MFO-ELM). In the MFO-ELM algorithm, MFO is used to optimize the hidden node parameters of ELM. Further, 5-fold stratified cross-validation is used to improve the generalization performance of the model. The proposed model is evaluated on two standard datasets, namely MIAS and DDSM. From the experiment, it is observed that the proposed CAD model obtains ideal results for the MIAS dataset and achieves an accuracy of 99.76% (normal vs. abnormal) and 98.80% (benign vs. malignant) for the DDSM dataset. Our proposed model also demands minimum computational time as compared to other existing models. The experimental results show that the proposed model is superior to other state-of-the-art models in terms of classification accuracy with a significantly reduced number of features.
An aggregative learning gravitational search algorithm with self-adaptive gravitational constants

The gravitational search algorithm (GSA) is a meta-heuristic algorithm based on the theory of Newtonian gravity. This algorithm uses the gravitational forces among individuals to move their positions in order to find a solution to optimization problems. Many studies indicate that the GSA is an effective algorithm, but in some cases, it still suffers from low search performance and premature convergence. To alleviate these issues of the GSA, an aggregative learning GSA called the ALGSA is proposed with a self-adaptive gravitational constant in which each individual possesses its own gravitational constant to improve the search performance. The proposed algorithm is compared with some existing variants of the GSA on the IEEE CEC2017 benchmark test functions to validate its search performance. Moreover, the ALGSA is also tested on neural network optimization to further verify its effectiveness. Finally, the time complexity of the ALGSA is analyzed to clarify its search performance.
Spatial-Temporal Cascade Autoencoder for Video Anomaly Detection in Crowded Scenes

Time-efficient anomaly detection and localization in video surveillance still remains challenging due to the complexity of “anomaly”. In this paper, we propose a cuboid-patch-based method characterized by a cascade of classifiers called a spatial-temporal cascade autoencoder (ST-CaAE), which makes full use of both spatial and temporal cues from video data. The ST-CaAE has two main stages, defined by two proposed neural networks: a spatial-temporal adversarial autoencoder (ST-AAE) and a spatial-temporal convolutional autoencoder (ST-CAE). First, the ST-AAE is used to preliminarily identify anomalous video cuboids and exclude normal cuboids. The key idea underlying ST-AAE is to obtain a Gaussian model to fit the distribution of the regular data. Then in the second stage, the ST-CAE classifies the specific abnormal patches in each anomalous cuboid with reconstruction error based strategy that takes advantage of the CAE and skip connection. A two-stream framework is utilized to fuse the appearance and motion cues to achieve more complete detection results, taking the gradient and optical flow cuboids as inputs for each stream. The proposed ST-CaAE is evaluated using three public datasets. The experimental results verify that our framework outperforms other state-of-the-art works.
Decentralized Stochastic Optimization and Machine Learning: A Unified Variance-Reduction Framework for Robust Performance and Fast Convergence

Decentralized methods to solve finite-sum minimization problems are important in many signal processing and machine learning tasks where the data samples are distributed across a network of nodes, and raw data sharing is not permitted due to privacy and/or resource constraints. In this article, we review decentralized stochastic first-order methods and provide a unified algorithmic framework that combines variance reduction with gradient tracking to achieve robust performance and fast convergence. We provide explicit theoretical guarantees of the corresponding methods when the objective functions are smooth and strongly convex and show their applicability to nonconvex problems via numerical experiments. Throughout the article, we provide intuitive illustrations of the main technical ideas by casting appropriate tradeoffs and comparisons among the methods of interest and by highlighting applications to decentralized training of machine learning models.
Optimizing Weighted Extreme Learning Machines for imbalanced classification and application to credit card fraud detection

The classification problems with imbalanced datasets widely exist in real word. An Extreme Learning Machine is found unsuitable for imbalanced classification problems. This work applies a Weighted Extreme Learning Machine (WELM) to handle them. Its two parameters are found to affect its performance greatly. The aim of this work is to apply various intelligent optimization methods to optimize a WELM and compare their performance in imbalanced classification. Experimental results show that WELM with a dandelion algorithm with probability-based mutation can perform better than WELM with improved particle swarm optimization, bat algorithm, genetic algorithm, dandelion algorithm and self-learning dandelion algorithm. In addition, the proposed algorithm is applied to credit card fraud detection. The results show that it can achieve high detection performance.
Deepfake Detection: Current Challenges and Next Steps

High quality fake videos and audios generated by AI- algorithms (the deep fakes) have started to challenge the status of videos and audios as definitive evidence of events. In this paper, we highlight a few of these challenges and discuss the research opportunities in this direction.
Linear Mode Connectivity and the Lottery Ticket Hypothesis

We study whether a neural network optimizes to the same, linearly connected minimum under different samples of SGD noise (e.g., random data order and augmentation). We find that standard vision models become stable to SGD noise in this way early in training. From then on, the outcome of optimization is determined to a linearly connected region. We use this technique to study iterative magnitude pruning (IMP), the procedure used by work on the lottery ticket hypothesis to identify subnetworks that could have trained in isolation to full accuracy. We find that these subnetworks only reach full accuracy when they are stable to SGD noise, which either occurs at initialization for small-scale settings (MNIST) or early in training for large-scale settings (ResNet-50 and Inception-v3 on ImageNet).
Industrial Security Solution for Virtual Reality

In order to protect industrial safety, improve the operation stability of the industrial control system, conduct the response measures for network environment attacked by the external world, and realize simulation in virtual reality environment, in this study, class and sample weighted C-support vector machine (CSWC-SVM) algorithm is first proposed using SVM. Then, the intrusion detection model of industrial control network is built based on the CSWC-SVM algorithm. Finally, KDD CUP 1999 data are introduced to carry out simulation experiments on the algorithm model constructed in this study in the virtual reality simulation environment. The results show when the penalty factor of the polynomial kernel function, radial basis kernel function, and sigmoid kernel function is 104, the average number of support vectors is 45, 46, and 37, respectively; the average training time are about 0.43, 0.45, and 0.47 s, and the average test time is about 9.7, 9.9, and 10.2 s, respectively; the average recognition accuracy is about 85.7%, 86.2%, and 86.7%, and the false positive rate is 3.8%, 2.8%, and 2.3%, respectively; the accuracy of the CSWC-SVM algorithm in different sample sizes (1000-6000) can be kept above 90%. The operation error rate of the CSWC-SVM algorithm is lower than that of C-SVM, C-SVM, and RS-SVM algorithms under different validation data sets. After dimension reduction, the classification accuracy of the CSWC-SVM algorithm is higher than that of C-SVM and WC-SVM algorithms. The weight value increases from 0 to 200, and the number of model errors on 1000, 2000, and 3000 pieces of data decreases significantly. When the weight value is 200, the number of errors drops to 0, and the classification accuracy reaches 100%. In a word, the CSWC-SVM algorithm constructed in this study performs well in response to the attack of the industrial control system in the virtual reality simulation environment, which provides practical significance for the application of virtual reality in industrial monitoring.
Unsupervised MR-to-CT Synthesis Using Structure-Constrained CycleGAN

Synthesizing a CT image from an available MR image has recently emerged as a key goal in radiotherapy treatment planning for cancer patients. CycleGANs have achieved promising results on unsupervised MR-to-CT image synthesis; however, because they have no direct constraints between input and synthetic images, cycleGANs do not guarantee structural consistency between these two images. This means that anatomical geometry can be shifted in the synthetic CT images, clearly a highly undesirable outcome in the given application. In this paper, we propose a structure-constrained cycleGAN for unsupervised MR-to-CT synthesis by defining an extra structure-consistency loss based on the modality independent neighborhood descriptor. We also utilize a spectral normalization technique to stabilize the training process and a self-attention module to model the long-range spatial dependencies in the synthetic images. Results on unpaired brain and abdomen MR-to-CT image synthesis show that our method produces better synthetic CT images in both accuracy and visual quality as compared to other unsupervised synthesis methods. We also show that an approximate affine pre-registration for unpaired training data can improve synthesis results.
DeepFaceDrawing

Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch. We take a local-to-global approach. We first learn feature embeddings of key face components, and push corresponding parts of input sketches towards underlying component manifolds defined by the feature vectors of face component samples. We also propose another deep neural network to learn the mapping from the embedded component features to realistic images with multi-channel feature maps as intermediate results to improve the information flow. Our method essentially uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches. Our tool is easy to use even for non-artists, while still supporting fine-grained control of shape details. Both qualitative and quantitative evaluations show the superior generation ability of our system to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.
Spatial Temporal Transformer Network for Skeleton-Based Action Recognition

Skeleton-based human action recognition has achieved a great interest in recent years, as skeleton data has been demonstrated to be robust to illumination changes, body scales, dynamic camera views, and complex background. Nevertheless, an effective encoding of the latent information underlying the 3D skeleton is still an open problem. In this work, we propose a novel Spatial-Temporal Transformer network (ST-TR) which models dependencies between joints using the Transformer self-attention operator. In our ST-TR model, a Spatial Self-Attention module (SSA) is used to understand intra-frame interactions between different body parts, and a Temporal Self-Attention module (TSA) to model inter-frame correlations. The two are combined in a two-stream network which outperforms state-of-the-art models using the same input data on both NTU-RGB+D 60 and NTU-RGB+D 120.
Bitcoin-energy markets interrelationships - New evidence

The annual electricity consumption of cryptocurrency transactions has grown substantially in recent years, partially driven by the increasing difficulty in mining, but also driven by the large number of new market participants that have been attracted by the elevated prices of this developing financial asset. Total carbon production from mining now likely exceeds that generated by individual developed nations. This is now a prevailing and accepted feature in cryptocurrency markets, however unsustainable it may be. This paper investigates as to how Bitcoin's price volatility and the underlying dynamics of cryptocurrency mining characteristics affect underlying energy markets and utilities companies. Further analysis of potential side-effects within the market for Exchange Traded Funds are considered. The results show a sustained and significant influence of cryptocurrency energy-usage on the performance of some companies in the energy sector as separated by jurisdiction, emphasising the importance of further assessment of environmental impacts of cryptocurrency growth. Robustness testing presents evidence that dynamic correlations peaked during the sharp Bitcoin price appreciation of late-2017 as investors re-evaluated how this increased energy usage would influence the profitability of utility companies.
Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularization

Deep neural networks (DNNs) have been proven vulnerable to backdoor attacks, where hidden features (patterns) trained to a normal model, which is only activated by some specific input (called triggers), trick the model into producing unexpected behavior. In this article, we create covert and scattered triggers for backdoor attacks, invisible backdoors, where triggers can fool both DNN models and human inspection. We apply our invisible backdoors through two state-of-the-art methods of embedding triggers for backdoor attacks. The first approach on Badnets embeds the trigger into DNNs through steganography. The second approach of a trojan attack uses two types of additional regularization terms to generate the triggers with irregular shape and size. We use the Attack Success Rate and Functionality to measure the performance of our attacks. We introduce two novel definitions of invisibility for human perception; one is conceptualized by the Perceptual Adversarial Similarity Score (PASS) and the other is Learned Perceptual Image Patch Similarity (LPIPS). We show that the proposed invisible backdoors can be fairly effective across various DNN models as well as four datasets MNIST, CIFAR-10, CIFAR-100, and GTSRB, by measuring their attack success rates for the adversary, functionality for the normal users, and invisibility scores for the administrators. We finally argue that the proposed invisible backdoor attacks can effectively thwart the state-of-the-art trojan backdoor detection approaches.
Eye tracking algorithms, techniques, tools, and applications with an emphasis on machine learning and Internet of Things technologies

Eye tracking is the process of measuring where one is looking (point of gaze) or the motion of an eye relative to the head. Researchers have developed different algorithms and techniques to automatically track the gaze position and direction, which are helpful in different applications. Research on eye tracking is increasing owing to its ability to facilitate many different tasks, particularly for the elderly or users with special needs. This study aims to explore and review eye tracking concepts, methods, and techniques by further elaborating on efficient and effective modern approaches such as machine learning (ML), Internet of Things (IoT), and cloud computing. These approaches have been in use for more than two decades and are heavily used in the development of recent eye tracking applications. The results of this study indicate that ML and IoT are important aspects in evolving eye tracking applications owing to their ability to learn from existing data, make better decisions, be flexible, and eliminate the need to manually re-calibrate the tracker during the eye tracking process. In addition, they show that eye tracking techniques have more accurate detection results compared with traditional event-detection methods. In addition, various motives and factors in the use of a specific eye tracking technique or application are explored and recommended. Finally, some future directions related to the use of eye tracking in several developed applications are described.
Visual Odometry Revisited: What Should Be Learnt?

In this work we present a monocular visual odometry (VO) algorithm which leverages geometry-based methods and deep learning. Most existing VO/SLAM systems with superior performance are based on geometry and have to be carefully designed for different application scenarios. Moreover, most monocular systems suffer from scale-drift issue. Some recent deep learning works learn VO in an end-to-end manner but the performance of these deep systems is still not comparable to geometry-based methods. In this work, we revisit the basics of VO and explore the right way for integrating deep learning with epipolar geometry and Perspective-n-Point (PnP) method. Specifically, we train two convolutional neural networks (CNNs) for estimating single-view depths and two-view optical flows as intermediate outputs. With the deep predictions, we design a simple but robust frame-to-frame VO algorithm (DF-VO) which outperforms pure deep learning-based and geometry-based methods. More importantly, our system does not suffer from the scale-drift issue being aided by a scale consistent single-view depth CNN. Extensive experiments on KITTI dataset shows the robustness of our system and a detailed ablation study shows the effect of different factors in our system. Code is available at here: DF-VO.
Digital competences for teacher professional development. Systematic review

Over the last years, Information and Communication Technologies have played a substantial role in improving both the quality of life for people. However, their potential in education is still developing. In this regard, this paper provides a systematic review of the literature on digital competencies and teacher professional development from 2008 to 2018. The purpose of this study is identifying research trends and potential directions on this topic for future research. The method followed is based on the PRISMA Statement Guidelines for Systematic Reviews and Meta-analysis. After applying criteria of inclusion, exclusion and thematic belonging, in Scopus and Wos databases, the sample was constituted in 21 studies. The selected studies emphasise the importance of digital competence as one of the challenges facing teachers today, being mainly qualitative methodology studies. Among the conclusions we highlight that most of the selected studies reveal a lack of teacher training and insufficient ICT training.