Reviewerly/exHarmony huggingface dataset:
Immersive virtual-reality computer-assembly serious game to enhance autonomous learning

Immersive virtual reality (VR) environments create a very strong sense of presence and immersion. Nowadays, especially when student isolation and online autonomous learning is required, such sensations can provide higher satisfaction and learning rates than conventional teaching. However, up until the present, learning outcomes with VR tools have yet to prove their advantageous aspects over conventional teaching. The project presents a VR serious game for teaching concepts associated with computer hardware assembly. These concepts are often included in any undergraduate's introduction to Computer Science. The learning outcomes are evaluated using a pre-test of previous knowledge, a satisfaction/usability test, and a post-test on knowledge acquisition, structured with questions on different knowledge areas. The results of the VR serious game are compared with another two learning methodologies adapted to online learning: (1) an online conventional lecture; and (2) playing the same serious game on a desktop PC. An extensive sample of students (n = 77) was formed for this purpose. The results showed the strong potential of VR serious games to improve student well-being during spells of confinement, due to higher learning satisfaction. Besides, ease of usability and the use of in-game tutorials are directly related with game-user satisfaction and performance. The main novelty of this research is related to academic performance. Although a very limited effect was noted for learning theoretical knowledge with the VR application in comparison with the other methodologies, this effect was significantly improved through visual knowledge, understanding and making connections between different concepts. It can therefore be concluded that the proposed VR serious game has the potential to increase student learning and therefore student satisfaction, by imparting a deeper understanding of the subject matter to students.The online version contains supplementary material available at 10.1007/s10055-021-00607-1.
FaceFormer: Speech-Driven 3D Facial Animation with Transformers

Speech-driven 3D facial animation is challenging due to the complex geometry of human faces and the limited availability of 3D audio-visual data. Prior works typically focus on learning phoneme-level features of short audio windows with limited context, occasionally resulting in inaccurate lip movements. To tackle this limitation, we propose a Transformer-based autoregressive model, Face-Former, which encodes the long-term audio context and autoregressively predicts a sequence of animated 3D face meshes. To cope with the data scarcity issue, we integrate the self-supervised pre-trained speech representations. Also, we devise two biased attention mechanisms well suited to this specific task, including the biased cross-modal multi-head (MH) attention and the biased causal MH self-attention with a periodic positional encoding strategy. The former effectively aligns the audio-motion modalities, whereas the latter offers abilities to generalize to longer audio sequences. Extensive experiments and a perceptual user study show that our approach outperforms the existing state-of-the-arts. The code and the video are available at: https://evelynfan.github.io/audio2face/
Learning Dual-Level Deep Representation for Thermal Infrared Tracking

The feature models used by existing Thermal InfraRed (TIR) tracking methods are usually learned from RGB images due to the lack of a large-scale TIR image training dataset. However, these feature models are less effective in representing TIR objects and they are difficult to effectively distinguish distractors because they do not contain fine-grained discriminative information. To this end, we propose a dual-level feature model containing the TIR-specific discriminative feature and fine-grained correlation feature for robust TIR object tracking. Specifically, to distinguish inter-class TIR objects, we first design an auxiliary multi-classification network to learn the TIR-specific discriminative feature. Then, to recognize intra-class TIR objects, we propose a fine-grained aware module to learn the fine-grained correlation feature. These two kinds of features complement each other and represent TIR objects in the levels of inter-class and intra-class respectively. These two feature models are constructed using a multi-task matching framework and are jointly optimized on the TIR object tracking task. In addition, we develop a large-scale TIR image dataset to train the network for learning TIR-specific feature patterns. To the best of our knowledge, this is the largest TIR tracking training dataset with the richest object class and scenario. To verify the effectiveness of the proposed dual-level feature model, we propose an offline TIR tracker (MMNet) and an online TIR tracker (ECO-MM) based on the feature model and evaluate them on three TIR tracking benchmarks. Extensive experimental results on these benchmarks demonstrate that the proposed algorithms perform favorably against the state-of-the-art methods.
A self-adaptive gradient descent search algorithm for fully-connected neural networks

Artificial neural networks have achieved great success in artificial intelligence. Back propagation is one of the most widely used parameter search techniques for artificial neural networks, and it has been widely used in many deep learning approaches such as convolutional neural networks (CNNs), generator adversarial networks (GANs) and recurrent neural networks (RNNs) to train their fully-connected layers. However, the fully-connected layers of the neural networks with back propagation technique depends on the gradient descent strategy, which has the disadvantage of easily falling into local optima. Meanwhile, many evolutionary computation (EC) techniques are applied to the parameters optimization of neural networks. They take advantage of populations to achieve global search. But their convergence speed is relatively slow since they use their original evolutionary search strategies. In this paper, we propose a self-adaptive gradient descent search algorithm (SaGDSA) to search the parameters for fully-connected neural networks. Different from the existing adaptive gradient descent algorithm, the proposed algorithm does not require the user to have any prior knowledge and manually design the learning rate of different stages to match the different search stages. In addition, four kinds of gradient descent strategies are used to optimize parameters of fully-connected neural networks. All used datasets are collected from the University of California Irvine (UCI) machine learning repository. The experimental results indicate that SaGDSA is better than the comparison algorithms on most of the datasets. By introducing the self-adaptive mechanism, SaGDSA not only has excellent global search capabilities from evolutionary computation but also has local search capabilities from gradient descent.
Variational transformer-based anomaly detection approach for multivariate time series

Due to the strategic importance of satellites, the safety and reliability of satellites have become more important. Sensors that monitor satellites generate lots of multivariate time series, and the abnormal patterns in the multivariate time series may imply malfunctions. The existing anomaly detection methods for multivariate time series have poor effects when processing the data with few dimensions or sparse relationships between sequences. This paper proposes an unsupervised anomaly detection model based on the variational Transformer to solve the above problems. The model uses the Transformer's self-attention mechanism to capture the potential correlations between sequences and capture the multi-scale temporal information through the improved positional encoding and up-sampling algorithm. Then, the model comprehensively considers the extracted features through the residual variational autoencoder to perform effective anomaly detection. Experimental results on a real dataset and two public datasets show that the proposed method is superior to the mainstream and state-of-the-art methods.
An image encryption algorithm based on new generalized fusion fractal structure

The design and utilization of suitable fractal structures is one of the prominent areas of security for the protection of digital data. This paper proposes a generalized fusion fractal structure by combining two one-dimensional fractals as seed functions from a larger spectrum of fractal functions. A fusion fractal termed as PLFF is formulated by combining traditional Phoenix and Lambda fractals. Improved randomized phase space, self-similar structure on various magnification scales, and fractional dimension are found in the resultant PLFF fractal. The capacity of PLFF to create a pseudo-random number (PRN) sequence in both integer and binary format is validated by its increased complexity and enhanced chaotic range. The generated PRN sequences feature a significant degree of uncorrelation and randomness. A novel image encryption algorithm based on the new PLFF fractal function is proposed which utilizes a generated PRN sequence as secret key. Standard security evaluations such as histogram variance, NPCR and UACI tests for plain-image sensitivity, key sensitivity, information entropy, pixel correlation, and noise and data loss, etc. are used to analyze the performance of the proposed encryption algorithm. The simulation results revealed performance indicators such as entropies > 7.997, NPCR > 96.6, UACI > 33.5, high throughput of ∼ 6MBps, and highly uncorrelated neighboring pixels in encrypted images. The findings are also compared with some current image encryption schemes, demonstrating that the proposed digital image encryption algorithm performs well.
Do we still need financial intermediation? The case of decentralized finance – DeFi

Purpose Decentralized finance (DeFi), enabled by blockchain, could bring about a new financial system, where peers will interact directly, with little or no place for traditional intermediation. However, some crucial tasks cannot be left solely to an algorithm and, consequently, most DeFi applications still require human decisions. The aim of this research is to assess the role of intermediation in the light of DeFi, analysing how humans and algorithms will interact. Design/methodology/approach The authors based their work on a twofold qualitative methodology, first analysing publicly available secondary data, particularly from white papers and DeFi Pulse (a website providing data on DeFi solutions) and then running two focus group discussions. Findings DeFi does not eliminate financial intermediation, but enables it to be performed in new ways, where decentralization means that no single entity can hold too much power or monopoly. DeFi has, however, inherited risks from the underlying technologies that unintentionally facilitate illegal behaviour and can hamper the authorities’ supervision. The complex duality algorithm- vs human-based actions will not be solved indisputably in favour of the former, as DeFi solutions can range from requiring algorithms to play a dominant role, to enabling greater human interaction by actively involving more people. Originality/value This research contributes to the emerging debate between algorithm- and human-based intermediation, especially in relation to the standing literature on financial intermediation, where considerations made in the light of the newest theories on blockchain and DeFi are still scarce.
Target Detection of Forward-Looking Sonar Image Based on Improved YOLOv5

Forward-looking sonar is a commonly used underwater detection device at present, but the detection accuracy is poor due to the complex underwater environment, small target highlight area and fuzzy feature details.Therefore, this paper proposes a forward sonar image target detection model based on You Only Look Once Version 5 (YOLOv5) network using transfer learning method.First, the YOLOv5 network is pretrained with COCO data set.Then the pre-training model is fine-tuned according to the training set of forward-looking sonar images.Before fine-tuning, the traditional k-means clustering is improved.The intersection over union (IoU ) value is used as the distance function to cluster the labeling information of the training set of the forward-looking sonar image.The results of clustering serve as the initial anchor frame of the training network.This operation greatly improves the detection speed.Second, due to the characteristics of weak echo intensity and small target area of forward-looking sonar image, an improved feature extraction method of CoordConv was proposed to give corresponding coordinate information to high-level features which improves the accuracy of network detection regression.Finally, the fine-tuned network is used to detect the target in the forward-looking sonar image.The experimental results show that the improved model based on YOLOv5 network is superior to the original YOLOv5 network and other popular deep neural networks for target detection in the forward-looking sonar image, which has a reference significance for underwater target detection.The CoordConv-YOLOv5 network based on transfer learning proposed in this paper shows the best performance in both detection accuracy and detection speed.Detection accuracy mAP@0.5:0.95 can reach 56.95%, and detection speed can reach 9ms.
Retrieving soil heavy metals concentrations based on GaoFen-5 hyperspectral satellite image at an opencast coal mine, Inner Mongolia, China

Soil heavy metals pollution has been becoming one of the severely environmental issues globally. Previous studies reported laboratory-measured spectra could be used to infer soil heavy metals concentrations to some extent. However, using field-obtained spectra to estimate soil heavy metals concentrations is still a great challenge due to the low precision and weak efficiency at large scales. The present study collected 110 topsoil samples from an Opencast Coal Mine of Ordos, Inner Mongolia, China. Then, the spectra and soil heavy metals concentrations of samples were measured under laboratory conditions. The direct standardization (DS) algorithm was introduced to calibrate the Gaofen-5 (GF-5) hyperspectral image based on the measured spectra of samples. The spectral reflectance of the GF-5 hyperspectral image was reconstructed using continuous wavelet transform (CWT) at different scales. The characteristic bands of GF-5 for estimating heavy metals concentrations were selected by the Boruta algorithm. Finally, the random forest (RF), the extreme learning machine (ELM), the support vector machine (SVM), and the back-propagation neural network (BPNN) algorithms were used to predict the heavy metals concentrations. Some findings were achieved. First, CWT can effectively eliminate the noise of satellite hyperspectral data. The characteristic bands of Zn (480-677, 827-1029, 1241-1334, 1435-1797, and 1949-2500 nm), Ni (514-630, 835-985, 1258-1325, 1460-1578, and 1949-2319 nm), and Cu (822-831; 1029-1300, 1486-1595, and 1730-2294 nm) can be effectively retrieved via the Boruta algorithm. Second, the estimation accuracy was significantly improved by using the DS algorithm. For zinc (Zn), nickel (Ni), and copper (Cu), the determination coefficients of the validation dataset (Rv2) were 0.77 (RF), 0.62 (RF), and 0.56 (ELM), respectively. Third, the distribution trends of heavy metals were almost consistent with the results of actual ground measurements. This paper revealed that the GF-5 can be one of the reliable satellite hyperspectral imagery for mapping soil heavy metals.
SMOTE-RkNN: A hybrid re-sampling method based on SMOTE and reverse k-nearest neighbors

In recent years, class imbalance learning (CIL) has become an important branch of machine learning. The Synthetic Minority Oversampling TEchnique (SMOTE) is considered to be a benchmark algorithm among CIL techniques. Although the SMOTE algorithm performs well on the vast majority of class-imbalance tasks, it also has the inherent drawback of noise propagation. Many SMOTE-variants have been proposed to address this problem. Generally, the improved solutions conduct a hybrid sampling procedure, i.e., carrying out an undersampling process after SMOTE to remove noises. However, owing to the complexity of data distribution, it is sometimes difficult to accurately identify real instances of noise, resulting in low modeling quality. In this paper, we propose a more robust and universal SMOTE hybrid variant algorithm named SMOTE-reverse k-nearest neighbors (SMOTE-RkNN). The proposed algorithm identifies noise based on probability density but not local neighborhood information. Specifically, the probability density information of each instance is provided by RkNN, a well-known KNN variant. Noisy instances are found and deleted according to their relevant probability density. In experiments on 46 class-imbalanced data sets, SMOTE-RkNN showed promising results in comparison with several popular SMOTE hybrid variant algorithms.
Rethinking the Backdoor Attacks’ Triggers: A Frequency Perspective

Backdoor attacks have been considered a severe security threat to deep learning. Such attacks can make models perform abnormally on inputs with predefined triggers and still retain state-of-the-art performance on clean data. While backdoor attacks have been thoroughly investigated in the image domain from both attackers’ and defenders’ sides, an analysis in the frequency domain has been missing thus far.This paper first revisits existing backdoor triggers from a frequency perspective and performs a comprehensive analysis. Our results show that many current backdoor attacks exhibit severe high-frequency artifacts, which persist across different datasets and resolutions. We further demonstrate these high-frequency artifacts enable a simple way to detect existing backdoor triggers at a detection rate of 98.50% without prior knowledge of the attack details and the target model. Acknowledging previous attacks’ weaknesses, we propose a practical way to create smooth backdoor triggers without high-frequency artifacts and study their detectability. We show that existing defense works can benefit by incorporating these smooth triggers into their design consideration. Moreover, we show that the detector tuned over stronger smooth triggers can generalize well to unseen weak smooth triggers. In short, our work emphasizes the importance of considering frequency analysis when designing both backdoor attacks and defenses in deep learning.
A reliable deep representation learning to improve trust-aware recommendation systems

Deep neural networks have been extensively employed in many applications such as natural language processing and computer vision. They have attracted a lot of attention in designing recommender systems due to their recent significant achievements. These models are mainly used in recommender systems to generate the users’ latent features according to the input data such as user-item rating and trust matrixes. However, such input data are significantly sparse leading to reduce the effectiveness of deep learning models in generating the latent features. In addition, the existing deep learning based recommendation models ignore the reliability measure in their recommendation process. To address these challenges, in this paper, a trust-aware recommendation method based on deep sparse autoencoder is proposed. Specifically, an effective probabilistic model is proposed to determine how many ratings are required for each user to produce an accurate prediction. Moreover, for enhancing the rating profiles of users with insufficient ratings, an efficient mechanism is developed based on the utilization of implicit ratings. In this mechanism, the implicit ratings are selected based on a reliability measure to guarantee their effectiveness. Then, the reliable enhanced rating profiles and trust statements are considered as the input data of deep sparse autoencoder to generate the users’ latent features. Finally, a similarity function that utilizes the generated latent features is developed to obtain similarity values between users and make recommendations. Massive experiments are carried out on two datasets to assess the effectiveness of the proposed method in comparison to the-state-of-the-art models. The results of experiments demonstrate that the proposed method can significantly beat other models in producing more accurate recommendations.
Traffic Sign Detection via Improved Sparse R-CNN for Autonomous Vehicles

Traffic sign detection is an important component of autonomous vehicles. There is still a mismatch problem between the existing detection algorithm and its practical application in real traffic scenes, which is mainly due to the detection accuracy and data acquisition. To tackle this problem, this study proposed an improved sparse R-CNN that integrates coordinate attention block with ResNeSt and builds a feature pyramid to modify the backbone, which enables the extracted features to focus on important information, and improves the detection accuracy. In order to obtain more diverse data, the augmentation method used is specifically designed for complex traffic scenarios, and we also present a traffic sign dataset in this study. For on-road autonomous vehicles, we designed two modules, self-adaption augmentation (SAA) and detection time augmentation (DTA), to improve the robustness of the detection algorithm. The evaluations on traffic sign datasets and on-road testing demonstrate the accuracy and effectiveness of the proposed method.
Z-Score Normalization, Hubness, and Few-Shot Learning

The goal of few-shot learning (FSL) is to recognize a set of novel classes with only few labeled samples by exploiting a large set of abundant base class samples. Adopting a meta-learning framework, most recent FSL methods meta-learn a deep feature embedding network, and during inference classify novel class samples using nearest neighbor in the learned high-dimensional embedding space. This means that these methods are prone to the hubness problem, that is, a certain class prototype becomes the nearest neighbor of many test instances regardless which classes they belong to. However, this problem is largely ignored in existing FSL studies. In this work, for the first time we show that many FSL methods indeed suffer from the hubness problem. To mitigate its negative effects, we further propose to employ z-score feature normalization, a simple yet effective trans-formation, during meta-training. A theoretical analysis is provided on why it helps. Extensive experiments are then conducted to show that with z-score normalization, the performance of many recent FSL methods can be boosted, resulting in new state-of-the-art on three benchmarks.