Reviewerly/exHarmony huggingface dataset:
Place Classification Algorithm Based on Semantic Segmented Objects

Scene or place classification is one of the important problems in image and video search and recommendation systems. Humans can understand the scene they are located, but it is difficult for machines to do it. Considering a scene image which has several objects, humans recognize the scene based on these objects, especially background objects. According to this observation, we propose an efficient scene classification algorithm for three different classes by detecting objects in the scene. We use pre-trained semantic segmentation model to extract objects from an image. After that, we construct a weight matrix to determine a scene class better. Finally, we classify an image into one of three scene classes (i.e., indoor, nature, city) by using the designed weighting matrix. The performance of our scheme outperforms several classification methods using convolutional neural networks (CNNs), such as VGG, Inception, ResNet, ResNeXt, Wide-ResNet, DenseNet, and MnasNet. The proposed model achieves 90.8% of verification accuracy and improves over 2.8% of the accuracy when comparing to the existing CNN-based methods.
A Survey of Recent Advances in CNN-Based Fine-Grained Visual Categorization

Fine-grained visual classification (FGVC) is an important task in the field of computer vision (CV), which aims to classify sub-categories that are hard to distinguish (e.g., identifying different species of birds). It has become a challenging problem due to the larger inter-class similarity and intra-class variability in fine-grained datasets. In recent years, with the booming of deep learning, convolutional neural networks (CNN) have brought a new opportunity to FGVC, and a large number of CNN-based algorithms have been proposed, which have advanced the rapid development of FGVC. In this paper, we introduce the characteristics and challenges of FGVC tasks first, then analyze the latest methods from the strongly-supervised to and weakly-supervised ones, and compare the performances of these algorithms on commonly used FGVC datasets. Finally, we categorize the FGVC related applications.
Multi-view Low-rank Preserving Embedding: A novel method for multi-view representation

In recent years, we have witnessed a surge of interest in multi-view representation learning. When facing multiple views that are highly related but sightly different from each other, most existing multi-view methods might fail to fully explore multi-view information. Additionally, pairwise correlations among multiple views often vary drastically, which makes multi-view representation challenging. Therefore, how to learn appropriate representation from multi-view information is still an open but challenging problem. To handle this issue, this paper proposes a novel multi-view learning method, named Multi-view Low-rank Preserving Embedding (MvLPE). It integrates all views into a common latent space, termed as centroid view, by minimizing the disagreement between centroid view and each view, which encourages different views to mutually learn from each other. Unlike existing methods with explicit weight definition, the proposed method could automatically allocate an ideal weight for each view according to its contribution. Besides, MvLPE could maintain its low-rank reconstruction structure for each view while integrating all views into centroid view. Since there is no closed-form solution for MvLPE, an effective algorithm based on iterative alternating strategy is provided to obtain the solution. The experiments on six benchmark datasets validate the effectiveness of the proposed method, which achieves superior performance over its counterparts.
Ablation Analysis to Select Wearable Sensors for Classifying Standing, Walking, and Running

The field of human activity recognition (HAR) often utilizes wearable sensors and machine learning techniques in order to identify the actions of the subject. This paper considers the activity recognition of walking and running while using a support vector machine (SVM) that was trained on principal components derived from wearable sensor data. An ablation analysis is performed in order to select the subset of sensors that yield the highest classification accuracy. The paper also compares principal components across trials to inform the similarity of the trials. Five subjects were instructed to perform standing, walking, running, and sprinting on a self-paced treadmill, and the data were recorded while using surface electromyography sensors (sEMGs), inertial measurement units (IMUs), and force plates. When all of the sensors were included, the SVM had over 90% classification accuracy using only the first three principal components of the data with the classes of stand, walk, and run/sprint (combined run and sprint class). It was found that sensors that were placed only on the lower leg produce higher accuracies than sensors placed on the upper leg. There was a small decrease in accuracy when the force plates are ablated, but the difference may not be operationally relevant. Using only accelerometers without sEMGs was shown to decrease the accuracy of the SVM.
An Efficient FPGA Accelerator Optimized for High Throughput Sparse CNN Inference

Pruning techniques can compress the CNN models by making the insignificant weights to be zeros to release the tremendous workload in large-scale CNNs. However, for hardware architecture, to efficiently load and operate on the nonzero data with high parallelism is a great challenge due to the random location of pruned weights. To address this issue, a sparsity aware CNN accelerator is proposed in this work to process the irregularly pruned CNN models. A candidate pool architecture is designed to only pick the randomly needed activations chosen by nonzero weights. It is set as a three-dimensional structure to relieve the problem of workload imbalance caused by random nonzero weight locations and high parallelism. Besides, a dedicated indexing method is designed to cooperate with the candidate pool architecture to accomplish the whole sparse dataflow. The proposed sparsity aware CNN accelerator is demonstrated on Intel Arria 10 FPGA for multiple popular CNN models that achieves up to 89.7% throughput improvement compared to the baseline design.
Performance Analysis of Cost and Accuracy for Whale Swarm and RMSprop Optimizer

Abstract The scientific fields’ deals with day to day problems, like economic planning and engineering design, mostly are disconnected, high dimensional, multimodal and oscillated optimization problems. These complex problems cannot be solved well within reasonable time using conventional process based on gradient. The natural phenomenon motivates and animal group behavior characteristics, the investigators have anticipated many efficient natural heuristic algorithms for high-dimensional complex optimization problems in real-world. This manuscript deals with one of the meta-heuristic algorithm, Whale Swarm Optimization algorithm (WSO) and compares the performance with RMS prop optimization techniques.
Hardware/Software Obfuscation against Timing Side-channel Attack on a GPU

GPUs are increasingly being used in security applications, especially for accelerating encryption/decryption. While GPUs are an attractive platform in terms of performance, the security of these devices raises a number of concerns. One vulnerability is the data-dependent timing information, which can be exploited by adversary to recover the encryption key. Memory system features are frequently exploited since they create detectable timing variations. In this paper, our attack model is a coalescing attack, which leverages a critical GPU microarchitectural feature the coalescing unit. As multiple concurrent GPU memory requests can refer to the same cache block, the coalescing unit collapses them into a single memory transaction. The access time of an encryption kernel is dependent on the number of transactions. Correlation between a guessed key value and the associated timing samples can be exploited to recover the secret key. In this paper, a series of hardware/software countermeasures are proposed to obfuscate the memory timing side channel, making the GPU more resilient without impacting performance. Our hardware-based approach attempts to randomize the width of the coalescing unit to lower the signal-to-noise ratio. We present a hierarchical Miss Status Holding Register (MSHR) design that can merge transactions across different warps. This feature boosts performance, while, at the same time, secures the execution. We also present a software-based approach to permute the organization of critical data structures, significantly changing the coalescing behavior and introducing a high degree of randomness. Equipped with our new protections, the effort to launch a successful attack is increased up to $1433X\times 178X$, while also improving encryption/decryption performance up to 7%.
Superimposition: Augmenting Machine Learning Outputs with Conceptual Models for Explainable AI

Machine learning has become almost synonymous with Artificial Intelligence (AI). However, it has many challenges with one of the most important being explainable AI; that is, providing human-understandable accounts of why a machine learning model produces specific outputs. To address this challenge, we propose superimposition as a concept which uses conceptual models to improve explainability by mapping the features that are important to a machine learning model’s decision outcomes to a conceptual model of an application domain. Superimposition is a design method for supplementing machine learning models with structural elements that are used by humans to reason about reality and generate explanations. To illustrate the potential of superimposition, we present the method and apply it to a churn prediction problem.
Bidirectional loss function for Label Enhancement and distribution learning

Label distribution learning (LDL) is an interpretable and general learning paradigm that has been applied in many real-world applications. In contrast to the simple logical vector in single-label learning (SLL) and multi-label learning (MLL), LDL assigns labels with a description degree to each instance. In practice, two challenges exist in LDL, namely, how to address the dimensional gap problem during the learning process of LDL and how to exactly recover label distributions from existing logical labels, i.e., Label Enhancement (LE). For most existing LDL and LE algorithms, the fact that the dimension of the input matrix is much higher than that of the output one is always ignored and it typically leads to the dimensional reduction owing to the unidirectional projection. The valuable information hidden in the feature space is lost during the mapping process. To this end, this study considers bidirectional projections function which can be applied in LE and LDL problems simultaneously. More specifically, this novel loss function not only considers the mapping errors generated from the projection of the input space into the output one but also accounts for the reconstruction errors generated from the projection of the output space back to the input one. This loss function aims to potentially reconstruct the input data from the output data. Therefore, it is expected to obtain more accurate results. Experiments on several real-world datasets are carried out to demonstrate the superiority of the proposed method for both LE and LDL. Specifically, BD-LE achieves optimal performance in 85.71% cases and renders sub-optimal in 13.09% cases. BD-LDL ranks 1st in 90.48% cases across six evaluation measurements. Compared with the baseline algorithms, the bidirectional projection methods can outperform the best baselines over 7.38% and 9.98% on average for LE and LDL, respectively.
Efficient Resampling for Fraud Detection During Anonymised Credit Card Transactions with Unbalanced Datasets

The rapid growth of e-commerce and online shopping have resulted in an unprecedented increase in the amount of money that is annually lost to credit card fraudsters. In an attempt to address credit card fraud, researchers are leveraging the application of various machine learning techniques for efficiently detecting and preventing fraudulent credit card transactions. One of the prevalent common issues around the analytics of credit card transactions is the highly unbalanced nature of the datasets, which is frequently associated with the binary classification problems. This paper intends to review, analyse and implement a selection of notable machine learning algorithms such as Logistic Regression, Random Forest, K-Nearest Neighbours and Stochastic Gradient Descent, with the motivation of empirically evaluating their efficiencies in handling unbalanced datasets whilst detecting credit card fraud transactions. A publicly available dataset comprising 284807 transactions of European cardholders is analysed and trained with the studied machine learning techniques to detect fraudulent transactions. Furthermore, this paper also evaluates the incorporation of two notable resampling methods, namely Random Under-sampling and Synthetic Majority Oversampling Techniques (SMOTE) in the aforementioned algorithms, in order to analyse their efficiency in handling unbalanced datasets. The proposed resampling methods significantly increased the detection ability, the most successful technique of combination of Random Forest with Random Under-sampling achieved the recall score of 100% in contrast to the recall score 77% of model without resampling technique. The key contribution of this paper is the postulation of efficient machine learning algorithms together with suitable resampling methods, suitable for credit card fraud detection with unbalanced dataset.
Perspective registration and multi-frame super-resolution of license plates in surveillance videos

One task often encountered in surveillance videos is the recognition of a target—e.g. the license plate of a vehicle. Often, the quality of a single video frame does not permit a reliable recognition. If multiple frames are available, it is possible to combine them in order to generate a single image with lower noise (frame averaging) and/or higher resolution (super-resolution). In order for these techniques to work, it is necessary to accurately estimate the motion of the object of interest in the recorded footage. In this paper, we introduce a method capable of accurately computing the perspective transformation that describes the motion of a planar object. The method minimizes the squared distance between the transformed image and a reference, computed over a user-defined region of interest, and uses the partial derivatives in order to significantly speed up the computation. This approach is inspired by the well known Kanade–Lucas–Tomasi feature tracker.
Gesture recognition and information recommendation based on machine learning and virtual reality in distance education

The dynamic and static gesture recognition in the distance education application scenario is not mature enough in theory at present, and still has a large space for development, and the application of gesture recognition in education is relatively insufficient. The purpose of this article is to combine gesture recognition with teacher classroom education and introduce a dynamic gesture recognition method. Moreover, this study introduces the data collection and preprocessing in detail and converts the data of the gesture action area into gray value images, and then uses the improved algorithm to perform classification. In addition, this study designs a control experiment to analyze the performance of the algorithm in this study and compares the accuracy of algorithm recognition from the perspective of simple background and complex background. The research results show that teaching gesture recognition in distance education can effectively improve education efficiency, with high accuracy, and can be directly applied to the system.
Combining finite mixture distribution, receptor model, and geostatistical simulation to evaluate heavy metals pollution in soils: Source and spatial pattern

Abstract Soil heavy metals (HMs) pollution has caused significant land degradation by affecting soil properties and functions. Clarifying sources and spatial distributions of soil HMs is necessary for the investigation of land degradation, but lacks accurate and efficient methods. This study proposes a combined method for improving source apportionment and spatial prediction of soil HMs. Finite mixture distribution modelling (FMDM) was used to explore the backgrounds and contamination thresholds, and can verify the source contribution calculated by positive matrix factorization (PMF). To improve the efficiency of traditional multivariate geostatistical simulation, independent component analysis and sequential Gaussian simulation were integrated into geostatistical independent simulation for revealing the spatial patterns and the polluted areas. The combined method was applied to an HMs dataset of Longkou City, eastern China as a case‐study. Both PMF and FMDM produced three sources of HMs. Cr and Ni were dominated by natural sources. Agricultural inputs mainly affected Cd, Cu, and Zn concentrations. Hg, 66.8%, and Pb, 8.1%, originated from industrial, mining, and traffic emissions. Hazardous areas exceeding moderate pollution cutoffs comprised 203.88 km 2 , accounting for 23.2% of the study area. Less than 0.1% of the areas for all HMs were identified as hazardous zones exceeding the high pollution thresholds. The reasonable management of industrial and agricultural activities, traffic emissions, and mining could have an important influence on the protection of land resources. Combining FMDM, PMF, and geostatistical independent simulation proved valid for investigating the land degradation due to HMs pollution.
A Novel and Reliable Framework of Patient Deterioration Prediction in Intensive Care Unit Based on Long Short-Term Memory-Recurrent Neural Network

The clinical investigation explored that early recognition and intervention are crucial for preventing clinical deterioration in patients in Intensive Care units (ICUs). Deterioration of patients is predictable and can be preventable if early risk factors are recognized and developed in the clinical setting. Timely detection of deterioration in ICU patients may also lead to better health management. In this paper, a new model was proposed based on Long Short-Term Memory-Recurrent Neural Network (LSTM-RNN) to predict deterioration of ICU patients. An optimisation model based on a modified genetic algorithm (GA) has also been proposed in this study to optimize the observation window, prediction window, and the number of neurons in hidden layers to increase accuracy, AUROC, and minimize test loss. The experimental results demonstrate that the prediction model proposed in this study acquired a significantly better classification performance compared with many other studies that used deep learning models in their works. Our proposed model was evaluated for two tasks: mortality and sudden transfer of patients to ICU. Our results show that the proposed model could predict deterioration before one hour of onset and outperforms other models. In this study, the proposed predictive model is implemented using the state-of-the-art graphical processing unit (GPU) virtual machine provided by Google Colaboratory. Moreover, the study uses a novel time-series approach, which is minute-by-minute. This novel approach enables the proposed model to obtain highly accurate results (i.e., an AUROC of 0.933 and an accuracy of 0.921). This study utilizes the individual and combined effectiveness of different types of variables (i.e., vital signs, laboratory measurements, GCS, and demographic data). In this study, data was extracted from MIMIC-III database. The ad-hoc frameworks proposed by previous studies can be improved by the novel and reliable prediction framework proposed in this research, which will result in predictions of more accurate performance. The proposed predictive model could reduce the required observation window (i.e., a reduction of 83%) for the prediction task while improving the performance. In fact, the proposed significant small size of observation window could obtain higher results which outperformed all previous works that utilize different sizes of observation window (i.e., 48 hours and 24 hours). Moreover, this research demonstrates the ability of the proposed predictive model to achieve accurate results (>80%) on `raw' data in an experimental work. This shows that the rule-based pre-processing of clinical features is unnecessary for deep learning predictive models.
Federated Learning Service Market: A Game Theoretic Analysis

Federated learning enables data owners in intelligent communication systems to share information without revealing actual data contents. In federated learning, each data owner trains data locally, and only uploads the corresponding trained gradient to a learning server. The learning server aggregates collected gradients and further trains a learning model by averaging all the gradients. The trained model can be returned to data owners for improving their performance of data utilization and analysis. In this work, we extend the two-layered federated learning architecture to a learning market with privacy preserving. In the learning market, data owners apply federated learning to trade information extracted from data, a learning service provider collects information from data owners and provides data services based on the learned model to arbitrary service users, who have no data but are willing to pay for data services. To analyze and solve the optimal behaviours of all the participants in the system, a market-oriented architecture is formulated with a Stackelberg game theoretic approach, considering social impacts among all the market participants in the data and model trading processes.
Developing an eXtended Reality platform for Immersive and Interactive Experiences for Cultural Heritage: Serralves Museum and Coa Archeologic Park

Digital Heritage and Digital Humanities focus on distinct typologies of heritage: tangible and intangible Cultural Heritage (CH) objects and their preservation, education, and research versus the application of digital technologies to support research in the humanities. Both allow scholars to go beyond textual sources to integrate digital tools into the humanistic study. This project aims at supporting a new way of experiencing CH in the Serralves Museum and Coa Archeologic Park through more involving and culturally-qualified user experience. The main goal is to understand the potential of eXtended Reality within CH while also proposing the idea of developing a digital experience platform: an authoring tool based on an engine with core experiences functions that can be applied for developing multiple experiences for CH. This platform will contribute to new approaches, technologies, and tools for creating, processing, and delivering immersive and interactive content for engaging and meaningful experiences in these specific CH environments.