Reviewerly/exHarmony huggingface dataset:
TopicStriKer: A topic kernels-powered approach for text classification

Topic models are unsupervised machine learning techniques that output clusters of “topics” represented as co-occurring words with their associated probability distributions. Topic modeling algorithms find latent themes from large document collections by understanding their context. On the other hand, string kernels are supervised machine-learning techniques that quantify string similarities without explicit string encoding. We propose TopicStriKer, a model combining the advantages of unsupervised topic modeling with supervised string kernels for text classification tasks. The co-occurring topic words per topic and topic proportions per document obtained are used to reduce the document corpus to a topic-word sequence. This reduced representation is then used for text classification with the aid of string kernels, significantly improving accuracy and reducing training time. Experiments on the bag-of-words kernel-based string embeddings using the proposed algorithm outperform the traditional text classification approaches. This work extensively compares string kernels with topic modeling on various performance metrics to establish our findings.
Few-shot human–object interaction video recognition with transformers

We propose a novel few-shot learning framework that can recognize human-object interaction (HOI) classes with a few labeled samples. We achieve this by leveraging a meta-learning paradigm where human-object interactions are embedded into compact features for similarity calculation. More specifically, spatial and temporal relationships of HOI in videos are constructed with transformers which boost the performance over the baseline significantly. First, we present a spatial encoder that extracts the spatial context and infers frame-level features of a human and objects in each frame. And then the video-level feature is obtained by encoding a series of frame-level feature vectors with a temporal encoder. Experiments on two datasets, CAD-120 and Something-Else, validate that our approach achieves 7.8% and 15.2% accuracy improvement on 1-shot task, 4.7% and 15.7% on 5-shot task, which outperforms the state-of-the-art methods.
An Information-Theoretic Perspective on Intrinsic Motivation in Reinforcement Learning: A Survey

The reinforcement learning (RL) research area is very active, with an important number of new contributions, especially considering the emergent field of deep RL (DRL). However, a number of scientific and technical challenges still need to be resolved, among which we acknowledge the ability to abstract actions or the difficulty to explore the environment in sparse-reward settings which can be addressed by intrinsic motivation (IM). We propose to survey these research works through a new taxonomy based on information theory: we computationally revisit the notions of surprise, novelty, and skill-learning. This allows us to identify advantages and disadvantages of methods and exhibit current outlooks of research. Our analysis suggests that novelty and surprise can assist the building of a hierarchy of transferable skills which abstracts dynamics and makes the exploration process more robust.
A study on the evaluation of tokenizer performance in natural language processing

Formulae display:?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.
Virtual reality training for police officers: a comparison of training responses in VR and real-life training

In policing, Virtual Reality scenario-based training (VR SBT) is being explored to complement real-life scenario-based training (RL SBT).We investigated whether relevant training responses in VR SBT, namely heart rate (HR), level of physical activity, mental effort, and perceived stress, resemble those in RL SBT.Utilizing a within-subject study design, we investigated the training responses of 237 police officers of the Dutch National Police.We found that the maximum HR and average level of physical activity were significantly higher in RL SBT, whereas invested mental effort was significantly higher in VR SBT.No significant differences were found in average HR and perceived stress.We also found that perceived stress in VR was predicted by participants' VR experiences such as engagement with VR and experience of negative effects, but not by participant characteristics.Participants' mental effort in VR was predicted by their VR experiences and participant characteristics, particularly gaming frequency.In conclusion, VR SBT can elicit perceived stress, mental effort, and average HR that resemble or exceed responses in RL SBT, providing a promising tool to complement police training.
Self-Adversarial Disentangling for Specific Domain Adaptation

Domain adaptation aims to bridge the domain shifts between the source and the target domain. These shifts may span different dimensions such as fog, rainfall, etc. However, recent methods typically do not consider explicit prior knowledge about the domain shifts on a specific dimension, thus leading to less desired adaptation performance. In this article, we study a practical setting called Specific Domain Adaptation (SDA) that aligns the source and target domains in a demanded-specific dimension. Within this setting, we observe the intra-domain gap induced by different domainness (i.e., numerical magnitudes of domain shifts in this dimension) is crucial when adapting to a specific domain. To address the problem, we propose a novel Self-Adversarial Disentangling (SAD) framework. In particular, given a specific dimension, we first enrich the source domain by introducing a domainness creator with providing additional supervisory signals. Guided by the created domainness, we design a self-adversarial regularizer and two loss functions to jointly disentangle the latent representations into domainness-specific and domainness-invariant features, thus mitigating the intra-domain gap. Our method can be easily taken as a plug-and-play framework and does not introduce any extra costs in the inference time. We achieve consistent improvements over state-of-the-art methods in both object detection and semantic segmentation.
Self-FuseNet: Data Free Unsupervised Remote Sensing Image Super-Resolution

Real-world degradations deviate from ideal degradations, as most deep learning-based scenarios involve the ideal synthesis of low-resolution (LR) counterpart images by popularly used bicubic interpolation. Moreover, supervised learning approaches rely on many high-resolution (HR) and LR image pairings to reconstruct missing information based on their association, developed by complex long hours of deep neural network training. Additionally, the trained model's generalizability on various image datasets with various distributions is not guaranteed. To overcome this challenge, we proposed our novel Self-FuseNet, particularly for extremely poor-resolution satellite images. Also, the network exhibits strong generalization performance on additional datasets (both “ideal” and “nonideal” scenarios). The network is especially for those image datasets suffering from the following two significant limitations: 1) nonavailability of ground truth HR images; 2) limitation of a large count of the unpaired dataset for deep neural network training. The benefit of the proposed model is threefold: 1) it does not require any significant extensive training data, either paired or unpaired but only a single LR image without prior knowledge of its distribution; 2) it is a simple and effective model for super-resolving very poor-resolution images, saving computational resources and time; 3) using UNet, the processing of data are accelerated by the network's wide skip connections, allowing image reconstruction with fewer parameters. Rather than using an inverse approach, as common in most deep learning scenarios, we introduced a forward approach to super-resolve exceptionally LR remote sensing images. This demonstrates its supremacy over recently proposed state-of-the-art methods for unsupervised single real-world image blind super-resolution.
Deep Gaussian Processes for Classification With Multiple Noisy Annotators. Application to Breast Cancer Tissue Classification

Machine learning (ML) methods often require large volumes of labeled data to achieve meaningful performance. The expertise necessary for labeling data in medical applications like pathology presents a significant challenge in developing clinical-grade tools. Crowdsourcing approaches address this challenge by collecting labels from multiple annotators with varying degrees of expertise. In recent years, multiple methods have been adapted to learn from noisy crowdsourced labels. Among them, Gaussian Processes (GPs) have achieved excellent performance due to their ability to model uncertainty. Deep Gaussian Processes (DGPs) address the limitations of GPs using multiple layers to enable the learning of more complex representations. In this work, we develop Deep Gaussian Processes for Crowdsourcing (DGPCR) to model the crowdsourcing problem with DGPs for the first time. DGPCR models the (unknown) underlying true labels, and the behavior of each annotator is modeled with a confusion matrix among classes. We use end-to-end variational inference to estimate both DGPCR parameters and annotator biases. Using annotations from 25 pathologists and medical trainees, we show that DGPCR is competitive or superior to Scalable Gaussian Processes for Crowdsourcing (SVGPCR) and other state-of-the-art deep-learning crowdsourcing methods for breast cancer classification. Also, we observe that DGPCR with noisy labels obtains better results ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{F}1=81.91$ </tex-math></inline-formula> %) than GPs ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{F}1=81.57$ </tex-math></inline-formula> %) and deep learning methods ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{F}1=80.88$ </tex-math></inline-formula> %) with true labels curated by experts. Finally, we show an improved estimation of annotators’ behavior.
Superpixel-Based Multiscale CNN Approach Toward Multiclass Object Segmentation From UAV-Captured Aerial Images

Unmanned aerial vehicles (UAVs) are promising remote sensors capable of reforming remote sensing applications. However, for artificial intelligence (AI)-guided tasks, like land cover mapping and ground-object mapping, most deep learning-based architectures fail to extract scale-invariant features, resulting in poor performance accuracy. In this context, the article proposes a superpixel-aided multiscale convolutional neural network (CNN) architecture to avoid misclassification in complex urban aerial images.The proposed framework is a two-tier deep learning-based segmentation architecture. In the first stage, a superpixel-based simple linear iterative cluster (SLIC) algorithm produces superpixel images with crucial contextual information. The second stage comprises a multiscale CNN architecture that uses these information-rich superpixel images to extract scale-invariant features for predicting the object class of each pixel. Two UAV-image-based aerial image datasets: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NITRDrone</i> dataset and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">urban drone dataset</i> (UDD) are considered to perform the experimentation. The proposed model outperforms the considered state-of-the-art methods with an intersection of union (IoU) of 76.39% and 86.85% on UDD and NITRDrone datasets, respectively. Experimentally obtained results prove that the proposed architecture performs superior by achieving better performance accuracy in complex and challenging scenarios.
A multi-step ahead global solar radiation prediction method using an attention-based transformer model with an interpretable mechanism

The conventional multi-step ahead solar radiation prediction method ignores the time-dependence of a future solar radiation time series. Therefore, according to sequence-to-sequence (seq2seq) model theory, this paper proposes the seq2seq long- and short-term memory model (seq2seq-LSTM), the seq2seq-LSTM model with an attention mechanism (seq2seq-at-LSTM), and a transformer model, which consists only of the attention mechanism. The hourly global solar radiation data between 2016 and 2018 from Shaanxi, China, is used to train and validate the models. The results show that the introduction of the attention mechanism can effectively improve the prediction accuracy of the seq2seq-LSTM model. However, the model is still not very good at capturing the long-distance dependence of the solar radiation time series due to the inherent properties of LSTM. In comparison, the transformer model, which is based entirely on the attention mechanism, performs much better at capturing the long-distance dependence of the solar radiation time series. Furthermore, as the number of time-steps increases, the performance of the solar radiation prediction decreases relatively smoothly and slowly. The obtained average coefficient of determination, root mean square error (RMSE), relative RMSE, and mean bias error are 0.9788, 72.91 W/m2, 25.25%, and 38.35 W/m2, respectively. In addition, the average skill score of the transformer model is around 44.9%, which is 20.54% higher than that of the seq2seq-at-LSTM model and about 40.84% higher than that of the seq2seq-LSTM model. Besides, the use of the attention mechanism can explain the improved prediction compared to other models. This model developed in this study could also be used for predictions in other fields, such as wind energy predictions and building energy predictions.
Transformer-Based Imitative Reinforcement Learning for Multirobot Path Planning

Multirobot path planning leads multiple robots from start positions to designated goal positions by generating efficient and collision-free paths. Multirobot systems realize coordination solutions and decentralized path planning, which is essential for large-scale systems. The state-of-the-art decentralized methods utilize imitation learning and reinforcement learning methods to teach fully decentralized policies, dramatically improving their performance. However, these methods cannot enable robots to perform tasks efficiently in relatively dense environments without communication between robots. We introduce the transformer structure into policy neural networks for the first time, dramatically enhancing the ability of policy neural networks to extract features that facilitate collaboration between robots. It mainly focuses on improving the performance of policies in relatively dense multirobot environments under conditions where robots do not communicate with each other. Furthermore, a novel imitation reinforcement learning framework is proposed by combining contrastive learning and double deep Q-network to solve the problem of difficulty training policy neural networks after introducing the transformer structure. We present results in the simulation environment and compare the resulting policy against advanced multirobot path-planning methods in terms of success rate. Simulation results show that our policy achieves state-of-the-art performance when there is no communication between robots. Finally, we experimented with a real-world case using a total of three robots in our robotic laboratory.
Diagnosis of Breast Cancer Using Random Forests

Breast cancer was the most diagnosed form of cancer in 2020. Early diagnosis of breast cancer results in a significant improvement in long-term survival rates. Current methods require consultation with experts, which is expensive and time-consuming and thus may not be accessible to all. This paper seeks to train and evaluate supervised machine learning models for the accurate and efficient detection of breast cancer. The Wisconsin Breast Cancer Database dataset describes 30 attributes of cell nuclei, including, but not limited to, their radius, texture, and concavity. It contains 569 instances, 212 of which are malignant tumors. The Random Forest algorithm outperforms other algorithms in classifying breast tumors as either malignant or benign and is thus selected as our primary model. It is trained on two different subsets of the dataset having 16 and 8 features, respectively, identified with the help of multiple feature selection methods. The Random Forest models are tested post hyperparameter tuning on a holdout set, and accuracies of 100% and 99.30% respectively. The models are also compared with four other machine learning classification algorithms: Support Vector Machine (SVM), Decision Tree, Multilayer Perceptron, and K-Nearest Neighbors. The results confirm that Random Forest is the superior method for breast cancer diagnosis.
Machine Learning Techniques for Breast Cancer Prediction

Breast cancer is main reason for mortality in woman. Prediction of breast cancer is a challenging task in medical data analysis. Doctors and pathologist required some automated tools to take decision and to differentiate between malignant and benign tumour. A machine learning (ML) algorithm helps lot to take decisions and to perform diagnosis from the data collected by medical field. Various researches show that ML techniques are helpful for decision making in breast cancer prediction. In this paper, we used various ML Classification techniques: Naïve Bayes(NB), Logistic regression (LR),Support vector machine(SVM),K-Nearest Neighbor (KNN), Decision Tree(DT), and ensemble techniques: Random forest(RF), Adaboost, XGBoost on breast cancer dataset and evaluated by using different performance measure. It has been found that both decision tree and XGBoost classifier has highest accuracy 97% among all model and highest AUC 0.999 obtained for XGBoost classifier.
Double integral‐enhanced Zeroing neural network with linear noise rejection for time‐varying matrix inverse

Abstract In engineering fields, time‐varying matrix inversion (TVMI) issue is often encountered. Zeroing neural network (ZNN) has been extensively employed to resolve the TVMI problem. Nevertheless, the original ZNN (OZNN) and the integral‐enhanced ZNN (IEZNN) usually fail to deal with the TVMI problem under unbounded noises, such as linear noises. Therefore, a neural network model that can handle the TVMI under linear noise interference is urgently needed. This paper develops a double integral‐enhanced ZNN (DIEZNN) model based on a novel integral‐type design formula with inherent linear‐noise tolerance. Moreover, its convergence and robustness are verified by derivation strictly. For comparison and verification, the OZNN and the IEZNN models are adopted to resolve the TVMI under multiple identical noise environments. The experiments proved that the DIEZNN model has excellent advantages in solving TVMI problems under linear noises. In general, the DIEZNN model is an innovative work and is proposed for the first time. Satisfyingly, the errors of DIEZNN are always less than 1 × 10 −3 under linear noises, whereas the error norms of OZNN and IEZNN models are not convergent to zero. In addition, these models are applied to the control of the controllable permanent magnet synchronous motor chaotic system to indicate the superiority of the DIEZNN.
Multi-scale hybrid attention graph convolution neural network for remote sensing images super-resolution

The super-resolution (SR) method of remote sensing image based on deep learning has achieved satisfactory performance. However, when faced with remote sensing images with large size difference and high complexity, the image SR algorithm based deep learning cannot restore more accurate information, resulting in the reconstructed images with blur problem. To solve this problem, we propose a multi-scale hybrid attention graph convolution neural network for remote sensing image SR (MAGSR). Specifically, we construct a chain module that is composed of a multi-scale hybrid attention module and graph convolution neural network to extract more multi-scale deep features and multi-scale high-frequency detail information. Moreover, we design a hybrid attention mechanism, which combines the advantages of spatial attention and channel attention to learn the spatial dependence on each channel, obtaining more abundant critical high-frequency information. Our proposed MAGSR algorithm obtains the highest PSNR and SSIM values on the Google dataset of SIRI-WHU_earth, UCMerced and NWPU-RESISC45. Compared with the suboptimal algorithm DRCAN, our proposed MAGSR achieve 0.03 dB improvements for PSNR. All experimental results show that our proposed MAGSR is superior to other mainstream SR methods and can reconstruct clear SR remote sensing image with better visual effect.
A privacy-preserving trading strategy for blockchain-based P2P electricity transactions

With the rising demand for energy in distribution networks and the growing development of communication technology, peer-to-peer (P2P) electricity trading realized by blockchain technology not only promotes the participation of distributed generation resources in the energy supply but also enhances the fairness of P2P electricity trading through the decentralized and transparent characteristics of the blockchain. However, owing to the openness and transparency of the on-chain data in the blockchain, the order information submitted by P2P market participants to the blockchain may encounter threats of information disclosure. To address this challenge, we have designed a novel three-layer architecture for P2P electricity trading, whose primary objective is to protect the private information submitted to the blockchain through a new privacy-preserving trading strategy. The proposed three-layer architecture is composed of a physical layer, information layer and market layer. An on-chain data protection method with homomorphic encryption and secure multi-party computing (SMPC) is used to sort and clear encrypted orders in the information layer, and the trading checking and safe settlement process of encrypted orders in continuous double auction (CDA) market are realized through grid security-constrained market mechanism in the market layer. In addition, the effectiveness of the proposed method is verified by case studies running on the IBM Hyperledger Fabric.
LiDAR-camera fusion: Dual transformer enhancement for 3D object detection

Recently, the progress in autonomous driving tries to leverage the strong complementarity of LiDAR point clouds and RGB images to realize a high-efficient 3D object detection task. However, some works just simply decorate the raw point clouds or point-cloud features with camera clues in a hard way, which cannot fully exploit the relevance between the two-modal data. In this paper, we propose a dual-feature interaction module that adopts a soft-fusion strategy to give guidance for the LiDAR-camera feature fusion by interacting the LiDAR and camera features with Transformer. Compared with the hard-fusion method, this soft-fusion method can decorate the LiDAR feature with a reliable image feature. Additionally, we design an uncertainty-based 3D Intersection over Union (IoU) metric in the training process. This strategy aims at modeling the unreliability of 3D IoU scores to alleviate the bad effects caused by the coupling problem of 3D properties. Experiments conducted on the KITTI dataset achieve significant improvements in the 3D object detection and bird's eye view tasks when compared with the previous arts. Especially for the task of 3D object detection, our approach obtains 0.68 and 0.45 gains for the metric of AP3D on the moderate level and hard level, respectively.