Reviewerly/exHarmony huggingface dataset:
Contrastive Learning for Fine-Grained Ship Classification in Remote Sensing Images

Fine-grained image classification can be considered as a discriminative learning process where images of different subclasses are separated from each other while the same subclass images are clustered. Most existing methods perform synchronous discriminative learning in their approaches. Although achieving promising results in fine-grained visual classification (FGVC) in natural images, these methods may fail in fine-grained ship classification (FGSC) problem in remote sensing (RS) images due to the highly &#x201C;imbalanced fineness" and &#x201C;imbalanced appearances" of ships among subclasses. To tackle the issue, we propose an asynchronous contrastive learning-based method for effective FGSC. The proposed method, which we refer to as &#x201C;Push-and-Pull Network (P2Net)", includes a &#x201C;push-out stage&#x201D; and a &#x201C;pull-in stage&#x201D;, where the first stage forces all the instances to be de-correlated and then the second one groups them into each subclass. A dual-branch network is designed to separate/de-correlate the images with each other, while an Integration Module is designed to aggregate the de-correlated images into their corresponding subclass together with a Proxy-based Module designed for acceleration. In this way, the correlation between subclasses can be decoupled, which in turn makes the final classification much easier. Our method can be trained end-to-end and requires no additional annotations other than category information. Extensive experiments are conducted on two large-scale FGSC datasets (FGSC-23 and FGSCR-42). Our method outperforms other state-of-the-art approaches. Ablation experiments also suggest the effectiveness of our design. Our code is available at https://github.com/WindVChen/Push-and-Pull-Network.
A robust subpixel refinement technique using self‐adaptive edge points matching for vision‐based structural displacement measurement

Abstract Applying a subpixel refinement technique in vision‐based displacement sensing can significantly improve the measurement accuracy. However, digital image signals from the camera are highly sensitive to drastically varying lighting conditions in the field measurements of structural displacement, causing pixels expressing a tracking target to have nonuniform grayscale intensity changes in different recording video frames. Traditional feature points‐based subpixel refinement techniques are neither robust nor accurate enough in this case, presenting challenges for accurate measurement. This paper proposes a robust subpixel refinement technique—self‐adaptive edge points matching (SEPM)—to obtain accurate subpixel‐level displacements under drastic illumination change conditions. Different from traditional feature points‐based methods, the gradient and shape information of the target edge contour are used in the SEPM calculation. Three tests under different illumination conditions were conducted to evaluate the performance of the SEPM. The results show that the SEPM is capable of producing accurate subpixel‐level displacement data with less 1/16‐pixel root mean‐square error.
Edge Intelligence: Federated Learning-Based Privacy Protection Framework for Smart Healthcare Systems

Federated learning methods offer secured monitor services and privacy-preserving paradigms to end-users and organisations in the Internet of Things networks such as smart healthcare systems. Federated learning has been coined to safeguard sensitive data, and its global aggregation is often based on a centralised server. This design is vulnerable to malicious attacks and could be breached by privacy attacks such as inference and free-riding, leading to inefficient training models. Besides, uploaded analysing parameters by patients can reveal private information and the threat of direct manipulation by the central server. To address these issues, we present a three-fold Federated Edge Aggregator, the so-called Edge Intelligence, a federated learning-based privacy protection framework for safeguarding Smart Healthcare Systems at the edge against such privacy attacks. We employ an iteration-based Conventional Neural Network (CNN) model and artificial noise functions to balance privacy protection and model performance. A theoretical convergence bound of Edge Intelligence on the trained federated learning model's loss function is also introduced here. We evaluate and compare the proposed framework with the recently established methods using model performance and privacy budget on popular and recent datasets: MNIST, CIFAR10, STL10, and COVID19 chest x-ray. Finally, the proposed framework achieves 90% accuracy and a high privacy rate demonstrating better performance than the baseline technique.
Deep Learning Model of Image Classification Using Machine Learning

Not only were traditional artificial neural networks and machine learning difficult to meet the processing needs of massive images in feature extraction and model training but also they had low efficiency and low classification accuracy when they were applied to image classification. Therefore, this paper proposed a deep learning model of image classification, which aimed to provide foundation and support for image classification and recognition of large datasets. Firstly, based on the analysis of the basic theory of neural network, this paper expounded the different types of convolution neural network and the basic process of its application in image classification. Secondly, based on the existing convolution neural network model, the noise reduction and parameter adjustment were carried out in the feature extraction process, and an image classification depth learning model was proposed based on the improved convolution neural network structure. Finally, the structure of the deep learning model was optimized to improve the classification efficiency and accuracy of the model. In order to verify the effectiveness of the deep learning model proposed in this paper in image classification, the relationship between the accuracy of several common network models in image classification and the number of iterations was compared through experiments. The results showed that the model proposed in this paper was better than other models in classification accuracy. At the same time, the classification accuracy of the deep learning model before and after optimization was compared and analyzed by using the training set and test set. The results showed that the accuracy of image classification had been greatly improved after the model proposed in this paper had been optimized to a certain extent.
Approximate inferring with confidence predicting based on uncertain knowledge graph embedding

Uncertainty is a natural character of knowledge, while it is still tough to be encoded into the knowledge graph embedding space that can be employed for machine learning tasks. However, the approximate inference could be performed in the embedding space, if confidence, real-value representation of the uncertainty of knowledge facts, can be learned by neural networks. To tackle this, a simple yet effective confidence predicting method is proposed, and several approximate inferring are efficiently performed based on these predictions. The model is a two-step model: knowledge elements embedding step, in which knowledge facts regarded as short sentences are fed into the natural language model to get entity and relation embedding vectors; and confidence learning step, in which the confidence distribution of knowledge facts in the knowledge graph are learned utilizing the recurrent neural network in order to carry out approximate inference. The experience demonstrates that the model achieves better results than state-of-the-art on the link prediction task over uncertain knowledge graph embedding. Uncertainty inferring grounded on predicted confidence is more accurate, feasible, and meaningful for serval knowledge inferring tasks: transitivity, composition inferring, and probabilistic soft logic inferring. Likewise, the proposed approach achieves the best tradeoff between efficiency and accuracy of uncertain knowledge graph embedding and inferring, and can be used to handle large size knowledge graphs at lower time consumption because of the simplicity.
VulSlicer: Vulnerability detection through code slicing

There has been a multitude of techniques proposed for identifying vulnerabilities in software. Forcing a program into a vulnerable state has become increasingly unscalable, given the size of the programs and the number of possible execution states. At the same time, techniques that are looking for vulnerability signatures are marred with weak and incomplete signatures. This is not to say that such techniques have failed to identify previously unknown vulnerabilities in the code. However, they have inherent weaknesses, which result in identifying vulnerabilities that are limited in type and complexity. We propose a novel technique to extract succinct vulnerability-relevant statements representing the self-contained nature of vulnerabilities and reproduce the vulnerable behavior independently of the rest of the program. We also introduce an innovative technique to slice target programs and search for similar vulnerability-relevant statements in them. We developed VulSlicer, a prototype system capable of extracting vulnerability-relevant statements from vulnerable programs and searching for them on target programs at scale. Furthermore, we have examined four candidate open-source projects and have been able to identify 118 potential vulnerabilities, out of which 94 were found to be silently patched, and from the remaining reported cases, three were confirmed by obtaining a CVE designation.
FSODS: A Lightweight Metalearning Method for Few-Shot Object Detection on SAR Images

At present, few-shot object detection research in the field of optical remote sensing images has been conducted, but few-shot object detection in the field of SAR images have rarely been explored. To this end, this paper proposes a lightweight meta-learning-based SAR image few-shot object detection method, which improves the accuracy and speed of SAR image few-shot object detection from a more balanced perspective. First, we introduce the latest FSODM method in optical remote sensing as a benchmark framework. Second, a lightweight meta-feature extractor named DarknetS is designed to enhance the feature representation of SAR images and improve detection timeliness. Furthermore, we build a new aggregation module called AggregationS, which encodes support features and query features into the same feature subspace via a novel transformer encoder. This module design can better extract the correlation and saliency between different classes in the support set, improve the detection accuracy of the query set, and enhance the detection generalization performance of new classes. Finally, we built several real-world SAR image few-shot object detection datasets to verify the effectiveness of the method. Experimental results show that FSODS can achieve a better object detection performance compared to the baseline model under the condition that only a small amount of labelled data is required for new classes of SAR image objects.
ROSEmark: Robust semi-blind ECG watermarking scheme using SWT-DCT framework

Coronary heart disease is a major cause of fatality worldwide. Online access to a patient's ECG record on networked hand-held devices enables its timely assessment by a cardiologist. However, the transmission of the ECG data on an unsecured network leads to a major concern about its authenticity. As a solution to this problem, this paper proposes ROSEmark – a robust semi-blind scheme for watermarking the ECG records. Each ECG record from the MIT-BIH Arrhythmia database is decomposed into equal-sized segments, and a set of key segments is selected randomly for watermarking. Each key segment is transformed using a 3-level stationary wavelet transform (SWT), and the resultant approximation vector is subjected to discrete cosine transform (DCT). An extreme learning machine (ELM) is applied for modulating the low-frequency coefficients, and the watermark is embedded in the modulated coefficients. Further, to ensure the security of the algorithm, the watermark is XORed with a randomly generated bitmask sequence. Finally, ROSEmark's performance is evaluated for fidelity and robustness using a variety of interferences. It is found that ROSEmark yields an average PSNR and PRD values of 50.72 dB and 0.7636, respectively. The experiments demonstrate that ROSEmark outperforms its competitors in terms of the standard evaluation metrics and is also time-efficient, thus making it suitable for real-time applications such as telemedicine.
Spaks: Self-paced multiple kernel subspace clustering with feature smoothing regularization

Considerable attention to kernel subspace clustering has emerged in recent years due to its excellent performance in grouping data points from multiple manifolds. In particular, multiple kernel learning (MKL) is commonly used in kernel subspace clustering models to alleviate the difficulty of choosing a suitable kernel function, which brings out numerous MKL-based subspace clustering methods. Despite these successes, these MKL methods cannot guarantee that the feature representation produced by the learned kernel function is easy-to-divide, which is critical for the clustering task. Besides, the weighting strategies in these methods ignore the respective fitting ability of each base kernel, which may result in inappropriate weight assignment and inferior performance. To address these problems, this paper proposes a Self-Paced Smooth Multiple Kernel Subspace Clustering (Spaks) method. In Spaks, we first incorporate a feature smoothing regularization into the kernel subspace clustering model to facilitate the smoothness of feature representation on the affinity graph, thus ensuring that the data points between different subspaces are sufficiently separable. Then we implement the multiple kernel mechanism in the self-paced learning framework. In this way, the weights of base kernels depend on their corresponding single-kernel optimization objective function value, which indicates that a greater weight will be assigned to a kernel with a smaller objective. At last, we optimize the formulated model by an alternative iteration scheme, so that the optimal affinity matrix is obtained for clustering. Extensive experiments on seven benchmark data sets validate the superiority of the proposed Spaks method on the clustering task.
Experimental demonstration of novel scheme of HZO/Si FeFET reservoir computing with parallel data processing for speech recognition

We propose a novel scheme of the reservoir computing using ferroelectric gate MOSFETs (FeFETs) in a parallel data processing manner and apply this scheme to speech recognition. The effectiveness of combining the time responses of the drain, source and substrate currents on the reservoir performance as well as that of the usage of pre-cycled devices on mitigation of the influence of the endurance characteristics is experimentally shown. Several ideas on the FeFET reservoir computing system such as optimization of the virtual node number per time step, analog/digital input waveform and combination of different frequency channels are introduced to enhance the accuracy of a speech recognition task. As a result, we experimentally demonstrate the classification accuracy of 95.9% in spoken digit recognition, which is comparable to that of a software reservoir.
Metaverse Space Ecological Scene Design Based on Multimedia Digital Technology

Yuan cosmos is a virtual world linked and created by scientific and technological means, which is mapped and interacted with the real world, and a digital living space with a new social system. With the increasing popularity of data acquisition and production equipment, people are increasingly convenient to produce multimedia data such as images, graphics, audio, video, animation, and three-dimensional models. In addition to the rapid development of digital technology itself, the biological information technology related to digital technology also greatly promotes the emergence of the metauniverse. This paper aims to study the application of multimedia digital technology to the ecological scene design of metauniverse space, introduces the related concepts of metauniverse and multimedia digital technology, expounds the related methods of multimedia digital technology and neural network related algorithms, and then takes the three-dimensional simulation of the auditory system in the interactive multisensory simulation system of the constituent elements of metauniverse as an example. The mel-frequency cepstrum coefficient (MFCC) is used to simulate the auditory characteristics of the auditory periphery (cochlea) as the perceptual end of the model. A variety of bionic mechanisms are used in the model, such as designing the connection mode of neurons, learning state and release effect, and the regeneration mechanism of neurons. For the verification of the performance of the model, the speech sample database, including English words and phrases, is recorded and the speech content information recognized by the model by means of speech recognition is experimented. The experimental results show that, in terms of phrase accuracy, the DN-1 model improves 2.59% and the DN-2 model improves 2.77% compared with MFCC feature on the basis of mixed features. When only DBN features are used, the performance improvement rate of the developmental network model is small.
Dense Oil Tank Detection and Classification via YOLOX-TR Network in Large-Scale SAR Images

Oil storage tank detection and classification in synthetic aperture radar (SAR) images play a vital role in monitoring energy distribution and consumption. Due to the SAR side-looking imaging geometry and multibouncing scattering mechanism, dense oil tank detection and classification tasks have faced more challenges, such as overlapping, blurred contours, and geometric distortion, especially for small-sized tanks. To address the above issues, this paper proposes YOLOX-TR, an improved YOLOX based on the Transformer encoder and structural reparameterized VGG-like (RepVGG) blocks, to achieve end-to-end oil tank detection and classification in densely arranged areas of large-scale SAR images. Based on YOLOX, the Transformer encoder, a self-attention-based architecture, is integrated to enhance the representation of feature maps and capture the region of interest of oil tanks in densely distributed scenarios. Furthermore, RepVGG blocks are employed to reparameterize the backbone with multibranch typologies to strengthen the distinguishable feature extraction of multi-scale oil tanks without increasing computation in inference time. Eventually, comprehensive experiments based on a Gaofen-3 1 m oil tank dataset (OTD) demonstrated the effectiveness of the Transformer encoder and RepVGG blocks, as well as the performance superiority of YOLOX-TR with a mAP and mAP0.5 of 60.8% and 94.8%, respectively.
Exploiting generative self-supervised learning for the assessment of biological images with lack of annotations

Computer-aided analysis of biological images typically requires extensive training on large-scale annotated datasets, which is not viable in many situations. In this paper, we present Generative Adversarial Network Discriminator Learner (GAN-DL), a novel self-supervised learning paradigm based on the StyleGAN2 architecture, which we employ for self-supervised image representation learning in the case of fluorescent biological images.We show that Wasserstein Generative Adversarial Networks enable high-throughput compound screening based on raw images. We demonstrate this by classifying active and inactive compounds tested for the inhibition of SARS-CoV-2 infection in two different cell models: the primary human renal cortical epithelial cells (HRCE) and the African green monkey kidney epithelial cells (VERO). In contrast to previous methods, our deep learning-based approach does not require any annotation, and can also be used to solve subtle tasks it was not specifically trained on, in a self-supervised manner. For example, it can effectively derive a dose-response curve for the tested treatments.Our code and embeddings are available at https://gitlab.com/AlesioRFM/gan-dl StyleGAN2 is available at https://github.com/NVlabs/stylegan2 .
The Influences of Virtual Reality Shopping Characteristics on Consumers’ Impulse Buying Behavior

With the rapid development of virtual reality (VR) hardware devices and virtual technologies, VR applications in retailing are increasingly becoming an inevitable trend of future shopping, particularly essential to the "metaverses" in the digitalization 4.0 era. Thanks to its superiority over traditional e-commerce, virtual stores can provide highly interactive and immersive sensory shopping experiences to system users or end customers. Todays, both retailers and scholars, pay much attention to impulse buying behavior. Nevertheless, this domain remains unaddressed in the context of virtual commerce. Drawing upon the Stimulus-Organism-Response paradigm, this article investigates the effectiveness of VR shopping platform factors, namely interactivity and vividness, that impacts consumers' internal stages (telepresence, perceived diagnosticity, and playfulness) and lead to the urge to buy impulsively in virtual shopping stores. A full 2 × 2 × 2 factorial design experiment was conducted with 227 participants. The empirical results show that interactivity and vividness positively impact telepresence, perceived diagnosticity, and playfulness, which trigger consumers' urge to buy impulsively. Furthermore, the role of the impulsiveness trait in driving the urge to buy impulsively was further explicitly investigated in the VR environment. Detailed discussions and future research directions were also provided.
Privacy‐preserving federated learning cyber‐threat detection for intelligent transport systems with blockchain‐based security

Abstract Artificial intelligence (AI) techniques implemented at a large scale in intelligent transport systems (ITS), have considerably enhanced the vehicles' autonomous behaviour in making independent decisions about cyber threats, attacks, and faults. While, AI techniques are based on data sharing among the vehicles, it is important to note that sensitive data cannot be shared. Thus, federated learning (FL) has been implemented to protect privacy in vehicles. On the other hand, the integrity of data and the safety of aggregation are ensured by using blockchain technology. This paper applied classification approaches to VANET and ITS cyber‐threats detection at the vehicle. Subsequently, by using blockchain and by applying an aggregation strategy to different models, models from the previous step were uploaded in a smart contract. Lastly, we returned the updated models to the vehicles. Furthermore, we conducted an experimental study to measure the effectiveness of the proposed prototype. In this paper, the VeReMi data set was distributed in a balanced manner into five parts in the experimental study. Thus, classification techniques were executed by each vehicle separately, and models were generated. Upon the aggregation of the models in blockchain, they were returned to the vehicles. Lastly, the vehicles updated their decision functions and accessed the precision and accuracy of cyber‐threat detection. The results indicated that the precision and accuracy decreased by 7.1% on average with comparable F 1‐score and recall. Our solution ensures the privacy preservation of vehicles whereas blockchain guarantees the safety of aggregation technique and low gas consumption.