Reviewerly/exHarmony huggingface dataset:
Dual Conditional Normalization Pyramid Network for Face Photo-Sketch Synthesis

Face photo-sketch synthesis has undergone remarkable progress with the rapid development of deep learning techniques. Cutting-edge methods directly learn the cross-domain mapping between photos and sketches, which ignores the available reference samples. We argue that the reference samples can provide adequate prior information on texture and content in this task and improve the visual performance of synthetic images. This paper proposes a Dual Conditional Normalization Pyramid (DCNP) network with a multi-scale pyramid structure. The core of the DCNP network is a Dual Conditional Normalization (DCN) based architecture, which can obtain prior information on different semantics from reference samples. Specifically, DCN contains two conditional normalization branches. The first branch allows for spatially-adaptive normalization of the reference image conditioned on the semantic mask of the input image. The second branch enables adaptive instance normalization of the input image conditioned on the reference image. DCN can emphasize the isolated importance of textural and spatial factors by disintegrating the entire cross-domain mapping into two branches. To avoid information redundancy and improve the final performance, we propose a Gated Channel Attention Fusion (GCAF) module to distill and fuse the helpful information of the two branches. Qualitative and quantitative experimental results demonstrate the superior performance of the proposed method over the state-of-the-art approaches in structural information preservation and realistic texture generation. The code is public in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Tony0720/DCNP</uri> .
Machine Learning for PIN Side-Channel Attacks Based on Smartphone Motion Sensors

Motion sensors are integrated into all mobile devices, providing information useful for several purposes.However, these sensor data can be read by any application and by websites accessed through a browser, without requiring security permissions.In this paper, we show that information about smartphone movements can lead to the identification of a Personal Identification Number (PIN) tapped by the user.To reduce the amount of sniffed data, we use an event-driven approach, where motion sensors are sampled only when a key is pressed.The acquired data are used to train a machine learning algorithm for the classification of the keystrokes in a supervised manner.We also consider that users insert the same PIN each time authentication is required, leading to further side-channel information available to the attacker.Numerical results show the feasibility of cyber-attacks based on motion sensors.For example, 4-digit PINs are correctly recognized at the first attempt with an accuracy of 37%, and in five attempts with an accuracy of 63%.
Robust Multi-View Clustering Through Partition Integration on Stiefel Manifold

Multi-view clustering aims at integrating information from different views to improve clustering performance. Recent methods integrate multiple view-specific partition matrices to seek a consensus one and have demonstrated promising clustering performance in various applications. However, the clustering performance of such methods heavily relies on the consensus partition matrix estimated by the arithmetic mean in euclidean space and thus is highly susceptible to noise corruption. To this end, this article proposes to learn a consensus partition matrix through the geometric mean on the manifold to achieve robust clustering. Specifically, the multiple view-specific partition matrices can be regarded as points residing in the Stiefel manifold and enable a manifold-based integration. Consequently, the view-specific partition matrices are integrated by estimating a consensus partition matrix as the center point on the Stiefel manifold. Such a partition integration boils down to the Fréchet mean problem on a manifold, which is solved by the intrinsic manifold-based optimization and proves effective in providing a more robust estimation against noise. Experimental results on seven benchmark datasets demonstrate the effectiveness and noise-robustness of our proposed method in comparison to eight competitive methods.
Two-Stream Prototype Learning Network for Few-Shot Face Recognition Under Occlusions

Few-shot face recognition under occlusion (FSFRO) aims to recognize novel subjects given only a few, probably occluded face images, and it is challenging and common in real-world scenarios. Unknown occlusions may deteriorate the class prototypes, while an occluded image in the support set may be critical for recognition if the query image is occluded. This motivates us to propose a novel Two-stream Prototype Learning Network (TSPLN) for FSFR under occlusions by simultaneously considering the quality of support images and their relevance to the query i mage. Specifically, we design a two-stream architecture, which mainly consists of a support-centered stream and query-centered stream, to learn the optimal class prototypes. The former stream is to reduce the negative impact of occluded images on the prototype. This is achieved by exploring the similarities between different images in the support set. In the query-centered stream, we exploit the relevance between the query and support set based on feature alignment (FA). We conduct extensive experiments on two popular datasets: CASIA-WebFace and RMFRD. The experimental results show that our proposed method achieves the state-of-the-art performance for occluded face recognition in the few-shot setting.
LayerNet: A One-Step Layered Network for Semantic Segmentation at Night

We have collected a novel, nighttime scene dataset, called Rebecca, including 600 real images captured at night with pixel-level semantic annotations, which is currently scarce and can be invoked as a new benchmark. In addition, we proposed a one-step layered network, named LayerNet, to combine local features rich in appearance information in the shallow layer, global features abundant in semantic information in the deep layer, and middle-level features in between by explicitly modeling multistage features of objects in the nighttime. In addition, a multihead decoder and a well-designed hierarchical module are utilized to extract and fuse features of different depths. Numerous experiments show that our dataset can significantly improve the segmentation ability of the existing models for nighttime images. Meanwhile, our LayerNet achieves the state-of-the-art accuracy on Rebecca (65.3% mIOU). The dataset is available at https://github.com/Lihao482/REebecca.
Spiking Neural Network Nonlinear Demapping on Neuromorphic Hardware for IM/DD Optical Communication

Neuromorphic computing implementing spiking neural networks (SNN) is a promising technology for reducing the footprint of optical transceivers, as required by the fast-paced growth of data center traffic. In this work, an SNN nonlinear demapper is designed and evaluated on a simulated intensity-modulation direct-detection link with chromatic dispersion. The SNN demapper is implemented in software and on the analog neuromorphic hardware system BrainScaleS-2 (BSS-2). For comparison, linear equalization (LE), Volterra nonlinear equalization (VNLE), and nonlinear demapping by an artificial neural network (ANN) implemented in software are considered. At a pre-forward error correction bit error rate of 2e-3, the software SNN outperforms LE by 1.5 dB, VNLE by 0.3 dB and the ANN by 0.5 dB. The hardware penalty of the SNN on BSS-2 is only 0.2 dB, i.e., also on hardware, the SNN performs better than all software implementations of the reference approaches. Hence, this work demonstrates that SNN demappers implemented on electrical analog hardware can realize powerful and accurate signal processing fulfilling the strict requirements of optical communications.
A VAE-Based User Preference Learning and Transfer Framework for Cross-Domain Recommendation

The core idea of cross-domain recommendation is to alleviate the problem of data scarcity. Previous methods have made brilliant successes. However, many of them mainly focus on learning an ideal mapping function across-domains, ignoring the user preferences within a specific domain, which leads to suboptimal results. In this paper, we propose a Cross-Domain Recommendation Variational AutoEncoder framework (CDRVAE), a novel extension of a variational autoencoder on cross-domain recommendations for user behaviour distribution modeling. It applies a new hybrid architecture of VAE as the backbone and simultaneously constructs two information flows, within-domain and cross-domain modeling. For the former, an asymmetric codec structure is designed to reconstruct preference distribution from domain-specific latent factors. To relieve the posterior collapse dilemma, a combined prior is employed to increase the distribution complexity. The equivalent transition by a transformation matrix and the unobserved interaction generation by cross-domain reconstruction contribute to the latter. We combine all the above components for the more accurate and reliable user features. Extensive experiments are conducted on three public benchmark datasets to validate the effectiveness of the proposed CDRVAE. Experimental results demonstrate that CDRVAE is consistently superior to other state-of-the-art alternative baseline models.
Integration of Prediction Scores From Various Automated Essay Scoring Models Using Item Response Theory

In automated essay scoring (AES), essays are automatically graded without human raters. Many AES models based on various manually designed features or various architectures of deep neural networks have been proposed over the past few decades. Each AES model has unique advantages and characteristics. Therefore, rather than using a single AES model, appropriate integration of predictions from various AES models is expected to achieve higher scoring accuracy. In the present paper, we propose a method that uses item response theory to integrate prediction scores from various AES models while taking into account differences in the characteristics of scoring behavior among models. It is found that the proposed method achieves higher accuracy than that of individual AES models and conventional score-integration methods. Furthermore, the proposed method facilitates interpreting each AES model's scoring characteristics and score-integration mechanism.
SCMA: A Scattering Center Model Attack on CNN-SAR Target Recognition

Convolutional neural networks (CNNs) have been widely used in SAR (Synthetic Aperture Radar) target recognition, which can extract feature automatically. However, due to its own structural flaws, CNNs are easy to be fooled by adversarial examples, even if they have excellent performance. In this letter, a novel attack named scattering center model attack (SCMA) is designed, and its generation process does not rely on the prior knowledge of any neural network. Therefore, we can get a stable way which can be applied to any neural network. In addition, an improved scattering center model extraction method, which is the pre-part of SCMA, can filter out the useless noise to optimize the stability of attack. In the experiment, SCMA is compared with advanced attack algorithms. From the experimental results, it is clear to find that SCMA has excellent performance in terms of transfer attack success rate. Furthermore, visualization and interpretability analysis underpin the theoretical feasibility of SCMA.
TB-Logger: Secure Vehicle Data Logging Method Using Trusted Execution Environment and Blockchain

With the development of IT technologies, event data recorder (EDR) devices are now installed in modern vehicles to record and analyze vehicle-related events. As data recorded in EDRs began to be used as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">conclusive proof</i> in courts, many researchers turned their to focus on developing methodologies that can protect data recorded in EDRs from data forgery attacks. In general, these existing methods generate verification values for EDR data by using a digital signature algorithm. However, these methods do not provide a way to protect the data stored in an EDR from data forgery attacks in the event of an emergency, such as a car accident, and it is not possible to properly create a verification value for the EDR data due to unexpected and sudden events like a power supply problem. Thus, in this paper, we propose TB-Logger, a novel method that protects EDR data even when there is an emergency situation. TB-Logger relies on the trusted execution environment (TEE) to protect EDR data from data forgery attacks, which ultimately result in data modification, reordering, and deletion. In addition, in the event of an emergency, TB-Logger utilizes a blockchain system to store verification values and publicly verify the data generated during the event. We evaluated the practicality of TB-Logger using two real vehicles: the Hyundai Avante CN7 and the Tesla Model 3. Through these tests, we confirmed that TB-Logger can generate verification values for EDR data without incurring any data loss.
HCSC: A Hierarchical Certificate Service Chain Based on Reputation for VANETs

In vehicular ad hoc networks (VANETs), there are many resource constrained communication nodes, distributed geographical locations, and low delay requirements. The authentication system is required to be distributed. However, the information interaction between traditional distributed authentication entities (AEs) is not transparent, and the use of blockchain is only to save certificate status records on the chain. There is a lack of trusted security management mechanisms between AEs, so there is a risk of collusion attacks and behavior record forgery and tamper attacks between AEs. Therefore, this paper mainly addresses the management problems of opaque interaction and insufficient security verification of distributed authentication entities in the VANETs scenario. In this paper, based on the security features of blockchain technology, such as public audit and tamper proof block structure, and distributed trust consensus, we propose a hierarchical and trust certificate service chain based on reputation value called HCSC. We have a set of safe and reliable management mechanisms between distributed authentication entities such as the Bitcoin system. First, we integrate the traditional hierarchical authentication system and the blockchain network into a new hierarchical certificate service chain. Then, we propose a reputation evaluation model based on the logistic regression model by quantifying the behavior records of the AEs. In addition, a new hierarchical reputation consensus based on delegated proof of stake (DPoS) and proof of work (PoW) is given. Then, a fast verification based on block height and security verification for the efficiency and security of certificate verification is proposed. Finally, the security analysis is given, and a prototype implementation of HCSC is developed based on Fabric. The experimental simulation and security analysis reveal that the HCSC is very efficient and suitable for the above issues.
Contrastive 3D Human Skeleton Action Representation Learning via CrossMoCo With Spatiotemporal Occlusion Mask Data Augmentation

Self-supervised learning methods for 3D skeleton-based action recognition via contrastive learning have obtained competitive achievements compared to classical supervised methods. Current researches show that adding a Multilayer Perceptron (MLP) to the top of the base encoder can extract high-level and global positive representations. Using a negative memory bank to store negative samples dynamically can balance the ample storage and feature consistency. However, these methods need to consider that the MLP lacks accurate encoding of fine-grained local features, and a memory bank needs rich and diverse negative sample pairs to match positive representations from different encoders. This paper proposes a new method called Cross Momentum Contrast (CrossMoCo), composed of three parts: ST-GCN encoder, ST-GCN encoder with MLP encoder (ST-MLP encoder), and two independent negative memory banks. The two encoders encode the input data into two positive feature pairs. Learning the cross representations of the two positive pairs is helpful for the model to extract both the global and the local information. Two independent negative memory banks update the negative samples according to different positive representations from two encoders, diversifying the negative samples' distribution and making negative representations close to the positive features. The increasing classification difficulty will improve the model's ability of contrastive learning. In addition, the spatiotemporal occlusion mask data augmentation method is used to enhance positive samples' information diversity. This method takes the adjacent skeleton joints that can form a skeleton bone as a mask unit, which can reduce the information redundancy after data augmentation since adjacent joints may carry similar spatiotemporal information. Experiments on the PKU-MMD Part II dataset, the NTU RGB+D 60 dataset, and the NW-UCLA dataset show that the CrossMoCo framework with spatiotemporal occlusion mask data augmentation has achieved a comparable performance.
A Modular Ice Cream Factory Dataset on Anomalies in Sensors to Support Machine Learning Research in Manufacturing Systems

A small deviation in manufacturing systems can cause huge economic losses, and all components and sensors in the system must be continuously monitored to provide an immediate response.The usual industrial practice is rather simplistic based on brute force checking of limited set of parameters often with pessimistic pre-defined bounds.The usage of appropriate machine learning techniques can be very valuable in this context to narrow down the set of parameters to monitor, define more refined bounds, and forecast impending issues.One of the factors hampering progress in this field is the lack of datasets that can realistically mimic the behaviours of manufacturing systems.In this paper, we propose a new dataset called MIDAS (Modular Ice cream factory Dataset on Anomalies in Sensors) to support machine learning research in analog sensor data.MIDAS is created using a modular manufacturing simulation environment that simulates the ice cream-making process.Using MIDAS, we evaluated four different supervised machine learning algorithms (Logistic Regression, Decision Tree, Random Forest, and Multilayer Perceptron) for two different problems: anomaly detection and anomaly classification.The results showed that multilayer perceptron is the most suitable algorithm with respect to model accuracy and execution time.We have made the data set and the code for the experiments publicly available, to enable interested researchers to enhance the state of the art by conducting further studies.
Resolving Power Equipment Data Inconsistency via Heterogeneous Network Alignment

This paper studies the problem of resolving data inconsistency from multiple sources in managing data related to power equipment for China’s state grid corporation. This paper proposes to automatically align inconsistent devices from multiple sources, i.e., the same devices that have multiple entries with different values in each source, by HENGE, a HEtetrogeneous Network GEneration model. HENGE builds multiple data sources into a heterogeneous graph, and captures complex physical and semantic relationships among devices. HENGE combines both feature and relational information and improves alignment accuracy by feature-enhanced residual graph encoder and disentangled representation learning. HENGE is capable to learn from a small amount of labeled data, through a uniformity autoencoder trained on an unsupervised generation task. Experiments on two real-world datasets demonstrate the capability of HENGE in resolving inconsistent device entries in multiple sources.
A Comprehensive Survey of Context-Aware Continuous Implicit Authentication in Online Learning Environments

User authentication is crucial in the digital learning environment to preserve the integrity and reliability of the learning process. Implicit authentication using biometrics has been proposed to improve the user experience while resolving the issues that password dominant authentication faces. Implicit authentication does not require explicit user actions as it is a background process that implicitly acquires a user’s identifying information through sensors embedded within the authenticating devices. To accommodate a variety of user contexts, context-aware implicit authentication has gained attention especially in the mobile device domain, but it has not been fully explored in digital learning environments. This study is motivated to determine how implicit authentication can observe students’ behaviour without causing disruption to the learning activity. The study provides a structured systematic review of the existing literature to identify and discuss the structure of context-aware continuous implicit authentication systems and future directions. The study found that requirements in the future will be: 1) to consider diverse authenticators to cover all possible user interactions with online learning environments, including coverage of course participants not engaged with online exams; 2) to investigate template adaptation to overcome template ageing issues with biometrics; and 3) to explore evaluation approaches of context-aware implicit authentication systems.
Shape-Aware Monocular 3D Object Detection

The 3D object detection is the key issue in the autonomous driving system. This issue is particularly challenging when the detection only relies on a single perspective camera. The anchor-free and keypoint-based models receive increasing attention recently due to their effectiveness and simplicity. However, most of these methods are vulnerable to the occlusion and truncation of objects. In this paper, a single-stage monocular 3D object detection model is proposed. An instance-segmentation head is integrated into the model training, which allows the model to be aware of the visible shape of a target object. Therefore, the detection largely avoids interference from irrelevant regions surrounding the target objects. In addition, we also reveal that the popular IoU-based evaluation metrics, which were originally designed for evaluating stereo or LiDAR-based detection methods, are insensitive to the improvement achieved by the monocular 3D object detection algorithms. A novel evaluation metric, namely average depth similarity (ADS) is proposed for the monocular 3D object detection models. Our method outperforms the comparison baseline in terms of both the popular and the proposed evaluation metrics while maintaining real-time efficiency.