Reviewerly/exHarmony huggingface dataset:
Pose Attention-Guided Profile-to-Frontal Face Recognition

In recent years, face recognition systems have achieved exceptional success due to promising advances in deep learning architectures. However, they still fail to achieve expected accuracy when matching profile images against a gallery of frontal images. Current approaches either perform pose normalization (i.e., frontalization) or disentangle pose information for face recognition. We instead propose a new approach to utilize pose as an auxiliary information via an attention mechanism. In this paper, we hypothesize that pose attended information using an attention mechanism can guide contextual and distinctive feature extraction from profile faces, which further benefits a better representation learning in an embedded domain. To achieve this, first, we design a unified coupled profile-to-frontal face recognition network. It learns the mapping from faces to a compact em-bedding subspace via a class-specific contrastive loss. Second, we develop a novel pose attention block (PAB) to specially guide the pose-agnostic feature extraction from profile faces. To be more specific, PAB is designed to explicitly help the network to focus on important features along both “channel” and “spatial” dimension while learning discriminative yet pose-invariant features in an embedding subspace. To validate the effectiveness of our proposed method, we conduct experiments on both controlled and in-the-wild benchmarks including Multi-PIE, CFP, IJB-C, and show superiority over the state-of-the-arts.
Reinforcement Learning Based Path Exploration for Sequential Explainable Recommendation

Recent advances in path-based explainable recommendation systems have attracted increasing attention thanks to the rich information from knowledge graphs. Most existing explainable recommendations only utilize static knowledge graphs and ignore the dynamic user-item evolutions, leading to less convincing and inaccurate explanations. Although some works boost the performance and explainability of recommendations through modeling the user's temporal sequential behavior, most of them either only focus on modeling the user's sequential interactions within a path or independently and separately of the recommendation mechanism. Moreover, some path-based explainable recommendations use random selection or traditional machine learning methods to decrease the volume of explainable paths, which cannot guarantee high quality of the explainable paths for the recommendation. To deal with the problem, recent path exploration use reinforcement learning to improve diversity and quality. However, unsupervised training leads to low-efficiency path exploration. Therefore, we propose a novel <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>T</u></b> emporal <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>M</u></b> eta-path Guided <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>E</u></b> xplainable <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>R</u></b> ecommendation leveraging <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>R</u></b> einforcement <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>L</u></b> earning ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TMER-RL</b> ), which utilizes supervised reinforcement learning to explore item-item paths between consecutive items with attention mechanisms to sequentially model dynamic user-item evolutions on a dynamic knowledge graph for the explainable recommendation. Extensive evaluations of TMER-RL on two real-world datasets show state-of-the-art performance compared to recent strong baselines.
FedCare: Federated Learning for Resource-Constrained Healthcare Devices in IoMT System

In social IoMT systems, resource-constrained devices face the challenges of limited computation, bandwidth, and privacy in the deployment of deep learning models. Federated learning (FL) is one of the solutions to user privacy and provides distributed training among several local devices. In addition, it reduces the computation and bandwidth of transferring videos to the central server in camera-based IoMT devices. In this work, we design an edge-based federated framework for such devices. In contrast to traditional methods that drop the resource-constrained stragglers in a federated round, our system provides a methodology to incorporate them. We propose a new phase in the FL algorithm, known as split learning. The stragglers train collaboratively with the nearest edge node using split learning. We test the implementation using heterogeneous computing devices that extract vital signs from videos. The results show a reduction of 3.6 h in the training time of videos using the split learning phase with respect to the traditional approach. We also evaluate the performance of the devices and system with key parameters, CPU utilization, memory consumption, and data rate. Furthermore, we achieve 87.29% and 60.26% test accuracy at the nonstragglers and stragglers, respectively, with a global accuracy of 90.32% at the server. Therefore, FedCare provides a straggler-resistant federated method for a heterogeneous system for social IoMT devices.
Unsupervised Face Morphing Attack Detection via Self-paced Anomaly Detection

The supervised-learning-based morphing attack detection (MAD) solutions achieve outstanding success in dealing with attacks from known morphing techniques and known data sources. However, given variations in the morphing attacks, the performance of supervised MAD solutions drops significantly due to the insufficient diversity and quantity of the existing MAD datasets. To address this concern, we propose a completely unsupervised MAD solution via self-paced anomaly detection (SPL-MAD) by leveraging the existing large-scale face recognition (FR) datasets and the unsupervised nature of convolutional autoencoders. Using general FR datasets that might contain unintentionally and unlabeled manipulated samples to train an autoencoder can lead to a diverse reconstruction behavior of attack and bona fide samples. We analyze this behavior empirically to provide a solid theoretical ground for designing our unsupervised MAD solution. This also results in proposing to integrate our adapted modified self-paced learning paradigm to enhance the reconstruction error separability between the bona fide and attack samples in a completely unsupervised manner. Our experimental results on a diverse set of MAD evaluation datasets show that the proposed unsupervised SPL-MAD solution outperforms the overall performance of a wide range of supervised MAD solutions and provides higher generalizability on unknown attacks. Training codes and pre-trained models are publicly released <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://github.com/meilfang/SPL-MAD.
Non-IID quantum federated learning with one-shot communication complexity

Federated learning refers to the task of machine learning based on decentralized data from multiple clients with secured data privacy. Recent studies show that quantum algorithms can be exploited to boost its performance. However, when the clients' data are not independent and identically distributed (IID), the performance of conventional federated algorithms is known to deteriorate. In this work, we explore the non-IID issue in quantum federated learning with both theoretical and numerical analysis. We further prove that a global quantum channel can be exactly decomposed into local channels trained by each client with the help of local density estimators. This observation leads to a general framework for quantum federated learning on non-IID data with one-shot communication complexity. Numerical simulations show that the proposed algorithm outperforms the conventional ones significantly under non-IID settings.
A deep learning approach for suppressing noise in livestream earthquake data from a large seismic network

SUMMARY Detecting and analyzing small earthquakes is important for many seismological studies. Signals of small earthquakes are often obscured by noise. Recent advances in signal processing and deep learning along with available computing resources provide a great opportunity to address this challenge. In this study, we present a time domain method of suppressing noise for processing livestream earthquake data from a large seismic network by applying a deep neural network Real-time Denoiser (RTDenoiser). This neural network is able to attenuate a variety of colored noise and non-earthquake signals and suppress noise in the overlapping frequency bandwidth with signals. Because of its simplicity in time domain without domain transformation and subsequent processing, the method is able to process continuous livestream three-component data from several hundreds of seismic stations simultaneously. We create ‘noise-free’ samples by scaling down the waveforms of relatively large events from ML 3.5–5.0 to ML 1.5–3.0 according to the Richter scaling relationship. We also select noise samples from the same seismic station and add to ‘noise-free’ data to generate samples at different signal-to-noise ratio (SNR) levels. These data samples are randomly split into training, validation, and test sets. We verify the trained network to process data recorded in Sichuan and Yunnan, China from 2013 to 2018. Results show that the RTDenoiser can help improve SNRs from 5 dB to 15 dB in averag. The number of detected small events at magnitude between ML 1.0 and 3.0 has been increased by 58.8 per cent. The method is currently applied in a seismic network of 300 stations in Sichuan and Yunnan, China for continuous processing. It takes about 10 ms on average to process three-component 60-s data from 300 seismic stations on a single GPU.
Redefining teaching-and-learning-process in TLBO and its application in cloud

Teaching-Learning-Based Optimization (TLBO) algorithm was developed to solve single-objective optimization problems. TLBO is inspired by the theory of teaching-learning mechanism. Basic TLBO works better for unimodal problems but poorly for multi-model problems because of its poor exploration. To provide a fair exploration for solving complex optimization problems, we redefined the learning strategy to the basic TLBO. This newly redefined variant is called Multi-Teacher Teaching-Learning Based Optimization (MT-TLBO). The performance of MT-TLBO was tested on the latest optimization benchmark functions, CEC-C06, 2019. It is observed that the performance of MT-TLBO is superb comparatively. After that, it is simulated to solve workflow scheduling problems by minimizing the execution cost of the standard workflow and maximizing the workload on computing resources in a cloud environment. Finally, it is found that MT-TLBO produces minimal execution cost and a fair workload distribution on resources on standard benchmark workflow, comparatively.
MG-MVSNet: Multiple granularities feature fusion network for multi-view stereo

The goal of Multi-View Stereo is to reconstruct the 3D point cloud model from multiple views. With the development of deep learning, more and more learning-based research has achieved remarkable results. However, existing methods ignore the fine-grained features of the bottom layer, which leads to the poor quality of model reconstruction, especially in terms of completeness. Besides, current methods still rely on a large amount of consumed memory resources because of the application of 3D convolution. To this end, this paper proposes a Multiple Granularities Feature Fusion Network for Multi-View Stereo, an end-to-end depth estimation network combining global and local features, which is characterized by fine-granularity multi-feature fusion. Firstly, we propose a dense feature adaptive connection module, which can adaptively fuse the global and local features in the scene, provide a more complete and effective feature map for inferring a more detailed depth map, and make the ultimate model more complete. Secondly, in order to further improve the accuracy and completeness of the reconstructed point cloud, we introduce normal and edge loss futead of only using depth loss functions as in the existing methods, which makes the network more sensitive to small depth structures. Finally, we propose distributed 3D convolution instead of traditional 3D convolution, which reduces memory consumption. The experimental results on the DTU and Tanks & Temples datasets demonstrate that the proposed method in this papaer achieves the state-of-the-art performance, which proves the accuracy and effectiveness of the MG-MVSNet proposed in this paper.
Performance Evaluation and Cyberattack Mitigation in a Blockchain-Enabled Peer-to-Peer Energy Trading Framework

With the electric power grid experiencing a rapid shift to the smart grid paradigm over a deregulated energy market, Internet of Things (IoT)-based solutions are gaining prominence, and innovative peer-to-peer (P2P) energy trading at a micro level is being deployed. Such advancement, however, leaves traditional security models vulnerable and paves the path for blockchain, a distributed ledger technology (DLT), with its decentralized, open, and transparency characteristics as a viable alternative. However, due to deregulation in energy trading markets, most of the prototype resilience regarding cybersecurity attack, performance and scalability of transaction broadcasting, and its direct impact on overall performances and attacks are required to be supported, which becomes a performance bottleneck with existing blockchain solutions such as Hyperledger, Ethereum, and so on. In this paper, we design a novel permissioned Corda framework for P2P energy trading peers that not only mitigates a new class of cyberattacks, i.e., delay trading (or discard), but also disseminates the transactions in a optimized propagation time, resulting in a fair transaction distribution. Sharing transactions in a permissioned R3 Corda blockchain framework is handled by the Advanced Message Queuing Protocol (AMQP) and transport layer security (TLS). The unique contribution of this paper lies in the use of an optimized CPU and JVM heap memory scenario analysis with P2P metric in addition to a far more realistic multihosted testbed for the performance analysis. The average latencies measured are 22 ms and 51 ms for sending and receiving messages. We compare the throughput by varying different types of flow such as energy request, request + pay, transfer, multiple notary, sender, receiver, and single notary. In the proposed framework, request is an energy asset that is based on payment state and contract in the P2P energy trading module, so in request flow, only one node with no notary appears on the vault of the node.Energy request + pay flow interaction deals with two nodes, such as producer and consumer, to deal with request and transfer of asset ownership with the help of a notary. Request + repeated pay flow request, on node A and repeatedly transfers a fraction of energy asset state to another node, B, through a notary.
The application of machine learning to predict high-cost patients: A performance-comparison of different models using healthcare claims data

Our aim was to predict future high-cost patients with machine learning using healthcare claims data. We applied a random forest (RF), a gradient boosting machine (GBM), an artificial neural network (ANN) and a logistic regression (LR) to predict high-cost patients in the following year. Therefore, we exploited routinely collected sickness funds claims and cost data of the years 2016, 2017 and 2018. Various specifications of each algorithm were trained and cross-validated on training data (n = 20,984) with claims and cost data from 2016 and outcomes from 2017. The best performing specifications of each algorithm were selected based on validation dataset performance. For performance comparison, selected models were applied to unforeseen data with features of the year 2017 and outcomes of the year 2018 (n = 21,146). The RF was the best performing algorithm measured by the area under the receiver operating curve (AUC) with a value of 0.883 (95% confidence interval (CI): 0.872-0.893) on test data, followed by the GBM (AUC = 0.878; 95% CI: 0.867-0.889). The ANN (AUC = 0.846; 95% CI: 0.834-0.857) and LR (AUC = 0.839; 95% CI: 0.826-0.852) were significantly outperformed by the GBM and the RF. All ML algorithms and the LR performed ´good´ (i.e. 0.9 > AUC ≥ 0.8). We were able to develop machine learning models that predict high-cost patients with 'good' performance facilitating routinely collected sickness fund claims and cost data. We found that tree-based models performed best and outperformed the ANN and LR.
Semantic Communication as a Signaling Game with Correlated Knowledge Bases

Semantic communication (SC) goes beyond technical communication in which a given sequence of bits or symbols, often referred to as information, is be transmitted reliably over a noisy channel, regardless of its meaning. In SC, conveying the meaning of information becomes important, which requires some sort of agreement between a sender and a receiver through their knowledge bases. In this sense, SC is closely related to a signaling game where a sender takes an action to send a signal that conveys information to a receiver, while the receiver can interpret the signal and choose a response accordingly. Based on the signaling game, we can build a SC model and characterize the performance in terms of mutual information in this paper. In addition, we show that the conditional mutual information between the instances of the knowledge bases of communicating parties plays a crucial role in improving the performance of SC.
Automatic fault detection in race walking from a smartphone camera via fine-tuning pose estimation

Automatic fault detection is an important issue in sports. In race walking, researchers have attempted it using sensors and machine learning. However, there exists problems with sensor attachments, conflicting with referee’s judgement, and the model’s interpretability. Here, we propose a fault detection system via fine-tuning pose estimation from a smartphone camera and logistic regression models trained by referee’s judgement. Using normal race-walk and walking with intentional faults, we showed that the proposed system detected faults with an average accuracy of over 90%, and the model used grounds for detecting faults according to the rules of race walking.
Research on data augmentation for vital data using conditional GAN

In the field of elderly care, detecting anomaly in condition of the elderly people is important for improving the quality of care and quality of life. However, anomaly detection technology using machine learning often requires a large amount of training data. In this study, we examined whether vital and lifestyle data can be extended appropriately using generative adversarial networks. As vital and lifestyle data, heart rate, respiration rate, and bed leaving status were collected from sheet-type sensors set on subject’s bed. Data augmentation was conducted on four nursing home users. As a result, the difference between the average values of the actual data and the extended data for heart rate and respiratory rate was less than 10% of the average of the actual data, which indicates that the extended data was successfully generated relatively close to the actual data. On the other hand, individual differences were shown in the extended performance of the bed leaving status. Future work is needed to consider how to extend the lifestyle data in consideration of individual differences.
Heterogeneous Tri-stream Clustering Network

Contrastive deep clustering has recently gained significant attention with its ability of joint contrastive learning and clustering via deep neural networks. Despite the rapid progress, previous works mostly require both positive and negative sample pairs for contrastive clustering, which rely on a relative large batch-size. Moreover, they typically adopt a two-stream architecture with two augmented views, which overlook the possibility and potential benefits of multi-stream architectures (especially with heterogeneous or hybrid networks). In light of this, this paper presents a new end-to-end deep clustering approach termed Heterogeneous Tri-stream Clustering Network (HTCN). The tri-stream architecture in HTCN consists of three main components, including two weight-sharing online networks and a target network, where the parameters of the target network are the exponential moving average of that of the online networks. Notably, the two online networks are trained by simultaneously (i) predicting the instance representations of the target network and (ii) enforcing the consistency between the cluster representations of the target network and that of the two online networks. Experimental results on four challenging image datasets demonstrate the superiority of HTCN over the state-of-the-art deep clustering approaches. The code is available at https://github.com/dengxiaozhi/HTCN .
Facial expression recognition based on improved residual network

Abstract Facial expressions are an important part of human emotional signals and their recognition has become an important topic of research in the field of pattern recognition. Deep learning based methods have achieved great success in the recognition of facial expressions. However, with the evolution of convolution neural networks and the increased network depth, these methods suffer from problems such as degraded network performance and loss of feature information. To address these problems, a novel facial expression recognition algorithm based on an improved residual neural network is proposed. First, a residual neural network is designed to extract deep features while retaining the shallow ones. This can effectively prevent the degradation of network performance. Moreover, when the gradient of the Rectified Linear Units activation function used in the residual module is 0, it will inactivate the neurons and cause a loss of feature information. To address this, the Mish activation function is used instead. The slight allowance for negative values in Mish improves the gradient flow. Next, an inception module is introduced to obtain richer feature information under the same receptive field. Finally, by conducting verification experiments on the public datasets CK+ and KDEF, the authors manage to solve the problems of degraded network performance and insufficient information from extracted features, achieving recognition accuracy rates of 96.37% and 93.38% on the two datasets, respectively.
Planning and Learning in Multi-Agent Path Finding

Abstract Multi-agent path finding arises, on the one hand, in numerous applied areas. A classical example is automated warehouses with a large number of mobile goods-sorting robots operating simultaneously. On the other hand, for this problem, there are no universal solution methods that simultaneously satisfy numerous (often contradictory) requirements. Examples of such criteria are a guarantee of finding optimal solutions, high-speed operation, the possibility of operation in partially observable environments, etc. This paper provides a survey of modern methods for multi-agent path finding. Special attention is given to various settings of the problem. The differences and between learnable and nonlearnable solution methods and their applicability are discussed. Experimental programming environments necessary for implementing learnable approaches are analyzed separately.