Reviewerly/exHarmony huggingface dataset:
Hyperspectral Image Denoising Using Factor Group Sparsity-Regularized Nonconvex Low-Rank Approximation

Hyperspectral image (HSI) mixed noise removal is a fundamental problem and an important preprocessing step in remote sensing fields. The low-rank approximation-based methods have been verified effective to encode the global spectral correlation for HSI denoising. However, due to the large scale and complexity of real HSI, previous low-rank HSI denoising techniques encounter several problems, including coarse rank approximation (such as nuclear norm), the high computational cost of singular value decomposition (SVD) (such as Schatten <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> -norm), and adaptive rank selection (such as low-rank factorization). In this article, two novel factor group sparsity-regularized nonconvex low-rank approximation (FGSLR) methods are introduced for HSI denoising, which can simultaneously overcome the mentioned issues of previous works. The FGSLR methods capture the spectral correlation via low-rank factorization, meanwhile utilizing factor group sparsity regularization to further enhance the low-rank property. It is SVD-free and robust to rank selection. Moreover, FGSLR is equivalent to Schatten <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> -norm approximation ( <xref ref-type="theorem" rid="theorem1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Theorem 1</xref> ), and thus FGSLR is tighter than the nuclear norm in terms of rank approximation. To preserve the spatial information of HSI in the denoising process, the total variation regularization is also incorporated into the proposed FGSLR models. Specifically, the proximal alternating minimization is designed to solve the proposed FGSLR models. Experimental results have demonstrated that the proposed FGSLR methods significantly outperform existing low-rank approximation-based HSI denoising methods.
Composable security for continuous variable quantum key distribution: Trust levels and practical key rates in wired and wireless networks

Continuous variable (CV) quantum key distribution (QKD) provides a powerful setting for secure quantum communications, thanks to the use of room-temperature off-the-shelf optical devices and the potential to reach much higher rates than the standard discrete-variable counterpart. In this paper, we provide a general framework for studying the composable finite-size security of CV-QKD with Gaussian-modulated coherent-state protocols under various levels of trust for the loss and noise experienced by the parties. Our paper considers both wired (i.e., fiber-based) and wireless (i.e., free-space) quantum communications. In the latter case, we show that high key rates are achievable for short-range optical wireless (LiFi) in secure quantum networks with both fixed and mobile devices. Finally, we extend our investigation to microwave wireless (WiFi) discussing security and feasibility of CV-QKD for very short-range applications.
Translating Images into Maps

We approach instantaneous mapping, converting images to a top-down view of the world, as a translation problem. We show how a novel form of transformer network can be used to map from images and video directly to an overhead map or bird's-eye-view (BEV) of the world, in a single end-to-end network. We assume a 1–1 correspondence between a vertical scanline in the image, and rays passing through the camera location in an overhead map. This lets us formulate map generation from an image as a set of sequence-to-sequence translations. Posing the problem as translation allows the network to use the context of the image when interpreting the role of each pixel. This constrained formulation, based upon a strong physical grounding of the problem, leads to a restricted transformer network that is convolutional in the horizontal direction only. The structure allows us to make efficient use of data when training, and obtains state-of-the-art results for instantaneous mapping of three large-scale datasets, including a 15% and 30% relative gain against existing best performing methods on the nuScenes and Argoverse datasets, respectively.
Skeleton Cloud Colorization for Unsupervised 3D Action Representation Learning

Skeleton-based human action recognition has attracted increasing attention in recent years. However, most of the existing works focus on supervised learning which requiring a large number of annotated action sequences that are often expensive to collect. We investigate unsupervised representation learning for skeleton action recognition, and design a novel skeleton cloud colorization technique that is capable of learning skeleton representations from unlabeled skeleton sequence data. Specifically, we represent a skeleton action sequence as a 3D skeleton cloud and colorize each point in the cloud according to its temporal and spatial orders in the original (unannotated) skeleton sequence. Leveraging the colorized skeleton point cloud, we design an auto-encoder framework that can learn spatial-temporal features from the artificial color labels of skeleton joints effectively. We evaluate our skeleton cloud colorization approach with action classifiers trained under different configurations, including unsupervised, semi-supervised and fully-supervised settings. Extensive experiments on NTU RGB+D and NW-UCLA datasets show that the proposed method outperforms existing unsupervised and semi-supervised 3D action recognition methods by large margins, and it achieves competitive performance in supervised 3D action recognition as well.
Semi-Decentralized Federated Learning With Cooperative D2D Local Model Aggregations

Federated learning has emerged as a popular technique for distributing machine learning (ML) model training across the wireless edge. In this paper, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">two timescale hybrid federated learning</i> ( <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TT-HF</monospace> ), a semi-decentralized learning architecture that combines the conventional device-to-server communication paradigm for federated learning with device-to-device (D2D) communications for model training. In <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TT-HF</monospace> , during each global aggregation interval, devices (i) perform multiple stochastic gradient descent iterations on their individual datasets, and (ii) aperiodically engage in consensus procedure of their model parameters through cooperative, distributed D2D communications within local clusters. With a new general definition of gradient diversity, we formally study the convergence behavior of <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TT-HF</monospace> , resulting in new convergence bounds for distributed ML. We leverage our convergence bounds to develop an adaptive control algorithm that tunes the step size, D2D communication rounds, and global aggregation period of <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TT-HF</monospace> over time to target a sublinear convergence rate of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(1/t)$ </tex-math></inline-formula> while minimizing network resource utilization. Our subsequent experiments demonstrate that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TT-HF</monospace> significantly outperforms the current art in federated learning in terms of model accuracy and/or network energy consumption in different scenarios where local device datasets exhibit statistical heterogeneity. Finally, our numerical evaluations demonstrate robustness against outages caused by fading channels, as well favorable performance with non-convex loss functions.
Uncertainty-aware Pseudo Label Refinery for Domain Adaptive Semantic Segmentation

Unsupervised domain adaptation for semantic segmentation aims to assign the pixel-level labels for unlabeled target domain by transferring knowledge from the labeled source domain. A typical self-supervised learning approach generates pseudo labels from the source model and then re-trains the model to fit the target distribution. However, it suffers from noisy pseudo labels due to the existence of domain shift. Related works alleviate this problem by selecting high-confidence predictions, but uncertain classes with low confidence scores have rarely been considered. This informative uncertainty is essential to enhance feature representation and align source and target domains. In this paper, we propose a novel uncertainty-aware pseudo label refinery framework considering two crucial factors simultaneously. First, we progressively enhance the feature alignment model via the target-guided uncertainty rectifying framework. Second, we provide an uncertainty-aware pseudo label assignment strategy without any manually de-signed threshold to reduce the noisy labels. Extensive experiments demonstrate the effectiveness of our proposed approach and achieve state-of-the-art performance on two standard synthetic-2-real tasks.
Real-time epileptic seizure recognition using Bayesian genetic whale optimizer and adaptive machine learning

The electroencephalogram (EEG) has been commonly used to identify epileptic seizures, but identification of seizures from EEG remains a challenging task that requires qualified neurophysiologists. It is important to detect seizures in real time, which can be achieved in an internet of things (IoT)-based cloud platform to alert patients of impending seizures. Therefore, in this study, we propose a new model to recognize seizure states (e.g., ictal, preictal, interictal) from EEG in the IoT framework to monitor patients remotely. The proposed model uses an efficient hybrid genetic whale optimization algorithm (GWOA) based on naïve Bayes (NB-GWOA) for feature selection, and an adaptive extreme learning machine (ELM) based on a differential evolutionary (DE) algorithm (DEELM) for classification. In the NB-GWOA method, the genetic algorithm serves to enhance the exploitation of the whale optimization algorithm in the search of the optimal solutions, while the naïve Bayes method is used to determine a fitness function to assess every agent in the search space. GWOA has strong robustness and is capable of finding the best solutions in less than five iterations, so it is suitable for selecting discriminative features from a huge number of neurofeatures obtained from EEG. Further, the classification model is constructed based on ELM, which uses the DE algorithm for a fast and efficient learning solution. Results show that the proposed NB-GWOA-DEELM model can avoid over- and under-fitting and can provide better and more accurate performance in classifying seizure states from EEG than its competitors.
FastHyMix: Fast and Parameter-Free Hyperspectral Image Mixed Noise Removal

The decrease in the widths of spectral bands in hyperspectral imaging leads to a decrease in signal-to-noise ratio (SNR) of measurements. The decreased SNR reduces the reliability of measured features or information extracted from hyperspectral images (HSIs). Furthermore, the image degradations linked with various mechanisms also result in different types of noise, such as Gaussian noise, impulse noise, deadlines, and stripes. This article introduces a fast and parameter-free hyperspectral image mixed noise removal method (termed FastHyMix), which characterizes the complex distribution of mixed noise by using a Gaussian mixture model and exploits two main characteristics of hyperspectral data, namely, low rankness in the spectral domain and high correlation in the spatial domain. The Gaussian mixture model enables us to make a good estimation of Gaussian noise intensity and the locations of sparse noise. The proposed method takes advantage of the low rankness using subspace representation and the spatial correlation of HSIs by adding a powerful deep image prior, which is extracted from a neural denoising network. An exhaustive array of experiments and comparisons with state-of-the-art denoisers was carried out. The experimental results show significant improvement in both synthetic and real datasets. A MATLAB demo of this work is available at https://github.com/LinaZhuang for the sake of reproducibility.
A Latent Transformer for Disentangled Face Editing in Images and Videos

High quality facial image editing is a challenging problem in the movie post-production industry, requiring a high degree of control and identity preservation. Previous works that attempt to tackle this problem may suffer from the entanglement of facial attributes and the loss of the person’s identity. Furthermore, many algorithms are limited to a certain task. To tackle these limitations, we propose to edit facial attributes via the latent space of a StyleGAN generator, by training a dedicated latent transformation network and incorporating explicit disentanglement and identity preservation terms in the loss function. We further introduce a pipeline to generalize our face editing to videos. Our model achieves a disentangled, controllable, and identity-preserving facial attribute editing, even in the challenging case of real (i.e., non-synthetic) images and videos. We conduct extensive experiments on image and video datasets and show that our model outperforms other state-of-the-art methods in visual quality and quantitative evaluation. Source codes are available at https://github.com/InterDigitalInc/latent-transformer.
Face Hallucination via Split-Attention in Split-Attention Network

Recently, convolutional neural networks (CNNs) have been widely employed to promote the face hallucination due to the ability to predict high-frequency details from a large number of samples. However, most of them fail to take into account the overall facial profile and fine texture details simultaneously, resulting in reduced naturalness and fidelity of the reconstructed face, and further impairing the performance of downstream tasks (e.g., face detection, facial recognition). To tackle this issue, we propose a novel external-internal split attention group (ESAG), which encompasses two paths responsible for facial structure information and facial texture details, respectively. By fusing the features from these two paths, the consistency of facial structure and the fidelity of facial details are strengthened at the same time. Then, we propose a split-attention in split-attention network (SISN) to reconstruct photorealistic high-resolution facial images by cascading several ESAGs. Experimental results on face hallucination and face recognition unveil that the proposed method not only significantly improves the clarity of hallucinated faces, but also encourages the subsequent face recognition performance substantially. Codes have been released at https://github.com/mdswyz/SISN-Face-Hallucination.
Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning

Safe and efficient navigation through human crowds is an essential capability for mobile robots. Previous work on robot crowd navigation assumes that the dynamics of all agents are known and well-defined. In addition, the performance of previous methods deteriorates in partially observable environments and environments with dense crowds. To tackle these problems, we propose decentralized structural-Recurrent Neural Network (DS-RNN), a novel network that reasons about spatial and temporal relationships for robot decision making in crowd navigation. We train our network with model-free deep reinforcement learning without any expert supervision. We demonstrate that our model outperforms previous methods in challenging crowd navigation scenarios. We successfully transfer the policy learned in the simulator to a real-world TurtleBot 2i.
Incomplete multi-view clustering with cosine similarity

Incomplete multi-view clustering partitions multi-view data suffering from missing views, for which matrix factorization approaches seek the latent representation of incomplete multi-view data and constitute one effective category of methods. To exploit data properties further, manifold structure preserving is also incorporated into matrix factorization. However, previous methods optimized the data similarity matrix in the manifold structure preserving term as an unknown variable, which is not guaranteed to faithfully represent the similarities of the original multi-view data and also increases the computational difficulty. To overcome these drawbacks, in this paper, we propose Incomplete Multi-view Clustering with Cosine Similarity (IMCCS). In IMCCS, we directly calculate the cosine similarity in the original multi-view space to strengthen the ability of preserving the manifold structure of the original multi-view data. There is no need to introduce the additional variable. The manifold structure preserving term with cosine similarity and the matrix factorization term are integrated into a unified objective function. An iterative algorithm with gradient descent is designed to solve this objective. Extensive experiments on multi-view datasets show that IMCCS outperforms state-of-the-art incomplete multi-view clustering methods.
Higher-Order Attribute-Enhancing Heterogeneous Graph Neural Networks

GNNs have been widely used in deep learning on graphs. They learn effective node representations. However, most methods ignore the heterogeneity. Methods designed for heterogeneous graphs, on the other hand, fail to learn complex semantic representations because they only use meta-paths instead of meta-graphs. Furthermore, they cannot fully capture the content-based correlations, as they either do not use the self-attention mechanism or only use it to consider the immediate neighbors of each node, ignoring the higher-order neighbors. We propose a novel Higher-order Attribute-Enhancing (HAE) framework enhancing node embedding in a layer-by-layer manner. Under the HAE framework, we propose a Higher-order Attribute-Enhancing GNN (HAE\textsubscript{GNN}) for heterogeneous network embeding. HAE\textsubscript{GNN} simultaneously incorporates meta-paths and meta-graphs for rich, heterogeneous semantics, and leverages the self-attention mechanism to explore content-based nodes' interactions. The unique higher-order architecture of HAE\textsubscript{GNN} allows examining the first-order as well as higher-order neighborhoods. Moreover, HAE\textsubscript{GNN} shows good explainability as it learns the importances of different meta-paths and meta-graphs. HAE\textsubscript{GNN} is also memory-efficient, for it avoids per meta-path based matrix calculation. Experimental results not only show HAE\textsubscript{GNN}'s superior performance against the state-of-the-art methods in node classification, node clustering, and visualization, but also demonstrate its superiorities in terms of memory efficiency and explainability.
Pain-FL: Personalized Privacy-Preserving Incentive for Federated Learning

Federated learning (FL) is a privacy-preserving distributed machine learning framework, which involves training statistical models over a number of mobile users (i.e., workers) while keeping data localized. However, recent works have demonstrated that workers engaged in FL are still susceptible to advanced inference attacks when sharing model updates or gradients, which would discourage them from participating. Most of the existing incentive mechanisms for FL mainly account for workers’ resource cost, while the cost incurred by potential privacy leakage resulting from inference attacks has rarely been incorporated. To address these issues, in this paper, we propose a contract-based personalized privacy-preserving incentive for FL, named Pain-FL, to provide customized payments for workers with different privacy preferences as compensation for privacy leakage cost while ensuring satisfactory convergence performance of FL models. The core idea of Pain-FL is that each worker agrees on a customized contract, which specifies a kind of privacy-preserving level (PPL) and the corresponding payment, with the server in each round of FL. Then, the worker perturbs her calculated stochastic gradients to be uploaded with that PPL in exchange for that payment. In particular, we respectively derive a set of optimal contracts analytically under both complete and incomplete information models, which could optimize the convergence performance of the finally learned global model, while bearing some desired economic properties, i.e., budget feasibility, individual rationality, and incentive compatibility. An exhaustive experimental evaluation of Pain-FL is conducted, and the results corroborate its practicability and effectiveness.
Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition

The task of skeleton-based action recognition remains a core challenge in human-centred scene understanding due to the multiple granularities and large variation in human motion. Existing approaches typically employ a single neural representation for different motion patterns, which has difficulty in capturing fine-grained action classes given limited training data. To address the aforementioned problems, we propose a novel multi-granular spatio-temporal graph network for skeleton-based action classification that jointly models the coarse- and fine-grained skeleton motion patterns. To this end, we develop a dual-head graph network consisting of two interleaved branches, which enables us to extract features at two spatio-temporal resolutions in an effective and efficient manner. Moreover, our network utilises a cross-head communication strategy to mutually enhance the representations of both heads. We conducted extensive experiments on three large-scale datasets, namely NTU RGB+D 60, NTU RGB+D 120, and Kinetics-Skeleton, and achieves the state-of-the-art performance on all the benchmarks, which validates the effectiveness of our method1.
Rationally Inattentive Behavior: Characterizing and Generalizing Shannon Entropy

We introduce three new classes of attention cost functions: posterior separable, uniformly posterior separable, and invariant posterior separable. As with the Shannon cost function, all can be solved using Lagrangian methods. Uniformly posterior-separable cost functions capture many forms of sequential learning and hence play a key role in many applications. Invariant posterior-separable cost functions make learning strategies depend exclusively on payoff uncertainty. We introduce two behavioral axioms, Locally Invariant Posteriors and Only Payoffs Matter, which identify posterior-separable functions as uniformly and invariant posterior separable, respectively. In combination, they pinpoint the Shannon cost function.