pwc-archive/papers-with-abstracts huggingface dataset:
GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis
Authors: Zhenhui Ye, Ziyue Jiang, Yi Ren, Jinglin Liu, Jinzheng He, Zhou Zhao
Date: 2023-01-31 00:00:00
Tasks: Face Generation, Lip Reading, NeRF, Talking Face Generation, Vocal Bursts Intensity Prediction

Generating photo-realistic video portrait with arbitrary speech audio is a crucial problem in film-making and virtual reality. Recently, several works explore the usage of neural radiance field in this task to improve 3D realness and image fidelity. However, the generalizability of previous NeRF-based methods to out-of-domain audio is limited by the small scale of training data. In this work, we propose GeneFace, a generalized and high-fidelity NeRF-based talking face generation method, which can generate natural results corresponding to various out-of-domain audio. Specifically, we learn a variaitional motion generator on a large lip-reading corpus, and introduce a domain adaptative post-net to calibrate the result. Moreover, we learn a NeRF-based renderer conditioned on the predicted facial motion. A head-aware torso-NeRF is proposed to eliminate the head-torso separation problem. Extensive experiments show that our method achieves more generalized and high-fidelity talking face generation compared to previous methods.
Contrast and Clustering: Learning Neighborhood Pair Representation for Source-free Domain Adaptation
Authors: Yuqi Chen, Xiangbin Zhu, Yonggang Li, Yingjian Li, Haojie Fang
Date: 2023-01-31 00:00:00
Tasks: Clustering, Contrastive Learning, Domain Adaptation, Source-Free Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation uses source data from different distributions to solve the problem of classifying data from unlabeled target domains. However, conventional methods require access to source data, which often raise concerns about data privacy. In this paper, we consider a more practical but challenging setting where the source domain data is unavailable and the target domain data is unlabeled. Specifically, we address the domain discrepancy problem from the perspective of contrastive learning. The key idea of our work is to learn a domain-invariant feature by 1) performing clustering directly in the original feature space with nearest neighbors; 2) constructing truly hard negative pairs by extended neighbors without introducing additional computational complexity; and 3) combining noise-contrastive estimation theory to gain computational advantage. We conduct careful ablation studies and extensive experiments on three common benchmarks: VisDA, Office-Home, and Office-31. The results demonstrate the superiority of our methods compared with other state-of-the-art works.
Ultrasound based Gas Detection: Analyzing Acoustic Impedance for High-Performance and Low-Cost Solutions
Authors: Ayush Singh, Harikrishnan Pisharody Gopalkrishnan, Mahesh Raveendranatha Panicker
Date: 2023-01-31 00:00:00

Gas leakage is a serious hazard in a variety of sectors, including industrial, domestic, and gas-powered vehicles. The installation of gas leak detection systems has emerged as a critical preventative measure to solve this problem. Traditional gas sensors, such as electrochemical, infrared point, and Metal Oxide Semiconductor sensors, have been widely used for gas leak detection. However, these sensors have limitations in terms of their adaptation to various gases, as well as their high cost and difficulties in scaling. In this paper, a novel non-contact gas detection technique based on a 40 kHz ultrasonic signal is described. The proposed approach employs the reflections of the emitted ultrasonic wave to detect the gas leaks and also able to identify the exact gas in real-time. To confirm the method's effectiveness, trials were carried out using Hydrogen, Helium, Argon, and Butane gas. The system identified gas flow breaches in 0.01 seconds, whereas the gas identification procedure took 0.8 seconds. An interesting extension of the proposed approach is real-time visualisation of gas flow employing an array of transducers.
Recurrent Structure Attention Guidance for Depth Super-Resolution
Authors: Jiayi Yuan, Haobo Jiang, Xiang Li, Jianjun Qian, Jun Li, Jian Yang
Date: 2023-01-31 00:00:00
Tasks: Depth Estimation, Super-Resolution

Image guidance is an effective strategy for depth super-resolution. Generally, most existing methods employ hand-crafted operators to decompose the high-frequency (HF) and low-frequency (LF) ingredients from low-resolution depth maps and guide the HF ingredients by directly concatenating them with image features. However, the hand-designed operators usually cause inferior HF maps (e.g., distorted or structurally missing) due to the diverse appearance of complex depth maps. Moreover, the direct concatenation often results in weak guidance because not all image features have a positive effect on the HF maps. In this paper, we develop a recurrent structure attention guided (RSAG) framework, consisting of two important parts. First, we introduce a deep contrastive network with multi-scale filters for adaptive frequency-domain separation, which adopts contrastive networks from large filters to small ones to calculate the pixel contrasts for adaptive high-quality HF predictions. Second, instead of the coarse concatenation guidance, we propose a recurrent structure attention block, which iteratively utilizes the latest depth estimation and the image features to jointly select clear patterns and boundaries, aiming at providing refined guidance for accurate depth recovery. In addition, we fuse the features of HF maps to enhance the edge structures in the decomposed LF maps. Extensive experiments show that our approach obtains superior performance compared with state-of-the-art depth super-resolution methods.
Ambient FSK Backscatter Communications using LTE Cell Specific Reference Signals
Authors: Jingyi Liao, Xiyu Wang, Kalle Ruttik, Riku Jantti, Phan-Huy Dinh-Thuy
Date: 2023-01-31 00:00:00

Long Term Evolution (LTE) signal is ubiquitously present in electromagnetic (EM) background environment, which make it an attractive signal source for the ambient backscatter communications (AmBC). In this paper, we propose a system, in which a backscatter device (BD) introduces artificial Doppler shift to the channel which is larger than the natural Doppler but still small enough such that it can be tracked by the channel estimator at the User Equipment (UE). Channel estimation is done using the downlink cell specific reference signals (CRS) that are present regardless the UE being attached to the network or not. FSK was selected due to its robust operation in a fading channel. We describe the whole AmBC system, use two receivers. Finally, numerical simulations and measurements are provided to validate the proposed FSK AmBC performance.
Structure Flow-Guided Network for Real Depth Super-Resolution
Authors: Jiayi Yuan, Haobo Jiang, Xiang Li, Jianjun Qian, Jun Li, Jian Yang
Date: 2023-01-31 00:00:00
Tasks: Depth Estimation, Depth Prediction, Super-Resolution

Real depth super-resolution (DSR), unlike synthetic settings, is a challenging task due to the structural distortion and the edge noise caused by the natural degradation in real-world low-resolution (LR) depth maps. These defeats result in significant structure inconsistency between the depth map and the RGB guidance, which potentially confuses the RGB-structure guidance and thereby degrades the DSR quality. In this paper, we propose a novel structure flow-guided DSR framework, where a cross-modality flow map is learned to guide the RGB-structure information transferring for precise depth upsampling. Specifically, our framework consists of a cross-modality flow-guided upsampling network (CFUNet) and a flow-enhanced pyramid edge attention network (PEANet). CFUNet contains a trilateral self-attention module combining both the geometric and semantic correlations for reliable cross-modality flow learning. Then, the learned flow maps are combined with the grid-sampling mechanism for coarse high-resolution (HR) depth prediction. PEANet targets at integrating the learned flow map as the edge attention into a pyramid network to hierarchically learn the edge-focused guidance feature for depth edge refinement. Extensive experiments on real and synthetic DSR datasets verify that our approach achieves excellent performance compared to state-of-the-art methods.
Millimetre-wave Radar for Low-Cost 3D Imaging: A Performance Study
Authors: Han Cui, Jiacheng Wu, Naim Dahnoun
Date: 2023-01-31 00:00:00
Tasks: Super-Resolution

Millimetre-wave (mmWave) radars can generate 3D point clouds to represent objects in the scene. However, the accuracy and density of the generated point cloud can be lower than a laser sensor. Although researchers have used mmWave radars for various applications, there are few quantitative evaluations on the quality of the point cloud generated by the radar and there is a lack of a standard on how this quality can be assessed. This work aims to fill the gap in the literature. A radar simulator is built to evaluate the most common data processing chains of 3D point cloud construction and to examine the capability of the mmWave radar as a 3D imaging sensor under various factors. It will be shown that the radar detection can be noisy and have an imbalance distribution. To address the problem, a novel super-resolution point cloud construction (SRPC) algorithm is proposed to improve the spatial resolution of the point cloud and is shown to be able to produce a more natural point cloud and reduce outliers.
A Knowledge-Driven Meta-Learning Method for CSI Feedback
Authors: Han Xiao, Wenqiang Tian, Wendong Liu, Zhi Zhang, Zhihua Shi, Li Guo, Jia Shen
Date: 2023-01-31 00:00:00
Tasks: Meta-Learning

Accurate and effective channel state information (CSI) feedback is a key technology for massive multiple-input and multiple-output (MIMO) systems. Recently, deep learning (DL) has been introduced to enhance CSI feedback in massive MIMO application, where the massive collected training data and lengthy training time are costly and impractical for realistic deployment. In this paper, a knowledge-driven meta-learning solution for CSI feedback is proposed, where the DL model initialized by the meta model obtained from meta training phase is able to achieve rapid convergence when facing a new scenario during the target retraining phase. Specifically, instead of training with massive data collected from various scenarios, the meta task environment is constructed based on the intrinsic knowledge of spatial-frequency characteristics of CSI for meta training. Moreover, the target task dataset is also augmented by exploiting the knowledge of statistical characteristics of channel, so that the DL model initialized by meta training can rapidly fit into a new target scenario with higher performance using only a few actually collected data in the target retraining phase. The method greatly reduces the demand for the number of actual collected data, as well as the cost of training time for realistic deployment. Simulation results demonstrate the superiority of the proposed approach from the perspective of feedback performance and convergence speed.
Few-Shot Object Detection via Variational Feature Aggregation
Authors: Jiaming Han, Yuqiang Ren, Jian Ding, Ke Yan, Gui-Song Xia
Date: 2023-01-31 00:00:00
Tasks: Few-Shot Object Detection, Meta-Learning, Object, object-detection, Object Detection, Object Localization

As few-shot object detectors are often trained with abundant base samples and fine-tuned on few-shot novel examples,the learned models are usually biased to base classes and sensitive to the variance of novel examples. To address this issue, we propose a meta-learning framework with two novel feature aggregation schemes. More precisely, we first present a Class-Agnostic Aggregation (CAA) method, where the query and support features can be aggregated regardless of their categories. The interactions between different classes encourage class-agnostic representations and reduce confusion between base and novel classes. Based on the CAA, we then propose a Variational Feature Aggregation (VFA) method, which encodes support examples into class-level support features for robust feature aggregation. We use a variational autoencoder to estimate class distributions and sample variational features from distributions that are more robust to the variance of support examples. Besides, we decouple classification and regression tasks so that VFA is performed on the classification branch without affecting object localization. Extensive experiments on PASCAL VOC and COCO demonstrate that our method significantly outperforms a strong baseline (up to 16\%) and previous state-of-the-art methods (4\% in average). Code will be available at: \url{https://github.com/csuhan/VFA}
Energy-Efficient Vehicular Edge Computing with One-by-one Access Scheme
Authors: Youngsu Jang, Seongah Jeong, Joonhyuk Kang
Date: 2023-01-31 00:00:00
Tasks: Edge-computing, Scheduling

With the advent of ever-growing vehicular applications, vehicular edge computing (VEC) has been a promising solution to augment the computing capacity of future smart vehicles. The ultimate challenge to fulfill the quality of service (QoS) is increasingly prominent with constrained computing and communication resources of vehicles. In this paper, we propose an energy-efficient task offloading strategy for VEC system with one-by-one scheduling mechanism, where only one vehicle wakes up at a time to offload with a road side unit (RSU). The goal of system is to minimize the total energy consumption of vehicles by jointly optimizing user scheduling, offloading ratio and bit allocation within a given mission time. To this end, the non-convex and mixed-integer optimization problem is formulated and solved by adopting Lagrange dual problem, whose superior performances are verified via numerical results, as compared to other benchmark schemes.
GaitSADA: Self-Aligned Domain Adaptation for mmWave Gait Recognition
Authors: Ekkasit Pinyoanuntapong, Ayman Ali, Kalvik Jakkala, Pu Wang, Minwoo Lee, Qucheng Peng, Chen Chen, Zhi Sun
Date: 2023-01-31 00:00:00
Tasks: Contrastive Learning, Domain Adaptation, Gait Recognition, User Identification

mmWave radar-based gait recognition is a novel user identification method that captures human gait biometrics from mmWave radar return signals. This technology offers privacy protection and is resilient to weather and lighting conditions. However, its generalization performance is yet unknown and limits its practical deployment. To address this problem, in this paper, a non-synthetic dataset is collected and analyzed to reveal the presence of spatial and temporal domain shifts in mmWave gait biometric data, which significantly impacts identification accuracy. To mitigate this issue, a novel self-aligned domain adaptation method called GaitSADA is proposed. GaitSADA improves system generalization performance by using a two-stage semi-supervised model training approach. The first stage employs semi-supervised contrastive learning to learn a compact gait representation from both source and target domain data, aligning source-target domain distributions implicitly. The second stage uses semi-supervised consistency training with centroid alignment to further close source-target domain gap by pseudo-labelling the target-domain samples, clustering together the samples belonging to the same class but from different domains, and pushing the class centroid close to the weight vector of each class. Experiments show that GaitSADA outperforms representative domain adaptation methods with an improvement ranging from 15.41\% to 26.32\% on average accuracy in low data regimes. Code and dataset will be available at https://exitudio.github.io/GaitSADA
Study of Optical Networks, 5G, Artificial Intelligence and Their Applications
Authors: Quanda Zhang, Qi Zhang
Date: 2023-01-31 00:00:00

This paper discusses the application of artificial intelligence (AI) technology in optical communication networks and 5G. It primarily introduces representative applications of AI technology and potential risks of AI technology failure caused by the openness of optical communication networks, and proposes some coping strategies, mainly including modeling AI systems through modularization and miniaturization, combining with traditional classical network modeling and planning methods, and improving the effectiveness and interpretability of AI technology. At the same time, it proposes response strategies based on network protection for the possible failure and attack of AI technology.
Massive MIMO in 5G: How Beamforming, Codebooks, and Feedback Enable Larger Arrays
Authors: Ryan M. Dreifuerst, Robert W. Heath Jr
Date: 2023-01-31 00:00:00
Tasks: Management

Massive multiple-input multiple-output (MIMO) is an important technology in fifth generation (5G) cellular networks and beyond. To help design the beamforming at the base station, 5G has introduced new support in the form of flexible feedback and configurable antenna array geometries. In this article, we present an overview of MIMO throughout the mobile standards, highlight the new beam-based feedback system in 5G NR, and describe how this feedback system enables massive MIMO through beam management. Finally, we conclude with challenges related to massive MIMO in 5G.
When Source-Free Domain Adaptation Meets Learning with Noisy Labels
Authors: Li Yi, Gezheng Xu, Pengcheng Xu, Jiaqi Li, Ruizhi Pu, Charles Ling, A. Ian McLeod, Boyu Wang
Date: 2023-01-31 00:00:00
Tasks: Domain Adaptation, Learning with noisy labels, Source-Free Domain Adaptation

Recent state-of-the-art source-free domain adaptation (SFDA) methods have focused on learning meaningful cluster structures in the feature space, which have succeeded in adapting the knowledge from source domain to unlabeled target domain without accessing the private source data. However, existing methods rely on the pseudo-labels generated by source models that can be noisy due to domain shift. In this paper, we study SFDA from the perspective of learning with label noise (LLN). Unlike the label noise in the conventional LLN scenario, we prove that the label noise in SFDA follows a different distribution assumption. We also prove that such a difference makes existing LLN methods that rely on their distribution assumptions unable to address the label noise in SFDA. Empirical evidence suggests that only marginal improvements are achieved when applying the existing LLN methods to solve the SFDA problem. On the other hand, although there exists a fundamental difference between the label noise in the two scenarios, we demonstrate theoretically that the early-time training phenomenon (ETP), which has been previously observed in conventional label noise settings, can also be observed in the SFDA problem. Extensive experiments demonstrate significant improvements to existing SFDA algorithms by leveraging ETP to address the label noise in SFDA.
Iterative Loop Method Combining Active and Semi-Supervised Learning for Domain Adaptive Semantic Segmentation
Authors: Licong Guan, Xue Yuan
Date: 2023-01-31 00:00:00
Tasks: Active Learning, Domain Adaptation, Semantic Segmentation, Semi-Supervised Semantic Segmentation

Semantic segmentation is an important technique for environment perception in intelligent transportation systems. With the rapid development of convolutional neural networks (CNNs), road scene analysis can usually achieve satisfactory results in the source domain. However, guaranteeing good generalization to different target domain scenarios remains a significant challenge. Recently, semi-supervised learning and active learning have been proposed to alleviate this problem. Semisupervised learning can improve model accuracy with massive unlabeled data, but some pseudo labels containing noise would be generated with limited or imbalanced training data. And there will be suboptimal models if human guidance is absent. Active learning can select more effective data to intervene, while the model accuracy can not be improved because the massive unlabeled data are not used. And the probability of querying sub-optimal samples will increase when the domain difference is too large, increasing annotation cost. This paper proposes an iterative loop method combining active and semisupervised learning for domain adaptive semantic segmentation. The method first uses semi-supervised to learn massive unlabeled data to improve model accuracy and provide more accurate selection models for active learning. Secondly, combined with the predictive uncertainty sample selection strategy of active learning, manual intervention is used to correct the pseudo-labels. Finally, flexible iterative loops achieve the best performance with minimal labeling cost. Extensive experiments show that our method establishes state-of-the-art performance on tasks of GTAV to Cityscapes, SYNTHIA to Cityscapes, improving by 4.9% mIoU and 5.2% mIoU, compared to the previous best method, respectively.
Inference Time Evidences of Adversarial Attacks for Forensic on Transformers
Authors: Hugo Lemarchant, Liangzi Li, Yiming Qian, Yuta Nakashima, Hajime Nagahara
Date: 2023-01-31 00:00:00
Tasks: image-classification, Image Classification

Vision Transformers (ViTs) are becoming a very popular paradigm for vision tasks as they achieve state-of-the-art performance on image classification. However, although early works implied that this network structure had increased robustness against adversarial attacks, some works argue ViTs are still vulnerable. This paper presents our first attempt toward detecting adversarial attacks during inference time using the network's input and outputs as well as latent features. We design four quantifications (or derivatives) of input, output, and latent vectors of ViT-based models that provide a signature of the inference, which could be beneficial for the attack detection, and empirically study their behavior over clean samples and adversarial samples. The results demonstrate that the quantifications from input (images) and output (posterior probabilities) are promising for distinguishing clean and adversarial samples, while latent vectors offer less discriminative power, though they give some insights on how adversarial perturbations work.
Continuous Spatiotemporal Transformers
Authors: Antonio H. de O. Fonseca, Emanuele Zappala, Josue Ortega Caro, David van Dijk
Date: 2023-01-31 00:00:00

Modeling spatiotemporal dynamical systems is a fundamental challenge in machine learning. Transformer models have been very successful in NLP and computer vision where they provide interpretable representations of data. However, a limitation of transformers in modeling continuous dynamical systems is that they are fundamentally discrete time and space models and thus have no guarantees regarding continuous sampling. To address this challenge, we present the Continuous Spatiotemporal Transformer (CST), a new transformer architecture that is designed for the modeling of continuous systems. This new framework guarantees a continuous and smooth output via optimization in Sobolev space. We benchmark CST against traditional transformers as well as other spatiotemporal dynamics modeling methods and achieve superior performance in a number of tasks on synthetic and real systems, including learning brain dynamics from calcium imaging data.