pwc-archive/papers-with-abstracts huggingface dataset:
BSRA: Block-based Super Resolution Accelerator with Hardware Efficient Pixel Attention
Authors: Dun-Hao Yang, Tian-Sheuan Chang
Date: 2022-05-02 00:00:00
Tasks: Image Reconstruction, Super-Resolution

Increasingly, convolution neural network (CNN) based super resolution models have been proposed for better reconstruction results, but their large model size and complicated structure inhibit their real-time hardware implementation. Current hardware designs are limited to a plain network and suffer from lower quality and high memory bandwidth requirements. This paper proposes a super resolution hardware accelerator with hardware efficient pixel attention that just needs 25.9K parameters and simple structure but achieves 0.38dB better reconstruction images than the widely used FSRCNN. The accelerator adopts full model block wise convolution for full model layer fusion to reduce external memory access to model input and output only. In addition, CNN and pixel attention are well supported by PE arrays with distributed weights. The final implementation can support full HD image reconstruction at 30 frames per second with TSMC 40nm CMOS process.
Point Cloud Compression with Sibling Context and Surface Priors
Authors: Zhili Chen, Zian Qian, Sukai Wang, Qifeng Chen
Date: 2022-05-02 00:00:00

We present a novel octree-based multi-level framework for large-scale point cloud compression, which can organize sparse and unstructured point clouds in a memory-efficient way. In this framework, we propose a new entropy model that explores the hierarchical dependency in an octree using the context of siblings' children, ancestors, and neighbors to encode the occupancy information of each non-leaf octree node into a bitstream. Moreover, we locally fit quadratic surfaces with a voxel-based geometry-aware module to provide geometric priors in entropy encoding. These strong priors empower our entropy framework to encode the octree into a more compact bitstream. In the decoding stage, we apply a two-step heuristic strategy to restore point clouds with better reconstruction quality. The quantitative evaluation shows that our method outperforms state-of-the-art baselines with a bitrate improvement of 11-16% and 12-14% on the KITTI Odometry and nuScenes datasets, respectively.
Exposing Deepfake Face Forgeries with Guided Residuals
Authors: Zhiqing Guo, Gaobo Yang, Jiyou Chen, Xingming Sun
Date: 2022-05-02 00:00:00
Tasks: DeepFake Detection, Face Swapping

Residual-domain feature is very useful for Deepfake detection because it suppresses irrelevant content features and preserves key manipulation traces. However, inappropriate residual prediction will bring side effects on detection accuracy. In addition, residual-domain features are easily affected by image operations such as compression. Most existing works exploit either spatial-domain features or residual-domain features, while neglecting that two types of features are mutually correlated. In this paper, we propose a guided residuals network, namely GRnet, which fuses spatial-domain and residual-domain features in a mutually reinforcing way, to expose face images generated by Deepfake. Different from existing prediction based residual extraction methods, we propose a manipulation trace extractor (MTE) to directly remove the content features and preserve manipulation traces. MTE is a fine-grained method that can avoid the potential bias caused by inappropriate prediction. Moreover, an attention fusion mechanism (AFM) is designed to selectively emphasize feature channel maps and adaptively allocate the weights for two streams. The experimental results show that the proposed GRnet achieves better performances than the state-of-the-art works on four public fake face datasets including HFF, FaceForensics++, DFDC and Celeb-DF. Especially, GRnet achieves an average accuracy of 97.72% on the HFF dataset, which is at least 5.25% higher than the existing works.
Dual networks based 3D Multi-Person Pose Estimation from Monocular Video
Authors: Yu Cheng, Bo wang, Robby T. Tan
Date: 2022-05-02 00:00:00
Tasks: 3D Human Pose Estimation, 3D Multi-Person Pose Estimation, 3D Multi-Person Pose Estimation (absolute), 3D Multi-Person Pose Estimation (root-relative), 3D Pose Estimation, Human Detection, Monocular 3D Human Pose Estimation, Multi-Person Pose Estimation, Pose Estimation

Monocular 3D human pose estimation has made progress in recent years. Most of the methods focus on single persons, which estimate the poses in the person-centric coordinates, i.e., the coordinates based on the center of the target person. Hence, these methods are inapplicable for multi-person 3D pose estimation, where the absolute coordinates (e.g., the camera coordinates) are required. Moreover, multi-person pose estimation is more challenging than single pose estimation, due to inter-person occlusion and close human interactions. Existing top-down multi-person methods rely on human detection (i.e., top-down approach), and thus suffer from the detection errors and cannot produce reliable pose estimation in multi-person scenes. Meanwhile, existing bottom-up methods that do not use human detection are not affected by detection errors, but since they process all persons in a scene at once, they are prone to errors, particularly for persons in small scales. To address all these challenges, we propose the integration of top-down and bottom-up approaches to exploit their strengths. Our top-down network estimates human joints from all persons instead of one in an image patch, making it robust to possible erroneous bounding boxes. Our bottom-up network incorporates human-detection based normalized heatmaps, allowing the network to be more robust in handling scale variations. Finally, the estimated 3D poses from the top-down and bottom-up networks are fed into our integration network for final 3D poses. To address the common gaps between training and testing data, we do optimization during the test time, by refining the estimated 3D human poses using high-order temporal constraint, re-projection loss, and bone length regularizations. Our evaluations demonstrate the effectiveness of the proposed method. Code and models are available: https://github.com/3dpose/3D-Multi-Person-Pose.
3D Object Detection with a Self-supervised Lidar Scene Flow Backbone
Authors: Ekim Yurtsever, Emeç Erçelik, MingYu Liu, Zhijie Yang, Hanzhen Zhang, Pınar Topçam, Maximilian Listl, Yılmaz Kaan Çaylı, Alois Knoll
Date: 2022-05-02 00:00:00
Tasks: 3D Object Detection, Object, object-detection, Object Detection, Scene Flow Estimation, Self-Supervised Learning, Self-supervised Scene Flow Estimation

State-of-the-art lidar-based 3D object detection methods rely on supervised learning and large labeled datasets. However, annotating lidar data is resource-consuming, and depending only on supervised learning limits the applicability of trained models. Self-supervised training strategies can alleviate these issues by learning a general point cloud backbone model for downstream 3D vision tasks. Against this backdrop, we show the relationship between self-supervised multi-frame flow representations and single-frame 3D detection hypotheses. Our main contribution leverages learned flow and motion representations and combines a self-supervised backbone with a supervised 3D detection head. First, a self-supervised scene flow estimation model is trained with cycle consistency. Then, the point cloud encoder of this model is used as the backbone of a single-frame 3D object detection head model. This second 3D object detection model learns to utilize motion representations to distinguish dynamic objects exhibiting different movement patterns. Experiments on KITTI and nuScenes benchmarks show that the proposed self-supervised pre-training increases 3D detection performance significantly. https://github.com/emecercelik/ssl-3d-detection.git
Unsupervised Denoising of Optical Coherence Tomography Images with Dual_Merged CycleWGAN
Authors: Jie Du, Xujian Yang, Kecheng Jin, Xuanzheng Qi, Hu Chen
Date: 2022-05-02 00:00:00
Tasks: Denoising, Image Denoising

Nosie is an important cause of low quality Optical coherence tomography (OCT) image. The neural network model based on Convolutional neural networks(CNNs) has demonstrated its excellent performance in image denoising. However, OCT image denoising still faces great challenges because many previous neural network algorithms required a large number of labeled data, which might cost much time or is expensive. Besides, these CNN-based algorithms need numerous parameters and good tuning techniques, which is hardware resources consuming. To solved above problems, We proposed a new Cycle-Consistent Generative Adversarial Nets called Dual-Merged Cycle-WGAN for retinal OCT image denoiseing, which has remarkable performance with less unlabeled traning data. Our model consists of two Cycle-GAN networks with imporved generator, descriminator and wasserstein loss to achieve good training stability and better performance. Using image merge technique between two Cycle-GAN networks, our model could obtain more detailed information and hence better training effect. The effectiveness and generality of our proposed network has been proved via ablation experiments and comparative experiments. Compared with other state-of-the-art methods, our unsupervised method obtains best subjective visual effect and higher evaluation objective indicators.
Revisiting Classical Multiclass Linear Discriminant Analysis with a Novel Prototype-based Interpretable Solution
Authors: Sayed Kamaledin Ghiasi-Shirazi
Date: 2022-05-02 00:00:00
Tasks: Dimensionality Reduction

Linear discriminant analysis (LDA) is a fundamental method for feature extraction and dimensionality reduction. Despite having many variants, classical LDA has its own importance, as it is a keystone in human knowledge about statistical pattern recognition. For a dataset containing C clusters, the classical solution to LDA extracts at most C-1 features. Here, we introduce a novel solution to classical LDA, called LDA++, that yields C features, each interpretable as measuring similarity to one cluster. This novel solution bridges dimensionality reduction and multiclass classification. Specifically, we prove that, for homoscedastic Gaussian data and under some mild conditions, the optimal weights of a linear multiclass classifier also make an optimal solution to LDA. In addition, we show that LDA++ reveals some important new facts about LDA that remarkably changes our understanding of classical multiclass LDA after 75 years of its introduction. We provide a complete numerical solution for LDA++ for the cases 1) when the scatter matrices can be constructed explicitly, 2) when constructing the scatter matrices is infeasible, and 3) the kernel extension.
An Application to Generate Style Guided Compatible Outfit
Authors: Debopriyo Banerjee, Harsh Maheshwari, Lucky Dhakad1, Arnab Bhattacharya1, Niloy Ganguly, Muthusamy Chelliah, Suyash Agarwal1
Date: 2022-05-02 00:00:00

Fashion recommendation has witnessed a phenomenal growth of research, particularly in the domains of shop-the-look, contextaware outfit creation, personalizing outfit creation etc. Majority of the work in this area focuses on better understanding of the notion of complimentary relationship between lifestyle items. Quite recently, some works have realised that style plays a vital role in fashion, especially in the understanding of compatibility learning and outfit creation. In this paper, we would like to present the end-to-end design of a methodology in which we aim to generate outfits guided by styles or themes using a novel style encoder network. We present an extensive analysis of different aspects of our method through various experiments. We also provide a demonstration api to showcase the ability of our work in generating outfits based on an anchor item and styles.
Deep fiber clustering: Anatomically informed fiber clustering with self-supervised deep learning for fast and effective tractography parcellation
Authors: Yuqian Chen, Chaoyi Zhang, Tengfei Xue, Yang song, Nikos Makris, Yogesh Rathi, Weidong Cai, Fan Zhang, Lauren J. O'Donnell
Date: 2022-05-02 00:00:00
Tasks: Anatomy, Clustering, Computational Efficiency, Representation Learning, Self-Supervised Learning

White matter fiber clustering is an important strategy for white matter parcellation, which enables quantitative analysis of brain connections in health and disease. In combination with expert neuroanatomical labeling, data-driven white matter fiber clustering is a powerful tool for creating atlases that can model white matter anatomy across individuals. While widely used fiber clustering approaches have shown good performance using classical unsupervised machine learning techniques, recent advances in deep learning reveal a promising direction toward fast and effective fiber clustering. In this work, we propose a novel deep learning framework for white matter fiber clustering, Deep Fiber Clustering (DFC), which solves the unsupervised clustering problem as a self-supervised learning task with a domain-specific pretext task to predict pairwise fiber distances. This process learns a high-dimensional embedding feature representation for each fiber, regardless of the order of fiber points reconstructed during tractography. We design a novel network architecture that represents input fibers as point clouds and allows the incorporation of additional sources of input information from gray matter parcellation to improve anatomical coherence of clusters. In addition, DFC conducts outlier removal naturally by rejecting fibers with low cluster assignment probability. We evaluate DFC on three independently acquired cohorts, including data from 220 individuals across genders, ages (young and elderly adults), and different health conditions (healthy control and multiple neuropsychiatric disorders). We compare DFC to several state-of-the-art white matter fiber clustering algorithms. Experimental results demonstrate superior performance of DFC in terms of cluster compactness, generalization ability, anatomical coherence, and computational efficiency.
Using a novel fractional-order gradient method for CNN back-propagation
Authors: Mundher Mohammed Taresh, Ningbo Zhu, Talal Ahmed Ali Ali, Mohammed Alghaili, Weihua Guo
Date: 2022-05-01 00:00:00
Tasks: COVID-19 Diagnosis

Computer-aided diagnosis tools have experienced rapid growth and development in recent years. Among all, deep learning is the most sophisticated and popular tool. In this paper, researchers propose a novel deep learning model and apply it to COVID-19 diagnosis. Our model uses the tool of fractional calculus, which has the potential to improve the performance of gradient methods. To this end, the researcher proposes a fractional-order gradient method for the back-propagation of convolutional neural networks based on the Caputo definition. However, if only the first term of the infinite series of the Caputo definition is used to approximate the fractional-order derivative, the length of the memory is truncated. Therefore, the fractional-order gradient (FGD) method with a fixed memory step and an adjustable number of terms is used to update the weights of the layers. Experiments were performed on the COVIDx dataset to demonstrate fast convergence, good accuracy, and the ability to bypass the local optimal point. We also compared the performance of the developed fractional-order neural networks and Integer-order neural networks. The results confirmed the effectiveness of our proposed model in the diagnosis of COVID-19.
COUCH: Towards Controllable Human-Chair Interactions
Authors: Xiaohan Zhang, Bharat Lal Bhatnagar, Vladimir Guzov, Sebastian Starke, Gerard Pons-Moll
Date: 2022-05-01 00:00:00
Tasks: Human-Object Interaction Detection, Object

Humans interact with an object in many different ways by making contact at different locations, creating a highly complex motion space that can be difficult to learn, particularly when synthesizing such human interactions in a controllable manner. Existing works on synthesizing human scene interaction focus on the high-level control of action but do not consider the fine-grained control of motion. In this work, we study the problem of synthesizing scene interactions conditioned on different contact positions on the object. As a testbed to investigate this new problem, we focus on human-chair interaction as one of the most common actions which exhibit large variability in terms of contacts. We propose a novel synthesis framework COUCH that plans ahead the motion by predicting contact-aware control signals of the hands, which are then used to synthesize contact-conditioned interactions. Furthermore, we contribute a large human-chair interaction dataset with clean annotations, the COUCH Dataset. Our method shows significant quantitative and qualitative improvements over existing methods for human-object interactions. More importantly, our method enables control of the motion through user-specified or automatically predicted contacts.
Deep vs. Shallow Learning: A Benchmark Study in Low Magnitude Earthquake Detection
Authors: Akshat Goel, Denise Gorse
Date: 2022-05-01 00:00:00
Tasks: regression, Time Series, Time Series Analysis

While deep learning models have seen recent high uptake in the geosciences, and are appealing in their ability to learn from minimally processed input data, as black box models they do not provide an easy means to understand how a decision is reached, which in safety-critical tasks especially can be problematical. An alternative route is to use simpler, more transparent white box models, in which task-specific feature construction replaces the more opaque feature discovery process performed automatically within deep learning models. Using data from the Groningen Gas Field in the Netherlands, we build on an existing logistic regression model by the addition of four further features discovered using elastic net driven data mining within the catch22 time series analysis package. We then evaluate the performance of the augmented logistic regression model relative to a deep (CNN) model, pre-trained on the Groningen data, on progressively increasing noise-to-signal ratios. We discover that, for each ratio, our logistic regression model correctly detects every earthquake, while the deep model fails to detect nearly 20 % of seismic events, thus justifying at least a degree of caution in the application of deep models, especially to data with higher noise-to-signal ratios.
The Best of Both Worlds: Combining Model-based and Nonparametric Approaches for 3D Human Body Estimation
Authors: Zhe Wang, Jimei Yang, Charless Fowlkes
Date: 2022-05-01 00:00:00
Tasks: 3D Absolute Human Pose Estimation, 3D Human Pose Estimation, Prediction

Nonparametric based methods have recently shown promising results in reconstructing human bodies from monocular images while model-based methods can help correct these estimates and improve prediction. However, estimating model parameters from global image features may lead to noticeable misalignment between the estimated meshes and image evidence. To address this issue and leverage the best of both worlds, we propose a framework of three consecutive modules. A dense map prediction module explicitly establishes the dense UV correspondence between the image evidence and each part of the body model. The inverse kinematics module refines the key point prediction and generates a posed template mesh. Finally, a UV inpainting module relies on the corresponding feature, prediction and the posed template, and completes the predictions of occluded body shape. Our framework leverages the best of non-parametric and model-based methods and is also robust to partial occlusion. Experiments demonstrate that our framework outperforms existing 3D human estimation methods on multiple public benchmarks.
Preserve Pre-trained Knowledge: Transfer Learning With Self-Distillation For Action Recognition
Authors: Yang Zhou, Zhanhao He, Keyu Lu, Guanhong Wang, Gaoang Wang
Date: 2022-05-01 00:00:00
Tasks: Action Recognition, Representation Learning, Transfer Learning

Video-based action recognition is one of the most popular topics in computer vision. With recent advances of selfsupervised video representation learning approaches, action recognition usually follows a two-stage training framework, i.e., self-supervised pre-training on large-scale unlabeled sets and transfer learning on a downstream labeled set. However, catastrophic forgetting of the pre-trained knowledge becomes the main issue in the downstream transfer learning of action recognition, resulting in a sub-optimal solution. In this paper, to alleviate the above issue, we propose a novel transfer learning approach that combines self-distillation in fine-tuning to preserve knowledge from the pre-trained model learned from the large-scale dataset. Specifically, we fix the encoder from the last epoch as the teacher model to guide the training of the encoder from the current epoch in the transfer learning. With such a simple yet effective learning strategy, we outperform state-of-the-art methods on widely used UCF101 and HMDB51 datasets in action recognition task.
A Dataset-free Deep learning Method for Low-Dose CT Image Reconstruction
Authors: Qiaoqiao Ding, Hui Ji, Yuhui Quan, Xiaoqun Zhang
Date: 2022-05-01 00:00:00
Tasks: Bayesian Inference, Image Reconstruction

Low-dose CT (LDCT) imaging attracted a considerable interest for the reduction of the object's exposure to X-ray radiation. In recent years, supervised deep learning (DL) has been extensively studied for LDCT image reconstruction, which trains a network over a dataset containing many pairs of normal-dose and low-dose images. However, the challenge on collecting many such pairs in the clinical setup limits the application of such supervised-learning-based methods for LDCT image reconstruction in practice. Aiming at addressing the challenges raised by the collection of training dataset, this paper proposed a unsupervised deep learning method for LDCT image reconstruction, which does not require any external training data. The proposed method is built on a re-parametrization technique for Bayesian inference via deep network with random weights, combined with additional total variational~(TV) regularization. The experiments show that the proposed method noticeably outperforms existing dataset-free image reconstruction methods on the test data.
Reinforced Swin-Convs Transformer for Underwater Image Enhancement
Authors: Tingdi Ren, Haiyong Xu, Gangyi Jiang, Mei Yu, Ting Luo
Date: 2022-05-01 00:00:00
Tasks: Image Enhancement, UIE

Underwater Image Enhancement (UIE) technology aims to tackle the challenge of restoring the degraded underwater images due to light absorption and scattering. To address problems, a novel U-Net based Reinforced Swin-Convs Transformer for the Underwater Image Enhancement method (URSCT-UIE) is proposed. Specifically, with the deficiency of U-Net based on pure convolutions, we embedded the Swin Transformer into U-Net for improving the ability to capture the global dependency. Then, given the inadequacy of the Swin Transformer capturing the local attention, the reintroduction of convolutions may capture more local attention. Thus, we provide an ingenious manner for the fusion of convolutions and the core attention mechanism to build a Reinforced Swin-Convs Transformer Block (RSCTB) for capturing more local attention, which is reinforced in the channel and the spatial attention of the Swin Transformer. Finally, the experimental results on available datasets demonstrate that the proposed URSCT-UIE achieves state-of-the-art performance compared with other methods in terms of both subjective and objective evaluations. The code will be released on GitHub after acceptance.