pwc-archive/papers-with-abstracts huggingface dataset:
Training Deep Normalizing Flow Models in Highly Incomplete Data Scenarios with Prior Regularization
Authors: Edgar A. Bernal
Date: 2021-04-03 00:00:00
Tasks: Missing Values, Unity

Deep generative frameworks including GANs and normalizing flow models have proven successful at filling in missing values in partially observed data samples by effectively learning -- either explicitly or implicitly -- complex, high-dimensional statistical distributions. In tasks where the data available for learning is only partially observed, however, their performance decays monotonically as a function of the data missingness rate. In high missing data rate regimes (e.g., 60% and above), it has been observed that state-of-the-art models tend to break down and produce unrealistic and/or semantically inaccurate data. We propose a novel framework to facilitate the learning of data distributions in high paucity scenarios that is inspired by traditional formulations of solutions to ill-posed problems. The proposed framework naturally stems from posing the process of learning from incomplete data as a joint optimization task of the parameters of the model being learned and the missing data values. The method involves enforcing a prior regularization term that seamlessly integrates with objectives used to train explicit and tractable deep generative frameworks such as deep normalizing flow models. We demonstrate via extensive experimental validation that the proposed framework outperforms competing techniques, particularly as the rate of data paucity approaches unity.
"Forget" the Forget Gate: Estimating Anomalies in Videos using Self-contained Long Short-Term Memory Networks
Authors: Habtamu Fanta, Zhiwen Shao, Lizhuang Ma
Date: 2021-04-03 00:00:00
Tasks: Anomaly Detection, Computational Efficiency, Event Detection, Optical Flow Estimation

Abnormal event detection is a challenging task that requires effectively handling intricate features of appearance and motion. In this paper, we present an approach of detecting anomalies in videos by learning a novel LSTM based self-contained network on normal dense optical flow. Due to their sigmoid implementations, standard LSTM's forget gate is susceptible to overlooking and dismissing relevant content in long sequence tasks like abnormality detection. The forget gate mitigates participation of previous hidden state for computation of cell state prioritizing current input. In addition, the hyperbolic tangent activation of standard LSTMs sacrifices performance when a network gets deeper. To tackle these two limitations, we introduce a bi-gated, light LSTM cell by discarding the forget gate and introducing sigmoid activation. Specifically, the LSTM architecture we come up with fully sustains content from previous hidden state thereby enabling the trained model to be robust and make context-independent decision during evaluation. Removing the forget gate results in a simplified and undemanding LSTM cell with improved performance effectiveness and computational efficiency. Empirical evaluations show that the proposed bi-gated LSTM based network outperforms various LSTM based models verifying its effectiveness for abnormality detection and generalization tasks on CUHK Avenue and UCSD datasets.
MR-Contrast-Aware Image-to-Image Translations with Generative Adversarial Networks
Authors: Jonas Denck, Jens Guehring, Andreas Maier, Eva Rothgang
Date: 2021-04-03 00:00:00
Tasks: Data Augmentation, Generative Adversarial Network, Style Transfer

Purpose A Magnetic Resonance Imaging (MRI) exam typically consists of several sequences that yield different image contrasts. Each sequence is parameterized through multiple acquisition parameters that influence image contrast, signal-to-noise ratio, acquisition time, and/or resolution. Depending on the clinical indication, different contrasts are required by the radiologist to make a diagnosis. As MR sequence acquisition is time consuming and acquired images may be corrupted due to motion, a method to synthesize MR images with adjustable contrast properties is required. Methods Therefore, we trained an image-to-image generative adversarial network conditioned on the MR acquisition parameters repetition time and echo time. Our approach is motivated by style transfer networks, whereas the "style" for an image is explicitly given in our case, as it is determined by the MR acquisition parameters our network is conditioned on. Results This enables us to synthesize MR images with adjustable image contrast. We evaluated our approach on the fastMRI dataset, a large set of publicly available MR knee images, and show that our method outperforms a benchmark pix2pix approach in the translation of non-fat-saturated MR images to fat-saturated images. Our approach yields a peak signal-to-noise ratio and structural similarity of 24.48 and 0.66, surpassing the pix2pix benchmark model significantly. Conclusion Our model is the first that enables fine-tuned contrast synthesis, which can be used to synthesize missing MR contrasts or as a data augmentation technique for AI training in MRI.
Aggregated Contextual Transformations for High-Resolution Image Inpainting
Authors: Yanhong Zeng, Jianlong Fu, Hongyang Chao, Baining Guo
Date: 2021-04-03 00:00:00
Tasks: Image Inpainting, Texture Synthesis, Vocal Bursts Intensity Prediction

State-of-the-art image inpainting approaches can suffer from generating distorted structures and blurry textures in high-resolution images (e.g., 512x512). The challenges mainly drive from (1) image content reasoning from distant contexts, and (2) fine-grained texture synthesis for a large missing region. To overcome these two challenges, we propose an enhanced GAN-based model, named Aggregated COntextual-Transformation GAN (AOT-GAN), for high-resolution image inpainting. Specifically, to enhance context reasoning, we construct the generator of AOT-GAN by stacking multiple layers of a proposed AOT block. The AOT blocks aggregate contextual transformations from various receptive fields, allowing to capture both informative distant image contexts and rich patterns of interest for context reasoning. For improving texture synthesis, we enhance the discriminator of AOT-GAN by training it with a tailored mask-prediction task. Such a training objective forces the discriminator to distinguish the detailed appearances of real and synthesized patches, and in turn, facilitates the generator to synthesize clear textures. Extensive comparisons on Places2, the most challenging benchmark with 1.8 million high-resolution images of 365 complex scenes, show that our model outperforms the state-of-the-art by a significant margin in terms of FID with 38.60% relative improvement. A user study including more than 30 subjects further validates the superiority of AOT-GAN. We further evaluate the proposed AOT-GAN in practical applications, e.g., logo removal, face editing, and object removal. Results show that our model achieves promising completions in the real world. We release code and models in https://github.com/researchmm/AOT-GAN-for-Inpainting.
Graph Contrastive Clustering
Authors: Huasong Zhong, Jianlong Wu, Chong Chen, Jianqiang Huang, Minghua Deng, Liqiang Nie, Zhouchen Lin, Xian-Sheng Hua
Date: 2021-04-03 00:00:00
Proceeding: ICCV 2021 10
Tasks: Clustering, Contrastive Learning

Recently, some contrastive learning methods have been proposed to simultaneously learn representations and clustering assignments, achieving significant improvements. However, these methods do not take the category information and clustering objective into consideration, thus the learned representations are not optimal for clustering and the performance might be limited. Towards this issue, we first propose a novel graph contrastive learning framework, which is then applied to the clustering task and we come up with the Graph Constrastive Clustering~(GCC) method. Different from basic contrastive clustering that only assumes an image and its augmentation should share similar representation and clustering assignments, we lift the instance-level consistency to the cluster-level consistency with the assumption that samples in one cluster and their augmentations should all be similar. Specifically, on the one hand, the graph Laplacian based contrastive loss is proposed to learn more discriminative and clustering-friendly features. On the other hand, a novel graph-based contrastive learning strategy is proposed to learn more compact clustering assignments. Both of them incorporate the latent category information to reduce the intra-cluster variance while increasing the inter-cluster variance. Experiments on six commonly used datasets demonstrate the superiority of our proposed approach over the state-of-the-art methods.
IDOL-Net: An Interactive Dual-Domain Parallel Network for CT Metal Artifact Reduction
Authors: Tao Wang, Wenjun Xia, Zexin Lu, Huaiqiang Sun, Yan Liu, Hu Chen, Jiliu Zhou, Yi Zhang
Date: 2021-04-03 00:00:00
Tasks: Computed Tomography (CT), Disentanglement, Metal Artifact Reduction

Due to the presence of metallic implants, the imaging quality of computed tomography (CT) would be heavily degraded. With the rapid development of deep learning, several network models have been proposed for metal artifact reduction (MAR). Since the dual-domain MAR methods can leverage the hybrid information from both sinogram and image domains, they have significantly improved the performance compared to single-domain methods. However,current dual-domain methods usually operate on both domains in a specific order, which implicitly imposes a certain priority prior into MAR and may ignore the latent information interaction between both domains. To address this problem, in this paper, we propose a novel interactive dualdomain parallel network for CT MAR, dubbed as IDOLNet. Different from existing dual-domain methods, the proposed IDOL-Net is composed of two modules. The disentanglement module is utilized to generate high-quality prior sinogram and image as the complementary inputs. The follow-up refinement module consists of two parallel and interactive branches that simultaneously operate on image and sinogram domain, fully exploiting the latent information interaction between both domains. The simulated and clinical results demonstrate that the proposed IDOL-Net outperforms several state-of-the-art models in both qualitative and quantitative aspects.
Learning Mobile CNN Feature Extraction Toward Fast Computation of Visual Object Tracking
Authors: Tsubasa Murate, Takashi Watanabe, Masaki Yamada
Date: 2021-04-03 00:00:00
Tasks: Object Tracking, Visual Object Tracking

In this paper, we construct a lightweight, high-precision and high-speed object tracking using a trained CNN. Conventional methods with trained CNNs use VGG16 network which requires powerful computational resources. Therefore, there is a problem that it is difficult to apply in low computation resources environments. To solve this problem, we use MobileNetV3, which is a CNN for mobile terminals.Based on Feature Map Selection Tracking, we propose a new architecture that extracts effective features of MobileNet for object tracking. The architecture requires no online learning but only offline learning. In addition, by using features of objects other than tracking target, the features of tracking target are extracted more efficiently. We measure the tracking accuracy with Visual Tracker Benchmark and confirm that the proposed method can perform high-precision and high-speed calculation even in low computation resource environments.
Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensing
Authors: Ioannis Kakogeorgiou, Konstantinos Karantzalos
Date: 2021-04-03 00:00:00
Tasks: Explainable artificial intelligence, Explainable Artificial Intelligence (XAI), Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Although deep neural networks hold the state-of-the-art in several remote sensing tasks, their black-box operation hinders the understanding of their decisions, concealing any bias and other shortcomings in datasets and model performance. To this end, we have applied explainable artificial intelligence (XAI) methods in remote sensing multi-label classification tasks towards producing human-interpretable explanations and improve transparency. In particular, we utilized and trained deep learning models with state-of-the-art performance in the benchmark BigEarthNet and SEN12MS datasets. Ten XAI methods were employed towards understanding and interpreting models' predictions, along with quantitative metrics to assess and compare their performance. Numerous experiments were performed to assess the overall performance of XAI methods for straightforward prediction cases, competing multiple labels, as well as misclassification cases. According to our findings, Occlusion, Grad-CAM and Lime were the most interpretable and reliable XAI methods. However, none delivers high-resolution outputs, while apart from Grad-CAM, both Lime and Occlusion are computationally expensive. We also highlight different aspects of XAI performance and elaborate with insights on black-box decisions in order to improve transparency, understand their behavior and reveal, as well, datasets' particularities.
Interpretable Unsupervised Diversity Denoising and Artefact Removal
Authors: Mangal Prakash, Mauricio Delbracio, Peyman Milanfar, Florian Jug
Date: 2021-04-03 00:00:00
Proceeding: ICLR 2022 4
Tasks: Denoising, Diversity, Image Denoising, Image Restoration, valid

Image denoising and artefact removal are complex inverse problems admitting multiple valid solutions. Unsupervised diversity restoration, that is, obtaining a diverse set of possible restorations given a corrupted image, is important for ambiguity removal in many applications such as microscopy where paired data for supervised training are often unobtainable. In real world applications, imaging noise and artefacts are typically hard to model, leading to unsatisfactory performance of existing unsupervised approaches. This work presents an interpretable approach for unsupervised and diverse image restoration. To this end, we introduce a capable architecture called Hierarchical DivNoising (HDN) based on hierarchical Variational Autoencoder. We show that HDN learns an interpretable multi-scale representation of artefacts and we leverage this interpretability to remove imaging artefacts commonly occurring in microscopy data. Our method achieves state-of-the-art results on twelve benchmark image denoising datasets while providing access to a whole distribution of sensibly restored solutions. Additionally, we demonstrate on three real microscopy datasets that HDN removes artefacts without supervision, being the first method capable of doing so while generating multiple plausible restorations all consistent with the given corrupted image.
Generation of Gradient-Preserving Images allowing HOG Feature Extraction
Authors: Masaki Kitayama, Hitoshi Kiya
Date: 2021-04-03 00:00:00
Tasks: BIG-bench Machine Learning, Face Recognition, Privacy Preserving

In this paper, we propose a method for generating visually protected images, referred to as gradient-preserving images. The protected images allow us to directly extract Histogram-of-Oriented-Gradients (HOG) features for privacy-preserving machine learning. In an experiment, HOG features extracted from gradient-preserving images are applied to a face recognition algorithm to demonstrate the effectiveness of the proposed method.
Recursively Refined R-CNN: Instance Segmentation with Self-RoI Rebalancing
Authors: Leonardo Rossi, Akbar Karimi, Andrea Prati
Date: 2021-04-03 00:00:00
Tasks: Instance Segmentation, Object Detection, Semantic Segmentation

Within the field of instance segmentation, most of the state-of-the-art deep learning networks rely nowadays on cascade architectures, where multiple object detectors are trained sequentially, re-sampling the ground truth at each step. This offers a solution to the problem of exponentially vanishing positive samples. However, it also translates into an increase in network complexity in terms of the number of parameters. To address this issue, we propose Recursively Refined R-CNN (R^3-CNN) which avoids duplicates by introducing a loop mechanism instead. At the same time, it achieves a quality boost using a recursive re-sampling technique, where a specific IoU quality is utilized in each recursion to eventually equally cover the positive spectrum. Our experiments highlight the specific encoding of the loop mechanism in the weights, requiring its usage at inference time. The R^3-CNN architecture is able to surpass the recently proposed HTC model, while reducing the number of parameters significantly. Experiments on COCO minival 2017 dataset show performance boost independently from the utilized baseline model. The code is available online at https://github.com/IMPLabUniPr/mmdetection/tree/r3_cnn.
DARCNN: Domain Adaptive Region-based Convolutional Neural Network for Unsupervised Instance Segmentation in Biomedical Images
Authors: Joy Hsu, Wah Chiu, Serena Yeung
Date: 2021-04-03 00:00:00
Proceeding: CVPR 2021 1
Tasks: Domain Adaptation, Instance Segmentation, Medical Image Segmentation, scientific discovery, Segmentation, Semantic Segmentation, Unsupervised Instance Segmentation

In the biomedical domain, there is an abundance of dense, complex data where objects of interest may be challenging to detect or constrained by limits of human knowledge. Labelled domain specific datasets for supervised tasks are often expensive to obtain, and furthermore discovery of novel distinct objects may be desirable for unbiased scientific discovery. Therefore, we propose leveraging the wealth of annotations in benchmark computer vision datasets to conduct unsupervised instance segmentation for diverse biomedical datasets. The key obstacle is thus overcoming the large domain shift from common to biomedical images. We propose a Domain Adaptive Region-based Convolutional Neural Network (DARCNN), that adapts knowledge of object definition from COCO, a large labelled vision dataset, to multiple biomedical datasets. We introduce a domain separation module, a self-supervised representation consistency loss, and an augmented pseudo-labelling stage within DARCNN to effectively perform domain adaptation across such large domain shifts. We showcase DARCNN's performance for unsupervised instance segmentation on numerous biomedical datasets.
Efficient DETR: Improving End-to-End Object Detector with Dense Prior
Authors: Zhuyu Yao, Jiangbo Ai, Boxun Li, Chi Zhang
Date: 2021-04-03 00:00:00
Tasks: Decoder, Object, object-detection, Object Detection

The recently proposed end-to-end transformer detectors, such as DETR and Deformable DETR, have a cascade structure of stacking 6 decoder layers to update object queries iteratively, without which their performance degrades seriously. In this paper, we investigate that the random initialization of object containers, which include object queries and reference points, is mainly responsible for the requirement of multiple iterations. Based on our findings, we propose Efficient DETR, a simple and efficient pipeline for end-to-end object detection. By taking advantage of both dense detection and sparse set detection, Efficient DETR leverages dense prior to initialize the object containers and brings the gap of the 1-decoder structure and 6-decoder structure. Experiments conducted on MS COCO show that our method, with only 3 encoder layers and 1 decoder layer, achieves competitive performance with state-of-the-art object detection methods. Efficient DETR is also robust in crowded scenes. It outperforms modern detectors on CrowdHuman dataset by a large margin.
Multimedia Technology Applications and Algorithms: A Survey
Authors: Palak Tiwary, Sanjida Ahmed
Date: 2021-04-03 00:00:00
Tasks: Disparity Estimation, Survey

Multimedia related research and development has evolved rapidly in the last few years with advancements in hardware, software and network infrastructures. As a result, multimedia has been integrated into domains like Healthcare and Medicine, Human facial feature extraction and tracking, pose recognition, disparity estimation, etc. This survey gives an overview of the various multimedia technologies and algorithms developed in the domains mentioned.
Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation
Authors: Astuti Sharma, Tarun Kalluri, Manmohan Chandraker
Date: 2021-04-03 00:00:00
Proceeding: CVPR 2021 1
Tasks: Clustering, Domain Adaptation, Unsupervised Domain Adaptation

Domain adaptation deals with training models using large scale labeled data from a specific source domain and then adapting the knowledge to certain target domains that have few or no labels. Many prior works learn domain agnostic feature representations for this purpose using a global distribution alignment objective which does not take into account the finer class specific structure in the source and target domains. We address this issue in our work and propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. ILA-DA simultaneously accounts for intra-class clustering as well as inter-class separation among the categories, resulting in less noisy classifier boundaries, improved transferability and increased accuracy. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets and provide insights into the proposed alignment approach. Code will be made publicly available at https://github.com/astuti/ILA-DA.
Multi-class motion-based semantic segmentation for ureteroscopy and laser lithotripsy
Authors: Soumya Gupta, Sharib Ali, Louise Goldsmith, Ben Turney, Jens Rittscher
Date: 2021-04-02 00:00:00
Tasks: Data Augmentation, Segmentation, Semantic Segmentation

Kidney stones represent a considerable burden for public health-care systems. Ureteroscopy with laser lithotripsy has evolved as the most commonly used technique for the treatment of kidney stones. Automated segmentation of kidney stones and laser fiber is an important initial step to performing any automated quantitative analysis of the stones, particularly stone-size estimation, that helps the surgeon decide if the stone requires more fragmentation. Factors such as turbid fluid inside the cavity, specularities, motion blur due to kidney movements and camera motion, bleeding, and stone debris impact the quality of vision within the kidney and lead to extended operative times. To the best of our knowledge, this is the first attempt made towards multi-class segmentation in ureteroscopy and laser lithotripsy data. We propose an end-to-end CNN-based framework for the segmentation of stones and laser fiber. The proposed approach utilizes two sub-networks: HybResUNet, a version of residual U-Net, that uses residual connections in the encoder path of U-Net and a DVFNet that generates DVF predictions which are then used to prune the prediction maps. We also present ablation studies that combine dilated convolutions, recurrent and residual connections, ASPP and attention gate. We propose a compound loss function that improves our segmentation performance. We have also provided an ablation study to determine the optimal data augmentation strategy. Our qualitative and quantitative results illustrate that our proposed method outperforms SOTA methods such as UNet and DeepLabv3+ showing an improvement of 5.2% and 15.93%, respectively, for the combined mean of DSC and JI in our invivo test dataset. We also show that our proposed model generalizes better on a new clinical dataset showing a mean improvement of 25.4%, 20%, and 11% over UNet, HybResUNet, and DeepLabv3+, respectively, for the same metric.