pwc-archive/papers-with-abstracts huggingface dataset:
Bayesian Matrix Decomposition and Applications
Authors: Jun Lu
Date: 2023-02-18 00:00:00
Tasks: Variational Inference

The sole aim of this book is to give a self-contained introduction to concepts and mathematical tools in Bayesian matrix decomposition in order to seamlessly introduce matrix decomposition techniques and their applications in subsequent sections. However, we clearly realize our inability to cover all the useful and interesting results concerning Bayesian matrix decomposition and given the paucity of scope to present this discussion, e.g., the separated analysis of variational inference for conducting the optimization. We refer the reader to literature in the field of Bayesian analysis for a more detailed introduction to the related fields. This book is primarily a summary of purpose, significance of important Bayesian matrix decomposition methods, e.g., real-valued decomposition, nonnegative matrix factorization, Bayesian interpolative decomposition, and the origin and complexity of the methods which shed light on their applications. The mathematical prerequisite is a first course in statistics and linear algebra. Other than this modest background, the development is self-contained, with rigorous proof provided throughout.
GLUECons: A Generic Benchmark for Learning Under Constraints
Authors: Hossein Rajaby Faghihi, Aliakbar Nafar, Chen Zheng, Roshanak Mirzaee, Yue Zhang, Andrzej Uszok, Alexander Wan, Tanawan Premsri, Dan Roth, Parisa Kordjamshidi
Date: 2023-02-16 00:00:00

Recent research has shown that integrating domain knowledge into deep learning architectures is effective -- it helps reduce the amount of required data, improves the accuracy of the models' decisions, and improves the interpretability of models. However, the research community is missing a convened benchmark for systematically evaluating knowledge integration methods. In this work, we create a benchmark that is a collection of nine tasks in the domains of natural language processing and computer vision. In all cases, we model external knowledge as constraints, specify the sources of the constraints for each task, and implement various models that use these constraints. We report the results of these models using a new set of extended evaluation criteria in addition to the task performances for a more in-depth analysis. This effort provides a framework for a more comprehensive and systematic comparison of constraint integration techniques and for identifying related research challenges. It will facilitate further research for alleviating some problems of state-of-the-art neural models.
Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition
Authors: Chen Guo, Tianjian Jiang, Xu Chen, Jie Song, Otmar Hilliges
Date: 2023-02-22 00:00:00
Proceeding: CVPR 2023 1
Tasks: 3D Human Reconstruction, global-optimization, Surface Reconstruction

We present Vid2Avatar, a method to learn human avatars from monocular in-the-wild videos. Reconstructing humans that move naturally from monocular in-the-wild videos is difficult. Solving it requires accurately separating humans from arbitrary backgrounds. Moreover, it requires reconstructing detailed 3D surface from short video sequences, making it even more challenging. Despite these challenges, our method does not require any groundtruth supervision or priors extracted from large datasets of clothed human scans, nor do we rely on any external segmentation modules. Instead, it solves the tasks of scene decomposition and surface reconstruction directly in 3D by modeling both the human and the background in the scene jointly, parameterized via two separate neural fields. Specifically, we define a temporally consistent human representation in canonical space and formulate a global optimization over the background model, the canonical human shape and texture, and per-frame human pose parameters. A coarse-to-fine sampling strategy for volume rendering and novel objectives are introduced for a clean separation of dynamic human and static background, yielding detailed and robust 3D human geometry reconstructions. We evaluate our methods on publicly available datasets and show improvements over prior art.
Uncovering Bias in Face Generation Models
Authors: Cristian Muñoz, Sara Zannone, Umar Mohammed, Adriano Koshiyama
Date: 2023-02-22 00:00:00
Tasks: Attribute, Decision Making, Face Generation, Fairness

Recent advancements in GANs and diffusion models have enabled the creation of high-resolution, hyper-realistic images. However, these models may misrepresent certain social groups and present bias. Understanding bias in these models remains an important research question, especially for tasks that support critical decision-making and could affect minorities. The contribution of this work is a novel analysis covering architectures and embedding spaces for fine-grained understanding of bias over three approaches: generators, attribute modifier, and post-processing bias mitigators. This work shows that generators suffer from bias across all social groups with attribute preferences such as between 75%-85% for whiteness and 60%-80% for the female gender (for all trained CelebA models) and low probabilities of generating children and older men. Modifier and mitigators work as post-processor and change the generator performance. For instance, attribute channel perturbation strategies modify the embedding spaces. We quantify the influence of this change on group fairness by measuring the impact on image quality and group features. Specifically, we use the Fr\'echet Inception Distance (FID), the Face Matching Error and the Self-Similarity score. For Interfacegan, we analyze one and two attribute channel perturbations and examine the effect on the fairness distribution and the quality of the image. Finally, we analyzed the post-processing bias mitigators, which are the fastest and most computationally efficient way to mitigate bias. We find that these mitigation techniques show similar results on KL divergence and FID score, however, self-similarity scores show a different feature concentration on the new groups of the data distribution. The weaknesses and ongoing challenges described in this work must be considered in the pursuit of creating fair and unbiased face generation models.
Word level Bangla Sign Language Dataset for Continuous BSL Recognition
Authors: Md Shamimul Islam, A. J. M. Akhtarujjaman Joha, Md Nur Hossain, Sohaib Abdullah, Ibrahim Elwarfalli, Md Mahedi Hasan
Date: 2023-02-22 00:00:00
Tasks: Sign Language Recognition

An robust sign language recognition system can greatly alleviate communication barriers, particularly for people who struggle with verbal communication. This is crucial for human growth and progress as it enables the expression of thoughts, feelings, and ideas. However, sign recognition is a complex task that faces numerous challenges such as same gesture patterns for multiple signs, lighting, clothing, carrying conditions, and the presence of large poses, as well as illumination discrepancies across different views. Additionally, the absence of an extensive Bangla sign language video dataset makes it even more challenging to operate recognition systems, particularly when utilizing deep learning techniques. In order to address this issue, firstly, we created a large-scale dataset called the MVBSL-W50, which comprises 50 isolated words across 13 categories. Secondly, we developed an attention-based Bi-GRU model that captures the temporal dynamics of pose information for individuals communicating through sign language. The proposed model utilizes human pose information, which has shown to be successful in analyzing sign language patterns. By focusing solely on movement information and disregarding body appearance and environmental factors, the model is simplified and can achieve a speedier performance. The accuracy of the model is reported to be 85.64%.
K-Diag: Knowledge-enhanced Disease Diagnosis in Radiographic Imaging
Authors: Chaoyi Wu, Xiaoman Zhang, Yanfeng Wang, Ya zhang, Weidi Xie
Date: 2023-02-22 00:00:00
Tasks: Anatomy, Contrastive Learning, Implicit Relations, Zero-Shot Learning

In this paper, we consider the problem of disease diagnosis. Unlike the conventional learning paradigm that treats labels independently, we propose a knowledge-enhanced framework, that enables training visual representation with the guidance of medical domain knowledge. In particular, we make the following contributions: First, to explicitly incorporate experts' knowledge, we propose to learn a neural representation for the medical knowledge graph via contrastive learning, implicitly establishing relations between different medical concepts. Second, while training the visual encoder, we keep the parameters of the knowledge encoder frozen and propose to learn a set of prompt vectors for efficient adaptation. Third, we adopt a Transformer-based disease-query module for cross-model fusion, which naturally enables explainable diagnosis results via cross attention. To validate the effectiveness of our proposed framework, we conduct thorough experiments on three x-ray imaging datasets across different anatomy structures, showing our model is able to exploit the implicit relations between diseases/findings, thus is beneficial to the commonly encountered problem in the medical domain, namely, long-tailed and zero-shot recognition, which conventional methods either struggle or completely fail to realize.
A study on the invariance in security whatever the dimension of images for the steganalysis by deep-learning
Authors: Kévin Planolles, Marc Chaumont, Frédéric Comby
Date: 2023-02-22 00:00:00
Tasks: Steganalysis

In this paper, we study the performance invariance of convolutional neural networks when confronted with variable image sizes in the context of a more "wild steganalysis". First, we propose two algorithms and definitions for a fine experimental protocol with datasets owning "similar difficulty" and "similar security". The "smart crop 2" algorithm allows the introduction of the Nearly Nested Image Datasets (NNID) that ensure "a similar difficulty" between various datasets, and a dichotomous research algorithm allows a "similar security". Second, we show that invariance does not exist in state-of-the-art architectures. We also exhibit a difference in behavior depending on whether we test on images larger or smaller than the training images. Finally, based on the experiments, we propose to use the dilated convolution which leads to an improvement of a state-of-the-art architecture.
Slim U-Net: Efficient Anatomical Feature Preserving U-net Architecture for Ultrasound Image Segmentation
Authors: Deepak Raina, Kashish Verma, SH Chandrashekhara, Subir Kumar Saha
Date: 2023-02-22 00:00:00
Tasks: Anatomy, Computational Efficiency, Image Segmentation, Segmentation, Semantic Segmentation

We investigate the applicability of U-Net based models for segmenting Urinary Bladder (UB) in male pelvic view UltraSound (US) images. The segmentation of UB in the US image aids radiologists in diagnosing the UB. However, UB in US images has arbitrary shapes, indistinct boundaries and considerably large inter- and intra-subject variability, making segmentation a quite challenging task. Our study of the state-of-the-art (SOTA) segmentation network, U-Net, for the problem reveals that it often fails to capture the salient characteristics of UB due to the varying shape and scales of anatomy in the noisy US image. Also, U-net has an excessive number of trainable parameters, reporting poor computational efficiency during training. We propose a Slim U-Net to address the challenges of UB segmentation. Slim U-Net proposes to efficiently preserve the salient features of UB by reshaping the structure of U-Net using a less number of 2D convolution layers in the contracting path, in order to preserve and impose them on expanding path. To effectively distinguish the blurred boundaries, we propose a novel annotation methodology, which includes the background area of the image at the boundary of a marked region of interest (RoI), thereby steering the model's attention towards boundaries. In addition, we suggested a combination of loss functions for network training in the complex segmentation of UB. The experimental results demonstrate that Slim U-net is statistically superior to U-net for UB segmentation. The Slim U-net further decreases the number of trainable parameters and training time by 54% and 57.7%, respectively, compared to the standard U-Net, without compromising the segmentation accuracy.
Evaluation of Extra Pixel Interpolation with Mask Processing for Medical Image Segmentation with Deep Learning
Authors: Olivier Rukundo
Date: 2023-02-22 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Semantic Segmentation

Current mask processing operations rely on interpolation algorithms that do not produce extra pixels, such as nearest neighbor (NN) interpolation, as opposed to algorithms that do produce extra pixels, like bicubic (BIC) or bilinear (BIL) interpolation. In our previous study, the author proposed an alternative approach to NN-based mask processing and evaluated its effects on deep learning training outcomes. In this study, the author evaluated the effects of both BIC-based image and mask processing and BIC-and-NN-based image and mask processing versus NN-based image and mask processing. The evaluation revealed that the BIC-BIC model/network was an 8.9578 % (with image size 256 x 256) and a 1.0496 % (with image size 384 x 384) increase of the NN-NN network compared to the NN-BIC network which was an 8.3127 % (with image size 256 x 256) and a 0.2887 % (with image size 384 x 384) increase of the NN-NN network.
A Global and Patch-wise Contrastive Loss for Accurate Automated Exudate Detection
Authors: Wei Tang, Kangning Cui, Raymond H. Chan
Date: 2023-02-22 00:00:00
Tasks: Contrastive Learning, Segmentation

Diabetic retinopathy (DR) is a leading global cause of blindness. Early detection of hard exudates plays a crucial role in identifying DR, which aids in treating diabetes and preventing vision loss. However, the unique characteristics of hard exudates, ranging from their inconsistent shapes to indistinct boundaries, pose significant challenges to existing segmentation techniques. To address these issues, we present a novel supervised contrastive learning framework to optimize hard exudate segmentation. Specifically, we introduce a patch-wise density contrasting scheme to distinguish between areas with varying lesion concentrations, and therefore improve the model's proficiency in segmenting small lesions. To handle the ambiguous boundaries, we develop a discriminative edge inspection module to dynamically analyze the pixels that lie around the boundaries and accurately delineate the exudates. Upon evaluation using the IDRiD dataset and comparison with state-of-the-art frameworks, our method exhibits its effectiveness and shows potential for computer-assisted hard exudate detection. The code to replicate experiments is available at github.com/wetang7/HECL/.
Selective experience replay compression using coresets for lifelong deep reinforcement learning in medical imaging
Authors: Guangyao Zheng, Samson Zhou, Vladimir Braverman, Michael A. Jacobs, Vishwa S. Parekh
Date: 2023-02-22 00:00:00
Tasks: Brain Tumor Segmentation, Deep Reinforcement Learning, Lifelong learning, Tumor Segmentation

Selective experience replay is a popular strategy for integrating lifelong learning with deep reinforcement learning. Selective experience replay aims to recount selected experiences from previous tasks to avoid catastrophic forgetting. Furthermore, selective experience replay based techniques are model agnostic and allow experiences to be shared across different models. However, storing experiences from all previous tasks make lifelong learning using selective experience replay computationally very expensive and impractical as the number of tasks increase. To that end, we propose a reward distribution-preserving coreset compression technique for compressing experience replay buffers stored for selective experience replay. We evaluated the coreset compression technique on the brain tumor segmentation (BRATS) dataset for the task of ventricle localization and on the whole-body MRI for localization of left knee cap, left kidney, right trochanter, left lung, and spleen. The coreset lifelong learning models trained on a sequence of 10 different brain MR imaging environments demonstrated excellent performance localizing the ventricle with a mean pixel error distance of 12.93 for the compression ratio of 10x. In comparison, the conventional lifelong learning model localized the ventricle with a mean pixel distance of 10.87. Similarly, the coreset lifelong learning models trained on whole-body MRI demonstrated no significant difference (p=0.28) between the 10x compressed coreset lifelong learning models and conventional lifelong learning models for all the landmarks. The mean pixel distance for the 10x compressed models across all the landmarks was 25.30, compared to 19.24 for the conventional lifelong learning models. Our results demonstrate that the potential of the coreset-based ERB compression method for compressing experiences without a significant drop in performance.
S3I-PointHop: SO(3)-Invariant PointHop for 3D Point Cloud Classification
Authors: Pranav Kadam, Hardik Prajapati, Min Zhang, Jintang Xue, Shan Liu, C. -C. Jay Kuo
Date: 2023-02-22 00:00:00
Tasks: 3D Point Cloud Classification, Classification, Point Cloud Classification

Many point cloud classification methods are developed under the assumption that all point clouds in the dataset are well aligned with the canonical axes so that the 3D Cartesian point coordinates can be employed to learn features. When input point clouds are not aligned, the classification performance drops significantly. In this work, we focus on a mathematically transparent point cloud classification method called PointHop, analyze its reason for failure due to pose variations, and solve the problem by replacing its pose dependent modules with rotation invariant counterparts. The proposed method is named SO(3)-Invariant PointHop (or S3I-PointHop in short). We also significantly simplify the PointHop pipeline using only one single hop along with multiple spatial aggregation techniques. The idea of exploiting more spatial information is novel. Experiments on the ModelNet40 dataset demonstrate the superiority of S3I-PointHop over traditional PointHop-like methods.
On The Role of Alias and Band-Shift for Sentinel-2 Super-Resolution
Authors: Ngoc Long Nguyen, Jérémy Anger, Lara Raad, Bruno Galerne, Gabriele Facciolo
Date: 2023-02-22 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

In this work, we study the problem of single-image super-resolution (SISR) of Sentinel-2 imagery. We show that thanks to its unique sensor specification, namely the inter-band shift and alias, that deep-learning methods are able to recover fine details. By training a model using a simple $L_1$ loss, results are free of hallucinated details. For this study, we build a dataset of pairs of images Sentinel-2/PlanetScope to train and evaluate our super-resolution (SR) model.
Magnification Invariant Medical Image Analysis: A Comparison of Convolutional Networks, Vision Transformers, and Token Mixers
Authors: Pranav Jeevan, Nikhil Cherian Kurian, Amit Sethi
Date: 2023-02-22 00:00:00
Tasks: Breast Cancer Histology Image Classification, Deep Learning, Image Classification, Medical Image Analysis

Convolution Neural Networks (CNNs) are widely used in medical image analysis, but their performance degrade when the magnification of testing images differ from the training images. The inability of CNNs to generalize across magnification scales can result in sub-optimal performance on external datasets. This study aims to evaluate the robustness of various deep learning architectures in the analysis of breast cancer histopathological images with varying magnification scales at training and testing stages. Here we explore and compare the performance of multiple deep learning architectures, including CNN-based ResNet and MobileNet, self-attention-based Vision Transformers and Swin Transformers, and token-mixing models, such as FNet, ConvMixer, MLP-Mixer, and WaveMix. The experiments are conducted using the BreakHis dataset, which contains breast cancer histopathological images at varying magnification levels. We show that performance of WaveMix is invariant to the magnification of training and testing data and can provide stable and good classification accuracy. These evaluations are critical in identifying deep learning architectures that can robustly handle changes in magnification scale, ensuring that scale changes across anatomical structures do not disturb the inference results.
Transformer-Based Sensor Fusion for Autonomous Driving: A Survey
Authors: Apoorv Singh
Date: 2023-02-22 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, object-detection, Object Detection, Sensor Fusion, Survey

Sensor fusion is an essential topic in many perception systems, such as autonomous driving and robotics. Transformers-based detection head and CNN-based feature encoder to extract features from raw sensor-data has emerged as one of the best performing sensor-fusion 3D-detection-framework, according to the dataset leaderboards. In this work we provide an in-depth literature survey of transformer based 3D-object detection task in the recent past, primarily focusing on the sensor fusion. We also briefly go through the Vision transformers (ViT) basics, so that readers can easily follow through the paper. Moreover, we also briefly go through few of the non-transformer based less-dominant methods for sensor fusion for autonomous driving. In conclusion we summarize with sensor-fusion trends to follow and provoke future research. More updated summary can be found at: https://github.com/ApoorvRoboticist/Transformers-Sensor-Fusion
Gap-closing Matters: Perceptual Quality Evaluation and Optimization of Low-Light Image Enhancement
Authors: Baoliang Chen, Lingyu Zhu, Hanwei Zhu, Wenhan Yang, Linqi Song, Shiqi Wang
Date: 2023-02-22 00:00:00
Tasks: Image Enhancement, Image Quality Assessment, Low-Light Image Enhancement

There is a growing consensus in the research community that the optimization of low-light image enhancement approaches should be guided by the visual quality perceived by end users. Despite the substantial efforts invested in the design of low-light enhancement algorithms, there has been comparatively limited focus on assessing subjective and objective quality systematically. To mitigate this gap and provide a clear path towards optimizing low-light image enhancement for better visual quality, we propose a gap-closing framework. In particular, our gap-closing framework starts with the creation of a large-scale dataset for Subjective QUality Assessment of REconstructed LOw-Light Images (SQUARE-LOL). This database serves as the foundation for studying the quality of enhanced images and conducting a comprehensive subjective user study. Subsequently, we propose an objective quality assessment measure that plays a critical role in bridging the gap between visual quality and enhancement. Finally, we demonstrate that our proposed objective quality measure can be incorporated into the process of optimizing the learning of the enhancement model toward perceptual optimality. We validate the effectiveness of our proposed framework through both the accuracy of quality prediction and the perceptual quality of image enhancement. Our database and codes are publicly available at https://github.com/Baoliang93/IACA_For_Lowlight_IQA.