pwc-archive/papers-with-abstracts huggingface dataset:
APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking
Authors: Yuxiang Yang, Junjie Yang, Yufei Xu, Jing Zhang, Long Lan, DaCheng Tao
Date: 2022-06-12 00:00:00
Tasks: Animal Pose Estimation, Domain Generalization, Pose Estimation, Transfer Learning

Animal pose estimation and tracking (APT) is a fundamental task for detecting and tracking animal keypoints from a sequence of video frames. Previous animal-related datasets focus either on animal tracking or single-frame animal pose estimation, and never on both aspects. The lack of APT datasets hinders the development and evaluation of video-based animal pose estimation and tracking methods, limiting real-world applications, e.g., understanding animal behavior in wildlife conservation. To fill this gap, we make the first step and propose APT-36K, i.e., the first large-scale benchmark for animal pose estimation and tracking. Specifically, APT-36K consists of 2,400 video clips collected and filtered from 30 animal species with 15 frames for each video, resulting in 36,000 frames in total. After manual annotation and careful double-check, high-quality keypoint and tracking annotations are provided for all the animal instances. Based on APT-36K, we benchmark several representative models on the following three tracks: (1) supervised animal pose estimation on a single frame under intra- and inter-domain transfer learning settings, (2) inter-species domain generalization test for unseen animals, and (3) animal pose estimation with animal tracking. Based on the experimental results, we gain some empirical insights and show that APT-36K provides a valuable animal pose estimation and tracking benchmark, offering new challenges and opportunities for future research. The code and dataset will be made publicly available at https://github.com/pandorgan/APT-36K.
STD-NET: Search of Image Steganalytic Deep-learning Architecture via Hierarchical Tensor Decomposition
Authors: Shunquan Tan, Qiushi Li, Laiyuan Li, Bin Li, Jiwu Huang
Date: 2022-06-12 00:00:00
Tasks: Model Compression, Steganalysis, Tensor Decomposition

Recent studies shows that the majority of existing deep steganalysis models have a large amount of redundancy, which leads to a huge waste of storage and computing resources. The existing model compression method cannot flexibly compress the convolutional layer in residual shortcut block so that a satisfactory shrinking rate cannot be obtained. In this paper, we propose STD-NET, an unsupervised deep-learning architecture search approach via hierarchical tensor decomposition for image steganalysis. Our proposed strategy will not be restricted by various residual connections, since this strategy does not change the number of input and output channels of the convolution block. We propose a normalized distortion threshold to evaluate the sensitivity of each involved convolutional layer of the base model to guide STD-NET to compress target network in an efficient and unsupervised approach, and obtain two network structures of different shapes with low computation cost and similar performance compared with the original one. Extensive experiments have confirmed that, on one hand, our model can achieve comparable or even better detection performance in various steganalytic scenarios due to the great adaptivity of the obtained network architecture. On the other hand, the experimental results also demonstrate that our proposed strategy is more efficient and can remove more redundancy compared with previous steganalytic network compression methods.
Preprocessing Enhanced Image Compression for Machine Vision
Authors: Guo Lu, Xingtong Ge, Tianxiong Zhong, Jing Geng, Qiang Hu
Date: 2022-06-12 00:00:00
Tasks: Image Compression, object-detection, Object Detection, Quantization

Recently, more and more images are compressed and sent to the back-end devices for the machine analysis tasks~(\textit{e.g.,} object detection) instead of being purely watched by humans. However, most traditional or learned image codecs are designed to minimize the distortion of the human visual system without considering the increased demand from machine vision systems. In this work, we propose a preprocessing enhanced image compression method for machine vision tasks to address this challenge. Instead of relying on the learned image codecs for end-to-end optimization, our framework is built upon the traditional non-differential codecs, which means it is standard compatible and can be easily deployed in practical applications. Specifically, we propose a neural preprocessing module before the encoder to maintain the useful semantic information for the downstream tasks and suppress the irrelevant information for bitrate saving. Furthermore, our neural preprocessing module is quantization adaptive and can be used in different compression ratios. More importantly, to jointly optimize the preprocessing module with the downstream machine vision tasks, we introduce the proxy network for the traditional non-differential codecs in the back-propagation stage. We provide extensive experiments by evaluating our compression method for two representative downstream tasks with different backbone networks. Experimental results show our method achieves a better trade-off between the coding bitrate and the performance of the downstream machine vision tasks by saving about 20% bitrate.
TileGen: Tileable, Controllable Material Generation and Capture
Authors: Xilong Zhou, Miloš Hašan, Valentin Deschaintre, Paul Guerrero, Kalyan Sunkavalli, Nima Kalantari
Date: 2022-06-12 00:00:00
Tasks: Inverse Rendering

Recent methods (e.g. MaterialGAN) have used unconditional GANs to generate per-pixel material maps, or as a prior to reconstruct materials from input photographs. These models can generate varied random material appearance, but do not have any mechanism to constrain the generated material to a specific category or to control the coarse structure of the generated material, such as the exact brick layout on a brick wall. Furthermore, materials reconstructed from a single input photo commonly have artifacts and are generally not tileable, which limits their use in practical content creation pipelines. We propose TileGen, a generative model for SVBRDFs that is specific to a material category, always tileable, and optionally conditional on a provided input structure pattern. TileGen is a variant of StyleGAN whose architecture is modified to always produce tileable (periodic) material maps. In addition to the standard "style" latent code, TileGen can optionally take a condition image, giving a user direct control over the dominant spatial (and optionally color) features of the material. For example, in brick materials, the user can specify a brick layout and the brick color, or in leather materials, the locations of wrinkles and folds. Our inverse rendering approach can find a material perceptually matching a single target photograph by optimization. This reconstruction can also be conditional on a user-provided pattern. The resulting materials are tileable, can be larger than the target image, and are editable by varying the condition.
Indirect-Instant Attention Optimization for Crowd Counting in Dense Scenes
Authors: Suyu Han, Guodong Wang, Donghua Liu
Date: 2022-06-12 00:00:00
Tasks: Crowd Counting

One of appealing approaches to guiding learnable parameter optimization, such as feature maps, is global attention, which enlightens network intelligence at a fraction of the cost. However, its loss calculation process still falls short: 1)We can only produce one-dimensional 'pseudo labels' for attention, since the artificial threshold involved in the procedure is not robust; 2) The attention awaiting loss calculation is necessarily high-dimensional, and decreasing it by convolution will inevitably introduce additional learnable parameters, thus confusing the source of the loss. To this end, we devise a simple but efficient Indirect-Instant Attention Optimization (IIAO) module based on SoftMax-Attention , which transforms high-dimensional attention map into a one-dimensional feature map in the mathematical sense for loss calculation midway through the network, while automatically providing adaptive multi-scale fusion to feature pyramid module. The special transformation yields relatively coarse features and, originally, the predictive fallibility of regions varies by crowd density distribution, so we tailor the Regional Correlation Loss (RCLoss) to retrieve continuous error-prone regions and smooth spatial information . Extensive experiments have proven that our approach surpasses previous SOTA methods in many benchmark datasets.
A Fast Alternating Minimization Algorithm for Coded Aperture Snapshot Spectral Imaging Based on Sparsity and Deep Image Priors
Authors: Qile Zhao, Xianhong Zhao, Xu Ma, Xudong Chen, Gonzalo R. Arce
Date: 2022-06-12 00:00:00
Tasks: Compressive Sensing

Coded aperture snapshot spectral imaging (CASSI) is a technique used to reconstruct three-dimensional hyperspectral images (HSIs) from one or several two-dimensional projection measurements. However, fewer projection measurements or more spectral channels leads to a severly ill-posed problem, in which case regularization methods have to be applied. In order to significantly improve the accuracy of reconstruction, this paper proposes a fast alternating minimization algorithm based on the sparsity and deep image priors (Fama-SDIP) of natural images. By integrating deep image prior (DIP) into the principle of compressive sensing (CS) reconstruction, the proposed algorithm can achieve state-of-the-art results without any training dataset. Extensive experiments show that Fama-SDIP method significantly outperforms prevailing leading methods on simulation and real HSI datasets.
An Unsupervised Deep-Learning Method for Bone Age Assessment
Authors: Hao Zhu, Wan-Jing Nie, Yue-Jie Hou, Qi-Meng Du, Si-Jing Li, Chi-Chun Zhou
Date: 2022-06-12 00:00:00
Tasks: Deep Learning

The bone age, reflecting the degree of development of the bones, can be used to predict the adult height and detect endocrine diseases of children. Both examinations of radiologists and variability of operators have a significant impact on bone age assessment. To decrease human intervention , machine learning algorithms are used to assess the bone age automatically. However, conventional supervised deep-learning methods need pre-labeled data. In this paper, based on the convolutional auto-encoder with constraints (CCAE), an unsupervised deep-learning model proposed in the classification of the fingerprint, we propose this model for the classification of the bone age and baptize it BA-CCAE. In the proposed BA-CCAE model, the key regions of the raw X-ray images of the bone age are encoded, yielding the latent vectors. The K-means clustering algorithm is used to obtain the final classifications by grouping the latent vectors of the bone images. A set of experiments on the Radiological Society of North America pediatric bone age dataset (RSNA) show that the accuracy of classifications at 48-month intervals is 76.15%. Although the accuracy now is lower than most of the existing supervised models, the proposed BA-CCAE model can establish the classification of bone age without any pre-labeled data, and to the best of our knowledge, the proposed BA-CCAE is one of the few trails using the unsupervised deep-learning method for the bone age assessment.
Deep Learning Models for Automated Classification of Dog Emotional States from Facial Expressions
Authors: Tali Boneh-Shitrit, Shir Amir, Annika Bremhorst, Daniel S. Mills, Stefanie Riemer, Dror Fried, Anna Zamansky
Date: 2022-06-11 00:00:00

Similarly to humans, facial expressions in animals are closely linked with emotional states. However, in contrast to the human domain, automated recognition of emotional states from facial expressions in animals is underexplored, mainly due to difficulties in data collection and establishment of ground truth concerning emotional states of non-verbal users. We apply recent deep learning techniques to classify (positive) anticipation and (negative) frustration of dogs on a dataset collected in a controlled experimental setting. We explore the suitability of different backbones (e.g. ResNet, ViT) under different supervisions to this task, and find that features of a self-supervised pretrained ViT (DINO-ViT) are superior to the other alternatives. To the best of our knowledge, this work is the first to address the task of automatic classification of canine emotions on data acquired in a controlled experiment.
Synthetic PET via Domain Translation of 3D MRI
Authors: Abhejit Rajagopal, Yutaka Natsuaki, Kristen Wangerin, Mahdjoub Hamdi, Hongyu An, John J. Sunderland, Richard Laforest, Paul E. Kinahan, Peder E. Z. Larson, Thomas A. Hope
Date: 2022-06-11 00:00:00
Tasks: MRI Reconstruction, Translation

Historically, patient datasets have been used to develop and validate various reconstruction algorithms for PET/MRI and PET/CT. To enable such algorithm development, without the need for acquiring hundreds of patient exams, in this paper we demonstrate a deep learning technique to generate synthetic but realistic whole-body PET sinograms from abundantly-available whole-body MRI. Specifically, we use a dataset of 56 $^{18}$F-FDG-PET/MRI exams to train a 3D residual UNet to predict physiologic PET uptake from whole-body T1-weighted MRI. In training we implemented a balanced loss function to generate realistic uptake across a large dynamic range and computed losses along tomographic lines of response to mimic the PET acquisition. The predicted PET images are forward projected to produce synthetic PET time-of-flight (ToF) sinograms that can be used with vendor-provided PET reconstruction algorithms, including using CT-based attenuation correction (CTAC) and MR-based attenuation correction (MRAC). The resulting synthetic data recapitulates physiologic $^{18}$F-FDG uptake, e.g. high uptake localized to the brain and bladder, as well as uptake in liver, kidneys, heart and muscle. To simulate abnormalities with high uptake, we also insert synthetic lesions. We demonstrate that this synthetic PET data can be used interchangeably with real PET data for the PET quantification task of comparing CT and MR-based attenuation correction methods, achieving $\leq 7.6\%$ error in mean-SUV compared to using real data. These results together show that the proposed synthetic PET data pipeline can be reasonably used for development, evaluation, and validation of PET/MRI reconstruction methods.
Federated Learning with Research Prototypes for Multi-Center MRI-based Detection of Prostate Cancer with Diverse Histopathology
Authors: Abhejit Rajagopal, Ekaterina Redekop, Anil Kemisetti, Rushi Kulkarni, Steven Raman, Kirti Magudia, Corey W. Arnold, Peder E. Z. Larson
Date: 2022-06-11 00:00:00
Tasks: Deep Learning, Federated Learning

Early prostate cancer detection and staging from MRI are extremely challenging tasks for both radiologists and deep learning algorithms, but the potential to learn from large and diverse datasets remains a promising avenue to increase their generalization capability both within- and across clinics. To enable this for prototype-stage algorithms, where the majority of existing research remains, in this paper we introduce a flexible federated learning framework for cross-site training, validation, and evaluation of deep prostate cancer detection algorithms. Our approach utilizes an abstracted representation of the model architecture and data, which allows unpolished prototype deep learning models to be trained without modification using the NVFlare federated learning framework. Our results show increases in prostate cancer detection and classification accuracy using a specialized neural network model and diverse prostate biopsy data collected at two University of California research hospitals, demonstrating the efficacy of our approach in adapting to different datasets and improving MR-biomarker discovery. We open-source our FLtools system, which can be easily adapted to other deep learning projects for medical imaging.
Machine learning approaches for COVID-19 detection from chest X-ray imaging: A Systematic Review
Authors: Harold Brayan Arteaga-Arteaga, Melissa delaPava, Alejandro Mora-Rubio, Mario Alejandro Bravo-Ortíz, Jesus Alejandro Alzate-Grisales, Daniel Arias-Garzón, Luis Humberto López-Murillo, Felipe Buitrago-Carmona, Juan Pablo Villa-Pulgarín, Esteban Mercado-Ruiz, Simon Orozco-Arias, M. Hassaballah, Maria de la Iglesia-Vaya, Oscar Cardona-Morales, Reinel Tabares-Soto
Date: 2022-06-11 00:00:00
Tasks: Articles, Decision Making, Diagnostic

There is a necessity to develop affordable, and reliable diagnostic tools, which allow containing the COVID-19 spreading. Machine Learning (ML) algorithms have been proposed to design support decision-making systems to assess chest X-ray images, which have proven to be useful to detect and evaluate disease progression. Many research articles are published around this subject, which makes it difficult to identify the best approaches for future work. This paper presents a systematic review of ML applied to COVID-19 detection using chest X-ray images, aiming to offer a baseline for researchers in terms of methods, architectures, databases, and current limitations.
MammoFL: Mammographic Breast Density Estimation using Federated Learning
Authors: Ramya Muthukrishnan, Angelina Heyler, Keshava Katti, Sarthak Pati, Walter Mankowski, Aprupa Alahari, Michael Sanborn, Emily F. Conant, Christopher Scott, Stacey Winham, Celine Vachon, Pratik Chaudhari, Despina Kontos, Spyridon Bakas
Date: 2022-06-11 00:00:00
Tasks: Density Estimation, Federated Learning

In this study, we automate quantitative mammographic breast density estimation with neural networks and show that this tool is a strong use case for federated learning on multi-institutional datasets. Our dataset included bilateral CC-view and MLO-view mammographic images from two separate institutions. Two U-Nets were separately trained on algorithm-generated labels to perform segmentation of the breast and dense tissue from these images and subsequently calculate breast percent density (PD). The networks were trained with federated learning and compared to three non-federated baselines, one trained on each single-institution dataset and one trained on the aggregated multi-institution dataset. We demonstrate that training on multi-institution datasets is critical to algorithm generalizability. We further show that federated learning on multi-institutional datasets improves model generalization to unseen data at nearly the same level as centralized training on multi-institutional datasets, indicating that federated learning can be applied to our method to improve algorithm generalizability while maintaining patient privacy.
A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training
Authors: Zhihao Fan, Zhongyu Wei, Jingjing Chen, Siyuan Wang, Zejun Li, Jiarong Xu, Xuanjing Huang
Date: 2022-06-11 00:00:00

Multi-modal pre-training and knowledge discovery are two important research topics in multi-modal machine learning. Nevertheless, none of existing works make attempts to link knowledge discovery with knowledge guided multi-modal pre-training. In this paper, we propose to unify them into a continuous learning framework for mutual improvement. Taking the open-domain uni-modal datasets of images and texts as input, we maintain a knowledge graph as the foundation to support these two tasks. For knowledge discovery, a pre-trained model is used to identify cross-modal links on the graph. For model pre-training, the knowledge graph is used as the external knowledge to guide the model updating. These two steps are iteratively performed in our framework for continuous learning. The experimental results on MS-COCO and Flickr30K with respect to both knowledge discovery and the pre-trained model validate the effectiveness of our framework.
Surround-View Cameras based Holistic Visual Perception for Automated Driving
Authors: Varun Ravi Kumar
Date: 2022-06-11 00:00:00
Tasks: Decision Making, Multi-Task Learning, Self-Driving Cars

The formation of eyes led to the big bang of evolution. The dynamics changed from a primitive organism waiting for the food to come into contact for eating food being sought after by visual sensors. The human eye is one of the most sophisticated developments of evolution, but it still has defects. Humans have evolved a biological perception algorithm capable of driving cars, operating machinery, piloting aircraft, and navigating ships over millions of years. Automating these capabilities for computers is critical for various applications, including self-driving cars, augmented reality, and architectural surveying. Near-field visual perception in the context of self-driving cars can perceive the environment in a range of $0-10$ meters and 360{\deg} coverage around the vehicle. It is a critical decision-making component in the development of safer automated driving. Recent advances in computer vision and deep learning, in conjunction with high-quality sensors such as cameras and LiDARs, have fueled mature visual perception solutions. Until now, far-field perception has been the primary focus. Another significant issue is the limited processing power available for developing real-time applications. Because of this bottleneck, there is frequently a trade-off between performance and run-time efficiency. We concentrate on the following issues in order to address them: 1) Developing near-field perception algorithms with high performance and low computational complexity for various visual perception tasks such as geometric and semantic tasks using convolutional neural networks. 2) Using Multi-Task Learning to overcome computational bottlenecks by sharing initial convolutional layers between tasks and developing optimization strategies that balance tasks.
A Simplified Un-Supervised Learning Based Approach for Ink Mismatch Detection in Handwritten Hyper-Spectral Document Images
Authors: Muhammad Farhan Humayun, Hassan Waseem Malik, Ahmed Ahsan Alvi
Date: 2022-06-11 00:00:00

Hyper-spectral imaging has become the latest trend in the field of optical imaging systems. Among various other applications, hyper-spectral imaging has been widely used for analysis of printed and handwritten documents. This paper proposes an efficient technique for estimating the number of different but visibly similar inks present in a Hyper spectral Document Image. Our approach is based on un-supervised learning and does not require any prior knowledge of the dataset. The algorithm was tested on the iVision HHID dataset and has achieved comparable results with the state of the algorithms present in the literature. This work can prove to be effective when employed during the early stages of forgery detection in Hyper-spectral Document Images.
A Two-stage Method for Non-extreme Value Salt-and-Pepper Noise Removal
Authors: Renwei Yang, Yike Liu, Bing Zeng
Date: 2022-06-11 00:00:00
Tasks: Denoising, Salt-And-Pepper Noise Removal

There are several previous methods based on neural network can have great performance in denoising salt and pepper noise. However, those methods are based on a hypothesis that the value of salt and pepper noise is exactly 0 and 255. It is not true in the real world. The result of those methods deviate sharply when the value is different from 0 and 255. To overcome this weakness, our method aims at designing a convolutional neural network to detect the noise pixels in a wider range of value and then a filter is used to modify pixel value to 0, which is beneficial for further filtering. Additionally, another convolutional neural network is used to conduct the denoising and restoration work.