pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning-based Occluded Person Re-identification: A Survey
Authors: Yunjie Peng, Saihui Hou, Chunshui Cao, Xu Liu, Yongzhen Huang, Zhiqiang He
Date: 2022-07-29 00:00:00
Tasks: Deep Learning, Occluded Person Re-Identification, Person Re-Identification, Survey

Occluded person re-identification (Re-ID) aims at addressing the occlusion problem when retrieving the person of interest across multiple cameras. With the promotion of deep learning technology and the increasing demand for intelligent video surveillance, the frequent occlusion in real-world applications has made occluded person Re-ID draw considerable interest from researchers. A large number of occluded person Re-ID methods have been proposed while there are few surveys that focus on occlusion. To fill this gap and help boost future research, this paper provides a systematic survey of occluded person Re-ID. Through an in-depth analysis of the occlusion in person Re-ID, most existing methods are found to only consider part of the problems brought by occlusion. Therefore, we review occlusion-related person Re-ID methods from the perspective of issues and solutions. We summarize four issues caused by occlusion in person Re-ID, i.e., position misalignment, scale misalignment, noisy information, and missing information. The occlusion-related methods addressing different issues are then categorized and introduced accordingly. After that, we summarize and compare the performance of recent occluded person Re-ID methods on four popular datasets: Partial-ReID, Partial-iLIDS, Occluded-ReID, and Occluded-DukeMTMC. Finally, we provide insights on promising future research directions.
PC-GANs: Progressive Compensation Generative Adversarial Networks for Pan-sharpening
Authors: Yinghui Xing, Shuyuan Yang, Song Wang, Yan Zhang, Yanning Zhang
Date: 2022-07-29 00:00:00
Tasks: Generative Adversarial Network, Pansharpening

The fusion of multispectral and panchromatic images is always dubbed pansharpening. Most of the available deep learning-based pan-sharpening methods sharpen the multispectral images through a one-step scheme, which strongly depends on the reconstruction ability of the network. However, remote sensing images always have large variations, as a result, these one-step methods are vulnerable to the error accumulation and thus incapable of preserving spatial details as well as the spectral information. In this paper, we propose a novel two-step model for pan-sharpening that sharpens the MS image through the progressive compensation of the spatial and spectral information. Firstly, a deep multiscale guided generative adversarial network is used to preliminarily enhance the spatial resolution of the MS image. Starting from the pre-sharpened MS image in the coarse domain, our approach then progressively refines the spatial and spectral residuals over a couple of generative adversarial networks (GANs) that have reverse architectures. The whole model is composed of triple GANs, and based on the specific architecture, a joint compensation loss function is designed to enable the triple GANs to be trained simultaneously. Moreover, the spatial-spectral residual compensation structure proposed in this paper can be extended to other pan-sharpening methods to further enhance their fusion results. Extensive experiments are performed on different datasets and the results demonstrate the effectiveness and efficiency of our proposed method.
Dataset and Evaluation algorithm design for GOALS Challenge
Authors: Huihui Fang, Fei Li, Huazhu Fu, Junde Wu, Xiulan Zhang, Yanwu Xu
Date: 2022-07-29 00:00:00
Tasks: Segmentation

Glaucoma causes irreversible vision loss due to damage to the optic nerve, and there is no cure for glaucoma.OCT imaging modality is an essential technique for assessing glaucomatous damage since it aids in quantifying fundus structures. To promote the research of AI technology in the field of OCT-assisted diagnosis of glaucoma, we held a Glaucoma OCT Analysis and Layer Segmentation (GOALS) Challenge in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2022 to provide data and corresponding annotations for researchers studying layer segmentation from OCT images and the classification of glaucoma. This paper describes the released 300 circumpapillary OCT images, the baselines of the two sub-tasks, and the evaluation methodology. The GOALS Challenge is accessible at https://aistudio.baidu.com/aistudio/competition/detail/230.
Graph-Based Small Bowel Path Tracking with Cylindrical Constraints
Authors: Seung Yeon Shin, SungWon Lee, Ronald M. Summers
Date: 2022-07-29 00:00:00

We present a new graph-based method for small bowel path tracking based on cylindrical constraints. A distinctive characteristic of the small bowel compared to other organs is the contact between parts of itself along its course, which makes the path tracking difficult together with the indistinct appearance of the wall. It causes the tracked path to easily cross over the walls when relying on low-level features like the wall detection. To circumvent this, a series of cylinders that are fitted along the course of the small bowel are used to guide the tracking to more reliable directions. It is implemented as soft constraints using a new cost function. The proposed method is evaluated against ground-truth paths that are all connected from start to end of the small bowel for 10 abdominal CT scans. The proposed method showed clear improvements compared to the baseline method in tracking the path without making an error. Improvements of 6.6% and 17.0%, in terms of the tracked length, were observed for two different settings related to the small bowel segmentation.
Paired Cross-Modal Data Augmentation for Fine-Grained Image-to-Text Retrieval
Authors: Hao Wang, Guosheng Lin, Steven C. H. Hoi, Chunyan Miao
Date: 2022-07-29 00:00:00
Tasks: Cross-Modal Retrieval, Data Augmentation, Image to text, Image-to-Text Retrieval, Retrieval, Text Retrieval

This paper investigates an open research problem of generating text-image pairs to improve the training of fine-grained image-to-text cross-modal retrieval task, and proposes a novel framework for paired data augmentation by uncovering the hidden semantic information of StyleGAN2 model. Specifically, we first train a StyleGAN2 model on the given dataset. We then project the real images back to the latent space of StyleGAN2 to obtain the latent codes. To make the generated images manipulatable, we further introduce a latent space alignment module to learn the alignment between StyleGAN2 latent codes and the corresponding textual caption features. When we do online paired data augmentation, we first generate augmented text through random token replacement, then pass the augmented text into the latent space alignment module to output the latent codes, which are finally fed to StyleGAN2 to generate the augmented images. We evaluate the efficacy of our augmented data approach on two public cross-modal retrieval datasets, in which the promising experimental results demonstrate the augmented text-image pair data can be trained together with the original data to boost the image-to-text cross-modal retrieval performance.
3D Cartoon Face Generation with Controllable Expressions from a Single GAN Image
Authors: Hao Wang, Wenhao Shen, Guosheng Lin, Steven C. H. Hoi, Chunyan Miao
Date: 2022-07-29 00:00:00
Tasks: Face Generation, Face Model

In this paper, we investigate an open research task of generating 3D cartoon face shapes from single 2D GAN generated human faces and without 3D supervision, where we can also manipulate the facial expressions of the 3D shapes. To this end, we discover the semantic meanings of StyleGAN latent space, such that we are able to produce face images of various expressions, poses, and lighting conditions by controlling the latent codes. Specifically, we first finetune the pretrained StyleGAN face model on the cartoon datasets. By feeding the same latent codes to face and cartoon generation models, we aim to realize the translation from 2D human face images to cartoon styled avatars. We then discover semantic directions of the GAN latent space, in an attempt to change the facial expressions while preserving the original identity. As we do not have any 3D annotations for cartoon faces, we manipulate the latent codes to generate images with different poses and lighting conditions, such that we can reconstruct the 3D cartoon face shapes. We validate the efficacy of our method on three cartoon datasets qualitatively and quantitatively.
Deep learning for understanding multilabel imbalanced Chest X-ray datasets
Authors: Helena Liz, Javier Huertas-Tato, Manuel Sánchez-Montañés, Javier Del Ser, David Camacho
Date: 2022-07-28 00:00:00
Tasks: Deep Learning

Over the last few years, convolutional neural networks (CNNs) have dominated the field of computer vision thanks to their ability to extract features and their outstanding performance in classification problems, for example in the automatic analysis of X-rays. Unfortunately, these neural networks are considered black-box algorithms, i.e. it is impossible to understand how the algorithm has achieved the final result. To apply these algorithms in different fields and test how the methodology works, we need to use eXplainable AI techniques. Most of the work in the medical field focuses on binary or multiclass classification problems. However, in many real-life situations, such as chest X-rays, radiological signs of different diseases can appear at the same time. This gives rise to what is known as "multilabel classification problems". A disadvantage of these tasks is class imbalance, i.e. different labels do not have the same number of samples. The main contribution of this paper is a Deep Learning methodology for imbalanced, multilabel chest X-ray datasets. It establishes a baseline for the currently underutilised PadChest dataset and a new eXplainable AI technique based on heatmaps. This technique also includes probabilities and inter-model matching. The results of our system are promising, especially considering the number of labels used. Furthermore, the heatmaps match the expected areas, i.e. they mark the areas that an expert would use to make the decision.
Low Cost Embedded Vision System For Location And Tracking Of A Color Object
Authors: Diego Ayala, Danilo Chavez, Leopoldo Altamirano Robles
Date: 2022-07-28 00:00:00
Tasks: Object

This paper describes the development of an embedded vision system for detection, location, and tracking of a color object; it makes use of a single 32-bit microprocessor to acquire image data, process, and perform actions according to the interpreted data. The system is intended for applications that need to make use of artificial vision for detection, location and tracking of a color object and its objective is to have achieve at reduced terms of size, power consumption, and cost.
A Deep Generative Approach to Oversampling in Ptychography
Authors: Semih Barutcu, Aggelos K. Katsaggelos, Doğa Gürsoy
Date: 2022-07-28 00:00:00

Ptychography is a well-studied phase imaging method that makes non-invasive imaging possible at a nanometer scale. It has developed into a mainstream technique with various applications across a range of areas such as material science or the defense industry. One major drawback of ptychography is the long data acquisition time due to the high overlap requirement between adjacent illumination areas to achieve a reasonable reconstruction. Traditional approaches with reduced overlap between scanning areas result in reconstructions with artifacts. In this paper, we propose complementing sparsely acquired or undersampled data with data sampled from a deep generative network to satisfy the oversampling requirement in ptychography. Because the deep generative network is pre-trained and its output can be computed as we collect data, the experimental data and the time to acquire the data can be reduced. We validate the method by presenting the reconstruction quality compared to the previously proposed and traditional approaches and comment on the strengths and drawbacks of the proposed approach.
Pro-tuning: Unified Prompt Tuning for Vision Tasks
Authors: Xing Nie, Bolin Ni, Jianlong Chang, Gaomeng Meng, Chunlei Huo, Zhaoxiang Zhang, Shiming Xiang, Qi Tian, Chunhong Pan
Date: 2022-07-28 00:00:00
Tasks: Adversarial Robustness, image-classification, Image Classification, object-detection, Object Detection, Out-of-Distribution Generalization, Semantic Segmentation

In computer vision, fine-tuning is the de-facto approach to leverage pre-trained vision models to perform downstream tasks. However, deploying it in practice is quite challenging, due to adopting parameter inefficient global update and heavily relying on high-quality downstream data. Recently, prompt-based learning, which adds a task-relevant prompt to adapt the downstream tasks to pre-trained models, has drastically boosted the performance of many natural language downstream tasks. In this work, we extend this notable transfer ability benefited from prompt into vision models as an alternative to fine-tuning. To this end, we propose parameter-efficient Prompt tuning (Pro-tuning) to adapt frozen vision models to various downstream vision tasks. The key to Pro-tuning is prompt-based tuning, i.e., learning task-specific vision prompts for downstream input images with the pre-trained model frozen. By only training a few additional parameters, it can work on diverse CNN-based and Transformer-based architectures. Extensive experiments evidence that Pro-tuning outperforms fine-tuning in a broad range of vision tasks and scenarios, including image classification (generic objects, class imbalance, image corruption, adversarial robustness, and out-of-distribution generalization), and dense prediction tasks such as object detection and semantic segmentation.
Eye Gaze Estimation Model Analysis
Authors: Aveena Kottwani, Ayush Kumar
Date: 2022-07-28 00:00:00
Tasks: Gaze Estimation, model

We explore techniques for eye gaze estimation using machine learning. Eye gaze estimation is a common problem for various behavior analysis and human-computer interfaces. The purpose of this work is to discuss various model types for eye gaze estimation and present the results from predicting gaze direction using eye landmarks in unconstrained settings. In unconstrained real-world settings, feature-based and model-based methods are outperformed by recent appearance-based methods due to factors like illumination changes and other visual artifacts. We discuss a learning-based method for eye region landmark localization trained exclusively on synthetic data. We discuss how to use detected landmarks as input to iterative model-fitting and lightweight learning-based gaze estimation methods and how to use the model for person-independent and personalized gaze estimations.
Training a universal instance segmentation network for live cell images of various cell types and imaging modalities
Authors: Tianqi Guo, Yin Wang, Luis Solorio, Jan P. Allebach
Date: 2022-07-28 00:00:00
Tasks: Cell Tracking, Instance Segmentation, Segmentation, Semantic Segmentation, Universal Segmentation

We share our recent findings in an attempt to train a universal segmentation network for various cell types and imaging modalities. Our method was built on the generalized U-Net architecture, which allows the evaluation of each component individually. We modified the traditional binary training targets to include three classes for direct instance segmentation. Detailed experiments were performed regarding training schemes, training settings, network backbones, and individual modules on the segmentation performance. Our proposed training scheme draws minibatches in turn from each dataset, and the gradients are accumulated before an optimization step. We found that the key to training a universal network is all-time supervision on all datasets, and it is necessary to sample each dataset in an unbiased way. Our experiments also suggest that there might exist common features to define cell boundaries across cell types and imaging modalities, which could allow application of trained models to totally unseen datasets. A few training tricks can further boost the segmentation performance, including uneven class weights in the cross-entropy loss function, well-designed learning rate scheduler, larger image crops for contextual information, and additional loss terms for unbalanced classes. We also found that segmentation performance can benefit from group normalization layer and Atrous Spatial Pyramid Pooling module, thanks to their more reliable statistics estimation and improved semantic understanding, respectively. We participated in the 6th Cell Tracking Challenge (CTC) held at IEEE International Symposium on Biomedical Imaging (ISBI) 2021 using one of the developed variants. Our method was evaluated as the best runner up during the initial submission for the primary track, and also secured the 3rd place in an additional round of competition in preparation for the summary publication.
Aztec curve: proposal for a new space-filling curve
Authors: Diego Ayala, Daniel Durini, Jose Rangel-Magdaleno
Date: 2022-07-28 00:00:00
Tasks: compressed sensing

Different space-filling curves (SFCs) are briefly reviewed in this paper, and a new one is proposed. A century has passed between the inception of this kind of curves, since then they have been found useful in computer science, particularly in data storage and indexing due to their clustering properties, being Hilbert curve the most well-known member of the family of fractals. The proposed Aztec curve, with similar characteristics to the Hilbert's curve, is introduced in this paper, accompanied by a grammatical description for its construction. It yields the possibility of creating bi-dimensional clusters, not available for Hilbert nor Peano curves. Additional to this, a case of application on the scope of Compressed Sensing is implemented, in which the use of Hilbert curve is contrasted with Aztec curve, having a similar performance, and positioning the Aztec curve as viable and a new alternative for future exploitation on applications that make use of SFC's.
SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation
Authors: Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, Onkar Dabeer
Date: 2022-07-28 00:00:00
Tasks: Anomaly Detection, Anomaly Segmentation, Self-Supervised Learning

Visual anomaly detection is commonly used in industrial quality inspection. In this paper, we present a new dataset as well as a new self-supervised learning method for ImageNet pre-training to improve anomaly detection and segmentation in 1-class and 2-class 5/10/high-shot training setups. We release the Visual Anomaly (VisA) Dataset consisting of 10,821 high-resolution color images (9,621 normal and 1,200 anomalous samples) covering 12 objects in 3 domains, making it the largest industrial anomaly detection dataset to date. Both image and pixel-level labels are provided. We also propose a new self-supervised framework - SPot-the-difference (SPD) - which can regularize contrastive self-supervised pre-training, such as SimSiam, MoCo and SimCLR, to be more suitable for anomaly detection tasks. Our experiments on VisA and MVTec-AD dataset show that SPD consistently improves these contrastive pre-training baselines and even the supervised pre-training. For example, SPD improves Area Under the Precision-Recall curve (AU-PR) for anomaly segmentation by 5.9% and 6.8% over SimSiam and supervised pre-training respectively in the 2-class high-shot regime. We open-source the project at http://github.com/amazon-research/spot-diff .
Graph Inverse Reinforcement Learning from Diverse Videos
Authors: Sateesh Kumar, Jonathan Zamora, Nicklas Hansen, Rishabh Jangir, Xiaolong Wang
Date: 2022-07-28 00:00:00
Tasks: Diversity, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Robot Manipulation

Research on Inverse Reinforcement Learning (IRL) from third-person videos has shown encouraging results on removing the need for manual reward design for robotic tasks. However, most prior works are still limited by training from a relatively restricted domain of videos. In this paper, we argue that the true potential of third-person IRL lies in increasing the diversity of videos for better scaling. To learn a reward function from diverse videos, we propose to perform graph abstraction on the videos followed by temporal matching in the graph space to measure the task progress. Our insight is that a task can be described by entity interactions that form a graph, and this graph abstraction can help remove irrelevant information such as textures, resulting in more robust reward functions. We evaluate our approach, GraphIRL, on cross-embodiment learning in X-MAGICAL and learning from human demonstrations for real-robot manipulation. We show significant improvements in robustness to diverse video demonstrations over previous approaches, and even achieve better results than manual reward design on a real robot pushing task. Videos are available at https://sateeshkumar21.github.io/GraphIRL .
Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learning
Authors: Shuang Qiu, Lingxiao Wang, Chenjia Bai, Zhuoran Yang, Zhaoran Wang
Date: 2022-07-29 00:00:00
Tasks: Contrastive Learning, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL), Representation Learning, Self-Supervised Learning

In view of its power in extracting feature representation, contrastive self-supervised learning has been successfully integrated into the practice of (deep) reinforcement learning (RL), leading to efficient policy learning in various applications. Despite its tremendous empirical successes, the understanding of contrastive learning for RL remains elusive. To narrow such a gap, we study how RL can be empowered by contrastive learning in a class of Markov decision processes (MDPs) and Markov games (MGs) with low-rank transitions. For both models, we propose to extract the correct feature representations of the low-rank model by minimizing a contrastive loss. Moreover, under the online setting, we propose novel upper confidence bound (UCB)-type algorithms that incorporate such a contrastive loss with online RL algorithms for MDPs or MGs. We further theoretically prove that our algorithm recovers the true representations and simultaneously achieves sample efficiency in learning the optimal policy and Nash equilibrium in MDPs and MGs. We also provide empirical studies to demonstrate the efficacy of the UCB-based contrastive learning method for RL. To the best of our knowledge, we provide the first provably efficient online RL algorithm that incorporates contrastive learning for representation learning. Our codes are available at https://github.com/Baichenjia/Contrastive-UCB.
Encoder-Decoder Architecture for 3D Seismic Inversion
Authors: Maayan Gelboim, Amir Adler, Yen Sun, Mauricio Araya-Polo
Date: 2022-07-29 00:00:00
Tasks: Decoder, Seismic Inversion, SSIM

Inverting seismic data to build 3D geological structures is a challenging task due to the overwhelming amount of acquired seismic data, and the very-high computational load due to iterative numerical solutions of the wave equation, as required by industry-standard tools such as Full Waveform Inversion (FWI). For example, in an area with surface dimensions of 4.5km $\times$ 4.5km, hundreds of seismic shot-gather cubes are required for 3D model reconstruction, leading to Terabytes of recorded data. This paper presents a deep learning solution for the reconstruction of realistic 3D models in the presence of field noise recorded in seismic surveys. We implement and analyze a convolutional encoder-decoder architecture that efficiently processes the entire collection of hundreds of seismic shot-gather cubes. The proposed solution demonstrates that realistic 3D models can be reconstructed with a structural similarity index measure (SSIM) of 0.8554 (out of 1.0) in the presence of field noise at 10dB signal-to-noise ratio.