pwc-archive/papers-with-abstracts huggingface dataset:
Cut-and-Approximate: 3D Shape Reconstruction from Planar Cross-sections with Deep Reinforcement Learning
Authors: Azimkhon Ostonov
Date: 2022-10-22 00:00:00
Tasks: 3D Object Reconstruction, 3D Shape Reconstruction, Deep Reinforcement Learning, Imitation Learning, Object Reconstruction, Surface Reconstruction

Current methods for 3D object reconstruction from a set of planar cross-sections still struggle to capture detailed topology or require a considerable number of cross-sections. In this paper, we present, to the best of our knowledge the first 3D shape reconstruction network to solve this task which additionally uses orthographic projections of the shape. Our method is based on applying a Reinforcement Learning algorithm to learn how to effectively parse the shape using a trial-and-error scheme relying on scalar rewards. This method cuts a part of a 3D shape in each step which is then approximated as a polygon mesh. The agent aims to maximize the reward that depends on the accuracy of surface reconstruction for the approximated parts. We also consider pre-training of the network for faster learning using demonstrations generated by a heuristic approach. Experiments show that our training algorithm which benefits from both imitation learning and also self exploration, learns efficient policies faster, which results the agent to produce visually compelling results.
A Flexible-Frame-Rate Vision-Aided Inertial Object Tracking System for Mobile Devices
Authors: Yo-Chung Lau, Kuan-Wei Tseng, I-Ju Hsieh, Hsiao-Ching Tseng, Yi-Ping Hung
Date: 2022-10-22 00:00:00
Tasks: 3D Pose Estimation, Object, Object Tracking, Pose Estimation

Real-time object pose estimation and tracking is challenging but essential for emerging augmented reality (AR) applications. In general, state-of-the-art methods address this problem using deep neural networks which indeed yield satisfactory results. Nevertheless, the high computational cost of these methods makes them unsuitable for mobile devices where real-world applications usually take place. In addition, head-mounted displays such as AR glasses require at least 90~FPS to avoid motion sickness, which further complicates the problem. We propose a flexible-frame-rate object pose estimation and tracking system for mobile devices. It is a monocular visual-inertial-based system with a client-server architecture. Inertial measurement unit (IMU) pose propagation is performed on the client side for high speed tracking, and RGB image-based 3D pose estimation is performed on the server side to obtain accurate poses, after which the pose is sent to the client side for visual-inertial fusion, where we propose a bias self-correction mechanism to reduce drift. We also propose a pose inspection algorithm to detect tracking failures and incorrect pose estimation. Connected by high-speed networking, our system supports flexible frame rates up to 120 FPS and guarantees high precision and real-time tracking on low-end devices. Both simulations and real world experiments show that our method achieves accurate and robust object tracking.
SLAMs: Semantic Learning based Activation Map for Weakly Supervised Semantic Segmentation
Authors: Junliang Chen, Xiaodong Zhao, Minmin Liu, Linlin Shen
Date: 2022-10-22 00:00:00
Tasks: Segmentation, Semantic Segmentation, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

Recent mainstream weakly-supervised semantic segmentation (WSSS) approaches mainly relies on image-level classification learning, which has limited representation capacity. In this paper, we propose a novel semantic learning based framework, named SLAMs (Semantic Learning based Activation Map), for WSSS.
Neural Distortion Fields for Spatial Calibration of Wide Field-of-View Near-Eye Displays
Authors: Yuichi Hiroi, Kiyosato Someya, Yuta Itoh
Date: 2022-10-22 00:00:00

We propose a spatial calibration method for wide Field-of-View (FoV) Near-Eye Displays (NEDs) with complex image distortions. Image distortions in NEDs can destroy the reality of the virtual object and cause sickness. To achieve distortion-free images in NEDs, it is necessary to establish a pixel-by-pixel correspondence between the viewpoint and the displayed image. Designing compact and wide-FoV NEDs requires complex optical designs. In such designs, the displayed images are subject to gaze-contingent, non-linear geometric distortions, which explicit geometric models can be difficult to represent or computationally intensive to optimize. To solve these problems, we propose Neural Distortion Field (NDF), a fully-connected deep neural network that implicitly represents display surfaces complexly distorted in spaces. NDF takes spatial position and gaze direction as input and outputs the display pixel coordinate and its intensity as perceived in the input gaze direction. We synthesize the distortion map from a novel viewpoint by querying points on the ray from the viewpoint and computing a weighted sum to project output display coordinates into an image. Experiments showed that NDF calibrates an augmented reality NED with 90$^{\circ}$ FoV with about 3.23 pixel (5.8 arcmin) median error using only 8 training viewpoints. Additionally, we confirmed that NDF calibrates more accurately than the non-linear polynomial fitting, especially around the center of the FoV.
Diversity-Promoting Ensemble for Medical Image Segmentation
Authors: Mariana-Iuliana Georgescu, Radu Tudor Ionescu, Andreea-Iuliana Miron
Date: 2022-10-22 00:00:00
Tasks: Diversity, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Medical image segmentation is an actively studied task in medical imaging, where the precision of the annotations is of utter importance towards accurate diagnosis and treatment. In recent years, the task has been approached with various deep learning systems, among the most popular models being U-Net. In this work, we propose a novel strategy to generate ensembles of different architectures for medical image segmentation, by leveraging the diversity (decorrelation) of the models forming the ensemble. More specifically, we utilize the Dice score among model pairs to estimate the correlation between the outputs of the two models forming each pair. To promote diversity, we select models with low Dice scores among each other. We carry out gastro-intestinal tract image segmentation experiments to compare our diversity-promoting ensemble (DiPE) with another strategy to create ensembles based on selecting the top scoring U-Net models. Our empirical results show that DiPE surpasses both individual models as well as the ensemble creation strategy based on selecting the top scoring models.
S2WAT: Image Style Transfer via Hierarchical Vision Transformer using Strips Window Attention
Authors: Chiyu Zhang, Xiaogang Xu, Lei Wang, Zaiyan Dai, Jun Yang
Date: 2022-10-22 00:00:00
Tasks: Style Transfer

Transformer's recent integration into style transfer leverages its proficiency in establishing long-range dependencies, albeit at the expense of attenuated local modeling. This paper introduces Strips Window Attention Transformer (S2WAT), a novel hierarchical vision transformer designed for style transfer. S2WAT employs attention computation in diverse window shapes to capture both short- and long-range dependencies. The merged dependencies utilize the "Attn Merge" strategy, which adaptively determines spatial weights based on their relevance to the target. Extensive experiments on representative datasets show the proposed method's effectiveness compared to state-of-the-art (SOTA) transformer-based and other approaches. The code and pre-trained models are available at https://github.com/AlienZhang1996/S2WAT.
MS-DCANet: A Novel Segmentation Network For Multi-Modality COVID-19 Medical Images
Authors: Xiaoyu Pan, Huazheng Zhu, Jinglong Du, Guangtao Hu, Baoru Han, Yuanyuan Jia
Date: 2022-10-22 00:00:00
Tasks: Image Segmentation, Learning Theory, Medical Image Segmentation, Segmentation, Semantic Segmentation

The Coronavirus Disease 2019 (COVID-19) pandemic has increased the public health burden and brought profound disaster to humans. For the particularity of the COVID-19 medical images with blurred boundaries, low contrast and different infection sites, some researchers have improved the accuracy by adding more complexity. Also, they overlook the complexity of lesions, which hinder their ability to capture the relationship between segmentation sites and the background, as well as the edge contours and global context. However, increasing the computational complexity, parameters and inference speed is unfavorable for model transfer from laboratory to clinic. A perfect segmentation network needs to balance the above three factors completely. To solve the above issues, this paper propose a symmetric automatic segmentation framework named MS-DCANet. We introduce Tokenized MLP block, a novel attention scheme that use a shift-window mechanism to conditionally fuse local and global features to get more continuous boundaries and spatial positioning capabilities. It has greater understanding of irregular lesions contours. MS-DCANet also uses several Dual Channel blocks and a Res-ASPP block to improve the ability to recognize small targets. On multi-modality COVID-19 tasks, MS-DCANet achieved state-of-the-art performance compared with other baselines. It can well trade off the accuracy and complexity. To prove the strong generalization ability of our proposed model, we apply it to other tasks (ISIC 2018 and BAA) and achieve satisfactory results.
NeuPhysics: Editable Neural Geometry and Physics from Monocular Videos
Authors: Yi-Ling Qiao, Alexander Gao, Ming C. Lin
Date: 2022-10-22 00:00:00
Tasks: 3D geometry, Video Reconstruction

We present a method for learning 3D geometry and physics parameters of a dynamic scene from only a monocular RGB video input. To decouple the learning of underlying scene geometry from dynamic motion, we represent the scene as a time-invariant signed distance function (SDF) which serves as a reference frame, along with a time-conditioned deformation field. We further bridge this neural geometry representation with a differentiable physics simulator by designing a two-way conversion between the neural field and its corresponding hexahedral mesh, enabling us to estimate physics parameters from the source video by minimizing a cycle consistency loss. Our method also allows a user to interactively edit 3D objects from the source video by modifying the recovered hexahedral mesh, and propagating the operation back to the neural field representation. Experiments show that our method achieves superior mesh and video reconstruction of dynamic scenes compared to competing Neural Field approaches, and we provide extensive examples which demonstrate its ability to extract useful 3D representations from videos captured with consumer-grade cameras.
A Task-aware Dual Similarity Network for Fine-grained Few-shot Learning
Authors: Yan Qi, Han Sun, Ningzhong Liu, Huiyu Zhou
Date: 2022-10-22 00:00:00
Tasks: Few-Shot Learning

The goal of fine-grained few-shot learning is to recognize sub-categories under the same super-category by learning few labeled samples. Most of the recent approaches adopt a single similarity measure, that is, global or local measure alone. However, for fine-grained images with high intra-class variance and low inter-class variance, exploring global invariant features and discriminative local details is quite essential. In this paper, we propose a Task-aware Dual Similarity Network(TDSNet), which applies global features and local patches to achieve better performance. Specifically, a local feature enhancement module is adopted to activate the features with strong discriminability. Besides, task-aware attention exploits the important patches among the entire task. Finally, both the class prototypes obtained by global features and discriminative local patches are employed for prediction. Extensive experiments on three fine-grained datasets demonstrate that the proposed TDSNet achieves competitive performance by comparing with other state-of-the-art algorithms.
Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets
Authors: Xiangyu Chen, Qinghao Hu, Kaidong Li, Cuncong Zhong, Guanghui Wang
Date: 2022-10-22 00:00:00

Vision Transformers has demonstrated competitive performance on computer vision tasks benefiting from their ability to capture long-range dependencies with multi-head self-attention modules and multi-layer perceptron. However, calculating global attention brings another disadvantage compared with convolutional neural networks, i.e. requiring much more data and computations to converge, which makes it difficult to generalize well on small datasets, which is common in practical applications. Previous works are either focusing on transferring knowledge from large datasets or adjusting the structure for small datasets. After carefully examining the self-attention modules, we discover that the number of trivial attention weights is far greater than the important ones and the accumulated trivial weights are dominating the attention in Vision Transformers due to their large quantity, which is not handled by the attention itself. This will cover useful non-trivial attention and harm the performance when trivial attention includes more noise, e.g. in shallow layers for some backbones. To solve this issue, we proposed to divide attention weights into trivial and non-trivial ones by thresholds, then Suppressing Accumulated Trivial Attention (SATA) weights by proposed Trivial WeIghts Suppression Transformation (TWIST) to reduce attention noise. Extensive experiments on CIFAR-100 and Tiny-ImageNet datasets show that our suppressing method boosts the accuracy of Vision Transformers by up to 2.3%. Code is available at https://github.com/xiangyu8/SATA.
Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model
Authors: Zhiyuan Ren, Zhihong Pan, Xin Zhou, Le Kang
Date: 2022-10-22 00:00:00
Tasks: Denoising, Image Generation, Motion Synthesis

We propose a simple and novel method for generating 3D human motion from complex natural language sentences, which describe different velocity, direction and composition of all kinds of actions. Different from existing methods that use classical generative architecture, we apply the Denoising Diffusion Probabilistic Model to this task, synthesizing diverse motion results under the guidance of texts. The diffusion model converts white noise into structured 3D motion by a Markov process with a series of denoising steps and is efficiently trained by optimizing a variational lower bound. To achieve the goal of text-conditioned image synthesis, we use the classifier-free guidance strategy to fuse text embedding into the model during training. Our experiments demonstrate that our model achieves competitive results on HumanML3D test set quantitatively and can generate more visually natural and diverse examples. We also show with experiments that our model is capable of zero-shot generation of motions for unseen text guidance.
Tools for Extracting Spatio-Temporal Patterns in Meteorological Image Sequences: From Feature Engineering to Attention-Based Neural Networks
Authors: Akansha Singh Bansal, Yoonjin Lee, Kyle Hilburn, Imme Ebert-Uphoff
Date: 2022-10-22 00:00:00
Tasks: Feature Engineering, Topological Data Analysis, Transfer Learning

Atmospheric processes involve both space and time. This is why human analysis of atmospheric imagery can often extract more information from animated loops of image sequences than from individual images. Automating such an analysis requires the ability to identify spatio-temporal patterns in image sequences which is a very challenging task, because of the endless possibilities of patterns in both space and time. In this paper we review different concepts and techniques that are useful to extract spatio-temporal context specifically for meteorological applications. In this survey we first motivate the need for these approaches in meteorology using two applications, solar forecasting and detecting convection from satellite imagery. Then we provide an overview of many different concepts and techniques that are helpful for the interpretation of meteorological image sequences, such as (1) feature engineering methods to strengthen the desired signal in the input, using meteorological knowledge, classic image processing, harmonic analysis and topological data analysis (2) explain how different convolution filters (2D/3D/LSTM-convolution) can be utilized strategically in convolutional neural network architectures to find patterns in both space and time (3) discuss the powerful new concept of 'attention' in neural networks and the powerful abilities it brings to the interpretation of image sequences (4) briefly survey strategies from unsupervised, self-supervised and transfer learning to reduce the need for large labeled datasets. We hope that presenting an overview of these tools - many of which are underutilized - will help accelerate progress in this area.
Learning a Grammar Inducer from Massive Uncurated Instructional Videos
Authors: Songyang Zhang, Linfeng Song, Lifeng Jin, Haitao Mi, Kun Xu, Dong Yu, Jiebo Luo
Date: 2022-10-22 00:00:00
Tasks: Language Acquisition, Video Alignment

Video-aided grammar induction aims to leverage video information for finding more accurate syntactic grammars for accompanying text. While previous work focuses on building systems for inducing grammars on text that are well-aligned with video content, we investigate the scenario, in which text and video are only in loose correspondence. Such data can be found in abundance online, and the weak correspondence is similar to the indeterminacy problem studied in language acquisition. Furthermore, we build a new model that can better learn video-span correlation without manually designed features adopted by previous work. Experiments show that our model trained only on large-scale YouTube data with no text-video alignment reports strong and robust performances across three unseen datasets, despite domain shift and noisy label issues. Furthermore our model yields higher F1 scores than the previous state-of-the-art systems trained on in-domain data.
Feature selection intelligent algorithm with mutual information and steepest ascent strategy
Authors: Elkebir Sarhrouni, Ahmed Hammouch, Driss Aboutajdine
Date: 2022-10-21 00:00:00
Tasks: feature selection

Remote sensing is a higher technology to produce knowledge for data mining applications. In principle hyperspectral images (HSIs) is a remote sensing tool that provides precise classification of regions. The HSI contains more than a hundred of images of the ground truth (GT) map. Some images are carrying relevant information, but others describe redundant information, or they are affected by atmospheric noise. The aim is to reduce dimensionality of HSI. Many studies use mutual information (MI) or normalised forms of MI to select appropriate bands. In this paper we design an algorithm based also on MI, and we combine MI with steepest ascent algorithm, to improve a symmetric uncertainty coefficient-based strategy to select relevant bands for classification of HSI. This algorithm is a feature selection tool and a wrapper strategy. We perform our study on HSI AVIRIS 92AV3C. This is an artificial intelligent system to control redundancy; we had to clear the difference of the result's algorithm and the human decision, and this can be viewed as case study which human decision is perhaps different to an intelligent algorithm. Index Terms - Hyperspectral images, Classification, Fea-ture selection, Mutual Information, Redundancy, Steepest Ascent. Artificial Intelligence
Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination
Authors: Yue Yang, Wenlin Yao, Hongming Zhang, Xiaoyang Wang, Dong Yu, Jianshu Chen
Date: 2022-10-21 00:00:00
Tasks: Image Generation, Language Modeling, Language Modelling, Retrieval, Text to Image Generation, Text-to-Image Generation, Visual Commonsense Tests

Large-scale pretrained language models have made significant advances in solving downstream language understanding tasks. However, they generally suffer from reporting bias, the phenomenon describing the lack of explicit commonsense knowledge in written text, e.g., ''an orange is orange''. To overcome this limitation, we develop a novel approach, Z-LaVI, to endow language models with visual imagination capabilities. Specifically, we leverage two complementary types of ''imaginations'': (i) recalling existing images through retrieval and (ii) synthesizing nonexistent images via text-to-image generation. Jointly exploiting the language inputs and the imagination, a pretrained vision-language model (e.g., CLIP) eventually composes a zero-shot solution to the original language tasks. Notably, fueling language models with imagination can effectively leverage visual knowledge to solve plain language tasks. In consequence, Z-LaVI consistently improves the zero-shot performance of existing language models across a diverse set of language tasks.
Score-based Denoising Diffusion with Non-Isotropic Gaussian Noise Models
Authors: Vikram Voleti, Christopher Pal, Adam Oberman
Date: 2022-10-21 00:00:00
Tasks: Denoising, Diversity

Generative models based on denoising diffusion techniques have led to an unprecedented increase in the quality and diversity of imagery that is now possible to create with neural generative models. However, most contemporary state-of-the-art methods are derived from a standard isotropic Gaussian formulation. In this work we examine the situation where non-isotropic Gaussian distributions are used. We present the key mathematical derivations for creating denoising diffusion models using an underlying non-isotropic Gaussian noise model. We also provide initial experiments with the CIFAR-10 dataset to help verify empirically that this more general modeling approach can also yield high-quality samples.
FIND: An Unsupervised Implicit 3D Model of Articulated Human Feet
Authors: Oliver Boyne, James Charles, Roberto Cipolla
Date: 2022-10-21 00:00:00
Tasks: Disentanglement

In this paper we present a high fidelity and articulated 3D human foot model. The model is parameterised by a disentangled latent code in terms of shape, texture and articulated pose. While high fidelity models are typically created with strong supervision such as 3D keypoint correspondences or pre-registration, we focus on the difficult case of little to no annotation. To this end, we make the following contributions: (i) we develop a Foot Implicit Neural Deformation field model, named FIND, capable of tailoring explicit meshes at any resolution i.e. for low or high powered devices; (ii) an approach for training our model in various modes of weak supervision with progressively better disentanglement as more labels, such as pose categories, are provided; (iii) a novel unsupervised part-based loss for fitting our model to 2D images which is better than traditional photometric or silhouette losses; (iv) finally, we release a new dataset of high resolution 3D human foot scans, Foot3D. On this dataset, we show our model outperforms a strong PCA implementation trained on the same data in terms of shape quality and part correspondences, and that our novel unsupervised part-based loss improves inference on images.
Imbalanced Classification in Medical Imaging via Regrouping
Authors: Le Peng, Yash Travadi, Rui Zhang, Ying Cui, Ju Sun
Date: 2022-10-21 00:00:00
Tasks: Classification, image-classification, Image Classification, imbalanced classification, Medical Image Classification

We propose performing imbalanced classification by regrouping majority classes into small classes so that we turn the problem into balanced multiclass classification. This new idea is dramatically different from popular loss reweighting and class resampling methods. Our preliminary result on imbalanced medical image classification shows that this natural idea can substantially boost the classification performance as measured by average precision (approximately area-under-the-precision-recall-curve, or AUPRC), which is more appropriate for evaluating imbalanced classification than other metrics such as balanced accuracy.