pwc-archive/papers-with-abstracts huggingface dataset:
Implementing a Real-Time, YOLOv5 based Social Distancing Measuring System for Covid-19
Authors: Narayana Darapaneni, Shrawan Kumar, Selvarangan Krishnan, Hemalatha K, Arunkumar Rajagopal, Nagendra, Anwesh Reddy Paduri
Date: 2022-04-07 00:00:00
Tasks: Transfer Learning

The purpose of this work is, to provide a YOLOv5 deep learning-based social distance monitoring framework using an overhead view perspective. In addition, we have developed a custom defined model YOLOv5 modified CSP (Cross Stage Partial Network) and assessed the performance on COCO and Visdrone dataset with and without transfer learning. Our findings show that the developed model successfully identifies the individual who violates the social distances. The accuracy of 81.7% for the modified bottleneck CSP without transfer learning is observed on COCO dataset after training the model for 300 epochs whereas for the same epochs, the default YOLOv5 model is attaining 80.1% accuracy with transfer learning. This shows an improvement in accuracy by our modified bottleneck CSP model. For the Visdrone dataset, we are able to achieve an accuracy of upto 56.5% for certain classes and especially an accuracy of 40% for people and pedestrians with transfer learning using the default YOLOv5s model for 30 epochs. While the modified bottleneck CSP is able to perform slightly better than the default model with an accuracy score of upto 58.1% for certain classes and an accuracy of ~40.4% for people and pedestrians.
PSTR: End-to-End One-Step Person Search With Transformers
Authors: Jiale Cao, Yanwei Pang, Rao Muhammad Anwer, Hisham Cholakkal, Jin Xie, Mubarak Shah, Fahad Shahbaz Khan
Date: 2022-04-07 00:00:00
Proceeding: CVPR 2022 1
Tasks: Decoder, Human Detection, Person Search

We propose a novel one-step transformer-based person search framework, PSTR, that jointly performs person detection and re-identification (re-id) in a single architecture. PSTR comprises a person search-specialized (PSS) module that contains a detection encoder-decoder for person detection along with a discriminative re-id decoder for person re-id. The discriminative re-id decoder utilizes a multi-level supervision scheme with a shared decoder for discriminative re-id feature learning and also comprises a part attention block to encode relationship between different parts of a person. We further introduce a simple multi-scale scheme to support re-id across person instances at different scales. PSTR jointly achieves the diverse objectives of object-level recognition (detection) and instance-level matching (re-id). To the best of our knowledge, we are the first to propose an end-to-end one-step transformer-based person search framework. Experiments are performed on two popular benchmarks: CUHK-SYSU and PRW. Our extensive ablations reveal the merits of the proposed contributions. Further, the proposed PSTR sets a new state-of-the-art on both benchmarks. On the challenging PRW benchmark, PSTR achieves a mean average precision (mAP) score of 56.5%. The source code is available at \url{https://github.com/JialeCao001/PSTR}.
Learning to Sieve: Prediction of Grading Curves from Images of Concrete Aggregate
Authors: Max Coenen, Dries Beyer, Christian Heipke, Michael Haist
Date: 2022-04-07 00:00:00

A large component of the building material concrete consists of aggregate with varying particle sizes between 0.125 and 32 mm. Its actual size distribution significantly affects the quality characteristics of the final concrete in both, the fresh and hardened states. The usually unknown variations in the size distribution of the aggregate particles, which can be large especially when using recycled aggregate materials, are typically compensated by an increased usage of cement which, however, has severe negative impacts on economical and ecological aspects of the concrete production. In order to allow a precise control of the target properties of the concrete, unknown variations in the size distribution have to be quantified to enable a proper adaptation of the concrete's mixture design in real time. To this end, this paper proposes a deep learning based method for the determination of concrete aggregate grading curves. In this context, we propose a network architecture applying multi-scale feature extraction modules in order to handle the strongly diverse object sizes of the particles. Furthermore, we propose and publish a novel dataset of concrete aggregate used for the quantitative evaluation of our method.
Sparse Optical Flow-Based Line Feature Tracking
Authors: Qiang Fu, Hongshan Yu, Islam Ali, Hong Zhang
Date: 2022-04-07 00:00:00
Tasks: Camera Pose Estimation, Optical Flow Estimation, Pose Estimation

In this paper we propose a novel sparse optical flow (SOF)-based line feature tracking method for the camera pose estimation problem. This method is inspired by the point-based SOF algorithm and developed based on an observation that two adjacent images in time-varying image sequences satisfy brightness invariant. Based on this observation, we re-define the goal of line feature tracking: track two endpoints of a line feature instead of the entire line based on gray value matching instead of descriptor matching. To achieve this goal, an efficient two endpoint tracking (TET) method is presented: first, describe a given line feature with its two endpoints; next, track the two endpoints based on SOF to obtain two new tracked endpoints by minimizing a pixel-level grayscale residual function; finally, connect the two tracked endpoints to generate a new line feature. The correspondence is established between the given and the new line feature. Compared with current descriptor-based methods, our TET method needs not to compute descriptors and detect line features repeatedly. Naturally, it has an obvious advantage over computation. Experiments in several public benchmark datasets show our method yields highly competitive accuracy with an obvious advantage over speed.
A Comprehensive Review of Sign Language Recognition: Different Types, Modalities, and Datasets
Authors: Dr. M. Madhiarasan, Prof. Partha Pratim Roy
Date: 2022-04-07 00:00:00
Tasks: Sign Language Recognition

A machine can understand human activities, and the meaning of signs can help overcome the communication barriers between the inaudible and ordinary people. Sign Language Recognition (SLR) is a fascinating research area and a crucial task concerning computer vision and pattern recognition. Recently, SLR usage has increased in many applications, but the environment, background image resolution, modalities, and datasets affect the performance a lot. Many researchers have been striving to carry out generic real-time SLR models. This review paper facilitates a comprehensive overview of SLR and discusses the needs, challenges, and problems associated with SLR. We study related works about manual and non-manual, various modalities, and datasets. Research progress and existing state-of-the-art SLR models over the past decade have been reviewed. Finally, we find the research gap and limitations in this domain and suggest future directions. This review paper will be helpful for readers and researchers to get complete guidance about SLR and the progressive design of the state-of-the-art SLR model
Multi-Sample $ζ$-mixup: Richer, More Realistic Synthetic Samples from a $p$-Series Interpolant
Authors: Kumar Abhishek, Colin J. Brown, Ghassan Hamarneh
Date: 2022-04-07 00:00:00
Tasks: Data Augmentation, Diversity, image-classification, Image Classification, Medical Image Classification

Modern deep learning training procedures rely on model regularization techniques such as data augmentation methods, which generate training samples that increase the diversity of data and richness of label information. A popular recent method, mixup, uses convex combinations of pairs of original samples to generate new samples. However, as we show in our experiments, mixup can produce undesirable synthetic samples, where the data is sampled off the manifold and can contain incorrect labels. We propose $\zeta$-mixup, a generalization of mixup with provably and demonstrably desirable properties that allows convex combinations of $N \geq 2$ samples, leading to more realistic and diverse outputs that incorporate information from $N$ original samples by using a $p$-series interpolant. We show that, compared to mixup, $\zeta$-mixup better preserves the intrinsic dimensionality of the original datasets, which is a desirable property for training generalizable models. Furthermore, we show that our implementation of $\zeta$-mixup is faster than mixup, and extensive evaluation on controlled synthetic and 24 real-world natural and medical image classification datasets shows that $\zeta$-mixup outperforms mixup and traditional data augmentation techniques.
Swarm behavior tracking based on a deep vision algorithm
Authors: Meihong Wu, Xiaoyan Cao, Shihui Guo
Date: 2022-04-07 00:00:00
Tasks: object-detection, Object Detection

The intelligent swarm behavior of social insects (such as ants) springs up in different environments, promising to provide insights for the study of embodied intelligence. Researching swarm behavior requires that researchers could accurately track each individual over time. Obviously, manually labeling individual insects in a video is labor-intensive. Automatic tracking methods, however, also poses serious challenges: (1) individuals are small and similar in appearance; (2) frequent interactions with each other cause severe and long-term occlusion. With the advances of artificial intelligence and computing vision technologies, we are hopeful to provide a tool to automate monitor multiple insects to address the above challenges. In this paper, we propose a detection and tracking framework for multi-ant tracking in the videos by: (1) adopting a two-stage object detection framework using ResNet-50 as backbone and coding the position of regions of interest to locate ants accurately; (2) using the ResNet model to develop the appearance descriptors of ants; (3) constructing long-term appearance sequences and combining them with motion information to achieve online tracking. To validate our method, we construct an ant database including 10 videos of ants from different indoor and outdoor scenes. We achieve a state-of-the-art performance of 95.7\% mMOTA and 81.1\% mMOTP in indoor videos, 81.8\% mMOTA and 81.9\% mMOTP in outdoor videos. Additionally, Our method runs 6-10 times faster than existing methods for insect tracking. Experimental results demonstrate that our method provides a powerful tool for accelerating the unraveling of the mechanisms underlying the swarm behavior of social insects.
Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU
Authors: Alessandro Lotti, Dario Modenini, Paolo Tortora, Massimiliano Saponara, Maria A. Perino
Date: 2022-04-07 00:00:00
Tasks: Pose Estimation, Spacecraft Pose Estimation

Pose estimation of an uncooperative space resident object is a key asset towards autonomy in close proximity operations. In this context monocular cameras are a valuable solution because of their low system requirements. However, the associated image processing algorithms are either too computationally expensive for real time on-board implementation, or not enough accurate. In this paper we propose a pose estimation software exploiting neural network architectures which can be scaled to different accuracy-latency trade-offs. We designed our pipeline to be compatible with Edge Tensor Processing Units to show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space. The neural networks were tested both on the benchmark Spacecraft Pose Estimation Dataset, and on the purposely developed Cosmo Photorealistic Dataset, which depicts a COSMO-SkyMed satellite in a variety of random poses and steerable solar panels orientations. The lightest version of our architecture achieves state-of-the-art accuracy on both datasets but at a fraction of networks complexity, running at 7.7 frames per second on a Coral Dev Board Mini consuming just 2.2W.
Multi-scale Context-aware Network with Transformer for Gait Recognition
Authors: Duowang Zhu, Xiaohu Huang, Xinggang Wang, Bo Yang, Botao He, Wenyu Liu, Bin Feng
Date: 2022-04-07 00:00:00
Proceeding: ICCV 2021 10
Tasks: Gait Recognition, Multiview Gait Recognition, Relation

Although gait recognition has drawn increasing research attention recently, since the silhouette differences are quite subtle in spatial domain, temporal feature representation is crucial for gait recognition. Inspired by the observation that humans can distinguish gaits of different subjects by adaptively focusing on clips of varying time scales, we propose a multi-scale context-aware network with transformer (MCAT) for gait recognition. MCAT generates temporal features across three scales, and adaptively aggregates them using contextual information from both local and global perspectives. Specifically, MCAT contains an adaptive temporal aggregation (ATA) module that performs local relation modeling followed by global relation modeling to fuse the multi-scale features. Besides, in order to remedy the spatial feature corruption resulting from temporal operations, MCAT incorporates a salient spatial feature learning (SSFL) module to select groups of discriminative spatial features. Extensive experiments conducted on three datasets demonstrate the state-of-the-art performance. Concretely, we achieve rank-1 accuracies of 98.7%, 96.2% and 88.7% under normal walking, bag-carrying and coat-wearing conditions on CASIA-B, 97.5% on OU-MVLP and 50.6% on GREW. The source code will be available at https://github.com/zhuduowang/MCAT.git.
Deep learning-based approach to reveal tumor mutational burden status from whole slide images across multiple cancer types
Authors: Siteng Chen, Jinxi Xiang, Xiyue Wang, Jun Zhang, Sen yang, Junzhou Huang, Wei Yang, Junhua Zheng, Xiao Han
Date: 2022-04-07 00:00:00
Tasks: whole slide images

Tumor mutational burden (TMB) is a potential genomic biomarker of immunotherapy. However, TMB detected through whole exome sequencing lacks clinical penetration in low-resource settings. In this study, we proposed a multi-scale deep learning framework to address the detection of TMB status from routinely used whole slide images for a multiple cancer TMB prediction model (MC- TMB). The MC-TMB achieved a mean area under the curve (AUC) of 0.818 (0.804-0.831) in the cross-validation cohort, which showed superior performance to each single-scale model. The improvements of MC-TMB over the single-tumor models were also confirmed by the ablation tests on x10 magnification, and the highly concerned regions typically correspond to dense lymphocytic infiltration and heteromorphic tumor cells. MC-TMB algorithm also exhibited good generalization on the external validation cohort with an AUC of 0.732 (0.683-0.761), and better performance when compared to other methods. In conclusion, we proposed a deep learning-based approach to reveal tumor mutational burden status from routinely used pathological slides across multiple cancer types.
MC-UNet Multi-module Concatenation based on U-shape Network for Retinal Blood Vessels Segmentation
Authors: Ting Zhang, Jun Li, Yi Zhao, Nan Chen, Han Zhou, Hongtao Xu, Zihao Guan, Changcai Yang, Lanyan Xue, Riqing Chen, Lifang Wei
Date: 2022-04-07 00:00:00
Tasks: Segmentation

Accurate segmentation of the blood vessels of the retina is an important step in clinical diagnosis of ophthalmic diseases. Many deep learning frameworks have come up for retinal blood vessels segmentation tasks. However, the complex vascular structure and uncertain pathological features make the blood vessel segmentation still very challenging. A novel U-shaped network named Multi-module Concatenation which is based on Atrous convolution and multi-kernel pooling is put forward to retinal vessels segmentation in this paper. The proposed network structure retains three layers the essential structure of U-Net, in which the atrous convolution combining the multi-kernel pooling blocks are designed to obtain more contextual information. The spatial attention module is concatenated with dense atrous convolution module and multi-kernel pooling module to form a multi-module concatenation. And different dilation rates are selected by cascading to acquire a larger receptive field in atrous convolution. Adequate comparative experiments are conducted on these public retinal datasets: DRIVE, STARE and CHASE_DB1. The results show that the proposed method is effective, especially for microvessels. The code will be put out at https://github.com/Rebeccala/MC-UNet
L2G: A Simple Local-to-Global Knowledge Transfer Framework for Weakly Supervised Semantic Segmentation
Authors: Peng-Tao Jiang, YuQi Yang, Qibin Hou, Yunchao Wei
Date: 2022-04-07 00:00:00
Proceeding: CVPR 2022 1
Tasks: Object, Semantic Segmentation, Transfer Learning, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

Mining precise class-aware attention maps, a.k.a, class activation maps, is essential for weakly supervised semantic segmentation. In this paper, we present L2G, a simple online local-to-global knowledge transfer framework for high-quality object attention mining. We observe that classification models can discover object regions with more details when replacing the input image with its local patches. Taking this into account, we first leverage a local classification network to extract attentions from multiple local patches randomly cropped from the input image. Then, we utilize a global network to learn complementary attention knowledge across multiple local attention maps online. Our framework conducts the global network to learn the captured rich object detail knowledge from a global view and thereby produces high-quality attention maps that can be directly used as pseudo annotations for semantic segmentation networks. Experiments show that our method attains 72.1% and 44.2% mIoU scores on the validation set of PASCAL VOC 2012 and MS COCO 2014, respectively, setting new state-of-the-art records. Code is available at https://github.com/PengtaoJiang/L2G.
MDA GAN: Adversarial-Learning-based 3-D Seismic Data Interpolation and Reconstruction for Complex Missing
Authors: YiMin Dou, Kewen Li, Hongjie Duan, Timing Li, Lin Dong, Zongchao Huang
Date: 2022-04-07 00:00:00

The interpolation and reconstruction of missing traces is a crucial step in seismic data processing, moreover it is also a highly ill-posed problem, especially for complex cases such as high-ratio random discrete missing, continuous missing and missing in fault-rich or salt body surveys. These complex cases are rarely mentioned in current works. To cope with complex missing cases, we propose Multi-Dimensional Adversarial GAN (MDA GAN), a novel 3-D GAN framework. It keeps anisotropy and spatial continuity of the data after 3D complex missing reconstruction using three discriminators. The feature stitching module is designed and embedded in the generator to retain more information of the input data. The Tanh cross entropy (TCE) loss is derived, which provides the generator with the optimal reconstruction gradient to make the generated data smoother and continuous. We experimentally verified the effectiveness of the individual components of the study and then tested the method on multiple publicly available data. The method achieves reasonable reconstructions for up to 95% of random discrete missing and 100 traces of continuous missing. In fault and salt body enriched surveys, MDA GAN still yields promising results for complex cases. Experimentally it has been demonstrated that our method achieves better performance than other methods in both simple and complex cases.https://github.com/douyimin/MDA_GAN
Low-Dose CT Denoising via Sinogram Inner-Structure Transformer
Authors: Liutao Yang, Zhongnian Li, Rongjun Ge, Junyong Zhao, Haipeng Si, Daoqiang Zhang
Date: 2022-04-07 00:00:00
Tasks: Denoising, Image Reconstruction

Low-Dose Computed Tomography (LDCT) technique, which reduces the radiation harm to human bodies, is now attracting increasing interest in the medical imaging field. As the image quality is degraded by low dose radiation, LDCT exams require specialized reconstruction methods or denoising algorithms. However, most of the recent effective methods overlook the inner-structure of the original projection data (sinogram) which limits their denoising ability. The inner-structure of the sinogram represents special characteristics of the data in the sinogram domain. By maintaining this structure while denoising, the noise can be obviously restrained. Therefore, we propose an LDCT denoising network namely Sinogram Inner-Structure Transformer (SIST) to reduce the noise by utilizing the inner-structure in the sinogram domain. Specifically, we study the CT imaging mechanism and statistical characteristics of sinogram to design the sinogram inner-structure loss including the global and local inner-structure for restoring high-quality CT images. Besides, we propose a sinogram transformer module to better extract sinogram features. The transformer architecture using a self-attention mechanism can exploit interrelations between projections of different view angles, which achieves an outstanding performance in sinogram denoising. Furthermore, in order to improve the performance in the image domain, we propose the image reconstruction module to complementarily denoise both in the sinogram and image domain.
Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality
Authors: Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, Candace Ross
Date: 2022-04-07 00:00:00
Proceeding: CVPR 2022 1
Tasks: Visual Reasoning

We present a novel task and dataset for evaluating the ability of vision and language models to conduct visio-linguistic compositional reasoning, which we call Winoground. Given two images and two captions, the goal is to match them correctly - but crucially, both captions contain a completely identical set of words, only in a different order. The dataset was carefully hand-curated by expert annotators and is labeled with a rich set of fine-grained tags to assist in analyzing model performance. We probe a diverse range of state-of-the-art vision and language models and find that, surprisingly, none of them do much better than chance. Evidently, these models are not as skilled at visio-linguistic compositional reasoning as we might have hoped. We perform an extensive analysis to obtain insights into how future work might try to mitigate these models' shortcomings. We aim for Winoground to serve as a useful evaluation set for advancing the state of the art and driving further progress in the field. The dataset is available at https://huggingface.co/datasets/facebook/winoground.