pwc-archive/papers-with-abstracts huggingface dataset:
Enhanced Correlation Matching based Video Frame Interpolation
Authors: Sungho Lee, Narae Choi, Woong Il Choi
Date: 2021-11-17 00:00:00
Tasks: 4k, Optical Flow Estimation, Video Frame Interpolation

We propose a novel DNN based framework called the Enhanced Correlation Matching based Video Frame Interpolation Network to support high resolution like 4K, which has a large scale of motion and occlusion. Considering the extensibility of the network model according to resolution, the proposed scheme employs the recurrent pyramid architecture that shares the parameters among each pyramid layer for optical flow estimation. In the proposed flow estimation, the optical flows are recursively refined by tracing the location with maximum correlation. The forward warping based correlation matching enables to improve the accuracy of flow update by excluding incorrectly warped features around the occlusion area. Based on the final bi-directional flows, the intermediate frame at arbitrary temporal position is synthesized using the warping and blending network and it is further improved by refinement network. Experiment results demonstrate that the proposed scheme outperforms the previous works at 4K video data and low-resolution benchmark datasets as well in terms of objective and subjective quality with the smallest number of model parameters.
TYolov5: A Temporal Yolov5 Detector Based on Quasi-Recurrent Neural Networks for Real-Time Handgun Detection in Video
Authors: Mario Alberto Duran-Vega, Miguel Gonzalez-Mendoza, Leonardo Chang, Cuauhtemoc Daniel Suarez-Ramirez
Date: 2021-11-17 00:00:00
Tasks: Data Augmentation, Image Augmentation, object-detection, Object Detection, Video Object Detection

Timely handgun detection is a crucial problem to improve public safety; nevertheless, the effectiveness of many surveillance systems still depends of finite human attention. Much of the previous research on handgun detection is based on static image detectors, leaving aside valuable temporal information that could be used to improve object detection in videos. To improve the performance of surveillance systems, a real-time temporal handgun detection system should be built. Using Temporal Yolov5, an architecture based on Quasi-Recurrent Neural Networks, temporal information is extracted from video to improve the results of handgun detection. Moreover, two publicly available datasets are proposed, labeled with hands, guns, and phones. One containing 2199 static images to train static detectors, and another with 5960 frames of videos to train temporal modules. Additionally, we explore two temporal data augmentation techniques based on Mosaic and Mixup. The resulting systems are three temporal architectures: one focused in reducing inference with a mAP$_{50:95}$ of 55.9, another in having a good balance between inference and accuracy with a mAP$_{50:95}$ of 59, and a last one specialized in accuracy with a mAP$_{50:95}$ of 60.2. Temporal Yolov5 achieves real-time detection in the small and medium architectures. Moreover, it takes advantage of temporal features contained in videos to perform better than Yolov5 in our temporal dataset, making TYolov5 suitable for real-world applications. The source code is publicly available at https://github.com/MarioDuran/TYolov5.
Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities
Authors: Xintong Shi, Wenzhi Cao, Sebastian Raschka
Date: 2021-11-17 00:00:00
Tasks: regression

In recent times, deep neural networks achieved outstanding predictive performance on various classification and pattern recognition tasks. However, many real-world prediction problems have ordinal response variables, and this ordering information is ignored by conventional classification losses such as the multi-category cross-entropy. Ordinal regression methods for deep neural networks address this. One such method is the CORAL method, which is based on an earlier binary label extension framework and achieves rank consistency among its output layer tasks by imposing a weight-sharing constraint. However, while earlier experiments showed that CORAL's rank consistency is beneficial for performance, it is limited by a weight-sharing constraint in a neural network's fully connected output layer, which may restrict the expressiveness and capacity of a network trained using CORAL. We propose a new method for rank-consistent ordinal regression without this limitation. Our rank-consistent ordinal regression framework (CORN) achieves rank consistency by a novel training scheme. This training scheme uses conditional training sets to obtain the unconditional rank probabilities through applying the chain rule for conditional probability distributions. Experiments on various datasets demonstrate the efficacy of the proposed method to utilize the ordinal target information, and the absence of the weight-sharing restriction improves the performance substantially compared to the CORAL reference approach. Additionally, the suggested CORN method is not tied to any specific architecture and can be utilized with any deep neural network classifier to train it for ordinal regression tasks.
HARA: A Hierarchical Approach for Robust Rotation Averaging
Authors: Seong Hun Lee, Javier Civera
Date: 2021-11-16 00:00:00
Proceeding: CVPR 2022 1
Tasks: Triplet, valid

We propose a novel hierarchical approach for multiple rotation averaging, dubbed HARA. Our method incrementally initializes the rotation graph based on a hierarchy of triplet support. The key idea is to build a spanning tree by prioritizing the edges with many strong triplet supports and gradually adding those with weaker and fewer supports. This reduces the risk of adding outliers in the spanning tree. As a result, we obtain a robust initial solution that enables us to filter outliers prior to nonlinear optimization. With minimal modification, our approach can also integrate the knowledge of the number of valid 2D-2D correspondences. We perform extensive evaluations on both synthetic and real datasets, demonstrating state-of-the-art results.
DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds
Authors: Ruben Wiersma, Ahmad Nasikun, Elmar Eisemann, Klaus Hildebrandt
Date: 2021-11-16 00:00:00
Tasks: 3D Part Segmentation, 3D Point Cloud Classification, Classification, Deep Learning, Point Cloud Classification, Semantic Segmentation

Learning from 3D point-cloud data has rapidly gained momentum, motivated by the success of deep learning on images and the increased availability of 3D~data. In this paper, we aim to construct anisotropic convolution layers that work directly on the surface derived from a point cloud. This is challenging because of the lack of a global coordinate system for tangential directions on surfaces. We introduce DeltaConv, a convolution layer that combines geometric operators from vector calculus to enable the construction of anisotropic filters on point clouds. Because these operators are defined on scalar- and vector-fields, we separate the network into a scalar- and a vector-stream, which are connected by the operators. The vector stream enables the network to explicitly represent, evaluate, and process directional information. Our convolutions are robust and simple to implement and match or improve on state-of-the-art approaches on several benchmarks, while also speeding up training and inference.
Detecting AutoAttack Perturbations in the Frequency Domain
Authors: Peter Lorenz, Paula Harder, Dominik Strassel, Margret Keuper, Janis Keuper
Date: 2021-11-16 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: image-classification, Image Classification

Recently, adversarial attacks on image classification networks by the AutoAttack (Croce and Hein, 2020b) framework have drawn a lot of attention. While AutoAttack has shown a very high attack success rate, most defense approaches are focusing on network hardening and robustness enhancements, like adversarial training. This way, the currently best-reported method can withstand about 66% of adversarial examples on CIFAR10. In this paper, we investigate the spatial and frequency domain properties of AutoAttack and propose an alternative defense. Instead of hardening a network, we detect adversarial attacks during inference, rejecting manipulated inputs. Based on a rather simple and fast analysis in the frequency domain, we introduce two different detection algorithms. First, a black box detector that only operates on the input images and achieves a detection accuracy of 100% on the AutoAttack CIFAR10 benchmark and 99.3% on ImageNet, for epsilon = 8/255 in both cases. Second, a whitebox detector using an analysis of CNN feature maps, leading to a detection rate of also 100% and 98.7% on the same benchmarks.
Finding the Right Moment: Human-Assisted Trailer Creation via Task Composition
Authors: Pinelopi Papalampidi, Frank Keller, Mirella Lapata
Date: 2021-11-16 00:00:00

Movie trailers perform multiple functions: they introduce viewers to the story, convey the mood and artistic style of the film, and encourage audiences to see the movie. These diverse functions make trailer creation a challenging endeavor. In this work, we focus on finding trailer moments in a movie, i.e., shots that could be potentially included in a trailer. We decompose this task into two subtasks: narrative structure identification and sentiment prediction. We model movies as graphs, where nodes are shots and edges denote semantic relations between them. We learn these relations using joint contrastive training which distills rich textual information (e.g., characters, actions, situations) from screenplays. An unsupervised algorithm then traverses the graph and selects trailer moments from the movie that human judges prefer to ones selected by competitive supervised approaches. A main advantage of our algorithm is that it uses interpretable criteria, which allows us to deploy it in an interactive tool for trailer creation with a human in the loop. Our tool allows users to select trailer shots in under 30 minutes that are superior to fully automatic methods and comparable to (exclusive) manual selection by experts.
Computer Vision for Supporting Image Search
Authors: Alan F. Smeaton
Date: 2021-11-16 00:00:00
Tasks: Image Retrieval, Medical Image Analysis

Computer vision and multimedia information processing have made extreme progress within the last decade and many tasks can be done with a level of accuracy as if done by humans, or better. This is because we leverage the benefits of huge amounts of data available for training, we have enormous computer processing available and we have seen the evolution of machine learning as a suite of techniques to process data and deliver accurate vision-based systems. What kind of applications do we use this processing for ? We use this in autonomous vehicle navigation or in security applications, searching CCTV for example, and in medical image analysis for healthcare diagnostics. One application which is not widespread is image or video search directly by users. In this paper we present the need for such image finding or re-finding by examining human memory and when it fails, thus motivating the need for a different approach to image search which is outlined, along with the requirements of computer vision to support it.
Automatic Semantic Segmentation of the Lumbar Spine: Clinical Applicability in a Multi-parametric and Multi-centre Study on Magnetic Resonance Images
Authors: Jhon Jairo Saenz-Gamboa, Julio Domenech, Antonio Alonso-Manjarrés, Jon A. Gómez, Maria de la Iglesia-Vayá
Date: 2021-11-16 00:00:00
Tasks: Anatomy, Image Segmentation, Medical Image Segmentation, Semantic Segmentation

One of the major difficulties in medical image segmentation is the high variability of these images, which is caused by their origin (multi-centre), the acquisition protocols (multi-parametric), as well as the variability of human anatomy, the severity of the illness, the effect of age and gender, among others. The problem addressed in this work is the automatic semantic segmentation of lumbar spine Magnetic Resonance images using convolutional neural networks. The purpose is to assign a class label to each pixel of an image. Classes were defined by radiologists and correspond to different structural elements like vertebrae, intervertebral discs, nerves, blood vessels, and other tissues. The proposed network topologies are variants of the U-Net architecture. Several complementary blocks were used to define the variants: Three types of convolutional blocks, spatial attention models, deep supervision and multilevel feature extractor. This document describes the topologies and analyses the results of the neural network designs that obtained the most accurate segmentations. Several of the proposed designs outperform the standard U-Net used as baseline, especially when used in ensembles where the output of multiple neural networks is combined according to different strategies.
Two-step adversarial debiasing with partial learning -- medical image case-studies
Authors: Ramon Correa, Jiwoong Jason Jeong, Bhavik Patel, Hari Trivedi, Judy W. Gichoya, Imon Banerjee
Date: 2021-11-16 00:00:00
Tasks: Decision Making, image-classification, Image Classification, Vocal Bursts Valence Prediction

The use of artificial intelligence (AI) in healthcare has become a very active research area in the last few years. While significant progress has been made in image classification tasks, only a few AI methods are actually being deployed in hospitals. A major hurdle in actively using clinical AI models currently is the trustworthiness of these models. More often than not, these complex models are black boxes in which promising results are generated. However, when scrutinized, these models begin to reveal implicit biases during the decision making, such as detecting race and having bias towards ethnic groups and subpopulations. In our ongoing study, we develop a two-step adversarial debiasing approach with partial learning that can reduce the racial disparity while preserving the performance of the targeted task. The methodology has been evaluated on two independent medical image case-studies - chest X-ray and mammograms, and showed promises in bias reduction while preserving the targeted performance.
CNN Filter Learning from Drawn Markers for the Detection of Suggestive Signs of COVID-19 in CT Images
Authors: Azael M. Sousa, Fabiano Reis, Rachel Zerbini, João L. D. Comba, Alexandre X. Falcão
Date: 2021-11-16 00:00:00

Early detection of COVID-19 is vital to control its spread. Deep learning methods have been presented to detect suggestive signs of COVID-19 from chest CT images. However, due to the novelty of the disease, annotated volumetric data are scarce. Here we propose a method that does not require either large annotated datasets or backpropagation to estimate the filters of a convolutional neural network (CNN). For a few CT images, the user draws markers at representative normal and abnormal regions. The method generates a feature extractor composed of a sequence of convolutional layers, whose kernels are specialized in enhancing regions similar to the marked ones, and the decision layer of our CNN is a support vector machine. As we have no control over the CT image acquisition, we also propose an intensity standardization approach. Our method can achieve mean accuracy and kappa values of $0.97$ and $0.93$, respectively, on a dataset with 117 CT images extracted from different sites, surpassing its counterpart in all scenarios.
Automated skin lesion segmentation using multi-scale feature extraction scheme and dual-attention mechanism
Authors: G Jignesh Chowdary, G V S N Durga Yathisha, Suganya G, Premalatha M
Date: 2021-11-16 00:00:00
Tasks: Lesion Segmentation, Segmentation, Skin Lesion Segmentation

Segmenting skin lesions from dermoscopic images is essential for diagnosing skin cancer. But the automatic segmentation of these lesions is complicated due to the poor contrast between the background and the lesion, image artifacts, and unclear lesion boundaries. In this work, we present a deep learning model for the segmentation of skin lesions from dermoscopic images. To deal with the challenges of skin lesion characteristics, we designed a multi-scale feature extraction module for extracting the discriminative features. Further in this work, two attention mechanisms are developed to refine the post-upsampled features and the features extracted by the encoder. This model is evaluated using the ISIC2018 and ISBI2017 datasets. The proposed model outperformed all the existing works and the top-ranked models in two competitions.
Automated Atlas-based Segmentation of Single Coronal Mouse Brain Slices using Linear 2D-2D Registration
Authors: Sébastien Piluso, Nicolas Souedet, Caroline Jan, Cédric Clouchoux, Thierry Delzescaux
Date: 2021-11-16 00:00:00
Tasks: Segmentation

A significant challenge for brain histological data analysis is to precisely identify anatomical regions in order to perform accurate local quantifications and evaluate therapeutic solutions. Usually, this task is performed manually, becoming therefore tedious and subjective. Another option is to use automatic or semi-automatic methods, among which segmentation using digital atlases co-registration. However, most available atlases are 3D, whereas digitized histological data are 2D. Methods to perform such 2D-3D segmentation from an atlas are required. This paper proposes a strategy to automatically and accurately segment single 2D coronal slices within a 3D volume of atlas, using linear registration. We validated its robustness and performance using an exploratory approach at whole-brain scale.
Identifying the Factors that Influence Urban Public Transit Demand
Authors: Armstrong Aboah, Lydia Johnson, Setul Shah
Date: 2021-11-16 00:00:00
Tasks: regression

The rise in urbanization throughout the United States (US) in recent years has required urban planners and transportation engineers to have greater consideration for the transportation services available to residents of a metropolitan region. This compels transportation authorities to provide better and more reliable modes of public transit through improved technologies and increased service quality. These improvements can be achieved by identifying and understanding the factors that influence urban public transit demand. Common factors that can influence urban public transit demand can be internal and/or external factors. Internal factors include policy measures such as transit fares, service headways, and travel times. External factors can include geographic, socioeconomic, and highway facility characteristics. There is inherent simultaneity between transit supply and demand, thus a two-stage least squares (2SLS) regression modeling procedure should be conducted to forecast urban transit supply and demand. As such, two multiple linear regression models should be developed: one to predict transit supply and a second to predict transit demand. It was found that service area density, total average cost per trip, and the average number of vehicles operated in maximum service can be used to forecast transit supply, expressed as vehicle revenue hours. Furthermore, estimated vehicle revenue hours and total average fares per trip can be used to forecast transit demand, expressed as unlinked passenger trips. Additional data such as socioeconomic information of the surrounding areas for each transit agency and travel time information of the various transit systems would be useful to improve upon the models developed.
The Neural Correlates of Image Texture in the Human Vision Using Magnetoencephalography
Authors: Elaheh Hatamimajoumerd, Alireza Talebpour
Date: 2021-11-16 00:00:00
Tasks: Object Recognition

Undoubtedly, textural property of an image is one of the most important features in object recognition task in both human and computer vision applications. Here, we investigated the neural signatures of four well-known statistical texture features including contrast, homogeneity, energy, and correlation computed from the gray level co-occurrence matrix (GLCM) of the images viewed by the participants in the process of magnetoencephalography (MEG) data collection. To trace these features in the human visual system, we used multivariate pattern analysis (MVPA) and trained a linear support vector machine (SVM) classifier on every timepoint of MEG data representing the brain activity and compared it with the textural descriptors of images using the Spearman correlation. The result of this study demonstrates that hierarchical structure in the processing of these four texture descriptors in the human brain with the order of contrast, homogeneity, energy, and correlation. Additionally, we found that energy, which carries broad texture property of the images, shows a more sustained statistically meaningful correlation with the brain activity in the course of time.
Image-based monitoring of bolt loosening through deep-learning-based integrated detection and tracking
Authors: Xiao Pan, T. Y. Yang
Date: 2021-11-16 00:00:00
Tasks: Friction, Object Tracking, Optical Flow Estimation

Structural bolts are critical components used in different structural elements, such as beam-column connections and friction damping devices. The clamping force in structural bolts is highly influenced by the bolt rotation. Much of the existing vision-based research about bolt rotation estimation relies on traditional computer vision algorithms such as Hough Transform to assess static images of bolts. This requires careful image preprocessing, and it may not perform well in the situation of complicated bolt assemblies, or in the presence of surrounding objects and background noise, thus hindering their real-world applications. In this study, an integrated real-time detect-track method, namely RTDT-Bolt, is proposed to monitor the bolt rotation angle. First, a real-time convolutional-neural-networks-based object detector, named YOLOv3-tiny, is established and trained to localize structural bolts. Then, the target-free object tracking algorithm based on optical flow is implemented, to continuously monitor and quantify the rotation of structural bolts. In order to enhance the tracking performance against background noise and potential illumination changes during tracking, the YOLOv3-tiny is integrated with the optical flow tracking algorithm to re-detect the bolts when the tracking gets lost. Extensive parameter studies were conducted to identify optimal tracking performance and examine the potential limitations. The results indicate the RTDT-Bolt method can greatly enhance the tracking performance of bolt rotation, which can achieve over 90% accuracy using the recommended range for the parameters.