pwc-archive/papers-with-abstracts huggingface dataset:
A User Study to Investigate Semantically Relevant Contextual Information of WWW Images
Authors: Fariza Fauzi, Mohammed Belkhatir
Date: 2020-04-29 00:00:00

The contextual information of Web images is investigated to address the issue of enriching their index characterizations with semantic descriptors and therefore bridge the semantic gap (i.e. the gap between the low-level content-based description of images and their semantic interpretation). Although we are highly motivated by the availability of rich knowledge on the Web and the relative success achieved by commercial search engines in indexing images using surrounding text-based information in webpages, we are aware that the unpredictable quality of the surrounding text is a major limiting factor. In order to improve its quality, we highlight contextual information which is relevant for the semantic characterization of Web images and study its statistical properties in terms of its location and nature considering a classification into five semantic concept classes: signal, object, scene, abstract and relational. A user study is conducted to validate the results. The results suggest that there are several locations that consistently contain relevant textual information with respect to the image. The importance of each location is influenced by the type of webpage as the results show the different distribution of relevant contextual information across the locations for different webpage types. The frequently found semantic concept classes are object and abstract. Another important outcome of the user study shows that a webpage is not an atomic unit and can be further partitioned into smaller segments. Segments containing images are of interest and termed as image segments. We observe that users typically single out textual information which they consider relevant to the image from the textual information bounded within the image segment.
An Unsupervised Semantic Sentence Ranking Scheme for Text Documents
Authors: Hao Zhang, Jie Wang
Date: 2020-04-28 00:00:00
Tasks: Clustering, Diversity, Sentence

This paper presents Semantic SentenceRank (SSR), an unsupervised scheme for automatically ranking sentences in a single document according to their relative importance. In particular, SSR extracts essential words and phrases from a text document, and uses semantic measures to construct, respectively, a semantic phrase graph over phrases and words, and a semantic sentence graph over sentences. It applies two variants of article-structure-biased PageRank to score phrases and words on the first graph and sentences on the second graph. It then combines these scores to generate the final score for each sentence. Finally, SSR solves a multi-objective optimization problem for ranking sentences based on their final scores and topic diversity through semantic subtopic clustering. An implementation of SSR that runs in quadratic time is presented, and it outperforms, on the SummBank benchmarks, each individual judge's ranking and compares favorably with the combined ranking of all judges.
Psychophysical Evaluation of Deep Re-Identification Models
Authors: Hamish Nicholson
Date: 2020-04-28 00:00:00
Tasks: Autonomous Driving

Pedestrian re-identification (ReID) is the task of continuously recognising the sameindividual across time and camera views. Researchers of pedestrian ReID and theirGPUs spend enormous energy producing novel algorithms, challenging datasets,and readily accessible tools to successfully improve results on standard metrics.Yet practitioners in biometrics, surveillance, and autonomous driving have not re-alized benefits that reflect these metrics. Different detections, slight occlusions,changes in perspective, and other banal perturbations render the best neural net-works virtually useless. This work makes two contributions. First, we introducethe ReID community to a budding area of computer vision research in model eval-uation. By adapting established principles of psychophysical evaluation from psy-chology, we can quantify the performance degradation and begin research thatwill improve the utility of pedestrian ReID models; not just their performance ontest sets. Second, we introduce NuscenesReID, a challenging new ReID datasetdesigned to reflect the real world autonomous vehicle conditions in which ReIDalgorithms are used. We show that, despite performing well on existing ReIDdatasets, most models are not robust to synthetic augmentations or to the morerealistic NuscenesReID data.
A fast and memory-efficient algorithm for smooth interpolation of polyrigid transformations: application to human joint tracking
Authors: K. Makki, B. Borotikar, M. Garetier, S. Brochard, D. Ben Salem, F. Rousseau
Date: 2020-04-28 00:00:00
Tasks: Image Registration

The log Euclidean polyrigid registration framework provides a way to smoothly estimate and interpolate poly-rigid/affine transformations for which the invertibility is guaranteed. This powerful and flexible mathematical framework is currently being used to track the human joint dynamics by first imposing bone rigidity constraints in order to synthetize the spatio-temporal joint deformations later. However, since no closed-form exists, then a computationally expensive integration of ordinary differential equations (ODEs) is required to perform image registration using this framework. To tackle this problem, the exponential map for solving these ODEs is computed using the scaling and squaring method in the literature. In this paper, we propose an algorithm using a matrix diagonalization based method for smooth interpolation of homogeneous polyrigid transformations of human joints during motion. The use of this alternative computational approach to integrate ODEs is well motivated by the fact that bone rigid transformations satisfy the mechanical constraints of human joint motion, which provide conditions that guarantee the diagonalizability of local bone transformations and consequently of the resulting joint transformations. In a comparison with the scaling and squaring method, we discuss the usefulness of the matrix eigendecomposition technique which reduces significantly the computational burden associated with the computation of matrix exponential over a dense regular grid. Finally, we have applied the method to enhance the temporal resolution of dynamic MRI sequences of the ankle joint. To conclude, numerical experiments show that the eigendecomposition method is more capable of balancing the trade-off between accuracy, computation time, and memory requirements.
Transfer Learning for sEMG-based Hand Gesture Classification using Deep Learning in a Master-Slave Architecture
Authors: Karush Suri, Rinki Gupta
Date: 2020-04-27 00:00:00
Tasks: Diagnostic, General Classification, Transfer Learning

Recent advancements in diagnostic learning and development of gesture-based human machine interfaces have driven surface electromyography (sEMG) towards significant importance. Analysis of hand gestures requires an accurate assessment of sEMG signals. The proposed work presents a novel sequential master-slave architecture consisting of deep neural networks (DNNs) for classification of signs from the Indian sign language using signals recorded from multiple sEMG channels. The performance of the master-slave network is augmented by leveraging additional synthetic feature data generated by long short term memory networks. Performance of the proposed network is compared to that of a conventional DNN prior to and after the addition of synthetic data. Up to 14% improvement is observed in the conventional DNN and up to 9% improvement in master-slave network on addition of synthetic data with an average accuracy value of 93.5% asserting the suitability of the proposed approach.
Printing and Scanning Attack for Image Counter Forensics
Authors: Hailey James, Otkrist Gupta, Dan Raviv
Date: 2020-04-27 00:00:00
Tasks: Image Manipulation

Examining the authenticity of images has become increasingly important as manipulation tools become more accessible and advanced. Recent work has shown that while CNN-based image manipulation detectors can successfully identify manipulations, they are also vulnerable to adversarial attacks, ranging from simple double JPEG compression to advanced pixel-based perturbation. In this paper we explore another method of highly plausible attack: printing and scanning. We demonstrate the vulnerability of two state-of-the-art models to this type of attack. We also propose a new machine learning model that performs comparably to these state-of-the-art models when trained and validated on printed and scanned images. Of the three models, our proposed model outperforms the others when trained and validated on images from a single printer. To facilitate this exploration, we create a dataset of over 6,000 printed and scanned image blocks. Further analysis suggests that variation between images produced from different printers is significant, large enough that good validation accuracy on images from one printer does not imply similar validation accuracy on identical images from a different printer.
ExpDNN: Explainable Deep Neural Network
Authors: Chi-Hua Chen
Date: 2020-04-26 00:00:00
Tasks: regression

In recent years, deep neural networks have been applied to obtain high performance of prediction, classification, and pattern recognition. However, the weights in these deep neural networks are difficult to be explained. Although a linear regression method can provide explainable results, the method is not suitable in the case of input interaction. Therefore, an explainable deep neural network (ExpDNN) with explainable layers is proposed to obtain explainable results in the case of input interaction. Three cases were given to evaluate the proposed ExpDNN, and the results showed that the absolute value of weight in an explainable layer can be used to explain the weight of corresponding input for feature extraction.
Global Wheat Head Detection (GWHD) dataset: a large and diverse dataset of high resolution RGB labelled images to develop and benchmark wheat head detection methods
Authors: E. David, S. Madec, P. Sadeghi-Tehran, H. Aasen, B. Zheng, S. Liu, N. Kirchgessner, G. Ishikawa, K. Nagasawa, M. A. Badhon, C. Pozniak, B. de Solan, A. Hund, S. C. Chapman, F. Baret, I. Stavness, W. Guo
Date: 2020-04-25 00:00:00
Tasks: Benchmarking, Head Detection

Detection of wheat heads is an important task allowing to estimate pertinent traits including head population density and head characteristics such as sanitary state, size, maturity stage and the presence of awns. Several studies developed methods for wheat head detection from high-resolution RGB imagery. They are based on computer vision and machine learning and are generally calibrated and validated on limited datasets. However, variability in observational conditions, genotypic differences, development stages, head orientation represents a challenge in computer vision. Further, possible blurring due to motion or wind and overlap between heads for dense populations make this task even more complex. Through a joint international collaborative effort, we have built a large, diverse and well-labelled dataset, the Global Wheat Head detection (GWHD) dataset. It contains 4,700 high-resolution RGB images and 190,000 labelled wheat heads collected from several countries around the world at different growth stages with a wide range of genotypes. Guidelines for image acquisition, associating minimum metadata to respect FAIR principles and consistent head labelling methods are proposed when developing new head detection datasets. The GWHD is publicly available at http://www.global-wheat.com/ and aimed at developing and benchmarking methods for wheat head detection.
Detecting Electric Devices in 3D Images of Bags
Authors: Anthony Bagnall, Paul Southam, James Large, Richard Harvey
Date: 2020-04-25 00:00:00
Tasks: Computed Tomography (CT)

The aviation and transport security industries face the challenge of screening high volumes of baggage for threats and contraband in the minimum time possible. Automation and semi-automation of this procedure offers the potential to increase security by detecting more threats and improve the customer experience by speeding up the process. Traditional 2D x-ray images are often extremely difficult to examine due to the fact that they are tightly packed and contain a wide variety of cluttered and occluded objects. Because of these limitations, major airports are introducing 3D x-ray Computed Tomography (CT) baggage scanning. We investigate whether we can automate the process of detecting electric devices in these 3D images of luggage. Detecting electrical devices is of particular concern as they can be used to conceal explosives. Given the massive volume of luggage that needs to be screened for this threat, the best way to automate the detection is to first filter whether a bag contains an electric device or not, and if it does, to identify the number of devices and their location. We present an algorithm, Unpack, Predict, eXtract, Repack (UXPR), which involves unpacking through segmenting the data at a range of scales using an algorithm known as the Sieve, predicting whether a segment is electrical or not based on the histogram of voxel intensities, then repacking the bag by ensembling the segments and predictions to identify the devices in bags. Through a range of experiments using data provided by ALERT (Awareness and Localization of Explosives-Related Threats) we show that this system can find a high proportion of devices with unsupervised segmentation if a similar device has been seen before, and shows promising results for detecting devices not seen at all based on the properties of its constituent parts.
AutoTune: Automatically Tuning Convolutional Neural Networks for Improved Transfer Learning
Authors: S. H. Shabbeer Basha, Sravan Kumar Vinakota, Viswanath Pulabaigari, Snehasis Mukherjee, Shiv Ram Dubey
Date: 2020-04-25 00:00:00
Tasks: Bayesian Optimization, Transfer Learning

Transfer learning enables solving a specific task having limited data by using the pre-trained deep networks trained on large-scale datasets. Typically, while transferring the learned knowledge from source task to the target task, the last few layers are fine-tuned (re-trained) over the target dataset. However, these layers are originally designed for the source task that might not be suitable for the target task. In this paper, we introduce a mechanism for automatically tuning the Convolutional Neural Networks (CNN) for improved transfer learning. The pre-trained CNN layers are tuned with the knowledge from target data using Bayesian Optimization. First, we train the final layer of the base CNN model by replacing the number of neurons in the softmax layer with the number of classes involved in the target task. Next, the pre-trained CNN is tuned automatically by observing the classification performance on the validation data (greedy criteria). To evaluate the performance of the proposed method, experiments are conducted on three benchmark datasets, e.g., CalTech-101, CalTech-256, and Stanford Dogs. The classification results obtained through the proposed AutoTune method outperforms the standard baseline transfer learning methods over the three datasets by achieving $95.92\%$, $86.54\%$, and $84.67\%$ accuracy over CalTech-101, CalTech-256, and Stanford Dogs, respectively. The experimental results obtained in this study depict that tuning of the pre-trained CNN layers with the knowledge from the target dataset confesses better transfer learning ability. The source codes are available at https://github.com/JekyllAndHyde8999/AutoTune_CNN_TransferLearning.
PF-cpGAN: Profile to Frontal Coupled GAN for Face Recognition in the Wild
Authors: Fariborz Taherkhani, Veeru Talreja, Jeremy Dawson, Matthew C. Valenti, Nasser M. Nasrabadi
Date: 2020-04-25 00:00:00
Tasks: Face Recognition, Generative Adversarial Network, Retrieval

In recent years, due to the emergence of deep learning, face recognition has achieved exceptional success. However, many of these deep face recognition models perform relatively poorly in handling profile faces compared to frontal faces. The major reason for this poor performance is that it is inherently difficult to learn large pose invariant deep representations that are useful for profile face recognition. In this paper, we hypothesize that the profile face domain possesses a gradual connection with the frontal face domain in the deep feature space. We look to exploit this connection by projecting the profile faces and frontal faces into a common latent space and perform verification or retrieval in the latent domain. We leverage a coupled generative adversarial network (cpGAN) structure to find the hidden relationship between the profile and frontal images in a latent common embedding subspace. Specifically, the cpGAN framework consists of two GAN-based sub-networks, one dedicated to the frontal domain and the other dedicated to the profile domain. Each sub-network tends to find a projection that maximizes the pair-wise correlation between two feature domains in a common embedding feature subspace. The efficacy of our approach compared with the state-of-the-art is demonstrated using the CFP, CMU MultiPIE, IJB-A, and IJB-C datasets.
Computer Vision-Based Health Monitoring of Mecklenburg Bridge Using 3D Digital Image Correlation
Authors: Mehrdad S. Dizaji, Devin K. Harris, Bernie Kassner, Jeffrey C. Hill
Date: 2020-04-25 00:00:00

A collaborative investigation between the University of Virginia (UVA) and the Virginia Transportation Research Council was performed on the Mecklenburg Bridge (I-85 over Route 1 in Mecklenburg County). The research team aided the Virginia Department of Transportation - Richmond District in the characterization of the bridge behavior of one of the bridge beams that had been repaired due to a previous web buckling and crippling failure. The investigation focused on collecting full-field three-dimensional digital image correlation (3D-DIC) deformation measurements during the dropping sequence (removal of jacking to support beam on bearing/pier). Additionally, measurements were taken of the section prior to and after dropping using a handheld laser scanner to assess the potential of lateral deformation or out-of-plane buckling. Results from the study demonstrated that buckling of the tested beam did not occur, but did provided a series of approaches that can be used to evaluate the effectiveness of repaired steel beam ends. Specifically, the results provided an approach that could estimate the dead load distribution through back-calculation.
Facial Action Unit Detection on ICU Data for Pain Assessment
Authors: Subhash Nerella, Azra Bihorac, Patrick Tighe, Parisa Rashidi
Date: 2020-04-24 00:00:00
Tasks: Action Unit Detection, Facial Action Unit Detection

Current day pain assessment methods rely on patient self-report or by an observer like the Intensive Care Unit (ICU) nurses. Patient self-report is subjective to the individual and suffers due to poor recall. Pain assessment by manual observation is limited by the number of administrations per day and staff workload. Previous studies showed the feasibility of automatic pain assessment by detecting Facial Action Units (AUs). Pain is observed to be associated with certain facial action units (AUs). This method of pain assessment can overcome the pitfalls of present-day pain assessment techniques. All the previous studies are limited to controlled environment data. In this study, we evaluated the performance of OpenFace an open-source facial behavior analysis tool and AU R-CNN on the real-world ICU data. Presence of assisted breathing devices, variable lighting of ICUs, patient orientation with respect to camera significantly affected the performance of the models, although these showed the state-of-the-art results in facial behavior analysis tasks. In this study, we show the need for automated pain assessment system which is trained on real-world ICU data for clinically acceptable pain assessment system.
Prominent Attribute Modification using Attribute Dependent Generative Adversarial Network
Authors: Naeem Ul Islam, Sungmin Lee, Jaebyung Park
Date: 2020-04-24 00:00:00
Tasks: Attribute, Generative Adversarial Network

Modifying the facial images with desired attributes is important, though challenging tasks in computer vision, where it aims to modify single or multiple attributes of the face image. Some of the existing methods are either based on attribute independent approaches where the modification is done in the latent representation or attribute dependent approaches. The attribute independent methods are limited in performance as they require the desired paired data for changing the desired attributes. Secondly, the attribute independent constraint may result in the loss of information and, hence, fail in generating the required attributes in the face image. In contrast, the attribute dependent approaches are effective as these approaches are capable of modifying the required features along with preserving the information in the given image. However, attribute dependent approaches are sensitive and require a careful model design in generating high-quality results. To address this problem, we propose an attribute dependent face modification approach. The proposed approach is based on two generators and two discriminators that utilize the binary as well as the real representation of the attributes and, in return, generate high-quality attribute modification results. Experiments on the CelebA dataset show that our method effectively performs the multiple attribute editing with preserving other facial details intactly.
Expanding Sparse Guidance for Stereo Matching
Authors: Yu-Kai Huang, Yueh-Cheng Liu, Tsung-Han Wu, Hung-Ting Su, Winston H. Hsu
Date: 2020-04-24 00:00:00
Tasks: Domain Adaptation, Stereo Matching

The performance of image based stereo estimation suffers from lighting variations, repetitive patterns and homogeneous appearance. Moreover, to achieve good performance, stereo supervision requires sufficient densely-labeled data, which are hard to obtain. In this work, we leverage small amount of data with very sparse but accurate disparity cues from LiDAR to bridge the gap. We propose a novel sparsity expansion technique to expand the sparse cues concerning RGB images for local feature enhancement. The feature enhancement method can be easily applied to any stereo estimation algorithms with cost volume at the test stage. Extensive experiments on stereo datasets demonstrate the effectiveness and robustness across different backbones on domain adaption and self-supervision scenario. Our sparsity expansion method outperforms previous methods in terms of disparity by more than 2 pixel error on KITTI Stereo 2012 and 3 pixel error on KITTI Stereo 2015. Our approach significantly boosts the existing state-of-the-art stereo algorithms with extremely sparse cues.
Raccoons vs Demons: multiclass labeled P300 dataset
Authors: V. Goncharenko, R. Grigoryan, A. Samokhina
Date: 2020-04-22 00:00:00

We publish dataset of visual P300 BCI performed in Virtual Reality (VR) game Raccoons versus Demons (RvD). Data contains reach labels incorporating information about stimulus chosen enabling us to estimate model's confidence at each stimulus prediction stage. Data and experiments code are available at https://gitlab.com/impulse-neiry_public/raccoons-vs-demons