pwc-archive/papers-with-abstracts huggingface dataset:
Neural Code Summarization
Authors: Piyush Shrivastava
Date: 2021-02-26 00:00:00
Tasks: Code Summarization, Source Code Summarization, Transfer Learning

Code summarization is the task of generating readable summaries that are semantically meaningful and can accurately describe the presumed task of a software. Program comprehension has become one of the most tedious tasks for knowledge transfer. As the codebase evolves over time, the description needs to be manually updated each time with the changes made. An automatic approach is proposed to infer such captions based on benchmarked and custom datasets with comparison between the original and generated results.
If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques
Authors: Mark T Keane, Eoin M Kenny, Eoin Delaney, Barry Smyth
Date: 2021-02-26 00:00:00
Tasks: counterfactual, Counterfactual Explanation, Explainable Artificial Intelligence (XAI), Survey

In recent years, there has been an explosion of AI research on counterfactual explanations as a solution to the problem of eXplainable AI (XAI). These explanations seem to offer technical, psychological and legal benefits over other explanation techniques. We survey 100 distinct counterfactual explanation methods reported in the literature. This survey addresses the extent to which these methods have been adequately evaluated, both psychologically and computationally, and quantifies the shortfalls occurring. For instance, only 21% of these methods have been user tested. Five key deficits in the evaluation of these methods are detailed and a roadmap, with standardised benchmark evaluations, is proposed to resolve the issues arising; issues, that currently effectively block scientific progress in this field.
Optimal Linear Combination of Classifiers
Authors: Georgi Nalbantov, Svetoslav Ivanov
Date: 2021-03-01 00:00:00
Tasks: BIG-bench Machine Learning, General Classification

The question of whether to use one classifier or a combination of classifiers is a central topic in Machine Learning. We propose here a method for finding an optimal linear combination of classifiers derived from a bias-variance framework for the classification task.
Blockchain-Based Federated Learning in Mobile Edge Networks with Application in Internet of Vehicles
Authors: Rui Wang, Heju Li, Erwu Liu
Date: 2021-03-01 00:00:00
Tasks: Edge-computing, Federated Learning, Privacy Preserving

The rapid increase of the data scale in Internet of Vehicles (IoV) system paradigm, hews out new possibilities in boosting the service quality for the emerging applications through data sharing. Nevertheless, privacy concerns are major bottlenecks for data providers to share private data in traditional IoV networks. To this end, federated learning (FL) as an emerging learning paradigm, where data providers only send local model updates trained on their local raw data rather than upload any raw data, has been recently proposed to build a privacy-preserving data sharing models. Unfortunately, by analyzing on the differences of uploaded local model updates from data providers, private information can still be divulged, and performance of the system cannot be guaranteed when partial federated nodes executes malicious behavior. Additionally, traditional cloud-based FL poses challenges to the communication overhead with the rapid increase of terminal equipment in IoV system. All these issues inspire us to propose an autonomous blockchain empowered privacy-preserving FL framework in this paper, where the mobile edge computing (MEC) technology was naturally integrated in IoV system.
Deep Learning with a Classifier System: Initial Results
Authors: Richard J. Preen, Larry Bull
Date: 2021-03-01 00:00:00
Tasks: Deep Learning, Handwritten Digit Recognition

This article presents the first results from using a learning classifier system capable of performing adaptive computation with deep neural networks. Individual classifiers within the population are composed of two neural networks. The first acts as a gating or guarding component, which enables the conditional computation of an associated deep neural network on a per instance basis. Self-adaptive mutation is applied upon reproduction and prediction networks are refined with stochastic gradient descent during lifetime learning. The use of fully-connected and convolutional layers are evaluated on handwritten digit recognition tasks where evolution adapts (i) the gradient descent learning rate applied to each layer (ii) the number of units within each layer, i.e., the number of fully-connected neurons and the number of convolutional kernel filters (iii) the connectivity of each layer, i.e., whether each weight is active (iv) the weight magnitudes, enabling escape from local optima. The system automatically reduces the number of weights and units while maintaining performance after achieving a maximum prediction error.
Statistically Significant Stopping of Neural Network Training
Authors: J. K. Terry, Mario Jayakumar, Kusal De Alwis
Date: 2021-03-01 00:00:00

The general approach taken when training deep learning classifiers is to save the parameters after every few iterations, train until either a human observer or a simple metric-based heuristic decides the network isn't learning anymore, and then backtrack and pick the saved parameters with the best validation accuracy. Simple methods are used to determine if a neural network isn't learning anymore because, as long as it's well after the optimal values are found, the condition doesn't impact the final accuracy of the model. However from a runtime perspective, this is of great significance to the many cases where numerous neural networks are trained simultaneously (e.g. hyper-parameter tuning). Motivated by this, we introduce a statistical significance test to determine if a neural network has stopped learning. This stopping criterion appears to represent a happy medium compared to other popular stopping criterions, achieving comparable accuracy to the criterions that achieve the highest final accuracies in 77% or fewer epochs, while the criterions which stop sooner do so with an appreciable loss to final accuracy. Additionally, we use this as the basis of a new learning rate scheduler, removing the need to manually choose learning rate schedules and acting as a quasi-line search, achieving superior or comparable empirical performance to existing methods.
Between Post-Flaneur and Smartphone Zombie Smartphone Users Altering Visual Attention and Walking Behavior in Public Space
Authors: Gorsev Argin, Burak Pak, Handan Turkoglu
Date: 2021-02-26 00:00:00

The extensive use of smartphones in our everyday lives has created new modes of appropriation and behavior in public spaces. Recognition of these are essential for urban design and planning practices which help us to improve the relationship between humans, technologies, and urban environment. This study aims to research smartphone users in public space by observing their altering visual attention and walking behavior, and, in this way, to reveal the emergent new figures. For this purpose, Korenmarkt square in Ghent, Belgium, was observed for seven days in 10-min time intervals. The gaze and walking behavior of smartphone users were encoded as geo-located and temporal data, analyzed and mapped using statistical and spatial analysis methods. Developing and implementing new methods for identifying the characteristics of smartphone users, this study resulted in a nuanced characterization of novel spatial appropriations. The findings led to a better understanding and knowledge of the different behavior patterns of emergent figures such as post-flaneurs and smartphone zombies while uncovering their altering visual interactions with and movements in the public space. The results evoked questions on how researchers and designers can make use of spatial analysis methods and rethink the public space of the future as a hybrid construct integrating the virtual and the physical.
Learning Transferable Visual Models From Natural Language Supervision
Authors: Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever
Date: 2021-02-26 00:00:00
Tasks: Action Recognition, Benchmarking, Few-Shot Image Classification, geo-localization, Hateful Meme Classification, Image Classification, Image-to-Text Retrieval, Long-tail Learning, Meme Classification, Natural Language Understanding, Object Categorization, Object Recognition, Open Vocabulary Attribute Detection, Out-of-Distribution Generalization, Preference Mapping, Prompt Engineering, Semi-Supervised Image Classification, Temporal Relation Extraction, Text-based Person Retrieval with Noisy Correspondence, Text Generation, Visual Reasoning, Zero-Shot Cross-Modal Retrieval, Zero-Shot Learning, Zero-shot Text-to-Image Retrieval, Zero-Shot Transfer Image Classification

State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories. This restricted form of supervision limits their generality and usability since additional labeled data is needed to specify any other visual concept. Learning directly from raw text about images is a promising alternative which leverages a much broader source of supervision. We demonstrate that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet. After pre-training, natural language is used to reference learned visual concepts (or describe new ones) enabling zero-shot transfer of the model to downstream tasks. We study the performance of this approach by benchmarking on over 30 different existing computer vision datasets, spanning tasks such as OCR, action recognition in videos, geo-localization, and many types of fine-grained object classification. The model transfers non-trivially to most tasks and is often competitive with a fully supervised baseline without the need for any dataset specific training. For instance, we match the accuracy of the original ResNet-50 on ImageNet zero-shot without needing to use any of the 1.28 million training examples it was trained on. We release our code and pre-trained model weights at https://github.com/OpenAI/CLIP.
Recursive Training for Zero-Shot Semantic Segmentation
Authors: Ce Wang, Moshiur Farazi, Nick Barnes
Date: 2021-02-26 00:00:00
Tasks: Segmentation, Semantic Segmentation, Zero-Shot Semantic Segmentation

General purpose semantic segmentation relies on a backbone CNN network to extract discriminative features that help classify each image pixel into a 'seen' object class (ie., the object classes available during training) or a background class. Zero-shot semantic segmentation is a challenging task that requires a computer vision model to identify image pixels belonging to an object class which it has never seen before. Equipping a general purpose semantic segmentation model to separate image pixels of 'unseen' classes from the background remains an open challenge. Some recent models have approached this problem by fine-tuning the final pixel classification layer of a semantic segmentation model for a Zero-Shot setting, but struggle to learn discriminative features due to the lack of supervision. We propose a recursive training scheme to supervise the retraining of a semantic segmentation model for a zero-shot setting using a pseudo-feature representation. To this end, we propose a Zero-Shot Maximum Mean Discrepancy (ZS-MMD) loss that weighs high confidence outputs of the pixel classification layer as a pseudo-feature representation, and feeds it back to the generator. By closing-the-loop on the generator end, we provide supervision during retraining that in turn helps the model learn a more discriminative feature representation for 'unseen' classes. We show that using our recursive training and ZS-MMD loss, our proposed model achieves state-of-the-art performance on the Pascal-VOC 2012 dataset and Pascal-Context dataset.
ASMNet: a Lightweight Deep Neural Network for Face Alignment and Pose Estimation
Authors: Ali Pourramezan Fard, Hojjat Abdollahi, Mohammad Mahoor
Date: 2021-02-27 00:00:00
Tasks: Face Alignment, Head Pose Estimation, Pose Estimation, Transfer Learning

Active Shape Model (ASM) is a statistical model of object shapes that represents a target structure. ASM can guide machine learning algorithms to fit a set of points representing an object (e.g., face) onto an image. This paper presents a lightweight Convolutional Neural Network (CNN) architecture with a loss function being assisted by ASM for face alignment and estimating head pose in the wild. We use ASM to first guide the network towards learning a smoother distribution of the facial landmark points. Inspired by transfer learning, during the training process, we gradually harden the regression problem and guide the network towards learning the original landmark points distribution. We define multi-tasks in our loss function that are responsible for detecting facial landmark points as well as estimating the face pose. Learning multiple correlated tasks simultaneously builds synergy and improves the performance of individual tasks. We compare the performance of our proposed model called ASMNet with MobileNetV2 (which is about 2 times bigger than ASMNet) in both the face alignment and pose estimation tasks. Experimental results on challenging datasets show that by using the proposed ASM assisted loss function, the ASMNet performance is comparable with MobileNetV2 in the face alignment task. In addition, for face pose estimation, ASMNet performs much better than MobileNetV2. ASMNet achieves an acceptable performance for facial landmark points detection and pose estimation while having a significantly smaller number of parameters and floating-point operations compared to many CNN-based models.
Successive Subspace Learning: An Overview
Authors: Mozhdeh Rouhsedaghat, Masoud Monajatipoor, Zohreh Azizi, C. -C. Jay Kuo
Date: 2021-02-27 00:00:00

Successive Subspace Learning (SSL) offers a light-weight unsupervised feature learning method based on inherent statistical properties of data units (e.g. image pixels and points in point cloud sets). It has shown promising results, especially on small datasets. In this paper, we intuitively explain this method, provide an overview of its development, and point out some open questions and challenges for future research.
PRISM: A Rich Class of Parameterized Submodular Information Measures for Guided Subset Selection
Authors: Suraj Kothawade, Vishal Kaushal, Ganesh Ramakrishnan, Jeff Bilmes, Rishabh Iyer
Date: 2021-02-27 00:00:00
Tasks: Image Classification

With ever-increasing dataset sizes, subset selection techniques are becoming increasingly important for a plethora of tasks. It is often necessary to guide the subset selection to achieve certain desiderata, which includes focusing or targeting certain data points, while avoiding others. Examples of such problems include: i)targeted learning, where the goal is to find subsets with rare classes or rare attributes on which the model is underperforming, and ii)guided summarization, where data (e.g., image collection, text, document or video) is summarized for quicker human consumption with specific additional user intent. Motivated by such applications, we present PRISM, a rich class of PaRameterIzed Submodular information Measures. Through novel functions and their parameterizations, PRISM offers a variety of modeling capabilities that enable a trade-off between desired qualities of a subset like diversity or representation and similarity/dissimilarity with a set of data points. We demonstrate how PRISM can be applied to the two real-world problems mentioned above, which require guided subset selection. In doing so, we show that PRISM interestingly generalizes some past work, therein reinforcing its broad utility. Through extensive experiments on diverse datasets, we demonstrate the superiority of PRISM over the state-of-the-art in targeted learning and in guided image-collection summarization
Disentangling Geometric Deformation Spaces in Generative Latent Shape Models
Authors: Tristan Aumentado-Armstrong, Stavros Tsogkas, Sven Dickinson, Allan Jepson
Date: 2021-02-27 00:00:00
Tasks: Disentanglement, Pose Transfer, Representation Learning, Retrieval

A complete representation of 3D objects requires characterizing the space of deformations in an interpretable manner, from articulations of a single instance to changes in shape across categories. In this work, we improve on a prior generative model of geometric disentanglement for 3D shapes, wherein the space of object geometry is factorized into rigid orientation, non-rigid pose, and intrinsic shape. The resulting model can be trained from raw 3D shapes, without correspondences, labels, or even rigid alignment, using a combination of classical spectral geometry and probabilistic disentanglement of a structured latent representation space. Our improvements include more sophisticated handling of rotational invariance and the use of a diffeomorphic flow network to bridge latent and spectral space. The geometric structuring of the latent space imparts an interpretable characterization of the deformation space of an object. Furthermore, it enables tasks like pose transfer and pose-aware retrieval without requiring supervision. We evaluate our model on its generative modelling, representation learning, and disentanglement performance, showing improved rotation invariance and intrinsic-extrinsic factorization quality over the prior model.
Pedestrian Motion State Estimation From 2D Pose
Authors: Fei Li, Shiwei Fan, Pengzhen Chen, Xiangxu Li
Date: 2021-02-27 00:00:00
Tasks: State Estimation

Traffic violation and the flexible and changeable nature of pedestrians make it more difficult to predict pedestrian behavior or intention, which might be a potential safety hazard on the road. Pedestrian motion state (such as walking and standing) directly affects or reflects its intention. In combination with pedestrian motion state and other influencing factors, pedestrian intention can be predicted to avoid unnecessary accidents. In this paper, pedestrian is treated as non-rigid object, which can be represented by a set of two-dimensional key points, and the movement of key point relative to the torso is introduced as micro motion. Static and dynamic micro motion features, such as position, angle and distance, and their differential calculations in time domain, are used to describe its motion pattern. Gated recurrent neural network based seq2seq model is used to learn the dependence of motion state transition on previous information, finally the pedestrian motion state is estimated via a softmax classifier. The proposed method only needs the previous hidden state of GRU and current feature to evaluate the probability of current motion state, and it is computation efficient to deploy on vehicles. This paper verifies the proposed algorithm on the JAAD public dataset, and the accuracy is improved by 11.6% compared with the existing method.
FisheyeSuperPoint: Keypoint Detection and Description Network for Fisheye Images
Authors: Anna Konrad, Ciarán Eising, Ganesh Sistu, John McDonald, Rudi Villing, Senthil Yogamani
Date: 2021-02-27 00:00:00
Tasks: Autonomous Driving, Homography Estimation, Keypoint Detection

Keypoint detection and description is a commonly used building block in computer vision systems particularly for robotics and autonomous driving. However, the majority of techniques to date have focused on standard cameras with little consideration given to fisheye cameras which are commonly used in urban driving and automated parking. In this paper, we propose a novel training and evaluation pipeline for fisheye images. We make use of SuperPoint as our baseline which is a self-supervised keypoint detector and descriptor that has achieved state-of-the-art results on homography estimation. We introduce a fisheye adaptation pipeline to enable training on undistorted fisheye images. We evaluate the performance on the HPatches benchmark, and, by introducing a fisheye based evaluation method for detection repeatability and descriptor matching correctness, on the Oxford RobotCar dataset.
Remote Sensing Image Change Detection with Transformers
Authors: Hao Chen, Zipeng Qi, Zhenwei Shi
Date: 2021-02-27 00:00:00
Tasks: Building change detection for remote sensing images, Change Detection

Modern change detection (CD) has achieved remarkable success by the powerful discriminative ability of deep convolutions. However, high-resolution remote sensing CD remains challenging due to the complexity of objects in the scene. Objects with the same semantic concept may show distinct spectral characteristics at different times and spatial locations. Most recent CD pipelines using pure convolutions are still struggling to relate long-range concepts in space-time. Non-local self-attention approaches show promising performance via modeling dense relations among pixels, yet are computationally inefficient. Here, we propose a bitemporal image transformer (BIT) to efficiently and effectively model contexts within the spatial-temporal domain. Our intuition is that the high-level concepts of the change of interest can be represented by a few visual words, i.e., semantic tokens. To achieve this, we express the bitemporal image into a few tokens, and use a transformer encoder to model contexts in the compact token-based space-time. The learned context-rich tokens are then feedback to the pixel-space for refining the original features via a transformer decoder. We incorporate BIT in a deep feature differencing-based CD framework. Extensive experiments on three CD datasets demonstrate the effectiveness and efficiency of the proposed method. Notably, our BIT-based model significantly outperforms the purely convolutional baseline using only 3 times lower computational costs and model parameters. Based on a naive backbone (ResNet18) without sophisticated structures (e.g., FPN, UNet), our model surpasses several state-of-the-art CD methods, including better than four recent attention-based methods in terms of efficiency and accuracy. Our code is available at https://github.com/justchenhao/BIT\_CD.