pwc-archive/papers-with-abstracts huggingface dataset:
Adaptive and Iteratively Improving Recurrent Lateral Connections
Authors: Barak Battash, Lior Wolf
Date: 2019-10-16 00:00:00
Tasks: Action Recognition

The current leading computer vision models are typically feed forward neural models, in which the output of one computational block is passed to the next one sequentially. This is in sharp contrast to the organization of the primate visual cortex, in which feedback and lateral connections are abundant. In this work, we propose a computational model for the role of lateral connections in a given block, in which the weights of the block vary dynamically as a function of its activations, and the input from the upstream blocks is iteratively reintroduced. We demonstrate how this novel architectural modification can lead to sizable gains in performance, when applied to visual action recognition without pretraining and that it outperforms the literature architectures with recurrent feedback processing on ImageNet.
Label-Conditioned Next-Frame Video Generation with Neural Flows
Authors: David Donahue
Date: 2019-10-16 00:00:00
Tasks: Video Generation

Recent state-of-the-art video generation systems employ Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs) to produce novel videos. However, VAE models typically produce blurry outputs when faced with sub-optimal conditioning of the input, and GANs are known to be unstable for large output sizes. In addition, the output videos of these models are difficult to evaluate, partly because the GAN loss function is not an accurate measure of convergence. In this work, we propose using a state-of-the-art neural flow generator called Glow to generate videos conditioned on a textual label, one frame at a time. Neural flow models are more stable than standard GANs, as they only optimize a single cross entropy loss function, which is monotonic and avoids the circular convergence issues of the GAN minimax objective. In addition, we also show how to condition Glow on external context, while still preserving the invertible nature of each "flow" layer. Finally, we evaluate the proposed Glow model by calculating cross entropy on a held-out validation set of videos, in order to compare multiple versions of the proposed model via an ablation study. We show generated videos and discuss future improvements.
Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network
Authors: Dimitrios Kollias, Viktoriia Sharmanska, Stefanos Zafeiriou
Date: 2019-10-15 00:00:00
Tasks: Emotion Recognition, Few-Shot Learning

Automatic facial behavior analysis has a long history of studies in the intersection of computer vision, physiology and psychology. However it is only recently, with the collection of large-scale datasets and powerful machine learning methods such as deep neural networks, that automatic facial behavior analysis started to thrive. Three of its iconic tasks are automatic recognition of basic expressions (e.g. happy, sad, surprised), estimation of continuous emotions (e.g., valence and arousal), and detection of facial action units (activations of e.g. upper/inner eyebrows, nose wrinkles). Up until now these tasks have been mostly studied independently collecting a dataset for the task. We present the first and the largest study of all facial behaviour tasks learned jointly in a single multi-task, multi-domain and multi-label network, which we call FaceBehaviorNet. For this we utilize all publicly available datasets in the community (around 5M images) that study facial behaviour tasks in-the-wild. We demonstrate that training jointly an end-to-end network for all tasks has consistently better performance than training each of the single-task networks. Furthermore, we propose two simple strategies for coupling the tasks during training, co-annotation and distribution matching, and show the advantages of this approach. Finally we show that FaceBehaviorNet has learned features that encapsulate all aspects of facial behaviour, and can be successfully applied to perform tasks (compound emotion recognition) beyond the ones that it has been trained in a zero- and few-shot learning setting.
Supervised Encoding for Discrete Representation Learning
Authors: Cat P. Le, Yi Zhou, Jie Ding, Vahid Tarokh
Date: 2019-10-15 00:00:00
Tasks: Decoder, Representation Learning, Style Transfer

Classical supervised classification tasks search for a nonlinear mapping that maps each encoded feature directly to a probability mass over the labels. Such a learning framework typically lacks the intuition that encoded features from the same class tend to be similar and thus has little interpretability for the learned features. In this paper, we propose a novel supervised learning model named Supervised-Encoding Quantizer (SEQ). The SEQ applies a quantizer to cluster and classify the encoded features. We found that the quantizer provides an interpretable graph where each cluster in the graph represents a class of data samples that have a particular style. We also trained a decoder that can decode convex combinations of the encoded features from similar and different clusters and provide guidance on style transfer between sub-classes.
Scale-Equivariant Steerable Networks
Authors: Ivan Sosnovik, Michał Szmaja, Arnold Smeulders
Date: 2019-10-14 00:00:00
Proceeding: ICLR 2020 1
Tasks: Computational Efficiency, Image Classification, Translation

The effectiveness of Convolutional Neural Networks (CNNs) has been substantially attributed to their built-in property of translation equivariance. However, CNNs do not have embedded mechanisms to handle other types of transformations. In this work, we pay attention to scale changes, which regularly appear in various tasks due to the changing distances between the objects and the camera. First, we introduce the general theory for building scale-equivariant convolutional networks with steerable filters. We develop scale-convolution and generalize other common blocks to be scale-equivariant. We demonstrate the computational efficiency and numerical stability of the proposed method. We compare the proposed models to the previously developed methods for scale equivariance and local scale invariance. We demonstrate state-of-the-art results on MNIST-scale dataset and on STL-10 dataset in the supervised learning setting.
Spatial and Colour Opponency in Anatomically Constrained Deep Networks
Authors: Ethan Harris, Daniela Mihai, Jonathon Hare
Date: 2019-10-14 00:00:00

Colour vision has long fascinated scientists, who have sought to understand both the physiology of the mechanics of colour vision and the psychophysics of colour perception. We consider representations of colour in anatomically constrained convolutional deep neural networks. Following ideas from neuroscience, we classify cells in early layers into groups relating to their spectral and spatial functionality. We show the emergence of single and double opponent cells in our networks and characterise how the distribution of these cells changes under the constraint of a retinal bottleneck. Our experiments not only open up a new understanding of how deep networks process spatial and colour information, but also provide new tools to help understand the black box of deep learning. The code for all experiments is avaialable at \url{https://github.com/ecs-vlc/opponency}.
Interpretable Text Classification Using CNN and Max-pooling
Authors: Hao Cheng, Xiaoqing Yang, Zang Li, Yanghua Xiao, Yu-Cheng Lin
Date: 2019-10-14 00:00:00
Tasks: Classification, General Classification, Sentence, text-classification, Text Classification

Deep neural networks have been widely used in text classification. However, it is hard to interpret the neural models due to the complicate mechanisms. In this work, we study the interpretability of a variant of the typical text classification model which is based on convolutional operation and max-pooling layer. Two mechanisms: convolution attribution and n-gram feature analysis are proposed to analyse the process procedure for the CNN model. The interpretability of the model is reflected by providing posterior interpretation for neural network predictions. Besides, a multi-sentence strategy is proposed to enable the model to beused in multi-sentence situation without loss of performance and interpret ability. We evaluate the performance of the model on several classification tasks and justify the interpretable performance with some case studies.
Real-World Image Datasets for Federated Learning
Authors: Jiahuan Luo, Xueyang Wu, Yun Luo, Anbu Huang, Yun-Feng Huang, Yang Liu, Qiang Yang
Date: 2019-10-14 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning, object-detection

Federated learning is a new machine learning paradigm which allows data parties to build machine learning models collaboratively while keeping their data secure and private. While research efforts on federated learning have been growing tremendously in the past two years, most existing works still depend on pre-existing public datasets and artificial partitions to simulate data federations due to the lack of high-quality labeled data generated from real-world edge applications. Consequently, advances on benchmark and model evaluations for federated learning have been lagging behind. In this paper, we introduce a real-world image dataset. The dataset contains more than 900 images generated from 26 street cameras and 7 object categories annotated with detailed bounding box. The data distribution is non-IID and unbalanced, reflecting the characteristic real-world federated learning scenarios. Based on this dataset, we implemented two mainstream object detection algorithms (YOLO and Faster R-CNN) and provided an extensive benchmark on model performance, efficiency, and communication in a federated learning setting. Both the dataset and algorithms are made publicly available.
Rethinking Exposure Bias In Language Modeling
Authors: Yifan Xu, Kening Zhang, Haoyu Dong, Yuezhou Sun, Wenlong Zhao, Zhuowen Tu
Date: 2019-10-13 00:00:00
Tasks: Language Modeling, Language Modelling, Reinforcement Learning, Reinforcement Learning (RL)

Exposure bias describes the phenomenon that a language model trained under the teacher forcing schema may perform poorly at the inference stage when its predictions are conditioned on its previous predictions unseen from the training corpus. Recently, several generative adversarial networks (GANs) and reinforcement learning (RL) methods have been introduced to alleviate this problem. Nonetheless, a common issue in RL and GANs training is the sparsity of reward signals. In this paper, we adopt two simple strategies, multi-range reinforcing, and multi-entropy sampling, to amplify and denoise the reward signal. Our model produces an improvement over competing models with regards to BLEU scores and road exam, a new metric we designed to measure the robustness against exposure bias in language models.
DeepPCO: End-to-End Point Cloud Odometry through Deep Parallel Neural Network
Authors: Wei Wang, Muhamad Risqi U. Saputra, Peijun Zhao, Pedro Gusmao, Bo Yang, Changhao Chen, Andrew Markham, Niki Trigoni
Date: 2019-10-13 00:00:00
Tasks: Translation, Visual Odometry

Odometry is of key importance for localization in the absence of a map. There is considerable work in the area of visual odometry (VO), and recent advances in deep learning have brought novel approaches to VO, which directly learn salient features from raw images. These learning-based approaches have led to more accurate and robust VO systems. However, they have not been well applied to point cloud data yet. In this work, we investigate how to exploit deep learning to estimate point cloud odometry (PCO), which may serve as a critical component in point cloud-based downstream tasks or learning-based systems. Specifically, we propose a novel end-to-end deep parallel neural network called DeepPCO, which can estimate the 6-DOF poses using consecutive point clouds. It consists of two parallel sub-networks to estimate 3-D translation and orientation respectively rather than a single neural network. We validate our approach on KITTI Visual Odometry/SLAM benchmark dataset with different baselines. Experiments demonstrate that the proposed approach achieves good performance in terms of pose accuracy.
Self-enhancement of automatic tunnel accident detection (TAD) on CCTV by AI deep-learning
Authors: Kyu-Beom Lee, Hyu-Soung Shin
Date: 2019-10-11 00:00:00
Tasks: Deep Learning

The deep-learning-based tunnel accident detection (TAD) system (Lee 2019) has installed a system capable of monitoring 9 CCTVs at XX site in November, 2018. The initial deep-learning training was started by studying 70,914 labeled images and label data. However, sunlight, the tail light of a vehicle, and the warning light of the working vehicle were recognized as a fire, and many pedestrians were detected in the lane of the tunnel or a black elongated black object. To solve these problems, as shown in Fig. 1, the false detection data detected in the field were trained with labeled data and reapplied in the field. As a result, false detection of pedestrians and fire could be significantly reduced.
Towards DeepSpray: Using Convolutional Neural Network to post-process Shadowgraphy Images of Liquid Atomization
Authors: Geoffroy Chaussonnet, Christian Lieber, Yan Yikang, Wenda Gu, Andreas Bartschat, Markus Reischl, Rainer Koch, Ralf Mikut, Hans-Jörg Bauer
Date: 2019-10-11 00:00:00
Tasks: Data Augmentation

This technical report investigates the potential of Convolutional Neural Networks to post-process images from primary atomization. Three tasks are investigated. First, the detection and segmentation of liquid droplets in degraded optical conditions. Second, the detection of overlapping ellipses and the prediction of their geometrical characteristics. This task corresponds to extrapolate the hidden contour of an ellipse with reduced visual information. Third, several features of the liquid surface during primary breakup (ligaments, bags, rims) are manually annotated on 15 experimental images. The detector is trained on this minimal database using simple data augmentation and then applied to other images from numerical simulation and from other experiment. In these three tasks, models from the literature based on Convolutional Neural Networks showed very promising results, thus demonstrating the high potential of Deep Learning to post-process liquid atomization. The next step is to embed these models into a unified framework DeepSpray.
Emotion Generation and Recognition: A StarGAN Approach
Authors: Aritra Banerjee, Dimitrios Kollias
Date: 2019-10-12 00:00:00
Tasks: 4k, Emotion Recognition, Face Detection

The main idea of this ISO is to use StarGAN (A type of GAN model) to perform training and testing on an emotion dataset resulting in a emotion recognition which can be generated by the valence arousal score of the 7 basic expressions. We have created an entirely new dataset consisting of 4K videos. This dataset consists of all the basic 7 types of emotions: Happy, Sad, Angry, Surprised, Fear, Disgust, Neutral. We have performed face detection and alignment followed by annotating basic valence arousal values to the frames/images in the dataset depending on the emotions manually. Then the existing StarGAN model is trained on our created dataset after which some manual subjects were chosen to test the efficiency of the trained StarGAN model.
DeepACEv2: Automated Chromosome Enumeration in Metaphase Cell Images Using Deep Convolutional Neural Networks
Authors: Li Xiao, Chunlong Luo, Tianqi Yu, Yufan Luo, Manqing Wang, Fuhai Yu, Yinhao Li, Chan Tian, Jie Qiao
Date: 2019-10-12 00:00:00
Tasks: object-detection, Object Detection, Region Proposal

Chromosome enumeration is an essential but tedious procedure in karyotyping analysis. To automate the enumeration process, we develop a chromosome enumeration framework, DeepACEv2, based on the region based object detection scheme. The framework is developed following three steps. Firstly, we take the classical ResNet-101 as the backbone and attach the Feature Pyramid Network (FPN) to the backbone. The FPN takes full advantage of the multiple level features, and we only output the level of feature map that most of the chromosomes are assigned to. Secondly, we enhance the region proposal network's ability by adding a newly proposed Hard Negative Anchors Sampling to extract unapparent but essential information about highly confusing partial chromosomes. Next, to alleviate serious occlusion problems, besides the traditional detection branch, we novelly introduce an isolated Template Module branch to extract unique embeddings of each proposal by utilizing the chromosome's geometric information. The embeddings are further incorporated into the No Maximum Suppression (NMS) procedure to improve the detection of overlapping chromosomes. Finally, we design a Truncated Normalized Repulsion Loss and add it to the loss function to avoid inaccurate localization caused by occlusion. In the newly collected 1375 metaphase images that came from a clinical laboratory, a series of ablation studies validate the effectiveness of each proposed module. Combining them, the proposed DeepACEv2 outperforms all the previous methods, yielding the Whole Correct Ratio(WCR)(%) with respect to images as 71.39, and the Average Error Ratio(AER)(%) with respect to chromosomes as about 1.17.
An application of a deep learning algorithm for automatic detection of unexpected accidents under bad CCTV monitoring conditions in tunnels
Authors: Kyu-Beom Lee, Hyu-Soung Shin
Date: 2019-10-11 00:00:00
Tasks: Deep Learning, Object, object-detection, Object Detection, Object Tracking

In this paper, Object Detection and Tracking System (ODTS) in combination with a well-known deep learning network, Faster Regional Convolution Neural Network (Faster R-CNN), for Object Detection and Conventional Object Tracking algorithm will be introduced and applied for automatic detection and monitoring of unexpected events on CCTVs in tunnels, which are likely to (1) Wrong-Way Driving (WWD), (2) Stop, (3) Person out of vehicle in tunnel (4) Fire. ODTS accepts a video frame in time as an input to obtain Bounding Box (BBox) results by Object Detection and compares the BBoxs of the current and previous video frames to assign a unique ID number to each moving and detected object. This system makes it possible to track a moving object in time, which is not usual to be achieved in conventional object detection frameworks. A deep learning model in ODTS was trained with a dataset of event images in tunnels to Average Precision (AP) values of 0.8479, 0.7161 and 0.9085 for target objects: Car, Person, and Fire, respectively. Then, based on trained deep learning model, the ODTS based Tunnel CCTV Accident Detection System was tested using four accident videos which including each accident. As a result, the system can detect all accidents within 10 seconds. The more important point is that the detection capacity of ODTS could be enhanced automatically without any changes in the program codes as the training dataset becomes rich.
BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search
Authors: Colin White, Willie Neiswanger, Yash Savani
Date: 2019-10-25 00:00:00
Tasks: Bayesian Optimization, Hyperparameter Optimization, Neural Architecture Search, Reinforcement Learning

Over the past half-decade, many methods have been considered for neural architecture search (NAS). Bayesian optimization (BO), which has long had success in hyperparameter optimization, has recently emerged as a very promising strategy for NAS when it is coupled with a neural predictor. Recent work has proposed different instantiations of this framework, for example, using Bayesian neural networks or graph convolutional networks as the predictive model within BO. However, the analyses in these papers often focus on the full-fledged NAS algorithm, so it is difficult to tell which individual components of the framework lead to the best performance. In this work, we give a thorough analysis of the "BO + neural predictor" framework by identifying five main components: the architecture encoding, neural predictor, uncertainty calibration method, acquisition function, and acquisition optimization strategy. We test several different methods for each component and also develop a novel path-based encoding scheme for neural architectures, which we show theoretically and empirically scales better than other encodings. Using all of our analyses, we develop a final algorithm called BANANAS, which achieves state-of-the-art performance on NAS search spaces. We adhere to the NAS research checklist (Lindauer and Hutter 2019) to facilitate best practices, and our code is available at https://github.com/naszilla/naszilla.
Mobile Robot Navigation Using Fuzzy-GA Approaches Along with Three Path Concept
Authors: Ngangbam Herojit Singh1 • Khelchandra Thongam1
Date: 2019-06-01 00:00:00
Proceeding: Iranian Journal of Science and Technology, Transactions of Electrical Engineering 2019 6
Tasks: Robot Navigation

This paper presents a technique of navigation of a mobile robot using fuzzy computing and genetic algorithm along with three path concept. The information about the distances and angles of obstacles from the robot is acquired by using the concept of three paths. Fuzzy system is used to avoid obstacles when all the three paths are blocked by obstacles; otherwise, the collision-free path is selected by using three path method. Genetic algorithm is used to find optimal range of the linguistic values of the variables for the membership functions. Results show that the method of using fuzzy-GA along with the three path concept is computationally efficient as compared to other hybrid methods.
PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs
Authors: Yunbo Wang Mingsheng Long Jianmin Wang Zhifeng Gao Philip S. Yu
Date: 2017-12-01 00:00:00
Proceeding: Conference 2017 12
Tasks: Video Prediction

The predictive learning of spatiotemporal sequences aims to generate future images by learning from the historical frames, where spatial appearances and temporal variations are two crucial structures. This paper models these structures by presenting a predictive recurrent neural network (PredRNN). This architecture is enlightened by the idea that spatiotemporal predictive learning should memorize both spatial appearances and temporal variations in a unified memory pool. Concretely, memory states are no longer constrained inside each LSTM unit. Instead, they are allowed to zigzag in two directions: across stacked RNN layers vertically and through all RNN states horizontally. The core of this network is a new Spatiotemporal LSTM (ST-LSTM) unit that extracts and memorizes spatial and temporal representations simultaneously. PredRNN achieves the state-of-the-art prediction performance on three video prediction datasets and is a more general framework, that can be easily extended to other predictive learning tasks by integrating with other architectures.
Bias-Variance Tradeoff in a Sliding Window Implementation of the Stochastic Gradient Algorithm
Authors: Yakup Ceki Papo
Date: 2019-10-25 00:00:00

This paper provides a framework to analyze stochastic gradient algorithms in a mean squared error (MSE) sense using the asymptotic normality result of the stochastic gradient descent (SGD) iterates. We perform this analysis by taking the asymptotic normality result and applying it to the finite iteration case. Specifically, we look at problems where the gradient estimators are biased and have reduced variance and compare the iterates generated by these gradient estimators to the iterates generated by the SGD algorithm. We use the work of Fabian to characterize the mean and the variance of the distribution of the iterates in terms of the bias and the covariance matrix of the gradient estimators. We introduce the sliding window SGD (SW-SGD) algorithm, with its proof of convergence, which incurs a lower MSE than the SGD algorithm on quadratic and convex problems. Lastly, we present some numerical results to show the effectiveness of this framework and the superiority of SW-SGD algorithm over the SGD algorithm.