pwc-archive/papers-with-abstracts huggingface dataset:
LeagueAI: Improving object detector performance and flexibility through automatically generated training data and domain randomization
Authors: Oliver Struckmeier
Date: 2019-05-28 00:00:00
Tasks: Dataset Generation, object-detection, Object Detection

In this technical report I present my method for automatic synthetic dataset generation for object detection and demonstrate it on the video game League of Legends. This report furthermore serves as a handbook on how to automatically generate datasets and as an introduction on the dataset generation part of the LeagueAI framework. The LeagueAI framework is a software framework that provides detailed information about the game League of Legends based on the same input a human player would have, namely vision. The framework allows researchers and enthusiasts to develop their own intelligent agents or to extract detailed information about the state of the game. A big problem of machine vision applications usually is the laborious work of gathering large amounts of hand labeled data. Thus, a crucial part of the vision pipeline of the LeagueAI framework, the dataset generation, is presented in this report. The method involves extracting image raw data from the game's 3D models and combining them with the game background to create game-like synthetic images and to generate the corresponding labels automatically. In an experiment I compared a model trained on synthetic data to a model trained on hand labeled data and a model trained on a combined dataset. The model trained on the synthetic data showed higher detection precision on more classes and more reliable tracking performance of the player character. The model trained on the combined dataset did not perform better because of the different formats of the older hand labeled dataset and the synthetic data.
Learning robust control for LQR systems with multiplicative noise via policy gradient
Authors: Benjamin Gravell, Peyman Mohajerin Esfahani, Tyler Summers
Date: 2019-05-28 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The linear quadratic regulator (LQR) problem has reemerged as an important theoretical benchmark for reinforcement learning-based control of complex dynamical systems with continuous state and action spaces. In contrast with nearly all recent work in this area, we consider multiplicative noise models, which are increasingly relevant because they explicitly incorporate inherent uncertainty and variation in the system dynamics and thereby improve robustness properties of the controller. Robustness is a critical and poorly understood issue in reinforcement learning; existing methods which do not account for uncertainty can converge to fragile policies or fail to converge at all. Additionally, intentional injection of multiplicative noise into learning algorithms can enhance robustness of policies, as observed in ad hoc work on domain randomization. Although policy gradient algorithms require optimization of a non-convex cost function, we show that the multiplicative noise LQR cost has a special property called gradient domination, which is exploited to prove global convergence of policy gradient algorithms to the globally optimum control policy with polynomial dependence on problem parameters. Results are provided both in the model-known and model-unknown settings where samples of system trajectories are used to estimate policy gradients.
Recurrent Existence Determination Through Policy Optimization
Authors: Baoxiang Wang
Date: 2019-05-29 00:00:00

Binary determination of the presence of objects is one of the problems where humans perform extraordinarily better than computer vision systems, in terms of both speed and preciseness. One of the possible reasons is that humans can skip most of the clutter and attend only on salient regions. Recurrent attention models (RAM) are the first computational models to imitate the way humans process images via the REINFORCE algorithm. Despite that RAM is originally designed for image recognition, we extend it and present recurrent existence determination, an attention-based mechanism to solve the existence determination. Our algorithm employs a novel $k$-maximum aggregation layer and a new reward mechanism to address the issue of delayed rewards, which would have caused the instability of the training process. The experimental analysis demonstrates significant efficiency and accuracy improvement over existing approaches, on both synthetic and real-world datasets.
A Survey on Biomedical Image Captioning
Authors: Vasiliki Kougia, John Pavlopoulos, Ion Androutsopoulos
Date: 2019-05-26 00:00:00
Proceeding: WS 2019 6
Tasks: Image Captioning, Survey

Image captioning applied to biomedical images can assist and accelerate the diagnosis process followed by clinicians. This article is the first survey of biomedical image captioning, discussing datasets, evaluation measures, and state of the art methods. Additionally, we suggest two baselines, a weak and a stronger one; the latter outperforms all current state of the art systems on one of the datasets.
Hangul Fonts Dataset: a Hierarchical and Compositional Dataset for Investigating Learned Representations
Authors: Jesse A. Livezey, Ahyeon Hwang, Jacob Yeung, Kristofer E. Bouchard
Date: 2019-05-23 00:00:00
Tasks: BIG-bench Machine Learning, Representation Learning

Hierarchy and compositionality are common latent properties in many natural and scientific datasets. Determining when a deep network's hidden activations represent hierarchy and compositionality is important both for understanding deep representation learning and for applying deep networks in domains where interpretability is crucial. However, current benchmark machine learning datasets either have little hierarchical or compositional structure, or the structure is not known. This gap impedes precise analysis of a network's representations and thus hinders development of new methods that can learn such properties. To address this gap, we developed a new benchmark dataset with known hierarchical and compositional structure. The Hangul Fonts Dataset (HFD) is comprised of 35 fonts from the Korean writing system (Hangul), each with 11,172 blocks (syllables) composed from the product of initial consonant, medial vowel, and final consonant glyphs. All blocks can be grouped into a few geometric types which induces a hierarchy across blocks. In addition, each block is composed of individual glyphs with rotations, translations, scalings, and naturalistic style variation across fonts. We find that both shallow and deep unsupervised methods only show modest evidence of hierarchy and compositionality in their representations of the HFD compared to supervised deep networks. Supervised deep network representations contain structure related to the geometrical hierarchy of the characters, but the compositional structure of the data is not evident. Thus, HFD enables the identification of shortcomings in existing methods, a critical first step toward developing new machine learning algorithms to extract hierarchical and compositional structure in the context of naturalistic variability.
Technical Report of the Video Event Reconstruction and Analysis (VERA) System -- Shooter Localization, Models, Interface, and Beyond
Authors: Junwei Liang, Jay D. Aronson, Alexander Hauptmann
Date: 2019-05-26 00:00:00
Tasks: Gunshot Detection, Shooter Localization, Temporal Localization, Video Synchronization

Every minute, hundreds of hours of video are uploaded to social media sites and the Internet from around the world. This material creates a visual record of the experiences of a significant percentage of humanity and can help illuminate how we live in the present moment. When properly analyzed, this video can also help analysts to reconstruct events of interest, including war crimes, human rights violations, and terrorist acts. Machine learning and computer vision can play a crucial role in this process. In this technical report, we describe the Video Event Reconstruction and Analysis (VERA) system. This new tool brings together a variety of capabilities we have developed over the past few years (including video synchronization and geolocation to order unstructured videos lacking metadata over time and space, and sound recognition algorithms) to enable the reconstruction and analysis of events captured on video. Among other uses, VERA enables the localization of a shooter from just a few videos that include the sound of gunshots. To demonstrate the efficacy of this suite of tools, we present the results of estimating the shooter's location of the Las Vegas Shooting in 2017 and show that VERA accurately predicts the shooter's location using only the first few gunshots. We then point out future directions that can help improve the system and further reduce unnecessary human labor in the process. All of the components of VERA run through a web interface that enables human-in-the-loop verification to ensure accurate estimations. All relevant source code, including the web interface and machine learning models, is freely available on Github. We hope that researchers and software developers will be inspired to improve and expand this system moving forward to better meet the needs of human rights and public safety.
Graph Attention Memory for Visual Navigation
Authors: Dong Li, Qichao Zhang, Dongbin Zhao, Yuzheng Zhuang, Bin Wang, Wulong Liu, Rasul Tutunov, Jun Wang
Date: 2019-05-11 00:00:00
Tasks: Deep Reinforcement Learning, Graph Attention, Reinforcement Learning, Reinforcement Learning (RL), Visual Navigation

Visual navigation in complex environments is inefficient with traditional reactive policy or general-purposed recurrent policy. To address the long-term memory issue, this paper proposes a graph attention memory (GAM) architecture consisting of memory construction module, graph attention module and control module. The memory construction module builds the topological graph based on supervised learning by taking the exploration prior. Then, guided attention features are extracted with the graph attention module. Finally, the deep reinforcement learning based control module makes decisions based on visual observations and guided attention features. Detailed convergence analysis of GAM is presented in this paper. We evaluate GAM-based navigation system in two complex 3D environments. Experimental results show that the GAM-based navigation system significantly improves learning efficiency and outperforms all baselines in average success rate.
Vehicle Detection in Deep Learning
Authors: Yao Xiao
Date: 2019-05-29 00:00:00
Tasks: Deep Learning, Region Proposal, vehicle detection

Computer vision is developing rapidly with the support of deep learning techniques. This thesis proposes an advanced vehicle-detection model based on an improvement to classical convolutional neural networks. The advanced model was applied against a vehicle detection benchmark and was built to detect on-road objects. First, we propose a high-level architecture for our advanced model, which utilizes different state-of-the-art deep learning techniques. Then, we utilize the residual neural networks and region proposal network to achieve competitive performance according to the vehicle detection benchmark. Lastly, we describe the developing trend of vehicle detection techniques and the future direction of research.
Cumulative link models for deep ordinal classification
Authors: Víctor-Manuel Vargas, Pedro-Antonio Gutiérrez, César Hervás-Martínez
Date: 2019-05-27 00:00:00
Tasks: Classification, General Classification, Ordinal Classification

This paper proposes a deep convolutional neural network model for ordinal regression by considering a family of probabilistic ordinal link functions in the output layer. The link functions are those used for cumulative link models, which are traditional statistical linear models based on projecting each pattern into a 1-dimensional space. A set of ordered thresholds splits this space into the different classes of the problem. In our case, the projections are estimated by a non-linear deep neural network. To further improve the results, we combine these ordinal models with a loss function that takes into account the distance between the categories, based on the weighted Kappa index. Three different link functions are studied in the experimental study, and the results are contrasted with statistical analysis. The experiments run over two different ordinal classification problems and the statistical tests confirm that these models improve the results of a nominal model and outperform other robust proposals considered in the literature.
Deep Representation Learning for Road Detection through Siamese Network
Authors: Huafeng Liu, Xiaofeng Han, Xiangrui Li, Yazhou Yao, Pu Huang, Zhenming Tang
Date: 2019-05-26 00:00:00
Tasks: Autonomous Driving, Representation Learning

Robust road detection is a key challenge in safe autonomous driving. Recently, with the rapid development of 3D sensors, more and more researchers are trying to fuse information across different sensors to improve the performance of road detection. Although many successful works have been achieved in this field, methods for data fusion under deep learning framework is still an open problem. In this paper, we propose a Siamese deep neural network based on FCN-8s to detect road region. Our method uses data collected from a monocular color camera and a Velodyne-64 LiDAR sensor. We project the LiDAR point clouds onto the image plane to generate LiDAR images and feed them into one of the branches of the network. The RGB images are fed into another branch of our proposed network. The feature maps that these two branches extract in multiple scales are fused before each pooling layer, via padding additional fusion layers. Extensive experimental results on public dataset KITTI ROAD demonstrate the effectiveness of our proposed approach.
Autonomous Human Activity Classification from Ego-vision Camera and Accelerometer Data
Authors: Yantao Lu, Senem Velipasalar
Date: 2019-05-28 00:00:00
Tasks: General Classification, Multimodal Activity Recognition

There has been significant amount of research work on human activity classification relying either on Inertial Measurement Unit (IMU) data or data from static cameras providing a third-person view. Using only IMU data limits the variety and complexity of the activities that can be detected. For instance, the sitting activity can be detected by IMU data, but it cannot be determined whether the subject has sat on a chair or a sofa, or where the subject is. To perform fine-grained activity classification from egocentric videos, and to distinguish between activities that cannot be differentiated by only IMU data, we present an autonomous and robust method using data from both ego-vision cameras and IMUs. In contrast to convolutional neural network-based approaches, we propose to employ capsule networks to obtain features from egocentric video data. Moreover, Convolutional Long Short Term Memory framework is employed both on egocentric videos and IMU data to capture temporal aspect of actions. We also propose a genetic algorithm-based approach to autonomously and systematically set various network parameters, rather than using manual settings. Experiments have been performed to perform 9- and 26-label activity classification, and the proposed method, using autonomously set network parameters, has provided very promising results, achieving overall accuracies of 86.6\% and 77.2\%, respectively. The proposed approach combining both modalities also provides increased accuracy compared to using only egovision data and only IMU data.
DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning
Authors: Xiawu Zheng, Chenyi Yang, Shaokun Zhang, Yan Wang, Baochang Zhang, Yongjian Wu, Yunsheng Wu, Ling Shao, Rongrong Ji
Date: 2019-05-28 00:00:00
Tasks: AutoML, GPU, Neural Architecture Search

Neural Architecture Search (NAS) has demonstrated state-of-the-art performance on various computer vision tasks. Despite the superior performance achieved, the efficiency and generality of existing methods are highly valued due to their high computational complexity and low generality. In this paper, we propose an efficient and unified NAS framework termed DDPNAS via dynamic distribution pruning, facilitating a theoretical bound on accuracy and efficiency. In particular, we first sample architectures from a joint categorical distribution. Then the search space is dynamically pruned and its distribution is updated every few epochs. With the proposed efficient network generation method, we directly obtain the optimal neural architectures on given constraints, which is practical for on-device models across diverse search spaces and constraints. The architectures searched by our method achieve remarkable top-1 accuracies, 97.56 and 77.2 on CIFAR-10 and ImageNet (mobile settings), respectively, with the fastest search process, i.e., only 1.8 GPU hours on a Tesla V100. Codes for searching and network generation are available at: https://openi.pcl.ac.cn/PCL AutoML/XNAS.
High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
Authors: Haohan Wang, Xindi Wu, Zeyi Huang, Eric P. Xing
Date: 2019-05-28 00:00:00
Tasks: Adversarial Attack, Vocal Bursts Intensity Prediction

We investigate the relationship between the frequency spectrum of image data and the generalization behavior of convolutional neural networks (CNN). We first notice CNN's ability in capturing the high-frequency components of images. These high-frequency components are almost imperceptible to a human. Thus the observation leads to multiple hypotheses that are related to the generalization behaviors of CNN, including a potential explanation for adversarial examples, a discussion of CNN's trade-off between robustness and accuracy, and some evidence in understanding training heuristics.
Learning Robust Global Representations by Penalizing Local Predictive Power
Authors: Haohan Wang, Songwei Ge, Eric P. Xing, Zachary C. Lipton
Date: 2019-05-29 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Domain Adaptation, Domain Generalization, General Classification

Despite their renowned predictive power on i.i.d. data, convolutional neural networks are known to rely more on high-frequency patterns that humans deem superficial than on low-frequency patterns that agree better with intuitions about what constitutes category membership. This paper proposes a method for training robust convolutional networks by penalizing the predictive power of the local representations learned by earlier layers. Intuitively, our networks are forced to discard predictive signals such as color and texture that can be gleaned from local receptive fields and to rely instead on the global structures of the image. Across a battery of synthetic and benchmark domain adaptation tasks, our method confers improved generalization out of the domain. Also, to evaluate cross-domain transfer, we introduce ImageNet-Sketch, a new dataset consisting of sketch-like images, that matches the ImageNet classification validation set in categories and scale.
Sketch2code: Generating a website from a paper mockup
Authors: Alex Robinson
Date: 2019-05-09 00:00:00
Tasks: Code Generation, Deep Learning, Semantic Segmentation

An early stage of developing user-facing applications is creating a wireframe to layout the interface. Once a wireframe has been created it is given to a developer to implement in code. Developing boiler plate user interface code is time consuming work but still requires an experienced developer. In this dissertation we present two approaches which automates this process, one using classical computer vision techniques, and another using a novel application of deep semantic segmentation networks. We release a dataset of websites which can be used to train and evaluate these approaches. Further, we have designed a novel evaluation framework which allows empirical evaluation by creating synthetic sketches. Our evaluation illustrates that our deep learning approach outperforms our classical computer vision approach and we conclude that deep learning is the most promising direction for future research.
Convolutional Restricted Boltzmann Machine Based-Radiomics for Prediction of Pathological Complete Response to Neoadjuvant Chemotherapy in Breast Cancer
Authors: Li Wang, Lihui Wang, Qijian Chen, Caixia Sun, Xinyu Cheng, Yue-Min Zhu
Date: 2019-05-23 00:00:00
Tasks: feature selection, Prediction

We proposed a novel convolutional restricted Boltzmann machine CRBM-based radiomic method for predicting pathologic complete response (pCR) to neoadjuvant chemotherapy treatment (NACT) in breast cancer. The method consists of extracting semantic features from CRBM network, and pCR prediction. It was evaluated on the dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data of 57 patients and using the area under the receiver operating characteristic curve (AUC). Traditional radiomics features and the semantic features learned from CRBM network were extracted from the images acquired before and after the administration of NACT. After the feature selection, the support vector machine (SVM), logistic regression (LR) and random forest (RF) were trained to predict the pCR status. Compared to traditional radiomic methods, the proposed CRBM-based radiomic method yielded an AUC of 0.92 for the prediction with the images acquired before and after NACT, and an AUC of 0.87 for the pretreatment prediction, which was increased by about 38%. The results showed that the CRBM-based radiomic method provided a potential means for accurately predicting the pCR to NACT in breast cancer before the treatment, which is very useful for making more appropriate and personalized treatment regimens.
An Algorithmic Framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-Gradient Method
Authors: Li Shen, Peng Sun, Yitong Wang, Wei Liu, Tong Zhang
Date: 2018-05-16 00:00:00
Proceeding: ICML 2018 7

We propose a novel algorithmic framework of Variable Metric Over-Relaxed
Hybrid Proximal Extra-gradient (VMOR-HPE) method with a global convergence
guarantee for the maximal monotone operator inclusion problem. Its iteration
complexities and local linear convergence rate are provided, which
theoretically demonstrate that a large over-relaxed step-size contributes to
accelerating the proposed VMOR-HPE as a byproduct. Specifically, we find that a
large class of primal and primal-dual operator splitting algorithms are all
special cases of VMOR-HPE. Hence, the proposed framework offers a new insight
into these operator splitting algorithms. In addition, we apply VMOR-HPE to the
Karush-Kuhn-Tucker (KKT) generalized equation of linear equality constrained
multi-block composite convex optimization, yielding a new algorithm, namely
nonsymmetric Proximal Alternating Direction Method of Multipliers with a
preconditioned Extra-gradient step in which the preconditioned metric is
generated by a blockwise Barzilai-Borwein line search technique (PADMM-EBB). We
also establish iteration complexities of PADMM-EBB in terms of the KKT
residual. Finally, we apply PADMM-EBB to handle the nonnegative dual graph
regularized low-rank representation problem. Promising results on synthetic and
real datasets corroborate the efficacy of PADMM-EBB.
Sparse optimal control of networks with multiplicative noise via policy gradient
Authors: Benjamin Gravell, Yi Guo, Tyler Summers
Date: 2019-05-28 00:00:00

We give algorithms for designing near-optimal sparse controllers using policy gradient with applications to control of systems corrupted by multiplicative noise, which is increasingly important in emerging complex dynamical networks. Various regularization schemes are examined and incorporated into the optimization by the use of gradient, subgradient, and proximal gradient methods. Numerical experiments on a large networked system show that the algorithms converge to performant sparse mean-square stabilizing controllers.