pwc-archive/papers-with-abstracts huggingface dataset:
Differential Equation Units: Learning Functional Forms of Activation Functions from Data
Authors: MohamadAli Torkamani, Shiv Shankar, Amirmohammad Rooshenas, Phillip Wallis
Date: 2019-09-06 00:00:00

Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural networks, which enables each neuron to learn a particular nonlinear activation function from a family of solutions to an ordinary differential equation. Specifically, each neuron may change its functional form during training based on the behavior of the other parts of the network. We show that using neurons with DEU activation functions results in a more compact network capable of achieving comparable, if not superior, performance when is compared to much larger networks.
Distributed creation of Machine learning agents for Blockchain analysis
Authors: Zvezdin Besarabov, Todor Kolev
Date: 2019-09-06 00:00:00
Tasks: Bayesian Optimization, BIG-bench Machine Learning, Neural Architecture Search

Creating efficient deep neural networks involves repetitive manual optimization of the topology and the hyperparameters. This human intervention significantly inhibits the process. Recent publications propose various Neural Architecture Search (NAS) algorithms that automate this work. We have applied a customized NAS algorithm with network morphism and Bayesian optimization to the problem of cryptocurrency predictions, where it achieved results on par with our best manually designed models. This is consistent with the findings of other teams, while several known experiments suggest that given enough computing power, NAS algorithms can surpass state-of-the-art neural network models designed by humans. In this paper, we propose a blockchain network protocol that incentivises independent computing nodes to run NAS algorithms and compete in finding better neural network models for a particular task. If implemented, such network can be an autonomous and self-improving source of machine learning models, significantly boosting and democratizing the access to AI capabilities for many industries.
Compact Autoregressive Network
Authors: Di Wang, Feiqing Huang, Jingyu Zhao, Guodong Li, Guangjian Tian
Date: 2019-09-06 00:00:00
Tasks: TAR

Autoregressive networks can achieve promising performance in many sequence modeling tasks with short-range dependence. However, when handling high-dimensional inputs and outputs, the huge amount of parameters in the network lead to expensive computational cost and low learning efficiency. The problem can be alleviated slightly by introducing one more narrow hidden layer to the network, but the sample size required to achieve a certain training error is still large. To address this challenge, we rearrange the weight matrices of a linear autoregressive network into a tensor form, and then make use of Tucker decomposition to represent low-rank structures. This leads to a novel compact autoregressive network, called Tucker AutoRegressive (TAR) net. Interestingly, the TAR net can be applied to sequences with long-range dependence since the dimension along the sequential order is reduced. Theoretical studies show that the TAR net improves the learning efficiency, and requires much fewer samples for model training. Experiments on synthetic and real-world datasets demonstrate the promising performance of the proposed compact network.
"Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understanding
Authors: Ben Zhou, Daniel Khashabi, Qiang Ning, Dan Roth
Date: 2019-09-06 00:00:00

Understanding time is crucial for understanding events expressed in natural language. Because people rarely say the obvious, it is often necessary to have commonsense knowledge about various temporal aspects of events, such as duration, frequency, and temporal order. However, this important problem has so far received limited attention. This paper systematically studies this temporal commonsense problem. Specifically, we define five classes of temporal commonsense, and use crowdsourcing to develop a new dataset, MCTACO, that serves as a test set for this task. We find that the best current methods used on MCTACO are still far behind human performance, by about 20%, and discuss several directions for improvement. We hope that the new dataset and our study here can foster more future research on this topic.
Training Deep Neural Networks Using Posit Number System
Authors: Jinming Lu, Siyuan Lu, Zhisheng Wang, Chao Fang, Jun Lin, Zhongfeng Wang, Li Du
Date: 2019-09-06 00:00:00
Tasks: image-classification, Image Classification

With the increasing size of Deep Neural Network (DNN) models, the high memory space requirements and computational complexity have become an obstacle for efficient DNN implementations. To ease this problem, using reduced-precision representations for DNN training and inference has attracted many interests from researchers. This paper first proposes a methodology for training DNNs with the posit arithmetic, a type- 3 universal number (Unum) format that is similar to the floating point(FP) but has reduced precision. A warm-up training strategy and layer-wise scaling factors are adopted to stabilize training and fit the dynamic range of DNN parameters. With the proposed training methodology, we demonstrate the first successful training of DNN models on ImageNet image classification task in 16 bits posit with no accuracy loss. Then, an efficient hardware architecture for the posit multiply-and-accumulate operation is also proposed, which can achieve significant improvement in energy efficiency than traditional floating-point implementations. The proposed design is helpful for future low-power DNN training accelerators.
DublinCity: Annotated LiDAR Point Cloud and its Applications
Authors: S. M. Iman Zolanvari, Susana Ruano, Aakanksha Rana, Alan Cummins, Rogerio Eduardo da Silva, Morteza Rahbar, Aljosa Smolic
Date: 2019-09-06 00:00:00
Tasks: 3D Reconstruction, Scene Understanding

Scene understanding of full-scale 3D models of an urban area remains a challenging task. While advanced computer vision techniques offer cost-effective approaches to analyse 3D urban elements, a precise and densely labelled dataset is quintessential. The paper presents the first-ever labelled dataset for a highly dense Aerial Laser Scanning (ALS) point cloud at city-scale. This work introduces a novel benchmark dataset that includes a manually annotated point cloud for over 260 million laser scanning points into 100'000 (approx.) assets from Dublin LiDAR point cloud [12] in 2015. Objects are labelled into 13 classes using hierarchical levels of detail from large (i.e., building, vegetation and ground) to refined (i.e., window, door and tree) elements. To validate the performance of our dataset, two different applications are showcased. Firstly, the labelled point cloud is employed for training Convolutional Neural Networks (CNNs) to classify urban elements. The dataset is tested on the well-known state-of-the-art CNNs (i.e., PointNet, PointNet++ and So-Net). Secondly, the complete ALS dataset is applied as detailed ground truth for city-scale image-based 3D reconstruction.
Testing Deep Learning Models for Image Analysis Using Object-Relevant Metamorphic Relations
Authors: Yongqiang Tian, Shiqing Ma, Ming Wen, Yepang Liu, Shing-Chi Cheung, Xiangyu Zhang
Date: 2019-09-06 00:00:00
Tasks: General Classification, image-classification, Image Classification, Image Generation, Object, object-detection, Object Detection

Deep learning models are widely used for image analysis. While they offer high performance in terms of accuracy, people are concerned about if these models inappropriately make inferences using irrelevant features that are not encoded from the target object in a given image. To address the concern, we propose a metamorphic testing approach that assesses if a given inference is made based on irrelevant features. Specifically, we propose two novel metamorphic relations to detect such inappropriate inferences. We applied our approach to 10 image classification models and 10 object detection models, with three large datasets, i.e., ImageNet, COCO, and Pascal VOC. Over 5.3% of the top-5 correct predictions made by the image classification models are subject to inappropriate inferences using irrelevant features. The corresponding rate for the object detection models is over 8.5%. Based on the findings, we further designed a new image generation strategy that can effectively attack existing models. Comparing with a baseline approach, our strategy can double the success rate of attacks.
Linear Context Transform Block
Authors: Dongsheng Ruan, Jun Wen, Nenggan Zheng, Min Zheng
Date: 2019-09-06 00:00:00
Tasks: image-classification, Image Classification, object-detection, Object Detection

Squeeze-and-Excitation (SE) block presents a channel attention mechanism for modeling global context via explicitly capturing dependencies across channels. However, we are still far from understanding how the SE block works. In this work, we first revisit the SE block, and then present a detailed empirical study of the relationship between global context and attention distribution, based on which we propose a simple yet effective module, called Linear Context Transform (LCT) block. We divide all channels into different groups and normalize the globally aggregated context features within each channel group, reducing the disturbance from irrelevant channels. Through linear transform of the normalized context features, we model global context for each channel independently. The LCT block is extremely lightweight and easy to be plugged into different backbone models while with negligible parameters and computational burden increase. Extensive experiments show that the LCT block outperforms the SE block in image classification task on the ImageNet and object detection/segmentation on the COCO dataset with different backbone models. Moreover, LCT yields consistent performance gains over existing state-of-the-art detection architectures, e.g., 1.5$\sim$1.7% AP$^{bbox}$ and 1.0$\sim$1.2% AP$^{mask}$ improvements on the COCO benchmark, irrespective of different baseline models of varied capacities. We hope our simple yet effective approach will shed some light on future research of attention-based models.
Calibrating Wayfinding Decisions in Pedestrian Simulation Models: The Entropy Map
Authors: Luca Crociani, Giuseppe Vizzari, Stefania Bandini
Date: 2019-09-06 00:00:00

This paper presents entropy maps, an approach to describing and visualising uncertainty among alternative potential movement intentions in pedestrian simulation models. In particular, entropy maps show the instantaneous level of randomness in decisions of a pedestrian agent situated in a specific point of the simulated environment with an heatmap approach. Experimental results highlighting the relevance of this tool supporting modelers are provided and discussed.
Data Sanity Check for Deep Learning Systems via Learnt Assertions
Authors: Haochuan Lu, Huanlin Xu, Nana Liu, Yangfan Zhou, Xin Wang
Date: 2019-09-06 00:00:00
Tasks: Deep Learning

Reliability is a critical consideration to DL-based systems. But the statistical nature of DL makes it quite vulnerable to invalid inputs, i.e., those cases that are not considered in the training phase of a DL model. This paper proposes to perform data sanity check to identify invalid inputs, so as to enhance the reliability of DL-based systems. We design and implement a tool to detect behavior deviation of a DL model when processing an input case. This tool extracts the data flow footprints and conducts an assertion-based validation mechanism. The assertions are built automatically, which are specifically-tailored for DL model data flow analysis. Our experiments conducted with real-world scenarios demonstrate that such an assertion-based data sanity check mechanism is effective in identifying invalid input cases.
SEdroid: A Robust Android Malware Detector using Selective Ensemble Learning
Authors: Ji Wang, Qi Jing, Jianbo Gao
Date: 2019-09-06 00:00:00
Tasks: Android Malware Detection, Ensemble Learning, Malware Detection

For the dramatic increase of Android malware and low efficiency of manual check process, deep learning methods started to be an auxiliary means for Android malware detection these years. However, these models are highly dependent on the quality of datasets, and perform unsatisfactory results when the quality of training data is not good enough. In the real world, the quality of datasets without manually check cannot be guaranteed, even Google Play may contain malicious applications, which will cause the trained model failure. To address the challenge, we propose a robust Android malware detection approach based on selective ensemble learning, trying to provide an effective solution not that limited to the quality of datasets. The proposed model utilizes genetic algorithm to help find the best combination of the component learners and improve robustness of the model. Our results show that the proposed approach achieves a more robust performance than other approaches in the same area.
Recovery of Future Data via Convolution Nuclear Norm Minimization
Authors: Guangcan Liu, Wayne Zhang
Date: 2019-09-06 00:00:00
Tasks: compressed sensing, Time Series, Time Series Analysis, Time Series Forecasting

This paper studies the problem of time series forecasting (TSF) from the perspective of compressed sensing. First of all, we convert TSF into a more inclusive problem called tensor completion with arbitrary sampling (TCAS), which is to restore a tensor from a subset of its entries sampled in an arbitrary manner. While it is known that, in the framework of Tucker low-rankness, it is theoretically impossible to identify the target tensor based on some arbitrarily selected entries, in this work we shall show that TCAS is indeed tackleable in the light of a new concept called convolutional low-rankness, which is a generalization of the well-known Fourier sparsity. Then we introduce a convex program termed Convolution Nuclear Norm Minimization (CNNM), and we prove that CNNM succeeds in solving TCAS as long as a sampling condition--which depends on the convolution rank of the target tensor--is obeyed. This theory provides a meaningful answer to the fundamental question of what is the minimum sampling size needed for making a given number of forecasts. Experiments on univariate time series, images and videos show encouraging results.
Efficient Automatic Meta Optimization Search for Few-Shot Learning
Authors: Xinyue Zheng, Peng Wang, Qigang Wang, Zhongchao shi, Feiyu Xu
Date: 2019-09-06 00:00:00
Tasks: Few-Shot Learning, GPU, Meta-Learning, Neural Architecture Search

Previous works on meta-learning either relied on elaborately hand-designed network structures or adopted specialized learning rules to a particular domain. We propose a universal framework to optimize the meta-learning process automatically by adopting neural architecture search technique (NAS). NAS automatically generates and evaluates meta-learner's architecture for few-shot learning problems, while the meta-learner uses meta-learning algorithm to optimize its parameters based on the distribution of learning tasks. Parameter sharing and experience replay are adopted to accelerate the architectures searching process, so it takes only 1-2 GPU days to find good architectures. Extensive experiments on Mini-ImageNet and Omniglot show that our algorithm excels in few-shot learning tasks. The best architecture found on Mini-ImageNet achieves competitive results when transferred to Omniglot, which shows the high transferability of architectures among different computer vision problems.
On Data-Selective Learning
Authors: Hamed Yazdanpanah
Date: 2019-09-06 00:00:00

Adaptive filters are applied in several electronic and communication devices like smartphones, advanced headphones, DSP chips, smart antenna, and teleconference systems. Also, they have application in many areas such as system identification, channel equalization, noise reduction, echo cancellation, interference cancellation, signal prediction, and stock market. Therefore, reducing the energy consumption of the adaptive filtering algorithms has great importance, particularly in green technologies and in devices using battery. In this thesis, data-selective adaptive filters, in particular the set-membership (SM) adaptive filters, are the tools to reach the goal. There are well known SM adaptive filters in literature. This work introduces new algorithms based on the classical ones in order to improve their performances and reduce the number of required arithmetic operations at the same time. Therefore, firstly, we analyze the robustness of the classical SM adaptive filtering algorithms. Secondly, we extend the SM technique to trinion and quaternion systems. Thirdly, by combining SM filtering and partial-updating, we introduce a new improved set-membership affine projection algorithm with constrained step size to improve its stability behavior. Fourthly, we propose some new least-mean-square (LMS) based and recursive least-squares based adaptive filtering algorithms with low computational complexity for sparse systems. Finally, we derive some feature LMS algorithms to exploit the hidden sparsity in the parameters.
A Variational Bayes Approach to Adaptive Radio Tomography
Authors: Donghoon Lee, Georgios B. Giannakis
Date: 2019-09-05 00:00:00
Tasks: Management

Radio tomographic imaging (RTI) is an emerging technology for localization of physical objects in a geographical area covered by wireless networks. With attenuation measurements collected at spatially distributed sensors, RTI capitalizes on spatial loss fields (SLFs) measuring the absorption of radio frequency waves at spatial locations along the propagation path. These SLFs can be utilized for interference management in wireless communication networks, environmental monitoring, and survivor localization after natural disasters such as earthquakes. Key to the success of RTI is to accurately model shadowing as the weighted line integral of the SLF. To learn the SLF exhibiting statistical heterogeneity induced by spatially diverse environments, the present work develops a Bayesian framework entailing a piecewise homogeneous SLF with an underlying hidden Markov random field model. Utilizing variational Bayes techniques, the novel approach yields efficient field estimators at affordable complexity. A data-adaptive sensor selection strategy is also introduced to collect informative measurements for effective reconstruction of the SLF. Numerical tests using synthetic and real datasets demonstrate the capabilities of the proposed approach to radio tomography and channel-gain estimation.
On Learning Disentangled Representations for Gait Recognition
Authors: Ziyuan Zhang, Luan Tran, Feng Liu, Xiaoming Liu
Date: 2019-09-05 00:00:00
Tasks: Computational Efficiency, Disentanglement, Face Recognition, Gait Recognition

Gait, the walking pattern of individuals, is one of the important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and viewing angle. To remedy this issue, we propose a novel AutoEncoder framework, GaitNet, to explicitly disentangle appearance, canonical and pose features from RGB imagery. The LSTM integrates pose features over time as a dynamic gait feature while canonical features are averaged as a static gait feature. Both of them are utilized as classification features. In addition, we collect a Frontal-View Gait (FVG) dataset to focus on gait recognition from frontal-view walking, which is a challenging problem since it contains minimal gait cues compared to other views. FVG also includes other important variations, e.g., walking speed, carrying, and clothing. With extensive experiments on CASIA-B, USF, and FVG datasets, our method demonstrates superior performance to the SOTA quantitatively, the ability of feature disentanglement qualitatively, and promising computational efficiency. We further compare our GaitNet with state-of-the-art face recognition to demonstrate the advantages of gait biometrics identification under certain scenarios, e.g., long distance/lower resolutions, cross viewing angles.
Large-Scale Local Causal Inference of Gene Regulatory Relationships
Authors: Ioan Gabriel Bucur, Tom Claassen, Tom Heskes
Date: 2019-09-03 00:00:00
Tasks: Causal Inference, Triplet

Gene regulatory networks play a crucial role in controlling an organism's biological processes, which is why there is significant interest in developing computational methods that are able to extract their structure from high-throughput genetic data. Many of these computational methods are designed to infer individual regulatory relationships among genes from data on gene expression. We propose a novel efficient Bayesian method for discovering local causal relationships among triplets of (normally distributed) variables. In our approach, we score covariance structures for each triplet in one go and incorporate available background knowledge in the form of priors to derive posterior probabilities over local causal structures. Our method is flexible in the sense that it allows for different types of causal structures and assumptions. We apply our approach to the task of learning causal regulatory relationships among genes. We show that the proposed algorithm produces stable and conservative posterior probability estimates over local causal structures that can be used to derive an honest ranking of the most meaningful regulatory relationships. We demonstrate the stability and efficacy of our method both on simulated data and on real-world data from an experiment on yeast.
Performance Evaluation of Histogram Equalization and Fuzzy image Enhancement Techniques on Low Contrast Images
Authors: E Onyedinma, I Onyenwe, H Inyiama
Date: 2019-09-01 00:00:00
Tasks: Image Enhancement

Image enhancement aims at improving the information content of original image for a specific purpose. This purpose could be for visual interpretation or for effective extraction of required details. Nevertheless, some acquired images are often associated with pixels of low dynamic range and as such result in low contrast images. Enhancing the contrast therefore tends to increase the dynamic range of the gray levels in the acquired image so as to span the full intensity range. Techniques such as Histogram Equalization (HE) and fuzzy technique can be adopted for contrast enhancement. HE adjusts the contrast of an input image by modifying the intensity distribution of its histogram. It is characterized by providing a global approach to image enhancement, computationally fast and easy to implement approach but can introduce unnatural artifacts and other undesirable elements to the resulting image. Fuzzy technique on its part enhances image by mapping the image gray level intensities into a fuzzy plane using membership functions; modifying the membership functions as desired and mapping back into the gray level plane. Thus, details at desired areas can be enhanced at the expense of increase in computational cost. This paper explores the effect of the use of HE and fuzzy technique to enhance low contrast images. Their performances are evaluated using the Mean squared error (MSE), Peak to signal noise ratio (PSNR), entropy and Absolute mean brightness error (AMBE).