pwc-archive/papers-with-abstracts huggingface dataset:
Feasibility-Guided Learning for Robust Control in Constrained Optimal Control Problems
Authors: Wei Xiao, Calin A. Belta, Christos G. Cassandras
Date: 2019-12-06 00:00:00

Optimal control problems with constraints ensuring safety and convergence to desired states can be mapped onto a sequence of real time optimization problems through the use of Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in these approaches is ensuring the feasibility of the resulting quadratic programs (QPs) if the system is affine in controls. The recently proposed penalty method has the potential to improve the existence of feasible solutions to such problems. In this paper, we further improve the feasibility robustness (i.e., feasibility maintenance in the presence of time-varying and unknown unsafe sets) through the definition of a High Order CBF (HOCBF) that works for arbitrary relative degree constraints; this is achieved by a proposed feasibility-guided learning approach. Specifically, we apply machine learning techniques to classify the parameter space of a HOCBF into feasible and infeasible sets, and get a differentiable classifier that is then added to the learning process. The proposed feasibility-guided learning approach is compared with the gradient-descent method on a robot control problem. The simulation results show an improved ability of the feasibility-guided learning approach over the gradient-decent method to determine the optimal parameters in the definition of a HOCBF for the feasibility robustness, as well as show the potential of the CBF method for robot safe navigation in an unknown environment.
Geometric Capsule Autoencoders for 3D Point Clouds
Authors: Nitish Srivastava, Hanlin Goh, Ruslan Salakhutdinov
Date: 2019-12-06 00:00:00
Tasks: Object, Retrieval

We propose a method to learn object representations from 3D point clouds using bundles of geometrically interpretable hidden units, which we call geometric capsules. Each geometric capsule represents a visual entity, such as an object or a part, and consists of two components: a pose and a feature. The pose encodes where the entity is, while the feature encodes what it is. We use these capsules to construct a Geometric Capsule Autoencoder that learns to group 3D points into parts (small local surfaces), and these parts into the whole object, in an unsupervised manner. Our novel Multi-View Agreement voting mechanism is used to discover an object's canonical pose and its pose-invariant feature vector. Using the ShapeNet and ModelNet40 datasets, we analyze the properties of the learned representations and show the benefits of having multiple votes agree. We perform alignment and retrieval of arbitrarily rotated objects -- tasks that evaluate our model's object identification and canonical pose recovery capabilities -- and obtained insightful results.
Element Level Differential Privacy: The Right Granularity of Privacy
Authors: Hilal Asi, John Duchi, Omid Javidbakht
Date: 2019-12-05 00:00:00

Differential Privacy (DP) provides strong guarantees on the risk of compromising a user's data in statistical learning applications, though these strong protections make learning challenging and may be too stringent for some use cases. To address this, we propose element level differential privacy, which extends differential privacy to provide protection against leaking information about any particular "element" a user has, allowing better utility and more robust results than classical DP. By carefully choosing these "elements," it is possible to provide privacy protections at a desired granularity. We provide definitions, associated privacy guarantees, and analysis to identify the tradeoffs with the new definition; we also develop several private estimation and learning methodologies, providing careful examples for item frequency and M-estimation (empirical risk minimization) with concomitant privacy and utility analysis. We complement our theoretical and methodological advances with several real-world applications, estimating histograms and fitting several large-scale prediction models, including deep networks.
Reinforcement Learning with Convolutional Reservoir Computing
Authors: Hanten Chang, Katsuya Futagami
Date: 2019-12-05 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Time Series, Time Series Analysis

Recently, reinforcement learning models have achieved great success, mastering complex tasks such as Go and other games with higher scores than human players. Many of these models store considerable data on the tasks and achieve high performance by extracting visual and time-series features using convolutional neural networks (CNNs) and recurrent neural networks, respectively. However, these networks have very high computational costs because they need to be trained by repeatedly using the stored data. In this study, we propose a novel practical approach called reinforcement learning with convolutional reservoir computing (RCRC) model. The RCRC model uses a fixed random-weight CNN and a reservoir computing model to extract visual and time-series features. Using these extracted features, it decides actions with an evolution strategy method. Thereby, the RCRC model has several desirable features: (1) there is no need to train the feature extractor, (2) there is no need to store training data, (3) it can take a wide range of actions, and (4) there is only a single task-dependent weight parameter to be trained. Furthermore, we show the RCRC model can solve multiple reinforcement learning tasks with a completely identical feature extractor.
Asymptotic Unbiasedness of the Permutation Importance Measure in Random Forest Models
Authors: Burim Ramosaj, Markus Pauly
Date: 2019-12-05 00:00:00
Tasks: Econometrics, regression, Variable Selection

Variable selection in sparse regression models is an important task as applications ranging from biomedical research to econometrics have shown. Especially for higher dimensional regression problems, for which the link function between response and covariates cannot be directly detected, the selection of informative variables is challenging. Under these circumstances, the Random Forest method is a helpful tool to predict new outcomes while delivering measures for variable selection. One common approach is the usage of the permutation importance. Due to its intuitive idea and flexible usage, it is important to explore circumstances, for which the permutation importance based on Random Forest correctly indicates informative covariates. Regarding the latter, we deliver theoretical guarantees for the validity of the permutation importance measure under specific assumptions and prove its (asymptotic) unbiasedness. An extensive simulation study verifies our findings.
Learning to run a power network challenge for training topology controllers
Authors: Antoine Marot, Benjamin Donnot, Camilo Romero, Luca Veyrin-Forrer, Marvin Lerousseau, Balthazar Donon, Isabelle Guyon
Date: 2019-12-05 00:00:00
Tasks: Management, Reinforcement Learning, RTE

For power grid operations, a large body of research focuses on using generation redispatching, load shedding or demand side management flexibilities. However, a less costly and potentially more flexible option would be grid topology reconfiguration, as already partially exploited by Coreso (European RSC) and RTE (French TSO) operations. Beyond previous work on branch switching, bus reconfigurations are a broader class of action and could provide some substantial benefits to route electricity and optimize the grid capacity to keep it within safety margins. Because of its non-linear and combinatorial nature, no existing optimal power flow solver can yet tackle this problem. We here propose a new framework to learn topology controllers through imitation and reinforcement learning. We present the design and the results of the first "Learning to Run a Power Network" challenge released with this framework. We finally develop a method providing performance upper-bounds (oracle), which highlights remaining unsolved challenges and suggests future directions of improvement.
Solving Bayesian Inverse Problems via Variational Autoencoders
Authors: Hwan Goh, Sheroze Sheriffdeen, Jonathan Wittmer, Tan Bui-Thanh
Date: 2019-12-05 00:00:00
Tasks: Uncertainty Quantification, Variational Inference

In recent years, the field of machine learning has made phenomenal progress in the pursuit of simulating real-world data generation processes. One notable example of such success is the variational autoencoder (VAE). In this work, with a small shift in perspective, we leverage and adapt VAEs for a different purpose: uncertainty quantification in scientific inverse problems. We introduce UQ-VAE: a flexible, adaptive, hybrid data/model-informed framework for training neural networks capable of rapid modelling of the posterior distribution representing the unknown parameter of interest. Specifically, from divergence-based variational inference, our framework is derived such that most of the information usually present in scientific inverse problems is fully utilized in the training procedure. Additionally, this framework includes an adjustable hyperparameter that allows selection of the notion of distance between the posterior model and the target distribution. This introduces more flexibility in controlling how optimization directs the learning of the posterior model. Further, this framework possesses an inherent adaptive optimization property that emerges through the learning of the posterior uncertainty.
EviDense: a Graph-based Method for Finding Unique High-impact Events with Succinct Keyword-based Descriptions
Authors: Oana Balalau, Carlos Castillo, Mauro Sozio
Date: 2019-12-05 00:00:00

Despite the significant efforts made by the research community in recent years, automatically acquiring valuable information about high impact-events from social media remains challenging. We present EviDense, a graph-based approach for finding high-impact events (such as disaster events) in social media. One of the challenges we address in our work is to provide for each event a succinct keyword-based description, containing the most relevant information about it, such as what happened, the location, as well as its timeframe. We evaluate our approach on a large collection of tweets posted over a period of 19 months, using a crowdsourcing platform. Our evaluation shows that our method outperforms state-of-the-art approaches for the same problem, in terms of having higher precision, lower number of duplicates, and presenting a keyword-based description that is succinct and informative. We further improve the results of our algorithm by incorporating news from mainstream media. A preliminary version of this work was presented as a 4-pages short paper at ICWSM 2018.
Towards countering hate speech against journalists on social media
Authors: Polychronis Charitidis, Stavros Doropoulos, Stavros Vologiannidis, Ioannis Papastergiou, Sophia Karakeva
Date: 2019-12-05 00:00:00
Tasks: Active Learning, Hate Speech Detection

The damaging effects of hate speech on social media are evident during the last few years, and several organizations, researchers and social media platforms tried to harness them in various ways. Despite these efforts, social media users are still affected by hate speech. The problem is even more apparent to social groups that promote public discourse, such as journalists. In this work, we focus on countering hate speech that is targeted to journalistic social media accounts. To accomplish this, a group of journalists assembled a definition of hate speech, taking into account the journalistic point of view and the types of hate speech that are usually targeted against journalists. We then compile a large pool of tweets referring to journalism-related accounts in multiple languages. In order to annotate the pool of unlabeled tweets according to the definition, we follow a concise annotation strategy that involves active learning annotation stages. The outcome of this paper is a novel, publicly available collection of Twitter datasets in five different languages. Additionally, we experiment with state-of-the-art deep learning architectures for hate speech detection and use our annotated datasets to train and evaluate them. Finally, we propose an ensemble detection model that outperforms all individual models.
PhoneBit: Efficient GPU-Accelerated Binary Neural Network Inference Engine for Mobile Phones
Authors: Gang Chen, Shengyu He, Haitao Meng, Kai Huang
Date: 2019-12-05 00:00:00
Tasks: GPU

Over the last years, a great success of deep neural networks (DNNs) has been witnessed in computer vision and other fields. However, performance and power constraints make it still challenging to deploy DNNs on mobile devices due to their high computational complexity. Binary neural networks (BNNs) have been demonstrated as a promising solution to achieve this goal by using bit-wise operations to replace most arithmetic operations. Currently, existing GPU-accelerated implementations of BNNs are only tailored for desktop platforms. Due to architecture differences, mere porting of such implementations to mobile devices yields suboptimal performance or is impossible in some cases. In this paper, we propose PhoneBit, a GPU-accelerated BNN inference engine for Android-based mobile devices that fully exploits the computing power of BNNs on mobile GPUs. PhoneBit provides a set of operator-level optimizations including locality-friendly data layout, bit packing with vectorization and layers integration for efficient binary convolution. We also provide a detailed implementation and parallelization optimization for PhoneBit to optimally utilize the memory bandwidth and computing power of mobile GPUs. We evaluate PhoneBit with AlexNet, YOLOv2 Tiny and VGG16 with their binary version. Our experiment results show that PhoneBit can achieve significant speedup and energy efficiency compared with state-of-the-art frameworks for mobile devices.
Analysis of Deep Neural Networks with Quasi-optimal polynomial approximation rates
Authors: Joseph Daws, Clayton Webster
Date: 2019-12-04 00:00:00

We show the existence of a deep neural network capable of approximating a wide class of high-dimensional approximations. The construction of the proposed neural network is based on a quasi-optimal polynomial approximation. We show that this network achieves an error rate that is sub-exponential in the number of polynomial functions, $M$, used in the polynomial approximation. The complexity of the network which achieves this sub-exponential rate is shown to be algebraic in $M$.
Forward and Backward Feature Selection for Query Performance Prediction
Authors: Sébastien Déjean, Radu Tudor Ionescu, Josiane Mothe, Md Zia Ullah
Date: 2019-12-04 00:00:00
Tasks: feature selection, Model Selection, Retrieval

The goal of query performance prediction (QPP) is to automatically estimate the effectiveness of a search result for any given query, without relevance judgements. Post-retrieval features have been shown to be more effective for this task while being more expensive to compute than pre-retrieval features. Combining multiple post-retrieval features is even more effective, but state-of-the-art QPP methods are impossible to interpret because of the black-box nature of the employed machine learning models. However, interpretation is useful for understanding the predictive model and providing more answers about its behavior. Moreover, combining many post-retrieval features is not applicable to real-world cases, since the query running time is of utter importance. In this paper, we investigate a new framework for feature selection in which the trained model explains well the prediction. We introduce a step-wise (forward and backward) model selection approach where different subsets of query features are used to fit different models from which the system selects the best one. We evaluate our approach on four TREC collections using standard QPP features. We also develop two QPP features to address the issue of query-drift in the query feedback setting. We found that: (1) our model based on a limited number of selected features is as good as more complex models for QPP and better than non-selective models; (2) our model is more efficient than complex models during inference time since it requires fewer features; (3) the predictive model is readable and understandable; and (4) one of our new QPP features is consistently selected across different collections, proving its usefulness.
Learning to Recommend via Meta Parameter Partition
Authors: Liang Zhao, Yang Wang, daxiang dong, Hao Tian
Date: 2019-12-04 00:00:00
Tasks: Meta-Learning

In this paper we propose to solve an important problem in recommendation -- user cold start, based on meta leaning method. Previous meta learning approaches finetune all parameters for each new user, which is both computing and storage expensive. In contrast, we divide model parameters into fixed and adaptive parts and develop a two-stage meta learning algorithm to learn them separately. The fixed part, capturing user invariant features, is shared by all users and is learned during offline meta learning stage. The adaptive part, capturing user specific features, is learned during online meta learning stage. By decoupling user invariant parameters from user dependent parameters, the proposed approach is more efficient and storage cheaper than previous methods. It also has potential to deal with catastrophic forgetting while continually adapting for streaming coming users. Experiments on production data demonstrates that the proposed method converges faster and to a better performance than baseline methods. Meta-training without online meta model finetuning increases the AUC from 72.24% to 74.72% (2.48% absolute improvement). Online meta training achieves a further gain of 2.46\% absolute improvement comparing with offline meta training.
Shared Visual Abstractions
Authors: Tom White
Date: 2019-11-19 00:00:00

This paper presents abstract art created by neural networks and broadly recognizable across various computer vision systems. The existence of abstract forms that trigger specific labels independent of neural architecture or training set suggests convolutional neural networks build shared visual representations for the categories they understand. Computer vision classifiers encountering these drawings often respond with strong responses for specific labels - in extreme cases stronger than all examples from the validation set. By surveying human subjects we confirm that these abstract artworks are also broadly recognizable by people, suggesting visual representations triggered by these drawings are shared across human and computer vision systems.
FaultNet: Faulty Rail-Valves Detection using Deep Learning and Computer Vision
Authors: Ramanpreet Singh Pahwa, Jin Chao, Jestine Paul, Yiqun Li, Ma Tin Lay Nwe, Shudong Xie, Ashish James, ArulMurugan Ambikapathi, Zeng Zeng, Vijay Ramaseshan Chandrasekhar
Date: 2019-11-09 00:00:00
Tasks: Defect Detection, Fault Detection, Image Segmentation, Segmentation, Semantic Segmentation

Regular inspection of rail valves and engines is an important task to ensure the safety and efficiency of railway networks around the globe. Over the past decade, computer vision and pattern recognition based techniques have gained traction for such inspection and defect detection tasks. An automated end-to-end trained system can potentially provide a low-cost, high throughput, and cheap alternative to manual visual inspection of these components. However, such systems require a huge amount of defective images for networks to understand complex defects. In this paper, a multi-phase deep learning based technique is proposed to perform accurate fault detection of rail-valves. Our approach uses a two-step method to perform high precision image segmentation of rail-valves resulting in pixel-wise accurate segmentation. Thereafter, a computer vision technique is used to identify faulty valves. We demonstrate that the proposed approach results in improved detection performance when compared to current state-of-theart techniques used in fault detection.
A Neural Network Based on the Johnson $S_\mathrm{U}$ Translation System and Related Application to Electromyogram Classification
Authors: Hideaki Hayashi, Taro Shibanoki, Toshio Tsuji
Date: 2019-11-14 00:00:00
Tasks: Classification, General Classification, Hyperparameter Optimization, Translation

Electromyogram (EMG) classification is a key technique in EMG-based control systems. The existing EMG classification methods do not consider the characteristics of EMG features that the distribution has skewness and kurtosis, causing drawbacks such as the requirement of hyperparameter tuning. In this paper, we propose a neural network based on the Johnson $S_\mathrm{U}$ translation system that is capable of representing distributions with skewness and kurtosis. The Johnson system is a normalizing translation that transforms non-normal data to a normal distribution, thereby enabling the representation of a wide range of distributions. In this study, a discriminative model based on the multivariate Johnson $S_\mathrm{U}$ translation system is transformed into a linear combination of coefficients and input vectors using log-linearization. This is then incorporated into a neural network structure, thereby allowing the calculation of the posterior probability of the input vectors for each class and the determination of model parameters as weight coefficients of the network. The uniqueness of convergence of the network learning is theoretically guaranteed. In the experiments, the suitability of the proposed network for distributions including skewness and kurtosis is evaluated using artificially generated data. Its applicability for real biological data is also evaluated via an EMG classification experiment. The results show that the proposed network achieves high classification performance without the need for hyperparameter optimization.
Ultra-Reliable and Low-Latency Vehicular Communication: An Active Learning Approach
Authors: Mohamed K. Abdel-Aziz, Sumudu Samarakoon, Mehdi Bennis, Walid Saad
Date: 2019-11-27 00:00:00
Tasks: Active Learning, GPR

In this letter, an age of information (AoI)-aware transmission power and resource block (RB) allocation technique for vehicular communication networks is proposed. Due to the highly dynamic nature of vehicular networks, gaining a prior knowledge about the network dynamics, i.e., wireless channels and interference, in order to allocate resources, is challenging. Therefore, to effectively allocate power and RBs, the proposed approach allows the network to actively learn its dynamics by balancing a tradeoff between minimizing the probability that the vehicles' AoI exceeds a predefined threshold and maximizing the knowledge about the network dynamics. In this regard, using a Gaussian process regression (GPR) approach, an online decentralized strategy is proposed to actively learn the network dynamics, estimate the vehicles' future AoI, and proactively allocate resources. Simulation results show a significant improvement in terms of AoI violation probability, compared to several baselines, with a reduction of at least 50%.