pwc-archive/papers-with-abstracts huggingface dataset:
SQLR: Short-Term Memory Q-Learning for Elastic Provisioning
Authors: Constantine Ayimba, Paolo Casari, Vincenzo Mancuso
Date: 2019-09-12 00:00:00
Tasks: Blocking, Q-Learning, Reinforcement Learning

As more and more application providers transition to the cloud and deliver their services on a Software as a Service (SaaS) basis, cloud providers need to make their provisioning systems agile enough to meet Service Level Agreements. At the same time they should guard against over-provisioning which limits their capacity to accommodate more tenants. To this end we propose SQLR, a dynamic provisioning system employing a customized model-free reinforcement learning algorithm that is capable of reusing contextual knowledge learned from one workload to optimize resource provisioning for other workload patterns. SQLR achieves results comparable to those where resources are unconstrained, with minimal overhead. Our experiments show that we can reduce the amount of provisioned resources by almost 25% with less than 1% overall service unavailability (due to blocking) while delivering similar response times as those of an over-provisioned system.
Generating Data using Monte Carlo Dropout
Authors: Kristian Miok, Dong Nguyen-Doan, Daniela Zaharie, Marko Robnik-Šikonja
Date: 2019-09-12 00:00:00

For many analytical problems the challenge is to handle huge amounts of available data. However, there are data science application areas where collecting information is difficult and costly, e.g., in the study of geological phenomena, rare diseases, faults in complex systems, insurance frauds, etc. In many such cases, generators of synthetic data with the same statistical and predictive properties as the actual data allow efficient simulations and development of tools and applications. In this work, we propose the incorporation of Monte Carlo Dropout method within Autoencoder (MCD-AE) and Variational Autoencoder (MCD-VAE) as efficient generators of synthetic data sets. As the Variational Autoencoder (VAE) is one of the most popular generator techniques, we explore its similarities and differences to the proposed methods. We compare the generated data sets with the original data based on statistical properties, structural similarity, and predictive similarity. The results obtained show a strong similarity between the results of VAE, MCD-VAE and MCD-AE; however, the proposed methods are faster and can generate values similar to specific selected initial instances.
Ensemble Learning Based Convex Approximation of Three-Phase Power Flow
Authors: Ren Hu, QiFeng Li, Feng Qiu
Date: 2019-09-12 00:00:00
Tasks: Computational Efficiency, Ensemble Learning

Though the convex optimization has been widely used in power systems, it still cannot guarantee to yield a tight (accurate) solution to some problems. To mitigate this issue, this paper proposes an ensemble learning based convex approximation for AC power flow equations that differs from the existing convex relaxations. The proposed approach is based on quadratic power flow equations in rectangular coordinates and it can be used in both balanced and unbalanced three-phase power networks. To develop this data-driven convex approximation of power flows, the polynomial regression (PR) is first deployed as a basic learner to fit convex relationships between the independent and dependent variables. Then, ensemble learning algorithms such as gradient boosting (GB) and bagging are introduced to combine learners to boost model performance. Based on the learned convex approximation of power flows, optimal power flow (OPF) is formulated as a convex quadratic programming problem. The simulation results on IEEE standard cases show that, in the context of solving OPF, the proposed data-driven convex approximation outperforms the conventional SDP relaxation in both accuracy and computational efficiency, especially in the cases that the conventional SDP relaxation fails.
Sams-Net: A Sliced Attention-based Neural Network for Music Source Separation
Authors: Tingle Li, Jia-Wei Chen, Haowen Hou, Ming Li
Date: 2019-09-12 00:00:00
Tasks: Audio Source Separation, Music Source Separation

Convolutional Neural Network (CNN) or Long short-term memory (LSTM) based models with the input of spectrogram or waveforms are commonly used for deep learning based audio source separation. In this paper, we propose a Sliced Attention-based neural network (Sams-Net) in the spectrogram domain for the music source separation task. It enables spectral feature interactions with multi-head attention mechanism, achieves easier parallel computing and has a larger receptive field compared with LSTMs and CNNs respectively. Experimental results on the MUSDB18 dataset show that the proposed method, with fewer parameters, outperforms most of the state-of-the-art DNN-based methods.
Rethinking the CSC Model for Natural Images
Authors: Dror Simon, Michael Elad
Date: 2019-09-12 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Color Image Denoising, Denoising, model

Sparse representation with respect to an overcomplete dictionary is often used when regularizing inverse problems in signal and image processing. In recent years, the Convolutional Sparse Coding (CSC) model, in which the dictionary consists of shift-invariant filters, has gained renewed interest. While this model has been successfully used in some image processing problems, it still falls behind traditional patch-based methods on simple tasks such as denoising. In this work we provide new insights regarding the CSC model and its capability to represent natural images, and suggest a Bayesian connection between this model and its patch-based ancestor. Armed with these observations, we suggest a novel feed-forward network that follows an MMSE approximation process to the CSC model, using strided convolutions. The performance of this supervised architecture is shown to be on par with state of the art methods while using much fewer parameters.
A tale of two toolkits, report the first: benchmarking time series classification algorithms for correctness and efficiency
Authors: Anthony Bagnall, Franz Király, Markus Löning, Matthew Middlehurst, George Oastler
Date: 2019-09-12 00:00:00
Tasks: Benchmarking, General Classification, Time Series, Time Series Analysis, Time Series Classification

sktime is an open source, Python based, sklearn compatible toolkit for time series analysis developed by researchers at the University of East Anglia (UEA), University College London and the Alan Turing Institute. A key initial goal for sktime was to provide time series classification functionality equivalent to that available in a related java package, tsml, also developed at UEA. We describe the implementation of six such classifiers in sktime and compare them to their tsml equivalents. We demonstrate correctness through equivalence of accuracy on a range of standard test problems and compare the build time of the different implementations. We find that there is significant difference in accuracy on only one of the six algorithms we look at (Proximity Forest). This difference is causing us some pain in debugging. We found a much wider range of difference in efficiency. Again, this was not unexpected, but it does highlight ways both toolkits could be improved.
CvxNet: Learnable Convex Decomposition
Authors: Boyang Deng, Kyle Genova, Soroosh Yazdani, Sofien Bouaziz, Geoffrey Hinton, Andrea Tagliasacchi
Date: 2019-09-12 00:00:00
Proceeding: CVPR 2020 6
Tasks: 3D Reconstruction, Image to 3D, Retrieval

Any solid object can be decomposed into a collection of convex polytopes (in short, convexes). When a small number of convexes are used, such a decomposition can be thought of as a piece-wise approximation of the geometry. This decomposition is fundamental in computer graphics, where it provides one of the most common ways to approximate geometry, for example, in real-time physics simulation. A convex object also has the property of being simultaneously an explicit and implicit representation: one can interpret it explicitly as a mesh derived by computing the vertices of a convex hull, or implicitly as the collection of half-space constraints or support functions. Their implicit representation makes them particularly well suited for neural network training, as they abstract away from the topology of the geometry they need to represent. However, at testing time, convexes can also generate explicit representations -- polygonal meshes -- which can then be used in any downstream application. We introduce a network architecture to represent a low dimensional family of convexes. This family is automatically derived via an auto-encoding process. We investigate the applications of this architecture including automatic convex decomposition, image to 3D reconstruction, and part-based shape retrieval.
Propagation of spiking moments in linear Hawkes networks
Authors: Matthieu Gilson, Jean-Pascal Pfister
Date: 2018-09-18 00:00:00

The present paper provides exact mathematical expressions for the high-order moments of spiking activity in a recurrently-connected network of linear Hawkes processes. It extends previous studies that have explored the case of a (linear) Hawkes network driven by deterministic intensity functions to the case of a stimulation by external inputs (rate functions or spike trains) with arbitrary correlation structure. Our approach describes the spatio-temporal filtering induced by the afferent and recurrent connectivities (with arbitrary synaptic response kernels) using operators acting on the input moments. This algebraic viewpoint provides intuition about how the network ingredients shape the input-output mapping for moments, as well as cumulants. We also show using numerical simulation that our results hold for neurons with refractoriness implemented by self-inhibition, provided the corresponding negative feedback for each neuron only mildly alters its mean firing probability.
VeREFINE: Integrating Object Pose Verification with Physics-guided Iterative Refinement
Authors: Dominik Bauer, Timothy Patten, Markus Vincze
Date: 2019-09-12 00:00:00
Tasks: Object, Pose Estimation

Accurate and robust object pose estimation for robotics applications requires verification and refinement steps. In this work, we propose to integrate hypotheses verification with object pose refinement guided by physics simulation. This allows the physical plausibility of individual object pose estimates and the stability of the estimated scene to be considered in a unified optimization. The proposed method is able to adapt to scenes of multiple objects and efficiently focuses on refining the most promising object poses in multi-hypotheses scenarios. We call this integrated approach VeREFINE and evaluate it on three datasets with varying scene complexity. The generality of the approach is shown by using three state-of-the-art pose estimators and three baseline refiners. Results show improvements over all baselines and on all datasets. Furthermore, our approach is applied in real-world grasping experiments and outperforms competing methods in terms of grasp success rate. Code is publicly available at github.com/dornik/verefine.
GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended Animation
Authors: Jiawei Zhang, Lin Meng
Date: 2019-09-12 00:00:00
Tasks: Graph Neural Network, Node Classification

The existing graph neural networks (GNNs) based on the spectral graph convolutional operator have been criticized for its performance degradation, which is especially common for the models with deep architectures. In this paper, we further identify the suspended animation problem with the existing GNNs. Such a problem happens when the model depth reaches the suspended animation limit, and the model will not respond to the training data any more and become not learnable. Analysis about the causes of the suspended animation problem with existing GNNs will be provided in this paper, whereas several other peripheral factors that will impact the problem will be reported as well. To resolve the problem, we introduce the GResNet (Graph Residual Network) framework in this paper, which creates extensively connected highways to involve nodes' raw features or intermediate representations throughout the graph for all the model layers. Different from the other learning settings, the extensive connections in the graph data will render the existing simple residual learning methods fail to work. We prove the effectiveness of the introduced new graph residual terms from the norm preservation perspective, which will help avoid dramatic changes to the node's representations between sequential layers. Detailed studies about the GResNet framework for many existing GNNs, including GCN, GAT and LoopyNet, will be reported in the paper with extensive empirical experiments on real-world benchmark datasets.
InstructableCrowd: Creating IF-THEN Rules for Smartphones via Conversations with the Crowd
Authors: Ting-Hao 'Kenneth' Huang, Amos Azaria, Oscar J. Romero, Jeffrey P. Bigham
Date: 2019-09-12 00:00:00

Natural language interfaces have become a common part of modern digital life. Chatbots utilize text-based conversations to communicate with users; personal assistants on smartphones such as Google Assistant take direct speech commands from their users; and speech-controlled devices such as Amazon Echo use voice as their only input mode. In this paper, we introduce InstructableCrowd, a crowd-powered system that allows users to program their devices via conversation. The user verbally expresses a problem to the system, in which a group of crowd workers collectively respond and program relevant multi-part IF-THEN rules to help the user. The IF-THEN rules generated by InstructableCrowd connect relevant sensor combinations (e.g., location, weather, device acceleration, etc.) to useful effectors (e.g., text messages, device alarms, etc.). Our study showed that non-programmers can use the conversational interface of InstructableCrowd to create IF-THEN rules that have similar quality compared with the rules created manually. InstructableCrowd generally illustrates how users may converse with their devices, not only to trigger simple voice commands, but also to personalize their increasingly powerful and complicated devices.
Lost in Evaluation: Misleading Benchmarks for Bilingual Dictionary Induction
Authors: Yova Kementchedjhieva, Mareike Hartmann, Anders Søgaard
Date: 2019-09-12 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Cross-Lingual Word Embeddings, Word Embeddings

The task of bilingual dictionary induction (BDI) is commonly used for intrinsic evaluation of cross-lingual word embeddings. The largest dataset for BDI was generated automatically, so its quality is dubious. We study the composition and quality of the test sets for five diverse languages from this dataset, with concerning findings: (1) a quarter of the data consists of proper nouns, which can be hardly indicative of BDI performance, and (2) there are pervasive gaps in the gold-standard targets. These issues appear to affect the ranking between cross-lingual embedding systems on individual languages, and the overall degree to which the systems differ in performance. With proper nouns removed from the data, the margin between the top two systems included in the study grows from 3.4% to 17.2%. Manual verification of the predictions, on the other hand, reveals that gaps in the gold standard targets artificially inflate the margin between the two systems on English to Bulgarian BDI from 0.1% to 6.7%. We thus suggest that future research either avoids drawing conclusions from quantitative results on this BDI dataset, or accompanies such evaluation with rigorous error analysis.
Coarse-scale PDEs from fine-scale observations via machine learning
Authors: Seungjoon Lee, Mahdi Kooshkbaghi, Konstantinos Spiliotis, Constantinos I. Siettos, Ioannis G. Kevrekidis
Date: 2019-09-12 00:00:00
Tasks: BIG-bench Machine Learning, Gaussian Processes

Complex spatiotemporal dynamics of physicochemical processes are often modeled at a microscopic level (through e.g. atomistic, agent-based or lattice models) based on first principles. Some of these processes can also be successfully modeled at the macroscopic level using e.g. partial differential equations (PDEs) describing the evolution of the right few macroscopic observables (e.g. concentration and momentum fields). Deriving good macroscopic descriptions (the so-called "closure problem") is often a time-consuming process requiring deep understanding/intuition about the system of interest. Recent developments in data science provide alternative ways to effectively extract/learn accurate macroscopic descriptions approximating the underlying microscopic observations. In this paper, we introduce a data-driven framework for the identification of unavailable coarse-scale PDEs from microscopic observations via machine learning algorithms. Specifically, using Gaussian Processes, Artificial Neural Networks, and/or Diffusion Maps, the proposed framework uncovers the relation between the relevant macroscopic space fields and their time evolution (the right-hand-side of the explicitly unavailable macroscopic PDE). Interestingly, several choices equally representative of the data can be discovered. The framework will be illustrated through the data-driven discovery of macroscopic, concentration-level PDEs resulting from a fine-scale, Lattice Boltzmann level model of a reaction/transport process. Once the coarse evolution law is identified, it can be simulated to produce long-term macroscopic predictions. Different features (pros as well as cons) of alternative machine learning algorithms for performing this task (Gaussian Processes and Artificial Neural Networks), are presented and discussed.
Closed-loop Model Selection for Kernel-based Models using Bayesian Optimization
Authors: Thomas Beckers, Somil Bansal, Claire J. Tomlin, Sandra Hirche
Date: 2019-09-12 00:00:00
Tasks: Bayesian Optimization, Model Selection

Kernel-based nonparametric models have become very attractive for model-based control approaches for nonlinear systems. However, the selection of the kernel and its hyperparameters strongly influences the quality of the learned model. Classically, these hyperparameters are optimized to minimize the prediction error of the model but this process totally neglects its later usage in the control loop. In this work, we present a framework to optimize the kernel and hyperparameters of a kernel-based model directly with respect to the closed-loop performance of the model. Our framework uses Bayesian optimization to iteratively refine the kernel-based model using the observed performance on the actual system until a desired performance is achieved. We demonstrate the proposed approach in a simulation and on a 3-DoF robotic arm.
Augmented Data Science: Towards Industrialization and Democratization of Data Science
Authors: Huseyin Uzunalioglu, Jin Cao, Chitra Phadke, Gerald Lehmann, Ahmet Akyamac, Ran He, Jeongran Lee, Maria Able
Date: 2019-09-12 00:00:00
Tasks: Diversity

Conversion of raw data into insights and knowledge requires substantial amounts of effort from data scientists. Despite breathtaking advances in Machine Learning (ML) and Artificial Intelligence (AI), data scientists still spend the majority of their effort in understanding and then preparing the raw data for ML/AI. The effort is often manual and ad hoc, and requires some level of domain knowledge. The complexity of the effort increases dramatically when data diversity, both in form and context, increases. In this paper, we introduce our solution, Augmented Data Science (ADS), towards addressing this "human bottleneck" in creating value from diverse datasets. ADS is a data-driven approach and relies on statistics and ML to extract insights from any data set in a domain-agnostic way to facilitate the data science process. Key features of ADS are the replacement of rudimentary data exploration and processing steps with automation and the augmentation of data scientist judgment with automatically-generated insights. We present building blocks of our end-to-end solution and provide a case study to exemplify its capabilities.
UER: An Open-Source Toolkit for Pre-training Models
Authors: Zhe Zhao, Hui Chen, Jinbin Zhang, Xin Zhao, Tao Liu, Wei Lu, Xi Chen, Haotang Deng, Qi Ju, Xiaoyong Du
Date: 2019-09-12 00:00:00
Proceeding: IJCNLP 2019 11

Existing works, including ELMO and BERT, have revealed the importance of pre-training for NLP tasks. While there does not exist a single pre-training model that works best in all cases, it is of necessity to develop a framework that is able to deploy various pre-training models efficiently. For this purpose, we propose an assemble-on-demand pre-training toolkit, namely Universal Encoder Representations (UER). UER is loosely coupled, and encapsulated with rich modules. By assembling modules on demand, users can either reproduce a state-of-the-art pre-training model or develop a pre-training model that remains unexplored. With UER, we have built a model zoo, which contains pre-trained models based on different corpora, encoders, and targets (objectives). With proper pre-trained models, we could achieve new state-of-the-art results on a range of downstream datasets.
A Camera That CNNs: Towards Embedded Neural Networks on Pixel Processor Arrays
Authors: Laurie Bose, Jianing Chen, Stephen J. Carey, Piotr Dudek, Walterio Mayol-Cuevas
Date: 2019-09-12 00:00:00
Proceeding: ICCV 2019 10

We present a convolutional neural network implementation for pixel processor array (PPA) sensors. PPA hardware consists of a fine-grained array of general-purpose processing elements, each capable of light capture, data storage, program execution, and communication with neighboring elements. This allows images to be stored and manipulated directly at the point of light capture, rather than having to transfer images to external processing hardware. Our CNN approach divides this array up into 4x4 blocks of processing elements, essentially trading-off image resolution for increased local memory capacity per 4x4 "pixel". We implement parallel operations for image addition, subtraction and bit-shifting images in this 4x4 block format. Using these components we formulate how to perform ternary weight convolutions upon these images, compactly store results of such convolutions, perform max-pooling, and transfer the resulting sub-sampled data to an attached micro-controller. We train ternary weight filter CNNs for digit recognition and a simple tracking task, and demonstrate inference of these networks upon the SCAMP5 PPA system. This work represents a first step towards embedding neural network processing capability directly onto the focal plane of a sensor.
A Channel-Pruned and Weight-Binarized Convolutional Neural Network for Keyword Spotting
Authors: Jiancheng Lyu, Spencer Sheen
Date: 2019-09-12 00:00:00
Tasks: Binarization, General Classification, Keyword Spotting

We study channel number reduction in combination with weight binarization (1-bit weight precision) to trim a convolutional neural network for a keyword spotting (classification) task. We adopt a group-wise splitting method based on the group Lasso penalty to achieve over 50% channel sparsity while maintaining the network performance within 0.25% accuracy loss. We show an effective three-stage procedure to balance accuracy and sparsity in network training.