pwc-archive/papers-with-abstracts huggingface dataset:
FiniteNet: A Fully Convolutional LSTM Network Architecture for Time-Dependent Partial Differential Equations
Authors: Ben Stevens, Tim Colonius
Date: 2020-02-07 00:00:00

In this work, we present a machine learning approach for reducing the error when numerically solving time-dependent partial differential equations (PDE). We use a fully convolutional LSTM network to exploit the spatiotemporal dynamics of PDEs. The neural network serves to enhance finite-difference and finite-volume methods (FDM/FVM) that are commonly used to solve PDEs, allowing us to maintain guarantees on the order of convergence of our method. We train the network on simulation data, and show that our network can reduce error by a factor of 2 to 3 compared to the baseline algorithms. We demonstrate our method on three PDEs that each feature qualitatively different dynamics. We look at the linear advection equation, which propagates its initial conditions at a constant speed, the inviscid Burgers' equation, which develops shockwaves, and the Kuramoto-Sivashinsky (KS) equation, which is chaotic.
Blind Source Separation for NMR Spectra with Negative Intensity
Authors: Ryan J. McCarty, Nimish Ronghe, Mandy Woo, Todd M. Alam
Date: 2020-02-07 00:00:00
Tasks: Benchmarking, blind source separation

NMR spectral datasets, especially in systems with limited samples, can be difficult to interpret if they contain multiple chemical components (phases, polymorphs, molecules, crystals, glasses, etc...) and the possibility of overlapping resonances. In this paper, we benchmark several blind source separation techniques for analysis of NMR spectral datasets containing negative intensity. For benchmarking purposes, we generated a large synthetic datasbase of quadrupolar solid-state NMR-like spectra that model spin-lattice T1 relaxation or nutation tip/flip angle experiments. Our benchmarking approach focused exclusively on the ability of blind source separation techniques to reproduce the spectra of the underlying pure components. In general, we find that FastICA (Fast Independent Component Analysis), SIMPLISMA (SIMPLe-to-use-Interactive Self-modeling Mixture Analysis), and NNMF (Non-Negative Matrix Factorization) are top-performing techniques. We demonstrate that dataset normalization approaches prior to blind source separation do not considerably improve outcomes. Within the range of noise levels studied, we did not find drastic changes to the ranking of techniques. The accuracy of FastICA and SIMPLISMA degrades quickly if excess (unreal) pure components are predicted. Our results indicate poor performance of SVD (Singular Value Decomposition) methods, and we propose alternative techniques for matrix initialization. The benchmarked techniques are also applied to real solid state NMR datasets. In general, the recommendations from the synthetic datasets agree with the recommendations and results from the real data analysis. The discussion provides some additional recommendations for spectroscopists applying blind source separation to NMR datasets, and for future benchmark studies.
A Novel Evolution Strategy with Directional Gaussian Smoothing for Blackbox Optimization
Authors: Jiaxin Zhang, Hoang Tran, Dan Lu, Guannan Zhang
Date: 2020-02-07 00:00:00
Tasks: global-optimization

We propose an improved evolution strategy (ES) using a novel nonlocal gradient operator for high-dimensional black-box optimization. Standard ES methods with $d$-dimensional Gaussian smoothing suffer from the curse of dimensionality due to the high variance of Monte Carlo (MC) based gradient estimators. To control the variance, Gaussian smoothing is usually limited in a small region, so existing ES methods lack nonlocal exploration ability required for escaping from local minima. We develop a nonlocal gradient operator with directional Gaussian smoothing (DGS) to address this challenge. The DGS conducts 1D nonlocal explorations along $d$ orthogonal directions in $\mathbb{R}^d$, each of which defines a nonlocal directional derivative as a 1D integral. We then use Gauss-Hermite quadrature, instead of MC sampling, to estimate the $d$ 1D integrals to ensure high accuracy (i.e., small variance). Our method enables effective nonlocal exploration to facilitate the global search in high-dimensional optimization. We demonstrate the superior performance of our method in three sets of examples, including benchmark functions for global optimization, and real-world science and engineering applications.
Renofeation: A Simple Transfer Learning Method for Improved Adversarial Robustness
Authors: Ting-Wu Chin, Cha Zhang, Diana Marculescu
Date: 2020-02-07 00:00:00
Tasks: Adversarial Attack, Adversarial Robustness, Transfer Learning

Fine-tuning through knowledge transfer from a pre-trained model on a large-scale dataset is a widely spread approach to effectively build models on small-scale datasets. In this work, we show that a recent adversarial attack designed for transfer learning via re-training the last linear layer can successfully deceive models trained with transfer learning via end-to-end fine-tuning. This raises security concerns for many industrial applications. In contrast, models trained with random initialization without transfer are much more robust to such attacks, although these models often exhibit much lower accuracy. To this end, we propose noisy feature distillation, a new transfer learning method that trains a network from random initialization while achieving clean-data performance competitive with fine-tuning. Code available at https://github.com/cmu-enyac/Renofeation.
DropCluster: A structured dropout for convolutional networks
Authors: Liyan Chen, Philip Gautier, Sergul Aydore
Date: 2020-02-07 00:00:00

Dropout as a regularizer in deep neural networks has been less effective in convolutional layers than in fully connected layers. This is due to the fact that dropout drops features randomly. When features are spatially correlated as in the case of convolutional layers, information about the dropped pixels can still propagate to the next layers via neighboring pixels. In order to address this problem, more structured forms of dropout have been proposed. A drawback of these methods is that they do not adapt to the data. In this work, we introduce a novel structured regularization for convolutional layers, which we call DropCluster. Our regularizer relies on data-driven structure. It finds clusters of correlated features in convolutional layer outputs and drops the clusters randomly at each iteration. The clusters are learned and updated during model training so that they adapt both to the data and to the model weights. Our experiments on the ResNet-50 architecture demonstrate that our approach achieves better performance than DropBlock or other existing structured dropout variants. We also demonstrate the robustness of our approach when the size of training data is limited and when there is corruption in the data at test time.
Mario Level Generation From Mechanics Using Scene Stitching
Authors: Michael Cerny Green, Luvneesh Mugrai, Ahmed Khalifa, Julian Togelius
Date: 2020-02-07 00:00:00

This paper presents a level generation method for Super Mario by stitching together pre-generated "scenes" that contain specific mechanics, using mechanic-sequences from agent playthroughs as input specifications. Given a sequence of mechanics, our system uses an FI-2Pop algorithm and a corpus of scenes to perform automated level authoring. The system outputs levels that have a similar mechanical sequence to the target mechanic sequence but with a different playthrough experience. We compare our system to a greedy method that selects scenes that maximize the target mechanics. Our system is able to maximize the number of matched mechanics while reducing emergent mechanics using the stitching process compared to the greedy approach.
Learning Whole-body Motor Skills for Humanoids
Authors: Chuanyu Yang, Kai Yuan, Wolfgang Merkt, Taku Komura, Sethu Vijayakumar, Zhibin Li
Date: 2020-02-07 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning

This paper presents a hierarchical framework for Deep Reinforcement Learning that acquires motor skills for a variety of push recovery and balancing behaviors, i.e., ankle, hip, foot tilting, and stepping strategies. The policy is trained in a physics simulator with realistic setting of robot model and low-level impedance control that are easy to transfer the learned skills to real robots. The advantage over traditional methods is the integration of high-level planner and feedback control all in one single coherent policy network, which is generic for learning versatile balancing and recovery motions against unknown perturbations at arbitrary locations (e.g., legs, torso). Furthermore, the proposed framework allows the policy to be learned quickly by many state-of-the-art learning algorithms. By comparing our learned results to studies of preprogrammed, special-purpose controllers in the literature, self-learned skills are comparable in terms of disturbance rejection but with additional advantages of producing a wide range of adaptive, versatile and robust behaviors.
Manipulating Reinforcement Learning: Poisoning Attacks on Cost Signals
Authors: Yunhan Huang, Quanyan Zhu
Date: 2020-02-07 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This chapter studies emerging cyber-attacks on reinforcement learning (RL) and introduces a quantitative approach to analyze the vulnerabilities of RL. Focusing on adversarial manipulation on the cost signals, we analyze the performance degradation of TD($\lambda$) and $Q$-learning algorithms under the manipulation. For TD($\lambda$), the approximation learned from the manipulated costs has an approximation error bound proportional to the magnitude of the attack. The effect of the adversarial attacks on the bound does not depend on the choice of $\lambda$. In $Q$-learning, we show that $Q$-learning algorithms converge under stealthy attacks and bounded falsifications on cost signals. We characterize the relation between the falsified cost and the $Q$-factors as well as the policy learned by the learning agent which provides fundamental limits for feasible offensive and defensive moves. We propose a robust region in terms of the cost within which the adversary can never achieve the targeted policy. We provide conditions on the falsified cost which can mislead the agent to learn an adversary's favored policy. A case study of TD($\lambda$) learning is provided to corroborate the results.
Attentive Group Equivariant Convolutional Networks
Authors: David W. Romero, Erik J. Bekkers, Jakub M. Tomczak, Mark Hoogendoorn
Date: 2020-02-07 00:00:00
Proceeding: ICML 2020 1

Although group convolutional networks are able to learn powerful representations based on symmetry patterns, they lack explicit means to learn meaningful relationships among them (e.g., relative positions and poses). In this paper, we present attentive group equivariant convolutions, a generalization of the group convolution, in which attention is applied during the course of convolution to accentuate meaningful symmetry combinations and suppress non-plausible, misleading ones. We indicate that prior work on visual attention can be described as special cases of our proposed framework and show empirically that our attentive group equivariant convolutional networks consistently outperform conventional group convolutional networks on benchmark image datasets. Simultaneously, we provide interpretability to the learned concepts through the visualization of equivariant attention maps.
Overview of chemical ontologies
Authors: Christian Pachl, Nils Frank, Jan Breitbart, Stefan Bräse
Date: 2020-02-07 00:00:00

Ontologies order and interconnect knowledge of a certain field in a formal and semantic way so that they are machine-parsable. They try to define allwhere acceptable definition of concepts and objects, classify them, provide properties as well as interconnect them with relations (e.g. "A is a special case of B"). More precisely, Tom Gruber defines Ontologies as a "specification of a conceptualization; [...] a description (like a formal specification of a program) of the concepts and relationships that can exist for an agent or a community of agents." [1] An Ontology is made of Individuals which are organized in Classes. Both can have Attributes and Relations among themselves. Some complex Ontologies define Restrictions, Rules and Events which change attributes or relations. To be computer accessible they are written in certain ontology languages, like the OBO language or the more used Common Algebraic Specification Language. With the rising of a digitalized, interconnected and globalized world, where common standards have to be found, ontologies are of great interest. [2] Yet, the development of chemical ontologies is in the beginning. Indeed, some interesting basic approaches towards chemical ontologies can be found, but nevertheless they suffer from two main flaws. Firstly, we found that they are mostly only fragmentary completed or are still in an architecture state. Secondly, apparently no chemical ontology is widespread accepted. Therefore, we herein try to describe the major ontology-developments in the chemical related fields Ontologies about chemical analytical methods, Ontologies about name reactions and Ontologies about scientific units.
Anomaly Detection using Deep Autoencoders for in-situ Wastewater Systems Monitoring Data
Authors: Stefania Russo, Andy Disch, Frank Blumensaat, Kris Villez
Date: 2020-02-07 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Analysis

Due to the growing amount of data from in-situ sensors in wastewater systems, it becomes necessary to automatically identify abnormal behaviours and ensure high data quality. This paper proposes an anomaly detection method based on a deep autoencoder for in-situ wastewater systems monitoring data. The autoencoder architecture is based on 1D Convolutional Neural Network (CNN) layers where the convolutions are performed over the inputs across the temporal axis of the data. Anomaly detection is then performed based on the reconstruction error of the decoding stage. The approach is validated on multivariate time series from in-sewer process monitoring data. We discuss the results and the challenge of labelling anomalies in complex time series. We suggest that our proposed approach can support the domain experts in the identification of anomalies.
Machine Education: Designing semantically ordered and ontologically guided modular neural networks
Authors: Hussein A. Abbass, Sondoss Elsawah, Eleni Petraki, Robert Hunjet
Date: 2020-02-07 00:00:00

The literature on machine teaching, machine education, and curriculum design for machines is in its infancy with sparse papers on the topic primarily focusing on data and model engineering factors to improve machine learning. In this paper, we first discuss selected attempts to date on machine teaching and education. We then bring theories and methodologies together from human education to structure and mathematically define the core problems in lesson design for machine education and the modelling approaches required to support the steps for machine education. Last, but not least, we offer an ontology-based methodology to guide the development of lesson plans to produce transparent and explainable modular learning machines, including neural networks.
Exploiting Temporal Coherence for Multi-modal Video Categorization
Authors: Palash Goyal, Saurabh Sahu, Shalini Ghosh, Chul Lee
Date: 2020-02-07 00:00:00
Tasks: object-detection, Object Detection, Scene Understanding

Multimodal ML models can process data in multiple modalities (e.g., video, images, audio, text) and are useful for video content analysis in a variety of problems (e.g., object detection, scene understanding). In this paper, we focus on the problem of video categorization by using a multimodal approach. We have developed a novel temporal coherence-based regularization approach, which applies to different types of models (e.g., RNN, NetVLAD, Transformer). We demonstrate through experiments how our proposed multimodal video categorization models with temporal coherence out-perform strong state-of-the-art baseline models.
CIFAR-10 Image Classification Using Feature Ensembles
Authors: Felipe O. Giuste, Juan C. Vizcarra
Date: 2020-02-07 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Transfer Learning

Image classification requires the generation of features capable of detecting image patterns informative of group identity. The objective of this study was to classify images from the public CIFAR-10 image dataset by leveraging combinations of disparate image feature sources from both manual and deep learning approaches. Histogram of oriented gradients (HOG) and pixel intensities successfully inform classification (53% and 59% classification accuracy, respectively), yet there is much room for improvement. VGG16 with ImageNet trained weights and a CIFAR-10 optimized model (CIFAR-VGG) further improve upon image classification (60% and 93.43% accuracy, respectively). We further improved classification by utilizing transfer learning to re-establish optimal network weights for VGG16 (TL-VGG) and Inception ResNet v2 (TL-Inception) resulting in significant performance increases (85% and 90.74%, respectively), yet fail to surpass CIFAR-VGG. We hypothesized that if each generated feature set obtained some unique insight into the classification problem, then combining these features would result in greater classification accuracy, surpassing that of CIFAR-VGG. Upon selection of the top 1000 principal components from TL-VGG, TL-Inception, HOG, pixel intensities, and CIFAR-VGG, we achieved testing accuracy of 94.6%, lending support to our hypothesis.
Geometric Formulation for Discrete Points and its Applications
Authors: Yuuya Takayama
Date: 2020-02-07 00:00:00
Tasks: BIG-bench Machine Learning

We introduce a novel formulation for geometry on discrete points. It is based on a universal differential calculus, which gives a geometric description of a discrete set by the algebra of functions. We expand this mathematical framework so that it is consistent with differential geometry, and works on spectral graph theory and random walks. Consequently, our formulation comprehensively demonstrates many discrete frameworks in probability theory, physics, applied harmonic analysis, and machine learning. Our approach would suggest the existence of an intrinsic theory and a unified picture of those discrete frameworks.
Parallel Performance-Energy Predictive Modeling of Browsers: Case Study of Servo
Authors: Rohit Zambre, Lars Bergstrom, Laleh Aghababaie Beni, Aparna Chandramowliswharan
Date: 2020-02-06 00:00:00

Mozilla Research is developing Servo, a parallel web browser engine, to exploit the benefits of parallelism and concurrency in the web rendering pipeline. Parallelization results in improved performance for pinterest.com but not for google.com. This is because the workload of a browser is dependent on the web page it is rendering. In many cases, the overhead of creating, deleting, and coordinating parallel work outweighs any of its benefits. In this paper, we model the relationship between web page primitives and a web browser's parallel performance using supervised learning. We discover a feature space that is representative of the parallelism available in a web page and characterize it using seven key features. Additionally, we consider energy usage trade-offs for different levels of performance improvements using automated labeling algorithms. Such a model allows us to predict the degree of parallelism available in a web page and decide whether or not to render a web page in parallel. This modeling is critical for improving the browser's performance and minimizing its energy usage. We evaluate our model by using Servo's layout stage as a case study. Experiments on a quad-core Intel Ivy Bridge (i7-3615QM) laptop show that we can improve performance and energy usage by up to 94.52% and 46.32% respectively on the 535 web pages considered in this study. Looking forward, we identify opportunities to apply this model to other stages of a browser's architecture as well as other performance- and energy-critical devices.
Multilevel Acyclic Hypergraph Partitioning
Authors: Merten Popp, Sebastian Schlag, Christian Schulz, Daniel Seemaier
Date: 2020-02-06 00:00:00
Tasks: hypergraph partitioning, Scheduling

A directed acyclic hypergraph is a generalized concept of a directed acyclic graph, where each hyperedge can contain an arbitrary number of tails and heads. Directed hypergraphs can be used to model data flow and execution dependencies in streaming applications. Thus, hypergraph partitioning algorithms can be used to obtain efficient parallelizations for multiprocessor architectures. However, an acyclicity constraint on the partition is necessary when mapping streaming applications to embedded multiprocessors due to resource restrictions on this type of hardware. The acyclic hypergraph partitioning problem is to partition the hypernodes of a directed acyclic hypergraph into a given number of blocks of roughly equal size such that the corresponding quotient graph is acyclic while minimizing an objective function on the partition. Here, we contribute the first n-level algorithm for the acyclic hypergraph partitioning problem. Our focus is on acyclic hypergraphs where hyperedges can have one head and arbitrary many tails. Based on this, we engineer a memetic algorithm to further reduce communication cost, as well as to improve scheduling makespan on embedded multiprocessor architectures. Experiments indicate that our algorithm outperforms previous algorithms that focus on the directed acyclic graph case which have previously been employed in the application domain. Moreover, our experiments indicate that using the directed hypergraph model for this type of application yields a significantly smaller makespan.
Continuous Silent Speech Recognition using EEG
Authors: Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik
Date: 2020-02-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), EEG, Electroencephalogram (EEG), General Classification, Silent Speech Recognition, speech-recognition, Speech Recognition

In this paper we explore continuous silent speech recognition using electroencephalography (EEG) signals. We implemented a connectionist temporal classification (CTC) automatic speech recognition (ASR) model to translate EEG signals recorded in parallel while subjects were reading English sentences in their mind without producing any voice to text. Our results demonstrate the feasibility of using EEG signals for performing continuous silent speech recognition. We demonstrate our results for a limited English vocabulary consisting of 30 unique sentences.