pwc-archive/papers-with-abstracts huggingface dataset:
A Novel Framework for Neural Architecture Search in the Hill Climbing Domain
Authors: Mudit Verma, Pradyumna Sinha, Karan Goyal, Apoorva Verma, Seba Susan
Date: 2021-02-22 00:00:00
Tasks: GPU, Neural Architecture Search, Reinforcement Learning (RL)

Neural networks have now long been used for solving complex problems of image domain, yet designing the same needs manual expertise. Furthermore, techniques for automatically generating a suitable deep learning architecture for a given dataset have frequently made use of reinforcement learning and evolutionary methods which take extensive computational resources and time. We propose a new framework for neural architecture search based on a hill-climbing procedure using morphism operators that makes use of a novel gradient update scheme. The update is based on the aging of neural network layers and results in the reduction in the overall training time. This technique can search in a broader search space which subsequently yields competitive results. We achieve a 4.96% error rate on the CIFAR-10 dataset in 19.4 hours of a single GPU training.
SPINN: Sparse, Physics-based, and partially Interpretable Neural Networks for PDEs
Authors: Amuthan A. Ramabathiran, Prabhu Ramachandran
Date: 2021-02-25 00:00:00

We introduce a class of Sparse, Physics-based, and partially Interpretable Neural Networks (SPINN) for solving ordinary and partial differential equations (PDEs). By reinterpreting a traditional meshless representation of solutions of PDEs we develop a class of sparse neural network architectures that are partially interpretable. The SPINN model we propose here serves as a seamless bridge between two extreme modeling tools for PDEs, namely dense neural network based methods like Physics Informed Neural Networks (PINNs) and traditional mesh-free numerical methods, thereby providing a novel means to develop a new class of hybrid algorithms that build on the best of both these viewpoints. A unique feature of the SPINN model that distinguishes it from other neural network based approximations proposed earlier is that it is (i) interpretable, in a particular sense made precise in the work, and (ii) sparse in the sense that it has much fewer connections than typical dense neural networks used for PDEs. Further, the SPINN algorithm implicitly encodes mesh adaptivity and is able to handle discontinuities in the solutions. In addition, we demonstrate that Fourier series representations can also be expressed as a special class of SPINN and propose generalized neural network analogues of Fourier representations. We illustrate the utility of the proposed method with a variety of examples involving ordinary differential equations, elliptic, parabolic, hyperbolic and nonlinear partial differential equations, and an example in fluid dynamics.
Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling
Authors: Gregory W. Benton, Wesley J. Maddox, Sanae Lotfi, Andrew Gordon Wilson
Date: 2021-02-25 00:00:00

With a better understanding of the loss surfaces for multilayer networks, we can build more robust and accurate training procedures. Recently it was discovered that independently trained SGD solutions can be connected along one-dimensional paths of near-constant training loss. In this paper, we show that there are mode-connecting simplicial complexes that form multi-dimensional manifolds of low loss, connecting many independently trained models. Inspired by this discovery, we show how to efficiently build simplicial complexes for fast ensembling, outperforming independently trained deep ensembles in accuracy, calibration, and robustness to dataset shift. Notably, our approach only requires a few training epochs to discover a low-loss simplex, starting from a pre-trained solution. Code is available at https://github.com/g-benton/loss-surface-simplexes.
Iterative Bounding MDPs: Learning Interpretable Policies via Non-Interpretable Methods
Authors: Nicholay Topin, Stephanie Milani, Fei Fang, Manuela Veloso
Date: 2021-02-25 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Current work in explainable reinforcement learning generally produces policies in the form of a decision tree over the state space. Such policies can be used for formal safety verification, agent behavior prediction, and manual inspection of important features. However, existing approaches fit a decision tree after training or use a custom learning procedure which is not compatible with new learning techniques, such as those which use neural networks. To address this limitation, we propose a novel Markov Decision Process (MDP) type for learning decision tree policies: Iterative Bounding MDPs (IBMDPs). An IBMDP is constructed around a base MDP so each IBMDP policy is guaranteed to correspond to a decision tree policy for the base MDP when using a method-agnostic masking procedure. Because of this decision tree equivalence, any function approximator can be used during training, including a neural network, while yielding a decision tree policy for the base MDP. We present the required masking procedure as well as a modified value update step which allows IBMDPs to be solved using existing algorithms. We apply this procedure to produce IBMDP variants of recent reinforcement learning methods. We empirically show the benefits of our approach by solving IBMDPs to produce decision tree policies for the base MDPs.
Truncated Log-concave Sampling with Reflective Hamiltonian Monte Carlo
Authors: Apostolos Chalkis, Vissarion Fisikopoulos, Marios Papachristou, Elias Tsigaridas
Date: 2021-02-25 00:00:00

We introduce Reflective Hamiltonian Monte Carlo (ReHMC), an HMC-based algorithm, to sample from a log-concave distribution restricted to a convex body. We prove that, starting from a warm start, the walk mixes to a log-concave target distribution $\pi(x) \propto e^{-f(x)}$, where $f$ is $L$-smooth and $m$-strongly-convex, within accuracy $\varepsilon$ after $\widetilde O(\kappa d^2 \ell^2 \log (1 / \varepsilon))$ steps for a well-rounded convex body where $\kappa = L / m$ is the condition number of the negative log-density, $d$ is the dimension, $\ell$ is an upper bound on the number of reflections, and $\varepsilon$ is the accuracy parameter. We also developed an efficient open source implementation of ReHMC and we performed an experimental study on various high-dimensional data-sets. The experiments suggest that ReHMC outperfroms Hit-and-Run and Coordinate-Hit-and-Run regarding the time it needs to produce an independent sample and introduces practical truncated sampling in thousands of dimensions.
Towards Robust Graph Contrastive Learning
Authors: Nikola Jovanović, Zhao Meng, Lukas Faber, Roger Wattenhofer
Date: 2021-02-25 00:00:00
Tasks: Adversarial Robustness, Contrastive Learning, Self-Supervised Learning

We study the problem of adversarially robust self-supervised learning on graphs. In the contrastive learning framework, we introduce a new method that increases the adversarial robustness of the learned representations through i) adversarial transformations and ii) transformations that not only remove but also insert edges. We evaluate the learned representations in a preliminary set of experiments, obtaining promising results. We believe this work takes an important step towards incorporating robustness as a viable auxiliary task in graph contrastive learning.
Even your Teacher Needs Guidance: Ground-Truth Targets Dampen Regularization Imposed by Self-Distillation
Authors: Kenneth Borup, Lars N. Andersen
Date: 2021-02-25 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Knowledge Distillation, Self-Knowledge Distillation

Knowledge distillation is classically a procedure where a neural network is trained on the output of another network along with the original targets in order to transfer knowledge between the architectures. The special case of self-distillation, where the network architectures are identical, has been observed to improve generalization accuracy. In this paper, we consider an iterative variant of self-distillation in a kernel regression setting, in which successive steps incorporate both model outputs and the ground-truth targets. This allows us to provide the first theoretical results on the importance of using the weighted ground-truth targets in self-distillation. Our focus is on fitting nonlinear functions to training data with a weighted mean square error objective function suitable for distillation, subject to $\ell_2$ regularization of the model parameters. We show that any such function obtained with self-distillation can be calculated directly as a function of the initial fit, and that infinite distillation steps yields the same optimization problem as the original with amplified regularization. Furthermore, we provide a closed form solution for the optimal choice of weighting parameter at each step, and show how to efficiently estimate this weighting parameter for deep learning and significantly reduce the computational requirements compared to a grid search.
On The Effect of Auxiliary Tasks on Representation Dynamics
Authors: Clare Lyle, Mark Rowland, Georg Ostrovski, Will Dabney
Date: 2021-02-25 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

While auxiliary tasks play a key role in shaping the representations learnt by reinforcement learning agents, much is still unknown about the mechanisms through which this is achieved. This work develops our understanding of the relationship between auxiliary tasks, environment structure, and representations by analysing the dynamics of temporal difference algorithms. Through this approach, we establish a connection between the spectral decomposition of the transition operator and the representations induced by a variety of auxiliary tasks. We then leverage insights from these theoretical results to inform the selection of auxiliary tasks for deep reinforcement learning agents in sparse-reward environments.
Task-Agnostic Morphology Evolution
Authors: Donald J. Hejna III, Pieter Abbeel, Lerrel Pinto
Date: 2021-02-25 00:00:00
Proceeding: ICLR 2021 1
Tasks: Deep Reinforcement Learning

Deep reinforcement learning primarily focuses on learning behavior, usually overlooking the fact that an agent's function is largely determined by form. So, how should one go about finding a morphology fit for solving tasks in a given environment? Current approaches that co-adapt morphology and behavior use a specific task's reward as a signal for morphology optimization. However, this often requires expensive policy optimization and results in task-dependent morphologies that are not built to generalize. In this work, we propose a new approach, Task-Agnostic Morphology Evolution (TAME), to alleviate both of these issues. Without any task or reward specification, TAME evolves morphologies by only applying randomly sampled action primitives on a population of agents. This is accomplished using an information-theoretic objective that efficiently ranks agents by their ability to reach diverse states in the environment and the causality of their actions. Finally, we empirically demonstrate that across 2D, 3D, and manipulation environments TAME can evolve morphologies that match the multi-task performance of those learned with task supervised algorithms. Our code and videos can be found at https://sites.google.com/view/task-agnostic-evolution.
Federated Multi-armed Bandits with Personalization
Authors: Chengshuai Shi, Cong Shen, Jing Yang
Date: 2021-02-25 00:00:00
Tasks: Federated Learning, Multi-Armed Bandits

A general framework of personalized federated multi-armed bandits (PF-MAB) is proposed, which is a new bandit paradigm analogous to the federated learning (FL) framework in supervised learning and enjoys the features of FL with personalization. Under the PF-MAB framework, a mixed bandit learning problem that flexibly balances generalization and personalization is studied. A lower bound analysis for the mixed model is presented. We then propose the Personalized Federated Upper Confidence Bound (PF-UCB) algorithm, where the exploration length is chosen carefully to achieve the desired balance of learning the local model and supplying global information for the mixed learning objective. Theoretical analysis proves that PF-UCB achieves an $O(\log(T))$ regret regardless of the degree of personalization, and has a similar instance dependency as the lower bound. Experiments using both synthetic and real-world datasets corroborate the theoretical analysis and demonstrate the effectiveness of the proposed algorithm.
Entanglement Diagnostics for Efficient Quantum Computation
Authors: JoonHo Kim, Yaron Oz
Date: 2021-02-24 00:00:00
Tasks: Diagnostic

We consider information spreading measures in randomly initialized variational quantum circuits and introduce entanglement diagnostics for efficient variational quantum/classical computations. We establish a robust connection between entanglement measures and optimization accuracy by solving two eigensolver problems for Ising Hamiltonians with nearest-neighbor and long-range spin interactions. As the circuit depth affects the average entanglement of random circuit states, the entanglement diagnostics can identify a high-performing depth range for optimization tasks encoded in local Hamiltonians. We argue, based on an eigensolver problem for the Sachdev-Ye-Kitaev model, that entanglement alone is insufficient as a diagnostic to the approximation of volume-law entangled target states and that a large number of circuit parameters is needed for such an optimization task.
Auto-Detection of Tibial Plateau Angle in Canine Radiographs Using a Deep Learning Approach
Authors: Masuda Akter Tonima, F M Anim Hossain, Austin Dehart, Youmin Zhang
Date: 2021-02-24 00:00:00
Tasks: object-detection, Object Detection

Stifle joint issues are a major cause of lameness in dogs and it can be a significant marker for various forms of diseases or injuries. A known Tibial Plateau Angle (TPA) helps in the reduction of the diagnosis time of the cause. With the state of the art object detection algorithm YOLO, and its variants, this paper delves into identifying joints, their centroids and other regions of interest to draw multiple line axes and finally calculating the TPA. The methods investigated predicts successfully the TPA within the normal range for 80 percent of the images.
Maximum Likelihood Constraint Inference from Stochastic Demonstrations
Authors: David L. McPherson, Kaylene C. Stocking, S. Shankar Sastry
Date: 2021-02-24 00:00:00

When an expert operates a perilous dynamic system, ideal constraint information is tacitly contained in their demonstrated trajectories and controls. The likelihood of these demonstrations can be computed, given the system dynamics and task objective, and the maximum likelihood constraints can be identified. Prior constraint inference work has focused mainly on deterministic models. Stochastic models, however, can capture the uncertainty and risk tolerance that are often present in real systems of interest. This paper extends maximum likelihood constraint inference to stochastic applications by using maximum causal entropy likelihoods. Furthermore, we propose an efficient algorithm that computes constraint likelihood and risk tolerance in a unified Bellman backup, allowing us to generalize to stochastic systems without increasing computational complexity.
Robust SleepNets
Authors: Yigit Alparslan, Edward Kim
Date: 2021-02-24 00:00:00
Tasks: BIG-bench Machine Learning, Data Augmentation, Face Recognition

State-of-the-art convolutional neural networks excel in machine learning tasks such as face recognition, and object classification but suffer significantly when adversarial attacks are present. It is crucial that machine critical systems, where machine learning models are deployed, utilize robust models to handle a wide range of variability in the real world and malicious actors that may use adversarial attacks. In this study, we investigate eye closedness detection to prevent vehicle accidents related to driver disengagements and driver drowsiness. Specifically, we focus on adversarial attacks in this application domain, but emphasize that the methodology can be applied to many other domains. We develop two models to detect eye closedness: first model on eye images and a second model on face images. We adversarially attack the models with Projected Gradient Descent, Fast Gradient Sign and DeepFool methods and report adversarial success rate. We also study the effect of training data augmentation. Finally, we adversarially train the same models on perturbed images and report the success rate for the defense against these attacks. We hope our study sets up the work to prevent potential vehicle accidents by capturing drivers' face images and alerting them in case driver's eyes are closed due to drowsiness.
The Logical Options Framework
Authors: Brandon Araki, Xiao Li, Kiran Vodrahalli, Jonathan DeCastro, Micah J. Fry, Daniela Rus
Date: 2021-02-24 00:00:00
Tasks: Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

Learning composable policies for environments with complex rules and tasks is a challenging problem. We introduce a hierarchical reinforcement learning framework called the Logical Options Framework (LOF) that learns policies that are satisfying, optimal, and composable. LOF efficiently learns policies that satisfy tasks by representing the task as an automaton and integrating it into learning and planning. We provide and prove conditions under which LOF will learn satisfying, optimal policies. And lastly, we show how LOF's learned policies can be composed to satisfy unseen tasks with only 10-50 retraining steps. We evaluate LOF on four tasks in discrete and continuous domains, including a 3D pick-and-place environment.
A Comprehensive Review on the NILM Algorithms for Energy Disaggregation
Authors: Akriti Verma, Adnan Anwar, M. A. Parvez Mahmud, Mohiuddin Ahmed, Abbas Kouzani
Date: 2021-02-20 00:00:00
Tasks: energy management, Management, Non-Intrusive Load Monitoring

The housing structures have changed with urbanization and the growth due to the construction of high-rise buildings all around the world requires end-use appliance energy conservation and management in real-time. This shift also came along with smart-meters which enabled the estimation of appliance-specific power consumption from the buildings aggregate power consumption reading. Non-intrusive load monitoring (NILM) or energy disaggregation is aimed at separating the household energy measured at the aggregate level into constituent appliances. Over the years, signal processing and machine learning algorithms have been combined to achieve this. Incredible research and publications have been conducted on energy disaggregation, non-intrusive load monitoring, home energy management and appliance classification. There exists an API, NILMTK, a reproducible benchmark algorithm for the same. Many other approaches to perform energy disaggregation has been adapted such as deep neural network architectures and big data approach for household energy disaggregation. This paper provides a survey of the effective NILM system frameworks and reviews the performance of the benchmark algorithms in a comprehensive manner. This paper also summarizes the wide application scope and the effectiveness of the algorithmic performance on three publicly available data sets.
Railway Anomaly detection model using synthetic defect images generated by CycleGAN
Authors: Takuro Hoshi, Yohei Baba, Gaurang Gavai
Date: 2021-02-24 00:00:00
Tasks: Anomaly Detection, BIG-bench Machine Learning, Defect Detection

Although training data is essential for machine learning, railway companies are facing difficulties in gathering adequate images of defective equipment due to their proactive replacement of would be defective equipment. Nevertheless, proactive replacement is indispensable for safe and undisturbed operation of public transport. In this research, we have developed a model using CycleGAN to generate artificial images of defective equipment instead of real images. By adopting these generated images as training data, we verified that these images are indistinguishable from real images and they play a vital role in enhancing the accuracy of the defect detection models.
Dynamic Social Media Monitoring for Fast-Evolving Online Discussions
Authors: Maya Srikanth, Anqi Liu, Nicholas Adams-Cohen, Jian Cao, R. Michael Alvarez, Anima Anandkumar
Date: 2021-02-24 00:00:00
Tasks: Decision Making, Time Series, Time Series Analysis

Tracking and collecting fast-evolving online discussions provides vast data for studying social media usage and its role in people's public lives. However, collecting social media data using a static set of keywords fails to satisfy the growing need to monitor dynamic conversations and to study fast-changing topics. We propose a dynamic keyword search method to maximize the coverage of relevant information in fast-evolving online discussions. The method uses word embedding models to represent the semantic relations between keywords and predictive models to forecast the future time series. We also implement a visual user interface to aid in the decision-making process in each round of keyword updates. This allows for both human-assisted tracking and fully-automated data collection. In simulations using historical #MeToo data in 2017, our human-assisted tracking method outperforms the traditional static baseline method significantly, with 37.1% higher F-1 score than traditional static monitors in tracking the top trending keywords. We conduct a contemporary case study to cover dynamic conversations about the recent Presidential Inauguration and to test the dynamic data collection system. Our case studies reflect the effectiveness of our process and also points to the potential challenges in future deployment.
Sequential Learning-based IaaS Composition
Authors: Sajib Mistry, Sheik Mohammad Mostakim Fattah, Athman Bouguettaya
Date: 2021-02-24 00:00:00
Tasks: Clustering, Q-Learning

We propose a novel IaaS composition framework that selects an optimal set of consumer requests according to the provider's qualitative preferences on long-term service provisions. Decision variables are included in the temporal conditional preference networks (TempCP-net) to represent qualitative preferences for both short-term and long-term consumers. The global preference ranking of a set of requests is computed using a \textit{k}-d tree indexing based temporal similarity measure approach. We propose an extended three-dimensional Q-learning approach to maximize the global preference ranking. We design the on-policy based sequential selection learning approach that applies the length of request to accept or reject requests in a composition. The proposed on-policy based learning method reuses historical experiences or policies of sequential optimization using an agglomerative clustering approach. Experimental results prove the feasibility of the proposed framework.
Modular Object-Oriented Games: A Task Framework for Reinforcement Learning, Psychology, and Neuroscience
Authors: Nicholas Watters, Joshua Tenenbaum, Mehrdad Jazayeri
Date: 2021-02-25 00:00:00
Tasks: Reinforcement Learning (RL)

In recent years, trends towards studying simulated games have gained momentum in the fields of artificial intelligence, cognitive science, psychology, and neuroscience. The intersections of these fields have also grown recently, as researchers increasing study such games using both artificial agents and human or animal subjects. However, implementing games can be a time-consuming endeavor and may require a researcher to grapple with complex codebases that are not easily customized. Furthermore, interdisciplinary researchers studying some combination of artificial intelligence, human psychology, and animal neurophysiology face additional challenges, because existing platforms are designed for only one of these domains. Here we introduce Modular Object-Oriented Games, a Python task framework that is lightweight, flexible, customizable, and designed for use by machine learning, psychology, and neurophysiology researchers.