pwc-archive/papers-with-abstracts huggingface dataset:
Deep Neural Network Models Compression
Authors: Anonymous
Date: 2021-03-04 00:00:00
Tasks: Knowledge Distillation, Quantization, Transfer Learning

AbstractModern  Deep  Neural  Networks  (DNNs)  models  used  in  computer  vision  applications  are  compelling.They are widely used to solve a variety of problems and the increase in data size implies that the modelcould be very large and complex, and therefore increased in computational requirements.  The number ofparameters in recent state-of-the-art networks makes them difficult to deploy in edges devices like mobilephones, watches or drones where memory and energy are limited.  We are working on the implementationof techniques that significantly reduce the size of a very large and powerful vision model while preservingas much of its performance as possible.  We built classification models on the MNIST dataset and used itwith pre-trained models on ImageNet on the Cats & Dogs dataset.  We performed a closer examinationof the effectiveness (mathematical and implementation aspects) of Knowledge distillation (KD), Pruningand Quantization techniques.  Firstly, we implemented transfer learning which consists on modifying theparameters  of  an  already-trained  network  to  adapt  to  a  new  task  on  a  new  dataset,  then  secondly,we  trained  this  network  by  using  a  gradual  pruning  approach  that  requires  minimal  tuning  and  can  beseamlessly  incorporated  within  the  training  process.   Thirdly,  the  Quantization  has  helped  us  reducethe  number  of  bits  required  to  represent  each  parameter  from  32-bit  floats  to  8  bits.  We  significantlyreduced  bandwidth  and  storage.  On  MNIST,  we  reduced  the  model  from  12.52  MB  to  0.57  MB  withno loss of accuracy.  After the transfer learning and pruning step, we reduced the MobileNet from 12.48MB with an accuracy of 0.9556 to 2.91 MB with an accuracy of 0.9516.  We also empirically show ourmethod’s adaptability for classification based architecture VGG16 and VGG19 on datasets Cat & Dogsobserving that the entire pruning pipeline plus post-quantification at 8 bits works well up to 70% level ofsparsity, suffer only very small losses in accuracy and the size of the model obtained by transfer learningare divided by 10.
Redundant Information Neural Estimation
Authors: Michael Kleinman, Alessandro Achille, Stefano Soatto, Jonathan Kao
Date: 2021-03-04 00:00:00
Proceeding: ICLR Workshop Neural_Compression 2021 5
Tasks: image-classification, Image Classification

We introduce the Redundant Information Neural Estimator (RINE), a method that allows efficient estimation for the component of information about a target variable that is common to a set of sources, previously referred to as the “redundant information.” We show that existing definitions of the redundant information can be recast in terms of an optimization over a family of deterministic or stochastic functions. In contrast to previous information decompositions, which can only be evaluated for discrete variables over small alphabets, by optimizing over functions we show empirically that we can recover the redundant information on simple benchmark tasks and that we can approximate the redundant information for high-dimensional predictors on image classification tasks, paving the way for application in different domains.
SCALE SPACE FLOW WITH AUTOREGRESSIVE PRIORS
Authors: Ruihan Yang, Yibo Yang, Joseph Marino, Stephan Mandt
Date: 2021-03-04 00:00:00
Proceeding: ICLR Workshop Neural_Compression 2021 5
Tasks: Dimensionality Reduction, Open-Ended Question Answering, Video Compression

There has been a recent surge of interest in neural video compression models that combines data-driven dimensionality reduction with learned entropy coding. ScaleSpace Flow (SSF) is among the most popular variants due to its favorable rate-distortion performance.  Recent work showed that this approach could be further improved by structured priors and stochastic temporal autoregressive transforms on the frame level.   However,  as of early  2021,  most state-of-the-art compression approaches work with time-independent priors. Assuming that frame patents are still temporally correlated, further compression gains should be expected by conditioning the priors on temporal information.  We show that the naive way of conditioning priors on previous stochastic latent states degrades performance, but temporal conditioning on a deterministic quantity does lead to a consistent improvement over all baselines.  Evaluating the benefits of the temporal prior given the involved challenges in training and deployment remains an open question.
NETWORK COMPRESSION FOR MACHINE-LEARNT FLUID SIMULATIONS
Authors: Anonymous
Date: 2021-03-04 00:00:00
Tasks: Physics-informed machine learning, Quantization

Multi-scale, multi-fidelity numerical simulations form the pillar of scientific applications
related to numerically modeling fluids. However, simulating the fluid
behavior characterized by the non-linear Navier Stokes equations are often times
computational expensive. Physics informed machine learning methods is a viable
alternative and as such has seen great interest in the community [refer to
Kutz (2017); Brunton et al. (2020); Duraisamy et al. (2019) for a detailed review
on this topic]. For full physics emulators, the cost of network inference is often
trivial. However, in the current paradigm of data-driven fluid mechanics models
are built as surrogates for complex sub-processes. These models are then used in
conjunction to the Navier Stokes solvers, which makes ML model inference an
important factor in the terms of algorithmic latency. With the ever growing size
of networks, and often times overparameterization, exploring effective network
compression techniques becomes not only relevant but critical for engineering
systems design. In this study, we explore the applicability of pruning and quantization
(FP32 to int8) methods for one such application relevant to modeling fluid
turbulence. Post-compression, we demonstrate the improvement in the accuracy
of network predictions and build intuition in the process by comparing the compressed
to the original network state.
Lossless Compression using Continuously-Indexed Normalizing Flows
Authors: Adam Golinski, Anthony L. Caterini
Date: 2021-03-04 00:00:00
Proceeding: ICLR Workshop Neural_Compression 2021 5
Tasks: Density Estimation, Variational Inference

Recently, a class of deep generative models known as continuously-indexed flows (CIFs) have expanding the modelling capacity of normalizing flows (NFs) in the context of both density estimation and variational inference.
CIFs are provably more general and expressive than NFs, but do not induce a closed-form density model and thus require additional considerations when applying in the same contexts that NFs have shown promise.
One such area is lossless compression, where NFs have been used as the density model to develop a compression scheme, known as local bits-back, with expected codelength approximately equal to the average negative log-likelihood of the NF density model.
Here, we propose to extend the local bits-back scheme to CIF-based density models, as the improved expressiveness inherent in CIFs stands to reduce the expected codelength of compressed data.
We also leverage recent work on compression schemes built with hierarchical variational auto-encoders -- as hierarchical CIFs can themselves be seen as interpolating between these and NFs -- gaining further expressiveness in our density models and effectiveness in our compression scheme.
Exhibit: Converting my PhD thesis to HTML
Authors: Damien Desfontaines
Date: 2021-03-18 00:00:00
Proceeding: ICLR Workshop Rethinking_ML_Papers 2021 5

I converted my PhD thesis, titled Lowering the cost of anonymization, into HTML; the result is available at the URL desfontain.es/thesis. The motivation and process are described in a blog post (desfontain.es/privacy/latex-tohtml.html); and I was told that this work could constitute an interesting exhibit submission to the 2021 Rethinking ML Papers workshop at ICLR.
Diagrammatic summaries for neural architectures
Authors: Guy Clarke Marshall, Caroline Jay, André Freitas
Date: 2021-03-18 00:00:00
Proceeding: ICLR Workshop Rethinking_ML_Papers 2021 5
Tasks: BIG-bench Machine Learning

This paper advocates for diagrammatic summary publications for machine learning system architecture papers. We review existing diagram-centric scholarly practices, and summarise relevant studies on neural network system architecture diagrams. We subsequently propose three opportunities: Diagram guidelines, diagrammatic system summary publications, and the community creation of a formal diagram standards, which could be integrated with existing LaTeX + PDF publication processes.
ModulOM: Disseminating Deep Learning Research with Modular Output Mathematics
Authors: Maxime Istasse, Kim Mens, Christophe De Vleeschouwer
Date: 2021-03-18 00:00:00
Proceeding: ICLR Workshop Rethinking_ML_Papers 2021 5
Tasks: Data Augmentation, Deep Learning

Solving a task with a deep neural network requires an appropriate formulation of the underlying inference problem. A formulation defines the type of variables output by the network, but also the set of variables and functions, denoted output mathematics, needed to turn those outputs into task-relevant predictions. Despite the fact that the task performance may largely depend on the formulation, most deep learning experiment repositories do not offer a convenient solution to explore formulation variants in a flexible and incremental manner. Software components for neural network creation, parameter optimization or data augmentation, in contrast, offer some degree of modularity that has proved to facilitate the transfer of know-how associated to model development. But this is not the case for output mathematics. Our paper proposes to address this limitation by embedding the output mathematics in a modular component as well, by building on multiple inheritance principles in object-oriented programming. The flexibility offered by the proposed component and its added value in terms of knowledge dissemination are demonstrated in the context of the Panoptic-Deeplab method, a representative computer vision use case.
SPICES: SURVEY PAPERS AS INTERACTIVE CHEATSHEET EMBEDDINGS
Authors: Vinay Uday Prabhu, Matthew McAteer, Ryan Teehan
Date: 2021-03-18 00:00:00
Proceeding: ICLR Workshop Rethinking_ML_Papers 2021 5
Tasks: Cultural Vocal Bursts Intensity Prediction, Out-of-Distribution Detection, Survey

Papers are hard to write. Survey papers are just that much harder. From the authors' perspective, challenges include the responsibility to not erase out important work being done by (sometimes) adversarially aligned research groups, finding the right semantic clustering to sub-categorize individual contributions, controlling for the verbosity and length of the final paper, ensuring an optimal mixing of personal opinion and the innate narratives in the paper(s) being cited, version controlling, ease of updating, and also the aesthetics of presentation. From the reader's viewpoint, challenges include ease of reading, single-snapshot summarizability, portability, and being given the agency to edit or fork their own copies.
Taking cues from the emergence of the cheat-sheet culture in machine learning and the virtues of living editable documentation and version control, we propose an interactive and live SVG format based methodology that we term SPICE: Survey Papers as Interactive Cheat-sheet Embedding. We cover the technical details behind constructing SPICEs and present an example gallery covering `hot button' areas in machine learning such as Out of distribution detection, the `All you need' histrionics and Transformer architectures. We have open-sourced all of the code with regards to this project here: 
\url{https://anonymous.4open.science/r/7c5ee736-c876-4b90-97bd-49870eb6b63f/}.
Curating Publications as Artefacts — Exploring Machine Learning Research in an Interactive Virtual Museum
Authors: Beatrice Gobbo, Mennatallah El-Assady
Date: 2021-03-18 00:00:00
Proceeding: ICLR Workshop Rethinking_ML_Papers 2021 5
Tasks: Articles, BIG-bench Machine Learning

The need for innovating scientific publications is felt across various research fields. In the last thirty years, the publishing process is accelerating, and new research is coming out every day. In Machine Learning, some attempts have been made to keep track of recent publications and to enhance open access articles for allowing authors to integrate multimedia contents. However, it is still hard to compare selected articles exhaustively. Thus, we envision a Machine Learning Papers Museum as a metaphorical solution for providing a digital space where users can explore publication collections in guided and serendipitous ways.
Fairness and Friends
Authors: Falaah Arif Khan, Eleni Manis, Julia Stoyanovich
Date: 2021-03-18 00:00:00
Proceeding: ICLR Workshop Rethinking_ML_Papers 2021 5
Tasks: Fairness, Philosophy

Recent interest in codifying fairness in Automated Decision Systems (ADS) has resulted in a wide range of formulations of what it means for an algorithm to be “fair.” Most of these propositions are inspired by, but inadequately grounded in, scholarship from political philosophy. This comic aims to correct that deficit. We begin by setting up a working definition of an 'Automated Decision System' (ADS) and explaining 'bias' in outputs of an ADS. We then critically evaluate different definitions of fairness as Equality of Opportunity (EOP) by contrasting their conception in political philosophy (such as Rawls’s fair EOP and formal EOP) with the proposed codification in Fair-ML (such as statistical parity, equality of odds and accuracy) to provide a clearer lens with which to view existing results and to identify future research directions. We use this framing to reinterpret the impossibility results as the incompatibility between different EOP doctrines and demonstrate how political philosophy can provide normative guidance as to which notion of fairness is applicable in which context. We conclude by highlighting justice considerations that the fair-ML literature currently overlooks or underemphasizes, such as Rawls's broader theory of justice, which supplements his EOP principle with a
principle guaranteeing equal rights and liberties to all citizens in a free and democratic society.
On augmenting the references section with a citation network visualization
Authors: Putra Manggala, Tigran Atoyan, Gracia Samosir, Jan Varsava, Johannes Ruf
Date: 2021-03-18 00:00:00
Proceeding: ICLR Workshop Rethinking_ML_Papers 2021 5
Tasks: Articles

Researchers often find relevant articles by looking at the references section. We conducted user interviews with researchers about their workflow and their needs when carrying out literature research. Based on this study, we identify a set of problems encountered by researchers. We then propose to embed classified citation networks into articles as a solution, and to complement this graph with optional comments about references by authors.  We demonstrate this idea by implementing it for this article. We argue that our solution helps increase inclusivity and improves the efficiency of reading scientific articles.
Optimism is All You Need: Model-Based Imitation Learning From Observation Alone
Authors: Rahul Kidambi, Jonathan Daniel Chang, Wen Sun
Date: 2021-03-09 00:00:00
Proceeding: ICLR Workshop SSL-RL 2021 5
Tasks: All, Imitation Learning, OpenAI Gym

This paper studies Imitation Learning from Observations alone (ILFO) where the learner is presented with expert demonstrations that only consist of states encountered by an expert (without access to actions taken by the expert). This paper presents a provably efficient model-based framework MobILE to solve the ILFO problem. MobILE uses self-supervision towards (a) training a dynamics model and (b) designing an intrinsic reward signal for exploration. Using these ideas, MobILE carefully trades off exploration against imitation by integrating the idea of optimism in the face of uncertainty into the distribution matching imitation learning (IL) framework. We provide a unified analysis for MobILE, and demonstrate that MobILE enjoys strong performance guarantees for classes of MDP dynamics that satisfy certain well studied notions of complexity. We also show that the ILFO problem is strictly harder than the standard IL problem by reducing ILFO to a multi-armed bandit problem indicating that strategic exploration is necessary for solving ILFO efficiently. We complement  these theoretical results with experimental simulations on benchmark OpenAI Gym tasks that indicate the efficacy of MobILE.
Relevant Action Matters : Motivating agent with action usefulness
Authors: Mathieu Seurin, Florian Strub, Philippe Preux, Olivier Pietquin
Date: 2021-03-09 00:00:00
Proceeding: ICLR Workshop SSL-RL 2021 5

Sparse rewards are double-edged training signals in reinforcement learning: easy to design but hard to optimize. Intrinsic motivation guidances have thus been developed toward alleviating the resulting exploration problem. They usually incentivize agents to look for new states through novelty signals. Yet, such methods encourage exhaustive exploration of the state space rather than focusing on the environment's salient interaction opportunities. We propose a new exploration method, called Relevant Action Matters (RAM), shifting the emphasis from state novelty to state with relevant actions. While most actions consistently change the state when used, e.g. moving the agent, some actions are only effective in specific states, e.g., opening a door, grabbing an object. RAM detects and rewards actions that seldom affect the environment.
We evaluate RAM on the procedurally-generated environment MiniGrid, against state-of-the-art methods. Experiments consistently show that RAM greatly reduces sample complexity, installing the new state-of-the-art in MiniGrid.
Learning to Infer Unseen Contexts in Causal Contextual Reinforcement Learning
Authors: Hamid Eghbal-zadeh, Florian Henkel, Gerhard Widmer
Date: 2021-03-09 00:00:00
Proceeding: ICLR Workshop SSL-RL 2021 5
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In Contextual Reinforcement Learning (CRL), a change in the context variable can cause a change in the distribution of the states. Hence contextual agents must be able to learn adaptive policies that can change when a context changes. Furthermore, in certain scenarios agents have to deal with unseen contexts, and be able to choose suitable actions. In order to generalise onto unseen contexts, agents need to not only detect and adapt to previously observed contexts, but also reason about how a context is constructed, and what are the causal factors of context variables. In this paper, we propose a new task and environment for Causal Contextual Reinforcement Learning (CCRL), where the performance of different agents can be compared in a causal reasoning task. Furthermore, we introduce a Contextual Attention Module that allows the agent to incorporate disentangled features as the contextual factors, which results in performance improvement of the agent in unseen contexts. Finally, we demonstrate that non-causal agents fail to generalise onto unseen contexts, while the agents incorporating the proposed module can achieve better performance in unseen contexts.
Out-of-distribution generalization of internal models is correlated with reward
Authors: Khushdeep Singh Mann, Steffen Schneider, Alberto Chiappa, Jin Hwa Lee, Matthias Bethge, Alexander Mathis, Mackenzie W Mathis
Date: 2021-03-09 00:00:00
Proceeding: ICLR Workshop SSL-RL 2021 5
Tasks: Out-of-Distribution Generalization, reinforcement-learning, Reinforcement Learning (RL)

We investigate the behavior of reinforcement learning (RL) agents under morphological distribution shifts. Similar to recent robustness benchmarks in computer vision, we train algorithms on selected RL environments and test transfer performance on perturbed environments. We specifically test perturbations to popular RL agent's morphologies by changing the length and mass of limbs, which in biological settings is a major challenge (e.g., after injury or during growth). In this setup, called PyBullet-M, we compare the performance of policies obtained by reward-driven learning with self-supervised models of the observed state-action transitions. We find that out-of-distribution performance of self-supervised models is correlated to degradation in reward.
Variational Model-Based Imitation Learning in High-Dimensional Observation Spaces
Authors: Rafael Rafailov, Tianhe Yu, Aravind Rajeswaran, Chelsea Finn
Date: 2021-03-09 00:00:00
Proceeding: ICLR Workshop SSL-RL 2021 5
Tasks: Imitation Learning, Vocal Bursts Intensity Prediction

We consider the problem setting of imitation learning where the agent is provided a fixed dataset of demonstrations. While the agent can interact with the environment for exploration, it is oblivious to the reward function used by the demonstrator. This setting is representative of many applications in robotics where task demonstrations may be straightforward while reward shaping or conveying stylistic aspects of human motion may be difficult. For this setting, we develop a variational model-based imitation learning algorithm (VMIL) that is capable of learning policies from visual observations. Through experiments, we find that VMIL is more sample efficient compared to prior algorithms in several challenging vision-based locomotion and manipulation tasks, including a high-dimensional in-hand dexterous manipulation task.
Pretraining Reward-Free Representations for Data-Efficient Reinforcement Learning
Authors: Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, R Devon Hjelm, Philip Bachman, Aaron Courville
Date: 2021-03-09 00:00:00
Proceeding: ICLR Workshop SSL-RL 2021 5
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Representation Learning

Data efficiency poses a major challenge for deep reinforcement learning.
We approach this issue from the perspective of self-supervised representation learning, leveraging reward-free exploratory data to pretrain encoder networks.
We employ a novel combination of latent dynamics modelling and goal-reaching objectives, which exploit the inherent structure of data in reinforcement learning.
We demonstrate that our method scales well with network capacity and pretraining data.
When evaluated on the Atari 100k data-efficiency benchmark, our approach significantly outperforms previous methods combining unsupervised pretraining with task-specific finetuning, and approaches human-level performance.
Solipsistic Reinforcement Learning
Authors: Mingtian Zhang, Peter Noel Hayes, Tim Z. Xiao, Andi Zhang, David Barber
Date: 2021-03-09 00:00:00
Proceeding: ICLR Workshop SSL-RL 2021 5
Tasks: Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We introduce a new model-based reinforcement learning framework that aims to tackle environments with high dimensional state spaces. In contrast to existing approaches, agents under our framework learn a low dimensional internal representation of the environment while avoiding the need to learn a generative model of the environment itself. This solipsistic representation is trained to encode a belief that is consistent with the dynamics of the environment and is then exploited for effective planning. We present specific cases of our framework with choices of model and corresponding planning algorithms that can deal with both discrete and continuous state environments. We demonstrate empirically gains in efficiency over existing model-free methods when learning directly from pixels and analyze the properties of our learned representations.