pwc-archive/papers-with-abstracts huggingface dataset:
A visual exploration of Gaussian Processes and Infinite Neural Networks
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Bayesian Inference, Gaussian Processes

Gaussian Processes generalize Gaussian distributions to functons as random variables (stochastic processes). As such, they are a powerful tool for regression and Bayesian inference. In the last years, they have received increased attention due to the interest in machine learning. An interesting connection between Gaussian Processes and Neural Networks exists when the width of feed-forward neural networks tends towards infinity. For a specific choice of priors over the network parameters, the resulting network samples will be distributed according to a Gaussian Process.

In this blog post, I intend to explore this connection between Gaussian Processes and infinite Neural Nets, using Julia code and visual examples. I believe that there is some value in creating an intuitive understanding of the connection between those two methods as Gaussian Processes may help us understand how well Neural Networks can perform in the limit of infinite neurons per layer. Moreover, they may do so while keeping the probabilistic nature of the process in mind, as network training and the resulting performance will heavily depend on the initialization of the network weights. Finally, I discuss the limitations of this approach and how it may be extended by using Bayesian Neural Networks.
Designing realistic RL environment for power systems
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Reinforcement Learning (RL)

Power grids are critical infrastructure: ensuring they are reliable, robust and secure is essential to humanity,to everyday life, and to progress. With increasing renewable generation, growing electricity demand, and more severe weather events due to climate change, the task of maintaining efficient and robust power distribution poses a tremendous challenge to grid operators. In recent years, Reinforcement Learning (‘RL’) has shown substantial progress in solving highly complex, nonlinear problems, such as AlphaGo and it is now feasible that an RL agent could address the growing challenge of grid control.
Learning to Run a Power Network (‘L2RPN’) is one competition–organized by Réseau de Transport d’Electricité and
Electric Power Research Institute–aimed at testing out the capabilities of RL and other algorithms to safely control
electricity transportation in power grids. In 2020, L2RPN’s winners used a Semi Markov Afterstate Actor-Critic (‘SMAAC’)
approach to successfully manage a grid.  L2RPN represents an important first step in commercializing AI for the power
grid, but additional refinement of the RL environment is necessary to make it realistic for application in the real world.
Implementations that Matter in Cooperative Multi-Agent Reinforcement Learning
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Multi-agent reinforcement learning would like to implement some techniques that are known to have superior improvements in reinforcement learning. However, it is still unclear which of these techniques are complementary and can be fruitfully combined. This post dives into some proper experimental techniques that are controversial in multi-agent settings and gives out some advice to enhance the performance of MARL algorithms.
11 Summaries of Papers on Explainable Reinforcement Learning With Some Commentary
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Model interpretability was a bullet point in Concrete Problems in AI Safety (2016). Since then, interpretability has come to comprise entire research directions in technical safety agendas (2020). It is safe to say that interpretability is now a very popular area of research. In fact, the topic is sufficiently mainstream that there are books on the topic and corporate services promising to provide it.

Interpretability for reinforcement learning, however, has received much less attention than for supervised learning. So what's the state of research on this topic? What does progress in interpretable RL look like, and are we making progress?

What is this post? This post summarizes 11 recent papers on explaining reinforcement learning agents (in ICLR and related conferences), then provides commentary on the research. The summaries - and not the commentary - are the main point of this post. Though people like paper summaries, this is the kind of interpretive labor that isn't traditionally awarded space in research venues. We primarily select papers appearing between 2018 and 2020, in order to bridge the gap between foundational papers published in 2010-2017 and the more recent and diverse directions of research in the field.
Vision Transformer
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5

In last 10 years there has been significant development in Computer vision after development of Convolutional neural network(ConvNets). There has been research around combining self-attention with CNN after success seen in NLP with transformer models. In the blog-post we discuss the model “Vision transformer”(ViT) in detail and its new intuitions.
Emergent Symbols through Binding in External Memory
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Logical Reasoning

In this blog post, we describe the paper 'Emergent Symbols through Binding in External Memory', which introduces the Emergent Symbol Binding Network (ESBN), a recurrent neural network with factorized augmented memory to solve several logical reasoning tasks. We describe the motivation and design choices of the ESBN in great detail, compare it to strong baselines, and perform a qualitative validation of the generalization claims from the original paper.
From Generalization to Inner Activations
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5

One of the side effects of deep learning models becoming increasingly large is the amount of data that they intermediately generate which is then often ignored.  Researchers have used this information to create explainability methods which show importance relative to a specific input it relates to, but often the intermediate values are disregarded and mostly underutilized.  Although this intermediate data can become increasingly unwieldily in size as the networks grow, the ability to monitor specific layers is a valuable tool that provides insight into how the model is learning as well as vague generalities about how the model performs overall (e.g. where are specific features begun to be extracted or where is noise filtered out).
An Understanding of Learning from Demonstrations for Neural Text Generation
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: reinforcement-learning, Reinforcement Learning (RL), Text Generation

In this blog post, we will go over the ICLR 2021 paper titled Text Generation by Learning from Demonstrations. This paper introduces a learning method based on offline, off-policy reinforcement learning (RL) which addresses two key limitations of a training objective used in neural text generation models: Maximum Likelihood Estimate (MLE).

Goal of this blog post: Our main goal with this blog post is to provide researchers and practitioners in both NLP and RL with (1) a better understanding of algorithm presented in this paper (GOLD), and (2) an understanding of how RL is used for text generation.
The 37 Implementation Details of Proximal Policy Optimization
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Deep Reinforcement Learning

Proximal policy optimization (PPO) has become one of the most popular deep reinforcement learning (DRL) algorithms. Yet, reproducing the PPO's results has been challenging in the community.  While recent works conducted ablation studies to provide insight on PPO's implementation details, these works are not structured as tutorials and only focus on details concerning robotics tasks. As a result, reproducing PPO from scratch can become a daunting experience. Instead of introducing additional improvements, or doing further ablation studies, this blog post takes a step back and focuses on delivering a thorough reproduction of PPO in all accounts, as well as aggregating, documenting, and cataloging its most salient implementation details. This blog post also points out software engineering challenges in PPO and further efficiency improvement via the accelerated vectorized environments. With these, we believe this blog post will help people understand PPO faster and better, facilitating customization and research upon this versatile RL algorithm.
PPLM Revisited: Steering and Beaming a Lumbering Mammoth to Control Text Generation
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Language Modeling, Language Modelling, Text Generation

Dathathri et al., 2020 introduced the Plug and Play Language Model (PPLM) as a configurable method for controlled language generation. Since then, the publication has been cited more than 200 times and the official implementation received >800 stars on GitHub. Such popularity can be explained by the fact that PPLM achieves fine-grained control of attributes such as topics and sentiment while retaining fluency without retraining the language model (LM).

In a blogpost accompanying the original paper, the authors compared any LM with an “unguided mammoth” whilst PPLM was presented as a mouse sitting on top of the mammoth who steers it in the right direction. However, the interplay between the prompt and the BoW is underexplored in the paper and the original blog post. Moreover, to the best of our knowledge, the impact of the interplay between the prompt and the BoW on the controllability of the text has also not yet been described elsewhere.

In this blog post, we address both of the issues as well as the issue of the reproducibility and sensitivity to hyperparameter settings of PPLM. We aim to provide more insights into the workings of PPLM and, to some extent, into Natural Language Generation (NLG) controllability in general.

Based on the results of the experiments we conclude that producing topic related texts is difficult with the hyperparameters provided in the original paper, but pushing the distribution of the generated words more towards the targeted subject leads to enhanced results. On top of that, we also conclude that carefully picking the words in the BoW is important to control the PPLM, and modifying the weight might help control the PPLM to generate the texts that are relevant to the exact words that we like.
When Do Curricula Work?
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5

Inspired by human learning, researchers have proposed ordering examples during training based on their difficulty. Both curriculum learning, exposing a network to easier examples early in training, and anti-curriculum learning, showing the most difficult examples first, have been suggested as improvements to the standard i.i.d. training. In this work, we set out to investigate the relative benefits of ordered learning. We first investigate the implicit curricula resulting from architectural and optimization bias and find that samples are learned in a highly consistent order. Next, to quantify the benefit of explicit curricula, we conduct extensive experiments over thousands of orderings spanning three kinds of learning: curriculum, anti-curriculum, and random-curriculum -- in which the size of the training dataset is dynamically increased over time, but the examples are randomly ordered. We find that for standard benchmark datasets, curricula have only marginal benefits, and that randomly ordered samples perform as well or better than curricula and anti-curricula, suggesting that any benefit is entirely due to the dynamic training set size. Inspired by common use cases of curriculum learning in practice, we investigate the role of limited training time budget and noisy data in the success of curriculum learning. Our experiments demonstrate that curriculum, but not anti-curriculum or random ordering can indeed improve the performance either with limited training time budget or in the existence of noisy data.
Exploration by Random Network Distillation
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Atari Games, Deep Reinforcement Learning, Montezuma's Revenge, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We introduce an exploration bonus for deep reinforcement learning methods that is easy to implement and adds minimal overhead to the computation performed. The bonus is the error of a neural network predicting features of the observations given by a fixed randomly initialized neural network. We also introduce a method to flexibly combine intrinsic and extrinsic rewards. We find that the random network distillation (RND) bonus combined with this increased flexibility enables significant progress on several hard exploration Atari games. In particular we establish state of the art performance on Montezuma's Revenge, a game famously difficult for deep reinforcement learning methods. To the best of our knowledge, this is the first method that achieves better than average human performance on this game without using demonstrations or having access the underlying state of the game, and occasionally completes the first level. This suggests that relatively simple methods that scale well can be sufficient to tackle challenging exploration problems.
Representation Change in Model-Agnostic Meta-Learning
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Few-Shot Learning, Meta-Learning, model

Last year, an exciting adaptation of one of the most popular optimization-based meta-learning approaches, model-agnostic meta-learning (MAML) [Finn et al., 2017], was proposed in 

- Jaehoon Oh, Hyungjun Yoo, ChangHwan Kim, Se-Young Yun (ICLR, 2021) BOIL: Towards Representation Change for Few-shot Learning

The authors adapt MAML by freezing the last layer to force body only inner learning (BOIL). Interestingly, this is complementary to ANIL (almost no inner loop) proposed in

- Aniruddh Raghu, Maithra Raghu, Samy Bengio, Oriol Vinyals (ICLR, 2020) Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML

Both papers attempt to understand the success of MAML and improve it.
Oh et al. [2021] compare BOIL, ANIL, and MAML and show that both improve the performance of MAML, but BOIL outperforms ANIL, especially when the task distribution varies between training and testing.
Deep Learning Optimization Theory - Trajectory Analysis of Gradient Descent
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Deep Learning, Learning Theory

In recent years an obvious yet mysterious fact that stood across various experiments is the ability of gradient descent, a relatively simple first-order optimization method, to optimize an enormous number of parameters on highly non-convex loss functions. In some sense, this practical observation stands in contrast to classical statistical learning theory. This post will discuss the significant progress researchers are making in bridging this theory gap and demystifying gradient descent.
Generating Molecular Conformations via Normalizing Flows and Neural ODEs
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Diversity

We provide an in-depth overview of methods outlined in the paper "Learning Neural Generative Dynamics for Molecular Conformation Generation," discuss the impact of the work in the context of other conformation generation approaches, and additionally discuss future potential applications to improve the diversity and stability of generated conformations.
GAN2Shape - Create 3D Shape with 2D GANs
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: 3D Shape Reconstruction

In this blog post, we discuss the key points of the paper “Do 2D GANs Know 3D Shape? Unsupervised 3D Shape Reconstruction from 2D Image GANs” (GAN2Shape) by Pan et al. We will discuss both the theory and code (in the author's GitHub repository and use a demo Colab notebook to show how GAN2Shape is able to transform 2D images to 3D images in multiple view images format.

We gave a brief introduction to 3D deep learning, GANs, StyleGAN2, and Unsup3D . Then we discussed in detail the key steps of GAN2Shape: how to transform a 2D image into 3D shapes, by using Unsup3D and StyleGAN2D. We demonstrated the training process with a Colab notebook to show the input image and the generated 3D images.
The Annotated S4
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: State Space Models

An annotated implementation of a series of papers developing state-space models for very long-term sequence modeling. Covers "HiPPO: Recurrent Memory with Optimal Polynomial Projections" and ends with "Efficiently Modeling Long Sequences with Structured State Spaces".
Euclidean geometry meets graph, a geometric deep learning perspective
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Deep Learning, Graph Learning

Graph neural networks (GNN) have been an active area of machine learning research to tackle various problems in graph data. Sometimes nodes and edges in a graph can have spatial features, such as 3D coordinates of nodes and directions along edges. How do we reason over the topology of graphs while considering those geometric features? In this post, we discuss a ICLR 2021 paper "Learning from Protein Structure with Geometric Vector Perceptrons (GVP)". We take a geometric deep learning perspective to explain the intuition and advantage of GVP-GNN, the novel GNN architecture described in the paper, to reason over geometric graphs. We inspire an idea along physics-oriented interpretation. We also highlight under-appreciated applications of geometric graph learning in solving problems from multiple domains, including biochemistry, geography, and network analysis.
State of the Art of Reinforcement Learning
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Yeah so having offered introductions, the focus of this paper will now be to offer a few quick discussions surrounding reinforcement learning domain papers published at next week’s ICLR conference. To keep the project manageable we’ll limit our attention to those papers chosen for spotlight and oral presentations, which is a proxy for selecting papers deemed to have the most significant findings by conference chairs. In some cases the discussions may be more involved than others, this is partly indicative of my limited background in the field as in a few cases may be out of my depth. Hopefully a reader may expect to pick up a few points about current state of the art for the field. Yeah so without further ado.
FCDD - Explainable Anomaly Detection
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Anomaly Detection

Anomaly detection (AD) is the task of identifying anomalies in a corpus of data. There are many real-life applications where we find anomalies including applications in healthcare and manufacturing. We briefly discuss some of the existing approaches for solving this problem before focusing on one specific approach. FCDD, an approach introduced in a previous ICLR paper , provides an explainable way to solve anomaly detection, while providing SOTA performance. In this blog, we go over the paper and discuss the advantages and disadvantages of the methods involved in it. We also review the various additional experiments from the authors of the paper which were included in the appendix to the main paper.
"Auction Learning as a Two Player Game": GANs (?) for Mechanism Design
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Deep Learning, Vocal Bursts Valence Prediction

Designing strategyproof, revenue-maximizing auctions is an important task, but it is surprisingly difficult -- even in some seemingly trivial cases, nothing is known about the optimal auction design. Motivated by this lack of progress, a number of recent works have proposed the use of deep neural networks as function approximators for learning strategyproof mechanisms. One of these works is "Auction Learning as a Two-Player Game", which appeared at ICLR 2021. We discuss this work, situate it in the broader context of deep learning for auctions, explain how it improves over prior techniques, and discuss the future outlook for interactions between modern deep learning and mechanism design.
Rule and Convention Transfer in Context
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5

We discuss context and other work surrounding conventions in Human-AI Collaboration, motivated by Shih et al.'s "On the Critical Role of Conventions in Adaptive Human-AI Collaboration", published at ICLR 2021. We also discuss some room for improvement in the usage of conventions in Human-AI Collaboration.
Moving Beyond Navigation with Active Neural SLAM
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Domain Generalization, motion prediction, Navigate, Scene Understanding

The ability to effectively harness autonomous control in real-world 3D environments largely depends on learning realistic navigation techniques for embodied agents. Advances to classical robotics call for efficient methods to navigate, viz., explore unseen simulations while transferring the attained performance to the real-world domain. Any autonomous agent in the wild should have the ability to make decisions on the fly, having explored its surroundings and being cognizant of sudden changes, as in the real world. This blog post breaks down the method of Active Neural SLAM for exploring 3D environments, moving away from end-to-end exploration, by learning policies in a modular and hierarchical manner. Active Neural SLAM enabled Domain Generalization to previously unseen environments and Task Generalization in navigation while curbing the high sample complexity of previous methods. We also suggest ways to leverage this method for scene understanding and learning agent-scene interaction by combining navigation with motion prediction or synthesis.
Learning to Coarsen Graphs with Graph Neural Networks
Authors: Anonymous
Date: 2022-01-17 00:00:00
Proceeding: ICLR Track Blog 2022 5
Tasks: Relational Reasoning

With the rise of large-scale graphs for relational learning, graph coarsening emerges as a computationally viable alternative. We revisit the principles that aim to improve data-driven graph coarsening with adjustable coarsened structures.