TASK DEFINITION: In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
PROBLEM: Abstraction of Markov Decision Processes is a useful tool for solving complex problems, as it can ignore unimportant aspects of an environment, simplifying the process of learning an optimal policy. In this paper, we propose a new algorithm for finding abstract MDPs in environments with continuous state spaces. It is based on MDP homomorphisms, a structure-preserving mapping between MDPs. We demonstrate our algorithm's ability to learns abstractions from collected experience and show how to reuse the abstractions to guide exploration in new tasks the agent encounters. Our novel task transfer method beats a baseline based on a deep Q-network.

SOLUTION: We create abstract models of environments from experience and use them to learn new tasks faster.

PROBLEM: We propose a rejection sampling scheme using the discriminator of a GAN to
 approximately correct errors in the GAN generator distribution. We show that
 under quite strict assumptions, this will allow us to recover the data distribution
 exactly. We then examine where those strict assumptions break down and design a
 practical algorithm—called Discriminator Rejection Sampling (DRS)—that can be
 used on real data-sets. Finally, we demonstrate the efficacy of DRS on a mixture of
 Gaussians and on the state of the art SAGAN model. On ImageNet, we train an
 improved baseline that increases the best published Inception Score from 52.52 to
 62.36 and reduces the Frechet Inception Distance from 18.65 to 14.79. We then use
 DRS to further improve on this baseline, improving the Inception Score to 76.08
 and the FID to 13.75.

SOLUTION: We use a GAN discriminator to perform an approximate rejection sampling scheme on the output of the GAN generator.

PROBLEM: The problem of exploration in reinforcement learning is well-understood in the tabular case and many sample-efficient algorithms are known. Nevertheless, it is often unclear how the algorithms in the tabular setting can be extended to tasks with large state-spaces where generalization is required. Recent promising developments generally depend on problem-specific density models or handcrafted features. In this paper we introduce a simple approach for exploration that allows us to develop theoretically justified algorithms in the tabular case but that also give us intuitions for new algorithms applicable to settings where function approximation is required. Our approach and its underlying theory is based on the substochastic successor representation, a concept we develop here. While the traditional successor representation is a representation that defines state generalization by the similarity of successor states, the substochastic successor representation is also able to implicitly count the number of times each state (or feature) has been observed. This extension connects two until now disjoint areas of research. We show in traditional tabular domains (RiverSwim and SixArms) that our algorithm empirically performs as well as other sample-efficient algorithms. We then describe a deep reinforcement learning algorithm inspired by these ideas and show that it matches the performance of recent pseudo-count-based methods in hard exploration Atari 2600 games.

SOLUTION:
We propose the idea of using the norm of the successor representation an exploration bonus in reinforcement learning. In hard exploration Atari games, our the deep RL algorithm matches the performance of recent pseudo-count-based methods.