In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

Ex Input:
Optimistic initialisation is an effective strategy for efficient exploration in reinforcement learning (RL). In the tabular case, all provably efficient model-free algorithms rely on it. However, model-free deep RL algorithms do not use optimistic initialisation despite taking inspiration from these provably efficient tabular algorithms. In particular, in scenarios with only positive rewards, Q-values are initialised at their lowest possible values due to commonly used network initialisation schemes, a pessimistic initialisation. Merely initialising the network to output optimistic Q-values is not enough, since we cannot ensure that they remain optimistic for novel state-action pairs, which is crucial for exploration. We propose a simple count-based augmentation to pessimistically initialised Q-values that separates the source of optimism from the neural network. We show that this scheme is provably efficient in the tabular setting and extend it to the deep RL setting. Our algorithm, Optimistic Pessimistically Initialised Q-Learning (OPIQ), augments the Q-value estimates of a DQN-based agent with count-derived bonuses to ensure optimism during both action selection and bootstrapping. We show that OPIQ outperforms non-optimistic DQN variants that utilise a pseudocount-based intrinsic motivation in hard exploration tasks, and that it predicts optimistic estimates for novel state-action pairs.

Ex Output:
We augment the Q-value estimates with a count-based bonus that ensures optimism during action selection and bootstrapping, even if the Q-value estimates are pessimistic.


Ex Input:
Audio signals are sampled at high temporal resolutions, and learning to synthesize audio requires capturing structure across a range of timescales. Generative adversarial networks (GANs) have seen wide success at generating images that are both locally and globally coherent, but they have seen little application to audio generation. In this paper we introduce WaveGAN, a first attempt at applying GANs to unsupervised synthesis of raw-waveform audio. WaveGAN is capable of synthesizing one second slices of audio waveforms with global coherence, suitable for sound effect generation. Our experiments demonstrate that—without labels—WaveGAN learns to produce intelligible words when trained on a small-vocabulary speech dataset, and can also synthesize audio from other domains such as drums, bird vocalizations, and piano. We compare WaveGAN to a method which applies GANs designed for image generation on image-like audio feature representations, finding both approaches to be promising.

Ex Output:
Learning to synthesize raw waveform audio with GANs


Ex Input:
In many robotic applications, it is crucial to maintain a belief about the state of 
 a system, like the location of a robot or the pose of an object.
 These state estimates serve as input for planning and decision making and 
 provide feedback during task execution. 
 Recursive Bayesian Filtering algorithms address the state estimation problem,
 but they require a model of the process dynamics and the sensory observations as well as 
 noise estimates that quantify the accuracy of these models. 
 Recently, multiple works have demonstrated that the process and sensor models can be 
 learned by end-to-end training through differentiable versions of Recursive Filtering methods.
 However, even if the predictive models are known, finding suitable noise models 
 remains challenging. Therefore, many practical applications rely on very simplistic noise 
 models. 
 Our hypothesis is that end-to-end training through differentiable Bayesian 
 Filters enables us to learn more complex heteroscedastic noise models for
 the system dynamics. We evaluate learning such models with different types of 
 filtering algorithms and on two different robotic tasks. Our experiments show that especially 
 for sampling-based filters like the Particle Filter, learning heteroscedastic noise 
 models can drastically improve the tracking performance in comparison to using 
 constant noise models.

Ex Output:

We evaluate learning heteroscedastic noise models within different Differentiable Bayes Filters