instruction:
In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
question:
Normalization methods are a central building block in the deep learning toolbox. They accelerate and stabilize training, while decreasing the dependence on manually tuned learning rate schedules. When learning from multi-modal distributions, the effectiveness of batch normalization (BN), arguably the most prominent normalization method, is reduced. As a remedy, we propose a more flexible approach: by extending the normalization to more than a single mean and variance, we detect modes of data on-the-fly, jointly normalizing samples that share common features. We demonstrate that our method outperforms BN and other widely used normalization techniques in several experiments, including single and multi-task datasets.
answer:
We present a novel normalization method for deep neural networks that is robust to multi-modalities in intermediate feature distributions.


question:
Disentangling underlying generative factors of a data distribution is important for interpretability and generalizable representations. In this paper,  we introduce two novel disentangling methods. Our first method, Unlabeled Disentangling GAN (UD-GAN, unsupervised), decomposes the latent noise by generating similar/dissimilar image pairs and it learns a distance metric on these pairs with siamese networks and a contrastive loss. This pairwise approach provides consistent representations for similar data points. Our second method (UD-GAN-G, weakly supervised) modifies the UD-GAN with user-defined guidance functions, which restrict the information that goes into the siamese networks. This constraint helps UD-GAN-G to focus on the desired semantic variations in the data. We  show  that  both  our  methods  outperform  existing  unsupervised approaches in quantitative metrics that measure semantic accuracy of the learned representations. In addition, we illustrate that simple guidance functions we use in UD-GAN-G allow us to directly capture the desired variations in the data.
answer:
We use Siamese Networks to guide and disentangle the generation process in GANs without labeled data.


question:
In high-dimensional reinforcement learning settings with sparse rewards, performing
 effective exploration to even obtain any reward signal is an open challenge.
 While model-based approaches hold promise of better exploration via planning, it
 is extremely difficult to learn a reliable enough Markov Decision Process (MDP)
 in high dimensions (e.g., over 10^100 states). In this paper, we propose learning
 an abstract MDP over a much smaller number of states (e.g., 10^5), which we can
 plan over for effective exploration. We assume we have an abstraction function
 that maps concrete states (e.g., raw pixels) to abstract states (e.g., agent position,
 ignoring other objects). In our approach, a manager maintains an abstract
 MDP over a subset of the abstract states, which grows monotonically through targeted
 exploration (possible due to the abstract MDP). Concurrently, we learn a
 worker policy to travel between abstract states; the worker deals with the messiness
 of concrete states and presents a clean abstraction to the manager. On three of
 the hardest games from the Arcade Learning Environment (Montezuma's,
 Pitfall!, and Private Eye), our approach outperforms the previous
 state-of-the-art by over a factor of 2 in each game. In Pitfall!, our approach is
 the first to achieve superhuman performance without demonstrations.
answer:

We automatically construct and explore a small abstract Markov Decision Process, enabling us to achieve state-of-the-art results on Montezuma's Revenge, Pitfall!, and Private Eye by a significant margin.