TASK DEFINITION: In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
PROBLEM: Deploying machine learning systems in the real world requires both high accuracy on clean data and robustness to naturally occurring corruptions. While architectural advances have led to improved accuracy, building robust models remains challenging, involving major changes in training procedure and datasets.   Prior work has argued that there is an inherent trade-off between robustness and accuracy, as exemplified by standard data augmentation techniques such as Cutout, which improves clean accuracy but not robustness, and additive Gaussian noise, which improves robustness but hurts accuracy. We introduce Patch Gaussian, a simple augmentation scheme that adds noise to randomly selected patches in an input image.   Models trained with Patch Gaussian achieve state of the art on the CIFAR-10 and ImageNet Common Corruptions benchmarks while also maintaining accuracy on clean data. We find that this augmentation leads to reduced sensitivity to high frequency noise (similar to Gaussian) while retaining the ability to take advantage of relevant high frequency information in the image (similar to Cutout). We show it can be used in conjunction with other regularization methods and data augmentation policies such as AutoAugment.   Finally, we find that the idea of restricting perturbations to patches can also be useful in the context of adversarial learning, yielding models without the loss in accuracy that is found with unconstrained adversarial training.

SOLUTION: Simple augmentation method overcomes robustness/accuracy trade-off observed in literature and opens questions about the effect of training distribution on out-of-distribution generalization.

PROBLEM: In complex transfer learning scenarios new tasks might not be tightly linked to previous tasks. Approaches that transfer information contained only in the final parameters of a source model will therefore struggle. Instead, transfer learning at at higher level of abstraction is needed. We propose Leap, a framework that achieves this by transferring knowledge across learning processes. We associate each task with a manifold on which the training process travels from initialization to final parameters and construct a meta-learning objective that minimizes the expected length of this path. Our framework leverages only information obtained during training and can be computed on the fly at negligible cost. We demonstrate that our framework outperforms competing methods, both in meta-learning and transfer learning, on a set of computer vision tasks. Finally, we demonstrate that Leap can transfer knowledge across learning processes in demanding reinforcement learning environments (Atari) that involve millions of gradient steps.

SOLUTION: We propose Leap, a framework that transfers knowledge across learning processes by  minimizing the expected distance the training process travels on a task's loss surface.

PROBLEM: Program synthesis is the task of automatically generating a program consistent with
 a specification. Recent years have seen proposal of a number of neural approaches
 for program synthesis, many of which adopt a sequence generation paradigm similar
 to neural machine translation, in which sequence-to-sequence models are trained to
 maximize the likelihood of known reference programs. While achieving impressive
 results, this strategy has two key limitations. First, it ignores Program Aliasing: the
 fact that many different programs may satisfy a given specification (especially with
 incomplete specifications such as a few input-output examples). By maximizing
 the likelihood of only a single reference program, it penalizes many semantically
 correct programs, which can adversely affect the synthesizer performance. Second,
 this strategy overlooks the fact that programs have a strict syntax that can be
 efficiently checked. To address the first limitation, we perform reinforcement
 learning on top of a supervised model with an objective that explicitly maximizes
 the likelihood of generating semantically correct programs. For addressing the
 second limitation, we introduce a training procedure that directly maximizes the
 probability of generating syntactically correct programs that fulfill the specification.
 We show that our contributions lead to improved accuracy of the models, especially
 in cases where the training data is limited.

SOLUTION:
Using the DSL grammar and reinforcement learning to improve synthesis of programs with complex control flow.