TASK DEFINITION: In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
PROBLEM:   Analysis methods which enable us to better understand the
  representations and functioning of neural models of language are
  increasingly needed as deep learning becomes the dominant approach
  in NLP. Here we present two methods based on Representational
  Similarity Analysis (RSA) and Tree Kernels (TK) which allow us to
  directly quantify how strongly the information encoded in neural
  activation patterns corresponds to information represented by
  symbolic structures such as syntax trees. We first validate our
  methods on the case of a simple synthetic language for arithmetic
  expressions with clearly defined syntax and semantics, and show that
  they exhibit the expected pattern of results. We then apply our methods to
  correlate neural representations of English sentences with their
  constituency parse trees.

SOLUTION: Two methods based on Representational Similarity Analysis (RSA) and Tree Kernels (TK) which directly quantify how strongly information encoded in neural activation patterns corresponds to information represented by symbolic structures.

PROBLEM: We show that information about whether a neural network's output will be correct or incorrect is present in the outputs of the network's intermediate layers. To demonstrate this effect, we train a new "meta" network to predict from either the final output of the underlying "base" network or the output of one of the base network's intermediate layers whether the base network will be correct or incorrect for a particular input. We find that, over a wide range of tasks and base networks, the meta network can achieve accuracies ranging from 65% - 85% in making this determination.


SOLUTION: Information about whether a neural network's output will be correct or incorrect is somewhat present in the outputs of the network's intermediate layers.

PROBLEM: Performing controlled experiments on noisy data is essential in thoroughly understanding deep learning across a spectrum of noise levels. Due to the lack of suitable datasets, previous research have only examined deep learning on controlled synthetic noise, and real-world noise has never been systematically studied in a controlled setting. To this end, this paper establishes a benchmark of real-world noisy labels at 10 controlled noise levels. As real-world noise possesses unique properties, to understand the difference, we conduct a large-scale study across a variety of noise levels and types, architectures, methods, and training settings. Our study shows that: (1) Deep Neural Networks (DNNs) generalize much better on real-world noise. (2) DNNs may not learn patterns first on real-world noisy data. (3) When networks are fine-tuned, ImageNet architectures generalize well on noisy data. (4) Real-world noise appears to be less harmful, yet it is more difficult for robust DNN methods to improve. (5) Robust learning methods that work well on synthetic noise may not work as well on real-world noise, and vice versa. We hope our benchmark, as well as our findings, will facilitate deep learning research on noisy data.


SOLUTION:
We establish a benchmark of controlled real noise and reveal several interesting findings about real-world noisy data.