TASK DEFINITION: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
PROBLEM: Concept maps can be used to concisely represent important information and bring structure into large document collections. Therefore, we study a variant of multidocument summarization that produces summaries in the form of concept maps. However, suitable evaluation datasets for this task are currently missing. To close this gap, we present a newly created corpus of concept maps that summarize heterogeneous collections of web documents on educational topics. It was created using a novel crowdsourcing approach that allows us to efficiently determine important elements in large document collections. We release the corpus along with a baseline system and proposed evaluation protocol to enable further research on this variant of summarization.

SOLUTION: Connecting the dots: Summarizing and Structuring Large Document Collections Using Concept Maps

PROBLEM: We introduce word vectors for the construction domain. Our vectors were obtained by running word2vec on an 11M-word corpus that we created from scratch by leveraging freely-accessible online sources of construction-related text. We first explore the embedding space and show that our vectors capture meaningful constructionspecific concepts. We then evaluate the performance of our vectors against that of ones trained on a 100B-word corpus (Google News) within the framework of an injury report classification task. Without any parameter tuning, our embeddings give competitive results, and outperform the Google News vectors in many cases. Using a keyword-based compression of the reports also leads to a significant speed-up with only a limited loss in performance. We release our corpus and the data set we created for the classification task as publicly available, in the hope that they will be used by future studies for benchmarking and building on our work.

SOLUTION: Word Embeddings for the Construction Domain

PROBLEM: The activation function of Deep Neural Networks (DNNs) has undergone many changes during the last decades. Since the advent of the well-known non-saturated Rectified Linear Unit (ReLU), many have tried to further improve the performance of the networks with more elaborate functions. Examples are the Leaky ReLU (LReLU) to remove zero gradients and Exponential Linear Unit (ELU) to reduce bias shift. In this paper, we introduce the Parametric ELU (PELU), an adaptive activation function that allows the DNNs to adopt different non-linear behaviors throughout the training phase. We contribute in three ways: (1) we show that PELU increases the network flexibility to counter vanishing gradient, (2) we provide a gradient-based optimization framework to learn the parameters of the function, and (3) we conduct several experiments on MNIST, CIFAR-10/100 and ImageNet with different network architectures, such as NiN, Overfeat, All-CNN, ResNet and Vgg, to demonstrate the general applicability of the approach. Our proposed PELU has shown relative error improvements of 4.45% and 5.68% on CIFAR10 and 100, and as much as 7.28% with only 0.0003% parameter increase on ImageNet, along with faster convergence rate in almost all test scenarios. We also observed that Vgg using PELU tended to prefer activations saturating close to zero, as in ReLU, except at last layer, which saturated near -2. These results suggest that varying the shape of the activations during training along with the other parameters helps to control vanishing gradients and bias shift, thus facilitating learning.

SOLUTION:
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks