TASK DEFINITION: In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
PROBLEM: Recent work has demonstrated that neural networks are vulnerable to adversarial examples, i.e., inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against a well-defined class of adversaries. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest robustness against a first-order adversary as a natural security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models.

SOLUTION: We provide a principled, optimization-based re-look at the notion of adversarial examples, and develop methods that produce models that are adversarially robust against a wide range of adversaries.

PROBLEM: Bayesian optimization (BO) is a popular methodology to tune the hyperparameters of expensive black-box functions. Despite its success, standard BO focuses on a single task at a time and is not designed to leverage information from related functions, such as tuning performance metrics of the same algorithm across multiple datasets. In this work, we introduce a novel approach to achieve transfer learning across different datasets as well as different metrics. The main idea is to regress the mapping from hyperparameter to metric quantiles with a semi-parametric Gaussian Copula distribution, which provides robustness against different scales or outliers that can occur in different tasks. We introduce two methods to leverage this estimation: a Thompson sampling strategy as well as a Gaussian Copula process using such quantile estimate as a prior. We show that these strategies can combine the estimation of multiple metrics such as runtime and accuracy, steering the optimization toward cheaper hyperparameters for the same level of accuracy. Experiments on an extensive set of hyperparameter tuning tasks demonstrate significant improvements over state-of-the-art methods.

SOLUTION: We show how using semi-parametric prior estimations can speed up HPO significantly across datasets and metrics.

PROBLEM: Recurrent neural networks (RNNs) are widely used to model sequential data but
 their non-linear dependencies between sequence elements prevent parallelizing
 training over sequence length. We show the training of RNNs with only linear
 sequential dependencies can be parallelized over the sequence length using the
 parallel scan algorithm, leading to rapid training on long sequences even with
 small minibatch size. We develop a parallel linear recurrence CUDA kernel and
 show that it can be applied to immediately speed up training and inference of
 several state of the art RNN architectures by up to 9x.  We abstract recent work
 on linear RNNs into a new framework of linear surrogate RNNs and develop a
 linear surrogate model for the long short-term memory unit, the GILR-LSTM, that
 utilizes parallel linear recurrence.  We extend sequence learning to new
 extremely long sequence regimes that were previously out of reach by
 successfully training a GILR-LSTM on a synthetic sequence classification task
 with a one million timestep dependency.


SOLUTION:
use parallel scan to parallelize linear recurrent neural nets. train model on length 1 million dependency