In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

Input: Consider Input: We describe two end-to-end autoencoding models for semi-supervised graph-based dependency parsing. The first model is a Local Autoencoding Parser (LAP) encoding the input using continuous latent variables in a sequential manner; The second model is a Global Autoencoding Parser (GAP) encoding the input into dependency trees as latent variables, with exact inference. Both models consist of two parts: an encoder enhanced by deep neural networks (DNN) that can utilize the contextual information to encode the input into latent variables, and a decoder which is a generative model able to reconstruct the input. Both LAP and GAP admit a unified structure with different loss functions for labeled and unlabeled data with shared parameters. We conducted experiments on WSJ and UD dependency parsing data sets, showing that our models can exploit the unlabeled data to boost the performance given a limited amount of labeled data.

Output: We describe two end-to-end autoencoding parsers for semi-supervised graph-based dependency parsing.


Input: Consider Input: Asynchronous distributed gradient descent algorithms for training of deep neural
 networks are usually considered as inefficient, mainly because of the Gradient delay
 problem. In this paper, we propose a novel asynchronous distributed algorithm
 that tackles this limitation by well-thought-out averaging of model updates, computed
 by workers. The algorithm allows computing gradients along the process
 of gradient merge, thus, reducing or even completely eliminating worker idle time
 due to communication overhead, which is a pitfall of existing asynchronous methods.
 We provide theoretical analysis of the proposed asynchronous algorithm,
 and show its regret bounds. According to our analysis, the crucial parameter for
 keeping high convergence rate is the maximal discrepancy between local parameter
 vectors of any pair of workers. As long as it is kept relatively small, the
 convergence rate of the algorithm is shown to be the same as the one of a sequential
 online learning. Furthermore, in our algorithm, this discrepancy is bounded
 by an expression that involves the staleness parameter of the algorithm, and is
 independent on the number of workers. This is the main differentiator between
 our approach and other solutions, such as Elastic Asynchronous SGD or Downpour
 SGD, in which that maximal discrepancy is bounded by an expression that
 depends on the number of workers, due to gradient delay problem. To demonstrate
 effectiveness of our approach, we conduct a series of experiments on image
 classification task on a cluster with 4 machines, equipped with a commodity communication
 switch and with a single GPU card per machine. Our experiments
 show a linear scaling on 4-machine cluster without sacrificing the test accuracy,
 while eliminating almost completely worker idle time. Since our method allows
 using commodity communication switch, it paves a way for large scale distributed
 training performed on commodity clusters.

Output: A method for an efficient asynchronous distributed training of deep learning models along with theoretical regret bounds.


Input: Consider Input: We study the error landscape of deep linear and nonlinear neural networks with the squared error loss. Minimizing the loss of a deep linear neural network is a nonconvex problem, and despite recent progress, our understanding of this loss surface is still incomplete. For deep linear networks, we present necessary and sufficient conditions for a critical point of the risk function to be a global minimum. Surprisingly, our conditions provide an efficiently checkable test for global optimality, while such tests are typically intractable in nonconvex optimization. We further extend these results to deep nonlinear neural networks and prove similar sufficient conditions for global optimality, albeit in a more limited function space setting.


Output: We provide efficiently checkable necessary and sufficient conditions for global optimality in deep linear neural networks, with some initial extensions to nonlinear settings.