In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

Ex Input:
Asynchronous distributed gradient descent algorithms for training of deep neural
 networks are usually considered as inefficient, mainly because of the Gradient delay
 problem. In this paper, we propose a novel asynchronous distributed algorithm
 that tackles this limitation by well-thought-out averaging of model updates, computed
 by workers. The algorithm allows computing gradients along the process
 of gradient merge, thus, reducing or even completely eliminating worker idle time
 due to communication overhead, which is a pitfall of existing asynchronous methods.
 We provide theoretical analysis of the proposed asynchronous algorithm,
 and show its regret bounds. According to our analysis, the crucial parameter for
 keeping high convergence rate is the maximal discrepancy between local parameter
 vectors of any pair of workers. As long as it is kept relatively small, the
 convergence rate of the algorithm is shown to be the same as the one of a sequential
 online learning. Furthermore, in our algorithm, this discrepancy is bounded
 by an expression that involves the staleness parameter of the algorithm, and is
 independent on the number of workers. This is the main differentiator between
 our approach and other solutions, such as Elastic Asynchronous SGD or Downpour
 SGD, in which that maximal discrepancy is bounded by an expression that
 depends on the number of workers, due to gradient delay problem. To demonstrate
 effectiveness of our approach, we conduct a series of experiments on image
 classification task on a cluster with 4 machines, equipped with a commodity communication
 switch and with a single GPU card per machine. Our experiments
 show a linear scaling on 4-machine cluster without sacrificing the test accuracy,
 while eliminating almost completely worker idle time. Since our method allows
 using commodity communication switch, it paves a way for large scale distributed
 training performed on commodity clusters.

Ex Output:
A method for an efficient asynchronous distributed training of deep learning models along with theoretical regret bounds.


Ex Input:
Recurrent neural networks (RNNs) can model natural language by sequentially ''reading'' input tokens and outputting a distributed representation of each token. Due to the sequential nature of RNNs, inference time is linearly dependent on the input length, and all inputs are read regardless of their importance. Efforts to speed up this inference, known as ''neural speed reading'', either ignore or skim over part of the input. We present Structural-Jump-LSTM: the first neural speed reading model to both skip and jump text during inference. The model consists of a standard LSTM and two agents: one capable of skipping single words when reading, and one capable of exploiting punctuation structure (sub-sentence separators (,:), sentence end symbols (.!?), or end of text markers) to jump ahead after reading a word.
 A comprehensive experimental evaluation of our model against all five state-of-the-art neural reading models shows that 
 Structural-Jump-LSTM achieves the best overall floating point operations (FLOP) reduction (hence is faster), while keeping the same accuracy or even improving it compared to a vanilla LSTM that reads the whole text.

Ex Output:
We propose a new model for neural speed reading that utilizes the inherent punctuation structure of a text to define effective jumping and skipping behavior.


Ex Input:
Deep latent variable models have seen recent success in many data domains. Lossless compression is an application of these models which, despite having the potential to be highly useful, has yet to be implemented in a practical manner. We present '`Bits Back with ANS' (BB-ANS), a scheme to perform lossless compression with latent variable models at a near optimal rate. We demonstrate this scheme by using it to compress the MNIST dataset with a variational auto-encoder model (VAE), achieving compression rates superior to standard methods with only a simple VAE. Given that the scheme is highly amenable to parallelization, we conclude that with a sufficiently high quality generative model this scheme could be used to achieve substantial improvements in compression rate with acceptable running time. We make our implementation available open source at https://github.com/bits-back/bits-back .

Ex Output:

We do lossless compression of large image datasets using a VAE, beat existing compression algorithms.