In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

Input: Consider Input: We propose a framework to model the distribution of sequential data coming from
 a set of entities connected in a graph with a known topology. The method is
 based on a mixture of shared hidden Markov models (HMMs), which are trained
 in order to exploit the knowledge of the graph structure and in such a way that the
 obtained mixtures tend to be sparse. Experiments in different application domains
 demonstrate the effectiveness and versatility of the method.

Output: A method to model the generative distribution of sequences coming from graph connected entities.


Input: Consider Input: Understanding the optimization trajectory is critical to understand training of deep neural networks. We show how the hyperparameters of stochastic gradient descent influence the covariance of the gradients (K) and the Hessian of the training loss (H) along this trajectory. Based on a theoretical model, we predict that using a high learning rate or a small batch size in the early phase of training leads SGD to regions of the parameter space with (1) reduced spectral norm of K, and (2) improved conditioning of K and H. We show that the point on the trajectory after which these effects hold, which we refer to as the break-even point, is reached early during training. We demonstrate these effects empirically for a range of deep neural networks applied to multiple different tasks. Finally, we apply our analysis to networks with batch normalization (BN) layers and find that it is necessary to use a high learning rate to achieve loss smoothing effects attributed previously to BN alone.

Output: In the early phase of training of deep neural networks there exists a "break-even point" which determines properties of the entire optimization trajectory.


Input: Consider Input: Deep latent variable models are powerful tools for representation learning. In this paper, we adopt the deep information bottleneck model, identify its shortcomings and propose a model that circumvents them. To this end, we apply a copula transformation which, by restoring the invariance properties of the information bottleneck method, leads to disentanglement of the features in the latent space. Building on that, we show how this transformation translates to sparsity of the latent space in the new model.   We evaluate our method on artificial and real data.


Output: We apply the copula transformation to the Deep Information Bottleneck which leads to restored invariance properties and a disentangled latent space with superior predictive capabilities.