In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

Input: Consider Input: Recurrent neural networks (RNNs) are particularly well-suited for modeling long-term dependencies in sequential data, but are notoriously hard to train because the error backpropagated in time either vanishes or explodes at an exponential rate. While a number of works attempt to mitigate this effect through gated recurrent units, skip-connections, parametric constraints and design choices, we propose a novel incremental RNN (iRNN), where hidden state vectors keep track of incremental changes, and as such approximate state-vector increments of Rosenblatt's (1962) continuous-time RNNs. iRNN exhibits identity gradients and is able to account for long-term dependencies (LTD). We show that our method is computationally efficient overcoming overheads of many existing methods that attempt to improve RNN training, while suffering no performance degradation. We demonstrate the utility of our approach with extensive experiments and show competitive performance against standard LSTMs on LTD and other non-LTD tasks.

Output: Incremental-RNNs resolves exploding/vanishing gradient problem by updating state vectors based on difference between previous state and that predicted by an ODE.

Input: Consider Input: Understanding the groundbreaking performance of Deep Neural Networks is one
of the greatest challenges to the scientific community today. In this work, we
introduce an information theoretic viewpoint on the behavior of deep networks
optimization processes and their generalization abilities. By studying the Information
Plane, the plane of the mutual information between the input variable and
the desired label, for each hidden layer. Specifically, we show that the training of
the network is characterized by a rapid increase in the mutual information (MI)
between the layers and the target label, followed by a longer decrease in the MI
between the layers and the input variable. Further, we explicitly show that these
two fundamental information-theoretic quantities correspond to the generalization
error of the network, as a result of introducing a new generalization bound that is
exponential in the representation compression. The analysis focuses on typical
patterns of large-scale problems. For this purpose, we introduce a novel analytic
bound on the mutual information between consecutive layers in the network.
An important consequence of our analysis is a super-linear boost in training time
with the number of non-degenerate hidden layers, demonstrating the computational
benefit of the hidden layers.

Output: Introduce an information theoretic viewpoint on the behavior of deep networks optimization processes and their generalization abilities

Input: Consider Input: Neural networks for structured data like graphs have been studied extensively in recent years.
To date, the bulk of research activity has focused mainly on static graphs.
However, most real-world networks are dynamic since their topology tends to change over time.
Predicting the evolution of dynamic graphs is a task of high significance in the area of graph mining.
Despite its practical importance, the task has not been explored in depth so far, mainly due to its challenging nature.
In this paper, we propose a model that predicts the evolution of dynamic graphs.
Specifically, we use a graph neural network along with a recurrent architecture to capture the temporal evolution patterns of dynamic graphs.
Then, we employ a generative model which predicts the topology of the graph at the next time step and constructs a graph instance that corresponds to that topology.
We evaluate the proposed model on several artificial datasets following common network evolving dynamics, as well as on real-world datasets.
Results demonstrate the effectiveness of the proposed model.
Output: Combining graph neural networks and the RNN graph generative model, we propose a novel architecture that is able to learn from a sequence of evolving graphs and predict the graph topology evolution for the future timesteps