In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

Input: Consider Input: We consider new variants of optimization algorithms. Our algorithms are based on the observation that mini-batch of stochastic gradients in consecutive iterations do not change drastically and consequently may be predictable. Inspired by the similar setting in online learning literature called Optimistic Online learning, we propose two new optimistic algorithms for AMSGrad and Adam, respectively, by  exploiting the predictability of gradients.   The new algorithms combine the idea of momentum method, adaptive gradient method, and algorithms in Optimistic Online learning, which leads to speed up in training deep neural nets in practice.

Output: We consider new variants of optimization algorithms for training deep nets.


Input: Consider Input: Inspired by neurophysiological discoveries of navigation cells in the mammalian
 brain, we introduce the first deep neural network architecture for modeling Egocentric
 Spatial Memory (ESM). It learns to estimate the pose of the agent and
 progressively construct top-down 2D global maps from egocentric views in a spatially
 extended environment. During the exploration, our proposed ESM network
 model updates belief of the global map based on local observations using a recurrent
 neural network. It also augments the local mapping with a novel external
 memory to encode and store latent representations of the visited places based on
 their corresponding locations in the egocentric coordinate. This enables the agents
 to perform loop closure and mapping correction. This work contributes in the
 following aspects: first, our proposed ESM network provides an accurate mapping
 ability which is vitally important for embodied agents to navigate to goal locations.
 In the experiments, we demonstrate the functionalities of the ESM network in
 random walks in complicated 3D mazes by comparing with several competitive
 baselines and state-of-the-art Simultaneous Localization and Mapping (SLAM)
 algorithms. Secondly, we faithfully hypothesize the functionality and the working
 mechanism of navigation cells in the brain. Comprehensive analysis of our model
 suggests the essential role of individual modules in our proposed architecture and
 demonstrates efficiency of communications among these modules. We hope this
 work would advance research in the collaboration and communications over both
 fields of computer science and computational neuroscience.

Output: first deep neural network for modeling Egocentric Spatial Memory inspired by neurophysiological discoveries of navigation cells in mammalian brain


Input: Consider Input: In recent years several adversarial attacks and defenses have been proposed. Often seemingly robust models turn out to be non-robust when more sophisticated attacks are used. One way out of this dilemma are provable robustness guarantees. While provably robust models for specific $l_p$-perturbation models have been developed, we show that they do not come with any guarantee against other $l_q$-perturbations. We propose a new regularization scheme, MMR-Universal, for ReLU networks which enforces robustness wrt $l_1$- \textit{and} $l_\infty$-perturbations and show how that leads to the first provably robust models wrt any $l_p$-norm for $p\geq 1$.


Output: We introduce a method to train models with provable robustness wrt all the $l_p$-norms for $p\geq 1$ simultaneously.