In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.

[EX Q]: A variety of real-world processes (over networks) produce sequences of data whose complex temporal dynamics need to be studied. More especially, the event timestamps can carry important information about the underlying network dynamics, which otherwise are not available from the time-series evenly sampled from continuous signals. Moreover, in most complex processes, event sequences and evenly-sampled times series data can interact with each other, which renders joint modeling of those two sources of data necessary. To tackle the above problems, in this paper, we utilize the rich framework of (temporal) point processes to model event data and timely update its intensity function by the synergic twin Recurrent Neural Networks (RNNs). In the proposed architecture, the intensity function is synergistically modulated by one RNN with asynchronous events as input and another RNN with time series as input. Furthermore, to enhance the interpretability of the model, the attention mechanism for the neural point process is introduced. The whole model with event type and timestamp prediction output layers can be trained end-to-end and allows a black-box treatment for modeling the intensity. We substantiate the superiority of our model in synthetic data and three real-world benchmark datasets.
[EX A]: Joint Modeling of Event Sequence and Time Series with Attentional Twin Recurrent Neural Networks

[EX Q]: Policies for complex visual tasks have been successfully learned with deep reinforcement learning, using an approach called deep Q-networks (DQN), but relatively large (task-specific) networks and extensive training are needed to achieve good performance. In this work, we present a novel method called policy distillation that can be used to extract the policy of a reinforcement learning agent and train a new network that performs at the expert level while being dramatically smaller and more efficient. Furthermore, the same method can be used to consolidate multiple task-specific policies into a single policy. We demonstrate these claims using the Atari domain and show that the multi-task distilled agent outperforms the single-task teachers as well as a jointly-trained DQN agent.
[EX A]: POLICY DISTILLATION

[EX Q]: Given recent deep learning results that demonstrate the ability to effectively optimize high-dimensional non-convex functions with gradient descent optimization on GPUs, we ask in this paper whether symbolic gradient optimization tools such as Tensorflow can be effective for planning in hybrid (mixed discrete and continuous) nonlinear domains with high dimensional state and action spaces? To this end, we demonstrate that hybrid planning with Tensorflow and RMSProp gradient descent is competitive with mixed integer linear program (MILP) based optimization on piecewise linear planning domains (where we can compute optimal solutions) and substantially outperforms state-of-the-art interior point methods for nonlinear planning domains. Furthermore, we remark that Tensorflow is highly scalable, converging to a strong policy on a largescale concurrent domain with a total of 576,000 continuous actions over a horizon of 96 time steps in only 4 minutes. We provide a number of insights that clarify such strong performance including observations that despite long horizons, RMSProp avoids both the vanishing and exploding gradients problem. Together these results suggest a new frontier for highly scalable planning in nonlinear hybrid domains by leveraging GPUs and the power of recent advances in gradient descent with highly optmized toolkits like Tensorflow. Introduction Many real-world hybrid (mixed discrete continuous) planning problems such as Reservoir Control [Yeh, 1985], Heating, Ventilation and Air Conditioning (HVAC) [Erickson et al., 2009; Agarwal et al., 2010], and Navigation [Faulwasser and Findeisen, 2009] have highly nonlinear transition and (possibly nonlinear) reward functions to optimize. Unfortunately, existing state-of-the-art hybrid planners [Ivankovic et al., 2014; Löhr et al., 2012; Coles et al., 2013; Piotrowski et al., 2016] are not compatible with arbitrary nonlinear transition and reward models. Monte Carlo Tree Search (MCTS) methods [Coulom, 2006; Kocsis and Szepesvári, 2006; Keller and Helmert, 2013] 0.600 0.750 .900 0.600 0.750 .900 0.600 0.750 0.9 00 0.600 0.750 .900 0.600 0.750 .900 0.600 0.750 .900 0.600 0.750 0.9 00 Epochs:10 Epochs:20 Epochs:40 Epochs:80 Epochs:160 Epochs:320 Figure 1: The evolution of RMSProp gradient descent based Tensorflow planning in a 2D Navigation domain with nested central rectangles indicating nonlinearly increasing resistance to robot movement. (top) In initial RMSProp epochs, the plan evolves directly towards the goal shown as a star. (bottom) As later epochs of RMSProp descend the objective cost surface, the fastest path evolves to avoid the central obstacle entirely. including AlphaGo [Silver et al., 2016] that can use any (nonlinear) black box model of transition dynamics do not inherently work with continuous action spaces due to the infinite branching factor. While MCTS with continuous action extensions such as HOOT [Weinstein and Littman, 2012] have been proposed, their continuous partitioningmethods do not scale to high-dimensional continuous action spaces (e.g., 100’s or 1,000’s of dimensions as used in this paper). Finally, offline model-free reinforcement learning (e.g., Q-learning) with function approximation [Sutton and Barto, 1998; Szepesvári, 2010] and deep extensions [Mnih et al., 2013] do not require any knowledge of the (nonlinear) transition model or reward, but they also do not directly apply to domains with high-dimensional continuous action spaces. I.e., offline learning methods like Q-learning require action maximization for every update, but in high-dimensional continuous action spaces such nonlinear function maximization is non-convex and computationally intractable at the scale of millions or billions of updates. To address the above scalability and expressivity limitations of existing methods, we turn to Tensorflow [Abadi et al., 2015], which is a symbolic computation platform used in the machine learning community for deep learning due to its compilation of complex layered symbolic functions into a representation amenable to fast GPU-based reverse-mode automatic differentiation [Linnainmaa, 1970] for gradient-based optimization. Given recent results in gradient descent optimization with deep learning that demonstrate the ability to effectively optimize high-dimensional non-convex functions, we ask whether Tensorflow can be effective for planning in hybrid (mixed discrete and continuous) nonlinear domains with high dimensional state and action spaces? Our results answer this question affirmatively, where we demonstrate that hybrid planning with Tensorflow and RMSProp gradient descent [Tieleman and Hinton, 2012] is surprisingly effective at planning in complex hybrid nonlinear domains. As evidence, we reference figure 1, where we show Tensorflow with RMSProp optimizing a path in a 2d nonlinear Navigation domain. In general, Tensorflow with RMSProp planning results are competitive with optimal MILPbased optimization on piecewise linear planning domains and directly extend to nonlinear domains, where they substantially outperform interior point methods for nonlinear function optimization. Furthermore, we remark that Tensorflow converges to a strong policy on a large-scale concurrent domain with 576,000 continuous actions spread over a horizon of 96 time steps in 4 minutes. To explain such excellent results, we note that gradient descent algorithms such as RMSProp are highly effective for the non-convex function optimization that occurs in deep learning. Further, we provide an analysis of many transition functions in planning domains that suggest gradient descent on these domains will not suffer from either the vanishing or exploding gradient problems and hence provide a strong signal for optimization over long horizons. Together these results suggest a new frontier for highly scalable planning in nonlinear hybrid domains by leveraging GPUs and the power of recent advances in gradient descent with the Tensorflow toolkit. Hybrid Nonlinear Planning via Tensorflow
[EX A]:
Scalable Planning with Tensorflow for Hybrid Nonlinear Domains