In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.

Let me give you an example: We propose a local coherence model based on a convolutional neural network that operates over the entity grid representation of a text. The model captures long range entity transitions along with entity-specific features without loosing generalization, thanks to the power of distributed representation. We present a pairwise ranking method to train the model in an end-to-end fashion on a task and learn task-specific high level features. Our evaluation on three different coherence assessment tasks demonstrates that our model achieves state of the art results outperforming existing models by a good margin.
The answer to this example can be: A Neural Local Coherence Model
Here is why: This statement "A Neural Local Coherence Model" is taken from the subtext "convolutional neural network" and its explanation in the passage. This is a positive example because the title belongs to the mentions in the passage

OK. solve this:
In this paper we present the first empirical study of the emphatic temporaldifference learning algorithm (ETD), comparing it with conventional temporaldifference learning, in particular, with linear TD(0), on on-policy and off-policy variations of the Mountain Car problem. The initial motivation for developing ETD was that it has good convergence properties under off -policy training (Sutton, Mahmood & White 2016), but it is also a new algorithm for the on-policy case. In both our on-policy and off-policy experiments, we found that each method converged to a characteristic asymptotic level of error, with ETD better than TD(0). TD(0) achieved a still lower error level temporarily before falling back to its higher asymptote, whereas ETD never showed this kind of "bounce". In the off-policy case (in which TD(0) is not guaranteed to converge), ETD was significantly slower. 1 Emphatic Temporal Difference Learning We consider the problem of learning the value function for a Markov decision process and a given policy. An agent and environment interact at discrete time steps, t = 0, 1, 2, . . ., at each of which the environment is in a state St, the agent selects an action At and as a result the environment emits a reward Rt+1 and a next state St+1. States are represented to the agent as feature vectors t = (St) R. We seek to find a parameter vector, t R such that the inner product > t t approximates the expected return E [ Rt+1 + Rt+2 + Rt+3 + | At: ] , where : A S [0, 1] is a policy for selecting the future actions. In fact, all actions are selected by an alternate policy . If = , then the training is called on-policy, whereas if the two policies are different the training is called off-policy. We consider the special case of the emphatic temporal difference learning algorithm (ETD) in which bootstrapping is complete ((s) = 0,s) and there is no discounting ((s) = 1,s). Studying TD and ETD methods with complete bootstrapping is suitable because in this case the differences between them are maximized. As approaches 1, the methods behave more similarly up to the point where they become equivalent when = 1. By setting = 0 and = 1, the ETD algorithm can be 30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain. ar X iv :1 70 5. 04 18 5v 2 [ cs .A I] 1 2 M ay 2 01 7 completely described by: t+1 . = t + tFt ( Rt+1 + T t t+1 t t ) t, Ft . = t1Ft1 + 1, with F0 . = 1, t . = (At|St) (At|St) , where > 0 is a step size parameter. F is the followon trace according to which the update at each time step is emphasized or de-emphasized. TD is obtained by removing the F from the first equation. Because of F , ETD is different from TD even in the on-policy case in which is always 1. For a thorough explanation of ETD see (Sutton, Mahmood & White 2016). 2 Stability of On-policy TD with Variable : A Counterexample In this section we show that although the initial motivation for developing ETD was that it has good convergence properties under off-policy training (Yu 2015), it is also a different algorithm under on-policy training. To emphasize the difference between the two, we present a simple example for which TD() is not convergent under on-policy training but ETD is. It has long been known that TD() converges with any constant value of under on-policy training (Tsitsiklis & Van Roy 1997). Surprisingly, TD() is not assured to converge with varying even under on-policy training. Yu has recently presented a counterexample (personal communication) with state dependent for which on-policy TD() is not convergent. The example is a simple Markov decision process consisting of two states in which the system simply moves from one state to another in a cycle. The process starts in each of the states with equal probability. Let (S1) = 0 and (S2) = 1, (S1) = (3, 1) and (S2) = (1, 1) and = 0.95. As shown below, the TD() key matrix for this problem is not positive definite. Moreover, both eigenvalues of the key matrix have negative real parts and thus TD() diverges in this case. S1 S2 Key matrix = ( 0.4862 0.1713 0.7787 0.0738 ) This is while ETD is convergent under both off-policy and on-policy training with variable . This example appears in more detail in the supplementary material. 3 Fixed-policy Mountain Car Testbed For our experimental study, we used a new variation of the mountain car control problem (Sutton & Barto 1998) to form a prediction problem. The original mountain car problem has a 2-dimensional space, position (between -1.2 and 0.6), and velocity (between -0.07 and 0.07) with three actions, full throttle forward, full throttle backward, and 0 throttle. Each episode starts around the bottom of a hill (a uniform random number between -0.4 and -0.6). The reward is -1 on all time steps until the car pasts its goal at the top of the hill, which ends the episode. The task is undiscounted. Our variation of the mountain car problem has a fixed target policy which is to always push towards the direction of the velocity and not to push in any direction when the velocity is 0. We call the new variation of the mountain car problem, the fixed-policy mountain car testbed. The performance measure we used is an estimation of the mean squared value error (MSVE) which reflects the mean squared difference between the true value function and the estimated value function, weighted according to how often each state is visited in the state space following the behavior policy:
Answer:
A First Empirical Study of Emphatic Temporal Difference Learning