TASK DEFINITION: In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.
PROBLEM: Firstly, LastStateRNN is the classic RNN model, where the last state passes through an MLP and then the LR Layer estimates the corresponding probability. In contrast, in the AvgRNN model we consider the average vector of all states that come out of the cells. The AttentionRNN model is the one that it has been presented in BIBREF9.

SOLUTION: What was the baseline?

PROBLEM: First, we propose a class of recurrent-like neural networks for NLP tasks that satisfy the differential equation DISPLAYFORM0

where DISPLAYFORM0

and where INLINEFORM0 and INLINEFORM1 are learned functions. INLINEFORM2 corresponds to traditional RNNs, with INLINEFORM3 . For INLINEFORM4 , this takes the form of RNN cells with either nested internal memories or dependencies that extend temporally beyond the immediately previous hidden state.

SOLUTION: What novel class of recurrent-like networks is proposed?

PROBLEM: Three clear domains can be noticed in the behavior of $\langle cc
angle $ versus $\wp $, at $t_f$, as shown in Fig. FIGREF15 (blue squares). Phase I: $\langle cc
angle $ increases smoothly for $\wp < 0.4$, indicating that for this domain there is a small correlation between word neighborhoods. Full vocabularies are attained also for $\wp < 0.4$; Phase II: a drastic transition appears at the critical domain $\wp ^* \in (0.4,0.6)$, in which $\langle cc
angle $ shifts abruptly towards 1. An abrupt change in $V(t_f)$ versus $\wp $ is also found (Fig. FIGREF16) for $\wp ^*$; Phase III: single-word languages dominate for $\wp > 0.6$. The maximum value of $\langle cc
angle $ indicate that word neighborhoods are completely correlated.

SOLUTION:
What are three possible phases for language formation?