Original: F AST AND ACCURATE TEXT CLASSIFICATION : SKIMMING , REREADING AND EARLY STOPPING
Revised: F AST AND ACCURATE TEXT CLASSIFICATION : SKIMMING , REREADING AND EARLY STOPPING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Recent advances in recurrent neural nets (RNNs) have shown much promise in many applications in natural language processing.
Revised: Recent advances in recurrent neural nets (RNNs) have shown much promise in many applications in natural language processing.

Original: For most of these tasks, such as sentiment analysis of customer reviews, a recurrent neural net model parses the entire review before forming a decision.
Revised: For most of these tasks, such as sentiment analysis of customer reviews, a recurrent neural net model parses the entire review before forming a decision.

Original: We argue that reading the entire input is not always necessary in practice, since a lot of reviews are often easy to classify, i.e., a decision can be formed after reading some crucial sentences or words in the provided text.
Revised: We argue that reading the entire input is not always necessary in practice, since a lot of reviews are often easy to classify, i.e., a decision can be formed after reading some crucial sentences or words in the provided text.

Original: In this paper, we present an approach of fast reading for text classiﬁcation.
Revised: In this paper, we present an approach of fast reading for text classiﬁcation.

Original: Inspired by several well-known human reading techniques, our approach implements an intelligent recurrent agent which evaluates the importance of the current snippet in order to decide whether to make a prediction, or to skip some texts, or to re-read part of the sentence.
Revised: Inspired by several well-known human reading techniques, our approach implements an intelligent recurrent agent which evaluates the importance of the current snippet in order to decide whether to make a prediction, or to skip some texts, or to re-read part of the sentence.

Original: Our agent uses an RNN module to encode information from the past and the current tokens, and applies a policy module to form decisions.
Revised: Our agent uses an RNN module to encode information from the past and the current tokens, and applies a policy module to form decisions.

Original: With an end-to-end training algorithm based on policy gradient, we train and test our agent on several text classiﬁcation datasets and achieve both higher efﬁciency and better accuracy compared to previous approaches.
Revised: With an end-to-end training algorithm based on policy gradient, we train and test our agent on several text classiﬁcation datasets and achieve both higher efﬁciency and better accuracy compared to previous approaches.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Recurrent neural nets (RNNs), including GRU nets (Chung et al., 2014) and LSTM nets (Hochreiter & Schmidhuber, 1997), have been increasingly applied to many problems in natural language processing.
Revised: Recurrent neural nets (RNNs), including GRU nets (Chung et al., 2014) and LSTM nets (Hochreiter & Schmidhuber, 1997), have been increasingly applied to many problems in natural language processing.

Original: Most of the problems can be divided into two categories: sequence to sequence (seq2seq) tasks (Sutskever et al., 2014)
Revised: Most of the problems can be divided into two categories: sequence to sequence (seq2seq) tasks (Sutskever et al., 2014)

Original: (e.g., language modeling (Bengio et al., 2003; Mikolov et al., 2010), machine translation (Cho et al., 2014; Bahdanau et al., 2014; Kalchbrenner & Blunsom, 2013), conversational/dialogue modeling (Serban et al., 2016), question answering (Hermann et al., 2015;
Revised: (e.g., language modeling (Bengio et al., 2003; Mikolov et al., 2010), machine translation (Cho et al., 2014; Bahdanau et al., 2014; Kalchbrenner & Blunsom, 2013), conversational/dialogue modeling (Serban et al., 2016), question answering (Hermann et al., 2015;

Original: Weston et al., 2015; Lee et al., 2016), and document summarization (Rush et al., 2015; Nallapati et al., 2016)); and thesequence to classiﬁcation tasks (e.g., part-of-speech tagging (Santos & Zadrozny, 2014), chunking, named entity recognition (Collobert et al., 2011), sentimental analysis (Socher et al., 2011), and document classiﬁcation (Kim, 2014; Sebastiani, 2002)).
Revised: Weston et al., 2015; Lee et al., 2016), and document summarization (Rush et al., 2015; Nallapati et al., 2016)); and the classiﬁcation tasks (e.g., part-of-speech tagging (Santos & Zadrozny, 2014), chunking, named entity recognition (Collobert et al., 2011), sentimental analysis (Socher et al., 2011), and document classiﬁcation (Kim, 2014; Sebastiani, 2002)).

Original: To solve these problems, our models often need to read every token or word of the text from beginning to the end, which is necessary for most seq2seq problems.
Revised: To solve these problems, our models often need to read every token or word of the text from beginning to the end, which is necessary for most seq2seq problems.

Original: However, for classiﬁcation problems, we do not have to treat each individual word equally, since certain words or chunks are more relevant to the classiﬁcation task at hand.
Revised: However, for classiﬁcation problems, we do not have to treat each individual word equally, since certain words or chunks are more relevant to the classiﬁcation task at hand.

Original: For instance, for sentiment analysis it is sufﬁcient to read the ﬁrst half of a review like “this movie is amazing” or “it is the best I have ever seen,” to provide an answer even without reading the rest of the review.
Revised: For instance, for sentiment analysis it is sufﬁcient to read the ﬁrst half of a review like “this movie is amazing” or “it is the best I have ever seen,” to provide an answer even without reading the rest of the review.

Original: In other cases, we may want to skip or skim some text without carefully checking it.
Revised: In other cases, we may want to skip or skim some text without carefully checking it.

Original: For example, sentences such as “it’s worth to try” are usually more important than irrelevant text such as “we got here while it’s still raining outside” or “I visited on Saturday.”
Revised: For example, sentences such as “it’s worth to try” are usually more important than irrelevant text such as “we got here while it’s still raining outside” or “I visited on Saturday.”

Original: On the other hand, sometimes, we want to re-read some sentences to ﬁgure out the actual hidden message of the text.
Revised: On the other hand, sometimes, we want to re-read some sentences to ﬁgure out the actual hidden message of the text.

Original: All of these techniques enable us to achieve fast and accurate reading.
Revised: All of these techniques enable us to achieve fast and accurate reading.

Original: Similarly, we expect RNN models to intelligently determine the importance or the relevance of the current sentence in order to decide whether to make a prediction, whether to skip some texts, or whether to re-read the current sentence.
Revised: Similarly, we expect RNN models to intelligently determine the importance or the relevance of the current sentence in order to decide whether to make a prediction, whether to skip some texts, or whether to re-read the current sentence.

Original: In this paper, we aim to augment existing RNN models by introducing efﬁcient partial reading for classiﬁcation, while maintaining a higher or comparable accuracy compared to reading the full text.
Revised: In this paper, we aim to augment existing RNN models by introducing efﬁcient partial reading for classiﬁcation, while maintaining a higher or comparable accuracy compared to reading the full text.

Original: To do so, we introduce a recurrent agent which uses an RNN module to encode information from the past and the current tokens, and applies a policy module to decide what token to read next
Revised: To do so, we introduce a recurrent agent which uses an RNN module to encode information from the past and the current tokens, and applies a policy module to decide what token to read next

Original: LSTM
Revised: LSTM

Original: Classifier
Revised: Classifier

Original: Stop?
Revised: Stop?

Original: Next Move
Revised: Next Move

Original: Yes
Revised: Yes

Original: No
Revised: No

Original: Policy
Revised: Policy

Original: Module
Revised: Module

Original: Output Label action … …
Revised: Output Label action … …

Original: K h t labelprobability
Revised: K h t labelprobability

Original: Reread
Revised: Reread

Original: Read Sequentially
Revised: Read Sequentially

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Skip K-

Revised: Skip K-

Original: Tokens π s π c π n
Revised: Tokens π s π c π n

Original: Token
Revised: Token

Original: Encoder
Revised: Encoder

Original: Skip 2 Tokens
Revised: Skip 2 Tokens

Original: Token Token
Revised: Token Token

Original: LSTM
Revised: LSTM

Original: Output Label
Revised: Output Label

Original: Encoder Encoder
Revised: Encoder Encoder

Original: Read Sequentially
Revised: Read Sequentially

Original: Reread
Revised: Reread

Original: LSTM LSTM
Revised: LSTM LSTM

Original: Stop
Revised: Stop

Original: (e.g., rereading the current token, reading the next one, or skipping the next few tokens) or whether the model should stop reading and form a decision.
Revised: (e.g., rereading the current token, reading the next one, or skipping the next few tokens) or whether the model should stop reading and form a decision.

Original: To encourage fast and accurate reading, we incorporate both classiﬁcation accuracy and the computational cost as a reward function to score classiﬁcation or other actions made by the agent during reading.
Revised: To encourage fast and accurate reading, we incorporate both classiﬁcation accuracy and the computational cost as a reward function to score classiﬁcation or other actions made by the agent during reading.

Original: We expect that our agent will be able to achieve fast reading for classiﬁcation with both high computational efﬁciency and good classiﬁcation performance.
Revised: We expect that our agent will be able to achieve fast reading for classiﬁcation with both high computational efﬁciency and good classiﬁcation performance.

Original: To train this model, we develop an end-to-end approach based on the policy gradient method which backpropagates the reward signal into both the policy module (also including the classiﬁcation policy) and the recurrent encoder.
Revised: To train this model, we develop an end-to-end approach based on the policy gradient method which backpropagates the reward signal into both the policy module (also including the classiﬁcation policy) and the recurrent encoder.

Original: We evaluate our approach on four different sentiment analysis and document topic classiﬁcation datasets.
Revised: We evaluate our approach on four different sentiment analysis and document topic classiﬁcation datasets.

Original: By comparing to the standard RNN models and a recent LSTM-skip model which implements a skip action (Yu et al., 2017), we ﬁnd that our approach achieves both higher efﬁciency and better accuracy.
Revised: By comparing to the standard RNN models and a recent LSTM-skip model which implements a skip action (Yu et al., 2017), we ﬁnd that our approach achieves both higher efﬁciency and better accuracy.

Original:  M ETHODS 2. M ODEL O VERVIEW
Revised:  M ETHODS 2. M ODEL O VERVIEW

Original: Given an input sequence x 1: T with length T , our model aims to predict a single label y for the entire sequence, such as the topic or the sentiment of a document.
Revised: Given an input sequence x 1: T with length T , our model aims to predict a single label y for the entire sequence, such as the topic or the sentiment of a document.

Original: We develop a technique for skimming, re-reading, early stopping and prediction, with the goal of (i) skipping irrelevant information and reinforcing the important parts, and (ii) to enable fast and accurate text classiﬁcation.
Revised: We develop a technique for skimming, re-reading, early stopping and prediction, with the goal of (i) skipping irrelevant information and reinforcing the important parts, and (ii) to enable fast and accurate text classiﬁcation.

Original: Speciﬁcally, the model will read the current token/chunk x i t at time step t , encode the data x i t and previous information h t − 1 into a feature h t , and then decide the next token to read by skimming/skipping or to stop to form a ﬁnal prediction (see Figure 1).
Revised: Speciﬁcally, the model will read the current token/chunk x i t at time step t , encode the data x i t and previous information h t − 1 into a feature h t , and then decide the next token to read by skimming/skipping or to stop to form a ﬁnal prediction (see Figure 1).

Original: Such a model can be fully deﬁned on top of a
Revised: Such a model can be fully deﬁned on top of a

Original: RNN structure and trained in an end-to-end fashion via back-propagation of a well deﬁned reward signal.
Revised: RNN structure and trained in an end-to-end fashion via back-propagation of a well deﬁned reward signal.

Original: Both skimming and re-reading actions can be deﬁned similarly by ﬁrst choosing a step size k ∈ { 0 , 1 , 2 , · · · , K } and then setting i t
Revised: Both skimming and re-reading actions can be deﬁned similarly by ﬁrst choosing a step size k ∈ { 0 , 1 , 2 , · · · , K } and then setting i t

Original: +1 =
Revised: +1 =

Original: i t + k .
Revised: i t + k .

Original: When k = 0 , the model rereads the current token; when k = 1 , the model moves to the next token sequentially; when k > 1 , the model skips the next k − 1 tokens.
Revised: When k = 0 , the model rereads the current token; when k = 1 , the model moves to the next token sequentially; when k > 1 , the model skips the next k − 1 tokens.

Original: If the current action is to stop or the next token to read is after the last token of the input sequence text, the model will stop and output a label.
Revised: If the current action is to stop or the next token to read is after the last token of the input sequence text, the model will stop and output a label.

Original: All of these actions are deﬁned by a policy module Π which takes the recurrent feature h t as input and outputs a stop signal and a label or generates a step size k and moves to the next token x i t +

Revised: All of these actions are deﬁned by a policy module Π which takes the recurrent feature h t as input and outputs a stop signal and a label or generates a step size k and moves to the next token x i t +

Original: =
Revised: =

Original: i t + k .
Revised: i t + k .

Original: 2. M ODEL SPECIFICATION AND TRAINING
Revised: 2. M ODEL SPECIFICATION AND TRAINING

Original: The design of the policy module Π plays an critical role in our framework.
Revised: The design of the policy module Π plays an critical role in our framework.

Original: It should (i) read as much signiﬁcant text as possible to ensure a conﬁdent classiﬁcation output and (ii) be computationally efﬁcient, e.g., avoiding reading to the end of the text if sufﬁcient information is already obtained and skipping irrelevant or unimportant part of the text.
Revised: It should (i) read as much signiﬁcant text as possible to ensure a conﬁdent classiﬁcation output and (ii) be computationally efﬁcient, e.g., avoiding reading to the end of the text if sufﬁcient information is already obtained and skipping irrelevant or unimportant part of the text.

Original: More speciﬁcally, for each step, the policy module Π should decide whether the information collected is convincing enough to stop reading and make a prediction.
Revised: More speciﬁcally, for each step, the policy module Π should decide whether the information collected is convincing enough to stop reading and make a prediction.

Original: Otherwise it will need to evaluate the importance of the current semantic unit or token just read to decide which token to be read in the next step.
Revised: Otherwise it will need to evaluate the importance of the current semantic unit or token just read to decide which token to be read in the next step.

Original: By formulating this process as a sequential decision process, at each time step t , the policy module takes the output h t of an encoder, which summarizes the text read before and the current token x i t , and outputs a probability distribution π t deﬁned over actions.
Revised: By formulating this process as a sequential decision process, at each time step t , the policy module takes the output h t of an encoder, which summarizes the text read before and the current token x i t , and outputs a probability distribution π t deﬁned over actions.

Original: It is worth noting that to save computation, the actions are determined only by the latent representation h t .
Revised: It is worth noting that to save computation, the actions are determined only by the latent representation h t .

Original: At each time step t , a sequence of actions are generated by ﬁrst sampling a stopping decision in the form of a binary variable s from a Bernoulli distribution π S ( ·| h t ) .
Revised: At each time step t , a sequence of actions are generated by ﬁrst sampling a stopping decision in the form of a binary variable s from a Bernoulli distribution π S ( ·| h t ) .

Original: If s = 1 , the model stops and draws a label ˆ y from a conditional multinomial distribution speciﬁed by a classiﬁer π C ( ·| h t , s = 1) ; otherwise, the model draws a step size k ∈ { 0 , . .
Revised: If s = 1 , the model stops and draws a label ˆ y from a conditional multinomial distribution speciﬁed by a classiﬁer π C ( ·| h t , s = 1) ; otherwise, the model draws a step size k ∈ { 0 , . .

Original: , K } from another conditional multinomial distribution π N ( ·| h t , s = 0) to jump to the token x i t
Revised: , K } from another conditional multinomial distribution π N ( ·| h t , s = 0) to jump to the token x i t

Original: +

Revised: +

Original: =
Revised: =

Original: i t + k .
Revised: i t + k .

Original: Thus the probability of a sequence of actions that reads text X i 1 : i t = { x i 1 , x i 2 , ..., x i t } , stops at time t , and outputs a label ˆ y can be written as the joint distribution Π( X i 1 : i t , ˆ y ) =
Revised: Thus the probability of a sequence of actions that reads text X i 1 : i t = { x i 1 , x i 2 , ..., x i t } , stops at time t , and outputs a label ˆ y can be written as the joint distribution Π( X i 1 : i t , ˆ y ) =

Original: π S ( s = 1 | h t ) π C (ˆ y | h t , s = 1)
Revised: π S ( s = 1 | h t ) π C (ˆ y | h t , s = 1)

Original: [Equation]
Revised: [Equation]

Original: (cid:89)
Revised: (cid:89)

Original: [Equation]
Revised: [Equation]

Original: or simply as
Revised: or simply as

Original: [Equation]
Revised: [Equation]

Original: Hereby, k j = i j +1 − i j is the step size sampled at time j which ensures the model moves from token x i j to x i j +1 , and h j = RNN ( x i j , h j − 1 ) is computed by the RNN module.
Revised: Hereby, k j = i j +1 − i j is the step size sampled at time j which ensures the model moves from token x i j to x i j +1 , and h j = RNN ( x i j , h j − 1 ) is computed by the RNN module.

Original: To encourage fast and accurate text reading, we want to minimize the difference between true label and predicted while ensuring a low computational cost, which is measured by the length of the assessed text.
Revised: To encourage fast and accurate text reading, we want to minimize the difference between true label and predicted while ensuring a low computational cost, which is measured by the length of the assessed text.

Original: Hence, as the reward for the last output action, we use −L (ˆ y, y ) , where L is a loss function that measures the accuracy between predicted label ˆ y and true label y .
Revised: Hence, as the reward for the last output action, we use −L (ˆ y, y ) , where L is a loss function that measures the accuracy between predicted label ˆ y and true label y .

Original: For other actions we use a negative computational cost −F .
Revised: For other actions we use a negative computational cost −F .

Original: Hereby, F is the normalized FLOP count used at each time step which is approximately constant.
Revised: Hereby, F is the normalized FLOP count used at each time step which is approximately constant.

Original: Note that the FLOP count for the last step, F t , differs, since it also includes the cost of the classiﬁcation.
Revised: Note that the FLOP count for the last step, F t , differs, since it also includes the cost of the classiﬁcation.

Original: Overall, the reward signal is deﬁned as:
Revised: Overall, the reward signal is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: j
Revised: j

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: the ﬁnal time
Revised: the ﬁnal time

Original: [Equation]
Revised: [Equation]

Original: otherwise , where α is a trade-off parameter between accuracy and efﬁciency.
Revised: otherwise , where α is a trade-off parameter between accuracy and efﬁciency.

Original: Assume that the entire policy Π θ is parameterized by θ = { θ π S , θ π C , θ N , θ RNN } , where θ RNN subsumes the parameters for the encoder.
Revised: Assume that the entire policy Π θ is parameterized by θ = { θ π S , θ π C , θ N , θ RNN } , where θ RNN subsumes the parameters for the encoder.

Original: Our ﬁnal goal is to ﬁnd the optimal θ which maximize the expected return deﬁned by:
Revised: Our ﬁnal goal is to ﬁnd the optimal θ which maximize the expected return deﬁned by:

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst summation is used for integrating all possible sequences with different lengths to ensure the normalization of the distribution Π , and γ ∈ (0 , 1) is a discount factor.
Revised: where the ﬁrst summation is used for integrating all possible sequences with different lengths to ensure the normalization of the distribution Π , and γ ∈ (0 , 1) is a discount factor.

Original: It is not hard to see that J is infeasible to compute by enumerating all possibilities in the summation and expectation.
Revised: It is not hard to see that J is infeasible to compute by enumerating all possibilities in the summation and expectation.

Original: Fortunately, we can apply the policy gradient algorithm (Williams, 1992) to optimize this objective by estimating the gradient using Monte Carlo rollout samples, without doing expensive integration or enumeration.
Revised: Fortunately, we can apply the policy gradient algorithm (Williams, 1992) to optimize this objective by estimating the gradient using Monte Carlo rollout samples, without doing expensive integration or enumeration.

Original: The REINFORCE policy gradient of the objective on data ( x, y ) can be derived as follows:
Revised: The REINFORCE policy gradient of the objective on data ( x, y ) can be derived as follows:

Original: [Equation]
Revised: [Equation]

Original: Considering that the length of the rollout sequence can differ signiﬁcantly, the space for policy exploration is very large, thus making the variance of the gradient estimation very high.
Revised: Considering that the length of the rollout sequence can differ signiﬁcantly, the space for policy exploration is very large, thus making the variance of the gradient estimation very high.

Original: To remedy this, we also implement the advantage actor-critic algorithm (Konda & Tsitsiklis, 2000), which couples partial future return with each action and estimates a value function as the baseline for variance reduction.
Revised: To remedy this, we also implement the advantage actor-critic algorithm (Konda & Tsitsiklis, 2000), which couples partial future return with each action and estimates a value function as the baseline for variance reduction.

Original: We ﬁnd this procedure to provide better performance than the vanilla REINFORCE algorithm.
Revised: We ﬁnd this procedure to provide better performance than the vanilla REINFORCE algorithm.

Original: It is worth noting that this policy gradient method eventually is able to backpropagate both classiﬁcation accuracy and computational cost signals to every module in our model, including the stopping/skipping distributions, the label distribution and even the recurrent encoder, thus providing an end-to-end solution to text classiﬁcation problems.
Revised: It is worth noting that this policy gradient method eventually is able to backpropagate both classiﬁcation accuracy and computational cost signals to every module in our model, including the stopping/skipping distributions, the label distribution and even the recurrent encoder, thus providing an end-to-end solution to text classiﬁcation problems.

Original: Overall, our model aims to accelerate text classiﬁcation while still achieving a high accuracy.
Revised: Overall, our model aims to accelerate text classiﬁcation while still achieving a high accuracy.

Original: The hyperparameter α is used to control the trade-off between accuracy and time cost.
Revised: The hyperparameter α is used to control the trade-off between accuracy and time cost.

Original: If we set α to be a relatively large value, our model will be more boldly to skip tokens, stop reading and output a label.
Revised: If we set α to be a relatively large value, our model will be more boldly to skip tokens, stop reading and output a label.

Original: If α is small, our model would like to (re)read more tokens.
Revised: If α is small, our model would like to (re)read more tokens.

Original: Actually, the reward for penalizing the computational cost can be seen as a Lagrangian multiplier which is used to constrain the average cost of the computation.
Revised: Actually, the reward for penalizing the computational cost can be seen as a Lagrangian multiplier which is used to constrain the average cost of the computation.

Original: Therefore, there is a mapping between α and the amortized computational budget allocated for each sample.
Revised: Therefore, there is a mapping between α and the amortized computational budget allocated for each sample.

Original: Given a budget, we can tune α to provide a model with best classiﬁcation accuracy with the amortized cost within the budget.
Revised: Given a budget, we can tune α to provide a model with best classiﬁcation accuracy with the amortized cost within the budget.

Original: This is desirable for many cost-sensitive applications, such as those on mobile devices.
Revised: This is desirable for many cost-sensitive applications, such as those on mobile devices.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we illustrate our approach using two representative text classiﬁcation tasks: sentiment analysis and topic classiﬁcation.
Revised: In this section, we illustrate our approach using two representative text classiﬁcation tasks: sentiment analysis and topic classiﬁcation.

Original: To perform a solid demonstration on re-reading and skimming, we conduct experiments on three different syntactic levels.
Revised: To perform a solid demonstration on re-reading and skimming, we conduct experiments on three different syntactic levels.

Original: We will ﬁrst introduce the results on the word level before discussing character and sentence level performance.
Revised: We will ﬁrst introduce the results on the word level before discussing character and sentence level performance.

Original: General Experimental Settings: In our experiments, we use the IMDB and Yelp dataset for sentiment analysis, and the AG news and DBpedia for topic classiﬁcation.
Revised: General Experimental Settings: In our experiments, we use the IMDB and Yelp dataset for sentiment analysis, and the AG news and DBpedia for topic classiﬁcation.

Original: To evaluate each classiﬁer, we use predictive accuracy as the performance metric and average per-data ﬂoating point operations (FLOPs) as the computational cost metric.
Revised: To evaluate each classiﬁer, we use predictive accuracy as the performance metric and average per-data ﬂoating point operations (FLOPs) as the computational cost metric.

Original: We also take the FLOPs of the policy module into account, even though they are much lower than the classiﬁer.
Revised: We also take the FLOPs of the policy module into account, even though they are much lower than the classiﬁer.

Original: The energy cost for the policy module is about 1 to 2 million FLOPs per sentence, which is much smaller than the total FLOPs needed for the recurrent module and the classiﬁer.
Revised: The energy cost for the policy module is about 1 to 2 million FLOPs per sentence, which is much smaller than the total FLOPs needed for the recurrent module and the classiﬁer.

Original: Hyper-parameters: We use the Adam (Kingma & Ba, 2014) optimizer with a learning rate of 0.001 in all experiments.
Revised: Hyper-parameters: We use the Adam (Kingma & Ba, 2014) optimizer with a learning rate of 0.001 in all experiments.

Original: For the recurrent network structure, we use a convolution layer withkernels of size 5 and stack it as input to an LSTM with a hidden size of 128.
Revised: For the recurrent network structure, we use a convolution layer withkernels of size 5 and stack it as input to an LSTM with a hidden size of 128.

Original: For π S and π N policy network, we use three-hidden-layer MLPs with 128 hidden units per layer.
Revised: For π S and π N policy network, we use three-hidden-layer MLPs with 128 hidden units per layer.

Original: For π C and value network, we use single layer MLPs with 128 hidden units.
Revised: For π C and value network, we use single layer MLPs with 128 hidden units.

Original: K is set to 3.
Revised: K is set to 3.

Original: 3. R ESULTS
Revised: 3. R ESULTS

Original: We ﬁrst evaluate our method on the IMDB movie dataset (Maas et al., 2011).
Revised: We ﬁrst evaluate our method on the IMDB movie dataset (Maas et al., 2011).

Original: We randomly split it into 20,000 training, 5,000 validation and 25,000 test samples.
Revised: We randomly split it into 20,000 training, 5,000 validation and 25,000 test samples.

Original: The average length in the dataset is 240 words.
Revised: The average length in the dataset is 240 words.

Original: We adopt the same preprocessing method as Yu et al.
Revised: We adopt the same preprocessing method as Yu et al.

Original: (2017), either padding or truncating each sentence to 400 words.
Revised: (2017), either padding or truncating each sentence to 400 words.

Original: We use a chunk-size of 20 words, i.e., at each step, the classiﬁer reads 20 words.
Revised: We use a chunk-size of 20 words, i.e., at each step, the classiﬁer reads 20 words.

Original: When the action is rereading or skipping, it rereads or skips several chunks of 20 words.
Revised: When the action is rereading or skipping, it rereads or skips several chunks of 20 words.

Original: To demonstrate the effectiveness of re-reading and skimming, we design three baseline models: (1)
Revised: To demonstrate the effectiveness of re-reading and skimming, we design three baseline models: (1)

Original: The early stopping model , which has only a stopping module to decide when to terminate reading the paragraph, the classiﬁer and policy module are jointly trained on the entire training corpus; (2)
Revised: The early stopping model , which has only a stopping module to decide when to terminate reading the paragraph, the classiﬁer and policy module are jointly trained on the entire training corpus; (2)

Original: The partial reading model, which is a classiﬁer with same architecture trained on the truncated sentences decided by the stopping model (same as the one in the early stopping model).
Revised: The partial reading model, which is a classiﬁer with same architecture trained on the truncated sentences decided by the stopping model (same as the one in the early stopping model).

Original: Thus, although the partial reading model has the same computational budget as the early stopping model , the prediction performance may differ; (3) The whole reading model , which tries to use the whole corpus as training data.
Revised: Thus, although the partial reading model has the same computational budget as the early stopping model , the prediction performance may differ; (3) The whole reading model , which tries to use the whole corpus as training data.

Original: [Figure]
Revised: [Figure]

Original: Figure 2 shows our comparison on the IMDB dataset, where the blue line indicates our proposed model while green and red one denote early-stopping and partial reading , respectively.
Revised: Figure 2 shows our comparison on the IMDB dataset, where the blue line indicates our proposed model while green and red one denote early-stopping and partial reading , respectively.

Original: The x-axis denotes the FLOP count (in millions) and the y-axis indicates the accuracy.
Revised: The x-axis denotes the FLOP count (in millions) and the y-axis indicates the accuracy.

Original: Here the FLOP count is determined by the choice of the hyper-parameter α .
Revised: Here the FLOP count is determined by the choice of the hyper-parameter α .

Original: As α increases, we obtain a curve indicating the trade-off between accuracy and energy cost.
Revised: As α increases, we obtain a curve indicating the trade-off between accuracy and energy cost.

Original: From this plot, we observe that both blue line and green line outperform the red line signiﬁcantly.
Revised: From this plot, we observe that both blue line and green line outperform the red line signiﬁcantly.

Original: Furthermore, rereading and skipping further improve the performance of the model with only the early stopping mechanism.
Revised: Furthermore, rereading and skipping further improve the performance of the model with only the early stopping mechanism.

Original: This observation implies that training the classiﬁer jointly with the policy model improves both computational efﬁciency and accuracy.
Revised: This observation implies that training the classiﬁer jointly with the policy model improves both computational efﬁciency and accuracy.

Original: Besides the word-level evaluation, we also conduct experiments on a smaller scale syntactic unit: character-level.
Revised: Besides the word-level evaluation, we also conduct experiments on a smaller scale syntactic unit: character-level.

Original: In detail, we perform topic classiﬁcation on two large-scale text datasets: the AG news and the DBpedia dataset (Zhang et al., 2015) contains four topics, 120,000 training news, 10,000 validation news, 7,600 testing news.
Revised: In detail, we perform topic classiﬁcation on two large-scale text datasets: the AG news and the DBpedia dataset (Zhang et al., 2015) contains four topics, 120,000 training news, 10,000 validation news, 7,600 testing news.

Original: The DBpedia dataset contains 14 topics, 560,000 training entities, 10,000 validation entities and 70,000 testing entities.
Revised: The DBpedia dataset contains 14 topics, 560,000 training entities, 10,000 validation entities and 70,000 testing entities.

Original: The results are summarized in
Revised: The results are summarized in

Original: Figure 3.
Revised: Figure 3.

Original: We observe that our proposed model outperforms the partial reading baseline by a signiﬁcant margin.
Revised: We observe that our proposed model outperforms the partial reading baseline by a signiﬁcant margin.

Original: Furthermore, we evaluate our proposed model on a larger syntactic level: sentence level.
Revised: Furthermore, we evaluate our proposed model on a larger syntactic level: sentence level.

Original: We use
Revised: We use

Original: Yelp review sentiment analysis dataset for this experiment.
Revised: Yelp review sentiment analysis dataset for this experiment.

Original: The Yelp dataset includes 500,000 training reviews, 20,000 validation reviews and 40,000 testing reviews.
Revised: The Yelp dataset includes 500,000 training reviews, 20,000 validation reviews and 40,000 testing reviews.

Original: To evaluate on the larger semantic unit, we treat each sentence as a token , which will be read sequentially by the RNN encoder.
Revised: To evaluate on the larger semantic unit, we treat each sentence as a token , which will be read sequentially by the RNN encoder.

Original: The performance is provided in Figure 4.
Revised: The performance is provided in Figure 4.

Original: We observe that our proposed model achieves superior performance while being signiﬁcantly faster.
Revised: We observe that our proposed model achieves superior performance while being signiﬁcantly faster.

Original: We summarize the obtained performance improvements in Table 1.
Revised: We summarize the obtained performance improvements in Table 1.

Original: On four different datasets and for three different syntactic levels we observe signiﬁcant speedups when using the proposed techniques for skimming, rereading and early stopping, while maintaining the accuracy.
Revised: On four different datasets and for three different syntactic levels we observe signiﬁcant speedups when using the proposed techniques for skimming, rereading and early stopping, while maintaining the accuracy.

Original: A partial reading model which has the same computational cost achieves results that are less accurate, which illustrates the beneﬁts of a ﬂexible model.
Revised: A partial reading model which has the same computational cost achieves results that are less accurate, which illustrates the beneﬁts of a ﬂexible model.

Original: In addition, our model achieves about 0.5-1 percent accuracy improvement compared to the full-reading model.
Revised: In addition, our model achieves about 0.5-1 percent accuracy improvement compared to the full-reading model.

Original: Finally, we compare our model to a recently published baseline (Yu et al., 2017), which only implements the skipping actions with k ∈ { 1 , 2 , ..., K } but without  early stopping or rereading.
Revised: Finally, we compare our model to a recently published baseline (Yu et al., 2017), which only implements the skipping actions with k ∈ { 1 , 2 , ..., K } but without rereading, and simply do early stopping when k = 0 .

Original: We implemented their algorithm for a fair comparison.
Revised: We implemented their algorithm for a fair comparison.

Original: Results in Table 2 show that our model is much more efﬁcient than their LSTM-skip model at the same-level of accuracy.
Revised: Results in Tableshow that our model is much more efﬁcient than their LSTM-skip model at the same-level of accuracy, which is marginally better than full reading baseline.

Original: These results demonstrated that our proposed rereading and skimming mechanisms are effective on a variety of text classiﬁcation tasks including sentiment analysis and topic classiﬁcation.
Revised: These results demonstrated that our proposed rereading and skimming mechanisms are effective on a variety of text classiﬁcation tasks including sentiment analysis and topic classiﬁcation.

Original: Also it is effective on different level of semantics: character-level, word-level or even sentence-level.
Revised: Also it is effective on different level of seman- [Figure] [Figure] tics: character-level, word-level or even sentence-level.

Original: With the help of those two mechanisms, we could achieve both higher accuracy and faster speed.[Figure] [Figure]
Revised: With the help of our mechanism, we could achieve both higher accuracy and faster speed.

Original: 3. C ASE STUDIES
Revised: 3. C ASE STUDIES

Original: To obtain a more detailed understanding of our model [Figure] we ﬁrst show the actions taken by our model on a sentiment analysis example (Figure 5), on which the LSTM full-reading model failed to give the right classiﬁcation.
Revised: To obtain a more detailed understanding of our model, we ﬁrst show the actions taken by our model on a sentiment analysis example (Figure 5), on which the LSTM full-reading model failed to give the right classiﬁcation.

Original: We show the degree of positiveness given by LSTM model encoded in color, from green representing positiveness to brown representing negativeness.
Revised: We show the degree of positiveness given by LSTM model encoded in color, from green representing positiveness to brown representing negativeness.

Original: The paragraph starts with a sentence with strong positiveness of a dinner, then followed by a few sentence on some confusing description on the dinner.
Revised: The paragraph starts with a sentence with strong positiveness of a dinner, then followed by a few sentence on some confusing description on the dinner.

Original: Many trivial or even negative words show up in the explanation.
Revised: Many trivial or even negative words show up in the explanation.

Original: As a result, the output of the full-reading model gradually changes from positive to negative and ﬁnally results in a negative signal.
Revised: As a result, the output of the full-reading model gradually changes from positive to negative and ﬁnally results in a negative signal.

Original: Importantly, after our model reads the ﬁrst two sentences, the policy module decides that it is conﬁdent enough to make a decision yielding the correct answer.
Revised: Importantly, after our model reads the ﬁrst two sentences, the policy module decides that it is conﬁdent enough to make a decision yielding the correct answer.

Original: Next we illustrate how the rereading and skimming actions are useful to identify important information in the text.
Revised: Next we illustrate how the rereading and skimming actions are useful to identify important information in the text.

Original: As shown in Figure 6, our model ﬁrst reads a key word “stake” and is conﬁdent  that the document is about money.
Revised: As shown in Figure 6, our model ﬁrst reads a key word “stake” and is conﬁdent [Table] [Table] that the document is about money.

Original: Then it skims a few irrelevant tokens to read about “buying stake in Biotechnology” the following two tokens.
Revised: Then it skims a few irrelevant tokens to read about “buying stake in Biotechnology” the following two tokens.

Original: The phrase “5 percent stake” showed up twice.
Revised: The phrase “5 percent stake” showed up twice.

Original: Our model consider it to be important so it re-reads this token.
Revised: Our model consider it to be important so it re-reads this token.

Original: At this time, the model basically knows 
Revised: At this time, the model basically knows this text is about business with a reasonable conﬁdence.

Original: [Table] [Table]
Revised: Then it skips to read about “collaboration deal” and stops to make a conﬁdent prediction.

Original: 3. C OMPONENT A NALYSIS
Revised: 3. C OMPONENT A NALYSIS

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we conducted an ablation study to demonstrate the effectiveness of each action mechanism in our method: skimming, rereading and early-stopping.
Revised: we conducted an ablation study to demonstrate the effectiveness of each action mechanism in our method: skimming, rereading and early-stopping.

Original: Our experiment was performed on the word-level IMDB dataset, and the result is presented in Figure 8.
Revised: Our experiment was performed on the word-level IMDB dataset, and the result is presented in Figure 7.

Original: The blue curve denotes the performance of the model with all actions (skimming, rereading and early-stopping) enabled.
Revised: The blue curve denotes the performance of the model with all actions (skimming, rereading and early-stopping) enabled.

Original: The green one denotes the performance of the model with only the early-stopping actions.
Revised: The green one denotes the performance of the model with only the early-stopping actions.

Original: Between these two curves, the red curve represents a model with rereading and early-stopping action, and the yellow line represents a model with skimming and early-stopping actions.
Revised: Between these two curves, the red curve represents a model with rereading and early-stopping action, and the yellow line represents a model with skimming and early-stopping actions.

Original: Note that the performance of the green curve is the worst, indicating that rereading and skimming mechanisms are useful.
Revised: Note that the performance of the green curve is the worst, indicating that rereading and skimming mechanisms are useful.

Original: Furthermore, the blue curve is better than all other ones, indicating that combining skimming and rereading together can further improve the performance of the policy model.
Revised: Furthermore, the blue curve is better than all other ones, indicating that combining skimming and rereading together can further improve the performance of the policy model.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: The idea of improving time efﬁciency with adaptive computation has been studied extensively recently and throughout the years (e.g., Schwing et al., 2011; Weiss & Taskar, 2013).
Revised: The idea of improving time efﬁciency with adaptive computation has been studied extensively recently and throughout the years (e.g., Schwing et al., 2011; Weiss & Taskar, 2013).

Original: For example, the adaptive computation time algorithm (Graves, 2016) on recurrent neural networks is similar to our work.
Revised: For example, the adaptive computation time algorithm (Graves, 2016) on recurrent neural networks is similar to our work.

Original: However their model only implements the early stopping action, which can be easily trained by augmenting the loss with computational cost metrics.
Revised: However their model only implements the early stopping action, which can be easily trained by augmenting the loss with computational cost metrics.

Original: Spatially adaptive computation time (Figurnov et al., 2016) was proposed for image classiﬁcation and object detection tasks.
Revised: Spatially adaptive computation time (Figurnov et al., 2016) was proposed for image classiﬁcation and object detection tasks.

Original: Different from our approach, the actions deﬁned in their models are very simple.
Revised: Different from our approach, the actions deﬁned in their models are very simple.

Original: In contrast, in our model, the combinatorial complexity of actions make it more powerful but much harder to train.
Revised: In contrast, in our model, the combinatorial complexity of actions make it more powerful but much harder to train.

Original: We adopt the efﬁcient policy gradient methods and provide an efﬁcient algorithm for end-to-end training.
Revised: We adopt the efﬁcient policy gradient methods and provide an efﬁcient algorithm for end-to-end training.

Original: The idea of learning visual attention with neural networks (Mnih et al., 2014; Ba et al., 2014; Sermanet et al., 2014), which combine RNN or LSTM models with Reinforcement learning is closely related to our work.
Revised: The idea of learning visual attention with neural networks (Mnih et al., 2014; Ba et al., 2014; Sermanet et al., 2014), which combine RNN or LSTM models with Reinforcement learning is closely related to our work.

Original: However, their decision making process is continues and can be sampled from a continuous distribution, while our model’s actions are discrete, so it is more difﬁcult to learn the policy when the sequences are long.
Revised: However, their decision making process is continues and can be sampled from a continuous distribution, while our model’s actions are discrete, so it is more difﬁcult to learn the policy when the sequences are long.

Original: Attention mechanism applied to text are related.

Original: ReasonNet (Shen et al., 2017) trains a policy network with two actions, continue and stop, which is similar to our proposed model.

Original: However their work is mainly evaluated on the question answering task and they use early stopping to increase accuracy.

Original: Our model’s target is to minimize calculate cost while maintaining the same accuracy,

Original: They walk around with three What a great dinner!
Revised: They walk around with three What a great dinner!

Original: I had the pasta trio special.
Revised: I had the pasta trio special.

Original: and they did not bat an eye.
Revised: and they did not bat an eye.

Original: They even had the pasta prepared the same way to
Revised: They even had the pasta prepared the same way to

Original: and I kick myself for it on my way out of town every pastas and just re-fill your plate.
Revised: and I kick myself for it on my way out of town every pastas and just re-fill your plate.

Original: I even wanted one of them without an ingredient , refill me!
Revised: I even wanted one of them without an ingredient , refill me!

Original: I haven't been back, time I visit there!
Revised: I haven't been back, time I visit there!

Original: STOP
Revised: STOP

Original: Medarex Chief Executive Donald Drakeman.
Revised: Medarex Chief Executive Donald Drakeman.

Original: in Medarex BOSTON (CBS.MW) - Pharmaceutical Pfizer buys 5 percent stake researcher Medarex under their newly Pfizer is buying a 5 percent stake in biotechnology signed collaboration deal, according to powerhouse
Revised: in Medarex BOSTON (CBS.MW) - Pharmaceutical Pfizer buys 5 percent stake researcher Medarex under their newly Pfizer is buying a 5 percent stake in biotechnology signed collaboration deal, according to powerhouse

Original: Powerhouse Pfizer is buying a 5 percent stake in biotechnology
Revised: Powerhouse Pfizer is buying a 5 percent stake in biotechnology

Original: Pfizer buys 5 percent stake signed collaboration deal , according to 5 percent stake in biotechnology Reread
Revised: Pfizer buys 5 percent stake signed collaboration deal , according to 5 percent stake in biotechnology Reread

Original: Skip Skip
Revised: Skip Skip

Original: Stop
Revised: Stop

Original: Output
Revised: Output

Original: Scan so as to realize fast processing.
Revised: Scan Attention mechanism applied to text are related.

Revised: ReasonNet (Shen et al., 2017) trains a policy network with two actions, continue and stop, which is similar to our proposed model.

Revised: However their work is mainly evaluated on the question answering task and they use early stopping to increase accuracy.

Revised: Our model’s target is to minimize calculate cost while maintaining the same accuracy, so as to realize fast processing.

Original: Similarly, the model of Dulac-Arnold et al.
Revised: Similarly, the model of Dulac-Arnold et al.

Original: (2011) learns earlystopping for text classiﬁcation but they do not provide a skimming and rereading mechanism.
Revised: (2011) learns earlystopping for text classiﬁcation but they do not provide a skimming and rereading mechanism.

Original: Choi et al.
Revised: Choi et al.

Original: (2016) focus on an approach to improve efﬁciency of the question answering task.
Revised: (2016) focus on an approach to improve efﬁciency of the question answering task.

Original: They pursue a hierarchical approach for joint training.
Revised: They pursue a hierarchical approach for joint training.

Original: Rationalizing Neural Predictions (Lei et al., 2016) also uses a generator-encoder framework to select some fragments from a given document.
Revised: Rationalizing Neural Predictions (Lei et al., 2016) also uses a generator-encoder framework to select some fragments from a given document.

Original: But those fragments are used to facilitate text understanding rather than speeding up document processing.
Revised: But those fragments are used to facilitate text understanding rather than speeding up document processing.

Original: Hahn & Keller (2016) use a similar action space.
Revised: Hahn & Keller (2016) use a similar action space.

Original: In contrast to our work they use unsupervised training and mainly focus on action re-read, since their main target is to improve the performance of seq2seq tasks like language modeling.
Revised: In contrast to our work they use unsupervised training and mainly focus on action re-read, since their main target is to improve the performance of seq2seq tasks like language modeling.

Original: They skim words in order to not confuse the model rather than materializing computational beneﬁts.
Revised: They skim words in order to not confuse the model rather than materializing computational beneﬁts.

Original: Other ways to reduce the evaluation time for new examples have been considered.
Revised: Other ways to reduce the evaluation time for new examples have been considered.

Original: Bengio et al.
Revised: Bengio et al.

Original: (2015) propose a scheme to selectively activate parts of the network.
Revised: (2015) propose a scheme to selectively activate parts of the network.

Original: Bolukbasi et al.
Revised: Bolukbasi et al.

Original: (2017) present two schemes to adaptively utilize the network during evaluation:
Revised: (2017) present two schemes to adaptively utilize the network during evaluation:

Original: The ﬁrst is to choose components of a deep network, the second is a network selection system that adaptively selects the network to be evaluated for each example.
Revised: The ﬁrst is to choose components of a deep network, the second is a network selection system that adaptively selects the network to be evaluated for each example.

Original: But both of these mostly focus on model efﬁciency not on data efﬁciency.
Revised: But both of these mostly focus on model efﬁciency not on data efﬁciency.

Original: Closely related is work by Yu et al.
Revised: Closely related is work by Yu et al.

Original: The authors train their policy network end-to-end with reinforcement learning.
Revised: The authors train their policy network end-to-end with reinforcement learning.

Original: In contrast to their work which only implemented the skipping mechanism, our model has two other human-like mechanisms, rereading and early-stopping, thus leading to further improved efﬁciency and accuracy.
Revised: In contrast to their work which only implemented the skipping mechanism, our model has two other human-like mechanisms, rereading and early-stopping, thus leading to further improved efﬁciency and accuracy.

Original: Furthermore, we hardly rely on many hyper-parameters and only use a simple reward structure consisting of classiﬁcation accuracy and computational cost.
Revised: Furthermore, we hardly rely on many hyper-parameters and only use a simple reward structure consisting of classiﬁcation accuracy and computational cost.

Original: Finally, we illustrate the advantage of the actor-critic formulation (Konda & Tsitsiklis, 2000) to reduce the variance and improve performance.
Revised: Finally, we get an advanced performance with better reward design which incorporates the negative energy cost explicitly and implement a value network to reduce the variance.

Original: [Figure]
Revised: [Figure]

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We develop an end-to-end trainable approach for skimming, rereading and early stopping applicable to seq2scalar tasks.
Revised: We develop an end-to-end trainable approach for skimming, rereading and early stopping applicable to classiﬁcation tasks.

Original: By mimicking human fast reading, we introduce a policy module to decide what token to read next (e.g., rereading the current token, reading the next one, or skipping the next few tokens) or whether the model should stop reading and form a decision.
Revised: By mimicking human fast reading, we introduce a policy module to decide what token to read next (e.g., rereading the current token, reading the next one, or skipping the next few tokens) or whether the model should stop reading and form a decision.

Original: To encourage fast and accurate reading, we incorporate both classiﬁcation accuracy and the computational cost as a reward function to score classiﬁcation or other actions made by the agent during reading.
Revised: To encourage fast and accurate reading, we incorporate both classiﬁcation accuracy and the computational cost as a reward function to score classiﬁcation or other actions made by the agent during reading.

Original: An endto-end training algorithm based on the policy gradient method backpropagates the reward signal into both the policy module (also including the classiﬁcation policy) and the recurrent encoder.
Revised: An endto-end training algorithm based on the policy gradient method backpropagates the reward signal into both the policy module (also including the classiﬁcation policy) and the recurrent encoder.

Original: We demonstrate the efﬁcacy of the proposed approach on four different datasets and demonstrate improvements for both accuracy and computational performance.
Revised: We demonstrate the efﬁcacy of the proposed approach on four different datasets and demonstrate improvements for both accuracy and computational performance.

Original:  APPENDIX 6.A C OMPARISON OF DIFFERENT CHUNK SIZE
Revised:  APPENDIX 6.A C OMPARISON OF DIFFERENT CHUNK SIZE

Original: To illustrate that our model’s performance is robust to the choice of chunk size, we explore the model performance with a variety of chunk sizes onthe IMDB dataset.
Revised: To illustrate that our model’s performance is robust to the choice of chunk size, we explore the model performance with a variety of chunk sizes onthe IMDB dataset.

Original: The result is shown in Figure 8.
Revised: The result is shown in Figure 8.

Original: Here the red curve denotes the performance of the partial reading baseline, and the other three curves denote the performance of our full-action model with three chunk sizes 8, 20, 40, respectively.
Revised: Here the red curve denotes the performance of the partial reading baseline, and the other three curves denote the performance of our full-action model with three chunk sizes 8, 20, 40, respectively.

Original: It is clear that our model outperformed the baseline signiﬁcantly with different choices of chunck size.
Revised: It is clear that our model outperformed the baseline signiﬁcantly with different choices of chunck size.

Original: In addition, we found that if the chunk size is too small, there were more decision steps inside each sentence, making the policy optimization more difﬁcult.
Revised: In addition, we found that if the chunk size is too small, there were more decision steps inside each sentence, making the policy optimization more difﬁcult.

Original: For example, the performance of the chunk size 8 seems worse that two larger chunk sizes.
Revised: For example, the performance of the chunk size 8 seems worse that two larger chunk sizes.

Original: We believe this issue may be overcome by applying more advanced policy optimization algorithms such as proximal policy optimization (Schulman et al., 2017).
Revised: We believe this issue may be overcome by applying more advanced policy optimization algorithms such as proximal policy optimization (Schulman et al., 2017).

Original: On the other hand, if the chunk size is too large, there were fewer decision steps, making the model not ﬂexible enough.
Revised: On the other hand, if the chunk size is too large, there were fewer decision steps, making the model not ﬂexible enough.

Original: Among all three choices, we found that setting chunk size 20 leads to best practice in our experiments.
Revised: Among all three choices, we found that setting chunk size 20 leads to best practice in our experiments.

Original: [Figure]
Revised: [Figure]