Summary: The paper proposes a new dataset for reading comprehension, called DuoRC.  The questions and answers in the DuoRC dataset are created from different versions of a movie plot narrating the same underlying story.  The DuoRC dataset offers the following challenges compared to the existing reading comprehension (RC) datasets –  1) low lexical overlap between questions and their corresponding passages,  2) requires use of common-sense knowledge to answer the question,  3) requires reasoning across multiples sentences to answer the question,  4) consists of those questions as well that cannot be answered from the given passage.  The paper experiments with two types of models  – 1) a model which only predicts the span in a document and  2) a model which generates the answer after predicting the span.  Both these models are built off of an existing model on SQuAD – the Bidirectional Attention Flow (BiDAF) model.  The experimental results show that the span based model performs better than the model which generates the answers.  But the accuracy of both the models is significantly lower than that of their base model (BiDAF) on SQuAD, demonstrating the difficulty of the DuoRC dataset.  Strengths:1.	The data collection process is interesting.  The challenges in the proposed dataset as outlined in the paper seem worth pushing for. 2.	The paper is well written making it easy to follow. 3.	The experiments and analysis presented in the paper are insightful. Weaknesses:1.	It would be good if the paper can throw some more light on the comparison between the existing MovieQA dataset and the proposed DuoRC dataset, other than the size. 2.	The dataset is motivated as consisting of four challenges (described in the summary above) that do not exist in the existing RC datasets. However, the paper lacks an analysis on what percentage of questions in the proposed dataset belong to each category of the four challenges.  Such an analysis would helpful to accurately get an estimate of the proportion of these challenges in the dataset. 3.	It is not clear from the paper how should the questions which are unanswerable be evaluated.  As in, what should be the ground-truth answer against which the answers should such questions be evaluated.  Clearly, string matching would not work  because a model could say “don’t know” whereas some other model could say “unanswerable”.  So, does the training data have a particular string as the ground truth answer for such questions, so that a model can just be trained to spit out that particular string when it thinks it can’t answer the questions?   4.	One of the observations made in the paper is that “training on one dataset and evaluating on the other results in a drop in the performance.”  However, in table 4, evaluating on Paraphrase RC is better when trained on Self RC as opposed to when trained on Paraphrase RC.  This seems to be in conflict with the observation drawn in the paper.  Could authors please clarify this?  Also, could authors please throw some light on why this might be happening? 5.	In the third phase of data collection (Paraphrase RC), was waiting for 2-3 weeks the only step taken in order to ensure that the workers for this stage are different from those in stage 2, or was something more sophisticated implemented which did not allow a worker who has worked in stage 2 to be able to participate in stage 3? 6.	Typo: Dataset section, phrases --> phases Overall: The challenges proposed in the DuoRC dataset are interesting.  The paper is well written  and the experiments are interesting.  However, there are some questions (as mentioned in the Weaknesses section) which need to be clarified before I can recommend acceptance for the paper.