Paper:
Beyond understanding what is being discussed, human communication requires an awareness of what someone is feeling. One challenge for dialogue agents is recognizing feelings in the conversation partner and replying accordingly, a key communicative skill that is trivial for humans. Research in this area is made difficult by the paucity of suitable publicly available datasets both for emotion and dialogues. This work proposes a new task for empathetic dialogue generation and EMPATHETICDIALOGUES, a dataset of 25k conversations grounded in emotional situations to facilitate training and evaluating dialogue systems. Our experiments indicate that dialogue models that use our dataset are perceived to be more empathetic by human evaluators, while improving on other metrics as well (e.g. perceived relevance of responses, BLEU scores), compared to models merely trained on large-scale Internet conversation data. We also present empirical comparisons of several ways to improve the performance of a given model by leveraging existing models or datasets without requiring lengthy re-training of the full model.

Review:
Overall this paper contributes many interesting insights into the specific application of empathetic dialog into chatbot responses.  The paper in particular is contributing its collected set of 25k empathetic dialogs, short semi-staged conversations around a particular seeded emotion and the results of various ways of incorporating this training set into a generative chatbot.
While the results clearly do not solve the problem of automating emapthy, the paper does give insights into which methods perform better than others (Generation vs Retrieval) and explicitly adding emotion predictions vs using an ensemble of encoders.
There is a lot in this paper, and I think it could have been better organized.
I am more familiar with emotion related research and not language to language translation, so I would have appreciated a better explanation of the rationale for using BLEU scores.  I did some online research to understand these Bilingual Evaluation Understudy Scores and while it seems like they measure sentence similarity, it is unclear how they capture ”relevance” at least according to the brief tutorial that I read (https://machinelearningmastery.com/calculate-bleu-score-for-text-python/).  I did not see the paper describing the use of this score in the references but perhaps I missed it – could you please clarify why this is a good metric for relevance?  It seems that these scores are very sensitive to sentence variation.  I am not sure if you can measure empathy or appropriateness of a response using this metric.
For your data collection you have 810 participants and 24,850 conversations.  Are the 810 participants all speakers or speakers and listeners combined?  How many conversations did each speaker/listener pair perform 32?  (one for each emotion) or 64? (two for each emotion) Was the number variable?  If so what is the distribution of the contribution – e.g. did one worker generate 10,000 while several hundred workers did only three of four?  Was it about even?  Just for clarity – how did you enroll participants?  Was it through AMT?  What were the criteria for the workers?  E.g. Native English speaker, etc.
In your supplemental material, I found the interchanging of the words “context” and “emotion” confusing.  The word context is used frequently throughout your manuscript: “dialog context,” “situational context” - emotions are different from situations, the situational utterance is the first utterance describing the emotion if I read your manuscript correctly.  Table 6 should use “Label” or “Emotion” instead of the more ambiguous “Context.”  
My understanding is that speakers were asked to write about a time when they experienced a particular feeling and they were given a choice of three feelings that they could write about.  You then say that workers are forced to select from contexts they had not chosen before to ensure that all of the categories were used.  From this I am assuming that each speaker/listener worker pair had to write about all 32 emotions – is this correct?  Another interpretation of this is that you asked new workers to describe situations involving feelings that had not been chosen by other workers as data collection progressed to ensure that you had a balanced data set.  This would imply that some emotional situations were less preferred and potentially more difficult to write about.  It would be interesting if this data was presented.  It might imply that some emotion labels are not as strong if people were forced to write about them rather than being able to choose to write about them.  
Were these dialogs ever actually annotated?  You state in section 2, Related Work “we train models for emotion detection on conversation data that has been explicitly labeled by annotators” – please describe how this was done.  Did independent third party annotators review the dialogs for label correctness?  Was a single rater or a majority vote used to decide the final label.  For example, in Table 1, the label “Afraid” is given to a conversation that could also have reasonable been generated by the label “Anxious” a word explicitly used in the dialog.  I am guessing that the dialogs are just labeled according to the label / provocation word and that they were not annotated beyond that, but please make this clear.  
In the last paragraph you state “A few works focus..” and then list 5.  This should rather be “several other works have focused on “ …  
Conversely, you later state in section 3 “Speaker and Listener”, “We include a few example conversations from the training data in Table 1,” this should more explicitly be “two.”
Also in section 3 when you describe your cross validation process, you state “We split the conversations into approximately 80/10/10 partitions.  To prevent overlap of <<discussed topics>> we split the data so that all the sets of conversations with the same speaker providing the prompt would be in the same partition.  
In your supplemental material you state that workers were paired.  Each worker is asked to write a prompt, which also seems to be the first utterance in the dialog they will start.  You state each worker selects one emotion word from a list of three which is somehow generated (randomly?) form your list of 32 .  I am assuming each worker in the pair does this, then the pair has a two “conversations” one where the first worker is the speaker and another where the second worker is the speaker – is this correct?  It is not entirely clear from the description. Given that you have 810 workers and 24,850 conversations, I am assuming that each worker had more than one conversation.  My question is  - did they generate a new prompt / first utterance for each conversations.  I am assuming yes since you say there are 24,850 prompts/conversations.  For each user are all of the situation/prompts they generate  describing the same emotion context?  E.g. would one worker write ~30 conversations on the same emotion.  This seems unlikely, and it seems more likely that given the number of conversations ~30 per participant is similar to the number of emotion words that you asked each worker to cycle through nearly all of the emotions or that given they were able to select, they might describe the same emotion, e.g. “fear” several times.  If the same worker was allowed to select the same emotion context multiple times was it found that they re-used the same prompt several times?  I am assuming that this is the case and that this is what you mean when you say that you “prevent overlap of discussed topics” between sets when you exclude particular workers.  Is this correct?  Or did you actually look and code the discussed topics to ensure no overlap even across workers (e.g. several people might have expressed fear of heights or fear of the dark).
In section 4, Empathetic dialog generator, you state that the dialog model has access to the situation description given by the speaker (also later called the situational prompt) but not the emotion word prompt.  Calling these both prompts makes the statement about 24,850 prompts/conversations a bit ambiguous.  A better statement would be 24,850 conversations based on unique situational prompts/descriptions (if they are in fact unique situational prompts.  I am assuming they are not if you are worried about overlapping “discussed topics” which I am assuming are the situational prompts since the dialogs are very short and heavily keyed off these initial situational prompts)
In your evaluation of the models with Human ratings you describe two sets of tests.  In one test you say you collect 100 annotations per model.  More explicitly, did you select 100 situational prompts and then ask workers to rate the response of each model?  Was how many responses was each worker shown?  How many workers were used?  Are the highlighted numbers the only significant findings or just the max scores?  Annotations is probably not the correct word here.
Please also describe your process for assigning workers to the second human ratings task.     
Since the two novel aspects of your paper are the new dataset and the use of this dataset to create more empathetic chatbot responses ("I know the feeling") I have focused on these aspects of the paper in my review.
I found the inclusion of Table 7 underexplained in the text.  The emotion labels for all these datasets are not directly comparable so I would have liked to have seen more explanation around how these classifications were compared.  It would also be helpful to know how more similar emotions such as "afraid" and "anxious" were scored vs "happy" and "sad" confusions