SYNTHEMPATHY: A Scalable Empathy Corpus Generated Using LLMs Without Any Crowdsourcing
Authors: Run Chen, Jun Shin, Julia Hirschberg


Reviewer paper summaries: This work aims to generate empathic responses using large language models (LLM). The main claim of this paper is to introduce a pipeline that does not need any "crowdsourcing or web crawling" and is capable of generating large-scale corpus.  The method involves several key steps: 1) using LLMs to generate stories from the English Stress Annotated Dataset (SAD); 2) prompting LLMs for First-Person Explanation, where LLMs are asked to mimic several therapeutic styles; 3) generating empathic responses with LLMs. The authors also conduct experiments to show that the LLM fine-tuned with the proposed dataset is more empathic. 1. The paper is clearly written, and easy to follow. The suggested pipeline is generally reasonable. 
2. The first-person explanation approach is an interesting way to generate user utterances.
Summary of strengths: 1. The paper is clearly written, and easy to follow. The suggested pipeline is generally reasonable. 
2. The first-person explanation approach is an interesting way to generate user utterances.
Summary of weaknesses: 1. The use of four therapeutic styles is interesting. However, it is not clear why the prompts are reasonably corresponding to each of the therapeutic styles.
2. Further, this paper does not analyze whether using these therapeutic styles is useful. 
3. The evaluation of Table 2 is not clear. How can a "RoBERTa-based" assign three emotional-metric scores?
Comments, suggestions and typos: 1. Lines 42--54 discuss the potential issues of crowdsourcing, but LLMs have also been observed to have various biases. The implication of this argument is that LLMs are better than crowdsourcing. Personally, I am not disagreeing, but I feel there may need to be more justification. 
2. L124, Welivita et al. (2020)'s approach may not be considered LLM-based.
3. In Table 1, what exactly are "topics evenly distributed," and why does the proposed approach guarantee it?
4. L505, please elaborate more on "by inspection." Does this step also require human intervention? 
5. The step of "Remove Offensive" (depicted in Figure 1) is not explained in Section 3.5. The only mention in the paper I can find is "manual keyword search" in L194, which is not clear.
6. Figure 3 suggests a "25"-word limit for every generation. Is it applied to any story, regardless of the length?
Overall assessment scores: 3