EP4MGT: Entailment Progressions for Machine Generated Text Detection
Authors: Rishabh Shastry, Patricia Chiril, Joshua Charney, David Uminsky


Reviewer paper summaries: The paper starts from the idea of incorporating entailment into the task of detecting machine-generated text. In other words, the premise is that determining inference relations between sentences in a text is a component of identifying authentic human
authorship.
Summary of strengths: - interesting and novel idea on the integration of entailment into MGT
Summary of weaknesses: - the idea is not supported by enough evidence, eg texts (even from a single author) might present poor coherence scores among sentences
- the paper presents some problems in defining the concepts and the motivation of the proposed approach. For example, the authors present the scenario where “an individual is tasked with evaluating the coherence of a 10-sentence argument.”. To me, this is a weak point. For instance, authentic human text may contain poor coherence among sequential sentences. So the method fails if starts from this assumption.
- Overall, I’m not convinced about the use of entailment and coherence as “features” to detect machine-generated text. I find this claim quite strong and not well-connected with a) the technological features of language models and b) the obtained results.
Comments, suggestions and typos: - Figure 1 is not immediately comprehensible due to some problems with ambiguous colors
- In the introduction it is not well explained the context of white and black-box MGT systems, their inner working and differences  
- It is not clear and well defined what the authors state with this claim: "a model’s integration of entailment into its outputs is a latent feature of its authorship that can be compared with humans and other models." Language models do not work specifically on entailment, while there is a correlation between word probabilities at inference time and logical entailment of the content expressed by the words
Overall assessment scores: 2.5