Evaluating the performance of ChatGPT in writing autopsy clinicopathological correlations
Authors: Paz Matt Andrew , Turner Kevin , Racila Emilian 
Published: November 2023

Abstract
ChatGPT is a chatbot launched by OpenAI in November 2022 that has generated excitement due to its utility in several technical fields including medicine. The application has gained media attention due to concerns of academic plagiarism, including its use by students for writing projects. In scientific writing, a preprint in bioarxiv reports human reviewers failing to identify a portion of ChatGPT generated abstracts. Several articles published in early 2023 list ChatGPT as a coauthor, generating debate. In medicine, Gilson et al found ChatGPT crossed the 60% threshold for the NBME Free Step-1 dataset. Patel and Lam wrote a commentary examining the value of ChatGPT in writing discharge summaries. A literature review in PubMed yielded articles further examining the utility of ChatGPT in scientific writing, however none looking at its use in pathology report writing was found. To examine this, we investigated the capabilities of ChatGPT in writing autopsy clinicopathologic reports. The February 13 version of ChatGPT was used to generate autopsy clinicopathologic correlations using prompts with the same patient demographic, same medical history of hypertension and coronary disease, and varying autopsy findings. These findings include cardiac infarct, saddle emboli, basilar artery thrombus, liver cyst, and leukoplakia. A prompt with no significant autopsy findings was also included. The prompts note a lack of imaging, labs, and toxicology. The generated responses were examined and compared. ChatGPT generated multi-sentence responses incorporating patient information, history, and major autopsy findings per prompt, ranging in length from 8 to 14 sentences. Each output varied in style, with some responses including sections for gross and microscopic findings, while others only consisting of clinicopathologic correlation. Each response was generated between 42 seconds to 1 minute. The application generated plausible correlations between the medical histories and autopsy findings. ChatGPT correlated the findings to the patient’s supplied history in prompts with a cardiovascular autopsy finding. For the liver cyst, ChatGPT linked cyst formation to portal hypertension. For leukoplakia, ChatGPT suggested the presence of an upper GI cancer with cardiovascular history also likely contributing to death. For the prompt without significant findings, ChatGPT noted that the autopsy was inconclusive. ChatGPT debuted with an increasing amount of media attention due to its capability of producing natural language responses and surprising knowledge within multiple technical fields. ChatGPT is currently being used in academic writing at multiple levels within multiple fields. In medicine, ChatGPT has done well in multiple USMLE question banks. This study found that ChatGPT generated coherent clinicopathologic correlations commensurate to the amount of information supplied in the initial prompt. While useful as an initial framework, human review is still necessary to refine the response to make a more coherent report and to review made-up findings and dubious correlations.