## 1 Introduction  
Significant advances have been made in Generative Artificial Intelligence over the last couple of years. Large language models (LLMs) such as GPT2 [1], GPT3 [2], LLaMA [3] are capable of generating surprisingly high-quality texts [4]. These language models are used in a variety of useful applications, including chatbots (ChatGPT, Bard), translation [5], text summarization [6], public health interventions [7], and code generation [8], among others. Recent rapid advances also raise concerns about potential abuses of these models, especially in social media settings, such as the large-scale generation of disinformation [9], manipulation campaigns that use AI-generated content [10], and personalized scans [11]. As LLMs become ubiquitous, social media platforms will be inundated with LLM-generated content. Although some LLM-generated content would be benign and helpful, such as social media bots that post valuable information, some could be low quality, spam, or even harmful. Therefore, social media platforms will need to implement LLM detectors. These detectors could be used as a signal in their recommendation models to down-rank low-quality content and indicate AI-generated content via user-facing labels. We are already seeing that social media platforms are experimenting with labels for AI-generated content (see [12; 13]). For a reader and especially for a novice user who does not have prior experience in engaging with LLMs, AI-generated texts are almost indistinguishable from human-written texts [14]. Even if one can identify AI written text, it is often hard to identify which linguistic patterns are unique to LLMs. There are several approaches that can be used to detect LLM-generated texts [9; 14; 15; 16; 17]. While detecting LLM-generated texts is important, it is also important to understand what characteristics of LLM-generated texts make them different from human-written texts. This problem was approached from a stylometry perspective. Stylometry is the study of linguistic style. It has a long history of applications [18; 19], including the identification of anonymous document authors, the inferring of author demographics, plagiarism detection, and in a variety of digital forensic settings. One area of study in stylometry is authorship verification (AV). AV is the task of predicting whether two documents are written by the same author [20; 21]. In this paper, we use an existing AV model from [PERSON] et al. [22] and apply it to both human-written and LLM-generated texts to analyze the similarities and differences between them. This AV approach has been used successfully on fanfiction, multiauthor documents [23], and on social media settings including Reddit and Twitter to link cross-platform accounts and socksockpuppet accounts [24]. We used a fanfiction dataset [25], a subset of which was originally used to train the AV model, and excerpts from the fanfictions as prompts to different LLMs (GPT2, GPT3, ChatGPT, and LLaMA) to create \"LLM-generated fanfictions\". The AV model score was then used as a measure of the similarity in writing style between the human-written and LLM-generated fanfictions. A feature analysis was performed to understand the features driving the model scores. Specifically, in this paper, the following questions were answered:  
* Q1: What features distinguish the language style of AI and human texts? * Q2: What features are different across different AI models? * Q3: How do the features change over different AI model versions? We found that the language style of the fanfictions generated using GPT3 and ChatGPT is very different from the language style of the source fanfiction, but the language style across all the fanfictions generated by both GPT3 and ChatGPT were very similar. This indicates that GPT3 and ChatGPT, both belonging to the same GPT3.5 family, have a unique language style.