# Synthetic Text as Data: On Usefulness and Limitations
## 5 Conclusions and Discussion  
This study set out to evaluate the utility of GPT-generated text as training data for supervised learning tasks. We approached this question from two complementary angles: first, whether synthetic text can effectively supplement real data in settings of class imbalance or data scarcity; and second, whether it can substitute for human-written text by providing comparable informational depth. Using MBTI personality classification as a benchmark task, we conducted a series of experiments under controlled low-resource conditions. Our findings indicate that GPT-generated text can serve as a practical augmentation resource--improving model performance when added to underrepresented classes. In particular, models trained on mixed datasets (real + synthetic) consistently outperformed those trained on real data alone in imbalanced settings. These results affirm the value of GPT outputs as a cost-effective means of extending datasets in situations where collecting real samples is difficult. However, our analysis also reveals a clear gap between synthetic and real data when it comes to substitutive value. In few-shot learning experiments, models trained solely on GPT-generated text underperformed those trained on equivalent real data, even when the amount of synthetic data was substantially increased. This suggests that surface-level fluency does not guarantee task-relevant informativeness. Further investigations showed that while GPT-generated text may have failed to capture the full richness of real samples, it retained partial signal strength--enough to distinguish between personality types that are more dissimilar, but insufficient for subtle, fine-grained classification tasks. These findings point to an important distinction in how synthetic data should be interpreted and deployed. Rather than viewing GPT-generated text as either inherently  
Figure 6: This chart depicts the AUC metrics for differentiating ENFJ from various MBTI categories, with steps of data augmentation displayed on the \(x\)-axis (+0, +40, +80, +120 samples), and the corresponding AUC performance on the \(y\)-axis. The outcomes when incorporating real-world data samples are traced by the blue line, while the red line illustrates the effects of including GPT-generated data into the analysis  
valid or entirely inadequate, we argue that its effectiveness is highly dependent on the granularity of the classification task. Coarse distinctions may be well supported by generative models, whereas nuanced tasks likely demand the semantic depth found only in real human-authored data. In conclusion, GPT-generated text shows promise as a conditional training resource--useful in scenarios with limited real data, particularly when the classification target is broad or well-separated. Future research should explore the generalizability of the findings to other NLP tasks, the linguistic characteristics of synthetic text, comparisons with newer language models, and strategies for mitigating potential biases and fairness risks. While MBTI classification serves as a convenient benchmark, additional validation is needed to determine whether the results generalize to other NLP tasks such as sentiment analysis or intent classification. Moreover, this study did not conduct an in-depth analysis of the linguistic properties of synthetic text, nor did it include comparisons between GPT-3.5 and newer models such as GPT-4o. The observed variation in performance across personality types and models also warrants systematic investigation. Finally, exploring approaches to mitigate the potential biases and fairness risks introduced by synthetic data (e.g., as discussed in [23]) remains an important direction for future work. Conceptualization, G.C. ; Methodology, S.C.; Formal analysis, S.C.; Data curation, J.S. ; Writing--original draft, S.C. and J.S. ; Writing--review & editing, G.C. ; Visualization, J.S. All authors have read and agreed to the published version of the manuscript.