Evaluating Large Language Models on Wikipedia-Style Survey Generation (2024)

Abstract: Educational materials such as survey articles in specialized fields like computer science traditionally require tremendous expert inputs and are therefore expensive to create and update. Recently, Large Language Models (LLMs) have achieved significant success across various general tasks. However, their effectiveness and limitations in the education domain are yet to be fully explored. In this work, we examine the proficiency of LLMs in generating succinct survey articles specific to the niche field of NLP in computer science, focusing on a curated list of 99 topics. Automated benchmarks reveal that GPT-4 surpasses its predecessors, inluding GPT-3.5, PaLM2, and LLaMa2 by margins ranging from 2% to 20% in comparison to the established ground truth. We compare both human and GPT-based evaluation scores and provide in-depth analysis. While our findings suggest that GPT-created surveys are more contemporary and accessible than human-authored ones, certain limitations were observed. Notably, GPT-4, despite often delivering outstanding content, occasionally exhibited lapses like missing details or factual errors. At last, we compared the rating behavior between humans and GPT-4 and found systematic bias in using GPT evaluation.

heading: 1 Introduction
text: Recently, large language models (LLMs) have attracted significant attention due to their strong performance on general natural language processing (NLP) tasks (Shaib et al., 2023; Feng et al., 2022). Especially, the GPT family (Brown et al., 2020) shows great ability in various applications. While it has been demonstrated that they perform well in many general tasks, their effectiveness in domainspecific tasks continues to be under scrutiny (Tian et al., 2023). Specifically, the text produced by LLMs can sometimes exhibit issues like creating
false information and hallucination (Zhao et al., 2023; Yang et al., 2024a).
In the context of scientific education, automatic survey generation aims to employ machine learning or NLP techniques to create a structured overview of a specific concept (Sun and Zhuge, 2022; Li et al., 2022; Yang et al., 2024b). Automating this process not only alleviates the manual effort but also ensures timely updates at a reduced cost. A common approach involves an initial information retrieval phase to select pertinent documents or sentences based on the query topic. This is followed by a summarization or simplification phase to produce the final survey (Jha et al., 2013; Li et al., 2022). While LLMs have the potential to be an alternative method for writing scientific surveys, their effectiveness and limitations are not yet thoroughly investigated.
Existing work focuses on applying LLMs to similar scenarios, including aiding scientific writing (Shen et al., 2023; Altmäe et al., 2023), questionanswering with scientific papers (Tahri et al., 2022), writing paper reviews (Liang et al., 2023), and answering quiz or exam questions (Song et al., 2023; Wang et al., 2023). This study pushes the boundary of this research area as the first to evaluate the capability of LLMs in generating education surveys within the scientific domain of NLP (Li et al., 2022). Our primary objective is to understand whether LLMs can be used to explain concepts in a more structured manner. To this end, we aim to answer the following research questions (RQs):
• RQ1: How proficient are LLMs in generating survey articles on NLP concepts?
• RQ2: Can LLMs emulate human judgment when provided with specific criteria?
• RQ3: Do LLMs introduce a noticeable bias in evaluating machine-generated texts compared to human-written texts? We empirically conduct experiments on LLaMa2 (Touvron et al., 2023), PaLM2 (Anil et al., 2023),
5405
GPT-3.5 (Brown et al., 2020) and GPT-4 (OpenAI, 2023) across four different settings. Furthermore, we engage human experts to provide a qualitative dimension, ensuring that our results not only reflect the technical performance but also incorporate subjective human perspectives. We release the LLMs-generated surveys of all these works1.

heading: 2 Method

heading: 2.1 Dataset
text: We adopt the Surfer100 dataset (Li et al., 2022), which contains 100 manually written Wikipediastyle articles on NLP concepts. Each survey is structured into five sections: Introduction, History, Key Ideas, Uses/Applications, and Variations, with each section containing between 50 and 150 words.

heading: 2.2 Experiment Setup
text: We compare three settings: zero-shot (ZS), oneshot (OS) and description prompt (DP). For zeroshot, we directly ask the model to generate the arti-
1https://github.com/astridesa/EDULLM/ tree/master
cle by providing the following prompt: Generate a survey about <Topic>. There should be five subsections: Introduction, History, Key Ideas, Variations and Applications. Each subsection should contain 50-150 words. For the one-shot setting, we add a ground-truth article of Word2Vec as the sample survey; for the description prompt setting, we add a detailed description to each section explaining what should be included. For example, SECTION 1: INTRODUCTION Describe what the topic is (a method, a model, a task, a dataset), which field/subfield it is part of, quick overview of applications and motivation behind the concept and related ideas). To further enrich the provided information, we also introduce a combination of one-shot and description prompt (OSP). The full prompt is shown in Figure 1. By employing a single ground truth for one-shot learning, we generate 99 surveys per setting. Moreover, we evaluate a special retrieval augmented generation (RAG) (Gao et al., 2023; Ram et al., 2023; Yang et al., 2023; Aksitov et al., 2023) setting (denoted as OS+IR) which enables GPT-4 link to Wikipedia pages and
access to web data. We elaborate more experimental details in Appendix B

heading: 2.3 Evaluation Metrics
text: Automatic Evaluation We evaluate the generated surveys using a range of automatic metrics including ROUGE, BERTScore (Zhang* et al., 2020), MoverScore (Zhao et al., 2019), UniEval (Zhong et al., 2022) and BARTScore (Yuan et al., 2021). Tab. 1 provides an overview of results for the following LLMs: LLaMa2 (13B, 70B), PaLM2 (textbison), GPT-3.5 (Turbo-0613) as well as GPT-4 (0613) across different prompt settings. We first notice that GPT-4 consistently outperforms other baselines, obtaining a significant improvement of around 2% to 20% when enhancing prompts. Specifically, GPT-4 OSP achieves the top spot under most situations. However, it is not to say that prompt enrichment always yields positive results. For instance, in the case of LLaMa2, oneshot and description prompts perform better than OSP. As for PaLM2, four types of prompts obtain similar results. When we add external knowledge (GPT-4 OS+IR), there is some improvement compared to GPT-4 OS. As our primary goal is to study the extent of knowledge LLMs possess in this task, we mainly focus on analyses in settings without external data. However, additional analysis about GPT-4 OS+IR setting can be found in Appendix B.
Human and GPTs Evaluation We employ two NLP experts, GPT-4 and G-Eval (Liu et al., 2023a) to evaluate surveys generated by the best GPT-4 OSP setting, focusing on 6 perspectives: Readability, Relevancy, Hallucination, Completeness, Factuality. Both GPT models and humans are required to score each aspect on a scale from 1 to 5, following the same guidelines. The detailed guidance can be found in Appendix A. It’s important to note that we implement a pre-selection stage in the choice of human experts (Appendix A). Tab. 2 shows that both human experts and GPTs agree that the generated surveys perform well across most aspects, though the completeness exhibits marginally lowest scores. According to IAA, we can observe that human experts demonstrate a high consistent quality of the generated surveys whlie GPT-4 and G-Eval have more randomness. To better understand the degree of agreement between human experts and GPT-4 on ratings, we also calculate Kendall’s τ and p-value as shown in Tab. 3. We can
observe that the Factuality possesses the highest degree of correlation. In contrast, Redundancy displays the lowest correlation while the other aspects exhibit relatively lower correlation levels. This difference is largely because Factuality is based on objective ground truth, while Redundancy is more dependent on subjective judgment. Notably, we can conclude that in most scenarios, GPT-4 showcases similar evaluative opinions as humans, despite showing a higher degree of variability across different independent sessions. Regarding RQ1 and RQ2, we find that 1) LLMs can produce highquality survey articles, and 2) with specific guidance, there’s a strong consistency between GPT outputs and human judgment.

heading: 3 Analysis
text: In this section, we provide an in-depth analysis of the LLMs’ internal knowledge on survey writing ability, and compare the evaluation scores of human and LLM assessments.
Error Types We have shown that both automated and manual evaluations demonstrated that LLMs excel in crafting survey articles on scientific concepts. We analyze the best setting, GPT-4 OSP, assessing errors identified by two experts, and summarize error types and distributions in Fig. 2. We classify these errors into four categories: Verbose, Wrong Fact, Missing Information, and No Error (indicating flawless content). It shows that most errors are missing information, followed by verbosity and factual inaccuracies. Furthermore, the History and Introduction sections of the generated articles contained the highest number of errors, while the Application section exhibited the best.
Novel Entity Mention To further investigate how interesting the generated content is, we look at the mentions of novel entities following (Lee et al., 2022). Specifically, we examine the survey content by comparing the entities it contains with those in the ground truth. We employ Stanza (Qi et al., 2020) to identify all entities in both the LLMgenerated text and the ground truth. Subsequently, we quantify the number of unique entities found in the LLM-generated content. For a fair comparison, we analyze the one-shot with prompt settings of LLaMa2-13b, PaLM2, and GPT-4, in addition to the ZS setting of GPT-3.5, as depicted in Fig. 3. Our findings reveal that PaLM2 exhibited the least variation in entity mentions, while LLaMa2-13b showcased the most. Despite GPT-4’s outstanding performance in both automated and human evalu-
ations, we didn’t discern a marked novelty in its entity mentions. We speculate that this might be an inherent compromise when generating high-fidelity content in relation to the ground truth. So far, regarding RQ1, although LLMs register commendable results based on predefined criteria, certain shortcomings are evident. Specifically, we observe some omitted details, particularly within the Introduction and History sections. While LLMs often introduce new entities, we don’t find a significant correlation between this tendency and their performance. More case studies are in Appendix C.
LLM and Human Preference Previous studies have indicated that LLM-based evaluation methods tend to favor content generated by LLMs (Liu et al.,
2023b). To test the veracity of this assertion within the context of survey generation tasks, we took the opportunity to investigate whether a similar observation holds in the context of survey generation tasks. Hence, we recruited two human experts in a blind side-by-side comparison of both the ground truth survey articles and articles generated using the best GPT-4 settings, and they assessed the content based on ‘Likeability’(Chiang and Lee, 2023). Subsequently, we categorized the survey articles into three groups: a) (human experts) Liked, b) (human experts) Disliked, and c) Equal (equally good). The experts reached a significant agreement, reflected in a Cohen’s Kappa score of 0.68 (Cohen, 1960). In instances of disagreement, we randomly selected a score to reach a final consensus. We then apply the GPT-4 evaluation scores on the first four criteria except for Factuality and Completeness because both are impossible to do a blind test. We show the average ratings on all 99 concepts in Fig. 4. One main observation is the bias of GPT-4 towards texts generated by itself and consistently conferring high ratings – an observation consistent with other studies (Liu et al., 2023b). When evaluating the ground truth, GPT-4 consistently assigns marginally lower ratings across all three categories. Intriguingly, GPT-4 shows a preference for the Disliked group over the Liked group when considering the ground truth, a tendency that diverges from human inclinations. This suggests that when assessing human-composed text, such as ground truth survey articles, GPT-4 might not yet be an impeccable substitute for human discernment. Thus, in response to RQ3, we found that GPT-4 exhibits a notable preference for machine-generated texts with specific biases. Furthermore, we contend that the complete replacement of human experts by GPT-4 is a challenging prospect. For instance, human expertise remains indispensable for manual content fact checking.

heading: 4 Discussion and Conclusion
text: In this work, we evaluate the ability of LLMs to write surveys on NLP concepts. We find that LLMs, particularly GPT-4, can author surveys following specific guidelines that rival the quality of human experts, even though there are shortcomings such as incomplete information. Our findings also indicate that GPT-4 may not be a perfect replacement for human judgment when evaluating human-composed texts, and certain biases exist when asking it to rate machine-generated texts. Nevertheless, the results imply that these advanced generative LLMs could play a transformative role in the realm of education. They hold the promise of effectively structuring domain-specific knowledge tailored to general learners. This adaptability could potentially lead to a more interactive and personalized learning experience, enabling students to engage in query-driven studies that cater directly to their unique curiosities and learning objectives.

heading: Acknowledgements
text: This work was partially supported by the Deutsche Forschungsgemeinschaft (DFG) through the priority program RATIO (SPP-1999), Grant No. 376059226. It was also supported by the Japan Society for the Promotion of Science (JSPS) KAKENHI, Grant Number 24K20832. We would like to thank these funding agencies for their generous support. Additionally, we would like to thank the anonymous reviewers for their helpful feedback.

heading: Limitations and Ethical Considerations
text: GPT-4 can generate contemporary, accessible content, but sometimes compromises depth and detail, leading to potential information gaps and occasional factual inaccuracies. This requires extra verification. Comparing ratings between human experts and GPT-4 revealed a systematic bias in GPT evaluations, which can skew outcomes and mislead quality perception.
The primary objective of this work is to explore the potential applicability of LLMs in the field of education, with a specific emphasis on enhancing the understanding of LLMs’ generative capabilities within computer science. Our focus is on assessing the efficacy and identifying the boundaries of the generated texts. It’s important to note that the generated texts are devoid of any harmful content, and all data used and produced in this study contains no private information.

heading: A Human Evaluation Guidance
text: The detailed human evaluation guidance is listed in the following:
1. Readability:
• 1 (bad): The text is highly difficult to read, full of grammatical errors, and lacks coherence and clarity.
• 5 (good): The text is easy to read, well-structured, and flows naturally.
2. Relevancy:
• 1 (bad): The generated text is completely irrelevant to the given context or prompt. • 5 (good): The generated text is highly relevant and directly addresses the given context or
prompt.
3. Redundancy:
• 1 (bad): The text is excessively repetitive, containing unnecessary repetitions of the same information. For example, each section should have 50-150 tokens. If it is too long, we should give a low rating.
• 5 (good): The text is concise and free from redundancy, providing only essential information.
4. Hallucination:
• 1 (bad): The generated text includes false or misleading information that does not align with the context or is factually incorrect.
• 5 (good): The generated text is free from hallucinations and provides accurate and contextually appropriate information.
5. Completeness/Accuracy:
• 1 (bad): The generated text is incomplete (missing key information), leaving out crucial details or providing inaccurate information.
• 5 (good): The generated text is comprehensive, accurate, and includes all relevant information.
6. Factuality:
• 1 (bad): The text contains a significant number of factual inaccuracies or false statements, especially in History and Main Idea. For example, Year or people are wrong.
• 5 (good): The text is factually accurate, supported by evidence, and free from misinformation.
Pre-selection We initially engaged four NLP specialists to assess the surveys produced by GPT on 20 handpicked topics, as listed in Tab. 4. The evaluation scores across four model configurations are showcased in Tab. 5. Noting the considerable standard deviations among the evaluations of the four judges, we subsequently opted for two judges with a higher alignment in their scores to assess the entirety of the concepts.

heading: B Comparisons with External Knowledge
text: We conduct further evaluations with the inclusion of links to Wikipedia articles (GPT-4 OS+Wiki) and information retrieval (GPT OS+IR). In the GPT-4 OS+Wiki set up, we apply Embedchain (Taranjeet Singh, 2023) which supports embedding open sources for LLM querying. We crawl Wikipedia articles to concepts in Surfer 100 datasets, yielding 87 effective links. We then prompt GPT-4 to generate survey articles for the respective 87 topics, providing corresponding Wikipedia links and a sample survey as references. As for the setting GPT-4 OS+IR, we ask GPT-4 to ’Search on the web for helpful information’
2ZPS means zero-shot with description prompt.
in the prompt and utilitze the web search APIs. Table 6 shows the comparison results between the GPT-4 with and without external knowledge. It’s clear to see that both Wikipedia links and the information retrieval component significantly improve the Rouge scores. Notably, searching for web sources efficiently improve both the MoverScore and UniEval. In summary, external knowledge aids GPT-4 in generating higher-quality survey articles than using internal knowledge only. This suggests that LLMs possess limited proficiency when functioning as an academic search engine.

heading: C More Case Study and Observations

heading: C.1 Understanding of “Survey”
text: When we give the prompt to GPT models by asking them to write a “survey”, they sometimes generate survey articles as desired, but they will write other types of content. For example, as indicated in Fig. 5, it appears that GPT would understand the term “Survey” as the questionnaire. Moreover, even if they are able to generate a survey article in the format, there is still the situation that the generated content is not a typical survey. As shown in Fig. 6, there are inconsequential sentences in an attempt to extend and explain the provided text. For example, it repeats saying this section and participants. But this is mostly observed in the GPT-3.5 zero-shot setting.
C.2 Incomplete Information
In the “History” section, GPT models occasionally produce incomplete evolutionary history, and thus, potentially result in misleading information. For instance, in Fig. 7, when discussing the Knowledge Graph topic, GPT-4 model simply asserts that the term was invented by Google, while the reality is that the concept of Knowledge Graph has a long history, and it is Google that popularized the term. Similarly, in the case of the topic on Decision Trees, although the GPT model yields accurate context, it ignores landmark events and consequently causes misunderstandings.

heading: C.3 Nebulous Sentence Structure
text: We observe that GPT models frequently construct sentences, especially within the “Application” Section, that employ a rather vague sentence structure, which lacks specificity and can be used in different NLP topics. As shown in Fig. 8, it is evident that GPT models tend to generate similar sentences, such as “The Topic has a wide spectrum of applications” and “The Topic plays a vital role in Natural Language Processing and Natural Language Understanding”; These statements hold significant meaning when “The Topic” is substituted with any NLP topics.

heading: Section: History

heading: 3. Are you aware of the person who originally proposed the concept of GAN? If yes, could you please provide the name? 4. When was the GAN model first introduced? a)

heading: 2005 b) 2010 c) 2014 d) 2018

heading: C.4 High-quality Survey
text: We also present a high-quality generated survey in Fig 9. It is designed to read and understand easily, providing readers with comprehensive and detailed information. The example survey on LSTM is wellstructured, with a summary provided in the first sentence and followed by the detailed explanation in each section. Specially, when discussing applications, it demonstrates a high level of domain specificity. Most importantly, the generated information is both accurate and concise.

heading: C.5 Going Beyond the Ground Truth
text: GPT-4 based methods maintain an overall high-quality response regarding all aspects. We show two examples by comparing the GPT-4 one-shot with prompt setting result with the ground truth in Fig. 10. In the first topic, multilingual BERT (mBERT), GPT successfully points out that the key idea behind mBERT is mapping words from distinct languages into a shared embedding space. However, the ground truth only mentions shared vocabulary, which is superficial. In the second example, Hidden Markov Models (HMMs), the GPT response is more precise and more complete than the ground truth. One can find that the content flow is present as algorithm category → features → applications → motivation (highlighted in bold and italicized words). In contrast, the ground truth texts spend a lot of words to which category HMM belongs, including many terminologies which is less informative.
Limitations of Ground Truth We refer to this previous work (Li et al., 2022) on how the ground truth was generated. In general, the human writer was asked to rely on web data when writing the survey article; while these data were collected in the year 2021, it may be hard to say if it is a fair ROUGE score comparison with GPT models in Tab. 1. While the ground truth may not be a perfect reference, in this work, we focus more on human evaluation and case studies.

heading: D Potential Risks
text: Sole dependence on LLMs for educational content can lead to a homogenization of information and lack the nuanced understanding that human experts bring. If not properly vetted, the occasional factual errors made by GPT-4 could lead to the propagation of misinformation, especially detrimental in an educational context. Over time, heavy reliance on automated systems might diminish the role of human experts in content creation, leading to a potential loss of rich, experience-based insights. The observed systematic bias in GPT evaluations can lead researchers to draw incorrect conclusions about the quality of content, potentially impacting future research and educational endeavors.

heading: E Experimental Details
text: In our work, we mainly utilize the paid GPT-4 model to generate Wiki-style survey articles and further explore its capabilities to score the generations. The total cost of these experiments is around 230 USD. During the automatic evaluation stage, we compute the ROUGE score and BERTScore using the officially provided APIs: rouge3 and bert_score4. For calculations involving MoverScore, UniEval, and BARTScore, we directly download their source codes. All experiments were performed using the highperformance machine with 4 A100 40GB NVIDIA cards. As the experiments do not involve fine-tuning, for each setting, we were able to finish in a few hours. As for the human evaluation stage, we calculate the Krippendorff’s and Kendall’s scores with the authorized APIs krippendorff5 and scipy6.
3https://pypi.org/project/rouge/ 4https://github.com/Tiiiger/bert_score 5https://pypi.org/project/krippendorff/ 6https://scipy.org/

heading: Ground Truth

heading: Ground Truth

heading: Future Work