# The Impact of Large Language Models on Programming Education and Student Learning Outcomes
## 2 Research Background  
Recent years have witnessed a surge in studies exploring the integration of LLMs in educational settings. These investigations have delved into the potential benefits and challenges of utilizing LLMs, such as ChatGPT, across various domains of learning. In this subsection, we provide an overview of existing research to contextualize the current study's focus on the impact of informal LLM usage on programming education and student learning outcomes. As this paper explores the use of modern LLMs accessible to the public without specialized hardware or ML knowledge, the research background specifically examines the use of modern LLMs, excluding any reference to self-hosted pre-ChatGPT models. In a 2023 study [4], the transformative potential of LLMs in higher education was examined, emphasizing opportunities and challenges. The authors concluded that while LLMs offer personalized learning and on-demand support through customized learning plans, reduced human interaction, bias, and ethical considerations were observed. To mitigate these challenges, universities should integrate such models as supplements rather than replacements for human interaction, establish ethical guidelines, involve students in model development, and provide faculty training and student support. Hence, universities must balance leveraging LLMs for enhanced education quality with addressing associated challenges to ensure a high standard of education delivery. With a focus on ChatGPT [5], 50 articles published in 2023 were analyzed, focusing on the original version based on GPT-3.5. While recognizing the potential of ChatGPT to enhance teaching and learning, the study highlighted shortcomings in knowledge andperformance across subject domains and identified potential issues such as generating incorrect or fake information and facilitating student plagiarism. To address these challenges, immediate actions were recommended, including refining assessment tasks to incorporate multimedia resources, updating institutional policies, providing instructor training on identifying ChatGPT use, and educating students about its limitations and the importance of academic integrity. It was noted that leveraging ChatGPT in teaching and learning could involve creating course materials and assisting with active learning approaches, albeit with a need for accuracy verification. However, challenges related to accuracy, bias, and student plagiarism must be addressed through proactive measures such as incorporating digital-free assessment components and establishing anti-plagiarism guidelines. Additionally, instructor training and student education on ChatGPT's limitations and academic integrity policies were deemed essential for effective integration into education. [PERSON] [6] acknowledged the increasing prevalence of AI within the educational domain as well. Despite debates and technological limitations, AI's presence in education is undeniable and promises substantial transformations in teaching and learning methodologies. Central to ongoing discussions is the concern over AI's potential misuse, especially in academic assignments, leading some to advocate for bans on AI tools like ChatGPT in educational settings. However, others argue for integrating AI technologies into educational structures, emphasizing the need to address student dependency and implement guidelines to mitigate risks. The advancement of AI technology, exemplified by the evolution of ChatGPT, poses challenges to safeguarding against potential misuse. Rethinking assessment strategies was deemed imperative once again. On the other hand, integrating AI applications into education does offer students valuable hands-on experience while preparing them for an AI-dominated future. Negotiating AI's swift transformations entails developing effective strategies and customized training modules for teachers and students to maximize the benefits of AI tools in education.