# Large Language Models in Computer Science Classrooms: Ethical Challenges and Strategic Solutions
## 5 Tools and Strategies for Ensuring Academic Integrity
### Plagiarism Detection Methods
#### 5.3.2 Automated Detection Software
However, as noted in [5], at present, no automated tool is good enough for plagiarism detection considering AI-generated code. Regarding AI-generated code detection, [PERSON] [5] noted that ChatGPT's ability to produce original-seeming text poses a significant challenge. Studies have shown that conventional plagiarism detection tools such as Turnitin often fail to identify ChatGPT-generated content due to low similarity scores with existing work. This ability to bypass detection raises concerns about academic integrity and the potential misuse of ChatGPT for cheating. [PERSON] et al. [34] tested ChatGPT-generated essays against various plagiarism detection tools. They found that these tools often fail to recognize AI-generated content due to its originality and its close resemblance to human writing. Interestingly, [PERSON] et al. also discovered that a method involving asking ChatGPT directly whether a given text was generated by itself or not yielded the most accurate results, correctly identifying 46 out of 50 articles as ChatGPT-generated, thus surpassing the performance of conventional plagiarism detection tools. [PERSON] et al. [50] focused on AI-generated code detection and explored the effectiveness of AI-generated content detectors within programming education, particularly concerning the use of LLMs such as ChatGPT, driven by concerns over potential academic misconduct involving these models. The study generated a dataset of 5069 samples by prompting ChatGPT to solve various coding problems, resulting in a mix of human-written solutions and AI-generated codes from sources such as Quescol, Kaggle, and LeetCode. For the study, 13 different code problem variants were created to test the detectors. The performance of five AIGC detectors--GPTZero, Sapling, GPT-2 Detector, DetectGPT, and Giant Language Model Test Room (GLTR)--was evaluated. The findings revealed that these detectors generally struggle to distinguish between human-written and AI-generated code, with accuracy hovering around 0.5. This suggests a significant challenge in identifying AI-generated content, attributed to their primary training on natural language data and potential issues such as over-fitting. Notably, the GLTR detector showed improvements in detection accuracy across different code variants, with some surpassing an accuracy threshold of 0.6, indicating better performance in specific scenarios. The Sapling detector also demonstrated higher effectiveness, especially in detecting specific variants, with increased true negative rates (TNR). These results underscore the need for further research and development to enhance the precision and reliability of AIGC detectors in educational settings. In our previous research [11], we assessed 9 free code-origin detection tools, as detailed in [60], through the evaluation of 20 Python programming examples--10 generated by ChatGPT and 10 authored by humans. The detection tools evaluated included SEO.ai, Copyleaks, GPTZero, GPT-2 Output Detector, Corrector, Content at Scale, Roberta OpenAI Detector--Huggingface, ChatGPT Detector--Huggingface, and Writer. In our study, we additionally explored the capability of ChatGPT-01 to determine the origin of code using a specific prompt. For each tool, we conducted tests with 10 pairs of Python codes, each pair addressing a distinct programming challenge with one solution generated by ChatGPT and the other crafted by a human. The performance of each tool on the 20 examples is documented in Table 7.  
In this study, inspired by [34], we tasked ChatGPT-01 with determining the recognition percentages for code generated by GenAI, even though the examples presented were mixed. Specifically, we requested that ChatGPT provide recognition percentages for GenAI-generated code. The response highlighted that there is no robust, industry-wide benchmark or proven method that consistently yields reliable \"recognition percentages\" for GenAI-generated code.