# The Impact of Large Language Models on Programming Education and Student Learning Outcomes
## 4 Results
### Results and Interpretations  
From a statistical standpoint, the analysis revealed varying degrees of correlation between LLM usage for different purposes (generating code, seeking additional explanations, and debugging) and the final grades of the participants. The [PERSON]'s rho values indicated the strength and direction of these relationships, with negative values pointing towards an inverse relationship between LLM usage and final grades. The significance of these correlations was determined by the \(p\)-values, with values less than 0.05 considered statistically significant. To further substantiate the robustness and reliability of these correlations, bootstrap confidence intervals were calculated for each correlation coefficient. Bootstrap analysis provides a non-parametric way to estimate the sample distribution of statistics based on random sampling with replacement. This method is particularly important in studies like ours, where the sample size is relatively modest. It allows us to estimate how [PERSON]'s rho might vary due to sampling variability and, thus, provides a more comprehensive understanding of the reliability and stability of our results. The inclusion of bootstrap confidence intervals helps demonstrate the precision of the resulting estimates and highlights that the observed relationships are not artifacts of the random sample of students. Moreover, generating these intervals contributes significantly to validating the statistical inference made in our study, presenting a clear picture of how certain we are about the existence and magnitude of the described correlations. The results of [PERSON]'s correlation test and its bootstrap results are presented in Table 2.  
The results provided empirical support for H1, indicating a significant, though modest, inverse correlation between average LLM use and final grades ([PERSON]'s rho \(=-0.347\), \(p=0.026\)), suggesting that overall, higher engagement with LLMs may detract from the desired learning outcomes in programming education. This firm finding suggests that extensive engagement with LLMs might detract from the educational outcomes desired in programming education. The bootstrap confidence interval for this correlation (\(-0.626\), \(-0.044\)) excludes zero, cementing the reliability of this result. For H2a, the analysis revealed a significant inverse relationship between the use of LLMs for generating code and final grades ([PERSON]'s rho \(=-0.305\), \(p=0.045\)), supporting the concern that reliance on LLMs for code generation can undermine independent coding skills. The confidence interval (\(-0.595\), \(-0.058\)) again excludes zero, confirming the reliability of these implications. Conversely, [PERSON]'s exploration of LLMs for seeking additional explanations did not yield a statistically significant impact on grades ([PERSON]'s rho \(=-0.201\), \(p=0.135\)), implying this form of LLM use might not hinder, and could potentially aid, student performance. The broad confidence interval (\(-0.523\), \(0.220\)) encompasses zero, reflecting this non-significance and corroborating the nuanced role LLMs play in educational settings. H2c was strongly supported by a significant inverse correlation between debugging with LLMs and final grades ([PERSON]'s rho \(=-0.360\), \(p=0.021\)), underscoring the importance of fostering independent debugging skills in programming education. The entirely negative confidence interval (\(-0.628\), \(-0.011\)) robustly supports this outcome. These results indicate a significant impact of LLM usage on learning outcomes in the context of programming education. The significant inverse correlation associated with code generation and debugging suggests that reliance on LLMs for these critical thinking-intensive activities could be detrimental to students' ability to independently solve programming tasks.