## 4 Results
### Results and Interpretations
This may help explain the relatively higher variability and lower effectiveness seen with CU-tagged tasks, which may sometimes lack sufficient clarity or constraint. This interpretation aligns with educational research on prompt scaffolding. [PERSON] et al. [21] demonstrate that multi-step, conversational prompting improves contextual understanding and reduces cognitive load in LLM interactions, supporting logical reasoning and user engagement. These characteristics are particularly helpful in tasks like EF, where iterative refinement and stepwise logic are essential. Therefore, the superior performance observed in EF-tagged queries may be attributed to the benefits of such scaffolding structures, which help both the user and the model focus on discrete sub-tasks in complex problem-solving contexts. Finally, the study by [PERSON] and [PERSON] [23] provides additional insight into LLM behaviour. They note that models may drift from their initial instruction over longer outputs unless the prompt is explicitly anchored. This phenomenon of \"attention dilution\" supports the interpretation that well-anchored tasks, such as EF, are more robust to drift and yield better outcomes, whereas ambiguous prompts like those in CU queries are more vulnerable to deviation from intended goals. Together, these theoretical insights suggest that the observed EF/CU differences stem from a combination of prompt design (specificity and scaffolding) and intrinsic LLM behaviour. These findings carry practical implications: they highlight the importance of precise prompt engineering and suggest that educators and developers should prioritize anchored and scaffolded prompts for tasks requiring code comprehension or error resolution. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Avg Ranks**} & \multicolumn{3}{c}{**Mann-Whitney**} \\ \cline{2-6}
**Tag** & **No Tag** & **Tag** & **U** & **Z** & \(\mathbf{p}\) \\ \hline FI & 14.4 & 19.7 & 90.50 & \(-\)0.98 & 0.328 \\ BP & 18.3 & 24.5 & 56.50 & \(-\)0.40 & 0.688 \\ DOC & 18.7 & 30.0 & 9.00 & \(-\)0.24 & 0.813 \\ CC & 19.2 & 18.9 & 140.00 & \(-\)0.77 & 0.439 \\ EF & 28.5 & 17.2 & 135.50 & \(-\)2.03 & 0.042 \\ CU & 30.3 & 17.6 & 105.00 & \(-\)2.17 & 0.030 \\ CO & 22.5 & 14.4 & 189.50 & \(-\)1.13 & 0.260 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Average ranks and Mann-Whitney U test for each query type. For the other query types, no statistically significant differences were observed. While FI and CO showed trends toward better outcomes with tagging, their \(p\)-values were above the 0.05 threshold, indicating insufficient evidence to conclude a significant impact. The DOC category had a particularly small sample size, limiting the interpretability of its results. This outcome aligns with findings in recent prompt engineering research, which suggest that not all coding tasks benefit equally from LLM assistance. [PERSON] et al. [22] note that tasks requiring open-ended generation, such as feature implementation, often depend heavily on the user's ability to frame precise constraints and goals. Without detailed specification, LLMs may default to generic or suboptimal solutions, reducing the marginal benefit of tagging. Similarly, [PERSON] and [PERSON] [23] show that prompt anchoring is most effective when the task involves error resolution or explanation, while its utility diminishes in tasks that require creative or strategic planning, as is common in optimization and feature design. Finally, for DOC, the limited sample size constrains the statistical power of the test, but this task type may also inherently require less LLM assistance, as it often depends more on context familiarity than reasoning or generation complexity. To assess the impact of ChatGPT on developers' coding performance, we compared the difference between grades received on the two separate exams in the controlled and experimental phase: one completed without the use of ChatGPT (an easier exam) and one with ChatGPT's assistance (a more challenging exam).