Kan AI blive gymnasielærernes nye kollega?
Creators: name: Kristiansen, Maria Rom
name: Jakobsen, Rasmus
name: Spies, Adam Faust
Publisher: name: Zenodo
Year: 2024
Subjects: AI
karaktergivning
støj
bias

description: This study seeks to discover the extent to which Danish high school teachers are subjective in their grading of papers and furthermore how artificial intelligence can be helpful as a worktool in the day-to-day grading throughout the school year. The study has been conducted using an experimental design that has been implemented through a survey. The survey has been con-structed through the program SurveyXact developed by Rambøll and had a total of 119 re-spondents who are all high school teachers in Danish on the A-level. Herein the respondents were set to grade a paper and give feedback. In addition to this a selection of the participants were given an AI support tool. The experiment was conducted through an intervention that con-tained feedback written on behalf of Google DeepMind’s generative artificial intelligence, Gem-ini. Within the survey the respondents were divided into three different groups wherein two of them took part in the actual experiment and received the intervention. In addition to the quantita-tive method the study also performs qualitative analysis on the individual feedback that the re-spondents perform. Thus, the study performs mixed-method research. To further analyze the outcome of the survey the study uses different theories from both within psychology as well as information technology. These theories are Kahneman et al.’s theory of the presence of noise in human judgement (2021) as well as the principles of the Technology Acceptance Model (TAM) as understood and perceived by Venkatesh and Davis (2000). The analysis was created around the following research question: To what degree will the use of artificial intelligence as a decision-making-tool increase the correctness of feedback and grading within the Danish high school and specifically in the subject Danish on the A-level?Furthermore, to specify the effects of the intervention the study created multiple hypotheses to understand and explain the outcome of the survey. These are as follows: H1a: High school teachers that use AI as a decision-making-tool will make fewer mistakes in their grading compared to the control-group.H1b: High school teachers that have participated as external graders on other assignments and use the AI decision-making-tool will collectively have a lower level of mistakes compared to those who have not.H2: The group ‘AI bias’ will be more likely to disagree with the feedback and grading gener-ated by artificial intelligence compared to the group ‘Social Opinion Influence’.Through analysis on behalf of the research question as well as the multiple hypothesis the study finds that the collective level of mistakes made by high school teachers are reduced when they use the AI decision-making-tool. Secondly, the teachers that have not acted as external graders become as good graders as those who have, when they use the AI tool. Thirdly, the study finds that a large group of respondents that can’t side with the AI generated feedback in reality actual-ly agree with the AI but renounce it purely due to the fact that it is artificial intelligence. This shows the importance of perceived usefulness within the Technology Acceptance Model and its crucial role to the final acceptance of a given technology. In summary the use of artificial intelli-gence will increase the correctness of feedback and grading within the Danish high school when looking at the subject Danish on the A-level.