Confidence assessment marking

Firstly, I thank Larz for taking this debate. His debate record is quite strong, and I always prefer to lose to a strong opponent than win easily. With that said, I don’t intend to lose! Con Case Con raises a few main arguments –my system increases mark inflation (as opposed to decreases; he agress mark inflation is bad) and can lead to good students being grouped with bad students, no current problem to fix (doctor argument) , smarter students will answer everything 3, extra pressure on students and that it complicates the system. Mark Inflation I was quite interested when my opponent brought up this point as it had never come up in any articles I read. I kept researching and still couldn’t find a note on whether it went into negative scores, so I’ll take my own path and state there is a floor at 0. This means that one can’t get a mark below 0 (although a note would be taken on how negative the score is as teachers / computers have to mark till a test’s end). The other solution would be to change pass marks and such, but that’s a very messy process which I’m not advocating. So, returning to my opponent’s 4/20 example on confidence 1 (effectively a guess), his guessing doesn’t give him 20%, it gives him 6.66%. Essentially, the benefits of guessing are mostly eliminated, and mark inflation is lower under confidence assessment, making many of my points (such as feedback) true. On regrading, there would be no change as grading is done on percentages and percentages aren’t negative. On the good attempt people, they clearly aren’t so good and the system is based on risk and return. Also, there is strong evidence to show that students, as a whole, don’t rate their confidence high enough, but more on that later. Effectively, these people are rare and, due to confidence assessment actually better realise their lack of knowledge (again, later elaboration). System Complications Put simply, this is a near non-issue. There’s no complications with adding a multiplier in computer programs. My friend could easily write this program in a few minutes (he’s wrote a program to work through the preferential voting process) , no problem, meaning this is no complication. Furthermore, as testing is increasingly moving in the direction towards computers (particularly in the case of the questions we are debating), issues with teacher time (which are minor) are a temporary problem that is minor and is a reasonable concession for the other positives of the system. Extra Pressure? This is based on Con’s second last paragraph. A few points. 1, universities aren’t harmed by voluntary decisions, plus this is the physiology choosing this type of assessment! Second, while exams may be stressful, it’s necessary to have such pressure. If a person can’t pass an exam with a reasonable amount of preparation, then are they really the right person for the job when confronted with sudden problems (with no preparation)? Will they make errors under stressful situations in real life when mistakes matter? Furthermore, once could always allow 5 min extra time in a test for confidence. And here’s an entertaining statistic for my opponent to puzzle over – how come when students who had used confidence assessment marking were asked on keeping it in exams voted 52% in favour to 35% against (13% no reply or no preference)? http://www.ucl.ac.uk... If my marking scheme creates extra pressure, yet students support it, then pressure isn’t really much of an issue. No problem to fix? Con’s 4th paragraph (not including his 1 line intro) countered. Firstly, I never justified not taking risks. I clearly stated I still supported risk taking when necessary (see R1). If a patient’s going to die if you go consult somebody then you’ll take your chances trying to save them without help. Secondly, Con states risk taking is not a ‘problem’ as a reason not to change systems. That’s like me saying ‘the current speed of modern computers is not a problem, so we needn’t improve their speed’. The lack of a ‘problem’ is no reason not to support improvement. Statistical Reliability Con earlier raises a point about teachers having to do more marking and this taking more time and being bad. Fair point, but if the tests they were marking were more statistically reliable and so could be made significantly shorter would not this make up this deficit? Now, readers may be saying ‘but why is this true?’ Well, it’s due to the confidence factor. Now let’s demonstrate this: “Reliability analysis of exams (GardnerMedwin, 2006b) suggests in any case that the number of questions needed for equally reliable assessment data can fall by a third or more with CBM compared with conventional marking.” - http://www.reap.ac.uk... “An improvement in the ability to discriminate between students means that confidence-based data can give results of equivalent statistical reliability to conventional marking with fewer questions, by a factor of 2.0 for the lowest third of our groups and 2.2 for the top third” - http://www.ucl.ac.uk... “It corresponds to an increase in a test length of about 58%” (see the first source of this round) Effectively, confidence assessment marking can allow for a halving of marking or double the statistical reliability. More on Guessing There’s many things to cover here. Firstly, my opponent fears that smart students will just blindly guess ‘3’. I can say that intelligent people know when they are right and when they are wrong. Furthermore: “In the exams, 41% of answers were entered at C=3 with 95% correct, 19% at C=2 with 79% correct and 40% at C=1 with 62% correct. The percentages correct were within the optimal ranges but for C=2 and C=3 were near the top of these ranges, reflecting caution or under-confidence. Only 2 students (1F, 1M; both weak) were over-confident” - http://www.ucl.ac.uk... It should be noted that the ‘optimal ranges’ are over 80% for C=3, 67-80% for C=2 and 67<% for C=1. Students don’t think in probabilities like this but pick up the idea of the system very quickly and intuitively (in sources already mentioned). Furthermore, confidence assessment marking is helpful for learning as it promotes understanding as opposed to rote learning. As evidenced: “By failing to think critically and identify points of weakness, students lose the opportunity to embed their learning deeply and to find connections between different elements of their knowledge” - http://www.ucl.ac.uk... . Given that this is what confidence assessment does through students thinking about understanding, this is a positive. Educationally, the issue is to teach students to distinguish between confident and unconfident answers rather than handle marking schemes with optimally calibrated answers. Another point is the psychological jolt this system provides in destroying bad habits. Because no consideration is given in normal marking as to how to answer (you just answer) , students don’t pick up on bad habits when they answer. However, when one answers with a ‘3’ , indicating about an 80% confidence or higher (on average) and is wrong (-6), it indicates a serious misconception with their thinking, prompting a rethink in future (vs. just having another question wrong and doing nothing) , as shown in some student comments from an evaluation study (same source as the one directly above) : ‘(Issroff & Gardner-Medwin, 1998) 4 : "It .. stops you making rush answers.", "You can assess how well you really understand a topic.", "It makes one think .. it can be quite a shock to get a -6 .. you are forced to concentrate".’ Conclusion The above disproves Con’s contentions, expands upon the benefits of the proposed system and reinforces R1 contentions. Resolution affirmed. (apologies if this is disjointed, low char. left)