Confidence assessment marking

Resolved: Confidence assessment marking is a better test marking system than the one currently in place Introduction Often people discuss the merits of standardised tests, how to improve them etc. One thing most people forget to think of is of whether an ideal marking scheme is currently in place, as how something is marked effects how student understanding is evaluated. While it would be entertaining to compare 2 idealistic systems, this debate will focus on the preliminary step of establishing faults in the current system. A few points and rules before arguments: Resolution 1. ‘Better’ refers to being on the whole superior. 2. ‘Confidence assessment marking’ and ‘the one currently in place’ will refer to the systems Pro illustrates. 3. If there are issues, ask in the comments prior to the debate 4. No ridiculous semantics! Other points 1. Confidence assessment marking is only being assessed on it’s merits for the following type of questions: a) Multiple choice b) 1 word answer (say, a science definition) c) Questions like those in maths – ie. Sqrt. 196 where the student supplies their own answer This is because of problems in marking of things like essays where deduction isn’t very fair given the way essays are scored. Repeat: Only the above types of questions are what we are debating on. Clarify in the comments if needed. Background reading This is not essential reading, yet I recommend it. http://www.caacentre.ac.uk... (where I first got this idea/ explored other ideas, useful for understanding the proposed system in 2.3) http://www.debate.org... (previous debate of mine on a slightly different topic; I’m using similar arguments so this might be a useful read) The 2 Systems Current marking system To get an idea of the 2 systems, we’ll start with the standard system. Say there’s 100 1 mark questions for a score of 100. Let’s also say that the students gets 50 correct and 50 wrong. 50 correct questions = 50 marks 50 wrong answers = 0 Total = 50/100 Note that the current system rewards guessing, and does not effectively distinguish between differing levels of student belief in their answer. Confidence Assessment Testing occurs as usual in that students answer questions and such. Also similar is that questions are given a rank of some sort, or put simply, a number of marks which the question is worth (difference between a 1 and a 4 mark question indicates complexity for example) . For example, say a student says the answer to 2+2 is 4 on a test that uses the above system and the question is worth 1 mark. The student is awarded 1 mark. What differs in confidence assessment is that a student’s faith in their answer is also tested. This is done by another mark (a ‘confidence level’) attached to their question answer - 1,2 or 3. This confidence level is what the student is awarded for a correct answer (times whatever the question value is) , while 0, -2 and -6 are awarded (respectively) otherwise . Why confidence assessment is superior Now that everything is cleared up (hopefully) , I’ll begin to try and demonstrate why confidence assessment is superior. The main points to consider are the problems with rewarding guessing, the benefits of assessing student belief in their answer (as opposed to not) and how feedback to both students and teachers is affected by the marking schemes. Where necessary though, other points will be raised. Problems with rewarding guessing Put simply, the reason rewarding guessing is bad is mark inflation. Ie. If it’s multiple choice with A,B,C and D and you don’t know the answer to 4 questions, you’ll likely get 1 right through pure guesswork. Try this in a full blown test and to get 40% one need only know 20% and on average (using 4 choices) they will get to 40% through guesswork. Obviously, students in reality make informed guesses normally, but this demonstrates a point at worst. First point is that mark inflation skews feedback. While I cover this in greater depth later, put simply this means that rewarding guessing creates skewed results for teachers when they are trying to assess student ability, and also can make students overconfident in their own abilities. Secondly, a mindset of immediate guessing should not be promoted. That’s not to say that guessing should be discouraged. Say you had a 20% chance to win a million dollars for $10 . You’d do it. It’s also not to say that one shouldn’t do anything unless they are 100% sure. Rather, it is best to refrain from taking some actions when one’s chances can be improved. Let’s demonstrate this: David wants to buy shares. Due to various reasons he can only be 2/3 sure he is making a sound financial investment. Let’s also say that due to financial hardship David can only make one investment; ie. He can’t throw caution to the wind and invest with these odds in 3 stocks hoping to win on 2 and lose on 1 while still making money. Now, the question is should David invest? No. Why? Because in real life David could instead improve his chances to make a sound investment (say, research deeper, learn terms he doesn’t understand etc.) . It’s much better for testing to cultivate a mindset of learning more as opposed to instinctive guessing. Benefits with testing confidence Confidence is key in everyday life, affecting whether we do something, whether we get help, research etc. Therefore, it makes sense to diagnose a student’s confidence. First, a medical student. Let’s say the student gives a very confident yet incorrect diagnosis. This could have ramifications if done on a patient (try wrong drugs for treatment), so they are penalised heavily. This is opposed to the student being very unsure of their answer. If a doctor was unsure of their diagnosis, they won’t operate and will consult others for help or think things over again as opposed to blindly operating. Secondly, confidence assessment helps differentiate bright from average students. In the case of an easy test where many would do well, it is more likely that the brighter students would have a higher confidence level, resulting in higher scores. This would also apply in more difficult tests where worse students might tentatively put a ‘1’ on a guess of theirs, yet a good student would put a ‘3’. It seems only fair to award certainty in decision making. Thirdly, confidence assessment is very helpful in teaching. As A.R. Gardner Medwin from the Dept. Physiology, University College London states in http://www.ucl.ac.uk... : “In many situations, university teachers are faced with a wide range of student backgrounds and abilities. It is important to identify and assist students early on, where knowledge is weak or absent. Because of the diversity of student problems, this is an area where automated techniques can help a great deal. One of the merits of confidence assessment is that it is possible to give students relatively easy basic tests without seeming condescending to good students who get them mostly correct. These students get a kick out of getting the answers right at maximum confidence, while the weak students can answer questions initially with low confidence and then increase confidence as they repeat exercises in an area. We use the system in this way, for example, to help with the basic mathematical skills necessary for a BSc degree.” Feedback Testing is typically done to assess student understanding of various things taught to them. To succeed in this, teachers must be able to get accurate results. Mark inflation skews this and so makes the quality of feedback less. For students, they are given a false indication of their own abilities, letting them down when content is revisited (as is often done in high school for instance) when they think they know something, don’t practice and then do badly. More on this point in later rounds when I have more characters. Conclusion For the reasons above, confidence assessment marking is a superior choice.