Can you analyze this?

ukka Kaisla
Doctor in psychological economics
This article explains how assessment of course reports 
operates when assignments are open-ended and when there 
are no simple “right or wrong” settings where the correct 
“outcome” is well known in advance.
When we say that someone succeeds well in a 
qualitative task, what do we mean by that – what does “well” 
actually imply and where do we take these concepts from to 
start with?
There is a common mistake that many people, 
including professional teachers, make when they try to 
understand how assessment of qualitative assignments 
operate, so some clarity is needed to benefit us all. 
Universal criteria of goodness
Let’s first focus on open-ended assignments of critical 
thinking. What do we look for when we assess outcomes? 
The first issue we need to understand is that by “we” we 
actually mean the entire humanity. There is no separation 
between a teacher assessing an assignment and any other 
competent human being. This is to say that the criteria for 
assessing qualitative assignments is universally shared 
among us all. What is considered “good” is therefore based 
on criteria that we all follow, whether we know it or not. 
The universal criteria of goodness are as natural for
thinking as air is natural for our breathing. We want to be 
able to rely on the reasoning of the writer. How does that 
come about? We follow consistency of the train of thought 
presented in the assignment. Is thinking coherent and based 
on credible evidence, are conclusions drawn logically from 
2
the premises and evidence, etc? In a nutshell the question is 
about how convincing the writer is in the eyes of the reader.
There is also another component which typically does 
not receive too much attention among professional teachers, 
but which is of utmost importance to humanity in general. 
The originality of thinking is something that many people 
have difficulties in assessing. But remember that critical 
thinking is essentially a creative process because everything 
must be constructed anew in one’s mind. Thinking critically 
involves a cognitive process by which we connect different 
things and the outcome is truly our own creation. 
Normal distribution is natural
There is some confusion about what normal distribution 
means in assessment. I have seen even strict prohibitions of 
the use of normal distribution at our school. In the following 
I will show that such prohibitions are based on 
misunderstanding and that what they really mean to 
prohibit is the distribution within sample which refers to a 
small group such as single class of students taking a course. 
Below you can see the form of normal distribution. It is 
a natural phenomenon in many things, such as the height of 
humans or their learning results. Because of its shape it is 
also called the “bell curve”. It simply shows how most of us 
and our products fall near the average and the further you 
move away from the average to 
either direction, the less people 
and their results can be found 
there. So, there are few whose 
learning outcomes are far less 
than desired, and such is also 
the case for those who excel. 
There is an understandable reason why this 
distribution is not to be used within a small group, such as a 
single course implementation. This is because an entire 
3
group of students can all excel or at least most of them can 
outperform the universal average earning high scores. It 
would then be unfair to force their grades to follow the 
normal distribution. Those who criticise the use of normal 
distribution mistake the distribution within a small sample 
for the universal normal distribution which is the only way 
we can assess any qualitative work, as I will show next.
To make things clear I better reiterate the above:
1. Distribution within a sample (e.g. single group of 
students) should not be aimed at as it potentially 
misrepresents learning results (an entire group can 
outperform the universal average).
2. Normal distribution in learning is a natural 
phenomenon and all our criteria of goodness are 
based on it, even if some people do not understand 
this.
Why is assessment of qualitative things like learning
necessarily based on normal distribution? Short answer is: 
because we are humans and human capabilities tend to 
follow normal distribution. This means that learning 
outcomes of most people are close to the average (3 in scale 
1-5) and that symmetrically there are fewer people at either 
end of the spectrum. 
If so, why do some people want to reject this very 
natural phenomenon? I have pondered this a lot in my career 
while observing professional teachers trying to falsify this 
phenomenon (but always failing, yet without understanding 
it by themselves). How does this happen?
In all cases I have observed, people who try to reinvent 
the wheel come with the following argument: “We should 
not use normal distribution, because it wrongly compares
learning outcomes of students. Learning outcomes should be 
assessed individually.” Since there is no way for us to inspect 
the mind of the student before they take a course, we cannot 
4
accurately assess the volume and quality of learning that has 
taken place in everyone’s mind during a course. 
Therefore, a handicap system needs to be set in the 
following way: instead of comparing everyone’s learning 
results to the universal average, they set up qualitative 
criteria using sentences like: “Good grade requires that a 
student knows certain things and is able to apply them well.” 
By introducing these qualitative sentences some people 
assume that they have now created an objective set of 
criteria in which comparison or normal distribution no 
longer exists. They are badly mistaken, as is shown next.
When we say that we know certain things, what do we 
mean? Knowingly or unknowingly, we refer to the level of 
knowledge. That level is necessarily based on normal 
distribution because all human knowing follows it. How 
about our competence in applying anything? Same thing, 
ability of application becomes meaningful only by 
understanding how well we apply whatever. Therefore, the 
last qualitative term in the sentence, “well” itself needs to be
qualitatively assessed against our peers, that is, against the 
normal distribution. Otherwise, the term “well” has no 
meaning whatever. 
There are always some who dislike or undervalue the 
qualitative nature of human life so much that they try 
everything to escape this very natural distribution of human 
endeavour. They want to see reality as certain and static, 
where things are either “right” or else they must be “wrong”. 
In the world of mathematics at introductory level things are 
like that. If you don’t get the correct answer, you must be 
wrong. Some become excited about this and believe that now 
we have escaped normal distribution. Not so.
Like all human capabilities, also mathematical 
capabilities follow normal distribution, and besides, when 
you start using more complex mathematics, the concepts of 
“right” and “wrong” are no longer tautologically applicable. 
5
So, to sum up what is explained here, those who try to 
reject normal distribution actively endorse it and utilise it 
every time they make assessments. This of course happens 
without them understanding it.
Genuine learning is personal
What I have said about assessing learning outcomes, deals 
with grading qualitative course work. But as I emphasise to 
all my students, learning is way more important than 
hunting for high scores. 
Earlier in this article I mentioned that an outsider 
cannot inspect your mind and get “before and after” images 
of your knowledge (even you are unable to produce such 
images). This is why we are bound to apply normal 
distribution, whether we like it or not. But when it comes to 
our personal learning, it is all subjective and none of it has 
anything to do with comparison with others. When 
assessment looks at our competences from the outside, 
learning is what happens within us. 
Every single human being creates their own unique 
path (of knowledge and understanding) by which they 
traverse through their lives. We are the ones both choosing 
and experiencing the way our journey affects us. So, even 
though this may appear paradoxical to some readers, what 
we need to concentrate on is expanding our own 
understanding of reality, which is something that grading 
can never catch in its entirety. 
Those who understand this correctly will grow and as
a by-product they will also enjoy high scores because these 
two correlate.