Notional Machines and The Karl Hassan Question
Published: 2020-05-04 18:04:53

---
title: Notional Machines and The Karl Hassan Question
published: true
description: A discussion of the paper Thinking out of the box: comparing metaphors for variables in programming education by Hermans et al.
---

In the article [_Thinking out of the box: comparing metaphors for variables in programming education_](https://dl.acm.org/doi/10.1145/3265757.3265765), by Hermans et al., they present a study that compares the effectiveness of metaphors when teaching novices about programming variables. For their study, they used two metaphors: variables as a box, and variables as labels that one can place on one value. So the _box_ metaphors leads to `x contains 5`, while the label metaphors leads to `x is 5`.

Once the subjects had their basic programming lessons, they were asked questions to test their understanding. Using Scratch as programming language, the researchers asked questions about the results of a program, and also asked people for explanation of their reasoning:

```
set points to 0
set points to 2
```

>Question: What is stored in points now?

In that case, some people arrived at the right answer, but their reasoning was wrong. While `2` is the right answer, justifying it by saying that 0 + 2 = 2 is not the right reasoning. This problem with mutation is discussed in Joe Armstrong's book Programming Erlang. There he explains how by having a single assignment of variables, Erlang is more familiar to the math one learns in school. 

A more interesting example happens when they asked people to reason about strings and variables:

```
set name to Karl
set name to Hassan
```

>Question: What is stored in name?

Some people answered `KarlHassan`, while others replied `KarlSan`. I want to focus on the latter, since the authors consider this reply to be a misconception.

Earlier in the paper the authors mention the idea of a "Notional Machine", which is explained as: 

>the general properties that a student assumes of the machine executing their code. It involves various aspects related to the program: compiler, memory management, etc. Having an incorrect understanding of the notional machine of a programming language is believed to be the cause of many misconceptions.

I think this idea of _notional machine_ is critical when it comes to understanding if _KarlSan_ qualifies as a misconception. To speak of a misconception, means that there is a correct answer according to some _notional machine_. Subjects received basic programming lessons, which are not included in the paper, so it's not clear in the paper if the researchers shared their own (correct?) notional machine with their subjects.

Code executes in a particular context, and there are contexts for which KarlSan is a correct answer. 

Let's talk about Java, and its [Non-Atomic Treatment of `double` and `long`](https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.7). As Goetz explains in Java Concurrency in Practice, the JVM could "treat a 64-bit read and write as two separate 32-bit operations." In a multithreaded environment, it's possible to read 32 bits of one value, and 32 bits of another! Transposing this to our example, there's a notional machine for which KarlSan is the right answer. We could easily create such a scenario with an hypothetical 8 bit computer, where unless the programming language updates strings of bytes atomically, there would be no guarantee that _KarlSan_ wouldn't happen.

Without a shared notional machine, it's hard to evaluate the subject's answers--KarlSan--as invalid or "misconceptions".

## Overwriting data metaphor

In section 5.3 of the papers the authors write:

>we observed that participants often tried to calculate with the given values in Q7, i.e. adding 0 and 2 together, rather than understanding that 2 _overwrites_ the existing value 0. [emphasis mine]

The word _overwrite_ is very interesting because it's a metaphor that semantically departs from the notion of _variables as labels_ or _variables as boxes_. Would it be better to explain variable assignment as "overwriting" sectors of memory?

## Metaphors and Notional Machines

A quick note on _notional machines_. In [So… what is a notional machine?](https://www.felienne.com/archives/6375) Hermans talks of these devices as being scaffolding that help us understand other ideas. How close is this to Lakoff's & Núñez idea of metaphorical scaffolding, as presented in their book [Where Mathematics Comes From](https://en.wikipedia.org/wiki/Where_Mathematics_Comes_From)? 

When explaining concepts like Sets, the line number (another metaphor), and so on, they say that we understand Sets (or any other mathematical concept), because we have built over the years a scaffolding of metaphors that finally let us support the idea of a Set. It'd be interesting to perform a similar exercise for programming, selecting one programming language and going over the many metaphors that help us understand what a function does, or what a loop does, and so on.

## while(feedback) learn

Another question for the study is to ask if a one shot exam is a good measurement for understanding which metaphor is better, the box one or the label one. Because learning is not a one shot opportunity. We learn through feedback, by applying what we have learnt, even if through a misconception, and then later receiving correcting nudges that put us in the right direction. In the long run, which metaphor is actually _better_? 

Another question is _embodiment_. Certain metaphors make sense due to our past experiences, our own body, our abilities, our culture, mother language, and so on. While it's clear that the study was targeted at either Dutch or English speakers, it’d be interesting to find out which metaphors work better in other cultures, like in Chinese or Ewe. In Fundamentals of Comparative And Intercultural Philosophy, Ma and Van Brakel explain how the concept of "to be" is divided among several verbs in the African language Ewe. How would this affect the idea of variable assignment, and which metaphors would be more apt in such cases?

Due to embodiment, objects present us with affordances--clues that tell us how an object can be used. In _Vision in Design_, Hekkert and van Dijk explain that a ceiling is a place to hang things from, like lightbulbs, but not a place to walk. Not for a fly though, for them a ceiling is a walking surface. “What is the purpose of a pen for someone who has lost his hands? Is it still a ‘tool for writing’?”. 

In the same vein we can ask which life experiences, abilities, cultural background, and so on, make a box or a label more _fit_ as metaphors.

## Conclusion

The paper by Hermans definitely opens some interesting questions, from metaphors to the _rightness_ of notional machines. Sharing notional machines seems to be a precondition for evaluating the effectiveness of metaphors, perhaps to the point where finding Lakoff's scaffolding for programming could be a crucial development to reason about how to best teach and learn programming. When discussing the _rightness_ of metaphors, it would be interesting to be one the lookout for metaphors that might pop up when explaining concepts, like the _overwrite_ metaphor presented above. Perhaps that was the _right_ metaphor all along?