Measuring Mathematical Formula Writing Competence: An Application of Graphical Protocol Analysis

Measuring Mathematic Formula Writing Competence: An Application of Graphical Protocol Analysis Peter C-H. Cheng (p.c.h.cheng@sussex.ac.uk) Hector Rojas-Anaya (h.rojas@sussex.ac.uk) Representation and Cognition Research Group Department of Informatics, University of Sussex Brighton, BN1 9QH, UK Abstract those studies participants memorised simple geometrical patterns, sequences of numbers or word phrases. The stim- uli were created with predetermined structure and the spe- cific stimuli learning procedures ensured the participants possessed chunks with those structures. A pause duration for a given graphical element is the time between the lifting of the pen at the end of the previous element and the placing of the pen to begin the given element. During the graphical production the pattern of pauses reflects the induced chunk structure. This temporal chunk signal is apparent in data for individual participants doing a single task/trial. Meaningful patterns can be found without the need to aggregate data over trials and participants. The strength and robustness of the temporal chunk signal suggests that it has the potential to be the basis for methods to probe the structure of chunks when they are not known a priori. Three different ranges of pause durations can be distin- guished: long pauses typically indicate the inter-chunk re- call and preparation to write a new chunk (L2); medium length pauses corresponding to the intra-chunk production of sub-chunks that are symbols within a chunk (L1); short pauses occur with strokes within a particular symbol (L0), such as the second line of a ‘=’ sign. In the previous study with the drawing of simple geometric objects it was found that L1≈410 ms and L2≈620 ms (Cheng, McFadzean & Copeland, 2001). For the writing of simple number se- quences L1≈280 ms and L2≈440 ms (Cheng & Rojas- Anaya, 2005). For familiar and unfamiliar word phrases L1≈270 ms and L2≈400 ms (Cheng & Rojas-Anaya, 2006). The similarity between the pairs of times for the written tasks is noteworthy. One possible explanation for the longer pauses with the drawing tasks is the mode of graphical pro- duction used, i.e. drawing versus writing. Another explana- tion is a task difference, as the drawing of geometric objects was cued by names of the object whereas the production of the written sequences was through direct recall. Hence, the greater duration, particularly for L1, may be due to the extra step of retrieving the geometric pattern into working mem- ory. The implication is that specific differences in the in- formation processing steps needed for particular forms of graphical production may results in diagnostically useful differences in the inter- and intra-chunk pause durations, which could be identified by GPA. The overall goal of the present study was to demon- strate and extend the utility of GPA. It had three related aims. First, it investigated the copying of meaningful mathematical formulas rather than the production of arbi- Graphical protocol analysis (GPA) is a novel method for studying chunk-based cognitive performance using semi- automated analysis of a temporal chunk signal in writing pro- tocols. This study applies GPA to the writing of mathemati- cal equations by participants with different levels of expertise. Multiple levels of competence can be distinguished on a sin- gle individual and single task basis. Keywords: chunks, graphical protocol analysis, novice- expertise difference, mathematical formulas. Introduction Chunking is a key theoretical concept in Cognitive Science. It is well understood that temporal patterns in behaviour may reveal the structure of chunks in memory (e.g., McLean & Gregg, 1967; Egan & Schwartz, 1979; Chase & Simon,). In particular, when a series of actions are executed the dura- tion of the pause before a given action is typically taken to be indicative of the amount of processing required to pro- duce the output, with longer pauses indicating boundaries between different chunks in memory. Such patterns of such pauses can be a rich and valuable source of evidence to ad- dress many issues in cognitive science. However, the use of this temporal signal is hampered by the laboriousness of extracting and analysing pause data and the theoretical un- certainties of interpreting such data with respect to complex task contexts. The study reported here is a further step in an ongoing research programme that is attempting to make the extrac- tion and interpretation of such temporal chunking signals more practical and reliable. The programme is developing Graphical Protocol Analysis (GPA) as a method to identify the structure of chunks in an individual’s memory by ana- lysing the processes of writing and drawing. The potential benefits of GPA include: the use of modern, economical, simple to use graphics tablet technology; raw data that is rich (hi-frequency), accurate and precise; automatic initial extraction, analysis and coding of digital behaviour proto- cols by computer (although current tools are research proto- types); the capture and analysis of continuous extended be- haviour sequences encompassing multiple chunks; the use of relatively naturalistic tasks even in an experimental con- text. Our previous work on GPA has demonstrated the exis- tence of a strong and robust temporal signal that reveals the structure of chunks in memory (Cheng, McFadzean & Copeland, 2001; Cheng & Rojas-Anaya, 2005, 2006). In