Effects of knowledge representation format and hypermedia instruction on metacognitive accuracy

Abstract Assessing the students' metacognitive accuracy was not without some difficulties. Many of the students' responses included multiple ideas that seemed to target multiple levels of the taxonomy. This seemed to provide an incomplete picture of the students' cognitive activity when a single noun was chosen to represent these multiple ideas. Conversely, there were times that the responses were quite brief, and students applied multiple nouns as cognitive-level labels. For example, one student responded “What is the meaning of default as it is used in the computer context? The general understanding of default is ‘failure to do something’. How does this fit into what it means in computerese?” This student also labeled this response using three nouns that were at three cognitive levels — the levels of analysis, knowledge, and comprehension. The raters based their scores on the evidence provided in the response. However, with the student evidently feeling that multiple ideas were represented and failing to provide evidence of these ideas in the form of a more complete response, the discrepancy, again, seemed to result in an incomplete picture of the activity we were attempting to measure. Based on Bloom's explanation of the taxonomy, if a higher-order cognitive-level label was appropriate, it would imply the presence of the lower-order levels. We did not operate with this assumption. Again, the researchers scored the responses based on the evidence provided and their combined subjective experiences with these students. The researchers' experience of the readings became an asset by allowing specific insight into some of the responses. Again, there is the issue of importance of the type of structure chosen during the knowledge-format-selection exercise and the relationship to other variables (in this case, concept-map complexity). A relationship between the two variables at the same data collection points would be anticipated, but this was not evidenced. It took the predominant format selected to reveal any significant findings. This fact, especially when combined with the fact that 9 of the 12 students changed formats at least once, may indicate that the students were continually tuning and restructuring in cyclical-type phases as suggested by Rumelhart and Norman (1978). The fact that acquiring information and building new knowledge structures (i.e., learning) involves numerous changes is clearly evidenced by the lack of singularly unidirectional patterns in the data of this study. For example, there was no clear trend of progressing to any of the three knowledge formats; students seemed to be trying different formats based on whatever experiences they might have had, at that point in time, in order to make sense of the information. There was clearly not a uniform pattern of metacognitive progression for the students as a whole. Some increased, and some actually decreased. The rhythm and flow evident in cyclical-models of learning may be responsible for the mixed results which produced a lack of vertical progression to (a) higher levels of metacognitive accuracy and (b) less linear, structure-free models for representing structural relationships of information. It has been recognized that development of conceptual understanding more often follows a zigzag as opposed to a linear path (Prawat, 1993). The fluctuation evident in this study may be more indicative of cyclical phases of learning rather than clear hierarchical progressions from low to high levels of learning.