Analyzing Human Tutorial Dialogues for Cohesion and Coherence During Hypermedia Learning of a Complex Science Topic

Analyzing Human Tutorial Dialogues for Cohesion and Coherence During Hypermedia Learning of a Complex Science Topic Moongee Jeon (MJEON1@memphis.edu) Department of Psychology | Institute for Intelligent Systems | University of Memphis, 3693 Norriswood Avenue Memphis, TN 38152 USA Roger Azevedo (RAZEVEDO@memphis.edu) Department of Psychology | Institute for Intelligent Systems | University of Memphis, 3693 Norriswood Avenue Memphis, TN 38152 USA Abstract We analyzed cohesion and coherence of 79 think-aloud transcription logs from a human tutorial dialogue study investigating the effect of tutoring on college students’ learning about the circulatory system with hypermedia. The study involves randomly assigning 82 non-science majors to either the self-regulated learning (SRL, control condition) or the externally-regulated learning (ERL, experimental condition with a human tutor). The corpus we examined contained a total of 1,445 pages. We used Coh-Metrix, an automated web-based tool developed to evaluate text and discourse, in order to assess cohesion and coherence of the tutorial dialogues produced between a human tutor and low- domain knowledge college students during 79 tutoring sessions. Our findings showed that there were significant differences in the tutorial dialogues of the ERL tutoring condition versus those of the SRL control condition in the co- referential cohesion, the semantic/conceptual overlap, the causal ratio, the standard readability formulas, the incidence scores of connectives, the number of words, the number of sentences, the number of turns, the average sentences per turn, and the average words per sentence. Our findings have implications for the design of intelligent tutorial dialogue in hypermedia systems developed to improve learners’ deep conceptual understanding of complex and challenging science topics. Keywords: Cohesion; Coherence; Human Tutorial Dialogue; Learning; Hypermedia; Human Tutoring Cohesion and Coherence in Tutorial Dialogues Cohesion and coherence of text and discourse are critical components affecting text processing and comprehension. According to Graesser et al. (2004), coherence indicates characteristics of mental models that readers establish during comprehension, whereas cohesion refers to features of text-based information such as argument overlap, discourse markers, anaphora, and connectives. Coherence, according to Graesser et al. (2003), is the final psychological construct that readers build in their mind. Readers normally construct a coherent representation in terms of various cohesion and coherence relations during comprehension. For example, they attain coherence of a text by identifying various coherence relations (Sanders, Spooren, & Noordman, 1992) or rhetorical structures (Mann & Thompson, 1988) in the text, linking text-based segments, and combining the text-based information with their prior background knowledge (Kintsch, 1988, 1998; McNamara et al., 1996). Similarly, students can construct coherent mental models for subject matters while learning with the help of a human tutor in a classroom or while interacting with a computer-based learning environment such as a hypermedia environment (e.g., Azevedo et al., 2004, 2005, in press). Recent empirical studies have shown that cohesion and coherence are cardinal to examine whether readers generate inferences to link text units during on-line comprehension (Graesser, Singer, & Trabasso, 1994; Graesser & Bertus, 1998; Millis & Graesser, 1994), to combine the text-based information with the readers’ prior knowledge in their long- term memory (McNamara et al, 1996), and to construct mental models successfully in their mind (Gernsbacher, 1997; Graesser, Singer, & Trabasso, 1994; Kintsch, 1988, 1998; Zwaan, Langston, & Graesser, 1995). The mental models established by the readers promote deeper processing and understanding of the text. Thus, the coherent mental representation constructed based on the text-based information together with the readers’ prior knowledge is essential to understand the text successfully at a deep level. While these indices have been adopted mainly by cognitive psychologists in text comprehension studies, they are now being used by members of the Cognitive Science community to examine the quality and linguistic features found in tutoring dialogues. The aim is to understand the nature of the student-tutor dialogues in order to better design automated conversational dialogue in learning technology-based systems aimed at fostering conceptual understanding of complex and challenging science topics (e.g., see Azevedo, 2005; Graesser, McNamara, & VanLehn, 2005). With these theoretical frameworks and recent advance in computational linguistics and natural language technologies, our research group has developed Coh-Metrix, an automated web-based tool to analyze cohesion and coherence of various types of text, discourse, and tutorial dialogue in terms of over 400 measures (Graesser, McNamara, Louwerse, & Cai, 2004). The Coh-Metrix tool has been widely used by our research group. For example, the Coh- Metrix tool was recently used by Jeon and Graesser (2006) to examine cohesion, coherence, and readability of the tutorial dialogues of high-knowledge students who had