# Conceptual Integration and Measurement of Epistemological and Ontological Beliefs in Educational Research

Authors: Gregory Schraw
Journal: ISRN Education
 (2013)

---

## Abstract

This paper examines the conceptualization and measurement of epistemological and ontological phenomena and makes recommendations for improving the conceptual framework and methodological assessment of these phenomena. I discuss the ways educational researchers have studied beliefs and how this research can be improved through a comprehensive conceptual framework and better measurement. This paper provides definitions of epistemological and ontological beliefs and world views, discusses six complementary strategies for assessing these beliefs, compares the strengths of these strategies, and provides examples of how they have been used in the research literature. This paper discusses challenges related to the development of a comprehensive theoretical framework for beliefs, as well as ways to improve measurement of these beliefs and summarizes six emergent themes.


---

## Body

## 1. Introduction

This paper reviews issues surrounding the conceptualization and measurement of epistemological and ontological phenomena (e.g., beliefs and world views) and makes a number of recommendations for future research regarding the need for a unified conceptual framework and better methodological assessment of these phenomena. Epistemology is the study of beliefs about the origin and acquisition of knowledge [1–5]. Ontology is the study of beliefs about the nature of reality and being [6–9]. Research in education and psychology over the past three decades has focused primarily on the structure and development of high school and college students’ epistemological beliefs [10–20]. However, a number of more recent studies have examined teacher’s epistemological beliefs [21–24], epistemological world views [25–29], as well as a broader epistemology of teacher education [30–37]. A handful of studies have also addressed the role of students’ and teachers’ ontological beliefs as well [8, 38–40].Research on personal epistemologies and ontologies has been difficult to conduct and interpret at times for conceptual and methodological reasons [36]. From a conceptual standpoint, there is disagreement about the underlying structure of epistemological beliefs [12, 29, 41–43]. In addition, and of special significance in this paper, there has not been a clear articulation of the relationship between personal epistemology and ontology [38, 39, 44]. From a methodological standpoint, findings have been inconsistent and measurement problems have been a persistent problem for researchers [43, 45]. Future research would benefit from a unified conceptual framework that links epistemology and ontology, as well as a systematical methodological measurement plan that facilitates comparisons of epistemological and ontological phenomena across studies [29, 46, 47].Researchers have used six different measurement strategies for assessing epistemological and ontological beliefs and world views, including multi-item questionnaires [19, 48–50], interviews [20, 25, 26, 51, 52], vignettes that characterize prototypical epistemological world views [29, 53], essays, journals, and storyboards [54–56], concept maps [28, 57], and multidimensional scaling of epistemological and ontological relativism [58]. A number of questions arise regarding the strengths and weaknesses of these measurements, as well as their interrelationships. Very few studies have addressed these questions; thus, it is unclear whether the measurement strategies used in contemporary epistemological and ontological research measure the same constructs and phenomena, and in the event they measure different constructs, the extent to which these constructs are related.This paper addresses three main goals. The first is to provide definitions for terms frequently encountered in epistemological and ontological research. The second goal is to review existing measurement strategies. I do so to better articulate what these different instruments and strategies purport to measure, how well they do so, and how they may be used together in future studies in a complementary fashion to better inform our understanding of personal epistemology and ontology. A third goal is to elaborate on challenges of conceptual integration for future research. The main conclusion I wish to draw is the need for a unified framework and program of research that uses a variety of methods to link empirical findings across epistemological and ontological phenomena.The remainder of the paper is divided into six sections. Section2 provides general working definitions for the most important terms used in this paper. Section 3 describes six measurement strategies used in previous research, including questionnaires, interviews, vignettes, essays, concept maps, and multidimensional scaling methods. Section 4 compares the constructs and measurement strategies used in previous research. Section 5 considers six challenges for conceptual integration in the study of epistemological and ontological beliefs. Section 6 makes several suggestions for improving measurement. Section 7 provides a summary of emergent themes.


## 2. Definitions

This section defines a variety of terms used in the literature, including epistemology, epistemological beliefs, epistemological world views, ontology, ontological beliefs, and ontological world views [1, 5, 59–61]. Table 1 provides each term and its definition.Table 1
Key terms, definitions, and sources.
Term
Definition
Recent sources
Epistemology
The study of knowledge and beliefs about knowledge
Audi, 2003 [60]; Hofer [41] Feldman, 2003 [1]; Rescher, 2003 [5]
Epistemological beliefs
Specific belief about some aspect of knowledge that is part of a broader epistemology (e.g., origin of knowledge)
Hofer [41] Schommer, 1990 [19]
Epistemological world views
A set of beliefs or theory about knowledge, acquisition of knowledge, and knowledge justification
Lincoln and Guba, 2000 [6]; Mertens, 2005 [4]; Schraw and Olafson, 2002 [29]
Ontology
The study of the nature of reality and being
Mertens, 2005 [4]; Packer and Goicoechea, 2000 [39]; Merricks, 2007 [7]
Ontological beliefs
A specific belief about some aspect of reality (e.g., realism)
Lincoln and Guba, 2000 [6]; Merricks, 2007 [7]; Shadish et al., 2002 [9]
Ontological world views
A set of beliefs or theory about reality or being (e.g., social constructivism)
Lincoln and Guba, 2000 [6]; Mertens, 2005 [4]; Ponterotto, 2005 [8]
Realism
The view that all individuals share a common knowledge base and experience a similar reality
Cunningham and Fitzgerald, 1996 [61]; Feldman, 2003 [1]; Rescher, 2003 [5]
Contextualism
The view that knowledge is constructed base and that individuals experience a negotiated, consensual reality within a specific context
Cunningham and Fitzgerald, 1996 [61]; Feldman, 2003 [1]; Rescher, 2003 [5]
Relativism
The view that individuals construct a unique knowledge base and experience a unique reality even within a shared context
Cunningham and Fitzgerald, 1996 [61]; Feldman, 2003 [1]; Rescher, 2003 [5]Epistemology is used in its broadest sense to refer to a theory of knowledge and rationality [1, 62, 63]. Hofer [41, page 4] defines epistemology as being “concerned with the origin, nature, limits, methods, and justification of human knowledge.” Philosophical accounts of epistemology traditionally distinguish between kinds of knowledge (e.g., propositional or nonevidential) and justification of knowledge (e.g., weak or strong arguments) claims [5]. I use the term in this paper to refer to the study of knowledge and beliefs about knowledge. I use this somewhat more restricted definition because most studies of student and teacher beliefs focus primarily on beliefs about knowledge rather than justification of knowledge claims.The termepistemological belief has been used widely for over a decade to refer to a specific belief about some aspect of knowledge that is part of a broader epistemology. This implies that individuals may have more than one epistemological belief that is part of a set of beliefs that constitute a personal epistemology. Schommer [19] proposed five independent beliefs pertaining to certain knowledge (i.e., absolute knowledge exists and will eventually be known), simple knowledge (i.e., knowledge consists of discrete facts), omniscient authority (i.e., authorities have access to otherwise inaccessible knowledge), quick learning (i.e., learning occurs in a quick or not-at-all fashion), and innate ability (i.e., the ability to acquire knowledge is endowed at birth). Currently, there is debate as to whether Schommer’s five beliefs constitute genuine epistemological dimensions [12, 43]. Most researchers agree that beliefs about the certainty and simplicity of knowledge constitute genuine epistemological beliefs. Hofer [41] also has suggested the origin of knowledge as a genuine epistemological belief.I use the termepistemological world view to refer to an individual’s collective beliefs about the nature and acquisition of knowledge. I use term synonymously with similar terms used in the literature such as personal epistemology [22, 49] and epistemological stances [25] that collectively refer to a set of beliefs or a personal theory about knowledge and knowledge justification. I assume that an epistemological world view includes all of one’s explicit and implicit beliefs, attitudes, and assumptions about the acquisition, structure, representation, and application of knowledge [12, 29, 59, 64].It is important to distinguish clearly between epistemological beliefs and epistemological world views. The former consist of specific beliefs about a particular dimension of knowledge such as its certainty, simplicity, origin, or justification. The latter consist of aset of beliefs that collectively define one’s attitudes about the nature and acquisition of knowledge. Each adult presumably has a set of epistemological beliefs that are included within an epistemological world view, which also may include other beliefs such as how epistemological beliefs are acquired and develop, and how these beliefs change over time [65–67]. Both epistemological beliefs and world views may be tacit or explicit [1]. Presently, it is unclear whether tacit beliefs affect cognition and decision making more or less than explicit beliefs, or whether they are easier or more difficult to change [24, 68].Ontology is used in its broadest sense to refer to the nature of reality and being [4, 6–8]. Typical ontological questions include what is reality and what can be known about reality? Ontology usually is discussed independent of epistemology, but at some level, the two are related because beliefs about how we come to know reality necessarily involve epistemological assumptions. I make the assumption that epistemological and ontological beliefs are related to some extent in most teachers’ minds, although this issue has not been resolved empirically [44].The termontological belief has not been used widely in educational and psychological research, at least compared to the study of epistemological beliefs. However, just as researchers have argued for the existence of separate epistemological beliefs such as the origin or certainty of knowledge, it seems plausible that individuals would hold multiple ontological beliefs about the origin, permanence, and changeability of reality and being [69].I use the termontological world view to refer to an individual’s collective beliefs about the nature of reality and being. I assume that an individual’s ontological beliefs collectively comprise a personal ontology. Like epistemology, ontological beliefs and world views may be tacit or explicit in part or whole. I also assume the epistemological and ontological world views work in tandem to determine an individual’s beliefs about learning and instruction. Indeed, many graduate textbooks in education research specifically discuss and compare methodological approaches that presume competing epistemological assumptions. For example, modes of research such as positivism, postpositivism, structuralism, and postmodernism assume mutually exclusive, incommensurate assumptions about knowledge and reality [4, 6, 8, 9, 61]. Philosophers of science also have proposed theories of scientific change that draw on different epistemological and ontological underpinnings [70–72].Philosophers and researchers often distinguish between three competing theories of justification referred to asrealist, contextualist, and relativist positions that are relevant to a better understanding of epistemological and ontological research [1, 5, 15, 16, 46, 61, 73]. A realist position assumes that there is a fixed, core body of knowledge that is best acquired through experts via transmission and reconstruction, which are in a more or less unchanging reality. For example, teachers with a realist world view may tend to view students as passive recipients of a preestablished knowledge base. A contextualist position assumes that learners construct shared understandings in supportive contexts in which teachers serve as facilitators. Teachers with a contextualist world view are less concerned with the type of knowledge that students construct, than the process by which they construct that knowledge, and the degree to which that knowledge has authentic application to the context it is learned in [4, 29]. Contextualist teachers also assume that a student’s reality is idiosyncratic and changing, even though they would likely endorse the view that reality is a consensual social agreement in most contexts [8, 51, 74]. For this reason, a student may possess “multiple epistemologies” that change as the context of understanding changes [75, 76]. In contrast, a relativist position assumes that each learner constructs a unique knowledge base that is different but equal to other learners’ knowledge. Teachers with relativist world views deny the primacy of their own knowledge and emphasize their role in creating an environment where students can learn to think independently. They also assume that knowledge must differ among students to the extent that the reality in which students construct and apply knowledge is different among students. For present purposes, we use the terms realist, contextualist, and relativist as equivalent to the terms dualist, multiplist, and relativist which have been used widely in the literature [16, 18, 69, 77]. We use the former because these terms are used exclusively in extended philosophical treatments of epistemology [1, 5, 34, 59].


## 3. Strategies for Measuring Epistemological and Ontological Constructs

Researchers have used one of six different measurement strategies for assessing epistemological and ontological beliefs and world views, including questionnaires [19, 41, 48, 50], interviews [25], vignettes [29, 53], essays [56], concept maps [28], and multidimensional scaling methods [58]. The vast majority of studies have used questionnaires, interviews [78], with several studies using vignettes [29, 53], essays [54–56], and concept maps [28, 57]. Only one study has used multidimensional scaling [58].Many studies have used two or more of the six measurement strategies, typically with questionnaires or interviews as the primary data collection strategy. In addition, measurement strategies appeared to be chosen based on the research question and goals of the research. The majority of studies have focused on a set of separate epistemological beliefs and therefore have used multi-item questionnaires to assess these beliefs.


### 3.1. Measuring Epistemological Beliefs: Questionnaires

A number of researchers have measured epistemological beliefs using self-report questionnaires [16, 19, 48–50, 79, 80]. The majority of this research has focused on college students’ epistemological beliefs, although more recent research has examined either preservice teachers [81–90] or inservice teachers’ epistemological beliefs [22, 54, 91–93].Schommer [19] proposed five independent beliefs based on the work of Perry [18] pertaining to certain knowledge (i.e., absolute knowledge exists and will eventually be known), simple knowledge (i.e., knowledge consists of discrete facts), omniscient authority (i.e., authorities have access to otherwise inaccessible knowledge), quick learning (i.e., learning occurs in a quick or not-at-all fashion), and innate ability (i.e., the ability to acquire knowledge is endowed at birth). Currently, there is debate as to whether Schommer’s five beliefs constitute genuine epistemological dimensions [12]. Most researchers agree that beliefs about the certainty and simplicity of knowledge constitute genuine epistemological beliefs. In contrast, many researchers argue that beliefs about innate ability reflect some other, nonepistemological dimension [12].Schommer [19] developed the Epistemological Questionnaire (EQ) to assess the five dimensions described above. The EQ consisted of 62 simple statements that individuals responded to using a five-point Likert scale, indicating the extent to which they agreed or disagreed with the statement (see Appendix A, e.g.). The EQ has been used frequently over the past 15 years by Schommer and other researchers [22, 67, 94, 95]. Results have been somewhat mixed in a variety of ways. First, factor analyses typically reported less than five interpretable factors [49, 50]. Second, some analyses have yielded factors with an interpretation that was not predicted by Schommer [19]. For example, a factor might pertain to the source of knowledge or have item loadings that are difficult to interpret in a manner consistent with Schommer’s proposed five-factor structure. Third, the factors reported in the analyses often had a small number of items with acceptable factor loadings and therefore had low or unacceptable reliability coefficients. Fourth, solutions usually explained a relatively small proportion of sample variance (e.g., 20% to 35%), which raised concerns about the construct validity of the questionnaire.Schraw et al. [50] developed a modified version of Schommer’s EQ called the Epistemic Beliefs Inventory (EBI) to address these methodological problems. The EBI contained 32 items based on the five factors proposed by Schommer [19] (see Appendix A, e.g.). The EBI typically yielded the five proposed factors; however, reliabilities tended to be low (e.g., .50 to .65) and results varied depending on the age and gender of the sample. In addition, like the EQ, the EBI explained a relatively small proportion of sample variation (e.g., usually less than 40%).Hofer [49] proposed an alternative four-factor framework and developed an instrument called the Epistemological Beliefs Questionnaire (EBQ) to assess these factors. She proposed four factors, which were subsumed under two general dimensions referred to as the nature of knowing and the process of knowing. The former refers to what knowledge is presumed to be, while the latter refers to how one comes to know and understand knowledge. The “nature” dimension included two factors called certainty of knowledge (i.e., the degree to which one sees knowledge as fixed versus fluid and changeable) and simplicity of knowledge (i.e., the degree to which knowledge is viewed as individual facts versus complex, interrelated concepts). The “process” dimension included two factors called source of knowledge (i.e., the extent to which credible knowledge is self- or other generated) and justification of knowing (i.e., the rules and criteria that individuals use to evaluate knowledge claims).Hofer [49] reported four empirically derived factors that differed somewhat from the four proposed factors described above (see Appendix A, e.g.). Her four observed factors included certainty of knowledge, source of knowledge, justification, and attainment of truth. A “simplicity of knowledge” factor was not observed, whereas an “attainment of truth” factor was observed. The attainment of truth factor was interpreted as the extent to which experts can attain deep knowledge (i.e., “truth”) within their area of expertise. Like the EQ and EBI, the EBQ explained approximately 45% of sample variance, had several factors with few items that loaded satisfactorily, and had factors with low reliability coefficients.Bromme and colleagues [48, 96] have taken a somewhat different approach to measuring epistemological beliefs. They created a 24-item semantic differential type instrument that measures connotative aspects of student’s epistemological beliefs using bipolar adjective pairs that were subsumed under three hypothesized epistemological factors, including certainty, simplicity, and source. For example, the certainty included the certain-uncertain pair; simplicity included the simple-complex pair; and the source factor included the constructed-preexisting pair. Two studies using university students enrolled in either plant biology or chemistry yielded two reliable factors referred to as texture and variability. Texture assessed the complexity and sophistication of beliefs, whereas variability assessed the changeability and permanence of beliefs.Thus far, self-report questionnaires that have been developed to assess multiple epistemological beliefs have contributed to the literature in two important ways. One contribution of this research is that there has been a great deal of productive discussion regarding the set of constructs that comprise the domain of epistemological beliefs [2, 12, 49, 97–99]. These discussions are crucial to mapping the possible domain of epistemological beliefs both conceptually and methodologically. A second contribution has been the preliminary findings concerning the relationships among epistemological beliefs and a variety of outcomes variables such as age, education level, gender, moral reasoning skills, and academic achievement [67, 100–102]. Epistemological beliefs that are more relativistic tend to be associated with better problem solving [20, 80, 86, 103], higher self-regulation [2, 99, 104–107], metacognition [91, 96, 108], constructivist teaching practices [29, 89, 109, 110], and academic career choices [111, 112]. However, despite positive correlations, the effect size tends to be small in most cases, usually in the .05 to .15 range in terms of variance explained in the outcome measure of interest.Questionnaires also have experienced a number of difficulties [84]. One is disagreement about the number and content of constructs being investigated. Of the four instruments described above, two proposed five factors, one proposed four factors, and one proposed three factors. There has not been a systematic comparison of the instruments; thus, it is unclear which factor structure is best supported by data. A second problem is that factor analytic solutions frequently have failed to replicate [50]. This suggests that regardless of proposed factor structure, instruments vary considerably from sample to sample. A third problem is that each factor usually explains a small proportion of sample variance (i.e., less than 15%), while the overall instrument (i.e., all factors combined) likewise explains a small proportion of sample variance (i.e., less than 50%). A fourth problem is that individual factors frequently have marginal reliabilities (i.e., less than .70). Overall, these findings suggest that questionnaires should be interpreted carefully within the context in which the data is collected [44].


### 3.2. Measuring Epistemological World Views: Interviews, Vignettes, Essays, and Concept Maps

A growing body of research has investigated epistemological beliefs and world views using interviews [25, 26, 52, 77, 109, 113–117], vignettes [29, 53], essays, journals, and storyboards [54–56], and concept maps [28, 57]. These studies usually differ in terms of methodology and the main research question from studies relying primarily on questionnaire data. Questionnaire studies have focused on using quantitative independent and dependent variables to examine the correlations between variables. In contrast, interview and essay studies have focused on qualitative outcome variables that are used to understand beliefs and world views in greater detail [118, 119]. These studies often focus on how different beliefs and world views affect curriculum and pedagogical choices from the perspective of teachers or students. However, it is important to note that many studies use some combination of the above [53], sometimes with questionnaires as well [77].Interviews have been used extensively in epistemological research, accounting for approximately 50 to 60 percent of the research in the current review of the literature. Johnston et al. [25] interviewed teachers as part of four case studies to examine teaching differences between teachers with a constructed knowing versus a received knowing epistemological stance. The former emphasizes context-bound, person-constructed knowledge in a student-centered classroom, while the later emphasizes universal, factual knowledge in a teacher-centered classroom. Teachers with constructed knowing stances were more likely to emphasize the role of student interaction with teachers and other students. In addition, constructivist teachers used a wider repertoire of teaching strategies and content sources in the classroom [120, 121]. Similar results have been reported by a variety of other studies that have reported a positive relationship between relativist epistemologies and constructivist classroom practices [51, 109, 113, 116, 122–124].Brownlee et al. [54] examined preservice teachers’ epistemological beliefs and whether the promotion of reflective practice in the classroom changed beliefs in a yearlong teacher education program. Teachers in the reflective practice condition reported more growth in epistemological beliefs than teachers in the nonreflective condition. Beliefs changed with respect to teachers expressing more uncertainty, more subjectivity, and greater flexibility in personal beliefs. Although beliefs became more relativistic for all students, changes among the reflection group were significantly greater than the comparison group. Marra [51] reported similar findings in a study of how constructivist instruction affected the development of graduate student teachers at a university. Teachers reported a variety of changes after the course, but especially changes in epistemological and pedagogical beliefs. Teachers adopted constructivist beliefs that emphasized the role of student interactions. In addition, graduate student teachers adopted stronger contextualist beliefs about the complexity and certainty of knowledge. Overall, a wide variety of studies have reported that reflection promoted more sophisticated epistemological beliefs [11, 14, 31, 118, 125].Wilson [77] used interviews and surveys to examine whether the amount of education was related to teachers’ epistemological stances. Teachers were classified as dualist, multiplist, or relativist. Teachers with graduate degrees tended to be in the multiplist and relativist groups rather than the dualist group. In contrast, teachers with a baccalaureate or less were more likely to be in the dualist group. Tsai and colleagues [52, 126, 127] reported similar findings in a study of Taiwanese science teachers. Teachers with more sophisticated epistemological beliefs and world views were more likely to endorse student-centered instructional practices that emphasized critical reasoning. Teachers with less sophisticated beliefs focused on traditional curriculum, student testing, and mastery of basic science concepts. Kang and Wallace [26] also reported parallel findings when examining the relationship between epistemological beliefs and laboratory activities in American high school science classrooms. Lidar et al. [27] found that teachers with more sophisticated personal epistemologies used a greater number of epistemological moves in their science classrooms. Epistemological moves consisted of cognitive activities designed to promote deeper learning and reflection, including generating, constructing, and reconstructing. They also reported that the relative success of different epistemological moves depended in large part on contextually specific factors such as student knowledge, complexity of activity, and sophistication of students’ conceptual understanding.Vignettes have been used by several researchers in conjunction with interviews and essays. Joram [53] examined preservice teachers’, practicing teachers’, and professors’ beliefs about what counts as legitimate knowledge. Participants read two different vignettes that described the effect of research on classroom teaching. In the teaching vignette, research supported classroom practices, whereas in the research vignette, research contradicted classroom practice. Participants read each vignette individually and responded to probe questions about the credibility of evidence and the extent to which research informs teaching. Two findings were of special importance. One was that preservice and practicing teachers emphasized the situation specific nature of knowledge, whereas professors emphasized the general nature or knowledge and general learning principles to a much greater extent. A second finding was that students were much more likely to believe that knowledge is transmitted, whereas professors strongly believed that knowledge is constructed or discovered. Joram [53] concluded that students and beginning teachers hold a view of educational knowledge as noncumulative, specific, and transmitted, and difficult to falsify; whereas experienced teachers and professors hold a view of educational knowledge as cumulative, changeable, generalizable, and falsifiable.Schraw and Olafson [29] created three vignettes shown in Appendix B that assessed realist, contextualist, and relativist world views of knowledge and teaching. Practicing teachers read each vignette and rated the degree to which they agreed with its position. Teachers also completed the Epistemological Belief Inventory [50], the Need for Cognition Scale [128], and a teacher motivation scale that assessed intrinsic and extrinsic motivation to teach. Three findings were of special note. One was that teachers strongly supported the contextualist and relativist world views, but not the realist world view. A second finding was that the need for cognition was correlated positively with the contextualist and relativist world views, but not the realist world view. A third finding was that years of teaching were correlated positively with the realist world view, but negatively with the contextualist and relativist world view. This suggested that teachers with long service (i.e., greater than 15 years) are more likely to endorse a realist world view and that world views may migrate over time from a contextualist to a realist position.Essays, journals, and pictorial storyboards have been used as well to explore world views and justification of beliefs. Sandoval [56] investigated whether instruction and electronic journaling using Explanation Constructor promoted change in epistemological beliefs. High school biology students completed five hours of computer-based instruction and used an electronic journal in small groups to explain and justify responses. Students produced increasingly sophisticated explanations and justifications as their opportunity to engage in group inquiry increased [118]. Several other researchers have used written explanations as data. Brownlee et al. [54] asked teachers to write a description and explanation of their epistemological beliefs. She found that students experienced difficulty doing so and often reported beliefs that conflicted with classroom practice. Asli Özgün-Koca and Ilhan Şen  [28] used journals for preservice teachers to justify beliefs outlined in self-generated concept maps about “effective instruction.” Olafson and Schraw [55] asked teachers to construct storyboards of a salient teaching episode and to reflect on the teaching experience. Storyboards consisted of digital photographs and a written description of the progression of events. Participants also were asked to provide written reflections that considered the ways in which their practices were indicative of their beliefs about teaching and learning. The vast majority of participants endorsed contextualist world views of teaching in their reflections and provided evidence in the storyboards (e.g., collaborative work, student choice) that was consistent with their views.Concept maps also have been used in several studies as a way to measure conceptual understanding of a domain and to assess epistemological world views [20, 28, 57]. Asli Özgün-Koca and Ilhan Şen  [28] investigated preservice teachers’ beliefs about effective instruction. Participants were asked to construct concept maps that linked important concepts such as content and pedagogical knowledge together. Interviews and journals were used after the construction of concept maps to better understand beliefs embedded within the concept map, the development of beliefs, as well as justification for each participant’s beliefs. Results suggested that preservice teachers’ have incomplete knowledge about teaching, value student-centered learning, and are concerned about their lack of domain expertise, especially in math and science.Two other studies utilized concept maps, primarily as a means to evaluate content knowledge and conceptual change over time [20, 57]. Liu [57] studies 15 high school science students who constructed concept maps as part of their chemistry classroom. Liu used these maps to evaluate the number and sophistication of ontological categories (e.g., chemical processes, interactions). He concluded that ongoing mapping led to greater conceptual change and a richer understanding of information within the targeted ontological categories. Yang [20] used concept maps in conjunction with a questionnaire to determine the conceptual understanding of 10th grade science students. An analysis of concept maps revealed growth over time due to instruction but was not linked to the relative sophistication of each student’s epistemological views.


### 3.3. Measuring Ontological Beliefs

Ontology is a core field of study in philosophy [7]. Nevertheless, little theoretical or empirical research has been conducted on students’ or teachers’ ontological beliefs by educational and psychological researchers. Notable exceptions include review and analysis articles by Greene et al. [38], Packer and Goicoechea [39], and Ponterotto [8]. Greene et al. [38] proposed a four-group classification system in which individuals are characterized by realism, dogmatism, skepticism, or rationalism based on different modes of epistemological and ontological cognition. Packer and Goicoechea [39] explored the epistemological and ontological differences between sociocultural and constructivists views of learning. Ponterotto [8] compared the epistemological and ontological differences among four competing paradigms used in counseling psychology.An electronic literature search revealed three empirical studies that examined ontological beliefs. Three of these studies considered the conceptual role of ontological beliefs [57, 110, 129] but did not provide quantitative measures of ontological beliefs or world views. Liu [57] investigated whether instruction about how to construct concept maps led to conceptual change in a 12th grade chemistry classroom as measured by changes in student’s concept maps of the target domain. In addition, final concept maps were analyzed with respect to the sophistication of ontological categories within the map. Liu [57] concluded that instruction produced more sophisticated ontological conceptual of the domain of knowledge. She [129] examined whether a Dual-Situated Learning Model (DSLM) that challenged students’ beliefs fostered radical conceptual change in ontological categories in among 9th grade students studying heat transfer.Slotta and Chi [40] investigated the effect of ontological training in promoting conceptual change among physics students. Ontological training consisted of a computerized instructional module that helped students understand four ontological principles related to emergent processes in physical phenomena, which focused on changes that are system wide, equilibrium seeking, simultaneous and independent, and ongoing. College students were classified based on interviews into expert and novice levels of ontological commitment. Results indicated that experts performed better than novices on the posttest as well as the completeness of verbal explanations. The experimental group also outperformed the control on the posttest and verbal explanations. These findings indicated that explicit training helps students become of implicit ontological categories and to learn information and evaluation skills better than the control group.Olafson and colleagues [55, 58, 118] asked teachers to situate themselves using a two-dimensional Cartesian coordinate system with respect to epistemological and ontological beliefs. Olafson et al. [118] referred to this method as the four-quadrant scale (see Figure 1). Teachers enrolled in college graduate classes were asked to read written instructions, listen to additional verbal instructions, then situate themselves in one of the four quadrants by making an “X” that best corresponded to the extent to which they endorse relativist versus realist epistemological and ontological world views. Teachers also participated in interviews to explain and justify their choices. Several findings were of interest. One was a statistically significant correlation of .47 between the epistemological and ontological dimensions. A second finding was that approximately 90 percent of teachers reported being ontological relativists to one degree or another, whereas about 40 percent reported being epistemological realists to one degree or another. A third finding was that teachers were inconsistent between their four-quadrant ratings and verbal justifications. Specifically, although many teachers professed to be highly relativist on the four-quadrant rating, they described teacher-centered realist classroom practices in the verbal interviews. This inconsistency between stated beliefs and practices has been noted previously by researchers [23, 24, 130, 131].Figure 1
The four-quadrant scale.


### 3.4. Summary of Measurement Strategies

This section identified six different measurement strategies used in previous research, including questionnaires, interviews, vignettes, essays, concept maps, and multidimensional scaling methods. Most studies used questionnaires or interviews, with some studies using two or more of these strategies together. Measurement strategies were selected based on the main goals of the study; thus, it is difficult to compare across participants (i.e., students or teachers) phenomena of interest (i.e., beliefs or world views). The vast majority of the studies focused on epistemological phenomena, and only one of these studies investigated both epistemology and ontology simultaneously.Overall, the extant literature suggests six general conclusions. One is that beliefs and world views can be measured using different types of measurement strategies. Questionnaires, vignettes, and scales were used to measure both epistemological and ontological beliefs and world views. Interviews, journals, and concept maps were used to explore these beliefs in greater detail. Although the bulk of the existing research has focused on epistemological beliefs measured through questionnaires, researcher could use any of the six measurement strategies with either epistemological or ontological phenomena.A second conclusion is that beliefs and world views are multidimensional and complex [48, 49, 67, 132]. For example, many practicing teachers endorsed both contextualist and relativist stances [118, 133, 134]. Both students and teachers clearly hold a variety of beliefs that are situated within epistemological and ontological world views. These beliefs appear to change over time due to a variety of factors. In addition, beliefs and world views often are not fully explicit. Several studies suggest that instruction makes beliefs more explicit and more amenable to change.A third conclusion is that beliefs and world views affect teaching practice [21, 31, 46]. Most studies consistently report that realist perspectives are linked to teacher-centered classroom that emphasize a core body of knowledge and teacher-directed instruction. In contrast, relativist perspectives tend to be link to student-centered classrooms characterized by collaborative learning, tutors, and independent learning opportunities. Nevertheless, a number of studies also reported important inconsistencies between stated beliefs and practices [24, 74, 118, 131]. Specifically, some teachers profess relativist, student-centered beliefs, but report a strong reliance on teacher-directed instruction, off-the-shelf curriculum, and an emphasis on testing and accountability, though inconsistencies exist. Many practicing teachers also face significant dilemmas with respect to identifying and implementing their personal beliefs [134].A fourth conclusion is that beliefs and world views are context bound [10, 69, 92, 135]. Many teachers and student reported that learning goals and practices were affected by institutional and classroom expectations and developed slowly throughout their university training [136–138] and first years of teaching [92, 139]. In addition, objective measurements across multiple domains usually suggested that beliefs changed as a function of the academic domain, as well as the relative sophistication of the materials being studied within that domain [55, 97, 140].A fifth conclusion is that instruction and short-term intervention designed to promote conceptual change, reflection, and examination of personal beliefs has a positive effect on epistemological beliefs and classroom practice [141]. For example, Olafson et al. [118] found that a 16-week college-level class that included discussion of beliefs, reflective writing and discussion activities, and action research projects in school settings affected beliefs and significantly changes attitudes about teaching practices. Generally, individuals become more likely to use contextualist, student-centered teaching practices and focus on large-grain thinking skills [21, 42, 73, 74, 114]. A variety of studies have reported similar findings [142–145].A sixth conclusion is that teachers’ beliefs and classroom practices affect students’ beliefs and learning [146–148]. A variety of studies have found that teachers’ epistemological beliefs are related positively to student epistemological development and learning outcomes and shape students’ emerging conceptions of knowledge [27, 51, 127]. Weinstock and Roth [149] also showed that teachers’ epistemological beliefs are related positively to student autonomy.


## 3.1. Measuring Epistemological Beliefs: Questionnaires

A number of researchers have measured epistemological beliefs using self-report questionnaires [16, 19, 48–50, 79, 80]. The majority of this research has focused on college students’ epistemological beliefs, although more recent research has examined either preservice teachers [81–90] or inservice teachers’ epistemological beliefs [22, 54, 91–93].Schommer [19] proposed five independent beliefs based on the work of Perry [18] pertaining to certain knowledge (i.e., absolute knowledge exists and will eventually be known), simple knowledge (i.e., knowledge consists of discrete facts), omniscient authority (i.e., authorities have access to otherwise inaccessible knowledge), quick learning (i.e., learning occurs in a quick or not-at-all fashion), and innate ability (i.e., the ability to acquire knowledge is endowed at birth). Currently, there is debate as to whether Schommer’s five beliefs constitute genuine epistemological dimensions [12]. Most researchers agree that beliefs about the certainty and simplicity of knowledge constitute genuine epistemological beliefs. In contrast, many researchers argue that beliefs about innate ability reflect some other, nonepistemological dimension [12].Schommer [19] developed the Epistemological Questionnaire (EQ) to assess the five dimensions described above. The EQ consisted of 62 simple statements that individuals responded to using a five-point Likert scale, indicating the extent to which they agreed or disagreed with the statement (see Appendix A, e.g.). The EQ has been used frequently over the past 15 years by Schommer and other researchers [22, 67, 94, 95]. Results have been somewhat mixed in a variety of ways. First, factor analyses typically reported less than five interpretable factors [49, 50]. Second, some analyses have yielded factors with an interpretation that was not predicted by Schommer [19]. For example, a factor might pertain to the source of knowledge or have item loadings that are difficult to interpret in a manner consistent with Schommer’s proposed five-factor structure. Third, the factors reported in the analyses often had a small number of items with acceptable factor loadings and therefore had low or unacceptable reliability coefficients. Fourth, solutions usually explained a relatively small proportion of sample variance (e.g., 20% to 35%), which raised concerns about the construct validity of the questionnaire.Schraw et al. [50] developed a modified version of Schommer’s EQ called the Epistemic Beliefs Inventory (EBI) to address these methodological problems. The EBI contained 32 items based on the five factors proposed by Schommer [19] (see Appendix A, e.g.). The EBI typically yielded the five proposed factors; however, reliabilities tended to be low (e.g., .50 to .65) and results varied depending on the age and gender of the sample. In addition, like the EQ, the EBI explained a relatively small proportion of sample variation (e.g., usually less than 40%).Hofer [49] proposed an alternative four-factor framework and developed an instrument called the Epistemological Beliefs Questionnaire (EBQ) to assess these factors. She proposed four factors, which were subsumed under two general dimensions referred to as the nature of knowing and the process of knowing. The former refers to what knowledge is presumed to be, while the latter refers to how one comes to know and understand knowledge. The “nature” dimension included two factors called certainty of knowledge (i.e., the degree to which one sees knowledge as fixed versus fluid and changeable) and simplicity of knowledge (i.e., the degree to which knowledge is viewed as individual facts versus complex, interrelated concepts). The “process” dimension included two factors called source of knowledge (i.e., the extent to which credible knowledge is self- or other generated) and justification of knowing (i.e., the rules and criteria that individuals use to evaluate knowledge claims).Hofer [49] reported four empirically derived factors that differed somewhat from the four proposed factors described above (see Appendix A, e.g.). Her four observed factors included certainty of knowledge, source of knowledge, justification, and attainment of truth. A “simplicity of knowledge” factor was not observed, whereas an “attainment of truth” factor was observed. The attainment of truth factor was interpreted as the extent to which experts can attain deep knowledge (i.e., “truth”) within their area of expertise. Like the EQ and EBI, the EBQ explained approximately 45% of sample variance, had several factors with few items that loaded satisfactorily, and had factors with low reliability coefficients.Bromme and colleagues [48, 96] have taken a somewhat different approach to measuring epistemological beliefs. They created a 24-item semantic differential type instrument that measures connotative aspects of student’s epistemological beliefs using bipolar adjective pairs that were subsumed under three hypothesized epistemological factors, including certainty, simplicity, and source. For example, the certainty included the certain-uncertain pair; simplicity included the simple-complex pair; and the source factor included the constructed-preexisting pair. Two studies using university students enrolled in either plant biology or chemistry yielded two reliable factors referred to as texture and variability. Texture assessed the complexity and sophistication of beliefs, whereas variability assessed the changeability and permanence of beliefs.Thus far, self-report questionnaires that have been developed to assess multiple epistemological beliefs have contributed to the literature in two important ways. One contribution of this research is that there has been a great deal of productive discussion regarding the set of constructs that comprise the domain of epistemological beliefs [2, 12, 49, 97–99]. These discussions are crucial to mapping the possible domain of epistemological beliefs both conceptually and methodologically. A second contribution has been the preliminary findings concerning the relationships among epistemological beliefs and a variety of outcomes variables such as age, education level, gender, moral reasoning skills, and academic achievement [67, 100–102]. Epistemological beliefs that are more relativistic tend to be associated with better problem solving [20, 80, 86, 103], higher self-regulation [2, 99, 104–107], metacognition [91, 96, 108], constructivist teaching practices [29, 89, 109, 110], and academic career choices [111, 112]. However, despite positive correlations, the effect size tends to be small in most cases, usually in the .05 to .15 range in terms of variance explained in the outcome measure of interest.Questionnaires also have experienced a number of difficulties [84]. One is disagreement about the number and content of constructs being investigated. Of the four instruments described above, two proposed five factors, one proposed four factors, and one proposed three factors. There has not been a systematic comparison of the instruments; thus, it is unclear which factor structure is best supported by data. A second problem is that factor analytic solutions frequently have failed to replicate [50]. This suggests that regardless of proposed factor structure, instruments vary considerably from sample to sample. A third problem is that each factor usually explains a small proportion of sample variance (i.e., less than 15%), while the overall instrument (i.e., all factors combined) likewise explains a small proportion of sample variance (i.e., less than 50%). A fourth problem is that individual factors frequently have marginal reliabilities (i.e., less than .70). Overall, these findings suggest that questionnaires should be interpreted carefully within the context in which the data is collected [44].


## 3.2. Measuring Epistemological World Views: Interviews, Vignettes, Essays, and Concept Maps

A growing body of research has investigated epistemological beliefs and world views using interviews [25, 26, 52, 77, 109, 113–117], vignettes [29, 53], essays, journals, and storyboards [54–56], and concept maps [28, 57]. These studies usually differ in terms of methodology and the main research question from studies relying primarily on questionnaire data. Questionnaire studies have focused on using quantitative independent and dependent variables to examine the correlations between variables. In contrast, interview and essay studies have focused on qualitative outcome variables that are used to understand beliefs and world views in greater detail [118, 119]. These studies often focus on how different beliefs and world views affect curriculum and pedagogical choices from the perspective of teachers or students. However, it is important to note that many studies use some combination of the above [53], sometimes with questionnaires as well [77].Interviews have been used extensively in epistemological research, accounting for approximately 50 to 60 percent of the research in the current review of the literature. Johnston et al. [25] interviewed teachers as part of four case studies to examine teaching differences between teachers with a constructed knowing versus a received knowing epistemological stance. The former emphasizes context-bound, person-constructed knowledge in a student-centered classroom, while the later emphasizes universal, factual knowledge in a teacher-centered classroom. Teachers with constructed knowing stances were more likely to emphasize the role of student interaction with teachers and other students. In addition, constructivist teachers used a wider repertoire of teaching strategies and content sources in the classroom [120, 121]. Similar results have been reported by a variety of other studies that have reported a positive relationship between relativist epistemologies and constructivist classroom practices [51, 109, 113, 116, 122–124].Brownlee et al. [54] examined preservice teachers’ epistemological beliefs and whether the promotion of reflective practice in the classroom changed beliefs in a yearlong teacher education program. Teachers in the reflective practice condition reported more growth in epistemological beliefs than teachers in the nonreflective condition. Beliefs changed with respect to teachers expressing more uncertainty, more subjectivity, and greater flexibility in personal beliefs. Although beliefs became more relativistic for all students, changes among the reflection group were significantly greater than the comparison group. Marra [51] reported similar findings in a study of how constructivist instruction affected the development of graduate student teachers at a university. Teachers reported a variety of changes after the course, but especially changes in epistemological and pedagogical beliefs. Teachers adopted constructivist beliefs that emphasized the role of student interactions. In addition, graduate student teachers adopted stronger contextualist beliefs about the complexity and certainty of knowledge. Overall, a wide variety of studies have reported that reflection promoted more sophisticated epistemological beliefs [11, 14, 31, 118, 125].Wilson [77] used interviews and surveys to examine whether the amount of education was related to teachers’ epistemological stances. Teachers were classified as dualist, multiplist, or relativist. Teachers with graduate degrees tended to be in the multiplist and relativist groups rather than the dualist group. In contrast, teachers with a baccalaureate or less were more likely to be in the dualist group. Tsai and colleagues [52, 126, 127] reported similar findings in a study of Taiwanese science teachers. Teachers with more sophisticated epistemological beliefs and world views were more likely to endorse student-centered instructional practices that emphasized critical reasoning. Teachers with less sophisticated beliefs focused on traditional curriculum, student testing, and mastery of basic science concepts. Kang and Wallace [26] also reported parallel findings when examining the relationship between epistemological beliefs and laboratory activities in American high school science classrooms. Lidar et al. [27] found that teachers with more sophisticated personal epistemologies used a greater number of epistemological moves in their science classrooms. Epistemological moves consisted of cognitive activities designed to promote deeper learning and reflection, including generating, constructing, and reconstructing. They also reported that the relative success of different epistemological moves depended in large part on contextually specific factors such as student knowledge, complexity of activity, and sophistication of students’ conceptual understanding.Vignettes have been used by several researchers in conjunction with interviews and essays. Joram [53] examined preservice teachers’, practicing teachers’, and professors’ beliefs about what counts as legitimate knowledge. Participants read two different vignettes that described the effect of research on classroom teaching. In the teaching vignette, research supported classroom practices, whereas in the research vignette, research contradicted classroom practice. Participants read each vignette individually and responded to probe questions about the credibility of evidence and the extent to which research informs teaching. Two findings were of special importance. One was that preservice and practicing teachers emphasized the situation specific nature of knowledge, whereas professors emphasized the general nature or knowledge and general learning principles to a much greater extent. A second finding was that students were much more likely to believe that knowledge is transmitted, whereas professors strongly believed that knowledge is constructed or discovered. Joram [53] concluded that students and beginning teachers hold a view of educational knowledge as noncumulative, specific, and transmitted, and difficult to falsify; whereas experienced teachers and professors hold a view of educational knowledge as cumulative, changeable, generalizable, and falsifiable.Schraw and Olafson [29] created three vignettes shown in Appendix B that assessed realist, contextualist, and relativist world views of knowledge and teaching. Practicing teachers read each vignette and rated the degree to which they agreed with its position. Teachers also completed the Epistemological Belief Inventory [50], the Need for Cognition Scale [128], and a teacher motivation scale that assessed intrinsic and extrinsic motivation to teach. Three findings were of special note. One was that teachers strongly supported the contextualist and relativist world views, but not the realist world view. A second finding was that the need for cognition was correlated positively with the contextualist and relativist world views, but not the realist world view. A third finding was that years of teaching were correlated positively with the realist world view, but negatively with the contextualist and relativist world view. This suggested that teachers with long service (i.e., greater than 15 years) are more likely to endorse a realist world view and that world views may migrate over time from a contextualist to a realist position.Essays, journals, and pictorial storyboards have been used as well to explore world views and justification of beliefs. Sandoval [56] investigated whether instruction and electronic journaling using Explanation Constructor promoted change in epistemological beliefs. High school biology students completed five hours of computer-based instruction and used an electronic journal in small groups to explain and justify responses. Students produced increasingly sophisticated explanations and justifications as their opportunity to engage in group inquiry increased [118]. Several other researchers have used written explanations as data. Brownlee et al. [54] asked teachers to write a description and explanation of their epistemological beliefs. She found that students experienced difficulty doing so and often reported beliefs that conflicted with classroom practice. Asli Özgün-Koca and Ilhan Şen  [28] used journals for preservice teachers to justify beliefs outlined in self-generated concept maps about “effective instruction.” Olafson and Schraw [55] asked teachers to construct storyboards of a salient teaching episode and to reflect on the teaching experience. Storyboards consisted of digital photographs and a written description of the progression of events. Participants also were asked to provide written reflections that considered the ways in which their practices were indicative of their beliefs about teaching and learning. The vast majority of participants endorsed contextualist world views of teaching in their reflections and provided evidence in the storyboards (e.g., collaborative work, student choice) that was consistent with their views.Concept maps also have been used in several studies as a way to measure conceptual understanding of a domain and to assess epistemological world views [20, 28, 57]. Asli Özgün-Koca and Ilhan Şen  [28] investigated preservice teachers’ beliefs about effective instruction. Participants were asked to construct concept maps that linked important concepts such as content and pedagogical knowledge together. Interviews and journals were used after the construction of concept maps to better understand beliefs embedded within the concept map, the development of beliefs, as well as justification for each participant’s beliefs. Results suggested that preservice teachers’ have incomplete knowledge about teaching, value student-centered learning, and are concerned about their lack of domain expertise, especially in math and science.Two other studies utilized concept maps, primarily as a means to evaluate content knowledge and conceptual change over time [20, 57]. Liu [57] studies 15 high school science students who constructed concept maps as part of their chemistry classroom. Liu used these maps to evaluate the number and sophistication of ontological categories (e.g., chemical processes, interactions). He concluded that ongoing mapping led to greater conceptual change and a richer understanding of information within the targeted ontological categories. Yang [20] used concept maps in conjunction with a questionnaire to determine the conceptual understanding of 10th grade science students. An analysis of concept maps revealed growth over time due to instruction but was not linked to the relative sophistication of each student’s epistemological views.


## 3.3. Measuring Ontological Beliefs

Ontology is a core field of study in philosophy [7]. Nevertheless, little theoretical or empirical research has been conducted on students’ or teachers’ ontological beliefs by educational and psychological researchers. Notable exceptions include review and analysis articles by Greene et al. [38], Packer and Goicoechea [39], and Ponterotto [8]. Greene et al. [38] proposed a four-group classification system in which individuals are characterized by realism, dogmatism, skepticism, or rationalism based on different modes of epistemological and ontological cognition. Packer and Goicoechea [39] explored the epistemological and ontological differences between sociocultural and constructivists views of learning. Ponterotto [8] compared the epistemological and ontological differences among four competing paradigms used in counseling psychology.An electronic literature search revealed three empirical studies that examined ontological beliefs. Three of these studies considered the conceptual role of ontological beliefs [57, 110, 129] but did not provide quantitative measures of ontological beliefs or world views. Liu [57] investigated whether instruction about how to construct concept maps led to conceptual change in a 12th grade chemistry classroom as measured by changes in student’s concept maps of the target domain. In addition, final concept maps were analyzed with respect to the sophistication of ontological categories within the map. Liu [57] concluded that instruction produced more sophisticated ontological conceptual of the domain of knowledge. She [129] examined whether a Dual-Situated Learning Model (DSLM) that challenged students’ beliefs fostered radical conceptual change in ontological categories in among 9th grade students studying heat transfer.Slotta and Chi [40] investigated the effect of ontological training in promoting conceptual change among physics students. Ontological training consisted of a computerized instructional module that helped students understand four ontological principles related to emergent processes in physical phenomena, which focused on changes that are system wide, equilibrium seeking, simultaneous and independent, and ongoing. College students were classified based on interviews into expert and novice levels of ontological commitment. Results indicated that experts performed better than novices on the posttest as well as the completeness of verbal explanations. The experimental group also outperformed the control on the posttest and verbal explanations. These findings indicated that explicit training helps students become of implicit ontological categories and to learn information and evaluation skills better than the control group.Olafson and colleagues [55, 58, 118] asked teachers to situate themselves using a two-dimensional Cartesian coordinate system with respect to epistemological and ontological beliefs. Olafson et al. [118] referred to this method as the four-quadrant scale (see Figure 1). Teachers enrolled in college graduate classes were asked to read written instructions, listen to additional verbal instructions, then situate themselves in one of the four quadrants by making an “X” that best corresponded to the extent to which they endorse relativist versus realist epistemological and ontological world views. Teachers also participated in interviews to explain and justify their choices. Several findings were of interest. One was a statistically significant correlation of .47 between the epistemological and ontological dimensions. A second finding was that approximately 90 percent of teachers reported being ontological relativists to one degree or another, whereas about 40 percent reported being epistemological realists to one degree or another. A third finding was that teachers were inconsistent between their four-quadrant ratings and verbal justifications. Specifically, although many teachers professed to be highly relativist on the four-quadrant rating, they described teacher-centered realist classroom practices in the verbal interviews. This inconsistency between stated beliefs and practices has been noted previously by researchers [23, 24, 130, 131].Figure 1
The four-quadrant scale.


## 3.4. Summary of Measurement Strategies

This section identified six different measurement strategies used in previous research, including questionnaires, interviews, vignettes, essays, concept maps, and multidimensional scaling methods. Most studies used questionnaires or interviews, with some studies using two or more of these strategies together. Measurement strategies were selected based on the main goals of the study; thus, it is difficult to compare across participants (i.e., students or teachers) phenomena of interest (i.e., beliefs or world views). The vast majority of the studies focused on epistemological phenomena, and only one of these studies investigated both epistemology and ontology simultaneously.Overall, the extant literature suggests six general conclusions. One is that beliefs and world views can be measured using different types of measurement strategies. Questionnaires, vignettes, and scales were used to measure both epistemological and ontological beliefs and world views. Interviews, journals, and concept maps were used to explore these beliefs in greater detail. Although the bulk of the existing research has focused on epistemological beliefs measured through questionnaires, researcher could use any of the six measurement strategies with either epistemological or ontological phenomena.A second conclusion is that beliefs and world views are multidimensional and complex [48, 49, 67, 132]. For example, many practicing teachers endorsed both contextualist and relativist stances [118, 133, 134]. Both students and teachers clearly hold a variety of beliefs that are situated within epistemological and ontological world views. These beliefs appear to change over time due to a variety of factors. In addition, beliefs and world views often are not fully explicit. Several studies suggest that instruction makes beliefs more explicit and more amenable to change.A third conclusion is that beliefs and world views affect teaching practice [21, 31, 46]. Most studies consistently report that realist perspectives are linked to teacher-centered classroom that emphasize a core body of knowledge and teacher-directed instruction. In contrast, relativist perspectives tend to be link to student-centered classrooms characterized by collaborative learning, tutors, and independent learning opportunities. Nevertheless, a number of studies also reported important inconsistencies between stated beliefs and practices [24, 74, 118, 131]. Specifically, some teachers profess relativist, student-centered beliefs, but report a strong reliance on teacher-directed instruction, off-the-shelf curriculum, and an emphasis on testing and accountability, though inconsistencies exist. Many practicing teachers also face significant dilemmas with respect to identifying and implementing their personal beliefs [134].A fourth conclusion is that beliefs and world views are context bound [10, 69, 92, 135]. Many teachers and student reported that learning goals and practices were affected by institutional and classroom expectations and developed slowly throughout their university training [136–138] and first years of teaching [92, 139]. In addition, objective measurements across multiple domains usually suggested that beliefs changed as a function of the academic domain, as well as the relative sophistication of the materials being studied within that domain [55, 97, 140].A fifth conclusion is that instruction and short-term intervention designed to promote conceptual change, reflection, and examination of personal beliefs has a positive effect on epistemological beliefs and classroom practice [141]. For example, Olafson et al. [118] found that a 16-week college-level class that included discussion of beliefs, reflective writing and discussion activities, and action research projects in school settings affected beliefs and significantly changes attitudes about teaching practices. Generally, individuals become more likely to use contextualist, student-centered teaching practices and focus on large-grain thinking skills [21, 42, 73, 74, 114]. A variety of studies have reported similar findings [142–145].A sixth conclusion is that teachers’ beliefs and classroom practices affect students’ beliefs and learning [146–148]. A variety of studies have found that teachers’ epistemological beliefs are related positively to student epistemological development and learning outcomes and shape students’ emerging conceptions of knowledge [27, 51, 127]. Weinstock and Roth [149] also showed that teachers’ epistemological beliefs are related positively to student autonomy.


## 4. A Comparison of Constructs and Measurement Strategies Used in Previous Research

Research to date has focused primarily on individual epistemological beliefs, a global epistemological world view, or a global ontological world view. There are no empirical studies that we located that examined multidimensional ontological beliefs analogous to the multibelief models proposed by Schommer [19] or Hofer [49]. Most studies have relied on one main measurement strategy using self-report questionnaires; however, approximately 40 percent of studies used a second data collection method such as an interview. Different measurement strategies have been used for different research purposes. Table 2 shows that studies focusing on students’ epistemological beliefs have relied primarily on questionnaires. In contrast, studies that addressed teachers’ world views have relied primarily on vignettes, interviews, essays, or some combination of these methods [26, 28, 54]. The few available studies of ontology have used a variety of methods, including interviews [129], an analysis of concept maps [57], short answer responses and interviews [40], and multidimensional scales [58, 118]. There are no studies in the literature that systematically compared across different measurement strategies.Table 2
Constructs and instruments used in research.
Constructs
Epistemological beliefs
Epistemological world views and stances
Epistemological and ontological relativism
Instruments
Epistemological Questionnaire [19]
Interviews [25, 51, 52, 54]
Four quadrant scale [58]
Epistemological Belief Inventory [150]
Vignettes [29, 53]
Epistemological Beliefs Questionnaire [49]
Essays and journals [28, 56]
Connotative Aspects of Epistemological Beliefs [48]
Concept maps [28, 57]Table3 summarizes the construct being assessed, design of the measurement tool, and relative strengths for each of the six measurement strategies described here. Each measurement strategy differs with respect to the ease of administration, type of information, and richness of information that it provides. All of the strategies shown in Table 3 complement one another and can be used together in virtually any study. Indeed, a number of the studies reported above used multiple strategies such as questionnaires, interviews, and essays [114]; concept maps, interviews, and journals [28]; or questionnaires, multidimensional scales, and interviews [118].Table 3
Six measurement strategies used in research.
Construct that is measured
Examples
Design of the measurement tool
Strengths
Questionnaires
Questionnaires measure multiple, presumably independent epistemological beliefs
EQ, EBQ, EBI, CAEB
Agreement using a Likert scale with specific statements
Measures separate independent beliefs using same scale; statistical analyses
Interviews
These methods measure the structure, origin, and impact of beliefs
Interviews, verbal  responses, think-alouds
Usually a structured interview with probes
Depth of response; justification of beliefs, evidence, and examples
Vignettes
Vignettes measure commitment to different epistemological world views and stances
Vignettes that summarize a prototypical world view or situation
Agreement using a Likert scale to the vignette
Measures relative commitment to separate world views described in the vignette
Essays, journals, and storyboards
These methods measure the structure, origin, and impact of beliefs
Detailed question
Usually an essay focusing on one or several specific questions
Depth of response; justification of beliefs, evidence, and examples; can be revised
Concept maps
Measures the relationships among beliefs
Individuals create a concept map
Individual constructs the concept map; identifies key concepts and their links
Identifies key concepts and their relationships
Scales
Scales measure commitment to epistemological and ontological relativism
Four quadrant scale
Situate oneself at a specific point in the quadrant
Compares epistemological and ontological beliefs using same scale; measures strength on each dimensionQuestionnaires and interviews clearly are the strategies of choice. Questionnaires are quick and easy to administer and amenable to sophisticated statistical analyses of reliability, validity, and the relationship of each variable to other variables. In contrast, interviews are time consuming to administer and score, but provide extremely rich data about beliefs, belief change, and how beliefs are related to classroom practices.This review foregoes an in-depth discussion of each of the six strategies, which are described at length in many research textbooks [4, 151–153], focusing instead on three important omissions in the literature. One is the absence of a questionnaire that measures ontological beliefs. Hofer and Pintrich [12] noted previously that many epistemological questionnaires also appear to measure ontological beliefs and assumptions. Notwithstanding this claim, there is no scale in the current literature that focuses on this goal, provides psychometrically reliable and valid scales, and most importantly provides a detailed discussion of the ontological construct(s) being measured.A second omission is the lack of cross-validation research within and between studies. Studies that used multiple measurement strategies typically reported the results from each strategy in an autonomous manner. There was very limited discussion regarding the consistency across strategies and rarely any quantitative data (e.g., correlations or inter-rater agreement) across methods. In addition, it is not clear based on these studies whether one strategy is better suited to investigate different types of epistemological and ontological beliefs and world views.A third omission is the lack of what I refer to asuniversal measurement design principles, by which I mean using a measurement strategy in a consistent manner across studies. There are at least four ways this might be accomplished. One way is thorough definition of constructs being investigated and the domain of study. Many studies provided a poor or missing description of critical constructs that were being investigated. A second way is through content consistency, by which I mean using the same items on a questionnaire or structured interviews as other researchers. One strategy would be to use two separate self-report questionnaires with a common set of linking items. A third way is to use consistent administration procedures such as order of presentation, instructions provided to participants, amount of time to complete the assessments, and conditions under which the data is collected. A fourth way is through consistent use of scoring procedures. Indeed rubrics and scoring methods were poorly described in the research, either within a study or across studies using the same methods. Structured interviews typically differed and used very different coding schemes. For all of these reasons, it may be difficult to generalize across studies, even those using similar methodology and measurement strategies.


## 5. The Challenge of Conceptual Integration

This section considers six ways to promote better conceptual understanding of epistemological and ontological beliefs. These include developing a conceptual framework, distinguishing between tacit versus explicit beliefs, examining whether beliefs are domain specific versus domain general, examining in more detail the relationship between beliefs and learning, investigating how teachers’ beliefs are related to classroom practices, and tracking the development of teachers’ beliefs.


### 5.1. Develop an Integrated Conceptual Framework

Previous research has not provided an integrated conceptual framework for understanding epistemological and ontological beliefs. Hofer [41] and Hofer and Pintrich [12] presented a well-articulated conceptual framework for conducting epistemological research and raised important questions about the boundary between epistemology and ontology. Bendixen and Rule [66], Muis [105], and Schommer-Aikins [67] also provided models that link epistemological beliefs to other related constructs such as metacognition and self-regulation. However, with the exception of Packer and Goicoechea [39] and Ponterotto [8], there has not been a clear explication of ontological beliefs.I propose the framework shown in Figure2 as a preliminary guide for conceptualizing and investigating core areas of epistemology and ontology in future research. Figure 2 considers both epistemological and ontological beliefs under the general category of teacher and student beliefs, which might also include self-efficacy, goal orientation, and attributions beliefs [154, 155].Figure 2
Epistemological and ontological domains of inquiry.Three distinct categories of inquiry are relevant to understanding epistemological beliefs, including types of knowledge and ways of knowing, sources of knowledge and evidence, and justification of knowledge. Beliefs within each of these categories may differ in important ways [156]. Hofer and Pintrich [12] provided an in-depth discussion of these three questions a decade ago; however, the literature would benefit from an updated discussion based on research that has occurred over the last decade. Each of these categories could be partitioned further into more specific questions. For example, researchers could create a separate taxonomy of types of knowledge (e.g., [157]) and beliefs about different types of knowledge (e.g., [19]). A separate taxonomy could be created for ways of knowing (e.g., [158, 159]). Similarly, sources of knowledge and issues surrounding the justification of knowledge could be partitioned into finer grained subcategories as shown in Table 1.Two categories of inquiry are relevant to ontology, including beliefs about reality and beliefs about the nature of being [8, 39]. Beliefs about reality may include questions about permanence and evidential support for one or more perceived realities. Beliefs about personal being may include perceptions, beliefs, and attitudes teachers have about themselves as teachers, as well as the fundamental nature of schooling. As with epistemological categories, ontological categories could be partitioned into subcategories. For example, very little is known about teachers’ beliefs about the reality of the classroom. A codified taxonomy of these beliefs and their implications for classroom instruction would make a very important contribution. Similarly, researchers lack a comprehensive framework for understanding teachers’ sense of being teachers.Of equal importance to any conceptual framework of epistemology and ontology is what the two domains of belief have in common. Regarding education, virtually nothing has been written about the development and nature of teachers’ “metabeliefs,” by which I mean an integrated belief system that includes personal epistemology, ontology, self-concept, and other core beliefs that teachers bring to their classroom practice. I assume that similar experiences affect every aspect of a teacher’s beliefs. Unfortunately, researchers have little awareness of whether one set of beliefs (e.g., ontology) has a causal impact on other beliefs (e.g., epistemology), whether effects are fully reciprocal, or whether foundational beliefs such as epistemology and ontology are independent.


### 5.2. Tacit versus Explicit Beliefs

Students and teachers hold many beliefs, some of which are explicit, and some of which are more implicit. Little is known currently about the explicitness of epistemological and ontological beliefs, or whether explicitness affects reflection on beliefs and classroom decisions. Most researchers assume that these beliefs are at least partially implicit [12, 29, 66, 160].Of special interest is how implicit beliefs become explicit over time and what fosters this change. Pirttilä-Backman and Kajanne [64] found that Finnish adults’ beliefs became more explicit in large part due to continued education. This finding is consistent with Kuhn [16] and Baxter-Magolda [11]. A number of studies have reported epistemological change and greater explicitness awareness of one’s beliefs, due to classroom reflection between teachers and students [27], constructivist activities [51, 117], doubt [66, 161, 162], and cognitive conflict [68, 156, 163–165].


### 5.3. Domain-Specific versus Domain-General Beliefs

Researchers have addressed whether epistemological beliefs and world views are domain specific or domain general in nature. Some researchers argue that beliefs are domain specific [41]. On this view, one’s epistemological beliefs are inseparable from one’ domain-specific knowledge, and the beliefs that one holds in one domain are not related to beliefs in another domain, but especially when comparing familiar versus unfamiliar domains. In general, the domain-specific hypothesis is consistent with models of domain learning [166] and expertise [167] that suggest expertise is encapsulated within a domain and does not readily transfer to other domains.An alternative view is that epistemological beliefs tend to be domain general [16, 67]. On this view, epistemological beliefs develop as general dispositions about the world; thus, one’s assumptions about the origin and legitimacy of knowledge are likely to be similar in different domains. This view is consistent with model of domain general self-regulation [105, 168] and metacognition [78] that postulates higher-order skills such as planning and monitoring that generalize in skilled learners to all domains.Muis et al. [106] reviewed 19 empirical studies and concluded that beliefs are both domain specific and domain general. They proposed a model in which individuals hold domain specific beliefs that differ between domains, while also holding domain general beliefs that are more or less consistent across a unique culture. They also proposed a developmental sequence in which domain-specific beliefs may contribute to broader domain general beliefs.


### 5.4. Student Beliefs and Learning

Previous research has provided a great deal of information regarding the effect the epistemological beliefs have on student learning [105]. For example, epistemological beliefs are related to overall learning [2, 67, 169], deeper learning [26, 56, 86], evaluation of evidence [20], academic career choices [112], increased self-regulation [27, 104, 105, 170], and teacher-student interactions  [52]. In general, students and teachers with contextualist or relativist beliefs learn more and outperform others with realist beliefs [16, 25, 52, 67]. Similar findings have been reported with teachers, although fewer studies have been conducted compared to college students [55].Virtually all previous studies have focused on epistemological beliefs without considering ontological beliefs. One issue is whether the empirical relationships between ontological beliefs and learning outcomes are comparable to the effect of epistemological beliefs on learning. One possibility is that ontological beliefs, but especially teachers’ ontological beliefs about the reality of teaching, would play an instrumental role in curriculum and pedagogy decisions.


### 5.5. Teacher Beliefs and Classroom Practices

Few studies have examined teachers’ epistemological world views in detail, and none that we know of have examined ontological beliefs. It is possible in some cases that preservice training has little effect on preservice teachers’ world views, in part because younger students may not be developmentally ready to consider difficult to resolve epistemological and ontological issues [21, 24, 34, 51, 77, 155, 171]. However, it is more likely that the type of preservice training a student receives affect his or her beliefs differently [172, 173].Existing studies suggest that preservice teachers benefit from an explicit discussion of beliefs, especially those related to student-centered teaching [21, 54]. One of the most effective belief-change strategies is reflection on teaching practice through collaborative discussions and reflective journaling [32, 51, 114], in part, because it makes competing world views more public to preservice teachers [53]. These activities appear to give preservice teachers a greater awareness of their existing beliefs, as well as an opportunity to discuss alternative beliefs with other students and their teacher.


### 5.6. The Development of Teacher Beliefs

Previous research suggests that teachers develop over time regarding teaching practices and that their beliefs change as part of their development [54, 143, 155]. Very little is known currently regarding the development of teachers’ epistemological and ontological beliefs [65]. One issue is whether the two beliefs develop in tandem or follow separate trajectories. A second issue is the general trend in development [123, 141]. It seems reasonable to assume that beliefs become more relativist over time, but may become stable at some point, or perhaps move from a relativist to realist world views past a certain number of years of experience [174, 175]. Cross-sectional studies would help address these questions, although longitudinal studies would be especially useful for understanding the development of beliefs.One important need is a developmental model of belief development that includes a wide range of sociocultural factors [67]. Several questions seem especially germane, including the impact of different variables (e.g., family, education) on belief development, whether epistemological and ontological beliefs develop in synchrony, and whether there is a zone of readiness [51] that facilitates belief development. One requirement of such a model is that it is tested on epistemological and ontological beliefs. Such a model should explain change and development across the two domains of belief or demonstrate that the two domains differ with respect to their developmental trajectories.


## 5.1. Develop an Integrated Conceptual Framework

Previous research has not provided an integrated conceptual framework for understanding epistemological and ontological beliefs. Hofer [41] and Hofer and Pintrich [12] presented a well-articulated conceptual framework for conducting epistemological research and raised important questions about the boundary between epistemology and ontology. Bendixen and Rule [66], Muis [105], and Schommer-Aikins [67] also provided models that link epistemological beliefs to other related constructs such as metacognition and self-regulation. However, with the exception of Packer and Goicoechea [39] and Ponterotto [8], there has not been a clear explication of ontological beliefs.I propose the framework shown in Figure2 as a preliminary guide for conceptualizing and investigating core areas of epistemology and ontology in future research. Figure 2 considers both epistemological and ontological beliefs under the general category of teacher and student beliefs, which might also include self-efficacy, goal orientation, and attributions beliefs [154, 155].Figure 2
Epistemological and ontological domains of inquiry.Three distinct categories of inquiry are relevant to understanding epistemological beliefs, including types of knowledge and ways of knowing, sources of knowledge and evidence, and justification of knowledge. Beliefs within each of these categories may differ in important ways [156]. Hofer and Pintrich [12] provided an in-depth discussion of these three questions a decade ago; however, the literature would benefit from an updated discussion based on research that has occurred over the last decade. Each of these categories could be partitioned further into more specific questions. For example, researchers could create a separate taxonomy of types of knowledge (e.g., [157]) and beliefs about different types of knowledge (e.g., [19]). A separate taxonomy could be created for ways of knowing (e.g., [158, 159]). Similarly, sources of knowledge and issues surrounding the justification of knowledge could be partitioned into finer grained subcategories as shown in Table 1.Two categories of inquiry are relevant to ontology, including beliefs about reality and beliefs about the nature of being [8, 39]. Beliefs about reality may include questions about permanence and evidential support for one or more perceived realities. Beliefs about personal being may include perceptions, beliefs, and attitudes teachers have about themselves as teachers, as well as the fundamental nature of schooling. As with epistemological categories, ontological categories could be partitioned into subcategories. For example, very little is known about teachers’ beliefs about the reality of the classroom. A codified taxonomy of these beliefs and their implications for classroom instruction would make a very important contribution. Similarly, researchers lack a comprehensive framework for understanding teachers’ sense of being teachers.Of equal importance to any conceptual framework of epistemology and ontology is what the two domains of belief have in common. Regarding education, virtually nothing has been written about the development and nature of teachers’ “metabeliefs,” by which I mean an integrated belief system that includes personal epistemology, ontology, self-concept, and other core beliefs that teachers bring to their classroom practice. I assume that similar experiences affect every aspect of a teacher’s beliefs. Unfortunately, researchers have little awareness of whether one set of beliefs (e.g., ontology) has a causal impact on other beliefs (e.g., epistemology), whether effects are fully reciprocal, or whether foundational beliefs such as epistemology and ontology are independent.


## 5.2. Tacit versus Explicit Beliefs

Students and teachers hold many beliefs, some of which are explicit, and some of which are more implicit. Little is known currently about the explicitness of epistemological and ontological beliefs, or whether explicitness affects reflection on beliefs and classroom decisions. Most researchers assume that these beliefs are at least partially implicit [12, 29, 66, 160].Of special interest is how implicit beliefs become explicit over time and what fosters this change. Pirttilä-Backman and Kajanne [64] found that Finnish adults’ beliefs became more explicit in large part due to continued education. This finding is consistent with Kuhn [16] and Baxter-Magolda [11]. A number of studies have reported epistemological change and greater explicitness awareness of one’s beliefs, due to classroom reflection between teachers and students [27], constructivist activities [51, 117], doubt [66, 161, 162], and cognitive conflict [68, 156, 163–165].


## 5.3. Domain-Specific versus Domain-General Beliefs

Researchers have addressed whether epistemological beliefs and world views are domain specific or domain general in nature. Some researchers argue that beliefs are domain specific [41]. On this view, one’s epistemological beliefs are inseparable from one’ domain-specific knowledge, and the beliefs that one holds in one domain are not related to beliefs in another domain, but especially when comparing familiar versus unfamiliar domains. In general, the domain-specific hypothesis is consistent with models of domain learning [166] and expertise [167] that suggest expertise is encapsulated within a domain and does not readily transfer to other domains.An alternative view is that epistemological beliefs tend to be domain general [16, 67]. On this view, epistemological beliefs develop as general dispositions about the world; thus, one’s assumptions about the origin and legitimacy of knowledge are likely to be similar in different domains. This view is consistent with model of domain general self-regulation [105, 168] and metacognition [78] that postulates higher-order skills such as planning and monitoring that generalize in skilled learners to all domains.Muis et al. [106] reviewed 19 empirical studies and concluded that beliefs are both domain specific and domain general. They proposed a model in which individuals hold domain specific beliefs that differ between domains, while also holding domain general beliefs that are more or less consistent across a unique culture. They also proposed a developmental sequence in which domain-specific beliefs may contribute to broader domain general beliefs.


## 5.4. Student Beliefs and Learning

Previous research has provided a great deal of information regarding the effect the epistemological beliefs have on student learning [105]. For example, epistemological beliefs are related to overall learning [2, 67, 169], deeper learning [26, 56, 86], evaluation of evidence [20], academic career choices [112], increased self-regulation [27, 104, 105, 170], and teacher-student interactions  [52]. In general, students and teachers with contextualist or relativist beliefs learn more and outperform others with realist beliefs [16, 25, 52, 67]. Similar findings have been reported with teachers, although fewer studies have been conducted compared to college students [55].Virtually all previous studies have focused on epistemological beliefs without considering ontological beliefs. One issue is whether the empirical relationships between ontological beliefs and learning outcomes are comparable to the effect of epistemological beliefs on learning. One possibility is that ontological beliefs, but especially teachers’ ontological beliefs about the reality of teaching, would play an instrumental role in curriculum and pedagogy decisions.


## 5.5. Teacher Beliefs and Classroom Practices

Few studies have examined teachers’ epistemological world views in detail, and none that we know of have examined ontological beliefs. It is possible in some cases that preservice training has little effect on preservice teachers’ world views, in part because younger students may not be developmentally ready to consider difficult to resolve epistemological and ontological issues [21, 24, 34, 51, 77, 155, 171]. However, it is more likely that the type of preservice training a student receives affect his or her beliefs differently [172, 173].Existing studies suggest that preservice teachers benefit from an explicit discussion of beliefs, especially those related to student-centered teaching [21, 54]. One of the most effective belief-change strategies is reflection on teaching practice through collaborative discussions and reflective journaling [32, 51, 114], in part, because it makes competing world views more public to preservice teachers [53]. These activities appear to give preservice teachers a greater awareness of their existing beliefs, as well as an opportunity to discuss alternative beliefs with other students and their teacher.


## 5.6. The Development of Teacher Beliefs

Previous research suggests that teachers develop over time regarding teaching practices and that their beliefs change as part of their development [54, 143, 155]. Very little is known currently regarding the development of teachers’ epistemological and ontological beliefs [65]. One issue is whether the two beliefs develop in tandem or follow separate trajectories. A second issue is the general trend in development [123, 141]. It seems reasonable to assume that beliefs become more relativist over time, but may become stable at some point, or perhaps move from a relativist to realist world views past a certain number of years of experience [174, 175]. Cross-sectional studies would help address these questions, although longitudinal studies would be especially useful for understanding the development of beliefs.One important need is a developmental model of belief development that includes a wide range of sociocultural factors [67]. Several questions seem especially germane, including the impact of different variables (e.g., family, education) on belief development, whether epistemological and ontological beliefs develop in synchrony, and whether there is a zone of readiness [51] that facilitates belief development. One requirement of such a model is that it is tested on epistemological and ontological beliefs. Such a model should explain change and development across the two domains of belief or demonstrate that the two domains differ with respect to their developmental trajectories.


## 6. The Challenge of Improving Measurement

Most research described in this paper is based on the use of questionnaires to assess epistemological beliefs [12, 44, 67, 105, 106, 132, 150]. I make a variety of suggestions in the following for improving the measurement of beliefs, including the development of new instruments, utilizing new methods, improving reliability of assessments, and providing relevant validity evidence.


### 6.1. Develop New Instruments

Most research utilizes self-report questionnaires of epistemological beliefs. This situation could be improved by developing a questionnaire to assess ontological beliefs that assess beliefs about reality and beliefs about personal being as suggested in Figure2. This would enable researchers to assess both epistemological and ontological beliefs simultaneously from the same individuals. Researchers also should standardize other assessments such as interviews to compare epistemological and ontological beliefs within and between individuals. Specifically, interviews could use a common format and a common scoring rubric to more effectively compare results.


### 6.2. Utilize New Methods

Most studies underutilize the type of measurements used to study beliefs. Researchers should explore the use of concept maps [20, 28, 57], multidimensional scaling [58], Vignettes [53, 118], and essays, journals, and storyboards [54–56]. A variety of other methodologies could be employed as well such as Q-sorts [176, 177] in which teachers prioritized beliefs and beeper studies [178] in which individuals are stopped while engaged in vivo in real-time activities in order to report beliefs and ongoing measurable activities. Using a semantic differential tool like that used in Stahl and Bromme [48] provides a quick and easy data-collection strategy. Diaries, journals, videos, and portfolios [179] also have been underutilized in beliefs research in a manner that eliminates “context-stripping” from the interpretation of actions and teacher choices in actual classroom contexts.


### 6.3. Improve Reliability

The reliability of instruments used to assess beliefs often is low [44]. This could be improved by additional instrument development work that provides better construct definitions and additional assessment items [152]. Reliability could be improved as well as standardizing interview formats and rubrics used to score interviews [4].Another way to improve reliability and interpretability is to use auniversal measurement design across the five subdomains shown in Figure 2. Ideally, the same data collection method (e.g., interviews, concept maps) could be used to assess epistemological and ontological beliefs at the same time. This would enable researchers to compare beliefs in a more reliable manner.A third strategy is to use statistical techniques that partition difference types of variation among variables. For example, structural equation modeling enables researchers to simultaneously estimate the error variance in a variable (i.e., variable 1) and the variance that is shared between variable 1 and variable 2. Unfortunately, few studies have used statistical techniques to compare either the similarity between parallel measures of beliefs [84] or to model the measurement error of individual variables in a complex structural model.


### 6.4. Provide Relevant Validity Evidence

It is ironic that research on epistemological and ontological beliefs rarely discusses that extent to which constructs are clearly defined and assessed in an appropriate manner. Even a simple three-step approach can improve this situation immensely. The first step is to describe the organizational framework and dimensionality of beliefs in a manner consistent with Figure2. Doing so enables the researcher to describe the number of constructs being measured. A second step is to provide clear definitions of the constructs of interest such as those in Table 1. A third step is to describe theoretically the relationships among the variables in step 1; that is, how variables x, y, and z are related to one another. Ideally, this should be done using a structural model that illustrates the direction and strength of the predicted relationships between variables.Validity would be improved if researchers used multiple measures to gather convergent evidence of each type of targeted belief. I encourage researchers to do so to develop what has been known historically as anomothetic net [9] that integrates theory and measurement into a single conceptual framework that can be evaluated using a multiple-trait, multiple-measure validation strategy that uses multiple traits (e.g., strategy use, planning, monitoring, interest, and self-efficacy) and multiple methods (e.g., self-report and think-alouds). For example, on-line strategy use could be evaluated based on think-aloud reports, eye tracking data that supports or fails to support intended strategy use described before the study, interactions with tutors, as well as palette choices made in real time. Using multiple measures and methods should enhance the interpretative validity of conclusions by combining and synthesizing multiple sources of data and allowing researchers to examine the concurrent validity (i.e., real-time relationships) among different measures.


## 6.1. Develop New Instruments

Most research utilizes self-report questionnaires of epistemological beliefs. This situation could be improved by developing a questionnaire to assess ontological beliefs that assess beliefs about reality and beliefs about personal being as suggested in Figure2. This would enable researchers to assess both epistemological and ontological beliefs simultaneously from the same individuals. Researchers also should standardize other assessments such as interviews to compare epistemological and ontological beliefs within and between individuals. Specifically, interviews could use a common format and a common scoring rubric to more effectively compare results.


## 6.2. Utilize New Methods

Most studies underutilize the type of measurements used to study beliefs. Researchers should explore the use of concept maps [20, 28, 57], multidimensional scaling [58], Vignettes [53, 118], and essays, journals, and storyboards [54–56]. A variety of other methodologies could be employed as well such as Q-sorts [176, 177] in which teachers prioritized beliefs and beeper studies [178] in which individuals are stopped while engaged in vivo in real-time activities in order to report beliefs and ongoing measurable activities. Using a semantic differential tool like that used in Stahl and Bromme [48] provides a quick and easy data-collection strategy. Diaries, journals, videos, and portfolios [179] also have been underutilized in beliefs research in a manner that eliminates “context-stripping” from the interpretation of actions and teacher choices in actual classroom contexts.


## 6.3. Improve Reliability

The reliability of instruments used to assess beliefs often is low [44]. This could be improved by additional instrument development work that provides better construct definitions and additional assessment items [152]. Reliability could be improved as well as standardizing interview formats and rubrics used to score interviews [4].Another way to improve reliability and interpretability is to use auniversal measurement design across the five subdomains shown in Figure 2. Ideally, the same data collection method (e.g., interviews, concept maps) could be used to assess epistemological and ontological beliefs at the same time. This would enable researchers to compare beliefs in a more reliable manner.A third strategy is to use statistical techniques that partition difference types of variation among variables. For example, structural equation modeling enables researchers to simultaneously estimate the error variance in a variable (i.e., variable 1) and the variance that is shared between variable 1 and variable 2. Unfortunately, few studies have used statistical techniques to compare either the similarity between parallel measures of beliefs [84] or to model the measurement error of individual variables in a complex structural model.


## 6.4. Provide Relevant Validity Evidence

It is ironic that research on epistemological and ontological beliefs rarely discusses that extent to which constructs are clearly defined and assessed in an appropriate manner. Even a simple three-step approach can improve this situation immensely. The first step is to describe the organizational framework and dimensionality of beliefs in a manner consistent with Figure2. Doing so enables the researcher to describe the number of constructs being measured. A second step is to provide clear definitions of the constructs of interest such as those in Table 1. A third step is to describe theoretically the relationships among the variables in step 1; that is, how variables x, y, and z are related to one another. Ideally, this should be done using a structural model that illustrates the direction and strength of the predicted relationships between variables.Validity would be improved if researchers used multiple measures to gather convergent evidence of each type of targeted belief. I encourage researchers to do so to develop what has been known historically as anomothetic net [9] that integrates theory and measurement into a single conceptual framework that can be evaluated using a multiple-trait, multiple-measure validation strategy that uses multiple traits (e.g., strategy use, planning, monitoring, interest, and self-efficacy) and multiple methods (e.g., self-report and think-alouds). For example, on-line strategy use could be evaluated based on think-aloud reports, eye tracking data that supports or fails to support intended strategy use described before the study, interactions with tutors, as well as palette choices made in real time. Using multiple measures and methods should enhance the interpretative validity of conclusions by combining and synthesizing multiple sources of data and allowing researchers to examine the concurrent validity (i.e., real-time relationships) among different measures.


## 7. Summary

This paper addressed three main goals. The first was to provide definitions for terms frequently encountered in epistemological and ontological research. Table1 provides detailed definitions and appropriate sources. The second goal was to review existing measurement strategies. Table 3 compared and contrasted six strategies in current use. A third goal was to elaborate on challenges of conceptual integration for future research. Figure 2 provided an integrated conceptual framework.Six themes emerged from this work. One was the lack of clear, universally shared definitions of epistemological and ontological phenomena. Table1 provided definitions to resolve this problem. A second theme was that educational researchers have all but ignored the role of ontological beliefs in classroom practice, focusing instead on epistemological beliefs. I argued that epistemological and ontological beliefs differ from one another and must be taken into consideration collectively. A third theme was that researchers have underutilized assessment strategies, relying primarily on questionnaires and interviews. Several additional suggestions were made, including the use of Q-sorts, beepers, journals, and portfolios to augment existing measurements. A fourth theme was that a unified theoretical framework is needed to advance theory and research. I proposed a framework in Figure 2 consistent with the existing literature. A fifth theme was that future research should consider the roles of epistemological and ontological beliefs, booth collectively and separately. It is unclear currently whether either type of belief is more important in terms of classroom practice and develops in a similar manner. A sixth theme was that better measurement in the form of new and more valid measures would advance our understanding of beliefs. I summarized four ways that researchers may do so in the future.



---

*Source: 327680-2013-01-29.xml*