# Construct Validity of Assessing Interest in STEM Content Scale

Authors: Sanit Srikoon; Ronnachit Apaivatin; Piyapong Monsang; Sujinta Khamngoen; Thararat Malaitao
Journal: Education Research International
 (2020)

---

## Abstract

The purpose of this research was to validate the assessing interest in STEM content scale. The sample in this research was 552 students in upper secondary school at Fangchanupathum School, Chiang Mai, Thailand, who validated the assessing interest in STEM content scale, which used a rating scale of four factors consisting of (1) science, (2) mathematics, (3) laboratorial skills, and (4) technology and 20 items. Confirmatory factor analysis was used to confirm the construct validity of the assessing interest in STEM content scale adapted from the study by Tyler-Wood, Knezek, and Christensen. The results confirmed that the construct validity of this assessing interest in STEM content scale had an excellent fit. They showed that the fitness index of validating the assessing interest in STEM content scale wasX2 statistic of 163.679 (degree of freedom = 141, P value = 0.0928) and the X2/df ratio having a value of 1.160 indicating a good fit. The comparative fit index (CFI) was 0.991, and Tucker–Lewis coefficient (TLI) was 0.988. The root mean square error approximation (RMSEA) was 0.017. The standardized root mean residual (SRMR) was 0.047. All the indicators indicated that there was a good fit between the empirical data and the hypothetical measurement model.


---

## Body

## 1. Introduction

In recent years, STEM education has been a learning and teaching approach, integrating science, technology, engineering, and mathematics knowledge and skills [1]. This interdisciplinary field of study both supports and provides a positive contribution to students’ skills in the 21st century such as STEM literacy, problem solving, critical thinking, and creativity [2, 3]. In addition, STEM education is able to help students develop their research-questioning, logical reasoning and working behaviours in collaborative ways and let the students find solutions among simple or complex problems, local or global problems, and real-life situations. Hence, STEM education uses 21st century skills by assisting students to adapt or adjust themselves by applying their new knowledge through their perceived experience to new situations [4].The interest does not only mean that if someone has an interest in learning about something it becomes easy and even enjoyable to learn about that topic seeing that the students’ interest is the primary influence on perceived learning [5]. To know students’ interest in STEM education is vital for teachers to create a learning environment which suits students perfectly. Using an accurate assessment of interest in STEM education, we are able to identify each of the subject areas which are science, technology, engineering, and mathematics; thus, the teachers can use the results to assess students’ interest to design and fulfill their classroom activities or students’ learning experience. This also confirms the expectation that students learn more in environments in which instructional methods are congruent with their preferences [6].Because of the shortage of the assessing interest in STEM content scale in Thailand and the loss of interest in STEM field, this was the most frequently cited reason that students gave for switching out of STEM education [7]. It was essential to develop the instrument to assess the students’ interest in STEM content; therefore, the researchers studied the appropriate instrument for assessing interest in STEM content adapted from the study by Tyler-Wood et al. which was analysed and it was shown that it had excellent internal consistency reliability as well as good content, construct, and criterion-related validity [8]. In addition, the researchers adjusted the instrument to assess the students’ interest in STEM content based on the educational context of Thailand. So to know how well the measured variables represent the construct validation of this questionnaire scale, the researchers use the confirmatory factor analysis (CFA) to examine the reliability and validity and verify the data reliability of this instrument. The confirmatory factor analysis (CFA) provides a standardized loading factor of each indicator where the research variable is quantified from a series of statements known as manifest variable. The standardized loading factor indicates the contribution of each indicator to the respective research latent variable. This model or hypothesis specifies which variables will be correlated with which factors and which factors are correlated. The hypothesis is based on a strong theoretical and/or empirical foundation [9].The adapted assessing interest in STEM content scale had four factors incorporated as descriptors for target statements reflecting perceptions of the students’ interest in science, mathematics, laboratorial skill, and technology separately. Each of the four scales consists of five semantic perception adjective pairs used on this assessing which represented STEM interests. The purpose of this study was to test whether the factor structure of the assessing interest in STEM content scale could be reiterated as an independent sample using confirmatory factor analysis (CFA) with maximum likelihood estimates following the conceptual framework in Figure1.Figure 1
The conceptual framework.


## 2. Literature Review

STEM (science, technology, engineering, and mathematics) education is a modern educational approach that endeavours to train students to develop high problem-solving skills and improve their critical thinking as well as fostering collaborative working skills to establish a connection between real-life problems and the integrated contents. In place of learning the disciplines involved separately, STEM education focuses on skills such as problem solving, research, effective communication, and design [10]. It also aims for students to look at problems with some perspective in connection with other fields and acquire current information skills with an educational approach holistically [11]. It can contribute to students’ sustainable growth by improving their skills in science and technology.To enhance students’ performance, teachers have to know their interests so as to motivate them. The interest was defined as relatively stable preferences that focused on objects, activities, or experience [12]. Consequently, the interest in STEM education was satisfaction, performance, and persistence incorporated within STEM education. If they feel better about learning in STEM education, they will be more successful and more productive. The students have worked on problem solving activities in groups gaining more freedom and flexibility, and these activities can lead students to think about STEM disciplines and allow them to develop skills and collaborate for lifelong learning. As a result of their studies, these activities help them develop their skills and learn from one another and develop their interest towards STEM education [4]. Moreover, the interest in STEM education directs effort towards learning in a domain and correlates with a multitude of academic and occupational outcomes, including course selection, achievement, and persistence in a given field of study or career [13]. Hence, the interest in STEM content can be assessed by using the affective judgment related to how much an individual likes the domain and the tasks one enjoys.Getting the right students’ information by using the proper assessing tools is the most important thing for all teachers to know the students’ interest upon their perceptions in (1) science, (2) mathematics, (3) laboratorial skills, and (4) technology, and it can result in students’ successful outcome. To earn the accurate students’ information using the construct validity which is generally viewed as a unifying form of validity for psychological measurements, one should subsume both content and criterion validity. The construct validity had been traditionally treated as distinct forms of validity [14]. Due to the fact that this method can be said to be well-accepted to accumulate the evidence by taking a unified definition of construct validity, using content analysis, correlation coefficients, factor analysis, and ANOVA, the researchers are able to demonstrate the differences between groups or pretest and posttest intervention studies and multitrait/multimethod studies and so on. [15]. Thereby, the construct validity is an experimental demonstration that the questionnaire uses to measure the construct it claims to be measuring. It can reveal that the questionnaire is a representative sample of the content of either objectives or specifications it was originally designed to measure. In order to test all the variables of the questionnaire, the researchers should use the appropriate analysis to investigate whether the questionnaire used has the construct validity. The confirmatory factor analysis (CFA) is a statistical procedure for examining the hypotheses about the commonality among the variables. As a multivariate procedure, the confirmatory factor analysis (CFA) is used to simultaneously test multiple hypotheses that collectively constitute a measurement model [16]. As a result, it attempts to validate hypothesized factors that the researchers expect to emerge from data collected from test questions or questionnaire responses. This method can be used to validate the questionnaire of the assessing interest in STEM content scale.


## 3. Objective

This research was to validate the assessing interest in STEM content scale adapted from the study by Tyler-Wood et al. [8] by using confirmatory factor analysis (CFA).


## 4. Research Design

### 4.1. The Sample Group

The sample group was 552 students in upper secondary at Fangchanupathum School, Chiang Mai, Thailand. The purposive sampling was used in this research. It consisted of 148 grade 10 students (26.81%), 168 grade 11 students (30.43%). and 236 grade 12 students (42.75%). It could be divided into two programs of study which included 203 students being in a science and mathematics program (36.77%) and 349 students being in an arts (languages, social studies, arts, or physical education) program (63.23%).


### 4.2. Variable

The variables used were the students’ interest in STEM content consisting of reflecting perceptions appearing while the students were studying (1) science, (2) mathematics, (3) laboratorial skills, and (4) technology and whether they had an interest in STEM education which was assessed through their satisfaction, performance, and persistence.


### 4.3. Research Instrument

The instrument used in the research was the questionnaire with five rating scales divided into four sections having five questions for each section following the four variables which were adapted from the study by Tyler-Wood et al. [8]. The first section was about the reflecting perceptions of students while studying science. The second section was about the reflecting perceptions of students during mathematics class. The third section was about students’ reflecting perceptions when they were doing experiments in the laboratory. The last section was about students’ reflecting perceptions as they were studying computer studies or technology. The Index of Item-Objective Congruence (IOC) was used to find the content validity. This questionnaire was checked by three experts. Obviously, the results showed that all items in this questionnaire received the value of +1 which meant that this questionnaire was congruent. The assessing interest in STEM content scale is shown in Table 1.Table 1
The assessing interest in the stem content scale.
Section1: while studying science, I feel1Attracted①②③④⑤⑥⑦Unattracted2Appealed①②③④⑤⑥⑦Unappealed3Excited①②③④⑤⑥⑦Unexcited4Depressed①②③④⑤⑥⑦Delighted5Bored①②③④⑤⑥⑦InterestedSection2: while studying mathematics, I feel1Unattracted①②③④⑤⑥⑦Attracted2Unappealed①②③④⑤⑥⑦Appealed3Excited①②③④⑤⑥⑦Unexcited4Depressed①②③④⑤⑥⑦Delighted5Bored①②③④⑤⑥⑦InterestedSection3: while doing the experiment in the laboratory, I feel1Unattracted①②③④⑤⑥⑦Attracted2Unappealed①②③④⑤⑥⑦Appealed3Unexcited①②③④⑤⑥⑦Excited4Depressed①②③④⑤⑥⑦Delighted5Bored①②③④⑤⑥⑦InterestedSection4: while studying computer studies/technology, I feel1Attracted①②③④⑤⑥⑦Unattracted2Appealed①②③④⑤⑥⑦Unappealed3Unexcited①②③④⑤⑥⑦Excited4Delighted①②③④⑤⑥⑦Depressed5Interested①②③④⑤⑥⑦BoredTable 2
The assumptions in statistics of the four factors.
Factorx̅SDSkewnessKurtosisScienceS13.9601.5640.106−0.415S24.2461.5040.070−0.564S33.6611.5770.250−0.501S44.3171.582-0.219−0.554S54.2441.617-0.161−0.666MathematicsM13.8291.6520.000−0.736M23.6631.5160.066−0.522M33.9321.6040.110−0.589M43.9581.611−0.162−0.516M53.9031.669−0.021−0.688Laboratorial skillsL14.9811.609−0.573−0.364L24.6081.521−0.381−0.303L35.0051.620−0.561−0.431L44.8851.666−0.451−0.547L54.8911.694−0.510−0.529TechnologyT13.1431.7420.506−0.570T23.3841.5510.414−0.320T34.5881.560−0.476−0.162T43.1591.5900.386−0.391T53.1251.6030.360−0.520In Table2, the assumptions in statistics of the four factors of the assessing interest in STEM content scale showed that the mean (x̅) was from 3.125 to 5.005, the deviation (SD) was from 1.504 to 1.694, the skewness was from −0.021 to 0.506, and the kurtosis was from −0.162 to −0.736.(2)
The results of the examination of the goodness of fit index of confirmatory factor analysis (CFA) in the assessing interest in STEM content scale showed the following:(2.1)
The chi-square:X2 was 163.679 (degree of freedom = 141, P value = 0.0928), which meant that the model fit indices are based on the empirical data [18],(2.2)
TheX2/df ratio had a value of 1.160 being in the ratio of 2 : 1, which indicates that the model of assessing interest in STEM content scale had a good fit [19],(2.3)
The comparative fit index (CFI) was 0.991; the value was above 0.95 indicated that the model of assessing interest in STEM content scale was correlated with the factors [20],(2.4)
Tucker–Lewis coefficient (TLI) was 0.988; the value was above 0.95 indicated that the model of assessing interest in STEM content scale was correlated with the factors [20],(2.5)
The root mean square error approximation (RMSEA) was 0.017 which was less than 0.03, and it indicated that the model of assessing interest in STEM content scale was correlated with the factors [20],(2.6)
The standardized root mean residual (SRMR) was 0.047 which was less than 0.08, and it indicated that the model of assessing interest in STEM content scale was correlated with the factors [20].


### 4.4. Data Collection

The data was collected as follows:(1)
Asking permission to survey the students’ interest in STEM content by using the qualified assessing interest in STEM content scale from the school director and planning to collect the data together(2)
Administering to assist students to understand the research objective, letting a total of 552 students at Fangchanupathum School, Chiang Mai, Thailand, realize how important this questionnaire was and answering it truly(3)
Collecting the data following the scheduled plan during 8–12 July, 2019


### 4.5. Data Analysis

This research was to examine the construct validity of the assessing interest in STEM content scale in order to examine the item objective congruence and whether it was a good fit using the confirmatory factor analysis (CFA) by MPlus program [17].


## 4.1. The Sample Group

The sample group was 552 students in upper secondary at Fangchanupathum School, Chiang Mai, Thailand. The purposive sampling was used in this research. It consisted of 148 grade 10 students (26.81%), 168 grade 11 students (30.43%). and 236 grade 12 students (42.75%). It could be divided into two programs of study which included 203 students being in a science and mathematics program (36.77%) and 349 students being in an arts (languages, social studies, arts, or physical education) program (63.23%).


## 4.2. Variable

The variables used were the students’ interest in STEM content consisting of reflecting perceptions appearing while the students were studying (1) science, (2) mathematics, (3) laboratorial skills, and (4) technology and whether they had an interest in STEM education which was assessed through their satisfaction, performance, and persistence.


## 4.3. Research Instrument

The instrument used in the research was the questionnaire with five rating scales divided into four sections having five questions for each section following the four variables which were adapted from the study by Tyler-Wood et al. [8]. The first section was about the reflecting perceptions of students while studying science. The second section was about the reflecting perceptions of students during mathematics class. The third section was about students’ reflecting perceptions when they were doing experiments in the laboratory. The last section was about students’ reflecting perceptions as they were studying computer studies or technology. The Index of Item-Objective Congruence (IOC) was used to find the content validity. This questionnaire was checked by three experts. Obviously, the results showed that all items in this questionnaire received the value of +1 which meant that this questionnaire was congruent. The assessing interest in STEM content scale is shown in Table 1.Table 1
The assessing interest in the stem content scale.
Section1: while studying science, I feel1Attracted①②③④⑤⑥⑦Unattracted2Appealed①②③④⑤⑥⑦Unappealed3Excited①②③④⑤⑥⑦Unexcited4Depressed①②③④⑤⑥⑦Delighted5Bored①②③④⑤⑥⑦InterestedSection2: while studying mathematics, I feel1Unattracted①②③④⑤⑥⑦Attracted2Unappealed①②③④⑤⑥⑦Appealed3Excited①②③④⑤⑥⑦Unexcited4Depressed①②③④⑤⑥⑦Delighted5Bored①②③④⑤⑥⑦InterestedSection3: while doing the experiment in the laboratory, I feel1Unattracted①②③④⑤⑥⑦Attracted2Unappealed①②③④⑤⑥⑦Appealed3Unexcited①②③④⑤⑥⑦Excited4Depressed①②③④⑤⑥⑦Delighted5Bored①②③④⑤⑥⑦InterestedSection4: while studying computer studies/technology, I feel1Attracted①②③④⑤⑥⑦Unattracted2Appealed①②③④⑤⑥⑦Unappealed3Unexcited①②③④⑤⑥⑦Excited4Delighted①②③④⑤⑥⑦Depressed5Interested①②③④⑤⑥⑦BoredTable 2
The assumptions in statistics of the four factors.
Factorx̅SDSkewnessKurtosisScienceS13.9601.5640.106−0.415S24.2461.5040.070−0.564S33.6611.5770.250−0.501S44.3171.582-0.219−0.554S54.2441.617-0.161−0.666MathematicsM13.8291.6520.000−0.736M23.6631.5160.066−0.522M33.9321.6040.110−0.589M43.9581.611−0.162−0.516M53.9031.669−0.021−0.688Laboratorial skillsL14.9811.609−0.573−0.364L24.6081.521−0.381−0.303L35.0051.620−0.561−0.431L44.8851.666−0.451−0.547L54.8911.694−0.510−0.529TechnologyT13.1431.7420.506−0.570T23.3841.5510.414−0.320T34.5881.560−0.476−0.162T43.1591.5900.386−0.391T53.1251.6030.360−0.520In Table2, the assumptions in statistics of the four factors of the assessing interest in STEM content scale showed that the mean (x̅) was from 3.125 to 5.005, the deviation (SD) was from 1.504 to 1.694, the skewness was from −0.021 to 0.506, and the kurtosis was from −0.162 to −0.736.(2)
The results of the examination of the goodness of fit index of confirmatory factor analysis (CFA) in the assessing interest in STEM content scale showed the following:(2.1)
The chi-square:X2 was 163.679 (degree of freedom = 141, P value = 0.0928), which meant that the model fit indices are based on the empirical data [18],(2.2)
TheX2/df ratio had a value of 1.160 being in the ratio of 2 : 1, which indicates that the model of assessing interest in STEM content scale had a good fit [19],(2.3)
The comparative fit index (CFI) was 0.991; the value was above 0.95 indicated that the model of assessing interest in STEM content scale was correlated with the factors [20],(2.4)
Tucker–Lewis coefficient (TLI) was 0.988; the value was above 0.95 indicated that the model of assessing interest in STEM content scale was correlated with the factors [20],(2.5)
The root mean square error approximation (RMSEA) was 0.017 which was less than 0.03, and it indicated that the model of assessing interest in STEM content scale was correlated with the factors [20],(2.6)
The standardized root mean residual (SRMR) was 0.047 which was less than 0.08, and it indicated that the model of assessing interest in STEM content scale was correlated with the factors [20].


## 4.4. Data Collection

The data was collected as follows:(1)
Asking permission to survey the students’ interest in STEM content by using the qualified assessing interest in STEM content scale from the school director and planning to collect the data together(2)
Administering to assist students to understand the research objective, letting a total of 552 students at Fangchanupathum School, Chiang Mai, Thailand, realize how important this questionnaire was and answering it truly(3)
Collecting the data following the scheduled plan during 8–12 July, 2019


## 4.5. Data Analysis

This research was to examine the construct validity of the assessing interest in STEM content scale in order to examine the item objective congruence and whether it was a good fit using the confirmatory factor analysis (CFA) by MPlus program [17].


## 5. Research Result

The results of the data analysis showed the following:(1)
The results of the assumptions in statistics of the four factors were obtained using 20 questions which involved students’ interest in STEM content through reflecting perceptions in (1) science (five items), (2) mathematics (five items), (3) laboratorial skills (five items), and (4) technology (five items). The results are shown in Table2.Hence, these results confirmed that the construct validity of this assessing interest in STEM content scale had an excellent fit. The detailed data analysis result is shown in Table3.Table 3
Result of the confirmatory factor analysis (CFA) of the assessing interest in the stem content scale.
Empirical factorFactor loadingβFSR2BSEtFirst-order CFAScienceS11.0000.000<—>0.3180.0310.101∗∗S20.4890.1533.200∗∗0.162-0.0350.026∗∗S31.4580.2356.212∗∗0.4600.0760.212∗∗S41.7220.3974.340∗∗0.5410.0630.293∗∗S51.6600.3784.396∗∗0.5110.0520.261∗∗MathematicsM11.0000.000<—>0.7060.0780.498∗∗M20.8280.04418.712∗∗0.6390.0380.408∗∗M30.0950.0791.209∗∗0.0690.0010.005∗∗M41.2050.06917.516∗∗0.8720.2690.761∗∗M51.2810.06719.003∗∗0.9020.3660.814∗∗Laboratorial skillsL11.0000.000<—>0.7560.0510.572∗∗L20.9510.04322.348∗∗0.7600.1470.578∗∗L31.1410.06118.706∗∗0.8700.3110.758∗∗L41.1220.06517.233∗∗0.8200.1460.673∗∗L51.1270.06716.886∗∗0.8110.1370.658∗∗TechnologyT11.0000.000<—>0.7000.0750.491∗∗T20.8360.04120.181∗∗0.6580.0530.433∗∗T30.4300.0676.405∗∗0.3360.0330.113∗∗T41.1230.06118.567∗∗0.8620.2800.743∗∗T51.1760.06817.273∗∗0.8960.3750.803∗∗Second-order CFAInterest ⟶ science1.0000.000<—>0.1690.1550.112∗∗Interest ⟶ mathematics0.7800.2383.277∗∗0.3890.0270.152∗∗Interest ⟶ laboratorial skills1.0900.3393.214∗∗0.5200.0140.270∗∗Interest ⟶ technology0.3050.1551.975∗∗0.1450.0040.105∗∗∗∗p < 0.01, <—> not reporting SE and t since they were the constrained parameter.Table3 of the first-order confirmatory factor analysis model of the assessing interest in STEM content scale shows that the estimate (B) of 20 items was from 0.095 to 1.722, the standard error (SE) was from 0.00 to 0.397, the standard error of estimate (T) was from 1.209 to 22.348, the estimate of STDYX standardization (β) was from 0.069 to 0.902, the factor score information (FS) was from −0.035 to 0.375, and the R-square (R2) was from 0.005 to 0.814. The second-order confirmatory factor analysis showed that the estimate (B) of four factors was from 0.305 to 1.090, the standard error (SE) was from 0.00 to 0.339, the standard error of estimate (T) was from 1.975 to 3.277, the estimate of STDYX standardization (β) was from 0.145 to 0.520, the factor score information (FS) was from 0.004 to 0.155, and the R-square (R2) was from 0.105 to 0.270.


## 6. Conclusion and Discussion

The research of confirmatory factor model analysis of the assessing interest in STEM content scale adjusted from Tyler-Wood et al. [8] appropriate to the educational context in Thailand consisted of four factors composing of the reflecting perceptions in (1) science, (2) mathematics, (3) laboratorial skills, and (4) technology because most of Thai schools generally focus on these four disciplines and teach the engineering skill through doing the projects or experiments in the laboratory. The results of the examining the goodness of fit index of confirmatory factor analysis (CFA) found that the model fit indexes based on the empirical data were X2 = 163.679; df = 141; P value = 0.0928; X2/df ratio = 1.160; CFI = 0.991; TLI = 0.988; RMSEA = 0.017; SRMR = 0.047. These represented that this developed assessing interest in STEM content scale can be used to measure students’ interest in STEM content. It was related to the questionnaire of Tyler-Wood et al. [8] using the Exploratory Factor Analysis (Principle Component Extraction, Varimax Rotation, and Suppressed Display of Loadings were less than 0.5) which indicated that in every case the items loaded on the hypothesized factors meant that this questionnaire was most strongly associated with intended construct in every case. Consequently, the teachers are able to use the results of the students’ interest in STEM content to create or develop their classroom activities properly to carry out the goal of 21st century learning skills.Frequently cited reasons for inspiring students to enjoy STEM content were good teaching that involved capable teachers who could create splendid learning activities [21–23]. Therefore, the teachers have to create an exceptional learning environment not only focusing on students’ achievement; otherwise, the students might become turned off by STEM subjects due to the relative difficulty of receiving high grades in STEM classes in comparison to other subjects, and the students became often discouraged from taking STEM subjects. Hence, the students’ emphasis on achieving high grades outweighs their interest in taking subjects related to science, mathematics, technology, and engineering and result in them abandoning their interest in STEM [24]. Thus, this assessing interest in STEM content scale is able to be the accurate tool that the teachers could use to assist them to know the students’ interest and find out ways to encourage both their interest and attitudes towards STEM content and boost their classroom activities. The teachers know clearly which factors in relation to STEM content should be promoted and used to enhance students’ interest [25]. The student’s performance is an essential part of their success, and teachers must consider how to best help them achieve that success by determining what interests or motivates them to live in modern society based on the goal of education in the 21st century which is the mastery of information, engaged knowledge, and understanding and the advanced use of technology to solve complex problems they will face in their real lives.In conclusion, the teachers can use this questionnaire in their courses which are related to STEM content in order to evaluate the students’ interests based on the educational context in Thailand. Furthermore, it is better to adjust the questionnaire further to assess both the students’ interests and attitudes towards STEM content.



---

*Source: 1696494-2020-09-16.xml*