Analysis of Influential Factors on Computer Literacy Course Evaluation

The purpose of this paper is to examine the factors that influence the students’ satisfaction of college’s computer literacy courses. Covariance structure analysis was used to analyze the course evaluate data by students. Following six points should be concluded from the results of the analysis: (1) improving course understanding will enhance students’ satisfaction. (2) “Instructional Efforts” improve “Satisfaction” indirectly through “Factors of Students” and “Understanding”. (3) “Communion” improves “Satisfaction” indirectly through “Degree of Understanding” (4) students with no absences are influenced more by faculties than ones with absences to increase their satisfaction. (5) Total effects of “Instructional Efforts” in the “No absences group” is much stronger than those in “with absences” (6) Effect of “Communication” through “Degree of Understanding” is the strongest of all effects in both groups. Introduction The concern with Information Education has been growing for the last several years in Japan. To become a teacher it is necessary to acquire a teaching certificate. MEXT (Ministry of Education, Culture, Sports, Science and Technology) has revised the “Teaching Certificate Law” to adjust to the development of information technology in society in 1988. As a result of this revision, a new subject “Teaching Methods and Skills (including the use of information technology)”was required for students who want to acquire teaching certificates. In other words, improving “Computer Literacy” became necessary not only for college students but institutions. According to statistics of MEXT in 2003, the ratio of the teachers that are able to teach students with computers increased to be 72.7% in Elementary schools, 53.8% in Lower and 46.1% in Upper Secondary Schools (MEXT, 2004). The curriculum of Information Education in Secondary Schools has improved in recent years. The unit “Information and Computer” (included in the subject “Technology and Homemaking”) became required in Lower Secondary Schools in 2002. Next year a new required subject “Information Technology” was established in Upper Secondary Schools. Furthermore, MEXT directed all Elementary and Secondary schools to use computers effectively in the “Period for Integrated Study” and other subjects. The “Period for Integrated Study” was established in 1999 when National Curriculum Standards were revised for the last time. It aims at helping children develop capability and ability to discover problems by themselves and solve those problems properly (The Curriculum Council, 1998). In order to improve computer literacy of teachers in the future, institutions with teacher training courses have obligations to make their computer classes effective enough. It is reasonable that they use students’ course evaluation ratings to improve their courses because course evaluation by students has become popular among colleges and universities in Japan recently. Eble (1984) states “tangible measures if judgment (student ratings) gets preference, not because they may be better, but because they afford written evidence that may stand up in court” However, most of the institutions or faculties are not able to use the results to improve their courses efficiently. Braskamp and Ory (1994) suggest that most faculty view student ratings as one important indicator of teaching ability. Not only teaching abilities but students’ satisfaction with courses must be enhanced based on their course evaluation data. Thomas and Galambos (2004) investigated how students’ characteristics and experiences affect satisfaction using regression and decision tree analysis with the CHAID algorithm. They conclude that faculty preparedness is a principal determinant of satisfaction. Asaba, Inaura and Sato (2004) found that the Degree of Understanding affected the “Degree of Satisfaction”. They also suggested that students’ physical condition and motivation do not influence as much as the “Degree of Understanding. ” The purpose of this paper is to examine the factors that influence the satisfaction of college’s computer literacy courses based on students ratings. The Study Data for this analysis were drawn from student ratings of four computer literacy courses having the same curriculum in 2003. These courses are designed for first grade students, of whom almost one third hope to acquire teaching certificates. The sample of the survey was 171 except for the data including missing values. Covariance structure analysis was used to analyze the data of students’ course evaluations. To make the covariance structure models, we made the following five potential variables that were based on factor analysis. These potential variables are constructed of several observed variables. All observed variables of the questionnaires were rated on a 5-point scale from 1 (“strongly disagree” or “poor”) to 5 (“strongly agree” or “excellent”). Figure 1 shows relations considered in the course evaluation. It is difficult to change input variables for faculties, so we used only process and output variables to construct the model. Variables of faculties; Instructional Efforts Communication with Students Students’ efforts Expectation Satisfaction Understanding Learning performance Prior knowledge Motivation Facilities Faculties Input Output Process Courses Students Figure 1: Variables for course (1) “Instructional Efforts” from the following three observed variables a. Voice could be clearly heard b. Prepared for the class enough c. Demonstrates enthusiasm for the class (2) “Communication” from the following two observed variables a. Check students’ comments or reactions for the class b. Consider students’ Degree of Understanding Variables of students; (3) “Factors of Students” from the following two observed variables a. Degree of students’ effort for the class b. Degree of students’ expectation of the class Variables of output (4) “Degree of Understanding” from the following two observed variables a. Degree of Understanding of the class b. Increase of interest in the subject (5) “Degree of Satisfaction” from the following three observed variables a. Significant for students b. Took pleasure in attending the class c. Degree of Satisfaction of the class Findings With the above observed and potential variables, a covariance structure model named “Interactive Model I” was structured. ML (Maximum Likelihood) Method was used to estimate regression weight. The path diagram is presented in Figure 2. In the diagram oval shows potential variable and rectangle shows observed variable. At first there were paths both from “Factors or students” to “Degree of Satisfaction” and from “Communication” to “Factors or students”. However, modifying the model to improve the fit index, these paths were eliminated. GFI (goodness of fit index), AGFI (adjusted goodness of fit index) and RMSEA (root mean square error of approximation) were used to evaluate fitness of the model. A value of the RMSEA of about . 05 or less would indicate a close fit of the model in relation to the degrees of freedom. It is also indicated that a value of about 0.08 or less for the RMSEA would indicate a reasonable error of approximation and would not want to employ a model with a RMSEA greater than 0.1 (Browne and Cudeck, 1993). According to these standards, it is possible to be able to employ Model 1(GFI=.919 RMSEA=.073). While the direct effects of “Instructional Efforts” and “Communication” on “Satisfaction” are not significant, indirect effects through “Factors of Students” or /and “Understanding” are significant. “Instructional Efforts” could improve “Satisfaction” through “Factors of Students” and “Understanding”. The estimate value is calculated to be . 21(.32× .80× .81) that is greater than direct one (.14). On the other hand, “Communication” influences “Satisfaction” only through “Understanding”. The estimate value is calculated to be . 66(.81×.81) that is much greater than direct one (.15). GFI = . 919 AGFI = . 869 RMSEA= . 073 N=171 Understanding Understanding Interest