Predicting Student Performance in Web-Based Distance Education Courses Based on Survey Instruments Measuring Personality Traits and Technical Skills

Two common web-based surveys, “Is Online Learning Right for Me?’ and “What Technical Skills Do I Need?”, were combined into a single survey instrument and given to 228 oncampus and 83 distance education students. The students were enrolled in four different classes (business, computer information services, criminal justice, and early childhood development) on three different regional campuses of a mid-western community college. Multiple regression equations were developed with the survey scores and the type of class to determine the extent to which the survey scores predicted final semester grade percentages. Although distance education students scored significantly higher on the survey instruments than on-campus students, the survey scores explained only 8% of the observed variance in their final grade percentage. The course in which they enrolled (business, computer information services, criminal justice, and early childhood development) explained most of the variance in final grade percentage. There was no significant difference in survey scores between students that withdrew from their classes and those that remained to receive a final semester grade. Recommendations for further research are suggested. Introduction There is a general consensus that the number of students receiving an education through distance learning has continued to grow steadily (Saba, 2005). In 2004 the National Center for Education Statistics (NCES) reported that course enrollments in distance education increased from 1.7 million in 1997-98 to 3.1 million in the academic year 2000-01. Not only are institutions offering individual courses via distance education, but the number of entire programs has increased as well. Among institutions offering any distance education courses, the NCES reported that the proportion offering degree programs completed entirely via distance education increased between 1997-98 and 2000-01 from 22 to 30%. Certificate programs increased from 7 to 16% during the same period. Although various technologies can be used to transmit educational content, the dominant delivery system used is the Internet coupled with browser-based technology (Compora, 2003; Howell, William, & Lindsay, 2003). However, it seems reasonable to assume that not all students would find the web-based distance education learning environment as attractive as a traditional face-to-face environment (Lorenzetti, 2005a). A number of researchers have suggested that the retention rate for distance education students is lower than for traditional face-to-face students (Nash, 2005; O'Brien & Reneer, 2002; Scalese, 2001). Administrators interviewed for The Chronicle of Higher Education (Carr, 2000) agreed that course completion rates in distance education courses are often 10-20 percentage points lower than in traditional classes. The lower retention rates for distance education students would indicate that some type of preassessment might be useful in identifying students that could have potential problems in a distance education environment. Several researchers have discussed the need for self-assessments for potential distance education students (Biner, Bink, Huffman, & Dean, 1995; Buchanan, 1999; Lorenzetti, 2005b; Maki & Maki, 2003; Restauri, 2004; Valasek, 2001). Kidder (2004) reported that 48% of faculty surveyed indicated that pre-assessment of students' technological skills was not present, but that pre-assessment was important to the success of on-line classes. A total of 77% reported pre-assessment is important, while only 29% recorded that it was present in their institution. However, commonly-used readiness questionnaires given to students prior to the start of a course may not be useful in selecting students who will do well in and be satisfied with technology-mediated courses (Maki & Maki). Purpose of the Study The purpose of this study was to determine the extent to which two commonly available survey instruments, “Is Online Learning Right for Me?” and “What Technical Skills Do I Need?”, are accurate predictors of student performance in web-based distance education courses. The survey “Is Online Learning Right for Me?” was developed by the Northern Virginia Community College Extended Learning Institute. It is a 16-question, forced response, self-scoring guide that has face validity for individual traits and skills believed to contribute to potential success in an Internet-based course (Buchanan, 1999; Noah, 2001). Each multiple choice question has three answers. Students are advised to select only one of the three answers. Selecting answer “A” equates to 3 points; selecting “B” equates to 2 points; selecting “C” equates to 1 point. The total score is calculated by adding up the individual question scores. The survey is designed such that the higher the total score, the greater the presumption for success in a distance education course. A copy of this survey is provided in Appendix A. The survey “What Technical Skills Do I Need?”, provided in Appendix B, was developed by Palm Beach Community College. It is a 15-question, forced response, self-scoring guide with face validity for computer-self efficacy. Most of the skills assessed are directly related to Internet applications such as e-mail, chat rooms, and Internet searches. Each multiple choice question also has three answers. Students are advised to select only one of the three answers. Selecting answer “A” equates to 3 points; selecting “B” equates to 2 points; selecting “C” equates to 1 point. The total score is calculated by adding up the individual question scores. The survey is designed such that the higher the total score, the greater the presumption for success in a distance education course. A copy of this survey is provided in Appendix B. Both surveys provide one of three responses based on the total score achieved by the respondent. Tables 1 and 2 list the three responses and the total scores needed to receive the indicated response. Table 1 “Is Online Learning Right for Me?” Survey Responses and Total Score Ranges McMillan and Schumacher (2001) define face validity as “a judgment that the items appear to be relevant” (p. 241). An examination of the surveys (Appendices A and B) reveals that the items included have the appearance of relevancy to individual traits and skills believed necessary for success in distance education. Yet face validity is considered the weakest form of construct validity (Trochim, 2006, Face Validity section, ¶ 1).