The Effects of Alternative Interpretations of Incomplete and Withdrawal Grades on Course Placement Validity Indices.

In validating tests for course placement in college, the criterion variable is usually defined in terms of the grade earned in a particular standard course. For example, success may be defined as completing the standard course with a grade of C or higher. This study considered the issue of interpreting incomplete (1) and withdrawal (W) grades either as unsuccessful outcomes or as missing data. The effects of either type of interpretation on placement indices and optimum cutoff scores were studied. The data for the study were obtained from four two-year colleges through their participation in a pilot study of the ACT Course Placement Service. Courses included were mathematics and English/reading courses. ASSET test scores were used as predictor variables. The results of this study showed that interpreting I and W grades as unsuccessful outcomes, rather than as missing data, generally resulted in lower conditional probabilities of success, higher optimal cutoff scores, and higher estimated delta accuracy rates. The manner in which I and W grades are interpreted should depend on an institution's policy or philosophy on those grades. Ideally, grades of I should be changed to grades of A through F or S /U before being included in the analyses of course placement criteria, and only students who received grades of W because of academic reasons should be classified as unsuccessful. If course placement accuracy indices were determined using these criteria, then the resulting optimum cutoff scores would be more accurate and appropriate. THE EFFECTS OF ALTERNATIVE INTERPRETATIONS OF INCOMPLETE AND WITHDRAWAL GRADES ON COURSE PLACEMENT VALIDITY INDICES Although grading policies may differ from one institution to another, most postsecondary institutions use a grading scale of A to F to denote excellent to failing performance in a course, and grades of I and W to denote incomplete work and withdrawal from a course. Some institutions may also use combinations of grades, such as withdrawal pass (W ? ) or withdrawal fail (WF). Students may receive grades of I or W for academic problems, such as the inability to cope with the demands of a course; or for nonacademic problems, such as illness, emotional distress, or financial difficulty. Institutions may also differ in how grades of I are maintained. Grades of I may be permanently kept on transcripts, or the grades may be changed to failure (F) if supplementary assignments (or course requirements) are not completed by a prespecified deadline. Many colleges, particularly two-year community colleges, have an open admission policy for all high school graduates. The implementation of this policy, and attempts by colleges to provide opportunities for students to succeed, has resulted in course placement policies and developmental/remedial instruction. Because course placement cutoff scores are often developed from statistical analyses of placement test scores and course grades, the interpretation of I and W grades as either unsuccessful outcomes or missing data (deleted from the analysis) may affect the cutoff scores selected for course placement and the resulting course placement decisions made for future students. If course placement validity indices and optimum cutoff scores differ because of the way in which 1 and W grades are interpreted, then thoughtful consideration must be given to how these grades are interpreted when evaluating the accuracy of course placement cutoff scores. Correct course placement decisions promote student success and foster persistence among students. Incorrect course placement decisions, however, waste students' time in school and educational expenses, as well as institutions' personnel allocations and costs. The purpose of this paper was to compare optimal placement cutoff scores and placement validity indices resulting from two interpretations of 1 and W grades, where I and W grades were interpreted as unsuccessful outcomes or as missing data. The placement validity indices and cutoff scores were developed based on the logistic regression of placement variables (i.e., test scores) on dichotomous (successful or unsuccessful) course outcomes in a particular course of interest (Sawyer, 1989). It should be noted that because the grades assigned in a course depend on the policies of the institution and instructor, when and how I and W grades are assigned may vary from institution to institution and from instructor to instructor. Thus, the perception of what these grades mean varies among college educators and administrators. The use of I and W grades in evaluating course placement cutoff scores is therefore best determined by individual institutions and/or instructors, and cannot be addressed solely through statistical methodology. Logistic regression can be used to estimate the conditional probability that a student would be successful in a course (e.g., a grade of C-or-higher), given the student's score on a predictor variable (e.g., placement test). The conditional probability of success estimates are based on the test scores and course grades of students in a particular course of interest. Placement validity indices can then be estimated from the conditional probability of success and the distribution of test scores for a larger group of students, those who could have taken the course (the "placement group"). These validity indices can provide information about cutoff scores used to place students into particular courses, and about the probable results of modifying such cutoff scores. Consider the following potential outcomes for a given cutoff score: A. True positive: the student is placed in the standard-level course and is successful (Correct decision). B. False positive: the student is placed in the standard-level course and is unsuccessful (Incorrect decision). C. True negative: the student is placed in a lower-level course and would have been unsuccessful in the standard-level course (Correct decision). D. False negative: the student is placed in a lower-level course, but would have been successful in the standard-level course (Incorrect decision). The sum of outcomes A and C is the number of students who could have taken the course and for whom correct decisions would have been made using the corresponding cutoff score and success criterion. This ratio of A +C /A +B+C +D is referred to as the accuracy rate (AR). The value of AR 2 depends on the cutoff score, the distribution of scores, and the statistical relationship between the test score and the success criterion. The AR attains a maximum value at or around a probability of success of . 50, which corresponds to the optimum cutoff score. The delta accuracy rate (AAR) is an indicator of the effectiveness of the predictor variable for placing all students scoring above a specific cutoff score, and not others, in the standard course, compared to placing all students in the course. This statistic is equal to the difference between the maximum AR value and the "base line” AR value, which is the proportion of correct decisions associated with using the lowest possible score as a cutoff score. The success rate (5R) is the estimated proportion of students in the placement group who would be placed in the standard-level course and who would be successful, given the corresponding cutoff score and success criterion. This statistic is equal to the ratio of A /(A +B). The lower-level course placement rate (LPR) refers to the proportion of all students in the placement group who would not be admitted to the standard-level course, given the corresponding cutoff score and success criterion. This statistic is equal to the ratio of C +D /A +B+C +D .