Quality of Statistical Education: Should ASA Assist or Assess?

Quality statistical education-like quality in education in general-is a Good Thing. I am sure we all agree to that and agree too that quality statistical education is something to be striven for. But this may be as far as full agreement can be guaranteed-though we might go a little further and agree that a minimum criterion of quality education is that students be given no false information. Beyond that, we would probably find disagreement on how to define and hence evaluate the quality of statistical education. Are we concerned with evaluating curriculum-with determining whether the content of a program is appropriate? And appropriate for whom? For graduate students? For undergraduate majors? For students taking service courses on all levels and from all disciplines? What is the appropriate statistical curriculum? Or are we concerned with evaluating the formal qualifications of those teaching the courses? Is a PhD in statistics necessary for teaching every course in statistics? Is it sufficient? Or are we concerned with evaluating the institutional arrangements for the teaching of statistics across a campus? Minton and Freund (1977) have explored different sorts of institutional arrangements. While they come down in favor of a centralized arrangement, they also remind us that circumstances alter cases. On some campuses a decentralized model is more appropriate, as long as the faculty teaching statistics in various departments communicate with one another. Or are we concerned with evaluating what goes on within the classroom itself? Is our primary concern the accuracy of the information presented or the efficiency of the transmission of knowledge between teacher and students? It seems to me that these are judgment calls-the sort of operationalizing that each of us as a statistician is supposedly trained to do and the sort of question that each of us answers frequently in our professional life. But are they questions that we want our association to answer, to take an official position on? I think not, for I think the diversity in our educational system fostered by not having hard-and-fast, institutionalized answers to such questions is valuable enough to be nurtured, even at the risk of permitting what some of us might consider inadequate departments, curricula, courses, or teaching to exist. But let us assume for a moment, and for the sake of the argument, that we can agree on the dimensions of "quality education." We are then still faced with the problem of how to go about measuring that quality. One way to measure quality is to ask the consumers of the education. Students' course and teacher evaluations, using any one of several carefully constructed standard instruments now available, seem to be reasonably reliable when aggregated over a class of moderate size (more than 20 or so, c.f. Feldman 1977). They have been further aggregated to provide evaluations of entire departments (Stumpf 1979; Smock and Brandenburg 1978). It seems to me that such aggregated student ratings are useless in evaluating the closeness of a curriculum to some ideal, however defined. Aggregated student ratings seem useful, however, in giving a department an idea of how well it is managing to communicate with its students. ASA could provide a service to departments wishing to evaluate their communication in this way by providing a bibliography of student rating instruments and of the empirical work that has been done to validate them. A second way to assess quality is through the use of reputational measures. One simply asks members of a discipline (those members chosen randomly or in some deliberate manner) to rate each department offering degrees in that discipline (usually on a scale ranging from inadequate to outstanding). Reputational measures have been used in several large studies (Cartter 1966; Roose and Andersen 1970) over the last fifteen years. Another such study is currently underway, and it, unlike its predecessors, does include the discipline of statistics. This is a nationwide assessment of researchdoctorate programs. It is being conducted by the Conference Board of Associated Research Councils with administrative and analytic functions to be carried out by the National Research Council of the National Academy of Sciences. It differs from earlier studies in that it provides information to raters about the departments being rated-primarily a list of faculty members. Thus the quality of statistical education within statistics departments offering research doctorates is being evaluated by this reputational system, whether we like it or not. It has been claimed that the purpose of these largescale reputational studies is "neither to afflict the comfortable nor to comfort the afflicted" (Logan Wilson in the forward to Roose and Andersen 1970), but rather to offer guidance to students (in particular, graduate students) seeking training and to suggest remediation to those departments found wanting. I can understand the "consumer's guide" aspect of these studies. Clearly the reputation of the department in which one receives one's training has real effects on one's later career (though it is not necessarily true that these effects are mediated by the quality of the training received). I fail to understand, however, how such studies can lead to remediation within departments, unless they signal the need to hire new faculty of high repute-but this begs the question of whether such repute is related to teaching skill. The results of these reputational studies have been validated against such objective measures of faculty and student quality as number of Nobel Laureates, members of the National Academy, and winners of fel*Judith M. Tanur is Associate Professor, Sociology Department, State University of New York at Stony Brook, Stony Brook, NY 11794.