Classification by Multivariate Analysis with Objectives of Minimizing Risk, Minimizing Maximum Risk, and Minimizing Probability of Misclassification

THE PURPOSE of this study is to I describe and illustrate certain developments in the field of classificatory analysis which are po tentially useful in education. The problem un der consideration arises when a number of measurements are made on an individual and it is desired to classify the individual into one of sev eral categories on the basis of these measure ments. It is assumed that the i n d i v i d ual has been randomly drawn from one of the populations and that in each population there is a probability distribution of the measurements. In problems of this type it frequently may be observed that the consequences of wrong decisions are not equally undesirable. Somewhat unique is the assumption that it is possible to specify the loss of utility resulting from a wrong decision or misclassification. Loss may be thought of as the penalty paid by the statistician when his guess is incorrect. In this investiga tion it serves to differentiate the seriousness of the possible errors of classification. By the risk of committing a certain error in classification is meant the probability of that error multiplied by the corresponding loss. Risk thus corresponds to the expected value of the loss or the expected disutility in the long-r un use of some classification rule. In considering the problem of classifying an individual into one of several populations there is a risk or expect ed loss associated with each possible choice or classification. Important in the solution of problems of class ification are the a priori probabilities whose values postulate the respective chances of drawing an individual from each of the popu lations under consideration. The c r i t e rion by which classification rules are chosen va r i e s with the presence or absence of the a priori probabilities. If they are assumed to be known, rules are selected which minimize overall risk I of misclassification. If in a particular problem it is not possible or appropriate to es tima te a priori probabilities, the solution requires a dif ferent criterion of goodness. Under such condi tions it is possible to choose classification rules which minimize the maximum risk of misclassi fication. This is a conservative approach and represents an application of the minimax princi ple. Anderson (1) discusses these criteria in detail. Rao (6), Brown (2), and Welch (9) are useful ref erences. More formal reasoning is gener ally expressed as follows: Assume, for exam pie, that an individual I with the set of measure ments xx-,xq has been randomly drawn from one of two populations nx or n2 with prob ability distributions of the measurements ix(xlf ...., xq) and f2 (xx,_, Xq), respectively. The choice of a classification rule R corresponds to a division of the q-dimensional sample space into two regions Rx and R2. If the random point corresponding to the individual falls in Rx the de cision is made that I is from TLl9 and if the point falls in R2 the decision is made that I is from n2. Let L(2/l) be the loss incurred in classifying I into H2 when in fact I belongs to Ul9 and let L(l/2) be the loss incurred in classifying I into nx when in fact I belongs to n2. For any rule R let rx(R) be the conditional risk or ex pected loss when I belongs to Ux. It is defined as the probability of I falling in R2 multiplied by the loss associated with this mistake. Simi larly, let r2(R) be the conditional risk when I belongs to H2. It is defined as the probability of falling in Rx multiplied by the corresponding loss. Suppose there exist a priori probabilities that I comes from the two populations, say probabil ity Pi that I does in fact belong to Ul9 and prob ability p2 that I belongs to II2, where px + p2 =