Induction of Condensed Determinations

In this paper we suggest determinations as a representation of knowledge that should be easy to understand. We briefly review determinations, which can be clispiayecl in a tabular format, and their use in precliction, which involves a simple matching process. We describe CONDET, an algorithm that uses feature selection to construct determinations from training data, augmented by a condensation process that collapses rows to produce simpler structures. We report experiments that show condensation reduces complexity with no loss of accuracy, then discuss CONDET’S relation to other work and outline directions for future studies. ity. After this, we present experimental studies of these techniques that evaluate the accuracy and complexity of the learned structures. We close with comments on related work and directions for future research. The Nature of Determinations Introduction TT-^I^--LI-=^l-:,I1-. :^ ̂ --1-^^_^^__ :-. ,. -,. ,, A,, A:, u1IUeL-sllawmdJIIllJy IS a lllt%J”l c;“III;txII 111 nll”wlau~;t: u13covery and data mining. Although it is important to discover knowledge that is accurate, in many domains it is also essential that users find that knowledge easy to interpret. Most researchers assume that logical rules and decision trees are more understandable than other formalisms, such as neural networks or stored cases. Although the evidence supporting this belief is mainly anecdotal, we will not argue with it here. Davies and Russell (1987) introduced determinations as a form of background knowledge for use in analogical reasoning, but the idea has more general applications. Briefly, a determination expresses some functional dependency between a set of predictor attributes P and a set of predicted attributes Q, so that, given P, one can infer Q. Of course, such knowledge is useful only if one has information about particular combinations of those attributes’ values. Davies and Russell proposed obtaining this information through anaiogy with stored cases. However, one can also envision a knowledge base containing a separate rule for each combination of predictor values, and we will assume such structures here. Rather, we will assume its validity and focus on a special class of logical rules, known as determinations, that we maintain are particularly understandable. This representation differs from other rule frameworks in that all rules in the knowledge base refer to the same attributes. As a result, they can be graphically displayed as a ‘truth table’, with one column for each attribute (including the class) and one row for each combination of attribute values. We anticipate that users will like this regular structure, especially given its similarity to widely used spreadsheet formats. Such determinations are interesting from the perspective of understandability because they can be displayed in a tabular format. Table 1 shows a determination for a simple artificial domain, originally used by Quinlan (1993) to illustrate decision trees, that involves deciding whether to pursue an outdoor activity. This domain includes four predictor attributes OUTLOOK. HIJMTDITY. WINDY, and TEMPERATURE and I ----. -----7 one predicted attribute CLASS, which states whether to engage in the activity. This determination includes columns for only three of the predictor variables, because TEMPERATURE does not help to predict CLASS. In the following sections, we review the representation of determinations and their use in classification, followed by an algorithm for inducing these structures hnnnrl fin Fnocmt nrnrlr in fPrlfl17-0 an1nrtinn Nnvt lx70 “u,Us,,u “II LcIb%. aIU “““Lax 111 LUrUUUlU ““SUUVLVLA. A.UI. ” I. ” present a technique for condensing induced determinations, aimed at further improving their understandabilOne can use a determination for prediction or inference in the same way as any other formalism that involves logical rules. For a given instance, one finds the row (i.e., rule) that specifies a combination of predictor values that match the instance, then infers the value specified for the predicted attribute(s). For now, we will assume that all attributes in a determination are discrete, and that any continuous variabies have been transformed into discrete ones either by the knowledge base’s developer or through some automatic process. *Also affiliated with the Institute for the Study of Learning Although Davies and Russell focused on logical deand Expertise, 2164 Staunton Court, Palo Alto, CA 94306. terminations that always held, one can adapt them to Rule Induction Q Decision Tree Induction 327 From: KDD-96 Proceedings. Copyright © 1996, AAAI (www.aaai.org). All rights reserved.