Differential Negative Reinforcement Improves Classifier System Learning Rate in Two-Class Problems with Unequal Base Rates

The effect of biasing negative reinforcement levels on learning rate and classification accuracy in a learning classifier system (LCS) was investigated. Simulation data at five prevalences (base rates) were used to train and test the LCS. Erroneous decisions made by the LCS during training were punished differentially according to type: false positive (FP) or false negative (FN), across a range of four FP:FN ratios. Training performance was assessed by learning rate, determined from the number of iterations required to reach 95% of the maximum area under the receiver operating characteristic (ROC) curve obtained during learning. Learning rates were compared across the three biased ratios with those obtained at the unbiased ratio. Classification performance of the LCS at testing was evaluated by means of the area under the ROC curve. During learning, differences were found between the biased and unbiased penalty schemes, but only at unequal base rates. A linear relationship between bias level and base rate was suggested. With unequal base rates, biasing the FP:FN ratio improved the learning rate. Little effect was observed on testing the LCS with novel cases. 1.0 Introduction This paper reports on an investigation into one asp ect of using a learning classifier system (LCS), specifica lly the ability of a LCS to learn pre-classified cases and to classify previously unseen cases that appear in varying rati os in epidemiologic research. Epidemiology seeks to stud y he “. occurrence and distribution of diseases and oth er healthrelated conditions in populations. ” (Kelsey, Thom pson, and Evans 1986). Disease “occurrence” is measured in terms of incidence and prevalence. Incidence reflects the rate of change from a healthy state to a diseased s tate over time, indicating the occurrence of new cases of dis ease within a population per unit time. Incidence is us ually expressed as a rate, such as 20.4 cases per 100,000 persons per year. Prevalence is a quantitative measure of the existence of disease in a population at a given poi t in time and is expressed as a proportion, such as 50.2 case s per 100,000 per persons, or a 0.05. During the early course of an epidemiologic surveillance, it is extremely rare to find individu als afflicted with a given disease outcome at the same rat as one finds those without the disease. Even in the c ourse of investigating an infection outbreak in a small grou p of people exposed to an organism known to cause diseas e in the majority of those exposed to it (such as S lmonella), an investigator would ordinarily find relatively few i ncident cases early on, due to the time required to incubat e the organism. However, it is especially during this ea rly period of surveillance that a robust classification rule would be needed, so that appropriate interventions may be planned and implemented. To date, no one has addressed the issue of differen tial negative reinforcement in a LCS when applying them to a classification task in a domain with unequal number s of positive and negative cases, or base rates. This investigation focused on the performance of a LCS w hen confronted with epidemiologic data at various disea se prevalence levels, and the ability of the LCS to cl assify previously unseen cases. The LCS used in this stu dy follows the stimulus-response model used in BOOLE (Wilson 1987) and NEWBOOLE (Bonelli et al 1990).