Active Learning for Classifying a Spectrally Variable Subject

Classifying Egeria densa, Brazilian waterweed, in scan-digitized color infrared (CIR) airphotos by automated methods presents a challenging problem due to a number of variable and unfavorable conditions: changes in imaging conditions, problems associated with water-related subjects, and other environmental changes as well as expected lack of spectral separation between Egeria and other land cover classes in CIR imagery. To address these challenges, we are developing an interactive computer system based on data mining techniques with Active Learning capabilities. The key components of this system are: feature extraction, automatic classification, active learning, and experimental evaluation. We anticipate creating an interactive learning system that can learn from human analysts by relating results to extracted objects and that can learn analytic rules for classification. In this paper, we report the concept of the system, preliminary experimental results, and anticipated future work. I. BACKGROUND AND PROBLEM Rapid advances in remote sensing and in data storage have made huge quantities of image data available for analysis. The spatial, spectral, and radiometric resolutions of digital remotely-sensed imagery have greatly improved in recent years. However, automated classification of high-resolution imagery has not kept pace with developments in data collection and storage. The need for data collection by aerial photographic surveys also still exists for many applications. Improvements in orthorectification techniques have allowed such photography, recent and historic, to be digitized, geometrically corrected, and integrated into a database more rapidly and at reduced cost. Such progress has particularly aided applications that require high-resolution spatial data, for example: precision agricultural monitoring, individual-tree forest inventory, wetland and small waterway mapping, and invasive species monitoring. In many cases, human experts are heavily involved in the interpretation and analysis of high-resolution imagery. Visual/manual image interpretation and analysis procedures are often time-consuming and costly, not repeatable, and dependent on the varying abilities of the interpreters. Individual expertise is hard to transfer from one interpreter to another, which contributes to high training costs. There is a significant gap between fast routine data collection and the slow interpretation and analysis of complex and detailed images in multidimensional (spatial and temporal) space. Monitoring Egeria densa, an invasive submergent weed, by remote sensing represents such a case. Egeria densa, commonly called Brazilian waterweed, has grown uncontrolled in the Sacramento-San Joaquin Delta of Northern California for over 35 years and now covers about 2400 hectares of waterways. The presence of this exotic weed is disrupting navigation and recreational uses of waterways, clogging irrigation intake trenches, and causing reservoir-pumping problems. The cost of pumping Clifton Court Forebay reservoir alone has increased by over US$1 million per year. The Egeria invasion has displaced native flora and probably affected native fauna. In 1997, the California Department of Boating and Waterways (DBW) started developing a control program to manage Egeria in the Delta. Research scientists at the Romberg Tiburon Center for Environmental Studies (RTC) at San Francisco State University were hired to assess the effects of control protocols on fish and other fauna and to estimate the areal extent of the Egeria [1]. Monitoring the areal extent of Egeria using scan-digitized color infrared (CIR) aerial photographs has occurred every year since then. The database of photographs is online at: http://romberg.sfsu. edu/~egeria. Classifying Egeria in CIR airphotos by automated methods presents a challenging problem due to a number of variable and unfavorable conditions. These include changes in imaging conditions (e.g., film exposure, vignetting, scanning anomalies), problems associated with water-related subjects (e.g., turbidity, sun glint, surface reflectance due to wind), and other environmental changes (e.g., exposure of Egeria at extremely low tide, shadows falling upon the water, algae cover over Egeria). Figure 1, a detail of a scan-digitized CIR aerial photograph, illustrates the spectral variations in Egeria that may occur even within a short distance. Digital analyses also indicated that subtle changes—for example, in Egeria canopy density, film vignetting, or water turbidity—produced overlapping spectral response patterns. In addition, the spectral response patterns for Egeria do not separate well from those of other land cover classes in CIR imagery. For example, dense well-submerged Egeria appears black and is confused with shadows on land; Egeria exposed during very low tide appears reddish and is confused with terrestrial vegetation. Clearly, traditional computer-assisted Figure 1. Scan-digitized CIR aerial photograph showing spectral variations in Egeria. multispectral classification methods are problematic under these conditions, and visual/manual image interpretation and analysis procedures are daunting. Post-classification processing is frequently needed to clean up noisy patches in classification maps. To address these challenges, we are exploiting the latest developments in data mining, investigating novel combinations of effective methods in feature extraction, classification, and machine learning, and proposing a computer system that implements the novel combinations. Such a system that integrates feature extraction, classification, and machine learning is expected to accomplish routine tasks, highlight areas for verification or further analysis, minimize expert intervention, and consequently allow experts more time to concentrate on more vexing problems. In this paper, we report our research addressing some challenges outlined above by developing an interactive computer system with Active Learning. The key components of the system are as follows: Feature extraction to derive relevant and effective features for automatic classification [2]; Automatic classification – to achieve speed, accuracy, generalization, and automation; Active learning to use machine learning techniques to minimize expert intervention without performance deterioration; and Experimental evaluation – to compare the performance with and without the newly developed systems. We anticipate creating an interactive learning system that can learn from human analysts by relating results to extracted objects and that can learn analytic rules for classification. In the following, we introduce the system, elaborate on the key components, present preliminary results, and conclude with future work. II. INTEGRATED INTERACTIVE SYSTEM Human experts have been heavily engaged in the problem of detecting Egeria in CIR imagery for practical applications. Due to the variability of this landscape class, a significant amount of time and interpreter training has been needed to discriminate Egeria in this imagery. Characteristic and unusual colors, shapes and textures have had to be verified by ground surveys. Typical Egeria sites and associations have been cataloged over time. Many problems have only been identified by multitemporal information. Well-trained image analysts are very good at analyzing a new image. Manual processing, though, is inherently unscalable. Experts can process only a small portion of the available unprocessed images (unlabeled data) at any given time. As a first step, we aim to reduce the amount of involvement of human experts. Therefore, the new system uses Active Learning and is interactive in defining features and resolving uncertainties. The images are divided into two groups: a training set and a testing set. Ideally, we wish to have a very small training set because the smaller training set (i.e., a few subsets of images) will require less involvement of human experts. Also, we are trying to minimize the involvement of experts during testing. A flow chart of the interactive system is presented in Figure 2. The following sections describe the components of this system in more detail.