Artificial Neural Network Classification Using a Minimal Training Set : Comparison to Conventional Supervised Classification

Recent research has shown an artificial neural network (ANN) to be capable of pattern recognition and the classification of image data. This paper examines the potential for the application of neural network computing to satellite image processing. A second objective is to provide a preliminary comparison of training site data inputs and generalized land-cover classification results for conventional supervised classification and ANN classification. An artificial neural network can be trained to do land-cover classification of satellite imagery using selected sites representative of each class in a manner similar to conventional supervised classification. One of the major problems associated with recognition and classification of patterns from remotely sensed data is the time and cost of developing a set of training sites. This research compares the use of an ANN back propagation classification procedure with a conventional supervised maximum likelihood classification procedure using a minimal training set. When using a minimal training set, the neural network is able to provide a land-cover classification superior to the classification derived from the conventional classification procedure. This research is the foundation for developing application parameters for further prototyping of software and hardware implementations for artificial neural networks in satellite image and geographic information processing. INTRODUCTION TO ARTIFICIAL NEURAL NETWORKS A N ARTIFICIAL NEURAL NETWORK (ANN) is composed of a large number of simple, interconnected "processors" (neurons) working in parallel within a network. The ANN has the capability to develop an internal representation of a signal pattern that is presented as input to the network. This automatic programming or "learning" is accomplished through the dynamic adjustment of the network interconnection strengths (adaptive weights) associated with each neuron. This process, termed back propagation, uses a desired outcome pattern and a defined input (training set) to initiate feedback to the neural network. The network cycles through the training set until the synapse weights are such that the network correctly relates the defined input to the desired output. When presented with new data, the internal synapse weights (stored in local memory) excite or inhibit the firing of specific processing units (neurons). The pattern of these neuron firings segregates the input signals into one of the output classes. The most apparent advantage of the back propagation neural network is that the learning algorithm is not programmed, a prior, into the network. This eliminates the algorithm and computer code development so often vexing in expert systems and other artificial intelligence approaches. Furthermore, neural networks are relatively tolerant of missing data and noise within the data. Once learning has been completed, new input data do not have to be completely specified. Most neural networks have a non-linear adaptation component which allows the network to perform more realistically on non-linear data. When implemented in hardware, the massively parallel processing configuration of the neural network should permit exceptional real-time throughput. While artificial neural networks are not a panacea, they appear to be a very promising approach to problems that involve the association of elements in a set with elements in another set. This class of problems includes pattern recognition and spectral and textural classification (Fukushima, 1988; Hecht-Nielsen, 1988). This research employs a software simulation of an artificial neural network developed by the Cartographic Applications Group, Image Processing Laboratory at the Jet Propulsion Laboratory. Based initially upon procedures described in Parallel Distributed Processing (Rumelhart and McClelland et al., 1986), this software simulation differs from other neural network simulations in that it is embedded within the image processing and geographic information systems domain of VICARfiBIS (Castleman, 1979; Logan and Bryant, 1987). The input, output, and processing network layers are specified in image processing terms. Investigation of the inner functioning of the network is elihanced because the network outputs, such as the synapse files, can be viewed as images. The VICMDIS neural network simulation software consists of four programs: NETGEN, NF2, NETWORK, and NREP. These programs are used respectively for defining and building the synaptic network, initializing synapse values (if desired), training and/or running the network simulation, and graphically analyzing the behavior of the neural network. The "NETWORK" software suite is well-suited for examining both the internal parameters and functioning of the neural network in a general sense, and for direct application to problems involving large spatial databases. The software suite will handle up to ten input files, ten output files, ten neural layers, and ten synapse files. The supported neural network processing algorithms include back propagation, a Hopfield-like "cycle" option, Hebbian, and feed-forward. Additional approaches are currently being implemented. APPLICATION TO LAND-COVER CLASSIFICATION Application of the artificfal neural network approach to landcover classification was substantiated in Hepner and Ritter (1989). The research discussed in this paper is directed towards the determination of minimum training data set requirements for a neural network of a given architecture to perform at an acceptable level. Relative performance is estimated by comparing the neural network classification to a conventional classification using the same imagery and minimal sized training sites. The focus is on the capability of the ANN to accomplish the classification PHOTOGRAMMETRIC ENGINEERING AND REMOTE SENSING, Vol. 56, No. 4, April 1990, pp. 469-473. -0099-1112~90/5604-469$02.25/0 01990 American Society for Photogrammetry and Remote Sensing PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING, 1990 regardless of specific performance parameters. Evaluation is based on the generalized accuracy of the classification relative to the effects of minimal data inputs for training. These preliminary efforts are part of a broader research program to analyze the training site requirements and robustness of an artificial neural network classifier for spectral feature classification and pattern recognition. Conventional multispectral classification is the process of simplifying continuous spectral pattern (e-g., satellite imagery) into discreet groups of known identity. Classification techniques have been the subject of considerable research over the past 20 years largely due to the NASA Landsat program, and the desire to generate land-cover classifications from Landsat satellite imagery for general resource assessment, inventory, and management. Much of the earliest conventional classification research centered on exploring different types of algorithms. However, Hoffer (1979) and Scholz et al. (1979) compared several different classification algorithms and reported their performance to be about the same. Scholz ef al. (1979) concluded that ". the major variable affecting correct classification accuracy is not the classifier, but the training method. " (p. 1146). The procedures involved with neural network spectral classification are more closely associated with "supervised" classification as opposed to "unsupervised classification techniques. With supervised techniques, the analyst initially defines several training areas in the imagery which are representative of the desired classes. For many resource applications, supervised classification is often the desired approach because of its ability to focus on the specific resources of interest. In unsupervised classification the data are clustered in feature space, while the analyst labels the clusters after they are formed. Supervised classifications tend to be highly labor-intensive in order to obtain a sufficiently comprehensive set of training sites. Unsupervised classifications tend to produce more classes than can easily be managed, with many of them undesirable transition classes. An alternative compromise method is "multi-cluster blocks" (Fleming and Hoffer, 1977) which involves unsupervised clustering of selected ("supervised") training site blocks. Hoffer (1979) compared conventional supervised, unsupervised, and multi-cluster blocks and found the multi-cluster blocks method to be the most accurate method. Multi-cluster blocks, also known as "guided clustering," has also been used effectively by Gaydos and Newland (1976) and Mayer and Fox (1981). In this preliminary classification analysis research, conventional supervised classification was compared with the neural network back-propagation approach without the potential enhancements of "multi-cluster blocks" in order to evaluate the fundamental capabilities of neural network artificial intelligence technology. The study area was the Ft. Lewis Military Reservation near Tacoma, Washington, for which detailed ground truth was obtained from the Defense Mapping Agency (DMA). Thematic Mapper ( m ) imagery consisting of the visible spectral bands 1,2, and 3 and the near-IR band 4 were used in the study (Plate 1). The neural network was configured as a fully interconnected back propagation linkage of three layers. The first layer was composed of a 3 by 3 by 4 array of neurons used for processing input. This permitted a 3by 3-pixel window to move across the four TIVI images, assuring a simultaneous consideration of texture as well as spectral decision space parameters. Using the 3 by 3 window of input neurons allowed the network to assimilate data of spatially adjacent pixels in both the training and classification operations. The second layer was a single, ten neuron processing or "hidden" layer. Several researchers have found that the number of hidden layers and the number of neural unit