Fuzzy-Neural Cost Estimation for Engine Tests

This chapter discusses artificial computational intelligence methods as applied to cost prediction. We present the development of a suite of hybrid fuzzy-neural systems for predicting the cost of performing engine tests at NASA’s Stennis Space Center testing facilities. The system is composed of several adaptive network-based fuzzy inference systems (ANFIS), with or without neural subsystems. The output produced by each system in the suite is a rough order of magnitude (ROM) cost estimate for performing the engine test. Basic systems predict cost based solely on raw test data, whereas others use preprocessing of these data, such as principal components and locally linear embedding (LLE), before entering the fuzzy engines. Backpropagation neural networks and radial basis functions networks (RBFNs) are also used to aid in the cost prediction by merging the costs estimated by several ANFIS into a final cost estimate. Fuzzy-Neural Cost Estimation for Engine Tests 179 Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. INTRODUCTION John C. Stennis Space Center (SSC) is NASA’s primary center for testing and flight certification of rocket propulsion systems for the space shuttle and future generations of space vehicles. Because of its important role in engine testing for more than 3 decades, SSC has been designated NASA’s Center of Excellence for Rocket Propulsion Testing. SSC tests all space shuttle main engines (SSME). These high-performance, liquid-fueled engines provide most of the total impulse needed during the shuttle’s 8 1/2-minute flight into orbit. All SSME must pass a series of test firings at SSC prior to being installed in the back of the orbiter. Moreover, commercial engine and component tests are also performed at the SSC NASA facilities. A few operations management software systems, including cost estimating algorithms, have been developed in the past (Lockheed Martin Space Operations, 2001; Lockheed Martin Space Operations, 2000; Rocket Propulsion Testing Lead Center, 1997, 1998; Sundar, 2001) to aid in scheduling and managing tests as well as to predict the cost of performing component and engine tests at NASA’s John C. Stennis Space Center testing facilities: The cost estimating model (CEM), which includes cost estimating relationships (CER), the operations impact assessor (OIA), bottoms-up cost estimator (BUCE), and risk constrained optimized strategic planning (RCOSP). The results, however, have not been very encouraging and are not available in the open literature. OIA and RCOSP are very complex systems and require input data that are rarely, if ever, available before tests are performed. BUCE is a bottoms-up estimator and requires a level of detail for the input data (e.g., a complete list of parts and number of labor hours) that bans this tool from being used to generate a rough order of magnitude estimate. CEM is the simplest system and it prompts the user to input the same type of preliminary data as the systems presented in this Chapter. Results from CEM will be compared to the new computational intelligence systems which perform considerably better. CEM uses cost estimating relationships, parametric estimation, and statistics. In this chapter, we present a system for this same purpose (cost prediction), based on adaptive network-based fuzzy inference systems (ANFIS) and neural networks (NN). The hybrid software suite was developed in Matlab1 and combines the adaptive capabilities of neural networks and the ease of development and additional benefits of fuzzy logic based systems, detailed by the current authors in (Danker-McDermot, 2004; Kaminsky, 2002; Kaminsky & Douglas, 2003). The software-based system consists of several user-selectable subsystems ranging from simple fuzzy estimators, to medium complexity ANFIS systems that use normalized and transformed input data as well as more complex multistage fuzzy-neural or neural systems. We will discuss each here, and present comparative results indicating that these artificial intelligence procedures produce good cost estimates even when they are developed using very small sets of data. The accuracy of the predicted cost increases as the complexity of the system (as measured by number of processing routines, number of stages, and number of input variables) increases. The goal of the project2 was to develop a hybrid fuzzy-neural cost estimating system to obtain rough order of magnitude (ROM) estimates of the cost for both component and engine tests. A very small set of data, mainly from NASA’s Project Requirement Documents (PRD) (NASA, 2001; University of New Orleans, 2000), were available for component and engine tests performed at NASA’s John C. Stennis Space Center (SSC). 180 Kaminsky, Danker-McDermot, & Douglas Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. In this chapter, however, we detail only the hardest problem: predicting cost for engine tests. The available PRD data set for engine tests was much smaller and more incomplete than the component test sets. Results presented here are, therefore, easy to improve upon for component tests. For results of component tests, the reader may refer to Kaminsky (2002) and Kaminsky and Douglas (2003). A subset of the already small group of PRD data for engine tests was used to train the computational intelligence fuzzy-neural systems in the suite. The trained systems are then used to predict cost for unseen engine articles (the testing set). Several prototypes were developed and are described in the rest of this chapter: Simple ANFIS cost estimators (ANFIS), principal component analysis (PCA) ANFIS cost estimators (PCA-ANFIS), parallel/cascaded ANFIS systems (Parallel-ANFIS), locally linear embedding (LLE) ANFIS estimators (LLE-ANFIS), fuzzy-neural estimators (Parallel-ANFIS-NN), and radial basis function network estimators (RBFN). These differ in complexity and amount of preprocessing needed. Accuracy of predicted cost, although similar in order of magnitude, varies depending on the complexity of the system. Principal components and LLE are used as preprocessing stages to reduce the dimensionality of the data because we have many more variables (descriptors) than we have exemplars (articles in the training set). PCA yields a linear decomposition whereas LLE is a nonlinear reduction method. The rest of this chapter is organized as follows: In the next section, the engine test data, data analysis, and the preprocessing routines used are presented. We then briefly summarize ANFIS theory and present the various prototypes, followed by results for each of these prototypes and comparative results among the various systems. A summary and suggestions for further work are given along with conclusions. DATA DESCRIPTION AND PREPROCESSING This section discusses the data, collected and provided by NASA at Stennis Space Center, used to develop and test the fuzzy-neuro systems. We first describe the raw data and their limitations, and later analyze these data. We also discuss preprocessing of the raw data.