Monitoring software reliability in the shuttle mission simulator

The Shuttle Mission Training Facility (SMTF) is used to train astronauts and ground controllers for Space Shuttle missions. The complex contains three simulators driven by real-time software on large mainframe host computers and supported by as many as twelve super-mini computers. The Shuttle Mission Simulators are complex systems with a large number of hardware components and many diverse pieces of software running simultaneously and interacting with each other. The real-time executable code consists of approximately 365,000 lines of FORTRAN source in 956 modules and 275,000 lines of ASSEMBLER code in 686 modules.Software reliability monitoring has been used in the SMTF for three reasons: To notify management of any changes in the reliability of the system due to new requirements implemented in the code.To understand which functions of the simulation are the reliability problem areas.To identify trends in the reliability of the software over time.Two established data bases are being used to help meet these objectives. The first is called the Problem Incident Report database. It contains information such as type of problem (hardware, software, or procedural), level of incident (critical, degraded, no lost time), time and date of occurrence and resolution, amount of lost time, software load number, and scheduled activity. The second database contains information about facility scheduling. It contains the software load number, start and stop date and time, the simulator being used, and a user identifier. Combining these two sources of information yields the data necessary to meet the three objectives.The Littlewood and Verrall, Musa, and Shooman software reliability models were examined for use in the SMTF. Results from the Littlewood and Verrall model provided a good lower bound for reliability prediction. The Musa model, with its calendar-time component, provided schedule data important to management. Because of its computational simplicity the Shooman model was used to provide “quick look” estimates and starting points for the Musa model. Since each model has attributes that make it appealing under certain circumstances, an eclectic approach was chosen; all three models were used in the SMTF monitoring effort.Software reliability monitoring has produced the desired results. Management was able to identify reliability problem areas and understand trends in the SMTF. Further, based on the application of the software reliability measurement techniques proposed by Littlewood and Verrall, Musa, and Shooman NASA has begun an upgrade to the SMTF. The upgrade consists of the consolidation of the processing done in the super-mini computers into a single multi-processing computer and acquisition of larger more efficient host computer. Along with these hardware upgrades some of the software will be redesigned to be more fault-tolerant. Also, much of the ASSEMBLER code will be rewritten in FORTRAN to make it more maintainable.An important note is that software reliability models were not available during the early phases of SMTF development and have been incorporated only recently. However, due to the benefits in the SMTF software reliability measurement has been incorporated into the proposed day-to-day operations of the new Space Station Training Facility. A plan for measuring software reliability throughout the life-cycle has been proposed using several different types of software reliability models, and requirements for this measurement have been written into the system's preliminary requirements document.