This article introduces the method the Air Force Operational Test and Evaluation Center uses to determine whether a system's software is mature enough for the system to enter dedicated operational test and evaluation and to subsequently be fielded or procured. Our evaluation is a trend analysis of software changes identified during software development and testing. This evaluation currently does not provide any kind of projection of future software product maturity but provides an excellent snapshot of system maturity. Data that describes and tracks documented software changes serves as the key input to the evaluation. Following are the minimum data required for each software change to evaluate software product maturity.
Many organizations use different severity, criticality, or priority definitions.
Any reasonable ranking system is acceptable as long as a clear definition of product maturity is included. In our evaluations, we include software problem reports, software failure reports, software change requests, trouble reports, and any other data that fits the above definitions. If the software does not meet user requirements, the documentation of the unmet need is an input for the software product maturity evaluation. To correctly gauge readiness to deliver, developers must also evaluate test completeness, test rates, and requirements stability. Any of these factors can cause product maturity to look unrealistically good or bad.