Overall Conclusions and Lessons Learned  A “qualified product is typically held as proven, stable, and unchanging in a world of carefully assessed  program risk. Yet, change is necessary over a product’s lifespan as limitations in a design are discovered.  These limitations range from feedback on usability from customers to full-blown failures. How can this  contradiction - that change is both inevitable and discouraged - be resolved? A “qualified product should  not be considered frozen or static in its configuration, since change will be required as production  continues. The process of engineering changes to qualified hardware must be carefully managed. As  potential design issues are discovered, a difficult process of assessment of field units, and possible recall,  may be required. While design changes to established products are essential to improve reliability,  usability, and robustness, the risks of unintended consequences must be understood and discipline in the  assessment and incorporation of changes must be maintained.  Qualified, Flight heritage products are sensitive to a large number of variables. Proper engineering  controls (print requirements, inspections, screening, etc.) can reduce or detect variability, but not  eliminate them. Every design has areas of sensitivity waiting to be revealed by real-world variations. Any  problem which can potentially occur eventually will, given enough time and repetition. This is part of the  reason for holding ample design margins. However, sometimes these areas of sensitivity are not  analyzed and escape the design team’s awareness. The pin which failed in life-test had reported  mechanical margin of >IO. In a low-cycle fatigue environment, this part failed. Expect problems and  variations in performance, and expect a constant effort to identify and eliminate sources of variation. This  mindset supports overall product reliability more than a rigid notion that a design is qualified and therefore  not subject to variation.  The importance of thorough analysis and review of a design goes without saying. Design intent must be  clearly communicated in engineering drawings. We must understand all features of the design which  contribute to performance, not just the critical features, in order to understand the impact of changes. This  is not as easy as it appears. Discipline in assessment of changes is crucial, since well intentioned,  seemingly innocuous design changes can have unintended consequences.  Definition of a qualified product must go beyond passing a single test program:  Qualification programs are bounded by the realities of cost and schedule, and may be driven  by an individual risk assessment for a given program.  Number of qualification units is typically limited.  Testing is normally confined to one production lot.  Testing at extremes of piece part tolerances is not feasible. 0  0  0  0  The point is to understand the limitations of a “qualified” product. In addition to qualification testing the  product, the process for manufacturing and testing that product must also be qualified. Ask:  0  0  0  0 What are the critical design features, and how are they to be controlled and inspected?  What screening steps and process controls are needed ensure expected performance?  What are the greatest sources of variability in production and how can they be mitigated?  Can performance margins be verified in-process, or at least periodically?  A well developed process control plan is critical to keeping variability in check. Thorough in-process and  acceptance test programs combined with inspection plans can be effective at detecting process escapes  and discrepancies. Consider periodic re-qualification activities, such as a first article “qualification” with  new production lots of critical parts, especially if a new supplier or process was used.  Flight Heritage is only one factor in assessing reliability of Flight components. Other arguably more  important criteria are robustness of design, stability of processes, rigor of the test program (acceptance  and qualification) and demonstration of margins. Do not rely on analysis (or manufacturer claims) alone to  verify margins. Perform tests, and incorporate tests into the recurring process. Reality frequently has  more variability than the analysis, and its supporting assumptions, can capture. Though our analysis  87