good morning ladies and gentlemen the panel will come to order your attendance i specially want to thank the witnesses for their very diligent preparation had the opportunity to review the testimony had a time it looks like we're going to have a very engaging and meaningful discussion this morning this is the last of our series of hearings looking at the first question that we're going to be looking at in our inquiry the members will recall that we're proceeding on a series of questions first of which is can we design a series of metrics that accurately measure the difference if any between the price paid by the taxpayers and the value received by the taxpayers in the war fighters for the systems and services that we are buying we're going to proceed after today's hearing with our second category of questions which is really hypotheses as to what's gone wrong we're going to have a series of panels talk about their series an analysis of why we have a difference between the price paid in the value received the first of our two panels on this metrics question dealt with the sort of measuring the  orthodox algorithms that are used in the major weapon system side we had a panel on that and on the services side we had a panel on that the purpose of today's panel is to bring in some people who we think perhaps look at the whole question through a different prism they're willing to give us some new perspectives through which we can analyze the difference between the price we pay and the value that we receive we have three witnesses with a wealth and brett experience  the acquisition field but also i who i think benefit from what i would call a healthy distance from the daily responsibilities for that function so they can give us a perspective that marries experience with a fresh perspective and hopefully w be able to use the testimony as witnesses to go forward draw some conclusions about the best metrics to use in our work and then proceed with our work in analyzing the various hypotheses given for why we have suffered these cost overruns i also want to take a moment and thank mr conaway who can't be with us this morning but he is ably represented by mr kaufman and all the members of the panel for their excellent contributions to the work on bill which is passed the house will be meeting with the senate informal conference at four thirty this afternoon to i believe but the finishing touches on that conference report so we'll be able to proceed to the floor in short order on that get the bill on the president's desk each member of the panel has made a very significant and welcome contributions that after having said that as the chairman of mr mchugh have said we think that that effort covers about twenty percent of the landscape it looks at a major weapons acquisition which is certainly needed to be examined but there are so many other areas that fall outside of that that the panel has to pursue as well as frankly reviewing the early stage implementation of the law that we believe the president will sign this week so we are by no means at the conclusion of our work really at the outset and one thing we've asked the witnesses to do this morning  to think about the fact that although the congress is about to pass major legislation dealing with major weapons systems we have not yet addressed the hardware that doesn't fall into the major weapons systems category fifty five sixty percent of the acquisition of the services that we have to look at on behalf of those who wear the uniform and only half of the taxpayers we're glad that you're here and i just want to turn to mr kaufman for his opening statement will proceed with the witnesses after that contracts the purpose of this hearing is to think outside the box and how we should measure performance and about what we should be measuring and less about how we are doing it today our previous hearings revealed that current measures of performance tend to break down if the program baseline is unrealistic we want to know if the program can be corrected if the problem can be corrected furthermore are there metrics beyond cost and schedule performance of value such as how closely does the delivery system meet actual warfighter needs as a time of delivery of operational capability satisfy warfighter needs how do we determine whether optional or tradeable capabilities requested by the war fighter  affordable mr chairman we have a distinguished panel of witnesses in front of us today i have looked at their written testimony and i look forward and i look forward to their testimony there are a couple of points i would like to highlight although mr  oversight expertise imposing unique reporting requirements assessments and reviews he goes on to state that in regards to adding additional acquisition workforce professionals that these new people should not be in the pentagon but instead where the execution of programs occurring quote as chairman i think this is an important point that i believe we need to follow closely another point or observation i would like to make is in regards to requirements and joint programs we have many programs out there that have the word quote unquote joint in front of them but they are joint in name only i understand that the current requirements generation process is called the joint capability integration systems or jason it seems to me that in order to have a truly joint program that the requirements for that program must be born joint yet i do not believe our current system or as mr dylan describes current tetons or such an environment so i would be interested in hearing from our witness from our witness in this regard i also encourage our witnesses to share their views on existing laws and regulations that are particularly helpful or not helpful to the department's efforts to obtain th best value and capability for our fires we have heard from the department in two separate hearings about how they currently measure performance and value on contracts what we haven't heard enough about is how they should be measuring value consequently your be included in the record and without objection the statement each of the three witnesses plus the supplemental for statement will be included in the record of the hearing i think each of you at this process you know that we generally have a five minute rule where we ask for seeing summaries of your written testimony we're going to be a little bit more liberal with that this morning but then we're going to turn as quickly as we can to the questions from the panel and get into the give and take i'm going to read a brief biography of each witness and then mr patterson we're going to begin with you mr dave paterson is the executive director of the national defense business institute where which where he is establishing which he's establishing at the university of tennessee in the college of business administration it's an institution inspiring business innovation for both government and industry by providing practical sound assistance in creating economically efficient and effective defense business and acquisition his current duties he was the principal deputy undersecretary of defense comptroller as the principal deputy he was directly responsible for advising and assisting the undersecretary of defense with development execution oversight of the d o d budget exceeding five hundred fifteen billion dollars with annual supplemental request of more than one hundred sixty billion dollars from august two thousand and three to june two thousand and five mr pattison held duties as the special assistant to the deputy secretary defense in that capacity he was responsible for managing the deputy secretary the secretary defense as personal staff as well as providing direction advice to the office of the secretary defense staff on a wide range of national security operations and policy subjects he serves in the air force from one hundred seventy nine hundred ninety three retiring the rank of colonel during that time he held responsible leadership and management positions with assignments at the airwing level as a c five a aircraft commander and deputy operations group commander at major command headquarters u s air force the office of the chairman joint chiefs of staff the office of the secretary of defense and the specter general in one thousand nine hundred sixty was the air force fellow at the american enterprise institute and served in vietnam flying zero two days as forward air controllers thank you for your service to captain following a career that included three operational and major acquisition command assignments mr fitch culminated his tour in the navy as the major program manager for the international and joint multi functional information distribution system it was a nine hundred ninety seven recipient of both the david packard excellence an acquisition award and the department of defense value engineering award in two thousand and one after nearly three years in the defense industry mr fitz joined the faculty of the defense acquisition university teaching and consulting acquisition executives and program management and other acquisition disciplines in two thousand and six he led a major independent study of the coast guard deepwater program in july of two thousand and eight he assumed the position of director a t l leadership learning center of excellence after nearly seven years as the dean of the defense systems management college mr fitch holds degrees in business in industrial management from san jose state college and mass in education from the university of southern california and he is a graduate with the highest distinction from the naval war college in newport rhode island thank you mr fitz for your service to our country and we're glad you're with us this morning dr daniel nussbaum from two thousand and four to the present has been a visiting professor at the naval postgraduate school operations research department from one nine was the director of the naval center for cost analysis at the office of the assistant secretary of the navy for financial management and comptroller here in d c nine hundred eighty seven in one thousand nine hundred six he was division head of the naval center for cost very distinguished servant of our country three excellent people with experience and fresh insight we're happy to have you with us mr patterson we'd like you to begin with your testimony thank you mr chairman members of the defense acquisition reform panel i'm very pleased to be here this morning to participate in a discussion of a question of those clearly captured the attention of the current administration and congress should congress assist the department of defense in improving its acquisition of weapons and services so that it can meet the needs of the war fighter in the field while still being a good steward of the taxpayers dollars the first consideration for judging the success of an acquisition program is whether it fielded a weapons system or information system or service in time to make a positive impact for the war fighter system or service field and too late to need may as well not have been bought at all the phrase too little too late can mean lost lives but before we look at the measures of the acquisition system eric there is another consideration central to this discussion when secretary gates made his budget announcement on april sixth two thousand and nine i believe he was speaking from frustration that was as much about what has been the persistent problem with the acquisition systems that we depend on that is simply not responsive to the immediate warfighter needs as it was about winnowing bloated failed and unnecessary programs implicit in that expression of frustration is a clear lack of confidence in a system that actually produce programs with uncertainty and instability the most dramatic improvement metric will be when the senior leadership in the administration congress and the department of defense have seen such improvements results not words that they can say they have renewed confidence in the stability predictability and effectiveness of the defense acquisition system shit defense acquisition performance assessments report contended that program stability and predictability were singularly and uniquely crucial to managing programs that were on cost on schedule and performance to that end in the time i have allowed me to describe two areas of improvement for measuring program effectiveness worthy of attention first major defense acquisition programs or am doubt often started milestone be the beginning of engineering and manufacturing development with critical staff positions vacant percentage of critical staff positions filled at milestone b is an easy and important metric to be observed it makes little difference to implement programs to raise the level of skills of the program staff if they're missing in action second the acquisition strategy document that is to lay out how the weapon system is to be acquired the initial roadmap if you will is often flawed in that it focuses more on presenting a case for required capabilities and quantities then on laying out the reasoning for acquisition competition methodology for example how the prime contractor participates participants in an m dep competition will select subcontractors and how the winner of the competition will manage the subcontractors to gain improved efficiencies and effectiveness are generally given little consideration if the creation of the acquisition strategy document is one of if not the most important tasks that government acquisition program management can undertake the strategy should establish the template for all the activities that will take place throughout the source selection process engineering manufacturing development and follow on production and fielding more important it establishes how the program management team is thinking about the numerous events and activities that a program will encounter the defense acquisition executive  of strategy elements that all the military department service acquisition executives must include in them data acquisition strategy documents additionally a set of standards or metrics by which the strategy elements can be evaluated as effective must be part of that process in closing i'd be remiss if i didn't acknowledge the progress that has been made by the department in improving the acquisition system over the last four years though it is the general accountability office is headlined that the ninety six major acquisition programs have grown in cost by two hundred ninety six billion dollars that gets attention those numbers be a lie and equally worthy but overlook statistic published in the very same report the average increase in unit cost of the twenty eight m to program with less than five years since development started is only one percent compared with an average of fifty five percent unit cost of twenty five programs in the group with five to nine years since program development start there has been important improvement that should be recognized with that it's my privilege to be here thank you very much and i'd be happy to take any questions that you have thank you mr patterson mr fitch chairman andrews congressman coffman members of the panel thank you for the opportunity to appear here today i will address the subject of acquisition performance metrics and your questions about how to increase the realism of the program baselines making trades between affordability and performance and how to assess the value of systems that do not necessarily read my please recognize  do not necessarily reflect the views of the defense acquisition university the definitive defense or the administration is why were interested i suspected as much measurement of acquisition performance must encompass both strategic and tactical elements as emphasized in a recent defense science board report titled creating the d o d strategic acquisition platform the management execution and oversight of acquisitions programs is moot if we aren't spending taxpayer dollars to buy the right capabilities if we are demonstrating a strategic choice it is as important to decide what capabilities we won't buy as well as what we will buy i believe one of the root causes of funding instability is what i describe as too many programs chasing too few dollars too many programs chasing too few dollars is one of the root causes i believe of overly optimistic cost estimates the recently implemented material developed decision process provides a framework for making strategic investment decisions the m t d is the form point of entry into the acquisition process m d t willing crease integration of the three major acquisition support systems requirements resources and acquisition improving requirements management and initiatives supported by the congress includes providing training to requirements writers and managers to ensure that they have a sufficient understanding of the critical as systems engineering and testing the goal is to improve collaboration between acquisition requirements communities to exploit cost in performance trades and improve acquisition outcomes having a formal requirements and acquisition process is not unique to the d o d we can learn important lessons from commercial industry if you compare the d o d acquisition system with the process to develop electronic games there are marked similarities and differences notably the process to get games on the shelves for the december holiday season starts with a precise clarity of what will be developed by when and includes a corporate commitment to the resources required for the project precise clarity is a result of intense interaction between the people the divine the capabilities of the game people who will design and test the software turning to the subject of tactical  effective tools and templates incorporate metrics both quantitative and qualitative the question was raised are there metrics beyond cost and schedule performance that are of value the answer is yes and an ongoing example is a probability of program success metrics props that are currently being developed and deployed across the services and other federal agencies such as the department of homeland security starting with a blank sheet of paper a group of faculty comprised of experienced pro managers and other functional experts ask themselves questions including what conditions facilitate the success of programs what metrics or leading indicators of derailment the resulting tool plops uses a structured process to continually assess and display key elements of planning resource execution and external influences the promote or negatively impact program success still evolving perhaps boxes being used in the army air force navy marine corps and coast guard family accurate and transparent metrics integrated into management oversight process will produce better program outcomes another question has been asked and we ensure improved realistic baselines again i believe the answer is yes and there are ongoing issues that will yield more realistic baselines these include increased emphasis on technology readiness piece of before starting a major development emphasis on improved cost estimating which i'm sure that we'll talk about today and competitive prototyping which is included in the new five thousand two and pending legislation prototyping increases the opportunity to identify an assessor for debility capability creates competitive prototyping also allows the government to observe the performance of competing industry teams before making it down select for engineering and manufacturing development no matter how thoughtful we plan and discipline source selection a paper only source  like selection process is never good as demonstrated performance the ultimate assessment of whether we have delivered value to the needed capability the warfighter is feedback from the field from the war fighter at various times i have seen photos of messages written with felt tip pens on trucks and personnel carriers that have been turned return of the government or industry depos one of those signed messages a picture is in my written testimony read truck saved my life as well as five others to april or wait at twenty three hundred lima was for iraq those kinds of tests monies those kinds of results with those of us that are in the acquisition aim to achieve on a daily basis by our efforts into minute before equipment is fielded it goes rigorous levels of development and operational testing the new d o t five thousand has increased emphasis on earlier testy mr chairman thank you very much for the opportunity participate in today's discussion as the secretary of defense has said there is no silver bullet it will be a combination of initiatives and collaboration amongst all the stakeholders to create an acquisition system that consistently produces successful acquisition programs i look forward your question mr fed's thank you very much for your testimony dr knot spawn mr chairman distinguished members of the panel i'd like to thank you for this opportunity to discuss my thoughts on how to improve acquisition and cost estimating processes in the department of defense these ideas are mine alone as the chairman said i'm a member of the faculty at the naval postgraduate school in monterey california and i've spent the last thirty years mainly doing it more recently teaching and researching in the defense acquisition management system with a focus on cost estimating i was a previous director of the naval center for cost analysis and past president of the society of cost estimating and analysis all my experiences and cost estimating confirm that three things are necessary for sound cost estimating acceptance of the underlying uncertainties in predicting the future accurate and plentiful historical data and professionally trained  certified personnel on uncertainty there's an intrinsic uncertainty in all estimates it derives from several sources mainly that we're usually designing or building or operating something that is substantially different from what we did before the difference could be in the product or in economic conditions or the programatic conditions an estimate reflects our knowledge at a point in time when we freeze problem and base the cost on the configuration and program as they are understood at that time and that baseline many things can change that can also change the cost estimate including labor rates overhead rates schedules and handsomest the capabilities or quantities changes when a particular technical solution to a problem doesn't work as planned then we need an alternative technical solution on data a hallmark of a necessary characteristic of a sound cost estimate is that it is based on historical program performance from similar or related ongoing or past programs historical data is variable not every aircraft cost the same as every other aircraft and the measurement of this variability is accomplished through statistical constructs things like standard error of the estimate things like confidence intervals we assume in our community of cost estimating that the patterns of the past will repeat in the future but these patterns are almost always statistically patterns modeled with the powerful and subtle techniques known collectively as regression further we know of no alternative approach to using the past as a guide to the future if we want to scientific that is a reproduc ought to blow approach not all estimating is done by government employees there really are three sub communities one is the government in house estimators one is employees of the large vendors who also design develop and build what we buy boeing's northrop lockheed martin for example and thirdly there are support contractors or consultants to the government those of the three communities and surely we need to increase the capacity in the quality of the numbers  the training of the government estimate is but so do we need to enhance the professionalism of the other two communities there are currently no undergraduate curricula in just estimating and there are only fo educational institutions that i'm aware of that teach at least one course in cost estimating and those are the naval postgraduate school in monterey where i am the air force institute of technology at wright patterson in ohio defense acquisition university diverse locations with a capital campus of fort belvoir and the massachusetts institute of technology which offers one elective course and cost estimating within its engineering curriculum the recent separation of the business cost estimating and financial management career field into two separate cost estimating and financial management tracks is a very welcome development and should be supported but note the dia you support is largely limited to military and d not the other two communities the society of cost estimating and analysis we say skia whose membership includes approximately one third of all cost estimated supporting d o d is a central and indispensable player in the training initial certification and periodic recertification of cost estimate as i know it with pleasure that the executive director of scale mr elma plague is in the room has collected a body of cost estimating knowledge that it provides the members of the cost estimating community provides training and cost estimating has developed and offers an examination and experience based certification program lockheed martin boeing north of grumman other vendors use skiis training and certification as their standard i appreciate very much what this committee seeks to accomplish mr chairman this concludes my prepared statement and i would be pleased to answer your questions for thank you d thank each of the three witnesses for their testimony as to say we've had the chance to review the written testimony we're now going to get to the questioning face mr patterson i was intrigued by your reference in your written testimony to two kinds of requirements cost which you express as customer requirements and derived requirements what's the difference between the two a customer requirements are stablished and key performance parameters which are requirements that are established for a particular weapon system that the weapon system then must perform against derived requirements or those requirements that the customer did not ask for but that as the name would suggest to arrive as a consequence of the design process oh look we could do this better if only we and of course that's taken to the program manager the program manager will say ok we can do that just bring money and they do now i notice on page seven of your testimony that you say that a recent study prepared by the monitor company group were based on selected acquisition report data estimates that approximately thirty three percent of the cost growth from two thousand two thousand and seven i think i read this is attributable to this second category of requirements you're talking about right the derived that's correct yes or no how how did how was that number reached where did that thirty three percent come from that comes from the selective acquisition reports in fact they laid that out in the stars and explained in each of the stars where the  one things that we want to do is to make sure that we discriminate between derive changes that are beneficial and those that may be suppressed phyllis how would you suggest that we might do that in other words i don't want to leave the impression that we're saying or the report saying that oh that thirty three percent was wasted oh absolutely not how do we draw the line between beneficial cost effective derived requirements and not so beneficial derive requests it's well let me give you a  example here in one thousand nine three i came to the c seventeen program it was a troubled problem plagued program the two program managers decided that much of the problem was that requirements were growing and then airplane they established a set of rules the set of rules was very simple if you have a requirement or an engineering change which is effectively a derived requirement then it must go through an engineering change board the engineering change board will evaluate the change or its intrinsic merit but if it doesn't meet the safety of flight or other driven requirements then it has to have a three to one pay back in savings and not part of a to schedule interest real simple now in terms of the size of the scope of that c seventeen program how how much were the derive requirements over on him another was how much was attributable to derive requirements i would only be guessing i can certainly get that for the record when would this method that you just described in your judgment what percent of the derived requirements were beneficial and which failed to make that three to one cut and didn't happen again let me give you an example at edwards air force base they were testing the airplanes one of the requirements was that one airplane needed to start another plane but it started it with two hoses from the domestic system somebody said i wonder if it'll start with one hose it didn't oh my gosh well now we have to go and figure out why that's the case  so that took a considerable amount of test time and money and as it turned out they really wanted to have it start with one hose i'm sorry that that would that would have taken to lose a son who was the someone who said that not the person name but where were they and that was a guest community out on the ramp and all right i just i'm going to go back for a second round but mr fitch mention this probability of success metrics program has there been built into that litmus test for a probability if you fall below it things stop other words to that program haven't had a build in go or no go line it does not have a go or go line it is is information that is updated and provided you think it should i think that there needs to be an informed review in decisions that look at each of the things that occur that is negative because some of the things that are in the probably success metrics in reporting are positive so i think that it needs to have the program manager when that information is put together on a monthly basis look at that need to be reviewed by someone in his p potentially even at higher authority when there are negative occurrences could i add something about your and then i'm going to go to mr kaufman sure i just want to explain that the drive requirement process is actually part of the systems engineering process so that if for instance the war fighter says an aircraft i need to display that has color i need to display that has this amount of resolution in the first pass even by the war fighter or the cost estimate or some may say this amount of processing may be sufficient for that when you get into will by the way you're going to have these other software capabilities you put those together and they start to build on one another you can find all of a sudden that the processor that was planned maybe a commercial off the shelf item with x amount of memory through putting cetera is insufficient at that point is the point where very useful to have the requirements community have a real dialogue with the acquisition community to say do  i now take away some of the requirements so i continue to use that commercial off the shelf processor but is less expensive or is are those requirements really important now that we have figured out what they are but so the point is the derive requirements is a very important process and it just it is actually the process where you allocate features will go to hardware software thank you very much mr kaufman chairman of this  go into fixed cost production with with another entity maybe maybe that person that would be allowed to bid on it as well but it is it is better to bifurcate the process you know i would say we would do that mandated technology readiness levels t r l's of certain levels before the program can get beyond a milestone they've narrowed that cone of uncertainty by saying you have to have a tiara level six prefer seven all have the same strengths they're going to some are going to view that i want to do this more in a hardware function others may be more software intensive their different technologies involved their proposal and then watching them deliver on what they promise and they say it is possible and competitive prototyping is key to them being able to transition and make an award to that contractor who has proven that they did with they say they could do so having a different contractor if i understood your your premise correctly to develop technology and then awarded to somebody differently to develop the system i think that's not the intent of the comparative prototype system i think too there's a  problem and that that that arises when we talk about spiral development originally viral development evolutionary development that we're designing to to have a weapon system that effectively was fielded and then have block upgrades to improve the weapons system in many cases that is not how that works because you do have a parallel r and d program that is working on improvements and before the weapon is actually fielded you have the attempt to integrate improvements well the consequence of that often is that you have a stretched out program costs escalate and in the end performance degraded and it takes a while for the much longer than had been anticipated for that weapons system to be fielded so i think we need to have very specific and clear understood standards by which we will integrate or put in  upgrades or technology in facets into weapons systems it certainly can't be during engineering manufacturing development i think it was mentioned has been mentioned repeatedly that the changes drive a lot of the cost  this interior requirements you know i think that changes come from all sorts of places including changes in labor overhead rates requirements quantity absolutely everything just like building a house everything that changes has a contractor saying cement is gone up break is gone up my subs of gone up or down but they're always changing and so there's a great some of it is part of life and some of that we try to control by saying tell us what you're building in will cost that program but i think it's just intrinsic in the cost estimating program anyway to have more discipline i mean what what would be some us what you're building how many what the technologies are and will cost that program maybe that things change after that but that's the program we're going to cost because otherwise we're tracing our tail and we don't know until the very last moment what we're costing and in fact  takes time to do a cost estimate so part of the discipline at o s d is a cost analysis required it's document will be prepared six months before the milestone and the lack of availability of that card results on a day to day slip in the milestone so that's real discipline but it has to obvious downside too i think too that in dealing with the discipline and structure that i must i submit that it requires rules and the rules simply have to say that after a milestone you don't have any more requirements and it's i mean it's somewhat draconian but nonetheless if unless of course it's a safety of flight or it's an obvious design failure that needs to be corrected but those kinds of things are few and far between quite frankly and  and then this idea that a requirement or a change would give you a three to one or four to one payback in savings well not for debating the schedule it's not a bad idea either and as technology moves forward that's entirely possible over the course of the program there are times with the requirements i recall when i was developing a black box that when it was going to go in seventeen different platforms i would have this platform come to me and say it would really be easier if you changed it this way it will make me less expensive for my integration save my cost and schedule what i came to understand is the program manager has to be prepared to say  during the development phase now the configuration stream boards we have in place and to supply eights that there may be compelling reasons to change or requirement model right requirement during development but that's the process of good figuration steering board is to raise that if you would visibility to the pressures of changing requirements to have a senior level decision made about the requirement once the milestone has been approved thank you mr cooper thank you mr chairman not especially like to  paterson and congratulate him on starting defense business sense to today at the great university of tennessee and knoxville thank you not only for your past service but what you're doing right now can anyone provide me with enough historical perspective to help me understand how during world war two men like henry kaiser rival to produce destroyers and i forget exactly who produced airplanes but it was an amazingly productive period and i don't know how was there a lot of iraq received and hours of how it was so much able to be accomplished so quickly anybody now my reading of history is that there were a lot of failures and then we tend to remember the successes which were terrific but the fact they were false starts with a p fifty one was wonderfully successful but because it had a bigger gas tank so it could keep up with the other aircraft it gave us greater range but we didn't build it for that reason just the sort of locked into that if you will i think that things were much simpler we were able to turn technology generations around much faster and therefore absorb the failures today's systems of very complex they take a long time and we just won't accept that you and i think i think that will remember a lot is those ships going down to the wages ok da ok that was in the production phase even the p fifty one it was just mentioned by mr nussbaum you know when it was first deal that it didn't have the engine that ended up being what everybody remembers about the performance of that aircraft there were various modifications made afterwards feel that it was in the field and found that it was not what exactly was needed i think two you had a tremendous industrial base that was able to accommodate to the level of technology that it was asked to accommodate to today i'm not sure that we would be able to do that again in that amount of time we have a dr ron sega did a study  long ago when he was d d r any director of defense research and development in which he looked at the industrial base and whether what it could do and found that sixty two percent of all of the ph d candidate that are enrolled in disciplines critical to national security have temporary visas and that tends to put us at a disadvantage because they generally don't stay around and work for a lockheed or northrop or boeing in skunkworks particularly phantom work and if you'll look at the aerospace industries association data you'll find that in the one thousand nine hundred ninety one you had one point three million touch labor workers highly skilled workers today we have something less than seven hundred thousand those are statistics that should give us cause but paused to consider what we're going to do in the future to help me understand i know that we've had failures acquisition throughout our history and mr fitch mentions a success when steve posts the note that was on the call the same lies but on the front page of the washington post recently was a statement by secretary of defense gates when he attended the return of the remains of some of our troops he asked how they died and he was told they were in an inadequate vehicle and he cursed because that was symbolic of the fact that we've had difficulty fielding relatively simple platforms like m wraps up armored humvees milly's for some period of time there was only one manufacturer of up armored humvees in america and yet at the same time we have automobile companies going bankrupt and looking for vehicles to build so there seems to be a mismatch somehow between pretty basic demands for troops and our ability to field and source of those even when we have able and willing automobile companies you know who are looking for ways to keep their plants busy how why the mismatch oh i think it's it's it's a very fundamental problem in that you have a department of defense that is that understands fully that we're at war with terrorists and you have a country that doesn't so chrysler or g m or ford didn't want to bid on the farmers on the car we couldn't well actually not a pretty area of those folks did but they really were not in the business of of putting out those kinds of vehicles people who were international harvester obviously and but the design fortunately or unfortunately for us that was available were foreign designs of up armored vehicles that were designed to sustain the kinds of things that an emirate would have to sustain of and then the number of companies that have as a core competency and the technologies for for armor ok is not the same as we have for the for the auto industry in fact we're doing a lot of investments f today to try to find better armor cheaper armor especially lighter armor whether that be for the vehicles because when you put the armor on it it it puts a demand on the engine it puts a demand on the drive train you have if if you double the weight of the vehicle whatever the percentage is ok it has additional tolls upon upon the reliability the vehicle so we don't have the same industrial base as the auto industry is surveys already are in a sense the humvees the wrong vehicle to up armor but it's the only vehicle to up armor but it was built as a replacement for the jeep which didn't go in harm's way so it was zion to optimize all the functions that it was going to perform and we missed the fact it was going to with the panel as well so it fits your schedule thank you mr patterson just made a suggestion that we might have a rule that says no new requirements after milestone b and that prompted  this question again this piece of data you cite about thirty three percent of the program cost growth being attributable to these requirements changes how many of those requirements changes happened after milestone b in your oral testimony you talk about the fact that if i read this correctly the average increase in unit cost of the twenty eight m debt programs of less than five years since development is only one percent of correct that's comparable but it's fifty five percent from yours five to nine how many of that in the fifty five percent category the five to nine got a waiver through milestone  didn't meet the requirements to get the milestone even got wave passed it do you know you know i don't i don't know exactly how many but i mean it would be x to say it's probably most of the fifty five percent still run comes from that i would say a significant portion  one of the things that's in the the conference report that we'll be looking at in the way still today is what we call intensive care where if a program is permitted to go forward even though it didn't achieve milestones the criteria has waived past that there's a whole set of intensive requirements that are imposed upon that program to try to get it back under control i want to come back to again you're the spy for  create between derives and and customer requirements describe for us the process that you think ought to be instituted to determine whether a derive requirement is added to the package or not let's say what we're at we're at a point assuming for a moment that we except your proposition that they're not after milestone b which i assume you say there should be some exceptions as you said for true safety or emergency purposes but let's assume we're living in a world where except for those narrow situations they're going to be no changes in requirements after we had milestone be we're now in pre milestone b and all by the way comes up as you said earlier by the way this can this can do this who should make the decisions whether that gets added to the package and by what criteria i think that the program manager should have the initial cut whether or not they're going to include that into the program but program managers generally are colonels are very large programs for generals who have significant oversight within the department  think that that if they have a set of rules that say first of all if it's not a safety of flight or if it's not some sort of safety issue or if it doesn't give me a return on and then i'm going to have a thumbs down initially how do you measure the concept of returns well let's take for example you have again i'm going to return to two to the b seventeen there were parts of that airplane that were originally designed for aluminum lithium for example well the minimalist is is strong but it's brittle so an engineering change was made to change that to a different a oy we were breaking lumen i'm lift the cargo cult floor guides at a regular pace so changing that eliminated the problem of having to constantly replace that was a savings and those are the kinds of things that i would suggest are three to one and even the suppliers were given an opportunity to do that strikes me that this is really the essence of the twenty percent the secretary talks about in his eighty percent solution that what he really is aiming to get out here is to give us an adjudicatory mechanism that the term that draws a line between the eighty in the twenty when you get to this point and it's your suggestion the program manager should be the first person to weigh in on this who should evaluate his or her recommendation of then you have an then you have an engineering change board that would provide a corporate view of it and if it's a particularly expensive change and you're going to go to your service acquisition executive or if it's a cat one day and acquisition category that is then you have the the defense department acquisition executive who would have a kind of that and my final question is you confident that we can quantify this concept of value sufficiently to hit the three or four hundred other words are all of the values that we want to promote you gave a great example of you know saving replacing a piece of a plane that's going to go wrong but aren't do all of the value concepts lend themselves to that kind of quantification that would let us say well this fails to meet three to one so out no it's many are much more difficult than that and that's why it takes a lot of of research and study to set up standards and conditions whereby you can evalu and conclude this is one of the reasons why we're glad we have the three of you in the institutions that you represent because we really do turn to institutions like yours to assemble those data analyze them and give us a factual replace current capabilities it's always hard to measure the value of things that represent new capabilities they don't represent don't represent a savings for the operating and support tail that you're going to occur but if you are replacing a current capability it's pretty easy to do an estimate but it's still an estimate of what it cost to invest to make this thing happen and what do you save over time and the operating i agree the much more difficult proposition is where you have a new function that could be added by something that you discover how do you measure that and that requires tradeoff analysis that requires opportunity cost analysis requires a lot of broader inquiries and if you have that long tail the question then is do you do any discounting on it the technical issue of net present value and what rates m b helps us there but not to be hype not to be hyper technical but one of the problems is then matching up the federal credit scoring a net present value rules with the real world once it's very often decision here a classic example is in energy the department has guidance to hit twenty five percent alternative fuels by two thousand and twenty and in order for them to do that they need to do multi year contracts but to do multiply your contracts the l scores that as putting the whole net present value into one a year which makes it almost impossible to do which means we don't do much of it which means we're falling backward so marrying the c b l criteria would the real world criteria is a bit of a challenge to dollars to make this thing happen and the promise is that you will return when asked dollars later on that that's a nice conversation but it doesn't accord with the budgeting realities and it also does score for in our person or should i think mr fyshe you want to add one thing and i'm going to to deal with industry to build something those need to be translated into technical terms we also use the term the ride requirements for that process drive requirements are really there's a couple types we're talking about here that which are a part of the normal system that you have to  do the other one the other one i just say is your question is how do we know we get value as a program manager i had an acquisition program baseline that was my contract it was my contract with my milestone decision authority and with the american taxpayer to produce a understands some subset of derive requirements are quite legitimate necessary and desirable but we're led i think your test of the three ideas given us some interesting tools to discriminate between undesirable derive requirements and desirable ones which is what we're about mr kaufman famous chairman we talked about today about having discipline in terms of change requirements but do we also have to have discipline when programs get so far out o line that they become questionable and i'm thinking about and one if you can reflect on the president's i can't remember the name nomenclature of the fallen helicopter littoral combat ship is in question i wonder if you all could reflect on when a program gets so far out along but well it would be helpful i think in the total scheme of things if we never let them get out of line but nonetheless you're exactly right they do happen to happen and and one of the things that i think is difficult when you take for example the seventy one the president's helicopter and that is a perfect example of you came in with one set of criteria as requirements and over the course of the time it changed dramatically i think that we don't have a set of standards or conditions that raise a budget flag that say wait a second i'm sorry you you're read here and you've been read three reviews in a row and we're canceling your program and what we do is because we put ourselves in a position where it's the only game in town you have the president's helicopter that is arguably old and we don't have an alternative you chose a ban helicopter and there is no off ramp there's no plan b and we do this rather consistently and i'm almost of the opinion that we do it by design and i would offer the marines  e f e for example we don't have a way to walk the dog back down the path to get and fill b s vehicle that would replace the f a so we need to make that  work i don't have the particulars on either of the programs he asked about i was the systems engineer for the v h three d the current one of the two current presidential helicopters so unique mission i think there's a desire usually if the white house says  walk of work who ever it is says i need a capability that you find a way to do it going back to it most of the changes i thought i saw in the pressures and the surprises that i got occur in that first traditionally the old first twelve to fifteen months of the program when you got towards p t are the contractors off doing a lot of things the reason a lot of of object at the level is to figure out doesn't get it as it doesn't get it i think that the competitive prototyping system the focus on doing p p d are as earlier to have affective communications with with the contractor teams and oversight of the contractor teams during that period of time will do much more to get to a stable baseline at milestone b initiation going to sound like a professor on one hand on the other hand because there are some historical examples of things that failed failed failed and then were terrific he just was one and tomahawk was another they just took a long time to bring bring the board and for somehow some reason we had the fortitude to stick with them and not say this isn't a twelve or a gamma goat and get rid of it and now we come to the category of l c s is it a tomahawk or is it a gamma goat and i don't  know but i'm taken with my colleague's remark that once you get past a milestone b if you're if you have three reds in a row you have some serious explaining to do with the presumptive answer you're out it's a rule but the problem is knowing the future and that's always the problem i don't know whether l c s is a tomahawk which is going to be absolutely terrific after a long incubation period and the same for v twenty two you just don't know so we make decisions as people thank you gentlemen thank you very much for your expertise we will be your reward for doing such a good job as we'll have to call on you again as the committee goes forward our intention is to try to make legislative proposals for the f y two thousand and eleven authorization bill that will deal with the area of the problem that the waste legislation that we're dealing with today is not deal with i think you've given us some very intriguing ways to measure the the gap between what we pay and what we get it's also heartening to hear what i've heard this morning a consensus that this panel's contribution to the wast bill was essentially two concepts the first was to add a whole series of reviews and a lot of scrutiny pre milestone b with particular emphasis on the requirements process and we did make it into the conference report and that will become the law this week we think and second the panel is very interested in much more rigorous what we call intensive care of systems that pass milestone be by waiver that have not met the requirements or that fail none mccurdy standards and yet exempted from the penalties there and go forward anyway and i think if you look at the cost overruns a huge majority fall into one of those two categories so what we wanted to do was to take the best practices that you're very able to describe this morning and engage them as intensely as we can in the systems that again never met the criteria to get past milestone be to get past it anyway and those that fail none mccurdy but continue to live on and as as mr patterson said a few minutes ago that was mr patterson our ultimate goal is not to have any of those cases in the future by unraveling the requirements process and looking at it more intensely to intensify that pre milestone be analysis of what is going on the other point i'd make you know that's more on our side of the table than yours i think that the principal reason that we get these cost overruns is that once something passes milestone be enormous political constituency that develops around it and there are tens of thousands of people deriving their paycheck from a project hundreds or thousands of subcontractors you know dozens or hundreds of congressional districts  and as secretary gates and i think can attest making changes in those programs is very politically difficult if you get to these flawed programs earlier when their political constituencies are smaller and weaker the chance to do the right thing is a lot higher so not just for analytical reasons but we think given the dynamic of the way these decisions are made in the political world the more precisely our measurements and the more focus we are in our evaluation in the requirements phase and a little bit beyond that we think the better job will do so i would say to each of the three of you we welcome your continued participation  we're certainly going to call upon you for your feedback as we go forward in our drafting process and thank you very very much for your time and attention this morning members will have period of time by contacting either majority minority staff to supplement the record with written questions and we invite the witnesses do the same thing with that hearing is a