Human Capital: Opportunities Exist to Further Improve DOD's Methodology for Estimating the Costs of Its Workforces
Released: 2013-09-25T00:00:00

Letter

Background

DOD’s DTM 09-007 established business rules, or standard procedures, to estimate and compare the full cost of military and DOD civilian personnel and contractor support. These rules were incorporated with amendments into DOD Instruction 7041.04, which supersedes the DTM. According to the DTM and the instruction, when developing national security policies and making program commitments, DOD officials must be aware of the full costs of personnel and have a thorough understanding of the implications of those costs to DOD and, on a broader scale, the federal government. To facilitate this awareness, the DTM provided and the instruction provides business rules for DOD officials to estimate the full costs of the defense workforce and contracted support for tasks supporting planning, defense acquisition, and force structure. According to the instruction, the Office of the Secretary of Defense (OSD) and all DOD components are required to use these business rules when performing an economic analysis to support workforce decisions, such as determining the workforce mix of new or expanding mission requirements that are not inherently governmental or exempt from private-sector performance, and in-sourcing.

Table 1 shows how the DOD components can generally use the business rules established in DTM 09-007 and in DOD Instruction 7041.04 to estimate and compare personnel costs and to support workforce mix decisions.

DOD Instruction 7041.04 does not require DOD’s components to make workforce decisions based on cost alone, but it does require the components to consider cost in the decision making process when the function in question is not required by law, regulation, or policy to be performed by a certain workforce (e.g., inherently governmental or military essential functions) and other workforce factors are equal. In other cases, the cost of using personnel to perform work may not be part of the decision making process. For example, a June 2013 memorandum from the Assistant Secretary of Defense states that during civilian furloughs the use of either military personnel or contractors to compensate for workload resulting from the civilian furloughs is prohibited.

DOD Instruction 7041.04 states that the full costs of personnel include labor costs, current and deferred compensation costs paid in cash and in- kind, as well as direct and indirect non-labor costs. For contractor support, the full cost is the sum of the service contract, the cost of goods, services and benefits provided in-kind to contractors or reimbursed by DOD, and the costs to DOD in support of the contract and contract administration. See table 2 for a description of the full cost of performance by military and civilian personnel, and contractor support, as defined in DOD Instruction 7041.04.

Further, the Federal Accounting Standards Advisory Board Handbook defines full cost as the total amount of resources used to produce the output. More specifically, the full cost of an output is the sum of (1) the costs of resources consumed directly or indirectly that contribute to the output, and (2) the costs of identifiable supporting services provided by units within the reporting entity and by other reporting entities.

Roles and Responsibilities

The Director of CAPE is the principal advisor to the Secretary of Defense and other senior officials in DOD for independent cost assessment, program evaluation, and analysis. DOD Instruction 7041.04 states that CAPE, in collaboration with USD(P&R), and the Under Secretary of Defense (Comptroller), is responsible for developing a cost model for DOD-wide application to implement the business rules identified in the instruction. The instruction also states that CAPE, USD(P&R), the Under Secretary of Defense (Comptroller), and the heads of the DOD components (such as the heads of the military departments) and OSD components, are responsible for using the business rules identified in the instruction. This includes using the business rules to estimate the full costs of the defense workforce in support of planning, defense acquisition, force structure decisions, and when performing an economic analysis in support of workforce mix decisions.

The DOD Office of the Actuary provides actuarial expertise on all matters relating to military compensation and benefits, including performing annual valuations of the military retirement system, education benefits under the Montgomery G. I. Bill, and health care for the military retired population.

Previous GAO Work on Estimating the Full Cost of Personnel

We have previously made recommendations to DOD to develop a methodology to estimate the full cost of military and civilian personnel. In our May 2006 report, we found that from fiscal years 2005 through 2007, the Air Force, the Army, and the Navy collectively converted or planned to convert a total of 5,507 military health care positions to civilian positions.We found at that time that it was unknown whether these conversions would increase or decrease costs to DOD, primarily because the methodology each of the departments considered using did not include the full cost of military personnel. Accordingly, we recommended, and DOD generally agreed, that the Secretaries of the Air Force, the Army, and the Navy coordinate with CAPE to develop the full cost for military personnel and for federal civilian or contract replacement personnel in assessing whether anticipated costs to hire civilian replacement personnel will increase costs to DOD for defense health care. In response to this recommendation, DOD issued DTM 09-007 in January 2010, which provided a full cost methodology for assessing military, civilian, or contractor support personnel costs to inform workforce mix decisions, including military to civilian conversions.

In our February 2008 report on DOD’s efforts to address legislative requirements to use a full cost methodology to certify and report on planned conversions of military medical and dental positions to civilian medical and dental positions, we found that the Navy’s methodology was the only one that addressed the specific factors identified by the John Warner National Defense Authorization Act for Fiscal Year 2007 for positions planned for conversion for fiscal years 2007 and 2008. The Air Force and the Army relied on composite military rates, instead of using a full cost methodology. These composite rates did not include all of the required cost factors, such as training and recruiting costs. Accordingly, we recommended that DOD, among other things, develop operating guidance for the military departments to use when justifying future conversions of military medical and dental positions to civilian positions. In our recommendation we stated that this guidance should stipulate requirements to use a consistent full cost methodology for comparing the cost of military and civilian personnel. Officials in CAPE attributed the development of DTM 09-007, in part, to these recommendations.

DOD’s Methodology for Estimating and Comparing the Full Cost to the Taxpayer of Work Performed by Its Workforces is Limited in Certain Aspects

DOD Instruction 7041.04 reflects improvements to DOD’s methodology for estimating and comparing the full cost to the taxpayer of work performed by military and civilian personnel and contractor support since the initial issuance of DTM 09-007, but the instruction is still limited in certain areas. For example, DOD’s instruction provides limited guidance on estimating overhead costs and adjusting advertising and recruiting, and training costs. In addition, CAPE has not established business rules for estimating the cost of a part of DOD’s total workforce—Reserve and National Guard personnel. Further, CAPE has not yet evaluated certain retirement and retiree health benefit cost elements that it is using to reflect the full cost of currently employed military and civilian personnel.

DOD’s Instruction Includes Improvements, but Does Not Provide Guidance on Estimating and Adjusting Certain Cost Elements Related to Overhead, Advertising and Recruiting, and Training Costs

Improvements to DOD’s Methodology

DOD Instruction 7041.04 reflects a number of improvements in estimating certain cost elements in comparison to DTM 09-007, addressing some of the limitations users of the DTM and the interested parties we met with identified. While the DTM did not identify a responsible office for preparing clarifying guidance to assist users in applying the methodology, the instruction establishes that CAPE, among its other responsibilities, will be responsible for issuing such guidance. Also, CAPE has expanded the methodology to address specific elements that users of the DTM previously identified as missing, such as the cost of foregone taxes, lost productivity during periods of transition, and some other non-common costs associated with converting from contract to government performance. Lastly, CAPE has been developing, refining, and testing a DOD-wide software tool—the Full Cost of Manpower—that employs the business rules established in the instruction, and the instruction provides a link to this tool. However, while CAPE has addressed several of the limitations users of the preceding DTM and other interested parties identified, certain limitations still exist.

DOD’s instruction provides limited direction on estimating general and administrative and overhead costs, and adjusting advertising and recruiting, and training costs. DOD Instruction 7041.04 states that the cost elements contained in the instruction can be modified or augmented in each specific case as necessary, but that the DOD components should be prepared to support each decision with sufficient justification. Best practices as reflected in GAO’s Cost Estimating and Assessment Guide state, however, that establishing ground rules for cost estimating provides a common set of agreed on estimating standards that provide guidance and minimize conflicts in definitions.

The instruction directs users to include general and administrative and overhead costs in cost estimates. These costs include the goods, services, and benefits that support more than one organization. More specifically, these costs include a share of supplies and facilities and professional support services, and a fair share of the recurring costs of higher-level management and professional support services for organizations that produce or provide more than one product or service. The instruction provides a list of direct included for military and civilian personnel, but does not provide data sources or guidance on how to estimate them. Instead, CAPE identified subject-matter experts in each of the military departments to serve as points of contact to address these types of questions and provide assistance in the future.

DODI 7041.04 defines direct costs as costs that are related directly to the production of a specific good or the performance of a specific service. Typical direct costs are the compensation of employees for performance of work, fringe benefit costs, and the costs of materials consumed or expended in the performance of the work. to contractors through a public-private competition using OMB Circular A-76. However, both the DOD Inspector General and we have found that the standard rate of 12 percent of labor costs does not have a sound analytical basis, which leaves some uncertainty about whether the rate may be understated or overstated. Our past work acknowledged the difficulty of obtaining reliable cost data that could provide a sound basis for an overhead rate, but we concluded that until actual overhead costs are used to develop a more meaningful standard overhead rate, the magnitude of savings expected from public-private competitions will be imprecise and competition decisions could continue to be controversial.

End strength represents the actual number of personnel on board at the end of a fiscal year. advertising and recruiting costs and added that some functions or specialties would cost more to recruit and train. Specifically, the users told us that, for some specialties, the average training rates are too low, while for other specialties, the rates are too high. The users also said these costs vary depending on the rank or grade of the personnel, and depending on whether the specialty requires training or certifications. For example, the average rate for training in the Army for fiscal year 2012 was $6,490 per servicemember. However, according to Army data, the amortized cost of training for an officer with a general aviation area of concentration can range from about $6,500 to $93,600 a year, depending on rank. In contrast, the amortized cost of training an enlisted member with an infantryman area of concentration can range from about $4,600 to $8,000 a year depending on pay grade.

Due to the lack of explanatory guidance on these cost elements, users of the preceding DTM told us they have developed their own methods for estimating and adjusting these costs, or have not included the costs in their estimates. Some officials have requested more developed guidance on these cost elements, but CAPE did not provide more specific direction in its recently issued instruction. Rather, as previously mentioned, CAPE identified subject-matter experts in each of the military departments to serve as points of contact to address these types of questions. However, without more developed guidance that establishes a clear set of ground rules or standards, subject-matter experts in the military departments and cost estimators must make their own assumptions, which can lead to inconsistent estimates, as well as hinder DOD’s and congressional decision makers’ visibility over the costs of accomplishing work by the different workforces.

CAPE Has Not Established Business Rules for Estimating Reserve and National Guard Costs

Although Reserve and National Guard personnel are a significant part of the military workforce, comprising about 38 percent of the military workforce end strength in fiscal year 2012, CAPE did not establish business rules for developing cost estimates for these personnel in its instruction. The Federal Accounting Standards Advisory Board Handbook states that a cost methodology that captures full costs should include any resources directly or indirectly used to perform work. Furthermore, the instruction states that the DOD components should use the business rules to account for the full cost of personnel when developing independent cost estimates and pricing units in the force structure.

Military service officials told us that they currently use or are considering using Reserve and National Guard personnel to perform work, and that in the absence of business rules for estimating the cost of these personnel, some officials said they have generated cost estimates for these workforces using their own methods. For example, one of the Air Force commands we interviewed conducted a business case analysis to evaluate the pros and cons associated with alternative workforce structures to help them meet their requirements. This analysis included the consideration of three courses of action: (1) adding additional contractor support, (2) adding additional civilian and active duty military personnel or (3) using reservists to provide quick surge capacity. The command officials said they used the methodology in the preceding DTM to conduct part of the analysis, and used Air Force guidance for the reserve component.

DOD’s military workforce consists of both active military and Reserve and National Guard personnel, and we have previously reported the importance of DOD employing a strategic approach to managing its total workforce to achieve its missions.Reserve Forces Policy Board report to the Secretary of Defense recommended that CAPE establish DOD policy or guidance for calculating the full cost of the reserve components. The report concluded In addition, in January 2013, a that without such a policy, senior leaders within DOD will not have complete or uniform data on the total costs associated with active and reserve personnel to make informed workforce decisions. According to officials in CAPE, determining the best person–civilian or military–to fill a full-time position is a far different question than determining the best mix of active and reserve personnel, and that they will work to develop additional guidance and tools as needed. CAPE added that the Board’s singular focus on cost obscured the fact that many other factors such as peacetime and wartime demands, deployment frequency and duration, and unit readiness are of equal or greater importance. An official in the Office of the Assistant Secretary of Defense for Reserve Affairs, however, stated that the reserve components should be included in the instruction in order for DOD to make effective workforce decisions for its total workforce.

Accordingly, CAPE added “active duty” to the title of instruction 7041.04 to be clear that the instruction did not include discussion of Reserve and National Guard personnel costs. to provide engineer units to assist local and state agencies in, among other things, construction of roads, bridges, and emergency housing. Without establishing business rules for estimating the cost of Reserve and National Guard personnel, however, whether as part of the instruction or in some other venue, DOD cannot create estimates and comparisons to inform workforce mix and force structure decisions for its total workforce.

Portions of Certain Retirement-Related Cost Elements May Not Be Relevant

The instruction directs users to include payments the government makes into funds to support the retirement and health benefits that current military and civilian personnel who become eligible for retirement will receive upon retirement; however, a portion of these payments may not be relevant to current personnel. Retirement and health benefits are deferred benefits that current employees will receive in the future. These deferred benefits, regardless of when the employee receives them, are attributable to current service by an employee and are to be included in considering the full costs of a workforce. According to the Federal Accounting Standards Advisory Board Handbook, however, costs that are attributable to services rendered in prior years that are attributable to services previously rendered by current retirees and to past service of current active civilian and military personnel are sunk costs that should not be included in determining current costs.

Accordingly, the instruction appropriately includes several cost elements that capture the actuarial “normal cost” to DOD of future retirement and health benefits for military and civilian personnel—that is, the costs of such future benefits that are attributable to a current employee’s work in a In addition, the instruction includes several cost elements given year. that represent costs to other federal agencies for deferred benefits. For contractor support, the cost of retirement and health care are included in the cost of the contract. See table 3 for a list of cost elements in the instruction related to retirement, health, and veteran benefits for military and civilian personnel.

The “normal cost” is the actuarially determined cost of future benefits attributable to employee service during the fiscal year.

DOD’s Components Report that They Have Incorporated Business Rules into Their Workforce Mix Decisions, but Implementation Challenges Exist

The DOD components we examined, for the most part, including commands, offices, and defense agencies, reported to us that they have incorporated the business rules identified in the DTM and instruction into their workforce mix decisions. Still, implementation challenges exist. Some of the DOD components we examined used the business rules for estimating and comparing the full cost of DOD personnel and contract support to inform workforce mix decisions, while others reported they have had limited opportunities to use the business rules. However, some military service officials said they did not know the extent to which officials at more local levels were aware of the DTM or used it, and therefore their reported instances of using the business rules may underestimate the actual number of instances. In addition, at the time when we met with these organizations, CAPE had not completed development of a DOD- wide software tool for implementing the instruction and, in the meantime, the components we examined had developed their own cost tools to apply the business rules and develop cost estimates. Further, the instruction that replaced DTM 09-007 directs users to data that may not contain the most accurate information for determining contractor support costs.

DOD Components Reported Incorporating the Business Rules into Their Workforce Mix Decisions, but Opportunities for Use Have Been Limited

While the DOD components we examined generally reported to us that they have incorporated the business rules identified in the preceding DTM and current instruction to create cost estimates in support of workforce mix decisions, their opportunities to use the business rules have been limited. Among the 13 DOD components we included in our scope, 8 reported that they used the preceding DTM and provided documentation showing that cost was a factor in their decision making from January 2010 through June 2013, 4 told us that they had not had the opportunity to use the DTM but were aware of it, and 1, while it also had not had the opportunity to use the DTM, reported that it was not aware of it. According to the DTM and instruction, the policies and procedures established within it are applicable to all DOD components. Specifically, the DTM stated and the instruction states that, when developing national security policies and making program commitments, DOD officials must be aware of the full costs of personnel and have a thorough understanding of the implications of those costs to DOD and, on a broader scale, to the federal government.

All eight of the DOD components we examined that reported that they had used the DTM to inform workforce mix decisions were commands within the military services, and a majority of their cost estimates supported in-sourcing decisions—moving work performed by contractors to performance by DOD employees. For example, of the 649 instances in which the commands we examined reported using the DTM from January 2010 through June 2013, 639 or 98 percent were to inform in-sourcing decisions. Of those 639, the Air Force Materiel Command accounted for approximately 525 or 82 percent. These commands provided us with documentation such as decision memoranda, completed cost benefit analysis spread sheets, and briefing slides given to senior leadership showing that cost was a factor in their decision-making. Further, with the exception of one instance in the Naval Supply Systems Command, officials from DOD components we met told us that, since January 2010, they have converted neither military to civilian positions nor civilian to military positions. The Army Installation Management Command and the Air Force Materiel Command reported a few instances of new or expanding missions that required the use of the DTM. Table 4 shows the number of times the commands we examined used the DTM and for what reason.

Some DOD officials said they were not clear on uses of the DTM beyond in-sourcing, and others said their reported instances of using the business rules may underestimate the actual number of reported instances. Officials within OSD and headquarters elements said they did not know the extent to which officials throughout the department are aware of the instruction and the requirement to use the associated business rules. For example, it was not clear to some Army officials in one office that the business rules were to be used for costing out decisions beyond in- sourcing. Other officials noted, however, that the business rules identified in the DTM or references to the DTM have been incorporated into some service-level policies and procedures to support decisions other than in- sourcing. For example, the Army’s approval form for new service contracts includes a question asking if the cost of labor for new services contracts was determined using the business rules. Some military service officials said they did not know the extent to which the reported data represented the full degree to which organizations were, in fact, using the business rules, and therefore their reported instances of using the business rules may underestimate the actual number of instances. For example, one official acknowledged to us that it is difficult for his organization to identify the degree to which the business rules were being applied at the local levels.

Officials within the agencies and offices that were aware of the DTM but had not used it told us of several reasons why they did not conduct these types of workforce cost estimates since the initial issuance of the DTM. For example, officials said they had not used it, in part, due to issues related to the current fiscal environment, such as concerns about anticipated reductions in funds available for contractor support and limitations on the number of civilian full-time equivalents. Officials also said they did not use the DTM because their offices do not have direct access to military personnel and have had no new or expanded missions.

DOD Components Use Different Tools to Implement the DTM

CAPE recently completed development of a DOD-wide software tool for implementing DOD Instruction 7041.04. During the time we met with component organizations, however, the DOD-wide tool was not available, and in the absence of the tool the components we examined had developed their own tools to apply the business rules in the DTM to develop workforce cost estimates. When initially released on January 29, 2010, DTM 09-007 called for CAPE, within one year of its publication, to develop a cost model for DOD-wide application that employs the business rules set forth in the DTM. In addition, best practices in GAO’s Cost Estimating and Assessment Guide state that in order to be reliable, cost estimates should be derived using an overall process that produces high- quality cost estimates that are comprehensive and accurate and that can be easily and clearly traced, replicated, and updated.

According to officials in CAPE, they contracted out the development of a software tool for the required cost model to implement the business rules and, when DOD issued Instruction 7041.04 on July 3, 2013, CAPE also released its DOD-wide tool for use across the department. Officials in CAPE told us that the components’ use of the DOD-wide tool will not be required, enforced, or monitored. DTM 09-007 stated that CAPE would oversee compliance with both the DTM and the use of the DOD-wide tool. When DOD Instruction 7041.04 was issued, however, the requirement for CAPE to oversee use of its tool was removed.

In the absence of CAPE’s DOD-wide tool, the Air Force modified an existing software tool originally used to inform public-private competition decisions, and the Army, the Marine Corps, and the Navy developed tools using an existing off-the-shelf software program. While the services developed and use different tools, those tools generally incorporate the cost elements identified in the DTM and current instruction. The defense agencies we met with said they had not created tools to implement the business rules, and officials from these agencies said they would use the DOD-wide tool when it was made available to them.

According to the instruction, the business rules provide a consistent approach for all DOD components to estimate the cost of personnel. Accordingly, the instruction provides a list of potential cost factors associated with personnel that should be considered in the decision- making process even when personnel costs are not the only factor. Officials in CAPE stated, however, that they have not reviewed the services’ tools to ensure they are in compliance with the business rules and do not plan to review them. An assessment of these various tools would enable CAPE to identify the advantages and disadvantages of allowing multiple cost estimation tools. DOD decision makers who then use these various economic analyses or cost estimates would have greater assurance that they are using reliable results to make workforce mix decisions.

DOD Components May Be Relying on Data Sources That Do Not Contain the Most Accurate Data for Determining Contractor Costs

When estimating contractor support costs for a new or expanding mission, Instruction 7041.04 provides several options for users to consider, although these options may not be the most accurate data sources. For example, the instruction directs the user to begin with the negotiated price of an existing contract, which DOD and military department officials told us is their preferred option. If an existing contract is not available, those officials attempt to find another contract that is similar to the contactor support for which costs are being estimated. To facilitate this, most of the services, agencies, and offices we met with maintain their own database of historical contractor data.

If an existing contract is not available, the instruction directs the user to either a General Services Administration (GSA) website or the Army’s online database for contract management and reporting to obtain contractor support costs for services. We have previously highlighted limitations with GSA’s data, however, such as the fact that contractors’ published rates on the website do not reflect post-competition prices.The GSA website allows users to search for services, and then identifies a list of contractors that provide the service along with their rates. The listed rates, however, may be negotiated with the contractor. Further, DOD officials we spoke to during this review said that GSA’s website does not provide targeted data, such as actual contractor rates by function or geographic location. For example, officials with one command we spoke to noted their use of a contractor who possesses unique specialized nuclear weapons related knowledge. According to those officials, GSA’s website does not contain available data for unique positions like these. In addition, officials said that GSA’s website provides too large a range of rates for them to develop realistic estimates. Best practices from GAO’s Cost Estimating and Assessment Guide state that a basic characteristic of credible cost estimates is having multiple sources of suitable, relevant, and available data.

In July 2013, when the DOD Instruction replacing DTM 09-007 was issued, the Army’s Contractor Manpower Reporting Application was added as an additional source for contractor data. The Contractor Manpower Reporting Application is an online database that automates the Army’s contract management and reporting process for contract management personnel by allowing users to view contract information, track contract data, and view various reports based on contract data in the application. The Contractor Manpower Reporting Application business process captures information on funding source, contracting vehicle, organization supported, mission and function performed, labor hours, and labor costs for contracted efforts, among other things. Currently the application collects data for the Army, the Air Force, and the Navy on the number of contractor employees by using direct labor hours and associated cost data. However, DOD is still developing its department- wide Contractor Manpower Reporting Application system and we previously found that a number of factors limit the accuracy and completeness of inventory data.unaware of some of the limitations with the contractor support data sources provided in the instruction. In the absence of data sources that are consistent with established practices for developing cost estimates, DOD components may be using data that do not lead to credible contractor cost estimates.

Conclusions

With a total workforce of about 3 million individuals, as well as an estimated 710,000 contractor full time equivalents, DOD’s investment in personnel to accomplish its missions constitutes a substantial financial commitment. In those cases where the department can choose what workforce it wants to utilize to accomplish tasks, it is crucial that it have an accurate method for comparing the costs of its civilian, military, and contractor personnel. DOD has taken important steps to achieve this goal since our 2008 report on military to civilian conversions for health care personnel, including the development of DTM 09-007 and its successor Instruction 7041.04. The department’s current direction for estimating and comparing the cost of its available workforces, however, could be improved. For instance, users currently find it difficult to develop estimates of particular costs such as overhead, advertising and recruiting, and training, and the services have not received guidance on developing estimates for Reserve and National Guard personnel. Further, as we have noted, there is some disagreement within the department about the inclusion of certain retirement costs in current workforce cost estimates. More comprehensive guidance in these areas could improve the components’ ability to make accurate cost comparisons between workforces.

Similarly, the department currently lacks assurance that workforce cost estimates and comparisons are consistent across the department, but by evaluating the different cost estimation tools currently being used by the components, it could decide on the best course to ensure consistency and accuracy. Also, current and accurate data on contractor support costs is critical for the department in making workforce decisions, but due to limitations in some of the data DOD has identified for making contractor support cost estimates, components may be relying on data sources that do not produce intended results. While federal agencies always must make consistent and cost-effective choices in managing their resources, this is especially true given the ongoing fiscal challenges that have imposed budgetary constraints across the federal government and are likely to continue for some time.

Recommendations for Executive Action

To improve DOD’s estimates and comparisons of the full cost of its military, civilian, and contractor workforces, we are making the following five recommendations to the Secretary of Defense.

To improve DOD’s methodology for estimating and comparing the full cost of its various workforces, we recommend that the Secretary of Defense direct the Office of Cost Assessment and Program Evaluation office to take the following three actions:

Further develop guidance for cost elements that users have identified as challenging to calculate, such as general and administrative, overhead, advertising and recruiting, and training;

Develop business rules for estimating the full cost of National Guard and Reserve personnel; and In coordination with the department’s Office of the Actuary and appropriate federal actuarial offices, reevaluate the inclusion and quantification of pension, retiree health care costs, and other relevant costs of an actuarial nature and make revisions as appropriate.

To facilitate consistent workforce cost estimates and comparisons, we recommend that the Secretary of Defense direct the Office of Cost Assessment and Program Evaluation to assess the advantages and disadvantages of allowing the continued use of different cost estimation tools across the department or directing department-wide application of one tool, and revise its guidance in accordance with the findings of its analysis.

To improve DOD’s ability to estimate contractor support costs, we recommend that the Secretary of Defense direct the Office of Cost Assessment and Program Evaluation, consistent with established practices for developing credible cost estimates, to research the data sources it is currently using and reassess its contractor support data sources for use when determining contractor support costs.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD agreed with our findings that it must make cost-effective decisions in structuring and shaping its workforce of military personnel, government civilians, and contracted support. Specifically, DOD concurred with two of our recommendations and partially concurred with three. DOD’s comments are reprinted in appendix II. DOD also provided technical comments on the draft report, which we incorporated as appropriate.

DOD partially concurred with our recommendation to further develop guidance for the cost elements that users have identified as challenging to calculate, such as general and administrative, overhead, advertising and recruiting, and training. In commenting on our report, DOD stated that it will continue to review the methodology for calculating these cost elements and issue clarifying guidance where necessary or appropriate. DOD also stated that it is part of an inter-agency effort that is developing government-wide cost comparisons and that, if or when government-wide cost comparison guidance is published, DOD will adjust its own guidance accordingly. We continue to believe that fully addressing this recommendation would enhance the development of DOD’s methodology for estimating and comparing the cost of its workforces.

DOD also partially concurred with our recommendation to develop business rules for estimating the full cost of National Guard and Reserve personnel. In its comments, DOD stated that it is assessing the potential need for reserve manpower costing models, as well as the identification of questions that reserve costing policies and models would be used to address. Once it has gained a more thorough understanding of the questions to be addressed by reserve cost estimates, it will begin work on guidance as necessary, and development of reserve component costing models if appropriate. We continue to believe that fully addressing this recommendation would enhance the development of DOD’s methodology for estimating and comparing the cost of its workforces and DOD’s ability to make more informed workforce decisions.

DOD concurred with our recommendation to coordinate with the department’s Office of the Actuary and appropriate federal actuarial offices, reevaluate the inclusion and quantification of pension, retiree health care costs, and other relevant costs of an actuarial nature and make revisions as appropriate. In its response to our report, DOD stated that it will work with the Office of the Actuary, and others as necessary, to reevaluate the inclusion and quantification of these costs elements and, following evaluation, revisions will be made. We believe such actions, if implemented effectively, will fully address the intent of the recommendation.

DOD partially concurred with our recommendation to assess the advantages and disadvantages of allowing the continued use of different cost estimation tools across the department or directing department-wide application of one tool, and revise its guidance in accordance with the findings of its analysis. We acknowledge, as DOD stated in comments on our report, that the department is open to assessing the advantages and disadvantages of allowing the continued use of alternate cost estimation tools among the various components, and, if necessary, will revise its guidance based on this assessment. However, in order to satisfy the intent of this recommendation, DOD needs to take action to fully address this recommendation. Doing so will facilitate more consistent workforce cost estimates and comparisons when DOD’s components use the department’s methodology for estimating and comparing the cost of their workforces.

Finally, DOD concurred with our recommendation to research the data sources it is currently using and reassess its contractor support data sources for use when determining contractor support costs. In its comments, DOD stated that as the department increases its fidelity into contractor support costs through the collection of statutorily required information via the Enterprise-wide Contractor Manpower Reporting Application in support of the Inventory of Contracts for Services, the department will modify its guidance accordingly. We believe that this action, if implemented effectively, will address the intent of the recommendation.

We are sending copies of this report to the Secretary of Defense, the Under Secretary of Defense for Personnel and Readiness, the Director of DOD’s Office of Cost Assessment and Program Evaluation, the Office of Management and Budget, and appropriate congressional committees. In addition, this report will also be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions regarding this report, please contact me at (202) 512-3604 or farrellb@gao.gov. Contact points for the Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

To evaluate the extent to which the Department of Defense’s (DOD) methodology for estimating and comparing the cost of work performed by military, civilian, and contractors reflects the full cost to the taxpayer, we identified each of the cost elements contained in the most recent version of Directive Type-Memorandum (DTM) 09-007 and DOD Instruction 7041.04 and compared them to best practices in GAO’s Cost Estimating and Assessment Guide, guidance in the Federal Accounting Standards Advisory Board Handbook, and the Office of Management and Budget guidelines for Performance of Commercial Activities. We also reviewed pertinent DOD and service-specific policies and guidance related to workforce mix and economic analysis. In addition, we interviewed knowledgeable officials within the offices of the Under Secretary of Defense (Personnel and Readiness), the Director of Cost Assessment and Program Evaluation, DOD’s Office of the Actuary, and the military services. We also met with the Office of Management and Budget and other experts and other interested parties we selected based on their work on issues related to the DTM. Specifically, we met with the Center for Strategic and International Studies and the Center for Strategic and Budgetary Assessments, both of which are independent nonprofit policy research institutes. We also met with the American Federation of Government Employees, a federal government employee union, and the Professional Services Council, a trade association of the government professional and technical services industry. We obtained the perspective of the National Guard and Reserve community by meeting with the Office of the Secretary of Defense (Reserve Affairs) and the Reserve Forces Policy Board. We also reviewed published work by independent research institutes evaluating the business rules contained in the preceding DTM, such as those issued by the Center for Strategic and International Studies, the Project on Government Oversight, and the Reserve Forces Policy Board.

DOD Instruction 7041.04 was issued on July 3, 2013. Therefore, for a majority of our review, DTM 09-007 was the most current guidance.

Corps is reported to the Defense Manpower Data Center, we were unable to align numbers of civilian personnel with specific Marine Corps commands or offices. Therefore, the Marine Corps was excluded from our selection criteria and we did not select any Marine Corps commands as part of our non-probability sample. Once the data for the other components was compiled, we rank ordered the commands within each of the military departments and the defense agencies from largest to smallest based on civilian personnel counts. For each of the military departments and defense agencies, we divided the rank ordered list of commands and agencies into three groups: large, medium, and small. We considered commands and agencies with 10,000 or more full-time civilian employees as large, commands and agencies with at least 1,000 but less than 10,000 full-time civilian employees as medium; and commands and agencies with less than 1,000 full-time civilian employees as small. We generated a selection number based on the total number of commands and agencies and counted down the rank ordered list of commands and agencies to identify those commands and agencies that we would meet with. This method resulted in one selected command, office, or agency from each of the categories of large, medium, and small. In addition to discussing with each of these entities their implementation of the business rules identified in DTM 09-007, we requested the number of times they used the business rules to inform workforce decisions (e.g., in- sourcing, workforce conversions, and new or expanded missions) from January 2010 when the DTM was issued to June 2013. We did not independently validate the number of uses of the business rules reported to us by each of the entities or the cost estimates used to inform the workforce decisions. Table 5 shows the commands and agencies we met with to determine the extent to which DOD incorporated the business rules contained in DTM 09-007.

In addition, we met with officials in other DOD entities that are involved in guiding the implementation of the methodology. Although the Marine Corps was not part of our non-probability sample, we met with several Marine Corps commands and offices to discuss their implementation of the business rules identified in DTM 09-007. We also received a consolidated Marine Corps response to our questions on the implementation of the methodology. Table 6 shows the other DOD organizations we contacted to discuss the department’s efforts to implement the business rules contained in the preceding DTM.

Further, we received a demonstration by the Office of Cost Assessment and Program Evaluation of its DOD-wide Full Cost of Manpower tool to gain an understanding of DOD’s application of the business rules contained in DTM 09-007. We also requested and obtained copies of the tools from each of the commands, offices, and defense agencies we met with to develop an understanding of the software tools they developed to apply the business rules in the DTM and in the instruction. For the Air Force we obtained a copy of their software tool, DTM-COMPARE. For the Army, the Marine Corps, and Navy we obtained copies of the off-the-shelf software that was programmed to implement the business rules contained in the preceding DTM. In addition, we attended an Army Installation Management Command training on their Cost Analysis Workbook tool. Further, we obtained and reviewed sample documentation from the military commands that had applied the business rules identified in the preceding DTM to support workforce mix decisions.

We conducted this performance audit from December 2012 to September 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Brenda S. Farrell, (202) 512-3604 or Farrellb@gao.gov.

Staff Acknowledgments

In addition to the contact named above, David Moser, Assistant Director; Timothy Carr, Brian Pegram, Erin Preston, Frank Todisco, Erik Wilkins- McKee, and Michael Willems made key contributions to this report.

Related GAO Products

Defense Contractors: Information on the Impact of Reducing the Cap on Employee Compensation Costs. GAO-13-566. Washington, D.C.: June 19, 2013.

Human Capital: Additional Steps Needed to Help Determine the Right Size and Composition of DOD’s Total Workforce. GAO-13-470. Washington, D.C.: May 29, 2013.

Defense Acquisitions: Continued Management Attention Needed to Enhance Use and Review of DOD’s Inventory of Contracted Services. GAO-13-491. Washington, D.C.: May 23, 2013.

Pension Costs on DOD Contracts: Additional Guidance Needed to Ensure Costs Are Consistent and Reasonable. GAO-13-158. Washington, D.C.: January 22, 2013.

Federal Workers: Results of Studies on Federal Pay Varied Due to Differing Methodologies. GAO-12-564. Washington, D.C.: June 22, 2012.

Defense Workforce: DOD Needs to Better Oversee In-sourcing Data and Align In-sourcing Efforts with Strategic Workforce Plans. GAO-12-319. Washington, D.C.: February 9, 2012.

DOD Met Statutory Reporting Requirements on Public-Private Competitions. GAO-11-923R. Washington, D.C.: September 26, 2011.

Military Personnel: Comparisons between Military and Civilian Compensation Can Be Useful, but Data Limitations Prevent Exact Comparison. GAO-10-666T. Washington, D.C.: April 28, 2010.

Military Personnel: Military and Civilian Pay Comparisons Present Challenges and Are One of Many Tools in Assessing Compensation. GAO-10-561R. Washington, D.C.: April 1, 2010.

Military Personnel: Guidance Needed for Any Future Conversions of Military Medical Positions to Civilian Positions. GAO-08-370R. Washington, D.C.: February 8, 2008.

Military Personnel: Military Departments Need to Ensure That Full Costs of Converting Military Health Care Positions to Civilian Positions Are Reported to Congress. GAO-06-642. Washington, D.C.: May 1, 2006.