According to an Army Human Resource official, the Army uses the workforce-planning model—CIVFORS—for human resources management. CIVFORS is a collection of software programs that anticipate future impacts on the workforce so that management can plan for changes instead of reacting to them. The model is used to evaluate a number of critical areas in civilian workforce planning, including projected recruitment of personnel, impact of organizational realignments, and changes in workforce trends (such as aging, retention, and projected personnel shortfalls). It is a life-cycle modeling and projection tool that models the most significant events that describe the life-cycle path of personnel, which includes accessions, promotions, reassignments, retirements, and voluntary and involuntary separations over a 7-year period. Verification and validation of models are important steps to building credible models because they provide the foundation for the accreditation process to ensure the suitability of the models for their intended purposes, as stated in Army guidance, Management of Army Models and Simulations. The verification process evaluates the extent to which a model has been developed using sound and established software engineering techniques, and it establishes whether the model’s computer code correctly performs the intended functions. Model verification includes data verification, model documentation, and testing of the information technology structure that supports the model; model verification is contained in such documents as the programmer’s manual, installation’s manual, user’s guide, analyst’s manual, and trainer’s manual. According to Army guidance, assessment of the correctness and forecasting capability of the model is also required, and it should be performed by a subject matter expert independent from the model developer; however, the developer is expected to conduct in-house verification and testing to assist in the overall model development process. Validation is the process of determining the extent to which the model adequately represents the real world. The Army has taken steps to ensure the reliability of the historical personnel data used by the model and the adequacy of its information technology structure used to support the model, but it has not provided documentation that it has sufficiently tested and reviewed the most critical aspect of the model—its forecasting capability and the appropriateness of its assumptions. As a result, the forecasting credibility of the current version of the model is not sufficiently validated or documented. Without proper documentation of the abilities of the model, there is a risk that the forecasts it produces may be inaccurate or misleading and the suitability for use by other organizations may be difficult to determine. The Army’s review of the historical personnel data used to provide information for workforce planning was adequate to show that the data are sufficiently reliable for use in the workforce model. Data regarding personnel (such as date hired, education, age, grade level, and occupational series) are taken from the Army’s Workforce Analysis Support System (WASS). CIVFORS uses the most recent 5 years of historical data to forecast the civilian workforce planning needs during the next 7 years. According to Army guidance, to ensure that data are sufficiently reliable for use in the Army model, support documents should contain information about the overall characteristics of the database. Furthermore, the documents should show the intended range of appropriate uses for the model as well as constraints on its use. They should also include concise statements of the condition of the database for the purpose of indicating its stability. The Army provided most, but not all, of the documents referred to in Army guidance; we believe that the documents provided are key ones and are adequate to show that WASS data are sufficiently reliable for use in CIVFORS. In addition, the Army program manager for the CIVFORS workforce-planning model stated that the workforce data are checked by reviewing the arithmetic in the numerical algorithms to verify that there is no unexplained change in the size of the civilian personnel workforce contained in the database. Further, edit checks include matching social security numbers for personnel from one time period to another to account for actual personnel and personnel transactions processed. In addition, CIVFORS has automated checks for inappropriate numbers or characters. Such steps help to assure that the data contained in WASS accurately and completely reflect critical personnel aspects and transactions. The Army’s procedures for validating the information technology support structure (the software and hardware used to interface with and house the model) were also sufficient. For example, the Army (1) adequately documented the information technology structure to allow for continuity of operations, (2) tested its functionality, and (3) provided expertise for system modification and operation. Procedures used by the Army include documenting the model’s system description and hardware and software requirements, providing system and user manuals, planning for configuration management, and conducting functionality tests to help ensure the system’s usability and operability over time and to demonstrate the adequacy of the information technology structure to support use of the workforce model. The Army’s documentation cannot show that the forecasting ability of CIVFORS has been adequately evaluated and, therefore, we cannot fully assess the credibility of the model. According to Army guidance, validation is the process of determining the extent to which a model adequately represents the real world. According to the Army program manager, over a 7-year period, CIVFORS forecasts the anticipated impacts on the workforce based on the most significant events in the life-cycle path of personnel (to include accessions, promotions, reassignments, retirements, voluntary separations, and involuntary separations). Army guidance states that an independent, peer, and subject matter expert review of the model should be conducted. The Army guidance also suggests generally accepted methods, such as conducting a careful line-by-line examination of the model design and computer code and algorithms. The Army’s program manager said this had been done for the original certification of CIVFORS in 1987. However, no formal document of the reviews has been prepared in the years since, even though the Army has undertaken several model improvements, such as (1) an expanded scope to include more dimensions in the modeling process; (2) a more integrated, streamlined process that involves fewer steps; and (3) greater flexibility, achieved by generalizing the formulas and parameters. In addition, there is insufficient documentation regarding tests performed, since 1987, in which CIVFORS’s forecasts for prior years are compared against equivalent historical data (called an “out of sample” test) to measure the model’s forecasting capability. Such testing, which is one method to validate a model’s forecasting capability, would involve using the first 5 of the last 7 years of historical data to forecast the 2 subsequent years. The forecasts for the last 2 years could then be compared to the actual historical data. The Army, however, performed tests comparing patterns of forecasts against historical data (called “in sample” tests), showing that forecasts reflect the same patterns as the historical data used to develop them for a sample of three Army major commands. However, the draft document that was provided to us was inadequate to fully assess the sampling used by the Army and the value of the tests. Finally, the Army could not provide adequate documentation of an independent or peer review of the model. The Army’s CIVFORS program manager stated that the major commands served as peer reviewers by conducting a comparison of their workforce data to WASS and CIVFORS workforce data. We believe that such assessments by users provide important information but do not constitute a peer review as defined in Army guidance. Also, the results of these assessments were not available for us to review. The program manager also stated that an independent subject matter expert reviewed the functional design and the code in 1999, but a formal report of the activities performed and the specific changes or modifications implemented during the review were not produced. Documentation has often not been a priority for several reasons. According to the Army’s CIVFORS program manager, lack of documentation is primarily due to limited funding, which was spent on implementing changes to CIVFORS and WASS rather than on the production of formal documents. Further, a shortage of staff (only one staff person—the program manager) and loss of documents during the attack on the Pentagon on September 11, 2001, also affected the amount of documentation the Army could provide us. The program manager also stated that some documentation was not needed because CIVFORS’s design is predicated on proven methods in other Army active-duty, military manpower forecasting models. In addition, the program manager stated that the Army and contractors have primarily been adapting technology (upgrading from mainframe to personal computer to Web-based) to improve model functionality rather than creating new technology. However, without proper documentation of the abilities of the model, there exists a risk that the forecasts it produces may be inaccurate or misleading. Consequently, decisions about future workforce requirements may be questionable, and planning for the size, shape, and experience level of the future workforce may not adequately meet the Army’s needs. These issues may extend beyond the Army. In April 2002, DOD published a strategic plan for civilian personnel, which includes a goal to obtain management systems to support workforce planning. According to a DOD official responsible for civilian workforce planning tools, components within DOD have been requesting a modeling tool to assist them with civilian workforce planning. As a result, DOD has decided to test the Army’s civilian forecasting model. In October 2002, DOD purchased hardware, installed modified software, and provided training to a small number of personnel. Recently, DOD obtained a historical database of civilian personnel data from the Defense Management Data Center and provided the database to the contractor to load into the model. Two agencies have volunteered to test the model: the Defense Logistics Agency and the Washington Headquarters Service. DOD is working to develop a test for these organizations using their own civilian personnel data to test the model. At the end of the testing period, DOD will assess the model to obtain a better understanding of its logic and determine whether or not it should be implemented departmentwide.