Congress has established four military ranks above the rank of colonel (for the Army, Air Force, and Marine Corps) and captain (for the Navy). Table 1 displays the pay grade designation, title of rank, and insignia worn by officers at general and flag officer ranks. Within the Office of the Under Secretary of Defense for Personnel and Readiness, the Officer and Enlisted Personnel Management office is responsible for general and flag officer matters, including the application of related legislation. Each of the services and the Joint Staff has a general officer matters office that is responsible for management of general and flag officers within their organization. Management of reserve component general and flag officers falls under the service general officer matters offices, the service reserve chiefs, and the National Guard Bureau. Their responsibilities include managing the selection, promotion, assignment, and retirement of general /flag officers in addition to coordination with other DOD and non-DOD organizations. The Reserve Forces Policy Board, acting through the Assistant Secretary of Defense for Reserve Affairs, is the principal policy adviser to the Secretary of Defense on matters relating to the reserve components. The Board was created by an act of Congress in 1952. Among its duties, the Board evaluates proposals by its members or other agencies for changes to existing laws and policies and recommends appropriate actions. The Board is composed of 24 members, including a civilian chairman and the assistant secretaries of the Army, Navy, and Air Force responsible for reserve components. The Fiscal Year 1997 National Defense Authorization Act directed the Secretary of Defense to review existing statutory general and flag officer authorizations and submit a report to Congress. The act also directed GAO to evaluate DOD’s study. The military services and Joint Staff conducted their own studies. On the basis of these study results, DOD developed a draft report; however, it never issued a final report. We testified on DOD’s study in April 1997 and issued a report in June 1997 that discussed the draft DOD report. Congress included similar mandate language in the Fiscal Year 2003 National Defense Authorization Act. As in 1997, the Secretary of Defense was directed to review existing statutory general and flag officer authorizations and submit a report to Congress. The report was to include any recommendations (together with the rationale) concerning revision of the limitations on general and flag officer grade authorizations. The act also stated that certain provisions of the Fiscal Year 1997 National Defense Authorization Act would apply. One of the applicable provisions from the 1997 act directed GAO to evaluate DOD’s study. To respond to the 2003 mandate, the Office of the Secretary of Defense contracted with the Logistics Management Institute (LMI) to assess and validate general and flag officer requirements. The results of the requirements study were expected to serve as a baseline of validated general and flag officer positions and provide a basis for future efforts, such as identifying innovative ways to manage senior leadership. The Assistant Secretary of Defense (Force Management Policy) established a working panel and a senior panel to support LMI’s review. The working panel was chaired by the Director, Officer and Enlisted Personnel Management, and included representatives from the military services’ and Joint Staff’s general and flag officer management offices and from the Office of the Secretary of Defense. The senior panel was chaired by the Deputy Assistant Secretary of Defense for Military Personnel Policy, with participation by the service personnel chiefs, the Joint Staff’s Director for Manpower and Personnel, and representatives from the Office of the Secretary of Defense. The general and flag officer study was completed in March 2003. DOD subsequently submitted its report to Congress on March 31, 2003. DOD has conducted several previous reviews of its general and flag officer requirements. In the 1997 draft report, DOD identified a need for 1,472 general and flag officers, including 1,018 active component and 454 reserve component officers. According to DOD, the 1997 study was the first to integrate active and reserve component requirements. A 1988 DOD study found a requirement for 1,436 general and flag officer positions. A 1978 DOD study identified a requirement for 1,419 general and flag officers. In 1972, DOD identified a requirement for 1,304 general and flag officers. Congressional authorizations for general and flag officers typically have been lower than requirements identified in DOD’s studies. The general and flag officer study validated total requirements for 1,630 general and flag officer positions. Congress, since fiscal year 1996, has authorized the services a total of 1,311 general and flag officers, a difference of 319. DOD, in the March 2003 report to Congress, recommended no additional authorizations to meet validated requirements. DOD did not explain its rationale for keeping authorizations at current levels, except to say that other options for managing general and flag officers were being considered. The report also did not provide the requirements data generated by the general and flag officer study, nor did it address the magnitude or the impact of the gap between requirements and authorizations. The general and flag officer study validated requirements for a total of 1,630 general and flag officer positions, including 1,039 validated requirements in the active component and 591 validated requirements in the reserve component. Tables 2 and 3 display results from DOD’s study. Title 10 of the U.S. Code establishes service-specific ceilings for active duty general and flag officers that total 877. Title 10 also authorizes 12 general and flag officer positions to be allocated by the Chairman of the Joint Chiefs of Staff to the services for joint duty positions. These authorizations, called the “Chairman’s 12,” do not count against the service ceilings. In addition, DOD is authorized a maximum of 422 reserve component general and flag officers. Title 10 of the U.S. Code establishes service ceilings for reserve component general and flag officers. Congress last revised service ceilings in fiscal year 1996. Table 4 displays congressional authorizations for general and flag officers. On the basis of the general and flag officer study results, DOD found that the number of validated requirements for general and flag officer positions exceeded the number of general and flag officers currently authorized under law. DOD concluded in its March 2003 report to Congress that all current authorizations for active and reserve general and flag officers were needed. Although DOD’s report made several recommendations concerning management of general and flag officers, DOD recommended no change to the authorized number of general and flag officers or their grades at this time. DOD, in the March 2003 report, did not explain its rationale for keeping general and flag officer authorizations at current levels. DOD’s stated rationale for not requesting additional authorizations was that before the department considers proposing a legislative increase in authorizations, it must look at other innovative ways to meet these requirements. DOD did not specify the options it would consider except to say that it intended to review its overall senior leadership needs, including both civilian and military leaders, and might seek authority to manage senior leaders under an overall ceiling. DOD stated that until this review was complete, the services would continue to meet their internal and external general and flag officer requirements using available management tools as “workarounds,” such as use of individuals selected for promotion to higher grades and frocking authority. DOD did not state when this review would be complete. In the March 2003 report, DOD did not address the magnitude or the impact of the gap between validated requirements and authorizations. The general and flag officer study summarized the validated requirements by active and reserve component, by service, and by grade. More detailed data was provided in a database of validated requirements that was developed as part of the study. However, the March 2003 report did not disclose this data or address the magnitude of the gap between validated requirements and authorizations. The total gap between validated requirements and authorizations was 319. In the active component, the study validated requirements for 1,039 general and flag officer positions, compared with 889 authorizations (a difference of 150). In the reserve component, the study validated requirements for 591 general and flag officer positions, compared with 422 authorizations (a difference of 169). These gaps are illustrated in figure 1. While the March 2003 report noted that the services would continue to use frocking and other “workarounds” to meet requirements, it did not address the impact of these workarounds. Following are examples of workaround issues that DOD did not address in its report: How do the services determine which positions to fill with an officer at a lower grade? Are some validated requirements going unfilled altogether? Do officers at lower pay grade levels have legal authority to carry out all duties required of a general and flag officer position? Does the practice of assigning lower grade officers to general and flag officer positions affect the ability of an organization to perform its mission? What is the impact of these workarounds on the overall management of general and flag officers and of officers in lower grades? Are there benefits to using these workarounds, such as providing developmental opportunities for officers in lower grades? Do the benefits outweigh any negative effects of using these workarounds? To what extent have each of the services used these workarounds? DOD used an established methodology known as job evaluation to assess general and flag officer requirements. Job evaluation methodologies have been widely used in the United States to assess job value and rank one job against another. One limitation of job evaluation is that it has numerous subjective features, including the selection of job factors to be assessed. The factors used in DOD’s study were subject to differing interpretations. Another limitation is that the methodology is based on an existing organizational structure at a designated point in time. The methodology is not designed to capture new or emerging needs, such as those resulting from DOD transformation efforts or other changes in the department. Periodic updates could capture changes in requirements. Such limitations do not invalidate DOD’s methodology; however, an explicit acknowledgment and assessment of these limitations would have provided more context for the study results. In addition, the study did not clearly account for dual-hatted positions or assess how each service’s authorizations were affected by the need to contribute general and flag officers to fill external (joint) positions. Addressing these issues could have enhanced the precision and usefulness of DOD’s study. In addition, we noted that while Congress directed DOD to ensure the Reserve Forces Policy Board participated in development of the report’s recommendations, the Board played a minimal role in producing the DOD 2003 report. The Board registered strong objections to DOD’s recommendation not to seek additional authorizations now to meet validated requirements and to the limited role it played in the overall process. To assess and validate general and flag officer requirements, DOD applied an established methodology known as job evaluation. Job evaluation methodologies have been widely used in the United States and were used in past general and flag officer studies, including the 1997 and 1988 studies. Job evaluation is a formal procedure for hierarchically ordering a set of jobs in terms of their value or worth to the organization. No universally accepted approach to job evaluation exists, and several types of job evaluation methodologies may be used. DOD’s 2003 study was not explicit about the type of job evaluation methodology used but appeared to be a point-factor system. While point-factor systems may vary, the basic steps involved are gathering job descriptions for all jobs, selecting the factors to be used for measurement, identifying benchmarks for comparing jobs in a similar industry or field, assigning weights to the factors, assigning numerical values to the selected factors, and establishing a rank band. DOD’s study included all of these steps in its job evaluation methodology except for identifying benchmarks. Benchmarks are typically used when the purpose of the evaluation is to determine appropriate compensation. The steps in the general and flag officer study were as follows: At LMI’s request, the services and the Joint Staff submitted data on all of their general and flag officer positions and on a sample of colonel and Navy captain positions. The working panel established five functional groups to categorize the positions to be assessed. These five groups were operations, combat development, material, headquarters staff, and special staff. The working panel and LMI selected factors to be used in the assessment and weighted the factors for each functional group. LMI placed the submitted positions into the five functional groups. The senior panel validated the functional groups, the factors, and the weighting. For the positions to be assessed, the services and the Joint Staff gathered, validated, and entered information about each factor into a database created by LMI. Using the database information, nine LMI scorers assessed each position by assigning numerical values to the factors and rank-ordered the positions within their functional groups. The LMI scorers were retired military officers representing a cross section of the military services, as well as the active and reserve components. They were divided into three panels of three members each. The senior panel reviewed and validated the LMI scoring, including the breakpoints between pay grades (for example, between O-7s and O-8s). Results were then vetted through the service and Joint Staff senior leadership. A basic aim of job evaluation methodologies is to interject objectivity into what is inherently a subjective process—assigning the relative value of work to an organization. Nevertheless, subjectivity is commonly a part of job evaluation. For example, the factors selected for measurement and the weighting of the factors are based on subjective judgment. Also, the process of scoring jobs is subjective, as are management adjustments to the study results. A second limitation of job evaluation methodology is an assumption that an existing organizational structure can serve as an adequate starting point for the assessment. The methodology reviewed current positions and was not designed to capture new or emerging needs. According to the Joint Staff, the study methodology, as well as time constraints, did not give them the opportunity to identify unresourced general and flag officer requirements. DOD efforts to reorganize the Army, alter the overseas presence of U.S. troops, and transfer thousands of military jobs to the civilian workforce, if implemented, could change requirements for general and flag officers. Such changes could be captured through periodic updates to the requirements. According to an official in the Office of the Under Secretary of Defense for Personnel and Readiness, the services and Joint Staff have been asked to submit new general and flag officer requirements to keep the data current. We agree that identifying new requirements is a necessary step to keep the data up-to-date; however, these new requirements are not going through the kind of job evaluation assessment and validation DOD used in its study. We found that neither the DOD study nor the March 2003 report to Congress addressed the limitations in the methodology selected. For instance, neither document specified the subjective features of the methodology or the assumptions underlying the study. In addition, they did not explicitly state the steps taken to test the reliability or validity of the study. These issues do not invalidate the results of the general and flag officer study, but an explicit acknowledgement and assessment of these issues would have provided more context for the study results. A critical step in job evaluations is the selection of factors to be measured. Because the factors are used to assess job value, the selection of these factors can influence the study results. LMI reviewed the factors used in the 1997 general and flag officer study and found similarities among them that could be traced to a set of 16 factors that predated the 1997 study. The origin of the 16 factors, however, is unclear. According to the general and flag officer study, the 16 factors originated in the deliberations of the Bolte commission in the late 1950s. An LMI official told us neither LMI nor the services could locate the original source document for the 16 Bolte factors; hence, LMI used a secondary service source to identify these factors. Two service regulations—Marine Corps Order 5311.4, dated September 30, 1986, and OPNAV Instruction 5420.87A, dated February 12, 1981—listed the 16 factors. (See app. I for a list of the 16 factors.) The working panel made minor adjustments to the wording of the factors, and the senior panel approved their use for the study. DOD officials told us that the factors were broad enough to cover the significant aspects of general and flag officer positions, although one service official expressed concern that nuances between the services were not captured. The services and the Joint Staff were responsible for gathering the data on the factors for the positions and for entering this data into a database. The services and Joint Staff asked other offices (such as major commands and reserve component offices) and individuals (including general and flag officers) to provide the needed information. This decentralized process left the factors open to differing interpretations. We selected a small number of similar general and flag officers and, comparing the information provided, noted inconsistencies in how the services and Joint Staff applied the factors. For example, the superintendents of the three service academies (the U.S. Military Academy, the U.S. Naval Academy, and the U.S. Air Force Academy) have the same function (combat development and training of future officers), the same rank (O-9), and oversee roughly similar numbers of personnel. However, the position descriptions for the superintendents vary greatly, even in terms of quantitative criteria like the magnitude of personnel and physical resources. In one case, overall dollar figures are given; in the second case, a budget breakdown is provided; and in the last case, the number and type (but not value) of equipment is provided. Information ranged from vague (“the Superintendent currently has sufficient authority to make decisions and commit resources within the scope of his position”) to detailed (“formulates and directs the execution of policies, procedures, and programs required to accomplish the mission—cadet leader development system; cadet honor system and honor code; academic curriculum, military and physical programs; strategic vision, planning, and communications; manpower and resource allocation”). During our review, we noted other issues concerning the methodology used to conduct the requirements study. The methodology did not allow for clear-cut differentiation between O-6 and O-7 positions and between O-7 and O-8 positions. Differentiation among the higher grades was more clear-cut. One service official said the factors were easier to apply to the higher grades. Factors were weighted identically for a diverse array of general and flag officer positions. The functional groups contained between 123 and 562 positions, with the result that the same weighting was applied to a diverse array of positions. For example, the same weighting was applied to the following Army positions in the special staff functional group: the Deputy Chief of Chaplains, the Assistant Judge Advocate General for Military Law and Operations, and the Dean of the Academic Board at the U.S. Military Academy. Several service officials expressed concern about the short time frame for conducting the study. Officials from one service told us they had not had time to verify the information submitted for all the service’s general and flag officer positions and instead did spot checks of the information. Some general and flag officers are dual-hatted, simultaneously holding more than one position. One goal of the general and flag officer study was to identify dual-hatted positions and assess them as a single position. When LMI initially requested information on each position, it asked the military services and the Joint Staff to identify whether any other dual-hatted titles were associated with the position. LMI created a separate field—other titles (if dual hatted)—to capture this information in the requirements database. Our analysis of the 1,630 positions in the database showed that 113 positions were listed in this field as having other titles. For assessment purposes, each dual-hatted pair was considered as a single position and categorized as either an Army, an Air Force, a Navy, a Marine Corps, or an external (joint) position. Our review showed that despite these efforts, the study did not clearly account for dual-hatted positions. First, the study did not distinguish whether positions were dual-hatted for organizational efficiencies or due to limited manpower. Separate studies of general officers conducted for the Army and Marine Corps under the 1997 mandate stated that dual-hatted positions may be created to increase efficiency of operations or administration. On the other hand, they may also be created to maximize leadership or management within the constraint of available authorization ceilings. The Marine Corps study stated that “sorting out these two different reasons is an important part of any study which purports to determine requirements, rather than to accommodate to scarcities.” The general and flag officer study did not address these different reasons for dual-hatting. As a result, requirements for general and flag officers could be different than the study data showed. Second, we found a small number of dual-hatted positions that were counted as separate requirements. Although an LMI official acknowledged that one pair of dual-hatted positions was double-counted, our analysis of the 113 positions listed as dual-hatted showed that at least 6 pairs of positions were counted separately in the requirements database. Three pairs of positions were internal to the Marine Corps, two pairs were Army/external (joint) positions, and one pair was internal to the Army. In the Marine Corps, the following dual-hatted positions were included as separate positions: the Deputy Commanding General, 1st Marine Expeditionary Force, and the Commanding General, 1st Marine Expeditionary Brigade; the Deputy Commanding General, 2nd Marine Expeditionary Force, and the Commanding General, 2nd Marine Expeditionary Brigade; and the Deputy Commanding General, 3rd Marine Expeditionary Force, and the Commanding General, 3rd Marine Expeditionary Brigade. The Army/external (joint) dual-hatted positions that were included as separate positions were the Deputy Commanding General, Eighth U.S. Army, and C-3/J-3, United Nations Command/Combined Forces Command/U.S. Forces Korea; and the Deputy Assistant Chief of Staff C/J-4, United Nations Command/Combined Forces Command/U.S. Forces Korea and Deputy Commanding General (Support), Eighth U.S. Army. In the Army, the Program Executive Office for the Army’s Air and Missile Defense and the Deputy Commanding General (Acquisition), U.S. Army Space and Missile Defense Command, were dual-hatted but included as separate positions. A third problem was a lack of consistency in identifying dual-hatted positions. Although the requirements database had the other titles (if dual-hatted) data field, we identified at least 37 positions that listed nothing under this field but included one or more dual-hatted titles elsewhere in their position descriptions. Thus, while the database identified 113 dual-hatted positions, other dual-hatted positions included in the database were not clearly identified as such. DOD’s general and flag officer study did not assess how each service’s authorizations were affected by the need to contribute general and flag officers to fill external (joint) positions. As a result, direct comparisons between the requirements generated by the study and congressional authorizations are difficult. Without knowing how many general and flag officers each service contributes to the external (joint) community, we cannot be certain of the exact number of personnel each service needs to meet all of its commitments, internal and external. As shown in table 4, Congress establishes authorizations by service. Congress does not provide separate authorizations for external (joint) requirements, except for the Chairman’s 12 exemptions from the overall service ceilings. External (joint) requirements are filled from service authorizations. In the general and flag officer study, internal (service-specific) positions were separated from external (joint) positions. However, the study did not show the extent to which each service contributed general and flag officers from their authorizations to fill external (joint) requirements. While service contributions to external (joint) requirements fluctuate due to the rotational nature of these assignments, service contributions can be determined at a given point in time. We used Joint Staff data from December 2002 to determine which service was filling each of the 223 active component external (joint) requirements (see table 5) and each of the 58 reserve component external (joint) requirements (see table 6) validated by the general and flag officer study. The data in tables 5 and 6 allow for a more direct comparison between requirements and authorizations. For example, the Army’s authorization for active duty generals is 302 plus the Army’s 5 exemptions for the Chairman’s 12, for a total of 307 authorized general officers. According to DOD’s requirements data, the Army needs 286 general officers to meet its internal active component requirements. Based on the data in table 5, Army general officers also filled 66 external (joint) requirements in the active component. As a result, the Army had total active component requirements—including internal and external—of 352 general officers, compared with active component authorizations for 307 generals, a difference of 45 generals. Using these calculations for the other services, active component requirements exceeded active component authorizations by 20 for the Air Force, 47 for the Navy, and 18 for the Marine Corps. The actual differences between individual service requirements and authorizations may be greater because the Joint Staff data did not identify the service filling 20 of the external (joint) active component requirements. “The Secretary of Defense shall ensure that the Reserve Forces Policy Board participates in the internal Department of Defense process for development of the recommendations of the Secretary contained in the report under subsection (a).” We noted during our review that the Reserve Forces Policy Board played a minimal role in producing the DOD 2003 report. The Board was not involved with the preparation of the general and flag officer study but was given the opportunity to comment on the study and the draft DOD report prior to its submission to Congress. The Board provided written comments on the report in April 2003 concurring with DOD’s conclusion that all current congressional authorizations for general and flag officers were needed, but registering strong objections to DOD’s recommendation not to seek additional authorizations now to meet validated requirements and to the limited role it played in the overall process. The Reserve Forces Policy Board had a more active role in DOD’s 1997 general and flag officer study. In a memorandum to the Secretary of Defense, the Chairman of the Reserve Forces Policy Board at that time indicated the Board was pleased with its level of participation in the 1997 study. The Chairman stated, “We believe our ‘independent voice’ is captured in this report and trust our views will be appropriately considered. We are pleased to share with you that [the Office of the Secretary of Defense] and the Services wholeheartedly embraced our participation. We were well represented at all levels of participation and our views were objectively considered…. In our view, this report was truly a Total Force effort.” DOD, in conducting its 2003 general and flag officer requirements study, incorporated some of the lessons learned from a GAO review of DOD’s 1997 study. A notable improvement was the use of a single methodology and set of factors in the 2003 study to evaluate general and flag officer positions across all the services and the joint community. DOD also improved its tracking of study results and its documentation of adjustments made by the services and the Joint Staff. As in 1997, DOD did not identify and assess general and flag officer positions that could conceivably be converted to the civilian workforce. However, DOD has begun this assessment and expects to have results by July 2004. In 1997, the services and Joint Staff each conducted separate reviews of their general and flag officer requirements and, in doing so, used different job evaluation methodologies and factors. We noted in our review of the 1997 study that differing methodologies made cross-service comparisons difficult, even when comparing similar positions from one service to another. The separate service reviews together created at least 24 different definitions of a general or flag officer. At the time, the Office of the Secretary of Defense stated that it permitted the services to use their own methodologies for three reasons: (1) The methodology selected had to recognize the unique mission and structure of each service. (2) There was no single definition of a general and flag officer and no one way to conduct job evaluations. The different methodologies were deemed valid. (3) Using existing methodologies and recently completed studies saved time. We stated that a single methodology, consistently applied, would have been a better approach than using different methodologies. In 2003, DOD used a single methodology and one set of factors to assess all general and flag officer positions. The Office of the Secretary of Defense, the services, and the Joint Staff representatives on the working and senior panels worked together with LMI to conduct the study. DOD officials we interviewed generally agreed with the approach of using a single methodology and set of factors. One official who had also participated in the 1997 study said the services had to spend time defending their service-specific methodologies and findings. The use of a single methodology in the 2003 study avoided this problem. The process promoted interaction and discussion among the organizations represented on the working and senior panels. In job evaluations, judgments and decisions by executives about job value and study results are expected to be made as part of the process. Adjustments made to study results on the basis of executive judgment should be explained. In our review of DOD’s 1997 study, we found that there were discrepancies between what the service secretaries recommended and what their own service studies found and that these discrepancies were not explained in DOD’s draft 1997 report. The service secretaries recommended, in the aggregate, a total number of authorizations that was 123 less than the number of requirements identified by the service studies (995 recommended authorizations compared with 1,118 requirements). When the service secretaries developed their recommendations, they did not explain the basis for their adjustments to the study results. We recommended that the Secretary of Defense modify the draft 1997 report to include an explanation of the criteria used by the service secretaries to modify the results of the services’ studies. For the 2003 study, we found that DOD tracked the adjustments made to the requirements at each step in the process, from the time the positions were originally submitted through final vetting by service and Joint Staff senior leadership. DOD’s study showed the number of general and flag officer positions originally submitted by the services and Joint Staff (1,625), the number as assessed by LMI scorers (1,627), the number approved by the senior panel (1,629), and the final study results following a review by service and Joint Staff senior leadership (1,630). While net adjustments in the total number of requirements were minimal (a net increase of five positions), adjustments to the grades of individual positions were made at each step in the process. These adjustments were tracked in the requirements database. Our analysis of the requirements database showed that LMI scorers made adjustments to a total of 74 positions. Of these 74 positions, 20 had been submitted as O-7/O-8 positions. LMI scored these 20 positions as belonging at one grade or the other, with 8 positions being assessed as O-8 and 12 positions being assessed as O-7. The other 54 positions represented “grade inversions,” where LMI scorers assessed the position at a pay grade level different from that submitted. Of the 54 grade inversions, 47 were downgrades (that is, for example, going from an O-8 to an O-7), and 7 were upgrades. The senior panel then had an opportunity to adjust the results of LMI’s scoring, and service and Joint Staff senior leadership vetted the requirements and made their own adjustments before the study results were finalized. The senior panel and the service and Joint Staff senior leadership made adjustments to a total of 38 positions. All but 1 of these 38 adjustments involved positions scored as grade inversions by the LMI scorers. (See app. II for more information on these 38 adjustments.) We found that, in contrast with the 1997 study, the reasons for adjusting the LMI scoring results were documented in most cases. DOD was able to produce documentation, such as a spreadsheet and e-mail messages, to explain most of the adjustments. Our review of this documentation showed that an explanation was provided for 34 of the 38 adjustments and was lacking for the other 4 adjustments. In a few cases, the documented explanation was brief. For instance, the senior panel upgraded an Army reserve position from O-7 (LMI’s scoring assessment) back to O-8 (the grade as originally submitted). The panel’s rationale was as follows: “Keep as O-8; important info left out; Dep Vice Cmdr for Homeland Defense; $900m budget.” In other cases, the explanation for the adjustment was more detailed. For example, the senior panel upgraded the President of the Naval War College from O-8 (LMI’s scoring assessment) back to O-9 (the grade as originally submitted). To support this adjustment, the Navy submitted a 2-page addendum to expand upon the original position description. In 1997, we found that the services had not fully evaluated the potential for military to civilian conversions as part of their general and flag officer studies. We noted that the need for additional general and flag officers could be reduced by converting general and flag officer positions that do not require a uniformed servicemember to civilian status. At that time, we recommended that the Secretary of Defense modify the draft 1997 report to include an evaluation of the potential to convert nonmilitary essential general and flag officer positions to civilian status. As part of its 2003 study, DOD did not consider which positions could conceivably be converted from the military ranks to the civilian workforce. DOD stated in its March 2003 report to Congress that following completion of the general and flag officer review, DOD would look at its senior leadership requirements—military and civilian. “We must look at the entire pool of senior leaders, to include DOD civilians, to assess how many we need overall to effectively manage the Department,” the report stated. “Once we have agreement on an overall number, we will seek additional flexibility to manage within an overall ceiling and assign the best qualified individual where needed.” In July 2003, DOD approved a contract with LMI to assess general and flag officer positions for potential civilian conversion. The statement of work calls for LMI to perform a “billet-by-billet assessment of each validated general and flag officer requirement to determine whether the position could be filled by a senior civilian.” DOD has commissioned separate studies to assess Senior Executive Service requirements. An official in the Office of the Secretary of Defense said the LMI civilian conversion study will pave the way for analysis of future legislative and policy proposals. LMI is to provide study results within a year of the contract date. DOD’s March 2003 report to Congress did not provide complete information on whether general and flag officer authorizations were sufficient to meet all requirements. Specifically, DOD’s 2003 report to Congress did not fully disclose the results from its general and flag officer study or address the magnitude or the impact of the gap between requirements and authorizations. Fully disclosing the study results and discussing the implications of its findings would help to explain DOD’s recommendation not to seek additional authorizations. In addition, the methodology used in the study was not designed to capture new or emerging needs that could result from DOD transformation efforts or other changes in the department. Such changes could be captured through periodic updates to the requirements validated through the 2003 study. In the absence of periodic updates, the validated requirements could become out-of-date. DOD also could enhance the precision of the requirements data by more clearly accounting for dual-hatted positions and could enhance the usefulness of the results by assessing the impact of external (joint) requirements on service authorizations. Finally, the requirements data could be improved by incorporating the results of the ongoing civilian conversion study. Addressing these issues would help DOD establish a solid baseline of validated general and flag officer requirements that will assist DOD as it assesses senior leadership (civilian/military) requirements and manages general and flag officers. In addition, clarification of DOD’s requirements would assist congressional decision makers in making adjustments to the maximum number of general and flag officers authorized to each service. We recommend that the Secretary of Defense direct the Under Secretary of Defense for Personnel and Readiness to take the following four actions: Clarify the magnitude and impact of the gap between DOD’s validated requirements for general and flag officers and congressional authorizations. This assessment should include (1) an analysis of the impact caused by the workarounds DOD uses to fill the gap between requirements and authorizations and (2) a more complete explanation of its recommendation not to seek additional authorizations in light of the study results showing that requirements exceeded authorizations. Update general and flag officer requirements periodically by identifying, assessing, and validating new general and flag officer requirements that emerge from DOD transformation efforts or other changes in the department. Take steps to enhance the precision and usefulness of the general and flag officer requirements. At a minimum, DOD should more clearly account for all dual-hatted positions in terms of whether each position is dual-hatted for efficiency or out of necessity due to shortages in general and flag officer authorizations. Positions that are dual-hatted out of necessity should be treated as separate positions for purposes of identifying requirements. In addition, to the extent possible, DOD should track service contributions of general and flag officers to external (joint) requirements to assess whether each service’s authorizations are sufficient to meet both internal and external requirements. Incorporate the results of the ongoing civilian conversion study in a future update of general and flag officer requirements. DOD provided written comments on a draft of this report. In its comments, DOD concurred with our recommendations and indicated that it will (1) address the impact of the gap in requirements and the use of workarounds in a separate study on alternative methods for dealing with the gap in requirements, (2) review all dual-hatted positions and add the additional information to the established requirements database, and (3) incorporate the results of the ongoing civilian conversion study in a future update of general and flag officer requirements. Regarding our recommendation to periodically update general and flag officer requirements, DOD indicated that its current procedures were adequate. However, as we stated in our report, these additional requirements are not going through the kind of job evaluation assessment and validation DOD used in its study. DOD could use the methodology developed for the baseline study to ensure that any additional requirements are validated. As we noted in our report, the assessment process resulted in validated requirements that in some cases differed from the requirements originally submitted by the services and the Joint Staff. Regarding our recommendation that DOD track the military services’ contributions of general and flag officers to external (joint) requirements, DOD stated that it will closely monitor service participation in this arena and that current safeguards mitigate the impact of joint participation. We continue to believe that an assessment of whether general and flag officer authorizations are sufficient to meet all requirements necessitates the inclusion of both internal service as well as external (joint) requirements. Furthermore, this assessment should be made at the service level because Congress has established service-specific authorization levels. DOD’s comments are reprinted in their entirety in appendix III. DOD also provided technical comments, which we have incorporated as appropriate. To determine whether DOD fully disclosed the results of its general and flag officer study in its March 2003 report to Congress and explained the rationale for any recommendations, we obtained and analyzed the results of the general and flag officer study, the database of validated requirements, supporting documentation, and the March 2003 report. We also reviewed laws and regulations pertaining to general and flag officers. We discussed the study with, and obtained documentation from, representatives of the Under Secretary of Defense for Personnel and Readiness, the Office of General Counsel (Office of the Secretary of Defense), LMI, and the military services’ and Joint Staff’s general officer matters offices. We also reviewed the Reserve Forces Policy Board’s comments on the general and flag officer study, met with Board staff, and obtained documentation concerning the Board’s involvement in the study. The scope of our review did not include an assessment of recommendations made by DOD concerning the management of general and flag officers. We plan to conduct a separate review of these management issues and recommendations. To assess the methodology used in the 2003 study, we reviewed prior DOD studies of general and flag officer requirements and the results of our review of DOD’s 1997 study, for which we had conducted an extensive literature review on job evaluation methods. We also reviewed a study by a public policy research contractor, contacted another outside researcher who was familiar with the study, and reviewed other literature on job evaluation. We reviewed the steps used to conduct the 2003 study and the origin and application of factors used in the assessment. We discussed the methodology with DOD and LMI officials. We also determined how the study accounted for dual-hatted positions and external (joint) requirements. Regarding dual-hatted positions, we examined selected positions from the requirements database. We also compared dual-hatted positions in the database with service and DOD-wide rosters of general and flag officers. Regarding external (joint) requirements, we examined Joint Staff data to determine the services’ contributions to meeting validated external (joint) general and flag officer requirements. We did not review the study findings to determine whether requirements for individual positions were validated appropriately through the study. To determine whether DOD incorporated lessons learned from our review of DOD’s 1997 general and flag officer study, we reviewed the findings and recommendations from our 1997 review and compared these lessons learned to the approach used in the 2003 study. To assess the tracking and documentation of adjustments made to study results, we analyzed the requirements database. We then obtained from DOD documentation concerning adjustments that were made to the study results by the senior panel and the service and Joint Staff senior leadership. We also reviewed DOD’s plans for a follow-on study concerning conversion of general and flag officer positions to the civilian workforce. We are sending copies of this report to the Secretary of Defense and the Director, Office of Management and Budget. In addition, the report will be available at no charge on GAO’s Web site at http://www.gao.gov. If you or your staff have any questions regarding this report, please contact me at (202) 512-5559 (stewartd@gao.gov) or Brenda S. Farrell at (202) 512-3604 (farrellb@gao.gov). Major contributors to this report were Thomas W. Gosling, J. Paul Newton, and Bethann E. Ritter. 1. Characteristics of function a. Type (e.g., command, general or coordinating staff, special staff, manager, deputy, specialist, etc.) b. Scope (e.g., operational command, training command, installation command, personnel management, officer personnel management, legal affairs, information, etc.) c. Level of function (e.g., national, secretarial, service, theater, field command, etc.) 2. Grade and position of c. lateral points of coordination (relative position within the military or governmental structure within which the position’s function is performed) 3. Supervision over position a. Proximity (remoteness or closeness of supervision) b. Degree (independence of operation) 4. Official relations with U.S. and foreign governmental officials and with the public a. Nature (e.g., reports to, works for, keeps informed, provides liaison, etc.) b. Extent (e.g., primary function, frequent requirement, continuous additional duty, occasional requirement, etc.) c. Level of official relations with U.S. and foreign governmental officials and with the public (e.g., governmental department or agency, national or local government, civil organizations, industry, press, non-governmental organizations , private volunteer organizations , etc.) 5. Reflection of national emphasis and determination (relation of position to national objectives and programs, special conditions under which the position was first established or other reasons why the position reflects national will) 6. Special qualifications required by the position (any special qualifications such as advanced education, or particular training or experience, which are essential to the proper execution of positional responsibilities) 7. Mission(s) of organization and the special requirements of the position as it relates to the mission(s) (the nature of the responsibilities that are associated with the position and the need for multidimensional “executive skills.” The mission of the organization is the key, day-to-day activities that are accomplished.) 8. Number, type, and value of resources managed and employed. Data should be displayed within three categories: operational control, administrative control, and immediate staff within each subsection. a. Military forces (number and type of forces normally assigned or programmed for planned or special operations) b. Personnel (number of personnel by officer and warrant officer, enlisted, and civilian) c. Value of equipment and properties (total value of equipment, supplies, and real property displayed in millions) e. Foreign resources (scope and type of foreign resources involved, if any) 9. Geographical area of responsibilities (the size, location, and, if appropriate, the criticality of the land, sea, or air spaces involved) 10. Authority to make decisions and commit resources (the scope of the position with respect to specific authority delegated to or withheld from the position in either routine or emergency situations) 11. Development of policy (involvement in the development of policy within the specific functional areas associated with the position, e.g., budget, program, communications, or manpower) 12. National commitment to international agreements (authority to make commitments to foreign nations or involvement in negotiating such commitments for the United States) 13. Auxiliary (supporting) authorities and responsibilities inherent in the position (inherent requirements charged to the position by virtue of situation, location, proximity, tradition, etc.) 14. Impact on national security or other national interests (effect of mission accomplishment or position performance on the protection of national interests or the advancement of national programs) 15. Importance to present and future effectiveness and efficiency of the national defense establishment (effect on the force structure, operational capabilities, status of combat readiness, quality of personnel and equipment, cost effectiveness, command and control means, management procedures and techniques, responsiveness to national needs, or other factors) 16. Effect on the prestige of the nation or the armed forces (how effectiveness or accomplishment reflects on the stature of the nation and its armed forces, and influences the credibility of national aims and capabilities) This appendix lists the 38 general and flag officer positions where adjustments were made after the Logistics Management Institute (LMI) scorers conducted their assessment. The Department of Defense (DOD) components (the military services and Joint Staff) submitted the pay grade level for each general and flag officer position reviewed. LMI scorers assessed the positions to validate the grade level. The senior panel reviewed the results of LMI’s scoring assessment, made adjustments, and approved the validated requirements. The requirements were then vetted through the senior leadership of the services and the Joint Staff, who requested further adjustments. With these adjustments, the study results were finalized. Table 7 shows, for each of the 38 positions, the adjustments to the position’s pay grade as approved by the senior panel and vetted through the senior leadership.

Summary: The Fiscal Year 2003 National Defense Authorization Act directed the Department of Defense (DOD) to assess whether general and flag officer authorizations were sufficient to meet all requirements. GAO's objectives were to determine whether DOD (1) fully disclosed the results of its study in its March 2003 report to Congress and explained the rationale for any recommendations, (2) used an established methodology to meet the objectives of its study, and (3) incorporated lessons learned from a GAO review of DOD's 1997 general and flag officer study. The 2003 act also directed DOD to review legislation affecting general and flag officer management. DOD included the results of its review in the March 2003 report, making several recommendations. GAO plans a separate review of these issues and recommendations. DOD's March 2003 report to Congress did not fully disclose the results of the general and flag officer study or explain its recommendation not to seek additional authorizations (people) to meet validated requirements (positions). The general and flag officer study validated requirements for general and flag officer positions that exceeded congressional authorizations for both the active and reserve components. However, the validated requirements data generated by the study were not disclosed in the March 2003 report to Congress. In its report, DOD did not address the magnitude of the gap between validated requirements of 1,630 positions and congressional authorizations of 1,311--a difference of 319. DOD's report also did not address the impact of "workarounds" used to fill the gap between requirements and authorizations, such as the practice of assigning colonels and Navy captains to general and flag officer positions. Fully disclosing the study results and discussing the implications of these findings in the March 2003 report to Congress would have provided a more complete picture of DOD's general and flag officer requirements and may have helped to explain its recommendation not to seek additional authorizations. DOD used an established methodology to conduct a position-by-position validation of general and flag officer requirements. This methodology, known as job evaluation, has been widely used in the United States. Job evaluation, however, has numerous subjective features, including the selection of factors used for measurement. In addition, it is not designed to project emerging needs, such as those that could result from transformation efforts. Periodic updates could capture changes in requirements. Such limitations do not invalidate DOD's methodology; however, an explicit acknowledgment and assessment of these limitations would have provided more context for the study results. In addition, the study did not clearly account for dual-hatted positions (where one individual holds more than one position simultaneously) or assess how each service's authorizations were affected by the need to contribute general and flag officers to fill external (joint) positions. Addressing these issues could have enhanced the precision and usefulness of DOD's study. In addition, we noted that while Congress directed DOD to ensure the Reserve Forces Policy Board participated in development of the report's recommendations, the Board played a minimal role in producing DOD's 2003 report. The Board registered strong objections to DOD's recommendation not to seek additional authorizations now to meet validated requirements and to the limited role it played in the process. DOD, in conducting its 2003 general and flag officer study, incorporated some of the lessons learned from a GAO review of DOD's 1997 general and flag officer study. DOD recognized the need to identify general and flag officer positions that could conceivably be converted from the military ranks to the civilian workforce, although it deferred this assessment until after the general and flag officer study was complete. DOD is currently assessing civilian conversion of general and flag officer positions.