Results Act: Observations on CFTC's Annual Performance Plan
Released: 1998-10-08T00:00:00

Letter

Results Act: Observations on CFTC’S Annual Performance Plan

Mr. Chairman and Members of the Subcommittee: I am pleased to be here today to assist the Subcommittee in its review of the Commodity Futures Trading Commission’s (CFTC) fiscal year 2000 annual performance plan. Hearings like this one and the one you held last year on CFTC’s 1997-2002 strategic plan continue to be an important means of ensuring that the intent of the Government Performance and Results Act of 1993 (Results Act) is met.

As you know, annual performance plans can be an invaluable tool for making policy decisions, improving program management, enhancing accountability, and communicating to both internal and external audiences on how the long-term direction outlined in strategic plans is translated into the day-to-day activities of managers and staff. Successful implementation of a performance-based management system, as envisioned by the Results Act, represents a significant challenge requiring sustained agency attention.

My testimony today focuses on five areas in which CFTC could improve its performance plan to make it a more useful tool for congressional and executive branch decisionmakers. Although opportunities exist to improve CFTC’s fiscal year 2000 performance plan, CFTC actions to date clearly show a good faith effort to comply with the Results Act and the Office of Management and Budget (OMB) guidance in developing its plan. In our discussions with CFTC staff, we found CFTC fully committed to meeting both the requirements of the Act and congressional expectations that the plan inform Congress and the public about CFTC performance goals, including how the agency will accomplish these goals and measure results. In addition, the areas in which CFTC could improve its plan are some of the same areas in which we found that many other federal agencies, including federal financial regulators, could improve their plans.

Specifically, CFTC could improve its plan in the following five areas:

Performance goals, measures, and targets could provide a clearer picture of intended performance.

Mission, goals, and activities could be better connected to more fully demonstrate how CFTC will chart annual progress toward achieving its long-term strategic goals.

Crosscutting efforts could be addressed more fully if CFTC worked with the affected federal agencies to develop performance goals and measures that reflect the nature and extent of their common efforts.

Strategies and resources used to achieve goals could be discussed in greater detail to better enable congressional and other decisionmakers to judge their reasonableness.

The means for verifying and validating that performance information is sufficiently complete, accurate, and consistent, as well as the extent to which such information and the means for collecting, maintaining, and analyzing it are reliable, should be discussed.

My comments today apply to the fiscal year 2000 annual performance plan that CFTC prepared for OMB in September 1998. Our assessment of CFTC’s plan was based on knowledge of the agency’s operations and programs; past reviews of CFTC, including a review of its 1997-2002 strategic plan; results of work on other agencies’ performance plans and the Results Act; discussions with CFTC staff; and other information available at the time of our assessment. The criteria we used to determine whether CFTC’s plan complied with the requirements of the Results Act were the Results Act itself; OMB guidance on preparing strategic and performance plans (OMB Circular A-11, Part 2); and GAO guidance on assessing agency performance plans.

Background

CFTC, an independent agency created by Congress in 1974, administers the Commodity Exchange Act (CEA), as amended. The principal purposes of the CEA are to protect the public interest in the proper functioning of the market’s price discovery and risk-shifting functions. In administering the CEA, CFTC is responsible for fostering the economic utility of the futures market by encouraging its efficiency, monitoring its integrity, and protecting market participants from abusive trade practices and fraud. prepared—to a focus on the results of those activities, such as protecting the economic functioning of the commodity futures and option markets.

Under the Results Act, strategic plans are the starting point for setting goals and measuring progress towards them. The Results Act requires virtually every executive agency to develop a strategic plan, covering a period of at least 5 years forward from the fiscal year in which the plan is submitted. In September 1997, CFTC formally submitted its fiscal years 1997-2002 strategic plan to Congress and OMB. This plan established three strategic goals: (1) protect the economic functions of the commodity futures and option markets; (2) protect market users and the public; and (3) foster open, competitive, and financially sound markets.

The Results Act also requires a federal agency to prepare an annual performance plan covering the program activities set out in its budget. In establishing the requirement for a performance plan, the Results Act establishes the first statutory link between an agency’s budget request and its performance planning efforts. The performance plan is to reinforce the connections between the long-term strategic goals outlined in the agency’s strategic plan and the daily activities of program managers and staff.

Finally, the Results Act requires executive agencies to prepare annual reports on program performance for the previous fiscal year. The performance reports are to be issued by March 31 each year, with the first (for fiscal year 1999) to be issued by March 31, 2000. In each report, the agency is to compare its performance against its goals, summarize findings of program evaluations completed during the year, and describe the actions needed to address any unmet goals.

Performance Goals, Measures, and Targets Could Be Improved

results-oriented, and goals were provided for all measures and internal management challenges; (2) some performance goals were made self-measuring and others were made more objective; (3) certain performance measures were replaced, restated, or deleted; and (4) baselines were established against which annual targets could be compared.

Goals and Measures Could Be More Results-Oriented and Complete

Results-oriented, or outcome, goals and measures provide the clearest picture of intended and actual performance. However, most of CFTC’s performance goals and measures focus on program outputs—such as the number of meetings attended and number of research projects or reports completed. In our testimony before this Subcommittee last year, we highlighted a similar problem with CFTC’s strategic plan. Although we recognize that establishing outcome measures is particularly challenging for regulatory agencies as they move from a focus on the activities they undertake to the results they are trying to achieve, a key shortcoming of CFTC’s performance plan is that it relies on output measures that describe completed activities, not program results. Also, these measures are weighted toward measuring the quantity of completed activities, rather than the quality, cost, or timeliness of performance outcomes. As mentioned earlier, the Results Act is intended to shift the focus of government decisionmaking and accountability away from a preoccupation with completed activities to a focus on the results of such activities.

The focus of CFTC’s performance plan on output measures appears to flow from its strategy of deriving performance goals from program activities. For example, one performance goal is to aggressively identify, investigate, and take action against individuals engaged in fraudulent Internet and media activities. This goal is associated with the program activity of monitoring the Internet and other media for fraudulent activities and other possible violations of the CEA. The measure for the goal is the number of referrals to enforcement authorities generated from Internet and media monitoring—an output of the activity, not an outcome of a program. results-oriented performance goals. For example, the Enforcement Program accomplishment section discusses the effectiveness of its quick-strike ability—the ability to file injunctive actions quickly after detecting fraud—to, among other things, obtain timely injunctive relief and enhance the possibility that customer funds will be recovered. This accomplishment section describes cases, filed within days or weeks of CFTC’s discovering an illegal activity, that stopped fraud at an early stage and preserved customer funds. CFTC’s performance measures could be made more results-oriented by replacing measures, such as reports on activities related to bringing injunctive actions and sanctioning violators, with more outcome-related measures, such as the percentage of quick-strike cases filed within a certain number of days of starting an investigation that resulted in sanctions and the percentage of funds recovered.

CFTC could also learn from the plans of other federal financial regulators that are attempting the transition to results-oriented goals. For example, the National Credit Union Administration is developing new outcome performance goals. One outcome goal of the National Credit Union Administration is to ensure that federally insured credit unions are adequately capitalized. A performance goal is to reduce the percentage of federally insured credit unions that are undercapitalized by 10 percent, from 372 to 335.

CFTC’s plan could also be improved if performance goals were provided for all activities and performance targets as well as for internal management challenges. Currently, the plan has 16 activities for which no performance goals exist. For example, no performance goal exists for the activity of reviewing and overseeing self-regulatory organization audit and financial practices. Without a performance goal, it is not clear what performance is expected. Also, CFTC’s strategic plan identifies several internal management challenges that the performance plan does not address. These challenges include diminishing resources, recruiting and retaining qualified professionals, remaining abreast of current technology, and remaining educated and informed as innovation changes the industry. To better respond to the intent of the Results Act, CFTC could add agencywide performance goals to the plan to address these challenges or incorporate these challenges in existing performance goals and measures.

Some Goals Could Be Made Self-Measuring, and Others Could Be Made More Objective

Although not required by the Results Act, CFTC could redefine some performance goals so that they are self-measuring, thereby reducing the complexity of the plan. Currently, all but 2 of CFTC’s 31 performance goals are stated as abstract goals—that is, as goals requiring that specific performance measures be defined to assess progress toward their achievement.

Performance goals that can be redefined so that they are self-measuring generally have one measure or two or more measures that can be combined. For example, the performance goal for the activity of reviewing the adequacy of self-regulatory organization disciplinary actions might be restated in the following way: On an annual basis, review a certain percentage of self-regulatory organization disciplinary actions to ensure compliance with CFTC standards. This approach, which has been taken by other federal financial regulators, such as the National Credit Union Administration and Federal Deposit Insurance Corporation, clearly defines performance expectations. For example, the Federal Deposit Insurance Corporation has the following performance goal: Market 80 percent of a failing financial institution’s assets based on book value at the time of resolution or within 90 days.

CFTC’s performance plan could also be improved by reducing the extent to which performance goals require interpretation. To the extent possible, goals should not require subjective considerations to dominate measurement. For example, one performance goal is defined as follows: Bring important cases (including matters involving ongoing conduct and complex transactions) aggressively. The performance goal does not define what an important case is or what it means to aggressively bring a case.

Certain Performance Measures Could Be Replaced, Restated, or Deleted

CFTC’s performance plan has 31 performance goals with 228 performance measures to address them. CFTC could replace, restate, or delete performance measures for certain performance goals to strike a better balance between too few and too many measures and to enhance its ability to assess the progress made in achieving performance goals. measure captures the amount of time it takes to process applications or changes, they could be replaced with one measure that captures the percentage of applications and changes processed in 45 days or less.

Second, CFTC could restate or delete certain performance measures, because they do not appear to be clearly related to their performance goals and/or appear to be unduly affected by external factors. For example, the number of active futures and option markets is used to measure, in part, two goals: (1) identify traders who can influence futures prices and (2) determine whether traders are influencing futures prices. The number of active futures markets is determined by futures exchanges and other external factors and has little direct bearing on the two goals. As a result, the measure could be deleted.

Third, CFTC could restate or delete certain performance measures, because they may have limitations that preclude them from accurately capturing intended performance and may promote unintended consequences. For example, one of the plan’s performance goals is to conduct important investigations and refer potential violations to other authorities as appropriate. The performance measures for this goal include the number of documents obtained through subpoena or inspection, number of witnesses from whom testimony was taken, and number of witnesses interviewed. These output-oriented measures could provide an incentive for staff to conduct more interviews, take more testimonies, and obtain more documents than necessary, which could add cost and time to investigations without necessarily contributing commensurately to their success.

Baselines Could Be Established Against Which to Compare Annual Targets

Although CFTC’s performance plan includes annual targets for performance goals, it could make such information more useful by providing baselines, or a context, for assessing the reasonableness and appropriateness of such targets. As we recently reported, agencies that go beyond the requirements of the Results Act and include baseline or trend data for their goals provide a more informative basis for assessing expected performance. year covered by the plan for the number of cease and desist orders, registration sanctions, and trading prohibitions. However, the basis for setting the specific targets and the contributions of these targets to the outcome objective are not readily apparent. Without this contextual information, the reader does not know if CFTC’s output-oriented performance targets are reasonable.

Mission, Goals, and Activities Could Be Better Connected

Consistent with the Results Act, CFTC’s performance plan attempts to show the relationship between the agency’s annual performance goals and its fiscal years 1997 through 2002 mission and strategic goals. To do so, the plan uses tables that connect each strategic goal to its accompanying set of performance goals, measures, and annual targets. The plan also ranks the agency’s outcome objectives by dollars budgeted, which is a starting point for providing useful information about its priorities. However, the plan could better connect mission, goals, and activities by more fully demonstrating how CFTC will chart annual progress toward achieving its long-term strategic goals.

As we found with other agency performance plans, CFTC’s plan associates one performance goal with multiple program activities and strategic goals. Such associations make it difficult to determine whether all activities are substantially covered or to understand how specific program activities are intended to contribute to CFTC’s strategic goals. For example, the performance goal—assess sanctions that are remedial and deter violators—is associated with three different program activities and three different strategic goals. Moreover, the measures and targets for this performance goal differ with each program activity and strategic goal.

Similarly, the performance plan’s presentation of many separate program areas, program activities, and performance goals, measures, and targets makes it difficult to link CFTC’s mission and strategic goals to performance goals across the entire agency. The plan’s 31 performance goals and 228 performance measures support 3 strategic goals and 9 strategic objectives, covering 5 program areas. Although the plan’s ranking of outcome objectives offers a useful perspective on CFTC’s priorities, given the high level of complexity, it is difficult to (1) identify the agency’s key priorities among the many goals and measures, (2) differentiate efforts to meet these priorities, and (3) understand what will be achieved if all the performance goals are met.

Crosscutting Efforts Could Be More Fully Addressed

The Results Act seeks to ensure that crosscutting goals of federal programs are consistent; strategies are mutually reinforcing; and, as appropriate, progress is assessed through the use of common performance measures. OMB guidance tasks performance plans with identifying those performance goals that are being mutually undertaken with other federal agencies in support of programs or activities of a crosscutting nature.CFTC’s plan recognizes the need to address crosscutting efforts. However, it could more fully address such efforts if the agency worked with the cognizant federal agencies to develop performance goals and measures that better reflect the nature and extent of their common efforts.

CFTC’s performance plan could be expanded to include performance goals and measures to more adequately address crosscutting efforts, such as those identified in its budget justification that accompanied its performance plan to OMB. These include CFTC’s participating in the President’s Working Group on Financial Markets; sharing information with other financial regulators; working with the U.S. Department of Agriculture on a risk management education program; contributing to a Department of the Treasury initiative that encourages global financial stability; as well as cooperative enforcement efforts with the Department of Justice, the Federal Bureau of Investigation, the Federal Reserve Board, the Federal Trade Commission, the Securities and Exchange Commission, and the U.S. Postal Inspection Service.

CFTC’s performance plan briefly discusses CFTC’s need to work with other U.S. financial regulators through, among other means, the President’s Working Group on Financial Markets. However, the related performance goals and measures do not directly address the type of crosscutting performance that the Working Group was created to address. For example, CFTC’s performance goal covering the Working Group is for CFTC to contribute to the performance of the group. The measure for this goal is the number of meetings attended, and the fiscal year 2000 target is two meetings.

As supported by OMB guidance, CFTC could strengthen its performance plan by participating with the members of the Working Group and other federal regulators involved in crosscutting programs to develop common performance goals and measures. For example, the continued growth and development of the over-the-counter (off-exchange) derivatives market has raised a number of potential regulatory concerns that affect CFTC and other members of the Working Group. In addition, the potential need to develop a financial markets contingency plan to address the “Year 2000” computer dating problem could involve coordination among CFTC and other federal financial regulators.

Strategies and Resources Used to Achieve Goals Could Be Discussed in Greater Detail

CFTC’s plan provides important information on how strategies and resources will be used to achieve goals. However, expanding this discussion could better enable congressional and other decisionmakers to judge the reasonableness of CFTC’s strategies and anticipated resource deployment.

Consistent with the Results Act and OMB guidance, CFTC’s performance plan attempts to address the strategies that CFTC will use to achieve its performance goals. Although this discussion should cover operational processes, skills, and technologies, CFTC’s discussion focuses on the agency’s operational processes and, to a much lesser extent, on skills and technologies. Aligning its discussion with CFTC’s strategic goals, the plan briefly describes the major activities of each program in relation to its performance goals and measures. In a few cases, the plan also discusses skills or technologies that programs will use in relation to performance goals and measures. The plan could be made more useful by providing additional information on the skills and, if appropriate, technologies used in connection with operational processes to achieve program goals.

Also, consistent with the Results Act, CFTC’s performance plan discusses the resources that will be applied to achieve the agency’s performance goals. Using tables and graphics, the plan shows the amount of budget funding and the number of full-time equivalent employees that will be needed to achieve each strategic goal and the individual outcome objectives covering each strategic goal. CFTC’s plan could be further improved by describing the resources required to achieve each performance goal.

The Means for Verifying and Validating Performance Information Should Be Described

Although required by the Results Act and OMB guidance, CFTC’s fiscal year 2000 performance plan does not describe the procedures that the agency will use to verify and validate that performance information is sufficiently complete, accurate, and consistent. Nor does the plan discuss the extent to which the performance information and the means for collecting, maintaining, and analyzing it are reliable. CFTC’s performance plan should be expanded to address these requirements.

In addition, to assess progress toward achieving its goals, CFTC will need to collect information on the over 200 performance measures in its plan. For some of these measures, the amount of information to be collected is voluminous and covers activities across CFTC headquarters and regional offices. Errors can occur in collecting, maintaining, processing, and reporting such information—potentially introducing bias and resulting in inaccurate estimates of program performance. As a result, CFTC should have procedures for ensuring that its performance information is free of significant levels of error and that bias is not introduced. Such procedures can include internal controls over data collection, maintenance, and entry, as well as audits, evaluations, and peer reviews.

In summary, Mr. Chairman, it is important to recognize that although CFTC’s performance planning can be further improved, the Results Act anticipated that the process of developing an effective planning process and plans could take several planning cycles. We look forward to continuing to work with Congress and CFTC to ensure that the requirements of the Results Act are met.

Mr. Chairman, this concludes my prepared statement. My colleagues and I would be pleased to answer any questions that you or Members of the Subcommittee may have.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.