Community Development: Early Results of the Community Development Financial Institutions Fund's Programs
Released: 1998-05-13T00:00:00

Letter

Background

Currently located within the Department of the Treasury, the CDFI Fund was authorized in 1994 and has received appropriations totaling $225 million through fiscal year 1998. The 1995 Rescissions Act limited the Fund to 10 full-time equivalent (FTE) staff for fiscal years 1995 and 1996, but for fiscal year 1998, the Fund has a FTE ceiling of 35 staff. As of May 8, 1998, the Fund had 27 full-time and 2 part-time staff.

The Fund’s overall performance is subject to the Results Act. This act seeks to improve the management of federal programs and their effectiveness and efficiency by establishing a system for agencies to set goals for performance and measure the results. Under the act, federal agencies must develop a strategic plan that covers a period of at least 5 years and includes a mission statement, long-term general goals, and strategies for reaching those goals. Agencies must report annually on the extent to which they are meeting their annual performance goals and identify the actions needed to reach or modify the goals they have not met. The Fund completed its final plan in September 1997 and is currently considering revisions to that plan.

Stronger Performance Measures Would Provide a Better Basis for Monitoring and Evaluating the CDFI Program’s Accomplishments

While the CDFI Fund has established a system for measuring awardees’ performance in the CDFI program, this system emphasizes activities over accomplishments and does not always include measures for key aspects of goals. In addition, baseline information that was available to the Fund seldom appears in the Fund’s performance measurement schedule. A more comprehensive performance measurement system would provide better indicators for monitoring and evaluating the program’s results.

Progress in Developing Performance Measures Is Mixed

The CDFI Fund’s progress in developing performance goals and measures for awardees in the CDFI program is mixed. On the one hand, the Fund has entered into assistance agreements with most of the 1996 awardees. As the CDFI Act requires, these assistance agreements include performance measures that (1) the Fund negotiated with the awardees and (2) are generally based on the awardees’ business plans. On the other hand, the Fund’s performance goals and measures fall somewhat short of the standards for performance measures established in the Results Act. Although awardees’ assistance agreements are not subject to the Results Act, the act establishes performance measurement standards for the federal government, including the CDFI Fund. In the absence of specific guidance on performance measures in the CDFI Act, we drew on the Results Act’s standards for discussion purposes.

The assistance agreements called for under the CDFI Act require awardees to comply with multiple provisions, including the accomplishment of agreed-upon levels of performance by the final evaluation date, typically 5 years in the future. As of January 1998, the Fund had entered into assistance agreements with 26 of the 31 awardees for 1996. We found, on the basis of our six case studies, that the Fund had negotiated performance goals that met the statutory requirements and established goals for awardees that match the Fund’s intended purpose, extensively involved the awardees in crafting their planned performance, and produced a flexible schedule for designing goals and measures.

According to the Results Act, both activity measures, such as the number of loans made, and accomplishment measures, such as the number of new low-income homeowners, are useful measures. However, the act regards accomplishment measures as more effective indicators of a program’s results because such measures identify the impact of the activities performed. Our survey of CDFIs nationwide, including the 1996 awardees, and our review of six case study awardees’ business plans showed that CDFIs use both types of measures to assess their progress toward meeting their goals. Yet our review of the 1996 awardees’ assistance agreements revealed a far greater use of activity measures. As a result, the assistance agreements focus primarily on what the awardees will do, rather than on how their activities will affect the distressed communities. According to most of the case study awardees, difficulties in isolating and measuring the results of community development efforts and concerns about the effects of factors outside the awardees’ control inhibited the awardees’ use of accomplishment measures.

According to the Results Act, goals and measures should be related and clear. We found that most of the goals and measures were related; however, in some agreements, the measures did not address all key aspects of the goals. Finally, under the Results Act, clarity in performance measurement is best achieved through the use of specific units, well-defined terms, and baseline and target values and dates. While the measures in the agreements included most of these elements, they generally lacked baseline values and dates. Fund officials told us that they used baseline values and dates in negotiating the performance measures, but this information did not appear in the assistance agreements themselves. Therefore, without information contained in awardees’ files, it is difficult to determine the level of increase or contribution the investment is intended to achieve.

Refining the awardees’ goals and measures to meet the Results Act will facilitate the Fund’s assessment of the awardees’ progress over time. The Fund is taking steps to avoid some of the initial shortcomings in future agreements and is seeking to enhance its expertise and staffing.

Mandatory Monitoring and Evaluation Systems Are Not Yet in Place

Although the Fund has developed reporting requirements for awardees to collect information for monitoring their performance, it lacks documented postaward monitoring procedures for assessing their compliance with their assistance agreements, determining the need for corrective actions, and verifying the accuracy of the information collected. In addition, the Fund has not yet established procedures for evaluating the impact of awardees’ activities. The effectiveness of the Fund’s monitoring and evaluation systems will depend, in large part, on the quality of the information being collected through the required reports and the Fund’s assessment of awardees’ compliance and the impact of awardees’ activities. Primarily because of statutorily imposed staffing restrictions in fiscal years 1995 and 1996 and subsequent departmental hiring restrictions, the Fund has had a limited number of staff to develop and implement its monitoring and evaluation systems. In fiscal year 1998, it began to hire management and professional staff to develop monitoring and evaluation policies and procedures.

The Fund has established quarterly and annual reporting requirements for awardees in their assistance agreements. Each awardee is to describe its progress toward its performance goals, demonstrate its financial soundness, and maintain appropriate financial information. However, according to an independent audit recently completed by KPMG Peat Marwick, the Fund lacks formal, documented postaward monitoring procedures to guide Fund staff in their oversight of awardees’ activities. In addition, Fund officials indicated that they had not yet established a system to verify information submitted by awardees through the reporting processes.

Fund staff told us that they had not developed postaward monitoring procedures because of the CDFI program’s initial staffing limits. Now that additional staff are in place, they have begun to focus their attention on monitoring issues, including those identified by KPMG Peat Marwick.

The CDFI statute also specifies that the Fund is to annually evaluate and report on the activities carried out by the Fund and the awardees. According to the Conference Report for the statute, the annual reports are to analyze the leveraging of private assistance with federal funds and determine the impact of spending resources on the program’s investment areas, targeted populations, and qualified distressed communities. To date, the Fund has published two annual reports, the second of which contains an estimate of the private funding leveraged by the CDFI funding. This estimate is based on discussions with CDFIs and CDFI trade association representatives, not on financial data collected from the awardees. Anecdotal information from three of our six case study awardees indicates that the CDFI funding has assisted them in leveraging private funding. One awardee estimated that the Fund’s award generated more than three times its value in private investment.

In part because it has been only 15 months since the Fund made its first investment in a CDFI, information on performance in the CDFI program is not yet available for a comprehensive evaluation of the program’s impact, such as the Conference Report envisions. The two annual reports include anecdotes about individuals served by awardees and general descriptions of awardees’ financial services and initiatives, but they do not evaluate the impact of the program on its investment areas, targeted populations, and qualified distressed communities. Satisfying this requirement will entail substantial research and analysis, as well as expertise in evaluation and time for the program’s results to unfold.

Fund officials have acknowledged that their evaluation efforts must be enhanced, and they have planned or taken actions toward improvement. For instance, the Fund has developed preliminary program evaluation options, begun hiring staff to conduct or supervise the research and evaluations, and revised the assistance agreements for the 1997 awardees to require that they annually submit a report to assist the Fund in evaluating the program’s impact. However, because the Fund has not yet finished hiring its research and evaluation staff, it has not yet reached a final decision on what information it will require from the awardees to evaluate the program’s impact. The Fund also has to determine how it will integrate the results of awardees’ reported performance measurement or recent findings from related research into its evaluation plans.

Preliminary Reports Describe Early Postaward Activity

As to be expected, reports of accomplishments in the CDFI program are limited and preliminary. Because most CDFIs signed their assistance agreements between March 1997 and October 1997, the Fund has just begun to receive the required quarterly reports, and neither the Fund nor we have verified the information in them. Through February 1998, the Fund had received 41 quarterly reports from 19 CDFIs, including community development banks, community development credit unions, nonprofit loan funds, microenterprise loan funds, and community development venture capital funds. The different types of CDFIs support a variety of activities, whose results will be measured against different types of performance measures.

Given the variety of performance measures for the different types of CDFIs, it is difficult to summarize the performance reported by the 19 CDFIs. To illustrate cumulative activity in the program to date, we compiled the data reported for the two most common measures—the total number of loans for both general and specific purposes and the total dollar value of these loans. According to these data, the 19 CDFIs made over 1,300 loans totaling about $52 million. In addition, the CDFIs reported providing consumer counseling and technical training to 480 individuals or businesses.

Impact of the BEA Program Is Difficult to Assess

In the BEA program, as of January 1998, about 58 percent of the banks had completed the activities for which they received the awards and the Fund had disbursed almost 80 percent of the $13.1 million awarded in fiscal year 1996. Despite this level of activity, the impact of the program on banks’ investments in distressed communities is difficult to assess. Our case studies of five awardees and interviews with Fund officials indicate that although the BEA awards encouraged some banks to increase their investments, other regulatory or economic incentives were equally or more important for other banks. In addition, more complete data on some banks’ investments are needed to guarantee that the increases in investments in distressed areas rewarded by the BEA program are not being offset by decreases in other investments in these distressed areas. The Fund has tried to measure the program’s impact by estimating the private investments leveraged through the BEA awards. However, this estimate includes banks’ existing, as well as increased, investments in distressed areas. Furthermore, the Fund cannot be assured that the banks’ increased investments remain in place because it does not require banks to report any material changes in these investments. Although the CDFI statute does not require awardees to reinvest their awards in community development, banks have reported to the Fund that they have done so, thereby furthering the BEA program’s objectives, according to the Fund. Finally, the Fund does not have a postaward evaluation system for assessing the impact of the program’s investments.

Impact of Award Varied, but Regulatory and Economic Incentives Generally Had a Greater Influence on Banks’ Investments

Our analysis indicated that the impact of the BEA award varied at our five case study banks. One bank reported that it would not have made an investment in a CDFI without the prospect of receiving an award from the Fund. In addition, a CDFI Fund official told us that some CDFIs marketed the prospect of receiving a BEA award as an incentive for banks to invest in them. We found, however, that the prospect of an award did not influence other banks’ investment activity. For example, two banks received awards totaling over $324,000 for increased investments they had made or agreed to make before the fiscal year 1996 awards were made.

Banks have multiple incentives for investing in CDFIs and distressed areas. Therefore, it is difficult to isolate the impact of the BEA award from the effects of other incentives; however the receipt of a BEA award is predicated on a bank’s increasing investments in community development. Discussions with our five case study banks indicated, however, that regulatory and economic incentives have a greater influence on these banks’ investments than the prospect of a BEA award. A reason that the banks frequently cited for investing in CDFIs and distressed areas was the need to comply with the Community Reinvestment Act (CRA). Economic considerations also motivated the banks. One bank said that such investments lay the groundwork for developing new markets, while other banks said that the investments help them maintain market share in areas targeted by the BEA program and compete with other banks in these areas. Two banks cited improved community relations as reasons for their investments. Some banks indicated that, compared with these regulatory and economic incentives, the BEA award provides a limited incentive, especially since it is relatively small and comes after a bank has already made at least an initial investment.

Some Banks Do Not Maintain Data Needed for Evaluation

According to Fund officials, a small portion of the 1996 awardees do not maintain the geographic data needed to determine whether any new investments in distressed areas are coming at the expense of other investments—particularly agricultural, consumer, and small business loans—in such areas. Concerned about the validity of the net increases in investments in distressed areas reported by awardees, the Fund required the 1996 awardees that did not maintain such data to certify that, to the best of their knowledge, they had not decreased investments in distressed areas that were not linked to their BEA award. While most banks maintain the data needed to track their investments by census tract and can thus link their investments with distressed areas, a few do not do so for all types of investments.

Fund’s Estimate Includes Existing as Well as Increased Investments

In an attempt to measure an impact of the BEA program, the Fund has reported that awards of $13.1 million in 1996 leveraged over $125 million in private investment—a leveraging ratio of almost 10 to 1. This estimate includes banks’ existing investments in CDFIs and direct investments in distressed areas. When we included only the banks’ new direct investments, we calculated a leveraging ratio of 7 to 1.

Fund Does Not Receive Information on Postaward Activity

The Fund does not require awardees to notify the Fund of material changes in their investments after awards have been made. Therefore, it does not know how long investments made under the program remain in place. We found, for example, that a CDFI in which one of our case study banks had invested was dissolved several months after the bank received a BEA award. The CDFI later repaid a portion of the bank’s total investment. Because the Fund does not require banks to report their postaward activity, the Fund was not aware of this situation until we brought it to the attention of Fund officials. After hearing of the situation, a Fund official contacted the awardee and learned that the awardee plans to reinvest the funds in another CDFI. Even though this case has been resolved, Fund officials do not have a mechanism for determining whether investments made under the program remain in place.

The CDFI statute does not require awardees to reinvest their awards in community development; however, awardees have reported to the Fund, and we found through our case studies, that many of them are reinvesting at least a portion of their awards in community development. Reinvestment in community development is consistent with the goals of the BEA program.

While the Fund initially established reporting requirements for the 1996 awardees designed to assess the impact of their investments in CDFIs and distressed communities, it discontinued these requirements in 1997 when it found that the accomplishments reported by awardees could not be linked to outcomes in their communities. As a result, the Fund has no system in place for determining the program’s impact. As previously noted, accomplishments in community development are difficult to isolate and measure. For example, the effects of investment in community development may not be readily distinguishable from other influences and may not be observable for many years. Nevertheless, the banks we visited are using a variety of measures to assess the effects of their investments, some of which track accomplishments. Such measures include loan repayment rates and reports on the occupancy rates and financial performance of housing projects financed by the banks. However, the awardees are no longer required to report this information to the Fund.

Opportunities Exist for Improving the Fund’s Strategic Plan

The CDFI Fund has more work to do before its strategic plan can fulfill the requirements of the Results Act. Though the plan covers the six basic elements required by the Results Act, these elements are generally not as specific, clear, and well linked as the act prescribes. However, the Fund is not unique in struggling to develop its strategic plan. We have found that federal agencies generally require sustained effort to develop the dynamic strategic planning processes envisioned by the Results Act. Difficulties that the Fund has encountered—in setting clear and specific strategic and performance goals, coordinating cross-cutting programs, and ensuring the capacity to gather and use performance and cost data—have faced many other federal agencies.

Under the Results Act, an agency’s strategic plan must contain (1) a comprehensive mission statement; (2) agencywide strategic goals and objectives for all major functions and operations; (3) strategies, skill, and technologies and the various resources needed to achieve the goals and objectives; (4) a relationship between the strategic goals and objectives and the annual performance goals; (5) an identification of key factors, external to the agency and beyond its control, that could significantly affect the achievement of the strategic goals and objectives; and (6) a description of how program evaluations were used to establish or revise strategic goals and objectives and a schedule for future program evaluations. The Office of Management and Budget (OMB) has provided agencies with additional guidance on developing their strategic plans.

Mission

In its strategic plan, the Fund states that its mission is “to promote economic revitalization and community development through investment in and assistance to community development financial institutions (CDFIs) and through encouraging insured depository institutions to increase lending, financial services and technical assistance within distressed communities and to invest in CDFIs.” Overall, the Fund’s mission statement generally meets the requirements established in the Results Act by explicitly referring to the Fund’s statutory objectives and indicating how these objectives are to be achieved through two core programs.

Strategic Goals and Objectives

Each agency’s strategic plan is to set out strategic goals and objectives that delineate the agency’s approach to carrying out its mission. The Fund’s strategic plan contains 5 goals and 13 objectives, with each objective clearly related to a specific goal. However, OMB’s guidance suggests that strategic goals and objectives be stated in a manner that allows a future assessment to determine whether they were or are being achieved. Because none of the 5 goals (e.g. to strengthen and expand the national network of CDFIs) and 13 objectives (e.g. increase the number of organizations in training programs) in the strategic plan include baseline dates and values, deadlines, and targets, the Fund’s goals and objectives do not meet this criterion.

Strategies to Achieve Goals and Objectives

The act also requires that an agency’s strategic plan describe how the agency’s goals and objectives are to be achieved. OMB’s guidance suggests that this description address the skills and technologies, as well as the human, capital, information, and other resources, needed to achieve strategic goals and objectives. The Fund’s plan shows mixed results in meeting these requirements. On the positive side, it clearly lists strategies for accomplishing each goal and objective—establishing better linkages than the strategic plans of agencies that simply listed objectives and strategies in groups. On the other hand, the strategies themselves consist entirely of one-line statements. Because they generally lack detail, most are too vague or general to permit an assessment of whether their accomplishment will help achieve the plan’s strategic goals and objectives. For example, it is unclear how the strategy of “emphasizing high quality standards in implementing the CDFI program” will specifically address the objective of “strengthening and expanding the national network of CDFIs.”

Relationship Between Strategic and Annual Performance Goals

The Fund’s strategic plan lists 22 performance goals, which are clearly linked to specific strategic goals. However, the performance goals, like the Fund’s strategic goals and objectives, generally lack sufficient specificity, as well as baseline and end values. These details would make the performance goals more tangible and measurable. For example, one performance goal is to “increase the number of applicants in the BEA program.” This goal would be more useful if it specified the baseline number of applicants and projected an increase over a specified period of time. Also, some performance goals are stated more as strategies than as desired results. For example, it is not readily apparent how the performance goal of proposing legislative improvements to the BEA program will support the related strategic goal of encouraging investments in CDFIs by insured depository institutions.

Key External Factors

The Fund’s strategic plan only partially meets the requirement of the Results Act and of OMB’s guidance that it describe key factors external to the Fund and beyond its control that could significantly affect the achievement of its objectives. While the plan briefly discusses external factors that could materially affect the Fund’s performance, such as “national and regional economic trends,” these factors are not linked to specific strategic goals or objectives.

Program Evaluations

The Results Act defines program evaluations as assessments, through objective measurement and objective analysis, of the manner and extent to which federal programs achieve intended objectives. Although the Fund’s plan does discuss various evaluation options, it does not discuss the role of program evaluations in either setting or measuring progress against all strategic goals. Also, the list of evaluation options does not describe the general scope or methodology for the evaluations, identify the key issues to be addressed, or indicate when the evaluations will occur.

Other Matters

Our review of the Fund’s strategic plan also identified other areas that could be improved. For instance, OMB’s guidance on the Results Act directs that federal programs contributing to the same or similar outcomes should be coordinated to ensure that their goals are consistent and their efforts mutually reinforcing. The Fund’s strategic plan does not explicitly address the relationship of the Fund’s activities to similar activities in other agencies or indicate whether or how the Fund coordinated with other agencies in developing its strategic plan. Also, the capacity of the Fund to provide reliable information on the achievement of its strategic objectives at this point is somewhat unclear. Specifically, the Fund has not developed its strategic plan sufficiently to identify the types and the sources of data needed to evaluate its progress in achieving its strategic objectives. Moreover, according to a study prepared by KPMG Peat Marwick, the Fund has yet to set up a formal system, including procedures, to evaluate, continuously monitor, and improve the effectiveness of the management controls associated with the Fund’s programs.

In closing, Mr. Chairman, our preliminary review has identified several opportunities for the Fund to improve the effectiveness of the CDFI and BEA programs and of its strategic planning effort. In our view, these opportunities exist, in part, because the Fund is new and is experiencing the typical growing pains associated with setting up an agency—particularly one that has the relatively complex and long-term mission of promoting economic revitalization and community development in low-income communities. In addition, staffing limitations have delayed the development of monitoring and evaluations systems. Recently, however, the Fund has hired several senior staff—including a director; two deputy directors, one of whom also serves as the chief financial officer; an awards manager; a financial manager; and program managers—and is reportedly close to hiring an evaluations director. While it is too early to assess the impact of filling these positions, the new managers have initiated actions to improve the programs and the strategic plan. Our report may include any recommendations or options we may have to further improve the operations of the CDFI Fund.

Agency Comments

We provided a copy of a draft of this testimony to the Fund for its review and comment. The Fund generally agreed with the facts presented and offered several clarifying comments, which we incorporated.

We performed this review from September 1997 through May 1998 in accordance with generally accepted government auditing standards.

Mr. Chairman, this concludes our testimony. We would be pleased to answer any questions that you or Members of the Committee may have at this time.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.