The United States began providing limited assistance to the Soviet Union in December 1990 to support the reform effort and then increased assistance after the Soviet Union dissolved in December 1991. In October 1992, the Freedom for Russia and Emerging Eurasian Democracies and Open Markets Support Act of 1992 (P.L. 102-511), commonly known as the Freedom Support Act, was enacted. It further increased assistance to the former Soviet Union and established a multiagency approach for providing assistance. It also called for the designation of a coordinator within the Department of State whose responsibilities would include designing an assistance and economic strategy and ensuring program and policy coordination among federal agencies in carrying out the act’s policies. The Freedom Support Act sets forth the broad policy outline for helping former Soviet Union countries implement both political and economic reforms. It also authorized a bilateral assistance program that is being implemented primarily by USAID. In January 1994, the State Department Coordinator approved the first overall U.S. assistance strategy for the former Soviet Union, and in May 1994, the Coordinator approved the strategy specifically for Russia. This strategy has three core objectives: (1) help the transition to a market economy, (2) support the transition to a democratic political system, and (3) ease the human cost associated with the transition. As of December 1994, USAID had obligated $1.4 billion and spent $539 million for programs and projects in Russia since fiscal year 1990. (See app. I.) USAID’s assistance to Russia has focused on 13 sectors, such as health care and housing, that support the three U.S. objectives. Hundreds of U.S. contractors and grantees are responsible for implementing individual projects in the 13 sectors. The 10 projects we reviewed showed mixed results in meeting their objectives. Two projects—coal industry restructuring and housing sector reform—met or exceeded their objectives. Five projects—voucher privatization, officer resettlement, small business development, district heating, and agribusiness partnerships—met some but not other objectives. Three projects—health care, commercial real estate, and environmental policy—met few or none of their objectives. Three of the 10 projects we examined were contributing significantly to systemic reform—that is, they were making fundamental structural changes. These projects were effecting change because they had sustainability—benefits that extend beyond the project’s life span—built into their design and they focused on issues on a national or regional scale. The housing sector project helped Russian ministries and agencies implement 38 laws, regulations, and decrees to reform housing policies and practices. The Urban Institute, which implemented the project, also completed a series of pilot projects related to housing maintenance, mortgage lending, rent reform, and property rights. Many of the activities affect the entire country or could be replicated in additional cities. The project helped create new institutions, strengthened existing ones, and distributed procedural guides and manuals to local governments as a way to sustain the reforms. The contract for implementing the voucher privatization project called for Deloitte & Touche to establish 35 voucher clearing centers in cities throughout Russia. This project encountered some difficulty in meeting its initial time frames and establishing all the centers, but overall the project was successful. The active centers handled 70 million voucher transactions as part of Russia’s unprecedented privatization program, and over half the centers participate in ongoing capital market activities. Partners in Economic Reform (PIER), which implemented the coal restructuring project, has facilitated movement toward the transformation of the entire coal sector. PIER helped build a consensus for reform among the Russian government, mine labor unions, and mine management. It was also instrumental in facilitating a World Bank review that could lead to a restructuring loan. To sustain the project’s contribution to systemic reform, PIER helped establish long-term business relations between the Russian and U.S. coal industries, formed a consortium of U.S. coal-related business, and is involved in social safety net and new job-creation activities. Finally, PIER helped facilitate the sale of U.S. equipment in Russia. Seven projects we examined did not contribute significantly to systemic reform because they either did not meet their objectives, were narrowly focused, or lacked sustainability. The University of Alaska sought to help develop small businesses by establishing American Russian Centers in four cities across the Russian Far East. The centers’ purpose was to help train entrepreneurs, help form new businesses, and build lasting business ties between the region and the United States. To become self-supporting after USAID stopped funding, the centers planned to develop partnerships with counterpart institutions. However, the centers have so far been unable to attract alternative funding. CH2M Hill International Services, Inc., signed a contract in September 1993 for an environmental policy development and technology project. The contractor had difficulty filling critical staff positions in Russia and providing required work plans for the activities. Of the work plans due in November 1994, one was approved in May 1995, while the other was still being revised as of June 1995. The project to provide health care financing training in the United States to Russian health professionals was implemented by Partners for International Education and Training (PIET), several training institutions, and USAID. Although PIET and the institutions provided the training as required, the Russian participants did not have the authority, expertise, or resources to make systemic changes. In addition, changes in Russia’s health reform plans have made the training irrelevant. A commercial real estate project, implemented by International Business & Technical Consultants, Inc. (IBTCI), was intended to create a standard approach for increasing the availability of commercial real estate in six Russian cities. The project design called for a pilot project/roll-out concept, but IBTCI did not roll out the pilot in any of the cities and used a different approach in each city. Also, Russian officials said the project had little or no effect on the availability of commercial real estate in their cities. The district heating project, which USAID recommended we review, was implemented by RCG/Haggler Bailly and met its objectives primarily by conducting energy audits and training as well as providing energy efficiency equipment to two Russian cities. However, as of February 1995, some of the equipment in one of the cities had not been installed. Russian officials said the equipment may never be installed because Russian authorities never certified it. USAID had not monitored the use of the equipment or followed up on the impact of the studies produced for the project. Consequently, we found no indication that the project contributed to systemic reform in the energy efficiency area. The agribusiness partnerships project, implemented by Tri Valley Growers, helped two U.S. companies establish joint ventures in two Russian cities. Although the involvement of U.S. companies increased the probability that the business ventures would be sustained, the limited scope of the partnerships makes it unlikely that they will have a significant effect on reforming Russia’s agricultural sector. USAID has discontinued the entire agribusiness partnerships project in Russia. The Russian officer resettlement pilot project was not intended to be sustainable after its completion, but instead was motivated by the United States’ desire to encourage the withdrawal of Russian troops from the Baltic countries. The $6-million pilot project objective was to construct 450 housing units to resettle demobilized officers by July 1994. As of February 1995, 422 units were either occupied or available for use, so in that sense the project was successful. Successful projects (1) had strong support and involvement at all levels of the Russian government, (2) had a long-term physical presence by U.S. contractors in Russia, and (3) were designed to achieve maximum results by supporting Russian initiatives, having a broad scope, and including elements that made them sustainable. A critical element to a project’s success was the degree to which Russian officials were committed to reform in the particular sector. Russians at both the federal and local levels demonstrated a strong commitment to the projects that were contributing to systemic reform. The Russian government also provided financial or in-kind support, and Russian nationals held leadership roles in the projects. For example, PIER’s approach to implementing the coal project included working with officials in the Ministry of Fuel and Energy, Fund of Social Guarantees, and the federal coal company; academic institutions; oblast’ and city officials in the two targeted regions; local mine management; and representatives of two labor unions. Russian nationals served as codirectors, and PIER staff received free apartments and office space. To accomplish the housing sector reform project, the Urban Institute worked closely with officials in the Ministries of Finance and Economy and the State Committee on Architecture and Construction at the federal level, the Moscow city government, various maintenance firms, banks, and grass-roots condominium associations. Although office space in Moscow is expensive and scarce, the Institute received free office space. In addition, Russian nationals played a key role on the Institute’s staff. In contrast, many less successful projects lacked the buy-in of Russians at either the local or federal level and had little Russian involvement or contribution. For example, the State Committee of the Russian Federation for the Management of State Property (GKI), Russia’s federal agency overseeing the privatization effort, was instrumental in designing the voucher clearing and commercial real estate projects. However, in some cities, local officials were not involved in designing the projects and had little interest in them; as a result, these projects were not fully successful. The officer resettlement project established housing in several cities, but not in Novosibirsk, where city officials reneged on a previous administration’s commitment to provide needed infrastructure support. Because officials at the federal and oblast’ levels were not involved in the initial agreements, they had no authority to require the new city administration to fulfill the contract, nor were they willing to provide additional funding for the project. The district heating project was not completed in Yekaterinburg because local officials did not allow monitoring equipment to be installed. They said the proper Russian authorities had not certified the equipment. The successful projects usually had long-term advisers living in Russia, which enabled the advisers to build trust, learn about local conditions and plan accordingly, monitor progress closely, and correct problems as they occurred. In addition, successful projects involved contractors that had appropriate experience to carry out the project. For example, the Urban Institute has had two long-term advisers living in Moscow since 1992 who maintained close contact with Russians involved in housing reforms. PIER’s project director had lived in Moscow for 3 years. Other members of its American staff had lived in Kemerovo and Vorkuta, the key cities of the major coal mining oblasts, since 1993 and 1994, respectively. The two field staff have years of experience as coal mine engineers. Russian officials at all levels (1) praised PIER’s staff; (2) described PIER’s assistance as timely, well-targeted, and beneficial; and (3) wanted the project to continue and expand. Contractors implementing many of the less successful projects did not have staff living in the Russian cities being assisted. For example, neither IBTCI nor RCG/Haggler Bailly had permanently assigned American staff in the cities being assisted. IBTCI’s consultants would fly in, make rapid diagnoses, deal with problems quickly, and then leave. Many U.S. officials, Russians, and contractors said that relying on “fly-through” consultants rather than permanent staff was an ineffective approach. Successful projects—the housing reform, voucher privatization, and coal industry restructuring—were designed to be sustainable, have a widespread effect, and support existing initiatives. Each project supported ongoing Russian efforts at widespread reform, considered local conditions, and contained elements to sustain the effects of the project beyond its life span. In contrast, some projects were not designed to maximize their potential impact. For example, the project design required RCG/Haggler Bailly to provide energy efficiency equipment and audits but did not include methods to replicate the project in other cities, or extend monitoring efforts to determine how the equipment or studies were used. The USAID Inspector General reported that other projects did not include any follow-up steps to ensure that the assistance provided was used. In addition, projects focusing on health care training and commercial real estate leasing did not consider local needs and conditions and thus had limited impact. Several projects did not adequately identify outcomes or measurable results. For example, the Tri Valley Growers’ contract with USAID did not stipulate how many agribusiness partnerships were to be established. The design of the coal project also did not adequately identify outcomes or measurable results, but PIER developed an effective project nonetheless. The USAID Inspector General found similar problems when reviewing many projects in the region. It is widely acknowledged that the Russian people themselves will determine the ultimate success or failure of political and economic reforms. Without their involvement and commitment to change, outside assistance will have a limited effect. For example, the support and involvement of Russian federal agencies, such as GKI in the privatization effort and the ministries related to housing, ensure that projects in those sectors are likely to have a wide and sustained effect. The coal project’s impact depends on Russia’s commitment to restructure the coal industry. In several sectors, a Russian commitment to reform remains elusive. Powerful factions in the Russian legislative branch strongly oppose land reform, and the Ministry of Health has not demonstrated a commitment to health care reform. This lack of commitment raises concerns that projects in the agriculture and health sectors will not have widespread benefits. USAID is now working with the Ministry of Environmental Protection and Natural Resources, but the level of support from other important federal ministries, including the Ministry of Finance, is still questionable. Other domestic conditions will also influence a project’s success. Russia’s commitment to breaking up monopolies and its ability to attract capital for modernizing outdated equipment, restructuring existing state enterprises, and starting new businesses will affect the pace and scope of Russia’s transformation to a market economy. Moreover, projects such as introducing mortgage lending will depend on macroeconomic policy and land reforms. Russia is counting on foreign capital to help move the transition process forward, but such factors as the unstable economic situation, a poor and uncertain tax structure, an undeveloped financial market infrastructure, and an increased crime rate make foreign investors hesitant to invest. USAID responded quickly to assist Russia in undertaking its political and economic reforms, as called for in the Freedom Support Act. However, to respond quickly, USAID made certain exceptions to its normal procedures and processes. Although USAID provided a quick and flexible response to a fluid, unpredictable situation, we identified several management problems in addition to design problems that occurred, in part, because of the quick response. USAID officials agreed that management problems occurred, but they said the risks associated with not responding quickly were high. The large size of USAID’s program, the vast geographic area receiving assistance, and staff limitations have prevented adequate monitoring in some cases. We found that USAID officials were unaware of positive and negative aspects of the projects implemented by IBTCI, RCG/Haggler Bailly, and PIER. USAID officials had not visited some projects, and USAID did not have representatives located outside Moscow. USAID expected its Russian staff to conduct field monitoring, but the Russian nationals lacked the necessary training. USAID officials said they considered but rejected the idea of establishing field offices outside Moscow. Without adequate staff, USAID relied mainly on contractors’ written and oral reports to monitor the projects, but some contractors did not report all problems. The USAID Inspector General also found shortcomings in the reporting process: contractors were not required to report on their progress toward specific objectives or indicators. Moreover, USAID did not enforce some of its reporting requirements. For example, Deloitte & Touche did not provide the required lists of equipment purchased with USAID funds and brought into the country, and USAID did not enforce the requirement. Although the State Department allowed USAID/Moscow to increase U.S. direct-hire personnel and personal services contractors from 27 in fiscal year 1993 to 66 in fiscal year 1995, USAID officials said even more staff were needed to adequately monitor the program. However, State would not allow the USAID mission to grow further because, among other reasons, the USAID assistance program is scheduled to end by the end of the decade. In some cases, USAID had not determined the relative success or failure of projects so that it could apply lessons learned to other efforts. It did not conduct the required periodic assessments/evaluations of the coal and agribusiness projects. The omnibus contracts do not require an evaluation of the individual tasks, but instead evaluations are to be done at the end of the contracts, according to USAID officials. The omnibus contracts for USAID’s private sector initiatives alone have obligated approximately $200 million and are not scheduled to terminate until 1996, too late to apply lessons learned. In addition, an evaluation that was conducted was not accurate. A contractor evaluated the district heating project in June 1993 and gave it high marks. Our 1995 review of the project found major shortcomings, such as equipment still in boxes after being delivered in 1993, even though the evaluation report said the equipment had been installed and was being used. The USAID Inspector General also found that evaluations had not been conducted and that the quality and impact of some project evaluations were questionable. The devolution of management and monitoring responsibility from USAID’s Washington office to a rapidly growing Moscow office has not been smooth, and several problems have developed as a result. First, as USAID’s Moscow office assumed more management responsibility, contractors had to begin dealing with another layer of management review. This caused delays and confusion among some contractors. Second, there were tensions between the Washington and Moscow offices because of differences regarding program implementation. For example, the offices disagreed about which reformers and Russian government agencies to work with. Third, the USAID/Moscow office lacked some essential documents to enable officials to carry out their duties. We found that key contract and financial documents were not available in Moscow, a problem also reported by the USAID Inspector General. The State Department Coordinator opposed giving greater authority to USAID/Moscow because he believed USAID/Washington needed to maintain a more prominent role. He said that because assistance to Russia is an important foreign policy issue, key decisions should not be delegated to the field. State and USAID/Washington officials said they needed quick access to important project data for reporting purposes, but quick access to data could not be ensured when projects were managed by the USAID/Moscow office. USAID has not yet developed a good management information system for its Russia program. The USAID Inspector General reported that USAID lacked an information system with baseline data, targets, time frames, and quantifiable indicators by which to measure program progress and results. USAID’s Bureau for Europe and the New Independent States was exempted from a new agencywide management system because the program was intended to be short term and regional rather than long term and country-specific. USAID officials said the pressure to provide assistance quickly meant forgoing the traditional project design process, which included developing progress indicators. Part of USAID’s assistance strategies was to focus on areas where reformers were willing to make changes. USAID believed this would help create a synergy that could stimulate the overall impact of the projects. Some contractors were not aware of each others’ activities. USAID’s management information system did not list contractors by region, and USAID sometimes did not tell new contractors about other contractors’ activities. In some cities, contractors contacted each other on their own and started coordinating their efforts. However, this was being done on an ad hoc basis. In Vladivostok and Yekaterinburg, U.S. Consuls General facilitated contractor coordination. The USAID Inspector General found that many projects with similar goals were not linked to one another. Poor coordination reduced the opportunity to achieve synergy and targeted impact and gave some Russians the impression U.S. assistance was fragmented and uncoordinated. We recommend that the USAID Administrator focus assistance efforts on projects that (1) will contribute to systemic reforms; (2) are designed to be sustainable; (3) are supported by all levels of Russian government; and (4) whenever possible, use American contractors with an in-country presence. In commenting on a draft of this report, USAID said the three projects that we had deemed to have not met their primary objectives did produce some positive benefits or it was too early to tell the impact the projects would have. USAID also said it was aware of the problems that have occurred and has taken steps to correct them or terminate activities that could not be fixed. USAID pointed to a new computerized monitoring system that is expected to produce its first report in November 1995. USAID agreed with our recommendation regarding the focus of its assistance projects and said it was taking steps to implement it. In addition, USAID said it was taking corrective action to address the management problems we identified. However, it stressed that its assistance has had a positive impact and occurred in a difficult operating environment. USAID indicated that it had made progress in setting up its own monitoring, reporting, and evaluation system. It should be pointed out, however, that in November 1994, the USAID Inspector General reported that the system was still far from able to measure program results. USAID said that our report would have provided a more balanced and accurate view of the systemic impact and sustainability of a project’s activities if we had considered the activity in the context of the whole program. USAID stated that, in nearly every case, the individual projects we focused on were part of a larger project or program that would have substantial impact on reforming Russia’s economy. USAID is correct that the projects we examined were usually one component of a larger sector program; however, USAID is incorrect in its assertion that we evaluated projects in isolation and without considering the context of the whole program. The overriding objective of USAID’s program in Russia is to contribute to reforming both the political and economic systems. This is also the objective of the assistance program for each sector and, with few exceptions, of each project that supports a sector program. Our examination focused on the individual building blocks that support sector programs and ultimately support the reform effort in Russia. In some cases, we found that the individual building blocks will not contribute to systemic reform in the sector or in Russia overall. Even though this does not mean that an entire program, of which a less-than-successful project is a part, will fail in its systemic reform objective, it does mean that an unsuccessful project is not contributing to a program’s success. We also disagree with USAID’s assertion that significant systemic reform has resulted from USAID activities in all sectors. For example, the agribusiness partnerships project, including components reviewed by the USAID Inspector General, comprises most of the USAID funding going to the sector but is not expected to contribute significantly to systemic reforms. Only a limited degree of systemic reform has occurred in other sectors as well, including the health care and the environmental sectors. We believe that a sector evaluation, although useful in its own right, would not have allowed us to draw conclusions about the role and contribution of individual projects. USAID provided other comments that we incorporated into the report where appropriate. The full text of USAID’s comments is reprinted in appendix IV. We judgmentally selected 10 individual projects from 6 sectors to review as case studies. We selected projects based on their geographic distribution, focusing on regions where several projects were concentrated. We also considered the level of obligations and expenditures; the type of assistance provided (e.g., training, technical assistance, and product delivery); and the type of contracting vehicle (e.g., cooperative agreements, grants, and contracts). We generally did not review projects examined by the USAID Inspector General, although we analyzed its work to assess whether common themes emerged. (See app. II for a list of the 10 projects we studied and USAID Inspector General reports we reviewed.) We analyzed USAID and project documents and interviewed USAID and other U.S. government officials, U.S. contractors, Russian counterparts, and beneficiaries. We visited project sites in Western Russia, Siberia, and the Russian Far East in November 1994 and February 1995. Appendix III provides a detailed analysis of the 10 projects in our case study. We conducted our work from September 1994 to April 1995 in accordance with generally accepted government auditing standards. As agreed with your offices, unless you publicly announce this report’s contents earlier, we plan no further distribution of this report until 30 days after its issue date. At that time, we will send copies to the Secretary of State, the Administrator of USAID, and other interested congressional committees. Copies will also be made available to others on request. Please contact me at (202) 512-4128 if you or your staff have any questions concerning this report. Major contributors to this report are listed in appendix V. In his January 1995 annual report, the State Department Coordinator reported about $2.9 billion in obligations and $1.8 billion in expenditures for Russia through December 1994. (See table I.1.) Between fiscal years 1990 and 1994, the U.S. Agency for International Development (USAID) allocated assistance to the New Independent States (NIS) as a region. During that time, most projects spanned different countries and USAID did not track how much money was obligated or expended by country. Thus, USAID country attributions are estimates and should be treated as such. In fiscal year 1995, USAID began keeping country accounts. We reviewed 10 projects in depth as part of our review. In addition, we reviewed various reports that USAID’s Inspector General has issued on management issues and projects in Central and East Europe and the former Soviet Union. The 10 projects and the USAID Inspector General reports we reviewed are listed in table II.1. Coal project/Partners in Economic Reform Small business development/American Russian Center, University of Alaska Environmental policy & technology/ CH2M Hill International Services Health care training/Partners for International Education and Training Commercial real estate/International Business & Technical Consultants, Inc. The following are the USAID Inspector General reports GAO reviewed. Audit of the Bureau for Europe’s Technical Assistance Contracts (Report No. 8-180-93-05, June 30, 1993). Audit of the ENI Bureau’s Monitoring, Reporting and Evaluation System (Report No. 8-000-95-002, Nov. 28, 1994). Audit of ENI’s Strategy for Managing Its Privatization and Restructuring Activities in Russia (Report No. 8-118-95-009, Mar. 17, 1995). Audit of Selected Privatization and Restructuring Activities in Russia (Report No. 8-118-95-007, Mar. 10, 1995). Audit of the Reestablishment of Vaccine Production Activity Under the New Independent States Health Care Improvement Project No. 110-0004 (Report No. 8-110-94-004, Feb. 25, 1994). Audit of the Medical Partnerships in Russia and Health Information Clearing House Activities Under the New Independent States Health Care Improvement Project (Report No. 8-110-94-005, Feb. 28, 1994). Audit of the Distribution of Emergency Medical Supplies to the New Independent States Under USAID Cooperative Agreement With the People-to-People Health Foundation “Project Hope” (Report No. 8-110-94-006, Mar. 17, 1994). Audit of the Vulnerable Groups Assistance Program in Russia Under Project No. 8-110-0001 (Report No. 8-110-93-08, Sept. 24, 1993). Audit of Activities to Improve Crop Storage Systems in the New Independent States (Report No. 8-110-94-014, Aug. 31, 1994). Audit of the ENI Bureau’s Cooperative Agreement With World Learning, Inc., for Support to Non-Governmental Organizations in the New Independent States of the Former Soviet Union (Report No. 8-110-95-008, Mar. 10, 1995). Audit of the Department of Commerce’s Special American Business Internship Training Program in the New Independent States (Report No. 8-110-93-10, Sept. 24, 1993). Audit of the Department of Commerce’s Consortia of American Businesses in the New Independent States Program (Report No. 8-110-93-11, Sept. 24, 1993). Audit of the Nuclear Regulatory Commission’s Technical Assistance Activities in Russia (Report No. 8-118-94-012, June 28, 1994). Audit of the Department of Energy’s Nuclear Safety Technical Assistance Activities in Russia and Ukraine (Report No. 8-110-95-001, Oct. 7, 1994). The following provides a detailed analysis of each project that we reviewed. We selected one project in each of the following areas: (1) housing policy reform, (2) voucher privatization, (3) coal, (4) small business development, (5) environmental policy and technology, (6) heating, (7) health care, (8) commercial real estate, (9) agribusiness partnerships, and (10) officer resettlement. Each summary contains information on the problems in the sector, USAID’s project objectives for the selected contract, and the project approach used by USAID or the contractor. We also provide our assessment of the contractor’s performance, the impact on systemic reform, and USAID’s management of the contract. The projects are presented on the basis of their capacity to contribute to systemic reform. The Urban Institute housing project was successful. It supported reforms already underway, used an experienced contractor with staff in country, installed local nationals in high-level positions, focused its efforts on both the federal and local levels, and contained elements that made it sustainable. Therefore, this project will likely have sustained benefits as legislation is implemented and new Russian institutions expand the pilot projects into other areas. Russia’s housing sector has been beset by housing shortages, production inefficiencies, maintenance problems, and deterioration. This situation occurred primarily because the state had a monopoly on housing. For example, it (1) used standardized apartment buildings constructed by state-owned companies, (2) controlled apartment construction and maintenance, (3) financed all state housing from state assets, (4) almost totally subsidized housing and maintenance, (5) guaranteed low-cost housing, and (6) distributed housing through waiting lists. In addition, because the Soviet government had not raised rents since 1928, rents covered less than 5 percent of the cost of operating the apartments in 1990. The problem was exacerbated when the Russian Federation government stopped paying for maintenance cost of apartments and they fell into disrepair. In addition, the Federation devolved the housing assets and responsibilities to municipalities as a way of relieving itself of the burdens associated with managing the apartments. Russia initiated housing reforms in 1991 when it allowed its citizens to privatize their apartments at little to no cost. This action set the stage for establishing a private housing sector. USAID signed its first contract for housing sector reform with the Urban Institute in September 1992 for $5.8 million. This 2-year contract required the Institute to provide draft legislative and financial advisers to help Russia develop market-oriented housing programs and legislation. Other Institute advisers were expected to conduct pilot projects on (1) rent reforms and housing allowances for the poor, (2) privatized housing maintenance, (3) condominium associations, and (4) mortgage lending. It was also expected to provide targeted training to those implementing reforms and develop local institutions to sustain and expand the reforms. Specific objectives and milestones were incorporated into the project design. A USAID team that included Institute representatives met with their Russians counterparts in early 1992 to determine their reform priorities. From 12 to 15 meetings at both the national and municipal levels in Moscow were needed to clarify Russian reform priorities. To help focus Russian priorities, the team used a “menu” of reforms based on experience in housing reforms in Hungary and developing countries, and then focused on one or two priorities to demonstrate results quickly and build confidence. The Russian priorities were formalized through agreements signed in March 1992 with USAID, the City of Moscow, the State Committee on Architecture and Construction, the Ministry of Finance, and the Ministry of Economy. The team sought joint agreements with the three ministries agencies to (1) ensure it would not become captive to any one ministry, (2) ensure broad-based agreement on reform priorities, and (3) reduce governmental impediments to reform. The Russian counterparts showed their support for the project by providing the Institute with free office space, which is highly unusual due to the scarcity of office space in Moscow. The team’s strategy was to work at the national level to help draft legislation that would shape and codify reforms. In addition, it planned a series of local demonstration projects to determine the effectiveness of the designs in the Russian context. The team augmented these efforts by providing training in Russia and the United States. A key strategy was to take advantage of the Russian reforms already underway and try to create “win-win” situations for the government and its citizens. The Institute’s staffing policies were also important to its approach. It provided two long-term resident advisers, including the Program Director, who were located in Moscow. The Director said using advisers who were permanently located in Russia rather than “fly-through” consultants helped establish trust with their Russian counterparts and enabled them to respond immediately. The Institute also employed five Russian housing experts. Short-term U.S. advisers were used on an as-needed basis. The Director said that using local Russians in key positions was critical to establishing trust with the Institute’s Russian counterparts. The large Russian staff also was less expensive than U.S. consultants. The Institute achieved its objectives of helping to develop housing legislation. According to the Russian Federation Housing Director, the Institute’s assistance was critical in drafting the 38 laws, decrees, and regulations that have been implemented. These included laws and regulations on property rights, housing finance, rent reform, housing allowances for the poor, privatized maintenance, condominiums, and mortgage lending. The Institute is now the government’s principal housing adviser. The Institute also achieved its objectives of establishing pilot projects in four areas: rent reform and housing allowances for the poor, privatized maintenance, condominiums, and mortgage lending. The Institute helped the City of Moscow develop a program that would raise rents over a 5-year period until they covered all the costs of operating the apartments. To reduce resistance to rent increases, it tied maintenance improvements to the increases so citizens would see an immediate improvement in their housing conditions. In 1994, the Federation initiated a national 5-year program to increase tenant payments to cover the full operating costs. The Institute also helped the Federation structure a program in which the municipalities began providing housing allowances to the poor. The Institute helped introduce competitive private maintenance for municipal housing. It conducted training sessions, organized the competition to select private firms, and conducted a study tour to the United States so officials could see private maintenance activities. In March 1993, three private maintenance firms assumed management of 2,000 apartments in Moscow, and in October 1993, Moscow’s mayor extended the program to all areas of the city. By 1994, over 60,000 apartments were under privatized maintenance, far surpassing the project’s goal of 2,000. The Institute’s goal for the condominium pilot was to lay the legal and procedural groundwork by 1994. However, it surpassed this goal and helped to create 24 functioning condominium associations in Ryazan’. The regulations it helped develop were instrumental in registering the first condominium association in Moscow. To address mortgage-lending problems, the Institute developed mortgage-lending facilities at several banks; limited lending has begun. For example, the Institute helped Mosbusinessbank, Russia’s third largest commercial bank, to make home mortgage loans and provided assistance in all phases of operations, including legal documentation, underwriting, loan servicing, mortgage loan instrument development, and risk management. The Institute then expanded its work to eight other banks and provided the necessary materials to other banks to expand and sustain mortgage lending. However, hyperinflation has precluded lending to most Russians. The Institute’s critical assistance helped transform Russian priorities into workable legislation and pilot projects. Although the Russians are responsible for the pace of reforms, the Institute has helped effect systemic changes in Russia’s housing sector. It helped pass far-ranging laws that have codified reforms. The program to raise rents and provide subsidies for the poor, which is being implemented across the country, is a fundamental change for the government and its citizens. The project has a strong sustainability component. Over the next several years, it plans to institutionalize the reforms by expanding the number of demonstration projects and developing private maintenance organizations, condominium associations, and mortgage banks. In addition, it created procedures manuals, explanatory guides, and other necessary documentation on implementing rent increases, beginning privatized maintenance, creating condominium associations, and developing mortgage lending. The Institute has distributed more than 25,000 of these documents, mostly to local governments. The project has won high praise from USAID and Russian officials. The USAID Mission Director in Moscow called the project one of the most successful ones he had ever seen. A USAID official in Washington said that, for the money, no USAID project has had more macroeconomic impact. The Russian Federation Housing Director noted the Urban Institute’s tremendous influence on the government, and Russian citizens working in maintenance, condominium associations, and mortgage lending also praised the project. Despite the program’s progress, most Russians have yet to benefit from the reforms. This is because the reforms are relatively recent, are tremendously complex, and face opposition by antireformists; they are also being implemented in a country with no tradition of market-based decisions. Private land ownership rights are still generally uncertain, housing and construction mortgages are generally unavailable, additional laws and regulations are needed, and most apartment buildings are still maintained by state organizations. In addition, factors beyond the housing sector, such as macroeconomic and political instability, slow the transformation to a fully developed privatized housing sector. USAID successfully managed the contract. It determined the Russians’ reform priorities, incorporated these into its reform plan, and listed these in its contract. USAID selected a contractor with experience in both the sector and region and is effectively monitoring the reforms through regular contacts with the contractor and Russian counterparts. Both USAID/Washington and USAID/Moscow agreed on the housing strategy. USAID also had the contractor develop measurable goals in its annual work plan. USAID then measured the contractor’s progress by comparing its task orders to the deliverables. As part of Russia’s privatization effort, Deloitte & Touche established a national system of centers to process millions of vouchers that Russians received and used in the privatization process. Overall, the Deloitte & Touche voucher privatization project was successful, with a few exceptions. The project focused on national reforms, but some areas had lower Russian participation than expected. Deloitte & Touche kept USAID and the State Committee of the Russian Federation for the Management of State Property (GKI) informed of project progress but did not meet some of its reporting requirements. Deloitte & Touche met its amended objective of setting up 30 centers, but many were underused. Several factors contributed to the overall success of the project. The Russian GKI helped focus assistance efforts and identified problems when USAID had minimal field presence. Further, the omnibus contract system allowed the contractor to institute a rapid roll-out as well as adjust the scope of work when warranted. In addition, using existing Russian agencies and using staff and equipment for follow-on activities increased the project’s effects and sustainability. Because the state controlled Russian enterprises, which were generally large and monopolistic, the private sector was virtually nonexistent. The legal and regulatory framework to create the new system was not in place; few citizens had entrepreneurial experience or exposure to western management, accounting, and marketing concepts; and no capital market infrastructure existed. In August 1992, President Yeltsin announced plans to privatize Russia’s large and medium-size state-owned enterprises. Within weeks, distribution of privatization vouchers began, with each Russian citizen eligible to receive one voucher. The sale of the enterprises was expected to reduce the need for massive state subsidies, begin to reduce inefficiencies, and eventually lead to higher productivity and innovation as shareholders demand profits. Voucher privatization was the initial step in the overall privatization process and was used to transfer ownership from the state to private individuals. Unlike the approaches used in some Central European countries, Russia chose to privatize enterprises before restructuring them.The process is therefore not complete: restructuring must still take place before the enterprises can function in a market economy. This may be difficult because management and workers received a majority of shares and can resist taking the painful steps necessary for restructuring. The voucher-clearing centers allowed individual Russians and investment fund managers to more easily buy shares via electronic transfers in enterprises located in remote areas. Without the centers, people would have had to physically transport vouchers to other parts of the country. There was also a fear that regions would not let outsiders, including foreigners, buy shares in highly visible enterprises, thereby allowing insiders and local bureaucrats to control the process. The specific objectives of the project required Deloitte & Touche to establish 35 functioning centers in various Russian cities to verify, process, and cancel voucher receipts. The project was carried out under two separate contracts at a total cost of $4.1 million. The initial contract (as amended) required the contractor to establish 20 centers by the end of 1993, and a task order under the omnibus contract required 15 more before March 1994. This would give citizens enough time to process their vouchers before the privatization program ended in July 1994. USAID and GKI, the Russian agency overseeing the national privatization effort, hoped that many of the centers would develop into institutions, such as registrars and depositories, in the capital market infrastructure. The number of vouchers the centers were to process was not defined. USAID worked closely with GKI on project design, which called for Deloitte & Touche to develop 4 pilot centers and then establish 16 more after successfully setting up the pilot sites. To provide broader geographic coverage, USAID and GKI decided to extend the project and have Deloitte & Touche set up 15 more centers. Consultants from another USAID contractor, the Harvard Institute for International Development, worked with GKI to design and monitor the project. Deloitte & Touche established a permanent office in Moscow in June 1990 and opened a separate office for this project in early 1993. It worked closely with GKI in Moscow and GKI’s local offices in various Russian cities to identify appropriate cities for the centers and suitable partners. Deloitte & Touche then imported computer equipment, established accounting systems, and installed the software and telecommunications systems needed to facilitate voucher transactions. Teams of Deloitte & Touche staff then traveled to the cities to train center staff, install and test the equipment, and test the software and telecommunications systems. The contractor hired Russians to assist with these efforts and usually supplemented the work of Russian agencies, typically banks, already working in related fields. Under the first contract, Deloitte established all 20 centers before its deadline. Under the second contract, USAID and GKI reduced the number of centers from 15 to 10 and extended the deadline by 3-1/2 months because of implementation delays. The delays took place because of problems with equipment procurement and Russian government customs clearance; difficulties locating viable agencies to act as centers; and problems at the local level. For example, some centers collapsed when their leadership changed or chose not to participate on a national scale for local political reasons. Both parts of the project were completed under budget. Of the 30 centers Deloitte & Touche set up, only 23 were used, and many of these experienced relatively little activity. The lack of use was attributed to delays in setting up some centers; limited public awareness (centers were not responsible for advertising their services); limited local interest in voucher auctions in other areas; Russian reluctance to use electronic transfers; and a lack of compatibility between the project goals and individual center goals. Deloitte & Touche was responsive to GKI requests for project changes. The task order was revised once it became clear that all 15 centers would not be needed. In some cases, Deloitte & Touche went beyond the required tasks at GKI’s request. Deloitte & Touche generally kept USAID and GKI informed with monthly reports on progress and problems. However, some reports were not filed as required, and Deloitte & Touche did not provide an adequate inventory of the equipment it procured, which would have ensured the accountability and tracking as required by the company’s contracts. The project is considered a success although not cost-effective. A functioning national system was created in a short time, and according to GKI, it handled over 70 million vouchers, nearly half the vouchers processed in the program. People were able to buy shares in enterprises located in remote areas. GKI noted that over half of the centers have evolved into institutions that are now active in capital market activities, such as share registrars and depositories. Our visits to three centers verified that center staff and equipment are being used in follow-on activities. These centers intend to become self-financing on a fee-for-service basis when USAID assistance ends. The scope of voucher privatization in Russia was unprecedented in scale and speed. According to Russian and U.S. officials, USAID’s support of GKI and other Russian institutions involved in privatization activities was crucial to this phase of the program. The Russian Privatization Center estimated that 14,000 large and medium-sized enterprises were privatized by July 1994; they employed over 60 percent of the industrial workforce. Nevertheless, the overall effect of the privatization program on Russia has yet to be determined. Enterprise restructuring has only begun, monopolies still exist, and inadequate tax legislation makes foreigners reluctant to provide badly needed capital investment. USAID used an omnibus contract to plan and implement projects quickly. It allowed USAID to respond quickly to emerging needs through task orders that included specific objectives for narrowly focused, short-term projects. USAID officials said this gave them the flexibility to change directions quickly, move money into areas and projects making rapid progress in reform, and adjust projects to meet emerging needs. Omnibus contracts also allowed USAID to obligate a large amount of funding. Deloitte & Touche has an omnibus contract for $41.5 million, with subcontractors performing some of the work. However, the individual task orders lacked an evaluation requirement. USAID officials said an evaluation is planned only at the end of the omnibus contract, which ends June 30, 1996. We identified several USAID management problems. For example, USAID did not design the project with quantifiable indicators to measure progress. Although Deloitte & Touche set up 30 voucher clearing centers, the project design did not specify the amount of activity expected at each center or on a national scale. USAID/Moscow had limited information on the project, such as key documents on Deloitte & Touche’s initial contract or task order, and accurate financial data for the project. USAID officials said key documents had not been transferred to Moscow when management moved to Moscow from Washington. Also, the physical distances involved, the geographic distribution of project activities, and the lack of staff to visit the sites left USAID uninformed about some Deloitte & Touche activities. USAID officials said they relied heavily on GKI and Harvard consultants to help monitor the project. Finally, USAID did not require Deloitte & Touche to provide adequate inventory data on the $1.1-million worth of equipment purchased with USAID funds. Not having this data hindered USAID from using surplus equipment in other projects as planned. The coal project is achieving its primary objective of facilitating the restructuring of Russia’s coal industry and is opening the industry to American technology and companies. The Russian beneficiaries expressed appreciation for the assistance and found it useful. Due to the size and cost of the restructuring, however, the Russian government must complete the effort. If the World Bank approves a $500-million sector loan, this project will have played a key role in restructuring the coal sector. Although the project is meeting its objective, USAID did not provide adequate oversight, did not fully understand the beneficiaries’ needs or opinions about USAID assistance, and did not know the extent to which the project was meeting its goals. Coal is an important component of Russia’s economy. However, Russia’s coal sector suffers from declines in production and serious environmental and safety liabilities, in large part because of the centralized structures, subsidized pricing, Soviet-style management, and state allocation system. To solve these problems, the coal industry needs to be restructured. The process of restructuring is both a problem and a solution because it creates new challenges. The major areas that need to be addressed in restructuring Russia’s coal industry and transitioning from a centrally planned to a market economy include reducing the numbers of mines and miners as well as the amounts of coal produced and government subsidies. In addition, the coal monopoly must be broken up, mines must be privatized, and new relationships and agreements must be established between management and labor. Efforts to restructure the coal industry are complicated because the state-subsidized coal mines provide many social services and may be the only source of energy or employment in the areas where they are located. Coal industry restructuring will take a heavy toll on miners and their families as the industry streamlines its operations, mines are closed, and miners lose their jobs. These same miners, who could lose their jobs as a result of the Yeltsin reform program, were instrumental in bringing Yeltsin to power in 1991. The mining community in Russia is still considered a politically powerful force. President Yeltsin took a major step toward restructuring Russia’s coal industry in July 1993 when he freed coal prices. Since that time, the industry has made some progress. For example, approximately 72,000 of the 914,000 coal miners and others employed by the coal sector in 1992 left the industry between January 1993 and June 1994. In addition, coal production decreased by approximately 41 percent from 1988 to 1994. The government also reduced subsidies to the coal industry by approximately 20 percent in real terms (i.e., taking inflation into account) between 1993 and 1994. Finally, the Russian coal industry closed 2 of its approximately 273 mines, was in the process of closing 14 more mines in 1994, and is preparing to close 40 more in the future. As part of USAID’s broader effort to assist Russia’s energy sector, USAID signed a cooperative agreement with the Partners In Economic Reform (PIER), a private, nonprofit organization established to assist the coal industries in Russia, Ukraine, and Kazakstan. USAID signed the agreement with PIER for $6.9 million in June 1992 and has increased funding since then to $8 million. The project’s main objective is to facilitate the transformation of the centrally planned and controlled coal mining industry to an industry capable of operating in a market economy. The cooperative agreement did not specify any measurable goals or deliverables. In 1989, U.S. coal representatives visited some of the coal regions of the Soviet Union where miners were starting to form independent unions, and between 1989 and 1991, groups of independent miners met with leaders of the U.S. coal industry labor and management in the United States. In June 1991, a memorandum of understanding, pertaining to continued assistance, was signed by U.S. and Russian coal industry representatives. During 1991, circumstances changed drastically. Boris Yeltsin was elected President of Russia in June, the communists mounted a failed coup attempt in August, and the Soviet Union dissolved in December. These changes opened the door for a broad U.S. technical assistance program in Russia. As part of this effort, the State Department announced the coal project in a January 23, 1992, press release and signed a cooperative agreement on June 25, 1992. The coal project gained early acceptance because PIER targeted the project at a problem (i.e., coal industry restructuring) that the Russians had already identified and were struggling to address. In addition, PIER established good working relationships. For example, PIER established a coordinating office in Moscow and cooperation and development centers in the Russian cities of Kemerovo and Vorkuta. An American director heads the coordination office, and an American director and a Russian codirector head the cooperation and development centers. In addition, because the American staff lived in Russia, they were able to develop and maintain long-term relationships with the Russian government, coal industry management, and labor unions. The Russians further demonstrated their support by donating rare office space for the coordination offices and cooperation and development centers and donating apartments for the American directors in Kemerovo and Vorkuta. PIER staff worked closely with representatives from the Russian government, coal industry management, and labor to (1) reduce the number of mines and miners, (2) develop new sources of employment in coal-producing regions to absorb displaced laborers, and (3) develop a social safety net for those miners left unemployed during and after the transition. PIER has cultivated cooperative efforts between government, management, and labor to address problems associated with coal industry restructuring. In addition to these efforts, PIER staff has helped build commercial links between the Russian and American coal and coal-related industries. PIER made progress in facilitating the restructuring of Russia’s coal industry and opening the Russian market to U.S. mining technology. Specifically, PIER worked closely with the World Bank to evaluate Russia’s coal industry and develop a restructuring plan; conducted detailed studies of employment, unemployment, and social programs; government subsidies; labor demand; a social safety net, job creation, and mine planning; and enterprise debt in the Russian coal industry; established a coal-bed methane recovery center; mediated discussions between U.S. and Russian officials on equipment certification in an effort to open the Russian market to U.S. high-tech safety equipment; established a program to facilitate U.S. private sector investment; hosted approximately 150 representatives of the Russian government and coal industry in the United States where they participated in meetings and negotiations with World Bank officials, training seminars, and meetings with U.S. coal industry representatives; provided training material and conducted seminars in Russia concerning mine safety, labor-management relations, mining and mine management in a market economy, and small business development in Russia; implemented a transition assistance program focused on developing a viable social safety net and creating new jobs; and provided $200,000 worth of U.S.-manufactured mine health and safety equipment to Russian miners. The Russian beneficiaries (i.e., government, labor, and management) we contacted in Russia stated that the coal project was well-targeted, timely, and beneficial. In addition, they all wanted the project to be continued and expanded. PIER has made several contributions to systemic reform. One of the clearest contributions is its work in facilitating a $500-million World Bank loan. By providing U.S. coal industry experts, PIER facilitated the World Bank’s work in Russia; contributed extensive analysis of the coal industry’s problems; built consensus among Russian government, management, and labor representatives; and brought representative Russian delegations to the United States to negotiate with the World Bank. The World Bank acknowledged PIER in its 1994 report for contributing to the Bank’s work in Russia. PIER helped establish relationships between Russian and U.S. coal mining and equipment-manufacturing firms. According to the beneficiaries, these relationships will help Russia attract capital investors and gain greater access to U.S. expertise and technology so that it can begin to produce coal efficiently and compete in a market economy. PIER also facilitated the sale of millions of dollars of non-USAID-funded U.S. mining equipment in Russia. PIER formed a consortium of U.S. industry representatives to help create a viable private coal industrial sector. The consortium is to assist coal managers and technicians in operating in a market economy, identify ways that private U.S. firms can participate in restructuring the coal industry, establish NIS-U.S. joint ventures, and promote the consortium’s services so it can become self-sustaining. Finally, PIER worked with Russia’s Fund for Social Guarantees to implement a transition assistance program focused on developing a viable social safety net and creating new jobs. PIER also brought in U.S. experts to provide small business education to miners and helped mining communities develop business proposals that can be presented to the Russian-American Enterprise Fund, Russian banks, and other sources for eventual financing. USAID started to implement the coal project before it had established a USAID mission in Moscow; consequently, the USAID project officer in Washington managed the project. Since the coal project was established through a cooperative agreement, without quantifiable indicators, PIER designed and implemented the project without direct oversight and control by USAID. PIER provided the required monthly program performance reports, annual work plans, and annual progress reports to USAID, which then reviewed them. PIER’s staff communicated regularly with USAID and felt they had a good reciprocal working relationship. Despite some success with the project, USAID did not meet its monitoring and evaluation requirements. Although the USAID staff should have regularly monitored the project, they visited the Russian project sites three times between June 1992 and February 1995. Two of the visits occurred after we began our review. USAID officials said a lack of staff prohibited more frequent visits. In addition, USAID did not conduct the annual assessments or midterm evaluation as required and thus lacked an objective basis for evaluating PIER’s activities and accomplishments. This, coupled with a lack of quantifiable indicators, hindered USAID’s ability to independently determine the project’s impact on coal sector restructuring. The University of Alaska met most of its project objectives while encouraging systemic reform, but to date the project has not become self-sustaining. The American staff live in Russia and have built trust with Russian officials and institutions, and Russians support the project with personnel and in-kind contributions. New enterprises are a major source of new jobs for most economies. However, the development of new enterprises in Russia has been hampered by years of central control of the economy, excessive rules and procedures for establishing a business, and the lack of entrepreneurial skills. To help promote the growth of small, private businesses and alleviate unemployment, the United States supported the creation of multipurpose business development centers in several Russian cities. These centers provided training and advisory services to small businesses and worked with local governments to create a hospitable environment for private business growth. USAID’s goal is that the centers eventually be operated by trained Russians on a self-financing, fee-for-service basis. The American Russian Center (ARC), established by the University of Alaska in Anchorage through an USAID cooperative agreement, was one of the first contractors in USAID’s program to establish new businesses. The program’s two phases, conducted over 2 years, cost about $5.1 million. The agreement called for ARC to provide small business training, develop Russian business activities in specific geographic areas, and develop business ties between the Russian Far East and the United States. ARC’s initial objectives were to establish a headquarters at the University of Alaska and two field centers, as well as to train a specific number of people. A subsequent work plan called for ARC to establish two more centers while expanding its program in the two original centers. Specific objectives included increasing (1) the number of Russians trained in modern business methods, (2) the number of viable Russian small businesses, (3) access to both U.S. and Russian technologies, (4) U.S.-Russian business ties through ARC field and business information centers, and (5) U.S. business activity in the Russian Far East. Creating Russian institutions that would be sustainable after USAID assistance ended was also an objective. From its headquarters at the University of Alaska, ARC worked closely with Russian partners to establish business training centers in four Russian cities. In each city, ARC had a local educational or academic institution as a partner. This partnership was reflected in the American and Russian codirectors and staff at each center and in-kind contributions such as free office space from the institutions. ARC’s American staff have had a long-term commitment to Russia. Full-time staff spoke Russian fluently, lived in the cities where the centers were located, and had business experience in the region. They were complemented by short-term American teachers who taught a 1- or 2-month course as well as by itinerant teachers who taught a 1-week or weekend course in one city and then moved to another city. These courses were taught with interpreters. ARC’s core program was an evening course that taught such skills as accounting, marketing, and management that were necessary to write a business plan. This course lasted 1 or 2 months, depending on the center, the time of year, and the targeted clientele. It was supplemented by short seminars in the host cities and extension seminars in outlying cities and was targeted at specific business sectors, such as bankers lending to small businesses. Russians and Americans, both resident and visiting, taught the courses and seminars. The centers also provided business counseling for Russians trying to set up their own small businesses. The training centers charged a relatively low fee for its courses and seminars. Participants who excelled in the training center programs were invited for advanced training in Anchorage. They were selected, in large part, through the business plans they wrote during their core course. In Anchorage, they attended a 5-week course that explored topics from the earlier training in more depth, and toured stores, manufacturing facilities, and offices in the Anchorage area. The 5-week course was followed by 2 weeks of either internships in local small businesses or more extensive business tours tailored to the participants’ interests. ARC successfully fulfilled its first year’s work plan targets, and then received $3 million for a second year’s activities (fiscal year 1994) from USAID after a March 1994 evaluation of the initial $2.1-million project. USAID also stipulated that, in fiscal year 1995, ARC must match USAID’s funding. ARC established business training centers in four Russian cities: Yuzhno-Sakhalinsk, Yakutsk, Khabarovsk, and Magadan. It established the Yuzhno-Sakhalinsk and Yakutsk centers in the fall of 1993 and Khabarovsk and Magadan centers in the fall of 1994. In May 1995, USAID agreed to provide ARC with an additional $1.5 million, even though ARC had not raised any matching funds. Between the fall of 1993 and January 1995, ARC’s Yuzhno-Sakhalinsk and Yakutsk Business Training Centers offered four cycles of evening courses, lasting 1 or 2 months, that trained 211 Russians—thereby exceeding the first year’s work plan goal of 200 participants. The two centers also provided individual business counseling to 300 Russians; the work plan’s goal was 200. In addition, the two centers offered 7 extension seminars to 103 Russians in outlying cities. The training centers in Khabarovsk and Magadan had only recently completed their first evening courses. ARC sponsored 19 technical assistance seminars, meeting the first year’s work plan goal of 15 to 20 seminars. Four seminars on banking drew 180 Russian bankers, and 8 seminars on hair salon management drew 250 women from throughout the Russian Far East. Forty construction managers from Yakutsk participated in training on cold weather construction methods. This seminar led to the government of Yakutsk testing American-manufactured plastic piping to replace its existing steel piping. Between the fall of 1993 and January 1995, 71 Russians completed the advanced business training courses at the University of Alaska. This exceeded the first year’s work plan goal of 50. In total, ARC trained 1,646 Russians in its USAID-financed programs through January 1995. On a more systemic level, ARC developed a database of U.S. and Russian businesses in the Russian Far East and provided assistance or information to U.S. and Russian businesses working throughout the region. ARC generally coordinated its activities with other U.S. government programs located in cities of the Russian Far East, but there were a few exceptions, particularly when contractors worked in separate sectors. For example, in Khabarovsk, where ARC established a center in late 1994, the local American codirector did not know the local environment project director until we visited. ARC officials in Anchorage were, however, working with CH2M Hill staff to link the projects. The ARC project will contribute to systemic reform on a regional basis if it can become financially self-supporting. USAID recognizes that creating small businesses in the region will push the Russian government to be more responsive and further develop the area’s nascent capitalist economy. The centers help Russians who come to Anchorage from their relatively isolated cities to meet each other and develop business contacts with other Russians as well as Americans. By drawing entrepreneurs from cities throughout the Russian Far East, ARC has helped build a network of private small businesses that can generate business for one another and for the region. Once the USAID funding ends, ARC’s partnerships with Russian institutions will be the key ingredient to sustaining its work. The directors of the Russian institutions plan to continue the program. For example, the Rector of the Far Eastern State Transport Academy, ARC’s partner in Khabarovsk, plans to establish a permanent school based on the activities of the project. Kray and oblast’ government officials are also highly supportive of ARC. However, despite the Russians’ desire to continue the program, most of these institutions currently lack the means to support an entire local ARC operation. Further, in an April 1994 evaluation report, USAID questioned whether ARC would be able to support itself. Other donors have not yet stepped forward to replace USAID. According to ARC’s director, the problem lies in the newness of the project and the project’s focus on Alaska and its businesses. The ARC director said the project plans to include more business internships and tours in the rest of the United States. The director believes that this expanded scope will increase ARC funding because large U.S. institutions and enterprises have the funds and business interests in Russia to provide long-term support. If USAID assistance had ended with fiscal year 1994 funds, the U.S. side of ARC would have been curtailed and U.S. personnel in Russia would have been withdrawn. USAID officials played a significant role in designing the project. Because the University of Alaska had no previous USAID contract experience, USAID sent an official from Moscow to Anchorage to help with the project’s design, which has proven to be effective. Under a cooperative agreement, USAID has relatively limited management and monitoring responsibility. ARC provided good progress reports to USAID. USAID/Moscow has adequately managed and monitored the project. USAID/Washington has maintained a duplicate document set so that it can respond to U.S. inquiries. When it was considering further funding for ARC, USAID sent an evaluation team from Moscow to Yakutsk, Yuzhno-Sakhalinsk, Khabarovsk, and Anchorage in March 1994. The team’s report became the basis for USAID’s continued funding of ARC. Within the amendment that provided the fiscal year 1994 funding, USAID included a clause stating that USAID would provide $1.5 million in fiscal year 1995 if ARC raised $1.5 million in matching funds. However, in May 1995, USAID agreed to provide the $1.5 million even though ARC had not raised the matching funds. CH2M Hill is an integral part of USAID’s $35-million environmental policy and technology program. In September 1993, USAID awarded CH2M Hill a contract to serve as the program’s core contractor and provide the technical support for its environmental activities in Russia, Kazakstan, and Ukraine. In April 1994, an initial delivery order was signed to provide support for activities in Novokuznetsk and the Russian Far East. Detailed delivery orders were signed for these activities in September 1994. CH2M Hill also serves as the contractor or subcontractor on various components of the program. Although the environmental policy and technology project is still ongoing, USAID officials said its progress so far has been disappointing. Progress has been slow because CH2M Hill did not fill critical staff positions in Russia in a timely manner, and it relied on staff located in the United States to manage the projects. The expanded scope of the Far East component further contributed to the delay. Further, USAID field staff lacked authority and information to expedite project implementation. The projects’ expected contributions to systemic reform and long-term benefits are not likely to be significant. Severe environmental degradation threatens the physical health and socioeconomic well-being of people throughout Russia and deters economic and political restructuring efforts. Environmental problems range from nuclear safety issues; to pervasive mismanagement of natural resources; to some of the worst air, water, and land pollution problems in the world. The breadth and magnitude of the economic, health, and ecological costs are difficult to quantify, although remediation activities alone are expected to cost billions. Environmental problems are exacerbated by many factors, including inattention to environmental consequences, a lack of economic and political incentives to use resources efficiently, the inability of nongovernmental agencies to participate in environmental decision-making, and the inability of governmental institutions to effectively regulate state-owned monopolies and curb illegal economic activities. Our analysis focused on CH2M Hill’s performance as the core contractor and two projects where it serves as the primary contractor—the Multiple Pollution Source Management project in Novokuznetsk and the Sustainable Natural Resources Management and Biodiversity Protection project in the Russian Far East. Both projects are to run from September 1994 to September 1997. The objectives of the $7.4-million core contract are to coordinate all activities under the core contract, monitor and evaluate the activities and deliverables, and provide support functions as needed. The objectives of the Multiple Pollution Source Management project are to reduce pollution-related health risks and promote environmentally sustainable economic development; improve public health; reduce pollutant emissions from industries and cities; assist industries in restructuring in an environmentally sound and sustainable manner; and strengthen institutions and train individuals to continue improvements initiated during the project. The $13.4-million delivery order included $6.3 million for the Novokuznetsk project along with two other projects. The Sustainable Natural Resources Management project was expanded from a narrowly focused $3-million, 3-year project focused on fire prevention and control to a $16.7-million, 5-year project focused on sustainable forest management and biodiversity protection. This expansion responded to the Gore-Chernomyrdin Commission’s recommendations. Specific project objectives are to promote sustainable forest management in the Khabarovskiy and Primorskiy Territories and protect endangered species and critical habitats in the Khrebet Sikhote-Alin’ mountain region. To address these objectives, the contract specifies 25 tasks for CH2M Hill and multiple subcontractors. USAID approved a $9.4-million delivery order for CH2M Hill to implement and coordinate these activities. CH2M Hill worked mainly with local and oblast’ government officials to design and manage the programs. CH2M Hill consultants spent short periods of time in Russia to design the project proposals and then returned to the United States to complete the project design. Although USAID and CH2M Hill established rapport with local and oblast’ authorities in the affected cities, the Ministry of Environmental Protection and Natural Resources was only involved in the initial selection of project activities and their locations. Subcontractors, U.S. nongovernmental organizations, and other federal agencies helped implement parts of the project. The project approach includes providing technical assistance, demonstration projects, training seminars, and limited commodities. Several components in both projects continued efforts initiated by the U.S. Environmental Protection Agency, World Bank, and the City of Pittsburgh. CH2M Hill staff in Washington manage the project, and a regional director and site managers in Russia handle the day-to-day activities and coordinate with other implementers. CH2M Hill plans to hire and train Russian employees who can eventually manage the activities without assistance from its U.S. office. Project progress to date has been mixed. CH2M Hill met the requirements of its core contract by establishing field offices, monitoring project implementation, and providing support functions to its field staff. Even though it has made some progress toward addressing the Novokuznetsk project objectives, it has been slow to implement the Far East project. CH2M Hill has missed critical milestones for both projects. In Novokuznetsk, CH2M Hill established an air pollution database for the 180 heating plants in the city and developed a strategic plan to address particulate pollution from the heating plants. It also upgraded the city’s air pollution program, trained Russian counterparts in environmental auditing, and completed environmental audits of two large steel mills. CH2M Hill is currently assessing local water monitoring activities and has recommended laboratory improvements to more accurately measure the quality of drinking water. CH2M Hill is working with the Novokuznetsk Development Fund and local government officials to develop a strategic plan. However, CH2M Hill has not provided an acceptable work plan, which was due on November 30, 1994. The current work is based on the delivery order specifications. In the Far East project, CH2M Hill has been even slower getting started and, according to USAID and Russian officials, had produced almost no quantifiable results as of February 1995. Several factors have hindered the project’s implementation, including its increased complexity; the size of the geographic area; and the large numbers of governmental officials, local interest groups, and subcontractors involved. The project covers 2 large regions and will involve at least 16 implementing organizations, including 2 U.S. federal agencies, subcontractors, and U.S. nongovernmental organizations. Several problems have delayed the effective implementation of both CH2M Hill projects. One problem was that CH2M Hill experienced problems filling critical staffing positions in Washington and Moscow and at the field office level. Although the core contract was awarded in September 1993, the regional director did not arrive in Moscow until February 1994. Other positions in Moscow funded in September 1994 delivery orders were still being filled as of January 1995. The contract to implement field support functions in Novokuznetsk and the Far East was awarded in April 1994, but on-site managers did not arrive until September and October 1994, respectively. The Far East project manager position was authorized in September 1994, but the manager did not move to Russia until February 1995. CH2M Hill officials had difficulty finding qualified staff who were willing to relocate to these areas because of the acute environmental problems and remote locations. USAID and CH2M Hill officials agreed that the on-site presence is essential for making progress. USAID/Moscow officials said staffing delays and delays in producing an acceptable work plan have hurt the credibility of the program in the region. CH2M Hill also had difficulty developing acceptable work plans that define how and when the scope of work will be implemented. CH2M Hill was required to submit the work plans for both the Novokuznetsk and the Far East projects within 60 days after signing the contract on September 30, 1994. USAID approved the work plan for the Far East project on May 8, 1995, but the work plan for the Novokuznetsk project was still being revised as of June 1, 1995. According to an USAID official, the work plans originally submitted were incomplete and lacked specific indicators or other factors necessary to evaluate the activities. Additionally, USAID officials said CH2M Hill had done a poor job of providing them with the appropriate reporting documents for these activities. USAID expressed concern over CH2M Hill’s failure to provide timely delivery of tracking materials, such as monthly summaries of financial status by project, monthly presentations of progress on select tasks, and weekly briefings on overall project progress. According to USAID officials, CH2M Hill addressed their concerns and has recently improved its reporting. As of February 1995, the CH2M Hill projects had contributed little to systemic reforms, and they will not generally be sustainable without outside funding support. This limited contribution is due largely to the vast environmental needs in Russia and the massive amounts of capital investment needed to modernize or purchase equipment for restructuring Russia’s environmental sector. Also, USAID and CH2M Hill officials said that Russian monitoring and enforcement procedures will be extremely difficult to change and are not addressed in these projects. Finally, the Ministry of Environmental Protection and Natural Resources was not involved in designing the project, thus reducing the likelihood the project could be duplicated on a wider scale. USAID officials said the project will attempt to address systemic reform through efforts to maintain and restock the forestry base. Some components of the Novokuznetsk project are likely to address environmental sector restructuring. CH2M Hill expects to work with Novokuznetsk’s industry, citizens, and local government to develop a strategic plan and provide recommendations for creating an environmentally safe city by 2010. However, these recommendations could require large capital investments. For example, CH2M Hill recently conducted industrial audits for two steel companies. After spending 6 weeks and using 7 U.S. advisers and 25 Russian counterparts to conduct the audits, company officials said the audits did not provide any new information on major pollution sources. Further, the companies do not have the funding to make the recommended improvements and will have difficulty obtaining it. According to one steel mill executive, the environmental audit allowed the mill to fulfill a condition for a World Bank loan. The Novokuznetsk project places a considerable emphasis on the contractor delivering studies and does not establish any indicators to measure progress in reducing actual pollution. Some components of the Far East project are designed to address the region’s need to maintain and restock its important forestry base. Efforts are planned to (1) strengthen polices and develop an adequate environmental regulatory structure, (2) create economic and political incentives to use resources efficiently, (3) increase the participation of nongovernmental agencies in environmental decision-making, (4) promote U.S.-Russian partnerships, (5) promote the export of timber products made by Russian workers, and (6) conserve biodiversity. USAID’s decision to use a core contract and delivery orders has caused delays and excessive paperwork reviews for both CH2M Hill and USAID staff. Under this system, USAID must prepare delivery orders and CH2M Hill must submit detailed work plans for each project component within 60 days. The decision to expand the Far East program has also delayed project design and implementation. The expansion covers a larger geographic region and greatly increased the scope of work, including the number of activities and subcontractors involved. The division of responsibility between USAID/Washington and USAID/Moscow has affected the agency’s ability to manage the project. USAID/Washington maintains overall management authority, but has given USAID/Moscow increased monitoring and program responsibility. However, USAID/Moscow officials said they still had minimal authority to manage the project or make changes. USAID/Washington must approve all program decisions, including minor ones, such as country clearances for visitors and all purchases exceeding $500. In April 1995, USAID/Moscow submitted an initial request, which remained under review as of June 1, 1995, for delegation of authority to the field. USAID has had difficulty monitoring the projects. USAID staff said they have not regularly visited the project sites because of the difficulty of traveling to the sites and the lack of adequate staff. The USAID/Moscow project officer keeps apprised of the project activities primarily by talking to project staff over the telephone or in informal meetings and by reviewing reports by the contractor or visiting teams. The district heating project is one component of USAID’s Energy Efficiency and Market Reform Project for the NIS. The project began in January 1992 and is considered the first economic development effort undertaken by the United States in the region. With $5.3 million in funding, the project was designed to improve district heating systems in six countries: Armenia, Belarus, Kazakstan, Kyrgyzstan, Russia, and Ukraine. Although the contractor, RCG/Haggler Bailly (RCG/HB), met most of its objectives, we found no indication that the project was having a significant impact on the sector. Most of the Russian work was concentrated in two Russian cities, Yekaterinburg and Kostroma, and the project was not completed in Yekaterinburg. Because USAID did not adequately monitor the project, it was unaware of (1) problems that prevented the completion of the project and (2) any long-term benefits, if any, to the beneficiaries. An evaluation conducted by a consultant did not identify obvious problems, and USAID did not address the recommendations in this evaluation. Fuel and energy are an important part of Russia’s economy. The subsidies provided by the former Soviet government to Russian energy consumers, both residential and industrial, created artificially low prices and promoted the inefficient use of highly polluting energy. Since the dissolution of the Soviet Union, Russia has implemented several policies, including increasing or freeing coal, oil, and gas prices, to reform its energy sector. Although still below world market levels, the cost of domestic oil and oil products in Russia doubled in 1993 and 1994. Such increases in energy prices have a significant influence on inflation and social conditions. As energy prices increase, consumers must find ways to use energy more efficiently. In February 1992, RCG/HB was awarded a contract for $550,000 to complete the project in Russia. The project was amended in August 1992, increasing the total cost to $1.3 million. The project had five objectives: (1) foster improved management of energy use in heating plants by identifying and implementing cost-effective “low cost-no cost” energy efficiency improvements; (2) transfer energy auditing and management techniques, including financial and economic analysis techniques; (3) provide equipment support to implement low-cost options, improve monitoring and energy management, and identify additional energy efficiency opportunities; (4) support the World Bank’s efforts to reform Russia’s energy pricing policies; and (5) promote the emergence of an energy efficiency industry in Russia. RCG/HB and USAID worked with representatives from the Russian Ministry of Fuel and Power, the Commission for Humanitarian and Technical Assistance of the Russian Federation, nongovernmental organizations concerned with energy efficiency and conservation, municipal governments, and industrial enterprises. The two primary Russian cities selected for the project were Yekaterinburg and Kostroma. In these cities, extensive energy audits were conducted of the district heating facilities, and three sites (i.e., hospitals, apartment buildings, and heating plants) in each city were selected as demonstration sites for U.S. energy efficiency equipment, including flow meters, temperature sensors, and thermostatic control valves. The value of the equipment supplied to the demonstration sites was approximately $172,000. The project sites were intended to demonstrate the savings in using no-cost or low-cost technologies and also serve to promote American-made equipment. In addition, RCG/HB conducted energy audit training seminars and provided energy audit equipment to technicians in Yekaterinburg, Kostroma, Irkutsk, Moscow, Murmansk, and St. Petersburg. To complete its work, RCG/HB contacted more than 250 U.S. equipment manufacturers to determine their interest in conducting business in Russia. The 12 companies that responded participated (at their own expense) in “wrap-up” seminars in four Russian cities when the project ended. The information obtained at these seminars was published in a lessons learned document. RCG/HB completed most of the objectives stipulated in its contract. The products delivered to complete the objectives included energy audits in two cities, energy audit training and distribution of energy audit equipment, a study of natural gas pricing in Russia, and an energy efficiency industry development effort. It also produced a video about the project that was shown on Russian television. RCG/HB was also required to identify, purchase, and install low-cost energy efficiency equipment manufactured by U.S. companies. RCG/HB purchased this equipment; however, due to problems with local conditions, some of the equipment was not installed in Yekaterinburg. An RCG/HB official said that in June 1993, a Russian subcontractor assured RCG/HB that it would install the equipment in Yekaterinburg by the end of 1993. We visited three sites in Yekaterinburg in February 1995 and found all the equipment at one site was still in shipping containers. Russian officials said the equipment at the other two sites only began operating in September 1994 and January 1995, respectively. According to an RCG/HB official, the company had not paid the subcontractor and would not pay until the installation was completed. However, USAID had already paid for the equipment, valued at $8,000. Officials in Yekaterinburg stated that the equipment had not been installed in 1993 for two reasons. First, in two cases, the sites (a hospital and an apartment building) were under construction and the construction plans had to be altered to accommodate the equipment. Second, at the other installation site (a district heating facility), the equipment had not been installed, and most likely will not be installed, because the proper Russian authorities had not certified it. Officials in Yekaterinburg stated that it would be illegal to install and operate the equipment before it was certified. They explained that although the equipment can be used for demonstration purposes at consumer locations (e.g., apartment buildings), a public utility cannot use the equipment and the information (e.g., energy consumption data) it produces as a basis for charging customers. Similar equipment was installed in Kostroma, according to USAID and RCG/HB officials, even though it had not been certified. USAID officials told us that city officials were willing to install the equipment because they realized the potential benefits. We found no indication that the project had contributed to systemic reform in the area of energy efficiency. Most of the work was concentrated in two cities, and the project was not completed in either city. In addition, USAID did not adequately monitor the project and could not be certain of any long-term benefits. USAID used an independent consultant, Management Systems International, to evaluate the NIS district heating project, including RCG/HB’s work in Russia. The evaluation, published in July 1993, reported no serious problems and declared the project a success. Specifically, the study indicated that total equipment costs for the four cities in Russia amounted to $418,000 and would produce an annual savings of $1.4 million. It also noted that the equipment would reduce pollution. Furthermore, as a result of the energy efficiency industry development effort, 12 U.S. companies had sent representatives to the various countries to participate in seminars held at the end of the project. We found that the consultant’s evaluation was deficient. The evaluation did not mention the equipment installation problems in Yekaterinburg or the need to have foreign equipment certified by the Russian government. Instead, the evaluation stated that “by April 1993, all of the equipment was installed and operating.” In addition, USAID did not specifically direct Management Systems International to assess the products RCG/HB was required to produce, such as the natural gas pricing study for Russia or the lessons learned from the energy efficiency industry development effort. The evaluation did not discuss the quality of either of these products. USAID officials stated that the natural gas pricing study had been completed in a collaborative effort with the World Bank, which used it in its work pertaining to loans made to Russia’s natural gas sector. However, the consultant’s report contradicted USAID’s statement by noting that the World Bank did not make a serious attempt to involve RCG/HB in its work in Russia. Finally, the evaluation did not discuss the training seminars conducted by RCG/HB in Irkutsk, Moscow, Murmansk, and St. Petersburg, or the energy audit kit instrumentation supplied to technicians in these cities. The continued use of these deliverables is an important factor to consider when evaluating the success and sustained benefits of this project. USAID officials were not aware of the problems we identified in Yekaterinburg nor the shortcomings of the evaluation. They stated that in June 1993, an official from USAID/Washington visited all the NIS sites except Yekaterinburg. They stated that equipment had been installed at the sites visited. A local national employee from the USAID mission in Moscow also visited Yekaterinburg in June 1993 but did not report any problems at that site. USAID officials discovered the problems we found when they accompanied us to Yekaterinburg. USAID said it would take corrective action. Also, USAID has no mechanism to monitor various outcomes of the project, including (1) the success of U.S. industry in entering the NIS market, (2) policy reforms written into law, and (3) the rate of adoption of new technologies. Although USAID said that the installed equipment would produce annual savings at project sites, it did not record these savings during the 1993 or 1994 heating seasons. Furthermore, USAID had not determined the savings generated by either the energy audit kits provided to technicians in six cities or the energy audits conducted in Yekaterinburg or Kostroma. USAID initiated the NIS Exchange and Training program in the spring of 1993 to train NIS leaders about free-market economies and democratic governance. USAID hoped that training the participants in the United States would provide them with the technical skills and attitudes required to create similar policies, programs, and institutions in their own nations. We reviewed the health care training provided to Russians in late 1993. Our analysis indicated that the health care training had little likelihood of contributing to systemic reform and that USAID now considers the training to be irrelevant after Russia changed its direction for health care reform. The training’s primary objective—to facilitate Russia’s transformation to a democratic free-market system—was unrealistic for a 2-week training course. USAID did not follow up with participants to determine the training’s impact on systemic reforms. Although USAID officials said that most participants have been involved in follow-on projects, only 25 percent are slated for follow-on activities planned in 1995 and 1996. According to USAID, Russia’s health care industry has a number of problems. These problems include the virtual collapse of the pharmaceutical and medical supply industry, poor quality of care due to training and technical gaps, serious funding shortfalls, and a centralized system devoid of incentives for efficiency and cost control. Although Russian policies have produced an educated workforce with more doctors per capita than the United States, the workforce lacks many of the basic skills and institutions necessary to function in a democratic, free-market context. USAID contracted in June 1993 with its worldwide training support contractor, Partners for International Education and Training (PIET), to conduct training in the United States for 200 NIS leaders and professionals at a cost of $2.6 million. The training objectives were to facilitate the region’s rapid and sustainable transformation from authoritarian, centrally controlled regimes to pluralistic, democratic countries with free-market economies; provide participants with new skills and knowledge to contribute to economic and social development; promote the value of democratic decision-making; provide an understanding of U.S. programs; and lead to long-term relationships with U.S. institutions. USAID also hoped that participants would share their new skills and perceptions with their counterparts. Under the PIET contract, USAID missions identified the training topics and selected the participants. USAID/Moscow selected the participants based on their positions in oblast’ health care systems and their planned inclusion in follow-on projects. According to USAID, participants went to the United States before participating in follow-on projects so they would be more receptive to reforms. The training project encouraged missions to link training, if appropriate, to ongoing or planned developmental assistance by USAID and others. After course topics and participants were selected, PIET was expected to arrange training courses in the United States and provide administrative and logistical support for international travel, living expenses, medical insurance, tuition, books, and other needs. PIET was also expected to (1) ensure that training programs at U.S. training institutions were functioning properly, (2) monitor the participants’ progress, (3) provide USAID status reports, and (4) evaluate each training program. PIET subcontracted with Management Sciences for Health to provide the training in the United States; 42 Russians were trained in health finance and 20 were trained in pharmaceutical management. USAID subsequently contracted with the Academy for Educational Development to make training arrangements. PIET met its contractual requirements by providing training, transportation, and logistics, according to USAID officials. The participants we spoke with in Russia praised PIET’s support and assistance as well as the quality of the training they received in the United States. Our review of sample course assessments showed that other participants generally gave high marks to the training. For example, in the evaluation conducted by the USAID mission, most participants were satisfied with the course and believed it was applicable to their work conditions. PIET also met its monitoring and reporting requirements. PIET maintained contact with the training institutions, called a random sample of participants once a week, contacted the trainers on an as needed basis, and helped participants with general adjustment problems. PIET also provided all the required reports, including regular status reports and course assessments. USAID/Washington officials were satisfied with the quality of PIET’s support and monitoring during the training. USAID was unable to provide any evidence that the training will help Russia’s democratic or economic transformation. Although the training may have met some secondary goals, without follow-on activities, fulfilling these objectives will not likely result in systemic reform. USAID/Washington officials agreed that the training could not meet all of the contract’s objectives. They said that a 2-week training course could only “facilitate” reaching these objectives but not actually attain them. Further, they provided the training quickly as a political imperative to respond to the opening in the NIS, and they recognized training alone has limited usefulness. USAID was unable to substantiate that any of the 62 participants contributed to any reforms, partly because the participants lacked the authority, expertise, or resources to influence reforms. However, the participants who had taken PIET health-related courses said the training helped them understand U.S. programs and they had shared their training with others. USAID officials in Moscow and Washington said that training alone would not influence systemic change and that subsequent training was better integrated into follow-on activities. They said that the main purpose of the PIET training was to make participants receptive to follow-on reform projects, which USAID thought would occur. However, USAID later dropped plans for follow-on activities in Central Russia because the oblasts were not reform-minded and the contractor reported that only 25 percent of the Siberian participants would participate in follow-on activities. The Russians did not see health reform as a priority when the early training took place, and Russia has only recently begun to consider the direction of reforms, according to USAID/Washington officials. Further, this early training is now irrelevant because it was based on Russian policy directions that were later discarded as unworkable. The mission was forced to move much more quickly than it desired because it was under extreme congressional pressure to quickly establish the training program, according to USAID officials. As a result, the health care training was initiated before it could be integrated into follow-on projects more likely to facilitate systemic change. Further, because the Russians were unclear about what reforms they wanted, the mission had trouble targeting the training. The Russian officials began exploring reform options with the mission in December 1994. USAID/Moscow officials assessed the training after participants returned to Moscow; however, they have had no contact with the participants since then. They did not know which participants, if any, would be involved in any of the follow-on activities planned in Siberia. The International Business & Technical Consultants, Inc. (IBTCI) project did not achieve its goal to increase the availability of commercial real estate to small enterprises, although it did provide potentially useful technical assistance in three cities. The project did not contribute to systemic reform and was not sustainable. IBTCI did not replicate the pilot project—a project objective—in part because the roll-out cities were poorly chosen. IBTCI was responsible for choosing appropriate cities, but its short-term consultants lacked sufficient knowledge of Russia and local conditions to determine what cities would have cooperative officials and could benefit from the project. Much of Russia’s commercial real estate is still owned by the government. Rather than divesting its ownership rights, the central government has decentralized those rights to local governments, both regional and municipal. Although this practice is quite common among other countries in transition, Russia is different because local governments (1) have a virtual monopoly on commercial real estate and (2) have not moved toward commercial real estate leasing using market mechanisms. Highly inefficient users occupy valuable commercial space, contributing very little to local budgets, while private sector development is blocked by the unavailability of property. USAID and GKI recognized this problem and signed a task order with IBTCI to develop a solution. The $2-million task order for the rapid diagnosis pilot project and roll-out project was part of IBTCI’s $13.3-million omnibus contract. The initial deadline of May 1994 was extended to December 1994, but without any increase in the cost or level of effort required. The general purpose of the task order was to significantly increase the availability of commercial property in Russian cities. The specific goals of the task order were to examine the causes of limited access to retail space, implement a pilot project in a selected city, and then replicate the pilot project in other oblasts. IBTCI was to (1) deliver a report on the root causes of and solutions to the problem of commercial property access for one city; (2) design an implementation plan to address these issues, including procedural, legal, administrative, financial, policy, and other measures; (3) replicate the pilot project in at least five other oblasts; and (4) produce and nationally distribute publicity and instructional materials for local state property committees, local authorities, and entrepreneurs on how to increase the availability of retail property. IBTCI used a subcontractor, Boston Consulting Group (BCG), to perform the rapid diagnosis phase and conduct the pilot project in the City of Perm’.The goals of the pilot project were to (1) design and test a method for increasing the amount of commercial real estate available to small and start-up businesses and (2) identify any constraints or impediments that might exist. The pilot was intended to serve as a model for instituting the program in five other Russian cities. IBTCI instituted the roll-out in Irkutsk, Tver’, Novgorod, Yekaterinburg, and Vladivostok. Because it had previously worked in Perm’, BCG used staff who already had a relationship with municipal officials when it began the diagnosis and pilot phases of the project. In contrast, IBTCI relied on consultants who made short visits to the other cities to research, plan, and implement the roll-out. During the rapid diagnosis phase in Perm’, BCG identified three feasible ways of improving access to commercial space: convert residential premises to commercial use, develop a secondary real estate market, and optimize the leasing process. BCG, IBTCI, USAID, and GKI selected the leasing optimization method because they thought some concrete results were possible during the study period, even though it was predicted that this alternative would have limited support and low potential impact. Lease optimization means, among other things, (1) moving toward market-determined rents, (2) removing bureaucratic discretion in space allocation, and (3) creating incentives to sublease unused space. BCG conducted the rapid diagnosis and pilot phase of the project from November 1993 to March 1994 in Perm’. BCG devised a two-track auction system for making municipality-controlled real estate available to private businesses. The first track was an auction for the right to lease specific commercial real estate properties (i.e., a one-time premium). The second track was an auction for the rental rate at which a property would be leased. The purpose of this system was to introduce market mechanisms into the allocation and pricing of commercial real estate. Under this system, bidding for the right to lease and the rent to be paid replaced government bureaucrats with market mechanisms. The results of the first auction, which occurred on March 1, 1994, were not promising. In the first track auction, three properties were available. The right to lease them was sold for each property. In the second track auction, 15 properties, all basements, were available. Bids were made on only 3 of the 15, and each received only one bid. The rental rate for the three properties did not exceed the rent that started the bidding. The results of a second round of auctions, which occurred in May 1994, were also disappointing. In the first track auction, 10 properties were available, but the right to lease was sold for only 4 properties, although several parties bid on them. In the second track auction, three properties were available, but only one received a bid, and that was the starting bid. These results were not perceived to have significantly increased the amount of commercial real estate available in Perm’. IBTCI started work on the roll-out in mid-February, before the Perm’ pilot was completed or its results evaluated. IBTCI soon found that none of the five cities chosen for the roll-out had conditions that approximated, let alone duplicated, those in Perm’. The roll-out cities seem to have been chosen more for their geographic and population distributions than for any existing economic, political, and regulatory conditions that might make the Perm’ model replicable. Because of these differences, IBTCI had to deviate from the Perm’ model and basically develop five new pilot projects; nonetheless, it still experienced problems. Irkutsk officials were not cooperative with IBTCI and declared the information needed to assess the commercial real estate situation a state secret. Local officials were not ready to participate in the project. In Tver’, an auction system had been functioning between June 1992 and December 1993. The original investment tender process used in the auction was challenged in court and hopelessly compromised. IBTCI introduced a new tender process in Tver’, but the new system’s effectiveness has not yet been demonstrated. The Novgorod officials opposed conducting right-to-lease auctions because they feared losing future revenue and the city had experienced poor results from a similar auction in November 1993. IBTCI focused on establishing a market for municipal, oblast’, and private commercial space by creating a real estate listing center, developing a secondary market, and encouraging officials to allow increased and legalized subletting. The listing center’s effectiveness has not yet been demonstrated. In Yekaterinburg, an effective auction system has been in place since 1992. City officials were not interested in IBTCI’s original task of increasing the use of commercial leases. Instead, they wanted assistance in how to use retail and commercial space efficiently and increase the city’s revenues from property leases. Although IBTCI submitted some analyses and recommendations addressing their concerns, city officials told us that IBTCI came to town on different occasions, spent little time there, did not speak to the appropriate local officials, and presented an academic report that was of little use to them. Vladivostok city officials were interested in privatizing commercial real estate, but were unable to devise a method that would use mortgages to provide substantial revenue for the city. IBTCI devised a mortgage instrument that allowed the city to continue receiving income by holding the mortgages and allowed small business owners to bid for a property and provide as little as 5 percent of the final cost as a down payment. The city auctioned one property in August 1994 for under $7,000, but local officials doubted they would use an auction again because the city did not not have any more excess property. By the time IBTCI had completed its work in the five cities, none of the cities had participated in any activities that remotely resembled the Perm’ model. As a result, the objective of replicating the pilot project in other cities was not achieved. There were various reasons for IBTCI’s inability to replicate the Perm’ model. First, tensions between IBTCI and BCG caused some problems. BCG performed both the rapid diagnosis and the pilot phases, but IBTCI determined that BCG’s approach was not adequate. Russian officials monitoring the project were aware of tensions between IBTCI and BCG early in the project, but IBTCI was obligated to fulfill the contract and replicate the model. The tensions between IBTCI and BCG resulted in little continuity of personnel from the pilot to the roll-out phases. Second, a provision of Russia’s 1994 State Privatization Program Act and its implementing regulations caused problems in the Perm’ pilot. The provision gave lessees who obtained their leases competitively (i.e., at an auction) the right to buy the property at the end of the leases. The implementing regulation set an extremely low selling price for such privatized properties. IBTCI said that the act’s provision and the implementing regulation stopped the Perm’ model because city officials did not want to lose revenue from leases and did not want to be forced to sell leased property for extremely low prices. Even though reports identified the problem as early as January 1994, USAID, GKI, and the Russian Privatization Center took no effective action to address the issue. Third, although officials at the federal level agreed earlier that the project should be done and that the Perm’ model was viable, local officials in the roll-out cities did not agree with the Perm’ model or its usefulness in their cities. Fourth, although the consultants used by IBTCI for this project had some experience in Russia and some spoke Russian, Russian authorities questioned the level of some IBTCI consultants’ professional experience. In addition, the consultants did not have enough knowledge of the Russian localities and local politics to choose roll-out cities well. IBTCI staff did not reside in the cities during the roll-out. Instead, they would fly in, do a few days work, then leave. Thus, they were unable to identify what cities would be the best candidates for replicating the Perm’ model. Even BCG had problems carrying out a successful pilot, despite its knowledge of and relationships in Perm’. The project was not sustainable and did not contribute to systemic reform. Although IBTCI’s final report provided solutions to specific problems, the project did not implement the pilot or develop a method that could be replicated in other cities. USAID officials and the Russians who were in charge of disseminating the report did not know whether or where the IBTCI “solutions” had been applied in any but the six cities. City officials we interviewed in Yekaterinburg and Vladivostok were not using the concepts of the project. GKI and the Russian Privatization Center had originally proposed the project, which suppported a federal initiative. However, an existing GKI act and its implementing regulation potentially forced local governments to sell leased property at low prices to anyone who bought the lease at an auction. This regulation contributed to the poor results of the project. USAID managed this project—from Washington with limited help from USAID/Moscow—jointly with GKI and the Russian Privatization Center; it relied on consultants from the Harvard Institute for International Development and the Center to help monitor the project. Nonetheless, USAID did not monitor the project adequately. Even though IBTCI filed the required reports, these reports failed to describe how much the roll-out deviated from the Perm’ model. Center officials said they first became aware that the pilot was not being implemented in other cities in late May 1994, long after the roll-out could be redirected to other cities. When problems were known in some cases, USAID did not take any corrective action. For example, a Harvard consultant who visited some sites raised questions about the cities selected for the roll-out, but USAID took no corrective action. Similarly, the problem with the State Privatization Program Act and its implementing regulations was mentioned in reports in January 1994, but no action was taken to resolve it. Finally, USAID officials said they were aware of the tensions between BCG and IBTCI, but simply told IBTCI to work the problem out themselves. USAID/Moscow officials said they did not have enough staff to intervene when problems arose, visit the project sites, and talk with beneficiaries about how the project was progressing. The lack of quantifiable objectives or time frames in the Tri Valley Growers’ (TVG) project design makes it difficult to measure the project’s success. TVG helped to facilitate the work of two agribusiness partnerships in Russia; nevertheless, USAID concluded that TVG did not perform adequately. It is too early to determine the long-term economic viability of the partnerships; however, the involvement of U.S. companies increases the likelihood the partnerships will be maintained. The partnerships will probably not have any measurable effect on Russia’s agricultural sector because of their limited size and number. Agriculture plays an important role in the Russian economy. Although estimates vary, Russia has approximately 27,000 large state and collective farms, which cultivate approximately 90 percent of Russia’s arable land. Approximately 270,000 private farms cultivate 5 percent of the arable land. The remaining 5 percent is made up of private garden plots. The total farming population comprises about 26 percent of the country’s population. Subsidies and income transfers to the agricultural sector represent 25 percent of Russia’s public expenditures. Some of these subsides could be expected to be eliminated if the agricultural sector were privatized. Russian agriculture is a low-productivity sector. For example, milk cows and potato and grain crops yield about half of western levels, and labor productivity is probably as low as one-tenth. In addition to low productivity, Russia has been plagued by losses of up to 50 percent in its storage and handling systems. Finally, Russia’s food processing system suffers from poor management and a lack of quality produce, additives, ingredients, and packaging materials. Although the Russian government has begun reforming the agricultural sector, the actual transformation of farms and agribusiness enterprises into market-oriented, productive entities is moving slowly. In 1992, it reorganized state and collective farms and agricultural input and output distribution enterprises into joint stock companies. However, most farms have not altered their operations to increase productivity and competitiveness. In August 1992, USAID developed the agribusiness partnerships project as the cornerstone of its Food System Restructuring Project. The agribusiness partnerships project was designed to create efficient systems for providing inputs to agriculture and for processing and distributing agricultural products. USAID intended to catalyze NIS private sector activity by facilitating the involvement of private U.S. agribusinesses. Between January and May 1993, USAID signed cooperative agreements with three agribusiness cooperatives: Citizens Network for Foreign Affairs (CNFA), TVG, and Agricultural Cooperative Development International. We reviewed USAID’s cooperative agreement with TVG, which had obligated $5.2 million for the region. To achieve the project’s objective, TVG was to facilitate partnerships between American and NIS private agribusiness-related enterprises. However, the agreement did not specify the number of partnerships or the related time frames. TVG established an office in Moscow staffed by one American director and three Russian nationals. This office was supported by several TVG headquarters staff in California. The American director did not have an agribusiness background but was responsible for managing the office, identifying potential Russian and American agribusiness partners, reviewing partnership proposals, and submitting the proposals to USAID for final subgrant approval. According to TVG officials, TVG identified potential Russian agribusiness partners through a network of contacts at the Ministry of Agriculture, Association of Individual Farms and Agricultural Cooperatives of Russia, World Bank, European Bank for Reconstruction and Development, Peace Corps, investment funds, and regional and local administrations. To identify American partners, TVG canvassed its members in the United States, advertised in agricultural publications, contacted agribusinesses via telephone, and looked for firms already operating in Russia. Once identified, TVG worked with the American and Russian partners to develop proposals for USAID’s approval. After receiving USAID approval, TVG awarded subgrants to U.S. agribusinesses working in Russia primarily to provide technical assistance and agricultural training to help create efficient food systems. The American agribusiness partner was required to provide at least 2.5 times the level of funding provided by USAID, to ensure its commitment to the partnership and the long-term economic viability and sustainability of the joint venture. The items purchased with the USAID subgrants are referred to as “additionality” components, or those components that might otherwise not be included in the joint venture without USAID funding. Additionality components include additional training and facility expansion. TVG established six partnerships in five NIS countries, with two in Russia. As of March 1995, one additional partnership in Russia was awaiting USAID approval. The first partnership established in Russia was with Petoseed Company, Inc., and is located in Krasnodar. Petoseed produces vegetable seeds that will be sold in the NIS and internationally. During the 1994 growing season, Petoseed produced 11,000 pounds of seed in Russia. The second TVG Russian partnership involves CTC Foods Company, which is building a potato processing facility in Pushchino. If finished, the facility will produce dried potato flakes that will be sold primarily to hospitals and schools. The two American agribusiness partners exceeded the required level of partnership funding in both partnerships. Contributions by USAID, U.S. agribusiness partners, and Russian beneficiaries to the TVG partnerships in Russia are shown in table III.1. TVG had difficulty identifying partnerships. TVG staff had difficulty beginning work in Russia because of poor telecommunication and office facilities, the chaotic Russian business environment, the limited number of American firms willing to invest in Russia, limited funding, and a small staff. According to a TVG official, Petoseed and CTC Foods contacted TVG to participate in the project. However, both companies were already working in Russia before USAID had established the agribusiness project and located Russian partners on their own. He said they would have invested in Russia without USAID involvement. An official at the Association of Individual Farms and Agricultural Cooperatives of Russia told us that the Association tried to work with TVG to identify Russian partners but received only “empty promises.” Although USAID never specified the number of partnerships that it wanted to establish within a given time frame, it concluded that TVG had not performed adequately. Between May and December 1993, USAID expressed concern about the number of partnership proposals TVG was submitting and the quality of the proposals. A February 1995 USAID review of the agribusiness project stated that TVG required more support by USAID staff and was less committed to the project than CNFA. TVG closed its office in Moscow in August 1994 and has stopped the Russia part of its program because USAID terminated the agribusiness partnerships project in Russia. Nevertheless, USAID’s review noted that the partnerships to which TVG had made subgrants were doing well. However, a TVG official told us in May 1995 that because of financial problems, CTC Foods may not be able to continue its work in Russia. Consequently, the processing facility in Pushchino may never be constructed. According to a USAID official, TVG’s Moldova office now monitors the Russian subgrants. The agribusiness partnerships developed by TVG in Russia have not been operating long enough to adequately judge their impact. However, due to their limited scope, it is unlikely that the partnerships will have a significant effect on reforming Russia’s agricultural sector. USAID/Washington designed the agribusiness partnerships project in 1992, before the USAID/Moscow mission was opened. USAID/Washington and USAID/Moscow split the oversight responsibilities: Washington was primarily responsible for TVG’s compliance with the cooperative agreement and Moscow was responsible for subgrant proposal evaluation. Final grant approval was a joint Moscow/Washington effort. Although USAID/Moscow raised continued concerns about the agribusiness partnerships project’s ability to influence systemic reform, the project proceeded. USAID/Moscow officials called for a review of the project as early as November 1993, and they developed a statement of work for an evaluation team. However, USAID/Washington told USAID/Moscow to “forget the assessment and get on with the job.” Consequently, no assessment was conducted. According to USAID officials, an evaluation is planned for June 1995. According to USAID officials, the agency wanted to implement the project quickly and demonstrate results. TVG’s Moscow director stated she was pressured to submit proposals quickly because USAID was being pressured by Congress. However, both CNFA and TVG officials complained that USAID’s subgrant approval process was arduous and lengthy. They said it took several months for USAID to accept or reject a proposal and added that USAID/Washington caused most of the delay. USAID/Washington officials said the delays were caused by the time required to research legal issues, conduct environmental audits, and work through the Washington bureaucracy. The cooperative agreement with TVG called for quarterly program performance reports and annual progress reports. An independent accounting firm was to audit TVG’s financial statements. Although USAID officials said that TVG met all of its reporting requirements, our review indicated that TVG had not submitted annual reports. According to USAID officials responsible for the project in Russia, USAID staff visited only half the project sites established by TVG, CNFA, and Agricultural Cooperative Development International between May 1993 and November 1994. USAID was required to annually assess the performance and program direction of the cooperative agreement and contract for an independent external evaluation. As of March 1995, it had done neither. However, CNFA completed an evaluation of the agribusiness partnerships project in August 1994 at USAID/Moscow’s request. It reported that the project had not started agribusiness partnerships quickly, had not made a significant contribution to sectoral reform, and had little to show for the “additionality” purchased with USAID funds. CNFA’s internal evaluation did not address TVG’s performance. USAID completed an internal review in February 1995, but the review did not cover the additionality components. The review stated that it was unrealistic to expect the overall project to have a significant, measurable impact on the food system in the NIS. USAID has discontinued the agribusiness partnerships project in Russia and, as of September 1994, stopped accepting proposals for Russian agribusiness partnerships. In addition, USAID has decided not to obligate any additional funds for the project. Agency officials stated that the project itself cannot adequately address the obstacles of reforming the agricultural sector and indicated that other projects, such as the Russian-American Enterprise Fund, were better vehicles for financing joint ventures. The Russian officer resettlement pilot project has been successful in providing the required housing units, although not within the original time frames. The project’s secondary objectives—to provide job skills training for demobilized officers and to help facilitate housing sector reform—were only partially met. By implementing a pilot program, USAID was able to test the viability of a housing construction project and apply lessons learned to the $160-million follow-on project. Planning and Development Collaborative International (PADCO), the contractor tasked to provide construction management services, was successful in part because it (1) had experience in working on housing sector reform in Russia, (2) established a physical presence in Moscow and in the field, (3) obtained at least some buy-in and involvement from the local Russian government, and (4) employed Russian staff to oversee construction activities. The Russian Ministry of Defense has traditionally provided qualifying retired and demobilized military officers with a dwelling unit or plot of land and some job skills training. After the Soviet Union dissolved, between 200,000 and 350,000 officers needed housing; approximately 42,000 were located in the Baltic Republics of Estonia, Latvia, and Lithuania. However, since the dissolution, the Russian government has lacked the housing stock to resettle all the demobilized military officers. Further, Russia’s severe economic problems, housing shortages, and lack of “social guarantees” for these retired officers has delayed troop withdrawals. President Clinton announced the Russian officer resettlement program at the Vancouver Summit in April 1993. Later, in July 1993, he stated at the G-7 Heads of State meeting in Tokyo that the program should encourage rapid withdrawal of demobilized Russian officers from the Baltic Republics. The Russian Officer Resettlement Initiative is being conducted in two phases—a $6-million pilot and a $160-million follow-on project. The pilot’s primary objective was to construct 450 housing units by July 1994 for the resettlement of demobilized Russian military officers. The follow-on project was to provide up to 5,000 units (2,500 constructed and 2,500 voucher certificates) by November 30, 1996, for officers demobilized in the Baltics or other countries outside Russia. The pilot project’s secondary objectives were to provide job skills training, experiment with new housing technologies, assist private firms in housing development and construction, and expand the scope of housing choices. To implement the pilot project, USAID contracted with PADCO for construction management services. It also awarded fixed-price contracts to five Russian builders and one private voluntary agency to construct housing units in five cities. Finally, it provided a grant to the International Catholic Migration Commission for training. According to project officials, PADCO assisted the project design team that included officials from USAID/Washington and USAID/Moscow. This team visited potential project sites, evaluated projects, and negotiated construction contracts. PADCO was responsible for managing the construction activities and monitoring contractor performance. U.S. officials said the design team created the pilot with only minimal input from the Ministry of Defense or the Ministry of Construction. USAID officials added that the design team conducted its own field assessment to select participating cities and worked almost exclusively with the local authorities in these cities. The local authorities were to provide infrastructure services such as heating, water, and road access for the housing units. USAID relied on the Russian Ministry of Defense to select the officers to receive the housing. The initial design for the pilot program did not stipulate where the officers should come from, but as a result of the Tokyo G-7 meetings in July 1993, USAID established criteria that gave priority to demobilized officers living in the Baltics. The criteria also included housing for officers from other areas outside Russia because two cities were reluctant to provide infrastructure for officers exclusively from the Baltics. Under this criteria, officers demobilized in other areas would be included. USAID’s compromise with these cities allowed some demobilized officers from their own jurisdictions to receive housing. In Nizhniy Novgorod, half the officers could come from its jurisdiction, while in Volgograd, 40 percent of the officers could come from its jurisdiction. USAID and PADCO officials said beneficiary composition would also be an issue in the follow-on project. PADCO’s project staff established a long-term presence in Moscow and traveled regularly to the various building sites. It also hired and trained Russian construction specialists to supervise the construction in each city. USAID officials said PADCO’s experience in Russian housing issues helped facilitate this project. PADCO and Russian contractors generally met the program’s primary objective of providing housing units, although not within established time frames. As of July 1994, only 94 (21 percent) of the 452 units were completed, although as of February 15, 1995, the project had provided 422 units (93 percent) through a combination of construction and voucher certificate activities. (See table III.2 and figs. III.1 and III.2.) Of the 10 project sites in 5 cities, USAID terminated 3: one because newly elected local officials refused to meet the previous administration’s commitments to provide infrastructure support to the housing units and 2 because contractors defaulted on their building commitments. In Novosibirsk, USAID and PADCO officials said federal and oblast’ officials were not involved in the initial agreements. Therefore, they had no authority to require the local administration to abide by the contract, and they would not allocate additional funds for the infrastructure. In Lipetsk, the contractor was a private voluntary agency that subcontracted with a local Russian construction firm to execute the work. When the subcontractor defaulted, the agency was unable to find a replacement to complete the work. At the Nizhniy Novgorod 50-unit project site, project officials said the Russian contractor ran into financial problems and stopped work, claiming that the $8,500 per unit allowed in the contract was not enough to cover costs. Although the city offered several incentives, including a $300,000 letter of credit and land for additional construction, USAID and PADCO officials said the contractor was unwilling to spend his own funds and the contract was terminated by USAID. USAID officials said the contractor at the 128-unit Nizhniy Novgorod site was concerned that $8,500 per unit was not enough to cover the cost of construction. The contractor had completed almost 70 percent of construction when increased construction costs, caused by rapidly rising inflation (9 percent a month) and the devaluation of the ruble, forced him to stop work. According to USAID officials, because the contractor had done a good job, used his own funds from other projects, and was well-connected with city and oblast’ officials, he negotiated an agreement so that the oblast’ and USAID would cover the increased costs of the 128 units. To ensure the project’s completion, USAID and the oblast’ administration each provided an additional $700,000, thus increasing the per unit price to $19,500. Because contracts were terminated months after they were awarded, USAID developed a method to meet the housing requirements quickly. It awarded a contract to the Urban Institute to implement a voucher certificate program, which allowed officers to purchase existing local housing in a participating area or housing under construction. Because of increased construction costs, the inclusion of land, and infrastructure costs, the vouchers were increased from the $8,500 per unit in the construction program to a maximum of $25,000 per unit. According to USAID officials, using voucher certificates allowed the pilot program to provide housing units much quicker than through direct construction. As of January 30, 1995, 80 vouchers had been disseminated to the officers, and 76 (95 percent) of the them had been used to purchase units, which were turned over to the officers. The International Catholic Migration Commission’s efforts to address one of the project’s secondary goals of job skills training has shown limited results, according to USAID housing officials. As of December 1994, it had arranged training for 46 beneficiaries who attended business courses in Pskov, Novgorod, and Volgograd. The USAID official said construction delays and the subsequent delays in officers relocating to their respective cities affected start-up activities. Further, the official said the Commission did not adequately identify the officers’ training needs and failed to recognize that many of them were not interested in training. Project officials said only minimal progress was made in addressing other secondary goals, such as demonstrating new housing technologies, expanding customer choices, and implementing more stringent quality control standards. For example, in Tula, contractors constructed 14-, 16-, and 30-unit duplexes, which took as long or longer to build than traditional high rise structures. (See fig. III.2.) PADCO field representatives worked with local builders to ensure that quality control measures were introduced and achieved. The officer resettlement pilot project accomplished its objective of providing housing to demobilized officers. The project was not designed to address systemic reform or to be sustainable, and it did not do so. PADCO officials said the attempts to sustain the effects of the project’s secondary objectives were short-lived, although the lessons imparted by PADCO—new housing technologies, housing choices, and quality control measures— may have some positive effect on the building industry and contractors. USAID and Urban Institute officials said the lessons learned from implementing the voucher certificate activity by the banks, realtors, and local governments may be used to facilitate the local governments’ transition to a private housing market. As a result of the pilot project, USAID incorporated the lessons learned as it designed the $160-million follow-on initiative to provide 5,000 units to officers from the Baltics. The primary changes included (1) obtaining total support, involvement, and buy-in from all three levels of Russian government; (2) using the voucher certificate program to expedite the relocation of 2,500 officers; (3) stipulating that a maximum of 10 percent of the demobilized officers could come from local jurisdictions; (4) using a U.S. construction management firm as the prime construction contractor and subcontract to the individual builders; (5) using only experienced, well-connected Russian builders; (6) selecting partially completed buildings and sites with existing infrastructure; (7) using a traditional Russian housing design; and (8) providing a more realistic per unit cost ($25,000 versus $8,500). According to USAID officials, these changes are expected to allow the follow-on project to proceed more quickly and efficiently than the pilot. USAID/Moscow had management responsibility for the project and generally did a good job of managing, monitoring, and overseeing it and coordinating with USAID/Washington. PADCO officials said USAID/Moscow actively assisted the contractors in reaching acceptable compromises with government officials and contractors. The USAID/Moscow project team reviewed project status reports, visited project sites, and held regular meetings with contractors. Finally, AID terminated work when problems could not be overcome. The following are GAO’s comments on USAID’s letter dated June 1, 1995. 1. We have incorporated these comments into the report where appropriate. 2. Although we noted project shortcomings, we also recognized the contribution Deloitte & Touche made toward the privatization process and considered the project a success. Moreover, we recognized USAID’s positive contribution to the overall privatization effort. 3. We conducted a detailed review of Tri Valley Growers’ performance, one of the three contractors responsible for implementing the agribusiness partnerships project, to determine whether this expenditure of funds had any sustainable impact. We concluded that it did not. Although we did not draw any conclusions about the agribusiness partnerships project as a whole, our analysis casts doubt on whether the project can have a systemic impact if the individual partnerships are not having an impact. (See comments 29 and 30 for additional discussion.) 4. It is too early to know whether USAID’s prediction concerning the outcome of ongoing activities in the energy sector will result in significant sector reform. Many of these projects are just starting and must overcome many obstacles. For example, in our September 1994 report on nuclear safety, we reported that there are no guarantees that the international assistance effort will result in safer reactors or expedite the closure of the riskiest reactors. In fact, in the absence of a commitment to close down the reactors, the assistance may encourage their continued operation. We noted that donor countries face formidable challenges in promoting the closure of the Soviet-designed reactors because the countries operating them depend on the nuclear power to meet their needs for domestic energy and export income. 5. We agree that the new evaluation system is promising in that it should provide an improved basis to evaluate USAID’s programs in the NIS. However, since the first report is not due until November 1995, it is too early to know whether the system will fulfill its promise. The value of the system will depend on the indicators selected, the reliability of the data, and the subjective judgments of USAID officials preparing the reports. For the system to have credibility, USAID will have to be able to identify shortcomings as well as successes. 6. We have modified the report to reflect this information. 7. We were able to reconcile obligations and expenditures in the USAID financial report with other USAID documentation. Accordingly, we have deleted the examples from the report. 8. Our draft report included information on the work of the Consuls General. We have modified our report to update the information on increased site visits. 9. Although market forces played a role in the limited use of some of the centers, the lack of local support as well as other factors also caused the low activity levels at some centers. More importantly, it is questionable whether USAID should spend funds on activities without a market unless it has a strategy to create demand for the product it is financing. 10. We visited only one site (in Siberia during February) because of the limited amount of time we had in country. Vorkuta can be reached by plane in the winter. USAID can visit the sites at other times during the year. We believe that three site visits—two occurred after we began our audit—in 31 months is inadequate for monitoring purposes. Day-to-day contacts with PIER staff are important; however, they do not substitute for site monitoring or provide USAID with an objective basis for evaluating the project’s success. 11. The accomplishments noted in our report are those that have had the greatest impact. PIER did not provide us with any statistics that indicated increased mine safety or productivity. Moveover, a September 1994 study produced by the U.S. Department of the Interior indicates that mine safety in Russia is actually getting worse. The beneficiaries we spoke with indicated that they were implementing new mining methods introduced by PIER; however, they did not mention any measurable increases in productivity. In addition, although productivity and efficiency are important, overall production for the coal sector is still too high. Finally, PIER’s Moscow director stated that this project has had the greatest impact in the areas of restructuring, private sector involvement, and social safety net development. 12. Our report does not state that an interim evaluation is imminent and may lead to activities being redirected. Our report states that “USAID management admitted that no annual assessments or midterm evaluations were conducted,” even when required by the cooperative agreement. 13. USAID’s new procedures did not affect the program during the time frames we reviewed. Also, the work plan example should be taken in the context that several iterations of the plan have been submitted and revised since November 1994. 14. Our draft report did not recommend that more authority be delegated. 15. Our report was modified to show that the Ministry of Environmental Protection and Natural Resources was involved in the initial selection of project activities. However, the Ministry did not participate in designing the projects as USAID suggests. USAID acknowledges the almost immediate shift of its relationship from the central to the local government once the projects were selected. We remain concerned over the lack of federal involvement, especially regarding the provision of resources and the limited potential for replicability. We believe that without outside funding or support from the federal level, sustainability and replication will be difficult. 16. As indicated in the report, we found that as of February 1995, the Far East project had contributed little to systemic reform and is unsustainable without outside funding. The report discusses the project’s attempt to address systemic reform through efforts to maintain and restock the forestry base. 17. As indicated in the report, the deliverables identified in the delivery orders generally cited reports and studies as the results. We are unable to verify USAID’s statements regarding the project’s results. 18. Although we recognize that this project was only one of many in the energy sector, we found that the project is unlikely to contribute to systemic reform because of its design and the lack of monitoring and follow-up by USAID. 19. USAID suggested that we select this project in part because USAID represented it as a success, based on an independent evaluation. We visited only Yekaterinburg for two primary reasons: USAID suggested that it was a good site to visit and it was one of only two cities where equipment was installed and extensive energy audits and training were provided. However, as we reported, the equipment had not been installed. USAID’s assertion that our conclusion is based almost entirely on the site visit is wrong. Our conclusion is also based on discussions with representatives from RCG/Haggler Bailly, Joseph Technology, Honeywell, and USAID officials responsible for the project and our review of numerous documents on the entire project. 20. We did not make statements or draw conclusions about other projects in the program or about the overall program. We noted that there was no indication that the project we reviewed had contributed to systemic reform. The energy efficiency audits and demonstration sites can only have an impact on systemic reform if USAID ensures that (1) equipment is installed; (2) equipment and training is used; (3) the recommendations in the energy audits are implemented; (4) the results of the project are monitored, recorded, and publicized; (5) appropriate personnel have access to the demonstration sites; and (6) problems such as lack of equipment certification are corrected. At the time we conducted our fieldwork, USAID was not ensuring any of these elements because the project did not include any mechanisms for long-term monitoring or replicating the project. USAID, in its comments, acknowledged that the project alone is unlikely to have an impact on systemic reform. 21. We agree that the dollar value of the uninstalled equipment constitutes a relatively small percentage of all the equipment purchased. The fact that USAID was unaware of the problems in Yekaterinburg and had not monitored whether any cost savings had been achieved and did not know whether any systemic improvements had resulted from the equipment, energy audits, and training provided is the issue. 22. The consultant’s evaluation indicated that all of the equipment was installed in April 1993. The documents we reviewed indicate that the equipment was provided in April and May 1993. If the equipment did not arrive until June 1993 as USAID suggests, USAID should have known the evaluation had problems when it indicated that all the equipment was installed 1 or 2 months before the equipment ever arrived. 23. We contacted individuals who were identified as participants by USAID/Moscow. 24. Our conclusion that the training was irrelevant was not based on our discussions with the participants of the PIET training courses. It was based on statements by various USAID officials, including the USAID/Moscow Mission Director and the Chief of USAID/Washington’s Europe and the Newly Independent States/Health and Population Office. For example, USAID officials told us that a 2-week training course without follow-on activities could not be expected to result in any systemic reform. In addition, as we also noted in the report, because USAID had not conducted any long-term monitoring of the participants, it had no evidence that any of the participants instituted systemic changes based on the training. The opinions and views of the participants we interviewed were used to provide insight as to why no systemic changes had occurred. 25. Contrary to USAID’s comments, we referred to the course evaluations in our draft report. We stated that “. . . most participants were satisfied with the course and believed it was applicable to their work conditions.” However, our assessment was not concerned with whether the participants were satisfied with the course but with whether the goals of the PIET contract and the Freedom Support Act were fulfilled. 26. We question USAID’s assertion about the actual positive impact of the training and follow-on assistance. First, at the time of our review, follow-on assistance was not planned to begin for another 6 months and no assessment had been completed to confirm or deny USAID’s assertion. Second, despite repeated USAID statements that the Siberian participants were involved in follow-on projects, USAID project officials did not know how many were actually involved. When the contractor compiled this data for us, the USAID project official in charge of the program was surprised that 75 percent of the participants in Siberia were not involved with the planned follow-on activities. Finally, USAID health officials we spoke with were unanimous in their assessment that the follow-on project’s progress to date has been a disappointment. 27. Contrary to USAID’s assertion, we met with the Russian Privatization Center (RPC) official responsible for the project. This official questioned the competence of some of the consultants. We also believe that the characterization of this project as a qualified success is an overstatement. Project task orders were never modified, thus the focus of the project remained to increase the availability of commercial real estate. However, after project completion, large amounts of commercial real estate continued to be leased in the selected cities under conditions that encouraged inefficient use, and the municipalities failed to maximize revenues. At our most recent meeting, on June 2, Jay Kalotra presented a preliminary draft of the wrap-up memo for the project. At that time, I reminded Jay several times that deviations from the Perm’ model would have to be rigorously defended to both USAID and the senior management of the RPC. Jay’s response was that IBTCI deviated from the Perm’ model in large part because the model was ill-suited to the chosen roll-out cities. To a considerable extent, this may be true. However, this obviously does not exonerate IBTCI, since their task was to find cities where a pure roll-out could be performed. [underscoring supplied] In its most recent memos, IBTCI suggests that they told us at the outset they would adopt a broader approach than BCG took in Perm’. While it is true that IBTCI states some very ambitious goals, it is disingenuous to suggest that we agreed to replace the Perm’ model with something else. . . . . . . However, as noted above, even three weeks ago IBTCI was maintaining the pretense of Perm’-style results. And it was only when we actually visited the project sites that we could see the extent to which deviations had occurred. It is not realistic to expect the agribusiness partnerships program to have a significant, measurable impact on overall food systems in the NIS. The limited number of partnerships being supported suggests such national level impacts are unlikely during the life of the activity, if ever. USAID/Moscow staff stated that the agribusiness partnerships project could not by itself influence systemic reform. 31. Although TVG has established three partnerships in Russia, only one (Petoseed) is functioning. According to TVG staff, CTC Foods has run into financial problems; consequently, its potato processing facility may never be constructed. Finally, the third partnership (Big Sky Foods Trading, Inc.) has only recently been approved, and it is too early to determine whether this project will be successful. 32. USAID staff in Moscow and Washington characterized the project as discontinued because no more partnerships can be introduced and no more funds will be obligated. 33. USAID/Moscow staff said TVG had not performed adequately and had not identified appropriate partnerships. The documents we reviewed also indicated that TVG was not performing well. 34. Our report includes examples of the causes for delays in the approval process, including the need to deal with legal issues. 35. We modified the report to reflect this foreign policy goal. However, the primary objective for the pilot project announced in April 1993 did not focus on relocating officers from the Baltics. The announcement made in July 1993 focused the program on the removal of officers from the Baltics. 36. USAID is incorrect in stating that the oblast’ was involved in signing the memorandum for Novosibirsk. It was only signed by USAID and the municipality of Novosibirsk. Louis H. Zanardi Eugene D. Beye Edward J. George, Jr. Peter J. Bylsma David M. Bruno Jodi McDade Prosser The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (301) 258-4097 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Summary: Pursuant to a congressional request, GAO reviewed the Agency for International Development's (AID) assistance projects in Russia, focusing on whether: (1) individual AID projects met their objectives and contributed to systemic reforms; (2) the projects had common characteristics that contributed to their successful or unsuccessful outcomes; and (3) AID adequately managed the Russian projects. GAO found that: (1) some of the projects reviewed fully met or exceeded their objectives, while other projects met few or none of their objectives; (2) three AID projects contributed to fundamental structural changes in Russia because they had sustainability built into their design and they focused on national or regional issues; (3) the successful projects had broad and strong support from all levels of the Russian government, U.S. contractors with long-term physical presence in Russia, a broad scope to maximize benefits, and specific sustainability objectives, and complemented or supported Russian initiatives; (4) Russian officials' commitment to reform in certain sectors was critical to project success; (5) the unsuccessful projects were poorly designed and implemented and often had little or no impact on problems; (6) AID made certain exceptions to its normal procedures and processes in its desire to respond quickly to assist Russia; and (7) AID failed to adequately manage some projects because of problems in delegating management and monitoring responsibility to the Moscow AID office, inadequate staff, and inadequate management information systems.