International Exchange Programs: Open World Achieves Broad Participation; Enhanced Planning and Accountability Could Strengthen Program
Released: 2004-03-17T00:00:00

Letter

Background

While Congress originally envisioned Open World as a vehicle to bring Russia’s emerging political leaders to the United States, Open World has recently been authorized to expand the scope of its program. As shown in fig. 1, Open World has launched pilot programs in Lithuania, Ukraine, and Uzbekistan, with the first groups of about 50 participants from each country visiting the United States between December 2003 and February 2004. While Congress also made 11 other countries in the Newly Independent States and Baltic states eligible for funding, as of February 2004, there were no plans to extend the program to these countries.

Open World is governed by a Board of Trustees and works with numerous partners to carry out the program. U.S. embassies play a key role by nominating individuals for the program, vetting applicants for final selection, and processing visas for participants. In addition, Open World has contracts with several organizations, such as the American Councils for International Education, which provides logistical support, and Project Harmony, which coordinates alumni activities, along with a network of 26 Open World alumni coordinators throughout Russia. Grants are awarded to U.S. national host organizations that, in conjunction with local partners, develop programs for participants and arrange home stays. (See app. II for a list of national host organizations in 2003.) The following chart illustrates Open World’s program operations and activities, as well as the entities involved in carrying out the program.

Funding and Support

Congress appropriates an annual amount for Open World, which has ranged from $8 million in fiscal year 2002 to $13.5 million in fiscal year 2004. In addition, Open World is authorized to seek and accept private donations and reports that it has received current gifts and pledges of about $2 million. Figure 3 shows the program’s total expenditures (unaudited) for fiscal years 1999-2003 and expenditures by major category during the same period. Based on data for the program’s expenditures and the number of participants for 2003 provided by Open World, we estimated the average cost per participant, including facilitators, to be about $6,200.

When the program was established as an independent entity in December 2000, Congress provided the program with additional authority and support, including the authority to receive donations and appoint an executive director and to establish a trust fund in the Department of the Treasury to be credited with appropriations and donations approved by the program. In addition, Congress authorized the Library of Congress to provide the program with support services, including the ability to disburse appropriated funds; pay the program’s personnel; and provide administrative, legal, financial management, and other services. The Library was also authorized to collect the full costs of the services from the program’s trust fund. To formalize this arrangement, the Library and Open World entered into an interagency agreement. In addition to providing for support services, the agreement enables Open World to use Library personnel to conduct the program and the Library to recover the related salary and benefit costs of such personnel. According to Open World officials, the Library currently has 14 established positions assigned to assist the center in conducting its program.

As provided for in the interagency agreement, the Library of Congress provides financial management services to the Open World Leadership Center. Open World officials review and approve financial management documents before submitting them to the Library of Congress for processing. Much of Open World’s in-house financial management activities are performed by a financial management consultant who makes recommendations to both the Executive Director and the Program Administrator regarding approval of program disbursements.

Types of Exchanges

In an effort to reach emerging leaders in various sectors, the program for Russia focuses on three types of exchanges—parliamentary, civic, and cultural. Parliamentary visits match members of Russia’s two houses of parliament—the Duma (the lower house) and the Federation Council (the upper house)—with host U.S. senators, representatives, and governors. The civic program in 2003 featured eight themes: economic development, education reform, environment, federalism, health, rule of law, women as leaders, and youth issues (see fig. 4). It targets, among others, government officials and civic leaders at all levels, with an emphasis on regional and local levels, and other community leaders. The new cultural program for Russia is designed specifically for cultural leaders, including museum professionals, visual and performing artists and administrators, and librarians.

The typical exchange program runs for 10 days, including a 2-day orientation program upon arrival in the United States. Delegations usually consist of five people—four delegates and one paid facilitator who acts as a “bridge” between the Russian delegates and their American hosts. Most participants stay in private homes of American host volunteers for some or most of their local visit—a special feature of the program. In addition, unlike some other U.S. exchange programs, English is not a requirement for the Open World program. As a result, Open World has been able to send participants from each region of Russia to the United States—most for the first time.

Program Has Reached a Large, Diverse Audience, but Measuring Improvements in Mutual Understanding Is Difficult

As of December 2003, Open World reported bringing 6,800 Russian delegates from seven geographic regions to visit over 1,200 communities in all U.S. states. The percentage of delegates from each region is roughly comparable to the proportion of the Russian population that each region represents. Figure 5 illustrates the representation of delegates, in terms of the number and percentage, from each region in Russia and the number of delegates that traveled to each state within the United States between 1999 and 2003.

Fifty-eight percent of the delegates in 2003 were women. The average age of Open World delegates in 2003 was 39 years. As shown in figure 6, 77 percent of the delegates in 2003 were 45 years of age or younger.

Delegates come from a wide variety of academic and professional backgrounds. Ninety-four percent of 2003 delegates reported having completed higher education, 12 percent had the equivalent of a master’s degree, and 1 percent had the equivalent of a doctorate. Their degrees span a wide spectrum, ranging from the fields of law and medicine to agriculture and journalism. Fifty-eight percent of the delegates reported that they had authored publications. Professions varied from Duma members and judges to leaders of nongovernmental organizations. Twenty percent of the delegates reported they were elected officials. Open World records as of February 2004 showed that 140 members of the Duma and 20 members of the Federation Council—representing about 31 percent and 11 percent of the current Duma and Federation Council, respectively—have traveled to the United States through Open World. Also, 577 Russian judges have participated in the program. In addition, Open World officials noted that Russia’s diverse ethnic groups were substantially represented among program participants.

Stakeholders Generally Cite Positive Impacts of Program

Based on our analysis of responses to participant surveys conducted by Open World, as well as our interviews with Open World alumni in Russia, delegates generally hold highly favorable views of their experience in the program. Almost all of the delegates reported that the program was useful and had partially or completely met their expectations. They also reported a greater willingness to cooperate with Americans as a result of the exchange. While overall comments were positive, some delegates cited not having enough time to establish business contacts.

Delegates Give Open World High Marks Overall

Our analysis of Open World questionnaires that surveyed delegates for their experiences in and immediate reactions to the program, and our own interviews of past delegates, showed that Open World program alumni hold highly favorable views of their exchange experience. Almost all of them reported the program was either probably or definitely useful to them.

Many alumni with whom we met offered concrete examples of actions they had taken to implement what they learned from their U.S. visits in the context of the Russian environment. For example, several members of the Russian Ministry of Internal Affairs in Moscow used information they gained from their visits with State Department officials and nongovernmental organizations to draft legislation prohibiting the trafficking of women. The Chairman of the Judicial Council in Moscow and several judges at the Supreme Court in Petrozavodsk told us that they established court management structures modeled after U.S. courts and developed a judicial code of ethics in consultation with American judges. Librarians in Petrozavodsk told us that, after returning from the United States, they helped to establish a library association in the Republic of Karelia that is similar to the Russian Library Association, a counterpart to the American Library Association. Another alumnus joined the Rotary Club in Moscow after his Open World experience and was recently elected the president of his chapter. One woman said that upon learning about fund- raising efforts and philanthropy of private organizations in the United States, she began soliciting donations for her nongovernmental organization from private businesses in Russia.

In addition, almost all alumni said that since the exchange, they had contacted other Open World alumni in other regions within Russia. Many of them had been invited to and had attended alumni conferences or other alumni events such as computer training seminars and professional development workshops. In addition, over half the alumni we interviewed had used the Internet to view Open World’s Web site or keep in touch with contacts made during their exchange experience. The American Corners facilities that we visited in Petrozavodsk and Samara, whose directors had participated in Open World, have become a hub for alumni who use the facilities’ computers for Internet access and other services.

The majority of the alumni we interviewed said that their views of the United States changed in some way after the exchange. For example, one alumnus said she was unable to accurately visualize the United States before participating in the program and had developed the impression from Cold War propaganda that Americans had few opportunities and little hope for the future. Some alumni expected Americans to be hostile and were surprised by their friendliness.

Open World surveys showed that, for the most part, delegates reported that the program improved their understanding of American institutions to some degree. For example, in 2003, a large amount of delegates (74 percent) believed that their visit improved their understanding of ethnic and cultural diversity in the United States, as well as their understanding of democracy (74 percent), role of Congress (68 percent), higher education system (70 percent), legal system (67 percent), freedom of speech (62 percent), market economy (50 percent), and role of religious organizations (52 percent). For example, among the past delegates we interviewed, one was impressed with the religious diversity in America when he observed Amish communities in Pennsylvania. Another delegate was impressed by the large Russian immigrant population in Brighton Beach, New York. Many delegates were surprised by how open and transparent American government institutions are to the citizenry, citing, for example, the openness of and public access to city council meetings, congressional sessions, courtrooms, and public hearings.

According to Open World’s 2003 surveys, 89 percent of delegates reported they probably or definitely expect a positive long-term impact as a result of their visit to the United States. For example, 88 percent of delegates reported that, as a result of the exchange, their readiness to cooperate with American leaders had risen. Eighty-six percent also reported that their trip improved relations between Russian and U.S. citizens. Fifty-four percent indicated that they extended an invitation to their American counterparts to visit them in Russia during the next year. Seventy-six percent said they plan to stay in contact with persons they meet in the United States.

Some Disappointment about Contacts

While overall comments in Open World’s surveys were positive, some delegates cited areas that could be improved. For example, 34 percent of delegates in 2003 indicated they had not had sufficient time to establish individual business contacts with their professional counterparts. In addition, 34 percent indicated they had not had sufficient time for individual consultations on professional issues. Although the majority of the alumni we interviewed said they were satisfied with the contacts they made during the program, a few of them wished they had met with higher- level officials. For example, a deputy chief at the Moscow State Duma reported Open World does not facilitate enough contacts with high-level decision makers. Another said that because he did not meet with his American counterparts, he had the impression they did not have an interest in meeting Russians. Others said the program was simply too short to meet with everyone they had hoped. Nonetheless, Open World’s surveys indicated that 56 percent of alumni felt that a 2-week program would be optimal. Open World officials said they considered 10 days as a reasonable amount of time to expect participants to be away from their jobs.

Open World Seen as Valuable Tool to Enhance, Complement U.S. Mission Activities

Ambassadors and embassy officials with whom we met noted that Open World complements U.S. mission activities and enhances outreach efforts—citing, in particular, the program’s alumni as a valuable resource. Although the U.S. mission offers several State Department-administered exchange programs, none of these individual programs brings Russians to the United States on the same scale as Open World, particularly from the remote regions of the country. (See app. III for a list of selected State exchange programs.) According to U.S. mission officials in Russia, including those at the consulate in St. Petersburg, it is at locations outside of the major cities that Open World has an advantage and can best target potential delegates for the program who have not yet traveled to the United States on other U.S. exchange programs. In fact, when visiting various regions within the country, embassy officials find it especially useful to meet with Open World alumni, many of whom are in leadership positions. For example, while visiting Volgograd, the U.S. ambassador met with Open World alumni who shared some examples of direct results of their exchange experiences. One alumnus started several youth programs in his district, while another started public information programs on healthy lifestyles directed at Volgograd youth.

In addition to enhancing outreach efforts, State officials in Washington, D.C., and U.S. embassy officials with whom we spoke in Russia said Open World complements other U.S. mission activities. Similarly, Open World is able to build on relationships fostered by other U.S.-Russia assistance activities to further its own program. For example, the U.S. Agency for International Development (USAID) worked through Open World to send more Russian judges to the United States than it could have funded on its own, as part of the Vermont-Karelia Project. This project was initially established to bring representatives of the judiciary of Karelia, with which the judiciary of Vermont has a long-standing working relationship, to meet their counterparts and learn about the U.S. judicial system. The program has since grown to include an additional six Russian regions and six U.S. states and is now called the Russian-American Rule of Law Consortium, of which the Vermont-Karelia Project is a part. Also, the State Department targets Open World alumni for follow-up technical assistance and training upon their return to Russia. Thus, although Open World does not bring Americans to Russia, under State or USAID sponsorship, some American judges who had hosted Open World delegates later visited Russia to provide technical assistance and training.

Embassy officials and State officials in Washington, D.C., noted that, although there are other independent entities within the executive branch that carry out international activities such as exchange programs, Open World is the only exchange program within the legislative branch. (For illustrative purposes, app. IV provides information on some independent entities funded through the executive branch.) The officials told us that the program’s independent status and current placement within the legislative branch offered some advantages, noting that congressional sponsorship of Open World lends a certain cachet to the program and allows it to attract emerging leaders who otherwise might not participate. The officials also said that congressional involvement was important to sustaining the support of Congress and other decision makers.

Program Has No Formalized Strategy by which to Measure Progress

Although Open World does deliberate and decide on programmatic themes and target audiences that it would like to emphasize each coming year, it does not have formalized strategic and performance plans that define success, what it will take to succeed, and how it should be measured. Without a framework that identifies long-term goals, explicitly links them to U.S. mission priorities and plans overseas, and systematically identifies the incremental outcomes expected at each step, along with measurable indicators of such progress, it is difficult to gauge whether Open World is targeting and reaching the right people, whether it is providing delegates with the right types of experiences, and whether these experiences are resulting in improved mutual understanding. This also makes it more difficult for Open World to adjust its course of action, when necessary, and to determine whether it is using its resources in the most efficient and effective manner. Also, although Open World surveys all delegates on their experiences in and immediate reactions to the exchange program, it does not systematically compare delegate attitudes and knowledge both before and after their participation in the program. Open World has also administered several different alumni surveys; however, these surveys are of limited value in gauging improvements in mutual understanding.

Open World does provide nominating organizations with general criteria for determining a person’s eligibility for the program. To further screen nominees, the vetting committee considers such factors as a person’s active involvement in politics, the community, or teaching; the number of publications issued; the number of people the person supervises; whether the person is from outside the capital; and any prior visits to the United States to make subjective judgments about the applicant’s potential to influence change and apply the experience gained from participating in the program. Nevertheless, it is difficult to determine whether these are the optimal criteria for any given year, or whether they are being met, without explicit and measurable performance targets that are designed and sequenced to mesh with a larger strategy for achieving Open World’s long- term goals.

Similarly, the lack of a strategy makes it difficult to assess whether delegates are gaining the desired experiences from their involvement in the program. We found varied responses among the past alumni we interviewed regarding what would constitute an optimal mix of experiences. Some favored a more focused approach involving training that is narrowly targeted toward specific professional needs. Others said that a broad exposure to the United States and its institutions is all that can be expected during a 10-day visit. One program nominator said that the program should consist of two separate trips: On the first, delegates would simply gain an insight into the American political and economic systems, while the second trip would be more focused on specific professional experiences. Without an explicit strategy that links particular target groups with specific program content, approaches, and timing, it is difficult to determine whether the experiences that delegates are gaining are optimal at any given time.

Open World Surveys Are of Limited Value in Measuring Progress

Open World conducts surveys that attempt to capture delegates’ experiences in and immediate reactions to the exchange program, including their impressions about whether their attitudes had changed as a result of their participation in the program. However, it does not systematically compare delegate attitudes and knowledge both prior to and following their participation. Open World has also administered alumni surveys; however, these surveys were not designed to determine the long- term impact of the program, including whether improved mutual understanding has occurred. While measuring the impact of exchange programs is difficult because the full effects of such programs may not be known for years, Open World officials agree that such an evaluation is necessary and hope to conduct one in the near future.

Program Surveys

American Councils conducts three types of surveys for the Open World program—application, predeparture, and postprogram. The application survey primarily contains descriptive information regarding the applicant, such as place of residence and occupation. The predeparture survey, filled out just before the participant leaves for the United States, contains additional descriptive information, including the participant’s age, ethnicity, educational profile, employment, and views on democratic values. The postprogram survey, which is filled out immediately after the exchange program, contains information on delegates’ exchange experiences and their impressions of how the program affected them. These three surveys had very high response rates, all exceeding 90 percent.

Overall, Open World surveys do an adequate job of measuring delegates’ experiences and immediate reactions to the exchange program. However, the surveys miss the opportunity to measure whether a delegate’s attitude toward the United States and its institutions changed as a direct result of participation in the program by not measuring pre- and postparticipation attitudes using parallel questions. The postprogram survey asks retrospective questions about whether delegates’ attitudes changed and whether their expectations were met. From a methodological standpoint, this approach is useful but not as rigorous as measuring attitudes and expectations before and after the program because it relies on the delegates’ accurately recalling how they felt before the exchange program. Open World staff told us they had not regularly analyzed responses to the surveys for evaluative or management purposes; however, they have recently embarked on an effort to redesign the surveys to use them for these purposes.

Alumni Surveys

Open World alumni have been surveyed, but these surveys were primarily designed to aid in program management, not to measure the long-term impact of the program. In 2000, American Councils conducted a survey to gain constructive feedback on what 1999 alumni found useful about the program and to obtain their opinions on what kinds of people should be considered as future program candidates. This survey was distributed through alumni networks and had a response rate of less than 30 percent— too low to be representative of total delegates. In 2002, American Councils in Moscow administered a survey to elicit ideas from alumni on how the program could be improved and to prepare them for upcoming alumni activities. However, Open World officials reported that the survey was not helpful because the answers were too general or vague and contained few suggestions for cost-effective improvement. Open World officials informed us they conducted an alumni questionnaire in December 2003 to assist Open World in planning future activities for alumni, but as of January 2004, the results had not been analyzed. Open World and American Councils staff acknowledged that a full program evaluation of alumni to determine the program’s progress over the long term was necessary and that they hope to conduct such an evaluation in the future.

Open World Lacks Formalized Financial Management and Accountability Mechanisms

Open World does not have the formalized financial management and accountability mechanisms—formalized policies, audited financial statements, an audit or financial management advisory committee, or full program data—that would provide Congress and other decision makers with the timely, reliable cost and performance information that is especially important for a permanent, expanding program. Although Open World has established procedures for reviewing and approving transactions and analyzing financial reports, these procedures have not been formalized in written policies. For example, Open World has procedures for reviewing budget submissions that accompany grant applications, for awarding grants, and for reviewing grantee expenditures. It also has procedures for analyzing reports on program payroll and outstanding obligations. However, it has not evaluated whether these procedures provide adequate internal control or codified them into management-approved policies that Open World staff are required to follow. Documentation of policies and procedures covering an entity’s internal control structure and all significant transactions and events is fundamental to ensuring that all staff understand and consistently apply procedures, while management assessment of these procedures is an essential component of internal control. Management evaluation of controls, along with approval and documentation of procedures, is particularly important when financial management services are being performed by a contractor. According to federal government standards for internal control, written policies and related operating procedures should address key control activities such as approvals, verifications, reconciliations, and the creation and maintenance of related records that provide evidence and appropriate documentation of these activities.

The lack of formal policies, particularly in the grants management area, may leave some critical elements of grantee accountability inadequately addressed. For example, as long as the total grant amount is not exceeded, Open World allows up to a 10 percent variance between the actual and budgeted amounts by budget category on an approved grant, but it does not require grantees to report such variances as part of their reporting of grant expenses; it also does not have follow-up procedures to deal with variances of more than 10 percent. Also, according to Open World officials, grantees are required to submit receipts or other evidence for all grant expenses unless Open World agrees, as part of the grant agreement, to permit a grantee to submit the results of its “single audit” conducted pursuant to OMB Circular A-133. However, the officials acknowledged that Open World does not have a formal policy that clearly defines the conditions under which it will accept the results of a single audit in lieu of a grantee’s submission of receipts or other evidence for all grant expenses. For example, in 2003, Open World began requiring, as a condition for accepting the results of a single audit, that an audit’s coverage include a “significant sample” of the costs incurred under the Open World grant. However, Open World has neither defined what audit coverage represents a “significant sample” nor implemented procedures to ensure that the requirement has been followed. According to program officials, Open World has permitted only a few grantees to submit single audit results in lieu of submitting receipts and other evidence of grant expenses. However, as the program expands, it may become difficult to manage the detailed review of supporting documentation for grantee expenditures; thus, Open World’s use of audit reports as an oversight mechanism could increase.

Financial Statements Are Not Being Prepared and Audited

Open World obtains detailed accounting reports for the program from the Library of Congress. In addition, Open World has prepared for the Board of Trustees various schedules that separately present the program’s planned budget and actual obligations. However, Open World does not currently prepare summary financial statements that are subjected to independent audit and used by the board in its oversight. Program officials plan to prepare financial statements for Open World and initiate an audit by the summer of 2004. As discussed in our executive guide on best practices in financial management, a solid foundation of control and accountability requires a system of checks and balances that provides reasonable assurance that the entity’s transactions are appropriately recorded and reported, its assets protected, its established policies followed, and its resources used economically and efficiently for the purposes intended. This foundation is built and maintained largely through the discipline of routinely preparing periodic financial statements and subjecting them annually to an independent audit. In fact, the April 2003 bylaws of the Open World Board of Trustees require an annual audited financial statement for the Open World Leadership Center Fund. This requirement further underscores the importance of Open World developing formal financial management policies. The auditor would use the financial management policies and any related operating procedures to gain an understanding of and evaluate Open World’s internal control environment.

Governance Structure Does Not Include an Audit Committee

Open World’s governance structure does not include either an audit committee or financial management advisory committee to provide the Board of Trustees and management with independent advice on financial management, accountability, and internal control issues. Such a committee is a required element of the governance structure of publicly owned companies and a best practice for other types of organizations. The audit committee of a publicly owned company plays a particularly important role in assuring fair presentation and appropriate accountability in connection with financial reporting and related external audits and general oversight of an organization’s internal control. In the federal government, audit committees and advisory committees are intended to protect the public interest by promoting and facilitating effective accountability and financial management. This is accomplished by providing management with independent, objective, and experienced advice and counsel, including oversight of audit and internal control issues. In the case of Open World, use of an audit or financial management advisory committee could facilitate the process of formalizing financial management policies and procedures, including related internal controls, and preparing for the program’s first financial statement audit.

Value of Contributed Services Is Not Disclosed

Open World is not collecting data on the significant volume and value of contributed services from U.S. volunteers. According to Open World, 83 percent of program participants in 2003 stayed in the homes of American host volunteers, a contribution that considerably reduces program expenditures associated with housing participants during their stay. As a result, the amount expended by Open World does not reflect the full scope and cost of operating the program. The usefulness of information on the nature and extent of similar contributed services is recognized under generally accepted accounting principles, which encourage entities to disclose, if practicable, the fair value of contributed services received. Open World could obtain data, by geographic area, on the number of program participants that stay in the homes of American host volunteers and then apply standard per diem rates to estimate the value of meals and lodging provided by host volunteers. Collecting and disclosing this information would provide management, the Board of Trustees, and Congress with more complete information about the full scope of the program.

Conclusions

Since its launch in 1999, Open World has organized large numbers of diverse delegations from every region in Russia and brought them to the United States. Most delegates viewed their program experiences very favorably, and many say they have taken concrete actions to adapt what they learned from their U.S. visits to the Russian environment. Also, U.S. ambassadors and embassy officials consider Open World a valuable tool to complement U.S. mission activities and outreach efforts and noted that congressional sponsorship of Open World lends a certain cachet to the program, allowing it to attract emerging leaders who otherwise might not participate. However, because the program does not have formalized strategic and performance plans with systematic performance measurement indicators, it is difficult to determine the extent to which Open World is targeting and reaching the right people and providing delegates with the right types of experiences, including those that result in improved mutual understanding. Also, Open World lacks the formalized financial management and accountability mechanisms that would help provide decision makers with useful, relevant, timely, and reliable information. Open World began as a pilot project and was not established as an independent entity until 2001. Now that Open World has permanent status and is expanding its scope, it is appropriate for the program to turn its attention to enhancing its strategic and performance planning and financial management and accountability mechanisms. Such mechanisms are particularly important to ensure that Open World’s efforts and the related activities of embassies, contractors, grantees, and nominating organizations are systematically integrated and managed to achieve measurable progress toward Open World’s fundamental goals. Strengthening these mechanisms will become even more important as the program further expands.

Recommendations

To enhance Open World’s management, particularly in light of the program’s expansion, this report makes recommendations to the Chairman of the Board of Trustees of the Open World Leadership Center to (1) establish strategic and performance plans that articulate Open World’s direction and set measurable goals and indicators; (2) strengthen the program’s mechanisms for collecting data and reporting on program performance; (3) assess whether the current procedures provide adequate internal control over expenditures and grantee oversight; (4) develop and implement written, management-approved policies, procedures, and internal controls for Open World’s resources and expenditures; (5) develop and implement controls and requirements for grantees to provide accountability for grant expenditures to ensure that funds are spent for their intended purposes; (6) develop and implement plans for routinely preparing financial statements that are annually subject to an independent audit; (7) consider establishing an audit committee or financial management advisory committee to provide the Board of Trustees and management with independent advice on financial management, accountability, and internal control issues; and (8) estimate and disclose the value of contributed services from U.S. volunteers to better reflect the total scope of the program.

Agency Comments and Our Evaluation

Open World provided written comments on a draft of this report (see app. V). Open World generally concurred with the report’s observations, conclusions, and recommendations and acknowledged that the time has come for strategic planning and for considering options to strengthen the program’s administrative operations and financial reporting. Open World said that it is proceeding with measures to implement some of these recommendations. These measures include proceeding with plans to develop strategic and performance plans, review the program’s data collection efforts, and prepare financial statements and subject them to an independent audit. In response to other recommendations, Open World said it would ask the Board of Trustees to consider forming an audit committee for the board, evaluating the in-kind contributions of the program’s American volunteer hosts, and implementing a system for more regular summary financial statements for the board. However, Open World took issue with the report’s emphasis on measurable goals and indicators of success, noting that the results of its programs can only be validated in the medium or long term. Open World also said that improving mutual understanding is not a measurable, performance-based goal.

We recognize the long-term commitment required to measure the ultimate success of exchange programs. However, measuring short-term incremental progress toward a program’s goals is also an important component of any serious effort to assess progress over the long term. It is fundamental to making necessary course corrections along the way—a capability that will become even more important as Open World further expands. While it is sometimes difficult to establish direct causal links between exchange programs and their ultimate impact, we believe that establishing convincing correlations is a reasonable expectation. With respect to mutual understanding, there are a number of internationally recognized social science research and statistically valid methodologies that can be used with questionnaires, interviews, and focus groups for gauging changes in attitudes, knowledge, and behavior among exchange program participants.

The State Department also reviewed a draft of this report for technical accuracy. State’s comments have been incorporated into the report, as appropriate.

We are sending copies of this report to other interested members of Congress, the Librarian of Congress and Chairman of the Open World Leadership Center Board of Trustees, and the Secretary of State. We will also make copies available to others upon request. In addition, this report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact Jess Ford at (202) 512-4128. Other GAO contacts and staff acknowledgments are listed in appendix VI.

Scope and Methodology

To review the Open World Leadership Center’s progress toward achieving its overall purpose, we reviewed Open World’s organizational structure, operational policies and procedures, program documentation, and legislative history, and we observed key program activities, including the vetting process and a predeparture orientation in Moscow, as well as program orientation and other selected events in Washington, D.C. This provided us with an understanding of the nature of Open World activities and how they are carried out—from the time delegates are selected in Moscow to their actual visit in the United States. We also analyzed the results of program surveys that delegates completed from 2000 to 2003.

We reviewed the surveys, which were carried out by American Councils and translated into English for elements such as consistency, balance, tone, and adherence to common survey design standards. For example, we considered whether the response scales used (1) were balanced, (2) appeared to cover all possible response options, and (3) contained “double-barreled” options—that is, whether questions asked about more than one issue but allowed for only one response.

We studied the survey results to see if there was anything unusual or unexpected that might indicate potential problems in the surveys’ design or structure.

We met with knowledgeable Open World and American Councils staff, both in Washington, D.C., and in Moscow, to determine the completeness of the data set and the accuracy of required data elements. We also engaged a GAO native Russian speaker to review the translation of key questions from Russian to English.

Based on these assessments, we determined that data produced by many of the survey items were sufficiently reliable and generally usable for the purposes of our study. These data included descriptive information on program delegates, such as education level; information on participants’ satisfaction with certain aspects of their exchange experience; and opinions on how the experience affected them. While we determined that most data elements were sufficiently reliable, we did not use a few data elements that we had questions about.

We supplemented our analysis of the program surveys with interviews of 56 alumni during fieldwork in Moscow, St. Petersburg, Petrozavodsk, and Samara, Russia, in October and November 2003. We chose those four cities for our review because they represent not only three of the seven geographic regions in Russia where 60 percent of the Open World delegates have come from but also a mix of urban and rural areas, and a diverse group of alumni. Because our interviews were limited to a few locations in Russia (due to travel and time constraints), we did not generalize the results of our interviews to the universe of delegates. We recognize that the opinions and experience of the group of delegates with whom we met may not be representative of all program alumni; therefore, the results of our interviews should be used for illustrative purposes only. We conducted our interviews as follows: We developed a structured interview instrument for our meetings with program alumni with the assistance of GAO social science analysts and analysts fluent in Russian and in consultation with Open World. The interview instrument included questions regarding the contacts alumni made during their trips, changes in their attitudes toward the United States, and any actions they may have taken in Russia as a direct result of their participation in the exchange program.

We conducted individual and group interviews with program alumni from various years of the exchange program, including some who were among the early delegates in 1999 and others who participated in the program as recently as 2003.

Russian-English translators provided by the U.S. Embassy and the Open World Leadership Center facilitated the interviews.

To review whether the program has appropriate financial management and accountability mechanisms in place, We discussed the nature and scope of existing mechanisms with Open World officials.

We observed deliberations of an Open World panel that reviewed grant proposals from organizations interested in hosting Open World delegations.

We performed a “walk-through” of supporting documentation for a grant and a contractor payment.

We performed this work solely to gain an understanding of Open World’s existing financial management and accountability mechanisms, and as such, we did not conduct an audit of Open World’s financial reports or individual transactions.

To provide information on the statutory authorities and governing structures of selected independent organizations funded through the executive branch, we conducted legislative research on the purpose, statutory authority, governance, and funding of four such organizations that carry out various international activities, including exchange programs: the African Development Foundation, The Asia Foundation, the Inter-American Foundation, and the National Endowment for Democracy.

We conducted our work from July 2003 to January 2004 in accordance with generally accepted government auditing standards.

National Host Organizations

The Open World Leadership Center awards grants to U.S.-based nonprofit and governmental organizations to host visiting delegations. Some organizations carry out Open World visits themselves or through their local affiliates, while others develop and oversee a network of local organizations to provide this hosting. These local organizations include civic associations, academic institutions, and nonprofit international training providers. Open World selects its host organizations annually through a competitive grants process. Figure 7 provides information on the 16 national host organizations selected in the 2003 grants cycle, from May 2003 to April 2004, and includes hosting activities.

Russian American Rule of Law Consortium (including the Vermont-Karelia Rule of Law Project)

An organization for partnerships matching the legal communities of seven Russian regions with seven U.S. states to develop the rule of law in both countries Supports women's progress in building democracies, strong economies, and peace Focuses on improving municipal services, economic development, health, social welfare, and quality of life in targeted communities CEC ARTSLINK was formerly known as CEC International Partners or Citizen Exchange Council.

Selected International Exchange Programs for Russia and the Newly Independent States That Are Administered by the State Department

The State Department facilitates exchange programs—like Open World— with other parts of the U.S. government, the private sector, and foreign governments. State’s Bureau of Educational and Cultural Affairs is responsible for the management and oversight of U.S. international educational and cultural exchange activities, as authorized by the Mutual Educational and Cultural Exchange Act of 1961 (Fulbright-Hays Act). American embassies collaborate with the bureau in administering and supervising exchange activities. As shown in fig. 8, State offers a wide spectrum of academic, professional, and youth exchange programs in Russia and the Newly Independent States. The programs may run anywhere from 2 weeks to 2 or more years, according to State officials, and have varied in size from as few as 9 participants up to 675 participants. The State Department reported that the exchange programs operating in Russia, including those highlighted in this appendix and others, brought a combined total of more than 2,300 participants from Russia to the United States in fiscal year 2003.

Selected Independent Entities Funded through the Executive Branch

For illustrative purposes, figure 9 provides basic information, including statutory authority and governing structures, about selected grants-making organizations and entities that Congress supports. Like Open World, these programs are independent entities; however, unlike Open World, they are funded through the executive branch.

Comments from the Open World Leadership Center

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the person named above, Natalya Bolshun, Lyric Clark, Janey Cohen, Martin De Alteriis, David Dornisch, Etana Finkler, Maxine Hattery, Catherine Hurley, Ed Kennedy, Lori Kmetz, Joy Labez, David Merrill, John Reilly Jr., Mark Speight, and Heather Von Behren made key contributions to this report.

GAO’s Mission

The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e- mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail alerts” under the “Order GAO Products” heading.

Order by Mail or Phone

To Report Fraud, Waste, and Abuse in Federal Programs

Public Affairs