District of Columbia Opportunity Scholarship Program: Actions Needed to Address Weaknesses in Administration and Oversight
Released: 2013-11-15T00:00:00

Letter

Background

Laws Governing the Opportunity Scholarship Program

Congress passed the D.C. School Choice Incentive Act of 2003 in January 2004, which directed the Secretary of Education to award a grant on a competitive basis for up to 5 years to an entity or entities to use to make scholarship payments to parents of eligible students to be used for This Act private school tuition, fees, and transportation expenses.created the program known as the District of Columbia Opportunity Scholarship Program (OSP), the first private kindergarten-through-grade- 12 school choice program supported by federal funds. The purpose of the Act was to provide low-income parents of students in the District, particularly parents of students who attend public schools identified as in need of improvement under the Elementary and Secondary Education Act of 1965, as amended, “with expanded opportunities for enrolling their children in higher performing schools in the District” by providing annual scholarships to attend the private elementary or secondary schools of their choice.

In April 2011, Congress reauthorized OSP for 5 years under the Scholarships for Opportunity and Results Act (SOAR Act).

The Act sets forth requirements for participating private schools.provides that none of the funds may be used by an eligible student to enroll in a participating private school unless the participating school 1. has and maintains a valid certificate of occupancy issued by the 2. makes readily available to all prospective students information on its 3. if operating for 5 years or less, submits to the eligible entity administering the program proof of adequate financial resources and the ability to maintain operations throughout the school year; 4. agrees to submit to site visits as determined to be necessary; 5. has financial systems, controls, policies, and procedures to ensure that funds are used according to the statute; and 6. ensures that each teacher of core subject matter in the school has a baccalaureate degree or its equivalent, regardless of whether the degree was awarded in or outside of the United States.

Administration and Oversight of OSP

Education’s Office of Innovation and Improvement, which is charged with the management of the program and oversight of the program administrator, awards scholarship funds to a grantee and provides programmatic guidance and technical assistance. Soon after the D.C. School Choice Incentive Act passed in 2004, Education awarded the grant to operate the OSP to the Washington Scholarship Fund, a nonprofit organization in the District of Columbia that had experience providing privately funded scholarships to low-income students. In May 2010, Washington Scholarship Fund withdrew as the grantee and Education transferred the grant for administration of OSP to the DC Children and Youth Investment Trust Corporation (the Trust), a non-profit organization. As the program administrator, the Trust recruits new students and schools, receives and processes applications, and awards scholarship funds to program participants. The Trust is also responsible for providing oversight to participating private schools.

The SOAR Act, which was enacted about 11 months after the Trust assumed responsibility for administering the OSP grant, states that the grantee (the Trust) must ensure that scholarships be awarded through a random selection process if more eligible applicants seek admission to the program than the program can accommodate, or if more eligible applicants seek admission to a participating school than the school can accommodate, and the Trust has implemented this requirement through a lottery process. The groups of applicants that have a priority status as defined in the SOAR Act are (1) applicants who are currently attending a school that has been identified for improvement, corrective action, or restructuring under the Elementary and Secondary Education Act of 1965, as amended; (2) applicants who have a sibling currently participating in OSP; and (3) applicants who received an OSP scholarship award in a previous year but did not use it. At the time of our review, the lottery process was carried out by Westat, a contractor.

In addition, the SOAR Act updates the responsibilities for implementing OSP for the District and Education. It directs the Secretary of Education and the Mayor of the District to revise the MOU regarding the implementation of OSP that was required pursuant to the D.C. School Choice Incentive Act. The MOU governs the program for fiscal year 2011 and all subsequent years. It is intended to help ensure the efficient and effective continued implementation of the OSP in a manner that incorporates and is consistent with the roles and responsibilities of both Education and the District. According to the MOU, Education is responsible for overseeing the Trust as a recipient of federal funds and working with the Trust to ensure that it makes improvements to various aspects of program administration. The District also has a role in the administration of OSP. The MOU states that District agencies are directly responsible for conducting required building, health, and safety inspections of participating schools when notified by the Trust.

Federal Funding for OSP and Use of Funds

According to Education, from fiscal years 2004-2010, Congress appropriated between $13.2 and $14.8 million per year for OSP, as shown in figure 1. For fiscal year 2011, the appropriated amount increased to about $15.5 million, and in fiscal year 2012, the appropriated amount increased to $20 million. Most recently, Congress appropriated about $19 million for the program for fiscal year 2013. Since OSP’s creation, about $152 million has been appropriated in total.

The SOAR Act permits 11 percent of the grant funds each year to be used for certain administrative and other purposes as follows—with the remainder of the funds going towards student scholarships:

5 percent for the program’s evaluation,

The grantee (the Trust):

3 percent for administrative costs,

2 percent for parental assistance,

1 percent for student academic support services.

The law provides for an increase in the amount of the scholarship for eligible students. Specifically, it provides up to $8,000 for grades K-8 and $12,000 for grades 9-12 starting in school year 2011-2012. While tuition is paid first, any leftover scholarship funds can be used to pay for school fees—including uniforms, field trips, before and after care—and transportation expenses.

Student Eligibility and Enrollment in OSP

Id., § 3007(c).

Id., § 3007(d). thresholdbenefits (known as SNAP benefits) issued within the District. The student must also be at least 5 years old by September 30th of the school year. or that receive Supplemental Nutrition Assistance Program In the 2011-2012 school year, there were 53 participating schools in OSP (see fig. 2). These schools represent a wide range of grades, sizes, sectors, and tuition levels.

These schools are located throughout all eight wards of the District, with the greatest number of schools located in Ward 3. In accordance with District regulations for all private schools operating in the District, these schools must be accredited by, or be in the process of being accredited by, at least one of seven recognized accrediting organizations or any other approved accrediting body. Schools that are unaccredited must submit documentation to demonstrate satisfactory evidence of instruction.

Internal Controls Relevant to Implementation and Oversight of OSP

Internal control is broadly defined as a process—affected by an entity’s board of directors, management, and other personnel—designed to provide reasonable assurance that the following objectives are being achieved: (1) effectiveness and efficiency of operations, (2) reliability of financial reporting, and (3) compliance with applicable laws and regulations. Internal controls include management and program policies, procedures, and guidance that help ensure effective and efficient use of resources; compliance with laws and regulations; prevention and detection of fraud, waste, and abuse; and the reliability of financial reporting. Effective internal control is a major part of managing any organization to achieve desired outcomes and manage risk.

Standards for Internal Control in the Federal Government presents five standards that define the minimum level of quality acceptable for internal control in government and provide the basis against which internal control is to be evaluated. These standards—control environment, risk assessment, control activities, information and communication, and monitoring—apply to all aspects of an agency’s operations and should be an integral part of a system that managers use to regulate and guide an agency’s operations. The private sector also recognizes the importance of internal controls for executives to better manage their business enterprises. The Committee of Sponsoring Organizations of the Treadway Commission’s (COSO)the same five key elements stated previously, which are intended to promote efficiency, reduce risk of asset loss, and help ensure the reliability of financial reports and compliance with laws and regulations.

The Trust Provides Untimely Information and Scholarship Awards

The Trust Provides Information to Families through Many Channels, but Its Participating School Directory Provides Incomplete and Untimely Information

The Trust provides program information to prospective and current OSP families through a variety of outreach activities. To reach prospective OSP families, the Trust advertises through print, radio, and bus ads, as well as in newspapers and flyers posted in neighborhood libraries, recreation centers, and local government service centers. For example, one OSP parent told us she found out about OSP through a local Spanish-language newspaper. The Trust conducts several application events throughout the District where interested families can apply for the program. The Trust also holds a participating school fair in which OSP families can obtain additional information about participating schools. In addition, the Trust works with families directly, offering frequent personal contact to assist families through the scholarship application and renewal processes, according to Trust officials. The Trust provides this assistance via phone, e-mail, and through one-on-one interactions, and employs Spanish and Amharic speakers to assist non-English speaking families.Lastly, the Trust holds events to assist current OSP families, such as a workshop for students transitioning from 8th grade to high school. Most families we spoke with were generally happy with their children’s participation in the program, citing increased safety and security at their children’s OSP schools and improved quality of education.

The SOAR Act requires all OSP grantee applicants to describe how parents of eligible students will be notified of the expanded choice opportunities each year, and the Directory is a key source of information about schools in the program. The Trust publishes the Directory for each school year in both English and Spanish, and it is designed to provide families with basic information on each participating school. The Directory also includes guidance on how to apply to private schools and worksheets to help families in their school search. In addition, the Directory provides information on tuition, accreditation, admissions, and facilities for each school participating in OSP for a given school year.

The 2011-2012 Directory was used to assist families with selecting schools for 2012-2013. We found that this Directory lacked key information about schools’ tuition, fees, and accreditation that could help families make more informed school choices:

Tuition: The 2011-2012 Directory listed one tuition amount for each school. According to Trust officials, the published tuition amount is the tuition applicable to each student unless the student is in a category to receive a discount on tuition. Whether a school offered different tuition levels, however, was not noted in the Directory. We found that many schools have different tuition levels, varying by grade level and religious affiliation.

Grade level: Nine of the 53 participating schools in school year 2011-2012 offered different tuition levels for students in grades K- 8 versus students in grades 9-12. In addition, 10 of the 53 participating schools had tuition levels that differed within grades K-8.

Religious affiliation: Some religious schools also offered lowered or discounted tuition rates for their members. For example, in school year 2011-2012, 6 Catholic schools offered lowered tuition rates for Catholic families, but the Directory only listed the higher, non-Catholic tuition rate. In addition, a Seventh-day Adventist school offered three different tuition levels, with the lowest rates offered to families belonging to their church and the highest rate offered to families who were not affiliated with this denomination.

Only the highest tuition rate, however, was published in the Directory.

Fees: The Directory offers parents limited information about fees. For example, the Directory excluded the cost of commonly required fees at schools, such as registration and book fees. In addition, many OSP families may need services such as before and after care, and hot lunches. The Directory listed the availability of such services but did not include their cost. The Directory also excluded information about parental involvement or fundraising fees. Some schools can reimburse at least part of these fees if parents volunteer a certain number of hours, according to four school officials. If a family cannot fulfill these volunteer hours, then it must pay these fees out of pocket because these fees are not covered under the OSP scholarship. Most OSP students attend schools where the maximum OSP scholarship amount would cover the estimated total cost of attendance (see sidebar). But without having this information, it is difficult for families to know the full cost of attendance, and they may be responsible for paying fees that they were unaware of. This may be difficult, given this program’s target population.

Accreditation: The Directory also offers families incomplete information about schools’ accreditation status. According to representatives of accreditation organizations, school accreditation is important because it serves as a means of accountability and oversight. The Directory indicates which accrediting organizations are recognized by the District. However, the Directory lists some schools’ affiliated memberships as an accreditation organization. For example, one school was listed as having the National Association of Episcopal Schools as an accrediting organization, but this organization does not accredit schools. Three schools were listed as being accredited by Nativity Miguel, which is a network of schools serving low-income communities, not an accrediting organization.

Some families we spoke with told us that they did not receive the Directory or tuition information, and therefore, they were not aware of which costs would not be covered by OSP or which schools were participating in the program. Instead, they communicated directly with schools about whether they were participating in the program. Although the Directory is publicly available on the Trust’s website, not all program participants may have computer or Internet access. In addition, even if families have Internet access through smart phones, this does not necessarily mean that program participants can use them to access everything they need, according to one school official. Trust officials told us they provide direct assistance to families to help them select schools and discern their total cost of attendance, but not all families may be receiving this assistance.

In addition to having incomplete information, the Directory has also been published too late to truly assist families in selecting a school. Families need to select schools for their students before the school year starts in August or September. However, the Trust published the Directory for the 2012-2013 school year in May 2013, about 9 months after the start of that school year. According to Trust officials, the Directory was published late because the school site visits—during which Trust officials confirm and update key school data for the Directory—occurred later in the year than in previous years due to substantial staff changes. Trust officials also told us that these directories are primarily meant to inform families selecting schools for the upcoming school year. While the 2012-2013 Directory was issued too late to assist families for that same school year, it can assist families for the upcoming 2013-2014 school year. Trust officials stated that in the future they plan to publish the Directory by December of the current school year. The Trust plans for future Directories to include more accurate notation of the school accrediting bodies and a general notation about applicable discounts and/or additional fees.

According to Trust officials, they issued a list of participating schools in the fall for the 2012-2013 school year to assist families with school selection. This list of participating schools was intended to be a companion piece to the 2011-2012 Directory and included schools’ addresses and grades served, but it did not include key information such as updated tuition information and admissions requirements. According to Trust officials, the participating school list is updated as needed to reflect any school changes that may have occurred between the publication of the previous year’s Directory and the start of the school year. For example, the 2012-2013 list of participating schools excluded a school that had closed, even though this school was listed in the 2011-2012 Directory. While some families may have been able to fill any gaps between these two sources of information through the guidance and hands-on assistance provided by the Trust, the extent to which all families had access to this direct assistance is unclear.

The Timeframes for Awarding Scholarships Do Not Align with Many Schools’ Admissions and Enrollment Schedules

For both the 2012-2013 and the 2013-2014 school years, the Trust awarded scholarships several months after many schools’ application deadlines had passed. For the 2012-2013 school year, the Trust conducted its scholarship lottery in July 2012, about 1 month before classes started for many schools. In addition, the Trust did not hold its school placement and welcome fair, which it uses to assist families with school selection, until August 2012, the same month that classes began for 6 of the 10 schools we visited, as shown in figure 3. By that time, many of the schools in OSP had already completed their admissions and enrollment processes. We found that 9 of the 10 schools we visited did not have an application deadline or were still accepting applications after students had been notified that they had received a scholarship.

For the 2013-2014 school year, the Trust moved up its timeframes, giving families more time to select schools than in the previous year. First, the Trust awarded scholarships in late May 2013 and held its school placement and welcome fair in early June rather than August, as was done in 2012, according to Trust officials. Still, as in school year 2012- 2013, many schools had already completed their admissions and enrollment processes by that time. In addition, according to some schools we visited, awarding scholarships so close to the start of the school year can affect their ability to adequately plan for the coming school year. For example, officials at two schools we visited told us that as a result of the scholarship lottery and school fair being held in July and August 2012, respectively, they did not know how many students would be enrolled at their schools until just a few weeks before the school year started in August. This made it difficult to discern class sizes and plan for the number of teachers needed.

Officials at schools we visited told us that they have found ways to work around the scholarship time frame and accommodate OSP students. For example, one school held an additional open house and another school accepted applications as late as the first week of school, according to two school officials. However, several parents we spoke with mentioned that they only had about a month to find a school and enroll their students, at which time many schools were no longer accepting students, and three school officials told us that the scholarship timeframes do not give families enough time to research school options. Since the application and financial aid deadlines for some of the program’s most costly schools occur much earlier than the OSP scholarship timeframes, by the time a student is awarded a scholarship, there are neither financial aid dollars from the school nor space available for that student.

According to Trust officials, Education’s requirement to verify family income using the most recent complete year of income information delays scholarship awards. To verify income information, the Trust also told us they have to wait until after the April 15th tax filing deadline. Trust officials noted that some schools use prior year income information to determine provisional financial aid awards, reconcile these awards with the current year’s tax information once that is available, and rescind financial aid awards if the incomes exceed a certain level. Trust officials expressed concerns about having to potentially rescind any scholarships, even though the chance that incomes will change so substantially among program participants is low. Trust officials are currently exploring ways to use prior year income information to determine eligibility earlier in the year and enable them to award scholarships earlier. Education officials told us if the Trust wants to consider using a different year’s income for the purpose of making preliminary determinations regarding students’ eligibility, it would have to submit its proposal to Education for evaluation.

The Trust Has Not Developed Effective Internal Controls to Safeguard Program Funds

The Trust Has Not Developed Effective Policies and Procedures for Implementing and Overseeing the Program

The Trust’s policies and procedures lack detail in several areas related to school compliance and financial accounting, which may result in little overall accountability for program funds. The absence of detailed policies and procedures also reflect weak internal control in the areas of risk assessment, control activities, information and communication, and control environment. Internal control is broadly defined as a process designed to provide reasonable assurance that an organization can achieve its objectives with effective, efficient operations, reliable financial reporting, and compliance with laws and regulations. The Committee of Sponsoring Organizations of the Treadway Commission’s (COSO) Internal Control—Integrated Framework, includes five key elements or components, as shown in table 1. COSO is applicable to the Trust because it is a non-governmental entity.

Policies and procedures are a central part of control activities and help ensure necessary actions are taken to address risks to achievement of the entity’s objectives. In August 2013, the Trust made amendments to its policies and procedures to address the financial review performed on schools, administrative expenses, and bank reconciliations. However, these amendments do not address all weaknesses identified in this report, and they have not yet been fully implemented.

The Trust, as administrator of OSP, is responsible for ensuring that participating schools comply with the reporting and program requirements that are outlined in the SOAR Act. The Trust, however, does not have a process for independently verifying the information that schools submit as evidence of compliance, reflecting weaknesses in the Trust’s risk assessment process and internal control activities.

The SOAR Act requires a potential OSP grantee to demonstrate how it will ensure that participating schools meet certain reporting and program requirements specified in the Act. In addition, the SOAR Act requires the grantee applicants to ensure that participating schools report to parents at least once during the school year on students’ academic achievement, as well as the safety and accreditation status of the school. The Act also requires schools to meet certain standards, such as building occupancy and teacher credentialing requirements. The Trust’s policies and procedures state that participating schools must self-certify whether they have met several of these requirements.

Participating schools complete a School Participation Verification Form that is submitted to the Trust each year to attest to student academic performance, school safety, and all of the applicable requirements in the SOAR Act, including status of accreditation. According to District regulation, if a private school in the District is not accredited, the school can provide evidence of acceptable instruction to the District’s Office of the State Superintendent for Education (OSSE) by submitting information regarding teacher credentials and curriculum documentation. However, during our interview, officials from OSSE stated that they do not regularly ensure that schools have an acceptable curriculum. Officials we spoke with said that they are revisiting this policy, but they did not provide a timeline for whether and when any changes will occur. According to OSSE officials, the last time the Trust requested an accreditation review of participating OSP schools was before the start of the 2010-2011 school year.

The SOAR Act states that participating schools must maintain a valid certificate of occupancy. The Trust collects and reviews copies of the certificates of occupancy. However, if the validity of the certificate of occupancy changes, the Trust may not know of the change. We obtained and reviewed the certificates of occupancy for the 10 schools we selected for site visits. In 5 of the 10 schools, we could not discern the validity of their certificates of occupancy. For example, one school’s certificate of occupancy did not list “school” or “private school” for the use, but rather “child development center and infant preschool.” Enrollment at three other schools, as shown in the Directory, appeared to exceed the listed capacity, and one additional school’s certificate did not list a capacity so it is not known if the school has exceeded its current capacity. Additional follow-up is needed to determine if these certificates of occupancy are, in fact, valid, but the Trust does not make any inquiries with the District agency responsible for issuing certificates of occupancy, nor does the Trust follow up with the schools.

In addition, the Trust conducts site visits at participating schools but does not verify the documents or activities to which the schools attest. Participating schools complete a School Review Form to document each site visit. This form is also used by schools to attest to meeting reporting and program requirements and is not independently verified by the Trust.

As the program administrator, the Trust is responsible for mitigating potential program risks. Verifying the information reported by participating schools is important to ensuring that scholarships are awarded only to students to attend participating schools that are in compliance with the SOAR Act’s requirements and that the program is administered effectively. Without a mechanism or procedures for verifying the accuracy of the information provided by participating schools, the Trust cannot ensure that schools are eligible to participate in OSP and, therefore, risks providing federal dollars to students to attend schools that do not meet the educational and health and safety standards required by the District.

The Trust’s policies and procedures lack sufficient detail to ensure each participating school in OSP has the financial systems, controls, policies, and procedures in place to ensure federal funds are used according to the law, a requirement of the SOAR Act.

The Trust’s policies and procedures require the financial Controller of the Trust to review documentation that demonstrates the adequacy of each participating schools’ financial resources. Based on this financial review, the Trust has identified two schools as “high risk,” one for issues that occurred during the 2010-2011 school year, and the other for financial struggles in 2011. However, the policies and procedures for this financial stability review do not identify the specific risk factors that should be considered when assessing schools’ financial sustainability information.In addition, it is unclear what risk factors were considered during the prior Controller’s review of the schools’ documentation demonstrating adequate financial resources since the policies and procedures did not specify factors to consider and, according to current Trust officials, there was no documentation about the review.

According to the risk assessment and control activities components of COSO’s internal control framework, it is important that management carefully considers factors that contribute to or increase risk and that management creates policies and procedures that help ensure that necessary actions are taken to address these risks. For OSP, factors that contribute to or increase risk include whether a school can continue to reasonably meet its financial obligations as they become due, and how dependent a school is on OSP funds. If a school cannot reasonably meet its financial obligations, it could be monitored more closely before and after being accepted into the program. If a school is overly reliant on OSP funds, which would be determined by the Trust, further review or increased monitoring could be warranted. As a result of certain risk factors not being considered in assessing schools’ financial sustainability, schools that are not financially sustainable may be participating in the program.

Based on documents provided by the Trust, several schools we visited that participated in OSP during the 2010-2011 and 2011-2012 school years did not provide detailed financial statements necessary to assess their financial stability, and during this time, the Trust did not have a The financial practice of documenting its financial review of schools.information submitted by 6 of the 10 schools we visited did not include detailed financial information required by the Trust’s policies and procedures. There was not any documentation or guidance completed by the previous Controller on these financial reviews. Officials from participating OSP schools we visited also confirmed that the Trust did not ask for additional information or support for the financial condition of the schools. Despite the policies and procedures that exist for OSP financial review of schools, there is little documentation detailing the analyses performed or the conclusions reached by the Trust. In addition, exceptions to the policy, such as where schools submitted information that did not adhere to the policies and procedures, was not documented. As stated earlier, the Trust amended their policies and procedures in August 2013 to include more detail regarding the financial review of participating schools; however, these amendments have not yet been fully implemented.

The Trust does not have detailed policies and procedures for dealing with schools that are not in compliance with program rules. For example, if the Trust discovers that a school no longer possesses a valid certificate of occupancy or is not accredited and did not meet the District’s educational standards, the Trust lacks policies and procedures dictating what is to be done at what time in the school year to ensure all schools participating in the program meet program rules and all students in OSP are attending schools that comply with these standards of safety, educational quality, and financial stability. The Trust’s policies and procedures provide only one example of a policy directed at a school not meeting standards required by the SOAR Act: if a school is unable to provide supporting documentation for financial sustainability, the Trust’s policy is that scholarship payments to the school will be distributed on a month-to- month basis, as opposed to three times per year. It is a weakness in the Trust’s control activities that there is not a clearly defined, comprehensive written policy addressing schools out of compliance with program rules. Trust officials told us that they were not clear on what actions they could take to address non-compliant schools. However, Education officials told us that under certain circumstances the Trust can remove a school from participating in OSP.

The Trust’s policies and procedures for fiscal years 2010, 2011, and 2012 did not specify how to track administrative expenses, including what expenses should be included, and the Trust has little documentation to support administrative expenses incurred during the fiscal years 2010, 2011, and 2012. In addition, the Trust does not have detailed policies and procedures outlining the monthly bank reconciliation process, where staff review accounting system records and compare them to bank records. As stated earlier, it is important that an entity’s management have policies and procedures in place that address risks to achieving an entity’s objectives. Control activities include a range of activities such as verifications, reconciliations, as well as reviews of operating performance and security of assets.

The SOAR Act limits the Trust’s administrative expenses to 3 percent of the annual grant amount. The Trust’s policies and procedures state that they may only draw down federal funds from Education on a reimbursement basis after such expenses have been incurred and paid. While the Trust’s policy manual does provide guidance for tracking employee time that should be billed to the program, it does not provide guidance for other expenses such as rent, telephone, printing, or office supplies, which could possibly be counted as administrative expenses for purposes of seeking reimbursement. The cost of administering OSP could be higher or lower than the 3 percent designated for administrative costs. Because these expenses were not tracked prior to October 2012, the true cost of administering the OSP program is unknown. In part due to the outcome of their fiscal year 2010 financial audit, the Trust has begun to track administrative expenses more closely.

The Trust did not draw down federal reimbursement from Education for administrative expenses for fiscal years 2010, 2011, and 2012. In addition, the Trust did not document the decision or approval as to why it did not follow its policies and procedures or did not request reimbursement from Education for its administrative expenses. According to Education officials, grantees do not typically document why they have not drawn down funds. In addition, they explained that Education’s grants management system automatically flags funds that have not been drawn down within a particular time frame and the system has not flagged the Trust since funds had been drawn for the scholarships. Education officials also stated that the Trust classifies its obligations and expenditures in reports such as their monthly expense reports, which is sufficient for the department’s purposes.

In addition, as part of the finance department’s monthly bank reconciliation process, the Trust’s policies and procedures also refer to scholarship payment reconciliation as well as administrative expense drawdown reconciliation. However, the Trust’s policies and procedures do not specify certain aspects of the reconciliation, such as when the reconciliations should be completed, how exceptions should be identified and dispossessed, how the process should be documented, and when the reconciliation should be reviewed—and by whom.

Without the specific guidance in the policies and procedures, it will be difficult for the Trust to ensure that items are being appropriately and consistently tracked and recorded, as administrative expenses may be included that should not be (or not included that should be). In addition, without specificity in the policies and procedures regarding bank reconciliations, these reconciliations may not be completed in a consistent or timely manner and increases the potential for errors, and likelihood that errors will not be identified and corrected in a timely manner.

The Trust’s Database Is Not Efficiently Structured

The Online Reports and Invoicing System (the database), which is used by the Trust to manage the OSP program, is not structured well enough for effective program implementation and oversight. Information contained in the database includes current and past application information for students and guardians, school placement information for each student who received a scholarship award, payment information, and information on participating schools.

According to COSO’s Integrated Framework, information should be communicated to management and within the entity in a form and time frame that enables officials to carry out their responsibilities and determine whether they are meeting their stated objectives. For example, in OSP, it is important that Trust officials have access to accurate, up-to- date student application information in order to meet program objectives, such as determining eligibility and awarding OSP scholarships in an efficient and timely manner. Similarly, families of OSP scholarship award recipients, as consumers, need complete and timely information about participating schools to make informed decisions about what school is best for the student. The Trust’s database has several deficiencies including a lack of documentation and automated checks, and a deficient structure, which leaves the database open to errors and slows the Trust’s ability to manage the program on a day-to-day basis, and communicate information about the program to families and Education.

The Trust (including the contractor that created and maintains the database) does not have any documentation for this database. For example, there is no user guide or data dictionary, and potential users of the database must rely on the institutional memory of select users who have had more experience with the database to understand how to properly use the database or the definitions of key data fields. When we attempted to follow various verbal or informal written instructions for using the database, we found flaws in these instructions that necessitated several rounds of follow-up to successfully perform key functions and basic data manipulation. It is not known what specifically caused this condition, but Trust officials have noted financial constraints in requesting more support from the contractor.

In addition to lacking documentation, the database does not have key automated checks built into the system to ensure accuracy and efficient data entry. Automated checks help maintain integrity in a database reducing the risk that a significant mistake could occur and remain undetected and uncorrected. For example, the Trust’s database does not have checks to determine if a student’s application information has already been entered. When a student submits an application, before his or her information is entered into the system, a Trust staff person must run searches in the database for his or her name, the guardian’s name, both persons’ Social Security numbers and address to ensure the applicant is not already in the system. These checks are necessary because the system allows duplicate entry of the same student information, potentially rendering the data unreliable, even with the searches described above. To ensure data are accurate given the lack of automated checks, the Trust stated that one of their officials runs several queries and creates tables in Microsoft Excel to check for errors by hand on a weekly or more frequent basis during key periods for the program.

The database’s current structure hampers an administrator’s ability to look at historical trends and use them as an effective management tool. When the Trust inherited the database from the Washington Scholarship Fund, the older data in the system were not cleaned, and thus there are many records in the database with missing fields, or applications that are only partially entered, suggesting they are not valid records. The Trust has stated they cannot attest to the accuracy of these older data. Yet, the Trust is using these older data in reports without an explanation of the potential issues of inaccuracy.

The weaknesses in the database’s structure also affect key activities for the Trust, such as determining the priority groups of applications for the OSP lottery. As stated earlier in this report, the SOAR Act states that if more students apply to OSP than the program can accommodate, the grantee (the Trust) must ensure that applicants are selected for awards through a random selection process that gives weight to three priority groups (which the Trust has implemented through a lottery). The groups of applicants that have a priority status as defined in the SOAR Act are: (1) applicants who are currently attending a school that has been identified for improvement, corrective action, or restructuring under the Elementary and Secondary Act of 1965, as amended; (2) applicants who have a sibling currently participating in OSP; and (3) applicants who received an OSP scholarship award in a previous year but did not use it. Using the Trust’s database, we employed our own methodology to replicate the process to determine the number of students in each applicant priority group for the 2011-2012 school year because no documentation exists for these queries. However, we were unable to determine which students belonged in the third priority category of students for the 2011-2012 school year because a key variable necessary for this calculation appears to be unreliably populated. In other words, it does not appear possible to use the Trust’s database to derive this priority category and puts in question the Trust’s ability to provide accurate priority categories for the OSP lottery.

Lastly, data entry of application information is also problematic. In addition to basic eligibility information, OSP applications also include important information needed for the third-party evaluation of the OSP program. According to Education officials, the evaluation timelines have been negatively affected for the 2011-2012 and 2012-2013 school years because data entry was not completed in a timely manner. In one instance the Trust had to retrieve hard copies of the applications from storage and send relevant application data to the officials involved in the evaluation because they were so behind in data entry. Education officials stated that there may have been a lack of clarity regarding the specific information the Trust was supposed to enter into the database and that delays resulting from untimely data entry should not be an issue going forward.

The Trust Has Not Filed Its Mandatory Financial Reports on Time

The Single Audit Act requires that recipients submit their Single Audit reports to the federal government no later than 9 months after the end of the period being audited, which would be June 30th for an entity with a September 30th fiscal year end, such as the Trust. The Trust’s Financial Statements, Schedule of Expenditures of Federal Awards, and Independent Auditors’ Reports Required by Government Auditing Standards and OMB Circular A-133 for the Year Ended September 30, 2010, were issued on January 31, 2013—more than 2 years after the end of its 2010 fiscal year. Until this report was issued, Education did not have the financial reports required to properly account for the federal funds expended for OSP under the Trust’s administration. According to Trust officials, the 2010 audit was delayed because of a need to change audit firms and ongoing investigations regarding its finances related to other grants.

The issue of delayed reports has not been resolved. As of August 2013, the Trust’s audited financial statements for fiscal year ending September 30, 2011, and 2012 have not been issued yet (see fig. 4). On August 9, 2013, Trust officials told us that the final fiscal year 2011 audit is expected to be completed by the end of August 2013 and the fiscal year 2012 audit will not be completed before September 2013.delay in submitting mandatory financial reports there was no opportunity for formal oversight of federal dollars spent on OSP for almost 2 years under the Trust’s administration. In addition, the continued delay in the audit reports means the Trust has not taken actions on deficiencies found in each audit in a timely manner to mitigate further issues.

Agencies Have Not Fully Executed Responsibilities Outlined in Governing Documents

The SOAR Act requires that the Secretary of Education and the Mayor of the District of Columbia enter into a memorandum of understanding (MOU) that addresses how OSP will be implemented. The MOU, which was agreed upon by Education and the District in June 2012, is intended to help ensure the efficient and effective implementation of OSP in a manner that incorporates and is consistent with the roles and responsibilities of Education and the District. It describes, for example, offices and officials within Education and the District that have lead responsibility for implementing OSP, issues and areas on which Education and the District will collaborate, and specific activities that Education and District agencies, such as the Department of Consumer and Regulatory Affairs (DCRA), Fire and Emergency Medical Services (EMS), and the Department of Health will carry out. According to the MOU, Education is responsible for working with the Trust to improve certain aspects of the administration of OSP. The District is responsible for conducting regulatory inspections of participating schools—upon notification by the Trust—and providing the Trust with the results of those inspections (see fig. 5).

In addition, Education entered into a cooperative agreement with the Trust to set forth the responsibilities of each entity for implementing OSP as required by the SOAR Act and the Education Department General Administrative Regulations. Education and Trust officials signed the cooperative agreement in June 2013, 3 years after the grant for OSP had been transferred to the Trust. Education officials stated that, among other reasons, the cooperative agreement had not been developed any earlier because of substantial turnover in staff at the Trust. Education officials stated they wanted to wait to implement the agreement until the Trust’s staff had stabilized. Trust officials stated that the cooperative agreement was an opportunity to codify the various ways in which Education and the Trust would assist each other in implementing the program. Through the cooperative agreement, Education also agreed to assist the Trust in the same four areas stated in the MOU.

Education Has Provided Limited Assistance to the Trust

Education has provided limited assistance to the Trust as agreed upon in the MOU and cooperative agreement governing OSP. Trust officials acknowledged that they have not proactively sought Education’s assistance in these areas. They told us that although they have had several discussions with Education regarding the general administration of the program and operational issues, they have not had opportunities to discuss lessons learned or making improvements in certain areas of program administration.

Through the MOU and cooperative agreement, Education is responsible for helping the Trust ensure that it implements appropriate improvements to its financial system. However, according to Trust officials, Education has not undertaken any activity to help improve its financial systems and the Trust has not had any conversations with Education about doing so. Education officials told us that the Trust’s financial systems were an improvement over the previous administrator’s financial system and that they have not had to closely manage the Trust in this area. However, as previously discussed, the Trust’s policies and procedures do not provide specific guidance on allocating certain administrative expenses and the Trust does not have detailed policies and procedures outlining the bank reconciliation process. Further, as a result of the Trust’s tardiness in submitting mandatory financial reports, the Trust was unable to account for federal dollars spent on OSP for about 2 years after the end of fiscal year 2010.

Education has not assisted the Trust in developing, implementing, or updating its policies and procedures for conducting site visits, as specified in the MOU and cooperative agreement. According to Education officials, as long as the Trust maintained the procedures used by the previous administrator, there was no need to revise its site visit procedures. When we spoke to Trust officials, however, we were told that although Education has not assisted with its site visit policies and procedures, the Trust updates its procedures every year and provides them to Education for review. Additionally, Education told us that it did not have any concerns with the timing of the Trust’s site visits, even though the Trust did not conduct site visits for the 2012-2013 school year until February 2013—the second semester of the school year.

According to the MOU and cooperative agreement, Education agreed to assist the Trust in developing procedures to improve the accuracy of information provided to families before they choose a participating school and encourage schools to make such information available on an annual basis to families of enrolled students. Education officials stated that they meet with the Trust at the start of each school year to discuss marketing and recruitment, the participating school fair, and how to engage with families, and to provide feedback on these aspects of OSP as needed. For example, Education worked with the Trust to publish the school Directory for the 2012-2013 school year. Education did not have any concerns with the accuracy of the information provided to families for the purpose of selecting a school to attend. According to the Trust, Education relies on it to verify that schools are providing parents with such information. However, as discussed earlier, the school Directory does not include complete information about schools’ tuition, fees, and accreditation that families need to make informed school choices.

In addition to the responsibilities described above, Education is also responsible for providing oversight of the Trust as a recipient of federal funds. According to Education officials, its role is to identify a grantee to administer OSP and ensure that the grantee adheres to applicable federal law and regulations. Education monitors the Trust with regard to (1) progress in implementing OSP; (2) financial records; and (3) data records, including records on student placements and numbers of scholarship awards. The activities that Education conducts as part of its oversight of the Trust are documented in the cooperative agreement between Education and the Trust and it conducts ongoing reviews of the Trust’s activities to ensure satisfactory performance under the cooperative agreement. Education officials stated that the cooperative agreement was developed, in part, to increase its oversight of the Trust. For example, Education used the cooperative agreement to institute reporting requirements and clarify program rules. Officials said they also recognized the need for a closer partnership with the Trust and stated that the cooperative agreement was an opportunity to clearly describe the Trust’s expectations.

Education officials also told us that they provide guidance and technical assistance to the Trust. For example, Education officials told us they participate in conference calls with Trust officials, review the Trust’s annual performance report and monitoring plans, and provide ongoing technical assistance. Education officials also stated that they conduct regular, ongoing desk monitoring of the Trust, and will also conduct biannual site visits with the Trust beginning in the fall of 2013. If there were any problems with safeguarding OSP funds, Education officials stated they would work with the Trust to correct the problem or, if necessary, they could penalize the Trust through actions such as freezing and withholding funds, recovering funds, and ensuring that the grant cannot be renewed.

Requirements Under the MOU Are Not Being Met

The MOU requires that the District conduct inspections necessary for schools to participate in OSP “upon notification by the grantee (the Trust),” but required inspections are often not being conducted. For example, the MOU requires the District Government, upon notification by the Trust, to conduct regulatory inspections of all program schools to determine whether they have a current certificate of occupancy. The District’s DCRA issues certificates of occupancy and conducts inspections based on the types of building or trade permits that schools have. A DCRA official, however, stated that it relies on the permit holder to make requests for all required inspections. Also, the District’s Fire and EMS Department is responsible for conducting biennial inspections or otherwise determining whether participating schools comply with applicable District health and safety requirements, as required by the MOU. However, a Fire and EMS official stated that private schools are not subject to regular inspections. Trust officials told us they did not know whether participating schools had been inspected by the District agencies. Lastly, officials at schools we visited told us that certain District agencies had conducted some inspections but not specifically for OSP. Rather, they occurred for other reasons, such as general inspections for schools that offer special education or early childhood services.

The MOU also states that the District should provide information to the Trust regarding whether participating schools are in conformance with District requirements. The Trust’s understanding of this requirement is that the District should be providing the inspection results on a regular basis without being specifically asked by the Trust. However, Trust officials told us they do not receive any information from the District as a result of any inspections that may have been conducted at participating schools. And according to the Trust, it does not follow up with District agencies to inquire about the results of any of the inspections.

Although the MOU is a written agreement between Education and the District, it includes a responsibility for the Trust. Specifically, the MOU states that the Trust—as the grantee—is responsible for notifying District agencies to conduct these required inspections of participating private schools. Given that the Trust is responsible for ensuring that participating schools continue to be eligible to receive federal dollars through OSP, notifying the District agencies can be important in ensuring appropriate oversight of participating schools. This activity can be especially important for agencies like the District’s Fire and EMS department that do not normally inspect private schools. However, because the Trust is not a signatory to the MOU, Trust officials were not acutely aware that they were responsible for notifying the District agencies. Furthermore, the cooperative agreement between Education and the Trust does not include this requirement for the Trust to notify District agencies. Officials from the Trust stated that they assumed that District agencies conducted inspections of private schools as a matter of process, without the Trust prompting them to do so. As a result, activities that are crucial to the successful implementation of the program—such as building, zoning, health, and safety inspections—may not be occurring for all participating schools.

Conclusions

The OSP is intended to afford District families the opportunity to attend higher-performing schools of their choice and have all or part of the cost of attendance paid. In order to make well-informed decisions, however, families need complete and timely information on schools’ tuition, fees, and accreditation, as well as timely scholarship awards. Since taking over OSP, the Trust has not provided such information to families in a timely manner, requiring families to make less-informed decisions near the beginning of the school year, when many schools no longer have places for admission.

The Trust needs to improve program management and operations to assure efficiency and effectiveness. Currently, it does not effectively oversee participating schools, has not implemented effective policies and procedures, and is unable to efficiently manage day-to-day program operations. As a result, families may believe that participating schools fully meet program requirements, although the Trust cannot ensure that these schools meet them. In August 2013, the Trust made amendments to its policies and procedures manual regarding the financial review performed on schools, administrative expenses, and bank reconciliations. However, these amendments do not address all weaknesses identified in this report, and they have not yet been fully implemented. Without sufficiently detailed policies and procedures for all aspects of the Trust’s operation, the Trust cannot sufficiently monitor its own operation of the program and may not be able to account for all federal dollars spent on OSP. Additionally, because of the weaknesses in its program database, the Trust cannot efficiently manage day-to-day operations of the program.

The weaknesses that we have identified highlight the need for sustained oversight and management attention to ensure efficient and effective program implementation and accountability over federal funds. However, Education has provided the Trust with limited assistance in specific areas stated in the MOU and the cooperative agreement governing the program. Furthermore, the Trust is unsure whether key activities crucial to the successful implementation of the program—inspections for compliance with the District’s building, zoning, health, and safety requirements—are being conducted. Under the MOU, the inspections are to be done upon the Trust’s notification to District agencies, but the Trust is not a signatory to the MOU. Moreover, the cooperative agreement between Education and the Trust is silent regarding the responsibilities of the Trust on this issue. The Trust’s limited ability to effectively oversee participating schools coupled with the weaknesses in the Trust’s internal control environment underscore the need for Education to conduct more rigorous oversight and monitoring of the Trust. Not only would increased oversight help ensure that the Trust is administering the program effectively, but it would also help ensure that OSP achieves its intended purpose as outlined in the SOAR Act.

Recommendations for Executive Action

To ensure families receive expanded opportunities for school choice and OSP is implemented and overseen effectively, we recommend that the Secretary of Education take the following actions:

Revise the June 2013 cooperative agreement between Education and the Trust to include Education’s expectations for the Trust regarding collaboration with District agencies to ensure they conduct required building, zoning, health, and safety inspections.

Work with the Mayor of the District of Columbia to revise the memorandum of understanding that governs OSP implementation to include processes that will help ensure that the results of OSP school inspections, when they are conducted, are communicated to the Trust.

Explore ways to improve Education’s monitoring and oversight of the Trust. For example, Education could require the Trust to develop and implement a plan for how it will address its timeliness in mandatory financial reporting.

Conduct activities to ensure that the Trust: publishes the school Directory prior to participating schools’ application deadlines with more complete information. more closely aligns its scholarship timeframes with schools’ admissions and enrollment schedules. Education could find ways to assist the Trust in executing the OSP lottery earlier in the school year, for example, by allowing the Trust to use the prior year’s income information to make preliminary determinations regarding eligibility. improves its OSP database by creating and maintaining documentation for the use of the database, adding automated checks to the system, creating flags and/or variables for the key priority categories called for in the SOAR Act, and streamlining the data entry process for new applications in order to ensure there is sufficiently reliable data regarding the operation of the program.

Require the Trust to update its policies and procedures in several key respects, and monitor the Trust’s activities to ensure that these updates are made. These updates should include: identifying the steps the Trust will take to verify reported information on participating schools’ compliance with reporting and program requirements specified in the SOAR Act. identifying the specific risks that should be included in the evaluation performed when assessing participating schools’ financial sustainability information, including a record of the analyses performed, and ensure that the conclusions reached with regard to schools’ financial sustainability are documented. specifying a process for addressing schools that are noncompliant with program requirements. providing more detailed procedures in key areas including calculating yearly administrative expenses, and performing monthly and year end reconciliations of the OSP bank account, and ensure the document reflects current practices and that key decisions associated with administrative expenses are documented.

Agency Comments and Our Evaluation

We provided a draft of this report to Education, the Office of the Mayor of the District of Columbia (the District), and the DC Children and Youth Investment Trust Corporation (the Trust) for comment. Education officials provided technical comments, which were incorporated into the final report as appropriate. Written comments from Education and the Trust are reproduced in appendices V and VI, respectively, and are also summarized below. The District did not provide written comments on the draft report. In its comments, Education did not indicate agreement or disagreement with our recommendations. The Trust generally agreed with our recommendations, but disagreed with some of the findings. Both Education and the Trust provided additional information on issues raised in the report.

In its letter, Education stated that it took the issues raised in the GAO report seriously and would continue to consider them carefully, but noted that the report did not fully reflect the Trust’s efforts to provide complete and timely information about participating schools to OSP families. However, in our final report, we discuss a number of different ways in which the Trust provides information and assistance to current and prospective OSP families, and we believe we adequately reflected the Trust’s efforts. Education also noted that the report does not fully recognize the significant staff turnover in leadership at the Trust, which affected several aspects of the Trust’s implementation of the program in school year 2012-2013. We recognize that the Trust faced a number of challenges as it assumed responsibility for the program. However, several issues that we identified in the report are not new, but rather contribute to long-standing weaknesses in internal controls and oversight of the program. Therefore, we believe that if Education had provided more effective oversight of the program and more timely technical assistance to the Trust, many of the challenges identified in the report could have been addressed sooner. In addition, more effective internal controls, including robust policies and procedures for program administration and school oversight, could have helped to smooth the difficulties associated with the transition in leadership and program administration.

The Trust disagreed with our finding that its Participating School Directory provided incomplete and untimely information. More specifically, in its comments, the Trust emphasized that this condition does not exist under the current management and administration of OSP and that the Trust provides information through a comprehensive array of tools, events, and individualized assistance. We acknowledge the substantial change in staff that occurred at the Trust and discuss in our report the number of ways in which the Trust provides assistance to families through channels other than the Directory. However, the most recent Directory—a key source of school information for families—was still published too late and contained incomplete information about tuition, fees, and accreditation. Education’s written comments focus on the Trust making a deliberate decision to maintain a reasonably sized, user-friendly directory with straightforward information on average tuition and fees and on the efforts of the Trust to provide individualized ongoing assistance to families and assistance through school placement fairs. We agree that the Directory must be prepared for a non-technical audience, but continue to believe that for a program predicated on choice, it is imperative that the Trust provide families with complete and timely information needed to make informed decisions, especially with respect to tuition and fees. The Trust could accomplish this without overburdening families with excessive information by using a general notation for schools that have discounted or different tuition rates, as the Trust noted in its written comments, or providing a range of the potential total cost of attendance for families. As the Trust indicated in its letter, it could also include more accurate notation of school-accrediting bodies. We support the Trust’s efforts to update the Directory for future school years by including more complete information for OSP families.

In addition, in its comments, Education noted that substantial changes in the Trust’s staff and leadership transition affected the timing of the Trust’s site visits at participating schools. As we discuss in our report, these circumstances resulted in site visits occurring later in the year, which in turn affected the timeliness of the Directory’s publication. We acknowledge that this is not a routine practice of the Trust and encourage the Trust to continue to work toward conducting these site visits earlier in the school year and publishing the Directory earlier. We also support Education’s plans to work with the Trust to provide the Directory prior to participating schools’ application deadlines. The plan to publish the Directory by December 2013 for the 2013-2014 school year, and by November in future school years, represents a marked improvement in the timeliness of the school directories. However, we maintain that families need to have information about schools prior to participating schools’ application deadlines. We encourage Education and the Trust to consider publishing the Directory earlier than November or December to help families start making school decisions as early as possible in the scholarship application process, ensuring that families have the full range of school options available to them.

In response to GAO’s finding regarding the misalignment of scholarship awards with participating schools’ admissions and enrollment schedules, Education expressed concern that an earlier scholarship lottery date could, among other things, result in the Trust conducting less outreach to potential applicants. We acknowledge the steps that Education identified the Trust needs to take prior to the lottery, such as screening applications for eligibility, conducting multiple rounds of baseline testing, and ensuring that the lottery pool of eligible applicants is sufficient so that the evaluation can proceed with the necessary sample size. Education stated that the Trust would need to submit a proposal to Education if it wanted to use a different approach for screening student eligibility, such as using a different year’s income information, that could help expedite the steps leading up to the lottery. We support Education’s consideration of any proposal to use a different approach that would ultimately enable the Trust to award scholarships earlier and ensure that OSP families have the widest range of choice possible, and we would encourage the Trust to fully consider the issues, and if it deems it appropriate, develop such a proposal for Education’s consideration.

The Trust also disagreed with our finding regarding its lack of policies and procedures for implementing and overseeing OSP, stating that its policies and procedures manual clearly and accurately reflects the policies and procedures governing the program. Further, the Trust stated that although amendments to the policies and procedures regarding allocation of costs and reconciliation were codified in August 2013, these policies were in practice prior to that date. GAO applauds the Trust’s efforts to increase oversight of participating schools and update their policies and procedures accordingly. Despite the Trust’s claims that these policies and procedures were already in practice, GAO does not consider these policies and procedures fully implemented until these activities take place on a regular basis. In addition, the Trust’s revised policies and procedures do not address all of the weaknesses for which GAO recommended action, such as developing a process for addressing schools that are noncompliant with program requirements. Education commented that the Trust’s proposed revisions to the letter of agreement with participating schools reserves the Trust’s right to limit or suspend a school’s participation in OSP and that any OSP schools that require further discussion about compliance would be handled on a case-by-case basis. However, we continue to believe that the Trust needs to develop a more clearly defined, comprehensive written policy to address schools that are noncompliant.

With regard to weaknesses GAO identified in the structure of the Trust’s database, the Trust stated they would take GAO’s comments under advisement and consider key changes and improvements. The Trust stated that the database is proficient in areas such as determining student eligibility, student invoicing, school oversight documentation tracking, and notation of communication with individual families and schools. The Trust also noted that they must use outside spreadsheets to complete any reporting on historical information about the program, and that automated checks do not exist, but the database does not impede data collection or identification of student sub-groups for inclusion in the scholarship lottery (referred to in this report as “priority categories”). Based on our review, we continue to believe that the Trust’s ability to determine the third priority category described in the SOAR Act—applicants who received an OSP scholarship award in a previous year but did not use it—is questionable because, according to GAO’s analysis, a key variable in the database used to determine this priority category is not reliable.

As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies of this report to relevant congressional committees, the Secretary of Education, the Mayor of the District of Columbia, the Executive Director of the District of Columbia Children and Youth Investment Trust Corporation, and other interested parties. In addition, the report will be available at no charge on GAO’s web site at http://www.gao.gov.

If you or your staff should have any questions about this report, please contact me at (202) 512-7215 or scottg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VII.

Appendix I: Scope and Methodology

We reviewed relevant federal and District laws and regulations and analyzed key documents from the DC Children and Youth Investment Trust Corporation (Trust), the federal Department of Education (Education), and participating schools, as well as generally accepted guiding documents for internal controls including those published by GAO and the Committee of Sponsoring Organizations of the Treadway Commission (COSO). We also conducted site visits at 10 of the 53 participating private schools in the program and interviewed relevant school officials. In addition, we conducted two discussion groups—one in English and one in Spanish—with parents and guardians of students who received and used an Opportunity Scholarship Program (OSP) scholarship in the 2012-2013 school year. We also interviewed key officials at the Trust, Education, and select agencies in the District, as well as representatives of accreditation organizations and nonprofit tuition assistance organizations. In addition, we analyzed the structure of the Trust’s database and the Trust’s program data.

We conducted this performance audit from May 2012 to September 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings based on our audit objectives.

Review of Program Documents

To determine the types of information available to families and the types of schools participating in OSP, we reviewed the Trust’s Participating School Directory (the Directory) for school years 2010-2011, 2011-2012, and 2012-2013. We also reviewed the list of participating schools available to families at the beginning of the 2012-2013 school year and compared it to the Directory when it was issued in May 2013 to determine if there were any discrepancies. We also examined OSP application materials as well as documents given to OSP families at the 10 participating schools we visited. These documents included application materials, tuition and fees information, and notices sent to families, such as school newsletters and notices about safety incidents occurring in and around the school.

To determine the types of information participating schools submit to the Trust, we reviewed schools’ Key School Data Forms. These forms include information on a school’s enrollment, facilities, number of days of instruction, curriculum, and services offered and are used by the Trust to compile the Directory. We also reviewed schools’ Tuition and Fees forms to estimate a student’s total cost of attendance. Given the variation in fees among participating schools, we estimated the total cost of attendance using several assumptions, such as full-time attendance for one school year for a new OSP student. We also used information provided by schools about their admissions requirements and deadlines to determine how their application deadlines aligned with scholarship award timeframes implemented by the Trust.

To determine the extent to which the Trust implemented internal controls over financial transactions and program administration, we reviewed the Trust’s financial management policies and procedures. We also reviewed the Trust’s financial statements, schedule of expenditures of federal awards, and independent auditors’ reports required by government auditing standards and OMB Circular A-133 for the year ending September 30, 2010. In addition, we reviewed the Trust’s administrative expense report and copies of monthly and annual bank reconciliations and reviewed relevant federal laws and regulations. To determine the roles and responsibilities of the Trust, Education, and the District, we examined the grant transfer agreement between the Trust, the Washington Scholarship Fund, and Education; the memorandum of understanding between Education and the District of Columbia Government; and the cooperative agreement between Education and the Trust.

To determine the Trust’s assessment of participating schools’ financial stability, we reviewed fiscal year end 2010-2012 financial statements and other documentation submitted by schools, such as management letters and reports. We also reviewed forms submitted by schools, such as the School Participation Verification Forms, to determine the mechanism by which schools attest to the veracity of the information they submit. In addition, we reviewed COSO’s standards for internal controls for non- federal entities.

Analysis of Program Data from the OSP Database

To determine several aspects of the Trust’s administration of OSP, we requested program data from the Trust, including participating school characteristics, OSP student enrollment at participating schools, student eligibility determination, scholarship payments, and student characteristics, including the priority categories for awarding scholarships as stated in the SOAR Act. We received program data from the 2004- 2005 school year, the first year in which the program was administered by the Washington Scholarship Fund, to the 2011-2012 school year. We assessed the reliability of the data through performing electronic testing for obvious errors in accuracy and completeness and interviewing agency officials knowledgeable about the data. We found substantial problems with the data, including missing fields or partially entered applications. As we previously stated in this report, the lack of documentation for the database made it difficult to understand which fields were current or still in use. As a result, we limited the scope of our analyses to data from the 2011-2012 school year, the most recent and complete year of data available at the time of our review. We determined that these data were sufficiently reliable for the purposes of our report.

The Trust did not have a documented query or process to determine the number of OSP students enrolled at each participating school. Following the instructions from the Trust’s data contractor, we initially determined the number of OSP students at each school by using variables representing the year, student and school identifiers, student school placement, and the number of days the student was enrolled. We found, however, that there was an extensive number of missing values for several of the key variables needed to complete this analysis. As a result, we determined that we could not use this method of calculating OSP enrollment. Instead, we calculated OSP enrollment by using a count of student identifiers by school and payment information. We also analyzed OSP student enrollment by race and ethnicity to determine the demographic characteristics of OSP students enrolled at each participating school.

Under the SOAR Act, Education is required to give priority to grantee applicants that will most effectively give priority to: 1. Students who, in the preceding year, attended a public school in the District that was identified as a school in need of improvement, corrective action, or restructuring under the Elementary and Secondary Education Act of 1965, as amended. 2. Students who previously were awarded a scholarship in a preceding year but have not used the scholarship, specifically emphasizing students that were awarded a scholarship for the first time in 2009- 2010, when the 216 new scholarships awarded that year were rescinded by the Department of Education. 3. Students that have a sibling participating in OSP.

To assess the Trust’s application of these priority groups for the 2011- 2012 school year, we attempted to replicate the Trust’s process for determining the number of students in each priority group. Since the Trust has not documented these processes, we determined the number of students in two of these priority groups—those attending a school identified for improvement and those with a sibling in the program—using detailed queries and clarifications from the Trust’s contractor. Despite employing several different approaches, our results did not corroborate the Trust’s results for these two categories. We were unable to determine a reliable number of students in the third priority group—those who were previously awarded a scholarship but had not used it. We found that one of the key variables Trust officials use to determine this category was unreliable. As a result, we could not use other variables in its place to reliably determine the number of students belonging in this group.

Site Visits to Participating Schools

To review aspects of participating schools’ compliance with program rules and regulations, as well as their admissions and enrollment processes, we conducted site visits to 10 participating schools (see table 2). During these visits, we interviewed key school officials; reviewed documents, including certificates of occupancy, health certificates, and teacher credentials; and toured school facilities. We also interviewed finance officers at these schools to determine the schools’ financial stability and the types of financial information schools submit to the Trust.

We selected a nongeneralizable sample of 10 schools to visit based on two criteria: (1) the percentage of OSP students enrolled and (2) the largest number of OSP students enrolled. Using this approach allowed us to visit schools that potentially were almost entirely funded by OSP scholarships as well as schools that were receiving the most absolute OSP dollars. We used student enrollment data obtained from the Trust for the most recent school year available (2011-2012) as well as information from the 2011-2012 Directory.

Using these criteria, we determined the sample of 10 schools to visit by selecting 5 schools with the highest concentration of OSP students relative to the total school enrollment and 5 schools with the highest number of OSP students, as shown in table 2.

In order to develop these criteria, we made multiple requests for information from the Trust and made several clarifying inquiries to ensure the accuracy of our approach. After we completed this analysis, through the course of additional interviews with the Trust, we learned there were some additional types of records in the Trust’s database that we needed to exclude to execute a more exact query for OSP enrollment. However, we received this information several months after nearly all the site visits were conducted.

To provide balance, we also spoke with National Presbyterian School and Aidan Montessori School—two schools that historically had few or zero OSP students enrolled—to determine what factors impact their interest and ability to enroll OSP students. We selected these schools based on our analysis of the Trust’s 2011-2012 data on the number of OSP students enrolled at participating schools.

Discussion Groups with Parents and Guardians of OSP Students

To better understand families’ experiences with OSP, we conducted two semi-structured discussion groups with parents and guardians of students who received and used an OSP scholarship in the 2012-2013 school year. To ensure we had a diverse group of parents, we conducted one group in English and one group in Spanish. The Trust recruited parents and guardians to participate in these groups by randomly selecting 75 to 100 English-speaking and Spanish-speaking parents/guardians. Of these randomly selected parents and guardians, 14 participated in these discussion groups. Although asking the program administrator to recruit participants may result in a biased group of participants, the Trust staff knew this population best and could most effectively reach out to this group. As a result, the extent to which these participants represent all OSP families is unclear, and the results of these discussion groups are not generalizable to the entire population of OSP families.

The Trust e-mailed letters written by GAO in both English and Spanish to parents explaining the purpose of the discussion groups and encouraging their participation. Two discussion groups were held in December 2012 and were about 1 to 2 hours in length. To ensure participants’ privacy, parents and guardians were asked to use alternate first names during the discussion.

In each of the discussion groups, we covered the following topics: reasons why parents were drawn to the OSP; how they found out about the program; the school selection and application process; and their children’s experience at their OSP schools.

Following the discussion group, we asked participants to provide us with some basic background information in a short voluntary questionnaire. Thirteen of the 14 participants completed the questionnaire, which we provided in both English and Spanish. After we conducted the groups, we obtained transcripts of both groups to ensure the accuracy of the groups’ discussions. For the discussion group conducted in Spanish, we obtained a transcript in Spanish as well as a translation in English to ensure the accuracy of the translation.

Interviews

We interviewed officials at the Trust to determine how they administered OSP, their relationship with Education and the District, and their internal control activities. During one of these interviews with Trust officials, we conducted a walk-through where Trust staff demonstrated how they enter and maintain their database. To better understand how the Trust manages its database, we interviewed Trust officials and the Trust’s contractor, who is responsible for maintaining the database. To determine Education’s role in OSP and their relationship with OSP and the Trust, we interviewed officials at Education’s Office of Innovation and Improvement and Institute of Education Sciences. To understand the scholarship lottery process and how OSP is evaluated, we interviewed officials from Education and Westat, the contractor responsible for conducting the lottery. To understand the relationship between the District, Education, and the Trust, and to specifically understand the execution of the MOU, we interviewed officials at key agencies in the District, such as the Office of Deputy Mayor for Education and the Office of the State Superintendent for Education. We also requested and received written responses to key questions from certain District agencies, including the Department of Health, the Department of Consumer and Regulatory Affairs, and the Fire and Emergency Medical Services Department. To determine the significance and role of accreditation, we spoke with representatives from Middle States Association of Colleges and Schools Commissions on Elementary and Secondary Schools, Association of Independent Maryland & DC Schools, and AdvancED. We also spoke with Independent Education, an association of private independent District- area schools that requires its member schools to be accredited. To better understand the role played by nonprofit organizations that provide additional private school tuition assistance, we interviewed representatives from the Archdiocese of Washington, Latino Student Fund, Capital Partners for Education, and the Jack Kent Cooke Foundation’s Young Scholars Program.

Appendix II: Schools That Agreed to Accept Students in the District’s Opportunity Scholarship Program in School Years 2010-2011 and 2011-2012 (“Participating Schools”)

Appendix II: Schools That Agreed to Accept Students in the District’s Opportunity Scholarship Program in School Years 2010- 2011 and 2011-2012 (“Participating Schools”)

School name Academia De La Recta Porta International Christian Day School Beauvoir – National Cathedral Elementary School Bishop John T. Walker School for Boys Howard University Early Learning Program Kuumba Learning Center (MLK Campus)

School name National Cathedral School Our Lady of Victory School St. Ann’s Academy St. Anselm’s Abbey School St. John’s College High School Washington Middle School for Girls (THEARC & Washington View Campuses)

Appendix III: Total School and OSP Enrollment, School Year 2011-2012

Appendix III: Total School and OSP Enrollment, School Year 2011-2012 School name Academia De La Recta Porta Intl Christian Day School Beauvoir-National Cathedral Elem. School Bishop John T. Walker School for Boys Howard University Early Learning Program Kuumba Learning Center (MLK Campus)

School name Preparatory School of DC St. Ann’s Academy St. Anselm’s Abbey School St. John’s College High School Washington Middle School for Girls Dupont Park Adventist School has two campuses: Alabama Ave (grades PK-5) and Massachusetts Ave (grades 6-10). Washington International School has two campuses: Primary School Campus (grades PK-5) and Tregaron Campus (grades 6-12).

Washington Middle School for Girls has two campuses: THEARC Campus (grades 6-8) and Washington View Campus (grades 4-5).

Appendix IV: Internal Control Frameworks

Internal control is broadly defined as a process, effected by an entity’s board of directors, management and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories: effectiveness and efficiency of operations, reliability of financial reporting, and compliance with applicable laws and regulations.

According to the Committee of Sponsoring Organizations of the Treadway Commission (COSO), the five components of internal control for businesses and other entities are:

Control Environment: The control environment sets the tone of an organization, influencing the control consciousness of its people. It is the foundation for all other components of internal control, providing discipline and structure. Control environment factors include the integrity, ethical values and competence of the entity’s people; management’s philosophy and operating style; the way management assigns authority and responsibility, and organizes and develops its people; and the attention and direction provided by the board of directors. The control environment has a pervasive influence on the way business activities are structured, objectives established and risks assessed. It also influences control activities, information and communication systems, and monitoring activities. This is true not only of their design, but also the way they work day to day. The control environment is influenced by the entity’s history and culture. It influences the control consciousness of its people. Effectively controlled entities strive to have competent people, instill an enterprise-wide attitude of integrity and control consciousness, and set a positive “tone at the top.” They establish appropriate policies and procedures, often including a written code of conduct, which foster shared values and teamwork in pursuit of the entity’s objectives.

Risk Assessment: Every entity faces a variety of risks from external and internal sources that must be assessed. A precondition to risk assessment is establishment of objectives, linked at different levels and internally consistent. Risk assessment is the identification and analysis of relevant risks to achievement of the objectives, forming a basis for determining how the risks should be managed. Because economic, industry, regulatory and operating conditions will continue to change, mechanisms are needed to identify and deal with the special risks associated with change. All entities, regardless of size, structure, nature or industry, encounter risks at all levels within their organizations. Risks affect each entity’s ability to survive; successfully compete within its industry; maintain its financial strength and positive public image; and maintain the overall quality of its products, services and people. There is no practical way to reduce risk to zero. Indeed, the decision to be in business creates risk. Management must determine how much risk is to be prudently accepted, and strive to maintain risk within these levels. Objective setting is a precondition to risk assessment. There must first be objectives before management can identify risks to their achievement and take necessary actions to manage the risks. Objective setting, then, is a key part of the management process. While not an internal control component, it is a prerequisite to and enabler of internal control. This chapter first discusses objectives, followed by the discussion of risks.

Control Activities: Control activities are policies and procedures, which are the actions of people to implement the policies, to help ensure that management directives identified as necessary to address risks are carried out. They help ensure that necessary actions are taken to address risks to achievement of the entity’s objectives. Control activities occur throughout the organization, at all levels and in all functions. They include a range of activities including approvals, authorizations, verifications, reconciliations, reviews of operating performance, security of assets and segregation of duties. Control activities can be divided into three categories, based on the nature of the entity’s objectives to which they relate: operations, financial reporting, or compliance. Although some controls relate solely to one area, there is often overlap. Depending on circumstances, a particular control activity could help satisfy entity objectives in more than one of the three categories. For example, operations controls also can help ensure reliable financial reporting and financial reporting controls can serve to effect compliance. Although these categories are helpful in discussing internal control, the particular category in which a control happens to be placed is not as important as the role it plays in achieving a particular activity’s objectives.

Information and Communication: Pertinent information must be identified, captured and communicated in a form and time frame that enables people to carry out their responsibilities. Information systems produce reports, containing operational, financial and compliance- related information, that make it possible to run and control the business. They deal not only with internally generated data, but also information about external events, activities and conditions necessary to informed business decision-making and external reporting. Effective communication also must occur in a broader sense, flowing down, across and up the organization. All personnel must receive a clear message from top management that control responsibilities must be taken seriously. They must understand their own role in the internal control system, as well as how individual activities relate to the work of others. They must have a means of communicating significant information upstream. There also needs to be effective communication with external parties, such as customers, suppliers, regulators and shareholders. Every enterprise must capture pertinent information, financial and non-financial, relating to external as well as internal events and activities. The information must be identified by management as relevant to managing the business. It must be delivered to people who need it in a form and timeframe that enables them to carry out their control and other responsibilities.

Monitoring: Internal control systems need to be monitored, a process that assesses the quality of the system’s performance over time. This is accomplished through ongoing monitoring activities, separate evaluations or a combination of the two. Ongoing monitoring occurs in the course of operations. It includes regular management and supervisory activities, and other actions personnel take in performing their duties. The scope and frequency of separate evaluations will depend primarily on an assessment of risks and the effectiveness of ongoing monitoring procedures. Internal control deficiencies should be reported upstream, with serious matters reported to top management and the board. Internal control systems change over time. The way controls are applied may evolve. Once-effective procedures can become less effective, or perhaps are no longer performed. This can be due to the arrival of new personnel, the varying effectiveness of training and supervision, time and resource constraints or additional pressures. Furthermore, circumstances for which the internal control system originally was designed also may change, causing it to be less able to warn of the risks brought by new conditions. Accordingly, management needs to determine whether the internal control system continues to be relevant and able to address new risks. Monitoring ensures that internal control continues to operate effectively. This process involves assessment by appropriate personnel of the design and operation of controls on a suitably timely basis, and the taking of necessary actions. It applies to all activities within an organization, and sometimes to outside contractors as well. For example, with outsourcing of health claims processing to a third-party administrator, and such processing directly affects benefits’ costs, the entity will want to monitor the functioning of the administrator’s activities and controls.

GAO Standards for Internal Controlthe five standards of internal control: for federal agencies also comprise

Control Environment: Management and employees should establish and maintain an environment throughout the organization that sets a positive and supportive attitude toward internal controls and conscientious management. A positive control environment is the foundation for all other standards. Several key factors affect the control environment including the integrity and ethical values maintained and demonstrated by management and staff, management’s commitment to competence, and philosophy and operating style. In addition, the agency’s organizational structure and the manner in which the agency delegates authority and responsibility throughout the organization affect the control environment. Good human capital policies and practices are another critical environmental factor. A final factor affecting the control environment is the agency’s relationship with Congress and central oversight agencies such as the Office of Management and Budget. Congress mandates the programs that agencies undertake and monitors their progress and central agencies provide policy and guidance on many different matters. In addition, Inspectors General and internal senior management councils can contribute to a good overall control environment.

Risk Assessment: Internal control should provide for an assessment of the risks the agency faces from both external and internal sources. A precondition to risk assessment is the establishment of clear, consistent agency objectives. Risk assessment is the identification and analysis of relevant risks associated with achieving the objectives, such as those defined in strategic and annual performance plans developed under the Government Performance and Results Act, and forming a basis for determining how risks should be managed. Management needs to comprehensively identify risks using methods such as qualitative and quantitative ranking activities, management conferences, forecasting and strategic planning, and consideration of findings from audits and other assessments. Once risks have been identified, they should be analyzed for their possible effect. Because governmental, economic, industry, regulatory, and operating conditions continually change, mechanisms should be provided to identify and deal with any special risks prompted by such changes.

Control Activities: Internal control activities help ensure that management’s directives are carried out. The control activities should be effective and efficient in accomplishing the agency’s control objectives. Control activities are policies, procedures, techniques, and mechanisms that enforce management’s directives, such as the process of adhering to requirements for budget development and execution. They help ensure that actions are taken to address risks. Control activities are an integral part of an entity’s planning, implementing, reviewing, and accountability for stewardship of government resources and achieving effective results. Control activities occur at all levels and functions of the entity. They include a wide range of diverse activities such as approvals, authorizations, verifications, reconciliations, performance reviews, maintenance of security, and the creation and maintenance of related records which provide evidence of execution of these activities as well as appropriate documentation. Control activities may be applied in a computerized information system environment or though manual processes. Activities may be classified by specific control objectives, such as ensuring completeness and accuracy of information processing. Examples of control activities include top level reviews of actual performance, reviews by management at the functional or activity level, management of human capital, controls over information processing, physical control over vulnerable assets, establishment and review of performance measures and indictors, segregation of duties, proper execution of transactions and events, accurate and timely recording of transactions and events, access restrictions to and accountability for resources and records, and appropriate documentation of transactions and internal control.

Information and Communications: Information should be recorded and communicated to management and others within the entity who need it and in a form and within a time frame that enables them to carry out their internal control and other responsibilities. For an entity to run and control its operations, it must have relevant, reliable, and timely communications relating to internal as well as external events. Information is needed throughout the agency to achieve all of its objectives. Program managers need both operational and financial data to determine whether they are meeting their agencies’ strategic and annual performance plans and meeting their goals for accountability for effective and efficient use of resources. For example, operating information is required for development of financial reports. This covers a broad range of data from purchases, subsidies, and other transactions to data on fixed assets, inventories, and receivables. Operating information is also needed to determine whether the agency is achieving its compliance requirements under various laws and regulations. Financial information is needed for both external and internal uses. It is required to develop financial statements for periodic external reporting, and, on a day-to-day basis, to make operating decisions, monitor performance, and allocate resources. Pertinent information should be identified, captured, and distributed in a form and time frame that permits people to perform their duties efficiently. Effective communications should occur in a broad sense with information flowing down, across, and up the organization. In addition to internal communications, management should ensure there are adequate means of communicating with, and obtaining information from, external stakeholders that may have a significant impact on the agency achieving its goals. Moreover, effective information technology management is critical to achieving useful, reliable, and continuous recording and communication of information.

Monitoring: Internal control monitoring should assess the quality of performance over time and ensure that the findings of audits and other reviews are promptly resolved. Internal control should generally be designed to assure that ongoing monitoring occurs in the course of normal operations. It is performed continually and is ingrained in the agency’s operations. It includes regular management and supervisory activities, comparisons, reconciliations, and other actions people take in performing their duties. Separate evaluations of control can also be useful by focusing directly on the controls’ effectiveness at a specific time. The scope and frequency of separate evaluations should depend primarily on the assessment of risks and the effectiveness of ongoing monitoring procedures. Separate evaluations may take the form of self-assessments as well as review of control design and direct testing of internal control. Separate evaluations also may be performed by the agency Inspector General or an external auditor. Deficiencies found during ongoing monitoring or through separate evaluations should be communicated to the individual responsible for the function and also to at least one level of management above that individual. Serious matters should be reported to top management. Monitoring of internal control should include policies and procedures for ensuring that the findings of audits and other reviews are promptly resolved. Managers are to (1) promptly evaluate findings from audits and other reviews, including those showing deficiencies and recommendations reported by auditors and others who evaluate agencies’ operations, (2) determine proper actions in response to findings and recommendations from audits and reviews, and (3) complete, within established time frames, all actions that correct or otherwise resolve the matters brought to management’s attention. The resolution process begins when audit or other review results are reported to management, and is completed only after action has been taken that (1) corrects identified deficiencies, (2) produces improvements, or (3) demonstrates the findings and recommendations do not warrant management action.

Appendix V: Comments from the Department of Education

Appendix VI: Comments from the District of Columbia Children and Youth Investment Trust Corporation

Appendix VII: Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Gretta L. Goodwin, Assistant Director; Jamila Jones Kennedy; Grace Cho; and Michelle Loutoo Wilson made significant contributions to this report. In addition, key support was provided by Carl Barden, Hiwotte Amare, Kimberly McGatlin, Carla Craddock, David Chrisinger, Maria C. Belaval, Mimi Nguyen, Melinda Cordero, John Lopez, Helina Wong, Ramon Rodriguez, Alexander Galuten, Jean McSween, James Rebbe, Edward Bodine, Aron Szapiro, and Kristy Kennedy.