District of Columbia: Improved Reporting Could Enhance Management of the Tuition Assistance Grant Program
Released: 2018-09-06T00:00:00

Letter

Background

When DCTAG was created, there was no income eligibility requirement. However, in 2007, federal law limited eligibility to students from families with annual taxable incomes less than $1,000,000. In 2015, federal law further limited eligibility to students from families with annual taxable incomes less than $750,000; the law provided that this limit was to be subsequently adjusted for inflation as measured by the percentage increase, if any, in the Consumer Price Index for All Urban Consumers. For example, in academic year 2018, eligibility was limited to students from families with annual taxable incomes less than $762,000 (see textbox for selected eligibility requirements).

Complete the Free Application for Federal

Generally begin a course of study within 3 years of graduating high school or obtaining a General Equivalency Diploma. institution’s requirements for Satisfactory Academic Progress.

Universities (HBCU) nationwide and other participating private nonprofit institutions in the D.C. metropolitan area.

Populations Eligible for and Enrolled in DCTAG Remained Relatively Stable as Amounts Awarded Increased and Recipients Graduated at Higher Rates than Selected National Comparison Groups

We identified the following trends in eligibility for and enrollment in DCTAG and graduation rates for recipients:

DCTAG’s potentially eligible population. ACS data for calendar years 2007−2016 indicate that the population of high school students with incomes within DCTAG’s eligibility requirements has remained relatively stable. Over this time frame, about 25,000 students in D.C. were enrolled in high school each year, and about 90 percent of D.C. households had annual incomes less than $200,000. Additional households with annual incomes of $200,000 and above were also likely eligible for DCTAG based on income.

Enrollment in DCTAG. DCTAG program data indicate that the number of DCTAG recipients remained relatively stable over the last decade. DCTAG provided awards to an average of about 4,750 recipients annually over academic years 2007−2016 (see fig. 1). While enrollment in DCTAG peaked in academic year 2012, the number of DCTAG recipients in academic year 2016, the last year in our period of review, was similar to the number of recipients in academic year 2007, the first year in our period of review.

Enrollment in DCTAG by type of high school attended. DCTAG program data indicate the majority of recipients over academic years 2007–2016 graduated from D.C.’s public high school system—both traditional public schools and public charter schools. D.C.’s traditional public schools include six selective schools, or magnet schools, that limit admission to students that meet certain criteria or eligibility requirements. For example, in academic year 2016, more than 70 percent of DCTAG recipients graduated from D.C.’s public high school system (see fig. 2). Many DCTAG recipients have also graduated from private schools or schools outside D.C., were home schooled, or attained their General Equivalency Diploma. For academic years 2007−2016, between about 30 and 40 percent of DCTAG recipients came from high schools or programs outside the D.C. public school system.

Enrollment in DCTAG by taxable household income. Although in 2007 federal law limited eligibility for DCTAG to students from families with annual taxable incomes less than $1,000,000, DCTAG enrollment data show the program made awards to students from families with a wide range of household taxable incomes in academic years 2009−2016. At the same time, enrollment data indicate the program’s particular support for students from middle and lower income families. Nearly 60 percent of recipients over this time frame came from families with annual household taxable incomes of $50,000 or less (see fig. 3).

Enrollment in DCTAG by Ward. DCTAG program data indicate that for academic years 2007−2016, about 50 percent of recipients came from the three D.C. wards with the lowest median household incomes, according to American Community Survey estimates (see fig. 4).

Enrollment in DCTAG by attendance at 4-year and 2-year institutions. DCTAG program data show that for academic years 2007−2016, about 90 percent of DCTAG recipients attended 4-year institutions (see fig. 5). To counter the downward trend in enrollment at 2-year institutions that began in academic year 2013, OSSE officials told us they made programmatic changes to DCTAG for academic year 2018. Specifically, OSSE officials told us they determined out-of-state-tuition at 2-year public institutions attended by DCTAG recipients exceeded in-state tuition by an average of $4,500 per year. However, the maximum annual award for recipients attending these institutions was only $2,500. For academic year 2018, OSSE officials said they increased the maximum annual award to attend 2-year public institutions to $10,000 to close this gap.

Enrollment in DCTAG by amount awarded. For academic years 2007−2016, the percentage of recipients receiving DCTAG’s maximum annual awards increased from 40 percent to 62 percent (see fig. 6). OSSE officials linked an increase in the percentage of recipients receiving maximum awards to rising tuition at colleges and universities over this period. We analyzed data from IPEDS on average tuition and required fees at 4-year public institutions and our analysis confirmed that the average gap between out-of-state and in- state tuition exceeded DCTAG’s $10,000 maximum annual award starting in academic year 2015.

DCTAG graduation rates. College graduation rates are an important measure of performance for DCTAG. OSSE officials told us they maintain a program goal of helping recipients choose schools from which they are likely to graduate. For academic years 2012−2015, 6- year college graduation rates for DCTAG recipients were lower than those for students nationwide. However, OSSE officials reported that rates for recipients compare favorably to rates for national and regional groups of students with characteristics similar to those of DCTAG recipients. Our analysis confirmed that in academic year 2015, about 72 percent of DCTAG recipients were African-American and the DCTAG graduation rate was about 10 percentage points higher than for African-Americans nationwide. Similarly, in academic year 2015, nearly 40 percent of DCTAG recipients attended Historically Black Colleges and Universities (HBCU) and the DCTAG graduation rate was about 15 percentage points higher than for the nationwide population of students at these schools (see fig. 7). Additionally, OSSE officials estimated that more than 65 percent of DCTAG recipients were eligible for Pell Grants in academic year 2016. The National Center for Education Statistics recently started reporting graduation rates for Pell Grant recipients, beginning with the cohort of recipients that should have graduated by academic year 2016. Although not directly aligned, the academic year 2016 graduation rate for Pell Grant recipients nationwide was 48 percent— similar to the academic year 2015 graduation rate for DCTAG recipients.

DCTAG and Its Partners Help Recipients Prepare for College, Complete Applications for Financial Aid, and Stay on Track to Graduate

DCTAG partners with other entities to offer support services intended to help D.C. students prepare for college, apply for financial aid, and stay on track to graduate college. These partners include other entities within OSSE, as well as partners in the broader community such as public and private high school officials and college access providers. DCTAG provides some support services directly to students, such as individual counseling on how to complete a DCTAG application (see fig. 8). An OSSE official told us that DCTAG counselors instruct applicants and renewing recipients on tasks such as how to obtain required supporting documents to verify their residency in D.C. Additionally, to keep recipients on track to graduate, DCTAG emails recipients a quarterly newsletter with reminders to reapply for DCTAG and federal student aid so that they do not disrupt their studies by losing financial assistance. OSSE officials also said that DCTAG expands the reach of its support services by partnering with other entities within OSSE and in the community. For example, DCTAG works with OSSE’s Office of College and Career Readiness, whose mission is to increase D.C. public school students’ access to college. Through this collaboration, DCTAG helps eligible students prepare for higher education, such as through assistance to public schools to offer college entrance exams at no cost to students. Similarly, by partnering with college access providers, DCTAG supplements the support services it offers to help students stay on track to graduate. For example, DCTAG partners with the D.C. College Access Program, a privately funded scholarship program that offers support services for D.C. students in college. One of their services includes using scholarship recipients to mentor incoming D.C. students.

OSSE’s Reporting to Key Stakeholders on DCTAG Does Not Include Program Performance Information

We found that although OSSE communicates DCTAG’s program data and activities to internal stakeholders, Congress, and the public in various formats, these reporting methods do not include the program’s four goals (see textbox), relate performance information to these program goals, or describe progress toward achieving them (see table 1). For example, OSSE’s 2017 annual report to Congress on DCTAG did not include DCTAG’s four program goals, nor did OSSE relate information about the performance of the program to those goals. Instead, the 2017 annual report was comprised of descriptive statistics that were presented without explanation or sufficient context to allow readers to understand the significance of what was being reported. Specifically, this information was unrelated—quantitatively or qualitatively—to DCTAG’s program goals of ensuring D.C. residents are aware of and apply to DCTAG, or of helping DCTAG students make smarter college choices, which OSSE officials told us includes helping students select schools where they are more likely to graduate. As a result, it is unclear how to interpret the information presented in these reports and whether reported results indicate positive or negative program performance.

Federal standards for internal control state that program managers should communicate necessary quality information so both internal and external parties can help the program achieve its objectives. We have previously reported that annual reports are essential for managers of federal programs to communicate to decision makers the progress an agency has made toward achieving its goals during a given year and, in cases where goals are not met, identify opportunities for improvement or whether goals need to be adjusted. In addition, our prior work found that managers of these programs can increase the value of their reports to congressional decision makers and the public by relating annual performance information to the agency’s strategic goals and mission. Furthermore, we reported that performance measurement does not require establishing a causal link between program activities and program outcomes, but rather emphasizes that the nature of performance measurement is the ongoing monitoring and reporting of program accomplishments, particularly toward pre-established goals.

OSSE officials agreed on the importance of developing an annual report relating performance to program goals for the DCTAG program and concurred with our finding that they had not communicated DCTAG’s performance information, such as progress toward program goals, in a single annual report. They explained that developing performance measures is challenging. For example, they said DCTAG recipients have access to multiple support programs, which creates difficulties in establishing causal links between a program and the desired outcome. OSSE officials also stated that many DCTAG initiatives are new and, as a result, complete data on those initiatives are not yet available. Although we recognize that developing an annual report could be challenging, our prior work has found performance measurement guidelines would not require program managers to establish causal links as part of ongoing performance monitoring and reporting of progress toward program goals. Unless DCTAG’s stakeholders have access to an annual report that relates performance information to the program’s goals, they may be limited in their ability to judge the significance of what is being reported, determine whether the agency is making progress toward achieving its goals, or make informed program management and funding decisions.

The Design of Selected Scholarship Programs Reflects Unique State and Local Needs

Each of the three other selected scholarship programs we reviewed was created to meet unique state or local needs.

Boston Tuition-Free Community College Plan. Created to make college more affordable for the city’s low-income students.

Kalamazoo Promise. Created to promote the economic and social well-being of the community by expanding college access with full- tuition scholarships.

Washington State Opportunity Scholarship. Created to address shortfalls in the state’s Science, Technology, Engineering, and Mathematics (STEM) and health care workforce and increase educational opportunities for low-income and middle-income students.

Because each program was designed to address a unique state or local need, they differ with regard to eligibility, funding, recipient supports, and outcome measures. (For additional information on these three programs see appendix I.)

Eligibility. Each of the three selected scholarship programs established eligibility criteria, such as income requirements, residency requirements, and grade point average (GPA) requirements among others, that reflect program objectives. For example, to ensure that the Boston program serves the intended low-income population, the program requires students to be eligible for Pell grants to receive funding.

Funding. While the selected scholarship programs have dedicated funding streams, their funding sources reflect the origins of each program. For example, Boston’s program was initiated by the city’s mayor and is funded through a public charitable trust from fees for large-scale commercial building projects while the Kalamazoo promise was initiated by a group of anonymous donors who have funded the program in perpetuity, according to program officials. Alternately, the Washington program was initiated through cooperation between the state government and private sector companies and is funded by private donations that are matched by state funds up to an annual maximum of $50 million.

Recipient supports. Each of the selected scholarship programs have developed supports such as coaching and peer mentoring to help recipients transition to college and stay on track to graduate. For example, the Kalamazoo Promise partners with and provides funding to two local colleges to create counseling, coaching, or peer mentoring services for scholarship recipients, according to program managers.

Outcome measures. The selected scholarship programs have developed outcome measures to better understand the programs’ impact, such as whether students stay on track to graduate or find employment post-graduation. For example, program managers with the Washington program said they initiated a post-graduation survey in 2015 to better understand the employment status of graduates in STEM and health care fields, their job location, and annual salary.

Conclusions

Steady enrollment in DCTAG provides an encouraging signal that the program may be meeting the purpose set forth in federal law to expand access to higher education opportunities for D.C. students. However, without annual reports that relate DCTAG’s performance information to the program’s goals, it is difficult to assess the impact of the program and its support services. The information OSSE currently makes available about DCTAG does not provide the context needed for the program’s internal stakeholders, Congress, or the public to determine whether the program is meeting its goals or if any changes may be necessary.

Recommendation for Executive Action

OSSE should issue an annual report on DCTAG that relates information about the program’s performance to the program’s goals. (Recommendation 1)

Agency Comments and Our Evaluation

We provided a draft of this report to the Mayor of the District of Columbia for review and comment. Comments from the Mayor are reproduced in appendix II. In response to our recommendation, the Mayor stated that OSSE plans to expand DCTAG’s current annual reports to Congress with direct links to DCTAG’s annual strategic performance goals and the reports will combine data points to illustrate the program’s performance. The Mayor also raised a concern about the title of the draft report, stating that it implied OSSE is not meeting legislative requirements. We have modified the title and text of the report to avoid this implication.

We are sending copies of this report to the appropriate congressional committees, the District of Columbia Office of the State Superintendent of Education, and other interested parties. In addition, this report is available at no charge on the GAO website at http://www.gao.gov.

Appendix I: Other State and Local Scholarship Programs

Based on interviews with officials of three selected state and local scholarship programs and a review of program documents, we present a selection of information to provide additional context on these programs. They include the Boston Tuition-Free Community College Plan, the Kalamazoo Promise, and the Washington State Opportunity Scholarship. The following tables include information on these scholarship programs’ eligibility requirements, funding sources, recipient supports, and annual reports and performance measures. Table 2 presents a selection of eligibility requirements for the Boston Tuition-Free Community College Plan, the Kalamazoo Promise, and the Washington State Opportunity Scholarship.

Table 3 presents a summary of the three selected scholarship programs’ funding sources, as well as how students may use those funds.

Table 4 presents a summary of the supports developed by the three selected scholarship programs to support students, keep them on track to graduate from college, and help them begin their careers.

Table 5 presents a summary of the annual reports and selected performance measures developed by the three selected scholarship programs.

Appendix II: Comments from the Mayor of the District of Columbia

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Melissa Emrey-Arras, (617) 788-0534 or emreyarrasm@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Bill J. Keller (Assistant Director), Tom Moscovitch (Analyst-in-Charge), and Michael C. Duane made significant contributions. Also contributing to this report were James Bennett, Deborah K. Bland, Sheila R. McCoy, Benjamin A. Sinoff, Rachel R. Stoiko, and Kate van Gelder.