Performance-based Funding 

In Adult Education 

Prepared for: 

U.S. Department of Education 

Office of Vocational and Adult Education 

Division of Adult Education and Literacy 

Submitted by: 

MPR Associates, Inc. 

August 2007
This report was produced under U.S. Department of Education Contract No. ED04CO-0121/0001 with MPR 

Associates, Inc. Trudy R. Turner served as the contract officer’s representative. The views expressed herein do not 

necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. 

Department of Education of any product commodity services or enterprise mentioned in this publication is intended 

or should be inferred. 

U.S. Department of Education

Margaret Spellings 

Secretary 

Office of Vocational and Adult Education

Troy R. Justesen 

Assistant Secretary 

Division of Adult Education and Literacy

Cheryl L. Keenan 

Director 

August 2007 

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to 

reprint this publication is not necessary, the citation should be: U.S. Department of Education, Office of Vocational 

and Adult Education, Division of Adult Education and Literacy, Performance-based Funding in Adult Education, 

Washington, D.C., 2007. 

To order copies of this report, 

write to: ED Pubs, Education Publications Center, U.S. Department of Education, P.O. Box 1398, Jessup, MD 

20794-1398; 

or fax request to: 301-470-1244; 

or e-mail your request to: edpubs@inet.ed.gov; 

or call in your request toll free: 1-877-433-7827 (1-877-4-ED-PUBS). Those who use a telecommunications device 

for the deaf (TDD) or a teletypewriter (TTY) should call 1-877-576-7734. If 877 is not yet available in your area, 

call 1-800-872-5327 (1-800-USA-LEARN; TTY: 1-800-437-0833); 

or order online at: www.edpubs.ed.gov. 

This report is also available on the Department’s Web site at: www.ed.gov/offices/ovae/dael. 

On request, this publication is available in alternate formats, such as Braille, large print or computer diskette. For 

more information, please contact the Department’s Alternate Format Center 202-260-0852 or 202-260-0818. 
— iii — 

Contents 

Page 

Executive Summary................................................................................................................ ES-1 

Section 1: Purpose of the Study ....................................................................................................1 

Section 2: Overview of State Adult Education Funding Systems..............................................3 

Indiana........................................................................................................................................3 

Kansas........................................................................................................................................5 

Missouri .....................................................................................................................................6 

Section 3: Rationale for System Adoption...................................................................................9 

Contextual Factors Influencing Formula Adoption ...................................................................9 

Federal Accountability Requirements..................................................................................9 

State Programmatic Factors ...............................................................................................10 

State Goals in Introducing Performance-based Funding .........................................................10 

Equity.................................................................................................................................11 

Efficiency...........................................................................................................................11 

Accountability....................................................................................................................12 

Program Improvement .......................................................................................................13 

Section 4: Formula Design Process ............................................................................................15

Formula Development .............................................................................................................16 

Formula Construction ..............................................................................................................17 

Resource Allocation...........................................................................................................17 

Performance Measures.......................................................................................................17 

Category 1: Core and Secondary Measures Included in the 

National Reporting System...........................................................................................18 

Category 2: Process Measures of Program Performance .............................................19 

Category 3: Provider Attainment of Performance Standards .......................................20 

Formula Weighting ............................................................................................................21

Educational Gain......................................................................................................................21 

Core Versus Secondary Measures ...........................................................................................23 

Timing of Data Collection .................................................................................................23 
Contents 

— iv — 

Page 

Section 5: Implementation Issues ...............................................................................................25 

Formula Rollout.......................................................................................................................25 

Provider Reporting Capacity....................................................................................................26 

Data Quality ............................................................................................................................27 

Management Information Systems .........................................................................................28 

Technical Assistance................................................................................................................29 

Section 6: Effects of Performance-based Funding....................................................................31 

Advantages of Performance-based Funding ............................................................................31 

Evidence of Performance Improvement ..................................................................................34 

Changes in Local Adult Education Providers....................................................................37 

Section 7: Considerations in Transitioning to Performance-based Funding .........................39 

Key Considerations in Formula Design...................................................................................40 

How Much Funding Should Be Allocated Using Performance Criteria?..........................40 

How Should Funding Formulas Be Structured? ................................................................40 

What Factors Should Be Considered in Formula Implementation? ..................................44 

Appendix A: Indiana Case Study Report: Performance-based Funding in 

Adult Education ...........................................................................................................................47 

Section A-1: Overview of the State Adult Education Funding System...................................49 

State Allocations ......................................................................................................................51 

Organization of State Adult Education Services .....................................................................52 

Local Providers ..................................................................................................................53 

Section A-2: Rationale for System Development ......................................................................55 

State Adoption Process ............................................................................................................55 

Goals in Introducing Performance-based Funding ..................................................................56 

Section A-3: Formula Design Process ........................................................................................57 

Specification of Funding Criteria: State Funding ....................................................................57 

Specification of Funding Criteria: Federal Funding ................................................................58 

Assessing Formula Effects.......................................................................................................60 

Section A-4: Implementation Process ........................................................................................61 

Provider Reporting Capacity....................................................................................................61 

Data Quality.............................................................................................................................62 

Management Information Systems ..........................................................................................62 

Technical Assistance................................................................................................................63 
Contents 

— v — 

Page 

Section A-5: Effects of Performance-based Funding................................................................65 

Contributions to Statewide Performance .................................................................................65 

Drawbacks................................................................................................................................67 

Effect of PBF on Adult Education Providers...........................................................................69 

Fiscal Eligibility.................................................................................................................69 

Program Outcomes.............................................................................................................73

System Evolution.....................................................................................................................75 

Appendix A Summary .................................................................................................................77 

Appendix B: Kansas Case Study Report: Performance-based Funding in 

Adult Education ...........................................................................................................................81 

Section B-1: Overview of the State Adult Education Funding System...................................83 

State Allocations ......................................................................................................................84 

Organization of State Adult Education Services .....................................................................86 

Local Providers ..................................................................................................................86 

Section B-2: Rationale for System Development.......................................................................89 

Contextual Factors Influencing Formula Adoption .................................................................89 

State Adoption Process ............................................................................................................90 

Goals in Introducing Performance-based Funding ..................................................................91 

Section B-3: Formula Design Process ........................................................................................93 

Specification of Funding Criteria: Base Funding ....................................................................93 

Specification of Funding Criteria: Performance Funding........................................................94 

Specification of Funding Criteria: Program Quality................................................................95 

Assessing Formula Effect ........................................................................................................96 

Section B-4: Implementation Process.........................................................................................97 

Provider Reporting Capacity....................................................................................................97 

Data Quality.............................................................................................................................98 

Management Information Systems ..........................................................................................99 

Technical Assistance..............................................................................................................100 

Section B-5: Effects of Performance-based Funding..............................................................101 

Contributions to Statewide Performance ...............................................................................101 

Drawbacks..............................................................................................................................103 
Contents 

— vi — 

Page 

Effect of PBF on Adult Education Providers.........................................................................104 

Fiscal Eligibility...............................................................................................................105 

Program Outcomes...........................................................................................................105 

System Evolution...................................................................................................................108 

Appendix B Summary ...............................................................................................................111

Appendix C: Missouri Case Study Report: Performance-based Funding in 

Adult Education .........................................................................................................................115 

Section C-1: Overview of the State Adult Education Funding System.................................117 

State Allocations ....................................................................................................................118 

Organization of State Adult Education Services ...................................................................119 

Local Providers ................................................................................................................120 

Section C-2: Rationale for System Development ....................................................................123 

State Adoption Process ..........................................................................................................124 

Goals in Introducing Performance-based Funding ................................................................124 

Section C-3: Formula Design Process.............................................................................127 

Specification of Funding Criteria: Core Funding ..................................................................128 

Specification of Funding Criteria: Performance Funding......................................................130 

Assessing Formula Effect ......................................................................................................131 

Section C-4: Implementation Process.............................................................................133 

Provider Reporting Capacity..................................................................................................133 

Data Quality...........................................................................................................................134 

Management Information Systems ........................................................................................134 

Technical Assistance..............................................................................................................135 

Section C-5: Effects of Performance-based Funding..............................................................137

Contributions to Statewide Performance ...............................................................................137 

Drawbacks..............................................................................................................................139 

Effect of PBF on Adult Education Providers.........................................................................141 

Fiscal Eligibility...............................................................................................................141 

Program Outcomes...........................................................................................................146 

System Evolution...................................................................................................................147

Appendix C Summary ...............................................................................................................149 
Contents 

— vii — 

Page 

Appendix D: Study Methodology and Analysis Procedures..................................................153 

Section D-1: Study Methodology ..............................................................................................155

Case Study Visits ...................................................................................................................155 

Interviews...............................................................................................................................156 

Site Meetings .........................................................................................................................159 

Indiana..............................................................................................................................159 

Kansas..............................................................................................................................160 

Missouri ...........................................................................................................................161 
— viii — 

Tables 

Table Page 

ES-1 Selected Characteristics of State Performance-based Funding Systems for 

Indiana, Kansas, and Missouri: FY05............................................................................ES-2 

1 NRS Core and Secondary Measures Used in Three State Performance-based 

Funding Formulas, by State ...............................................................................................18 

2 Indiana Performance Standard Incentive Rates as Based on Federal Funding and 

Numbers of Statewide Outcomes: FY01 to FY05 .............................................................21 

3 Missouri Performance Outcome Rates, by Education Functioning Level: FY02 and 

FY03 to Present..................................................................................................................22 

4 Percentage of Adult Learners Nationally and in Three States Achieving Outcomes 

on the NRS Core Performance Measures by Types of Measure: Program Years 

2000–01 to 2003–04 ..........................................................................................................36 

5 Selected Indicators of Indiana Adult Education Programs: Years 2000–01 to 

2004–05..............................................................................................................................37 

A-1 Indiana Adult Education Funding Allocated as Base or Performance Funding: FY05.....52 

A-2 Selected Characteristics of Indiana Adult Education Providers and Participants: 

FY00 to FY05 ....................................................................................................................54 

A-3 Indiana Performance Outcome Rate and Performance Goal Rate: FY01 to FY05 ...........59 

A-4 Indiana Combined Adult Education Performance and Incentive Awards, Percentage 

and Dollar Amount Change by Provider: FY01 to FY05..................................................71 

A-5 Total Change in Indiana Adult Education Funding, by Comprehensive Program 

District: FY00 to FY06 ......................................................................................................72 

A-6 Indiana Statewide Adult Education Participants, Providers, and Performance 

Outcomes and Goals: FY01 to FY05.................................................................................73 

A-7 Total Number of Indiana Adult Education Performance Outcomes Achieved and 

Percentage Change, by Comprehensive Program District: FY01 to FY05 .......................74 
— ix — 

Table Page 

A-8 Total Number of Indiana Adult Education Performance Goals Achieved, by 

Comprehensive Program District: FY01 to FY05 .............................................................76 

B-1 Kansas Adult Education Federal and State Funding Allocated as Base or 

Performance Funding: FY05..............................................................................................85 

B-2 Selected Characteristics of Kansas Adult Education Providers and Participants: 

FY99 to FY04 ....................................................................................................................87 

B-3 Kansas Local Provider Funding Eligibilities and Percentage Change Between 

Selected Fiscal Years: FY00 to FY04..............................................................................107 

B-4 Selected Performance Characteristics of Kansas Adult Education Participants, 

FY99 to FY04, and Percentage Change, FY01 to FY04 .................................................108 

C-1 Missouri Adult Education Funding Allocated as Base or Performance Funding: 

FY05 ................................................................................................................................118 

C-2 Selected Characteristics of Missouri Adult Education Providers and Participants: 

FY01 to FY05 ..................................................................................................................120 

C-3 Missouri Contact-Hour Rate and Total State Obligation: FY02 to FY06 .......................128 

C-4 Missouri Levels of Advancement Performance Calculator Used to Award 

Performance-based Funding: FY02 .................................................................................131 

C-5 Missouri Levels of Advancement Performance Calculator Used to Award 

Performance-based Funding: FY05 .................................................................................131 

C-6 Missouri Adult Education Expenditures Prior to and After PBF and Percentage 

Change, by Provider: FY01 to FY05 ...............................................................................142 

C-7 Missouri Adult Education Core Expenditures to Providers: FY02 to FY05 ...................144 

C-8 Missouri Adult Education Performance-based Funding Expenditures, by Provider: 

FY02 to FY05 ..................................................................................................................145 

C-9 Selected Characteristics of Missouri Adult Education Level Completers: FY01 to 

FY05 ................................................................................................................................146 

C-10 Selected Participation Characteristics of Missouri Adult Education Participants by 

ABE, ESL, and ASE Status: FY01 to FY05....................................................................147 

D-1 Change in Performance Award for Indiana, by Provider: FY01 to FY05.......................159 
— x — 

Table Page 

D-2 Change in Performance Award for Kansas, by Provider: FY00 to FY04 .......................160 

D-3 Change in Performance Award for Missouri, by Provider: FY01 to FY05.....................161 
— xi — 

Figures 

Figure Page 

A-1 Percentage of Indiana Local Provider Allocation by Funding Type: FY05 ......................52 

A-2 Ages of Indiana Adult Education Participants: FY05........................................................54 

B-1 Percentage of Kansas Adult Education Funding Allocated by Funding Type: FY06 .......85 

B-2 Ages of Kansas Adult Education Participants: FY05........................................................87 

C-1 Percentage of Missouri Local Provider Allocation by Funding Type: FY05..................119 

C-2 Ages of Missouri Adult Education Participants: FY05 ...................................................121 

Exhibits 

Exhibit Page 

D-1: Letter to State Directors of Adult Education Requesting Selected Information for 

Cross-case PBF Study..........................................................................................................163 

D-2: Case Study Site Visit Agenda..............................................................................................167 

D-3: Audiotape Consent Form.....................................................................................................169 

— ES-1 — 

Executive Summary 

The Adult Education and Family Literacy Act (AEFLA), Title II of the Workforce Investment Act 

of 1998, requires states to establish a comprehensive performance accountability system to assess 

the effectiveness of eligible agencies in making continuous improvement in their adult education 

and literacy activities. To motivate local adult education providers to improve the quality and 

effectiveness of their services, some states are adopting performance-based funding (PBF) 

systems to allocate federal, state, or both, types of adult education resources. 

Performance-based funding systems are designed to distribute resources to local providers based 

on state-defined criteria, which may include learner, administrative, or other programmatic 

measures. Performance outcomes are used to determine how much money a provider receives 

and, in some circumstances, whether funds should be added or deducted from a provider’s 

allocation. 

To gain a better understanding of how these systems operate, the Office of Vocational and Adult 

Education (OVAE), U.S. Department of Education, commissioned a study of adult education 

fiscal policies in selected states using PBF to allocate program resources. This cross-case 

analysis report summarizes findings from case study site visits conducted in three states—

Indiana, Kansas, and Missouri—which are among the most experienced in using performancebased funding to allocate federal, state, or both, types of resources. It is intended to assist state 

policymakers and adult education administrators in making more informed decisions when 

designing state allocation formulas using performance funding. 

In particular, this report documents commonalities in the design and implementation of PBF 

systems across study states, including lessons learned and challenges faced during system startup. The paper opens with an overview of funding approaches used in study states and the 

rationale for formula adoption. A description of the formula design process follows, including an 

account of the individuals involved in system development and the measures included in formula 

construction. The report next reviews implementation issues affecting formula rollout and the 

effects of performance funding on state adult education providers. A description of key issues 

state administrators should consider when making the transition to performance funding 

concludes the report. 
Executive Summary 

— ES-2 — 

Organization of State Performance-based Funding Systems 

States using performance-based funding to allocate adult education resources have developed 

unique approaches for allocating resources among eligible providers. As documented below, 

states selected for this study are representative of the diverse funding approaches used in the 

field, both in the amount of resources allocated using performance criteria and in their types of 

measures and distribution approach (table ES-1). 

All three states support the funding of adult education with a base allocation combined with 

performance funding. State strategies for allocating federal and state resources using 

performance funding are outlined below. 

Indiana—limits performance-based awards to federal resources, so that only a small percentage 

(5 percent) of total state and federal resources are allocated based on performance. Federal 

Indiana Kansas Missouri

Adult Education Funding ($) $21,055,280 $4,458,990 $12,040,787

Number of Providers 43 31 44

 Community Colleges 1 17 10

 Secondary Districts 41 11 31

 Othera 133

Percentage of Resources Allocated Using PBF

 Total 5% 88% 19%

 Federal 15% 100% 22%

 State 0% 50% 10%

Performance Measures

 Core NRSb 15 15 11

 Secondary NRS 7 3 0

 Quality Criteriac 0 25 0

 Incentive Funding 15 0 0

c

 A set of 10 process measures encompassing 25 state-established submeasures that assess program adoption of strategies that are

Table ES-1—

Selected Characteristics of State Performance-based Funding Systems for Indiana, Kansas, and Missouri: 

FY05

b

 Includes 11 submeasures of educational gain in Adult Basic Education (ABE), Adult Secondary Education (ASE), and English as a

SOURCE: Data files provided by Indiana Department of Education, 2006; Kansas Board of Regents, 2006; Missouri Department of 

Elementary and Secondary Education, 2006.

associated with high performance.

a

 Other includes: community-based and faith-based providers.

Second Language (ESL) programs.
Executive Summary 

— ES-3 — 

performance resources are allocated based on the number of adult learners achieving outcomes 

on each of the core and secondary measures contained within the National Reporting System for 

Adult Education (NRS), the accountability system for the AEFLA. Incentive grants also are 

provided for agencies meeting their state-negotiated performance levels for the core NRS 

measures. To align outcomes with state system goals, the amount awarded for a secondary NRS 

performance outcome is half that awarded for an outcome on a core NRS measure. 

Kansas—allocates a large percentage (88 percent) of its total state and federal funds based on 

provider performance. Fifty percent of federal funds are allocated based on the number of learner 

outcomes on the core and selected secondary measures of the NRS, with educational gain 

outcomes doubled for learners in the five lowest educational functioning levels (i.e., indicators of 

skill attainment). Fifty percent of federal and 50 percent of state funds are allocated based on the 

number of quality points a program earns relative to the statewide total generated in that year. 

Quality points are awarded based on provider performance on 25 measures encompassing 10 

state-established quality indicators. 

Missouri—distributes a modest share (19 percent) of its federal and state resources using 

performance-based funding. Performance resources are distributed based on two core NRS 

measures: the number of individuals who achieved an educational gain and those who received a 

general equivalency diploma (GED) diploma. The state attaches different dollar amounts to 

provider outcomes, with additional resources provided for outcomes achieved by lower-level 

Adult Basic Education (ABE) and English as a Second Language (ESL) learners. 

Rationale for Performance-based Funding 

Before the introduction of PBF, the three case study states funded adult education primarily 

based on enrollment, with program eligibility determined by the total number of learner-contact 

hours generated during the preceding program year. During interviews, state administrators and 

program directors in all three states related their frustration with contact-hour formulas, 

suggesting that this mechanism provided educators a greater incentive to fill seats than to 

manage enrollments, undertake staff development, or initiate curricular reforms that could 

improve student outcomes. 

The introduction of PBF was initiated by state directors of adult education to improve reporting 

on the accountability requirements contained in AEFLA and to address programmatic concerns 

about the delivery of adult education services. To guide formula drafting efforts, state 

administrators specified statewide goals for the adoption of PBF systems. Generally, these goals 

called for the creation of a more open, transparent system that would allocate resources in a fair 

and efficient manner. State goals fell into four broad areas: 
Executive Summary 

— ES-4 — 

• Equity—Administrators sought to develop fair and equitable systems for 

allocating resources to ensure that all providers had equal opportunity to compete 

for federal, state, or both, resources. 

• Efficiency—Performance funding systems were intended to channel resources to 

the most effective providers of instructional services, thereby maximizing the 

return on state investment. 

• Accountability—Formula development teams selected performance measures that 

would support the state in achieving its federally negotiated performance targets 

on the NRS measures. 

• Program Improvement—Performance funding systems were intended to motivate 

program directors to initiate program improvement efforts. 

Formula Design Process 

State administrators in all three states championed performance funding as a means of improving 

program effectiveness and system efficiency. Formula development typically proceeded under 

the direction of the state director, who provided a vision for PBF; recruited influential program 

directors—representative of local providers throughout the state—to serve on an advisory 

council; supervised formula drafting efforts; and modeled funding scenarios under different 

assumptions. 

The construction of state funding formulas was a roughly yearlong process, with advisory 

committee members meeting between three to five times over a six- to 12-month period. To 

promote understanding and support in the field, state administrators kept local providers apprised 

of formula development efforts at statewide or regional adult education conferences. Advisory 

council members also were asked to share their experiences with their colleagues so that there 

would be no surprises once the formula design process was complete. 

Study states have combined performance measures contained in AEFLA with their own stateestablished measures to allocate performance funding. State measures fall into three categories: 

• Core and Secondary NRS Measures—All three states make use of the core and 

secondary NRS measures to allocate all or a substantial portion of their state 

resources. 

• Process Measures of Program Performance—Kansas is unique in using a set of 

25 subindicators of program quality to award performance funding, with points 
Executive Summary 

— ES-5 — 

awarded based on whether a program is rated as having a high, medium, or low 

level of quality on each measure.

• Provider Attainment of Performance Standards—Indiana is the only state to 

award incentive funding to providers who meet or exceed their state-negotiated 

performance standards. Incentive funding is based on 15 performance measures 

associated with the core NRS measures.

To ensure that providers offer services to all students, administrators in each state assigned 

different weights to learner outcomes. The weights vary by state, with Kansas and Missouri 

attaching additional resources for lower-level and ESL achievement, while Indiana assigns a 

different value to learners who achieve an outcome associated with a core versus a secondary 

NRS measure. 

Implementation Issues 

States fully implemented their PBF systems in the year following formula development. To 

offset funding losses associated with PBF adoption, state administrators supplemented providers’ 

initial performance allocations using additional federal resources made available to states with 

the adoption of AEFLA. Funding simulations conducted during formula development meetings in 

all three states also indicated that there would be few losers among providers. Those projected to 

experience a drop in resources were primarily larger providers that could absorb anticipated 

losses. 

Study states were among the earliest implementers of PBF in the country, adopting performance 

funding concurrent with or immediately following the enactment of the AEFLA in 1998. During 

this transition period, local providers initially struggled to collect accurate NRS performance 

outcome data. Provider challenges included correctly administering tests of educational gain to 

learners so that test results were valid, and correctly using new, state-supplied management 

information system software intended to assist them in administering learner files and in 

collecting accountability data. 

Although providers initially had some problems collecting performance data, state directors 

noted that, with time and training, reporting difficulties were surmounted. Moreover, because all 

three states used the NRS measures to distribute some or all of their performance funding, 

directors believed that initial difficulties resulted from provider misunderstanding of the new 

federal reporting requirements, rather than the operation of the funding formula itself. States now 

considering using the NRS (or other state) measures may be able to avoid much of the confusion 

associated with system start-up, since providers have had over half a decade of experience 

collecting data for the NRS. 
Executive Summary 

— ES-6 — 

State adult education directors reported that the adoption of PBF had little effect on state-level 

investment in technology or personnel, since the majority of state performance measures overlap 

those used for the AEFLA. To standardize data collection for the NRS measures, state adult 

education administrators in all three states supplied providers with database software and 

guidance on entering participant data into the system. Although program directors reported that 

the adoption of state NRS software required that they make changes to their information systems, 

no additional technology costs were associated with performance reporting. That is, had 

performance reporting not been introduced, providers still would have been required to 

standardize data administration to report on AEFLA-mandated accountability measures. 

State administrators did not explicitly provide technical assistance on the operation of 

performance funding formulas. Rather, state training initially focused on familiarizing local 

program provider staff with new AEFLA reporting requirements, explaining the appropriate 

strategies for administering tests and collecting performance data, and entering data into statesupplied management information system software. Over time, as local familiarity with reporting 

systems has increased, state administrators have been able to target technical assistance to 

address methodological issues undercutting program reporting. 

Effects of Performance-based Funding 

Although state personnel interviewed for this study ascribed both positive and negative 

consequences to performance funding, few if any data exist to substantiate their claims. In many 

cases, participant observations were based on general perceptions of program operations or 

extrapolations from a single experience. Some participants also had difficulty differentiating 

programmatic effects resulting from the adoption of PBF from those associated with state 

implementation of the AEFLA. As a consequence, it is difficult to draw conclusions about the 

effects of PBF on state adult education systems. 

Participants’ observations, however, suggest that performance funding helped states achieve their 

intended goals and also reaped unexpected benefits in terms of data quality and its use for 

program improvement purposes. Specific contributions included: 

• Improves Data Quality—Linking resources to the core and secondary NRS 

measures appears to have given local directors a fiscal incentive to review the 

accuracy and completeness of information submitted to the state. 

• Increases System Effectiveness—Compensating providers for results has focused 

local directors’ attention on program performance, and, in particular, on making 

connections between programs, instructors, and learner outcomes. 
Executive Summary 

— ES-7 — 

• Engenders Political Support—State directors reported that system adoption 

bolstered their credibility among state legislators and the public, while providing 

an effective way of winning legislative support for program operations. 

• Promotes Instructor Professionalism—Program directors believed that 

performance funding contributes to program quality by holding instructional staff 

accountable for their learners’ outcomes because they now must demonstrate 

results to be considered effective. 

State directors believe that the adoption of PBF contributed to improving state outcomes on the 

core NRS measures by encouraging providers to improve the quality of their instruction and 

administration of data. A review of national performance data conducted for this report appears 

to support this claim, with study states outperforming the national average on many of the NRS 

measures and registering substantial improvements on individual measures across years. 

Unfortunately, it is not possible using existing data to ascribe causality to PBF in improving state 

outcomes. That is, although performance funding is associated with positive changes in state 

performance, it is possible that a host of other factors, such as population changes, learner access 

to other educational services, curricular innovations, and other education reform efforts have 

influenced state outcomes as well. Consequently, while state performance suggests that there 

may be changes in how states were performing prior to and immediately following the adoption 

of performance funding, it is impossible to attribute program successes directly to its use. 

Considerations in Making the Transition to Performance-based Funding 

Introducing PBF into state adult education funding formulas is a complicated process, one that 

requires that state directors and their staff commit time and resources to overseeing formula 

drafting and implementation efforts. States that have succeeded in adopting PBF have developed 

various strategies for allocating adult education resources because no one formula will work in 

all state contexts. Regardless of the approach used to distribute resources, all states face similar 

issues in designing their funding systems. Administrators in states considering using PBF can 

improve the likelihood of success by answering some of the following questions during the 

development process.
Executive Summary 

— ES-8 — 

• How Much Funding Should Be Allocated Using Performance Criteria?—

Ultimately, funding allocated using PBF depends upon state conditions, including 

the amount of federal and state resources available, the manner in which funds are 

distributed, the perceived need for statewide adult education reform, and the 

appetite for risk among state administrators and providers. Funding levels also 

should account for state capacity to support providers transitioning to the new 

funding system, as well as state capability to increase funding concurrent with 

system rollout, should the state seek to hold harmless providers projected to lose 

funding.

• How Should Funding Formulas Be Structured?—On balance, the less complexity 

associated with PBF the easier state systems are to understand and administer. 

Ideally, program measures and formula operation will align with and reinforce 

state policy priorities that have been identified, while limiting the reporting 

burden on providers. Care also must be taken to ensure that providers with unique 

characteristics, including those serving small and rural populations or large 

numbers of lower-level or ESL learners, can compete for resources on a level 

playing field. 

• How Should Formulas Be Implemented?—State administrators should consider 

whether precautionary steps are warranted to protect providers from sudden 

changes in their resource eligibility, such as by attaching a hold-harmless 

provision to initial allocations. The decision of whether or not to adopt funding 

“circuit breakers” should be based on state conditions, including changes in 

federal and state resources for adult education, the availability of held-back 

federal funds, and results from preliminary modeling of PBF allocations. 

States also should sponsor technical assistance workshops to assist providers in 

understanding how the state funding formula operates, how to analyze 

performance data to identify areas in need of improvement, and how to initiate 

programmatic changes to improve their funding eligibility. Given the high stakes 

associated with provider data submissions, administrators considering adopting 

PBF also should plan to review the quality of program data and put providers on 

notice of the importance of submitting accurate information. 

Summary 

Performance-based funding has been adopted by states to improve the fairness and efficiency of 

state adult education resource allocation. This study of states experienced in using PBF suggests 

that both state administrators and local program directors believe that these systems can help 

improve learner outcomes and the provision of program services. Study findings suggest that 
Executive Summary 

— ES-9 — 

state administrators face similar issues when designing and implementing these systems, and that 

states successful in adopting these systems involve providers in system development and in 

modeling system effects, and provide technical and material support to program staff. 

— 1 — 

SECTION 1: 

Purpose of the Study 

The Adult Education and Family Literacy Act (AEFLA), Title II of the Workforce Investment Act 

of 1998 (WIA), requires states to establish a comprehensive performance accountability system to 

assess the effectiveness of eligible agencies in making continuous improvement in their adult 

education and literacy activities. To encourage adult education service providers to improve the 

quality and effectiveness of their instructional programs, states are adopting performance-based 

funding (PBF) to allocate some or all of their federal, state, or both, adult education resources. 

These funding systems distribute resources to local providers based on state-defined criteria, 

which may include learner, administrative, or other programmatic measures. Performance 

outcomes are used to determine how much money a provider receives and, in some 

circumstances, whether funds should be added or deducted from a provider’s allocation. 

To gain a better understanding of how PBF systems operate, in 2004 the Office of Vocational 

and Adult Education (OVAE), U.S. Department of Education, commissioned a study of fiscal 

policies in states allocating adult education resources using performance criteria. The study 

findings herein can be used to assist state policymakers and adult education administrators in 

making more informed decisions when designing and implementing PBF systems. Research 

focused on four questions: 

• Why have states decided to adopt PBF systems? 

• How have PBF systems been designed at the state level? 

• How have PBF systems been implemented at the state level? 

• What effect have PBF systems had on the attainment of state-identified 

performance goals and the delivery of services? 

To answer these questions, researchers conducted case study visits to three states—Indiana, 

Kansas, and Missouri—that were early implementers of performance funding. Site visits, 

conducted between January and March 2006, were used to solicit state adult education 

administrators’ and local program providers’ perspectives on the development and use of 

performance funding and to collect quantitative and qualitative data on system operation.1 To 

 

1 See Appendix D, Study Methodology and Analysis Procedures, for a complete description of titles and positions in the nation’s 

education system scheme (e.g., local provider, state director, state administrator, and program staff). 
SECTION 1 

Purpose of the Study 

— 2 —

preserve confidentiality of respondents, names of individuals quoted in the text have been 

withheld, but job titles provided. 

This three-state, cross-case analysis report synthesizes study findings to document lessons 

learned by state administrators in designing and implementing PBF systems. Effective strategies 

used to support development and adoption of performance-funding systems are profiled, along 

with formula considerations common across these states and unique to specific funding 

approaches. The report closes with a review of the effects of performance funding on service 

delivery and issues states should consider before initiating the development of a PBF system. 

Detailed descriptions of state systems and the methodology used to collect and analyze data are 

included in the appendix of this report. Interested readers may also wish to consult the 

background report Performance-based Funding in Adult Education: Literature Review and 

Theoretical Framework,2 which documents performance-based resource distribution approaches 

in adult and higher education programs, along with the conceptual and theoretical framework 

used to guide study activities. 

 

2 Performance-based Funding in Adult Education: Literature Review and Theoretical Framework. Steven Klein, MPR 

Associates. Prepared for U.S. Department of Education, OVAE, Division of Adult Education and Literacy. September 2005. 
— 3 — 

SECTION 2: 

Overview of State Adult Education Funding 

Systems 

States using PBF to allocate adult education resources have developed distinct approaches for 

allocating resources among eligible providers. To provide context for the research findings, this 

section briefly summarizes strategies of the three states profiled for allocating federal and state 

resources for adult education programs and the performance criteria used to determine provider 

eligibility. 

Indiana 

In FY01, the Indiana Department of Education updated its federal adult education finance 

formula to incorporate performance funding. Its existing state formula for allocating adult 

education funding remained unchanged. Under the revised system, local providers are funded as 

follows: 

Base Funding: Roughly 85 percent of federal and 100 percent of state resources are allocated as 

base funding using the following distribution criteria: 

• Federal Resources—Providers automatically qualify for 90 percent of the 

resources that were in their FY00 base budget. This base remains constant over 

time and is associated with a funding district, not an individual provider. Should 

an existing provider withdraw from the system, a new provider is identified 

within the district and resources transferred from the exiting to the entering 

agency. 

• State Resources—Providers automatically qualify for 90 percent of their 

reimbursed expenditures for the summer and fall terms of 1991 and the spring 

term of 1992, or, for new programs, a base amount established at the close of the 

first full year of operation. This base remains constant unless an agency’s 

expenditures fall below its base during a given period; when this occurs, the 

reduced reimbursement becomes the new base. Remaining appropriations or new 

state dollars are distributed on the basis of enrollments, so that over time a greater 

percentage of the state appropriation is based on learner demand. 
SECTION 2 

Overview of State Adult Education Funding Systems 

— 4 —

Performance Funding: Approximately 15 percent of federal resources are distributed based on 

provider performance on the core and secondary measures of the National Reporting System for 

Adult Education (NRS), the accountability system for the AEFLA. All states are required to 

report on the core measures in the NRS, which include learner outcomes, employment outcomes, 

and postsecondary participation measures, and states have the option of reporting on the 

secondary measures, which include outcomes related to family and community. 

Federal performance resources are allocated based on the number of adult learners achieving 

outcomes on state-identified NRS measures. Incentive grants also are provided for agencies 

meeting their state-negotiated performance levels for 12 measures of educational gains included 

in the NRS. Performance funds lag one year behind recorded outcomes, meaning that a local 

provider’s eligibility in FY06 is determined by its program performance in FY05. State measures 

include: 

• Performance—Number of provider outcomes, calculated by summing the number 

of adult learners who achieve any of the following: 

○ Completed a level or completed a level and advanced to a higher level; 

○ Entered employment, retained employment, obtained a general equivalency 

diploma (GED) or secondary school diploma, or entered postsecondary 

education or training; 

○ In family literacy programs, advanced an educational functioning level (i.e., 

an indicator of skill attainment), entered or retained employment, obtained 

a secondary diploma or GED, entered postsecondary education or training, 

or had increased involvement in their children’s education or literacy 

activities; 

○ In workplace literacy programs, advanced an educational functioning level, 

entered or retained employment, obtained a secondary diploma or GED, or 

enrolled in postsecondary education or training; 

○ In corrections programs, advanced an educational functioning level, entered 

or retained employment, obtained a secondary diploma or GED, or entered 

postsecondary education or training; or 

○ Achieved a work-based project learning goal, left public assistance, 

acquired citizenship skills, increased involvement in their children’s 

education or literacy activities, voted or registered to vote, or increased 

involvement in community activities. 
SECTION 2 

Overview of State Adult Education Funding Systems 

— 5 —

Provider resource eligibility is calculated by multiplying the total number of individuals 

achieving a positive outcome on the identified NRS measures by a state-established rate for each 

measure. To align outcomes with state goals, the amount awarded for a secondary NRS 

performance outcome is half the amount awarded for an outcome on a core NRS measure. 

• Incentive—Funds are awarded for providers meeting their state-negotiated 

performance levels for the NRS core measures, including: 

○ Made an educational gain (12 levels); 

○ Entered further education and training; 

○ Entered employment; 

○ Retained employment; and 

○ Completed high school or GED. 

Local provider eligibility is calculated by multiplying the total number of negotiated 

performance targets achieved by a state-established rate equivalent for each measure. 

Kansas 

In FY00, the Kansas Board of Regents overhauled its formula for allocating federal and state 

adult education resources among eligible providers. Under the revised system, local providers are 

guaranteed a base level of funding from state resources, which is equally distributed across all of 

these providers. Providers compete for federal and remaining state resources allocated on the 

basis of learner outcomes, and for quality points awarded on the basis of provider administrative 

and programmatic operations. 

Base Funding: Each fiscal year, 50 percent of state adult education resources are evenly divided 

as base funding among all eligible providers in the state. 

Performance Funding: Performance awards are distributed based on provider performance 

during the fiscal year preceding the current one, meaning that funds lag one year behind recorded 

outcomes. Resources are distributed based on providers’ performance using the following 

distribution formula: 

• Learner Outcomes—Fifty percent of federal funds are allocated based on the 

number of learner outcomes on selected NRS core and secondary measures. 

These include: 
SECTION 2 

Overview of State Adult Education Funding Systems 

— 6 —

○ Completed one of the functional academic performance levels identified for 

Adult Basic Education (ABE), Adult Secondary Education (ASE), and 

English as a Second Language (ESL) learners as specified in the NRS; 

○ Entered employment; 

○ Retained employment; 

○ Received a GED; 

○ Entered postsecondary education or training; 

○ Acquired U.S. citizenship skills; 

○ Increased involvement in children’s education; and 

○ Increased involvement in children’s literacy activities. 

Educational gain outcomes are doubled for learners in the five lowest educational functioning 

levels of the NRS (i.e., the two lowest ABE levels and three lowest ESL levels). 

• Program Quality—Fifty percent of federal and 50 percent of state funds are 

allocated based on the number of quality points a program earns relative to the 

statewide total generated that year. Quality points are awarded based on provider 

performance on 25 submeasures encompassing 10 process indicators.3 Points are 

awarded based on evidence presented during a state monitoring visit, data that are 

in programs’ end-of-year reports or that are derived from learner follow-up 

surveys, and records maintained by state staff. 

Missouri 

In FY02, the Missouri Department of Elementary and Secondary Education adopted a 

performance-based funding system to allocate its federal and state adult education resources 

among eligible service providers. Under this system, local providers are guaranteed a base level 

of funding from federal and state resources, which is distributed across providers based on 

audited learner-contact hours. Providers compete for remaining federal and state resources that 

are allocated on the basis of provider performance on two NRS measures of student progress, as 

indicated below. Additional categorical funding is also available for state-specified uses. 

Base Funding: Each fiscal year, the state establishes a base funding rate that it uses to allocate 

50 percent of available resources. These funds are distributed based on a provider’s total audited 

 

3 A complete listing of the Kansas Indicators of a Quality Adult Education Program can be downloaded from the Internet at: 

http://www.kansasregents.org/download/adultEd/IndicatorsFY05.pdf 
SECTION 2 

Overview of State Adult Education Funding Systems 

— 7 —

learner-contact hours in the first preceding fiscal year for which audited totals are available (i.e., 

FY04 for FY06). The remaining 50 percent of base resources are allocated based on a program’s 

previous three-year average of its total audited contact hours (i.e., using FY02–04 for FY06). 

Performance Funding: Performance resources are distributed based on the number of 

individuals who made academic progress on two measures defined in the NRS: 

• Completed one of the functional academic performance levels identified for ABE, 

ASE, and ESL learners in the NRS; and 

• Received a GED. 

Performance resources also lag two years behind the recording of outcomes, meaning that local 

provider eligibility is determined by program performance in the fiscal year two years preceding 

the current one. Local provider eligibility is calculated by multiplying the total number of 

individuals achieving a positive outcome by a state-established rate for each measure. 

Categorical Funding: Federal and state resources are earmarked for use in specific functions 

identified by the state. These include data collection funds to offset data entry staff salaries and 

computing upgrades, marketing funds to offset provider advertising expenses, technology funds 

to purchase computing hardware and software, and One-Stop funds to support coordination with 

One-Stop Career Centers. Local provider eligibility is calculated by averaging providers’ threeyear expenditures in each category, with a state-established, fixed dollar amount awarded based 

on the level of program expenditures. 

— 9 — 

The locals didn’t like the old formula

but they didn’t know why. Nobody

ever got enough money. Nobody ever

got as much as they should.

Everybody got less money than they

should, compared to what they knew

their neighbor got, or suspected their

neighbor got, and they knew that their

neighbor wasn’t working nearly as

hard as they were. 

—State Director, Kansas 

SECTION 3: 

Rationale for System Adoption 

Performance-based funding systems distribute resources to local providers based on measurable, 

state-defined criteria of program performance. A review of the performance-funding literature 

reveals that these systems increasingly have been integrated into federal and state government 

programs since the early 1990s, where they have been used as a means of holding public 

agencies accountable for the quality, efficiency, and effectiveness of program services. 

Contextual Factors Influencing Formula Adoption 

Unlike the higher education sector, where early systems were imposed legislatively, often 

without input from postsecondary educators, adult education administrators in profiled states 

report that they encountered little, if any, pressure to adopt PBF from external sources. Rather, 

the introduction of PBF was initiated by state directors of adult education to respond to 

accountability requirements contained in AEFLA and to address programmatic concerns about 

the delivery of state adult education services. 

Federal Accountability Requirements 

Accountability provisions in the AEFLA catalyzed state 

adoption of PBF. To ensure an optimal return on federal 

investment, the AEFLA legislation requires that states 

negotiate levels of performance for core indicators contained 

in the act, and report annually on state and local progress in 

making continuous improvement toward achieving these 

targets. Performance data are used to produce state-by-state 

comparisons that are reported to Congress and the general 

public, and to establish state eligibility for incentive grants 

awarded through the Workforce Investment Act (WIA) legislation. 

State directors in study states believed that the common measures of performance contained in 

AEFLA would give adult education greater visibility, enabling policymakers to compare 

performance outcomes across states, as well as within states across individual providers. 

Although federal resources comprise a substantial proportion of program allocations, directors 

believed that, absent a fiscal incentive, providers would perceive new AEFLA reporting 

requirements as a bureaucratic exercise that did not need to be taken seriously. Directors 
SECTION 3 

Rationale for System Adoption 

— 10 —

assumed that attaching funding to program performance would create a high-stakes funding 

environment that would motivate providers to initiate instructional reforms to boost their 

resource eligibility, and, in so doing, increase the likelihood that their state would achieve its 

federally-negotiated AEFLA performance levels. 

State Programmatic Factors 

State directors played a pivotal role in introducing PBF into their state funding formula. In both 

Kansas and Missouri, the decision to adopt PBF coincided with leadership changes at the state 

director position. That is, incoming directors related that they were dissatisfied with the statefunding approach they were inheriting, believing that their existing state formulas concealed how 

provider resources were allocated, causing local providers to question the fairness of the process. 

Before the introduction of PBF, the three case study states funded adult education providers 

primarily based on enrollment, with program eligibility determined by the total number of 

learner-contact hours generated during the preceding program year. Resource allocations were 

calculated by multiplying a program’s contact hours by a state-established dollar amount, which 

varied across years based on the number of contact hours reported by providers and the 

availability of federal and state funds. 

During interviews, state administrators and program directors in all three states related their 

frustration with contact-hour formulas, suggesting that this mechanism provided educators a 

greater incentive to fill seats than to manage enrollments, undertake staff development, or initiate 

curricular reforms that could improve student outcomes. The Missouri state director also voiced 

his belief that funding based on contact hours destabilized resource allocations because year-toyear fluctuations in program participation had implications for future year allocations. To 

illustrate, he described how one year of low enrollment could force institutional cuts the 

following year, which, in turn, could make it difficult for a provider to generate additional 

contact hours to earn back lost resources. 

Although state administrators reported no external pressure to adopt PBF, prior to adopting 

performance funding, staff in Indiana recalled being asked by a new state board member whether 

performance was considered in determining provider allocations. This line of questioning 

sensitized state administrators to the lack of accountability in their existing funding system and 

helped to build their resolve to remedy the situation. 

State Goals in Introducing Performance-based Funding 

To guide formula drafting efforts, state administrators consulted with representatives of local 

providers to specify statewide goals for the adoption of PBF systems. Generally, these goals 
SECTION 3 

Rationale for System Adoption 

— 11 —

called for the creation of a more open, transparent system that would allocate resources in a fair 

and efficient manner. State goals fell into four broad areas. 

Equity 

Administrators sought to develop a fair and equitable system for allocating resources across 

providers. This translated into ensuring that all providers had equal opportunity to compete for 

PBF resources. Specific considerations identified by state administrators and advisory council 

members included: 

• Equalizing resource distribution across sites—Adult education service providers 

differ within states in a number of respects, including program size, program 

offerings, learner demographics, and geographic location. To ensure equitable 

distributions, advisory council members in all three states sought to ensure that 

providers, irrespective of their characteristics, could compete to earn sufficient 

resources to maintain their existing programs. Political realities also dictated 

equalizing basic grant allocations across agencies in Kansas, because, according 

to the director, state legislators would have been unwilling to advocate for 

increased adult education funding unless they believed their constituents were 

getting a fair share of available resources. 

• Linking resources to regional delivery systems—Learners are not evenly 

distributed within a state, and so advisory council members in Indiana set out to 

align regional allocations with population needs. That is, unlike Kansas and 

Missouri, Indiana associates adult education resources with a funding district 

rather than an individual provider. Providers within regions automatically qualify 

for 90 percent of the federal and state resources they received in preceding years, 

so that this base remains constant over time. Should an existing provider 

withdraw from the system, the state identifies a new provider, located nearby, so 

that the regional eligibility remains constant. 

Efficiency 

Performance-funding systems are intended to channel resources to the most effective providers 

of instructional services, thereby maximizing the return on state investment. At the same time, 

formula-drafting teams recognized that new state systems must permit providers to have an 

opportunity to improve, should their performance outcomes unexpectedly fall short of 

anticipated levels. All three states sought to promote system efficiency by: 
SECTION 3 

Rationale for System Adoption 

— 12 —

We certainly have seen an

improvement in our goals. We don’t

have a formal study to prove that it’s

a direct result of the performancebased funding, but I think that as the

dollars get tighter and [local

providers] need the performance to

continue their programs. ... They’re

all working harder to get to that point

of meeting the goals and improving. 

—Assistant State Director, Indiana 

• Guaranteeing providers a minimum level of 

funding—Formula development teams in all three 

states set out to structure formulas so that programs 

would receive an adequate base of funding to 

maintain instructional services. In Kansas and 

Missouri, this support was provided through a 

combination of basic grant and performance 

funding; in Indiana, advisory council members 

decided that providers should start off the year with 

sufficient funding to maintain their core program services, even if they were 

unable to qualify for performance funding. 

• Differentially weighting performance outcomes—Believing that not all learners 

are capable of making equal education progress, advisory council members in 

Kansas and Missouri structured their funding formulas to provide additional 

resources for outcomes achieved by lower-level learners. In Indiana, advisory 

members attached lower compensation to learner outcomes on the NRS secondary 

measures, which were not considered central to achieving state goals. 

• Allocating resources based on program quality—State administrators in Kansas 

believed that successful programs followed certain administrative procedures and 

processes when providing program services. Accordingly, advisory council 

members determined that the state formula should include quality indicators, such 

as measures of participant outcomes, program intensity, and credentialing of 

instructional staff, as an integral part of the state funding formula. 

Accountability 

State administrators recognized that new federal reporting requirements could be used to hold 

programs responsible for their performance, as well as to facilitate state-by-state comparisons. 

Formula development teams sought to identify performance measures that would support the 

state in achieving its AEFLA performance targets by: 

• Allocating resources based on the NRS measures—Team members in all three 

states employed the core NRS and, in some instances, secondary NRS measures 

as the criteria to award performance resources. It was anticipated that linking 

funding to the NRS measures would help motivate local providers to achieve 

outcomes on key indicators identified in the AEFLA, increasing the likelihood that 

the state would retain its eligibility for incentive funding identified in the WIA

legislation. 
SECTION 3 

Rationale for System Adoption 

— 13 —

Program Improvement 

Advisory council members in all three states believed that performance-funding systems should 

not only reward successful providers, but also motivate program directors to initiate program 

improvement efforts. While advisory council members believed that the use of performance 

funding would lead providers to critique their program operations, the formula development 

team in Missouri was the only one to build program improvement incentives directly into its 

state formula. This was accomplished by: 

• Earmarking performance funding for improvement activities—To encourage 

providers to improve upon their existing programs, advisory council members in 

Missouri determined that local providers should earmark a minimum of 10 

percent of their programs’ performance allocation for instructor stipends for 

planning and preparation and 25 percent for support of professional development 

activities. However, this provision was dropped in the second year of program 

operation due to complaints from program directors, particularly those with small 

performance allocations who objected to being told how to spend their resources. 

— 15 — 

SECTION 4: 

Formula Design Process 

State directors of adult education took direct responsibility for overseeing the development of 

state PBF formulas. To begin the design process, the state director either formed a performancefunding committee or consulted with his or her standing advisory council, a group typically 

composed of between 12 to 15 program directors identified as representative of the field. 

Considerations in selecting formula drafting members included: 

• Provider type—representatives from secondary and postsecondary agencies, 

community-based organizations, and other providers eligible for federal, state, or 

both, types of adult education funding; 

• Provider size—number of learners participating in program services; 

• Program type—extent of ABE, ASE, and ESL services offered within providers; 

and 

• Learner demographics—population characteristics, including education 

attainment and language ability, and personal characteristics, such as age, sex, and 

race-ethnicity. 

Directors reported recruiting highly visible individuals who were respected for their judgment 

and communication skills, as well as for their ability to contribute to council discussions. 

State directors believed that the advisory groups they created were representative of the state. For 

example, in Kansas, the state director’s advisory council included directors from six community 

college programs, four school district programs, and two community-based organizations. 

Directors were geographically and demographically distributed, with consideration given to their 

job tenure, program urbanicity and size, level of local program support, and type of program 

offering, with program directors including those administering programs serving predominantly 

ABE and ASE learners, along with those serving predominantly ESL learners. 

To promote understanding and support from the field, state administrators communicated 

formula development efforts to local providers using a variety of approaches. To forewarn 

program directors, Indiana advised them of the state’s intent to adopt performance funding by 

placing the issue on the agenda of a statewide administrators’ conference in December 1998, two 
SECTION 4 

Formula Design Process 

— 16 —

years before the state implemented performance funding. In Kansas, the state director met with 

all local providers for a half-day meeting the month before formula development was initiated to 

explain the rationale for formula development and to respond to their questions and concerns. 

In all states, once development was underway, state directors updated providers on council 

actions during statewide or regional adult education conferences. Advisory council members also 

were asked to share their experiences with their colleagues so that there would be no surprises 

once the formula design process was complete. 

Formula Development 

The construction of state funding formulas was a roughly yearlong process, with advisory 

committee members meeting between three to five times over a six- to 12-month period. To 

illustrate, the state director of Kansas reported that her advisory committee met four times 

between October 1998 and March 1999, with each meeting lasting approximately six hours. 

State directors used advisory council kick-off meetings to educate members about the operation 

of PBF formulas and to permit them to voice their concerns about the funding system. Before 

convening the first meeting of Missouri’s advisory committee, the state director sent all 

participants a copy of the book Who Moved My Cheese? Written by Spencer Johnson, this 

motivational best seller describes strategies to help individuals cope with potentially difficult 

changes in their work setting. 

Anticipating that team members would arrive prepared to delve into the meeting agenda, the 

Missouri state director was surprised to discover how deeply the book had affected people. 

Participants discussed personal situations and work experiences to such an extent that the first 

session took on the tone of a professional development seminar. But, as it turned out, these 

discussions helped members relieve anxiety and build trust, enabling the group to tackle difficult 

formula development issues at an earlier stage than they might have otherwise. 

State directors used subsequent advisory council meetings to draft performance-funding goals 

and to consider key issues relating to formula development, such as the amount of funding and 

criteria that should be used to allocate resources. Once the mechanics of formula operation were 

formalized, advisory council members modeled how provider allocations would change under 

various assumptions and, based on empirical evidence, agreed to a final formula construction. 

As an example, Indiana state administrators modeled resource allocations assuming base funding 

was set at 60 percent, 65 percent, and 70 percent of total adult education resources. These 

estimates were shared with committee members, who analyzed the amount of money their own 
SECTION 4 

Formula Design Process 

— 17 —

and other programs would receive. Attaching dollar amounts to formula scenarios helped 

program directors understand the effect of different assumptions on provider resources, and in 

the case of Indiana, ultimately convinced advisory council members that the state should limit 

the amount of funding it distributed based on performance. 

Formula Construction 

Although states followed similar development approaches, there is considerable variation in how 

state funding formulas operate. This is due to philosophical differences among state 

administrators on the appropriate uses of performance funding to motivate local providers, as 

well as state policies relating to how data are collected and applied. 

Resource Allocation 

Among the three states studied, Kansas has instituted the most aggressive use of PBF, allocating 

roughly 88 percent of its combined federal and state adult education funding based on provider 

outcomes. This commitment to performance has driven all providers within the state to strive to 

maximize their funding eligibilities, since basic grant funding allocated to local providers is not 

itself sufficient to maintain program operations. 

Missouri has opted for a more modest approach to performance funding, allocating roughly 19 

percent of its FY05 resources based on learner outcomes. Unlike other states in the study, over 

time Missouri has increased the amount of funding it has allocated using performance criteria. 

When first adopted in FY02, performance funding accounted for just 6 percent of combined 

federal and state allocations to local providers. Over time, as state adult education resources have 

fallen, the state has maintained its level of spending for PBF by lowering the amount of 

resources allocated based on contact hours; as a result, the overall proportion of adult education 

funding allocated using performance criteria has increased. 

Indiana has opted for the most limited use of PBF, with roughly 5 percent of its federal funds 

allocated on performance. Since performance funding accounts for only a fraction of overall 

resources, program directors report that they create their annual budgets assuming that they will 

only receive their guaranteed base. Directors use any performance funding they receive to 

provide supplemental services, such as purchasing professional development or additional 

equipment and supplies. 

Performance Measures 

Study states have combined performance measures contained in AEFLA with their own stateestablished measures to allocate performance funding. State measures fall into three categories: 
SECTION 4 

Formula Design Process 

— 18 —

CATEGORY 1: CORE AND SECONDARY MEASURES INCLUDED IN THE 

NATIONAL REPORTING SYSTEM 

All three states make use of the core and secondary NRS measures of learner outcomes to 

allocate PBF resources to local providers. Of the three states, Indiana has made the greatest 

commitment to the NRS measures, adopting all of the core and secondary measures to determine 

provider resource eligibility (table 1). In contrast, Missouri limits its funding criteria to evidence 

of learner educational gain and GED receipt. 

Indiana Kansas Missouri

Core Measures

 Made educational gainb √√√

 Received GED √√√

 Entered employment √ √

 Retained employmentc √ √

 Entered postsecondary education or training √ √

Secondary Measures

 Achieved work-based project learning goal √

 Received reduction in receipt of public assistance √

 Achieved citizenship skillsd √ √

 Voted or registered to vote √

 Increased general involvement in community activities √

 Increased involvement in children’s education √ √

 Increased involvement in children’s literacy activitiese √ √

a

 NRS—National Reporting System for Adult Education

exit.

d

 Whether a learner is has gained the skills required to pass the U.S. citizenship test.

education process, which is captured by the preceding measure. 

e

 Whether a learner increases his or her involvement in literacy-related activities (e.g., reading) with his or her children, as opposed to 

Table 1—

NRSa

 Core and Secondary Measures Used in Three State Performance-based Funding Formulas, by State

b

 Includes 11 submeasures of educational gain in Adult Basic Education (ABE), Adult Secondary Education (ASE), and English as a 

Second Language (ESL) programs.

c

 Whether a learner who finds a job in the first quarter following program exit remains employed in the third quarter following program 
SECTION 4 

Formula Design Process 

— 19 —

To allocate resources, states multiply the number of performance outcomes that a provider 

achieves by a state-established performance rate or, alternatively, assign each provider a 

proportional share of the resources available for performance distribution (i.e., a program 

accounting for 5 percent of statewide learner outcomes would qualify for 5 percent of the 

resources allocated for this factor). 

During interviews,4 state administrators and local program directors generally supported the use 

of the core measures of educational gain and GED receipt as valid outcomes of program 

performance. Moreover, since all providers are required to report outcome data to comply with 

AEFLA accountability requirements, state directors report that the use of the core and secondary 

NRS measures imposes no additional reporting burden on local providers. 

Program directors in Indiana and Kansas raised some concerns, however, with the use of retained 

employment by learners as a program funding measure, in part, because of the difficulty in 

collecting post-program data, and, in part, because of the belief that instructional staff can have 

little effect on learners’ behaviors once they had departed a program. Program directors in 

Indiana also did not universally support using secondary NRS measures to award performance 

funding. According to one program director, adult education agencies are focused on imparting 

basic literacy skills to prepare learners to earn a GED credential. He and others suggested that 

secondary NRS measures, such as registering to vote or increasing involvement in children’s 

literacy activities, are difficult to measure and less important than those dealing with learner 

progress. 

CATEGORY 2: PROCESS MEASURES OF PROGRAM PERFORMANCE 

To encourage providers to make constructive programmatic change, Kansas has identified a set 

of 25 measures of program quality. Points are awarded to providers based on whether their 

program is rated by a state representative as having a high, medium or low level of quality for 

each measure. Points are awarded on the basis of evidence presented during a state monitoring 

visit, data in programs’ end-of-year reports, data derived from follow-up surveys, and records 

maintained by state administrators. 

Awarding points based on program quality allows smaller programs to compete on equal footing 

for performance-funding resources. Since providers are awarded points for achieving particular 

criteria, such as maintaining standardized testing protocols confirmed by random review of 

selected learners’ permanent files, all programs have equal opportunity to earn a share of state 

resources. Use of program quality measures also can ensure that programs are following stateapproved data collection or testing procedures. 

 

4 A copy of the study methodology is included in Appendix D. 
SECTION 4 

Formula Design Process 

— 20 —

Since quality points are awarded based on external reviews of program offerings, state 

representatives in Kansas must visit all service providers in the state annually to audit 

administrative records and award quality points. This need for annual monitoring visits has 

added a substantial, recurring cost to PBF formula operations. State expenses include paying for 

monitoring team salaries and travel, coordinating with team members on the conduct of visits, 

and administering award data to calculate performance allocations. 

CATEGORY 3: PROVIDER ATTAINMENT OF PERFORMANCE STANDARDS 

Indiana is the only state to award incentive funding to providers who meet or exceed their statenegotiated performance standards. Incentive funding is based on 15 performance measures 

associated with the NRS core measures. These include 11 measures of educational gain and four 

measures associated with program completion and post-program outcomes: 

Measures of Educational Gain Measures of Program 

Completion and Outcomes

Beginning literacy ABE (0–1.9) Completed high school or GED 

Beginning basic ABE (2–3.9) Entered further education and training 

Low intermediate ABE (4–5.9) Entered employment 

High intermediate ABE (6–8.9) Retained employment 

Low ASE (9–10.9) 

High ASE (11–12) 

Beginning literacy ESL (0–0.9) 

Low intermediate ESL (2–3.9) 

High intermediate ESL (4–5.9) 

Low advanced ESL (6–8.9) 

High advanced ESL (9–10.9) 

To calculate a program’s incentive award, state administrators multiply the number of 

performance targets achieved by a predetermined dollar amount equivalent across measures. For 

example, in FY01 providers received $630.25 for each performance target they attained, 

meaning that a provider who achieved 10 targets would receive $6,302.50. Incentive rates are set 

each year based on the amount of federal resources that are available and the number of 

statewide outcomes that are reported by providers. 

This can cause the incentive rate to change over time, as illustrated in table 2, because if federal 

funding remains flat while the number of outcomes fluctuate, the value of a performance 

outcome must either rise or fall to keep expenditures constant. 
SECTION 4 

Formula Design Process 

— 21 —

As with process indicators, awarding resources based on provider attainment levels the playing 

field, since providers compete to meet or exceed their own negotiated performance targets. This 

can mean smaller providers earn as much as, or more than a larger provider. However, because 

the size of a performance award is a function of the number of providers achieving a positive 

outcome, programs are unable to accurately predict the size of their performance allocation when 

budgeting for the following fiscal year. Although project directors in Indiana were comfortable 

with this variability, its effect may be minimized in this state because only a fraction of adult 

education resources are allocated as incentive funding. 

Formula Weighting 

All three case study states assign different resources to learner outcomes. State approaches vary, 

however, with states attaching different levels of support for different learner or programmatic 

outcomes. 

Educational Gain 

Kansas and Missouri assign supplemental weighting for outcomes obtained by lower-level 

learners or those with limited English language ability. To establish funding rates, state 

administrators analyzed historical performance data to determine the rate at which learners made 

measurable advances. For example, in designing its funding formula, Kansas’ administrators 

determined that lower-level learners took approximately 1.8 hours to make an educational gain, 

compared to 1.0 hour for those at higher levels. Consequently, the state opted to double the value 

of an outcome for learners achieving gains in the five lowest educational functioning levels of 

the NRS (two lowest ABE levels and three lowest ESL levels). 

Administrators from Missouri also reviewed performance data when setting their performance 

reimbursement rates. Initial performance reimbursement rates provided differential weighting to 

lower-level learners participating in ABE and ASE programs, and for learners qualifying for 

Number of 

Fiscal Year Federal Funding Statewide Outcomes Incentive Rate

2001 $225,000 357 $630.25

2002 $225,000 373 $603.22

2003 $225,000 342 $657.89

2004 $224,997 349 $644.69

2005 $222,459 350 $635.59

SOURCE: Data provided by Indiana Department of Education, FY01 to FY05.

Table 2—

Indiana Performance Standard Incentive Rates as Based on Federal Funding and Numbers of Statewide 

Outcomes: FY01 to FY05
SECTION 4 

Formula Design Process 

— 22 —

GED award (table 3).5 Following PBF formula introduction, advisory committee members 

reconvened to consider whether outcome rates for ESL students required additional resources. 

As part of this exercise, state administrators compared the number of contact hours recorded for 

ESL students to those for other program participants. Based on a review of the relative effort 

required to achieve a learning gain, committee members recommended development of a 

separate rate for ESL students. 

Under the current approach, adopted in FY05, providers with ESL students making a level 

completion are compensated at a greater rater than for ABE or ASE students making a 

comparable gain. Providers with lower-level learners making educational gains also are 

compensated at a greater rater than those at higher levels; for example, providers receive $250 

for each ESL learner making a level 1 completion, compared to $100 for each learner making a 

level 3 or higher gain, as indicated in table 3. 

States adopt differentiated weighting to compensate providers for the additional cost of educating 

lower-level or limited English proficient learners, on the assumption that these individuals take 

longer to achieve a positive outcome. Left unanswered is whether supplemental weighting can 

actually discourage providers from undertaking needed reforms that might improve learner 

performance. It may be, for example, that the absence of a supplemental weight might encourage 

providers to intensify their efforts to improve programs serving lower-level learners, because 

students would otherwise not make educational gains at a rate sufficient to offset the fixed cost 

of their instruction. 

 

5 Adult Education and Literacy (AEL) levels of completion correspond to the six functional academic performance levels 

identified for ABE and ASE learners as specified in the NRS. Learners who complete an ASE high level are counted as attaining 

a GED credential. ESL levels also correspond to the six functional academic performance levels specified in the NRS. Learners 

who complete the advanced ESL level are included in level 3–5 for funding purposes. 

Level FY02 FY03 to Present

AEL Level Ia $150 $200

AEL Level 2 $100 $150

AEL Level 3–5 $50 $75

GED $150 $150

ESL Level 1 - $250

ESL Level 2 - $175

ESL Level 3–5 - $100

a

 AEL—Adult Education and Literacy

SOURCE: Missouri Department of Elementary and Secondary Education, 2006.

Table 3—

Missouri Performance Outcome Rates, by Education Functioning Level: FY02 and FY03 to Present
SECTION 4 

Formula Design Process 

— 23 —

Core Versus Secondary Measures 

Indiana has taken a somewhat different approach to weighting program outcomes. According to 

the assistant state director, 75 percent of adults in state programs function below the ninth-grade 

level or are ESL students, so differential weighting is unnecessary because all providers serve 

learners with high needs. Although the state does not provide additional resources for 

educational gains made by learners in different programs, the state differentially weights NRS 

outcomes, with secondary NRS outcomes reimbursed at half the amount of a core NRS measure. 

This change was justified for several reasons. First, only core NRS measures are used by the U.S. 

Department of Education to report to Congress, and by the Indiana Department of Education for 

the state’s adult education report card, meaning that these measures carry somewhat greater 

importance. Second, it is only through achieving the core outcome measures that Indiana, or any 

other state, may qualify for incentive funds through WIA. Finally, measurement and reporting of 

secondary outcomes is not consistent across programs within the state, undercutting the accuracy 

of these measures. 

Timing of Data Collection 

To accommodate state reporting cycles, provider allocations are based on retrospective 

performance data. In both Kansas and Indiana, allocations are based on program outcomes from 

the preceding fiscal year, meaning that providers’ eligibility in FY06 is determined by program 

performance in FY05. This one-year lag dictates that local program directors begin budgeting for 

the upcoming fiscal year without complete information on performance outcomes for the fourth 

quarter (April–June) of the current program year. 

Kansas asks that program providers develop their budgets based on their anticipated level of base 

and performance funding. To minimize the effect of fourth-quarter outcomes, when the PBF 

formula was originally introduced the state based funding on the first three quarters of the 

academic year, with $100,000 held out of the allocation formula to compensate providers for 

their fourth-quarter results. In the year following, the state increased this funding to $200,000 

and, beginning in FY06, this amount has been increased to $500,000. This action was taken 

because many providers appeared to be cutting back services in the fourth quarter, partly because 

these outcomes had lower value than ones recorded earlier in the year. 

Indiana also asks program directors to estimate their level of funding for the following fiscal year 

without information on their fourth-quarter results. However, since the state allocates only 5 

percent of provider funds based on performance, programs are at little risk of losing substantial 

resources midyear should they fail to achieve their predicted outcomes. 
SECTION 4 

Formula Design Process 

— 24 —

In contrast, program funding in Missouri is based upon performance data collected during the 

fiscal year two years preceding the current one. This two-year lag ensures that program directors 

have complete information when they begin their budgeting process near the start of the new 

calendar year. In return for the security that comes with full information, however, programs 

experience a delay between when they post outcomes and receive their resources. This can mean 

that programs have difficulty expanding programs to new sites, because a program experiencing 

substantial enrollment growth is forced to carry its increased costs for an additional year until 

state allocations catch up with its expanded needs. 
— 25 — 

SECTION 5: 

Implementation Issues 

States fully implemented their PBF systems in the year following formula development. 

Although states did not provide a hold-harmless clause to insulate providers from drastic funding 

changes during the first years of system operation, all sought to protect providers by increasing 

grant resources and training providers in proper data collection and reporting techniques. 

A review of the relevant higher education literature suggests that adult education providers might 

have been expected to face additional costs in collecting performance data, although no study 

exists quantifying the cost of information system start-up in the adult or higher education sectors. 

However, as detailed below, state agencies and local providers incurred minimal costs associated 

with PBF introduction, in part because data collection piggybacked on federal AEFLA reporting 

requirements. 

Formula Rollout 

To dampen negative effects associated with PBF adoption, state administrators supplemented 

providers’ initial performance allocations with increased federal funds made available to states 

following the adoption of AEFLA. To illustrate, the introduction of performance funding in 

Missouri coincided with an infusion of federal resources into the system, with statewide funding 

climbing from $11.6 million to $13.2 million with formula rollout. These additional resources 

helped reassure Missouri administrators that there was no need to shield providers from shifts in 

their resource eligibility. In Kansas, the state director also opted to release approximately 

$100,000 in held-back federal resources during the first two years of formula operation. 

The structure of Indiana’s PBF formula provided some measure of protection to local providers 

as well. According to the assistant state director, the fact that such a high percentage (95 percent) 

of funding was released as a guaranteed base negated the need for a hold-harmless or other type 

of protection mechanism. Moreover, at the time that PBF was first introduced, Indiana was 

experiencing a significant increase in federal funds due to population changes, with resources 

rising from $7.3 million in FY00 to $8.6 million in FY01. State administrators did, however, set 

aside a portion of new federal resources for use as a competitive enhancement grant, for which 

program directors could apply to support a variety of program development activities. The 

combination of guaranteed base funding, increased federal resources, performance-based 
SECTION 5 

Implementation Issues 

— 26 —

funding, and enhancement grants, meant that most Indiana providers were at little risk of losing 

resources with the shift to performance funding. 

Funding simulations conducted during formula development meetings in all three states also 

indicated that there would be few losers among providers. Those projected to experience a drop 

in resources were primarily larger providers that were thought to be able to absorb anticipated 

losses. Although initial allocations in Indiana and Missouri proved accurate, unanticipated 

problems associated with the award of quality points in Kansas adversely affected provider 

resource eligibility. Upon calculating program allocations for FY00, Kansas’ administrators 

discovered that they had grossly overestimated awards for many larger service providers, who 

were incapable of collecting data on all students or earning the median number of quality points 

assigned during modeling exercises. 

Once the magnitude of funding losses became apparent, Kansas state personnel contacted 

affected providers to advise them of anticipated shortfalls and to reassure them that, with 

improvements to data collection and attention to service delivery, their deficits could be 

corrected. Administrators also provided technical assistance to train provider staff on how to 

enter missing data into institutional databases, how to test using appropriate assessment 

protocols, and how to identify performance gaps adversely affecting their eligibility. 

Consequently, during the second year of performance funding many large providers recaptured 

funding, in some cases at the expense of smaller programs unable to generate equivalent 

performance gains. 

State administrators soon came to understand that their new funding formulas required continual 

review and modification. For example, in response to provider feedback that ESL programs were 

relatively more expensive to provide, Missouri reconvened its advisory committee in the year 

following PBF introduction to reconsider its formula allocation. As part of this exercise, advisory 

members compared the number of contact hours recorded for ESL students to those for ABE and 

ASE participants and, based on a review of the relative effort required to achieve a learning gain, 

proposed a separate, augmented rate for ESL students. 

Provider Reporting Capacity 

Study states were among the earliest implementers of PBF in the country, with system adoption 

occurring concurrent with or immediately following the adoption of the AEFLA legislation in 

1998. During this transition period, local providers initially struggled to collect accurate NRS 

performance outcome data. Provider challenges included incorrectly administering tests of 

educational gain to learners, which invalidated test results, and misusing new, state-supplied 
SECTION 5 

Implementation Issues 

— 27 —

management information system software intended to assist providers in administering learner 

program files and in collecting accountability data. 

States compensated for provider difficulties with a variety of strategies. In Missouri, state 

administrators held provider eligibilities constant for the first two years of PBF adoption, using 

program outcome data collected prior to the introduction of AEFLA to allocate resources. All 

three states also conducted statewide trainings to instruct provider staff on appropriate 

methodologies for administering education tests and for collecting and reporting provider 

outcome data. 

Providers in Missouri reportedly had difficulty assigning learner outcomes into the attainment 

levels of 1, 2, and 3–5 used by the state to calculate funding eligibility. While providers 

generally understood the difference between a level 1 and level 2 progression, staff had difficulty 

aggregating data to fit the levels 3–5 progressions, which were grouped by the state for formula 

purposes. With hindsight, state administrators reported that it would have been less confusing to 

have providers simply report learner progression for each level individually, with state 

administrators taking responsibility for combining these data to calculate program funding 

eligibility. 

Although providers initially had some problems collecting valid performance data, state directors 

related that, with time and training, reporting difficulties were eventually surmounted. Moreover, 

because all three states used the NRS measures to distribute some or all of their performance 

funding, directors believed that initial conflicts were due to provider misunderstanding of new 

federal reporting requirements contained in the NRS, rather than to the operation of the funding 

formula itself. This suggests that states considering using the NRS (or other state) measures may 

avoid much of the confusion associated with system start-up, since providers have had over half 

a decade of experience collecting data for the NRS. 

Data Quality 

Data quality gradually improved during the first years of formula use. State administrators and 

program directors reported that their greatest challenge was providers misapplying testing 

procedures, which invalidated learner outcomes. This occurred because service providers often 

were not following acceptable testing protocols for measuring participant outcomes. In Kansas, 

for example, counts of individuals making learning gains were initially rejected for some 

providers after it was determined that program staff had failed to enter pretest data or had 

incorrectly administered Comprehensive Adult Student Assessment System (CASAS) diagnostic 

pretest, posttest, or both, forms. 
SECTION 5 

Implementation Issues 

— 28 —

Similarly, the Missouri Department of Corrections, which had historically used its own 

assessment system to measure inmates’ education attainment, shifted to the Test of Adult Basic 

Education (TABE) to assess inmates’ level attainment for AEFLA reporting purposes. Once the 

prison system completed its transition to the new assessment system in the second year of PBF, 

its performance eligibility more than doubled. 

To ensure that provider allocations are fair and accurate, state administrators audit all adult 

education providers every year as part of each state’s official review process. In both Indiana and 

Missouri, provider data are subjected to an annual desk audit, which includes scrutinizing 

program paperwork and data submissions to ensure that program totals are consistent across 

years and reasonable given state administrators’ knowledge of program operations. Every five 

years, all adult education providers in these two states are subjected to an on-site audit, which 

includes a review of randomly selected learner program files and contact-hour and performanceoutcome records. State administrators report that these audits suggest that there is close 

alignment between reported data and program records, leading them to infer that providers are 

maintaining valid information in nonaccreditation years. 

In addition to conducting annual desk audits, Kansas employs a team of program reviewers who 

conduct yearly monitoring visits to each service provider in the state. Annual site visits are 

warranted because of the state’s use of program quality measures to award half of federal and 

state funds. Monitoring activities include auditing a sample of student files to confirm that data 

are properly documented and testing protocols were followed, and reviewing administrative and 

other program records stipulated in the standardized state monitoring instrument. Quality points 

used in the PBF formula are awarded as part of this review process. 

Management Information Systems 

State directors reported that the adoption of PBF had little effect on state-level investment in 

technology or personnel, since the majority of state performance measures overlap those used for 

the AEFLA. To standardize data collection for the NRS measures, state administrators in all three 

states supplied providers with database software and guidance for entering participant data into 

the system. With the exception of quality indicators used in Kansas, state administrators simply 

extract provider performance data from the state’s NRS management information system to 

calculate providers’ PBF eligibility. Consequently, determining statewide allocations is a routine 

task for a state fiscal administrator that is discharged typically in less than a day. 

Although program directors report that the adoption of state NRS software required that they 

make changes to their information system operations, no additional technology costs were 

associated with performance reporting. That is, had performance reporting not been introduced, 
SECTION 5 

Implementation Issues 

— 29 —

providers would still have been required to standardize data administration to report on AEFLAmandated accountability measures. 

Because of the high stakes associated with performance reporting, program directors relate that 

they now pay much closer attention to their data than if they were reporting solely for federal 

accountability purposes. Directors from a few large service providers in Kansas added a staff 

member to manage data entry and administration, while directors from smaller programs 

indicated that they either took on the data entry role themselves or delegated the task to existing 

secretarial staff. Directors in Indiana and Missouri also reported that they are more involved in 

the collection and review of performance data, but did not report hiring additional staff to 

administer data systems. 

Indiana providers believe that they now invest greater time entering information into their local 

databases because of state-instituted electronic data checks that were recently added to the NRS 

reporting system. According to the assistant state director, program staff was initially frustrated 

because they began encountering data flags, indicating a problem, when they tried to report their 

data. Since error warnings did not provide detailed information on what was wrong, agency staff 

had to invest time reviewing and cleaning data before it could be uploaded to the state database. 

Again, because this information was associated with the NRS data system and not the funding 

formula itself, providers would likely have faced this problem irrespective of whether 

performance funding existed. 

Technical Assistance 

State administrators recounted that they did not explicitly provide technical assistance on the 

operation of performance-funding formulas. Rather, state training initially focused on 

familiarizing program staff with new AEFLA reporting requirements, explaining the appropriate 

strategies for administering tests and collecting performance data, and entering data into statesupplied management information system software. This training occurred concurrent with PBF 

system adoption in Kansas and in the years preceding system adoption in Indiana and Missouri. 

Recognizing the importance of making statewide gains in achieving their negotiated performance 

targets, state administrators also provided technical assistance to equip program staff with 

analytic skills. For example, Missouri contracted with an education consulting firm that had 

helped develop the federal NRS to provide workshops on using data to make program 

improvement decisions. In Indiana, program directors attending statewide training meetings were 

given a set of performance outcomes for a fictitious district and taught how to disaggregate this 

data to identify issues affecting a particular program, teacher, or learner population. Professional 
SECTION 5 

Implementation Issues 

— 30 —

development facilitators from the state’s regional centers also 

met with program staff at state association conferences, and 

they met individually to help providers learn how to interpret 

data. 

Over time, as local familiarity with reporting systems has 

increased, state administrators have been able to target 

technical assistance to address methodological issues undercutting program reporting. For 

example, recognizing that there was a problem with measuring learner gains, Kansas’ 

administrators sponsored a series of technical assistance sessions on the proper use of CASAS 

assessments, including reviewing testing protocol, test selection, and data entry. More recently, 

with these issues resolved, the state has focused on improving lesson planning and instruction. 

Although state technical assistance helped program directors understand the procedural use of 

existing adult education databases for NRS reporting purposes, not all directors felt supported in 

the rollout of the new funding system. The sudden focus on performance outcomes meant that 

many directors had to immerse themselves in program data to understand why students weren’t 

performing and whether low outcomes were due to a failure on the part of faculty to follow 

appropriate testing protocols or due to deficiencies in instructional quality, program curricula, or 

some other factor. While shifting to performance funding was credited with helping motivate 

providers to improve program services, it also surfaced weaknesses that required closer scrutiny. 

As a consequence, program directors in Missouri reported that they would have benefited from 

additional training in how to interpret and use data to make more informed decisions.

It was a learning curve because at

first the directors didn’t use the data,

they just collected it. They didn’t

know how to use it. It took a year or

so to learn how to use that data, to

help the teachers along, to figure out

how to use it. It was a learning curve

for everybody. 

—Program Director, Kansas 
— 31 — 

I think PBF makes us pay much more

attention to the state standards and to

making sure that the teachers are

reporting the information. I

remember a big issue we had in our

first couple of years wasn’t that

students weren’t making progress; it

was getting the teachers to fill out the

forms in the data system. 

—Program Director, Missouri 

SECTION 6: 

Effects of Performance-based Funding 

During interviews, both state administrators and local provider staff commented on the effect 

PBF is having on adult education services. Although study participants ascribed both positive 

and negative consequences to performance funding, few if any data exist to substantiate their 

claims. In many cases, participant observations were based on general perceptions of program 

operations or extrapolations from a single experience. Some participants also had difficulty 

differentiating programmatic effects due to the adoption of PBF from those associated with the 

state’s implementation of the AEFLA. Therefore, readers should be cautious in drawing 

conclusions about the effects of PBF on state adult education systems. 

Advantages of Performance-based Funding 

As originally conceived, PBF was introduced to promote the effective delivery of adult education 

services, while simultaneously improving the equity and transparency of state allocations. Study 

participants’ observations suggest that performance funding has helped states achieve these 

intended goals, but also has reaped unexpected benefits in terms of data quality and its use for 

program improvement purposes. Specific contributions have included: 

• Improving Data Quality—During the implementation 

period of the NRS measures, providers in all states 

struggled with methodological issues related to 

measurement and practical issues associated with 

entering data into state-supplied databases. Linking 

resources to the NRS core and secondary measures 

appears to have given directors a fiscal incentive to 

review the accuracy and completeness of information 

submitted to the state. Prior to this time, program 

directors’ concentration had been on generating and maintaining contact hours, 

but now, with resources tied to results, directors refocused on improving reporting 

procedures lest they leave money on the table that they were otherwise eligible to 

receive. 

Reports of improved data quality were noted in all three study states, suggesting 

that PBF may have helped improve the reliability of data collected on federal 

NRS measures included in states’ formulas. Moreover, because performance 
SECTION 6 

Effects of Performance-based Funding 

— 32 —

funding was not introduced in Indiana until FY01 and Missouri until FY02, it 

appears that PBF can play a role in improving data quality years after the adoption 

of AEFLA. As one program director in Missouri observed, although required by 

federal law to collect outcome data, reporting requirements alone did not motivate 

him to collect accurate data. His behavior changed with the introduction of PBF, 

however, once he realized that he was losing money because he and his staff were 

not taking the time to ensure entering of correct information. 

• Increasing System Effectiveness—Compensating providers for results has focused 

directors’ attention on program performance, and, in particular, on making 

connections between programs, instructors, and learner outcomes. Before PBF, 

program directors reported that few of their colleagues ‘owned’ their data; that is, 

although information was routinely collected, few consulted it. It appears that the 

fiscal stakes associated with PBF compelled program directors to become more 

proactive in reviewing the information they collect. Perhaps the clearest evidence 

of this commitment came during interviews, when one Indiana director 

brandished a set of program outcomes that were less than two weeks old to 

illustrate his most recent accomplishments and to identify areas to which he was 

planning to devote his time in the coming weeks. 

Rewarding providers for achieving targeted learner outcomes also has encouraged 

program staff to actively manage student enrollments and service delivery. One 

program director in Indiana compared the new system to working at a doctor’s 

office: All new learners are initially pretested to diagnose their problems, and then 

services are closely tailored to meet their needs. New intake and processing 

procedures also have allowed instructors to make learner referrals to other 

agencies, where warranted, and to develop more realistic goals and services for 

those who enroll. 

Basing funding on performance has prompted most Kansas program providers to 

adopt managed enrollment, meaning that students may enroll and enter classes 

only at specific times rather than on a continuous basis (e.g., in the first three 

weeks of a 12-week course). According to the state director, although the state has 

no formal policy on management enrollment, providers voluntarily adopted the 

strategy because it allows them to more effectively serve the students they enroll. 

For example, this shift has been credited with helping to improve retention 

because providers realize that they must keep students enrolled if they are to 

achieve measurable outcomes, and because instructors are able to spend more 

time on instructional activities, rather than performing administrative functions 

(e.g., enrolling students piecemeal as opposed to altogether). 
SECTION 6 

Effects of Performance-based Funding 

— 33 —

I can take in a [performance outcome

report] like this and hand it to

everyone ... and it tells you how we

did on every one of those outcomes.

When I was showing that to the

senators and legislators and

congressman they were looking at

that and they had no idea that adult

education was touching that many

people, and that they were seeing

those kinds of gains ... they were

amazed when I first handed that out

... at the Kansas Ways and Means

Committee. 

—State Director, Kansas 

I think previously I would have

staff that would show up, teach; they

did their thing. They were here every

time they were supposed to be. ... But,

you know, if I had asked them at the

end of the year, ‘How successful were

you? How many GEDs were

awarded? How many students

improved their English?’ ... I don’t

think they could have told me. I don’t

think they, in many cases, had any

idea of whether they had really made

much of a difference. And I do think

that the fact that funding is tied to

gains ... causes a little more of that

edge, push, to improve. 

—Program Director, Missouri 

Program directors also reported that PBF made them more likely to consult with 

one another to share successful practices, which are easier to identify since 

programs now are using similar measures to assess program outcomes. 

• Engendering Political Support—State directors were 

cognizant of the state political environment when 

considering shifting to PBF, and, in particular, of the 

need to demonstrate positive returns to justify adult 

education’s continued support. Although state 

directors encountered no external pressure to introduce 

performance funding, following system adoption, they 

found that its use bolstered their credibility among 

state legislators and the public, while providing an 

effective way of countering legislative criticism of 

program operations. 

Similarly, Kansas state administrators credit PBF with increasing state and 

institutional investment in adult education, because legislators and the business 

and higher education communities in this state philosophically support allocating 

funding based on performance. Moreover, because accurate data on learner 

outcomes are now more readily available, state administrators are better able to 

communicate to policymakers the fiscal and social benefits of investment in adult 

education services. 

• Promoting Instructor Professionalism—Program 

directors believe that performance funding contributes 

to program quality by holding instructional staff 

accountable for their learners’ outcomes. When system 

resources were allocated using only contact hours, 

instructors were held responsible for filling seats, 

meaning that an underperforming instructor could 

actually generate greater resources by retaining 

learners than by demonstrating education progress. 

Under the new formula, instructors are judged by the 

number of outcomes they achieve, meaning they must 

demonstrate results to be considered effective. 

State directors relate that they are more likely to deliver professional development 

services to instructors, in the belief that trained staff are more likely to generate 

additional resources. Performance funding also has motivated local directors to 

replace low-performing instructors with those who relate to accountability and 
SECTION 6 

Effects of Performance-based Funding 

— 34 —

achieve results. Program directors in both Kansas and Missouri report that they 

are more likely to release underperformers to make room for more sophisticated 

individuals who understand the importance of accountability. Although program 

directors believe PBF has contributed to increased teacher quality, the state has 

not conducted any formal studies to assess the relationship of performance 

funding and instructor quality over time. 

Simply put, state administrators believe that staff at all provider levels—from 

program directors to classroom instructors—change how they go about their jobs 

in response to performance funding. In particular, program directors often spoke 

of the need to meet state performance targets and to obtain additional money to 

support program offerings, while program staff spoke of their desire to gain 

supplemental classroom materials or attend professional development conferences 

with the money they generated for their program. 

Evidence of Performance Improvement 

State directors believe that the adoption of PBF has contributed to improving state outcomes on 

the NRS core measures by encouraging providers to improve the quality of their instruction and 

administration of data. A review of national performance data appears to supports this claim, 

with the three study states outperforming the national average on many of the NRS measures, as 

well as registering substantial improvements on individual measures across years. 

Unfortunately, it is not possible using existing data to ascribe 

causality to PBF in improving state outcomes. Although 

performance funding is associated with positive changes in 

state performance, it is possible that a host of other factors, 

such as changes in the underlying population, learner access 

to other educational services, curricular innovations, and other 

education reform efforts have influenced state outcomes. 

Consequently, while state performance data can provide a 

snapshot of how states were performing prior to and 

immediately following the adoption of performance funding, 

it is impossible to attribute program successes directly to its use. 

Moreover, with the exception of Kansas, the first state to introduce PBF, in FY00, there is 

limited data to evaluate the effect that performance funding may have had on the delivery of 

I think our outcomes are coming

quicker. I think the students are

seeing their progress much faster,

which keeps them encouraged and

very positive. It keeps the teachers

more on task. It’s also easier to see a

weaker area that needs the fixing if

you’re really looking at your data and

your results and kind of looking for

things that you need to gather and

assess. 

—Program Director, Indiana 
SECTION 6 

Effects of Performance-based Funding 

— 35 —

adult education services.6 For example, though Missouri rolled out performance funding in 

FY02, state administrators held provider allocations constant for the first two years of formula 

operation, with funding based on assessment results collected prior to or concurrent with the 

introduction of new AEFLA reporting requirements. As state allocations shifted to include actual 

provider performance beginning in FY04 (based on FY02 data), differences in provider 

outcomes first became evident. 

A review of NRS outcome data indicates that between program year 2000–2001 and 2003–04, 

all three states improved their performance on measures used to allocate performance funding. 

Of the three states, Kansas has the most comprehensive data on the effects of PBF on statewide 

performance due to its early implementation. As illustrated in table 4, the percentage of ABE and 

ASE learners realizing learning gains in Kansas climbed from 53 percent in 2000–01 to 60 

percent in program year 2003–04. And although outcomes have fluctuated somewhat across 

years, state ABE and ASE performances consistently have outpaced the national average. 

Although program enrollments have fluctuated over time, Indiana adult education providers also 

have increased their performance outcomes achieved on the NRS measures over time. As 

illustrated in table 4, the percentage of participants achieving an NRS core outcome has 

increased dramatically; as a case in point, high school completion rates climbed from 46 percent 

in program year 2000–01 to 91 percent in program year 2003–04, a rate more than double the 

national average. 

At the same time, the number of participants in Indiana achieving state-identified core or 

secondary performance outcome on the NRS measures also improved, with participant successes 

climbing from 45,290 in PY00–01 to 56,043 in PY04–05, nearly a 24 percent increase compared 

to a much smaller population increase of roughly 3 percent (42,135 participants in PY00–01 

compared to 43,498 in PY04–05) (table 5). This suggests that the state was able to 

simultaneously increase its performance while expanding its participant base. 

Of particular interest is that since PY02–03, when Indiana halved its reimbursement rate for the 

secondary NRS measures, the number of secondary outcomes has fallen from 14,314 in PY02–

03 to 11,826 in PY04–05. At the same time, the number of core measures achieved climbed from 

39,787 to 44,217. This suggests that providers may be shifting away from pursuing lower value, 

though potentially more easily obtainable, secondary measures in favor of NRS core measures. 

 

6 Federal fiscal year funds are allocated to states for program operations the following year. For this reason, outcome data for federal program 

year 2000–01 would relate to federal resources allocated in FY00. To simplify discussion, this paper refers to state fiscal year and federal 

program year data, which cover the same reporting period. 
SECTION 6 

Effects of Performance-based Funding 

— 36 —

PY01 PY02 PY03 PY04

Educational Gain ABEb/ASEc

National 36 37 38 38 

Kansas 53 76 58 60 

Indiana 38 37 39 43 

Missouri 31 31 48 35 

Educational Gain ESL

National 32 34 36 36 

Kansas 42 68 55 59 

Indiana 34 35 40 39 

Missouri 29 16 31 31 

High School Completion

National 33 42 44 45 

Kansas 50 71 61 64 

Indiana 46 74 84 91 

Missouri 33 44 37 54 

Entered Postsecondary

National 25 29 30 30 

Kansas 25 69 62 58 

Indiana 53 77 83 90 

Missourid 19 27 19 22 

Entered Employment

National 31 39 37 36 

Kansas 50 59 52 62 

Indiana 62 82 96 90 

Missourid 51 44 54 55 

Retained Employment

National 62 63 69 63 

Kansas 53 74 54 69 

Indiana 63 87 62 91 

Missourid 70 34 67 68 

NOTE: Shaded areas reflect periods when performance funding was in use.

a Data reported in this table refer to federal NRS data for program years 2000–01 to 2003–04. These data correspond to state fiscal 

year data for FY01 to FY04. 

b

 ABE—Adult Basic Education.

c

 ASE—Adult Secondary Education.

d Did not have performance measures addressing this indicator.

Percent

SOURCE: U.S. Department of Education, Report to Congress on State Performance, Program Years 2000–01 to 2003–04 .

Table 4—

Percentage of Adult Learners Nationally and in Three States Achieving Outcomes on the NRS Core 

Performance Measures by Types of Measure: Program Years 2000–01 to 2003–04a 
SECTION 6 

Effects of Performance-based Funding 

— 37 —

If this interpretation is accurate, it may be that states can motivate providers to achieve desired 

state goals by carefully selecting the type of factors included in their adult education funding 

formula and the value attached to a given outcome. It is also likely that the difficulty associated 

with collecting different types of data and the amount of funding associated with a given 

performance outcome will affect the degree of provider response. 

Changes in Local Adult Education Providers 

Performance funding was associated with changes in the number of providers in Kansas and 

Missouri, although state directors did not believe that PBF itself was responsible for providers 

leaving the service network. Instead, it was suggested that the adoption of PBF highlighted 

existing problems within providers, which, in turn, led state administrators to take steps to 

address noted programmatic deficiencies. 

Before the adoption of PBF in Kansas in FY00, 36 providers were participating in the adult 

education system.7 In keeping with the state’s promise to protect programs in the first year of 

formula implementation, state administrators renewed funding for all providers for FY01. 

However, three programs were placed on probation and conditionally funded because of quality 

issues associated with their services. After failing to make progress in improving services, these 

programs were denied funding in the final year of the state’s three-year funding cycle. 

 

7 Kansas instituted performance funding in FY00, which was also the beginning of a three-year funding cycle for providers, meaning that 

providers competed for funding in FY99. The state then moved to a new two-year funding cycle for FY03 to FY04, with providers competing in 

FY02.

PY01 PY02 PY03 PY04 PY05

Participants 42,135 44,492 41,397 41,148 43,498

Providers Qualifying for Incentives 43 43 43 43 43

Performance Outcomes 45,290 50,620 54,101 54,370 56,043

Core NRS Measures — — 39,787 40,296 44,217

Secondary NRS Measures — — 14,314 14,074 11,826

Outcomes per Participant 1.1 1.1 1.3 1.3 1.3

Performance Goalsa

 Achieved 357 373 342 349 350

Average Goals Achieved 8.3 8.7 8.0 8.1 8.1

SOURCE: Indiana Department of Education.

Note: — Not applicable.

a Number of program targets achieved by local providers and statewide average targets achieved across providers.

Table 5—

Selected Indicators of Indiana Adult Education Programs: Years 2000–01 to 2004–05
SECTION 6 

Effects of Performance-based Funding 

— 38 —

Of the three programs, only one qualified for funding in the FY03 to FY04 competitive funding 

cycle. In the estimation of the state director, provider failure to qualify for funding resulted not 

because the agencies lacked the finances to offer services, but because program staff failed to 

offer adult education services of sufficient quality to warrant continued recognition as a state 

program. 

The state director in Missouri offered a similar assessment. Before the adoption of PBF in 

Missouri, 46 providers were awarded adult education funding. In the year following the adoption 

of PBF, which coincided with the start of a new three-year funding cycle for adult education 

(FY03–FY05), three providers were unsuccessful in competing for AEFLA funds, and one new 

program successfully competed and was funded. Of the three programs no longer participating, 

one was unable to generate sufficient contact hours or performance outcomes to offset its 

operational costs. According to the state director, the loss of the two remaining programs was 

due to errors and omissions by the provider in the application for funding, which resulted in the 

request being rejected.
— 39 — 

SECTION 7: 

Considerations in Transitioning to 

Performance-based Funding 

Introducing PBF into state adult education funding formulas is a complicated process, one that 

requires state directors and their staff to commit time and resources to overseeing formula 

drafting and implementation efforts. To gain providers’ approval in study states, state directors 

of adult education recruited directors of local program providers to serve on an advisory council 

that represented the views of the field in the formula drafting process. State directors were 

careful to select individuals who were respected for the quality of their programs, and who were 

not afraid to voice their opinion or to defend the system they helped to create. Directors also took 

care to ensure that providers were representative of the diversity of program types in the field, 

accounting for differences in provider type, size, learner demographics, and location. 

Formula development occurred using an open, consensusbased process. Advisory council meeting agendas provided 

time for participants to discuss their issues and concerns and 

to contribute their ideas to formula design. Building buy-in 

helped to ensure that council members supported project 

work. To illustrate, the former state director of Missouri 

recounted how a small group of program directors differed 

concerning PBF at a program directors’ retreat. Council 

members who had participated in the formula development 

process allayed dissent, pointing out that they had been 

involved in the process and had worked to ensure that providers’ viewpoints were incorporated 

into the final formula. As he observed, “We didn’t have to defend it. The people involved in the 

process defended it, and that was fantastic.” 

Although implementing PBF will likely require substantial investment of time on the part of state 

agency administrators, it bears repeating that states participating in this study were early 

implementers of PBF, and, as a consequence, faced the dual challenge of instituting new AEFLA

reporting requirements in conjunction with PBF systems. 

The most important concept in

implementing or rolling out a change

in funding ... is communication.

Because to just put it out there and

say, “Here it is,” won’t work. It takes

continually putting it out there and

saying, “This is what it is and this is

what it means to you.” ... Not only on

a state level—from state to program

directors—but program directors to

staff. 

—Program Director, Indiana
SECTION 7 

Considerations in Transitioning to Performance-based Funding 

— 40 —

Key Considerations in Formula Design 

States that have succeeded in adopting PBF have developed differing strategies to allocate adult 

education resources. This is because no one formula will work in all state contexts. Regardless of 

the approach used to distribute assets, all states face similar issues in designing their funding 

systems. Administrators in states considering using performance funding can improve the 

likelihood of success by answering some of the following questions during the development 

process. These include: 

How Much Funding Should Be Allocated Using Performance Criteria? 

States profiled in this report allotted different amounts of funding for distribution on the basis of 

performance funding. Funding levels ranged from a high of 88 percent of combined federal and 

state resources in Kansas to just 5 percent of available funds in Indiana. Conversations with 

program directors suggest that each approach was effective in changing provider behavior. As 

noted by the assistant state director in Indiana, even though only 5 percent of resources were 

allocated based on performance, providers acted as though 90 percent of their funds were at risk, 

so great was their desire to earn additional resources. 

The central question, then, is how much money is necessary to motivate providers to accomplish 

state goals and, from which source—federal, state, or both—should funding be tapped? 

Answering this question depends on a number of factors, including the relative amount of federal 

and state resources available to state administrators, the manner in which existing funds are 

distributed, the perceived need for statewide adult education reform, and the attitude toward risk 

among state administrators and providers. 

Funding levels also should take into account state capacity to support providers in transitioning 

to the new funding system. Since some providers will stand to lose resources, state 

administrators may wish to increase their adult education funding concurrent with system rollout. 

In study states, this occurred either through the addition of new adult education resources or the 

release of held-back federal funds. Providers also will need to be educated in how the new 

formula operates and how they can improve their funding eligibility; as such, when setting 

performance-funding levels, state administrators must consider their capacity to provide 

statewide and individualized technical assistance should the new formula place a substantial 

amount of provider resources at risk. 

How Should Funding Formulas Be Structured? 

Once resource levels have been established, states need to identify criteria for allocating 

resources among providers. This includes identifying a set of measures for formula inclusion and 
SECTION 7 

Considerations in Transitioning to Performance-based Funding 

— 41 —

determining their relative weight in the formula. State administrators also must consider provider 

capacity to report data in a timely manner and consider state capacity to audit program 

submissions for accuracy. 

• Selecting Performance Measures—On balance, the less complexity associated 

with performance funding the easier state systems are to understand and 

administer. Ideally, program measures and formula operation will align with and 

reinforce state policy priorities that have been identified, while limiting the 

reporting burden placed on providers. Care also must be taken to ensure that 

providers with unique characteristics, including those serving small and rural 

populations or large numbers of lower-level or ESL learners, are able to compete 

for resources on a level playing field. Study states used the following approaches 

to collect performance data: 

○ NRS Core and Secondary Measures—All states relied on the NRS core 

and, in some instances, secondary measures to allocate performance 

funding. Since providers are required to annually report on NRS measures, 

using a common set of testing protocols and data collection procedures, the 

use of these measures does not add an additional data burden on local 

agencies. During interviews, state administrators and program staff 

remarked on the value of the NRS measures, believing that they were good 

indicators that captured their program goals. And because states are held 

accountable by the federal government for their performance on these 

measures, state administrators believed that focusing provider attention on 

these measures helped their states achieve negotiated performance targets. 

Program directors did not support using all of the core and secondary NRS 

measures to award performance funding. In the case of the NRS core 

measures, provider dissatisfaction was due not to the measures themselves, 

but to the difficulty associated with collecting post-program outcomes 

particularly when follow-up survey methods were employed. 

Study participants were less satisfied with using secondary NRS measures 

to award program funding. During interviews, program directors suggested 

that, because many of the NRS secondary measures are self-reported, these 

measures lack the validity to be included in the state funding formula. 

Providers also felt that some of the secondary NRS measures, such as 

achieving citizenship skills or learner involvement in children’s literacyrelated activities, were less important because these measures did not 

always relate to programs’ education development goals. 
SECTION 7 

Considerations in Transitioning to Performance-based Funding 

— 42 —

○ State Process Indicators—Kansas is unique in that, in addition to the NRS 

measures, state administrators employ a set of 25 quality indicators to 

assess program administrative processes and procedures. These measures, 

built off an existing set of state-established indicators, are believed to 

capture the crucial elements that underlie a successful program. And since 

quality measurement criteria function independent of provider size, in most 

cases, small and large providers have an equal opportunity to earn points. 

While the decision to add quality measures to the funding formula offered 

the state a number of benefits, their addition requires that the state conduct 

annual site visits to all providers, as well as to collect and administer a large 

amount of program information. 

Conduct of annual monitoring to assess implementation of the 25 quality 

indicators has added a substantial, recurring cost to PBF formula operations 

in Kansas. State expenses include paying for monitoring team salaries and 

travel, and staff time spent coordinating with team members and 

administering quality point award data. Local program directors report that 

they also incur costs prepping for monitoring team visits, which include 

using the monitoring instrument to conduct self-assessments, updating and 

reviewing participant files for accuracy, and meeting with monitoring team 

members during the one-day visit. 

○ Incentive Funding—In an effort to even the playing field for providers, 

Indiana created incentive funding to allocate federal resources to providers 

meeting their performance targets on the NRS core measures. Here, 

providers qualify for a fixed dollar amount for achieving each of their statenegotiated NRS targets, meaning that all providers receive the same amount 

of resources for a positive result. This has helped to ensure that small and 

rural providers, which can have difficulty generating large numbers of 

participant outcomes, can effectively compete for performance funds. 

Since providers are reimbursed for the performance targets they achieve, 

program directors tend to encourage instructors to make fundable gains. 

Consequently, when state goals align with federal reporting measures, the 

design of PBF systems can actually contribute to a state achieving its 

federally negotiated levels of performance. And because incentive funding 

is based on the NRS core, providers face no additional burden in reporting 

on these measures. 

• Weighting Learner Outcomes—According to state administrators, not all learners 

make education progress at the same rate, meaning that instructors may have 
SECTION 7 

Considerations in Transitioning to Performance-based Funding 

— 43 —

difficulty achieving measurable gains for some students, particularly those with 

documented English language difficulties or learning disabilities. To ensure that 

providers offer services to all students, administrators in all three states assigned 

different weights to outcomes achieved by some populations of adult learners. 

The weights vary by state, with Kansas and Missouri attaching supplemental 

resources for lower-level and ESL learner achievements, while Indiana assigns 

different values to learners who complete a core versus a secondary NRS 

measure. 

Differentially weighting learner outcomes can be a logical decision if it can be 

proven that some learners are relatively more expensive to educate, and that to not 

take such action will prohibit the state from achieving its policy objectives. 

Before deciding to follow this route, it is imperative that state administrators 

analyze existing state data, for example, by calculating the average time it takes 

learners to complete each of the NRS functional levels to determine whether any 

action is necessary. Administrators also should assess the distribution of learners 

across providers to ensure that differentially weighting learner outcomes does not 

offer any programs a competitive advantage. 

If there is empirical evidence to support that there are differences in the cost of 

educating learners, then state administrators must assess the trade-offs of 

assigning supplemental weights to higher-cost learners. It may be, for example, 

that variation in educational gain may be due to the quality of instructors or 

because lower expectations are set for certain types of learners. If this is so, then 

adding a supplemental weight may actually discourage educators from 

undertaking needed reform. 

• Linking Outcome Data to Performance Allocations—Provider allocations in all 

three study states are determined by program and learner performance in earlier 

years. In both Indiana and Kansas, program allocations are a function of outcomes 

from the preceding fiscal year, meaning that FY06 expenditures are based on 

FY05 performance. This one-year lag means that program directors begin 

planning for the upcoming fiscal year without complete information on their 

fourth quarter outcomes.8 Should learners fail to achieve at their predicted rate, 

providers may fall short of their anticipated revenue, which may force them to cut 

programs or services to balance institutional budgets. 

 

8 For example, program directors in Indiana or Kansas would typically begin building their FY06 program budget in March 

2005, before they have information on their fourth-quarter performance. Consequently, providers must estimate the number of 

outcomes they anticipate for the fourth quarter of FY05 in order to produce their FY06 budget.
SECTION 7 

Considerations in Transitioning to Performance-based Funding 

— 44 —

Although a one-year lag between outcomes and resources can place providers at 

financial risk, immediate reimbursement reinforces the connection between 

performance and results. Moreover, linking funding to prior year performance 

enables providers to respond more rapidly to program changes, such as 

unanticipated growth in program enrollments. To lessen potential effects, states 

may opt to reduce the amount of funding associated with fourth-quarter outcomes; 

for example, in FY05 Kansas allocated roughly 13 percent of its PBF funding 

based on fourth-quarter results. 

Program funding in Missouri is based upon performance data collected during the 

fiscal year two years preceding the current one, meaning that FY06 expenditures 

are based on FY04 performance. This two-year lag provides program directors 

with more complete information when they begin their budgeting process. 

However, since programs experience a delay between when they record outcomes 

and when they are reimbursed, a program experiencing substantial enrollment 

growth must carry its increased costs until state allocations catch up with its 

expanded needs. 

Determining which data should be used to allocate performance resources 

depends on a number of issues, including state capacity to process provider data 

and provider capability of reporting information in a timely manner. Other factors 

also should be considered. For instance, states experiencing rapid demographic 

changes may wish to consider using data lagged one year, as in Indiana and 

Kansas, to ensure that programs undergoing substantial growth are funded based 

on their current conditions. 

What Factors Should Be Considered in Formula Implementation? 

State administrators reported that the design of their funding formulas was an iterative process, 

one that required that they model different funding scenarios assuming different allocation 

strategies. Program directors who participated in advisory council meetings reported that the use 

of historical data to project allocations permitted them to see the ramifications of different 

funding assumptions on different types of providers. For this reason, state administrators should 

plan to conduct formula simulations and data analyses to assess which program providers stand 

to lose funding, and, if so, the size of any anticipated losses. 

• Insulating Providers From Funding Decreases—Study states introduced PBF 

without taking precautionary steps to protect providers from sudden changes in 

their resource eligibility, such as by introducing a hold-harmless provision to 

initial allocations. States’ decision to fully-implement funding formulas were 
SECTION 7 

Considerations in Transitioning to Performance-based Funding 

— 45 —

based on a number of factors, including: funding formulas were introduced at a 

time of increasing federal and state resources for adult education, state 

administrators released held-back federal funding during the first years of formula 

use to augment available resources, and preliminary formula simulation suggested 

that few providers would be adversely affected. 

To avoid penalizing programs with below-average performance, states may wish 

to hold harmless providers that are projected to lose funding by compensating 

them up to the level of their loss. However, this hold-harmless provision should 

be phased out over time so that providers have an incentive to change; for 

example, states could provide affected programs with 75 percent of their funding 

in the first year, 50 percent in the second year, and so on, until the fourth year in 

which programs would receive no offsetting compensation.

Although Missouri did not attempt to protect providers from initial changes in 

their funding eligibility, over time the state increased its share of resources 

distributed based on performance. During the first year of PBF in FY02, the state 

allocated 6 percent of its resources as PBF; by FY05, the state allocation had 

climbed to 19 percent through a combination of increasing funding for learners 

achieving performance outcomes and reducing rates for learner-contact hours. 

Ramping up performance funding in this way allows providers to experience the 

impact that PBF can have on programs, without putting significant program 

resources in jeopardy. This can give providers time to address obvious data 

problems and initiate program improvement efforts so that they are prepared 

when PBF is fully implemented. 

• Offering Technical Assistance to Support Providers—Because states profiled in 

this report were early implementers of PBF, most state technical assistance 

activities initially focused on issues associated with NRS reporting, such as how 

to collect and enter program data into new, state-designed databases. 

Administrators also provided directors with an explanation of how the PBF 

formula operated so that they could understand the outcomes they would need to 

achieve to increase their funding eligibility. In addition to statewide trainings, 

outreach efforts targeted individual providers that had lost funding or that were 

having difficulty generating performance outcomes. 

Over time, as local familiarity with reporting system operations increased, the 

focus of training shifted from methodological issues undercutting program 

reporting to teaching provider staff how to analyze program data. For example, 

recognizing that there was a problem with measuring learner gains, early in the 

PBF process Kansas held a series of technical assistance training sessions on the 
SECTION 7 

Considerations in Transitioning to Performance-based Funding 

— 46 —

proper use of CASAS assessments, including reviewing testing protocol, test 

selection, and data entry. More recently, the state has focused on improving 

lesson planning and instruction to support program improvement efforts. 

Although by now most adult education providers have had experience reporting 

on the NRS core and secondary measures, it should not be assumed that all 

program staff understand or follow state reporting protocols. Consequently, states 

considering adopting PBF systems should plan to review data collection strategies 

with providers to prepare them to report accurate data that comply with state 

accountability provisions. 

States also should sponsor technical assistance workshops to assist providers in 

understanding how the state funding formula operates, how to analyze 

performance data to identify areas in need of improvement, and how to initiate 

programmatic changes to improve their funding eligibilities. This can be 

accomplished using a variety of strategies; for example, early in the PBF 

implementation process, program directors attending technical assistance 

workshops in Missouri were supplied with fictitious program outcome data and 

taught how to interpret it. Later, when individual program reports were finally 

available from the state data system, directors were again convened, but this time 

supplied with their own program data for analysis. 

• Auditing State Data—State administrators in all three study states instituted 

quality controls to ensure that local provider reports accurately reflect local 

conditions. These procedures included annual desk audits of program submissions 

to determine whether data were consistent across years and reasonable, given 

administrators’ knowledge of provider operations. Staff in Indiana and Missouri 

also conducted field visits to 20 percent of providers, and in Kansas, to 100 

percent of providers each year, using a standardized evaluation instrument to 

assess the reliability of program data. 

Given the high stakes associated with provider data submissions, administrators 

considering adopting PBF should plan to review the quality of program data and 

educate providers on the importance of submitting accurate information. If the 

experience of study states is any indication, it is likely that many programs are 

currently reporting data that are not collected using proper testing procedures 

(e.g., using different pre- and posttests). Accordingly, administrators must 

communicate to providers how their data will be evaluated and the steps they 

must take to guarantee reported outcomes are valid.
— 47 — 

APPENDIX A 

Indiana Case Study Report: 

Performance-based Funding In 

Adult Education 

— 49 — 

SECTION A-1: 

Overview of the State Adult Education Funding 

System 

In FY01, the Indiana Department of Education (IDE) adopted a new formula for allocating its 

federal adult education funding among eligible service providers to promote program 

improvement, while retaining its existing formula for allocating state funding. Under the current 

system, local providers are guaranteed a base level of funding, drawn from federal and state 

resources, that is sufficient to maintain their existing programs. Providers compete for remaining 

federal resources based on their performance. Allocations are distributed using the following 

criteria. 

Base Funding: Roughly 85 percent of federal and 100 percent of state resources are allocated as 

base funding using the following distribution criteria: 

• Federal Resources—Providers automatically qualify for 90 percent of the federal 

dollars that were in their 1999–2000 fiscal year (FY) base budget. This base 

remains constant over time and is associated with a funding district, not an 

individual provider. Should an existing provider withdraw from the system, the 

state would identify a new provider within the district and transfer the resource 

allocation from the old to new agency. 

• State Resources—Providers automatically qualify for 90 percent of their 

reimbursed expenditures for the summer and fall terms of 1991 and spring term of 

1992, or, for new programs, a base amount established at the close of the first full 

year of operation. This base remains constant unless an agency’s expenditures fall 

below its base during a given period; when this occurs, the reduced 

reimbursement becomes the new base. Remaining appropriations or new state 

dollars are distributed on the basis of enrollments, such that over time a greater 

percentage of the state appropriation is based on demand (i.e., enrollment growth 

or decline). 

Performance Funding: Approximately 15 percent of federal resources are distributed based on 

provider performance on the core and secondary measures of the National Reporting System for 

Adult Education (NRS), the accountability system for the federal Adult Education and Family 

Literacy Act (AEFLA). Nationwide, states are required to report on the core measures in the 
SECTION A-1 

Overview of the State Adult Education Funding System 

— 50 —

NRS, which include outcome, descriptive, and participation measures, and they have the option 

of reporting on the secondary measures, which include outcomes related to employment, family, 

and community. 

Federal performance-based funding (PBF) is allocated according to the number of adult learners 

achieving outcomes on state-identified core and secondary performance measures. Incentive 

grants also are provided for agencies meeting their state-negotiated performance levels for 15 

measures of educational gains included in the NRS core measures. Performance funds lag one 

year behind when outcomes are recorded, meaning that local providers’ eligibility in FY06 is 

determined by program performance in FY05. State allocation criteria include: 

• Performance—Number of provider outcomes, calculated by summing the number 

of adult learners who achieve any of the following: 

○ Completed a level or completed a level and advanced to a higher level; 

○ Entered employment, retained employment, obtained a secondary diploma 

or general equivalency diploma (GED), and entered postsecondary 

education or training; 

○ Achieved work-based project learning goals, left public assistance, 

acquired citizenship skills, increased involvement in their children’s 

education or literacy activities, voted or registered to vote, and increased 

involvement in community activities; 

○ In family literacy programs, advanced an educational functioning level, 

entered or retained employment, obtained a secondary diploma or GED 

diploma, entered postsecondary education or training, and increased 

involvement in their children’s education or children’s literacy activities; 

○ In workplace literacy programs, advanced an educational functioning level, 

entered or retained employment, obtained a secondary diploma or GED 

diploma, and enrolled in postsecondary education or training; and 

○ In corrections programs, advanced an educational functioning level, entered 

or retained employment, obtained a secondary diploma or GED diploma, 

and entered postsecondary education or training. 

Local provider resource eligibility is calculated by multiplying the total number of individuals 

achieving a positive outcome on the state-identified NRS core and secondary measures by a 

state-established rate for each measure. The amount awarded for a secondary NRS performance 

outcome is half the amount awarded for a core NRS measure. For example, in FY05, providers 
SECTION A-1 

Overview of the State Adult Education Funding System 

— 51 —

earned $16.45 for each learner outcome on a core NRS measure, and $8.22 for each learner 

outcome on a secondary NRS measure. 

• Incentive—Funding for districts meeting their state-negotiated performance levels 

for the following NRS core measures: 

○ Completed one of 11 education levels identified for adult basic education 

(ABE), adult secondary education (ASE), and English as a second language 

(ESL) learners as specified in the core NRS measures; 

○ Entered further education and training; 

○ Entered employment; 

○ Retained employment; and 

○ Completed high school diploma or GED. 

Local provider eligibility is calculated by multiplying the total number of negotiated 

performance targets achieved by a state-established rate that is equivalent for each measure. For 

example, providers earned $630.25 for each performance measure that they met or that exceeded 

their state-negotiated performance target in FY05. 

State Allocations 

In FY05, the state allocated a total of $21,055,280 in federal and state funding for adult 

education grants to providers (table A-1). Of this amount, $7,055,280 (34 percent) came from 

federal and $14,000,000 (66 percent) from state sources. 
SECTION A-1 

Overview of the State Adult Education Funding System 

— 52 —

Of the $21,055,280 allocated to local providers, IDE administrators distributed $20,007,828 (95 

percent) as base funding among the 43 eligible providers in the state. The remaining $1,1047,452 

was earmarked for performance funding, with $824,993 (4 percent) allocated as performance 

grants and $222,459 (1 percent) as incentive awards (fig. A-1). Overall, this accounted for 

approximately 5 percent of total adult education resources distributed in FY05 to local providers 

in the state. 

Organization of State Adult Education Services 

The Division of Adult Education of the Indiana Department of Education administers adult 

education and literacy services offered by 43 local providers, including 42 comprehensive 

programs located in secondary and postsecondary agencies, and the Indiana Department of 

 Total Federal State 

 Resources Resources Resources

Total Local Provider Allocation $21,055,280 $7,055,280 $14,000,000

Percentage of Total 100.0% 100.0% 100.0%

Base Funding $20,007,828 $6,007,828 $14,000,000

 Percentage of Total 95.0% 85.2% 100.0%

Performance Funding $1,047,452 $1,047,452 —

 Performance Grants $824,993 $824,993 —

 Percentage of Total 3.9% 11.7% —

 Incentive Awards $222,459 $222,459 —

 Percentage of Total 1.1% 3.2% —

SOURCE: Indiana Department of Education, 2006. Funding data provided by state.

Note: — Not applicable.

Table A-1—

Indiana Adult Education Funding Allocated as Base or Performance Funding: FY05

SOURCE: Indiana Department of Education, 2006. Funding data provided by state.

Figure A-1—

Percentage of Indiana Local Provider Allocation by Funding Type: FY05

1%

95%

4%

Incentive Awards

Performance Outcomes

Base Funding
SECTION A-1 

Overview of the State Adult Education Funding System 

— 53 —

Corrections. Ten program administrators work in the Division of Adult Education, two of whom 

directly administer the state’s PBF system. These individuals include: 

• State Director of Adult Education—The current state director, appointed in late 

1996, oversees division staff and is responsible for ensuring that the state agency 

and local providers comply with AEFLA provisions. The director is also 

responsible for administering state funding for adult education and for supervising 

the state’s GED testing program, the federal Even Start program, and services 

provided by the statutory Indiana Adult Literacy Coalition. 

• Assistant Director of Adult Education—The assistant state director, stateemployed since early 1991, manages the state’s adult education funding, 

coordinates data collection, and is responsible for evaluating service providers 

under the PBF system. 

Other division staff include three education consultants who provide program support services, a 

coordinator of the Even Start program, a GED administrator, and three support staff. Additional 

consultants and support staff work in the field on a part- or full-time basis directing the state’s 

regional professional development system, or delivering training and technical assistance to 

program directors and their staff. 

Local Providers 

With the exception of one university and the state Department of Corrections, federally funded 

adult education services are based in secondary school districts, with providers remaining 

unchanged since the adoption of performance-based funding in FY01. Providers are 

geographically distributed throughout the state, with classes offered at more than 350 adult 

education sites. 

State adult education enrollments have fluctuated slightly over time, with enrollment in FY05 

approaching 43,500. According to state administrators, any variations in adult participation (table 

A-2) are likely due to fluctuations in state economic conditions and eligible populations, with the 

exception of FY05, when the state corrections department added facilities, thereby increasing 

total enrollment. The gender balance is roughly evenly split between males and females, with 

little variation over time. The racial profile of participants has also remained relatively stable, 

with participants in FY05 primarily white (53 percent), followed by black (24 percent), Hispanic 

(19 percent), Asian (4 percent), and American Indian/Alaskan Native and Native Hawaiian/Other 

Pacific Islander (1 percent) accounting for remaining participants. Finally, roughly one-third (36 

percent) of participants in FY05 were between the ages of 25–44 years, with those aged 19–24 

accounting for a quarter (26 percent) of enrollments (fig. A-2). 
SECTION A-1 

Overview of the State Adult Education Funding System 

— 54 —

FY00 FY01 FY02 FY03 FY04 FY05

Providersa 43 43 43 43 43 43

Unified School Districts 41 41 41 41 41 41

Universities 111111

Corrections 1 1 1 1 1 1

Participants 41,760 42,135 44,492 41,397 41,148 43,498

 Gender

Male 53% 52% 51% 51% 50% 54%

Female 47% 48% 49% 49% 50% 46%

 Race-Ethnicity

White 57% 55% 55% 54% 54% 53%

Black 22% 21% 21% 21% 21% 24%

Hispanic 16% 19% 19% 20% 20% 19%

Asian 4% 4% 4% 4% 4% 4%

American Indian/Alaskan Native <1% <1% <1% <1% <1% <1%

Native Hawaiian/

Other Pacific Islander <1% <1% <1% <1% <1% <1%

SOURCE: U.S. Department of Education, Report to Congress on State Performance, program years 1999–2000 to program years

2004–05. http://www.doe.state.in.us/adulted/performancedata.html (last accessed June 18, 2007).

a Provider data included in this table is limited to Comprehensive Program Providers participating in the PBF system. The state also 

provides a small number of grants of federal resources to community-based organizations and state funding to small school corporations 

based on enrollment. Statewide totals for all providers are included in participants’ data.

Table A-2—

Selected Characteristics of Indiana Adult Education Providers and Participants: FY00 to FY05

SOURCE: U.S. Department of Education, Report to Congress on State Performance , program years 1999–2000 to program years

2004–05. http://www.doe.state.in.us/adulted/performancedata.html (last accessed June 18, 2007).

Figure A-2—

Ages of Indiana Adult Education Participants: FY05

27%

26%

36%

9% 2%

16-18

19-24

25-44

45-59

60+
— 55 — 

When you get the question, “How is

money allocated?” and you say, “In

part by performance,” that seems to

be the end of discussion. When

stakeholders out there hear the word

“performance,” there isn’t really a

lot of other questions that follow. 

—Indiana Assistant State Director 

of Adult Education 

SECTION A-2: 

Rationale for System Development 

Unlike other states profiled in this study, both the Indiana state director and assistant state 

director of adult education predated AEFLA. Therefore, the decision to implement performance 

funding in Indiana was not associated with a turnover in state leadership, but rather came as a 

calculated response to the new federal performance reporting provisions in AEFLA. Believing 

that the new legislation called for holding local providers accountable for their learners’ 

outcomes, state leaders opted to introduce PBF into their federal funds allocation formula as a 

way to improve their federal reporting capacity. 

The introduction of performance funding also was seen as a 

way to address statewide concerns about the way adult 

education funds were distributed. Although administrators 

reported feeling no legislative pressure to adopt PBF, they did 

recall questions in 1997 from a new state board member about 

whether performance was considered in the provider 

allocation. This question raised administrators’ awareness and 

contributed to their decision to introduce an accountability 

mechanism into the federal allocation formula. 

During interviews, program directors reported dissatisfaction with the existing contact-hour 

formula, with one questioning whether distribution of resources based on seat-time was the best 

method for holding programs accountable. Instead, some directors favored a funding formula 

that would reward more productive programs, believing that educators would be more likely to 

undertake staff development and curricular reforms if given financial incentives to improve. 

Directors also reported that many of their peers were resistant to making sweeping changes to the 

funding formula without a promise that any new system would be at least as fair as the current 

one, as well as simple to use, because directors were averse to adding yet another reporting 

burden to their workload. 

State Adoption Process 

To initiate discussion about PBF, the state director placed the issue of federal performance 

funding on the agenda of the statewide administrators’ conference in December 1998. Meeting 

discussions served to forewarn program directors of coming changes and provide initial 
SECTION A-2 

Rationale for System Development 

— 56 —

information on state planning efforts. In retrospect, state administrators and program directors 

did not recall any significant opposition to the planned adoption of performance funding in 

Indiana, although not all directors were enthusiastic about its introduction. Lack of resistance 

was attributed to state administrators’ efforts to communicate the need for formula change to the 

field (i.e., local providers) and a positive relationship between state administrators and program 

directors that helped dilute local opposition to the plan. 

Provider acceptance of PBF also may have reflected a growing awareness of the need for better 

information to promote the benefits of adult education. As the state director observed, studies at 

the time documented that many adult education and literacy programs were plagued by poor 

attendance and high dropout rates, evidence often used by critics to suggest that funding based 

on contact hours was not effective. State administrators also were beginning to anticipate 

legislative challenges to continued state, and even federal, adult education funding. Integrating 

performance funding into the state’s adult education funding formula was seen as a defensible 

alternative to status quo operations, because it promised to steer resources toward agencies that 

warranted support. 

Goals in Introducing Performance-based Funding 

The rationale for introducing performance funding in Indiana stems from language contained in 

Title II of the Workforce Investment Act of 1998, in which agencies administering adult basic 

education funds were directed to consider the past effectiveness of eligible providers in meeting 

or exceeding the performance measures contained in AEFLA. In interpreting this language, the 

division of adult education set out to create a new comprehensive federal funding formula that 

included performance indicators collected by the U.S. Department of Education for its annual 

report to Congress. 

Although formal goals for introducing PBF were not elaborated, state administrators sought to 

improve state capacity to report on the NRS measures. Administrators also focused on improving 

the quality of adult education programs offered in the state. In doing so, an overriding concern 

was that the new formula be based on a fair and equitable distribution of available resources. 
— 57 — 

SECTION A-3: 

Formula Design Process 

State and federal adult education resources are allocated using two discrete formulas. Although 

both formulas distribute a majority of resources based on learner-contact hours, state 

administrators reported that development committees sought to incorporate performance-funding 

criteria into both systems. 

Specification of Funding Criteria: State Funding 

The existing formula for allocating state resources dates back to 1991, when the state board of 

education assembled an adult education funding study committee, appointed by the 

superintendent of public instruction, to consider how state adult education funds were allocated 

to local school corporations. The study committee consisted of approximately 15 members, and 

included department of education personnel, adult educators, a school superintendent, and other 

school administrators. Committee members met several times during 1991 and later submitted a 

series of recommendations, including a new state funding formula for adult education, which the 

board adopted in 1992. 

Committee members opted to distribute roughly 90 percent of state funding in FY93 as a base 

grant dependent on agencies’ reimbursed FY92 expenditures for the summer and fall terms of 

1991 and spring term of 1992, with the remaining 10 percent based on program enrollments. The 

committee further stipulated that any remaining state appropriations or additional state funding 

for adult education was to be allocated based on enrollments. This was intended as a form of 

performance funding, since agencies losing or gaining students would be penalized or rewarded 

based on their ability to attract learners. Panelists reasoned that successful programs would be 

more likely to attract students, thereby creating an incentive for programs to undertake efforts to 

improve service quality. 

State resources continue to be allocated based on local providers’ FY92 expenditures, with 

funding appropriated by the Indiana legislature as part of its biennial budget. That is, because 

maintaining resources at FY92 levels is in keeping with the state’s goal of providing a fair and 

equitable distribution of available resources, in part, because each adult education district within 

the state is guaranteed a base level of funding that, when combined with federal resources, is 

sufficient to allow a provider to open their doors to offer a minimum level of program services. 
SECTION A-3 

Formula Design Process 

— 58 —

At the time, program directors were willing to earmark 90 percent of state funding as a 

guaranteed base to protect small rural providers unable to generate large enrollments. As one 

committee member noted, although it would have been easy for larger agencies to support 

contact-hour allocations, that would ultimately drive small agencies out of existence or force 

them to cut services to adults in need. In keeping with the goal of developing a fair and equitable 

formula, larger providers were willing to concede resources to ensure that no community was left 

out of the service network. 

Specification of Funding Criteria: Federal Funding

In FY99, to introduce performance funding into the state’s adult education formula, Indiana state 

administrators formed an advisory committee of 12 program directors, selected by the state 

director to represent the views of the field. Committee members met periodically throughout 

1999 to discuss potential federal funding mechanisms and to assess the effect of different 

measures on provider eligibility. Although willing to integrate PBF into the federal formula, 

members were reluctant to make radical changes to the allocation mechanism, believing that an 

effective adult education program faced fixed costs (e.g., facilities, teacher salaries, and benefits) 

regardless of the number of learners served or outcomes achieved. 

Ultimately, the committee decided to allocate a majority of federal resources—roughly 85 

percent—as a base grant conditioned on an agency’s 1999–2000 program year budget. 

Allocating such a large percentage of federal funding as base funding minimizes the risk of 

decreased funding to providers. Since providers are assured they will receive 90 percent of their 

federal resources in the coming fiscal year, they can budget realistically to support most of their 

program services. 

Committee members also adopted two approaches to allocating performance resources. Roughly 

80 percent of performance funds are awarded based on the number of learners who achieved an 

outcome on one of the NRS core or secondary measures. The remaining 20 percent is distributed 

as incentive funding, based on how many of the 15 state-negotiated NRS performance goals a 

program achieves. To reward programs for their performance, committee members agreed to 

award a majority of performance funds based on learner outcomes, ensuring that larger agencies 

would be able to generate sufficient resources to compensate them for serving more students. 

Indiana includes performance awards for all of the NRS measures, including so-called “soft” 

measures, which include entering another academic or vocational program, becoming a U.S. 

citizen, registering to vote or voting for the first time, exiting from public assistance rolls, 

reading more to children, and becoming more involved in children’s schooling. These NRS 

secondary outcomes are included because committee members were concerned that not all 
SECTION A-3 

Formula Design Process 

— 59 —

programs had equal capacity to achieve core NRS measures; for example, those serving 

institutionalized individuals may have difficulty showing learning gains or finding students 

employment. Consequently, members favored including alternative measures to give all 

programs the opportunity to achieve performance outcomes. 

To calculate a program’s performance award, state administrators multiply the number of 

outcomes or performance goals a provider achieves by a fixed dollar amount established by the 

state. An example of this fixed amount would be a learner who achieved a performance outcome 

in FY01, such as completing an education level, generated $23.74 for their local provider. 

Similarly, a local provider that attained one of 15 state-negotiated performance goals in FY01, 

such as demonstrating education gains for at least 20 percent of its Beginning Literacy ABE 

learners, earned $630.25 for each measure. 

Each year performance and incentive rates are adjusted to account for the level of federal support 

available and the number of outcomes reported by local providers. For example, between FY01 

and FY02, the funding rate for a performance outcome for an adult learner fell from $23.74 to 

$16.51, and for a performance goal it fell from $630.25 to $603.22, as a result of declining 

federal resources and statewide increases in provider performance (table A-3). 

All performance outcomes and performance goals were initially equally weighted, meaning that 

a student who advanced a level in a core or secondary performance measure generated the same 

amount as one identified as entering employment. Moreover, no adjustment was made based on 

student ability. According to the assistant state director, 75 percent of the adults served by state 

programs function below the ninth-grade level or are ESL students, so no effort was made to 

weight outcomes differentially, because all providers were serving learners with high needs. This 

decision initially troubled some program directors, particularly those serving relatively large 

populations of limited-English-proficient adults or those with documented disabilities. 

Beginning in FY03, the state modified its funding formula to offer differentiated payments for 

adults achieving core or secondary NRS performance outcomes. This change was justified for 

several reasons. First, only the core NRS measures are tracked by the U.S. Department of 

 Fiscal Year Performance Outcome Rate Performance Goal Rate 

2001 $23.74 $630.25 

2002 $16.51 $603.22 

2003 $17.81 core/$8.90 secondary $657.89 

2004 $17.66 core/$8.83 secondary $644.69 

2005 $16.45 core/$8.22 secondary $635.59 

Table A-3—

Indiana Performance Outcome Rate and Performance Goal Rate: FY01 to FY05
SECTION A-3 

Formula Design Process 

— 60 —

You have to provide [program

directors] some direction and you

have to provide them some models to

look at and to consider. You have to

place these models on the table for

them to react ... when it comes down

to working with our local providers,

they want to know about dollars and

cents. 

—Indiana Assistant State Director 

of Adult Education 

Education for reports to Congress and by the Indiana Department of Education for the state’s 

adult education report card, meaning that these measures carry somewhat greater importance. 

Second, it is only through achieving the core outcome measures that a state may qualify for 

incentive funds through the Workforce Investment Act, suggesting that programs should focus on 

tracking and reporting core outcomes to increase the likelihood of the state getting its award. 

Finally, measurement and reporting of secondary outcomes was not consistent across programs, 

undercutting the accuracy of these measures. 

Assessing Formula Effects 

To determine the effect of different scenarios on local 

providers, state administrators modeled how provider 

allocations would change under different assumptions. As an 

initial exercise, administrators looked at models in which base 

funding was set at 60 percent, 65 percent, and 70 percent. 

Resource estimates were shared with committee members, 

who immediately focused on the amount of money their own 

and other programs would receive. According to the assistant 

state director, much of the debate focused upon how much 

risk program directors were willing to incur. 

Since few directors proved to be risk-takers, the final model placed greater emphasis on 

allocating resources based on prior-year funding than on enrollment or performance outcomes. 

As one committee member pointed out, the existing system was too large to change overnight; 

although members supported the use of PBF in the abstract, it was felt that securing buy-in 

would require a slower, evolutionary process. 
— 61 — 

Prior to performance-based funding,

the Indiana Department of Education

already had other criteria that they

were looking at, so it was not a big

switch in the sense of their setting

new criteria for us or new standards

for us to meet. The standards were

already there, and now they were

going to attach dollars to them. 

—Indiana Program Director 

SECTION A-4: 

Implementation Process 

Indiana fully implemented performance funding in the 2000–01 program year. According to the 

assistant state director, although the state did not introduce a hold-harmless provision in the 

formula, the fact that such a high percentage of funding was released as a guaranteed base 

protected providers. Moreover, at the time that PBF was first introduced, the state was 

experiencing a significant funding increase in its allocation of federal funds due to demographic 

changes, with resources rising from $7.3 million in FY00 to $8.6 million in FY01. 

To ensure further that providers were insulated from sudden funding shifts following formula 

adoption, state administrators set aside a portion of new federal resources for use as a 

competitive enhancement grant, for which program directors could apply to support a variety of 

activities. The combination of guaranteed base funding, increased federal resources, performance 

funding, and enhancement grants meant that most providers were exposed to very little risk of 

losing resources with the shift to performance funding. 

Provider Reporting Capacity 

Indiana was an early implementer of computerized record 

keeping, with local providers electronically tracking program 

outcomes since 1993. According to the assistant state director, 

the introduction of AEFLA reporting requirements simply 

validated this existing system—although the state had not 

been actively using all of the information compiled by 

providers, data for most NRS measures were already being 

collected. So the introduction of AEFLA did not change the 

mechanics of reporting, only the importance attached to the 

information that already existed. 

Because state administrators wanted programs to receive immediate feedback for their 

performance, the state structured the funding formula to provide current year resources based on 

prior year outcomes. This requires that local providers quickly process and report data, so that 

state staff have the information needed to calculate provider eligibility. Some program directors 

have complained that this has required that they invest additional time to perform the extensive 

data entry required. 
SECTION A-4 

Implementation Process 

— 62 —

Data Quality 

Each year, state administrators perform a desk audit of program-reported data and conduct onsite reviews of 20 percent of providers as required by law. Desk audits consist of a review of 

program paperwork and data submissions to ensure that program totals are consistent and 

reasonable given state administrators’ knowledge of program operations. Field visits are 

conducted using a standardized evaluation instrument to ensure reliable assessments across sites. 

Programs failing to meet their state-negotiated performance targets are placed on probation and 

offered technical assistance to improve their outcomes. Underperforming agencies must 

demonstrate gains within a state-established time period or face defunding. 

Concurrent with the introduction of PBF, state administrators increased the number of data 

checks for the desk audit system to ensure that program reports were internally consistent. For 

example, providers were required to demonstrate that they had a performance goal and 

associated outcome, such as program exit, to receive credit for their data submission. Program 

evaluators also placed greater weight during field visits on performing spot checks of a sample of 

student records to ensure the accuracy of local data. 

During interviews, program directors suggested that poor performance is not always indicative of 

inadequate programs, but sometimes only reflects a temporary deficiency in the quality of data 

reported to the state. In particular, directors described the problems associated with “losing data 

in their systems,” meaning that fundable outcomes are lost because learner records are not 

properly updated, invalidating the file. To illustrate, one provider described how his program 

went from being one of the best to one of the worst in the state within two years. In retrospect, he 

determined that the reason for this decline had nothing to do with his program; instead, his data 

entry person was found to be suffering from a brain tumor that interfered with her ability to 

maintain program data accurately. 

Management Information Systems 

In FY00, to support programs in collecting NRS data, the state supplied providers with 

computing software and technical assistance in its use. Providers already had an existing data 

collection system in place when Indiana initiated performance funding in FY01. Since PBF 

reporting piggybacked on existing federal NRS measures, neither the state nor local providers 

incurred any additional technology costs in shifting to performance funding. 

Although providers incurred no quantifiable hardware or software costs related to the adoption of 

PBF, program directors reported that they and their staff now spend considerable time collecting 

and monitoring local program data. This increased attention is a direct result of the state 

attaching funding to agency performance on the NRS measures. As one state administrator 
SECTION A-4 

Implementation Process 

— 63 —

noted, before PBF, program directors often gave only perfunctory attention to their data; linking 

money to the NRS measures motivated directors to pay greater attention to their outcomes. 

Providers also were forced to invest greater time entering data because the state instituted 

electronic data checks in the reporting system. According to the assistant state director, program 

staff were initially frustrated because they encountered data flags, indicating a problem, when 

they tried to report their data. Since error warnings did not, at the time, provide detailed 

information on what was wrong, agency staff had to invest time reviewing and cleaning data so 

that it could be uploaded to the state database. 

Technical Assistance 

State administrators reported that they did not explicitly provide technical assistance to improve 

the collection of performance funding data. Instead, state technical assistance focused on 

assisting providers in using approved procedures for assessing student progress using the NRS 

measures and entering observed outcomes into their local databases. Training was targeted to 

different audiences, including program directors, lead teachers, veteran teachers, and data entry 

staff. Individualized training and one-on-one consultations also were provided. 

State administrators did, however, conduct roundtable sessions at semiannual statewide meetings 

to outfit program directors with data analysis skills. For example, directors were given a set of 

performance outcomes for a fictitious district and taught how to disaggregate data to identify 

issues affecting a particular program, teacher, or learner population. Professional development 

facilitators from the state’s regional centers also met with program staff at annual state 

association conferences and individually to help providers learn how to interpret data.

— 65 — 

Every annual performance report for

every program is posted. That has a

very interesting effect at the local

level. ... To have that information

accessible 24 hours a day, 365, is

very powerful. Peer pressure to

improve that program is probably

more than anything that we could

have said or [the federal government]

could have ever done. 

—Indiana Assistant State Director 

of Adult Education 

SECTION A-5: 

Effects of Performance-based Funding 

The introduction of PBF in Indiana was intended to add a level of performance accountability to 

adult education programs. Although focus group participants ascribed both positive and negative 

outcomes to the adoption of performance funding, few, if any, data exist to substantiate these 

claims. In many cases, participant observations were based on general perceptions of program 

operations or extrapolations of a single experience. Some participants also had difficulty 

differentiating programmatic effects due to the adoption of PBF from those associated with the 

state’s implementation of AEFLA. Consequently, readers should be cautious in drawing 

conclusions about the effects of PBF on state adult education systems. 

Contributions to Statewide Performance 

During site-visit interviews, both state administrators and local provider staff suggested that the 

adoption of PBF has contributed to achieving state goals, and, in particular, to the following: 

• Improving the Accuracy of Adult Education Data—

Aligning state PBF measures with those in the NRS 

has been credited with improving data quality, 

because program directors now have a financial 

incentive to review the information they report. As 

one program director observed, although providers 

were required by federal law to collect program data, 

reporting requirements alone would not have 

motivated directors to monitor their program 

outcomes or train staff to collect accurate data. 

Associating funding with the NRS core and secondary measures has caused 

directors to institute formal data collection procedures to ensure that information 

accurately captures program performance. 

Because of the resources and publicity associated with their data, program 

directors reported that they are more proactive in reviewing the information they 

report to the Indiana Department of Education. This has helped to create a 

“culture of data” within the state; that is, program directors are more likely to 

understand and proactively use data to manage program operations. Perhaps the 

clearest example of directors’ commitment to performance came during field 
SECTION A-5 

Effects of Performance-based Funding 

— 66 —

interviews, when one director brandished a set of program performance measures 

that were less than two weeks old, pointing out his most recent program 

accomplishments and identifying specific areas in which he was planning to focus 

his attention in the coming weeks. 

• Insulating Programs From Year-to-Year Fluctuations—Since the state allocates 

only a relatively small amount of funding based on performance, programs are 

better positioned to weather unanticipated enrollment or performance downturns. 

For example, one program director reported that rather than have to close a single 

site due to one year of unexpectedly poor performance, she was able to make 

small cutbacks across multiple sites to maintain program offerings. This enabled 

her to serve the same number of students and, thus, retain her enrollment 

eligibility. 

There are advantages to small rural providers, because only a small portion of 

their funding is contingent on learner outcomes. During case study interviews, 

program directors reported that they budgeted for programs using their base 

eligibility; performance funds were reserved for supplemental student services, 

and textbooks and supplies that could be suspended if performance funding were 

to drop. 

Providers did not believe, however, that increasing the amount of PBF resources 

would help to improve performance. As one program director explained, “It 

doesn’t matter if you put out 5 percent or you put out 50 percent, these people are 

still going to be out there hustling and trying to do what they are going to do for 

the students.” In the opinion of directors interviewed for the study, although there 

is a clear benefit to attaching some money to provider performance, there is a 

diminishing return to attaching too great a sum because of the potential for 

creating hardships from which programs may have difficulty recovering. 

• Improving Teacher Effectiveness—Rewarding providers for achieving learner 

outcomes has led program staff to manage student enrollments and service 

delivery more actively. One program director compared the new system to 

working at a doctor’s office: All new students are initially assessed to diagnose 

their problems, and then services closely tailored to their needs are provided. 

Monitoring intake and processing procedures also have allowed instructors to 

make referrals to other agencies, where warranted, and to develop more realistic 

goals and services for those who are enrolled. For one program, performance 

funding was credited with halving the dropout rate, as instructors became more 

adept at targeting services to program participants. 
SECTION A-5 

Effects of Performance-based Funding 

— 67 —

I think our outcomes are coming

quicker. I think the students are

seeing their progress much faster,

which keeps them encouraged and

very positive. It keeps the teachers

more on task. It’s also easier to see a

weaker area that needs fixing, if

you’re really looking at your data

and your results. 

—Indiana Adult Education 

Instructor 

As one director commented, performance funding 

“made our teachers more professional because they 

had to track the attendance records. They had to do the 

adult learning plan. They had to make sure that the 

goals were realistic.” Simply put, staff at all provider 

levels—from program directors to classroom 

instructors—changed how they went about their jobs 

in response to performance funding, although the 

motivations may have differed across groups. For 

example, program directors often spoke of the need 

to meet state performance targets to obtain additional money to support program 

offerings, while program staff spoke of their desire to gain supplemental 

classroom materials or attend professional development conferences with the 

money they earned for their program. 

Drawbacks 

Interviews with state administrators and local program directors indicate that PBF has been 

effectively institutionalized in Indiana. However, not all providers initially supported the new 

funding system or currently support all aspects of the funding formula enthusiastically. 

Drawbacks associated with implementing PBF include: 

• Large Base Undercuts Effect of Performance Award—A majority of state 

resources are awarded as base funding, with any additional resources provided by 

the legislature distributed based on enrollment. Since the state has not had an 

increase in funding in eight years, the assistant state director believes that formula 

allocations are, “just reshuffling the pot”—that is, very little money actually 

changes hands year-to-year since most providers are able to maintain their level of 

expenditure from the initial 1992 program year funding. Moreover, if state 

enrollment were to grow substantially, providers would face a potential loss of 

resources even if they maintain current enrollment, since the dollar amount 

attached to an adult learner would decrease. 

Since performance eligibility is not calculated until well after the fiscal year has 

started, state administrators are somewhat constrained in the amount of funding 

that can be awarded based on performance. For instance, if the state were to 

allocate 50 percent of its federal funding based on performance, programs would 

be at risk of losing a substantial amount of funding midyear, after annual 

expenditures have already been adopted, which could force program directors to 

make drastic cuts to balance their budgets. 
SECTION A-5 

Effects of Performance-based Funding 

— 68 —

Although directors interviewed for this study reported that the existing 

performance allocation motivated them to achieve outcomes, in discussions they 

pointed out that underperforming programs also are able to remain in the provider 

system because they are guaranteed a substantial amount of base funding if they 

maintain enrollments. Since programs are assured of their survival, it was 

suggested that some directors might be less likely to focus on performance, 

particularly if they were busy with other obligations. 

• Secondary NRS Measures Lack Utility—Program directors did not universally 

approve of using both core and secondary NRS measures to award performance 

funding. As one director pointed out, adult education agencies are not 

postsecondary institutions charged with providing advanced academic or technical 

skills, but rather instructional programs focused on imparting basic literacy skills 

to prepare learners to earn a GED credential. Providers felt that some of the 

secondary NRS measures, such as registering to vote or reading more to children, 

were less important than those dealing with learner progress. 

Directors also complained that follow-up efforts to report on some NRS measures 

consume a great deal of staff time, often with little payoff. Since some adult 

learners are highly mobile—one provider described one participant who reported 

four different addresses in a single week—many providers, and particularly those 

with highly mobile populations, may be disadvantaged in the funding formula. 

Although participants acknowledge that follow-up data can produce information 

useful for program planning purposes, a majority felt that post-program 

performance indicators should not be a basis for allocating funding. 

Core and secondary NRS performance measures initially were all counted equally 

in Indiana, meaning that a provider received equal funding whether a student 

completed a GED level or registered to vote. In some cases, this led some 

providers to focus program improvement efforts on secondary measures that were 

less difficult to document or achieve. Beginning in FY04, the state has halved 

payments on NRS secondary measures, which has helped to address this issue 

• High-needs Populations Are Not Differentially Weighted—Providers raised 

concerns with the equal weighting attached to the core NRS measures in the state 

funding formula. Since local programs may have to invest greater resources to 

produce outcomes for high-needs populations, such as those with limited English 

proficiency or disabilities, some directors felt that they were penalized for their 

student demographics. Providers also may have less incentive to operate high-cost 

programs. For example, one program instructor described how her program 

dropped a specialized program for developmentally disabled adults because, in 
SECTION A-5 

Effects of Performance-based Funding 

— 69 —

her estimation, these individuals were unlikely to achieve outcomes recognized in 

the performance funding formula. This observation, however, was the exception; 

program directors were unanimous in reporting that they served all students who 

entered their facility, regardless of their skills or abilities. 

• The Monetary Value of a Performance Outcome Cannot Be Predicted—The value 

of a performance outcome has fluctuated over time, making it difficult for 

providers to project their performance-funding eligibility for the subsequent fiscal 

year. For example, state payments for a learner outcome have fallen from $23.74 

per adult learner in FY01 to $16.45 in FY05, due, in part, to a decline in federal 

performance funding and, in part, to increases in the performance outcomes 

agencies have achieved (40,877 outcomes in FY01 compared to 44,217 in FY05). 

The valuation of performance outcomes and goals are set retrospectively by the 

state each fiscal year, with rates determined by available funding. Since agencies 

do not know ahead of time what the valuation will be, providers only have a 

ballpark estimate of their likely funding eligibility. 

Similar issues apply in the allocation of funds using state resources. Since 

enrollment rates are based on the amount of state resources remaining after basic 

grants are awarded, the value of a learner enrollment is a function of the total 

number of statewide enrollments recorded in the fiscal year. Therefore, if state 

enrollments were to substantially increase in a given year, a provider would have 

to increase enrollments by the statewide average increase simply to maintain 

stable resources. This also presents a challenge to annual budgeting. 

Effect of PBF on Adult Education Providers 

The number of program providers has remained unchanged since the adoption of performancebased funding in FY02. Although program directors described closing or moving some service 

locations, these changes were attributed to demographic shifts in the adult population rather than 

to the performance-funding formula. 

Fiscal Eligibility 

A review of program funding data indicates that federal allocations used to support PBF fell 18 

percent between FY01 and FY05, declining from approximately $1.27 million in FY01 to $1.05 

million in FY05 (table A-4). Of the 43 comprehensive program districts qualifying for 

performance funding in FY05, roughly two-fifths (18 of 43 providers) lost funding relative to the 

statewide average. However, since FY02, federal PBF has remained relatively stable, in keeping 

with the state’s goal of maintaining the amount of resources allocated using performance criteria. 
SECTION A-5 

Effects of Performance-based Funding 

— 70 —

Although funding changes have been pronounced in some sites, when PBF is expressed as a 

percentage of total adult education resources, the actual dollar amounts have been relatively 

small for most providers. For example, the New Albany area realized a 33 percent decline 

($12,614) in performance funding in the five years since PBF was adopted; however, when 

expressed as a percentage of its total adult education funding in FY00—the year before the 

adoption of performance funding—the agency actually lost less than 6 percent of its budget (see 

table A-5). Moreover, enrollment increases associated with increased state funding resulted in 

the area actually increasing its overall funding eligibility for FY06 by 14 percent, negating any 

losses due to diminished performance. Of the 19 providers losing performance funding between 

FY01 and FY05, half experienced a gain in overall resources as compared to the statewide 

average. 

Generally, although PBF appears to have reallocated resources from lower- to higher-performing 

providers over time, its effect is tempered by the relatively small proportion of federal resources 

distributed based on performance. As illustrated in table A-4, the total amount of funding for 

adult education has risen by 10 percent between FY00 (the last year of contact-hour funding) and 

FY06 (the most current year for which data are available). These funding increases are due to 

changes in the state eligibility for federal resources, as state funding has remained unchanged 

over the last eight years. 
SECTION A-5 

Effects of Performance-based Funding 

— 71 —

Percentage

Change Dollar

FY01 Amount

District / Program FY01 FY02 FY03 FY04 FY05 to FY05 Change

Statewide Total $1,271,873 $1,060,974 $1,060,968 $1,060,893 $1,047,452 -18% -$224,421

Richmond area 28,126 16,954 17,714 21,025 8,689 -69% -19,437

Crawfordsville area 27,860 17,015 21,194 15,639 8,675 -69% -19,185

Gary area 35,616 21,456 21,800 21,811 12,567 -65% -23,049

Central Nine area 40,340 37,057 20,002 18,041 16,187 -60% -24,153

 North White area 20,549 8,526 8,958 6,173 9,343 -55% -11,206

Warren Twp. Area 45,461 27,874 13,069 20,186 20,878 -54% -24,583

Evansville area 62,464 24,752 29,899 29,996 30,028 -52% -32,436

Wayne Twp. Area 61,963 46,388 45,334 37,423 34,695 -44% -27,268

Portage area 75,268 58,905 66,276 60,072 46,098 -39% -29,170

Kokomo area 24,814 17,113 14,891 10,208 15,408 -38% -9,406

IPS area 130,618 133,865 132,009 136,239 85,494 -35% -45,124

Michigan City area 39,047 32,621 30,808 29,086 25,760 -34% -13,287

New Albany area 38,325 32,069 12,906 17,466 25,711 -33% -12,614

Putnam Co. area 11,108 8,178 8,158 8,725 7,773 -30% -3,335

Vigo Co. area 31,732 9,704 15,417 12,027 22,307 -30% -9,425

Lafayette area 79,472 65,718 62,519 51,630 56,854 -28% -22,618

Blue River area 18,719 14,025 12,032 8,892 13,721 -27% -4,998

Columbus area 40,416 29,081 27,441 32,107 30,985 -23% -9,431

North Adams area 6,593 6,435 2,847 2,384 5,161 -22% -1,432

Muncie area 32,712 26,412 25,711 28,230 27,528 -16% -5,184

South Bend area 81,965 66,473 69,538 68,355 69,262 -15% -12,703

Four Co. area 27,000 23,934 23,682 23,939 22,982 -15% -4,018

Connersville area 17,978 15,321 9,714 15,365 15,346 -15% -2,632

Logansport area 14,407 9,738 12,466 10,164 12,791 -11% -1,616

Heartland area 8,104 6,708 5,710 8,001 7,504 -7% -600

Elwood area 11,677 10,466 11,692 11,348 11,032 -6% -645

Monroe Co. area 29,985 28,270 28,381 26,720 28,385 -5% -1,600

Fort Wayne area 46,796 32,622 48,227 46,632 45,783 -2% -1,013

Elkhart area 35,817 28,968 26,750 26,853 37,567 5% 1,750

New Castle area 21,046 19,054 24,786 20,045 22,381 6% 1,335

Hammond area 31,974 24,058 18,769 23,338 34,005 6% 2,031

Vincennes area 23,812 23,502 27,051 19,258 29,017 22% 5,205

Washington Twp. area 21,860 30,616 33,330 33,317 27,683 27% 5,823

Warsaw area 9,802 8,872 10,294 6,190 13,278 35% 3,476

South Vermillion area 5,203 3,452 4,776 4,812 7,274 40% 2,071

Southeastern area 10,499 5,839 8,410 15,912 16,457 57% 5,958

DOC 26,513 42,850 17,398 36,575 45,355 71% 18,842

North Spencer area 6,535 9,021 7,037 7,532 13,148 101% 6,613

Anderson area 8,761 5,178 14,178 21,502 18,296 109% 9,535

Plymouth area 3,302 6,254 6,280 5,369 7,451 126% 4,149

12 Benton Co. area 1,105 66 5,087 4,044 2,895 162% 1,790

Marion area 4,652 12,290 14,870 20,090 16,769 260% 12,117

East Chicago area 0 13,272 14,670 9,096 9,594 — 9,594

Lake Ridge area 0 0 3,312 3,320 1,934 — —

Goshen area 0 0 4,755 6,834 6,550 — —

Huntington area 0 0 1,104 1,086 2,057 — —

Frankfort area 0 0 3,953 4,724 4,048 — —

Jay County area 0 0 2,867 2,190 3,909 — —

Edinburgh area 0 0 12,893 10,922 10,837 — —

Mt. Vernon area 0 0 0 0 0 — —

SOURCE: Indiana Department of Education, 2006. Funding data provided by state.

Note: — Not applicable.

a Performance funding includes federal resources distributed based on outcomes achieved, while incentive awards are based on outcomes

achieved by providers.

Total Combined Performance and Incentive Award

Table A-4—

Indiana Combined Adult Education Performance and Incentive Awards,a

 Percentage and Dollar Amount 

Change by Provider: FY01 to FY05
SECTION A-5 

Effects of Performance-based Funding 

— 72 —

Percentage Change

District / Program FY00 FY06 FY00 to FY06

Statewide Total $17,522,107 $19,345,212 10%

Richmond area 175,537 175,590 0%

Michigan City area 237,703 238,846 0%

Columbus area 230,932 232,371 1%

Warsaw area 122,551 123,389 1%

Marion area 103,159 104,017 1%

Connersville area 112,986 113,945 1%

DOC 9,711,734 9,852,319 1%

East Chicago area 131,526 133,459 1%

Vigo Co. area 159,135 162,085 2%

Blue River area 73,741 77,225 5%

New Castle area 223,461 234,025 5%

Gary area 901,079 945,571 5%

Southeastern area 108,933 116,479 7%

Wayne Twp. Area 325,576 349,425 7%

Heartland area 35,741 38,534 8%

Evansville area 193,563 211,589 9%

Vincennes area 512,283 560,666 9%

Central Nine area 149,553 165,539 11%

IPS area 954,491 1,062,548 11%

North Adams area 34,937 39,181 12%

Logansport area 161,459 181,570 12%

Portage area 429,182 482,972 13%

South Vermillion area 32,161 36,218 13%

Anderson area 145,893 164,496 13%

Benton Co. area 37,821 43,059 14%

New Albany area 220,557 251,610 14%

Muncie area 188,977 219,371 16%

South Bend area 299,533 348,474 16%

Fort Wayne area 273,938 319,810 17%

Warren Twp. Area 150,660 179,836 19%

Lafayette area 244,669 328,329 34%

Elwood area 45,975 64,189 40%

Hammond area 187,350 268,402 43%

Monroe Co. area 133,450 197,691 48%

Elkhart area 199,976 301,753 51%

Four Co. area 87,262 132,819 52%

North White area 37,734 57,478 52%

Putnam Co. area 31,551 55,241 75%

Crawfordsville area 43,671 77,328 77%

Washington Twp. Area 71,667 151,211 111%

Plymouth area 0 46,330 —

Kokomo area 0 128,401 —

North Spencer area 0 371,820 —

SOURCE: Indiana Department of Education, 2006. Funding data provided by state.

Note: — Not applicable.

a Comprehensive program districts include providers competing for performance funding grants.

Total Funding

Table A-5—

Total Change in Indiana Adult Education Funding, by Comprehensive Program District:a

 FY00 to FY06
SECTION A-5 

Effects of Performance-based Funding 

— 73 —

Program directors report that they typically budget based on the 90 percent of funding that they 

are assured of receiving, using performance and incentive money to provide supplemental 

services, such as aides and professional development. Since performance-funding eligibility is 

not determined until late in the budgeting process, providers typically budget on the assumption 

that they will get approximately the same amount as in the preceding year and later make 

changes at the margins to align their budgets with final state allocations. 

Program Outcomes 

Although program enrollments have fluctuated over time, Indiana adult education providers have 

increased the number of performance outcomes achieved on the NRS measures since the 

adoption of performance funding in FY01. As illustrated in table A-6, the number of participants 

achieving a state-identified core or secondary performance outcome on the NRS measures 

climbed from 45,290 in FY01 to 56,043 in FY05, nearly a 24 percent increase compared to a 

participant population increase of roughly 3 percent (42,135 participants in FY01 compared to 

43,498 in FY05). 

Of particular interest is that since FY03, when the state halved its reimbursement rate for the 

secondary NRS measures, the number of secondary outcomes has fallen from 14,314 in FY03 to 

11,826 in FY05. At the same time, the number of core measures achieved climbed from 39,787 

to 44,217. This indicates that providers may be shifting away from pursuing lower value, though 

potentially more easily obtainable, secondary measures in favor of NRS core measures. Detailed 

provider data are available in table A-7. 

FY01 FY02 FY03 FY04 FY05

Participants 42,135 44,492 41,397 41,148 43,498

Providers Qualifying for Incentives 43 43 43 43 43

Performance Outcomes 45,290 50,620 54,101 54,370 56,043

Core NRS Measures — — 39,787 40,296 44,217

Secondary NRS Measures — — 14,314 14,074 11,826

Outcomes per Participant 1.1 1.1 1.3 1.3 1.3

Performance Goals Achieveda 357 373 342 349 350

Average Goals Achieved 8.3 8.7 8.0 8.1 8.1

SOURCE: Indiana Department of Education, 2006. Funding data provided by state.

Note: — Not applicable.

a

 Negotiated performance targets achieved by the local provider.

Table A-6—

Indiana Statewide Adult Education Participants, Providers, and Performance Outcomes and Goals: FY01 to 

FY05
SECTION A-5 

Effects of Performance-based Funding 

— 74 —

Percentage Change

District / Program FY01 FY02 FY03 FY04 FY05 FY01 to FY05

Statewide Total 45,290 50,620 54,101 54,370 56,043 23.7%

Gary area 1,235 1,007 1,334 1,052 667 -46.0%

Lake Ridge area 189 188 119 — 

Hammond area 1,055 1,055 1,491 1,804 2,385 126.1%

East Chicago area 0 621 872 487 775 — 

Portage area 2,879 3,092 3,419 3,125 2,496 -13.3%

Michigan City area 1,353 1,610 1,678 1,744 1,477 9.2%

South Bend area 3,347 3,879 4,411 4,092 4,274 27.7%

Elkhart area 1,270 1,608 1,589 1,977 3,075 142.1%

Goshen area 332 387 398 — 

Plymouth area 86 123 132 128 244 183.7%

Warsaw area 174 245 388 286 487 179.9%

Four Co. area 872 1,084 1,076 1,059 1,158 32.8%

Fort Wayne area 1,706 1,537 2,489 2,454 2,700 58.3%

Huntington area 62 67 126 — 

Benton Co. area 20 4 64 47 68 240.0%

North White area 494 151 209 131 196 -60.3%

Logansport area 368 334 482 320 498 35.3%

Heartland area 129 114 99 161 147 14.0%

North Adams area 145 134 93 62 147 1.4%

Lafayette area 3,003 3,468 3,326 2,740 3,145 4.7%

Frankfort area 246 287 281 — 

Kokomo area 833 744 693 426 584 -29.9%

Elwood area 253 305 394 371 421 66.4%

Marion area 196 452 543 752 710 262.2%

Anderson area 316 277 577 792 745 135.8%

Muncie area 1,033 1,161 1,046 1,184 1,330 28.8%

Jay County area 161 124 259 — 

Crawfordsville area 802 592 739 583 334 -58.4%

Putnam Co. area 229 203 200 202 203 — 

Wayne Twp. Area 2,345 2,261 2,479 2,224 2,400 2.3%

Washington Twp. Area 682 1,379 1,372 1,477 1,604 135.2%

Warren Twp. Area 1,517 1,213 672 827 794 -47.7%

IPS area 5,261 7,704 8,867 9,261 4,886 -7.1%

Central Nine area 1,434 1,696 757 779 863 -39.8%

Edinburgh area 727 629 667 — 

Blue River area 470 484 485 414 633 34.7%

Connersville area 545 599 243 624 674 23.7%

Richmond area 946 844 810 862 528 -44.2%

New Castle area 807 825 999 815 1,058 31.1%

Vigo Co. area 1,151 478 713 426 953 -17.2%

Monroe Co. area 865 1,310 1,415 1,568 1,662 92.1%

Columbus area 1,331 1,213 1,302 1,491 1,611 21.0%

Vincennes area 897 1,277 1,321 978 1,513 68.7%

Southeastern area 283 244 328 683 780 175.6%

Evansville area 2,313 1,097 1,459 1,533 1,675 -27.6%

Mt. Vernon area

New Albany area 1,296 1,467 548 674 1,009 -22.1%

South Vermillion area 113 136 91 90 133 17.7%

DOC 1,117 2,412 977 1,706 2,756 146.7%

North Spencer area 116 181 202 277 395 240.5%

SOURCE: Indiana Department of Education, 2006. Funding data provided by state.

Note: — Not applicable.

a Comprehensive program districts include providers competing for performance funding grants.

Performance Outcomes

Table A-7—

Total Number of Indiana Adult Education Performance Outcomes Achieved and Percentage Change, by 

Comprehensive Program District:a FY01 to FY05
SECTION A-5 

Effects of Performance-based Funding 

— 75 —

This funding formula, while it appears 

on the surface to be fairly constant over

the last five years ... is not constant. We 

have to continue to look at how we fund

programs. ... We have to continue to 

look at performance, and we have to 

wrestle with the question of, “What is 

an effective program and how do we 

deal with it?” 

—Indiana Assistant State Director of 

Adult Education 

Provider attainment of negotiated performance targets on the NRS core measures has remained 

relatively stable since the adoption of performance funding, with programs achieving an average 

of 8.1 of 15 goals in FY05. Since the number of performance outcomes achieved by providers 

has increased over the same period, it may be that providers are succeeding in achieving 

performance outcomes in program areas in which they are already meeting their performance 

goals. Detailed data disaggregated by provider are available in table A-8. 

A review of statewide performance targets and actual performance levels indicates that the state 

has consistently achieved its NRS performance targets since FY01, for example, achieving 14 of 

its 15 targets in FY03. Given that providers, on average, are attaining only 8 of 15 measures, it is 

likely that state success can be attributed to larger providers or those enrolling a disproportionate 

number of students in a given program area. Low rates of achievement on performance goals 

observed in some providers may be due to a lack of program offerings corresponding to some 

measures. 

While it is not possible to attribute statewide success directly to the adoption of PBF, state 

administrators reported that the adoption of PBF has supported the state in improving program 

services, and data show that the state has made substantial gains in the number of learners 

achieving outcomes on the core NRS measures. 

System Evolution 

In FY03, in response to provider concerns about the 

equivalent weight attached to both core and secondary NRS 

outcome measures, the state modified its funding formula to 

halve the weight of secondary outcomes reported in the 

2002–03 program year. This change was intended to align 

state performance funding more closely with the operation of 

the NRS system, given that only core outcome measures are 

tracked by the U.S. Department of Education for federal 

reporting purposes or to award federal incentive funds 

through the Workforce Investment Act. Perhaps as importantly, state administrators recognized 

that the measurement and reporting of secondary outcomes had not been as consistent or 

definitive across programs as those associated with the NRS core measures. 
SECTION A-5 

Effects of Performance-based Funding 

— 76 —

District / Program FY01 FY02 FY03 FY04 FY05

Statewide Total 357 373 342 349 350

Average Goals Achieved 8.3 8.7 8.0 8.1 8.1

Gary area 10 8 6 9 5

Lake Ridge area

Hammond area 11 11 0 3 11

East Chicago area 9 5 4 4 0

Portage area 11 13 14 14 13

Michigan City area 11 10 8 4 6

South Bend area 4 4 10 10 14

Elkhart area 9 4 6 3 2

Goshen area

Plymouth area 2 7 6 5 6

Warsaw area 9 8 8 4 11

Four Co. area 10 10 11 13 12

Fort Wayne area 10 12 14 12 8

Huntington area

Benton Co. area 1 0 6 5 3

North White area 14 10 8 6 10

Logansport area 9 7 8 7 10

Heartland area 8 8 6 8 8

North Adams area 5 7 2 2 5

Lafayette area 13 14 13 10 14

Frankfort area

Kokomo area 8 8 5 5 10

Elwood area 99887

Marion area 0 8 8 11 8

Anderson area 2 1 6 12 10

Muncie area 13 12 12 12 10

Jay County area

Crawfordsville area 14 12 15 11 5

Putnam Co. area 0 8 7 8 7

Wayne Twp. Area 10 15 14 8 6

Washington Twp. Area 9 13 15 14 5

Warren Twp. Area 15 13 2 9 13

IPS area 9 11 9 10 8

Central Nine area 10 15 14 12 9

Edinburgh area

Blue River area 12 10 6 3 6

Connersville area 89988

Richmond area 95590

New Castle area 3 9 11 9 8

Vigo Co. area 7 3 5 8 12

Monroe Co. area 15 11 14 9 11

Columbus area 14 15 10 14 11

Vincennes area 4 4 8 5 10

Southeastern area 6 3 4 9 8

Evansville area 12 11975

Mt. Vernon area

New Albany area 12 13 5 9 15

South Vermillion area 42558

DOC 0 5 0 10 0

North Spencer area 6 10 6 5 12

SOURCE: Indiana Department of Education, 2006. Funding data provided by state.

a Negotiated performance targets achieved by the local provider.

b Comprehensive program districts include providers competing for performance funding grants.

Measures

Table A-8—

Total Number of Indiana Adult Education Performance Goalsa Achieved, by Comprehensive Program 

District:b

 FY01 to FY05
— 77 — 

Appendix A Summary 

In FY04, Indiana allocated just 5 percent of its combined federal and state adult education 

resources based on provider performance. This low commitment to PBF can be traced, in part, to 

providers’ hesitancy to make radical changes to the existing state allocation mechanism. 

Believing that an effective adult education program faced fixed costs (e.g., facilities, teacher 

salaries and benefits) regardless of the number of learners served or outcomes achieved, program 

providers participating in the state director’s PBF advisory committee recommended limiting the 

amount of funding distributed based on performance criteria. 

Restricting the use of PBF is a logical decision given the manner in which the state funding 

system operates. Performance funding in Indiana lags one year, meaning that providers are 

funded for outcomes they achieved in the preceding academic year. Since final data on program 

performance are not available until well after the program year has begun, providers typically 

budget on the assumption that they will get approximately the same amount as in the preceding 

year and later make changes at the margins to align their budgets with final state allocations. 

The decision to implement performance funding in Indiana was not associated with a turnover in 

state leadership, but rather came as a calculated response to new federal performance reporting 

provisions contained in the AEFLA. Believing that the new legislation called for holding local 

providers accountable for their learners’ outcomes, state leaders opted to introduce PBF into their 

federal funds allocation formula as a way to improve their federal reporting capacity. As noted in 

other case study states, the decision to adopt PBF was based almost entirely on programmatic 

decisions; state administrators reported no political or other external pressures to adopt 

performance funding. 

To allocate federal resources using PBF, the state director and her staff convened an advisory 

committee of 12 program directors to represent the views of the field. These members 

collaborated with state administrators to create a state allocation formula. In retrospect, state 

administrators and local program directors did not recall any significant opposition to the 

planned adoption of performance funding, although not all directors were enthusiastic about its 

introduction. Lack of resistance was attributed to state administrators’ efforts to communicate the 

need for formula change to the field and a positive relationship between state administrators and 

program directors that helped dilute local opposition to the plan. 
Appendix A Summary 

— 78 —

To allocate resources, Indiana uses a fairly complex formula that includes three components. (1) 

All state and approximately 85 percent of federal funds are distributed as a base grant, with 

providers automatically qualifying for 90 percent of their budgets in prior program years (FY92 

for state and FY00 for federal resources). (2) Remaining federal resources are distributed based 

on provider performance on the core and secondary measures contained within the NRS, with the 

amount awarded for a core NRS outcome valued at twice the amount awarded for a secondary 

NRS measure. (3) Incentive grants also are provided for programs meeting their state-negotiated 

performance levels for 15 measures of educational gains included in the NRS core measures. 

Indiana fully implemented PBF in the 2000–01 program year. Although the state did not 

introduce a hold-harmless provision in the formula, the fact that such a high percentage of 

funding was released as a guaranteed base protected providers. Moreover, at the time that PBF 

was first introduced, the state was experiencing a significant increase in federal funds; to protect 

providers, state administrators set aside a portion of these new federal resources for use as a 

competitive enhancement grant, for which program directors could apply to support a variety of 

activities. The combination of guaranteed base funding, increased federal resources, performance 

funding, and enhancement grants meant that most providers were exposed to very little risk of 

losing resources with the shift to performance funding. 

Although case study participants ascribed both positive and negative outcomes to the adoption of 

performance funding, few, if any, data exist to substantiate these claims. Participant observations 

suggest that PBF improved the accuracy of adult education data, because financial incentives 

motivated program directors to review information more closely before submitting it to the state. 

Performance funding also was credited with helping to improve the effectiveness of instructional 

services. Moreover, the relatively small amount of funding distributed based on performance 

helped to insulate programs from year-to-year fluctuations in resources, since so little funding 

was at risk. 

Because state performance measures parallel those contained in the NRS, neither the state nor 

local providers incurred additional costs in collecting performance data. The number of program 

providers also has remained unchanged since the adoption of performance-based funding in 

FY02. Though program directors described closing or moving some service locations, these 

changes were attributed to demographic shifts in the adult population rather than to the PBF 

formula. 

Although directors interviewed for this study reported that PBF motivated them to achieve 

outcomes, they pointed out that underperforming programs also were able to remain in the 

provider system because they were guaranteed a substantial amount of base funding if they 

maintained enrollments. Providers also reported dissatisfaction with using the secondary NRS 
Appendix A Summary 

— 79 —

measures to award performance funding, in part because follow-up efforts to report on some 

NRS measures consume a great deal of staff time. Because the value of a performance outcome 

fluctuates depending upon available resources and the number of outcomes reported by 

providers, program directors do not know ahead of time the value of a positive outcome, making 

it difficult to project their resource eligibility in future years. 

A review of statewide performance goals and actual performance levels indicates that the state 

has consistently achieved its NRS performance targets since FY01, for example, achieving 14 of 

its 15 targets in FY03. While it is not possible to attribute statewide success directly to the 

adoption of PBF, state administrators believe that PBF has supported the state in improving 

program services, and data show that the state has made substantial gains in the number of 

learners achieving outcomes on the core NRS measures. 

— 81 — 

APPENDIX B 

Kansas Case Study Report: 

Performance-based Funding In 

Adult Education 

— 83 — 

SECTION B-1: 

Overview of the State Adult Education Funding 

System 

In FY00, the Kansas Board of Regents (KBOR) adopted a new formula for allocating its federal 

and state adult education resources among eligible service providers. This change was driven in 

part by state bureaucratic reorganization, and in part by the introduction of the federal Adult 

Education and Family Literacy Act (AEFLA). Under the current system, local providers are 

guaranteed a base level of funding, drawn from state resources, which is equally distributed 

across all providers. Providers compete for federal and remaining state resources allocated based 

on provider performance. Performance is assessed by the state by measuring specific effects of 

programs, including learner outcomes and other administrative and programmatic outcomes. 

Allocations to local providers are distributed using the following criteria. 

Base Funding: Each fiscal year, 50 percent of state adult education resources are evenly divided 

as base funding among all eligible programs in the state. 

Performance Funding: Performance awards are distributed based on provider performance 

during the fiscal year preceding the current one, meaning that funds lag one year behind recorded 

outcomes. Resources are distributed based on providers’ performance using the following 

distribution formula: 

• Learner Outcomes—Fifty percent of federal funds are allocated based on the 

number of successful learner outcomes on selected core and secondary measures 

contained within the National Reporting System for Adult Education (NRS), the 

accountability system created to monitor learner outcomes as specified in AEFLA. 

States are required to report on the core measures in the NRS, which include 

outcome, descriptive, and participation measures, and they have the option of 

reporting on the secondary measures, which include outcomes related to 

employment, family, and community. Resources are distributed based on 

providers’ performance on the following student indicators: 

○ Completed one of the functional academic performance levels identified for 

adult basic education (ABE), adult secondary education (ASE), and English 

as a second language (ESL) learners as specified in the NRS; 
SECTION B-1 

Overview of the State Adult Education Funding System 

— 84 — 

○ Entered employment; 

○ Retained employment; 

○ Received GED; 

○ Entered postsecondary education or training; 

○ Acquired U.S. citizenship skills; 

○ Increased involvement in children’s education; and 

○ Increased involvement in children’s literacy activities. 

Educational gain outcomes are doubled for learners in the five lowest educational functioning 

levels of the NRS (two lowest ABE levels and three lowest ESL levels). 

• Program Quality—Fifty percent of federal and 50 percent of state funds are 

allocated based on the number of quality points a program receives relative to the 

statewide total of quality points generated that year. Quality points are awarded 

based on provider performance on 25 measures encompassing 10 quality 

indicators that are believed to be associated with program success. A complete 

listing of the Kansas Indicators of a Quality Adult Education Program can be 

downloaded from the Internet at: 

http://www.kansasregents.org/download/adultEd/IndicatorsFY05.pdf. Points are 

awarded based on evidence presented during a state monitoring visit, data in 

programs’ end-of-year reports, data derived from follow-up surveys, or records 

maintained by KBOR staff. 

To illustrate, a program accounting for 5 percent of statewide learner outcomes in FY04 

would qualify for 5 percent of the 50 percent of federal resources allocated for this factor 

in FY05. Similarly, a provider accounting for 3 percent of the statewide quality points 

awarded in FY04 would qualify for 3 percent of the combined 50 percent of federal and 

50 percent of state resources allocated for this factor in FY05. 

State Allocations 

In FY05, the state allocated to local providers a total of $4,458,990 in federal and state funding 

for the provision of adult education services. Of this amount, $3,410,002 (76.5 percent) came 

from federal and $1,048,988 (23.5 percent) from state sources (table B-1). 
SECTION B-1 

Overview of the State Adult Education Funding System 

— 85 — 

Of the $4,458,990 allocated to local providers, KBOR administrators earmarked $524,499—half 

of its FY05 state adult education funding—for distribution as base funding among the 31 local 

providers approved to offer instructional services. This accounted for roughly 12 percent of total 

adult education resources distributed to local providers in the state (fig. B-1). Each provider in 

FY05 qualified for $16,919 as a base grant, regardless of program size or type of instructional 

services offered. 

The remaining $3,934,501 in state and federal funds—accounting for 88 percent of statewide 

resources for local providers—was released as performance funding. This included $524,499 (50 

percent) of state and $3,410,002 (100 percent) of federal resources. 

 Total Federal State 

 Resources Resources Resources

Local Provider Allocation $4,458,990 $3,410,002 $1,048,988

Percentage of Total 100.0% 100.0% 100.0%

Base Funding $524,499 $0 $524,499

Percentage of Total 11.8% 0.0% 50.0%

Performance Funding $3,934,501 $3,410,002 $524,499

Percentage of Total 88.2% 100.0% 50.0%

SOURCE: Kansas Board of Regents, 2006. Funding data provided by state.

Table B-1—

Kansas Adult Education Federal and State Funding Allocated as Base or Performance Funding: FY05

SOURCE: Kansas Board of Regents, 2006. Funding data provided by state.

Figure B-1—

Percentage of Kansas Adult Education Funding Allocated by Funding Type: FY06

12%

88%

Base Funding

Performance Funding
SECTION B-1 

Overview of the State Adult Education Funding System 

— 86 — 

Organization of State Adult Education Services 

Adult education programs are directly administered by the KBOR’s office of adult education, 

which has four staff, two of whom directly administer the state’s performance-based funding 

(PBF) system. Individuals interviewed for this report include: 

• State Director of Adult Education—Appointed as acting director in August 2000 

and state director in August 2001, the state director provides statewide oversight 

of adult education services. This includes offering policy guidance and technical 

assistance to service providers, reviewing state and provider data for accuracy and 

to allocate resources, and compiling federal and state accountability reports. 

• Associate Director of Adult Education for Professional Development—Hired in 

October 2005, the associate director oversees the professional development of 

faculty and staff from local programs. These duties include reviewing provider 

performance data and program improvement plans and providing technical 

assistance to underperforming providers. 

Other state office staff includes a state administrator for Kansas GED testing and a senior 

administrative assistant. The state also subcontracts with a team of independent consultants to 

conduct annual monitoring visits to audit local provider data and assign program quality points 

used to award performance funding. The site-visit team consists of a former local program 

director and her husband, both of whom were interviewed for this report. 

Local Providers 

Program services were offered in FY04 in 31 local agencies, including 17 community colleges, 

11 technical colleges and unified school districts, two community-based organizations, and one 

private, faith-based university (table B-2). Providers were geographically distributed throughout 

the state, with services offered at more than 80 locations. Within sites, adult participants have 

access to a range of services, including ABE, ASE, and ESL instruction. 

Although the number of adult education participants has steadily declined since FY99, state 

administrators do not believe that the adoption of performance funding was related to this 

decline. According to the state director, enrollment declines reflect providers’ efforts to retain 

learners until they reach their goals or are better prepared to reach their goals, rather than simply 

enrolling whomever walks in their door. 
SECTION B-1 

Overview of the State Adult Education Funding System 

— 87 — 

According to the most recent state data available, in FY05, adult education providers enrolled 

9,475 individuals, with roughly two-fifths (41 percent) of participants between the ages of 25 to 

44 years (fig. B-2). Adult participants tended to be Hispanic (42 percent) or white (38 percent), 

with black (10 percent), Asians (7 percent), and American Indians or Alaskan Natives (3 percent) 

accounting for the remainder of program participants (table B-2). 

FY99 FY00 FY01 FY02 FY03 FY04

Providers 35 34 35 33 31 31

Community Colleges 19 19 19 18 17 17

4-year Private College 1 1 1 1 1 1

School Districts/Technical Schools 11 11 12 11 11 11

Community-based Organizations 4 3 3 3 2 2

Participants 16,461 11,410 11,248 10,725 10,386 9,788

 Gender

Male NA 45% 46% 46% 44% 44%

Female NA 55% 54% 54% 56% 56%

 Race-Ethnicity

White NA 38% 39% 39% 38% 39%

Black NA 14% 13% 13% 11% 10%

Hispanic NA 37% 38% 38% 40% 41%

Asiana NA 9% 8% 8% 9% 8%

American Indian/Alaskan Native NA 2% 2% 2% 2% 2%

SOURCE: U.S. Department of Education, Report to Congress on State Performance, program years 1999–2000 to program years 

2003–04; and data provided by the Kansas Board of Regents.

Note: NA—Not available.

a In FY00, Pacific Islanders were included with Asians. In subsequent years they were reported in a separate category.

Table B-2—

Selected Characteristics of Kansas Adult Education Providers and Participants: FY99 to FY04

SOURCE: U.S. Department of Education, Report to Congress on State Performance , program years 1999–2000 to program years 

2003–04; and data provided by the Kansas Board of Regents.

Figure B-2—

Ages of Kansas Adult Education Participants: FY05 

20%

29%

41%

8% 2%

16-18

19-24

25-44

45-59

60+

— 89 — 

SECTION B-2: 

Rationale for System Development 

State bureaucratic reorganization, coupled with the introduction of the federal AEFLA, helped 

catalyze state efforts to introduce performance funding into adult education. Change began with 

the adoption of the Kansas Higher Education Coordination Act of 1999 (SB 345), which shifted 

state oversight of community colleges, technical schools and colleges, and adult education 

services from the Kansas State Department of Education to the KBOR. 

This organizational restructuring coincided with state efforts to develop a four-year state plan for 

adult education, as mandated by AEFLA. Under AEFLA, states were required to report on a 

consistent set of program outcome measures, which were used to hold local providers 

accountable for service delivery. The state administrator, at the time, who is the current state 

director of adult education, was given primary responsibility for devising the new state plan. 

Contextual Factors Influencing Formula Adoption 

Unlike in the case of the higher education sector, where state legislators initially pressed for the 

introduction of PBF, the impetus for performance funding within Kansas came almost entirely 

from within the KBOR and, in particular, through the vision and leadership of the current state 

director. Believing that the existing state funding system was failing to deliver resources in a fair 

and efficient manner, state administrators used the new federal reporting requirements as a 

springboard for introducing PBF into the state adult education funding formula. 

Recognizing that the new federal reporting system could impose a substantial data collection 

burden at the local level, the state director surmised that the state would need some sort of 

leverage to motivate providers to collect accurate program data and to use this information for 

program improvement. Believing that performance funding could fulfill these purposes, state 

administrators set out to link provider funding to the core measures contained in AEFLA. Fearing 

that basing funding on outcomes alone would disadvantage small and rural providers, which 

would be unable to generate a sufficient number of outcomes to offset their fixed program costs, 

the formula drafting committee also decided to incorporate a set of process indicators into the 

formula to allocate resources based on program quality. 

These process indicators, aligned with state adult education goals, were intended to assess 

whether local providers were employing state-identified practices and activities associated with 
SECTION B-2 

Rationale for System Development 

— 90 — 

high-quality programs. Constructed to permit all providers to compete for resources irrespective 

of their size, it was anticipated that these measures would help to level the playing field.9 It was 

also felt that incorporating these measures would drive providers to develop the infrastructure 

and supports critical to program success. 

Upon obtaining approval to introduce PBF into the new adult education funding formula, state 

administrators reached out to program directors to build support for the formula change. To 

secure buy-in, providers’ annual eligibilities were publicized—calculated using learner-contact 

hours—when allocating resources in FY99. Before this, program allocations were not routinely 

shared among providers. During case study interviews, program directors reported that though 

they were generally aware of the performance of their rivals, they did not know the actual dollar 

amounts that others received. Publicizing funding permitted local directors to compare their 

allocations to those of their neighbors, helping to raise awareness of funding inequities and to 

demystify the state funding process. 

State Adoption Process 

Before the adoption of PBF in FY00, adult education resources were allocated based on learnercontact hours reported by local providers. Although program directors reported that they 

preferred this system to its predecessor of the 1980s—in which the state director set program 

eligibility at his own discretion—during interviews, program directors questioned the fairness of 

this approach, believing that some providers were inflating their contact hours to qualify for 

additional funding. During interviews, the state director also reported that some providers were 

using inappropriate methods to quantify learner contact hours, such as counting homework hours 

or student time spent in GED testing, toward their contact totals.

Program directors also suggested that the contact-hour funding system was not in the best 

interest of their learners. Since contact hours were the sole criterion used to award funding, 

providers believed that staff often had a greater incentive to fill seats than to manage 

enrollments, with little attention paid to achieving positive learner outcomes. As one program 

director observed, “Under the previous formula ... directors were much more concerned with 

getting hours, simply because they had to get the funds ... and so their attention wasn’t 

necessarily focused where you want it focused, on actual student achievement.” 

 

9 For example, programs were awarded points for serving participants at the lowest levels of education attainment, with 8 points

awarded for serving 35 percent or more of participants at one of the five beginning levels (Levels 1=ABE Beginning Literacy; 

2=ABE Beginning Basic; 7=ESL Beginning Literacy; 8=ESL Beginning; or 9=ESL Intermediate Low); 4 points for serving 25–

34 percent of participants; and 0 points for serving less than 25 percent of participants. Since points are awarded based on the 

relative percentage of performance, and not the number of participants, all providers are able to compete for performance points. 
SECTION B-2 

Rationale for System Development 

— 91 — 

KBOR administrators capitalized on provider dissatisfaction with the existing funding formula 

and their positive relations with local directors to leverage support for change. Early on, state 

administrators made it clear that local providers were expected to be equal partners in the PBF 

formula development process, with their viewpoints communicated through the state director’s 

advisory council, a group of 12 program directors drawn from throughout the state. To ensure 

state views were represented, these program directors acted on behalf of the range of providers 

and programs in the state, with members selected based on their program type and size, 

population demographics, and program offerings. This advisory council met over a roughly 

yearlong period to consider different funding approaches. 

Although initially there was a great deal of provider resistance to PBF, positive relations between 

the state director and local program directors contributed to state adoption efforts; simply put, 

program directors trusted the state director and believed that she would treat them fairly when 

distributing state and federal funds earmarked for performance allocation. This point was 

underscored during focus group interviews by one program director who commented, “I think 

she deserves the credit and I think she’s taken a lot of heat for doing this. We’ve got a good 

strong relationship.” 

State efforts to introduce PBF were also strengthened by stable administrative leadership. In 

August 2000, soon after the introduction of the new formula, the state staffer overseeing 

development was promoted to acting state director of adult education, and, in August 2001, to 

her current position as state director. This continuity helped convince local providers that the 

newly introduced PBF system would be a permanent addition to the state allocation process. 

Goals in Introducing Performance-based Funding 

Performance-based funding was introduced in Kansas to encourage local programs to provide 

better services to their adult learners. As originally crafted, the state formula was intended to 

accomplish three broad goals: 

1. To build into funding decisions the core performance measures and state-established 

Indicators of a Quality Adult Education Program, including participant outcomes, 

intensity of learner participation, instructional staff credentials, and local support; 

2. To establish a minimum level of funding so that all adult education programs have an 

adequate base from which to maintain and enhance quality services; and 

3. To recognize the difference between urban and rural centers in cost per participant-hour 

to serve those participants at the five lowest education levels. 

Incorporating core performance measures and indicators of program quality into funding 

decisions was seen as a way of motivating service providers to pay more attention to their 
SECTION B-2 

Rationale for System Development 

— 92 — 

program outcomes and operations, while providing incentives to institute reforms. It also was 

anticipated that overlap with federal AEFLA core indicators would increase the likelihood that 

the state would attain its federal performance targets. 

To protect providers from severe funding shifts, advisory council members also determined that 

the new formula should provide all agencies achieving a minimal level of program success with 

a base level of resources sufficient to maintain program operations. Although the final formula 

allocates more than four-fifths of funds based on performance, members crafted quality 

indicators to ensure that all programs generated additional funding, beyond the base grant, to 

maintain program operations. 

Finally, given the relatively high costs of serving individuals in rural areas, it was felt that the 

funding formula should explicitly recognize and compensate providers for the costs associated 

with serving more disadvantaged learners. According to the state director, this goal was intended 

to encourage rural providers to continue serving lower-level learners, considered to be among the 

most expensive to educate. 
— 93 — 

Now, can you get a formula that is

fair to everybody? Probably not—I

don’t think it’s possible. But can you

get one that we can live with? ... You

can probably get one we can live

with. You can have clear indicators of

what a quality program is. 

—Kansas State Director of Adult

Education 

SECTION B-3: 

Formula Design Process 

Kansas state administrators employed a collaborative, consensus-based approach to creating a 

new state funding formula for adult education. The development of the state PBF formula was 

supervised by members of the state director’s advisory council, a group of 12 local program 

directors selected by the state director to represent the views of the field.10 Directors met 

periodically beginning in late 1998, with state administrators taking responsibility for convening 

and facilitating council meetings and for summarizing and acting on recommendations raised 

during group discussions. Council findings were shared with program directors at annual state 

meetings and in routine correspondence issued by the state agency. 

Advisory council discussions were designed to secure 

statewide consensus on the components and operation of the 

new funding formula. As an initial activity, program directors 

were asked to describe the characteristics of a quality adult 

education program, including the types of resources and 

supports needed to provide instruction. Directors also were 

asked to summarize the types of outcomes a program might 

be expected to produce and to differentiate those over which providers had control from those 

that they could not directly affect. 

Specification of Funding Criteria: Base Funding 

Early on, local directors supported moving away from contact hours to look at learner outcomes 

and other program characteristics. To ensure that all providers were guaranteed some funding, it 

was agreed that the state should guarantee a base level of funding—equivalent to 50 percent of 

state adult education resources—with all institutions allocated an equivalent amount regardless 

of program size or population demographics. Given that Kansas had limited state resources to the 

25 percent match mandated in AEFLA, just under $550,000 was available for base distribution 

across 36 providers participating in FY00, meaning that each agency qualified for a grant of 

roughly $15,275. 

 

10 During interviews, it was observed that state directors often convene an advisory council of program directors with whom they 

can work. Although there is no evidence to support the representativeness of this handpicked group, program directors observed 

that all program providers in the state were consulted in formula development, and, in particular, that 75 percent of the quality 

indicator model came directly from negotiations with program directors throughout the state. 
SECTION B-3 

Formula Design Process 

— 94 — 

Although this guaranteed base was not sufficient to fully offset the minimal cost of operations 

for any provider, equalizing resources did provide an advantage to smaller programs, which 

otherwise would have qualified for only a small amount of funding if resources were allocated 

based on student-contact hours or other criteria accounting for program size. Larger providers 

were willing to grant this concession because other aspects of the funding formula, which 

rewarded learner outcomes, ensured that larger programs could generate greater eligibility for 

funding. 

Political realities also may have supported equalizing allocations across agencies. According to 

the state director, state legislators would have been unwilling to advocate for increased state 

funding unless they believed their constituents were getting a fair share of available resources. 

Specification of Funding Criteria: Performance Funding

To allocate performance funding, council members agreed to award 50 percent of federal 

funding based on the number of learner outcomes a program achieved, using eight core and 

secondary measures paralleling those contained in the NRS. Points were to be awarded based on 

the following criteria: 

• NRS Core Measures 

○ Made lower educational level gain—all with outcomes doubled 

(completion of levels 1=ABE Beginning Literacy, 2=ABE Beginning 

Basic, 7=ESL Beginning Literacy, 8=ESL Beginning, or 9=ESL 

Intermediate Low); 

○ Made higher educational level gain (completion of levels 3=ABE 

Intermediate Low, 4=ABE Intermediate High, 5=ASE Low, 6=ASE High, 

10=ESL Intermediate High, or 11=ESL Low Advanced); 

○ Entered employment; 

○ Retained employment; and 

○ Received GED. 

○ NRS Secondary Measures 

○ Entered postsecondary education or other training; 

○ Acquired U.S. citizenship skills (skills needed to pass the U.S. citizenship 

exam); and 

○ Increased involvement in children’s literacy. 
SECTION B-3 

Formula Design Process 

— 95 — 

Council members recognized that basing funding on learner outcomes would steer a substantial 

percentage of the roughly $550,000 in federal resources available in FY00 to larger programs. 

Smaller providers were willing to support this approach because of the compromise guaranteeing 

them base funding equal to that of larger providers and their ability to compete for state 

resources based on program quality and other criteria unrelated to size. 

Advisory council members also agreed to double-weight educational gains for students in the 

two lowest levels in ABE and three lowest levels of ESL (i.e., a completion by a lower-level 

learner counts twice as much as one by a higher-level learner). This was a response to data 

supplied by the state indicating that students at lower levels took approximately 1.8 hours to 

make a learning gain, compared to 1.0 hour for those at higher levels. Double-weighting lowerlevel learners was seen by council members as a way of compensating providers serving 

relatively larger populations of lower-level learners, while avoiding financial disincentives that 

could cause some providers to curtail remedial instruction. 

Specification of Funding Criteria: Program Quality 

Since basing performance funding on outcomes alone could have a detrimental effect on smaller 

providers, council members agreed to the state director’s proposal to integrate a quality 

component into the funding formula. Working from a set of existing quality indicators used to 

evaluate state programs before adoption of AEFLA, participants went through four draft 

proposals before agreeing on nine program indicators encompassing 20 program quality 

measures. Quality criteria ranged from process measures used to capture how an agency went 

about implementing its programs, to outcome measures used to assess the percentage of 

participants who completed an achievement level in a program year. 

Council members agreed to weight quality indicators based on their perceived importance to 

adult education programs. For example, measures with a student focus were generally given a 

point value of between 8 and 10 points (10 points being the highest), while those dealing with 

administrative outcomes, such as the documentation of a program technology plan, were 

assigned lower point values (e.g., between 2 to 4 points). Care also was taken to ensure that all 

providers, regardless of program or learner characteristics, had an equal opportunity to attain 

similar point totals. 

Since not all programs were capable of earning points on each quality indicator, program 

directors also negotiated to assign point values that accounted for their differing administrative 

capabilities. This has meant that some programs do not attempt to earn quality points for all 

measures. For example, during case study focus group interviews, two directors of large urban 

agencies reported that they do not attempt to earn points for serving learners with documented 
SECTION B-3 

Formula Design Process 

— 96 — 

disabilities because a high proportion of their participants are classified as ESL learners—a 

population for which disabilities are seldom reported. Rather than devote substantial resources to 

identifying student disabilities, these directors relinquish these points, since they can successfully 

compete for points in other areas. 

Assessing Formula Effect 

To assess the potential impact on local providers, state administrators modeled how program 

allocations would change under different versions of the formula. Although historical provider 

data on many of the eight NRS program outcome measures were available, analysts lacked 

information to estimate provider point allocations across the 20 quality measures council 

members had developed; consequently, for modeling exercises, programmers assumed that all 

providers would earn the median level of points. This reflected the belief that, because program 

directors had negotiated an equitable distribution of points, all providers would earn similar point 

totals. 

Final PBF allocation models suggested that, with the exception of some large providers, most 

agencies would realize only a modest change in their overall funding eligibility. This was due, in 

part, to the careful deliberation that went into formula design and, in part, to a state commitment 

to allocate additional, held-back federal funding in the first two years of the PBF formula 

operation. At the close of the formula development process, state administrators and advisory 

council members believed that they had created a workable formula that would account for the 

diverse needs of providers throughout the state. 
— 97 — 

SECTION B-4: 

Implementation Process 

Kansas fully implemented PBF in the 1999–2000 program year. Although the new state funding 

formula did not contain any provisions to insulate providers from dramatic shifts in their 

resource eligibility, program directors were assured that their programs would not be shut down 

as long as they showed continued improvement on performance measures. To reduce initial 

potential negative effects, as stated previously, the state also decided to release approximately 

$100,000 in federal funds held back during the first two years of PBF formula operation. 

It was anticipated that local providers might have difficulty obtaining accurate participant 

outcome data in the first year of formula use, since program staff would be in the midst of 

shifting to a new data collection system and set of performance indicators. Accordingly, the state 

based its initial performance allocation of federal funds on the number of participant hours for 

those at the five lowest achievement levels (i.e., the two lowest levels in ABE and three lowest 

levels in ESL) times a program-size factor. Beginning in the second program year, this portion of 

the funding formula changed to reflect actual participant outcomes. 

Provider Reporting Capacity 

To prepare providers to report required AEFLA performance measures, in FY99 the state 

provided each program with Access database software and instructions on how to enter student 

performance data into state reporting frameworks. Although the state made a concerted effort to 

train program staff on database use, many providers initially were unable to collect valid 

information for key outcomes measures used to allocate PBF resources, undercutting their 

program funding eligibility. This occurred because service providers often were not following 

state testing protocols for measuring participant outcomes; for example, counts of individuals 

making learning gains were initially rejected for some providers because program staff had failed 

to enter pretest data or had incorrectly administered Comprehensive Adult Student Assessment 

System (CASAS) diagnostic pretest, posttest, or both, forms. 

Unanticipated problems associated with the award of quality points also adversely affected 

provider resource eligibility. Upon calculating program allocations for FY00, state administrators 

discovered that they had grossly overestimated awards for many larger service providers, who 

were incapable of earning the median level of quality points assumed during modeling exercises. 
SECTION B-4 

Implementation Process 

— 98 — 

The combination of incomplete performance outcome data and inaccurate quality data meant that 

some providers experienced unexpected funding losses. 

Once the magnitude of funding losses became apparent, state personnel contacted affected 

providers to advise them of anticipated shortfalls and to reassure them that, with improvements 

to data collection and attention to service delivery, funding deficits could be corrected. 

Administrators also provided technical assistance to local providers throughout the 2000–01 

program year to train them to enter missing data into institutional databases and to identify 

performance gaps adversely affecting their eligibility. Consequently, during the second year of 

performance funding, many large providers regained funding, in some cases at the expense of 

smaller programs unable to generate equivalent performance outcome gains.

Data Quality 

To validate local program data and award quality points, the state conducts annual monitoring 

visits to each service provider. Monitoring activities entail auditing individual student files to 

confirm that data are properly documented and testing protocols followed and reviewing 

administrative and other program records stipulated in the standardized state monitoring 

instrument. Quality points were initially based on provider performance on 20 measures of 

program operations, but over time the number of measures has increased to 25 to better align 

with state goals. 

During the first year of PBF, state staff performed local program visits. This approach was 

abandoned because it was found that program staff were unwilling to request technical assistance 

for fear of exposing program flaws. In the second year, the state tried contracting with program 

monitors from other states, but found that it took too long to train external evaluators to use the 

state measurement tool. 

Beginning in the third program year and continuing to the present, the state director has 

contracted with an in-state team to conduct program quality reviews. Because these in-state 

monitors understand how state adult education programs operate, they are able to audit 

performance and administrative data effectively to award quality points during a one-day site 

visit. 

To encourage statewide improvement, the monitoring team meets each year with the state 

director before initiating local visits. At these meetings, the state director identifies issues of 

concern and areas in which she would like the monitoring team to focus. For example, during the 

initial stages of PBF system adoption, the monitoring team was directed to concentrate on 

provider testing protocols to ensure that instructors were following appropriate procedures to 
SECTION B-4 

Implementation Process 

— 99 — 

measure participant gains (e.g., not administering the same form within six months of its first 

use). 

Management Information Systems 

Because Kansas’ shift to performance funding occurred along 

with the state’s rollout of new federal reporting requirements 

under AEFLA, program directors had difficulty differentiating 

technology changes due to the adoption of PBF from those 

due to the AEFLA requirements. For example, the four 

program directors interviewed agreed when a researcher 

observed, “It’s difficult to differentiate how much of the 

change and the discomfort was due to the performance-based funding system and how much was 

due to the NRS and performance funding piggybacking on it.” 

To standardize data for the NRS measures, the state office supplied all providers with Access 

database software and guidance for entering participant data into the system. State administrators 

also worked to clarify data entry instructions as it became apparent that local staff needed 

additional support; over time, these instructions were compiled into a state policy manual to 

structure annual data collection. The effort required to compile detailed guidelines had not been 

anticipated by state staff, who found themselves simultaneously devoting substantial resources to 

policy development and to providing technical support. 

Although providers were forced to make substantial changes to their management information 

systems, program directors reported that they did not incur any direct technology costs associated 

with performance reporting. That is, had performance reporting not been introduced, providers 

would still have been required to standardize data administration to comply with AEFLAmandated accountability measures. 

Because high stakes were now associated with performance, 

program directors reported that the new PBF formula forced 

them to pay much closer attention to their data than if 

reporting was solely for federal accountability purposes. 

During focus group interviews, directors from large service 

providers were more likely to report adding a staff member to 

manage data entry and administration, while directors from 

smaller programs reported that they either took on the data 

entry role themselves or delegated the task to existing staff. 

Your job as an educator is [to do]

whatever it takes to provide quality

services. And if part of what you do is

making sure the data you’re entering

is correct, so you can really have

good data to make decisions later on,

then that’s what it takes. You may not

like it—it may not be what you

thought education was about—but

that is part of your job now.

—Kansas State Director of Adult 

Education 

You show me any business or

company that doesn’t have somebody

who is collecting that data, how

they’re doing it, I'll show you

somebody who is in bankruptcy ...

because you can’t do your job if you

don’t have that data. 

—Kansas Program Director 
SECTION B-4 

Implementation Process 

— 100 — 

Technical Assistance 

To help provider staff understand how to use their new Access software, the state held statewide 

and regional training sessions throughout FY00. These technical assistance meetings were aimed 

at different audiences, with training directed toward data entry staff on how to use database 

software and toward program directors on how to interpret data to guide program improvement 

efforts. Smaller regional meetings also were held to allow providers to bring multiple staff for 

training and to address site-specific issues. 

Program directors credited the state agency for its efforts to assist them in the transition to the 

new reporting system. One particularly valuable resource was the hiring of an Access database 

expert who conducted statewide training sessions at statewide summer institutes and who was 

available for one-on-one consultation via phone or e-mail. State administrators also provided onsite technical assistance to resolve provider data issues. This frustrated some program directors, 

who reported that they were often unable to obtain timely feedback from state administrators, 

including the state director, because individuals were seldom in the office or were too busy to 

return telephone messages. 

Over time, as local familiarity with reporting system operations increased, state administrators 

were able to target technical assistance to methodological issues undercutting program reporting. 

For example, recognizing that there was a problem with measuring learner gains, the state held a 

series of technical assistance training sessions on the proper use of CASAS assessments, 

including reviewing testing protocol, test selection, and data entry. More recently, the state has 

focused on improving lesson planning and program instruction. 
— 101 — 

We have better teachers right

now at our center than we had

ten years ago. We have better

programs. ... We have people

who take pride in what they do.

We have very little turnover at

our program.

—Kansas Adult Education 

Program Director 

SECTION B-5: 

Effects of Performance-based Funding 

During interviews, both state administrators and local provider staff commented on the effect of 

PBF on adult education program services. Although case study participants ascribed both 

positive and negative outcomes to the adoption of performance funding, few, if any, data exist to 

substantiate these claims. In many cases, participant observations were based on general 

perceptions of program operations or extrapolations of a single experience. Some participants 

also had difficulty differentiating programmatic effects due to the adoption of PBF from those 

associated with the state’s implementation of AEFLA. Therefore, readers should be cautious in 

drawing conclusions about the effects of PBF on state adult education systems. 

Contributions to Statewide Performance 

As originally conceived, PBF was introduced in Kansas to promote more effective adult 

education services, while simultaneously improving the equity and transparency of state 

allocations. According to the state and local directors, the adoption of PBF has contributed to 

achieving state goals, particularly to: 

• Improving the Validity and Reliability of Adult Education Data—Linking 

resources to provider performance has helped to increase the quality of locally 

reported data. During the first year of implementation, state administrators 

primarily focused on getting local providers to enter participant outcomes, which 

typically were obtained from teacher reports. State analysis of quarterly data 

submitted by providers revealed that a great deal of this information was 

inaccurate, in part, because program staff were not yet familiar with new NRS 

reporting requirements, and, in part, because no importance had been attached yet 

to the submitted data. 

Monitoring visits to service providers, which were used both to 

audit administrative files and to award PBF quality points, helped 

to ensure that locally reported data were accurate and 

representative. In particular, since quality points were awarded 

based on whether randomly selected learners’ permanent files 

revealed any evidence of irregularities, service providers were 

motivated to improve their file administration and testing 

protocols. 
SECTION B-5 

Effects of Performance-based Funding 

— 102 — 

It made my institution step forward

with more cash. For years, the 10

percent minimum is what my

institution provided. And with

performance-based funding ... now

my institution is kicking up to 20

percent. So it’s made an impact on

my host agency to say this is worth

keeping, this is worth subsidizing. 

—Kansas Program Director 

• Professionalizing the Teaching Force—Performance funding has been credited 

with helping program directors assess instructor performance in relation to state 

goals. Since a substantial amount of funding is tied to student achievement, local 

directors reported that they were more motivated to provide professional 

development to individuals with a demonstrated need and, in cases where staff 

were unwilling to change, to release such staff to make room for more 

sophisticated individuals who understood the importance of accountability. 

Program directors also report that PBF made them more likely to collaborate with 

one another, because they are now able to identify successful programs to which 

they can turn for consultation and support. 

• Improving the Effectiveness of Instructional Services—Before the introduction of 

performance funding, providers were compensated for the number of contact 

hours generated, irrespective of whether a participant benefited from program 

services. Basing funding on performance has encouraged local agencies to shift to 

managed enrollment, meaning that students may only enroll and enter classes at 

specific times rather than on a continuous basis (e.g., in the first three weeks of a 

12-week course). 

This shift has helped improve retention because providers have realized that they 

must keep students enrolled if they are to achieve measurable outcomes, and 

because providers are more likely to focus intensive services on lower-level 

learners to support them in reaching their education goals. Indeed, since the 

introduction of PBF in FY00, the mean hours of attendance per learner have 

increased nearly 85 percent, rising to an average of 78 hours per participant in 

FY05, from roughly 42 mean hours in FY00. 

• Attaining Federal Reporting Targets—The state did 

not initially meet its negotiated AEFLA performance 

targets because program directors did not understand 

new federal reporting requirements or the importance 

of collecting accurate data. However, because state 

PBF measures overlapped those in the NRS, statewide 

performance increased as program directors worked to 

generate learner outcomes to qualify for additional 

performance resources. Consequently, Kansas made 

remarkable improvement on the NRS measures as program directors improved 

data collection and instituted program improvement strategies. 
SECTION B-5 

Effects of Performance-based Funding 

— 103 — 

To illustrate, in the second year of NRS reporting (FY01), Kansas reported that 53 

percent of adults enrolled in basic literacy programs succeeded in acquiring the 

skills needed to complete one or more levels of instruction in which they were 

initially enrolled. In the third year (FY02), the statewide percentage soared to 73 

percent. While statewide performance in basic literacy fell to 58 percent in the 

following year, overall state performance in all measures continues to exceed that 

of other states. 

• Contributing to Political and Fiscal Support for Adult Education—Performance 

funding has been credited with increasing state and institutional investment in 

adult education because legislators and the business and higher education 

communities philosophically support allocating funding based on performance. 

Moreover, because accurate data on learner participation and outcomes are now 

more readily available, state administrators are better able to communicate to 

policymakers the fiscal and social benefits of investment in adult education 

services. 

Drawbacks 

Interviews with state administrators and local program directors indicated that PBF has been 

effectively institutionalized in Kansas. However, not all providers initially supported the new 

funding system or enthusiastically support all aspects of the current funding formula. Drawbacks 

associated with implementing PBF include: 

• Costs Associated With Awarding Program Quality Points—Basing funding on 

multiple measures of program quality has required that state representatives visit 

all service providers in the state annually to audit administrative records and 

award quality points. This need for annual monitoring visits has added a 

substantial recurring cost to PBF formula operations. State expenses include 

paying for monitoring team salaries and travel, coordinating with team members 

on the conduct of visits, and administering award data to calculate performance 

allocations. 

Local provider staff also spend time preparing for monitoring team visits, 

including using a monitoring instrument to conduct self-assessments, updating 

and reviewing participant files for accuracy, and meeting with monitoring team 

members during the one-day visit. This focus on collecting data can steer 

resources away from instructional programs, as one program director observed, 

“We are devoting time and effort to collecting data and assessments. ... It’s 

possible that we’re losing a lot of clientele due to its structure.” 
SECTION B-5 

Effects of Performance-based Funding 

— 104 — 

• Challenges in Maintaining Small and Rural Programs—Since only a relatively 

small percentage of adult education funding is awarded as a basic grant, small and 

rural providers have had difficulty generating sufficient resources using the 

performance formula to maintain services at multiple locations in their service 

area. This occurs because smaller agencies are both unable to generate large 

numbers of participant outcomes and unable to effectively compete for quality 

points due to economies of scale. For example, smaller providers have had 

difficulty offering computerized instruction at all locations and providing services 

at a cost per outcome that conforms to the state average cost per outcome. Rural 

programs also may struggle to retain participants once service cuts are made. 

Because of distances between service centers, when one location is closed, 

learners are often unwilling or, with the increasing cost of gas, unable to drive 

long distances to continue course work or to take posttest exams to document 

their learning gains. 

• Focus on Funding Criteria Over Instructional Goals—Program directors 

suggested that performance funding can lead instructors to become overly 

conscious of achieving state-identified outcomes at the cost of other, 

nonquantifiable student skills. Complex data reporting guidelines also require that 

teachers invest time learning state data collection requirements, which can come 

at the expense of lesson planning or instructional support. It also was suggested 

that performance funding does little to promote student learning, since program 

participants are more concerned with achieving an end goal, such as earning their 

GED, then in attaining intermediate goals, such as making gains on the CASAS 

exam. 

Effect of PBF on Adult Education Providers 

Before the adoption of PBF in FY00, 36 providers were eligible to receive federal and state adult 

education funding. In keeping with their promise to protect programs in the first year of 

implementation, state administrators renewed funding for all 36 providers for FY01, although 

three programs—Let’s Help, Elkhart Unified School District, and Pratt Community College—

were placed on probation and conditionally funded because of quality issues associated with 

their services. 

After failing to make progress in FY01, these programs were denied funding for FY02, the final 

year of a three-year funding cycle for all programs. Of the three programs, only Let’s Help 

reapplied for funding in the next two-year competitive funding cycle (FY03 and FY04), and, 

since grant reviewers approved its application, the Let’s Help agency returned as an AEFLA

program in FY03. 
SECTION B-5 

Effects of Performance-based Funding 

— 105 — 

Since the introduction of PBF, a total of five providers have been dropped from the adult 

education provider network: two community colleges, two community-based organizations, and 

one school district program. In the estimation of the state director, provider withdrawal resulted 

not because the agencies lacked the finances to offer services, but rather because program staff 

failed to offer adult education services judged of sufficient quality to warrant continued 

recognition as an AEFLA program. Programs failing to make annual gains received low marks 

from grant reviewers during the two-year competitive funding cycle and consequently were 

denied eligibility to return as an AEFLA program. 

Fiscal Eligibility 

State administrators reported that the implementation of the new PBF formula produced 

unexpected resource gains and losses for many providers; however, since pre-PBF allocation 

data are no longer available, it is impossible to quantify changes in provider allocations for the 

last year of contact-hour funding in FY99 and the first year of PBF in FY00.11 Analysis of 

existing data indicates that between the first two years of program implementation, 25 of 35 

providers that existed in both years (70 percent) experienced resource losses of 25 percent or 

more of their base funding (see table B-3). Resource losses were greatest among larger providers 

who were, in the estimation of program directors interviewed for this study, unable to report 

valid participant outcomes in the initial stages of PBF implementation. 

Although many larger providers in FY04 were still below their PBF allocation levels in FY00, 

the first year of performance funding, with some exception, most had been able to gain back 

some of the resources they had lost in earlier years (not adjusted for inflation). For example, 

Labette Community College lost roughly 40 percent of its funding between the first and second 

years of the new formula, as it struggled to adopt to new funding requirements; however, by 

FY04 the college had posted a 56 percent gain over its FY00 allocation. 

Program Outcomes 

According to the state director, increases in statewide performance are directly attributable to the 

introduction of PBF and reflect a combination of increased performance by providers and 

improved collection of performance data. In particular, in FY03 the state instituted a policy of 

checking learners’ files to ensure the correct assessment was given, both pretest and posttest. In 

the director’s estimation, had the state originally introduced the same high standards for 

 

11 State administrators originally believed that they could provide funding data preceding and immediately following PBF 

introduction. State policy is to maintain two years of hard copy data on-site and to store records off-site for five years before their 

destruction. When they sought to retrieve hard copy files, state administrators were informed that paper records no longer existed 

at the state level, and that, due to changes in state data systems, electronic files were no longer available. Therefore, funding data 

are available only for the years immediately following PBF introduction. 
SECTION B-5 

Effects of Performance-based Funding 

— 106 — 

assessment instituted in FY03, the trend of increasing percentages of level completers would 

have been steady over time. 

As illustrated in table B-4, the percentage of participants realizing learning gains increases 

substantially beginning in FY01, and although outcomes have fluctuated somewhat across the 

years, state performance in FY04 is nearly double that posted in FY00, the second year of 

performance funding. The number of students achieving outcomes also has increased in absolute 

terms. For example, a total of 5,593 learners were reported as completing an ABE, ASE, or ESL 

level in FY04, up from 5,377 in FY01. 

Given the differential weighting of student outcomes whereby lower-level ABE and ESL student 

gains are doubled, it is possible that local providers might choose to target services to students 

with relatively higher point awards or to drop services to high-cost students. Indeed, declining 

enrollments over the years could be a result of providers shifting services away from specific 

student populations. However, since students who are more resource-intensive to serve also earn 

higher point awards in the state system, it does not appear that providers have great incentive to 

game the system to increase their resource eligibility, nor are they facing a disincentive for 

serving students requiring greater investment. 
SECTION B-5 

Effects of Performance-based Funding 

— 107 — 

 FY00 FY01 FY02 FY03 FY04

1st Year 2nd Year 3rd Year 4th Year 5thYear FY00 FY00

of PBF of PBF of PBF of PBF of PBF to to

 FY00 FY01 FY02 FY03 FY04 FY01 FY04

Total $3,591,514 $3,938,890 $4,244,996 $4,640,551 $4,782,816 9.7% 33.2%

Kansas City Kansas Com. College 276,863 152,808 180,244 181,533 156,579 (44.8) (43.45) 

Dodge City Com. College 227,930 162,621 167,591 128,389 131,311 (28.7) (42.39) 

Wichita Area Technical College 370,335 266,917 273,079 251,217 303,618 (27.9) (18.02) 

Emporia USD 210,753 185,744 197,751 181,175 190,030 (11.9) (9.83) 

Seward County Com. College 319,703 228,261 276,998 325,681 299,718 (28.6) (6.25) 

Butler County Com. College 90,712 88,003 93,322 118,275 97,623 (3.0) 7.62 

Independence Com. College 107,321 79,934 87,221 103,275 117,478 (25.5) 9.46 

Highland Com. College 101,264 93,033 91,971 124,232 132,352 (8.1) 30.70 

Garden City Com. College 196,570 210,461 278,396 285,287 280,692 7.1 42.79 

Salina Adult Education Center 110,046 131,870 121,740 151,782 160,911 19.8 46.22 

Labette Com. College 118,951 71,951 104,141 156,138 183,211 (39.5) 54.02 

Johnson County Com. College 254,306 239,337 404,233 371,060 397,932 (5.9) 56.48 

University of St. Mary Outfront 78205 84,117 92867 118457 127155 7.6 62.59 

Colby Com. College 84,170 107,157 128,892 119,529 146,154 27.3 73.64 

Topeka KATS 99,462 114,894 134,251 145,395 176,149 15.5 77.10 

Coffeyville County Com. College 58,957 100,354 93,215 101,976 106,534 70.2 80.70 

Wichita Indochinese Center 73,198 116,496 146,407 140,324 133,360 59.2 82.19 

Manhattan Adult Learning Center 64,600 105,034 104,338 117,737 122,956 62.6 90.33 

Osawatomie Learning Center 47,005 81,919 75,892 101,571 92,254 74.3 96.26 

Hutchinson Com. College 69,435 80,117 88,359 124,391 142,301 15.4 104.94 

Allen County Com. College 56,133 72,456 82,109 123,528 120,159 29.1 114.06 

Lawrence Adult Learning Center 61,689 77,867 93,839 133,230 133,018 26.2 115.63 

Newton Cooper Education Center 46,364 79,927 77,616 103,307 102,076 72.4 120.16 

Cowley County Com. College 53,715 60,548 79,518 110,587 123,187 12.7 129.33 

58,352 93,024 109,220 136,900 139,148 59.4 138.46 

Paola Adult Education Center 44,924 84,490 81,772 103,131 123,660 88.1 175.26 

Let’s Help, Inc. 43,363 88,093 NF 106,603 121,615 103.2 180.46 

Ottawa Adult Education Center 29,638 73,201 69,753 84,957 91,107 147.0 207.40 

Neosho County Com. College 26,753 59,131 83,912 112,785 91,894 121.0 243.49 

Pittsburg Adult Education Center 32,473 91,155 94,798 121,304 132,149 180.7 306.95 

Cloud County Com. College 16,667 59,374 82,894 66,120 106,485 256.2 538.90 

Fort Scott Com. College 57,336 77,567 75,050 NF NF 35.3 —

Junction City/Hauge Alt. Center 54,557 94,694 87,989 90,675 NF 73.6 —

Osborne Adult Literacy Project 31,043 87,159 85,618 NF NF 180.8 —

Pratt Com. College 18,721 75,513 NF NF NF 303.4 —

Elkhart — 63,661 NF NF NF — —

SOURCE: Kansas Board of Regents, 2006. Funding data provided by state.

Note: NF—Not funded.

Note: — Not applicable.

Table B-3—

Kansas Local Provider Funding Eligibilities and Percentage Change Between Selected Fiscal Years: FY00 

to FY04

Barton Co C

Funding in Percentage change
SECTION B-5 

Effects of Performance-based Funding 

— 108 — 

As evidence, state administrators reported that their own internal analyses reveal that providers 

with large ESL populations appear to be flourishing in the system, indicating that formula 

weighting, on average, adequately compensates providers for their instructional costs. Indeed, a 

review of state participation data indicates that enrollment rates for ESL students have declined 

at a slower pace than for other populations. Specifically, ESL participation rates have fallen by 

approximately 9 percent between FY01 and FY04 (falling from 4,272 to 3,873, respectively). In 

contrast, participation rates have declined by approximately 15 percent for participants in ABE 

or ASE populations over the same period. Moreover, the proportion of adults completing an ESL 

program actually has increased over time, perhaps indicating that providers are making 

improvements in their ability to serve limited-English populations. To illustrate, the percentage 

of adults completing an ESL level climbed by 28 percent between FY01 and FY04, compared to 

an 8 percent decline in program completion among ABE or ASE populations. 

System Evolution 

To standardize the collection of federal data, Kansas supplied all providers with Access database 

software and data entry instructions to guide local reporting. During the first few years of system 

implementation, providers were required to submit a quarterly performance report to the state. 

To improve data timeliness, in the last quarter of FY05 the state introduced a Web-based 

% Change

FY99 FY00 FY01 FY02 FY03 FY04 FY01–FY04

Participants 16,461 11,410 11,248 10,725 10,386 9,788 -13.0%

 Completed Education Level 53% 33% 49% 74% 58% 61% NA

Entered Employment 5% 47% 50% 59% 52% 62% NA

Retained Employment 6% 40% 53% 74% 54% 69% NA

Entered Postsecondary 5% 29% 25% 69% 60% 58% NA

Passed GED 31% 39% 50% 71% 61% 64% NA

Program Participation

Adult Basic/Secondary Education NA 7,070 6,976 6,577 6,326 5,915 -15.2%

English Literacy NA 4,340 4,272 4,148 4,060 3,873 -9.3%

Level or Outcome Completion 5,377 7,677 5,737 5,593 4.0%

Adult Basic/Secondary Education NA NA 3,592 4,842 3,492 3,310 -7.9%

English Literacy NA NA 1,785 2,835 2,245 2,283 28%

SOURCE: U.S. Department of Education, Report to Congress on State Performance, program years 1999–2000 to program years 

2003–04. http://www.doe.state.in.us/adulted/performancedata.html (last accessed June 18, 2007).

Note: NA—Not available.

a Initial years of PBF adoption.

Table B-4—

Selected Performance Characteristics of Kansas Adult Education Participants, FY99 to FY04, and 

Percentage Change, FY01 to FY04a
SECTION B-5 

Effects of Performance-based Funding 

— 109 — 

reporting system, known as PABLO, to permit providers to enter data directly into state data 

systems. Modeled after the state’s Access database, the PABLO system was developed by the 

state with the assistance of an outside consultant. Following a one-quarter fiscal year 

implementation period, in which agencies maintained dual information systems, the PABLO 

system was fully implemented in the FY06 program year. 

The state also makes continual updates to the performance benchmarks included in its funding 

formula. To motivate providers to make continuous improvement, each year the state evaluates 

its performance levels and, in some cases, increases performance benchmarks so that programs 

have an incentive to continue to review data and undertake reforms. 

Process indicators used to assess program quality also are 

reviewed constantly, with the state making changes to 

address perceived problems in program measures. As 

originally conceived, the Indicators of a Quality Adult 

Education Program included nine indicators and 20 

measurable standards; over time, the state has added 

measures to align the monitoring instrument better with state goals. 

Recent changes include adding a program cost-effectiveness measure to reward providers who 

operate efficient programs and eliminating the process requirement that programs update their 

professional development plans annually. Here, in response to feedback from local providers, the 

state modified the quality measure to require local agencies simply to meet their negotiated 

program improvement plan objectives, based on the assumption that an agency achieving its 

objectives must be doing something correct. 

Finally, to motivate providers to achieve learning gains throughout the year, the state has 

increased the amount of funding it awards for performance outcomes recorded in the fourth 

quarter of each academic year. When the PBF formula was originally introduced, the state based 

funding on the first three quarters of the academic year, with $100,000 held out of the allocation 

formula to compensate providers for their fourth-quarter results, which are not available until 

after the start of the subsequent fiscal year. In the year following, the state increased this funding 

to $200,000, and, beginning with FY06, this amount has been increased to $500,000. This action 

was taken because many providers were cutting back services in the fourth quarter, in part, 

because these outcomes were not seen as having as much value as those earlier in the year.

I know some people aren’t going to

like me to say this, but our Kansas

program is not perfect. That’s why we

continue to modify it and tweak it all

the time. 

—Kansas State Director 

— 111 — 

Appendix B Summary 

With 87 percent of its adult education resources in FY06 distributed based on provider 

performance, Kansas leads the nation in using PBF to allocate federal and state resources. This 

commitment illustrates state administrators’ belief, in particular, that of the state director of adult 

education, that performance funding can effectively motivate local program providers to collect 

accurate program data and to use this data for program improvement purposes. Unlike in the 

higher education sector, the decision to adopt PBF was made almost entirely based on 

programmatic decisions; state administrators reported no political or other external pressures to 

adopt performance funding. 

To build support for PBF and to design an allocation mechanism, the state director worked with 

members of her state advisory council, a group of 12 program directors drawn from throughout 

the state. These members worked with state administrators, in a collaborative, consensus-based 

approach to measure development. This support was crucial in winning statewide support for the 

adoption of PBF. Although initially there was a great deal of provider resistance to PBF, public 

support from council members combined with positive relations between the state director and 

local program directors contributed to state adoption efforts. 

To allocate resources, Kansas uses a fairly complex formula that draws on eight core and 

secondary performance outcome measures mandated in the federal NRS, along with 25 stateestablished indicators of program quality. Quality points are based on a system applied uniformly 

across programs, irrespective of size or population demographics. To award points, the state 

contracts with a team of program monitors, who visit each provider in the state yearly to assess 

program operations. Although the use of quality indicators adds a sizeable cost to program 

operations, state administrators believe that the data collected provide useful information that can 

support program directors in improving student and program performance. 

State administrators place considerable weight on the use of program quality points, allocating 

50 percent of both federal and state resources based on provider quality. Half of remaining state 

funds is distributed as a guaranteed base, provided equally to all qualified providers, and half 

based on provider performance on the selected NRS measures. The state also double-counts 

outcomes for learners in the five lowest educational functioning levels of the NRS to compensate 

providers for the additional costs associated with achieving learning gains with these groups. 

This decision was made based upon a review of state data that indicated that lower-level learners 

required nearly twice the investment of time to achieve a positive outcome. 
Appendix B Summary 

— 112 — 

Kansas fully implemented PBF in the 1999–2000 program year. Although the new state funding 

formula did not insulate providers from shifts in their resource eligibility, program directors were 

assured that their programs would not be shut down as long as they showed continued 

improvement on performance measures. To reduce potential negative effects, the state director 

also decided to release approximately $100,000 in federal funds held back during the first two 

years of PBF formula operation. This infusion of additional resources helped to soften the impact 

of system adoption. 

Although state administrators attempted to model formula effects prior to its implementation, 

unanticipated problems associated with the award of quality points and the collection of local 

data adversely affected provider resource eligibility. The combination of assumptions about 

program quality that proved, upon assessment, to be inaccurate and incomplete performance 

outcome data meant that some providers, particularly large ones, experienced unexpected 

funding losses. State personnel worked with affected providers to advise them of anticipated 

shortfalls and to provide them with technical assistance to remedy identified problems. 

Consequently, during the second year of performance funding, many large providers regained 

funding, in some cases, at the expense of smaller programs unable to generate equivalent 

performance gains. 

Although case study participants ascribed both positive and negative outcomes to the adoption of 

PBF, few, if any, data exist to substantiate these claims. Participant observations suggest that 

PBF contributed to improvements in the validity and reliability of adult education data, because 

program directors were more likely to review information for accuracy prior to submission. 

Performance funding also was credited with helping to professionalize the teaching force and 

improve the effectiveness of instructional services, to support the state in attaining federal 

AEFLA reporting targets, and to contributing to political and fiscal support for adult education. 

Due to the labor-intensive process used to award program quality points, state adoption of PBF 

has added a substantial cost to the collection of performance data, as well as imposed a burden 

on local provider staff, who must invest time to prepare for monitoring visits. Since only a 

relatively small percentage of adult education funding is awarded as a basic grant, small and 

rural providers have had difficulty generating sufficient resources using the performance formula 

to maintain services at multiple locations in their service area. Some program directors 

interviewed also commented that the focus on funding criteria over instructional goals can cause 

instructors to become overly conscious of achieving state-identified outcomes at the cost of 

other, nonquantifiable student outcomes. 

Since the introduction of PBF, a total of five providers have been dropped from the adult 

education provider network. In the estimation of the state director, provider withdrawal resulted 
Appendix B Summary 

— 113 — 

not because the agencies lacked the finances to offer services, but rather because program staff 

failed to offer adult education services judged of sufficient quality to warrant continued 

recognition as an AEFLA program. 

Review of state performance data indicates that, compared to other states, Kansas scores well 

above the national average on most NRS measures. According to the state director, increases in 

statewide performance are directly attributable to the introduction of PBF and reflect a 

combination of increased performance by providers and improved collection of performance 

data. 

— 115 — 

APPENDIX C 

Missouri Case Study Report: 

Performance-based Funding In 

Adult Education 

— 117 — 

SECTION C-I: 

Overview of the State Adult Education Funding 

System 

In FY02, the Missouri Department of Elementary and Secondary Education adopted a 

performance-based funding (PBF) system to allocate its federal and state adult education 

resources among eligible service providers. Under this current system, local providers are 

guaranteed a base level of funding, drawn from federal and state resources, which is distributed 

across providers based on audited learner-contact hours. Providers compete for remaining federal 

and state resources that are allocated based on provider performance on two measures of student 

progress contained in the National Reporting System for Adult Education (NRS). Allocations to 

local providers are distributed using the following criteria. 

Base Funding: Each fiscal year, the state establishes a base funding rate used to allocate 

resources based on a provider’s total audited contact hours. For example, in FY06, the state 

established a base funding rate of $2.40 per audited contact hour, with half of the rate ($1.20) 

distributed based on a program’s total number of audited contact hours in the first preceding 

fiscal year for which audited totals are available (i.e., FY04 for FY06). The remaining half of 

base resources ($1.20) are allocated based on a program’s previous three-year average of its total 

audited contact hours (i.e., using FY02–04 for FY06). 

Performance Funding: Performance resources are distributed based on the number of 

individuals who made academic progress on two measures defined in the Adult Education and 

Family Literacy Act (AEFLA): 

• Received GED. 

• Completed one of the functional academic performance levels identified for adult 

basic education (ABE), adult secondary education (ASE), and English as a second 

language (ESL) learners as specified in the NRS. 

Performance resources lag two years behind the recording of outcomes, meaning that local 

provider eligibility is determined by program performance in the fiscal year two years preceding 

the current one. Local provider funding eligibility is calculated by multiplying the total number 

of individuals achieving a positive outcome by a state-established rate for each measure. 
SECTION C-1 

Overview of the State Adult Education Funding System 

— 118 — 

Categorical Funding: Federal and state resources are earmarked for use in specific functions 

identified by the state. These include data collection funds to support data entry salaries and 

computing upgrades, marketing funds to offset provider advertising expenses, technology funds 

to purchase computing hardware and software, and One-Stop program funds to support 

coordination with One-Stop Career Centers. Local provider eligibility is calculated by averaging 

providers’ three-year average of total expenditures in each category and awarding a fixed dollar 

amount based on their expenditures. 

State Allocations 

In FY05, the state allocated a total of $12,040,787 in federal and state funding for the provision 

of adult education services. Of this amount, $8,437,216 (70 percent) came from federal and 

$3,603,571 (30 percent) from state sources. 

Of the $12,040,787 allocated to local providers, state administrators earmarked $8,666,383 for 

distribution as base funding among the 44 local providers approved to offer instructional services 

(table C-1 and fig. C-1). An additional $1,123,279 also was distributed as categorical funding to 

support data collection, technology purchases, and One-Stop program services. The remaining 

$2,251,125 in state and federal funds—accounting for 19 percent of statewide resources for local 

providers—was released as performance funding. 

 Total Federal State 

 Resources Resources Resources

Local Provider Allocation $12,040,787 $8,437,216 $3,603,571

Percentage of Total 100.0% 100.0% 100.0%

Base Funding $8,666,383 $6,295,091 $2,371,292

Percentage of Total 72.0% 74.6% 65.8%

Performance Funding $2,251,125 $1,878,600 $372,525

Percentage of Total 18.7% 22.3% 10.3%

Categorical Fundinga $1,123,279 $263,525 $859,754

Percentage of Total 9.3% 3.1% 23.9%

SOURCE: Missouri Department of Elementary and Secondary Education, 2006. Funding data provided by state.

a

 State resources earmarked for specific functions by state.

Table C-1—

Missouri Adult Education Funding Allocated as Base or Performance Funding: FY05
SECTION C-1 

Overview of the State Adult Education Funding System 

— 119 — 

Organization of State Adult Education Services 

Adult education and literacy programs are overseen by the Division of Career Education, which 

has eight administrators, four of whom manage the state’s PBF system. Individuals interviewed 

for this report include: 

• State Director of Adult Education—Appointed in February 2005, the state 

director’s primary responsibility is to ensure that the state agency and local 

service providers comply with provisions of the Missouri state plan for adult 

education. This includes tracking expenditures of federal and state resources and 

supervising adult education staff in the administration of program activities. 

• Adult Education Supervisor—Employed by the state for 11 years and within the 

Division of Career Education for seven years, the adult education supervisor is 

responsible for evaluating service providers and administering the state’s 

invitation for bid procedures for the special education and English literacy and 

civics education programs. 

• Finance Supervisor—Joining the division in January 2001, the finance supervisor 

generates budgets and reimburses providers for program expenditures according 

to data submitted to the state. He also administers the state’s PBF formula, and is 

responsible for calculating providers’ resource eligibility based on their contact 

hours and performance outcomes. 

• Management Information System Supervisor—Hired in March 2004, this staff 

member designed and administers the state’s Adult Computer Enrollment System 

(ACES), which is used to track provider performance on accountability measures 

contained in AEFLA and to award performance funding based on program 

outcomes. 

SOURCE: Missouri Department of Elementary and Secondary Education, 2006. Funding data provided by state.

Figure C-1—

Percentage of Missouri Local Provider Allocation by Funding Type: FY05

72%

19%

9%

Base Funding

Performance Funding

Categorical Funding
SECTION C-1 

Overview of the State Adult Education Funding System 

— 120 — 

Other office staff include the supervisor of GED online and teacher certification programs; a 

supervisor of Family Literacy and Adult Education Coordination and Planning pilot projects; and 

two administrative secretaries. Researchers also interviewed the former state director of adult 

education and current director of employment training, who introduced PBF to the adult 

education system, and the division coordinator for the Adult Education and Employment 

Training Unit, who also helped spearhead the PBF adoption effort. 

Local Providers 

As illustrated in table C-2, services were offered in FY05 by 44 local providers, including 10 

colleges, 31 unified school districts, two community-based organizations, and the Department of 

Corrections. With the exception of three providers that were defunded and one provider added at 

the start of the FY03 funding cycle, the number of adult education programs has remained 

constant over time. 

Providers are geographically distributed throughout the state, with classes offered at more than 

700 locations. Within sites, adult participants have access to a range of program services, 

including ABE, ASE, and ESL instruction. 

Program enrollments have fluctuated since the introduction of PBF, although state administrators 

did not believe there is any relationship between the introduction of performance funding and 

FY01 FY02 FY03 FY04 FY05

Providers 46 46 44 44 44

Community Colleges 10 10 10 10 10

Unified School Districts 33 33 31 31 31

Community-based Organizations 2 2 2 2 2

Corrections 1 1 1 1 1

Participants 41,089 39,821 41,928 37,729 37,052

 Gender

Male 50% 53% 53% 55% 52%

Female 50% 47% 47% 45% 48%

 Race-Ethnicity

White 53% 54% 52% 52% 55%

Black 28% 28% 28% 28% 31%

Hispanic 12% 12% 13% 21% 15%

Asian 6% 6% 6% 6% 6%

American Indian/Alaskan Native 1% 1% 1% 1% 1%

SOURCE: U.S. Department of Education, Report to Congress on State Performance, program years 2000–01 to program years 

2004–05. http://www.doe.state.in.us/adulted/performancedata.html (last accessed June 18, 2007).

Table C-2—

Selected Characteristics of Missouri Adult Education Providers and Participants: FY01 to FY05
SECTION C-1 

Overview of the State Adult Education Funding System 

— 121 — 

learner participation. Roughly two-fifths (45 percent) of participants in FY05 were between the 

ages of 25 to 44 years, with those ages 19–24 accounting for another 28 percent. This supports 

provider reports indicating that they are serving a predominantly older population (fig. C-2). 

Adult participants in FY05 tended to be white (55 percent) or black (31 percent), with Hispanic 

(15 percent), Asian (6 percent), and American Indian/ Alaskan Native (1 percent) accounting for 

the remaining participants (table C-2).

SOURCE: U.S. Department of Education, Report to Congress on State Performance, program years 2000–01 to program years 

2004–05. http://www.doe.state.in.us/adulted/performancedata.html (last accessed June 18, 2007).

Figure C-2—

Ages of Missouri Adult Education Participants: FY05 

12%

28%

45%

12%

3%

16–18

19–24

25–44

45–59

60+

— 123 — 

The direction that contracts 

[administration is] going from state 

governments to local governments—

everybody seems to be looking very 

heavily at ... performance-based

contracting. It just seemed like a 

normal step for our adult education 

program. 

—Missouri State Director 

SECTION C-2: 

Rationale for System Development 

Unlike in the case of the higher education sector, where state legislators initially pressed for the 

introduction of PBF, the impetus for performance funding stemmed from the appointment of a 

new state director of adult education in November 2000. Believing that budgeting based on 

contact hours made it difficult for providers to predict their annual funding eligibility or to 

recover from sudden enrollment drops, the new director decided to reform the state’s existing 

contact-hour formula and, concurrently, to introduce a performance-funding component to the 

system. His plans also reflected a deeper concern that year-to-year fluctuations in system 

enrollments were undermining legislative efforts to provide stable resources for adult education. 

The adoption of AEFLA and federal efforts to improve statewide reporting on the core measures 

contained in the NRS also influenced his decision. Following his appointment, the director 

increased his contact with administrative staff of the U.S. Department of Education, Office of 

Vocational and Adult Education, who, in his estimation, underscored the importance of injecting 

accountability into state systems. Convinced that the existing state funding formula did not 

promote accountability at the local level, because student-contact hours and not outcomes were 

driving resource allocations, he came to believe that performance funding was one means of 

motivating local providers to pursue federal priorities. 

State administrators also were concerned about the 

possibility of jeopardizing state eligibility for federal 

incentive grants provided by the Workforce Investment Act

(WIA). To qualify for a grant award, Missouri needed to 

exceed performance levels for outcomes in three programs: 

AEFLA, vocational education (Carl D. Perkins Vocational 

and Technical Education Act of 1998), and WIA, Title I. Two 

of these programs were based in the Missouri Division of 

Career Education, and administrators feared that it would reflect poorly on the division if adult 

education providers failed to reach state performance goals. Accordingly, in the state plan 

submitted under AEFLA, administrators indicated that they were planning to set aside 10 percent 

of state funding for PBF purposes. 
SECTION C-2 

Rationale for System Development 

— 124 — 

State Adoption Process 

To build support for the adoption of PBF, the state director used the state’s Adult Education 

Report Card to focus providers’ attention on program performance. Developed in FY01, report 

cards on each local provider present learner demographic and outcome data in a graphical 

format, using statewide averages and percentages to document comparative results. This 

information was used to raise provider awareness that the state had room for improvement. 

Program directors’ dissatisfaction with the existing contact-hour formula also helped diminish 

statewide opposition. According to providers interviewed for this study, year-to-year fluctuations 

in program enrollments could have dramatic effects on program funding, since one year of low 

enrollment could force cuts in subsequent years, which, in turn, would make it difficult to 

generate additional contact hours to earn back resources. Although fear of the unknown sparked 

some resistance to PBF among providers, there was sufficient frustration with the current system 

to support change. 

Provider opposition also was muted somewhat because the PBF component was introduced at a 

time of increasing federal funding for adult education and because the state offered assurances 

that federal carryover funding would be infused into the system. Since initial modeling indicated 

that performance funds would supplement program resources in most agencies, providers did not 

believe PBF would put them at risk of losing resources. Over time, as the amount of core funding 

distributed based on contact hours has declined, provider enthusiasm for PBF has dampened with 

the realization that some will receive less funding unless they initiate programmatic change. 

Goals in Introducing Performance-based Funding 

The statewide committee charged with developing a fair and equitable formula recommended 

that the state set aside performance funds to support six objectives: 

1. Sustaining instructional effectiveness; 

2. Improving student performance and progress; 

3. Assisting customers and partners in reaching their goals; 

4. Providing incentives to promote professional development; 

5. Enhancing financial support of instructional planning and preparation; and 

6. Fulfilling federal accountability requirements. 
SECTION C-2 

Rationale for System Development 

— 125 — 

These goals underscored providers’ desire to improve the quality of adult education programs, 

while providing support services to assist staff in achieving performance objectives. Given the 

overlap between state performance measures and federal AEFLA core indicators, committee 

members also recognized the possible contribution that PBF could make in helping the state to 

attain its federal performance targets.

— 127 — 

There’s an old saying: “Pay me now

or pay me later.” You either go

through all this trouble up front, put

lots of effort into facilitation or you

just go ahead and do it ... and then

pay the price later. And I think what

impresses me to this day is [the field

committee] was the best example I’ve

ever seen of involvement, facilitation

that resulted in [their] support. 

—Missouri Division Coordinator 

SECTION C-3: 

Formula Design Process 

Creation of the state PBF funding formula was supervised by a committee of approximately 20 

program directors convened by the state director to represent the views of the field. Directors met 

periodically beginning in January 2001, with state administrators taking responsibility for 

establishing a timeline for the formula review process, surveying program directors to determine 

statewide attitudes, convening and facilitating committee meetings, and running funding 

simulation models. Committee findings were shared with program directors at each of two 

annual statewide directors’ meetings and through representatives participating in the 

development committee. 

Before convening the first committee meeting, the state director sent all participants a copy of 

the book Who Moved My Cheese? Written by Spencer Johnson, this motivational best seller 

describes strategies to help individuals cope with potentially difficult changes in their work 

setting. Anticipating that everyone would arrive prepared to delve into the meeting agenda, the 

state director was surprised to discover how deeply the book had affected people. Participants 

discussed personal situations and work experiences to such an extent that the first session nearly 

became a professional development seminar. As it turned out, these discussions helped team 

members relieve anxiety and build trust, enabling the group to tackle difficult formula 

development issues at an earlier stage than they might have otherwise. 

Buy-in among committee members would later pay huge 

dividends for the state in reducing provider resistance to the new 

formula. Near the end of the formula development process, as 

the new system was being readied for implementation, a small 

group of program directors expressed concern regarding PBF at 

a program directors’ retreat. Although state administrators were 

not present at the meeting, committee members who had 

participated in the formula development process allayed 

concerns, pointing out that they had been involved in the process 

and had worked to ensure that providers’ viewpoints were incorporated into the final formula. As 

one state official observed, “We didn’t have to defend it. The people involved in the process 

defended it, and that was fantastic.” 
SECTION C-3 

Formula Design Process 

— 128 — 

Specification of Funding Criteria: Core Funding 

Historically, contact hours had been the sole criterion used to allocate provider resources, and 

although program directors were willing to rework this formula, some members of the state 

advisory committee were hesitant to make substantial changes to the existing system. Following 

group deliberations, which included a review of funding systems in other states, committee 

members agreed to allocate 80 percent of combined federal and state resources as core funding at 

a rate of $3.00 per audited contact hour for FY02. 

Since core funding was intended to support basic program operations, the formula development 

committee conducted a survey of all directors in the state to rate their program expenditure 

priorities. Committee members used directors’ responses to questions regarding key expenditure 

categories, including teacher and support staff salaries, instructional supplies, and testing 

supplies and equipment, to define a program funding core. 

To address directors’ concerns about year-to-year fluctuations in student enrollment, committee 

members agreed to the use of a three-year rolling average to allocate half of the core funding 

resources. Accordingly, the new formula for allocating core funding implemented for FY02 was 

specified as: 

• Fifty percent based on a program’s total number of audited contact hours from the 

immediate previous year (FY01); and 

• Fifty percent based upon a program’s previous three-year average of its total 

audited contact hours (FY98, FY99, FY00). 

To shift programs gradually away from core funding and into PBF, it was proposed that contacthour rates be based on a sliding scale so that, by FY06, allocations would be equivalent to 80 

percent of FY02 funding. See table C-3 for the contact-hour rate and state total obligation for 

FY02–06. 

 Fiscal Year Contact-Hour Rate Total Obligation 

2002 $3.00 (baseline FY02) $10,000,000

2003 $2.85 (95% of FY02) $9,500,000

2004 $2.70 (90% of FY02) $9,000,000

2005 $2.55 (85% of FY02) $8,500,000

2006 $2.40 (80% of FY02) $8,000,000

a Changes in resources has negated the FY06 proposed rate.

Table C-3—

Missouri Contact-Hour Rate and Total State Obligation: FY02 to FY06a
SECTION C-3 

Formula Design Process 

— 129 — 

The formula that we have evolved is

pretty straightforward. It’s simple in

design. It takes data that is already

there—the contact hours, the

progression, the GED award, ... the

matching for employment and

retention. So we’re using systems that

are already in place. 

—Missouri State Director 

Although the state has reduced its contact-hour rate over time, unanticipated changes in available 

federal and state resources have prevented the state from adhering to this schedule. Recently, 

rates for FY06 were fixed at $2.70 for general population and a rate of $2.20 for institutionalized 

learners. However, the overall amount of resources earmarked for core funding has kept pace 

with committee plans. As evidence, in FY05 the state allocated $8,666,383 in core funding, 

equivalent to roughly 72 percent of total adult education expenditures. 

Recognizing that PBF could cause structural changes, the committee also conducted a second 

survey in FY01 asking providers to identify the types of support they would need to incorporate 

performance funding into their institutional budgets. Locals requested additional financial 

assistance in three areas: 

• Data collection—support in building an electronic adult data collection system, 

including resources for data entry staff salaries, hardware and software upgrades, 

and materials and supplies; 

• Marketing—support in publicizing adult education programs and services, 

including resources for supplies, activities, and personnel; and 

• Technology—support for improving instructional technology, including resources 

for purchasing instructional software, operational software, and other materials 

and supplies. 

Accordingly, the state director earmarked just over 10 

percent of FY02 provider funds for support in these areas, 

with data collection and marketing resources based on a 

rolling three-year average of a program’s enrollment, and 

technology-based on a rolling three-year average of a 

program’s total expenditures. Therefore, of adult education 

funds allocated in FY02, 3.6 percent was earmarked for data 

collection, 2.2 percent for marketing, and 4.6 percent for 

technology. Providers also were required to develop a local plan, containing a proposed budget 

and expenditure narrative, to qualify for resources. The availability of these funds varies from 

year to year; for example, in FY03 the state dropped the marketing component from the funding 

formula to compensate for reductions in funding. 

To address other identified concerns, in FY03 the state earmarked funding to support programs 

offered in One-Stop Career Centers and created an Expansion and Enhancement grant program 

to support agencies in improving services. These grants were relatively modest, with funding for 
SECTION C-3 

Formula Design Process 

— 130 — 

One-Stop programs amounting to just 0.67 percent and Expansion and Enhancement grants to 

3.9 percent of total FY03 funding. 

Conducting a statewide survey added to the formula development process, and it was suggested 

that providing opportunities for director feedback offered a number of advantages. Soliciting 

input strengthened formula design by ensuring that all providers were able to share their insights 

and concerns with the formula drafting committee. Involving providers also helped promote buyin and short-circuit complaints. With additional funding, providers would not be justified in 

either complaining that the state was instituting PBF as an unfunded mandate or fighting 

categorical spending requirements, since they themselves had identified program priorities. 

Specification of Funding Criteria: Performance Funding 

Committee members identified two measures of student progress for performance funding that 

were most closely aligned with state objectives: GED attainment and literacy level advancement. 

Since these measures were included in AEFLA and providers had been collecting information on 

them since FY01, their selection meant that the state did not have to alter existing data collection 

procedures to generate local program performance eligibility. 

To support attainment of formula objectives, the statewide committee recommended that a 

portion of providers’ performance funds be set aside to support priorities identified through the 

FY01 statewide survey. Consequently, committee members recommended that a minimum of 25 

percent of each program’s local performance allocation be reserved for professional development 

and a minimum of 10 percent for instructional stipends for planning and preparation. This 

requirement was dropped in FY03 due to complaints from program directors, particularly those 

with small performance allocations who objected to being told how to spend their resources. 

Recognizing that lower-level learners often take longer to advance a level, committee members 

adopted a graduated funding scale, with supplemental resources attached to students making 

gains at the lowest learning levels.12 To establish rates, the state modeled resource allocations 

based on the number of outcomes recorded by providers in FY01 and the amount of performance 

funding available for distribution in FY02. Accordingly, provider performance funding 

eligibilities in FY02 were estimated using the calculator in table C-4. 

 

12 AEL levels of completion correspond to the six functional academic performance levels identified for ABE and ASE learners 

as specified in the NRS. Learners who complete an ASE high level are counted as attaining a GED credential. ESL levels also 

correspond to the six functional academic performance levels specified in the NRS. Learners who complete the Advanced ESL 

level are included in AEL levels 3–5 for funding purposes. 
SECTION C-3 

Formula Design Process 

— 131 — 

In the year following PBF introduction, committee members reconvened to consider whether 

outcome rates for ESL students required additional resources, since providers believed that ESL 

services were relatively more expensive to provide. As part of this exercise, state administrators 

compared the number of contact hours recorded for ESL students to those for ABE and ASE 

participants. Based on a review of the relative effort required to achieve a learning gain, 

committee members recommended development of a separate rate for ESL students. This new 

rate was added to the funding calculator in FY05, supported with resources transferred from an 

existing state-funded ESL grant program. Reimbursement rates for non-ESL students also were 

recalibrated. Since FY05, local agencies have qualified for PBF based on the number of students 

advancing in ESL as well as Adult Education and Literacy (AEL) and GED (table C-5). 

Assessing Formula Effect 

To assess potential effects on local providers, state administrators modeled how program 

allocations would change under different resource assumptions prior to formula rollout in FY02. 

These calculations were shared with committee members to provide context for their discussions. 

For example, to initiate discussions about an appropriate level of support for core funding, the 

Levels of Advancement Amount Students Advancing Total

AELa

 Level 1 Completion $150 0 $ ---

AEL Level 2 Completion $100 0 $ ---

AEL Level 3–5 Completion $50 0 $ ---

GED $150 0 $ ---

a Adult Education and Literacy

Table C-4—

Missouri Levels of Advancement Performance Calculator Used to Award Performance-based Funding: FY02

Levels of Advancement Amount Students Advancing Total

AELa

 Level 1 Completion $200 0 $ ---

AEL Level 2 Completion $150 0 $ ---

AEL Level 3–5 Completion $75 0 $ ---

GED $150 0 $ ---

ESL Level 1 Completion $250 0 $ ---

ESL Level 2 Completion $175 0 $ ---

ESL Level 3–5 Completion $100 0 $ ---

a Adult Education and Literacy

Table C-5—

Missouri Levels of Advancement Performance Calculator Used to Award Performance-based Funding: FY05
SECTION C-3 

Formula Design Process 

— 132 — 

state developed a budget simulation assuming that 70 percent of total state funding was allocated 

based on contact hours, using a rate of $3.10 per hour. Differential rates for performance 

outcomes also were proposed. Subsequent iterations of the core and performance components 

eventually resulted in adoption of the FY02 funding model. 

Final PBF allocation models suggested that most providers would realize an initial increase in 

their total resource eligibility in the first year of performance funding. This was due, in part, to 

the committee’s decision to distribute most federal and state resources as core funding, partially 

to categorical funding for data collection, marketing, and technology functions, and due, in part, 

to the release of federal carryover funding. Consequently, at the close of the formula 

development process, state administrators and committee members believed that they had created 

a realistic formula that would account for the diverse needs of providers throughout the state. 
— 133 — 

SECTION C-4: 

Implementation Process 

Missouri fully implemented performance funding in the 2001–02 program year (FY02). Because 

the introduction of performance funding coincided with a slated infusion of federal and state 

resources into the system—statewide funding climbed from $11.6 million to $13.2 million with 

formula rollout—the state did not attempt to insulate providers from shifts in their resource 

eligibility. This may have been because no action was needed: Simulations indicated that there 

would be few losers, and those projected to experience a drop in resources were generally larger 

providers with budgets big enough to offset any loss. 

Initial effects also may have been small because the state opted to phase in PBF gradually so that 

local providers would have time to adjust to the new formula. When introduced in FY02, PBF 

resources were limited to roughly 9 percent of total funding. Over time, as the state has cut back 

on its contact-hour rate while increasing the value of performance outcomes, the proportion of 

funding allocated based on performance has risen. In FY05, performance-related allocations 

amounted to nearly 19 percent of total adult education funding. 

Provider Reporting Capacity 

In FY01, to support providers in reporting federally mandated accountability data, the state 

rolled out ACES, its new management information system. Each provider was supplied with a 

state-developed database to be used to collect and administer program performance data. Each 

quarter, providers sent disk copies of their files to the state for use in NRS reporting. To support 

providers in using the ACES system, the state held a number of statewide technical assistance 

sessions, provided site-level support, and offered individual consultations via telephone. 

As in other states, not all providers were able to collect comprehensive data on student outcomes 

initially, most often because of technical issues related to testing. For example, the Missouri 

Department of Corrections had historically used its own assessment system to measure inmates’ 

education attainment. Although the prison system was required to shift to using the Test for 

Adult Basic Education (TABE) to assess inmates’ level attainment for AEFLA reporting 

purposes, this transition took time. Consequently, not all FY01 outcomes were recorded. After 

the prison system completed its transition to the new assessment system in FY02, its 

performance funding eligibility more than doubled in subsequent program years. 
SECTION C-4 

Implementation Process 

— 134 — 

It was observed that local providers have had some difficulty aggregating learner outcomes to fit 

the different attainment categories used in the funding calculator (i.e., AEL-ESL level 1, AELESL level 2, AEL-ESL levels 3–5). While providers understand the difference between a level 1 

and level 2 progression, provider staff have difficulty aggregating data to report on levels 3–5 

progressions. It was suggested that it would be simpler to have providers report learner 

progression for each level individually and then combine these data at the state level for formula 

purposes.

Data Quality 

Adult education providers are audited every year as part of the 

state’s official school review process. To validate adult 

education program data, the local auditor completes a form 

that includes a reference to contact hours reported by the 

agency. While auditors make a conscientious effort to review 

existing data, the audit is only a spot check that traces back 

through a sample of time slips to see if contact-hour reports are accurate. Programs also are 

required to submit an annual written evaluation of program performance. 

Additionally, every five years, as part of the secondary accreditation cycle, adult education 

providers housed in secondary districts are subjected to an intensive audit, which includes a 

review of contact-hour and performance-outcome records. Providers located in postsecondary 

institutions also are reviewed on a five-year cycle. State administrators report that these audits 

suggest that there is close alignment between reported data and program records, indicating that 

providers are maintaining valid contact and performance data in nonaccreditation years. 

Management Information Systems 

Missouri’s shift to performance funding occurred in FY02, the year following the state’s rollout 

of new federal reporting requirements. Although the current state information system supervisor 

was not present at the time the system was introduced, during interviews he suggested that the 

adoption of PBF did not add any additional hardware or software costs to formula adoption at the 

state level, because state performance measures simply duplicated those contained in AEFLA. 

In FY04, the state hired the current information system supervisor to update the ACES database. 

Until this time, the ACES data system was not designed to support agencies in undertaking 

program improvement efforts. Although providers collected and reported data on outcomes 

specified in the NRS core measures, local staff could not disaggregate their data to associate 

learner progressions with specific programs or teachers as needed for the ACES database. Unless 

providers were maintaining dual information systems (i.e., ACES along with a locally developed 

Having a good solid data collection

system and the ability to let them

[local program providers] see the

data, whether electronically or on

paper, that’s the key thing. 

—Former State Director of Adult 

Education 
SECTION C-4 

Implementation Process 

— 135 — 

A couple years ago, we were finally

... getting good reports out of ACES

and ... we were going through this

exercise and [directors] thought, “Oh

no, dummy data again.” But when we

gave them the real data, it was quiet.

They were really looking and they

were really starting to analyze their

own data. 

—Former Missouri State Director 

database), they were unable to discern how they were performing in different program areas. The 

updated ACES system enables providers to produce more detailed reports and analyze their data, 

so that they can initiate improvement efforts with underperforming programs or teachers.

Although reprogramming ACES entailed making software changes funded by the state, these 

changes were intended to provide program directors with better information for program 

management purposes. While some of this cost could be related to the operation of the state’s 

PBF system, state administrators believed it likely that database changes would have occurred 

irrespective of whether or not the state adopted performance funding to allocate resources. 

During case study interviews, however, one program director suggested that the rollout of PBF 

had placed an additional burden on his administrative aides and taken up a significant amount of 

his own time. Although much of this burden was associated with learning the new ACES 

software and compiling data required for the NRS, he and his staff also spent considerable time 

reviewing contact hours and tracking individual learning outcomes to qualify for performance 

funding. In his estimation, his focus on accurately quantifying fundable outcomes would not 

have occurred had the emphasis been on collecting and reporting federal data for NRS purposes 

alone. 

Although the state did provide additional funding in categorical areas to support providers in 

collecting data, these resources were based on program enrollment totals averaged over a threeyear period, with funding distributed based on nine different program sizes. For example, 

programs with 300–499 students qualified for an allocation of $2,500, while those with 1,500–

2,499 students qualified for $20,000. This meant that smaller programs were unable to hire even 

a part-time administrative aide to collect data, forcing existing staff to take on additional duties. 

Technical Assistance 

To support directors in using data to improve programs, the 

state hired an education consulting firm, which had helped 

develop the federal NRS, to provide workshops on using data 

to make program improvement decisions. Initial meetings 

were structured around fictitious data: Directors were supplied 

information and given strategies for interpreting it. Later, 

when program level reports finally were available from the 

state data system, directors were again convened, but this time 

given their own program data. 
SECTION C-4 

Implementation Process 

— 136 — 

Although the state provided technical assistance to help providers understand the ACES database 

and how to use data for program improvement, not all directors felt supported in the rollout of 

the new funding system. The sudden focus on performance outcomes meant that many directors 

had to immerse themselves in program data to understand why students were not performing and 

whether low outcomes were due to a failure on the part of faculty to follow appropriate testing 

protocols, instructional quality, program curricula, or some other factor. While shifting to 

performance funding was credited with helping motivate providers to improve program services, 

it also uncovered weaknesses in the state system that required closer scrutiny. That is, some 

directors reported that the state could have provided additional training in how to interpret and 

use data to make more informed decisions. 
— 137 — 

SECTION C-5: 

Effects of Performance-based Funding 

During case study interviews, both state administrators and local provider staff commented on 

the effect that PBF was having on adult education program services. Although focus group 

participants ascribed both positive and negative outcomes to the adoption of performance 

funding, few if any data exist to substantiate these claims. In many cases, participant 

observations were based on general perceptions of program operations or extrapolations of a 

single experience. Some participants also had difficulty differentiating programmatic effects due 

to the adoption of PBF from those associated with the state’s implementation of AEFLA. 

Consequently, readers should be cautious in drawing conclusions about the effects of PBF on 

state adult education systems. 

Contributions to Statewide Performance 

As originally conceived, PBF in Missouri was introduced to promote more effective adult 

education services, while simultaneously improving instructor professionalism and the state’s 

ability to respond to federal AEFLA reporting requirements. According to the state and local 

directors, the adoption of PBF has contributed to achieving state goals, and, in particular, to 

motivating providers to use data to improve their instructional effectiveness, to achieving student 

performance and instructor professional development goals, and to fulfilling federal 

accountability requirements. Specific benefits included:

• Improving the Accuracy of Adult Education Data—Linking resources to 

performance has motivated providers to align their assessment and data collection 

systems with state criteria and to do so more rapidly than if no funding were 

attached. Program directors reported that they were more willing to review data 

collection processes because improved reporting capacity translated into 

additional resources. In particular, it appears that performance funding helped 

improve data reliability in Missouri by ensuring that all providers conscientiously 

followed accepted state testing protocols when measuring learner progress. 

Evidence of changes in testing protocols is illustrated in data from larger 

programs, including Kansas City, St. Louis City, and the Department of 

Corrections, all of which might be expected to face greater challenges in 

standardizing assessment in a large number of sites. These three agencies 

registered substantial gains in their performance eligibility between FY03 to 
SECTION C-5 

Effects of Performance-based Funding 

— 138 — 

If a teacher is not doing well and not

getting students to progress, then the

local director knows they need to look

at that. Because if they’re not

performing, or progressing from one

level to another, for that ... factor in

the formula, they’re going to get less.

—Missouri State Director 

FY05, which, because funding lags two years behind, correlates with the 

introduction of new PBF measures. For example, performance eligibility in Saint 

Louis City, which remained constant between FY02 and FY03, then jumped 33 

percent in FY04 and another 20 percent between FY04 and FY05, presumably 

because local providers gained experience administering new testing protocols. 

• Building a Culture of Data—Compensating providers for generating learner 

outcomes has helped to focus directors’ attention on program performance, and, 

particularly on making connections between specific instructors and learner 

outcomes. Before PBF, program directors reported that few of their colleagues 

“owned” their data; that is, although information may have been collected, few 

had an incentive to use it, since funding was based on contact hours. 

Linking funding to performance (and subsequently 

updating ACES data fields) helped motivate 

directors to pay greater attention to program data, 

for example, encouraging them to review data 

system reports to identify areas where they could 

increase their funding. According to the state 

division coordinator, although only 10 percent of 

funds were allocated based on performance, providers treat the system as though 

90 percent of their funding were at risk, so program directors today are more 

responsible for, and thus responsive to, their data.

• Professionalizing the Teaching Force—Program directors believe that the shift to 

performance funding has helped improve instructor quality, in part because of the 

emphasis placed by state administrators on training local staff. To support the 

state goal of encouraging professional development, the state funding formula 

initially stipulated that program directors earmark a minimum of 10 percent of 

their programs’ performance allocation for instructor stipends for planning and 

preparation and 25 percent of PBF funds to support professional development 

activities. This funding can be used to ensure that teachers continue to improve 

their instructional skills. 

As one director observed, “There are workshops that have to be attended, there 

are a number of development hours they have to do every year to maintain that 

certification. ... We really strive to make sure our adult education teachers have 

the ability and the background and the training to be an adult teacher.” Another 

director reported using performance resources to provide financial awards to staff 

exhibiting high performance, while another used them to reward staff with trips to 
SECTION C-5 

Effects of Performance-based Funding 

— 139 — 

seminars and workshops, as well as to purchase materials and equipment for 

classrooms. 

The addition of targeted funding for marketing services also enabled providers to 

sell their programs more effectively to potential clients, which program directors 

reported helped them to attract higher quality staff. Program directors also 

reported that the introduction of PBF helped to remove teachers unwilling to be 

held accountable for their performance from the classroom. Although program 

directors believed PBF has contributed to teacher quality, the state has not 

conducted any formal studies to assess the relationship of the professional 

development expenditure requirements in the funding formula to changes in 

instructor quality over time. 

Drawbacks 

Interviews with state administrators and local program directors indicate that PBF has been 

effectively institutionalized in Missouri. However, not all providers initially supported the 

transition to the new funding system, nor do they enthusiastically support all aspects of the 

current formula. Drawbacks associated with implementing PBF include: 

• Lag Between Program Data and Funding Eligibility—Providers begin budgeting 

for the coming fiscal year in the early spring, before final fiscal year data on 

contact hours and performance outcomes are available. To permit the state to 

budget provider resources more accurately, the state agency calculates provider 

eligibilities for the upcoming fiscal year using data from the preceding year (i.e., 

data for FY04 is used to estimate funding for FY06). 

The delay between when programs post outcomes and receive their resources can 

mean that programs may have difficulty expanding programs to new sites, since 

providers do not realize any benefit for their successes until two years after 

posting their data. As the state fiscal supervisor explained, because a program 

experiencing substantial enrollment growth will not be rewarded for two years, 

the program is forced to carry its increased costs for one year until state 

allocations catch up with its expanded needs. The state has attempted to address 

this deficiency by offering one-time Expansion and Enhancement grants that 

providers can use to address site-specific issues related to program growth. 

• Administrative Burden on Instructional Staff—Tracking learner outcomes can 

create additional work for instructors, many of whom are more concerned with 

teaching than with keeping detailed records. Under the old system, teachers were 

simply required to track contact hours for funding purposes; with the new system, 
SECTION C-5 

Effects of Performance-based Funding 

— 140 — 

instructors must understand the content of the NRS performance measures and 

testing protocols, as well as monitor outcomes for a range of student activities. 

For example, although providers receive contact-hour funding for all students 

enrolled, they only are eligible for performance funding for participants meeting 

the 12-hour enrollment threshold. This has required staff to pay closer attention to 

tracking student participation. 

According to one program director, the paperwork associated with collecting 

accurate data has made it difficult to retain some experienced teachers, many of 

whom are part-time employees who would prefer to spend their limited time in 

student-centered activities. Although a number of directors shared this view, no 

empirical studies exist to quantify the extent to which the adoption of PBF has 

actually affected statewide staffing. 

• Insufficient Compensation for Learners With Special Needs—Not all learners are 

capable of making annual progress on state performance measures, meaning that 

instructors may have difficulty achieving measurable gains for special-needs 

students, particularly those requiring remedial instruction or with documented 

learning disabilities. In the words of one director, this “enrages some of the 

teachers because they are experienced enough to know that 60 hours of on-task 

work in a particular area may not remediate the student to the point that they may 

move up a grade level.” 

Although contact-hour funding helps offset some instructional costs, while 

ensuring that providers have an incentive to serve all students, other than the ESL 

component, the state formula does not compensate providers for high-cost 

learners. According to state administrators, the formula is intended to provide 

sufficient resources, on average, to educate all types of students. Administrators 

also have provided professional development to instructors to assist them in 

identifying and referring learners with special needs, as well as tools to help them 

improve instruction. 
SECTION C-5 

Effects of Performance-based Funding 

— 141 — 

Our focus is what’s going to be the

best for the customer: get them a job,

get them into postsecondary

education, get that GED, help them

with their kids, help them learn

English, and be part of society. It

seems to me that’s kind of where our

focus ought to be and not so much

how long they’re sitting in a chair

everyday or how many hours a week

they’re participating. What are they

able to do and how are we really

helping them? 

—Missouri State Director

Effect of PBF on Adult Education Providers 

Before the adoption of PBF in FY02, 46 providers were 

eligible to receive federal and state adult education funding. 

In the year following the adoption of PBF, which coincided 

with a new three-year funding cycle for adult education 

(FY03–FY05), three providers—Ava, Boonville R-I, and 

Fulton—were dropped and one program, Susanna Wesley, 

was added to the system. Of the programs no longer 

participating, two were small agencies that were subsumed 

into a neighboring provider, and one closed because it was 

unable to deliver its proposed program to inmates in state 

corrections facilities. 

According to the state director, with the exception of one program that was unable to generate 

sufficient contact hours or performance outcomes to qualify for resources, the introduction of 

PBF was not directly responsible for the loss of any adult education programs. Rather, the loss of 

programs was blamed on problems with the invitation to bid process, in which providers had 

problems with or forgot to submit required forms and, consequently, were declared ineligible for 

program funding in the new funding cycle. 

Fiscal Eligibility 

At the outset of performance funding in FY02, 38 of 46 providers realized funding gains over 

their contact-hour allotment in FY01 (see table C-6). For many smaller providers, resource 

increases were primarily due to the addition of state categorical grants for data collection, 

marketing, and technology, rather than to the actual introduction of performance funding. 

Although introduced in FY02, the full effects of PBF were not felt for at least two years due to 

the time lag between data collection and the application of data to provider allocations and the 

small amount of funding allocated based on performance criteria. 

Generally, community colleges and the Department of Corrections benefited the most under the 

new funding system. According to the state director, this was because community colleges were 

more successful in recruiting individuals preparing for their GED, while the corrections agency 

was able to control attendance and to measure inmate progress. In FY05, the state lowered its 

contact-hour rate for inmates from $2.70 to $2.20 to adjust for this relative advantage. 
SECTION C-5 

Effects of Performance-based Funding 

— 142 — 

Percent Percent

FY01 FY02 Change FY03 FY04 FY05 Change

Expenditure PBF FY01 to PBF PBF PBF FY01 to

AELa

 Program (Prior to PBF) Expenditure FY02 Expenditure Expenditure Expenditure FY05

Statewide Total $11,337,169 $12,394,697 9% $11,065,156 $10,882,354 $12,040,787 6%

Nevadab 454,979 382,810 -16% 220,317 98,586 105,023 -77%

Mexico 22,630 15,815 -30% 14,758 11,589 15,226 -33%

Poplar Bluff 313,928 372,197 19% 322,481 267,550 247,700 -21%

Sedalia Community College 234,021 202,043 -14% 199,225 179,805 192,399 -18%

Crowder College 230,064 232,357 1% 204,875 201,628 202,092 -12%

Rockwood 83,089 82,173 -1% 72,410 60,985 74,705 -10%

Moberly Community College 30,292 28,511 -6% 27,952 25,172 27,684 -9%

Parkway 915,275 895,044 -2% 804,405 788,125 839,890 -8%

University City 327,081 357,385 9% 303,177 295,969 302,679 -7%

Sikeston 127,165 168,903 33% 156,095 113,905 121,998 -4%

Cape Girardeau 152,927 205,968 35% 134,423 140,407 148,559 -3%

St. Joseph 470,630 417,633 -11% 439,250 394,756 459,908 -2%

Hannibal 71259 59198 -17% 77562 58103 71204 0%

St. Louis City 1,527,864 1,644,221 8% 1,514,196 1,506,349 1,529,543 0%

Waynesville 79,812 99,073 24% 81,613 63,655 81,146 2%

Della Lamb 404,342 435,916 8% 373,740 398,409 417,159 3%

Ritenour 87,899 100,669 15% 80,785 78,142 97,025 10%

Kansas City 968,956 1,018,129 5% 889,155 973,148 1,071,928 11%

Bonne Terre 383,198 392,090 2% 436,691 339,297 425,143 11%

Vandalia 505,279 604,370 20% 483,445 539,369 569,282 13%

North Kansas City 254,298 299,145 18% 275,803 262,165 289,472 14%

Carthage 65,010 72,695 12% 64,215 64,892 74,338 14%

NW Missouri State University 170,221 204,749 20% 179,710 190,688 204,144 20%

Blue Springs 27,931 35,394 27% 30,709 27,374 33,750 21%

Kirksville 44,641 71,583 60% 53,500 40,180 54,344 22%

Columbia 259,035 262,298 1% 241,407 285,190 327,863 27%

DOC 1,313,677 1,345,397 2% 1,446,683 1,464,158 1,666,641 27%

Rolla 120,224 154,652 29% 139,743 142,170 153,279 27%

Springfield Community College 365,785 450,834 23% 436,606 416,510 471,542 29%

Independence 183,764 253,101 38% 197,148 227,298 253,503 38%

Jefferson College 103,599 155,987 51% 102,413 116,372 143,023 38%

Joplin 118982 137180 15% 119541 139267 165657 39%

St. Louis Community College 80,440 110,662 38% 86,508 94,549 112,685 40%

Jefferson City 85,368 125,952 48% 108,806 111,212 123,405 45%

St. Charles Community College 151,125 171,923 14% 200,416 168,781 243,743 61%

East Central 76,961 101,619 32% 95,260 99,044 125,793 63%

West Plains 106,464 171,941 62% 195,941 202,915 195,176 83%

Macon 38,789 66,524 72% 54,517 61,579 72,387 87%

Linn Community College 1,696 7,569 346% 3,589 3,958 3,711 119%

Camdenton 19,484 25,853 33% 24,873 26,896 42,761 119%

Lebanon 38,414 45,801 19% 66,576 38,898 88,424 130%

Houston 16,015 34,348 114% 31,570 24,883 48,535 203%

Trenton Community College 15,282 45,770 200% 73,067 40,856 62,986 312%

Avac 32,637 43,307 33% — — — —

Boonvilled 3,549 12,143 242% — — — —

Fultone 253,088 273,765 8% — — — —

Susanna Wesleyf — — — — 97,570 83,332 —

SOURCE: Missouri Department of Elementary and Secondary Education, 2006. Funding data provided by state.

Note: — Not applicable.

a Adult Education and Literacy; corresponds to six functional academic performance levels for ABE and ASE learners.

b In FY03 the federal government closed a training camp located in Nevada, which severely impacted program services.

c Consolidated with West Plains in FY03.

dConsolidated with Sedilla in FY03.

e Program unable to deliver proposed programs; resources subsumed by Department of Corrections.

f New program added in FY03.

Table C-6—

Missouri Adult Education Expenditures Prior to and After PBF and Percentage Change, by Provider: FY01 

to FY05
SECTION C-5 

Effects of Performance-based Funding 

— 143 — 

With contact hours, the [provider’s]

attitude is, ‘We got someone in here,

we’re collecting the money.’ So the

question is: What is management

going to pay for, what do they want?

If you’re looking at retention, then

you go by contact hours. If you’re

going to want performance, then you

use the performance-based funding. 

—Missouri Fiscal Manager 

It appears that enrollment changes may have had a greater effect on provider eligibility than 

PBF. As indicated in table C-7, eight of 12 providers losing adult education funding between 

FY02 and FY05 also suffered above-average declines in core funding, indicating these programs 

were either experiencing relatively large enrollment declines or eliminating a larger-than-average 

number of duplicate enrollments as they improved the quality of their local data systems. In 

general, with the exception of 11 programs, all providers lost core funding between FY02 to 

FY05, as might be expected given the declining value attached to contact hours in the formula 

and improvements in providers’ ability to track students and eliminate duplications. 

In contrast, the contribution of PBF to providers’ expenditures took on greater importance as 

state performance rates increased and program staff simultaneously improved their assessment 

protocols and instructional approaches. As evident in table C-8, expenditures for provider 

performance outcomes was initially held constant in FY02 and FY03, because funding was based 

on assessment results collected prior to or concurrent with the introduction of new AEFLA

reporting requirements. As performance criteria were allowed to vary, differences in outcomes 

across providers became more evident. 

Performance-based funding changes appear particularly 

pronounced in some larger providers, such as Kansas City, 

which increased its performance eligibility by 293 percent 

between FY02 and FY05. According to the state director, such 

changes likely occurred because providers became more adept 

at collecting data on student progression; even so, by FY05, 

roughly half of the providers had gained performance funding 

relative to the statewide average, with gains slightly more 

pronounced in larger programs. Unfortunately, due to the introduction of a revamped ACES 

database in FY04, state staff were able to provide only total program allocation figures, and not 

raw performance data, for individual providers, making it impossible to control for the effect of 

learner population characteristics on program outcomes. 
SECTION C-5 

Effects of Performance-based Funding 

— 144 — 

FY02 to FY02 to

AEL Programb FY02 FY03 2004 2005 FY03 FY05

Statewide Total $10,003,559 $8,937,259 $8,641,407 $8,666,383 -11% -13%

Nevadac 322,109 289,898 80,961 59,064 -10% -82%

Sikeston 130,214 117,192 84,355 73,423 -10% -44%

Kirksville 41,166 28,870 25,490 24,591 -30% -40%

Poplar Bluff 292,980 263,682 213,222 177,133 -10% -40%

Cape Girardeau 147,507 132,756 105,340 99,342 -10% -33%

Waynesville 75,128 67,615 55,355 58,391 -10% -22%

Rockwood 60,934 54,841 45,610 48,055 -10% -21%

University City 310,940 297,846 259,594 247,804 -4% -20%

North Kansas C. 230,505 207,455 199,865 188,747 -10% -18%

Jefferson College 101,236 91,112 83,507 84,022 -10% -17%

Jefferson City 77,101 69,391 64,037 64,430 -10% -16%

Vandalia 532,569 446,834 462,169 445,869 -16% -16%

Parkway 774332 696899 675252 652008 -10% -16%

Mexico 9,326 8,393 7,114 7,976 -10% -14%

St. Louis City 1,447,383 1,302,644 1,266,857 1,246,581 -10% -14%

Crowder College 166,951 151,125 148,428 144,817 -9% -13%

Sedalia Community College 131,831 122,325 121,155 116,939 -7% -11%

Linn Community College 2,380 1,827 2,108 2,136 -23% -10%

East Central Community College 64,618 58,156 61,156 58,581 -10% -9%

Ritenour 70,008 63,007 63,717 63,750 -10% -9%

Blue Springs 21,399 19,259 19,199 19,625 -10% -8%

St. Joseph 328,044 295,239 290,976 301,765 -10% -8%

Macon 47,529 42,776 48,154 43,787 -10% -8%

Kansas City 923,634 831,271 845,524 858,236 -10% -7%

Rolla 111,791 100,612 112,760 104,127 -10% -7%

DOC 1,195,558 1,237,741 1,125,083 1,142,966 4% -4%

Springfield Community College 340,117 306,105 326,028 327,348 -10% -4%

NW Missouri State University 166,785 150,107 158,763 160,944 -10% -4%

St. Louis Community College 69,711 62,738 63,849 67,510 -10% -3%

Della Lamb 361,878 325,690 349,634 356,009 -10% -2%

Carthage 52,406 47,165 46,092 52,013 -10% -1%

Independence 170856 153771 165715 169784 -10% -1%

Bonne Terre 297,314 267,582 255,622 299,011 -10% 1%

Joplin 92,335 83,101 89,635 95,668 -10% 4%

Moberly Community College 17,272 15,545 14,772 18,046 -10% 4%

Camdenton 15,714 14,143 14,956 17,713 -10% 13%

West Plains 123,340 137,212 166,790 140,336 11% 14%

Columbia 195,792 176,213 209,562 234,002 -10% 20%

Hannibal 36,697 33,027 41,316 45,867 -10% 25%

St. Charles Community College 107,872 97,084 105,453 143,310 -10% 33%

Trenton Community College 26,319 23,687 24,781 36,136 -10% 37%

Houston 21,859 19,673 17,933 32,715 -10% 50%

Lebanon 28,501 25,650 33,198 68,149 -10% 139%

Avad 29,118 — — — — —

Boonvillee 5,754 — — — — —

Fultonf 226,746 — — — — —

Susanna Wesleyg

— — 90,320 67,657 — —

SOURCE: Missouri Department of Elementary and Secondary Education, 2006. Funding data provided by state.

Note: — Not applicable.

a Core expenditures include resources provided based on a provider’s total audited contact hours.

b Adult Education and Literacy; corresponds to six functional academic performance levels for ABE and ASE learners.

c In FY03 the federal government closed a training camp located in Nevada, which severely impacted program services.

dConsolidated with West Plains in FY03.

e Consolidated with Sedilla in FY03.

f

Program unable to deliver proposed programs; resources subsumed by Department of Corrections.

g New program added in FY03.

Percentage Change

Core Expenditures

Table C-7—

Missouri Adult Education Core Expendituresa to Providers: FY02 to FY05
SECTION C-5 

Effects of Performance-based Funding 

— 145 — 

FY04 to FY02 to

AEL Programa FY02 FY03 FY04 FY05 FY05 FY05

Statewide Total $1,135,050 $1,118,657 $1,383,375 $2,243,450 62.2% 97.7%

Linn Community College 450 450 350 75 -78.6% -83.3%

Nevadab 36,350 36,350 5,100 19,825 288.7% -45.5%

Crowder College 30,750 30,750 36,200 28,775 -20.5% -6.4%

Independence 43,200 43,200 32,100 47,000 46.4% 8.8%

Waynesville 8,900 8,900 2,550 10,425 308.8% 17.1%

Moberly Community College 4,000 4,000 3,000 4,950 65.0% 23.8%

Poplar Bluff 28,150 28,150 22,875 38,350 67.7% 36.2%

North Kansas City 30,050 30,050 32,300 43,025 33.2% 43.2%

Sedalia Community College 27,750 28,900 31,650 40,100 26.7% 44.5%

Cape Girardeau 20,500 20,500 16,100 30,300 88.2% 47.8%

Houston 3,750 3,750 3,950 5,600 41.8% 49.3%

Ritenour 9,200 9,200 6,175 14,025 127.1% 52.4%

Jefferson College 25400 25400 16525 39250 137.5% 54.5%

NW Missouri State University 20,100 20,100 19,925 31,200 56.6% 55.2%

Bonne Terre 57,050 57,050 53,675 89,025 65.9% 56.0%

St. Louis City 127,350 127,350 169,625 203,075 19.7% 59.5%

St. Louis CC-M 18,100 18,100 19,950 28,925 45.0% 59.8%

Kirksville 8,650 8,650 8,625 14,475 67.8% 67.3%

Springfield Community College 54,650 54,650 47,950 91,600 91.0% 67.6%

Jefferson City 19,500 19,500 30,175 33,475 10.9% 71.7%

Vandalia 46,250 46,250 48,700 87,050 78.7% 88.2%

Joplin 22,300 22,300 30,400 42,350 39.3% 89.9%

St. Jospeh 55,450 55,450 60,425 105,325 74.3% 89.9%

Della Lamb 16,250 16,250 18,775 31,150 65.9% 91.7%

Sikeston 12,450 12,450 15,050 25,575 69.9% 105.4%

Rolla 12,400 12,400 8,375 25,725 207.2% 107.5%

Blue Springs 3,950 3,950 3,925 8,375 113.4% 112.0%

St. Charles Community College 27,200 27,200 29,725 58,350 96.3% 114.5%

West Plains 11,750 19,200 14,125 25,275 78.9% 115.1%

Camdenton 2,900 2,900 3,900 6,550 67.9% 125.9%

Parkway 56,250 56,250 73,100 134,025 83.3% 138.3%

Columbia 21,850 21,850 42,875 54,575 27.3% 149.8%

Carthage 5,550 5,550 10,550 14,075 33.4% 153.6%

Trenton Community College 5,100 5,100 10,325 13,700 32.7% 168.6%

Mexico 1,750 1,750 1,975 4,750 140.5% 171.4%

Macon 5,950 5,950 9,675 16,350 69.0% 174.8%

University City 7,100 7,100 11,375 20,875 83.5% 194.0%

East Central Community College 14,150 14,150 24,100 41,900 73.9% 196.1%

Lebanon 5,300 5,300 1,450 16,025 1005.2% 202.4%

Rockwood 6,500 6,500 9,625 20,900 117.1% 221.5%

Hannibal 4,650 4,650 6,700 15,925 137.7% 242.5%

DOC 136,600 152,707 325,575 510,175 56.7% 273.5%

Kansas City 38,450 38,450 63,850 150,975 136.5% 292.7%

Avac 7,450 — — — — —

Boonvilled 1,150 — — — — —

Fultone 32,500 — — — — —

Susanna Wesleyf

— — 3,000 7,675 155.8% —

SOURCE: Missouri Department of Elementary and Secondary Education, 2006. Funding data provided by state.

Note: — Not applicable.

a Adult Education and Literacy; corresponds to six functional academic performance levels for ABE and ASE learners.

b In FY03 the federal government closed a training camp located in Nevada, which severely impacted program services.

c Consolidated with West Plains in FY03.

dConsolidated with Sedilla in FY03.

e Program unable to deliver proposed programs; resources subsumed by Department of Corrections.

f New program added in FY03.

Percentage Change

Performance Expenditures

Table C-8—

Missouri Adult Education Performance-based Funding Expenditures, by Provider: FY02 to FY05
SECTION C-5 

Effects of Performance-based Funding 

— 146 — 

Program Outcomes 

With the exception of FY03, in which adult education enrollments peaked, the number of 

learners who completed a level and either left the system or remained enrolled and moved to one 

or more higher levels of instruction has increased over time. Specifically, the number of level 

completers climbed from 11,213 in FY01 to 13,643 in FY05 (table C-9). Although program 

enrollments have fallen since FY03, the percentage of learners completing a level has remained 

relatively constant, which may indicate that providers are becoming increasingly effective in 

achieving state goals. 

Roughly a year into formula operation, a group of providers raised concerns about perceived 

funding inequities associated with the state’s ESL program, because it was suggested that ESL 

students do not progress to higher learning levels at the same rate as do other students. To 

address this issue, the state convened a special committee comprised of local program directors. 

After reviewing contact-hour data for ESL as compared to ABE and ASE participants, the 

committee proposed modifying the state formula to include a separate performance-funding 

category for ESL level advancement. Supplemental funding weights were incorporated into the 

funding formula beginning in the FY05 program year. 

A review of state data appears to support state administrators’ contention that certain populations 

of students require additional time to achieve an educational gain. For example, in FY05, 

roughly 33 percent of ESL participants achieved a level completion, compared to 37 percent of 

ABE and 40 percent of ASE participants (table C-10). Although completion rates for ESL 

populations were somewhat lower than for other learners, on average, ESL programs logged 

nearly 127 hours per participant, compared to roughly 98 hours per ABE and 63 hours per ASE 

participant. On average, this suggests that local providers must invest roughly 30 percent more 

time in ESL instruction than in ABE instruction to achieve a lower completion rate. 

FY01 FY02 FY03 FY04 FY05

Participants 41,089 39,821 41,928 37,729 37,052

Level Completers 11,213 12,350 17,952 12,889 13,643

Percentage Completing Level 27% 31% 43% 34% 37%

Average Attendance Hours/Participant 78.0 98.4 98.3 94.5 98.6

SOURCE: Missouri Department of Elementary and Secondary Education, 2006. Funding data provided by state.

Table C-9—

Selected Characteristics of Missouri Adult Education Level Completers: FY01 to FY05
SECTION C-5 

Effects of Performance-based Funding 

— 147 — 

Although local providers did not receive additional resources for achieving a performance 

outcome for ESL participants before FY05, it does not appear that providers were less likely to 

serve ESL learners immediately following the introduction of PBF. Between FY02 and FY05, 

the number of ABE and ESL participants each fell roughly 7 percent, and ASE participants fell 

roughly 5 percent, in keeping with an overall decline in statewide enrollment. However, since the 

addition of supplemental funding for ESL level completers was not introduced until FY05, with 

allocations based on FY03 data, it is impossible at the present time to assess the effect of 

additional weighting on provider behavior. 

System Evolution 

Performance funding is a dynamic system in Missouri. Although formula distribution criteria 

have remained stable over time, state administrators have modified the amount of resources 

allocated as core, performance, and categorical funding. Formula changes reflect state attempts 

to fine-tune the system to address provider concerns and to adjust for changing funding 

environments, as in FY04 when sharp reductions in federal funding diminished the state’s ability 

to maintain its resource allocations to programs. 

Specifically, in FY04, changes in federal census data reduced Missouri’s eligibility for AEFLA

resources, leading to a $1,000,000 reduction in funds. In an effort to protect local providers, the 

state dropped its allocation of state funding for marketing activities and Expansion and 

FY01 FY02 FY03 FY04 FY05

Adult Basic Education

Participants 24,986 25,321 26,264 24,186 23,518

Completers 7,392 7,603 12,621 8,360 8,788

Percent Completing 29.6% 30.0% 48.1% 34.6% 37.4%

Average Attendance Hours/Participant 84.8 91.4 95.0 96.0 97.5

English Literacy

Participants 9,704 8,583 9,361 8,036 7,955

Completers 1,532 2,453 2,906 2,487 2,625

Percent Completing 15.8% 28.6% 31.0% 30.9% 33.0%

Average Attendance Hours/Participant 63.3 136.3 125.4 113.7 126.9

Adult Secondary Education

Participants 6,399 5,917 6,303 5,507 5,579

Completers 2,289 2,294 2,425 2,042 2,230

Percent Completing 35.8% 38.8% 38.5% 37.1% 40.0%

Average Attendance Hours/Participant 74.1 73.6 71.6 60.3 62.7

Table C-10—

Selected Participation Characteristics of Missouri Adult Education Participants by ABE, ESL, and ASE 

Status: FY01 to FY05
SECTION C-5 

Effects of Performance-based Funding 

— 148 — 

Enhancement grants. In addition, the state funneled resources to local providers out of its 

statewide leadership projects to offset this unanticipated drop in federal resources. 

Another example of the system’s evolution is the state’s adjustment of its formula to 

differentially fund students with language challenges. The AEL Supplemental Literacy Funding, 

introduced in the FY04 program year, is intended to promote volunteerism and to support 

instruction for adults in the lowest literacy levels (below sixth grade). Because of where services 

are based, this program steers a large share of resources to Department of Corrections programs. 

Funding allocations under the new formula were based on local programs’ literacy hours, with 

rates of $3.00 per hour for volunteer tutors and $6.00 per hour for certified volunteer tutors in 

corrections facilities. All funds generated under the program are to be allocated to support 

literacy coordinator salaries, purchased services, and materials and supplies. 

The state is currently in the process of changing how it identifies sites requiring technical 

assistance. Under the existing system, local providers perform an annual self-evaluation using a 

state-supplied checklist. Technical assistance needs are determined through informal, anecdotal 

evidence collected by local program staff from telephone calls, conversations, and reviews of 

submitted data. In FY07, given the emphasis on meeting state performance targets, the state is 

shifting to an evaluation review process that aligns with components in the NRS. Providers will 

indicate whether or not they had a successful year, and, if not, offer reasons why they were 

unable to meet their NRS performance targets. Underperforming providers also will need to 

develop a corrective action plan indicating how they will address their deficiencies. Both state 

and local administrators must sign off on this plan, which will form the basis for targeting 

technical assistance to programs in areas of identified need. 
— 149 — 

Appendix C Summary 

In FY05, the most recent year for which data are available, Missouri allocated roughly 19 

percent of provider funding based on performance, drawing on both federal and state resources 

for both base funding and PBF. When introduced in FY02, PBF resources were limited to 

roughly 9 percent of total funding. Over time, as the state has cut back on its contact-hour rate 

while increasing the value of performance outcomes, the proportion of funding allocated based 

on performance has risen. Unlike in the higher education sector where PBF adoption was 

externally imposed, the decision to adopt PBF was made almost entirely based on programmatic 

factors, with the adoption process spearheaded by a state director newly appointed to his 

position. State administrators reported no political or other external pressures to adopt 

performance funding. 

To build support for PBF and to design an allocation mechanism, the state director worked with 

a group of approximately 20 program directors drawn from throughout the state. These members 

worked with state administrators, in a collaborative, consensus-based approach to funding 

formula development. Buy-in among committee members would later pay huge dividends for the 

state in reducing provider resistance to the new formula. Near the end of the formula 

development process, as the new system was being readied for implementation, a few program 

directors voiced concern regarding PBF, at a program directors’ retreat. Committee members 

who had participated in the formula development process pointed out that they had been 

involved in the process and had worked to ensure that providers’ viewpoints were incorporated 

into the final formula. 

The state allocates roughly 72 percent of provider resources as base funding according to 

providers’ total audited contact hours. The state also earmarks state and federal resources for use 

in specific functions identified by the state. These include data collection funding to support data 

entry staff salaries and computing upgrades; marketing funding to offset provider advertising 

expenses; technology funding to purchase computing hardware and software; and One-Stop 

program funding to support coordination with One-Stop Career Centers. The availability of these 

categorical funds is dependent upon state funding conditions; for example, in FY04, the state 

dropped the marketing component to compensate for decreased federal resources. 

In the belief that lower-level learners often take longer to achieve a level progression, the state 

has adopted a graduated funding scale, with supplemental resources attached to students making 

gains at the lowest learning levels. To establish initial rates for AEL level completions, the state 
Appendix C Summary 

— 150 — 

modeled resource allocations based on the number of outcomes recorded by providers in FY01 

and the amount of performance funding available for distribution in FY02. In FY05, award 

amounts increased and the state added a separate set of rates for students participating in ESL 

programs. 

Because the introduction of PBF coincided with an infusion of federal and state resources into 

the system, the state did not attempt to insulate providers from shifts in their resource eligibility. 

This may have been because no action was needed: Simulations indicated that there would be 

few losers, and those projected to experience a drop in resources were generally larger providers 

with budgets big enough to offset any loss. Initial effects also may have been small because the 

state initially distributed just 9 percent of provider funding using PBF criteria; over time, this 

amount has gradually increased to its current level of 19 percent. 

Although case study participants ascribed both positive and negative outcomes to the adoption of 

performance funding, few if any data exist to substantiate these claims. Participant observations 

suggest that PBF helped improve the validity and reliability of adult education data, both because 

program directors were more likely to review information for accuracy prior to submission and 

because providers conscientiously followed accepted testing protocols when measuring learner 

progression. Performance funding also was credited with helping to professionalize the teaching 

force, in part because of the emphasis placed by state administrators on training local staff. 

Finally, before PBF, program directors reported that few of their colleagues “owned” their data; 

that is, although information may have been collected, few had an incentive to use it, since 

funding was based on contact hours. It was suggested that the current system has helped to focus 

directors’ attention on program performance, and, in particular, on making connections between 

specific instructors and learner outcomes. 

Program directors also reported some drawbacks with PBF. The delay between when programs 

post outcomes and receive their resources can mean that programs may have difficulty 

expanding programs to new sites, since providers do not realize any benefit for their successes 

until two years after posting their data. Tracking learner outcomes also can create additional 

work for instructors, many of whom are more concerned with teaching than with keeping 

detailed records. Finally, directors suggested that the current system does not provide sufficient 

compensation for learners with special needs. Since not all learners are capable of making annual 

progress on state performance measures, directors reported that instructors have difficulty 

achieving measurable gains for special-needs students, particularly those requiring remedial 

instruction or with documented learning disabilities. 
Appendix C Summary 

— 151 — 

Since the introduction of performance funding, the total number of providers has declined by two 

programs. According to the state director, the introduction of PBF was not directly responsible 

for the loss of any adult education programs, although one program was unable to generate 

sufficient contact hours or performance outcomes to qualify for resources, Rather, the loss of 

programs was blamed on problems with the invitation to bid process, in which providers had 

problems with or forgot to submit required forms and consequently were declared ineligible for 

program funding in the new funding cycle. 

Review of state performance data indicates that in FY04, Missouri scored near the national 

average on most NRS measures. Although the state recorded a substantially higher percentage of 

learners receiving a GED credential, learner attainment of educational levels was slightly below 

the national average. Although it is difficult to determine the effect that PBF has had on state 

attainment of its negotiated federal performance targets, the state director and program directors 

interviewed for this study suggested that PBF has supported the state in improving data quality 

and provider use of data for program improvement purposes at the local level. 

— 153 — 

APPENDIX D 

Study Methodology and 

Analysis Procedures 

— 155 — 

SECTION D-1: 

Study Methodology 

Researchers conducted two-day visits to each case study state to interview state administrators 

and local service providers, and to collect fiscal allocation data and background materials.13 To 

ensure cross-case consistency, researchers developed interview guides organized around the 

research questions and the study conceptual framework (i.e., approach to the analysis project). 

Research team members used these guides and framework to structure case study interviews and 

focus groups, and to request state documents and fiscal data. 

Case Study Visits 

The first case study visit was conducted with three members of the research team—Steve Klein, 

MPR Associates; Garland Hankins, former Arkansas state director of adult education; and 

Raymond Maietta, a qualitative analysis consultant. Team members consulted with one another 

both prior to and following the first case study visit to review study protocols and research 

questions, and to review data submitted by the state prior to site visits. Klein and Hankins 

conducted subsequent case study visits. 

Researchers forwarded a list of requested documents and fiscal data at least four weeks prior to 

the site visit, and shared copies of interview or focus group questions with state administrators at 

least two weeks prior to the visit. State directors were asked to forward requested documents and 

fiscal data at least two weeks prior to the site visit. To narrow the volume of information 

collected, researchers provided state directors with a letter reiterating the study goals and 

research questions, and clarifying the intended use of the information collected. (See exhibit D-1 

for a copy of the letter.) Requested documents included: 

• State-developed descriptions of formula components and allocation procedures; 

• Examples of technical assistance and training materials distributed to service 

providers, only as they relate to the use of performance-based funding; 

• Formal studies of formula operation developed by state or independent agencies; 

 

13 States selected for study participation were identified as having among the more advanced PBF systems in the nation. 
APPENDIX D-1: 

Study Methodology 

— 156 — 

• Program education outcome data, including the number of students served, 

enrolled (12 hours or more), student contact hours, retention rate, cost per student, 

and percentage of education levels completed; and 

• Annual state fiscal allocation data, beginning two years preceding and three years 

following formula adoption. Data to be collected will include the number of 

providers funded, the type of providers funded, the amounts provided to each 

provider, and the data used to calculate provider eligibilities. 

Research team members reviewed submitted materials to supplement questions contained in the 

interview protocol and to inform analysis of the research questions. Analysis of submitted 

documents focused on identifying text that related to the specific questions contained within the 

interview guides or that answered the research questions guiding the study. 

Interviews 

Research team members interviewed key program staff within each state. (See exhibit D-2, Case 

Study Site Visit Agenda.) At least six weeks prior to case study visits, research team members 

conferred with the state director of adult education within each state to identify state-level 

participants and their job responsibilities. Although the mechanics of operation of state adult 

education formula varied across sites, research team members met with the following types of 

individuals: 

• State Director of Adult Education—responsible for overseeing policy 

development and system operation of state adult education services; 

• State Adult Education Administrators including assistant directors and staff 

reviewing local provider funding applications—responsible for conducting 

ongoing administration and monitoring of program operations, and for providing 

technical assistance to enable local providers to meet state requirements 

associated with performance-based funding; 

• State Fiscal Administrators—responsible for analyzing local data and for 

determining providers’ basic grant and performance-based funding grants; 

• State Management Information Staff—responsible for collecting, cleaning, and 

managing performance data from local providers; 

• Local Provider Directors—responsible for overseeing adult education program 

and fiscal operations; and 
APPENDIX D-1: 

Study Methodology 

— 157 — 

• Local Provider Staff who are key program staff, including institutional 

researchers and fiscal administrative staff—responsible for collecting learner 

outcomes and program data, analyzing program performance, and overseeing 

professional development activities. 

In practice, the number of individuals interviewed within each state varied, depending upon the 

organization and scale of state services. For example, in smaller states, the responsibility for 

administering program outcome and fiscal data were more likely to be performed by a single 

individual. 

All interviews were approximately one and a half hours and were conducted using open-ended 

questions, in a semi-structured interview format. This approach allowed participants to provide 

details that would not be accessible via a pre-designed survey format and to respond to research 

questions in their own words. Where appropriate, instead of individual interviews, data 

collection occurred using a focus group format, in which multiple individuals were asked to 

respond to interview questions. For example, where appropriate, focus group interviews were 

held with a team of two or more state administrators who shared common job duties or 

responsibilities (e.g., reviewing provider applications or conducting audits of local data). 

Research team members consistently applied protocols across sites, with questions individually 

tailored to specific job categories. Researchers were given the flexibility, however, to further 

tailor interview questions to participants’ job responsibilities and to follow up on potentially 

relevant lines of questioning. All interviews were audio taped, with written consent obtained 

from study participants. (See exhibit D-3 for a copy of consent form.) 

Research team members interviewed local providers both as a group and individually. To permit 

researchers to meet with multiple local providers, the state adult education director was asked to 

convene a group of directors from between three to five nearby local providers, with the meeting 

held either at the offices of the state agency or a local provider. Following the focus group 

meeting, researchers visited a single provider located within a one-hour drive of the state agency. 

Researchers consulted with states to identify providers that represented the types of organizations 

that offer adult education programs in the state. Every effort was made to include providers that 

offered a variety of adult education programs and services, such as, adult basic education, adult 

secondary education, English language literacy, workplace literacy, and family literacy. 

Researchers requested that state directors of adult education identify participants for the provider 

focus groups and interviews six weeks prior to the site visits. To assist state directors in selecting 
APPENDIX D-1: 

Study Methodology 

— 158 — 

providers, state directors were provided with a list of criteria for selecting participants. These 

criteria specified provider: 

• Type—sites should be representative of the type and diversity of adult education 

providers in the state; 

• Experience—sites should have participated in the state funding system since its 

inception and be staffed with administrators who have extensive experience using 

the state funding formula; 

• Clientele—sites should be representative of the different types of students being 

served, since some students (e.g., GED participants) may be easier to recruit or 

serve than other types of students (e.g., ABE participants); 

• Historical Data—sites should have complete fiscal records of state allocations 

beginning in the year preceding formula adoption, and be willing to share this 

information with case study researchers; 

• Satisfaction—participants should provide a fair and balanced view of the use of 

performance-based funding, and may include individuals supportive and critical 

of the funding approach; and 

• Geographic Proximity—sites should be within a one-hour drive of the state 

agency or other providers selected for study participation to allow researchers 

sufficient time to travel among sites. 

Care was taken that interviews with state administrators and local adult education program staff 

aligned with project research questions and the study conceptual framework. For example, 

interviews with state directors of adult education focused on global issues relating to the 

adoption and use of performance-based funding, ranging from the rationale for system 

development (e.g., what were the driving forces to introduce systems and who participated in the 

decision-making process), the processes used to design and implement funding formulas (e.g., 

who participated in system development and how were measures selected and integrated into the 

formula), and the perceived system effects on the provision of adult education services. In 

contrast, interviews with state fiscal staff focused primarily on how systems were implemented 

and their effects on provider operations. To avoid simply collecting anecdotal information, 

researchers formulated interview protocols so that local provider staff were more likely to supply 

concrete, detailed examples to support their responses. 
APPENDIX D-1: 

Study Methodology 

— 159 — 

Site Meetings 

To solicit local perspectives, researchers requested the state director or each state to identify 

program directors from at least four local providers representative of the type and diversity of 

adult education providers in the state. 

INDIANA 

The research team met with the state director and assistant state director of adult education at the 

state offices, located in Indianapolis. Group interviews were conducted off-site at a regional 

technical assistance center, with five program directors and one area director representing four of 

the eight adult education areas in the state. These off-site meetings included a one-hour interview 

with three program directors that was then repeated with a second group of three program 

directors. Researchers also met with two field-based representatives, including a state 

professional development consultant and a coaching consultant providing technical assistance to 

the field. 

A site visit also was conducted to a local adult education provider located in a fifth regional area 

approximately 50 miles from the state adult education offices. This local site visit included an 

interview with the program director and an adult education instructor. Researchers also toured 

the instructional facilities. 

Program directors selected by the Indiana state director of adult education for case study followup included four of 19 sites that lost performance funding at a rate exceeding the state average 

between FY01 to FY05, the first year and fifth years that PBF was awarded. Table D-1 contains 

data on the percentage and dollar amount change in provider funding eligibility for the six sites 

whose program directors participated in group interviews. 

Percentage Change Dollar Amount

Provider FY01 to FY05 Change

Statewide Total -18% -$224,421

Site 1 -60% -$24,153 

Site 2 -44% -$27,268 

Site 3 -39% -$29,170 

Site 4 (local case study visit)a -23% -$9,431 

Site 5 -16% -$5,184 

Site 6 6% $2,031 

SOURCE: Indiana Department of Education, 2006. Funding data provided by state.

a Case study team visited the local site indicated during the state visit.

Table D-1—

Change in Performance Award for Indiana, by Provider: FY01 to FY05
APPENDIX D-1: 

Study Methodology 

— 160 — 

KANSAS 

The research team met with the state director and associate director of adult education at the state 

offices, located in Topeka. To solicit local perspectives, researchers interviewed program 

directors from five local providers identified by the Kansas state director as representative of the 

type and diversity of adult education providers in the state. Group interviews were conducted 

off-site at a Topeka hotel hosting the state’s annual meeting of program directors. 

These off-site meetings included a 1.5-hour interview with four program directors and the 

independent field monitor; the state director joined the group for the final half hour of discussion. 

Following this group meeting, the research team met with the team of independent field monitors 

responsible for conducting provider visits and awarding program quality points. 

The research team also conducted a site visit to a local education provider located approximately 

55 miles from the state adult education offices. This local site visit included an interview with 

the program director and two adult education instructors. Researchers also toured the 

instructional facilities. 

Program directors selected by the Kansas state director of adult education for case study followup included one of six sites that lost performance funding at a rate exceeding the state average 

between FY00 to FY04, the first year and fifth years that PBF was awarded. The remaining four 

sites gained funding between FY00 and FY04. Table D-2 contains data on the percentage and 

dollar amount change in provider funding eligibility for the five sites whose program directors 

participated in group interviews. 

Provider Percentage Change Dollar Amount Change

Statewide Total 31% $223,333

Site 1 -43% -$120,284

Site 2 (local case study visit)a 56% $143,626

Site 3 74% $61,984

Site 4 105% $72,866

Site 5 243% $65,141

SOURCE: Kansas Board of Regents, 2006. Funding data provided by state.

a Case study team visited the local site indicated during the state visit.

Table D-2—

Change in Performance Award for Kansas, by Provider: FY00 to FY04
APPENDIX D-1: 

Study Methodology 

— 161 — 

MISSOURI 

Research team members met with the state director, former state director, and other state adult 

education administrators at the state office, located in Jefferson City. To solicit local 

perspectives, researchers interviewed program directors from five local providers identified by 

the Missouri state director as representative of the type and diversity of adult education providers 

in the state. Group interviews were conducted at the state offices with no members of the state 

administrative team in attendance. 

Provider meetings included a one and one half hour interview with the five program directors. 

The research team also conducted a site visit to a local education provider located within walking 

distance of the state adult education offices. This local site visit included an interview with the 

program director and a tour of the instructional facilities. 

Program directors selected by the Missouri Department of Elementary and Secondary Education 

for case study follow-up included three of 12 sites that lost performance funding at a rate 

exceeding the state average between FY01 to FY05, the last year of the old state funding formula 

year and the fourth year that PBF was awarded. The remaining three sites gained funding 

between FY01 and FY05. Table D-3 contains data on the percentage and dollar amount change 

in provider funding eligibility for the five sites whose program directors participated in group 

interviews. 

Percentage Change Dollar Amount

Provider FY01 to FY05 Change

Statewide Total 1% $18,788

Site 1 -18% -$41,622 

Site 2 -9% -$2,608 

Site 3 -7% -$24,402 

Site 4 0% $1,679 

Site 5 (local case study visit)a 45% $38,037 

Site 6 312% $47,704 

SOURCE: Missouri Department of Elementary and Secondary Education

a Case study team visited the local site indicated during the state visit.

Table D-3—

Change in Performance Award for Missouri, by Provider: FY01 to FY05

APPENDIX D-1: 

Study Methodology 

— 163 — 

Exhibit D-1: Letter to State Directors of Adult Education Requesting Selected Information 

for Cross-case PBF Study 

December 26, 2005 

State Director 

Director of Adult Education 

State Department 

Street 

City, State Zip 

Dear State Director: 

Thank you for agreeing to participate in the Office of Vocational and Adult Education, U.S. 

Department of Education study of performance-based funding (PBF) in adult education. As we 

discussed, MPR Associates, Inc., will be sending a team of case study researchers to your state to 

conduct a two-day site visit on Jan. 11–12, 2006. This visit will include interviews and focus 

groups with state adult education agency and local adult education provider staff. The purpose of 

these meetings will be to record administrators’ experiences designing and implementing PBF 

systems and providers’ perspectives on the use of PBF. We will be attempting to answer the 

following research questions: 

• Why have states decided to adopt performance-based funding systems? 

• How have performance-based funding systems been designed at the state level? 

• How have performance-based funding systems been implemented at the state 

level? 

• What effect have performance-based funding systems had on the attainment of 

state-identified performance goals and the delivery of services? 

To help prepare you for our case study visit, we have enclosed examples of the type of questions 

we will ask during our visit, as well as a list of background information that we are requesting 

you compile prior to our arrival. 

It is our hope that we will be able to speak to you and other individuals who can provide a 

comprehensive description of your state’s decision to adopt performance-based funding, as well 

as the outcomes of system adoption. While we are asking you to identify the specific individuals 
APPENDIX D-1: 

Study Methodology 

— 164 — 

with whom we can meet, we anticipate that we will need to talk with the following types of 

individuals: 

• State Adult Education Administrators—including assistant directors and staff 

responsible for reviewing local provider funding applications, for conducting 

ongoing administration and monitoring of program operations, and for providing 

technical assistance to enable local providers to meet state requirements 

associated with performance-based funding. 

• State Fiscal Administrators—responsible for analyzing local data and for 

determining providers’ basic grant and performance-based funding grant. 

• State Management Information Staff—responsible for collecting, cleaning, and 

managing performance data from local providers. 

• Local Provider Directors—responsible for overseeing adult education program 

and fiscal operations. 

• Local Provider Staff—key program staff, including institutional researchers and 

fiscal administrative staff responsible for collecting learner outcomes and 

program data, analyzing program performance, and overseeing professional 

development activities. 

All interviews will be approximately one and a half hours and will be conducted using openended questions, in a semi-structured interview format. The enclosed agenda, which we will 

tailor to fit your needs, provides an overview of the expected time we will need to spend with 

different groups. 

To permit researchers to meet with multiple local providers, we are also asking that you convene 

a group of directors from between three to five providers, with the meeting to be held either at 

the offices of the state agency or a local provider. Following the focus group meeting, 

researchers will visit a single provider, preferably located within a one-hour drive of the state 

agency. To assist in selecting providers, we ask that you consider the following criteria: 

• Type—sites should be representative of the type and diversity of adult education 

providers in the state. 

• Experience—sites should have participated in the state funding system since its 

inception and be staffed with administrators who have extensive experience using 

the state funding formula. 
APPENDIX D-1: 

Study Methodology 

— 165 — 

• Educational Clientele—sites should be representative of the different types of 

students being served, since some students (e.g., GED participants) may be easier 

to recruit or serve than other types of students (e.g., ABE participants). 

• Historical Data—sites should have complete fiscal records of state allocations 

beginning in the year preceding formula adoption and be willing to share this 

information with case study researchers. 

• Satisfaction—participants should provide a fair and balanced view of the use of 

performance-based funding and may include individuals both supportive and 

critical of the funding approach. 

• Geographic Proximity—sites should be within a one-hour drive of the state 

agency or other providers selected for study participation to allow researchers 

sufficient time to travel among sites. 

State Materials 

In addition to providing us with access to you and your staff, we are also requesting that you 

provide us with background information and fiscal data that will assist us in understanding how 

performance-based funding has been implemented in your state. In particular, we are asking that, 

prior to our visit, you compile the following types of information, where it exists: 

• State-developed descriptions of formula components and allocation procedures. 

• Examples of technical assistance and training materials distributed to service 

providers, only as they relate to the use of performance-based funding. 

• Formal studies of formula operation developed by state or independent agencies. 

• Program educational outcome data, including the number of students served, 

enrolled (12 hours or more), student contact hours, retention rate, cost per student, 

and percentage of education levels completed. 

• Annual state fiscal allocation data, beginning two years preceding and three years 

following formula adoption. Data to be collected will include the number of 

providers funded, the type of providers funded, the amounts provided to each 

provider, and the data used to calculate provider eligibilities. 

The site visit will not have any bearing on federal compliance issues and is intended only to 

collect information that can be used to assess the issues affecting state adoption of PBF. 
APPENDIX D-1: 

Study Methodology 

— 166 — 

Thank you in advance for your assistance. I anticipate speaking with you in early January to 

finalize our meeting plans. 

If you have any questions, please feel free to contact me by phone: 503-675-6619 or you may email me at sklein@mprinc.com. 

Sincerely, 

Steven Klein 

MPR Associates, Inc. 
APPENDIX D-1: 

Study Methodology 

— 167 — 

Exhibit D-2: Case Study Site Visit Agenda 

 Performance-based Funding 

Day 1: State Department of Adult Education 

8:30–9:00 a.m. Arrive on site: Introductions & Study Overview 

9:00–10:30 a.m. Interview State Director of Adult Education 

Focus on rationale for system development and issues relating to system 

design and implementation. 

10:30–10:45 a.m. Break 

10:45–12:00 p.m. Interview State Adult Education Administrators 

Meet with state agency personnel responsible for reviewing local provider 

funding applications, for conducting ongoing administration and 

monitoring of program operations, and for providing technical assistance 

to enable local providers to meet state requirements associated with 

performance-based funding. 

12:00–1:00 p.m. Lunch with State Director of Adult Education (if available) 

1:00–2:30 p.m. Interview with State Fiscal Staff 

Meet with individuals responsible for analyzing local data and for 

determining providers’ basic grant and performance-based funding grant. 

2:30–2:45 p.m. Break 

2:45–4:15 p.m. Interview with State Management Information Staff 

Meet with individuals responsible for collecting, cleaning, and managing 

performance data from local providers. 

4:15–4:45 p.m. Closing Interview State Director of Adult Education 

Follow up on issues identified by staff and solicit recommendations to 

guide system development in other states. 
APPENDIX D-1: 

Study Methodology 

— 168 — 

Day 2: Local Provider(s) of Adult Education (Focus Group and Site Visit) 

8:30–8:45 a.m. Arrive on site: Introductions & Study Overview 

8:45–10:45 a.m. Focus Group Interview: Local Provider Institutional Directors 

Meet with individuals responsible for overseeing adult education program 

and fiscal operations 

10:45–11:00 a.m. Break 

11:00–11:30 a.m. Focus Group Interview: Local Providers, Institutional Directors, and State 

Director of Adult Education 

Solicit recommendations to guide system development in other states. 

11:30–1:00 p.m. Lunch/Travel Time 

1:00–2:30 p.m. On-Site Interview: Service Provider Director 

Meet with individual responsible for overseeing adult education program 

and fiscal operations. 

2:30–3:00 p.m. Tour of Facility 

3:00–4:30 p.m. On-site Focus Group: Service Provider Administrative Staff 

Meet with key program staff, including institutional researchers and fiscal 

administrative staff responsible for collecting learner outcomes and 

program data, analyzing program performance, and overseeing 

professional development activities. 
APPENDIX D-1: 

Study Methodology 

— 169 — 

Exhibit D-3: Audiotape Consent Form 

Performance-based Funding in Adult Education 

1. Study Purpose 

This study has been commissioned by the Office of Vocational and Adult Education (OVAE), U.S. 

Department of Education to assess adult education fiscal policies in states using performance-based 

funding to allocate program resources. 

2. Study Benefits 

Findings from this study will be used to assist state policymakers in making more informed decisions 

when designing and implementing state allocation formulas to distribute federal and state resources to 

adult education and literacy service providers. Study participants will indirectly benefit if study 

findings are used by the state to improve formula operation. 

3. What You Will Be Asked to Do 

Study participants will be asked to participate in interviews with study researchers and to respond to a 

series of questions about their experiences using performance-based funding in their state. 

4. Confidentiality 

All information provided by study participants is considered confidential. Researchers will use 

information provided by participants in a case study report that will summarize state agency and local 

providers’ experiences working with performance-based funding. Audiotapes of interviews will be 

transcribed and analyzed using qualitative software, with access to tapes and transcriptions limited to 

study personnel and OVAE staff. 

I have read the above and agree to audiotaping at 

 (interview location) 

on . 

 

 (print name) 

 

 (signature)