Federal Disability Assistance: Stronger Federal Oversight Could Help Assure Multiple Programs' Accountability
Published: 2007-01-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Congress has created 20 federal employment-related programs that are aimed at helping people with disabilities obtain jobs. Little is known about the effectiveness and the management of some of these programs. GAO was asked to review four of these programs; the Department of Education (Education) oversees three--Projects with Industry (PWI), Supported Employment State Grants, and Randolph-Sheppard. An independent federal agency, the Committee for Purchase, oversees the fourth, Javits-Wagner-O'Day (JWOD). Specifically, GAO assessed the extent to which (1) performance goals and measures have been established for these programs and (2) the agencies responsible have established adequate oversight procedures. We reviewed program planning and performance information, interviewed agency officials, and visited each of the four programs in four states.
section_title: What GAO Found
paragraphs: Three of the four programs have federal performance goals. No federal performance goals or measures currently exist for the Randolph-Sheppard program, which provides opportunities for individuals who are blind to operate vending facilities on federal properties. Without goals, it is difficult to assess the program's performance, but Education officials told GAO they are developing them. Education has a goal and a measure for the Supported Employment State Grants program--a federal grant program that provides job coaching and other support to help individuals with severe disabilities secure jobs. The goal indirectly measures the program's performance because grant funds are mixed with other funding sources to provide supported employment services. Education has also developed one goal for the PWI program--a federal grant program that helps individuals with disabilities obtain competitive employment--that is consistent with the mission of the program. The goal is to create and expand job opportunities for individuals with disabilities in the competitive labor market by engaging business and industry, and one of the measures tracks the percentage of individuals placed in employment in work settings making at least minimum wage. The Committee for Purchase, which oversees the JWOD program--a program that helps to create jobs through the federal property management and procurement systems--first developed federal goals and measures for its fiscal year 2005-2007 strategic plan and has since revised them. The revised measures still have limitations, such as not being clearly defined or being difficult to measure. Education's and the Committee for Purchase's oversight of the four programs has been uneven. Education has established procedures, such as on-site reviews, for the PWI and Supported Employment State Grants programs that, if consistently followed, would provide reasonable assurance that the programs are in compliance with applicable laws and regulations. However, Education conducts limited oversight of the Randolph-Sheppard program. For example, Education does not routinely analyze or report the data it collects from states and has provided little guidance to ensure states comply with laws or consistently interpret program requirements. One area in which Education has not provided sufficient guidance is the circumstances under which federal agencies may charge fees to licensed vendors operating vending facilities on their properties. As a result, vendors in some locations were paying commissions or fees but those in other locationswere not. Finally, the Committee for Purchase delegates most of its oversight responsibilities to two central nonprofit agencies that also represent the interests of the JWOD nonprofit agencies they oversee. This arrangement, as well as the fact that they receive a percentage of the total value of the contracts from the JWOD nonprofit agencies, raises questions about their independence and gives them little incentive to identify instances of noncompliance that could result in the JWOD nonprofit agency losing its federal contract.

section_title: Letter
paragraphs: The employment rate for working-age people with disabilities is about half  of the employment rate for working-age people without disabilities. In  fiscal year 2003, at least $2.4 billion was spent on 20 federal programs that  are aimed to improve employment opportunities for people who have  disabilities, but little is known about how effectively some of these  programs are achieving their intended outcomes. You asked us to review  four of these programs: Projects with Industry (PWI), Supported  Employment State Grants, Randolph-Sheppard Vending Facility Program  (hereafter known as Randolph-Sheppard), and Javits-Wagner-O’Day  (JWOD). PWI and Supported Employment State Grants are federal grant  programs that fund services that help individuals with disabilities to obtain  competitive employment. The other two programs, Randolph-Sheppard  and JWOD, help create jobs for individuals with disabilities through the  federal property management and procurement systems. Randolph- Sheppard licenses people who are blind to operate vending facilities on  federal or other properties, and JWOD gives preference to nonprofit  agencies who employ people with disabilities to produce goods and  services for the federal government.
The Department of Education (Education) is responsible for administering  the Projects with Industry, Supported Employment State Grants, and  Randolph-Sheppard programs. By law, Randolph-Sheppard is a state- operated program with limited federal responsibilities. The Committee for  Purchase from People Who Are Blind or Severely Disabled (Committee for  Purchase), a small independent federal agency, is responsible for  administering the JWOD program. The Committee for Purchase utilizes  two central nonprofit agencies—National Industries for the Blind (NIB)  and NISH—to help carry out its responsibilities.
You asked us to determine to what extent (1) performance goals and  measures have been established for these programs and (2) the agencies  responsible for these programs have established adequate procedures for  overseeing program implementation and assuring laws and regulations are  followed.
To determine the extent to which performance goals have been  established for these programs, we conducted a review of program laws,  guidance, and performance documents. To obtain additional information  about the performance goals and measures and determine the extent that  the agencies responsible for these programs have established adequate  procedures for overseeing program implementation and assuring that laws  and regulations are followed, we reviewed agency policies and guidance,  and interviewed agency officials at Education and the Committee for  Purchase. We also met with officials of the two central nonprofit agencies  (NIB and NISH) that assist the Committee for Purchase in performing its  oversight responsibilities for JWOD. To supplement the overall program  information, we reviewed each of these programs at the local level in four  states—Arizona, Kansas, New York, and North Carolina—and analyzed  documentation (e.g., program guidance, monitoring protocols, etc.) to  ascertain how these programs were administered and monitored. We  selected these four states based on factors such as whether they had all  four programs operating in the state, geographic diversity, and whether  the states’ Vocational Rehabilitation (VR) programs had a separate blind  program (about half of all states have two agencies; one for the general VR  program and one for the blind program). We visited a total of 4 PWI  projects, 4 Supported Employment State Grants recipients, 7 Randolph- Sheppard vendors, and 13 JWOD nonprofit agencies. We reviewed  available audit reports on the state-operated Randolph-Sheppard program  for the states we visited and for other states. We also reviewed other  available audit reports for JWOD nonprofit agencies and PWI grantees.  Finally, we interviewed officials of agencies engaged in disability research  and advocacy at the national level including the Council of State  Administrators of Vocational Rehabilitation, National Council on  Disability, and the National Council of State Agencies for the Blind.   (App. I contains a more detailed discussion of our scope and  methodology.) We conducted our work between March 2006 and  December 2006 in accordance with generally accepted government  auditing standards.
subsections: section_title: Background
paragraphs: Over the years, Congress has established many employment-related  programs to help people with disabilities obtain jobs. Four of these  programs, PWI, Supported Employment State Grants, Randolph-Sheppard,  and JWOD, illustrate several different approaches taken by Congress to  create more employment opportunities for people with severe  disabilities—from providing job training and support to enabling  individuals to run businesses. Congress created two of the four programs  (PWI and Supported Employment State Grants) in the 1970s and the other  two (Randolph-Sheppard and JWOD) in the 1930s.
subsections: section_title: Projects with Industry
paragraphs: PWI was established in 1978, and is a discretionary grant program that  provides financial assistance for up to 5 years to organizations to assist  individuals with disabilities in obtaining competitive employment.    However, according to Education officials, recent grants have been  awarded for 3-year periods. Grantees of the PWI program include  community rehabilitation program providers, employers, labor unions,  nonprofit agencies or organizations, trade associations, and others. The  purposes of the PWI program are to (1) create and expand job and career  opportunities for individuals with disabilities in the competitive labor  market by engaging private industry as partners in the rehabilitation  process, (2) identify competitive job and career opportunities and the  skills needed to perform these jobs, (3) create practical settings for job  readiness and job training programs, and (4) provide job placements and  career advancements. PWI grantees must establish business advisory  councils (BAC) comprised of representatives of private industry,  organized labor, and individuals with disabilities and their representatives,  and others. BACs are required, among other things, to identify jobs and  careers available in the community, the skills necessary to perform them,  and prescribe appropriate training and job placement programs. Seventy- nine grantees received about $22 million in fiscal year 2005 and served  more than 10,000 individuals with significant disabilities.
The Department of Education is responsible for administering and  overseeing PWI and is required to:    Conduct annual on-site compliance reviews—Education’s primary  means of verifying the accuracy of the information grantees  submit—of at least 15 percent of grant recipients, chosen at  random.
Submit an annual report to Congress that analyzes the extent to  which the individual grant recipients have complied with the  evaluation standards. For example, the project must serve  individuals with disabilities that impair their capacity to obtain  competitive employment. In selecting persons to receive services,  priority must be given to individuals with significant disabilities.
Have a performance reporting system that grantees can use to  routinely submit program data that evaluates the grantees’ progress  in achieving the stated objectives, the effectiveness of the project in  meeting the purposes of the program, and the effect of the project  on its participants.
section_title: Supported Employment State Grants
paragraphs: Established in 1978, the Supported Employment State Grants program  provides funds to states and is designed to assist states in developing  collaborative programs with appropriate organizations to provide  supported employment services to individuals with the most severe  disabilities who require these services to enter or retain competitive  employment. Supported Employment State Grants funded services include  a wide array of employment-related activities ranging from intensive on- the-job skills training to discrete post-employment services such as job  station redesign or repair and maintenance of technology to help them  perform job functions, generally for up to 18 months after job placement.  In fiscal year 2005, the grant program was funded at approximately   $37 million.
The Supported Employment State Grants program is an integrated  component of state VR programs, which are also overseen by Education.  Title I of the Rehabilitation Act of 1973 authorizes a federal-state VR  program to provide services to persons with disabilities so that they may  prepare and engage in meaningful employment. Education provided   $2.6 billion in fiscal year 2005 in VR grants to the states and territories  based on a formula that considers the state’s population and per capita  income. Each state and territory designates a single VR agency to  administer the VR program, except where state law authorizes a separate  agency to administer VR services for individuals who are blind. State VR  agencies provide services to individuals in 22 service categories, such as  vocational counseling and guidance, job placement assistance, on-the-job  supports, college or university training, rehabilitation technology, and  interpreter services. State VR agencies that determine they will not be able  to serve all eligible individuals who apply for services must develop  criteria for prioritizing services to individuals with the most significant  disabilities. Education reported that, as of fiscal year 2006, 40 of the 80  state VR agencies had such an order of selection.
Oversight for Supported Employment State Grants is conducted as part of  oversight of state VR programs, and Education is required to:    conduct annual reviews of state VR programs that include collecting  and reporting information on budget and financial management  data, and an analysis of program performance, including relative  state performance, based on the standards and indicators; and    conduct periodic on-site monitoring of state VR programs.
section_title: Randolph-Sheppard
paragraphs: The Randolph-Sheppard program was created in 1936 to provide blind  persons with gainful employment, enlarge their economic opportunities,  and encourage their self-support. While Randolph-Sheppard is under the  authority of Education, the states are primarily responsible for operating  their programs, and every state except Wyoming has established a vendor  program. Each state that has a Randolph-Sheppard program is required to  have a state licensing agency (SLA), under the auspices of the state VR  program and approved by Education, to operate the program, including  the authority to promulgate rules and regulations that govern the program.  The SLAs are responsible for training, licensing, and placing people who  are blind as operators of vending facilities (machines, snack bars, and  cafeterias) located on federal and other properties. In addition, SLAs must  annually submit information about their Randolph-Sheppard programs to  Education, including information on the number of applicants and the  number accepted, the number of vending facilities and vendors, and the  total amount of vendor earnings.
In fiscal year 2005, SLAs spent about $37 million in federal and state VR  grant funds to help operate and support the program. In addition to VR  funds, some states fund the program through optional set-asides from  licensed vendors, which are a percentage of their revenues, and through  the profits from vending machines located on federal properties that are  not operated by licensed vendors. State funds are also used to operate the  program. In total, more than $76 million were used to operate and support  the Randolph-Sheppard program nationwide in fiscal year 2005.
In fiscal year 2005, the Randolph-Sheppard program generated   $661.3 million in total gross income and the average annual earnings of  vendors was $43,584. Over a 5-year period (fiscal year 2001 through fiscal  year 2005), the number of vending facilities have been in decline  nationwide, decreasing from 3,193 to 3,080. Over the same period, the  number of vendors decreased annually except in fiscal year 2005, as  shown in figure 1.
While states are responsible for operating their programs, among other  things, Education is required to:    approve applications from a state’s VR agency to serve as the SLA,  and approve the rules and regulations the SLA promulgates to  implement the Randolph-Sheppard Act;    conduct periodic evaluations of the program to determine whether  the program is being used to its maximum potential; and    convene arbitration panels and pay for arbitration to resolve vendor  and SLA disputes.
section_title: Javits-Wagner-O’Day
paragraphs: Established in 1938, JWOD is a federal procurement set-aside program  designed to increase employment and training opportunities for persons  who are blind or have other severe disabilities. Through this program, the  government purchases commodities and services from nonprofit agencies  employing workers who are blind or have severe disabilities. According to  Committee for Purchase officials, in fiscal year 2006, federal procurement  expenditures for goods and services provided by JWOD program suppliers  totaled about $2.3 billion, and provided employment for about   48,000 people who are blind or have severe disabilities at more than   600 participating JWOD nonprofit agencies. The types of employment  opportunities range from working in food service or providing janitorial  services in federal office buildings to producing and/or assembling boxes  and office supplies such as pens, notepads, file folders, and other goods.  For fiscal year 2005, Committee for Purchase officials reported that JWOD  workers earned an average of $9.49 per hour.
The Committee for Purchase, which administers the program, received  about $5 million in federal funds in fiscal year 2005 to support the  activities of a 15-member, presidentially appointed board and 29 full-time  program staff, including managing the JWOD procurement list. The  Committee for Purchase is required by law to designate one or more  central nonprofit agencies to facilitate the distribution of federal  procurement contracts among qualified nonprofit agencies, and has  designated two agencies for this purpose: NIB, which represents member  nonprofit agencies employing individuals who are blind, and NISH, which  represents its member nonprofit agencies that employ individuals with  other severe disabilities.
In addition to its duties related to establishing and maintaining a  procurement list of goods and services that must be purchased through  qualified JWOD suppliers, the Committee for Purchase is required to:    establish rules, regulations, and policies to carry out the purposes of  the JWOD program, and to provide that nonprofit agencies  employing individuals who are blind have priority in obtaining  JWOD contracts;    monitor nonprofit agency compliance with Committee for Purchase  inform federal agencies about the JWOD program and encourage  their participation, and, to the extent possible, monitor federal  agencies’ compliance with JWOD requirements; and    study and evaluate its activities on a continual basis to ensure the  effective and efficient administration of the JWOD Act.
The Committee for Purchase has also established regulations that require  the two central nonprofit agencies (NIB and NISH) to evaluate the  qualifications and capabilities of nonprofit agencies that apply for  contracts and provide pertinent data concerning the JWOD nonprofit  agencies, such as their status as qualified nonprofit agencies, and their  manufacturing or service capabilities. Additionally, NIB and NISH are to  monitor and inspect the activities of participating nonprofit agencies to  ensure compliance with the JWOD Act and appropriate regulations. For  example, to maintain its status as a qualified nonprofit agency organized  for the purposes of the JWOD program, an agency must employ persons  who are blind or have severe disabilities to perform at least 75 percent of  the work-hours of direct labor during the fiscal year to furnish such  commodities or services (whether or not the commodities or services are  procured under the JWOD Act).
The Committee for Purchase’s regulations require that each nonprofit  agency maintain employment files for persons with severe disabilities  participating in the JWOD Program. Each file must contain either a  certification by a state or local government entity or a written report  signed by a licensed physician, psychiatrist, or qualified psychologist,  reflecting the nature and extent of a participant’s disability or disabilities  that qualify as severe. These reports must also state whether an individual  with severe disabilities is capable of engaging in normal competitive  employment and be signed by persons qualified to evaluate their work  potential, interests, aptitudes, and abilities.
section_title: Three of the Four Programs Have Established Goals and Measures
paragraphs: Federal performance goals and measures have been established for three  of the four programs we reviewed. Education has not established  performance goals and measures for the Randolph-Sheppard program,  although two of the four states that we visited had their own goals and  measures. Education has one goal for the Supported Employment State  Grants program, but the goal only provides an indirect measure of the  program’s performance because the data also include individuals with  significant disabilities who receive supported employment services funded  under state VR programs. Education has established a performance goal  for PWI, which is consistent with the purpose of the program. Finally, for  the JWOD program, the Committee for Purchase recently revised its  performance goals and established some targets.
subsections: section_title: Education Has Not Established Federal Goals for Randolph-Sheppard
paragraphs: Education does not have GPRA performance goals for the Randolph- Sheppard program, and neither the Randolph-Sheppard Act nor its  implementing regulations require them. According to Education officials,  no formal federal performance goals or measures currently exist for the  Randolph-Sheppard program, but they are under development and  expected to be completed by April 2007. Although not specifically required  by law, Education does collect some information related to program  performance from the states. For example, Education collects information  on total vendor income, number of facilities, and vendors. Education also  collects information on the numbers of individuals who are blind or have  disabilities who are employed by vendors, although there is no  requirement for vendors to employ workers who have disabilities.
States may develop performance goals for their Randolph-Sheppard  programs, and two (Arizona and Kansas) of the four states we visited had  established performance goals. Arizona’s goals were to increase the  number of licensed vendors and vending facilities. In fiscal year 2005, the  state set a target of 32 vendors and five new facilities. However, Arizona  did not meet these targets and had 30 licensed vendors and added one new  facility. Kansas’ fiscal year 2005 goal was that at least 90 percent of the  licensed vendors maintain or increase their level of income from the prior  year, and the state reported that this goal had been exceeded in each of  the past 3 fiscal years. According to program officials in New York and  North Carolina, no state goals were established for their Randolph- Sheppard programs. In the four states we visited, nearly $10 million,  including more than $7 million in federal and state funds, were used to  support operations for about 215 licensed vendors, as shown in table 1.
In these four states, we also interviewed seven licensed vendors who  operated businesses that ranged from full-service cafeterias to small  convenience stores or canteens. We found that all of the licensed vendors  we met with worked full-time and most earned incomes that provided an  income that made them relatively self-sufficient. However, not all licensed  vendors nationwide receive incomes that allow them to support  themselves and their incomes may be subsidized through program  revenues generated by other vendors. Two states we visited were taking  steps to increase vendors’ income by consolidating facilities. Regardless of  their financial status, three of the seven licensed vendors we interviewed  continued to receive financial benefits from other federal disability  assistance programs, such as Social Security Disability Insurance. In  addition, at least 6 of the 7 vendors employed fewer than 10 workers, most  of whom were not blind or did not have severe disabilities. However, some  of the vendors we interviewed told us that they are interested in ways to  reach out to and employ more workers who are blind or have severe  disabilities. Further, states we visited told us about other program  challenges, such as a decline in customers as a result of increased security  in federal buildings and consolidation of unprofitable facilities that  reduced the number of opportunities available to vendors.
section_title: Education Has a Supported Employment State Grants Program Goal That Indirectly Measures Performance
paragraphs: Education has a GPRA goal for the Supported Employment State Grants  program that the department uses to indirectly measure the program’s  performance. According to program officials, Education has not sought  information that isolates the performance of federally-funded Supported  Employment State Grants because they are used together with state and  other federal funds to provide supported employment services, as is often  the case when funds from different sources are used to achieve an  outcome. Officials told us a separate measure for the Supported  Employment State Grants program would be an artificial distinction. The  performance goal is for individuals who have significant disabilities to  achieve high quality employment. For this goal, Education only includes  individuals with significant disabilities who have a goal of supported  employment, that is, achieving competitive employment with support  services such as rehabilitation technology or on-the-job supports. The  measure is the percentage of individuals who achieve competitive  employment, which they define as making minimum wage or higher, but  not less than the wages paid to workers without disabilities performing  similar work, and working alongside workers without disabilities in an  integrated setting.
During fiscal years 2003 and 2004, Education exceeded its performance  target for placing Supported Employment State Grants program  participants in competitive employment. For fiscal year 2005, Education  increased the performance target to 93 percent and achieved 92.6 percent,  as shown in table 2.
section_title: Education Has Established One Federal Performance Goal for the PWI Program
paragraphs: Education has established a GPRA performance goal that includes four  measures for the PWI program. The goal of the PWI program is to create  and expand job opportunities for people with disabilities in the  competitive labor market by engaging business and industry in the  rehabilitation process. The four performance measures are consistent with  the program’s goal. For example, one measure is the percentage of  individuals served by the program who were placed into competitive  employment. Another measure, cost per placement, was only recently  established for fiscal year 2006 and performance data are not yet available.  In recent years, Education has had mixed success in meeting the GPRA  targets. For example, in fiscal years 2003 to 2005, the PWI program did not  meet its target of increasing the percentage of individuals who were  placed into competitive employment. However, it consistently exceeded  its target for increased earnings over the same period, as shown in table 3.
Education has revised its GPRA performance measures for the PWI  program for fiscal year 2006. Specifically, Education will begin to measure  the percentage of PWI projects whose cost per placement is within a  specified range, which has yet to be determined. The agency will also  measure the percentage of all individuals who exit the program and are  placed in competitive employment. According to Education officials, this  measure was added in response to recommendations by the Office of  Management and Budget and will allow more accurate comparisons with  other job training programs throughout the government.
We visited four PWI grantees in the four states and found that these  projects set goals that are consistent with the goals of the PWI program,  such as placing people in competitive employment. For example, one PWI  grantee in Kansas is serving individuals with all types of disabilities ages  16 and older. One goal of the project is to place 75 percent of the people  they serve each year in competitive employment, which is higher than the  GPRA target set by Education. According to agency officials, clients are  being placed in jobs such as call centers and other customer service  positions, earning average salaries of $9.25 to $10.00 per hour. In another  example, one PWI grantee in New York has a goal to transition youth from  school to work and targets its services to individuals ages 16 to 25 with a  mental, physical, or emotional disability. One of the goals of the project is  to place about 67 percent of the people served in jobs over 3 years.  According to the grantee, a successful outcome in the program is  competitive employment in at least a part-time position paying at least the  federal minimum wage, and continued employment for at least 6 months.  The PWI projects in Kansas and New York just completed the first year of  operations.
section_title: The Committee for Purchase Recently Updated Its Strategic Plan for the JWOD Program; Progress toward Goals Is Not Yet Known
paragraphs: For the first time, the Committee for Purchase developed performance  goals and measures for the JWOD program in its fiscal year 2005-2007  strategic plan and updated this plan in October 2006, but it has not yet  reported progress toward meeting these goals. While JWOD’s enabling  legislation and regulations do not require goals, the strategic plan includes  five performance goals and a number of measures for each of these goals.  The current strategic plan includes some performance measures and  targets, but some of the measures that are more qualitative in nature do  not include targets, and it is unclear how JWOD will measure progress in  these areas. In keeping with the overall mission of the JWOD program, the  goals are aimed at increasing the number of job opportunities for people  who are blind or have severe disabilities. One of the plan’s five goals is to  expand employment opportunities. The other four goals include increasing  customer satisfaction (JWOD customers are federal agencies), improving  efficiency of operations, expanding program support and developing new  markets for its products and services. However, these goals do not  specifically address one part of the program’s mission, which is to increase  training opportunities.
Furthermore, some of the performance measures are not clearly defined  or may be difficult to measure, thus making it difficult to assess  performance. For example, there are several measures that involve using  “milestone tracking” although the milestones are not provided. One of  these measures will track progress toward annually updating and  implementing a plan to address communication and information sharing  with and among stakeholders. Further, JWOD has over 30 performance  measures, which may make it difficult to identify performance problems.  As we discussed in our June 1996 guide on implementing GPRA,  performance measures should be limited to the vital few. Limiting  measures to core program activities enables managers and other  stakeholders to assess accomplishments and make decisions without  having an excess of data that could obscure rather than clarify  performance issues. The JWOD performance goals and examples of  measures are shown in table 4. All of the JWOD performance measures are  listed in appendix II.
section_title: Uneven Federal Oversight Provides Little Assurance of Accountability for Two Programs
paragraphs: Education and the Committee for Purchase engage in a number of  oversight activities for the programs they are responsible for, but their  efforts to ensure compliance with applicable laws and regulations have  been uneven, and overall have provided little assurance of program  accountability for two of the four programs reviewed. Specifically,  Education has established oversight procedures for the PWI and the  Supported Employment State Grants programs that, if consistently  followed, should provide reasonable assurance of compliance with  relevant laws and regulations. The agency is just beginning to conduct on- site monitoring of PWI grantees that may be sufficient for testing the  accuracy of the information used to monitor compliance. Education’s  oversight of these two programs has generally been more active than its  oversight of the Randolph-Sheppard program. Education relies primarily  on self-reported data for its monitoring of the Randolph-Sheppard program  and does not routinely analyze or report the data it collects. Finally, the  Committee for Purchase has established procedures for monitoring and  overseeing the JWOD program, but has prescribed regulations that  delegate most of the responsibility for carrying out these procedures to  two central nonprofit agencies that are also responsible for representing  the interests of the JWOD nonprofit agencies they monitor, raising  questions about independence. Furthermore, there are no procedures in  place for the Committee for Purchase to address instances where the  central nonprofit agencies fail to carry out their oversight responsibilities.
subsections: section_title: Education Performs Various PWI Oversight Activities, and On-site Monitoring Is Improving
paragraphs: Education regularly performs a number of oversight activities to ensure  that PWI grantees are making progress toward project goals and  complying with applicable laws and regulations. Specifically, program  specialists told us they conduct quarterly monitoring calls with all PWI  grantees in which they ask a series of 30 questions that help them to  identify and proactively resolve problems with individual projects. The  questions address several areas, including progress toward meeting goals,  activities of the BACs, interaction with the state VR agency, and fiscal  management. Further, Education requires that PWI grantees submit annual  reports that include detailed information about project activities and  performance, and informs grantees of this requirement as part of the  application process. Education uses the project information it receives  from grantees to identify those grantees that may be at risk of being out of  compliance with program requirements and to target these grantees for  additional assistance or for on-site reviews. Education also relies on the  data it receives from grantees to provide information about grantees’  performance in its annual reports to Congress.
Although grantees are responsible for monitoring their own projects,  Education is required to conduct random on-site reviews of 15 percent of  PWI grantees annually. On-site reviews are the primary means by which  Education can assess the accuracy of the performance data submitted by  grantees. Education conducted 11 of the 12 required on-site reviews in  fiscal year 2006 and had scheduled the remaining review for November  2006. However, it conducted only 3 of the 12 required for 2005, and 0 in  2004, and therefore did not have enough information to provide reasonable  assurance of the accuracy of the data submitted by grantees in those  years. Although each of the PWI grantees that we visited had procedures  in place to review the data they submitted to Education, a research  organization conducted an evaluation of the program that raised doubts  about the accuracy of PWI data submitted by grantees in general.  Specifically, reviewers found that about one-fifth of PWI grantees surveyed  (19 out of 92) provided information on the number of persons placed in  fiscal year 2001 that was inconsistent with the information they had  submitted to Education.
Education is also required to submit an annual report to Congress  analyzing the extent to which the individual grant recipients have  complied with program evaluation standards. In fiscal years 2003, 2004,  and 2005, Education has met this requirement by providing summaries of  the extent to which grantees have met program performance targets in its  Performance and Accountability Report to Congress.
section_title: Education’s Oversight of Supported Employment Is Part of Overall VR Program Monitoring Efforts
paragraphs: Education’s oversight of the Supported Employment State Grants program  is integrated into its ongoing efforts to review and monitor state VR  programs. During fiscal year 2006, Education revised its annual state plan  review protocols and prepared a draft on-site monitoring plan for the VR  program. Annual reviews include examining each state’s VR program  plans and other documentation, such as required annual data reports on  VR customers, services, and outcomes; caseloads; and financial  accountability and data reporting procedures. Education’s October 2006  draft on-site monitoring protocols call for on-site reviews once every   3 years and are designed to verify and supplement the information it  receives from the states regarding program performance and compliance,  and include reviewing case files and holding public hearings or other  discussions with VR program consumers and advocates, as needed.  Education has not yet conducted any on-site reviews using the revised  protocols, but plans to conduct its first reviews beginning in fiscal year  2007. Once fully implemented, the annual reviews and on-site monitoring,  along with state-level activities, should offer reasonable assurance that the  Supported Employment State Grants program is in compliance with  applicable laws and regulations and the data that states submit to  Education annually are accurate.
In addition, we found that all four states we visited had their own  accountability procedures for ensuring that VR grant funds, including  Supported Employment State Grants funds, were being used in  accordance with federal laws and regulations. For example, the New York  state VR agency has configured its automated information management  system in a way that only authorizes payment for supported employment  services to providers that have a contract to provide these services, at the  contracted rates. In addition, three of the four states had adopted  performance-based contracting systems, whereby vendors providing  supported employment services, such as job coaching or training, are  required to demonstrate progress toward required milestones in order to  receive payment from the state agencies, and VR counselors monitor their  progress on a weekly, biweekly, or monthly basis.
section_title: Education Provides Little Oversight of the Randolph- Sheppard Program
paragraphs: Education provides little oversight of the Randolph-Sheppard program.  Despite being required to conduct periodic evaluations of the program and  being responsible for approving states’ rules and regulations for  implementing the Randolph-Sheppard Act, Education has no formal  procedures for evaluating state programs. In addition to lacking  procedures, Education has performed few on-site monitoring reviews of  SLAs in recent years. According to agency officials, Education has  performed five on-site monitoring reviews since the beginning of fiscal  year 2005 and had performed no recent site visits in the four states that we  visited. Education’s oversight activities primarily consist of collecting data  from states through annual reports from the SLAs that administer the  program and providing requested technical assistance. Although the states  report considerable information including earnings data; the number of  vendors, facilities, individuals employed by vendors; types of facilities;  costs; and sources of funding, Education does not test the accuracy of data  that it requires states to report, nor does the agency routinely analyze the  data to assess program performance and management. As a result,  Education cannot assess trends in performance, identify possible best  practices, or help states that may need assistance. Upon request,  Education also provides technical assistance to SLAs. According to  Education officials, technical assistance and guidance is regularly  provided to SLAs through telephone calls and written correspondence,  including e-mails, with staff on specific questions.
In its oversight role, Education has not provided clear guidance to states  on emerging issues that could have nationwide implications. Instead,  Education responds to individual state concerns and convenes panels to  arbitrate disputes that SLAs are unable to resolve. As a result, states have  different policies regarding the permissibility of teaming agreements,  which partner licensed vendors with commercial food operators in order  to help manage larger food service operations at dining facilities at  military bases. SLAs may have such agreements for various reasons, such  as state program officials’ lack of expertise or licensed vendors’  inexperience running such facilities. In these cases, the licensed vendor  generally does not operate the food service facilities, but rather manages  some aspects of food service operations. For example, one of the states  we visited (Kansas) had a teaming agreement. One of the states we visited  (New York) does not currently allow teaming agreements, while another  (Arizona) has no policy regarding teaming agreements. The fourth state,  North Carolina, permits teaming agreements but does not currently have  any as of June 2006.
Although Education has noted the increasing use of teaming agreements, it  has not issued guidance to the SLAs directly addressing whether these are  in keeping with the spirit of the Randolph-Sheppard Act, or whether they  should be subject to limitations, despite concerns expressed by states and  others. For example, the California State Auditor found that by allowing  teaming agreements, the SLA had inadequately protected the interests of  the state and licensed vendors. The SLA had not (1) ensured that written  contracts existed before beginning operations, (2) analyzed the investment  and return on investment of the teaming agreement to the program and  licensed vendors, (3) adequately reviewed the teaming agreements, or   (4) ensured that the commercial food service operators were paying their  fair share of program costs. In addition, the Georgia State Auditor  identified some concerns about teaming agreements, including the failure  to define the duties for participating licensed vendors, resulting in these  vendors having little, if any, responsibility for the overall operation and  success of subcontracted food services. Further, the auditor noted that the  program is not ensuring that the commercial food service operators are  making progress toward the program’s goal that licensed vendors  eventually assume responsibility for operating the facility.
Additionally, Education has not provided clear guidance or policies  regarding when federal agencies may charge fees or commissions to  licensed vendors as a condition of operating a vending facility on federal  property. The Randolph-Sheppard Act has been interpreted to prohibit  commissions unless federal agencies obtain written approval from the  Secretary of Education. We found that licensed vendors have paid  commissions or fees in some locations but not in others and the federal  agencies had not obtained approval from Education. For example, in one  state we visited (Kansas), at least one licensed vendor was required to pay  1.5 percent of total revenues to the U.S. Postal Service in exchange for  permission to operate vending facilities on the agency’s properties.  However, Education has not prohibited such practices or required the  Postal Service or other federal agencies charging commissions to obtain  written approval. Furthermore, officials in Kansas have chosen not to  dispute it. According to agency officials, Education has never approved  such a limitation and cannot routinely monitor state-level or vendor- specific business negotiations, but would intervene to bring the parties  together in an attempt to resolve disputes or make clear the requirements  of the Randolph-Sheppard Act.
Although Education has exercised little oversight for this program, the  four SLAs that we visited had certain procedures in place that should, if  consistently operated along with other certain complementary processes  and procedures such as management’s monitoring of performance over  time, help safeguard program assets. SLA officials obtained cash register  receipts, daily reports on business activities, or monthly reports submitted  by the vendors to review the financial operations for these programs.  However, audits of programs in other states have reported certain issues  relating to the accountability of state-operated programs under the  Randolph-Sheppard Act. For example, the Michigan Auditor General  reported that SLA staff did not comply with established equipment  inventory control procedures for program equipment and could not  account for equipment inventory, placing inventory at risk of  misappropriation. Further, the California State Auditor reported that,  among other things, the SLA has not followed up on missing financial  reports from licensed vendors and has not been able to monitor licensed  vendors’ financial problems properly. In addition, the auditor found that  the SLA was not adequately pursuing past-due commissions owed to the  program by private businesses operating vending machines on federal  properties.
section_title: Committee for Purchase Delegates Most Oversight Responsibilities for JWOD to Two Organizations That Represent the Interests of the Agencies They Oversee
paragraphs: The Committee for Purchase has established procedures for monitoring  and overseeing the JWOD program, but has delegated most of the  responsibility for monitoring participating JWOD nonprofit agencies to  two central nonprofit agencies. As of April 2006, NIB officials reported  that they worked with 81 participating JWOD nonprofit agencies that  employed individuals who are blind, and NISH officials reported that they  worked with 552 JWOD nonprofit agencies that employed individuals with  other severe disabilities. In particular, although the Committee for  Purchase must approve the nonprofit agencies’ participation in the  program, it relies on the central nonprofit agencies to certify that:    75 percent or more of the direct labor hours under JWOD contracts  are performed by individuals who are blind or have severe  disabilities, and if not, that there is a suitable plan in place to bring  this percentage up to the required level;    agencies maintain required documentation for each of these    agencies function as nonprofit entities serving individuals who are  blind or have severe disabilities;    agencies have a required job placement program; and    agencies comply with applicable occupational safety and health  standards.
The Committee for Purchase requires that the JWOD nonprofit agencies  certify annually that they are in compliance with program requirements  but does not routinely verify this information, relying instead on the  central nonprofit agencies to do so. According to agency officials, the  Committee for Purchase performs about 20 field visits annually, visiting up  to 3 agencies per visit, or about 60 of the more than 600 participating  nonprofit agencies. At this rate, the Committee for Purchase is unable to  satisfy its own requirements to perform on-site compliance reviews at  each fully compliant participating nonprofit agency every 5 years.
The Committee for Purchase’s regulations create at least two problems for  NIB and NISH: the potential for a conflict of interest resulting from a lack  of organizational independence as well as disincentives to perform their  monitoring duties effectively. Specifically, these regulations require that  NIB and NISH, on behalf of the Committee for Purchase, monitor the  compliance of JWOD nonprofit agencies, but, at the same time, represent  them in their dealings with the Committee for Purchase. Moreover, the  regulations also permit NIB and NISH to charge a fee based on JWOD  nonprofit agencies’ sales to the government that does not exceed the limit  set by the Committee for Purchase, and require the nonprofit agencies to  pay that fee in order to remain in good standing in the program. This  system of compensation may create a disincentive for NIB and NISH to  identify instances of noncompliance that could result in the JWOD  nonprofit agency losing its contract, especially for those JWOD nonprofit  agencies that are generating large volumes of JWOD sales. Finally,  although the regulations and procedures provide for a number of duties  that the central nonprofit agencies must perform, they do not specify  actions the Committee for Purchase can take if the central nonprofit  agencies fail to execute these duties.
NIB and NISH officials reported that they monitor JWOD nonprofit  agencies’ compliance with relevant laws and regulations by conducting on- site reviews of nonprofit agencies every 3 years, and require quarterly  statistical reports from the agencies they oversee. The Committee for  Purchase has established procedures for these reviews that require each  central nonprofit agency to use a standardized review sheet to assess  whether the JWOD nonprofit agency is compliant in 11 different program  areas, including the percentage of direct labor hours performed by  individuals who are blind or have severe disabilities, documentation of an  employee’s disability, and an evaluation of whether or not the individual is  capable of competitive employment. The on-site reviews are the primary  means for NIB and NISH to test the accuracy of the data that the JWOD  nonprofit agencies submit, but the scope of the reviews may not be  sufficient to provide reasonable assurance of the accuracy of all of the  data. For example, NIB and NISH officials reported that they test the  accuracy of the data for percentage of direct labor hours by reviewing a  sample of case files, but they do not verify other data, such as job  placement and upward mobility statistics. Further, they do not always  report instances of noncompliance they find to the Committee for  Purchase.
In the states we visited, reports from NIB’s and NISH’s on-site reviews  generally showed that the JWOD nonprofit agencies were in compliance  with program requirements, and most files contained the required  documentation. Eleven of 13 agencies that we visited provided  documentation of the results of their most recent on-site reviews showing  they were in compliance. However, in our limited reviews of 137 case files  at these 13 agencies, we found that 5 of 8 NISH agencies had at least 1 file  that lacked the required medical documentation of a worker’s disability,  and that 3 of these 8 NISH agencies had at least 1 file that lacked the  required documentation on competitive employment. We also found that  one of the five NIB agencies we visited had one case file that lacked the  required medical documentation. In sum, 11 percent of the files we  reviewed at the NIB and NISH agencies we visited lacked the required  medical or competitive employment documentation.
A serious instance of noncompliance escaped detection by the responsible  central nonprofit agency (NISH) and the Committee for Purchase. In this  case, the National Center for the Employment of the Disabled (NCED) in  El Paso, Texas, failed to use workers with severe disabilities to perform  the required percentage of direct labor hours on its JWOD contracts,  which were valued at over $200 million. Instead, NCED inflated its  reported percentage by improperly including economically disadvantaged  workers. The problems at NCED were detected not through routine  monitoring, but rather through an anonymous tip to the Committee for  Purchase, and resulted in as many as 1,144 JWOD jobs being lost to  individuals who did not have severe disabilities during fiscal years 2004  and 2005. The JWOD nonprofit agency took actions prescribed by the  Committee for Purchase to come into compliance, including dividing the  agency’s operations into two different units—one for JWOD work and one  for commercial activities—and the Committee for Purchase is satisfied  with the actions taken.
The definition of a severe disability in the law allows for differing  interpretations, which may complicate efforts to ensure compliance for  agencies that serve individuals who have severe disabilities. The statutory  definition of blindness is fairly straightforward: a lack of visual acuity of  not more than 20/200 in the better eye with correcting lenses or a limited  field of vision of not more than 20 degrees. In contrast, the definition of a  severe disability requires a diagnosis of a residual, physical or mental  impairment that limits functioning in one of five areas (mobility,  communication, self-care, self-direction, and work tolerance or work  skills), and a determination that the impairment has rendered the  individual unable to engage in normal competitive employment over an  extended period of time. Despite the fact that the definition is subject to  interpretation, the Committee for Purchase has offered little additional  guidance that would clarify when disabilities that may not normally be  considered severe could be, such as the conditions under which a  recovering alcoholic or a person with diabetes could be considered to  have a severe disability. During our review of case files at 13 JWOD  nonprofit agencies, we noted instances where it was unclear in the  medical documentation that the disability was severe, such as a case in  which the individual was diabetic, with no indicated symptoms, and  another in which the individual was diagnosed as having an aggressive  personality.
section_title: Conclusions
paragraphs: All four of these programs generally provide training and employment  opportunities that might not otherwise be available for individuals who are  blind or have severe disabilities. However, two are hampered by  weaknesses in performance management and program oversight that  signal a need for stronger federal leadership. Absent federal goals for the  Randolph-Sheppard program and routine analyses and reports from  Education on states’ program operations and performance, little is known  about how this program is improving the lives of participants. Having such  information about outcomes is an important component of any program,  and essential during times of fiscal austerity. Further, by not exercising  more oversight and issuing clear guidance to all states on emerging issues  that could affect program participants, Education may be missing an  important opportunity to help states improve program operations or  proactively respond to these issues. While recognizing that there may be  increased costs for improved oversight, these costs could be minimized by,  for example, monitoring Randolph-Sheppard activities as part of  Education’s oversight for the VR program. Although the Committee for  Purchase has made significant progress in developing goals for the JWOD  program, some of the goals lack key elements—clear measures and  performance targets. Also, the current approach for overseeing nonprofit  agencies operating under the JWOD program poses difficult challenges for  the two central nonprofit agencies in managing the conflicts of interest  that may exist because of their lack of organizational independence, and  therefore demands strong and effective oversight from the Committee for  Purchase. Ensuring program integrity is particularly important for JWOD  since nonprofit agencies are given a competitive advantage over private  business and industry in the federal procurement system to ensure that  opportunities are provided to individuals with severe disabilities.
section_title: Recommendations
paragraphs: 1.  To improve program performance management and oversight, we  recommend that the Secretary of Education provide more effective  leadership of the Randolph-Sheppard program by:    establishing performance goals to identify desired programwide  outcomes that assess states’ licensed vendor programs’  performance as a whole in achieving established goals;    being more proactive in disseminating clear, consistent and routine  guidance about program requirements and prohibited practices to  federal agencies and states; and    strengthening their monitoring of SLA and Randolph-Sheppard  program performance in a cost-effective manner.  2.  To improve program performance management, we recommend that  the Chairperson for the Committee for Purchase assess goals and  measures for JWOD to ensure that they are clear, measurable and  continue to capture key aspects of program performance as the  Committee for Purchase continues to develop its performance  management system.  3.  To help ensure that JWOD nonprofit agencies comply with program  laws and regulations, we recommend that the Chairperson of the  Committee for Purchase improve procedures for overseeing these  agencies. This could include requiring the central nonprofit agencies to  enter into written contracts with the Committee for Purchase that  clearly lay out their oversight responsibilities and the consequences for  failing to fulfill them, providing a means of compensating the central  nonprofit agencies for their services that provides an incentive for  effective enforcement, or having the Committee for Purchase assume  greater responsibility for oversight of JWOD nonprofit agencies, by  performing more on-site compliance reviews.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of this report from the  Department of Education and the Committee for Purchase. Education and  the Committee for Purchase generally agreed with our recommendations  and provided information on activities they had underway or planned.
Education agreed that it should provide more effective leadership of the  Randolph Sheppard program and commented that the actions we  recommended were consistent with the steps the program is taking to  improve program administration. Some of the steps Education cited  included developing appropriate performance goals, enhancing its efforts  to provide clear and consistent guidance, and improving program  monitoring. We believe these efforts will help to improve program  performance management and oversight.
The Committee for Purchase agreed that its performance goals and  measures for the JWOD program should be assessed to ensure that they  are clear and quantifiable. However, the Committee for Purchase  commented that its regulations did not intend for “training” to be taken  literally as a mission output and, therefore, the agency did not establish a  separate goal for training activities. Rather, the Committee for Purchase  stated that it viewed training as an important, but incremental activity that  equips persons who are blind or have severe disabilities with the  knowledge and skills necessary for employment, which it considers the  paramount goal of the program. Also, the Committee for Purchase believes  the reporting requirements for establishing a separate goal for training  would burden the nonprofit agencies. To avoid confusion over the  Committee for Purchase’s goals in the future, the agency plans to clarify  its regulations. While we believe that training is key to preparing persons  who are blind or have severe disabilities for employment, we can  understand the Committee for Purchase’s view that the paramount  program goal is employment. Clarifying the regulations regarding the  Committee for Purchase’s intent with respect to the role of training may  make it clear that a separate goal for training is not essential.
With respect to ensuring effective monitoring and oversight, the  Committee for Purchase agreed that more guidance was needed to help  ensure that JWOD nonprofit agencies comply with program laws and  regulations. Additionally, the Committee for Purchase commented that it  has recently begun taking steps to address possible conflicts of interest  between the two roles played by the central nonprofit agencies. The  Committee for Purchase also commented that it is considering  establishing other oversight and compliance mechanisms and in its  proposed fiscal year 2007 budget included three new positions and  additional funding for oversight, compliance monitoring, and program  review. We believe the Committee for Purchase’s proposed actions are  positive steps toward helping to ensure that JWOD nonprofit agencies  comply with program laws and regulations.
Education and The Committee’s comments appear in appendixes III and  IV, respectively. Both agencies also provided technical comments, which  we have incorporated into the report as appropriate.
We are sending copies of this report to the Secretary of Education,  Chairperson of the Committee for Purchase, relevant congressional  committees, and others who are interested. Copies will also be made  available at no charge on GAO’s Web site at http://www.gao.gov.
Please contact me on (202) 512-7215 if you or your staff have any  questions about this report. Contact points for our Offices of  Congressional Relations and Public Affairs can be found on the last page  of this report. Other major contributors to this report are listed in  appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of our study were to assess to what extent (1) performance  goals and measures have been established for these programs and (2) the  agencies responsible for these programs have established adequate  procedures for overseeing program implementation and assuring laws and  regulations are followed.
To determine what performance goals and measures have been  established for these programs, we reviewed program laws, guidance, and  performance-related documents. To obtain additional information about  the performance goals and measures, and determine the extent that the  agencies responsible for these programs have established adequate  procedures for overseeing program implementation and assuring that laws  and regulations are followed, we (1) reviewed federal laws, regulations,  and guidance to determine the programs’ requirements; and   (2) interviewed agency officials at the Rehabilitation Services  Administration (RSA) within the Department of Education, and the  Committee for Purchase from People Who Are Blind or Severely Disabled  (Committee for Purchase). In addition, we interviewed officials of the two  central nonprofit agencies—the National Industries for the Blind (NIB)  and NISH—that have been delegated certain oversight responsibilities for  Javits-Wagner-O’Day (JWOD) member nonprofit agencies by the  Committee for Purchase.
To obtain additional information about program goals and measures and  oversight for the four programs, we conducted site visits to four states  (Arizona, Kansas, New York, and North Carolina). During these visits, we  met with state VR agency officials to discuss the Supported Employment,  Projects with Industry (PWI), JWOD, and Randolph-Sheppard programs. In  addition, we visited 4 PWI grantees, 13 JWOD nonprofit agencies, and   7 Randolph-Sheppard vendors. Several criteria were used in selecting the  states to visit. States that were considered had all four of the employment- related disability programs currently present and operating in them. States  varied in the administration of their VR programs with about half of all  states having two separate programs—one for the general VR and a  separate one for the blind—and the remaining states having only one VR  program. We selected two states (New York and North Carolina) that had  both general and blind VR programs, and two states (Arizona and Kansas)  that had one VR program that served all people with disabilities seeking  employment-related assistance. In addition, we selected states based on a  review of information about the characteristics of NIB and NISH member  nonprofit agencies to include both large and small, urban and rural, and  different kinds of work performed (products and services). States were  also selected to include geographic diversity.
During the state visits, we met with officials representing each of the four  programs. For Education’s programs, we interviewed local officials of  nonprofit agencies with PWI grants, state program administrators for the  state-operated Randolph-Sheppard programs as well as licensed vendors,  and state VR officials responsible for administering the Supported  Employment State Grants program. For JWOD, we interviewed chief  executive officers or their representatives of the JWOD nonprofit agencies  with current federal contracts to provide goods and/or services. In  addition, during these meetings, we collected documentation to ascertain  how the federal and two central nonprofit agencies were monitoring their  respective programs. We also reviewed the records of 137 workers who  were blind or had severe disabilities, selecting some records at each of the  13 JWOD nonprofit agencies we visited. The records were randomly  selected from lists of current JWOD workers. A random number generator  was used to assign a number to each name on the active roster and  records were selected on the basis of the number—starting with the  lowest number. During the record review, we assessed whether the  agencies’ files of workers who were blind or had severe disabilities  contained required medical documentation and assessment of ability to  work in competitive employment.
We determined that the fiscal and program data we used in this report was  reliable for our purposes. To make this determination, we assessed the  reliability of fiscal and programmatic data by interviewing officials  knowledgeable about the data and the steps they take to ensure accuracy.  For Supported Employment, prior GAO work had systematically tested  relevant variables, including all 22 variables of the services provided. In  addition, for this engagement we obtained documentation from two states  (Kansas and New York) that described the states’ procedures for checking  the reliability of their data. Programmatic data collected by the Committee  for Purchase for the JWOD program, and Education for the PWI and  Randolph-Sheppard programs were self-reported by local program  officials. For example, reviews of the information reported by officials for  the PWI program were generally performed by project managers. For  fiscal reporting, however, we found that JWOD nonprofit agencies, PWI  grantees, and licensed vendors generally had systems and procedures for  stronger accounting of financial data. For example, state licensing agency  (SLA) officials used cash register receipts and routine reports on business  activities submitted by the licensed vendors to the SLAs to review the  financial information for these programs. In addition, licensed vendors  were also required to complete merchandise inventories at least once each  year.
We also reviewed other available reports on the Randolph-Sheppard  program of the states visited, and reviewed the findings and  recommendations of state audit reports from four other states—California,  Georgia, Michigan, and South Carolina—that had evaluated all or certain  aspects of their state-operated Randolph-Sheppard programs in recent  years. In addition, we interviewed officials of agencies engaged in  disability research and advocacy at the national level to learn more about  each of the objectives. These organizations were the Council of State  Administrators of Vocational Rehabilitation, Disability Policy  Collaboration, Easter Seals, Goodwill Industries International, National  Council on Disability, and the National Council of State Agencies for the  Blind. We also met with officials of the General Services Administration, a  federal procurement agency and partner of the JWOD program.
We conducted our work between March 2006 and December 2006 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: Performance Goals and Measures for the Javits-Wagner-O’Day (JWOD) Program
subsections: section_title: Continue to expand employment opportunities for people who are blind or have other severe disabilities under the JWOD program, including wage progression, benefits, upward mobility, and personal job satisfaction.
paragraphs: Percentage increase in direct labor hours performed by people who are blind or  have other severe disabilities on JWOD products and services.
Percentage increase in the number of people who are blind or have other severe  disabilities employed in direct labor positions on JWOD products/services.
Percentage increase in the number of people receiving benefits versus not  receiving benefits.
Percentage decrease in the number of people receiving less than the federally- mandated minimum wage or Service Contract Act wage rate, segmented by  disability.
The number of employees who are blind or have other severe disabilities who are  promoted into a direct labor job, other than supervisory or management positions,  which includes increased wages and/or fringe benefits, not attributed to cost of  living or productivity increases of less than 20 percent. Promotions can be  movement between JWOD and non-JWOD jobs.
The number of employees who are blind or have other severe disabilities who are  promoted into an indirect labor job requiring supervisory, management, or  technical skills, that included increased wages and/or fringe benefits, not attributed  to cost of living raises.
The number of employees who are blind or have other severe disabilities who  leave the nonprofit agency through competitive or supported employment  placements.
Partner with federal customers to increase  customer satisfaction and loyalty, so that the  JWOD program becomes their preferred  source for products and services.
Federal agency scorecard that evaluates the level of satisfaction with JWOD  products, services and/or customer experience among key federal agencies, using  a stoplight or similar summary format.
Time to resolve customer questions or complaints received via the central  customer feedback mechanism(s) or other means of communication.
Increased customer satisfaction with quality, timeliness, and price, based on  customer surveys and/or alternative qualitative research (e.g., focus groups).
Improve efficiency and effectiveness of the  JWOD program by streamlining and  automating processes and procedures, and  improving communication, while continuing to  ensure program integrity.
Overhead cost as a percentage of JWOD program direct labor hours, calculated  as total Committee for Purchase budget plus central nonprofit agencies’ operating  and supporting costs (excluding capital expenditures), divided by total number of  direct labor hours, segmented by overall program (Committee for Purchase plus  central nonprofit agencies’ overhead), National Industries for the Blind (NIB) and  NISH.
Reduction in the cycle time for the addition of a new JWOD product or service to  the procurement list.
Percentage increase in sales of products through commercial distribution  channels, segmented by product category.
Ranking of commercial distributors, evaluated against consistent program  performance expectations, including compliance with the Committee for  Purchase’s Essentially The Same (ETS) requirements, segmented by product  category.
Milestone tracking of evaluation of commercial distribution processes, including  staff resources and financial resources.
Percent of information technology projects on which the Committee for Purchase,  central nonprofit agencies, and nonprofit agencies collaborated to increase  efficiency and exchange of information.
Decrease in the percentage of JWOD nonprofit agencies found out of  compliance, segmented by reason.
Consider a future measure linked to the results of governance and executive  compensation actions.
Expand awareness, understanding, and  preference for the JWOD program within the  public, Congress, federal agencies, the  disability community, and other JWOD  stakeholders through effective  communication and information sharing.
Effectiveness of communication and information sharing measured by increased  percentages in awareness, familiarity (understanding) and preference,  segmented by key stakeholders.
Milestone tracking of annual update and implementation of a plan that addresses  communication and information sharing with and among both internal and  external stakeholders.
Analysis of program-level communications execution to ensure that program  resources are used in support of the strategic communications plan.
Analysis of program-level publications, events, and other communications tools to  evaluate message alignment.
Facilitate nonprofit agency adoption of program messaging and branding.
Awareness, understanding, preference among federal customers, segmented by  Department of Defense and civilian agencies.
Awareness, understanding, preference for “the disability community,” comprised  of government policy makers, academia, and private membership or advocacy  organizations for people who are blind or have other severe disabilities.
Among members of congressional committees or subcommittees with oversight  or other significance for the JWOD program, number who have been educated  about the JWOD program and/or actively engaged with their local JWOD- participating nonprofit agency(ies).
Strategically develop new markets and  expand existing markets in which the JWOD  program can provide best value products and  services to federal customers in order to  expand employment opportunities that meet  the needs of people who are blind or have  other severe disabilities.
Milestone tracking of establishment and implementation of a program market  development plan that addresses existing customers, existing products/services,  new customers, and new products/services.
Percent increase in the employment of people who are blind or have other severe  disabilities under the JWOD program, measured in (1) actual direct labor hours,  (2) actual jobs, (3) projected direct labor hours on procurement list additions, and  (4) projected jobs on procurement list additions by key market segment.
JWOD goal achievement, by agency and overall federal government.
Milestone tracking of establishment and implementation of a strategy for greater  cooperation between JWOD and the small business community, which may  explore counting appropriate JWOD awards toward the annual Small Business  procurement goals and/or the federal government’s inclusion of disability-owned  businesses within the small business measure categories (relates to leveraging  the JWOD program to create additional jobs in the commercial sector).
Milestone tracking of implementation of a strategy for greater cooperation  between JWOD and Randolph-Sheppard programs.
Milestone tracking of establishment and implementation of a strategy for greater  cooperation with service-disabled veterans’ businesses.
section_title: Appendix III: Comments from the Department of Education
section_title: Appendix IV: Comments from the Committee for Purchase from People Who Are Blind or Severely Disabled
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: The following individuals made important contributions to this report:  Shelia D. Drake, Timothy Hall, Regina Santucci, Don Allison,   Rachael Valliere, Daniel Schwimer, Walter Vance, and Robert Owens.