Welfare Reform: Information on Former Recipients' Status
Released: 1999-05-26T00:00:00

Letter

Background

Under the AFDC program, many states received waivers from federal rules to strengthen work requirements for adults. In addition, some states began experiments with time limits on receiving cash assistance. Under TANF, states generally must impose work and other program requirements on most adults receiving aid and, when an adult does not comply, reduce a family’s benefit or, at state option, terminate the benefit entirely. Moreover, families receiving TANF face a lifetime limit of 5 years, or less at state option, of federal assistance. These reforms represent significant departures from previous state and federal policies for needy families with children and have been accompanied by large declines in the number of families receiving cash assistance, from an all-time high in 1994 of about 5 million families to just over 3 million as of June 1998.

While numerous efforts are planned or under way to assess welfare reform nationally, currently little information is available on the status of families who have left welfare. Although families have always left welfare for a variety of reasons, including increased household income due to employment or marriage, once their cases were closed and the families no longer received assistance, they usually were not routinely tracked or monitored. However, in the new environment in which eligible needy families are no longer entitled to cash assistance and the emphasis is on moving families off welfare into employment, concern about the condition of families no longer receiving aid has increased. The Congress and others are interested in the employment status of former welfare recipients, changes in family composition resulting from marriage and pregnancy, and the overall well-being of these families and their children. While the Congress has earmarked $5 million for HHS to study the outcomes of welfare reform and has taken other steps to monitor the status of poor families as discussed below, states are not federally required to report on the condition of former welfare families.

States’ greater responsibility for welfare programs under PRWORA has increased states’ need for information to support program management and decision-making, as well as to respond to the information requests from a variety of interested parties, such as service providers, advocacy groups, and the media. For example, some state legislatures are requiring state welfare agencies to report on outcomes from their reformed welfare programs, including the status of former welfare families. Consequently, many states have begun to track former welfare families. The data these states are reporting are the major source of information currently available on the condition of families who have left welfare.

Only those families who actually become welfare recipients and then leave the rolls are included in most state tracking studies. However, the changes in welfare can also have the effect of decreasing the number of families coming onto the welfare rolls. For example, many states have diversion strategies designed to prevent families from coming onto the welfare rolls by providing a needed service, such as child care or transportation, providing a one-time cash payment to overcome a barrier to employment, or requiring that applicants conduct a job search before receiving cash assistance. As a result, a comprehensive assessment of the postreform status of poor families with children would include information on TANF-eligible families who did not become welfare recipients as well as former welfare recipients.

To provide information on the postreform status of all low-income families, not just former welfare families, the U.S. Census Bureau at the direction of the Congress is conducting a longitudinal survey of a nationally representative sample of families, paying particular attention to eligibility and participation in welfare programs, employment, earnings, out-of-wedlock births, and adult and child well-being. Data from this survey, called the Survey of Program Dynamics, will help researchers and policymakers understand the impact of welfare reform on the well-being of low-income families and children by providing information on whether welfare recipients are finding jobs, what their earnings are, and what types of support they need to make the transition from welfare to work. In addition, the Urban Institute is conducting a multiyear project monitoring program changes and fiscal developments, along with changes in the well-being of children and families. As part of this project, the Urban Institute has surveyed nearly 50,000 people to obtain comprehensive information on the well-being of adults and children as welfare reform is being implemented in the states. A second survey is planned for 1999. Full results from the Census Bureau and Urban Institute surveys may not be available until the year 2000. In addition, a plethora of studies are under way that will be providing information in the future on various aspects of welfare reform.

State Studies Reported Some Information on the Status of Families Who Have Left Welfare

Seventeen states have collected data and reported on the status of some former welfare families in the key areas of economic status, family composition, or family and child well-being. The state studies differed in important ways, such as categories of families tracked, the length of time families were tracked, and the sources of follow-up data. Some of the studies presented no information on a substantial portion of the sample families, limiting the usefulness of these studies for drawing conclusions about the status of most former welfare families in the state. We determined that studies in 7 of the 17 states had enough data on a sample of families who had left welfare to generalize sample findings to the population of former welfare families from which the sample was drawn.

Seventeen States Reported Some Information on Economic Status, Family Composition, or Family and Child Well-Being

We identified a total of 18 state-sponsored or -conducted studies in 17 states—2 studies in Wisconsin and 1 in each of the other states—that reported on the status of families who left welfare in 1995 or later. The reports contain a broad range of information on economic status, family composition, and family and child well-being. Figure 1 summarizes the kinds of information reported in each of the 17 states and classifies the information according to the three major areas of interest. All of the studies reported information on economic status, all but one reported on family and child well-being, and most reported some information on family composition. Overall, 15 of the 17 states reported information in all three areas. (App. II lists the 17 states and their study reports.)

Studies Differed in Coverage, Timing, and Data Sources

Because states generally initiated tracking studies to meet their own information needs, the 18 studies in the 17 states differed in a number of important ways, including the categories of families tracked, geographic coverage, the time periods covered, and the timing and frequency of follow-up. The studies also differed in the sources of data used for tracking families who had left welfare. Table 1 summarizes key information on the studies, including the categories of families studied, the time periods involved, the frequency of follow-up, the time between leaving and follow-up, and the method of data collection.

Fourteen of the studies reported data on a statewide sample of families who left welfare for a range of reasons, and one study reported on a sample of families who left welfare in the state’s major city. The remaining three studies focused primarily on families who left welfare because of an adult recipient’s failure to comply with program requirements. These three studies were conducted, at least in part, because of concerns about the potential impact on family well-being of the loss of the entire cash benefit, rather than just a reduction in benefits, as was typically required under AFDC for noncompliance.

None of the studies reported specifically on families who had left welfare because of time limits. While there is great interest in the status of these families, in most states few families have reached their time limits, and in states where they have, few families have lost benefits as a result of the time limits. However, as states’ programs mature and more families reach the federal 5-year time limit on TANF benefits or state-established time limits of shorter duration, more of these families will be included in the tracking studies.

The studies also differed in the time period during which families left welfare and the length of time between the family’s exit and the study follow-up. The time at which states initiated a study of families who had left welfare depended in part upon when states’ reforms were implemented and when they needed information on the status of families affected by the reforms. The time periods of the 18 state studies ranged from as early as 1995 (before federal welfare reform) to as late as 1998 (after TANF was implemented in most states). The amount of time between leaving welfare and the follow-up also varied, ranging from 1 to 24 months. There were also differences in the frequency of follow-up. At least one state, Maryland, has been tracking families who have left welfare for a number of years and plans to track monthly samples of families for 2 years after they leave the rolls, whereas other states planned a one-time follow-up effort.

In addition, the studies used different sources of data to locate and track families. The Maryland study and the first Wisconsin study relied solely on administrative data, while other states’ studies were based on surveys of the former recipients using in-home visits, the telephone, or the mail. Some states’ studies used both survey and administrative data. Administrative data are case-specific information from the files of various programs, services, or agencies, including state unemployment insurance, food stamps, Medicaid, child welfare, child support enforcement, and criminal and education agencies. Since administrative data are limited to data collected for program management purposes, they may not be as focused on the questions of interest as are the survey data. On the other hand, administrative data may be less expensive to collect and more accurate than the self-reported data and can more readily than a survey provide information on large numbers of individuals.

Findings From Most State Studies Were Not Generalizable to the Population of Former Welfare Families in the State

We determined that eight tracking studies, covering seven states, (1) were designed to include most families who left welfare in the state at the time of the study and (2) had sufficient data on the sample of families tracked for the sample to be considered representative of families studied. These studies were designed to include families who left welfare for a range of reasons, although the studies varied in the specific category of families covered. For example, the Maryland study included all families who had left welfare, while the South Carolina study included only families with a household member required to seek employment who subsequently left welfare and had not returned at the time of follow-up.

Although none of the 18 studies were able to locate all families included in the samples to be tracked, eight studies had sufficient data on a sample of families to conclude that the sample represented the population from which it was taken. The nonresponse rates ranged from 15 to 88 percent for the state surveys. For the two studies using administrative data only, information about 8 percent and 18 percent of the families being tracked could not be found in the data being used. (See app. I for the proportion of families located in all 18 studies.)

Missing information for some members of a sample raises concerns about the representativeness of the remaining sample and whether findings can be generalized to the population from which the sample was drawn. Families who left welfare and subsequently responded to a survey and families about whom information was available in administrative data may be different in important ways from families for whom no information is available; thus, results based on such families are not generalizable to the entire population of families who left welfare in a state. Some policymakers and researchers are concerned that families who do not answer surveys or whose current status is no longer reflected in administrative data might be worse off than families for whom there are data. While the families who were not located may have fared quite well in terms of employment or family formation, some missing families may be experiencing hardship. For the purpose of summarizing findings, we either included only those studies that had data on at least 70 percent of the sample of families from the population of interest in the state or included a nonresponse analysis that showed no important differences between respondents and nonrespondents.

The seven states that we determined to have studies with results generalizable to their welfare populations are Indiana, Maryland, Oklahoma, South Carolina, Tennessee, Washington, and Wisconsin. We estimated that these seven states accounted for about 8 percent of the number of families who left welfare nationwide between October 1993 and June 1997. Figure 2 highlights these 7 states, along with the 10 other states that reported information on former welfare recipients.

Studies in the seven states had either (1) data on a high enough percentage of the sample to reasonably generalize the results to the population from which the sample was drawn or (2) an analysis showing that the nonrespondents had some of the same key characteristics as the respondents, providing greater assurance that the results from the limited sample could be generalized to the population from which the sample was drawn. (See app. I for a more detailed discussion of our assessment.)

Most Adults in Former Welfare Families Were Employed at Some Time After Leaving Welfare; Little Else Is Known About Family Well-Being

Because the seven states’ studies differ in several ways, as discussed above, the results are not completely comparable across states. However, the studies provide an indication of the status of families who had left welfare in these states at the time of the studies and, to the extent that the results are consistent, suggest a pattern of what is happening to these families. The studies had consistent findings on employment and earnings. Most former welfare families had an adult who was or had been employed since leaving welfare. Although the studies indicated that former recipients often worked at low-wage jobs, little information was available on families’ total household incomes, which could include child support or earnings from a second worker. Some studies also reported that significant proportions of the families had returned to welfare. In general, the studies provided little information on family and child well-being.

Adults Had Employment Rates of 61 to 87 Percent, but Little Is Known About Household Income

Employment rates ranged from 61 to 87 percent for adults in the families who left welfare in the seven states; however, these employment rates were measured in different ways. Studies measuring employment at the time of follow-up reported employment rates from 61 to 71 percent. Studies measuring whether an adult in a family had ever been employed since leaving welfare reported employment rates from 63 to 87 percent. In the four studies reporting both employment measures, the percentage employed at some time since leaving welfare was considerably higher than the percentage reporting employment at the time of follow-up. (Table 2 summarizes employment and earnings data in seven states.) These employment rates generally exclude families who returned to welfare, which can be a substantial portion of the families who leave welfare. In the three studies for which such data were available, the percentage of the families who initially left welfare and then returned to the rolls ranged from 19 percent after 3 months in Maryland to 30 percent after 15 months in Wisconsin. Removing families who return to welfare from the employment rate calculations results in higher employment rates than when they are included, since many former recipients who return to the welfare rolls are not employed.

While all eight studies reported some information on former recipients’ earnings or wages, the studies did not provide a complete story on hourly wages or number of hours worked. Average quarterly earnings for former recipients ranged from $2,378 to $3,786 in the studies that either reported quarterly earnings or for which we estimated quarterly earnings. Extrapolating these quarterly earnings to a year results in average annual earned incomes ranging from $9,512 to $15,144. These amounts of annual earned income are greater than the maximum annual amount of cash assistance and food stamps that a three-person family with no other income could have received in these states. However, if these earnings were the only source of income for the families after they left welfare, many of them would remain below the federal poverty level.

The question of whether a family is economically better off after leaving welfare than when receiving cash assistance is quite complex. The answer depends on many factors, including the amount of the cash benefit while on welfare, which varies by state, family size, and earnings while on welfare; family earnings and other sources of income; and aid after leaving welfare, as well as any work-related expenses. For example, the 1995-96 Wisconsin study that tracked families for more than 15 months after they left welfare compared postwelfare earnings of these families to the maximum benefit they could have received under AFDC to see if families were economically better off after leaving welfare. The study found that whether postwelfare earnings exceeded the maximum AFDC benefit depended in part upon the number of children in the family. Postwelfare earnings exceeded the maximum AFDC cash benefit for 54 percent of the families with one child and for 41 percent of the families with three or more children. The study also noted that because some families combine welfare and work, the combination of the cash benefit and earnings could result in some families on welfare having more cash income than families with earnings only. The study showed that during their first year off welfare, less than half of the families had cash incomes higher than their incomes had been while on AFDC, including both benefits and earnings.

While the tracking studies provide information on individuals’ earned incomes, much remains unknown about families’ total household income. For example, the studies generally do not provide information on whether others in a household have earnings or on other sources of household income, such as child support payments or financial assistance from relatives and friends. Moreover, most of the studies do not include comprehensive information on the receipt of other noncash benefits, such as food stamps, Medicaid, and child care or transportation assistance, or what employment-related expenses, including child care and transportation, households may have. Only three of the eight state studies had some information on household income. In the Oklahoma study, 57 percent of the former welfare families reported household incomes at or below the official poverty level. In the Indiana study, 57 percent of the families off welfare at follow-up reported monthly household income below $1,000. In contrast, Washington reported average total family income, including child support payments, equal to 130 percent of the federal poverty level for a family of three. According to the Washington study, 35 percent of the families who left welfare and had children received some child support, and 36 percent had at least one worker in the family other than the respondent to the survey. The 1995-96 Wisconsin study found that the proportion of families who had left and remained off welfare for at least 1 year who had earnings above the official poverty level varied by family size. While 35 percent of the families with one child and 24 percent of the families with two children had earnings above the poverty level, only 11 percent of the families with three or more children did.

Although these studies do not provide a comprehensive picture of families’ financial situations, they consistently indicated that many of the families leaving welfare were employed at fairly low-paying jobs. Our recent report on TANF implementation in seven states and other studies indicate that many states and localities are providing support services, such as case management services and financial assistance with child care, to help former welfare recipients maintain their employment. Several states and localities have also undertaken efforts to help these low-wage workers upgrade their job skills to improve their job prospects. Moreover, the recently expanded earned income credit can increase the incomes of qualified low-income families by as much as $2,271 for families with one child and $3,756 for families with two or more children. Information on total household income and receipt of government supports is key to understanding the condition of former welfare recipients and the extent to which they continue to rely on government aid rather than becoming economically self-sufficient.

The studies in five states reported on the extent to which former welfare families say they receive noncash public assistance. As shown in table 3, in these states, between 44 and 83 percent of the families who left welfare received Medicaid benefits, and between 31 and 60 percent received food stamps. The Wisconsin study that tracked families who left welfare between July 1995 and July 1996 for 15 months found significant decreases in the use of noncash public assistance over time. Forty-six percent of the former recipients who remained off welfare for at least 1 year received both Medicaid and food stamps in the first quarter after leaving welfare, and 28 percent received both in the fifth quarter after leaving cash assistance. Four studies had information on the receipt of child care subsidies.

While receiving AFDC or TANF, families generally also receive Medicaid benefits to cover their health expenses. However, whether Medicaid benefits are retained after a family has left welfare depends on many factors, and health insurance coverage after leaving welfare varied in the states with these data. For example, about 9 percent of the children in families who left welfare in South Carolina, about 20 percent in Oklahoma, and 35 percent in Indiana did not have health insurance at the time of follow-up. For adults who left welfare in these states, 24 percent in Oklahoma, 32 percent in Washington, 48 percent in South Carolina, and 54 percent in Indiana did not have health insurance.

While much attention is paid to welfare recipients who become employed and stay off the rolls, there is also interest in how those who are not employed and have not returned to welfare are faring. The South Carolina and Wisconsin surveys asked nonworking former recipients what stopped them from working for pay. In both states, the most frequently mentioned reason was their own physical or mental illness, followed by the inability to find a job, lack of transportation, and lack of child care. The Wisconsin study attempted to determine how these families were supporting themselves. Of the 142 former recipients not currently working, 18 percent were living with employed spouses or partners. Sixty-five percent of the families of the remaining nonworking former recipients were receiving Social Security, state unemployment insurance, child support, or foster care payments; 23 percent were not receiving cash assistance but were receiving noncash assistance, such as free housing, rent subsidies, Medicaid, or food stamps.

Studies in Seven States Provided Limited Information on Family Composition and the Well-Being of Children and Families

The studies in seven states provided limited information on the family composition and well-being of former welfare families and the children in these families. Although a major goal of welfare reform was the promotion of two-parent families and the reduction of out-of-wedlock pregnancies, the tracking studies report only minimal information on family composition at the time of data collection, and no information on changes that may have occurred just before or after leaving welfare. The studies with surveys asked questions regarding family composition; however, these surveys did not provide information on changes in the number of children in a family, changes in marital status, or the formation of other two-parent families since a family left the welfare rolls.

Further, beyond any inferences that could be drawn from the employment and earnings of parents, the studies provided little information on how former welfare children and families were doing relative to housing, health, education, food security, substance abuse, crime, and victimization. While some of the studies provided limited information on some of these factors, there are no comprehensive data on family and child well-being.

Three studies—from Maryland, Oklahoma, and Washington—reported on the number of children in former recipient families that had ever been involved with child protective services. These studies found few cases in which children had been involved with child protective services since leaving welfare. For example, the Maryland study reviewed state data from its foster care program to determine the number of children placed in foster care after their families left welfare. This study reported that less than one-half of 1 percent of the children studied entered foster care after their families left cash assistance.

Two studies, South Carolina’s as well as Wisconsin’s recent survey of families leaving welfare during the first quarter of 1998, asked former recipients to compare several aspects of their general well-being after leaving welfare with their situation when they were on welfare. Because Wisconsin used a modified version of the interview schedule developed in South Carolina, the data are comparable, even though the programs that the recipients experienced are not. Table 4 shows the results from the two states’ surveys. Former welfare recipients in both states were more likely to experience some deprivations after leaving welfare than while on welfare. At the same time, in South Carolina and Wisconsin, 76 and 68 percent, respectively, disagreed or strongly disagreed with the statement that “life was better when you were getting welfare.” Regarding housing status, an important aspect of well-being, the limited information from the studies did not suggest increased incidence of homelessness at the time of follow-up.

State and Federal Efforts Are Under Way to Improve the Usefulness of State Tracking Studies

The number of states conducting or sponsoring studies that track the status of families leaving welfare has increased in recent years, and state and federal efforts are under way to improve the usefulness of the data being collected. Thirty-nine states, including the 17 we identified in this report, and the District of Columbia already are tracking or plan to track families leaving welfare. In an attempt to improve the quality and comparability of these studies, HHS has funded several states and other jurisdictions to conduct tracking studies and is providing them technical assistance in conducting these studies.

State Tracking Efforts Are Increasing in Number and Expanding in Scope

We have identified 39 states and the District of Columbia that are either planning to study former welfare families or are already doing so. Most of the 17 states that we discuss in this report are planning to continue their tracking and to enhance their efforts, in some cases with federal funds and in other cases with state funds. Maryland, for example, plans to survey former recipients to get the kind of detailed information about families’ lives that is not available in the program data upon which state officials currently rely in their ongoing longitudinal studies. Among the topics to be covered are how former welfare families are able to make ends meet; what enabled them to leave the welfare system; and, in the cases of those who returned to welfare, what brought them back. Idaho is trying to locate families that did not respond to its survey. Maryland, Massachusetts, South Carolina, Wisconsin, and Mecklenburg County in North Carolina have received funds from HHS to support their efforts to link administrative data systems for purposes of studying the effects of welfare reform on other state and federal public assistance programs. A South Carolina official told us that by linking TANF data to state unemployment insurance data, the state was able to locate many of the families that did not respond to its survey.

Federal Efforts Are Helping States to Improve the Usefulness of the Data Being Collected on Families Leaving Welfare

To increase the usefulness of state tracking efforts in providing a more complete picture of the status of former welfare families, HHS is supporting some states and counties with funds and technical assistance. As part of its overall strategy to evaluate welfare reform, HHS has awarded grants to 14 projects covering 16 jurisdictions—10 states, five counties in 2 other states, and the District of Columbia—to support efforts to track, through administrative data, surveys, or other methods, former TANF recipients’ work transitions and receipt of other benefits, including supportive services. Each of these tracking efforts plans to collect information on one or more of the following: families diverted from welfare, eligible families who do not apply for benefits, and families who have left welfare. All 14 grantees will collect both administrative data and survey data on former recipients. (See app. III for information on the 14 studies.)

In addition, HHS submitted its overall research plan for evaluating welfare reform to the National Academy of Sciences for guidance on research design and recommendations for further research. The National Academy has convened a panel of experts on program evaluation methods, survey design, administrative record analysis, state database development and analysis, and welfare policy and evaluation to review data needs and methods. One of the panel’s first activities was to conduct a workshop to review and assess the HHS grantees’ study methodologies. The workshop provided a forum in which representatives from states and counties that had been awarded grants were able to talk with experts about their planned tracking studies. The panel plans a 30-month study on a broad range of issues related to evaluating welfare, with an interim report to be ready in June 1999.

HHS expects to use data from the 14 funded projects to generate a picture of what is happening to families exiting welfare and families diverted from ever entering welfare. In recognition of the need for high-quality research and comparable findings, HHS is providing technical assistance to the states directly, and through the National Academy panel, and is encouraging grantees to share information with one another. The National Academy panel is providing advice on issues of data quality and comparability as well as policy relevance. Initially, the 14 grantees have agreed to work toward increasing comparability across studies by using a common definition of welfare “leaver.” They have also agreed to clarify which studies will be tracking only families with adults and which will also track welfare cases that only include children. Finally, with the encouragement of HHS and the National Academy, the grantees will be sharing common approaches to studying such areas as insecurity and deprivation, child well-being, and changes in household composition.

Observations

While we were able to learn about the status of former welfare recipients in several states, we could conclude little about the status of most families that have left welfare nationwide. However, the limited information on economic status of the families being tracked indicates that many families who leave welfare find jobs that are low-paying. The low wages of these jobs emphasize the importance that income supports, such as subsidized medical and child care and the earned income credit, can assume in these families’ total financial resources. As we noted in our earlier report on TANF implementation in seven states, federal and state policies and programs for assisting low-income working families are likely to play a critical role in helping these families remain off cash assistance and move toward economic self-sufficiency.

But much remains unknown about most families leaving welfare nationwide. In our attempt to describe the condition of former welfare families, we were constrained by the data currently available from these early state tracking studies. More specifically, the high nonresponse rates in many state studies limit the usefulness of the results because generalizations cannot be made to all families of interest. Because those families who do not respond to surveys, or who may not show up in administrative data for other programs, may be the ones at greatest risk of negative outcomes, some policymakers and program officials are particularly concerned about not having enough information to determine the status of these families. In addition, for policymakers to better understand whether states are making progress in meeting the goals of TANF, more comprehensive information is needed on household income; receipt of government assistance; and changes in family composition, including increases in the number of two-parent families and additional births, especially to teens. And, finally, the data often are not comparable among the states. Consequently, even if each state had collected generalizable data on a comprehensive range of topics, it would often be difficult to generate a national picture from such studies. More comparable data would also be useful to individual states that want to understand how former welfare families fare in their states as compared with those in other states. In addition, comparable data among the states could help policymakers and program administrators at all levels of government identify promising approaches and practices for assisting low-income families.

The limited nature of the information currently available emphasizes the importance of additional state efforts, such as those funded by HHS. The ongoing state efforts promise to provide a more complete picture in the future. Many more states have tracking studies in progress or planned and efforts are under way at the state and federal levels to improve the usefulness of these efforts. As HHS continues to work with states to support their efforts to collect data on families who have left welfare, it has an opportunity to help states develop more generalizable, comprehensive, and comparable data.

Agency Comments

We obtained comments on a draft of this report from HHS, which stated that the report provides useful information on the status of former welfare recipients and the varied efforts being made by states to follow up on the impacts of welfare reform. HHS also noted, however, how difficult it was to glean general results from such varied studies. It expressed concern that, while the report appropriately discusses the issues that preclude generalizing findings in many of the studies to the state level, the report does not address other factors, such as differences in the definitions of the populations studied and in states’ economic conditions, which make it difficult to report general results from any of the studies. HHS further suggested that rather than attempting to find areas of comparability in the studies, we should focus on the crucial differences between studies and emphasize the contribution that the HHS-funded state studies of families leaving welfare will make. Finally, HHS had concerns about our reliance upon studies with low response rates.

We agree with HHS about the difficulties involved in discussing general results from the varied studies. We did not suggest, however, that results from the eight studies could be generalized beyond the states from which the study samples were drawn. We also pointed out that much remains unknown about most families who have left welfare nationwide. In addition, we were not attempting to report program impacts, which would require controlling for other factors that could affect family status, such as varying economic conditions. Rather, we focused on what is currently known about the status of former welfare families given the extent to which a particular state study was generalizable to the study population within the state. We also agree with HHS’ concern regarding low response rates, and this was the reason most of the studies were determined to not be generalizable and no data from them were included in the report. Finally, the report had already noted that we believe the HHS-funded state efforts will make an important contribution toward improving the usefulness of future studies and increase understanding of the condition of former welfare families.

To address HHS’ concerns, we revised the report to place greater emphasis on the studies’ differences by moving detailed information on the studies’ varied populations, time periods, and methodologies from the appendix to the body of the report. We also added an additional caveat to the discussion about employment and earnings information, pointing out the lack of complete comparability among the studies. In addition, where results from several states were displayed, we added information on time periods and references to more detailed information on the studies’ populations and methodologies. HHS also made technical comments, which we incorporated where appropriate. (See app. IV for the text of HHS’ comments.)

We also provided copies of the draft report to the 17 states whose studies we reviewed and to an expert on welfare reform issues. We incorporated their technical comments where appropriate.

As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from the date of this report. At that time, we will send copies to the Honorable Donna E. Shalala, Secretary of the Department of Health and Human Services; state TANF directors; and other interested parties. We will also make copies available upon request.

If you or your staff have any questions about this report, please contact me on (202) 512-7215. Other staff who contributed to this report are listed in appendix V.

Scope and Methodology

This appendix provides more detail on how we (1) identified state studies of families who had left welfare, (2) assessed the extent to which the studies could support statewide generalizations of results, and (3) summarized the findings on employment and income from the studies with results generalizable to each state’s welfare population.

Identifying and Assessing the Studies

To obtain information to answer this request, we searched for state studies of families who left welfare during or after 1995 that had published results by September 30, 1998. We began with a list of state tracking efforts prepared by a joint effort of the staffs of the National Conference of State Legislatures, the National Governors’ Association, and the American Public Human Services Association. In addition, we talked to representatives of the 10 states with the largest welfare caseloads (California, Florida, Illinois, Michigan, New Jersey, New York, Ohio, Pennsylvania, Texas, and Washington) and asked if they had any studies of former welfare families for which results had been published. We also talked to welfare experts and asked them if they knew of any ongoing studies of former welfare recipients. Finally, we talked to representatives of those states that we identified as having published reports on the basis of their tracking efforts and asked what additional plans, if any, they had for tracking former welfare families and if they had updated their work. If a state had more recent information available, we included it in our analysis when possible, in some cases using a report published after September 30, 1998.

Through this process, we identified 18 separate tracking efforts in 17 states. The 18 studies varied in degree of data completeness and statewide generalizability. We were interested in summarizing results that could reasonably be generalized to most families who left welfare in the state at the time of the study. We considered a study to be of an acceptable level of statewide generalizability if the study successfully obtained data on at least 70 percent of the sample of families for which it sought such data, or if a nonresponse analysis of the data showed that were no important differences between families represented in the data and those missing from the data. Except for this assessment, we did not independently verify the data included in the state studies. Using this assessment, we identified eight studies representing seven states. The proportion of families that responded to surveys or for whom data were located in administrative databases in each study is shown in table I.1.

Response rate (percentage)

Families who left TANF July to Dec. 1997 Families receiving AFDC May 1995 to May 1996 who subsequently left AFDC AFDC families assigned to or who volunteered for the Limited Benefit Plan Nov. 1994 to Apr. 1995 Families who left TANF Jan. to Nov. 1997 Families in metropolitan New Orleans who left TANF Jan. to Mar. 1998 Families who left TANF Oct. 1996 to Sept. 1997 Families whose AFDC benefits were terminated in Apr. 1996 because they did not comply with program rules Families who received or left AFDC or TANF Mar. 1996 to Sept. 1997 Families whose TANF benefits were terminated Jan. to Feb. 1998 because of failure to comply with program rules Families who left AFDC July 1996 to June 1997 Families who left or were denied TANF Oct. 1996 to Nov. 1997 (continued)

Response rate (percentage)

Data not available.

Summarizing Data Related to Employment and Income

We summarized the results for the eight studies that we considered could reasonably be generalized to the state level in the three major areas of interest: economic status, family composition, and family and child well-being. This effort was constrained by the different sources of data used by the state studies, different categories of families tracked, and different questions asked of respondents in the surveys. The area for which each state study had somewhat comparable data was economic status. The state studies with surveys generally asked employment status at the time of follow-up, hourly wage rate, number of hours worked, and whether the respondent had ever worked since leaving welfare. The two studies relying on state unemployment insurance program data to track employment could report only whether an individual had worked at some point during a 3-month period and total earnings for the period. Further, since state unemployment insurance programs do not cover some categories of employed individuals, program data would not have information on these individuals. For example, self-employed individuals and certain agricultural workers are generally not covered.

The two studies reporting average quarterly earnings based on state unemployment insurance program data did not report average hourly wage or average number of hours worked. In order to make earnings data comparable among the eight studies, we estimated average quarterly earnings for the other studies for which we had, or could calculate, average hourly wage rates and average number of hours worked. We multiplied the average hourly wage in each of these studies by the study’s reported average number of hours worked in a week and multiplied by 13 to estimate a quarterly wage. This enabled us to compare estimated quarterly earnings with the reported quarterly earnings. We also had to make some adjustment to ensure that the employment rates were for comparable categories of families. Although some of the studies reported employment rates only for adults who left welfare and were still off the rolls at the time of follow-up, others included all families who left the rolls during the study period—even those who had returned to welfare at the time of follow-up. To estimate comparable employment rates, we removed from the calculation data on the individuals who returned to the rolls and assumed that those who returned to the rolls were not employed.

Reports From States’ Studies of Families Who Left Welfare

Idaho

Project Self-Reliance TAFI Participant Closure Study (II), Idaho Department of Health and Welfare, spring 1998.

Indiana

The Indiana Welfare Reform Evaluation: Assessing Program Implementation and Early Impacts on Cash Assistance, Abt Associates, Inc., Aug. 1997.

The Indiana Welfare Reform Evaluation: Who Is On and Who Is Off? Comparing Characteristics and Outcomes for Current and Former TANF Recipients, Abt Associates, Inc., Sept. 1997.

The Indiana Welfare Reform Evaluation: Program Implementation and Economic Impacts After Two Years, Abt Associates, Inc., and The Urban Institute, Nov. 1998.

Iowa

Iowa’s Limited Benefit Plan: Summary Report, Mathematica Policy Research, Inc., and the Institute for Social and Economic Development, May 1997.

A Study of Well-Being Visits to Families on Iowa’s Limited Benefit Plan, Mathematica Policy Research, Inc., June 1998.

Kentucky

From Welfare to Work: Welfare Reform in Kentucky, Welfare Reform Evaluation No. 1, Center for Policy Research and Evaluation, Urban Studies Institute, University of Louisville, Jan. 1998.

Louisiana

Exiting Welfare: The Experiences of Families in Metro New Orleans, School of Social Work, Southern University at New Orleans, June 1998.

Maryland

Life After Welfare: An Interim Report, University of Maryland School of Social Work, Sept. 1997.

Life After Welfare: Second Interim Report, University of Maryland School of Social Work, Mar. 1998.

Michigan

A Study of AFDC Case Closures Due to JOBS Sanctions April 1996, Michigan Family Independence Agency, May 1997.

Montana

Montana’s Welfare Reform Project: Families Achieving Independence in Montana FAIM, February 1998 Update, Montana Department of Public Health & Human Services, Feb. 12, 1998.

New Jersey

WFNJ (TANF) Sanction Survey, New Jersey Department of Human Services, July 2, 1998.

New Mexico

Survey of the New Mexico Closed-Case AFDC Recipients July 1996 to June 1997, Final Report, University of New Mexico, Sept. 1997.

Oklahoma

Family Health & Well-Being in Oklahoma: An Exploratory Analysis of TANF Cases Closed and Denied October 1996 to November 1997, Oklahoma Department of Human Services, Sept. 1998.

Pennsylvania

TANF Closed-Case Telephone Survey, Pennsylvania Department of Public Welfare, Mar. 1998.

South Carolina

Survey of Former Family Independence Program Clients: Cases Closed During January Through March 1997, South Carolina Department of Social Services, Division of Program Quality Assurance, Mar. 3, 1998.

Survey of Former Family Independence Program Clients: Cases Closed During July Through September 1997, South Carolina Department of Social Services, Division of Program Quality Assurance, Oct. 9, 1998.

Tennessee

Summary of Surveys of Welfare Recipients Employed or Sanctioned for Non-Compliance, University of Memphis, Mar. 1998.

Washington

Washington’s TANF Single Parent Families Shortly After Welfare: Survey of Families Which Exited TANF Between December 7 and March 1998, Washington DSHS Economic Services Administration, July 1998.

Washington’s TANF Single Parent Families After Welfare, Washington DSHS Economic Services Administration, Jan. 1999.

Wisconsin

Post-Exit Earnings and Benefit Receipt Among Those Who Left AFDC in Wisconsin, Institute for Research on Poverty, University of Wisconsin-Madison, Aug. 17, 1998.

Post-Exit Earnings and Benefit Receipt Among Those Who Left AFDC in Wisconsin, Institute for Research on Poverty, University of Wisconsin-Madison, Oct. 30, 1998.

Survey of Those Leaving AFDC or W-2 January to March 1998, Preliminary Report, State of Wisconsin, Department of Workforce Development, Jan. 13, 1999.

Wyoming

A Survey of Former POWER Recipients (Personal Opportunities With Employment Responsibilities), Western Management Services, LLC, for Wyoming Department of Family Services, May 1998.

Information on Selected Grants Awarded by the Department of Health and Human Services to Study Families Leaving Welfare

This appendix provides information on the study methodologies planned by the jurisdictions receiving grants from HHS to study families who have left welfare.

Timing of survey (months after exit)

Jan. to June 1997 and Dec. 1998 to Feb. 1999 Oct. to Dec. 1996 and Oct. to Dec. 1997 Nov. 1997 to Mar. 1998 and Jan. to Mar. 1999 Oct. to Dec. 1996 and Jan. to Mar. 1998 Jan. to Mar 1997 and Jan. to Mar. 1999 (continued)

Timing of survey (months after exit)

This category includes other data sources, such as sources of tax, welfare-to-work, and health information.

Comments From the Department of Health and Human Services

Major Contributors to This Report

Related GAO Products

Welfare Reform: States’ Experiences in Providing Employment Assistance to TANF Clients (GAO/HEHS-99-22, Feb. 26, 1999).

Domestic Violence: Prevalence and Implications for Employment Among Welfare Recipients (GAO/HEHS-99-12, Nov. 24, 1998).

Welfare Reform: Early Fiscal Effects of the TANF Block Grant (GAO/AIMD-98-137, Aug. 18, 1998).

Welfare Reform: Child Support an Uncertain Income Supplement for Families Leaving Welfare (GAO/HEHS-98-168, Aug. 3, 1998).

Welfare Reform: Many States Continue Some Federal or State Benefits for Immigrants (GAO/HEHS-98-132, July 31, 1998).

Welfare Reform: Changes Will Further Shape the Roles of Housing Agencies and HUD (GAO/RCED-98-148, June 25, 1998).

Welfare Reform: States Are Restructuring Programs to Reduce Welfare Dependence (GAO/HEHS-98-109, June 18, 1998).

Welfare Reform: Transportation’s Role in Moving From Welfare to Work (GAO/RCED-98-161, May 29, 1998).

Medicaid: Early Implications of Welfare Reform for Beneficiaries and States (GAO/HEHS-98-62, Feb. 24, 1998).

Welfare Reform: States’ Efforts to Expand Child Care Programs (GAO/HEHS-98-97, Jan. 13, 1998).

Welfare Reform: States’ Early Experiences With Benefit Termination (GAO/HEHS-97-74, May 15, 1997).

Welfare Waivers Implementation: States Work to Change Welfare Culture, Community Involvement, and Service Delivery (GAO/HEHS-96-105, July 2, 1996).

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.