1

Gaining the Skills for Employment: 

The Literacy of Female Public Assistance Recipients

Authors 

Andrea Amodeo 

Ying Jin 

Joanna Kling 

American Institutes for Research

June 2009 

Contents 

Introduction....................1

Profile of Women Who 

Were Recipients of 

Public Assistance............2

Background 

Characteristics ................ 2

Prose and Quantitative 

Literacy .........................4

Profile of Women with 

Lower Prose Literacy ....5

Background 

Characteristics ................ 5

Characteristics Most 

Associated with Low 

Prose Literacy Among 

Women Who Were 

Recipients of Public 

Assistance........................6

Summary ........................7

References.......................7

Appendix A: 

Methodology and 

Technical Notes ..........A-1

Descriptions of 

Background Variables. A-1

Statistical Procedures.. A-2

Estimation of Literacy 

Proficiency.................. A-3

Introduction 

The 2007 report Literacy in Everyday Life, based on the results of the 2003 National 

Assessment of Adult Literacy (NAAL), described respondents’ employment status and their 

self-assessments of how their literacy skills may limit job opportunities. According to the 

report, at least half the individuals with Below Basic prose, document, and quantitative literacy 

levels were not in the labor force. When women in particular do not have the literacy and job 

skills that are needed for today’s workforce, many must turn to public assistance (Wedgeworth 

2004). The 2003 NAAL data revealed that, in general, more literate women who received 

assistance reported that their need for it was shorter than women with lower levels of literacy 

(Kutner, Greenberg, Jin, Boyle, Hsu, and Dunleavy 2007). Therefore, in order for women 

receiving public assistance to enter or re-enter the workforce or to make the transition to jobs 

that will support their families, they must have the literacy skills necessary to locate and 

perform sustainable jobs. 

Using data collected from the 2003 NAAL, this report highlights findings that compare the 

literacy of women who reported receiving public assistance with the literacy of women who 

reported never receiving public assistance. Receipt of public assistance was measured using the 

item from the NAAL background questionnaire that asked respondents to report whether they 

received “Temporary Assistance to Needy Families (TANF), public assistance, or public 

welfare payments from the state or local welfare office” within the past 12 months. Women 

who responded that they had received assistance in the past 12 months are considered those 

who were current recipients for the purposes of these analyses. Analyses are restricted to 

women, because only women are eligible to receive TANF payments. Many of the background 

variables examined in this report are based on self-reported data, and because many of the 

variables are related to one another, complex interactions and relationships among them cannot 

be explored. Therefore, readers are cautioned not to draw causal inferences based solely on the 

results presented here. 

The 2003 NAAL assessed the English literacy of adults (ages 16 and older) in the United States 

for the first time since the 1992 National Adult Literacy Survey. The NAAL provided 

information on the literacy proficiency of approximately 18,000 adults living in households and 

1,200 prison inmates. In the household sample, 331 women reported that they had received 

public assistance in the past 12 months and formed the analysis sample for the current recipients 

of public assistance in this report. In addition to assessing the literacy skills of respondents, the 

NAAL gathered extensive background information on their demographic and socioeconomic 

characteristics (e.g., age, country of birth, schooling, labor force status), as well as on their 

literacy practices. 

The NAAL measured respondents’ proficiency on three literacy scales: prose, document, and 

quantitative. For each, proficiency was measured on a scale that ranged from 0 to 500. Scores 

on each of the three literacy scales were characterized in terms of four literacy proficiency 

levels: Below Basic, Basic, Intermediate, and Proficient.

1

 These analyses focused on the prose 

and quantitative literacy scales; in some sections, the analyses focused on the prose literacy 

scale only. A detailed description of background variables and methodology used in this report 

is provided in Appendix A: Methodology and Technical Notes. 

 

1

 For an interpretation of the literacy scales and performance levels on the NAAL assessment, see Kutner, M., Greenberg, E., Jin, Y., Boyle, B., 

Hsu, Y., and Dunleavy, E. (2007). Literacy in Everyday Life: Results From the 2003 National Assessment of Adult Literacy (NCES 2007-48). 

U.S. Department of Education. Washington, DC: National Center for Education Statistics. 
2

Profile of Women Who Were Recipients of 

Public Assistance 

Background Characteristics 

Demographic Characteristics 

The results of the 2003 NAAL shown in Table 1 indicate that 

the majority of women who were receiving public assistance 

were relatively young, White, U.S. natives. Nearly half (48%) 

of the women who were current recipients were ages 25-39. A 

higher percentage of current recipients were ages 19-39 than 

those who had never received public assistance. The 

race/ethnicity of the recipients was distributed among Whites 

(39%), Blacks (34%), and Hispanics (22%). Significantly 

more current recipients were Black or Hispanic than those 

who had never received public assistance in each of these 

racial/ethnic groups. As shown in Table 2, current recipients 

of public assistance who were White were underrepresented 

compared with the general household population, and Black 

and Hispanic current recipients were overrepresented. 

Among women who were current recipients of public 

assistance, 85 percent were native to the United States. 

Approximately the same percentages of women who were 

current recipients and those who had never received public 

assistance reported that their first language was English (only 

or with another language). 

Table 1. Percentage distribution of women who were 

currently receiving public assistance and women 

who had never received public assistance, by 

selected characteristics: 2003 

Characteristic 

Current 

public 

assistance 

recipients 

Never 

received

 public 

assistance 

Age 

16-18 — 6 

19-24 20 10* 

25-39 48 25* 

40-64 22 41* 

65 or older — 17 

Race/ethnicity 

White 39 73* 

Black 34 10* 

Hispanic 22 12* 

Other — 5 

Country of birth 

U.S. 85 87 

Other 15 13 

Language spoken before starting school

English only or with other 84 87 

Other only 16 13 

See notes at end of table. 

Table 1. Percentage distribution of women who were currently 

receiving public assistance and women who had never 

received public assistance, by selected characteristics: 

2003—Continued 

Characteristic 

Current 

public 

assistance 

recipients 

Never 

received

 public 

assistance 

Highest educational attainment 

Still in H.S. — 4 

Less than/some H.S. 40 13* 

H.S. grad/GED or equivalency diploma 34 31 

Postsecondary 24 53* 

Have you used a computer? 

Yes 65 78* 

No 35 23* 

Computer with Internet access 

Yes 37 69* 

No 63 31* 

Computer literacy 

Computer literate 52 71* 

Not computer literate 48 29* 

Employment status 

Employed full- or part-time 35 58* 

Unemployed 21 5* 

Out of labor force 44 37* 

Participation in job training 

Yes 37 39 

No 63 61 

Number of children 

No children 23 63* 

1 child 29 16* 

2 children 24 14* 

3 children or more 25 8* 

School involvement 

2 activities or fewer 34 33 

3 activities 30 28 

4 activities 37 39 

Voting 

Did not vote 59 32* 

Voted 41 68* 

*Significantly different from those who were currently receiving public 

assistance at the significance level of .05. 

— Sample size is insufficient to permit a reliable estimate. 

NOTES: Percentages may not sum to 100 because of rounding. The 

"Other" category includes Asians, Pacific Islanders, Native Hawaiians, 

American Indians, Alaska Natives, and Multiracial adults. 

Highest Educational Attainment and Computer Literacy 

Among women who were current recipients of public 

assistance, 40 percent had attained less than/some high school 

education. This represents a higher percentage of women with 

less than a high school diploma (or equivalency) than among 
3

women who had never received assistance. As shown in 

Table 2, women who were current recipients of welfare and 

had less than/some high school were overrepresented 

compared with the general household population. A higher 

percentage of women who had never received public 

assistance reported attaining postsecondary education, such as 

earning an Associate’s degree, graduating college, or earning a 

graduate degree, than women who were current recipients. 

A smaller percentage (65%) of women who were current 

recipients of public assistance reported having used a 

computer than those who had never received assistance (78%). 

Similarly, a smaller percentage (37%) of women who were 

current recipients of public assistance reported having a 

computer with Internet access than those women who had 

never received assistance (69%). Nearly half (48%) of the 

women who were current recipients of public assistance were 

not computer literate. In contrast, 71 percent of those who had 

never received assistance had at least some computer literacy. 

Table 2. Percentage distribution of women who were currently 

receiving public assistance, women who had never 

received public assistance, and the household 

population, by race/ethnicity and highest educational 

attainment: 2003 

Characteristic 

Current 

public 

assistance 

recipients 

Never 

received 

 public 

assistance 

Household 

adults 

Race/ethnicity 

White 39* 73 71 

Black 34* 10 11 

Hispanic 22* 12 12 

Other — 5 6 

Highest educational attainment 

Still in H.S. — 4 3 

Less than/some H.S. 40* 13* 15 

H.S. grad/GED or 

equivalency diploma 34 31 31 

Postsecondary 24* 53* 51 

*Significantly different from the household population distribution at 

the significance level of .05. 

— Sample size is insufficient to permit a reliable estimate. 

NOTES: Percentages may not sum to 100 because of rounding. The 

"Other" category includes Asians, Pacific Islanders, Native Hawaiians, 

American Indians, Alaska Natives, and Multiracial adults. 

Employment and Job Training 

The NAAL background questionnaire also collected 

information on respondents’ labor force and employment 

activities at the time of the assessment and during the previous 

12 months. Compared with those who had never received 

public assistance, a lower percentage of women who were 

current recipients were employed full- or part-time. Similarly, 

a higher percentage of current recipients reported that they 

were unemployed or out of the labor force2

 than those who 

had never received assistance. Among the out-of-the-laborforce respondents, the majority reported that they were 

“keeping house.” There were no significant differences 

between women who were current recipients of public 

assistance and women who had never received assistance in 

terms of their participation in job training or education,

including courses, workshops, formal on-the-job training, or 

apprenticeships that helped them perform their job better, 

obtain a job, or earn a promotion. 

Children, School Involvement, and Voting 

Among women who were current recipients of public 

assistance, higher percentages had one or more children than 

those women who had never received assistance. Parents with 

school-aged children were asked whether they had been 

involved in their children’s school(s) during the previous year 

in any of the following ways: 

 Volunteered to help out at the school, including in the 

classroom, on a field trip, or at a school event such as 

a party or school fair 

 Gone to a parent-teacher or other type of meeting at 

the school 

 Spoken individually with a teacher to see how their 

children were doing in school 

 Sent food or other items to share in the classroom 

Approximately one-third of both current recipients and those 

who had never received assistance participated in two or fewer 

activities. More than one third of the women in each group 

reported involvement with four activities during the past year. 

Among citizens of the United States, significantly more 

women who were current recipients of public assistance (59%) 

reported that they did not vote in the 2000 presidential election 

than those who had never received assistance (32%). 

Public Assistance Participation 

As shown in Table 3, in regard to the length of time for which 

women reported receiving assistance, the largest percentage of 

women (39%) who were current recipients reported receiving 

assistance for more than three years during their lifetime. 

Although the NAAL does not provide information regarding 

the specific courses taken, the majority (62%) of recipients 

had not taken any classes sponsored by a program to help 

them get off of public assistance. 

 

2

 For more information about the “out of labor force” category, please 

refer to the Labor Force Participation variable description in 

Appendix A. 
4

Table 3. Percentage distribution of women who were 

currently receiving public assistance, by selected 

public assistance participation characteristics: 

2003 

Public assistance participation 

Current public 

assistance 

recipients 

Length of time on public assistance 

1 year or less 27 

More than 1 year but less than 2 years 20 

2 to 3 years 14 

More than 3 years 39 

Taken classes to help get off public 

assistance 

Yes 38 

No 62 

NOTE: Percentages may not sum to 100 because of rounding. 

Prose and Quantitative Literacy 

Estimates of the mean prose and quantitative literacy scores of 

current recipients of public assistance and those who had 

never received assistance are displayed in Figure 1. Women 

who were current recipients of public assistance had 

significantly lower prose and quantitative literacy scores than 

those who had never received assistance. 

The literacy scores can also be used to assign individuals to 

one of the four levels of literacy performance. The percentage 

distributions across proficiency levels on the prose and 

quantitative scales are displayed in Figure 2. Higher 

percentages of women who were current recipients of public 

assistance had Below Basic or Basic prose literacy than 

women who had never received public assistance. A higher 

percentage of current recipients had Below Basic quantitative 

literacy than women who had never received public assistance. 

Studies have shown that the majority of public assistance 

recipients were not employed and that a substantial percentage 

of them had characteristics that made gaining and retaining 

employment difficult, such as low educational attainment, 

limited work experience, limited English proficiency, and low 

basic skills (Fagnoni 2001). A multivariate examination of 

TANF recipients’ background characteristics and level of 

literacy proficiency described in the sections above showed 

similar results. For example, women with children under the 

age of 18 (especially unmarried mothers), women who lacked 

a high school diploma, and unemployed women were more 

likely to receive public assistance. 
5

Profile of Women with Lower Prose Literacy 

This section examines selected characteristics of women 

receiving public assistance with the two lowest prose 

proficiency levels (i.e., Below Basic and Basic). In this report, 

these women are referred to as the lower literacy or less 

literate group. Special focus was paid to the lower literacy 

women who were recipients of public assistance because they 

were considered to be in greatest need of literacy interventions 

to improve their skills and employability. 

Background Characteristics 

Demographic Characteristics 

As shown in Table 4, nearly half (46%) of the less literate 

women who were current recipients were ages 25-39. The 

race/ethnicities of less literate women who were current 

recipients were represented by similar percentages of Blacks 

(37%) and Whites (34%). Nearly one-quarter of the women in 

this group were Hispanic. By contrast, a higher percentage 

(55%) of the less literate women who had never received 

assistance was White. Similarly, the percentage of lower 

literacy current recipients who were Black (37%) was larger 

than those who had never received assistance (16%). 

Table 4. Percentage distribution of women who were currently 

receiving public assistance with low prose literacy and 

women who had never received public assistance with 

low prose literacy, by selected characteristics: 2003 

Characteristic 

Current public 

assistance 

recipients with 

low prose 

literacy 

Never received 

public 

assistance with 

low prose 

literacy 

Age 

16-18 — 6 

19-24 17 10 

25-39 46 21* 

40-64 27 36* 

65 or older — 26 

Race/ethnicity 

White 34 55* 

Black 37 16* 

Hispanic 24 23 

Other — 6 

Country of birth 

U.S. 79 74 

Other 21 26 

Language spoken before starting school 

English only or with other 78 75 

Other only 22 25 

Highest educational attainment 

Still in H.S. — 4 

Less than/some H.S. 50 28* 

H.S. grad/GED or 

equivalency diploma 33 38 

Postsecondary 18 31* 

Have you used a computer? 

Yes 54 58 

No 46 42 

Computer with Internet access 

Yes 31 51* 

No 69 49* 

Computer literacy 

Computer literate 38 49 

Not computer literate 62 51 

Employment status 

Employed full- or parttime 28 45* 

Unemployed 25 6* 

Out of labor force 47 49 

See notes at end of table. 
6

Table 4. Percentage distribution of women who were currently 

receiving public assistance with low prose literacy and 

women who had never received public assistance with 

low prose literacy, by selected characteristics: 2003—

Continued 

Characteristic 

Current public 

assistance 

recipients with 

low prose 

literacy 

Never received 

public 

assistance with 

low prose 

literacy 

Number of children 

No children 27 64* 

1 child 27 16 

2 children 19 12 

3 children or more 27 8* 

School involvement 

2 activities or fewer 36 39 

3 activities 31 28 

4 activities 33 32 

Voting 

Did not vote 61 42* 

Voted 39 58* 

*Significantly different from low literacy women who were currently 

receiving public assistance at the significance level of .05. 

—Sample size is insufficient to permit a reliable estimate. 

NOTES: Percentages may not sum to 100 because of rounding. The 

"Other" category includes Asians, Pacific Islanders, Native Hawaiians, 

American Indians, Alaska Natives, and Multiracial adults. 

Highest Educational Attainment and Computer Literacy 

Half of the less literate women who were current recipients of 

public assistance had less than/some high school as their 

highest educational attainment. This is significantly higher 

than for those in this group who had never received public 

assistance. Conversely, and as expected, a smaller percentage 

of the lower literacy women who were current recipients 

(18%) had attained a postsecondary education than lower 

literacy women who had never received assistance (32%). 

Nearly half (46%) of the lower literacy women who were 

current recipients of public assistance had not used a 

computer, and more than two-thirds of women in this group 

(69%) did not have a computer with access to the Internet. 

Among both women who were current recipients and women 

who had never received assistance, more than half of the 

respondents (62%) were not computer literate. 

Employment 

Significantly fewer lower literacy women who were current 

recipients of public assistance were employed full- or parttime (28%) than those women who had never received public 

assistance (45%). 

Children and Voting 

Although the number of children varied among low literacy 

current recipients of public assistance, the majority of less 

literate women who had never received assistance had no 

children. A higher percentage of low literacy current 

recipients (27%) had three or more children than did women 

with low literacy who had never received assistance (8%). 

Among citizens of the United States, a higher percentage of 

lower literacy current recipients (61%) reported that they did 

not vote in the 2000 presidential election than did less literate 

women who had not received assistance (42%). 

Public Assistance Participation 

For these analyses, we examined lower literacy women who 

were current recipients of public assistance in comparison to 

their higher literacy counterparts. The length of time 

respondents received public assistance was similarly 

distributed across categories of time (see Table 5) for both 

lower and higher literacy recipients. Among both literacylevel groups, the highest percentages of recipients—44 percent 

of lower literacy recipients and 29 percent of higher literacy 

recipients—had received public assistance for more than three 

years. 

Table 5. Percentage distribution of women who were currently 

receiving public assistance with low prose literacy and 

women who were currently receiving public 

assistance with high prose literacy, by selected public 

assistance participation characteristics: 2003 

Public assistance 

participation 

Current public 

assistance 

recipients with 

low prose 

literacy 

Current public 

assistance 

recipients with 

high prose 

literacy 

Length of time on public assistance 

1 year or less 28 25 

More than 1 year but less 

than 2 years 18 26 

2 to 3 years 11 21 

More than 3 years 44 29 

Taken classes to help get off public 

assistance 

Yes 38 39 

No 63 61 

NOTE: Percentages may not sum to 100 because of rounding. 

Characteristics Most Associated with Low 

Prose Literacy Among Women Who Were 

Recipients of Public Assistance 

To identify the characteristics most associated with the low 

prose literacy of women who were current recipients of public 

assistance, we conducted multivariate analyses (see Appendix 

A for details on the methodology). Such analyses allowed us 

to disentangle differences in the characteristics among women 

who were current recipients with Below Basic, Basic, and 

Above Basic (i.e., Intermediate and Proficient) prose literacy. 

In general, among the factors investigated, educational 

attainment, race/ethnicity, computer literacy, and participation 
7

in job training were found to be associated with the literacy 

level of women receiving public assistance. 

For example, women who were current recipients lacking a 

high school diploma or a GED certificate were, not 

surprisingly, considerably more likely than those who had 

received postsecondary education to have Below Basic literacy 

relative to Basic or Above Basic literacy. Women who were 

current recipients whose highest level of education was high 

school or high school equivalency were six times more likely 

than those having received postsecondary education to have 

Below Basic literacy relative to Above Basic literacy. 

Among women who were current recipients of public 

assistance, Black women were nearly four times more likely 

than White women to have Below Basic literacy relative to 

Basic literacy. In contrast, women who were current 

recipients with at least some computer literacy were only 0.17 

times as likely as those who were not computer literate to have 

Below Basic literacy relative to Above Basic literacy. 

Women who were current recipients of assistance who had 

participated in job training were more than three times more 

likely than those who had never participated in job training to 

have Below Basic literacy relative to Basic literacy. 

Summary 

This report, based on the 2003 NAAL assessment data, 

examines the characteristics of women who were current 

recipients of public assistance as well as the relationship 

between various characteristics and the English literacy 

proficiency of this population group. The key findings in this 

report are as follows: 

 With regard to women who were current recipients of 

public assistance, whereas White women were 

underrepresented in comparison with the household 

population, Black and Hispanic women were 

overrepresented. 

 A higher percentage (42%) of women who were 

current recipients of public assistance had less 

than/some high school education than did women 

who had never received public assistance (17%). 

 The majority (44%) of women who were current 

recipients of public assistance reported that they were 

out of the labor force (e.g., keeping house). The 

majority (58%) of women who had never received 

assistance reported that they were employed full- or 

part-time. 

 A higher percentage of women who were current 

recipients of public assistance reported having 

children (78%) than those who had never received 

assistance (37%). 

 More than one third (39%) of women who were 

current recipients of public assistance reported 

receiving such for more than three years during their 

lifetime. The majority (62%) of these recipients had 

not taken classes sponsored by a program to help 

them get off of public assistance. 

 Women who were current recipients of public 

assistance had significantly lower prose and 

quantitative literacy scores than those who had never 

received assistance. 

 Compared with lower literacy women who had never 

received assistance (49%), a higher percentage (62%) 

of women who were current recipients with lower 

literacy reported they did not have a computer with 

Internet access. Among both current recipients and 

those who had never received assistance, more than 

half of the less literate women were not computer 

literate. 

 Among current recipients of public assistance, there 

were no differences in the length of time receiving 

public assistance for women with higher versus lower 

prose literacy. 

 Relative to more literate women who were receiving 

public assistance, women who were Black, were not 

computer literate, lacked participation in job training, 

and earned lower oral passage reading scores were 

more likely to have lower literacy. 

References 

Allen, N. L., Carlson, J. E., and Zelenak, C. A. (1999). The 

NAEP 1996 Technical Report (NCES 1999-452). 

U.S. Department of Education. Washington, DC: 

National Center for Education Statistics. 

Fagnoni, C. M. (2001). Welfare Reform: Moving the Hard-toEmploy Recipients Into the Workforce. Report to the 

Chairman, Subcommittee on Human Resources, 

Committee on Ways and Means, House of 

Representatives. Washington, DC: General 

Accounting Office. 

Kutner, M., Greenberg, E., Jin, Y., Boyle, B., Hsu, Y., and 

Dunleavy, E. (2007). Literacy in Everyday Life: 

Results From the 2003 National Assessment of Adult 

Literacy (NCES 2007-480). U.S. Department of 

Education. Washington, DC: National Center for 

Education Statistics. 

Mislevy, R.J. (1984). Estimating Latent Distributions. 

Psychometrika, 49(3): 359–381. 

Mislevy, R.J. (1985). Estimation of Latent Group Effects. 

Journal of the American Statistical Association, 

80(392): 993–997. 

Mislevy, R.J. (1991). Randomization-Based Inference About 

Latent Variables from Complex Samples. 

Psychometrika, 56(2): 177–196. 

Thomas, N. (1993). Asymptotic Corrections for Multivariate 

Posterior Moments with Factored Likelihood 

Functions. Journal of Computational and Graphical 

Statistics, 2(3): 309–322. 

Wedgeworth, R. (2004). State of Adult Literacy 2004. 

Syracuse, NY: ProLiteracy. 
A-1

Appendix A: Methodology and Technical Notes 

This section describes the background variables and statistical 

procedures used in this report. It also provides a brief 

explanation of the direct estimation method and the plausible 

values method used to estimate the NAAL proficiency scores. 

For information on survey methodology (e.g., sampling, data 

collection, weighting and variance estimation, scaling) for the 

NAAL, see Literacy in Everyday Life: Results from the 2003 

National Assessment of Adult Literacy (Kutner, Greenberg, 

Jin, Boyle, Hsu, and Dunleavy 2007). 

Descriptions of Background Variables 

Age 

All respondents were asked to report their birthdates, and this 

information was used to calculate their age. Age groups 

reported are 16 to 18, 19 to 24, 25 to 39, 40 to 49, 50 to 64, 

and 65 and older. Age groups were selected to correspond to 

key life stages of many adults: 

 16–18: Completion of secondary education 

 19–24: College or job training 

 25–39: Early career 

 40–49: Mid career 

 50–64: Late career 

 65 and older: Retirement 

Race and Ethnicity 

In 2003, all respondents were asked two questions about their 

race and ethnicity. The first question asked them to indicate 

whether they were Hispanic or Latino. Then, all respondents, 

including those who indicated they were Hispanic or Latino, 

were asked to choose one or more of the following groups to 

describe themselves: 

 White 

 Black or African American 

 Asian 

 American Indian or Alaska Native 

 Native Hawaiian or other Pacific Islander 

Individuals who responded “yes” to the first question were 

coded as Hispanic, regardless of their answer to the second 

question. Individuals who identified more than one group on 

the second question were coded as Multiracial. Respondents 

of Native Hawaiian or Pacific Islander origin were grouped 

with those of Asian origin. 

Language Spoken Before Starting School 

All respondents were asked what language or languages they 

learned to speak before starting school. Their responses were 

then used to divide respondents into five groups: English only, 

English and Spanish, English and other language, Spanish 

only, or Other language(s). The English and Spanish category 

includes adults who spoke languages in addition to both 

English and Spanish. 

Highest Educational Attainment 

All respondents were asked to indicate the highest level of 

education they had completed. The following options were 

provided: 

 Still in high school 

 Less than high school 

 Some high school 

 GED or high school equivalency 

 High school graduate 

 Vocational, trade, or business school after high 

school 

 College: less than 2 years 

 College: Associate’s degree (A.A.) 

 College: 2 or more years, no degree 

 College graduate (B.A. or B.S.) 

 Postgraduate, no degree 

 Postgraduate degree (M.S., M.A., Ph.D., M.D., etc.) 

Respondents who reported less than high school or some high 

school were asked how many years of education they had 

completed. For certain analyses, some of these groups were 

collapsed. 

Computer Literacy 

The NAAL background questionnaire collected data from 

respondents on performing various activities using a 

computer. Specifically, respondents were asked how often 

(every day, a few times a week, once a week, less than once a 

week, never) they: 

 Send or receive an email message 

 Write using a word processing program 

 Use a spreadsheet program or use a financial program 

 Look up information on a CD-ROM 

 Find information on the Internet 

On the basis of these questionnaire items, a computer literacy 

scale was created such that respondents who had never 

performed any of these five computer activities were 

considered to have no computer literacy, whereas those who 

had at least some experience with at least one of the five items 

were considered to have at least some computer literacy. 

Labor Force Participation 

The NAAL background questionnaire also collected 

information on respondents’ labor force and employment 

activities at the time of the assessment and during the previous 

12 months. Responses to the questions on current 

employment status at the time of the assessment were used to 

assign each respondent to one of the following labor force 

statuses: employed full-time; employed part-time; employed, 

not-at-work; unemployed, looking for work; unemployed, not 

looking for work; and out-of-the-labor-force. The out–of-thelabor-force group included individuals who were classified as 

neither employed nor unemployed: students not looking for 

work, retirees, persons keeping house, persons who were 

disabled, and those who did not wish to work at the present 

time for other reasons. 
A-2

Participation in Job Training 

Respondents were asked in separate questions whether during 

the past year they had participated in any training or 

education, including courses, workshops, formal on-the-job 

training, or apprenticeships, intended to help improve job 

performance, earn a promotion, or obtain a job. 

School Involvement 

Respondents were asked four questions to indicate the number 

of different types of activities they were involved in at their 

child’s or grandchild’s school. They were asked whether 

during the past year they had done the following: 

 Volunteered to help out at their child’s (one of their 

children’s/grandchild’s/grandchildren’s’) school(s), 

including in the classroom, on a field trip, or at a 

school event such as a party or school fair? 

 Gone to a PTA or other type of parent meeting at 

their child’s (one of their children’s/ 

grandchild’s/grandchildren’s’) school(s)? 

 Spoken individually with their child’s (one of their 

children’s/grandchild’s/grandchildren’s’) teacher(s) 

to see how he or she was doing in school? 

 Sent food, or other items to share in their child’s (one 

of their children’s/grandchild’s/ grandchildren’s’) 

classroom(s)? 

Respondents were grouped according to the number of 

questions to which they answered “yes” as none, one, two, 

three, or four. 

Voting 

All respondents who either were born in the United States or 

indicated in their response to a separate question (that was 

asked only of people not born in the United States) that they 

were citizens of the United States were asked whether they 

remembered whether or not they voted in the 2000 presidential 

election. If they said they remembered whether or not they 

voted in the election, they were asked whether they voted. 

Respondents who did not remember whether they voted were 

treated as missing data for this question. 

Participation in Public Assistance 

Respondents were asked whether they or anyone in their 

household had received TANF, public assistance, or public 

welfare payments from the state or local welfare office during 

the previous 12 months or whether they had ever received 

public assistance in the past. Respondents were identified as 

never, past, or current participants in public assistance. 

Time Receiving Public Assistance 

Respondents were asked about how long, in total, they had 

received public assistance payments in their lifetime: less than 

6 months, 6 months to 1 year, more than 1 year but less than 2 

years, 2 to 3 years, more than 3 years. 

Statistical Procedures 

Tests of Statistical Significance 

All comparisons discussed in this report have been tested for 

statistical significance using the t statistic. Statistical 

significance was determined by calculating a t value for the 

difference between a pair of means, or proportions, and 

comparing this value with published tables of values at a 

certain level of significance, called the alpha level. The alpha 

level is an a priori statement of the probability of inferring that 

a difference exists when, in fact, it does not. The alpha level 

used in this report is .05, based on a two-tailed test. 

Differences in the means and proportions between subgroups 

were calculated using the following t statistic: 

where p1 and p2 are the estimates to be compared and se1 and 

se2 are their corresponding standard errors. When a subgroup 

was compared to a total group, a modification of the standard 

error of difference was made to adjust for group dependence. 

The formula for the adjusted standard error of difference was 

as follows: 

where p is the proportion of the total group contained in the 

subgroup. 

Minimum Sample Sizes for Reporting Subgroup Results 

In the NAAL reports, the sample sizes were not always large 

enough to permit accurate estimates of proficiency and/or 

background results for one or more categories of variables. 

For results to be reported for any subgroup, a minimum 

sample size of 45 was required. This number was arrived at 

by determining the sample size needed to detect an effect size 

of 0.5 with a probability of 0.8 or greater, using a design effect 

of 1.5. This design effect implies a sample design-based 

variance 1.5 times that of a simple random sample. The effect 

size of 0.5 pertains to the true difference in a given mean 

estimate (e.g., mean proficiency) between the subgroup in 

question and the total population, divided by the standard 

deviation of that estimate in the total population. An effect 

size of 0.5 was chosen following Cohen (1988), who classifies 

effect size of this magnitude as “medium” as well as to be 

consistent with what was done in the 1992 National Adult 

Literacy Survey (NALS). 3

Multinomial Logistic Regression Analyses 

Proficiency on the NAAL literacy scales (i.e., prose, 

document, and quantitative) is measured on a scale that ranges 

from 0 to 500. The performance of adults on the assessment 

 

3 Cohen, J. (1988). Statistical Power Analysis for the Behavioral 

Sciences (second edition). Hillsdale, NJ: Lawrence Erlbaum 

Associates.
A-3

can be reported as either mean scores on the scale or on the 

basis of the distribution of adults across the NAAL 

performance levels (Below Basic, Basic, Intermediate, and 

Proficient). Each performance level describes the abilities 

associated with score ranges on the NAAL scale. 

In investigating the effects of multiple factors on low literacy 

for women who were current recipients of public assistance, 

we conducted multinomial logistic regression analyses. Such 

analyses allowed us to disentangle differences between the 

characteristics of women who were current recipients of 

public assistance with Below Basic literacy and those with 

Basic literacy, while holding constant a series of other 

explanatory variables. Similarly, it also allowed us to examine 

differences between the characteristics of adults with Basic

literacy and those of adults with literacy levels in the next 

highest categories (Intermediate and Proficient). 

Multinomial logistic regression is a form of regression used 

when the dependent variable is categorical with more than two 

classes and the independent variables are of any type.4

 It 

allows the simultaneous comparison of more than one contrast 

(e.g., the probability of Below Basic vs. Basic literacy, Basic

vs. Above Basic literacy, Below Basic vs. Above Basic 

literacy) and usually expresses the impact of predictor 

variables on dependent variables in terms of odds ratios. 

The odds ratio for a given independent variable represents the 

factor by which the odds change in the dependent variable for 

a one-unit change in the independent variable. For example, if 

the odds ratio for success in a given performance test for 

females versus male is 3.5, and if this odds ratio is statistically 

significant, we would say that the odds of success for females 

are 3.5 times as large as for males. 

The statistical significance of the odds ratio estimates are 

indicated by the confidence interval for the odds ratio. If the 

confidence interval around the odds ratio contains the value of 

1.0, then the change in the value of the independent variable is 

not associated with change in the odds of the dependent 

variable. Thus, that independent variable is not considered a 

useful predictor in the logistic model. 

In our multinomial logistic regression analyses, the outcome 

measure was the NAAL literacy performance level: Below 

Basic, Basic, and Above Basic (i.e., Intermediate and 

Proficient combined). Using the literacy levels rather than the 

NAAL scale scores as the dependent variables in the model 

made the analyses more easily interpretable. If the continuous 

NAAL scale scores had been used, the results would need to 

be discussed in terms of unit changes on the NAAL scale per 

unit change in an independent variable. The impact of specific 

variables would be more difficult to grasp in this approach, 

given the abstract nature of the NAAL scale. The predictor 

variables in the model were age, race/ethnicity, country of 

birth, educational attainment, length of time receiving public 

 

4

 For more information on multinomial logistic regression, see 

Hosmer, D., and Lemeshow, S. (2004). Applied Logistic Regression. 

New York: John Wiley & Sons. 

assistance, participation in courses to get off public assistance, 

computer literacy, employment status, participation in job 

training, and oral passage reading scores as measured in 

Fluency Addition to NAAL.5

Table A-1 reports the odds ratio estimates from the 

multinomial regression of the prose literacy performance level 

on the set of predictor variables described above. 

Estimation of Literacy Proficiency 

The NAAL used a complex assessment design that allowed 

maximum coverage of the broad domain of literacy while 

minimizing the time burden on any one respondent. Under 

this design, the NAAL administered only a fraction of the 

assessment items on each literacy scale to each respondent. 

Although individual respondents were required to take only a 

small portion of the entire pool of assessment questions, the 

aggregate results across the entire assessment allowed broad 

reporting of literacy for the targeted population. However, 

because respondents did not receive enough literacy tasks to 

provide reliable information about individual performance, 

traditional test scores for individual respondents would have 

resulted in biased estimates of population characteristics and 

therefore were not appropriate to use for estimates of 

population statistics. 

To obtain unbiased estimates of population statistics (e.g., 

subgroup means or percentages in each proficiency level), the 

NAAL used methods derived from Marginal Maximum 

Likelihood (MML) estimation. Such MML estimation 

procedures were available with AM software.6

 Estimates for 

average literacy scores and percentages in each literacy 

proficiency level in this report were all obtained using the 

direct estimation method with AM. The multinomial logistic 

regression analyses could not be conducted using MML direct 

estimation because the procedure is not available in AM. 

Instead, an alternative estimation procedure called plausible 

values methodology was used for the multinomial logistic 

regression analyses. Plausible values were initially developed 

for the National Assessment of Educational Progress (NAEP; 

Mislevy 1984, 1985, 1991; Thomas, 1993) to allow secondary 

users to estimate statistics derived from individual data. 

Plausible values are multiple imputations randomly drawn 

from a distribution derived from the MML parameter 

estimates for an extensive conditioning model (Allen, Carlson, 

and Zelenak 1999). 

 

5

 As part of the NAAL assessment, adults were asked to read a series 

of short passages aloud. Their responses were recorded and later 

scored for accuracy and speed. 

6

 For more information on direct estimation methodology followed 

for NAAL and the use of AM, see Baldi, S. (Ed.) et al. 

(Forthcoming). Technical Report and Data File User’s Manual for 

the 2003 National Assessment of Adult Literacy. U.S. Department of 

Education. Washington, DC: National Center for Education Statistics. 
A-4

Table A-1. Odds ratio estimates from multinomial regression analyses for women who were current recipients of public 

assistance 

Below Basic vs. Basic Basic vs. Above Basic 

Below Basic vs. Above 

Basic 

Effect 

Point 

Estimate 

95% Wald 

Confidence 

Limits 

Point 

Estimate 

95% Wald 

Confidence 

Limits 

Point 

Estimate 

95% Wald 

Confidence 

Limits 

Race/ethnicity: Black vs. White 3.83 1.28 12.13 -- -- -- 4.18 1.55 11.49 

Education: Less than/some high school vs. 

Postsecondary 9.42 3.96 23.87 2.71 1.04 7.15 14.18 4.35 49.89 

Education: High school/GED vs. 

Postsecondary 5.36 1.78 16.80 -- -- -- 6.45 1.78 23.86 

Computer literacy: Some vs. None 0.25 0.10 0.63 -- -- -- 0.17 0.06 0.48 

Job training participation: Yes vs. No 3.27 1.15 9.65 -- -- -- -- -- -- 

Oral passage reading score -- -- -- 0.98 0.97 0.99 0.98 0.96 0.99 

-- Estimates not significant and not shown. 

Note: Results were only shown for predictors with significant odds ratio estimates. 

It is important to recognize that plausible values are not test 

scores for individuals, and they should not be treated as such. 

Plausible values are randomly drawn from the distribution of 

scores that could be reasonably assigned to each individual. 

As such, the plausible values contain random error variance 

components and are not optimal as scores for individuals. 

In our multivariate analyses, five plausible values for each 

adult were obtained as estimates of scores on the prose literacy 

scale. These plausible values were then used to assign each 

individual to one of the NAAL performance levels. Five sets 

of multinomial regression analysis were conducted, using each 

of the five plausible values. The reported odds ratio estimates 

are the average of the five odds ratio estimates using each of 

the five plausible values. It should be noted, however, that the 

standard errors used in the significance tests for the reported 

odds ratio estimates were not adjusted for variation among the 

five sets of results given the complexity of the computations 

and the unavailability of an estimation procedure in the 

statistical software. Therefore, the confidence limits around 

the odds ratio estimates might be narrower than they would be, 

had the standard errors been corrected.