Monthly Labor Review May 1999 27

 Quality of Projections

The quality of BLS projections:

a historical account

Broad trends in occupational employment

have been projected fairly accurately,

but accuracy has not increased over time

Neal H. Rosenthal

Neal H. Rosenthal is

Associate Commissioner, Office of

Employment

Projections, Bureau

of Labor Statistics.

P rojections of employment change provide

the foundation for the statements on employment outlook presented in the Occupational

Outlook Handbook. Consequently, the quality of

employment outlook information presented in the

Handbook is very dependent on the accuracy of

the projections. Prior to 1960, discussions of employment outlook were based primarily on informed

judgments about the direction and magnitude of employment change rather than a formal set of numerical projections. Since 1960, the Handbook

statements on employment outlook have been based

on a set of statistical projections. The first numerical projections published by BLS covered the 1960–

70 period. However, data were published only for

major occupational groups.1 BLS did not publish

information on projected numerical change for detailed occupations until 1966.2 Since then, projections have been developed and published every

other year, conforming to the biennial production

of the Occupational Outlook Handbook.

The procedures for developing the projections

have changed significantly over the years stemming

from research devoted to improving methods for

developing employment projections. In addition,

more employment and related economic data of

higher quality have become available over time for

use in developing projections. BLS economists also

gained experience in developing projections and

gathered information from past efforts through detailed methodological and analytical memorandums

prepared by economists who preceded them.

This article discusses the changes in procedures

used to develop projections over the past 50 years

and presents data to see if the quality of the projections improved as the projection procedures changed.

Measurement of projection accuracy, however, is not

a simple task. One traditional measure is to compare

projected employment with actual employment and

compute the difference in percentage terms. For example, if employment were projected to increase

from 8,000 in the base year to 11,000 in the target

year and actual employment in the target year were

10,000, the projections would have an absolute error

of 1,000, or 10 percent. However, actual employment

estimates themselves may have error. For example, with

a standard error of 10 percent, true employment in the

base year could have varied from 7,200 to 8,800 and

in the target year from 9,000 to 11,000. Thus, in

any evaluation effort, significant weight also should

be assigned to a discussion of employment change

in reports presenting the projections, such as the Occupational Outlook Handbook, in addition to the

absolute levels of employment.

Space is not available in this article to present data

evaluating all detailed occupational projections ever

published by the BLS, so it focuses on projections of

the major occupational groups over time. It presents

data from five sets of projections prepared from 1960

to 1984 that roughly coincide with procedural

changes put in place. Detailed evaluations have been

published in the past for those projections and information from the articles presenting those evaluations was used in this article. They can be seen on

the BLS website3 by those seeking more information.

Changes in procedures

Mid 1940s through the mid-1960s. From the beginning of the occupational outlook program through
28 Monthly Labor Review May 1999

the mid-1960s, projections for each occupation were developed independently by the analyst responsible for preparing

the Occupational Outlook Handbook statement rather than as

part of a comprehensive model. In general, statistical extrapolations of historical employment data were used to develop

projections or simple regressions that related employment to

total population growth. For some occupations such as schoolteachers, the projections were based on statistical relationships,

such as pupil-teacher ratios to projections of the school-age

population. Although the projections were developed in numerical terms, the information was presented in the Handbook

only in qualitative terms, such as “continued rapid growth is

expected” or “growth is expected to be slow.” The only projections published were for major occupational groups.

Total employment growth was controlled by projections of

the labor force, minus an assumption about the unemployed.

Labor force and employment data, used as the base year of the

projections, were derived either from the decennial census or

the Current Population Survey (CPS). The projections, therefore, were on a “persons concept,” used in these surveys in

which individuals are counted once in occupational employment data in the job they worked the most hours during the

reference week of the survey.

Mid-1960s through the 1970s. A major procedural change

began with the 1964–75 projections released in 1966—the use

of an industry-occupation matrix. Although the matrix was

used in developing occupational projections for many occupations, some occupations covered in the Occupational Outlook Handbook were not projected in the matrix because the

necessary data on employment by industry were not available.

For the most part, projections for these occupations were developed as they were in earlier periods. Current and projected

employment estimates for these occupations were meshed with

the matrix projections in publications presenting the projections.4 In the mid-1970s, after the 1970 census data became

available, fewer occupations were developed independent of

the matrix because the number of occupations covered in the

1970 census was much greater than that in the 1960 census.

The main source of data on occupational staffing patterns

of industries used to develop the industry-occupation matrix

was the decennial census. Industry employment data were

derived from the Current Employment Statistics program. The

1960 census data were used until the 1970 census data became available to use in developing the 1974–85 projections.

The census data were adjusted to develop data for the base

year of the projections because the census industry classification was not directly comparable to the industry classification

used by BLS in the Current Employment Statistics Survey. The

matrix staffing patterns also were adjusted to make use of data

from a number of sources that were considered more preferable than census data. Among the sources used were scientists and engineers by industry from BLS surveys of employers;5 teacher and librarian employment collected by the Department of Education; and occupational employment data

collected by regulatory agencies for interstate commerce, including railroads, airlines, and telephone and telegraph communications. Data on Federal Government occupational employment were derived from payroll data obtained from the

Civil Service Commission, which later became the Office of

Personnel Management. Employment data collected from

sources other than the Government also were used, including

data for some medical and health occupations, compiled by

professional associations. After applying the census staffing

patterns to Current Employment Statistics industry employment, data from these sources were placed in the matrix and

the estimates of employment for the major occupational groups

in each major industry were forced to add to actual current

year data from the CPS.

With the use of an industry-occupation matrix to develop

occupational projections, the development of industry employment projections took on a great deal of importance. Three

approaches were used to develop detailed industry projections

from the mid-1960s through the 1970s. The first involved the

use of regression analysis to develop employment projections,

for each detailed industry, that were consistent with projections of total employment and overall economic growth. Equations were developed that related industry wage and salaryworker employment change to combinations of the following

variables: Real gross national product (GNP), national unemployment rate, Armed Forces personnel, Civilian noninstitutional population, and time. These variables were selected because they allowed cyclical and other factors to be separated

from secular trends.

A second approach to developing detailed industry projections involved the use of input-output analysis. Essentially,

this technique requires that final demand (GDP divided into its

components—investment, consumption, Government expenditures, and foreign trade) be specified by producing industry.

The demand is then traced back through the chain of production to determine the output from each industry supplying

materials or services to produce the end product. For example,

the final demand for automobiles creates an intermediate demand for steel, rubber, and so forth. The intermediate demand

for steel then will create a demand for iron ore, coal, and so

forth. By calculating total output requirements for each industry and relating it to projected output per worker-hour in each

industry, BLS derived a projection of employment.

The third approach was to conduct an in-depth study of

industries for which past employment growth was not considered indicative of future growth or for which the models described earlier had poor statistical results. In this approach,

BLS examined factors that were expected to influence future

employment growth and developed and tested a variety of regression equations. For example, in projecting employment

for the motor vehicle industry, various models were developed

Quality of Projections
 Monthly Labor Review May 1999 29

using such variables as personal disposable income, driving

age population, motor vehicle registrations, and number of

families with income over certain levels. The final projections took into account all the types of models used and, in

effect, the projected levels of employment that were used were

those judged to be best.

Projections of the occupational structure of each industry

in the industry-occupation matrix were based on an analysis of

the factors that could cause change in the structure. Initial

projections of occupational coefficients in each industry were

based on historical trends developed by extrapolating data from

the 1950 to 1960 census. These trends then were modified

based on an analysis of the factors that caused change in the

past, such as changes in technology, product mix, and business practices, and were assessed as to whether those factors

would continue to have an impact on occupational staffing of

industry in the future. Some occupations were analyzed independent of the matrix and the resulting projections were placed

into the matrix. For example, employment projections of elementary and secondary school teachers were based on analyses of pupil-teacher ratios and projections of school enrollments. Projections of automobile mechanics were based on

analyses of the growth in the number of motor vehicles.

The industry-occupation matrix only provided employment

of wage and salary workers. To develop projections of total

employment in each occupation, separate projections were

developed for self-employed workers and unpaid family workers. These projections were based on trends developed from

CPS data and trends in ratios of wage and salary workers to the

other classes of workers.

From the mid-1960s through the 1970s, as in earlier periods, the labor force projections were used as the major control

for projecting total employment. Unemployment, based on an

assumed unemployment rate used in developing the projections, was subtracted from the labor force and the resulting

number of persons was used as the control total for the sum of

wage and salary, self-employed, and unpaid family workers in

all occupations.

1980 through 1998. In 1980, the 1978–90 projections were

developed using the first national industry-occupation matrix

based on occupational staffing patterns of industries from the

Occupational Employment Statistics survey rather than decennial census data. Census data were deficient for analyzing

trends in industry staffing patterns because they were collected

only every 10 years and had limited occupational detail. BLS

initiated the Occupational Employment Statistics survey in the

early 1970s to collect data on occupational staffing patterns of

industries directly from establishments by mail survey and to

do so more frequently. This survey, which continues today, is

a Federal/State cooperative program in which data are collected by State employment security agencies according to standards, procedures, and methods developed by BLS. Until 1995,

all nonagricultural industries,6 except private households, were

covered in this survey on a 3-year cycle; manufacturing industries during the first year, and roughly half of nonmanufacturing

industries in each of the next 2 years.

Because data for all States were not available until the late

1970s, it was not until 1980 that a national matrix for 1978

based on data from the Occupational Employment Statistics

survey could be developed. Occupational employment estimates for 1978 were obtained by applying the occupational

staffing patterns of industries from the Occupational Employment Statistics survey to total wage-and salary employment

from the Bureau’s Current Employment Survey. The Occupational Employment Statistics survey included many more detailed industries and occupations than did census data. As a

result, employment projections for virtually all occupations in

the Handbook were developed through the industry-occupation matrix. Although independent analyses continued, such

as projecting teachers’ employment based on pupil-teacher ratios, those data were placed in the matrix and had to fit within

total employment and constraints of employment in all other

occupations in the educational services industry.

Wage-and-salary employment totals for agricultural and

private household industries continued to be obtained from

the CPS because the Occupational Employment Statistics survey did not cover employment in these industries. Occupational distributions of employment in these industries were developed from the census-based matrix and, therefore, detailed

occupations in the census-based matrix were reclassified into

the occupational classification used in the Occupational Employment Statistics survey. This was somewhat different from

the classification used in the census. The Occupational Employment Statistics surveys also do not cover self-employed

workers and unpaid family workers, but employment estimates

for these workers also were developed from CPS- and censusbased data.7 To develop total employment estimates by occupation, employment of wage-and-salary workers was added to

totals of self-employed and unpaid family workers.

Employment estimates for detailed occupations in the Occupational Employment Statistics survey-based matrices were

not comparable with those in previous census-based matrices

because of major differences in the underlying data sources.

The census counts persons, whereas the Occupational Employment Statistics survey counts jobs. Thus, wage and salary

worker employment in the Occupational Employment Statistics-based matrix is higher than that in the census-based matrix, because of multiple jobholders. The difference between

the numbers of jobs and of persons employed in 1978 was

roughly 10 percent, but it varied among occupations. Also, in

the census and CPS, individuals report themselves in the occupation in which they work the most hours. Respondents to

the Occupational Employment Statistics surveys are instructed

to report employees performing more than one job in the one

that requires the highest skill level. The definitions for each
30 Monthly Labor Review May 1999

occupation are listed on the questionnaire. In the census, the

titles reported by respondents are grouped into categories that

may include workers with greatly different skill levels. The

categories usually take the title of the most prominent occupation in that group. For example, the title “lawyer” includes

lawyers and law clerks. These are separate occupations in the

Occupational Employment Statistics survey.

In the 1978–90 occupational projections, alternative projections also were developed for the first time, and this practice continued until the 1996–2006 projections were issued in

1998. The alternatives included a base case scenario and highand low-growth scenario. For discussions in the Occupational

Outlook Handbook and related career guidance publications,

only the base case was used. It was believed that young people,

planning their career, could not easily differentiate the nuances

of high- and low-growth economic scenarios and would be

best served by discussions of employment change that seemed

most likely to BLS analysts—the base case scenario. In the

1996–2006 projections, a decision was made not to develop

alternative projections because very little use was made of those

projections and some uses that were made were not necessarily proper ones.8

Beginning with the 1978–90 projections, a major change

also was made in the development of industry projections. As

discussed earlier, three basic procedures were used to develop

industry projections from the mid-1960s through 1980. In

1980, all efforts concentrated on projections developed through

the input-output methodology described earlier, and abandoned

the regression analysis procedure. By placing more resources

in one effort, BLS economists believed that the quality of the

projections could be improved. In-depth analysis of specific

industries, however, continued and these analyses were coordinated with work on the models used to develop the projections.

Also during this period, the projections reflected a “jobs

concept” rather than a “persons concept.” Wage and salary

worker employment data reflect payroll employment and therefore include all jobs an individual may hold. CPS data on selfemployed include workers whose primary job was as a selfemployed worker or unpaid family worker. Thus, if those workers held a wage and salary worker job as well, all their jobs

would be included. Excluded from the job count are self-employed jobs that are not a workers’ primary job. Estimates of

those jobs will be added to the projections in the 1998–2008

projections to be issued in November 1999.

Despite changes in procedures, use of new data sources,

and other technical aspects of the methods used to develop

projections over from the 1950s to the 1990s, two factors were

constant. First, projections of the labor force always were a

significant determinant of growth in total employment, whether

the projections were of individuals or of jobs. This is very

significant as the projections of the labor force consistently

have been the most accurate of the many components of the

projections. Second, in-depth analysis of factors causing

change in employment of industries and occupation always

was used extensively in conjunction with the statistical models. Changes in technology, whether it affects industry–specific productivity or the staffing patterns of an industry, changes

in business practices, and the impact of social change can affect occupational employment significantly. Such changes do

not necessarily follow historical patterns in a manner that can

be captured in models without changing the statistical relationships in the models that reflect historical patterns. Consequently, when the results of the analyses conflicted with the

results of the statistical trends specified in the models, the independent analyses were given significant weight in the specification of the final projections.

Evaluating the projections

BLS considers evaluation to be an important component of its

projections program. Through evaluations, Bureau analysts

have been able to identify the strengths and weaknesses in procedures used to prepare the projections and that information

has been used to change procedures in developing later projections. The evaluations also have provided users of projections, especially in the fields of career guidance and education

planning, with information to enhance their understanding of

the problems faced in developing accurate projections.

Many obstacles are faced in evaluating occupational projections. One is that the occupational projections are highly

interrelated with the labor force, economic, and industry employment projections. For example, the labor force projections

have a great bearing on total employment projections, and industry projections influence occupational projections, because

occupations tend to be concentrated by industry. Thus, inaccurate industry projections that stem from inaccurate assumptions underlying the economic projections have a significant

impact on the growth of occupations in that industry. Also,

identifying the factors that caused differences between projected and actual data is sometimes difficult to uncover 10 or

more years after the projections were developed. Another

impediment faced in evaluating the projections stems from the

changes in occupational classification that occur between the

time the projections were developed and the target year. As a

result, the projected data are not often comparable with actual

data. With each decennial census, occupational classification

changes are made that affect data in the CPS, which uses the

census classification. In occupational surveys, such as the Occupational Employment Statistics survey, changes to occupational classification that are made to improve the survey’s quality can be made at any time and can cause havoc with data

comparability over time. Finally, as mentioned earlier, survey

data used to measure employment in both the base year and

target year of the projections are subject to normal sampling

and response error. Consequently, it is possible that the actual

Quality of Projections
 Monthly Labor Review May 1999 31

data used to measure error in projections have errors that could

distort conclusions of projections reliability.

The projections from the 1950s through 1994 were all made

to target years ending in 0 or 5, generally because of a belief

that projections to a specific year such as 1974 or 1977, would

imply more preciseness than projections to 1975 or to 1980.

In addition, the projections to a specific year are meant to imply changes in employment over the projected period rather

than precise time specific projections. The first publication

presenting comprehensive detailed occupational employment

projections stated:

“...the use of the target year 1975 is not meant to imply that

the projections of requirements will be realized in that year

and that year only, and regardless of the cyclical conditions

which prevail at that time. The projections are thus meant to

apply to a year in the mid-1970s when the stated assumptions

correctly describe the state of the economy.”9

The implied preciseness of the accuracy and error of the

occupational projections, however, cannot be avoided completely in evaluations that rely so much on statistics to measure error rates.

This article analyzes projections for five periods when procedures differed and for which evaluations have previously

been published. These periods are: 1960–70, the first projections issued; 1960–75, the first projections issued using an industry occupation matrix; the 1968–80 projections, the first

set of projections to 1980; the 1980–90 projections, the only

1990 projections used in the Occupational Outlook Handbook;

and the 1984–95 projections, the last set of projections made

before using the target year of 2000. No projections were

evaluated that use 1985 as the target year, because the occupational classification used in the 1980s was so different from

the classification used in the 1970s when the projections were

prepared, that the statistical analysis would be meaningless,

even for major occupational groups.

Accuracy and error

Total employment. Projections of total employment generally were more accurate than the projections of major occupational groups. In two of sets of projections, 1960–70 and 1980–

90, the error in total employment was less than 1 percent. The

1960–75 projections, with a projected growth rate of 33 percent and actual growth rate of 31 percent, certainly must be

considered very accurate. (See table 1.) The 1968–80 and

1984–95 projections were underprojected, –4.4 percent for the

earlier period and -5.6 percent for the later period. (See table

2.) However, even these projections cannot be considered significant errors. In view of the statement made earlier that projections were meant to apply to a year around the target year,

those projections would be highly accurate, with an error rate

of less than 1 percent, if both sets were measured by data 2

years before the actual target year. It could be assumed that

individuals using the projections were not misled by projections that were highly accurate 2-years prior to target years 12

and 11 years into the future. In terms of trends in projection

accuracy over time, no pattern is evident, but clearly no improvement can be seen.

Major groups. A few general patterns emerge in comparing

the projected and actual growth rates for the major occupational groups over time. The first is that the direction of change

was always projected correctly. This was no major feat, as all

Table 1. Projected and actual percent change in employment by major occupational group, selected projection periods

1960–70 1960–75 1968–80 1980–90 1984–95

Projected Actual Projected Actual Projected Actual Projected Actual Projected Actual

 Total employment ....................... 20 20 33 31 25 31 19 20 15 22

Executive and managerial ........... 23 17 28 27 22 43 17 12 20 33

Professional and technical ........... 41 49 73 73 50 55 (1

) (1) (1) (1)

Professional specialty .................. (1

) (1) (1) (1) (1) (1) 18 37 21 36

Technicians .............................. (1

) (1) (1) (1) (1) (1) 32 37 29 29

Clerical and sales ........................ 26 33 (1

) (1) (1) (1) (1) (1) (1) (1)

Sales ....................................... (1

) (1) 34 31 29 36 22 56 21 31

Clerical (administrative

 support, including clerical) ......... (1

) (1) 51 57 35 44 20 27 11 21

Service ........................................ 25 21 53 47 40 41 25 24 20 29

Agriculture, forestry, and fishing .. –18 –40 –38 –43 –33 –21 –7 –1 –4 –1

Precision production,

 operators, and laborers ............. (1

) (1) (1) (1) (1) (1) 18 1 (1) (1)

Precision production

 (craft-skilled) ............................... 23 19 33 30 22 38 (1) (1) 12 6

Operators and laborers ............... 18 14 18 11 8 7 (1

) (1) 79

1 The major groups were not comparable in each projection period and detailed occupational data were not available to develop comparable data.

Occupation group
32 Monthly Labor Review May 1999

groups except for agriculture, forestry, and fishing occupations were always projected to increase and all did so, and

that agriculture forestry, and fishing occupations, correctly,

were always projected to decline. Notable, however, is that

the projected decline was much slower in the last two projection periods than those in the earlier periods and the actual

decline also was much less.

Another general pattern is that 7 out of the 9 instances were

correctly projected in which employment in major groups grew

by less than 20 percent. Only in one of these cases was employment projected very inaccurately; professional specialty

workers from 1980 to 1990. A review of the detailed occupations comprising that group shows that projections of three

occupations contributed very significantly to that error—college teachers, vocational education teachers, and registered

nurses. Employment of college teachers was projected to decline based on Department of Education projections of declining college enrollments because of the shrinking of the

traditional college-population aged 18 to 24. During the 1980s,

however, colleges were successful in enrolling older individuals in greater numbers than in the past, and enrollment rates

of traditional-age college students also rose more than expected. As a result, employment of college teachers increased

rather than declined. Vocational education teachers also were

underprojected for similar reasons. Employment of registered

nurses was projected to grow rapidly, but it grew so much

faster than projected that it accounted for 10 percent of the

underprojection of professional specialty workers.

Employment of major groups was projected to increase

more than 40 percent in six instances and in all these cases,

employment did grow faster than 40 percent. Employment

was projected to increase faster than 30 percent in three other

cases, and in all cases it did.

The 1980–90 projection of salesworkers had the largest projection error. Employment was projected to increase by 22

percent, but actual data indicate it increased by 56 percent.

Much of the error was due to the underprojection of three occupations—self-employed salesworkers, cashiers, and stock

clerks. All of these occupations experienced a significant

change in definition from the occupational classification used

in the base year to that used in the target year. Considering

those changes, even though adjustments were made wherever

possible for classification changes, and the very rapid 56 percent growth, the projections may have been more accurate

than they appear.

From these data, a conclusion could be reached that the

fastest growing and slowest growing major occupational

groups have been projected, almost always, correctly. In comparing projected and actual growth rates and the percent error

in the level of projected employment for major groups, relatively few large discrepancies between projected and actual

data can be identified. It also is apparent that the quality of

projections as measured by the accuracy of projections has

not improved over time. For instance, the 1960–70 projections, the first set of projections published, were the most accurate and the last two sets, 1980–90 and 1984–95 were the

least accurate. Clearly, the accuracy of projections of major

occupational groups has not improved over time.

Detailed occupations. Significant detail has been published

in previous articles evaluating the BLS projections.10 Consequently, much of the information presented here was drawn

from articles presenting evaluations of the occupational projections. That information has been synthesized, allowing BLS

economists to determine if the quality of the detailed occupational projections has changed over time. As indicated above,

detailed occupational projections were not published before

the 1960–75 projections issued in 1996, and, therefore, the

1960–70 projections discussed in the section on major occupational groups cannot be discussed in this section.

The number of occupations for which projections were published increased significantly over time. In the 1960–75 projections, 162 occupations were projected through the industry-occupation matrix, but projections for other occupations

Table 2. Percent error in level of projected employment by major occupational group, selected projection periods

 Occupation group 1960–70 1960–75 1968–80 1980–-90 1984–95

Total employment ........................................................................ 0.0 3.2 –4.4 –0.9 –5.6

 Executive and managerial ......................................................... 4.6 .3 –17.2 4.4 –9.9

 Professional and technical ......................................................... –5.9 –.1 –3.0 (1) (1)

 Professional specialty ................................................................ (1) (1) (1) –14.4 –11.3

 Technicians and related support ............................................ (1

) (1) (1) –3.1 .5

 Clerical and sales ...................................................................... –5.4 (1

) (1) (1) (1)

 Sales ..................................................................................... (1

) 6.4 –5.1 –21.8 –7.4

 Clerical (administrative support, including clerical) .................... (1) –3.8 –6.8 6.0 –8.3

 Service ...................................................................................... 3.1 7.4 6.7 .9 –6.6

 Agriculture, forestry, and fishing ................................................ 26.3 11.8 –5.4 –6.0 –3.6

 Precision production, operators, and laborers ........................... (1

) (1) (1) 16.3 (1)

 Precision production (craft-skilled) ............................................ 3.4 2.2 –4.8 (1

) 6.2

 Operators and laborers .............................................................. 3.6 7.2 1.2 (1) –1.7

1 The major groups were not comparable in each projection period and detailed occupational data were not available to develop comparable data.

Quality of Projections
 Monthly Labor Review May 1999 33

included in the Handbook were not considered reliable enough

to be published at that time. In the 1970–80 projections, the

number projected through the matrix was 160, but an additional 80 occupations were independently projected and included in projections publications. The number of occupations projected increased significantly when the industry-occupation matrices shifted to the Occupational Employment

Statistics surveys as a source of employment data. In the 1980–

90 projections, 687 occupations were projected out of more

than 2,000 occupations for which employment data were collected in the Occupational Employment Statistics survey.11 By

the time the 1984–95 projections were developed, the Occupational Employment Statistics survey had reduced the number of occupations to about 750, roughly the same number of

occupations as in the Occupational Employment Statistics survey conducted in the late 1990s. All occupations in the Occupational Employment Statistics survey were not projected,

generally because of the small employment size of the occupations or because related occupations were aggregated for analytical purposes, such as college teachers by field of study to

all college teachers. The 750 occupations were aggregated into

500 occupations or occupation groups before projections were

developed. (See table 3.)

All occupations for which projections were developed

could not be evaluated because the occupational classification system used to collect the employment data changed before the target year of the projections was reached. For census–based matrices, 47 percent of the 1960–75 projections

and 40 percent of the 1970–80 projections could be evaluated. Although the number of occupations that could be evaluated increased significantly with the 1980–90 Occupational

Employment Statistics survey projections, the proportion decreased to 19 percent. During the 1980–90 period, the Occupational Employment Statistics survey classification underwent significant revisions. With the introduction of the 1980

Standard Occupational Classification in the 1983 Occupational

Employment Statistics survey, the classifications became more

stable and the number of occupations that could be evaluated

jumped to 348 in the 1984–95 projections—70 percent of the

total number of occupations projected.

In evaluating projection error, it is not clear what should be

considered a satisfactory or an acceptable level of error and

what is unsatisfactory. Some errors obviously indicate a poor

projection, such as actual growth of 100 percent when 5 percent growth or a decline was projected. A similarly poor projection is a percent error in level of 50 percent, when employment was projected to be 50,000 and it actually was to 75,000.

However, it is not clear what the statistical dividing line is

between a “good” and “poor” projection, whether evaluated

by percent change or percent error in employment. What is

significant is the impact the information stemming from the

projection has on the decisions made based on the projections. Using this criteria, it could be assumed that if employment goes in the opposite direction, decrease instead of the

projected increase, the projection is poor. But, this must also

be tempered if the projected increase is small. For example,

if employment is projected to increase by 5 percent, from

100,000 to 105,000 and employment declines from 100,000

to 95,000, the percent error in level would be only 10.5 percent. Also, because of measurement errors of employment

surveys the real direction of change could have been as projected. Despite the uncertainties about “real” quality, it is possible to evaluate differences in the accuracy of different sets of

projections, and to see how accuracy has changed over time.

The average absolute percent error of the detailed occupational projections was not significantly different among the

four sets of projections. The last set, 1984–95 had a slightly

higher error, but this evaluation covered many more occupations than did earlier evaluations. In terms of the direction of

change, the proportion of the occupations evaluated that were

projected in the correct direction was very similar in the first

three sets projections, and slightly lower in the latter.

The vast majority of occupations in all sets of projections

that were not projected in the correct direction had projected

increases, but actually declined. (See table 3.) A review of

the occupations projected to grow, but actually declined, uncovered some interesting information. In the 1960–75 projections, 6 of the 11 occupations were projected to increase,

but by less than 10 percent. In career guidance information,

this growth would not have been described as favorable for

job opportunities. Also, 7 of the 11 occupations had fewer

than 50,000 workers in 1960 and, consequently, employment

data would be subject to large variances. In the 1970–80 projections, 6 of the 12 occupations projected to increase that

actually declined according to matrix data, showed increases

from 1970–79 based on CPS data. In the 1980–90 projections,

Table 3. Number of detailed occupations projected and evaluated, selected projections periods

Average Not projected in

Occupations projected percent Correct direction correct direction

 Projection period error

Number Number Percent Number Percent Number Number that

evaluated evaluated of total declined

1960–75 .................................... 162 76 47 20.8 64 84 12 11

1970–80 .................................... 160 64 40 22.4 50 78 14 12

1980–90 .................................... 687 132 19 21.1 107 81 25 24

1984–95 .................................... 500 348 70 24.0 252 72 96 80

..................................................
34 Monthly Labor Review May 1999

almost half of the 24 occupations, projected to increase that

actually declined according to the evaluation, increased from

1983 to 1990 in the 1983–96 historical matrix. Significant

changes in occupational classification took place in the Occupational Employment Statistics survey, collected after 1983,

that could have caused significant differences in levels for

occupations that were comparable in definition between projections and the actual data. Thus, the projections could be

more accurate than depicted in the evaluations.

It is evident that projections for occupations with large employment numbers are more accurate as measured by the average percent error than are occupations with small employment numbers. As shown in table 4, in each set of projections, the percent error for occupations with more than 500,000

or 600,000 workers is significantly less than average, whereas

the projection error for occupations with less than 50,000 in

employment is much larger than average. Another point becomes clear in reviewing tables 3 and 4. As with the major

occupational groups discussed earlier, there was no clearly

significant improvement in the accuracy of the projections

over time, as data improved and more historical data were

available for analysis.

Conclusions

Significant improvements have been made in the procedures

used to develop occupational employment projections over the

past 50 years. In the 1950s and 1960s, the requirements for

inclusion of an occupation in the Occupational Outlook Handbook did not include statistically sound estimates of current

employment, and, consequently, projections were often developed outside of the projections model. Virtually all job outlook statements in recent editions of the Handbook have been

based on employment projections developed through complex

projections models.

Over time, the occupational projections developed by BLS

have captured most of the general trends in occupational employment. Projections of major occupational groups are consistently in the correct direction, and rapid growth and slow

growth projections for these groups have been very accurate.

For detailed occupations, projections have been less accurate

than for major occupational groups. The accuracy of the projections is better for occupations of large size than for small

occupations. The vast majority of occupations are projected in

the correct direction as measured in evaluation studies. It is

very likely that the proportion projected in the correct direction is even higher than that shown in evaluations of projections, because of occupational comparability problems in going from projected to actual data.

The accuracy of projections has not changed over time. The

projections prepared in the mid-1980s are no more accurate

than those prepared in the late 1960s, despite the availability

of more data and improved modeling. Biases found in the

projections in evaluations found in the late 1980s and 1990s

resulted in significant changes in analytical procedures to projections made to the year 2000 and beyond. The results could

be significant. For example, in the 1984–95 projections only

39 occupations were projected to decline as it seemed there

was a reluctance to project employment declines. In the 1996–

2006 projections, 125 occupations were projected to decline.

Time will tell if the analytical changes that were made will

have a real impact on projection accuracy. 

Table 4. Absolute average percent error for detailed occupations projected and evaluated, selected projections periods

 Absolute

 Projection period percent error occupations Less 50,000 Less 100,000 300,000 500,000 600,000

for all with a below than to than to to or or

occupations average error 50,000 99,999 100,000 299,000 599,999 more more

evaluated

1960–75 ........................ 20.8 66 32.4 20.3 (1

) 15.5 19.8 (1) 11.2

1970–80 ........................ 22.4 50 29.5 24.9 (1

) 26.3 17.2 (1) 10.1

1980–90 ........................ 21.1 60 (1) (1) 25.6 (1) (1) 14.5 (1)

1984–95 ........................ 24.0 60 (1

) (1) 29.0 (1) (1) 12.2 (1)

1 Not available.

Proportion of

Notes

1

 Manpower Challenge of the 1960s (U.S. Department of Labor, 1960).

2 The Outlook for Technological Change and Employment, Appendix

Volume I, Technology and the American Economy (Washington, National

Commission on Technology, Automation, and Economic Progress, February

1996).

3

 On the Internet at: http://stats.bls.gov/empbib06.htm

average

4

 Projections for occupations developed through an industry-occupation

matrix and independently that were merged for publication purposes can be

seen in Occupational Projections and Training Data, Bulletin 1824 (Bureau of Labor Statistics, 1974), pp. 79–87. 5

 BLS conducted a Survey of Scientific and Technical Personnel in Industry, funded by the National Science Foundation, from the late 1950s through

the 1960s. This survey program was the forerunner to the Occupational

Absolute average percent error for occupations by size of employment

Quality of Projections
 Monthly Labor Review May 1999 35

Employment Statistics survey. Similar information was collected in the Occupational Employment Statistics survey when it began. 6

 Agricultural Services was covered by the Occupational Employment

Statistics survey beginning in the early 1990s. 7

 Because of data limitations and resource constraints, the occupational

estimates for self-employed and unpaid family workers were not distributed

across industries. Consequently, occupation/industry cross-tabulations were

available only for wage and salary employment. 8

 Some users considered the difference between the high- and low-scenario projections to be a range rather than projections having different assumptions. 9

 See The Outlook for Technological Change and Employment, page I–

15. 10 The latest example in the Monthly Labor Review is, “Evaluating the

1995 projections,” September 1997, pp. 3–31. 11 Occupations not projected were listed, by size of employment, in the

Occupational Projections and Training Data, 1982 Edition, Bulletin 2202

(Bureau of Labor Statistics, December 1982), pp. 89–96.

Bureau of Labor Statistics Internet

The Bureau of Labor Statistics World Wide Web site on the Internet contains a range of data on consumer

and producer prices, employment and unemployment, occupational compensation, employee benefits, workplace injuries and illnesses, and productivity. The homepage can be accessed using any Web browser

 http://stats.bls.gov

Also, some data can be accessed through anonymous FTP or Gopher at

stats.bls.gov