Estimation Considerations for the Occupational 

Requirements Survey 

Bradley D. Rhein1, Chester H. Ponikowski1, and Erin McNulty1

1U.S.Bureau of Labor Statistics, 2 Massachusetts Ave., NE, Room 3160, 

Washington, DC 20212 

Abstract 

The Bureau of Labor Statistics (BLS) is working with the Social Security Administration 

(SSA) to carry out a series of tests to determine the feasibility of using the National 

Compensation Survey (NCS) platform to accurately and reliably capture data that are 

relevant to the SSA's disability program. The proposed new Occupational Requirements 

Survey (ORS) is envisioned to be an establishment survey that collects information on the 

vocational preparation and the cognitive and physical requirements of occupations in the 

U.S. economy, as well as the environmental conditions in which those occupations are 

performed. Many of the data elements are collected on the basis of presence and then are 

measured by duration (length of time). Some data elements collected are conditional on the 

presence of another data element. This paper discusses the considerations for developing 

the estimates to be produced for ORS and how the micro data may be adjusted to account 

for unit and item non-response. 

Key Words: estimation, binomial data, point estimates, establishment survey, 

descriptive statistics 

1. Introduction 

In the summer of 2012, the Social Security Administration (SSA) and the Bureau of Labor 

Statistics (BLS) signed an interagency agreement to begin the process of testing the 

collection of data on occupations. As a result, the Occupational Requirements Survey 

(ORS) began testing in late 2012. The goal of ORS is to collect and publish occupational 

information that will replace the outdated data currently used by SSA. All ORS products 

will be made public for use by non-profits, employment agencies, state or federal agencies, 

the disability community, and other stakeholders. More information on the background of 

ORS can be found in the next section. 

An ORS interviewer attempts to collect close to 70 data elements related to the 

occupational requirements of a job. The following four groups of information will be 

collected: 

 Physical demand characteristics/factors of occupations (e.g. strength, hearing, or 

stooping) 

 Educational requirements 

 Cognitive elements required to perform work 

 Environmental conditions in which the work is completed 
During the last two years of survey testing, there have been a number of changes to the 

details of collection. Some data elements were not considered necessary and were dropped. 

New data elements were added to the list. Many data element definitions were refined, 

while continuing to rely on definitions from the Revised Handbook for Analyzing Jobs 

(RHAJ) [1]. In some cases, the method of collection was changed. For instance, initially 

the duration of an occupational activity was captured categorically using ranges of time 

defined by SSA. Currently duration is captured by hours or percentage of the day, with 

fallbacks being a range specified by the respondent or use of a duration scale. 

This paper explores the estimation and some of the non-response options for ORS data. 

Section 2 provides background information on the Occupational Requirements Survey. 

Section 3 explains the types of ORS data elements captured and available for use in 

estimation. Section 4 explores the estimation possibilities for each data element, and 

Section 5 provides a glimpse of some non-response adjustment possibilities. The paper 

ends with a conclusion and description of further research to be completed. 

2. Background Information on ORS 

In addition to providing Social Security benefits to retirees and survivors, the Social 

Security Administration (SSA) administers two large disability programs, which provide 

benefit payments to millions of beneficiaries each year. Determinations for adult disability 

applicants are based on a five-step process that evaluates the capabilities of workers, the 

requirements of their past work, and their ability to perform other work in the U.S. 

economy. In some cases, if an applicant is denied disability benefits, SSA policy requires 

adjudicators to document the decision by citing examples of jobs the claimant can still 

perform despite restrictions (such as limited ability to balance, stand, or carry objects) [2]. 

For over 50 years, the Social Security Administration has turned to the Department of 

Labor's Dictionary of Occupational Titles (DOT) [3] as its primary source of occupational 

information to process the disability claims [4]. SSA has incorporated many DOT 

conventions into their disability regulations. However, the DOT was last updated in its 

entirety in the late 1970’s, although a partial update was completed in 1991. Consequently, 

the SSA adjudicators who make the disability decisions must continue to refer to an 

increasingly outdated resource because it remains the most compatible with their statutory 

mandate and is the best source of data at this time. 

When an applicant is denied SSA benefits, SSA must sometimes document the decision by 

citing examples of jobs that the claimant can still perform, despite their functional 

limitations. However, since the DOT has not been updated for so long, there are some jobs 

in the American economy that are not even represented in the DOT, and other jobs, in fact 

many often-cited jobs, no longer exist in large numbers in the American economy. For 

example, a job that is often cited is “envelope addressor,” because it is an example of a 

low-skilled job from the DOT with very low physical demands. There are serious doubts 

about whether or not this job still exists in the economy. 

SSA has investigated numerous alternative data sources for the DOT such as adapting the 

Employment and Training Administration’s Occupational Information Network (O*NET) 

[5], using the BLS Occupational Employment Statistics program [6] (OES), and 

developing their own survey. But SSA was not successful with any of these potential data 
sources and turned to the National Compensation Survey program at the Bureau of Labor 

Statistics.

3. Captured Data Elements 

ORS is designed to capture occupational information on educational requirements, 

cognitive and physical demands, and exposures to environmental conditions. Each of the 

data elements falls into two data types: categorical and continuous. Data elements that are 

categorical have a set of predetermined values, one of which will be selected as a response. 

Continuous data may be limited by a minimum, such as zero hours, or a maximum, such 

as 100 percent. 

Educational requirements are part of a category called “Specific Vocational Preparation” 

(SVP). SVP measures the amount of time it takes a typical worker to learn the techniques 

and acquire the information needed for average job performance. There are four 

components: 

1. Minimum level of education, including literacy 

2. Previous job experience required 

3. Time spent in earning certifications and licenses (pre- or post-employment 

training) 

4. On-the-job training (expressed as the time to average performance). 

Level of education is measured by the receipt of a degree, and an amount of time is assigned 

based on the degree. The four components are used to identify the following three 

occupational requirements, listed in Tables 1-3 below, desired by the SSA. 

Table 1: Specific Vocational Preparation 

SVP 

Level

Amount of Specific Vocational Preparation Time

1 Short demonstration only

2 Anything beyond short demonstration up to and including 1 month

3 Over 1 month up to and including 3 months

4 Over 3 months up to and including 6 months

5 Over 6 months up to and including 1 year

6 Over 1 year up to and including 2 years

7 Over 2 years up to and including 4 years

8 Over 4 years up to and including 10 years

9 Over 10 years
Table 2: Job Zone 

Job Zone Preparation Level SVP Level(s)

1 Little or no preparation needed 1-3

2 Some preparation needed 4-5

3 Medium preparation needed 6

4 Considerable preparation needed 7

5 Extensive preparation needed 8-9

Table 3: Skill Level 

Skill Level SVP Levels

Unskilled 1-2

Semi-skilled 3-4

Skilled 5-9

Cognitive abilities are captured entirely in categories. Information is gathered about the 

complexity of the job, the amount of oversight and controls associated with the job, the 

occurrence of deviations in work tasks, work schedules, and work location as well as the 

types and frequency of personal interactions required to perform typical job duties. 

The job complexity data element measures the level of decision-making, comprehension, 

memory, and application of information needed to perform the typical duties of an 

occupation. Work controls refer to the level of supervision and requirement for workers to 

adhere to established guidelines. Several data elements measure the work routine of an 

occupation, including the deviation in tasks, work schedule, and work location. Finally, the 

cognitive data elements explore the type and frequency of worker communications with 

established working relationships, labeled regular contacts, and other contacts. 

Table 4 summarizes the cognitive data elements and possible responses. 
Table 4: Cognitive Data Element Responses 

Cognitive Measure Possible Response

How complicated are the tasks of the 

occupation? 

Very simple, simple, moderate, 

complex, very complex

How closely controlled is the occupation’s 

work?

Very closely, closely, moderately, 

loosely, very loosely

How often are there deviations from the norm 

in work tasks? 

Hourly, daily, weekly, monthly, less 

than monthly

How often are there deviations from the norm 

in work schedule?

Hourly, daily, weekly, monthly, less 

than monthly

How often are there deviations from the norm 

in work location?

Hourly, daily, weekly, monthly, less 

than monthly

How often does the occupation verbally interact 

(work related) with regular contacts?

Hourly, daily, weekly, monthly, less 

than monthly

What type of work-related interactions does the 

occupation have with regular contacts? 

Very structured, structured, semistructured, unstructured, very 

unstructured

How often does the occupation verbally interact 

(work related) with people other than regular 

contacts? 

Hourly, daily, weekly, monthly, less 

than monthly

What type of work-related interactions does 

this occupation have with people other than 

regular contacts? 

Very structured, structured, semistructured, unstructured, very 

unstructured

Physical demands are captured in two ways: presence and duration. A physical demand is 

considered present if the worker is required to perform an activity as part of the job. 

Duration measures the amount of time in a typical work day a worker spends performing a 

physical demand. Some physical demands will only have information on presence, such as 

the demands for hearing or vision. The full list of physical demands is as follows: 

 Sitting 

 Standing/Walking 

 Sitting vs. Standing at Will 

 Lifting/Carrying 

 Reaching (overhead and at or below the shoulder) 

 Pushing and pulling with arms, legs, or feet only 

 Climbing stairs and ramps (job related or structural) 

 Climbing ladders and scaffolds 

 Crouching, kneeling, crawling, stooping 

 Use of hands (gross manipulation) 

 Use of fingers (fine manipulation) 

 Use of one or both feet or legs to move controls on machinery or equipment 
 Keyboarding (10-key, traditional, touch screen, other) 

 Hearing 

 Vision (near/far acuity, peripheral) 

 Driving and Vehicle Type 

 Communicating verbally (one-on-one, group, telephone, or other sounds) 

Some physical demands have sub-questions. One example is the reaching data element. 

Once reaching is found to be present and a duration is captured, an additional piece of 

information is collected: does the reaching require one arm or both arms? Other physical 

demands that have sub-questions include pushing and pulling, use of hands, and use of 

fingers. An example for reaching is found in Table 5. 

Table 5: Reaching with one or both arms 

Data Element Response

Reaching 2 hours

One arm or both arms One arm

Lifting and carrying weight is an exception. One data element measures the maximum 

amount a worker would have to lift and/or carry. After determining the most that a worker 

would have to lift, the amount of weight being lifted within a certain duration, such as 

between one-third and two-thirds of the time, would be captured. SSA uses the categories 

in Table 6, most of which are defined by the RHAJ [1]. 

Table 6: SSA Categories for Lifting and Carrying Weight 

How Often

Ranges for Amount of Weight

Never None None None None None

Seldom (not in the 

RHAJ) 0 to 10 lbs 11 to 20 lbs 21 to 50 lbs

51 to 100 

lbs

More than 

100 lbs

Up to 1/3 of the time 0 to 10 lbs 11 to 20 lbs 21 to 50 lbs

51 to 100 

lbs

More than 

100 lbs

1/3 up to 2/3 of the 

time Negligible 0 to 10 lbs 11 to 25 lbs 26 to 50 lbs

More than 

50 lbs

2/3 of the time or 

more None Negligible 0 to 10 lbs 11 to 20 lbs

More than 

20 lbs

One other physical demand data element measures the strength required for a job. This 

data element incorporates the amount of weight a worker must lift, push, and/or pull. 

Currently, the weight thresholds for each category of the strength data element are still 

under development. 

Environmental conditions are the specific surroundings and circumstances in which an 

occupation is typically performed. Data are captured on eleven environmental conditions. 

Ten such conditions are captured by presence and duration. Exposure should be coded as 

experienced with the use of personal protective equipment. Environmental conditions data 

are collected for the amount of time a worker is exposed to:
 Outdoors 

 Humidity 

 Extreme heat 

 Extreme cold 

 Heavy vibration 

 High, exposed places 

 Wetness 

 Proximity to moving parts 

 Toxic, caustic chemicals 

 Fumes, noxious odors, dusts, gases 

The eleventh environmental condition is noise intensity level. This condition is captured in 

four categories - quiet, moderate, loud, and very loud - considering the job’s typical noise 

intensity exposure and accounting for any protective equipment. 

4. Estimation Possibilities 

Estimation options will depend on the type of data element. For any categorical data 

element, a percentage of workers that fall into a given category could be calculated. For 

continuous data elements, descriptive statistics could be calculated, including the mean 

amount of time and percentiles. If the continuous data were placed within pre-specified 

ranges, they would become categorical values, and percentages could be calculated. Table 

7 presents the many of the ORS data elements and respective estimation possibilities. 
Table 7: Sample ORS Data Elements and Possible Estimates

Table 8 presents SSA’s requested categories for the duration of any activity or exposure. 

Table 8: SSA Duration Categories 

SSA Duration Category Activity/Exposure Occurrence in a Work Day

Not Present Not present

Seldom Occurs less than 2% of the time

Occasionally Occurs 2% to less than 1/3 of the time

Frequently Occurs 1/3 to less than 2/3 of the time

Constantly Occurs 2/3 of the time or more
For data elements that could be expressed as a category (see Table 7), a percentage of 

workers for whom the element is present will be calculated for each category among the 

total number of workers. 

The percentage will be calculated over some specified domain, such as an occupation, an 

occupation within an industry, or a group of occupations. The formula for the percentage 

of workers in the domain is as follows. 

100

OccFW

OccFW

ig

I

1i

G

1g

ig

igig

I

1i

G

1g

ig

i

i



































 

 

X

ZX

where: 

i = Establishment 

I = Total number of establishments 

g = Occupational quote within establishment i 

Gi = Total number of occupational quotes in establishment i 

OccFWig = Final occupational quote weight (a weight denoting the 

number employees in the frame that are represented by 

occupation g in establishment i) for occupation g in 

establishment i 

Xig = 1 if occupational quote ig is in the domain 

= 0 otherwise 

Zig = 1 if the data element is present for occupational quote ig 

= 0 otherwise 

To calculate the percentage of employees for whom a data element is present out of all 

employees in the domain, add the final quote weights across only those occupational quotes 

in the domain for whom the data element is present. Then divide that number by the sum 

of the final quote weights across quotes in the domain. Multiply the final quotient by 100 

to yield a percentage estimate. 

An example of a percentage estimate follows. Each estimate will have an associated 

measure of error. Calculation of standard errors will be left for a future paper. 
Example 1: Education Level for Salespersons 

Some data elements depend on the presence of a related data element. For instance, as 

noted above, once an occupation is known to require reaching overhead, an additional 

question captures whether the job requires the worker to reach overhead with one hand or 

with both hands. All of these circumstances necessitate a calculation of a percentage. A 

complete list of such data elements follows. 

 Reaching overhead 

 Reaching from the shoulder 

 Pushing or pulling 

 Use of hands 

The following example illustrates a sub-category of the reaching data element. Once the 

presence of reaching has been established for a particular occupation, a follow-up question 

records whether the reaching can be done with one hand/arm only, or if reaching with both 

hands/arms is necessary. In the example below, of the 75% of workers required to reach 

overhead, 20% could perform the reaching with one hand or arm. 

Example 2: Presence of Reaching Overhead for Salespersons 

Finally, a number of data elements are captured as continuous data. Mean and percentile 

values will be calculated for all continuous data elements (see Table 7). In addition, some 

continuous data will be converted into a range for use by SSA. These ranges will act as 

categories and a percentage of workers will be calculated, as described earlier for 

categorical data. 
A mean for a data element will be calculated over some specified domain, such as an 

occupation, and the formula is as follows: 

































 

 

igig

I

1i

G

1g

ig

igigig

I

1i

G

1g

ig

i

i

OccFW

OccFW

ZX

QZX

where: 

i = Establishment 

I = Total number of establishments in the survey 

g = Occupational quote within establishment i 

Gi = Total number of occupational quotes in establishment i 

OccFWig = Final quote weight for occupation g in establishment i 

Xig = 1 if occupational quote ig is in the domain 

= 0 otherwise 

Zig = 1 if the data element is present for occupational quote ig 

 = 0 otherwise 

Qig = Value of the continuous data element for occupational quote ig

To calculate the mean value of a continuous data element, multiply the final quote weight 

and the value of the element for those occupational quotes in the domain for whom the 

element is present; add these values across all contributing quotes to create the numerator. 

Divide this number by the sum of the final quote weights across only those quotes in the 

domain for whom the element is present. 

The following percentiles, designated as p, would be calculated: 10th, 25th, 50th (median), 

75th and 90th. The p-th percentile is the value of the continuous data element Qig such that: 

 the sum of final occupational quote weights (OccFWig) across quotes with a value 

less than Qigj is less than p percent of all final quote weights, and 

 the sum of final occupational quote weights (OccFWig) across quotes with a value 

more than Qigj is less than (100 – p) percent of all final quote weights. 

It is possible that there are no specific quotes ig for which both of these properties hold. 

This occurs when there exists a quote for which the OccFWig of records whose value is less 

than Qig equals p percent of all final quote weights. In this situation, the p-th percentile is 

the average of Qig and the value on the record with the next lowest value. Include only 

occupational quotes in the domain for whom the element is present – i.e., where: 

ZX igig  1
where: 

Xig = 1 if occupational quote ig is in the domain 

= 0 otherwise 

Zig = 1 if the data element is present for occupational quote ig 

 = 0 otherwise 

i = Establishment 

g = Occupational quote within establishment i 

5. Initial Non-response Adjustment Ideas 

Both unit and item non-response are expected to occur in the collection of the Occupational 

Requirements Survey. Unit non-response would entail a sampled establishment or 

occupation refusing to provide ORS data, while item non-response would refer to 

incomplete information within an occupation about the data elements described in the 

earlier sections. 

There is not much information on the expected amount of unit non-response for ORS at 

this point. Each of the various survey tests to this point have been focused on collection 

feasibility and protocols. ORS interviewers collected data from cooperating establishments 

found on an over-sampled list of units that was not fully representative of the sample frame. 

Notes detailing the reasons for refusal were taken for feasibility tests occurring in 2014, 

but refusal turnaround was not attempted. Additionally, ORS interviewers were not limited 

to a strict probability selection of occupations within an establishment but, instead, could 

collect data on occupations at the convenience of the respondent. However, completing a 

probability selection of establishments and occupations is vital for a statistically sound 

survey and so, generally, the additional respondent burden will result in increased nonresponse. 

Similar to the National Compensation Survey (NCS) [7], ORS plans to adjust for unit nonresponse by re-weighting establishments and occupations within a pre-specified nonresponse adjustment cell. Once a weighting adjustment has been made to the 

establishments and occupations within a cell where a unit refused, the weights of all viable 

units may be further adjusted by benchmarking the weights to the current national 

employment to obtain the Occupational Final Quote Weight used in the formulas above. 

Early testing of ORS has shown a low instance of item non-response. Once an 

establishment cooperates in collection, nearly all information for all items and quotes was 

collected in a single appointment. However, these circumstances are considered unusual as 

ORS interviewers will be attempting collection from less cooperative establishments in a 

production environment. As a result, some methods must be developed to alleviate the 

impending item non-response as ORS moves into production. 

One option currently under investigation is to group similar ORS data elements and impute 

all items in a group where at least one item is missing. The data elements naturally split 

into several imputation groups, though some groups are larger than others. For instance, all 

of the data elements regarding the lifting and carrying of weight could appear in an 
imputation group. Using groups may help keep imputed responses consistent with 

collected responses. Also, having a small number of elements within imputation groups 

would be ideal as less collected data would be over-written in the process of imputing for 

a single missing value within a group. Imputation could employ the nearest neighbor 

approach, where the nearest neighbor is determined by the establishment employment size. 

Imputation cells may need to be collapsed for situations where a cell lacks a viable number 

of donors. Further research on item non-response will occur in the next phase of survey 

testing. 

6. Conclusion and Next Steps 

Estimation for ORS data will likely include the use of descriptive statistics, dependent on 

whether the data element type is categorical or continuous. Some data element estimates 

are dependent on the presence of another data element. Percentages of workers in a 

category will be calculated for categorical data; the mean amount of time and percentiles 

will be calculated for continuous data. Standard errors, while not discussed in this paper, 

will accompany all estimates of percentages, means, and percentiles. 

While non-response is expected to be present in ORS, there are currently no final 

procedures for non-response adjustment. The behavior of ORS data is largely unknown 

and must be studied further. As a starting point, unit-level non-response will be adjusted 

for by adjusting the weights of both establishments and occupations. Possible methods for 

item non-response adjustment are under review. Before the production of estimates from 

the first full-scale production sample, an imputation method will be tested involving nearest 

neighbor imputation by employment size and within cell definitions based on available 

variables. Once a greater amount of ORS data has been amassed, more information will be 

available for use in developing appropriate item non-response adjustments. 

References/Footnotes 

[1] U.S. Department of Labor (1991), Revised Handbook for Analyzing Jobs (RHAJ), 

Washington, DC: Government Printing Office. 

[2] Social Security Administration, Occupational Information System Project, 

http://www.ssa.gov/disabilityresearch/occupational_info_systems.html.

[3] U.S. Department of Labor, Employment and Training Administration (1991), 

“Dictionary of Occupational Titles, Fourth Edition, Revised 1991” 

[4] Occupational Information Development Advisory Panel, 2010, 

http://www.socialsecurity.gov/oidap/index.htm

[5] U.S. Department of Labor, O*Net Online, http://www.onetonline.org/ 

[6] Bureau of Labor Statistics, Occupational Employment Statistics Program, 

http://www.bls.gov/oes/

[7] McCarthy, Christi, Ferguson, Gwyn R., and Ponikowski, Chester, (2011) "The 

Weighting Process Used in the Employer Costs for Employee Compensation 

Series for the National Compensation Survey ", 2011 Proceedings of the Section 

on Survey Research Methods, Alexandria, VA; American Statistical Association. 

Any opinions expressed in this paper are those of the authors and do not constitute 

policy of the Bureau of Labor Statistics or the Social Security Administration.