TECHNICAL NOTES FOR 1953--1954

Scope and Method of Survey

This survey of industrial research and development was conducted by the
United States Department of Labor's Bureau of Labor Statistics under
contract with the National Science Foundation. It is part of a broad
study of scientific and engineering research and development undertaken
by the National Science Foundation pursuant to its responsibilities for
making comprehensive studies of the Nation's research resources and
developing and recommending policies to strengthen scientific research.

The following is a brief nontechnical discussion of the coverage and
conduct of the survey and of certain problems of definition and
classification of data which were encountered. All this information is
important as background for interpreting the statistical findings
presented in the text and tables.

# How the Survey Was Made

Coverage of the Survey

The sample of companies included in the survey was drawn primarily from
the master list of those liable for OASI taxes, compiled by the Bureau
of Old Age and Survivors Insurance as of March 1951, the latest date for
which a listing was available when the survey was begun. The BOASI list
was augmented by separate lists of interstate railroads and Federal
Reserve Banks, which are not subject to OASI taxes. These combined lists
included nearly 2,800,000 organizations and represented the most
comprehensive roster of companies available in the United States.

Before the sample was selected from this listing, certain industries
were eliminated from it, for two main reasons---either because a
separate survey of the given industry was being conducted or sponsored
by the National Science Foundation or because the amount of research
performed there was believed to be negligible. The industries omitted
were those classified according to the Standard Industrial
Classification in the following major groups: 01---Farms;
73---Miscellaneous business services; 80---Medical and other health
services; 82---Educational services; 84---Museums, art galleries, and
botanical and zoological gardens; 86---Nonprofit membership
organizations; 88---Private households; 89---Miscellaneous services;
90---Government; and 99---Nonclassifiable establishments.

Companies below a specified minimum size, which was separately
determined for each major industry group, were also excluded from the
listing. In determining the size-of-company cutoffs, the Bureau of Labor
Statistics obtained information and advice from many trade associations
and other industry representatives. The aim was to reduce the number of
companies in the universe to be sampled and thus increase the efficiency
of the survey by eliminating as many as possible of the great numbers of
firms in the smallest size groups, but to do this without excluding
companies large enough so that there was any possibility of their
conducting a substantial amount of research and development or employing
a significant number of scientist or engineers. The size criterion used
was the company's total employment as of March 1951. In most branches of
manufacturing, mining, transportation and other public utilities,
agricultural services, forestry, and fisheries, only companies with less
than 8 employees were excluded. In radio broadcasting, every company, no
matter how small, was included in the universe. In many other
nonmanufacturing industries, the cutoff was set at 50 employees; the
most important exceptions were construction and railroads, for which the
cutoff was 100 employees.

An analysis of the information supplied by companies in the smallest
size groups covered by the survey suggests that in several industries
(including machinery, chemicals, electrical equipment, and professional
and scientific instruments) a considerable number of firms with less
than 8 employees may do a little R&D work, but that the aggregate cost
of their R&D activities is very small relative to the industry totals.
The available data also suggest that scientific and engineering
employment in companies below the size cutoffs is very small compared to
the industry totals---with a few possible exceptions, especially in the
case of engineers in the construction industry. Because of the
difficulty of sampling the great number of small construction companies
in the United States and of the financial and other limitations on the
survey, it seemed desirable to exclude all construction firms with less
than 100 employees from the universe to be sampled, although a sizable
number of engineers are undoubtedly employed in such organizations.

The exclusions outlined in the preceding paragraphs reduced the universe
to be sampled to a total of about 162,000 companies, with 27,000,000
employees. The list of companies was stratified by industry and size
group, and a sample of about 11,600 organizations was randomly selected
from the industry-size strata. The sampling ratio was varied in relation
to size of company and other factors, so as to obtain maximum
reliability at minimum cost. All of the approximately 3,000 companies
with 1,000 or more employees were included in the sample. In other
industry-size cells, sampling ratios ranging from 1 in 2 to 1 in 100
were used. In general, the larger the company and the more
research-intensive the industry, the higher the sampling ratio.

**Conduct of the Survey**

The questionnaires used in the survey were developed in consultation
with the National Science Foundation and with the advice and cooperation
of many industry representatives---especially the Research Committee of
the National Association of Manufacturers and the Industrial Research
Institute. The forms were also tested, when in preliminary draft, by
visits to a number of companies.

Since the survey covered a wide range of industries, including many in
which there was known to be little research activity, the same
questionnaire would not have been appropriate for all firms.
Accordingly, three different questionnaires were used. The longest and
most detailed version (BLS Form 2437a), which contained questions on R&D
cost and personnel and many related items, was sent to companies with
1,000 or more employees in practically all industries and to firms below
that size in industries where many small firms were known to conduct
research and development. The second questionnaire (Form 2437b) was sent
to companies in industry-size groups in which, it was presumed, some
companies conduct research and development but the great majority do no
or very little R&D work. This form permitted companies without R&D
activities or with a total R&D cost of less than \$25,000 to fill out
only a few questions, thus encouraging response, although companies with
larger R&D programs were asked to provide information on most of the
items included in Form 2437a. The third questionnaire (Form 2437c) was
mailed to companies in industry-size groups where the amount of R&D work
was assumed to be negligible, but which were included in the sample
because scientists and engineers might be employed in other types of
activities. The questions included in this brief form were practically
the same as those included in the abbreviated section of Form 2437b.

The questionnaires were mailed to most companies in the sample during
November 1954. Some 200 of the largest companies were visited in person,
in order to introduce the survey to these complex industrial
organizations which, in many cases, had special reporting problems and
to carry out a series of interviews with research officials on questions
important to the development of national science policy.

At least two followups were made of all nonrespondents, by mail or
telephone, beginning in December 1954 and continuing into the spring of
1955. When the tabulations on the survey were finally closed in July,
usable information had been received for 10,368 companies, or
approximately 90 percent of those in the sample. This count includes
several hundred companies for which partial information was obtained by
a final telephone followup of a subsample of the nonrespondents. The
rate of return is regarded as extremely high. Furthermore, bias owing to
small amounts of residual nonresponse was no problem in this survey.

The statistical findings derived from the sample data and presented in
this report are of three types, denoted as primary and secondary
estimates and descriptive data. Items classified as primary estimates
were: total R&D cost, total spending for research and development done
outside the reporting company, total scientists and engineers, and R&D
scientists and engineers. The process of obtaining the primary estimates
involved, first, adjustment of the sample data for each industry-size
cell for nonresponse and then inflation of these data by the reciprocal
of the applicable sampling ratio. At the same time and by the same
procedure, an estimate of employment, for the same data as that of the
control data on employment from BOASI (March 1951), was calculated for
each cell. The final estimate of the primary item was obtained for each
cell by multiplying the inflation estimate by the ratio of employment as
shown by the control data to employment as estimated in the preceding
step; the adjustments indicated by this final control procedure were
extremely small. The cell estimates were then added, to provide totals
for all industries and sizes of companies.

Items of estimate classified as secondary were, in general, components
of primary items and included, for example, cost of basic research and
employment in engineering and the various scientific fields. The method
of obtaining secondary estimates may be illustrated by that used in
deriving an estimate of the number of research chemists. Ratios of
research chemists to all research scientists and engineers were computed
for each estimating cell, based on information for only those companies
supplying data on both items, and then each ratio was multiplied by the
final estimate of the related primary item (in this case, the number of
research scientists and engineers) for the given cell.

Findings classified as descriptive data include the supporting personnel
ratios, the average cost ratios, and the data on patent ownership. The
descriptive statistics are in most cases presented as ratios of sample
aggregates, weighted by reciprocals of sampling rates. They have not
(except as indicated in the body of the report) been otherwise adjusted.
Thus their evidence, while analytically significant, is less certain
than that of the primary and secondary estimates.

The statistics which include companies with an employment of less than
1,000 are naturally subject in all instances to sampling errors.

The data from this as from all other surveys are subject to reporting,
editing, and tabulating errors. Such errors have been eliminated, to the
maximum extent possible, through checking procedures and through
correspondence with a number of companies whose reports appeared to
involve inconsistencies or misinterpretation of questions or
definitions.

# Definitions

Brief definitions of research-development and a number of other
important items were printed on the questionnaires. In addition, a
separate sheet containing much more detailed definitions of
research-development and cost were enclosed with the long form sent to
the larger companies.

These definitions, like other parts of the questionnaires, were
developed in consultation with many industry representatives and others
having an expert knowledge of industrial research. They were also tested
through visits to a number of companies before the start of the survey.
The aim was to arrive at definitions which would accurately and clearly
describe the desired information and would also conform insofar as
possible with customary accounting practices and thus make it feasible
for companies to supply the requested data. It was recognized, however,
that wide differences exist among industries, and even among companies
in the same industry, in the nature and organization of R&D activities,
in corporate structure, and in accounting practices and that these
differences would make some variations in interpretation and application
of the definitions inevitable. This was one of the major reasons for the
statement on the questionnaires that "Reasonable estimates will be
satisfactory."

A source of difficulty for many companies was the line of demarcation
between development and activities related to production. This problem
was specially acute in industries such as aircraft, where modifications
in design frequently continue after production of a model has begun.
Despite discussions with representatives of the aircraft industry aimed
at clarifying the problem, there were undoubtedly variations among the
aircraft companies in the types of engineering activities included in
the data on their R&D costs and personnel.

The definition of basic research was another item which was subject to
varying interpretations, as indicated earlier in this report. An
examination of the questionnaire returns indicated that a number of
companies classified as basic research activities not within the
intended meaning of the definition. Insofar as possible, letters were
written to companies which reported an unusually large proportion of
their R&D cost as basic research and corrections were made in the
figures in a number of cases on the basis of the replies received. In
addition, adjustments were made in the data for a number of other
companies which had apparently misinterpreted the definition.

Another important definition was that of the reporting unit for which it
was desired to obtain information. This definition was necessarily based
upon that used by the BOASI in the listing of companies from which the
sample was drawn. In accordance with the BOASI definition, separate
information was requested for each corporation or other legally separate
company in continental United States. It was recognized, however, that
some families of companies might find it difficult or impossible to
supply the requested figures for each separately incorporated
subsidiary, and it was therefore stated on the questionnaire that the
data might be submitted instead on a consolidated basis for the entire
corporate family. This alternative procedure was followed by about 200
parent companies with many subsidiaries, of which approximately 300 were
included in the designed sample. The data on numbers of companies in the
report therefore represent, in general, separate legal entities, but in
a relatively few cases represent corporate families. The definition of a
company also has an important bearing on the figures for different
industries and for different sizes of companies, as indicated in the
following section.

The employment controls had been established originally in terms of the
reporting unites as solicited. In those instances in which consolidated
returns were submitted by a corporate family, it was necessary to adjust
the controls. The adjustment process was carried out in a manner which
yielded an unbiased correction.

# Classification of Data

**Industry**

Since the sample of companies in this survey was drawn from the list
developed by the Bureau of Old Age and Survivors Insurance, and
stratified by industry according to the BOASI industrial classification
system, the classification of companies by industry in the survey is
based on that system. In accordance with it, the industry categories
used are those of the Standard Industrial Classification.[^1] The
industry of each establishment was determined on the basis of principal
activity or product; a company with more than one establishment was
given the industry code of the establishment (or establishments) having
the greatest number of employees.

The industry code assigned to a company by BOASI was changed in a
relatively small number of cases of the two following types: (1) Where
consolidated returns were received for an entire corporate family, all
the companies in that family were shown in the industry with which the
parent company was primarily associated; (2) the classification of a few
large companies was changed, because the industry codes given to them by
BOASI, based on criteria involving both employment and value of
shipments, seemed inappropriate to the present survey of research and
development. Thus, several oil companies, which had more employees in
merchandising than in refineries or production activities and were
classified by BOASI in trade, were shifted to the petroleum industry.

It should be noted that, as indicated in the preceding section, the
reporting unit in this survey was a company, rather than an
establishment as in many other types of economic data. In consequence,
all statistics for each company had to be classified in the single
industry which represented the company's principal field of employment.

**Size of Company**

Where data are shown separately for companies of different sizes, the
size categories used are based on total company employment in all
domestic activities as of March 1951. Since the sample was drawn from a
listing of companies compiled as of March 1951 and stratified on the
basis of company employment as of that date, use of these same
employment figures as the basis for classification of data in the report
greatly simplified the tabulations and reduced the variances of the
estimates. A careful comparison has been made of the distribution of
research cost and of the numbers of companies engaged in research, by
size of company, on the basis both of company employment in March 1951
and company employment in January 1954. In most estimating cells, there
was no significant difference in the findings from these two
tabulations. The figures for companies of different sizes given in the
report therefore portray, in general, the situation in early 1954.

[^1]: Executive Office of the President, Bureau of the Budget, *Standard
    Industrial Classification Manual,* Vol. I (Part 1 and 2, November
    and December 1945) and Vol. II (May 1949).