####### TECHNICAL NOTES FOR 1992

####### 

####### INTRODUCTION

This report is the third of three publications produced from the 1992
Survey of Industrial Research and Development. The first, a Data Brief
announcing the availability of survey results, contains some analytical
information and describes recent changes to the survey that are
discussed in detail below. The second, a Selected Data report, contains
16 of the most frequently requested tabulations and was published while
this more comprehensive report was being prepared. This report, the
Detailed Statistical Tables report, contains the full set of statistics
produced from the survey. All three of the publications provide
statistics on research and development (R&D) funding for the years
1982--92 and on R&D personnel for the period from January 1983 to
January 1994.

This report provides national estimates of the expenditures on R&D
performed within the United States by industrial firms, whether U.S. or
foreign owned. Among the statistics, classified various ways, are
estimates of total R&D, the portion of the total financed by U.S.
Government funds, and the portion financed by the companies themselves
(or by other non-Federal sources such as State and local governments or
other industrial firms under contracts or subcontracts). Total R&D is
also separated into its character-of-work components: basic research,
applied research, and development. Other R&D statistics include those on
the funds for R&D financed by the domestic firm but performed outside
the United States and on the funds spent to perform energy-related R&D.
Also in this report are statistics on R&D-performing companies including
domestic net sales, number of employees, number of R&D-performing
scientists and engineers, and cost per R&D scientist and engineer.

The Survey of Industrial Research and Development is a sample survey
that intends to include or represent all for-profit, nonfarm
R&D-performing companies, either publicly or privately held. The
survey's primary focus is on U.S. industry as a performer of, rather
than as a source of funds for, R&D. Thus, data on Federal support of R&D
activities performed by industry are collected and resulting statistics
appear in several tables, but statistics on industrial funding of R&D
undertaken at universities and colleges and other nonprofit
organizations are not collected and therefore are not included in the
tables.[^1]

Industry statistics are developed from data collected from individual
companies or enterprises. Since the survey is enterprise based rather
than establishment based, all data collected for the various subparts of
each enterprise (plants, divisions, or subdivisions) are tabulated in
the major standard industrial classification (SIC) of the company. The
resulting industry estimates are reported using the SIC of the companies
within the each industry. National totals are estimated by summing the
industry estimates.

All companies known, through previous surveys or through outside
information sources, to spend more than \$1 million annually on R&D in
the United States or to have 1,000 or more employees receive a survey
questionnaire every year. Remaining firms are subjected to probability
sampling and may or may not receive a questionnaire for a given survey
year. Among the organizations purposely excluded from the survey are
trade associations and not-for-profit consortia. Although their primary
mission is to serve industry, these associations are established as
nonprofit organizations.

Respondents receive detailed definitions to help them determine which
expenses to include or exclude from the R&D data they provide.
Nevertheless, the statistics presented in this report are subject to
response and concept errors caused by different respondent
interpretations of the definitions of R&D activities and by variations
in company accounting procedures. Consequently, the statistics are
better indicators of changes in, rather than absolute levels of, R&D
spending and personnel.

The National Science Foundation (NSF) has sponsored a survey of
industrial R&D since 1953. The two surveys covering the 1953--56 period
were conducted by the Bureau of Labor Statistics (BLS), U.S. Department
of Labor.[^2] Since 1957 the Bureau of the Census has conducted the
survey.[^3] NSF's Division of Science Resources Studies sponsors and
monitors the survey.

The content of the survey has been expanded and refined over the years
in response to an increasing need by policymakers for more detailed
information on the Nation's R&D effort. For example, questions on energy
R&D were added in the early seventies, following the first oil-shortage
crisis. On the other hand, the frequency of collection of certain data
items has been reduced in recent years in an attempt to alleviate some
of the respondent burden that has been placed on industry from all
sources. For large firms known to perform R&D, a detailed questionnaire,
Form RD-1L, is used to collect data for odd-numbered years and an
abbreviated version, Form RD-1S, is used to collect data for
even-numbered years. To further limit reporting burden on small R&D
performers and on firms that are included in the sample for the first
time, an even more abbreviated form, Form RD-1A, which collects only the
most crucial data, is used each year. This report provides data
collected from the abbreviated forms, RD-1S and RD-1A.

Several changes have been made to the survey recently that are of
special importance to users of this report. Prior to the 1992 survey,
statistics were based on samples selected at irregular intervals (i.e.,
1967, 1971, 1976, 1981, 1987). In intervening years a subset of the last
sample (called a "panel") was used. The most recent sample prior to the
1992 survey was selected and first used for survey year 1987. Estimates
for 1988 through 1991 were based on surveys of the panel of companies
that reported R&D activity in the 1987 survey. Beginning with the 1992
survey, statistics are based on samples selected annually. Also,
beginning with the 1992 survey, the sample size was increased from
approximately 14,000 to nearly 23,400 firms. This increase was made for
several reasons: (1) to better account for births of R&D-performing
establishments in the survey universe, (2) to survey more fully and
accurately R&D performed by nonmanufacturing firms (especially in the
service sector and by small firms in all industries), and (3) to gather
more current information about potential R&D performers. In this report
tables containing historical statistics are presented in two ways. For
the tables in section A, estimates from the 1992 survey are linked with
estimates from the 1987 survey. The linking was accomplished using an
algorithm that preserved to the greatest extent possible year-to-year
trends for each industry. A full explanation of the linking process used
is explained in this section, under "Comparability of Statistics." Also
under "Comparability of Statistics," there is a series of tables for
which no attempt was made to link the estimates derived from data
collected in the 1992 survey to estimates derived from data collected in
previous surveys.

####### SURVEY METHODOLOGY[^4]

######## Reporting Unit

The reporting unit for the Survey of Industrial Research and Development
is the enterprise, or company, defined as a business organization of one
or more establishments under common ownership or control. The survey
includes two groups of enterprises: (i) companies known to conduct
research and development (R&D) and (ii) a sample representation of
companies for which information on the extent of R&D activity is
uncertain.

######## Frame Creation

The Standard Statistical Establishment List (SSEL), a Bureau of the
Census compilation that contains information on over 3 million
establishments with paid employees, was the universe from which the
frame used to select the 1992 survey sample was created (see table B-1
for universe and sample sizes). For companies with more than one
establishment, data were summed to the company level. The firm was then
assigned a single standard industrial classification (SIC) code based on
the activity of the establishment(s) having the highest dollar value of
payroll. This assignment was done on a hierarchical basis. The
enterprise was first assigned to the economic division (manufacturing or
nonmanufacturing) with the highest payroll, then to the 2-digit SIC code
with the highest payroll within the assigned division, then to the
3-digit SIC code with the highest payroll within the assigned 2-digit
industry.

The frame from which the survey sample was drawn included all for-profit
companies classified in nonfarm industries. For surveys prior to 1992,
the frame was limited to companies above certain size criteria based on
number of employees. These criteria varied by industry. Also, some
industries were excluded from the frame because it was believed that
these industries contributed little or no R&D activity to the final
survey estimates. For the 1992 sample, new industries were added to the
frame[^5] and the size criteria were lowered considerably and applied
uniformly to firms in all industries. As a result, nearly 2 million
enterprises with 5 or more employees were given a chance of selection.
For comparison, the frame for the 1987 sample included 154,000 companies
of specified sizes and industries.

External information about the likelihood that a company conducted R&D
was used to identify nearly 10,000 companies that were included in the
survey sample with certainty. External sources included prior R&D
surveys, directories that include company information on R&D reported to
the Securities and Exchange Commission, commercially available
directories of R&D performing companies, Department of Defense
directories of contracts awarded for R&D, and various publications and
newsletters that highlight firms conducting R&D. In addition, all
companies in the frame with 1,000 employees or more were selected with
certainty.

######## Sample Selection

**Probability Proportionate to Size**

As with most types of economic surveys, the sample was selected using
probabilities proportionate to size. That is, large companies had a
higher probability of selection than did small companies. For this
survey it would have been ideal if company size could have been
determined by the amount of R&D expenditures. Unfortunately, except for
the companies that were in a previous survey or for which there was
information from external sources, it was impossible to know the R&D
expenditure values for firms in the universe. Consequently, most
companies' R&D expenditures had to be estimated, with the probability of
selection based on those estimated values.

Since total employment was known for each company in the universe, it
was possible to use an already-observed relationship between employment
and R&D to estimate R&D expenditure values for companies in the frame.
This was the same strategy employed in the 1981 and 1987 sampling
operations. For 1992 sampling, data collected in the 1991 survey was
used to derive this relationship separately for single-unit companies
and multiestablishment companies. The effect in all cases was to give
firms with a large number of employees higher probability of selection
since it was assumed that large companies were more likely to perform
R&D and that the amount of R&D was proportionate to the size of the
company.

######### Sample Stratification and Relative Standard Error Constraints

The particular sample selected was one of a large number of the same
type and size that by chance might have been selected. Statistics
resulting from the different samples would differ somewhat from each
other. These differences are represented by estimates of sampling error.
The smaller the sampling error, the more precise the statistic.

To control sampling error in the statistics resulting from this survey,
parameters were specified to allocate the sample across various levels,
or strata, that corresponded to industry groupings. These parameters
permitted the sample size to be varied to achieve a desired level of
sampling error for each stratum and were assigned so that estimated
errors of total R&D for industries in these strata did not exceed
certain levels. Sample sizes among the strata were only constrained by
the limit placed on the total sample size dictated by the available
budget.

For sample selections prior to 1992, the strata designations were the
published industry categories. The sample was allocated across these
industry categories to provide high, medium, and low levels of
precision. For the 1992 sample the criteria for this allocation were
modified. In order to gather information to review and evaluate the
appropriateness of the published industry groupings, the allocation of
the sample was controlled for levels of industry detail below those
traditionally published. The result was that the frame was partitioned
into 95 manufacturing industry strata and 25 nonmanufacturing strata.

Each industry was allocated to one of three groups. The first group was
formulated to analyze the distribution of data in manufacturing
industries. In this group each 3-digit manufacturing industry was
considered a separate stratum. The second group was formulated to
improve coverage and to identify emerging industries. In this group,
selected 2-digit and 3-digit nonmanufacturing industries each were
considered a separate stratum. The industries were identified as those
for which statistics had been published previously and those with high
concentrations of scientists and engineers as reported in occupational
surveys. The third group was a large stratum of companies in
nonmanufacturing industries that had not been included in previous
sampling frames or for which there was little indication of R&D
activity.

Once the strata were defined, the following criteria were used to
achieve the target sampling error for total R&D.

a\. Sampling error not to exceed 2 percent for: 44 3-digit manufacturing
industries that contribute to a current publication level below the
2-digit industry level and 15 3-digit nonmanufacturing industries that
represent a current publication level or for which there is a high
concentration of scientists or engineers.

b\. Sampling error not to exceed 5 percent for: 51 3-digit manufacturing
industries that are part of currently published 2-digit or aggregations
of 2-digit industries; 9 2-digit nonmanufacturing industries for which
R&D activity was likely, and 1 stratum of remaining nonmanufacturing
industries for which there was little prior indication of R&D activity.

Based on the desired precision represented by these sampling error
estimates, the criteria suggested a total sample size of approximately
23,000.

A limitation of the sample allocation process should be noted. Sampling
errors were controlled by using a universe total that, in large part,
was improvised. That is, as previously noted, an R&D value was assigned
to every company in the frame, even though many of these companies
actually may not have had R&D expenditures. The value assigned was
imputed for the majority of companies in the frame and, as a
consequence, the estimated universe and the distribution of individual
company values did not necessarily reflect the true distribution.
Estimates of sampling variability were nevertheless based on this
distribution. The presumption was---and this had been confirmed using
the previous sample selection---that actual variation in the sample
design would be less than that estimated, because many of the sampled
companies have true R&D values of zero, not the widely varying values
that were imputed using total employment as a predictor of R&D. Thus,
the 2-percent and 5-percent error levels described earlier are
conservative. (See table B-2 for a list by industry of the actual
standard error estimates for total R&D.)

In addition to sampling error, the estimates are subject to nonsampling
error. Errors are grouped into five categories: specification, coverage,
response, nonresponse, and processing. For detailed discussions on the
sources, control, and measurement of each of these types of error, see
the technical reports cited below.[^6]

######### Sample Size and Weighting

The sample was selected with a target sample size of 23,000 and with
other parameters set to ensure compliance with the standard error
constraints. An actual sample of 23,376 was selected. The actual sample
size differed from the target for two reasons. First, the sample frame
was subjected to independent sampling. Each company in the frame had an
independent chance of selection, based on its assigned probability,
i.e., selection of a company was completely independent of the selection
of any other company. In independent sampling, sample size itself is a
random variable. Theoretically, a sample of size zero or a sample the
size of the entire universe is possible, but the probabilities of these
extremes are so small that these are nearly impossible situations. The
actual sample size is usually quite close to the specified size. If
there is too much deviation, the selection is simply executed again.

Second, a minimum probability rule was imposed. As noted earlier,
probabilities of selection proportionate to size are assigned to each
company, where size is the imputed R&D value assigned each company.
Selected companies that report actual R&D expenditures vastly larger
than their assigned values can have adverse effects on the statistics,
which are based on the weighted value of survey responses.[^7] To lessen
the effects on the final statistics, the maximum weight a company could
assume was arbitrarily controlled by specifying the probability of the
company's selection. If the probability, based on company size, was less
than the arbitrarily set minimum, then the probability was set equal to
the minimum value. The consequence of raising these original
probabilities to the minimum probability was to raise the expected
sample size. It is likely that most of the difference between the size
of the target sample and the actually selected sample was because of
this rule.

####### Survey Questionnaires

Two questionnaires are used each year to collect data for the survey.
For large firms known to perform R&D, a detailed questionnaire, Form
RD-1L, is used to collect data for odd-numbered years and an abbreviated
version, Form RD-1S, is used to collect data for the even-numbered
years. The questionnaires are cycled in this manner to reduce reporting
burden on survey respondents.

The Form RD-1L requests data on sales or receipts, total employment,
employment of scientists and engineers, expenditures for R&D performed
within the company with Federal funds and with company and other funds,
character of work (basic research, applied research, and development),
company-sponsored R&D expenditures in foreign countries, R&D performed
under contract to others, expenditures for pollution abatement and
energy R&D, detail on R&D by product field, Federal R&D support to the
firm by contracting agency, domestic R&D expenditures by State, and
foreign R&D by country. The Form RD-1S requests the same information
except for the last four items. Because companies receiving the Forms
RD-1L and RD-1S generally have participated in previous surveys,
computer imprinted data reported by the company for the previous year is
supplied for reference. Companies are encouraged to revise or update
this imprinted data if they have more current information.

To further limit reporting burden on small R&D performers and on firms
that are included in the sample for the first time, an even more
abbreviated form is used each year. Form RD-1A collects data only on
R&D, sales, employment, and operational status and includes a screening
item that allows respondents to indicate that they do not perform R&D.
No prior-year information is available since the majority of the
companies have not reported previously.[^8]

For the 1992 survey, about 1,600 companies received Form RD-1S and
nearly 22,000 received Form RD-1A. Of the 22,000 firms, 1,760 reported
R&D expenditures.

####### Followup for Survey Nonresponse

The 1992 survey questionnaires were mailed in May 1993, and recipients
were asked to respond within 60 days. Thirty days later, letters were
mailed to all survey recipients reminding them that their completed
questionnaire was due within the next 30 days. After 60 days, followup
letters were sent to all nonresponding firms. Two additional followup
mailings were made to persistent nonrespondents, after 90 and 120 days.
The 90-day followup mailing included a replacement questionnaire.

In addition to the mailings, telephone followup was used to encourage
response from those firms ranked among the 300 largest R&D performers,
based on total R&D expenditures reported in the previous survey.
Telephone followup was also used for these firms during the initial data
edit phase of survey operations if data items were missing or unclear.

####### Imputation for Item Nonresponse

For various reasons, many firms chose to return the survey
questionnaires with one or more blank items.[^9] For instance, the
internal accounting procedures of the firm may not have allowed it to
quantify the pollution-abatement expenditures portion of R&D. In
addition, some firms, as a matter of policy, refused to answer any
voluntary questions.[^10]

When respondents did not provide the requested information, estimates
for the missing data were made using imputation algorithms. In general,
the imputation algorithms computed values for missing items by applying
the average percentage change for the target item in the nonresponding
firm's industry to the item's prior-year value for that firm, reported
or imputed. This approach, with minor variation, was used for most
items.[^11] Table B-3 contains imputation rates for the principal survey
items.

######### Character of work

Response to questions about character of work (basic research, applied
research, and development) declined in the mid-1980s and, as a result,
imputation rates increased. The general imputation procedure described
above became increasingly dependent upon information imputed in prior
years, thereby distancing current-year estimates from any reported
information. Because of the increasing dependence on imputed data, NSF
chose not to publish character-of-work estimates in 1986. Consequently,
the imputation procedure used to develop these estimates was revised in
1987 for use with 1986 and later data and differs from the general
imputation approach. The new method calculates the character-of-work
distribution for a nonresponding firm only if that firm reported a
distribution within a 5-year period, extending from 2 years before to 2
years after the year requiring imputation. Imputation for a given year
is initially performed in the year the data are collected and is based
on a character-of-work distribution reported in either of the 2 previous
years, if any. It is again performed using new data collected in the
next 2 years. Thus, character-of-work estimates are revised as newly
reported information becomes available and are not final for 2 years
following their initial publication.

If no reported data are available for a firm, character-of-work
estimates are not imputed. As a consequence, only a portion of the total
estimated R&D expenditures are distributed at the firm level. Those
expenditures not meeting the requirements of the new imputation
methodology are placed in a "not distributed" category. Tables B-4
through B-8 show the character-of-work estimates along with the "not
distributed" component for 1988--92, respectively.

NSF's objective in conducting the survey has always been to provide
estimates for the entire population of firms performing R&D in the
United States, however, the revised imputation procedure would no longer
produce such estimates because of the "not distributed" component. So, a
baseline estimation method was developed to allocate the "not
distributed" amounts among the character-of-work components. In the
baseline estimation method, the "not distributed" expenditures are
allocated, by industry group, to basic research, applied research, and
development categories, using the percentage splits in the distributed
category for that industry. The allocation is done at the lowest level
of published industry detail only; higher levels are derived by
aggregation (just as national totals are derived by aggregation of
individual industry estimates), and results in higher performance shares
for basic and applied research and lower estimates for development's
share than would have been calculated using the previous method.[^12]

####### Response Rates and Mandatory Versus Voluntary Reporting

Detailed unit and item response rates are shown in tables B-9 and B-10,
respectively. Table B-9 shows the number of companies in each industry
or group of industries that received a questionnaire and the percentage
that responded to the survey. Table B-10 shows the percentage of firms
with R&D expenditures that also reported data for selected items or
groups of items.

Current survey reporting requirements divide survey items into two
groups: mandatory and voluntary. Response to four data items on the
questionnaires (total R&D expenditures, Federal R&D funds, net sales,
and total employment) is mandatory, whereas response to the remaining
items is voluntary. During the 1990 survey cycle, NSF conducted a test
of the effect of reporting on a completely voluntary basis to determine
if combining both mandatory and voluntary items on one questionnaire
influences response rates. For this test, the 1990 sample was divided
into two panels of approximately equal size. One panel, the mandatory
panel, was asked to report as usual (four mandatory items and the
remainder voluntary), and the other panel, the voluntary panel, was
asked to report all items on a completely voluntary basis. The result of
the test was a decrease in the overall survey response rate to 80
percent from levels of 88 percent in 1989 and 89 percent in 1988. The
response rates for the mandatory and voluntary panels were 89 percent
and 69 percent, respectively. Detailed results of the test were
published in *Research and Development in Industry: 1990.*

####### COMPARABILITY OF STATISTICS

Summarized in this section are the statistical revisions that have been
made because of changes in survey procedures and practices.[^13] This
section is divided into two parts. The first focuses on the current-year
survey with a discussion of recent survey improvements and the effects
these have had on current-year and immediate prior-year statistics. The
second part describes revisions made to statistics produced from
pre-1992 surveys.

####### Current-Year Considerations

######### Recent Survey Improvements[^14]

Before the 1992 survey, the sample of firms surveyed was selected at
irregular intervals.[^15] In intervening years a panel of the largest
firms known to perform R&D was surveyed. For example, a sample of about
14,000 firms was selected for the 1987 survey. For the 1988 through 1991
studies, about 1,700 of these firms were annually resurveyed; the other
firms did not receive another questionnaire, and their R&D data were
estimated. This sample design was adequate during the early years of the
survey because the performance of R&D remained concentrated in the
manufacturing industries. However, as more and more firms began entering
the R&D performing arena, the old sample design proved increasingly
deficient because it did not capture births of new R&D performing firms;
the entry of fledgling R&D performers into the marketplace simply was
missed during panel years. Additionally, beginning in the early 1970s,
the need for more detailed R&D information for nonmanufacturers was
recognized. At that time, statistics for the broad industry
classifications "miscellaneous business services" and "miscellaneous
services" were added to the list of industry groups for which statistics
were published. By 1975 about 3 percent of total R&D was performed by
firms in nonmanufacturing industries.

During the mid-1980s there was evidence that an increasing number of
nonmanufacturing firms were conducting a significant amount of R&D, and
again the number of industries used to develop the statistics for
nonmanufacturers was increased. Consequently, the annual reports in this
series for 1987 and since have included separate R&D estimates for firms
in the communication, utility, engineering, architectural, research,
development, testing, computer programming, and data processing service
industries; hospitals; and medical labs. Approximately 9 percent of the
estimated industrial R&D performance during 1987 was undertaken by
nonmanufacturing firms.

In addition to adding to the list of industries for which statistics
were published, it became clear from these observations that the sample
design itself should be changed to reflect the widening population of
R&D performers among firms in the nonmanufacturing industries and small
firms in all industries, to account better for births of R&D performing
firms and to produce statistics that are generally more reliable. So,
beginning with the 1992 survey, NSF decided to (1) draw new samples with
broader coverage annually and (2) increase the sample size to
approximately 23,000 firms.[^16] As a result of the sample redesign, for
1992 the reported nonmanufacturing share was estimated to be 25 percent
of total R&D.[^17]

######### Revisions to Immediate Prior-Year Statistics

As has been the practice throughout the history of the survey, results
from the current-year survey are used not only to develop current-year
statistics, but also to revise immediate prior-year statistics.
Differences between originally developed statistics and revised
statistics occur for three reasons: industry shifts, data revisions,
and, of particular importance in the discussion of the 1992 survey
results, the effects of a new sample. Table B-11 quantifies these
effects for each industry and industry grouping.

**Industry shifts.---**The movement of a company from one industry into
another can be caused by several factors: changes in a company's payroll
composition, which is used to determine the industry classification code
(see discussion above under "Frame Creation"), changes in the industry
classification system itself, and changes in the way the industry
classification code is assigned or revised during survey processing.
These are described below.

**Payroll composition.---**A company's payroll composition changes
because of a number of events. Among them are (1) the growth or decline
of product or service lines; (2) the merger of two or more companies;
(3) the acquisition of one company by another; (4) divestitures; or (5)
the formation of conglomerates. With annual sampling, when it is
determined that a company's payroll composition and therefore its
industry classification has changed, the company's data are reclassified
into the new industry beginning in the year of the change. Prior to
annual sampling, firms were not subject to annual reclassification. Most
of the shifts in R&D performance between industries detailed in table
B-11 undoubtedly stemmed from changes in companies' payroll composition.

**Industry classification system.---**From time to time the standard
industrial classification (SIC) coding system, which is used by most
Federal Government agencies that publish industry statistics, is revised
to reflect the changing composition of U.S. industry. For statistics
developed for 1988--91 from the 1988--91 surveys, companies retained the
industry classifications assigned for the 1987 sample. These
classifications were based on the 1977 SIC system. The last major
revision of the SIC system was for 1987, so this new system was used to
classify companies in the 1992 survey. Consequently, the 1992 statistics
and revised 1991 statistics in this report were developed using the 1987
SIC system and minor data shifts are attributable to the system change.
For example, the 1987 system expanded SIC 30, rubber products, to
include a variety of specific plastic products that may have been
classified elsewhere using the 1977 system.

**Processing changes.---**Finally, in response to perceived changes in
the amount and dispersion of R&D among industries and findings of
various quality improvement initiatives and other research undertakings,
the sponsor of the survey, in consultation with the compiling agent,
from time to time seeks to improve the coverage of the survey by
revising the method used to classify firms. Research has shown that
there is no impact on the aggregated statistics because of these
processing changes and the impact on individual industry estimates is
minor.[^18] The current method used to classify firms is discussed above
under "Frame Creation." Methods used for past surveys are discussed in
the technical paper cited below.[^19]

As table B-11 shows, in the aggregate, industry shifts had no effect on
the revised 1991 estimate of total R&D. However, the effects are evident
among the industry groupings. Most affected were statistics for the
electrical equipment (SIC 36), transportation equipment (SIC 37), and
nonmanufacturing industries. Approximately \$6.9 billion of R&D
previously reported for manufacturing industries was shifted to
nonmanufacturing industries in the revised 1991 statistics.

**Data revisions.---**Changes to reported data can come from two
sources: from respondents (see discussion above under "Survey
Questionnaires") and from analysts involved in survey and statistical
processing. Respondents from companies that were in both the 1991 and
1992 surveys may have revised previously reported data for 1991.
Analysts, while performing followup, may have corrected incorrectly
reported or supplied missing 1991 data. Data revisions accounted for
\$1.0 billion or 6.8 percent of the \$14.7 billion revision to the 1991
estimate of total R&D.

**Sample design.---**Changes to the sample design can dramatically
affect revisions to immediate prior-year estimates. By far the most
profound influence on the revisions to the 1991 statistics was the new
sample design. It accounted for \$13.7 billion or 93.2 percent of the
\$14.7 billion revision to total R&D with most of this amount (\$11.4
billion) attributable to the wider sampling of the nonmanufacturing
industries.[^20]

To summarize, differences between originally published and revised 1991
statistics stem from industry shifts, data revisions, and the new
sample. Of the three, the new sample had the largest effect because it
included a larger number of industries, especially among
nonmanufacturing classifications, and potentially a larger number of
firms, especially small firms, in all classifications. Comparing the
1991 panel and the 1992 sample, the sample reflected changes in the
universe that could not be accounted for by the panel. The frame for the
sample included industries that were not represented previously under
the assumption that companies in those industries contributed little or
no R&D activity.[^21] Further, data for small R&D performers were
imputed for the panel years and used in the original 1991 statistics.
The revised 1991 estimates from the 1992 survey included actually
reported data for many more small companies.

####### Linking Current-Year Statistics with Statistics from Previous Surveys

######### Time Series Analyses

As discussed earlier, the statistics resulting from the survey are
better indicators of changes in, rather than absolute levels of, R&D
spending and personnel. Nevertheless, the statistics are often
considered as a continuous time series that has been prepared using the
same collection, processing, and tabulation methods. Such uniformity of
preparation has not been the case. Since the survey was first fielded,
improvements have been made to increase the reliability of the
statistics and to make the survey results more useful. To that end,
existing practices have been changed and new procedures have been
instituted. Preservation of the comparability of the statistics has been
an important consideration when improvements have been made, however.
Changes to survey definitions, the industry classification system, and
the procedure used to assign industry codes to multiestablishment
companies[^22] have had some, though not substantial, effects on the
comparability of statistics.[^23] The aspect of the survey that had a
greater effect on comparability was the selection of samples at
irregular intervals (i.e., 1967, 1971, 1976, 1981, 1987, 1992) and the
use of a subset or panel of the last sample drawn to develop statistics
for intervening years. As discussed above, this practice introduced
cyclical deterioration of the statistics.

To compensate for this deterioration, periodic revisions have been made
to the statistics produced from the panels surveyed between sample
years. Early in the survey's history, various methods were used to make
these revisions.[^24] Since 1976, a linking procedure called "wedging"
has been used.[^25] Simply described, in wedging the 2 sample years on
each end of a series of estimates serve as benchmarks in the algorithms
used to adjust the estimates for the intervening years.

######### Wedging Methodology

For a full discussion of the mathematical algorithm used for the wedging
process that linked statistics from the 1992 survey with those from the
1987 survey, see the technical memorandum cited below.[^26] In general,
the memorandum states that wedging---

> takes full advantage of the fact that in the first year of a new panel
> \[when a new sample is selected\], both current year and prior-year
> estimates are derived. Thus, two independent estimates exist for the
> prior year. The estimates from the new panel are treated as superior
> primarily because the new panel is based on updated classifications
> \[the industry classifications in the prior panel are frozen\] and is
> more fully representative of the current universe (the prior panel
> suffers from panel deterioration, especially a lack of birth
> updating). The limitations in the prior panel caused by these factors
> are naturally assumed to increase with time, so that in the revised
> series, we desire a gradual increase in the level or revision over
> time which culminates in the real difference observed between the two
> independent sample estimates of the prior year. At the same time, we
> desire that the annual movement of the original series be preserved to
> the degree possible in the revised series.

To that end, the wedging algorithm does not change estimates from sample
years and adjusts estimates from panel years, recognizing that
deterioration of the panel is progressive over time.

######### Wedged Versus Not-Wedged Statistics

One of the primary reasons for the decision to select a new sample
annually rather than at irregular intervals was to avoid the necessity
to apply global revision processes like wedging. Consequently, the 1992
survey is intended to be the last one for which wedging is an issue.
Tables N-1 through N-18 are identical to the section A tables except
that they contain statistics that are not wedged for 1988--90.

####### Revisions to Historical Statistics

Throughout the history of the survey, during regular survey processing,
all immediate prior-year statistics have been subject to revision with
results from the current year's survey. Changes to older statistics,
however, usually have been limited to revisions because of changes in
the industry classification of companies caused by changes in payroll
composition detected when a new sample was drawn. Various methodologies
have been adopted over the years to revise, or backcast, the data when
revisions to historical statistics have become necessary.

Documented revisions to the historical statistics from post-1967 surveys
are summarized in *Research and Development in Industry: 1991* (NSF
94-325). Detailed descriptions of the specific revisions made to the
statistics from pre-1967 surveys are scarce. However, summaries of some
of the major revisions are included in the technical paper cited
below.[^27]

####### SURVEY DEFINITIONS

######### *Company (and other) funds for R&D*---Cost of R&D actually performed within the company and funded by the company itself or by other non-Federal sources via contract, not including the cost of R&D supported by companies but contracted to outside organizations such as research institutions, universities and colleges, nonprofit organizations, or (to avoid double-counting) other companies.

######### *Cost per R&D scientist or engineer*---The arithmetic mean of the numbers of full-time-equivalent (FTE) scientists and engineers engaged in the performance of R&D reported for January in 2 consecutive years divided into the total R&D expenditures of the earlier year, with the ratio attributed to the earlier year (For example, the mean of the numbers of FTE R&D scientists and engineers in January 1991 and January 1992 is divided into total 1991 R&D expenditures for a total cost per R&D scientist or engineer in 1991.)

######### *Employment, FTE R&D scientists and engineers*---Persons employed by the company during the January following the survey year who are engaged in scientific or engineering work at a level that requires knowledge of physical, life, engineering, or mathematical science equivalent at least to that acquired through completion of a 4-year college program with a major in one of those fields. (The statistics in this report show the full-time-equivalent (FTE) employment. The FTE is the number of scientists and engineers in the company who are assigned full time to R&D projects plus the number of non-full-time R&D scientists and engineers prorated according to the fraction of their total work time spent on R&D projects.)

######### *Employment, total*---Number of persons domestically employed by R&D-performing companies in all activities during the pay period that includes the 12th of March.

*Federally funded research and development centers
(FFRDCs)***---**R&D-performing organizations administered by industrial,
educational, or other institutions on a nonprofit basis, exclusively or
substantially financed by the Federal Government (R&D expenditures of
the FFRDCs that are industry administered are included with the Federal
R&D data of the industry classification of each of the administering
firms. The industry-administered FFRDCs included in the 1992 survey are
listed below.)

FFRDCs supported by the Department of Energy:

Bettis Atomic Power Laboratory

Westinghouse Electric Corp.

West Mifflin, PA

Energy Technology Engineering Center

Rockwell International Corp.

Canoga Park, CA

Hanford Engineering Development Laboratory

Westinghouse-Hanford Corp.

Richland, WA

Idaho National Engineering Laboratory

EG&G Idaho, Inc.;

Argonne National Laboratory, West;

Rockwell International Corp.;

Westinghouse Electric Corp.

Idaho Falls, ID

Knolls Atomic Power Laboratory

General Electric Co.

Schenectady, NY

Oak Ridge National Laboratory

Martin Marietta Energy Systems, Inc.

Oak Ridge, TN

Sandia National Laboratories

Western Electric Co., Inc.---Sandia Corp.

Albuquerque, NM

Savannah River Laboratory

Westinghouse Electric Corp.

Aiken, SC

FFRDCs supported by the National Institutes of Health, Department of
Health and Human Services:

NCI Frederick Cancer Research Facility

Program Resources, Inc.

Frederick, MD

*Funds for R&D, Federal***---**Receipts for R&D performed by the company
under Federal R&D contracts or subcontracts and R&D portions of Federal
procurement contracts and subcontracts.

*Funds for R&D, total***---**Operating expenses incurred by a company in
the conduct of R&D in its own laboratories or other company owned or
operated facilities including wages and salaries, materials and
supplies, property and other taxes, maintenance and repairs,
depreciation, and an appropriate share of overhead, not including
capital expenditures.

*Net sales and receipts*---Dollar values for goods sold or services
rendered by R&D-performing companies to customers (outside the company),
including the Federal Government, less such items as returns,
allowances, freight, charges, and excise taxes. (Domestic intracompany
transfers and sales by foreign subsidiaries are excluded, but transfers
to foreign subsidiaries and export sales to foreign companies are
included.)

*Research and development*---Basic and applied research in the sciences
and engineering and the design and development of prototypes and
processes, excluding quality control, routine product testing, market
research, sales promotion, sales service, other nontechnological
activities or routine technical services, and research in the social
sciences or psychology.

> *Basic research*---Original investigations for the advancement of
> scientific knowledge not having specific immediate commercial
> objectives, although such investigations may be in fields of present
> or potential interest to the reporting company.
>
> *Applied research*---Investigations for the discovery of new
> scientific knowledge having specific commercial objectives with
> respect to products or processes. (Applied research differs from basic
> research chiefly in terms of the objectives of the reporting company.)
>
> *Development*---Technical activities not routine in nature concerned
> with translating research findings or other scientific knowledge into
> products or processes. (Not included are routine technical services to
> customers or other activities excluded above.)

[^1]: Data on R&D performed at universities and colleges are collected
    in the annual Survey of Scientific and Engineering Expenditures at
    Universities and Colleges. More information about this survey is
    available from NSF's Science and Engineering Activities Program in
    the Division of Science Resources Studies.

[^2]: See National Science Foundation, *Science and Engineering in
    American Industry: Final Report on a 1953--54 Survey* (NSF 56-16)
    and *Science and Engineering in American Industry: 1956* (NSF 59-50)
    (Washington, DC: Supt. of Documents, U.S. Government Printing
    Office, 1956 and 1960).

[^3]: Data obtained in the earlier BLS surveys are not directly
    comparable with Census figures because of methodological and other
    differences.

[^4]: Information for this section was provided by the Industry Division
    of the Bureau of the Census, the collecting and compiling agent for
    the National Science Foundation. Copies of the technical papers
    cited can be obtained by contacting NSF's Science and Engineering
    Activities Program in the Division of Science Resources Studies.

[^5]: These industries are listed and discussed under Comparability of
    Statistics later in this section.

[^6]: U.S. Department of Commerce, Bureau of the Census, *Documentation
    of Nonsampling Issues in the Survey of Industrial Research and
    Development,* RR94/03 (Washington, DC, September 1994) and U.S.
    Department of Commerce, Bureau of the Census, *A Study of Processing
    Error in the Survey of Industrial Research and Development,*
    ESMD-9403 (Washington, DC, September 1994).

[^7]: The weight given to a company selected for the survey is the
    inverse of its probability of selection. Companies selected for the
    sample with certainty (see "Frame Creation" above) represented only
    themselves and each had a weight of 1.0.

[^8]: For the 1992 survey, companies were asked to report R&D
    expenditures for both the current and previous years. For subsequent
    years, only current year data will be requested.

[^9]: For detailed discussions on the sources, control and measurement
    of error resulting from item nonresponse, see the technical report:
    U.S. Department of Commerce, Bureau of the Census, *Documentation of
    Nonsampling Error Issues in the Survey of Industrial Research and
    Development,* RR94/03 (Washington, DC, September 21, 1994). For a
    general discussion of the problems stemming from item nonresponse,
    see the technical report: National Science Foundation, *Estimating
    Basic and Applied Research and Development in Industry: A
    Preliminary Review of Survey Procedures,* NSF 90-322 (Washington,
    DC, 1990).

[^10]: All but four items---total R&D, Federal R&D, net sales, and total
    employment---which are included in the Census Bureau's annual
    mandatory statistical program, are voluntary. See further discussion
    under "Response Rates" and "Mandatory Versus Voluntary Reporting"
    below.

[^11]: For detailed descriptions and analyses of the imputation methods
    and algorithms used, see the technical report: U.S. Department of
    Commerce, Bureau of the Census, *An Evaluation of Imputation Methods
    for the Survey of Industrial Research and Development,* ESMD-9404
    (Washington, DC, September 1994).

[^12]: See the NSF technical report cited above for an explanation of
    the uncertainties in the data and to quantify their sensitivity to
    the choice of various possible imputation procedures.

[^13]: See also the technical paper: U.S. Department of Commerce, Bureau
    of the Census, *Documentation of the Survey Design for the Survey of
    Industrial Research and Development: A Historical Perspective*
    (Washington, DC, 1995).

[^14]: See also National Science Foundation, SRS Data Brief, "1992 R&D
    Spending by U.S. Firms Rises, NSF Survey Improved," Vol. 1994, No. 5
    (Arlington, VA, 09/09/94).

[^15]: During the early years of the survey, until 1967, samples were
    selected every 5 years. Subsequent samples were selected for 1967,
    1971, 1976, 1981, and 1987.

[^16]: Annual sampling also will remedy the cyclical deterioration of
    the statistics that results from changes in a company's payroll
    composition because of product line and corporate structural
    changes.

[^17]: For the 1992 survey, 25 new nonmanufacturing industry and
    industry groups were added to the sample frame: agricultural
    services (SIC 07); fishing, hunting, and trapping (09); wholesale
    trade-nondurables (51); stationery and office supply stores (5112);
    industrial and personal service paper (5113); groceries and related
    products (514); chemicals and allied products (516); miscellaneous
    nondurable goods (519); home furniture, furnishings, and equipment
    stores (57); radio, TV, consumer electronics, and music stores
    (573); eating and drinking places (581); miscellaneous retail (59);
    nonstore retailers (596); real estate (65); holding and other
    investment offices (67); hotels, rooming houses, camps and other
    lodging places (70); automotive repair, services, and parking (75);
    miscellaneous repair services (76); amusement and recreation
    services (79); health services (80); offices and clinics of medical
    doctors (801); offices and clinics of other health practitioners
    (804); miscellaneous health and allied services not elsewhere
    classified (809); engineering, accounting, research, management, and
    related services (87); and management, and public relations services
    (874).

[^18]: The effects of recent changes in the way companies are classified
    during survey processing are discussed in detail in a Bureau of the
    Census technical memorandum entitled "Reclassification of Companies
    in the 1992 Survey of Industrial Research and Development for the
    Generation of the 'Analytical' Series," October 25, 1994.

[^19]: U.S. Department of Commerce, Bureau of the Census, *Documentation
    of the Survey Design for the Survey of Industrial Research and
    Development: A Historical Perspective* (Washington, DC, 1995).

[^20]: Of the \$11.4 billion increase in the revised 1991
    nonmanufacturing R&D total, the 25 industry groups newly added to
    the survey frame accounted for over \$2.0 billion in R&D
    expenditures.

[^21]: That assumption could not be verified, nor an informed change
    made to it because these companies were not surveyed in panel years.

[^22]: For discussions of each of these, see the Bureau of the Census
    technical memorandum entitled "Wedging Considerations for the 1992
    Research and Development (R&D) Survey," June 10, 1994.

[^23]: See the Bureau of the Census technical memoranda entitled
    "Reclassification of Companies in the 1992 Survey of Industrial
    Research and Development (R&D) for the Generation of the
    'Analytical' Series," October 25, 1994 and "Effects of the 1987 SIC
    Revision on Company Classification in the Survey of Industrial
    Research and Development (R&D)," December 6, 1993.

[^24]: See U.S. Department of Commerce, Bureau of the Census,
    *Documentation of the Survey Design for the Survey of Industrial
    Research and Development: A Historical Perspective* (Washington, DC,
    1995).

[^25]: The process was dubbed "wedging" because of the wedgelike area
    produced on a graph that compares originally reported statistics
    with the revised statistics that result after linking.

[^26]: Bureau of the Census technical memorandum entitled "Wedging
    Considerations for the 1992 Research and Development (R&D) Survey,"
    June 10, 1994.

[^27]: U.S. Department of Commerce, Bureau of the Census, *Documentation
    of the Survey Design for the Survey of Industrial Research and
    Development: A Historical Perspective* (Washington, DC, 1995).