Establishment Respondents as Survey Managers: Using Survey Design 

Features to Empower Respondents to Find and Recruit Knowledgeable 

Company Personnel for Assistance1

Alfred D. Tuttle2

U.S. Census Bureau 

4600 Silver Hill Rd. 

Washington, DC 20233 

alfred.d.tuttle@census.gov 

I. Introduction

The National Science Foundation (NSF), as part of its original Congressional mandate, is charged with collecting data on 

science and engineering resources in the United States. Since 1953, NSF and the Census Bureau have co-sponsored a 

collection of industrial research and development (R&D) information via an annual enterprise-level survey. The Business 

R&D and Innovation Survey (BRDIS) is a new annual survey that replaced the former Survey of Industrial Research and 

Development starting in 2009. The BRDIS is being pilot-tested in a dress rehearsal with a full sample of 40,000 U.S. forprofit companies in 2009 and 2010. 

The BRDIS questionnaire is a paper booklet mailed to respondents, though they have the option of filling out and mailing 

back the booklet or entering their data into the Census Bureau’s Web-based collection instrument. Prior to mailout of the 

survey, previous respondents were mailed a letter announcing the new survey and asking that they verify their contact 

information on the Census Bureau’s website or provide updated information for a new contact. The respondents to the 

prior survey and to the BRDIS were typically personnel working in company finance or accounting areas. 

The new survey represents a substantial expansion over its predecessor in terms of the types of information it collects: 

additional information related to R&D finance and contracts, and new questions about R&D management and strategy, 

human resources (HR), and intellectual property (IP). These data requests require specialized knowledge beyond that 

possessed by the finance or accounting area respondents who typically were solely responsible for completing the previous 

survey. 

During pre-testing of the questionnaire, we learned of difficulties often faced by these finance area respondents at multiunit companies in 1) understanding the appropriate company personnel to provide the requested types of information, 2) 

finding those people among the myriad company units, and 3) obtaining the assistance of those people. In an attempt to 

address these difficulties, the BRDIS questionnaire provides explicit instructions for a “survey coordinator” to assist in 

understanding the types of data requested and the need to obtain assistance from personnel with specialized knowledge, 

along with the company units in which those personnel are likely to be found. These and other questionnaire design 

features are intended to encourage respondents to find specialized assistance rather than complete certain sections of the 

form themselves, and to address the concerns of other company personnel, in order to maximize the participation of 

appropriate company personnel. 

This paper will offer some preliminary findings from the pilot survey evaluating the effectiveness of various design features 

in empowering respondents in this way, based on results from respondent debriefings and initial analysis of reported data. 

II. Background

Our understanding of the establishment survey response process continues to evolve. At its core is the cognitive model

developed by Tourangeau (1984) that describes the individual’s interpretation of survey questions, retrieval of relevant 

information from memory, judgments as to whether what has been remembered matches the intent of the questions, and the 

1

 Draft version. Not for citation. 

2

 This report is released to inform interested parties of research and to encourage discussion. The views expressed on 

methodological, technical, or operational issues are those of the author and not necessarily those of the U.S. Census Bureau. 
communication of the information in the format provided by the survey. Edwards and Cantor (1991) adapted the cognitive 

model to organizations, expanding it to account for information being encoded in information management systems and the 

need for respondents with knowledge of those systems. Lorenc (2006) applies the concept of socially distributed cognition 

to the processes of inputting and transferring data via interactions between people and information systems – starting from 

the conditions in the real world that are being tracked, to the information being encoded in electronic systems, and finally to

being retrieved and reported on the questionnaire – in an attempt to quantify data quality and estimate measurement error. 

Tomaskovic-Devey et al. (1994) discuss the respondent’s authority, capacity, and motivation to respond to a survey request, 

all of which stem from the organizational context in which the survey is received. The organizational aspect is elaborated 

still further by Sudman et al. (2000) in their “hybrid” response model, which incorporates organizational dimensions of 

identification and selection of appropriate respondents, authority to delegate the task of responding to a survey and to 

release the requested information, and respondents’ competing job priorities. Willimack and Nichols (2001) and Willimack 

et al. (2002) describe the distribution within organizations of individuals with access to requested information and the 

necessity that some surveys requesting different types of data be fulfilled by multiple persons within an organization, as 

such representing multiple embedded cognitive processes. 

While becoming increasingly sophisticated in their ability to explain the response process for establishment surveys, the 

models of survey response cited above maintain focus on the individual respondents, with little discussion of other 

organizational factors beyond those directly affecting the individual. A more recent theory by Willimack (2007), drawing 

on theory and research in the areas of administrative science and organizational management, expands the organizational 

aspect of survey response to include interpersonal and behavioral factors that may impact the quality and likelihood of 

survey response. 

As noted in earlier work (Sudman et al. 2000), establishment surveys often require the participation of multiple persons 

with access to requested data. If the survey respondent is in a position of authority then he or she has the ability to compel 

cooperation and assistance from others, but in our experience this is often not the case. Respondents frequently have to 

utilize other forms of influence and persuasion to obtain data from others for completing a survey. Before additional 

respondents can be recruited to assist with the survey, they must be found, a difficult proposition in a large organization, 

made more difficult when the survey asks for information from areas with which the respondent may not be familiar. This 

is often the case with BRDIS, which asks questions related to R&D operations, human resources, and intellectual property, 

in addition to financial information. 

Based on findings from cognitive interviews conducted during the development of the survey, the questionnaire was 

designed to assist the process of finding and coordinating contributions from personnel in different parts of a respondent 

company. The paper form is a booklet divided into color-coded sections to distinguish the different types of questions. On 

page 2 of the booklet is a table of contents whose headers match the colors of the respective sections. Each section has a 

title page describing the types of questions the section contains and the likely location and type person in the company to 

answer them (see Figure 1). 

Figure 1: Example of section title page with instructions for finding the appropriate respondent 

A concern arose during the pre-testing phase that questions dealing with management and strategy of R&D activities might 

not always reach the appropriate person or persons in the company. These questions were intended to be answered by 

someone with a broad, high-level view of the company’s R&D activities who would be able to provide informed subjective 
responses. During pre-testing we heard from some finance area respondents that they would likely complete these questions 

themselves rather than trying to find someone from an operating unit or management organization to handle them, since the 

questions asked for details of funds spent on R&D as reported in the previous section completed by the same respondent. 

Also, some respondents reported that they had sought the assistance of others but were unable to find and/or secure the 

participation of appropriate respondents with knowledge of R&D operations for our cognitive interviews (Tuttle 2008), 

which may indicate comparable difficulty in getting their assistance with the actual survey request. In addition, being 

accountants concerned with the accuracy and validity of their reported data, these respondents described sometimes fairly 

burdensome records-based retrieval processes necessary to generate what they thought would be adequate answers to the 

questions. 

We took steps to increase the likelihood of the R&D management and strategy section of the survey reaching the intended 

respondents: 

 We changed the order of the questions, so that those found to be more difficult for finance area respondents appeared 

first, with the intent of discouraging them from completing this section themselves3

. 

 We asked for percentages rather than dollar amounts to make it more obvious that estimates were appropriate, to 

reduce the apparent burden of the questions, and to allay concerns about reporting sensitive data. 

 We also changed the title of the section and the instructions on who should complete it to emphasize the “technical, 

managerial, and strategic” aspects of R&D (see Figure 1). 

Some preliminary findings from debriefings with BRDIS respondents will be presented to describe the impact on some 

company respondents of these and other design features. 

III. Methodology

The findings presented in this paper come primarily from respondent debriefings, which are semi-structured in-person 

qualitative interviews conducted with respondents who recently completed and returned the BRDIS. The purpose of the 

respondent debriefings is to assist with evaluation of the BRDIS in terms of the quality of reported data and response error. 

The primary goal of the interviews is to understand the composition of reported data, respondents’ interpretations of the 

items, their response strategies, and respondent burden. 

The sample for the respondent debriefings is highly purposive, intended to augment other evaluations of the pilot through 

their depth, providing detailed explanations of response strategies and suspected response errors, especially among units 

critical to survey estimates. As is typical for business surveys, the BRDIS universe is skewed, with a relatively small 

number of very large companies accounting for the bulk of domestic private sector R&D activity. The first priority of the 

evaluations for the BRDIS pilot, respondent debriefings included, is understanding data quality issues affecting the large 

“certainty cases.” However, smaller companies were also recruited for debriefings. 

Companies with completed surveys were selected for respondent debriefings based on several considerations: 

 Industries of special interest with regard to R&D (e.g., technology, pharmaceuticals, aerospace, etc.); 

 Data variables of interest (e.g., R&D funded by the Federal government, outsourced R&D, etc.); 

 Inconsistent reporting within BRDIS or by comparison to accounting standards and publicly available information 

(e.g., reports required of publicly traded companies by the Securities and Exchange Commission), as identified by 

survey analysts; 

 Ownership by a foreign parent, or possession of foreign operations (to explore issues related to respondents’ 

interpretation of the desired reporting unit and inclusion of desired activities relative to affiliated foreign companies); 

 Significant item nonresponse (e.g., totals but no details) 

 Geographic locations with sufficient numbers of respondent companies to maximize efficiency of travel. 

 

3

 As Dillman et al. (2009, page 158) state, “A first question that is applicable to respondents communicates to them that the 

entire questionnaire is relevant to them and encourages their participation.” Our working hypothesis was that the converse 

is true: that a set of initial questions in the R&D Management and Strategy section found to be difficult to answer for 

finance area respondents would encourage them at least to seek guidance from persons in the company with the desired 

knowledge about company activities rather than attempt to answer it themselves without expert assistance. 
Once appropriate companies were identified, the survey coordinators were contacted and asked if they would be willing to 

participate in an interview. We determined whether the survey coordinators completed any portions of the form, and 

attempted to solicit contact information and/or referrals to respondents who completed other sections of the form, to attempt 

to recruit them for additional debriefings. Altogether we conducted 53 interviews with respondents at 47 companies. Five 

interviews were conducted by telephone, the rest in person. The duration of most interviews was 60 to 90 minutes, although 

in a few cases they lasted up to two hours. One or two subject matter specialists were present for nearly all interviews; 

frequently, others participated by telephone. 

Whenever possible, respondents were contacted and scheduled for debriefing within four weeks of receiving their 

responses. Our previous experience with conducting respondent debriefings suggests that respondents are more likely to 

remember specifics about their experiences with the survey within a month or so of completing it. The greater retention of 

response experiences results in richer interviews, and the lingering salience of the survey in respondents’ minds appears to 

increase the likelihood that they will agree to participate in debriefings. Although we were not always able to conduct 

debriefings within this time frame, the inability to remember some details was often mitigated to an extent by respondents’ 

routine retention of working papers and other documentation generated in the process of responding to the survey. This 

allowed us to understand with a great degree of accuracy the composition and sources of reported data and respondents’ 

interpretations of survey items. In the debriefings we also explored respondents’ terminology and record formation relative 

to the intended meanings of the survey items. Results will be used to inform recommendations on the design of the form for 

the next year of the survey collection. 

We interviewed four types of respondents: the coordinators who were usually also respondents for the R&D finance 

sections of the form; personnel from the companies’ human resources areas who provided responses for the human 

resources section; respondents from a legal department for the intellectual property section; and someone familiar with 

R&D operations, usually working in the office of Vice President of R&D (or equivalent) or in a financial analysis and 

forecasting role. We usually met with the various personnel separately, although in a few cases the survey coordinator was 

present as well. Many interviews involved multiple company personnel who assisted with completing or providing data for 

the section being discussed. 

It should be noted that because of the purposive nature of the respondent debriefing sample, it is not appropriate to make 

statistical inferences either to the survey sample or to the target population. Unlike structured survey interviews, respondent

debriefings rely on open-ended questions and follow-up probes of respondents’ initial answers, so instead of covering an 

identical list of topics across interviews, debriefings focus on emergent issues relevant to a specific company’s survey 

response. Although we attempt to cover specific debriefing protocol topics in every interview, it is not always possible, and 

there is no consistent denominator for a given protocol question across interviews. These debriefing findings are presented 

to illustrate observed organizational characteristics and their relevance for establishment surveys but not to make assertions 

about the distribution of these characteristics across the population of companies. 

IV. Findings

The coordinators and respondents we interviewed tended to have favorable impressions of the structure and appearance of 

the form, noting especially the separation of questions into topical sections and the use of color to distinguish one from 

another. Although it is difficult to gauge the usability of the form from statements about respondents’ impressions, we got

some indications that the color-coded sectional design assisted respondents in comprehending and navigating the form. 

Further, more than a few respondents reported that they saw and attended to the instructions describing the nature of the 

individual sections and the appropriate personnel in the company to provide the requested information. Some reported that 

these instructions were particularly helpful not only in terms of understanding what the sections were about, but also in that 

they provided specific key words on which to base their searches for likely company units where appropriate respondents 

might be found (e.g., “patenting, licensing, and other intellectual property”). 

Overall, the survey coordinators tended to be successful in finding help with the survey, especially from people in their 

companies’ human resources and legal departments. This success at securing assistance was in spite of the fact that many of 

them were not in a position of authority, either over the respondents or within the company as a whole. In many cases they 

had existing relationships with those personnel, since their day-to-day work sometimes requires that they request 

information from or share it with other units. 
Where prior contacts did not exist, respondents often reported starting with people they already knew either in their own 

offices or in other company locations. They described the nature of the survey and asked for a referral to a likely employee 

who might be able to help. Other respondents used company organizational charts or directories and looked for unit names 

and job titles that sounded promising, and made “cold calls” to those units. Some respondents described making several 

calls before finding someone to help with the survey. A few were not able to find assistance and completed those portions 

of the survey themselves as well as they could. 

A complication for several of the companies we visited was that they have no central R&D administration or authority 

structure, but conduct R&D in multiple units fairly independently. In these cases, there was no single person or unit where 

an appropriate respondent could be found. Instead, coordinators had to gather information from people at different 

locations, with varying degrees of success. Identifying locations where R&D was performed can be problematic since such 

activities are often co-located at manufacturing or other operating facilities, the names of which reflect activities other than 

R&D. Some respondents at such companies reported having to canvass their companies’ locations in order to identify 

those conducting R&D activities. By contrast, the human resources and legal functions appeared to be more often centrally 

located within the companies we visited. This fact may have contributed to the likelihood of finding those personnel to 

request their assistance with the survey. 

A key aspect of the process of finding help was the communication of the request, which often necessitated showing the 

questions that needed to be answered to potential respondents. Some coordinators made use of the electronic versions of the 

questionnaire offered on the Census Bureau’s website (pdf and Excel files), which they were able to send as email 

attachments or print out. However, not all coordinators were aware of the electronic options, and some of them created their 

own pdf or Excel versions of the form and sent those via email. Some coordinators made photocopies of the questionnaire 

sections and carried them to nearby respondents or sent them through inter-office mail to those located elsewhere. 

Coordinators tended to handle the various sections differently depending on the content of the sections and the people 

required to answer them. The human resources and intellectual property sections were more often “farmed out” 

completely, that is, sent to respondents in their entirety, and then returned to the coordinator for submission. This finding

leads us to hypothesize that intra-organizational requests for data are “normal” within some companies, or at least that 

cooperation is normal, especially among administrative units. Several respondents’ statements affirmed this. 

However, the R&D management section was often filled out by the coordinators or other finance area respondents, either 

completely or in part. The more frequent completion of the R&D management section by the finance area coordinators and 

respondents may be an indication that such data requests or other forms of cooperation are less common between 

administrative areas and operating units. Human resources and legal department personnel are likely more accustomed to 

fulfilling requests for information from within the company (e.g., management reporting, requests from survey 

coordinators) as well as from without (e.g., regulatory reports, patent applications). In contrast, personnel working in the 

“technical core” (Tomaskovic-Devey 1994) are focused on the company’s business operations. Interactions between them 

and administrative personnel (who are there, after all, to provide support, not make requests) may be uncommon and 

therefore less likely to be normal, since data requests of these types take them away from their “real” work. 

We did hear from some respondents to the R&D management section that they appreciated that the questions asked for 

percentages rather than precise dollar amounts. They said that being asked to report percentages made them think it was 

more acceptable to provide an informed “top-of-the-head” estimate rather than an exact figure which might have been far 

more burdensome or even impossible to provide. 

On the other hand, we did observe cases in which the survey was received by an executive in a position of authority over a 

company’s R&D activities or other operations, who then delegated one or more personnel to coordinate the completion of 

the survey. Those coordinators were formally mandated to carry out the survey task and thus were given access to 

appropriate personnel throughout the company, as well as the authority to compel their cooperation and/or sufficient 

credibility that it was an important task. 

V. Discussion

What is the significance of findings about organizational characteristics for the design and conduct of surveys? From a 

survey design perspective, there are many things that are beyond our control, but also some things that we may be able to 
influence. Knowing how people within companies interact and communicate with one another can assist us in designing 

questionnaires that may be able to take advantage of those existing processes. 

What organizational or inter-personal processes does the survey design need to activate or facilitate in order to maximize 

the likelihood of participation of appropriate respondents? An easy design feature for surveys that require a coordinator to 

solicit input from various types of respondents is information about who the appropriate respondents are and where they 

might be located, to facilitate more efficient search for those personnel. Just as the actual company data of interest to the 

survey program is (hopefully) encoded in company records and in the minds of its employees, so too is information about 

the personnel themselves, what they do for their companies and where they are located. Finding whether knowledgeable 

people exist in the company and where they are located is the first step in tapping their knowledge and access to records. 

Another easy design decision that will enable coordinators to find respondents more easily and quickly is to provide 

electronic versions that can be distributed via email, which is commonly described by respondents as the preferred means of 

communication, especially at larger companies. The apparent difficulty of completing survey questions may also be 

mitigated to some degree by asking for percentages rather than actual amounts, which may increase the likelihood that 

appropriate respondents will agree to provide information for the survey and thus reduce nonresponse. 

Other characteristics of organizations may not be so easily surmountable, such as the hypothesized normative barriers 

between administrative and operational personnel at some companies. Although some of the coordinators we interviewed 

reported greater difficulty in securing the assistance of personnel working in R&D areas than for the other sections of the 

survey, the likelihood of obtaining a completed survey may be increased by addressing the survey to an executive or highlevel manager with broad authority over parts of the company. However, a caveat is in order. In some industries, 

executives may be inundated with survey requests (e.g., from industry associations) and other mail, so a survey request 

must appear important enough to a gatekeeper to avoid being summarily dismissed. Federal government agencies may 

enjoy an advantage in this, but “name recognition” may not necessarily guarantee survey response. 

VI. Future Research

The BRDIS program staff embedded a small experiment in the survey collection to assess an alternative approach to 

eliciting responses to the new survey from historical non-responders to the previous survey. A sub-set of companies in the 

survey sample who did not respond to the previous survey in two of the last three years in which it was conducted were 

classified as "problematic". Instead of sending these companies a pre-survey notice asking for updated contact information 

to the most recent respondent on record for these companies, it was sent to the president or CEO, along with a letter 

explaining the nature of the survey and requesting that it be assigned to someone in the company capable of coordinating 

and returning a response. Anecdotal information from BRDIS staff offer some impressive examples of non-response 

turnaround using this approach, but empirical data from the experiment may provide more definitive evidence of its value

and will be available in the near future. Interviews with respondents at companies from which this strategy elicited 

response, as well as to those which still did not respond, would be useful in understanding reasons for which this approach 

did and did not work, and may shed more light on company processes that may help and hinder survey response, although 

such an evaluation is not planned at this time. 

Unit response is not the only issue under consideration in the collection of the BRDIS. Analyses of reported data are 

playing a central role in the evaluation of the BRDIS, both in terms of item non-response as well as data quality, the latter 

made possible through internal validity checks and comparisons to historical SIRD data and publicly available information. 

The BRDIS survey also collected the names and titles of persons who assisted with each section. It was intended that these 

data would be used to understand the types of company personnel involved in the survey and compare them to the quality 

of data they reported, to help determine whether the sections of the survey were being completed by appropriate 

respondents. These data analyses will be used as the bases for future investigations of problematic survey items, and of 

alternative questionnaire design features and contact strategies to address issues related to finding appropriate respondents. 

Another evaluation method under consideration for the next collection of the BRDIS is a response analysis survey (RAS), a 

survey administered to a statistical sub-sample of companies in a survey sample. A RAS can be used to explore topics 

similar to respondent debriefings, e.g., the composition of reported data, the perceived ease or difficulty of reporting data, 

the centralization/dispersal of requested data in company records, etc. Using a RAS in conjunction with respondent 

debriefings would enable the investigation of response-related issues at a detailed level as well as statistical statements 
about their distribution across the survey sample. 

Another investigative tool that would be particularly useful for exploring the organizational aspects of survey response is 

social network analysis (SNA). SNA makes use of both qualitative and quantitative techniques. Ethnographic-type 

interviews and observations, as well as analyses of company records, communications, etc., can be used to build a detailed 

understanding of the survey coordinator’s known contacts tapped for assistance with the survey, her/his attempts to find 

new contacts, and organizational characteristics that may promote or challenge obtaining assistance. Comparisons of 

coordinators’ and respondents’ within-company networks across different companies can be used to create typologies of 

networks and of company characteristics helpful and/or detrimental to survey response. Once a coordinator’s contacts are 

identified and her/his network is “mapped out,” the members of the network can be queried on issues related to their 

association with the coordinator (“ego” in SNA terms) and to their participation in the survey or lack thereof. Structured 

interviews of network members or questionnaires can be used to collect standardized data amenable to statistical analyses 

and the development of models to describe organizational response processes, and their distribution. 

References 

Dillman, D.A., Smyth J.D., and Christian L.M. (2009) Internet, Mail, and Mixed-Mode Surveys: The Tailored Design 

Method. New Jersey: Wiley. 

Edwards, W. S., & Cantor, D. (1991). “Toward a response model in establishment surveys.” In P. P. Biemer, R. M. Groves, 

L. E. Lyberg, N. Mathiowetz & S. Sudman (Eds.), Measurement Errors in Surveys. New York: Wiley. 

Tomaskovic-Devey, D., Leiter, J., & Thompson, S. (1994) "Organizational Survey Nonresponse." Administrative Science 

Quarterly. 39:439-457. 

Lorenc, Boris (2006) Propagation of Representational States in a Survey of Schools.” Paper presented at the European 

Conference on Quality in Survey Statistics, Cardiff, Wales, UK. 

Sudman, S., Willimack, D. K., Nichols, E., & Mesenbourg, T. (2000). “Exploratory research at the U.S. Census Bureau on 

the survey response process in large companies.” Paper presented at the Second International Conference on Establishment 

Surveys, Buffalo, New York, USA. 

Tourangeau, R. (1984) “Cognitive science and survey methods.” In T. B. Jabine, M. L. Straf, J. M. Tanur & R. Tourangeau 

(Eds.), Cognitive aspects of survey methodology: Building a bridge between disciplines. Washington DC: National 

Academy Press. 

Tuttle, A.D. (2008) “Establishment Respondents as Survey Managers: Evaluating and Applying an Organizational 

Model of Survey Response.” Poster presented at American Association of Public Opinion Research 63rd Annual 

Conference, New Orleans, Louisiana, USA. 

Willimack, D. K., & Nichols, E. (2001) “Building an alternative response process model for business surveys.” Paper 

presented at the American Association for Public Opinion Research 56th Annual Conference, Montreal, Quebec, Canada. 

Willimack, D. K. et al. (2002) “Understanding Unit and Item Nonresponse in Business Surveys,” in R.M. Groves et al. 

(Eds.) Survey Nonresponse. New York: Wiley. 

Willimack D. K. (2007) “Considering the Establishment Survey Response Process in the Context of the Administrative 

Sciences.” Paper presented at the Third International Conference on Establishment Surveys, Montreal, Quebec, Canada.