Introduction 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Determinants Of Web Reporting: 

A Qualitative Study Of Mode Selection

Tony Hak1

Erasmus University, P.O. Box 1738, 3000 DR Rotterdam, The Netherlands 

thak@fbk.eur.nl 

Amy E. Anderson and Diane K. Willimack2

Census Bureau, 4700 Silver Hill Road, Washington DC 20233, 

amy.e.anderson@census.gov, diane.k.willimack@census.gov 

Respondents are offered electronic reporting in an effort to increase the quality and timeliness of reporting. The cost 

of administering a survey electronically is substantially lower for the Census Bureau. Moving from paper-based to

electronic data collection is a priority for the Census Bureau. We have been introducing electronic survey options

into several existing data collection programs over the past decade. The goal of the research described in this paper 

was to identify factors that influence electronic reporting uptake rates in an effort to assess and improve current 

electronic reporting practices. The current electronic forms at the Census Bureau for establishments cover a variety 

of surveys. These surveys vary in frequency and complexity ranging from monthly indicator surveys to the detailed

economic census, which is administered every five years. In an effort to help capture a range of the different 

electronic reporting experiences, we conducted our research with two widely different survey programs: the monthly

Manufacturers’ Shipments, Inventories, and Orders Survey and the 2002 Economic Census. 

Manufacturers’ Shipments, Inventories, and Orders (M3) Survey 

The program. The Manufacturers’ Shipments, Inventories, and Orders (M3) Survey is a voluntary monthly survey

administered to approximately 3,500 manufacturing companies in the U.S. The M3 survey collects at most seven 

data items about manufacturers’ shipments, inventories, and orders. The Web reporting option for the M3 was 

offered to a subset of the M3 sample in 2000. In 2002, the Web survey was completely redesigned with a new

layout. The main reporting modes for the M3 have historically been fax and Touch Tone Data Entry (TDE). 

Companies can either be considered single form or multiple form companies. Single form companies are those who

primarily manufacture one type of good or goods that are very similar. These companies receive only one form per 

month. Multiple form companies are those that manufacture several different types of goods, and they receive more 

than one form per month. Because Web reporting was not available for multiple form companies at the time of our 

research, we will report here only about single form companies. 

The electronic form. The M3 electronic form was designed to be simple and to not add respondent burden. 

Respondents are taken to a main menu screen after entering their username and password using a login screen. The 

main menu screen allows users to go right to filling out their form, to review the form, print it, and view its status

(complete vs. not complete). 

Aim of our research. The Census Bureau wanted M3 respondents who were reporting in non-electronic modes (fax 

and mail-out/mail-back) to switch to Web reporting. The aim of this research was to look at respondents’ reasons for 

choosing or not choosing to switch reporting modes. 

1

This research was conducted while Tony Hak was an American Statistical Association / National Science Foundation Research 

Fellow at the U.S. Census Bureau (2002-2003). 

2

This paper reports the results of research and analysis undertaken by Census Bureau staff and their collaborators. It has 

undergone a Census Bureau review more limited in scope than that given to official Census Bureau publications. This report is 

released to inform interested parties of ongoing research and to encourage discussion of work in progress. The authors thank their 

reviewers for their helpful review comments. 

Page 55 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Methodology. In December 2002, M3 staff approached mail/fax reporters by telephone with a request to switch to 

Web reporting. Callers were instructed to probe respondents who refused the electronic option. Ten on-site visits

were conducted to observe how respondents handled reporting through the Web for the first time. For respondents, 

these visits were aimed at helping them adapt to this new method of reporting and to solve any problems that might

occur. For the researcher, it was an opportunity to watch real-time reporting of data using a Web mode. 

Arrangements were made for follow-up telephone calls with the companies that agreed to switch to an electronic 

mode but were not visited in-person. The purpose of the follow-up telephone call was to get additional feedback

about the electronic survey and any issues with converting to a new mode. These respondents were contacted as 

soon as possible after their data was submitted to the Census Bureau. 

Findings. Of the 77 mail/fax reporters that were approached by telephone, eleven respondents refused outright to

convert to Web reporting. A considerable number (22) of the 66 respondents that expressed their willingness to give 

Web reporting a try never did so. Respondents told us repeatedly that they had not tried electronic reporting for

practical reasons, and promised us to attempt it next time. They never did, therefore we conclude that these were

‘soft refusals’.

Refusal was mainly based on the assumption that fax reporting cannot be beaten in terms of objective response

burden. These respondents saw no good reason for changing a routine that was convenient to them and for 

substituting another system that takes more of their time, not accounting for the additional burden of the effort of 

switching modes itself. In contrast, some converters assumed that Web reporting would reduce their burden. Other 

converters did not find burden an issue and did not mind that Web reporting adds to burden. In other words, refusers

and converters differed mainly in terms of their expectations regarding the burden of Web reporting.

Eventually 37 companies (48%) were converted to Web reporting, with those respondents having received an offer 

of an onsite company visit netting 50% more adopters than those offered only a telephone debriefing call. Among 

converters, no difference was found between respondents who expected Web reporting to take less time than fax 

and/or mail and those who had no such expectations. All agreed that Web reporting is preferred, even though it took

more time. The main reasons respondents gave us for these preferences were: 

• Respondents do not need to move from their desk to file the report. 

• Web reporting fits with how they see their work developing in the future from paper-based to paper-less. 

• Some feel more confident that data have actually and safely been submitted.

Our main finding, thus, was that fax reporting continues to be the most attractive and easiest method of reporting for 

at least half of the reporters, but that for converters the perceived advantages of Web reporting seem to outweigh its

perceived disadvantages. 

The conversion process. Our findings revealed four steps to the conversion process, and we draw the following 

conclusions. 

The decision to try electronic reporting: The initial decision to convert to Web reporting was not dependent on

established company policies or technical restrictions. Respondents’ initial decisions were based on a comparison of

the level of burden associated with the current method of reporting to the level of perceived burden associated with

the new method of reporting. For the M3, respondents already believed that reporting through fax could not be

improved upon. There will be respondents who will assume that reporting through the Web reduces burden and

others who will not find issue with the added burden and will choose Web reporting for other reasons. 

A first attempt at reporting electronically: M3 Web reporting is a fairly straightforward task for most respondents. 

Some usability problems were uncovered during company visits, but none that affected adoption of Web reporting.

A decision to adopt this new mode of reporting: We found no difference between respondents who expected fax 

reporting to take less time and those who did not. All reporters that made the first attempt at Web reporting liked it, 

and told us that they preferred Web reporting, although it took more time. This decision was based on reasons 

mentioned previously, as well as respondents’ confidence that their data had safely been submitted. If respondents 

can be convinced to try Web reporting, it seems very likely that they will adopt this method long-term with

satisfaction.

Future use of this new mode: All of the respondents who tried Web reporting once stated that they would continue 

Page 56 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The 2002 Economic Census 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

using this mode in the future. Follow up research has found that the majority of these respondents did in fact remain

with the Web reporting. Very few respondents switched back to fax reporting in subsequent months. 

Conversion methods. The conversion rate in this study was 48%. In January 2003, a letter was sent to all 3500

companies in the entire M3 sample to inform them that mail reporting would be phased out. Additionally three 

remaining other modes (Web, TDE, and fax) were presented in the letter, of which Web reporting was the first in the 

list. After this mailing there were a number of ‘spontaneous’ Web reporters. We collected data on the reporting 

mode for all M3 sample companies for six consecutive monthly reports (December 2002 thru May 2003) after this

letter was sent. This allowed us to compare the eventual success of the personal conversion calls with that of 

‘spontaneous conversion’ invoked by the January letter. After six months there were 96 stable ‘spontaneous’ Web 

reporters (approximately 3%). It was concluded that an intensive approach (in this case a conversion request through 

telephone accompanied with an offer of a company visit or a telephone debriefing) is a much more effective, though 

more costly, conversion strategy than merely sending a letter. 

Recommendations. This study showed that an active conversion strategy helped in moving respondents from a 

nonelectronic mode (fax/mail) to an electronic mode. Completely phasing out fax reporting and converting to Web

reporting is not recommended at this time because likely resistance will occur with respondents who are content 

with fax reporting. To convert such resistant respondents, the Census Bureau may need to adopt an approach

offering continual support until the respondents demonstrate sufficient confidence in reporting via the Web. Such a 

supportive approach would demonstrate to respondents how important Web reporting is for the Census Bureau, as

well as how much these respondents’ contribution is appreciated. 

Knowledge of other non-electronic reporting modes (mail/fax) was an obstacle for converting respondents to the 

Web. With this in mind, we speculate that uptake of Web reporting might be more successful among respondents 

who are not accustomed to another mode – e.g., new sample cases. New respondents could be given the Web 

version as the only reporting option. Other options could be offered to respondents that had technical issues or were 

seriously resistant to Web reporting.

The program. Our second study was of the economic census, which provides a comprehensive look at most of the 

industries in the U.S. economy from all geographic levels. The economic census is an establishment-based survey 

conducted every five years, for years ending in “2” and “7”. It is a major component for many economic measures 

including U.S. Gross Domestic Product. In the past, the economic census was collected mainly by self-administered 

paper forms. Economic census questionnaires are tailored by industry, resulting in more than 550 different versions. 

For 2002, all respondents were offered the option of electronic reporting. The Census Bureau sees electronic 

reporting as a means of reducing survey costs while increasing survey data quality (using imbedded edits) and 

response timeliness (by allowing submission of responses using the Internet). 

The electronic reporting system. The electronic form for the 2002 Economic Census is a Windows based 

application that must be downloaded to a respondent’s personal computer (PC) from a diskette, CD, or through the 

Internet. Promotion of electronic reporting was done in two different ways. For very large companies in all 

industries, Census Bureau analysts made telephone, mail, or personal contact in an effort to recruit the company into

electronic reporting. The remaining companies were told about the electronic reporting option in the cover letter that 

accompanied the form. The cover letter listed the Internet address needed for downloading the application as well as 

instructions for finding respondents’ usernames and passwords, which had been pre-printed in the mailing label. 

When respondents first open the application, they see a window containing general information for using the 

electronic form. Respondents then move on to a “survey in-box,” where they receive a list of all the forms they are 

required to complete, one for each establishment. Respondents use the in-box to toggle between forms, print forms, 

and to submit forms. The in-box displays information about each form, including the form number, the 

establishment’s address, and whether or not the form contains errors or warnings. 

The most common way for companies to complete their forms using the software application is to navigate through 

the form keying in data where necessary. Companies also have the option of importing data into the electronic form

from a preformatted spreadsheet. In order to use this option, companies had to map questionnaire items to

Page 57 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

spreadsheet columns, selecting items relevant to their business. Such “import maps” had to be created by

respondents for each form type assigned to their company.

Aim of our research. The aim of this qualitative research was to understand user problems with the electronic 

instrument in the context of the response process in order to evaluate the software from the users’ perspective. We

wanted to develop specific recommendations for electronic reporting in the 2007 Economic Census, along with

general recommendations for other electronic establishment surveys at the Census Bureau. 

There was interest in a variety of topics related to the 2002 Economic Census electronic form: 

• How long did it take to download the software and prepare the software for reporting? Do respondents consider

these times reasonable?

• What are the major problems encountered when reporting electronically?

• If respondents chose to import their data, did they find the procedure user friendly?

• What were the most significant benefits (if any) of choosing the electronic reporting options?

• What were the least significant aspects of the electronic reporting option?

• What improvements or recommendations can respondents offer to enhance the software?

• Are respondents likely to use the software again to report for other economic programs? 

Methodology. To answer the many questions posed about this electronic software, we aimed at collecting as much 

detailed data as possible about the actual disparate but interconnected steps involved in responding electronically to

the 2002 Economic Census. We developed two approaches to data collection:

On-site real-time visits. Visits were arranged with nine very large companies to observe their complex response 

processes. In order to get this firsthand experience with tasks necessary for companies to complete the electronic 

census form, we identified respondents who were willing to give us direct access to the work they were doing. This 

access was facilitated by economic census analysts who were working one-on-one with companies. These analysts 

identified companies that would welcome on-site visits to discuss issues. These companies were offered support in

dealing with the electronic reporting system. Our visits were seen as an instrument for customer relations to help 

companies report timely quality data. 

A researcher who was accompanied by either an analyst or a member of the Electronic Reporting Branch (ERB) 

made all company visits. Analysts were helpful in answering questions about the form content, while ERB staff 

were the “experts” on the electronic reporting system. Meetings with companies were audio-recorded with the 

respondent’s permission, and some companies were visited more than once. Respondents were told during initial 

contact that researchers would be interested in discussing their response process and (hopefully) observing parts of

it. 

Retrospective on-site debriefing visits. Experiences of electronic reporters in smaller companies are typically 

different than those in large companies. Since it would have been very difficult to make similar real-time 

arrangements with small companies, we recruited businesses that submitted their 2002 Economic Census form

electronically. Companies were typically visited within a few weeks of completing the electronic form. These 

companies were debriefed about all of the different tasks that were necessary for them to complete the survey 

electronically. They were also asked for feedback. Fifteen companies were debriefed. These companies ranged in 

size from companies having one location to companies with 30 locations. Meetings were audio-recorded with the 

respondent’s permission. 

Findings: large companies. Large companies took the 2002 Economic Census very seriously. Companies spent

significant amounts of time preparing for this survey. They printed advance copies of the survey to identify data they

would need to gather. They spent a significant amount of time matching their lists of locations to the lists provided

by the Census Bureau. 

Most of the companies we visited chose to import their data into the electronic form from spreadsheets. They felt 

that although importing required a considerable amount of time up-front, it would save them a significant amount in

the end. Company respondents were very adept at working with spreadsheets. Many respondents were able to create 

programs or macros that automatically pulled data from their financial systems into the spreadsheet. However, 

Page 58 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

companies needed to spend additional time “cleaning-up” their spreadsheets to match the formatting required for 

importing. 

Many companies struggled with the importing feature initially. For some it was the case of unclear instructions that 

necessitated frequent calls to the Census Bureau for assistance. For others it was the tediousness of getting their 

spreadsheets ready for importing. The amount of time it took to create import maps and to actually import the data

into the electronic instrument was close to the amount of time required to actually gather the data. 

Early versions of the software did not function well for many of the companies visited. They experienced long wait 

times for downloading forms from the Internet or importing a handful of data cells. Later versions of the software 

were more efficient. 

The system was not built to be shared by multiple reporters within the company. There was neither ‘spawning’ 

functionality (i.e. a way of distributing forms to different persons in the company) nor consolidation functionality 

(i.e., a way of bringing different completed forms together). Large companies typically have more than one person

completing and gathering data for the economic census. The system was built to be the most beneficial for 

companies that had a centralized reporting process. If decentralized companies were interested in reporting 

electronically, they had to spend time either coordinating their efforts through one central contact person or come up

with other creative solutions for distributing the software. The decentralized companies included in our research 

spent many more hours than other types of companies in order to report electronically. One company we visited

abandoned reporting electronically because of this and reverted back to completing paper forms. This was a tedious

alternative they had been trying to avoid.

The software’s review (editing) function was less than ideal for very large companies, because editing was done on a 

form-by-form basis. Very large companies can have thousands of forms, and it is not feasible for them to perform

this review by individual location. Since companies were very interested in performing a review of their data, they

would return to their spreadsheets to look for problems. 

Findings: small and medium-size companies. The smaller companies that chose to report electronically had a 

much different reporting experience than larger companies. The reporting responsibility was typically given to one

person instead of several. It was not necessary for smaller companies to invest their time in the importing function of

the software. To complete their surveys, these smaller companies typically key entered their data location-bylocation. They tended to use paper forms as a reference during data retrieval, completing the paper forms first, and 

then using the electronic software as a data entry and submission device. 

These respondents ran into very few problems with the system itself and they were pleased with the performance 

and appearance of the 2002 Economic Census electronic form. However, it must be remembered that these 

respondents were self-selected, choosing to report electronically. It was not possible for us to interview companies 

that had downloaded the electronic form, but abandoned it, submitting a paper forms instead. 

Since smaller companies have fewer forms to handle, they were happier with the review functionality built into the 

software and felt this was a major asset to the electronic form. Regardless of their size, companies did not seem to 

have any problems with the submission process. The confirmation message, however, was unclear and many

respondents reported being unsure whether their submission had been successful. 

Some of the reasons respondents gave for choosing to report electronically included:

• Neatness and readability of report forms: Many respondents noted that, in the past, they had used mechanical 

typewriters to complete the forms. Although it was not mandatory for answers to be typed, many respondents felt it 

necessary. 

• Ease of a dding new locations was easy.

• No need to send the forms through registered mail, which saved time and money. 

• Fits with the priority of companies to go paper-less. 

• No fear of responses getting lost in the mail. 

• Sending in data that is higher in quality. 

Page 59 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Some of the recommendations that respondents gave for improving the system included:

• Giving a clear confirmation that submission had been successful. Some respondents suggested that an e-mail

confirmation would be ideal. Confirmations should be printable for respondents’ records.

• Making the electronic form more like the paper form. (We assume this recommendation is related to our 

observation that respondents repeatedly used the paper form as a reference throughout the reporting process.) 

• Assigning a status to forms. Companies managing large numbers of forms needed some additional help in figuring

out which of their forms were blank, in progress, contained errors, or complete. 

• Integrating the Census Bureau software with common payroll, financial, or spreadsheet software. 

Conclusions. The response process involved with reporting for the economic census varied widely across all 

companies and posed several different challenges to the electronic reporting system. The system was able to meet 

several of these challenges, but could not adequately deal with others. Our main conclusions are:

• The current system is adequate for self-selected electronic reporters in small companies. These companies 

perceived the rewards of electronic reporting outweighed the minimal added burden. The current software met 

respondents’ expectations. 

• The importing functionality of the system is too burdensome. The amount of effort and time needed for importing

was something that respondents in large companies did not anticipate. Added to this time was the long time that it 

took to actually import the data into the system. Although respondents did state that overall they probably did save

time by importing, they still felt strongly that the current importing process could be greatly improved.

• The current system is not useful for companies with a decentralized response process. Decentralized companies 

that chose to work with the system needed to make special arrangements. The system could not be distributed to

more than one PC to accommodate companies with multiple internal respondents. 

Our findings regarding large companies indicate that some assumptions of the software developers regarding 

respondents’ work practices and preferences were incorrect. Developers had assumed that respondents would prefer

the opportunity to create their own custom import maps rather than use generic import maps. Respondents, however, 

considered the need to create these maps as an unwarranted additional burden to their core response work (perceived

by them as the work of retrieving the requested information and making it available to the Census Bureau in some

form). Another incorrect assumption was that all large companies would want to make a clear distinction between

information providers in establishments on the one hand, and central reporters for the economic census on the other

hand, and that central reporters would want to consolidate information received from their various establishments in

spreadsheets before preparing this consolidated information for communication to the Census Bureau. In actual

practice, some central respondents only wanted to have a monitoring role and, therefore, preferred that informants 

from their various establishments be able to load their data directly onto the electronic forms. The latter was not

possible because the software had no spawning or consolidation functionality. 

Recommendations. Although research into business respondents’ preferences and response processes could lead to 

more realistic assumptions to guide software development, we assume there will remain unknown situations in

companies that will differ in significant ways. That is, we doubt it is possible to develop assumptions that cover all 

relevant response situations. Therefore, we recommend that further software development not be aimed at

developing an electronic reporting system that meets requirements connected to a limited number of known or

assumed circumstances and preferences, but at developing a system that is less dependent on such assumptions and 

is, instead, flexible enough to be adequate in circumstances not known beforehand. In our view, this would be a 

respondent-centered approach. 

Overall conclusions and recommendations 

Large companies in the economic census. Most respondents in most economic programs of the Census Bureau in

which an electronic reporting option is (or will be) available will have a choice regarding the reporting mode they 

want to use, either electronic or another mode (fax, mail, or TDE). The one exception is the group of very large 

companies for which completing the economic census using a non-electronic reporting option is not viable. For the

latter companies, it is of paramount importance that electronic reporting software for the 2007 Economic Census be 

improved significantly, particularly in terms of the importing, spawning and consolidation functions. The system

should be respondent-centered in the sense that it is flexible enough to adapt to many different circumstances that 

may not be known beforehand. 

Page 60 
 

 

 

 

 

 

 

 

 

 

 

 

 

Mode selection. Our main conclusion about mode selection is that respondents have subjective reasons for adopting 

or not adopting the electronic reporting option. Respondents’ attitudes can be influenced by means of an intensive 

approach in which support is offered to respondents (either through telephone or by on-site visits), or by promoting 

subjective reasons for adoption rather than objective (burden) reasons. Nevertheless, in the near term at least, we

suspect that substantial numbers of respondents in multi-mode surveys will likely exercise their right to opt for a 

non-electronic mode. 

Thinking outside the box. We observed that respondents repeatedly used the paper form as a reference throughout 

the reporting process and we assume this is the reason why many respondents want the paper and electronic 

instruments to look alike. However, this does not mean that electronic reporting instruments must be based on

current paper forms. Designing electronic instruments first will take advantage of the flexibility and capabilities of

the electronic medium, and the paper forms can subsequently be designed to resemble the electronic instrument. In

particular, we suggest enabling respondents to ‘build’ their own customized forms (in the manner by which routing

is managed in CAPI and CATI instruments) and to print the resulting form. Such a form-building functionality – as 

well as the spawning functionality recommended above – may be more easily supported in a Web-based

environment than in a downloadable system.

Page 61