# The Impact of Survey Response Rates on Research Validity and Reliability

Hamed Taherdoost, Mitra Madanchian

► To cite this version:

Hamed Taherdoost, Mitra Madanchian. The Impact of Survey Response Rates on Research Validity and Reliability. Journal of Advances in Library and Information Science, 2024, Advances in Library and Information Science, pp.177-206. 10.4018/979-8-3693-1135-6.ch009. hal-05087377

HAL Id: hal-05087377

https://hal.science/hal-05087377v1

Submitted on 27 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Chapter 9

The Impact of Survey
Response Rates on
Research Validity
and Reliability

Hamed Taherdoost

<figure><img src="image_2.png" /></figure>

https://orcid.org/0000-0002-6503-6739

University Canada West, Canada & Global University Systems (GUS), London,
UK

Mitra Madanchian

University Canada West, Canada

ABSTRACT

The response rate, which represents the number of individuals who answered the survey, is one of the factors that impact the validity and reliability of a research study. This chapter aims to investigate the factors that impact the response rate and acceptable values. A scoping literature review is conducted to investigate recent studies in different fields ranging from management to social science and information technology, to determine the ranges of response rates and the effect of factors to gain a higher number of responses. In business and management, the average response rate was 51.18%, while in information technology, it was 41.22%. Performance assessment studies had an average response rate of 50.83%, and in the management information system field, the overall average was 32.26%. Finally, in computer science, the average response rate was 56.55%.

DOI: 10.4018/979-8-3693-1135-6.ch009

This chapter published as an Open Access Chapter distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) which permits unrestricted use, distribution, and production in any medium, provided the author of the original work and original publication source are properly credited.
# INTRODUCTION

There are different data collection methods, and questionnaires and interviews are among the most frequent ones used to acquire information for research studies. The willingness of individuals to respond to the data collection tools is a vital factor that should be considered when conducting research and publishing the findings. This willingness can be described as a factor known as the response rate (Baruch & Holtom, 2008). There are different ways to define the response rate based on the denominator's choice. The response rate can be described as “the proportion of eligible individuals with whom the surveys are completed” (Holbrook, Krosnick, & Pfent, 2007). In other words, the response rate is calculated by dividing the number of people who completed the survey (questionnaire or interviews) by the total number of eligible people who were asked to participate in the survey. Alternatively, it can be calculated as the total number of participants who answered, divided by the total number of eligible individuals. The differences in data collection types make the description of the response rates also different (Morton, Bandara, Robinson, & Carr, 2012).

The response rate can be considered a function of both contacting the respondents and obtaining their cooperation using various strategies. Contact and cooperation rates are separate aspects of the interaction with participants, and surveys can be used to calculate them to assist researchers in increasing the response rate. For instance, researchers can increase the contact rate by targeting the contacts and increasing the number of call attempts. The contact rate is “the proportion of eligible individuals who were reached by the researcher.” On the other hand, the cooperation rate is “the pro-portion of all successfully contacted individuals who answered the questionnaire or agreed to participate in the interview.”

Indeed, the response rate can also be described based on the refusal rate, which is “the proportion of eligible individuals who refuse to participate in the study” (Holbrook et al., 2007). The main reasons for not responding can generally be categorized as follows:

* Issues that lead to the delivery failure of the questionnaires or difficulty in locating eligible participants;

* Reluctance of individuals to participate and answer the questions (Baruch, 1999; Morton et al., 2012).

The second issue is more challenging than the first one and can be addressed in many cases using different ways (Baruch & Holtom, 2008).
The response rate can be considered a quick proxy for survey quality, and there-
fore, it is essential to recognize the factors that can affect the response rate. Many
factors can influence the response rate, such as the length of the questionnaire, the
ordering and wording of questions, and the inclusion of leading questions (Cum-
mings, Savitz, & Konrad, 2001). Additionally, the method of data collection and
survey administration (online, face-to-face, etc.) can also affect the response rate.
For example, web-based methods are widely used, but response rates are often re-
ported to be lower than traditional methods in many cases. Finally, using reminders
can positively affect the response rate (Blumenberg et al., 2019). While some of
the main factors are listed here, it is important to discuss in more detail the strate-
gies that can be used to increase the response rate, which will be described in the
following sections. The primary aim of this chapter is to elucidate the various data
collection methods commonly employed in research endeavors. By delineating the
intricacies of each method, readers will gain a comprehensive understanding of their
applicability, advantages, and limitations in different research contexts.

In this chapter on enhancing response rates in research surveys, we embark on
a journey to understand the pivotal role of response rates in ensuring the reliability
and validity of survey research. Beginning with an exploration of “The Importance
of Response Rate,” we delve into the fundamental significance of response rates in
survey methodology, elucidating their impact on data quality, representativeness, and
research outcomes. Moving forward, our “Literature Review” synthesizes existing
scholarship on factors influencing response rates, providing insights into trends,
challenges, and emerging strategies in response rate optimization. Subsequently,
we offer practical guidance in “Increasing Response Rate,” where we discuss ac-
tionable strategies and best practices for improving response rates through effective
survey design, respondent engagement techniques, and follow-up protocols. In our
“Discussion,” we critically analyze the implications of response rates on research
methodology and interpretation, fostering a nuanced understanding of their broader
significance. Finally, in the “Conclusion,” we summarize key insights and recom-
mendations, emphasizing the ongoing importance of dialogue and collaboration in
advancing best practices in survey methodology. Through this explicit roadmap,
readers are guided through the interconnected sections of the chapter, facilitating
better coherence and flow in their exploration of response rate optimization in
survey research.
## THE IMPORTANCE OF RESPONSE RATE

The response rate level is a crucial factor that helps researchers assess the validity, reliability, and dependability of their study's findings. While some tactics can help researchers increase this factor (as discussed before), reaching valid and reliable findings is still a critical concern. In surveys, a non-random sample group is selected, which can differ from the population in terms of the variables of interest. These differences can lead to the distortion of true effects, and a high rate of non-response can cause more probable statistical biases (Baruch & Holtom, 2008). For a long time, researchers in survey studies have assumed that a high response rate is the best way to get unbiased results. However, many researchers have started questioning that assumption (Fosnacht et al., 2013).

Non-response at any level can also lead to bias due to non-response (Rindfuss et al., 2015). On the other hand, some researchers note a factor named response representativeness as more significant compared to the response rate in surveys, and therefore, they consider the impact of the response rate when this factor bears on the representativeness term (Baruch & Holtom, 2008).

To become more familiar with the effect and importance of this factor, the following discussion on response rates is based on the typology for validity described by Shadish et al. (Shadish et al., 2002). The authors defined validity as related to “the approximating truth of an inference,” which is often mistakenly used to refer to the quality of design or method. They also categorized validity into four different types, as shown in Figure 1.

<figure><img src="image_3.png" /><figcaption>Figure 1. Validity Types (Shadish et al., 2002)</figcaption></figure>

However, to account for the non-response effect, researchers should take into consideration to what extent these categories are related to the non-response error.
First, because non-response errors can result from pre-study attrition, they can be related to selection biases that also pertain to participant recruitment before the research. Both of these subjects share a primary concern about sample bias. Considering the taxonomy of Shadish et al. (Shadish et al., 2002), external validity is related to a specific concern, which is “how non-response biases a sample's representation of the target population because a finding drawn from the group of people studied might not hold if other kinds of people had been studied.” In other words, external validity “examines whether or not an observed causal relationship should be generalized to and across different measures, persons, settings, and times.” This type refers to generalizing the following:

* A population that is well specified which involves the generalizing of the result to a larger interest population.

* Across sub-populations which is related to conceptual replicability (to what extent the relationship between cause-effect in a particular study can be replicated in various time intervals, settings, and different subjects).

Therefore, a high response rate does not guarantee high external validity; however, it is not clear to researchers whether a low response rate can be seen in cases that report high external validity. For example, a poor response rate can cause difficulties in descriptive studies as the findings cannot be generalized with confidence and adequacy because the user cannot be certain of their capacity to generalize the results (Sivo et al., 2006).

Secondly, in addition to sample bias, a non-response error can also cause two factors that lead to a threat to statistical conclusion validity. These factors are defined as 1) inaccurate effect size estimation and 2) low power. Low power is related to relationship detection and the main reason behind low power is the inadequate sample size.

Therefore, a low response rate can jeopardize both external and statistical conclusion validities. However, it does not mean that the response rate does not affect construct or internal validities. Only the effect on former validity types is most direct (Sivo et al., 2006). To sum up, a low response rate can cause the discussed threats to increase the importance of considering different tactics and strategies by the researchers to address this issue. These strategies are defined below.
## INCREASING RESPONSE RATE

The main reasons for gaining a low response rate are discussed in the first section. In addition, in the 21st century, the complexity of individuals' lives and changes related to the definition of privacy, as well as laws and ethics on this subject, lead to reducing the cited participants in survey research. Another aspect of new laws that do not allow researchers to contact participants can cause selection bias. Therefore, manifold strategies based on the types of research, population, and rational additional costs should be considered to increase the response rate (Holbrook et al., 2007; Morton et al., 2012). Following this, first, the factors that affect the response rate value are summarized, and then some of the most significant strategies are discussed based on the ways they can affect the response rate.

The response rate can differ based on various aspects, such as the field of study. As discussed before, several factors can affect the response rate, including the survey's content, length, administration mode, and use of follow-up (Taherdoost, 2022). Therefore, considering all of these factors and implementing different strategies to address related issues can be useful in increasing the response rate. For instance, in primary care surveys, addressing all of these factors can boost the response rate by 30% (Booker, Austin, & Balasubramanian, 2021). Booker et al. (2021) described three strategies that can be useful in increasing the response rate in this field of study. These strategies are discussed below:

* Engage the “hard-to-reach” population: To better engage with this group, it is recommended to ask for the help of community members in these clinics to identify the population and invite them to participate in the survey.

* Ensure suitable content and length for the survey: It is essential to consider various aspects, such as suitable time, language, and avoiding vagueness. The instruments should be previously validated and tested to ensure relevance, validity, and reliability. Language and cultural aspects should also be considered, and the length of the survey should be appropriate (for example, a recommended time is 15-20 minutes).

* Consider a suitable mode of administration: Each mode of administration has both advantages and disadvantages. In-person and telephone modes have some advantages, such as being more familiar and easier to provide incentives. Web-based modes, by using smart devices, facilitate the filling process and provide a more comfortable environment for the participants as they can complete the survey at their own pace and in any location. Using mixed administration modes is also recommended where possible and can have a positive impact on the response rate. For example, using mixed modes (in-person and web-based together) by local research teams in a study increased the
response rate by up to 70% (Booker et al., 2021). Some other points recommended to increase the response rate are:

* To specify the purpose of your study to the participants;

* To specify the sponsor/s of your study;

* To send notifications in advance before sending the real survey;

* To reduce the participants' needed effort to complete as much as possible;

* To motivate the participants by using awards such as sending result summaries money, charity donation, etc.

* To ensure their confidentiality and privacy;

* To fulfill the promise related to the incentives.

The tactics mentioned above are also discussed in the literature review section of this paper, where we provide the results of related studies.

## What Is an Acceptable Response Rate?

Several factors are considered by authors when discussing acceptable response rates. For instance, several authors have considered the usage of gathered data, the field of study, and the mode of administration, which are discussed here. Additionally, some researchers have highlighted the time factor's importance in the response rate. They reported that over the past several decades, the response rate for epidemiological studies decreased dramatically by about 20% or even more (Morton et al., 2012).

Generally, an adequate response rate differs based on the use of data. For example, if the collected data is about the evaluation of a teacher, and would be used just for improvement purposes, even one response can be enough. However, if the response would be used as a summative appraisal of a teacher's performance, then gaining a single response is completely inadequate.

Considering the mode of administration, a study by Nulty (2008) compared eight different works and showed that the response rate given in online surveys is lower than in paper-based surveys. Therefore, the acceptable values are not even close. They reported an average difference of 23% between the response rate of the two different modes. Although they also reported an exception that showed close values for both modes, and even paper-based had a lower response rate. However, the point was that the different results were related to a study in which a paper-based survey was not conducted face-to-face. To sum up, they identified the acceptable response rate of face-to-face surveys as higher than online ones (Taherdoost, 2019).

In addition, another important aspect of identifying an acceptable response rate is the subject of the study. Although some reviews by researchers show that a minimum response rate is not agreed upon in different fields, some authors provide a range for response rates based on the fields of study. For example, a couple of studies in
1973 and 1981 in social science showed that 50% can be an acceptable rate for postal surveys (Nulty, 2008). For management, this value is reported as 55.6% overall by Baruch (Baruch, 1999), and also reported between 50% to 80% in another study by Baruch and Holtom (Baruch & Holtom, 2008). For surveys related to the course experience, 70% is known as desirable by an Australian career council (Nulty, 2008). On the other hand, some studies recommended more general values for the response rate. For instance, Goudy (Goudy, 1976) considered 30% to 70% as an acceptable range, and Malhotra and Grover (Malhotra & Grover, 1998) reported values below 20% as a very undesirable response rate. Finally, Gordon et al. (Gordon et al., 2002) reported that a higher is better for response rate. They listed 60%, 70%, 80%, and 90% as marginal, reasonable, good, and excellent, respectively

The values discussed and suggested above are not based on any theoretical justification, as they are mostly provided by reviewing different surveys in the selected fields. However, it would be beneficial if researchers could use a systematic way, such as a theoretically justified formula, to calculate the response rate (Nulty, 2008). Using a theoretical formula for calculating the acceptable rate of response requires many considerations, and it is not simply a matter of applying a simple and fixed theory. For example, a formula is suggested by Dillman (Dillman, 2000) in the teaching field under specific conditions. Therefore, it is recommended to be used with all those factors in mind. They listed several considerations to use the suggested formula, including the characteristics listed below:

* Knowing the total number of the population who are being surveyed;

* Surveying all of the students in the population (this case is necessary just for this argument which is related to response rate);

* Knowing a probability about any of the participants adds a certain answer to a question;

* knowing or setting a desired or required accuracy level for the findings;

* Knowing or choosing a desired or required confidence level. This level is related to gaining the same findings using other samples but with the same size sample, and with the same total group of participants (Nulty, 2008).

To sum up, many studies claim that there is no consensus on the acceptable value/range or a general formula to calculate the rate simply. On the other hand, some studies show that a desirable response rate can differ depending on the time aspect. For this purpose, another aim of this study is to review the works provided at a specific time on this subject to investigate the achieved ranges of response rates in different fields and changes during a selected period. The next section discusses and reviews recent academic research articles that report response rates in different fields.
# LITERATURE REVIEW

In this section, several survey research studies are reviewed to obtain the response rates achieved in various fields in the last 22 years (2000-2022). For this purpose, some related studies in five different areas, including Business and Management, Information Technology, Performance Assessment, Management Information System, and Computer Science are selected.

The studies were selected by identifying various journals that published survey studies in the selected fields on the “ScienceDirect” and “Web of Science” database. The keyword “response rate” was used, and the studies were first chosen based on screening their abstracts. Additionally, as there are different ranges of journals publishing works on the performance assessment area, the keywords “performance assessment” and “response rate” were first searched together, and then the selected journals were analyzed to reach the survey studies. This was considered for other fields as well. Initially, more than 90 papers were chosen based on their abstract fit. Subsequently, the papers were further filtered, and the unrelated ones, as well as those without a reported response rate, were eliminated.

The final results are summarized in the following tables (Table 1-5) based on the fields of study (considered in ScienceDirect and Web of Science filters). The tables represent the research objects and the response rates obtained from them.
<table><caption>Table 1. Studies in the business and management field</caption><thead><tr><th>Journal</th><th>Details</th><th>Respondents</th><th>Response Rate</th><th>Reference</th></tr></thead><tbody><tr><td rowspan="4">Industrial Marketing Management</td><td>Effect of different factors on response rate in a survey about the evaluation of public relations.</td><td>A business population</td><td>Normal= 51%<br/>After different treatments= 41.7% to 62.5%.</td><td>(McCoy & Hargie, 2007)</td></tr><tr><td>Effect of different factors on response rate in a survey on purchasing involvement.</td><td>Purchasing professionals</td><td>Overall= 27.3%.<br/>Using personalized mailings= 29.2% vs. Using non-personalized= 25.3%<br/>Follow-up after 4 weeks= 33.2%<br/>Follow-up after 2 weeks= 30.7% for follow-up letter at 2 weeks<br/>No follow-up= 18.0%<br/>$1.00 prepaid response incentive= 31.8%<br/>lottery incentive= 25.3%<br/>No incentive= 24.7%</td><td>(Larson & Chow, 2003)</td></tr><tr><td>A survey to investigate the effect of contextual factors on the links between buyers and suppliers.</td><td>Korean buyers and suppliers</td><td>80%</td><td>(Karande et al., 2008)</td></tr><tr><td>An online survey was conducted to study the effect of evaluating the performance of suppliers to face mitigating risks.</td><td>Buyers and suppliers</td><td>5.2%</td><td>(Hawkins et al., 2020)</td></tr><tr><td rowspan="2">The International Journal of Management Education</td><td>The entrepreneurial tendencies of students.</td><td>Malaysian polytechnics</td><td>84.3%</td><td>(Ahmad et al., 2014)</td></tr><tr><td>The intention to entrepreneurship.</td><td>Students in their final year at Bayero University Kano</td><td>71.4%</td><td>(Adekiya & Ibrahim, 2016)</td></tr><tr><td rowspan="3">International Journal of Hospitality Management</td><td>Using different aspects to increase response rate in a survey on the effect of wages in hospitality industries.</td><td>Employers of small hospitality organizations.</td><td>Increased to 19%</td><td>(Keegan & Lucas, 2005)</td></tr><tr><td>The impact of the level of organization and gender on job satisfaction and role stress.</td><td>Employees of hotels in Korea</td><td>64%</td><td>(Kim et al., 2009)</td></tr><tr><td>Investigating the psychological issues that non-supervisory employees face.</td><td>Employees in the full-service restaurant</td><td>46.2%</td><td>(Hancer & George, 2003)</td></tr></tbody></table>

continued on following page
<table><caption>Table 1. Continued</caption><thead><tr><td>Journal</td><td>Details</td><td>Respondents</td><td>Response Rate</td><td>Reference</td></tr></thead><tbody><tr><td>Journal of Management</td><td>Identifying the investment risk behavior determinants in specific retirement plans.</td><td>Employees of college and university</td><td>Approximately 20%</td><td>(Dulebohn, 2002)</td></tr><tr><td rowspan="2">Australasian Marketing Journal (AMJ)</td><td>Investigating the effect of qualifying respondents on response rate in the following Surveys: Case 1: Customer reference marketing. Case 2: the experience of alcohol drinking and Red Bull.</td><td>Case 1: Managers Case 2: A selected group of heterosexual men</td><td>Case 1: 39.6, 40.5, 45.9, 100 Case 2: 58.6, 60.4, 59.7, 52.5</td><td>(Basa-Martinez et al., 2018)</td></tr><tr><td>Quality of Journals in marketing fields in Australia and New Zealand.</td><td>Senior academics</td><td>45%</td><td>(Mort et al., 2004)</td></tr></tbody></table>

<table><caption>Table 2. Studies in the information technology field</caption><thead><tr><td>Journal</td><td>Details</td><td>Respondents</td><td>Response Rate</td><td>Reference</td></tr></thead><tbody><tr><td rowspan="2">Knowledge-Based Systems</td><td>Two-phase study Investigating the aspects impacting the diffusion and adoption of knowledge management systems in Western Australia.</td><td>Senior-level managers</td><td>Phase 1: 20% Phase 2: 24.8%</td><td>(Xu & Quaddus, 2012)</td></tr><tr><td>The impact of knowledge-management-fit on the performance of the business.</td><td>Top managers of different companies</td><td>15.73%</td><td>(Chen & Huang, 2012)</td></tr></tbody></table>

continued on following page
<table><caption>Table 2. Continued</caption><thead><tr><th>Journal</th><th>Details</th><th>Respondents</th><th>Response Rate</th><th>Reference</th></tr></thead><tbody><tr><td rowspan="3">Computers in Human Behavior</td><td>Survey related to online communities' participants and the different effects on response rate.</td><td>The members of an online health community (joint within 3 years)</td><td>Total aggregate response rate: 8.4%<br>Lowest = 5.1%<br>Highest = 22%</td><td>(Petrovčič et al., 2016)</td></tr><tr><td>Investigating the risk behavior of youth using different survey modes.</td><td>Students</td><td>Overall = 74.7%. In-class paper-and-pencil = 91.2%. In-class Web without skip = 90.1%. In-class Web with skip = 91.4%. "on your own" Web without skip = 28.0%.</td><td>(Denniston et al., 2010)</td></tr><tr><td>Survey on education quality considering different aspects of response rate such as time of the day and reminder.</td><td>School principals at public and private schools in Chile</td><td>Average without reminder = 12.79%<br>Average with reminder = 22.25%</td><td>(Madariaga et al., 2017)</td></tr></tbody></table>

continued on following page
<table><caption>Table 2. Continued</caption><thead><tr><td>Journal</td><td>Details</td><td>Respondents</td><td>Response Rate</td><td>Reference</td></tr></thead><tbody><tr><td rowspan="5">Computers &amp; Education</td><td>Impact of receiving embedded survey together with email on response rate in a demographic survey study.</td><td>Learners of Massive Open Online Courses (MOOCs)</td><td>Increased from 6.9% to 61.5%</td><td>(van de Oudeweetering &amp; Agirdag, 2018)</td></tr><tr><td>Using the interactive whiteboard (IWB) technology in schools.</td><td>Turkish middle school students</td><td>78.5%</td><td>(Türel, 2011)</td></tr><tr><td>The perceptions toward computer-based formative assessments (CBFA) usage and instructional technologies.</td><td>Academic teachers</td><td>63%</td><td>(Sullivan et al., 2021)</td></tr><tr><td>Different issues are faced by special education students and parents in cyber schools.</td><td>Students and parents</td><td>parents= 48.7%<br>Students= 53.7%</td><td>(Beck et al., 2014)</td></tr><tr><td>Determining perceptions, access, and usage of computer-based technologies and games for teaching.</td><td>American Primary and Secondary educators (who were winners of the Milken Educator Award from 1996 to 2009) were divided into different categories</td><td>Ranging from 15 to 25% based on categories</td><td>(Proctor &amp; Marks, 2013)</td></tr><tr><td>Computers &amp; Security</td><td>Investigating how abusive supervision moderates the compliance of information security policy.</td><td>Hotel Employees in China</td><td>Round 1= 90.20%<br>Round 2= 89.58%, but the total response rate was 80.80%.</td><td>(Xu et al., 2021)</td></tr></tbody></table>
<table><caption>Table 3. Studies on the performance assessment field</caption><thead><tr><td>Journal</td><td>Details</td><td>Respondents</td><td>Response Rate</td><td>Reference</td></tr></thead><tbody><tr><td rowspan="5">Performance Enhancement &amp; Health</td><td>Investigating the doping incentives and deterrents.</td><td>Elite athletes</td><td>43%</td><td>(Overbye et al., 2013)</td></tr><tr><td>Investigating the knowledge and needs of pharmacists about anti-doping.</td><td>Finnish pharmacists</td><td>97%</td><td>(Lemettilä et al., 2021)</td></tr><tr><td>The effect of using dietary supplements and medicines.</td><td>World-level bridge players</td><td>Approximately 33%</td><td>(Federico et al., 2014)</td></tr><tr><td>Collecting socio-demographic information.</td><td>Importers of doping product</td><td>30%</td><td>(Weber et al., 2016)</td></tr><tr><td>The knowledge and perceptions of athletes about anti-doping.</td><td>Jamaican athletes aged 12–19 years</td><td>89%</td><td>(Turfus et al., 2019)</td></tr><tr><td>The Journal of Academic Librarianship</td><td>Service improvement and performance assessment.</td><td>Administrators and library directors in the different Irish universities</td><td>Maximum= 20%<br>Minimum= 5%</td><td>(McCaffrey, 2013)</td></tr><tr><td>Accident Analysis &amp; Prevention</td><td>Safety performance assessment during travel and mobility surveys in Europe.</td><td>Different individuals such as owners of vehicles and households.</td><td>Ranging from 50 to 73%</td><td>(Papadimitriou et al., 2013)</td></tr><tr><td>Information &amp; Management</td><td>The assessment of the performance of E-commerce applications.</td><td>Chinese retailers</td><td>Approximately 30%</td><td>(Huang et al., 2009)</td></tr></tbody></table>

<table><caption>Table 4. Studies on the management information system field</caption><thead><tr><td>Journal</td><td>Details</td><td>Respondents</td><td>Response Rate</td><td>Reference</td></tr></thead><tbody><tr><td rowspan="3">Information &amp; Management</td><td>Investigating the impact of the integration of supply chain information and operational performance.</td><td>Individuals of some manufacturing companies</td><td>36%</td><td>(Vafaei-Zadeh et al., 2020)</td></tr><tr><td>Website quality in Northeastern University and Erasmus University.</td><td>Students</td><td>Approximately 5%</td><td>(van Iwaarden et al., 2004)</td></tr><tr><td>The association between distraction and extreme use of social media.</td><td>Chinese college students</td><td>97.7%</td><td>(Xie et al., 2021)</td></tr></tbody></table>

continued on following page
<table><caption>Table 4. Continued</caption><thead><tr><td>Journal</td><td>Details</td><td>Respondents</td><td>Response Rate</td><td>Reference</td></tr></thead><tbody><tr><td rowspan="4">International Journal of Information Management</td><td>The extent of the use of environmental scanning in collecting information.</td><td>Several CEOs of South African enterprises</td><td>32.5%</td><td>(Du Toit, 2016)</td></tr><tr><td>Investigating the aspects that impact the adoption of electronic health records.</td><td>Physicians</td><td>39.8%</td><td>(Gagnon et al., 2016)</td></tr><tr><td>Studying the effectiveness of using agricultural information service</td><td>Farmers in the Maseru district in Lesotho</td><td>70%</td><td>(Mokotjo & Kalusopa, 2010)</td></tr><tr><td>The nature of Information Management pedagogic practice</td><td>The academics in the fields of Information Systems and Management in the UK</td><td>30%</td><td>(Grant et al., 2010)</td></tr><tr><td rowspan="3">Information Systems</td><td>The grumbling behavior of end-user</td><td>Workers of five different manufacturing industries in Bangladesh</td><td>44.2%</td><td>(Mahmud et al., 2017)</td></tr><tr><td>Investigating the difficulties of moving the current legacy software systems to the cloud platform</td><td>Domain experts in manifold industries</td><td>20%</td><td>(Gholami et al., 2017)</td></tr><tr><td>The relationships and connections in Twitter (NavigTweet)</td><td>End users</td><td>Maximum 12 to 15%</td><td>(Gholami et al., 2017)</td></tr></tbody></table>

continued on following page
<table><caption>Table 4. Continued</caption><thead><tr><th>Journal</th><th>Details</th><th>Respondents</th><th>Response Rate</th><th>Reference</th></tr></thead><tbody><tr><td rowspan="4">International Journal of Accounting Information Systems</td><td>The impact of top management support, the mechanisms of IT governance, and ambidexterity in IT on companies</td><td>IT-related directors</td><td>65.2%</td><td>(Zhen et al., 2021)</td></tr><tr><td>Investigating suitable information technologies and self-perceptions of knowledge about these technologies</td><td>Professors and practitioners</td><td>Professors= 31.2%<br>Practitioners = 24.7%</td><td>(Greenstein & McKee, 2004)</td></tr><tr><td>The effectiveness of the governance of information technology</td><td>Actual and potential adopters of the cloud computing service</td><td>Overall, almost= 5.5%<br>1.05% for actual 4.44% for potential</td><td>(Prasad et al., 2010)</td></tr><tr><td>The effectiveness of the governance of cloud computing</td><td>Senior executives of different organizations selected from the ORBIS database</td><td>13.16%</td><td>(Prasad et al., 2014)</td></tr><tr><td rowspan="2">The Journal of Strategic Information Systems</td><td>The links between software developer and tester</td><td>Professional developers and testers, and organizational contacts from the subunits of development and testing</td><td>10.03%</td><td>(Dhaliwal et al., 2011)</td></tr><tr><td>The alignment between IT and business strategies</td><td>Senior IT executives</td><td>10%</td><td>(Queiroz et al., 2020)</td></tr></tbody></table>

<table><caption>Table 5. Studies on the computer science field</caption><thead><tr><th>Journal</th><th>Details</th><th>Respondents</th><th>Response Rate</th><th>Reference</th></tr></thead><tbody><tr><td rowspan="2">International Journal of Medical Informatics</td><td>The early clinical experience of medical students with an interest in technology</td><td>Students</td><td>84%</td><td>(Avidan et al., 2021)</td></tr><tr><td>A feasibility and value study of patient-directed interactive and intelligent computer medical history-gathering platforms in the emergency room</td><td>Patients</td><td>97%</td><td>(Benaroia et al., 2007)</td></tr></tbody></table>

continued on following page
<table><caption>Table 5. Continued</caption><thead><tr><td>Journal</td><td>Details</td><td>Respondents</td><td>Response Rate</td><td>Reference</td></tr></thead><tbody><tr><td rowspan="2">Computer Methods and Programs in Biomedicine</td><td>A study on how health workers in places with few resources could use delay-tolerant networks</td><td>Attendants at the International Conference on Global Health with health work experience in a poor connectivity context</td><td>74%</td><td>(Syed-Abdul et al., 2012)</td></tr><tr><td>Medical calculator use rates and obstacles</td><td>Physicians</td><td>13%</td><td>(Green et al., 2019)</td></tr><tr><td>Journal of the American Medical Informatics Association</td><td>Use and appreciated advantages of mobile computer-based clinical references</td><td>Physicians</td><td>42%</td><td>(Rothschild et al., 2006)</td></tr><tr><td>Journal of computer information systems</td><td>An empirical assessment of the variables influencing the amount of e-commerce adoption</td><td>Respondents in Malaysian organizations</td><td>27.25%</td><td>(Chong et al., 2009)</td></tr><tr><td>Information and Computer Security</td><td>Instrument and Internet of Things privacy problems for mobile users</td><td>Students</td><td>15.3%</td><td>(Foltz & Foltz, 2020)</td></tr><tr><td>Journal of Information Science</td><td>Is it possible for organizational knowledge capacities to influence knowledge sharing behavior?</td><td>Different industries</td><td>62.4%</td><td>(Yang & Chen, 2007)</td></tr><tr><td>Information and Software Technology</td><td>Findings from a regional industrial assessment on the current and potential of variability management techniques in software-intensive small and medium-sized enterprises</td><td>Software developing organizations in a region of Sweden</td><td>13%</td><td>(Thörn, 2010)</td></tr><tr><td>Program-Electronic Library and Information Systems</td><td>A review of United Kingdom universities' topic librarians and online learning environments</td><td>Subject librarians at seven United Kingdom universities</td><td>43%</td><td>(Corrall & Keates, 2011)</td></tr><tr><td>Online Information Review</td><td>Knowing the causes of helpful behavior among users of instant messaging</td><td>Employees from seven large companies</td><td>72.8%</td><td>(Lin & Chiu, 2011)</td></tr><tr><td>IEEE Access</td><td>From the viewpoint of healthcare consumers, factors influencing the effective deployment of e-health cloud based health systems</td><td>Healthcare consumers</td><td>76.9%</td><td>(Idoga et al., 2018)</td></tr></tbody></table>

continued on following page
<table><caption>Table 5. Continued</caption><thead><tr><td>Journal</td><td>Details</td><td>Respondents</td><td>Response Rate</td><td>Reference</td></tr></thead><tbody><tr><td rowspan="2">Internet Research</td><td>An investigation on how people aid one another in online communities</td><td>Participants in online communities in Taiwan</td><td>84%</td><td>(Chu, 2009)</td></tr><tr><td>Community-based innovation: its causes and effects on the success of innovations</td><td>Internet users</td><td>87%</td><td>(Chu & Chan, 2009)</td></tr></tbody></table>

## CROSS-CULTURAL STUDIES ON SURVEY RESPONSE RATES

Incorporating a dedicated section on cross-cultural studies in survey response rates is crucial for understanding the nuanced impact of cultural influences on respondents' behaviors and perceptions across diverse contexts. Cultural norms, attitudes towards research participation, and communication styles vary significantly across cultures, shaping respondents' willingness to engage with surveys. Researchers must navigate cultural nuances in question wording, tone, and phrasing to ensure clarity and relevance across diverse cultural settings. Building trust and rapport with respondents through transparent communication and ethical conduct is paramount, particularly in cultures where trust in institutions and researchers may vary widely. Moreover, addressing cultural perceptions of privacy and confidentiality is essential to alleviate concerns and foster trust among respondents in cross-cultural survey research.

To enhance the effectiveness and validity of cross-cultural survey research, researchers should demonstrate cultural sensitivity and adaptability in survey design, administration, and interpretation. Collaborating with local experts, translators, and community leaders can facilitate culturally appropriate approaches to survey research. Pilot testing and validation across diverse cultural groups enable researchers to identify and address potential cultural biases, language barriers, and comprehension issues in survey instruments. By acknowledging and adjusting for cultural differences in response patterns, researchers can enhance the relevance and accuracy of survey data across cultural contexts. Additionally, researchers engaging in cross-cultural survey research should undergo cross-cultural training and develop cultural competence to navigate diverse cultural landscapes sensitively and ethically. Understanding cultural values, communication styles, and social dynamics equips researchers with the knowledge and skills to foster meaningful engagement and collaboration across cultural boundaries, ultimately enhancing the validity, reliability, and cross-cultural applicability of survey findings.
DISCUSSION

The reviewed studies offer methodological insights into the intricate dynamics of survey response rates across various fields. They illuminate how different survey designs, administration methods, and sample characteristics influence response rates, thereby contributing to a more comprehensive understanding of the factors affecting survey response.

In the Business and Management field, studies such as those by McCoy & Hargie (2007) and Larson & Chow (2003) underscore the impact of personalized mailings, follow-up strategies, and incentives on response rates among business populations and purchasing professionals. Additionally, research by Karande et al. (2008) highlights high response rates among Korean buyers and suppliers, contrasting with Hawkins et al.'s (2020) findings of a low online survey response rate among buyers and suppliers.

In the Information Technology field, studies like those by Jun Xu & Quaddus (2012) and Chen & Huang (2012) explore the complexities of knowledge management system adoption and its relationship with response rates among senior-level managers. Petrovčič et al. (2016) shed light on response rates among members of online health communities, revealing variations based on survey modes and participant characteristics.

In the Performance Assessment field, studies such as those by Overbye et al. (2013) and Lemettilä et al., (2021) delve into response rates among elite athletes and pharmacists, respectively, unveiling factors influencing survey participation in doping and anti-doping studies.

Furthermore, in the Management Information System field, studies by Vafaei-Zadeh et al. (2020) and van Iwaarden et al., (2004) elucidate the impact of survey design and respondent characteristics on response rates among individuals of manufacturing companies and students, respectively.

In the Computer Science field, research by Avidan et al., (2021) and Syed-Abdul et al., (2012) provides insights into survey response rates among medical students and health workers, shedding light on the feasibility and value of survey methodologies in healthcare settings.

Collectively, these studies underscore the nuanced interplay between survey design, administration methods, and sample characteristics in shaping response rates across diverse research domains, enriching our understanding of the multifaceted factors influencing survey response behavior.

Several survey studies reporting response rates were reviewed in the previous section. These studies were conducted in different research areas with respondents ranging from professionals to normal individuals. The results showed different ranges of response rates in the selected studies, which were inconsistent with the contents
described in the first sections of this paper. In the field of business and manage-
ment, the reported range of response rates in the selected papers varied greatly. The
minimum response rate reported among the selected papers was 5.2, while higher
values of over 80% (even 100%) and average of 51.18% were also obtained. The
respondents in these studies were different individuals, such as students, buyers,
suppliers, and professionals. However, there was no clear link identified between the
response rate and the type of respondents in the papers in this field of study.In the
Information Technology area, the studies showed various ranges for response rates
as well (range of 6.9% to 90.2% and average = 41.22%), and the studies targeted
multiple groups such as students and learners, school principals, and managers. The
respondents in the performance assessment studies were mostly athletes, although
administrators, ordinary people, retailers, and pharmacists were also the target
populations of those papers with widely varying response rates and overall average
was 50.83% with range of 5% to 97%. The papers on the Management Information
System also obtained results targeting various groups of individuals (professors,
students, farmers, etc.) and overall average was 32.26% with range of 5% to 97.7%.
In the field of Computer Science, response rates were reported from 13% to 97%
with average of 56.55. The respondents were also different individuals ranging from
students, patients, physicians, residents, respondents in organizations, employees,
consumers, and users, etc. In addition to the results obtained in the selected fields,
it can also be observed that the response rate fell mostly within a specific range for
certain groups of respondents across all fields. This can be seen as follows

The response rates of senior managers, senior executives, professional staff, and
employers were reported at low levels.

Studies targeting students as respondents reported better response rates (except in
two cases with 5% and 15% response rates obtained from just complete responses).
The response rates in those papers were from 53.7% to 97.7%.

Overall, no correlation was observed between the types of journals and the
ranges of response rates in the published works, even when reviewing multiple
papers within a single journal. Another objective of this study is to investigate the
impact of changes in response rates within a specific field of study by taking the
time factor into account. Upon examining the results of each table (Tables 1 to 5),
no correlations were observed between the year of the studies and the response rate.
Therefore, it can be concluded that since various factors can influence the response
rate, it is not possible to draw a general conclusion about the time factor here. This
factor should be analyzed in more similar study conditions, such as the characteris-
tics of the respondents, administration mode, etc. Furthermore, based on the results
derived from the tables and by taking into account the papers that investigated the
effects of various criteria on response rate, the following conclusions can be drawn:
* The response rate can be maximized in a specific survey considering different strategies.

* The mode of administration can affect the response rate.

* Sending reminders increased the response rate in the related papers.

* Pre-notification can positively affect the response rate.

* Incentives can be effective in many cases to encourage the respondents.

* The time of the day that the surveys are sent can affect the number of responses.

* Qualifying is a very significant factor in the response rate, and a high degree of qualifying increased the results of the response rate to very high response rates as reported in the discussed papers.

ADVANCING SURVEY METHODOLOGY

In recent years, survey methodology has witnessed a transformative evolution driven by technological advancements and innovative approaches. Artificial Intel- ligence (AI) and machine learning algorithms have revolutionized survey design, administration, and data analysis by enabling automated text analysis, sentiment anal- ysis, and predictive modeling. AI-powered chatbots and virtual assistants streamline survey administration processes, offering personalized interactions and real-time support to respondents. The widespread adoption of online survey platforms and mobile applications has democratized access to survey participation, facilitating data collection across diverse populations and geographical boundaries. These platforms provide intuitive design interfaces, multimedia-rich features, and context-aware sur- vey delivery, enhancing respondent engagement and data quality. Furthermore, the integration of survey data with big data sources and the application of data fusion techniques enable researchers to uncover hidden patterns and associations, enriching survey research insights and driving methodological innovation. By embracing recent advances in survey methodology, including AI, machine learning, and technological innovations, researchers can leverage cutting-edge tools and techniques to overcome traditional methodological challenges, expand the scope of survey research inquiries, and advance the field of survey methodology in the digital age.

CONCLUSION

The literature review highlights the significance of response rates in research studies and investigates the variables affecting reaction time and acceptable standards. Initially, the review elucidates the critical components of the response rate, empha-
sizing its importance as a metric for assessing the validity and reliability of research findings. The scoping literature study conducted across various disciplines, including management, social science, and information technology, reveals diverse ranges of response rates and the impact of variables on increasing respondent participation.

Key findings indicate variations in response rates across disciplines, with the average rates ranging from 41.22% in information technology to 56.55% in computer science. While response rates serve as vital indicators of research validity, the review underscores that they are influenced by multifaceted factors such as the field of study and respondent characteristics. Moreover, the review suggests that response rates alone cannot determine research quality but are indicative of the degree of engagement and representativeness of the sample.

The literature review identifies several avenues for future research, including exploring the underlying reasons for disparities in response rates across disciplines and investigating innovative strategies to enhance respondent participation. Addressing gaps in the current understanding of survey response rates requires interdisciplinary collaboration and methodological advancements. Future research endeavors should prioritize the development of standardized protocols for measuring and reporting response rates, as well as the exploration of novel techniques, such as leveraging technology and data analytics, to optimize survey design and administration. Ultimately, advancing our understanding of response rates will contribute to the enhancement of research quality and the validity of research findings across diverse fields and contexts.
REFERENCES

Adekiya, A. A., & Ibrahim, F. (2016). Entrepreneurship intention among students. The antecedent role of culture and entrepreneurship training and development. *International Journal of Management Education, 14*(2), 116–132. DOI: 10.1016/j.ijme.2016.03.001

Ahmad, S. Z., Ismail, M. Z., & Buchanan, F. R. (2014). Examining the entrepreneur-ship curriculum in Malaysian polytechnics. *International Journal of Management Education, 12*(3), 397–406. DOI: 10.1016/j.ijme.2014.06.004

Avidan, A., Weissman, C., & Zisk-Rony, R. Y. (2021). Interest in technology among medical students early in their clinical experience. *International Journal of Medical Informatics, 153*, 104512. DOI: 10.1016/j.ijmedinf.2021.104512 PMID: 34107384

Baruch, Y. (1999). Response rate in academic studies-A comparative analysis. *Human Relations, 52*(4), 421–438. DOI: 10.1177/001872679905200401

Baruch, Y., & Holtom, B. C. (2008). Survey response rate levels and trends in organizational research. *Human Relations, 61*(8), 1139–1160. DOI: 10.1177/0018726708094863

Basa-Martinez, D. D. F., Cabrera, J. Y., Dionaldo, L. G., Orillo, J. G. R., Ramos, P. J. M., & Ocampo, L. A. (2018). An exploration of a respondent pre-qualifying framework to increase response rates in social media initiated online surveys. *Australian Marketing Journal, 26*(3), 239–261. DOI: 10.1016/j.ausmj.2018.04.001

Beck, D., Egalite, A., & Maranto, R. (2014). Why they choose and how it goes: Comparing special education and general education cyber student perceptions. *Computers & Education, 76*, 70–79. DOI: 10.1016/j.compedu.2014.03.011

Benaroia, M., Elinson, R., & Zarnke, K. (2007). Patient-directed intelligent and interactive computer medical history-gathering systems: A utility and feasibility study in the emergency department. *International Journal of Medical Informatics, 76*(4), 283–288. DOI: 10.1016/j.ijmedinf.2006.01.006 PMID: 16473548

Chen, Y.-Y., & Huang, H.-L. (2012). Knowledge management fit and its implications for business performance: A profile deviation analysis. *Knowledge-Based Systems, 27*, 262–270. DOI: 10.1016/j.knosys.2011.11.012

Chong, A. Y.-L., Lin, B., Ooi, K.-B., & Raman, M. (2009). Factors affecting the adoption level of c-commerce: An empirical study. *Journal of Computer Information Systems, 50*(2), 13–22. DOI: 10.1080/08874417.2009.11645380
Chu, K. M. (2009). A study of members' helping behaviors in online community. *Internet Research*, 19(3), 279-292. DOI: 10.1108/10662240910965351

Chu, K. M., & Chan, H. C. (2009). Community based innovation: Its antecedents and its impact on innovation success. *Internet Research*. Advance online publication.
DOI: 10.1108/10662240910998887

Corrall, S., & Keates, J. (2011). The subject librarian and the virtual learn-
ing environment: A study of UK universities. *Program*, 45(1), 29-49. DOI:
10.1108/00330331111107385

Denniston, M. M., Brener, N. D., Kann, L., Eaton, D. K., McManus, T., Kyle, T. M., Roberts, A. M., Flint, K. H., & Ross, J. G. (2010). Comparison of paper-and-pencil versus Web administration of the Youth Risk Behavior Survey (YRBS): Participation, data quality, and perceived privacy and anonymity. *Computers in Human Behavior*, 26(5), 1054–1060. DOI: 10.1016/j.chb.2010.03.006

Dhaliwal, J., Onita, C. G., Poston, R., & Zhang, X. P. (2011). Alignment within the software development unit: Assessing structural and relational dimensions between developers and testers. *The Journal of Strategic Information Systems*, 20(4), 323–342. DOI: 10.1016/j.jsis.2011.03.001

Dillman, D. A. (2000). *Mail and internet surveys: the tailored design method*. Wiley.

Du Toit, A. S. (2016). Using environmental scanning to collect strategic information: A South African survey. *International Journal of Information Management*, 36(1), 16–24. DOI: 10.1016/j.ijinfomgt.2015.08.005

Dulebohn, J. H. (2002). An investigation of the determinants of investment risk behavior in employer-sponsored retirement plans. *Journal of Management*, 28(1), 3–26. DOI: 10.1177/014920630202800102

Federico, B., Gabriele, P. W., & Capelli, G. (2014). Use of medicines and dietary supplements among Bridge players in two world-class tournaments. *Performance Enhancement & Health*, 3(3-4), 130–134. DOI: 10.1016/j.peh.2015.11.001

Foltz, C. B., & Foltz, L. (2020). Mobile users' information privacy concerns in-
strument and IoT. *Information and Computer Security*, 28(3), 359–371. Advance
online publication. DOI: 10.1108/ICS-07-2019-0090

Fosnacht, K., Howe, E., & Peck, L. (2013). How important are high Response Rates. *Annual Meeting of the Association for Institutional Research*. DOI: 10.1016/j.ijinfomgt.2015.12.002
Gholami, M. F., Daneshgar, F., Beydoun, G., & Rabhi, F. (2017). Challenges in migrating legacy software systems to the cloud—An empirical study. *Information Systems*, 67, 100–113. DOI: 10.1016/j.is.2017.03.008

Gordon, N., Davidoff, F., Tarnow, E., Reidenberg, M., & Endriss, K. (2002). A question of response rate. *Science Editor*, 25(1), 25.

Goudy, W. J. (1976). Nonresponse effects on relationships between variables. *Public Opinion Quarterly*, 40(3), 360–369. DOI: 10.1086/268312

Grant, K., Hackney, R., & Edgar, D. (2010). Informing UK Information Management pedagogic practice: The nature of contemporary higher education culture. *International Journal of Information Management*, 30(2), 152–161. DOI: 10.1016/j.ijinfomgt.2009.09.008

Green, T. A., Whitt, S., Belden, J. L., Erdelez, S., & Shyu, C.-R. (2019). Medical calculators: Prevalence, and barriers to use. *Computer Methods and Programs in Biomedicine*, 179, 105002. DOI: 10.1016/j.cmpb.2019.105002 PMID: 31443857

Greenstein, M., & McKee, T. E. (2004). Assurance practitioners’ and educators’ self-perceived IT knowledge level: An empirical assessment. *International Journal of Accounting Information Systems*, 5(2), 213–243. DOI: 10.1016/j.accinf.2004.04.002

Hancer, M., & George, R. T. (2003). Psychological empowerment of non-supervisory employees working in full-service restaurants. *International Journal of Hospitality Management*, 22(1), 3–16. DOI: 10.1016/S0278-4319(02)00032-4

Hawkins, T. G., Gravier, M. J., & Muir, W. A. (2020). The role of supplier performance evaluations in mitigating risk: Assessing evaluation processes and behaviors. *Industrial Marketing Management*, 87, 2–17. DOI: 10.1016/j.indmarman.2020.03.004

Huang, J., Jiang, X., & Tang, Q. (2009). An e-commerce performance assessment model: Its development and an initial test on e-commerce applications in the retail sector of China. *Information & Management*, 46(2), 100–108. DOI: 10.1016/j.im.2008.12.003

Idoga, P. E., Toycan, M., Nadiri, H., & Çelebi, E. (2018). Factors affecting the successful adoption of e-health cloud based health system from healthcare consumers’ perspective. *IEEE Access: Practical Innovations, Open Solutions*, 6, 71216–71228. DOI: 10.1109/ACCESS.2018.2881489

Karande, K., Ha, J., & Singhapakdi, A. (2008). The role of contextual factors in relationship commitment of buyers to foreign suppliers: A survey of Korean importers. *Industrial Marketing Management*, 37(7), 856–862. DOI: 10.1016/j.indmarman.2008.01.010
Keegan, S. N., & Lucas, R. (2005). Hospitality to hostility: Dealing with low response rates in postal surveys. *International Journal of Hospitality Management*, 24(2), 157–169. DOI: 10.1016/j.ijhm.2004.06.006

Kim, B. P., Murrmann, S. K., & Lee, G. (2009). Moderating effects of gender and organizational level between role stress and job satisfaction among hotel employees. *International Journal of Hospitality Management*, 28(4), 612–619. DOI: 10.1016/j.ijhm.2009.04.001

Larson, P. D., & Chow, G. (2003). Total cost/response rate trade-offs in mail survey research: Impact of follow-up mailings and monetary incentives. *Industrial Marketing Management*, 32(7), 533–537. DOI: 10.1016/S0019-8501(02)00277-8

Lemettilä, M., Leppä, E., Pohjanoksa-Mäntylä, M., Simula, A., & Koskelo, J. (2021). Anti-doping knowledge and educational needs of Finnish pharmacists. *Performance Enhancement & Health*, 9(2), 100195. DOI: 10.1016/j.peh.2021.100195

Lin, C. P., & Chiu, C. K. (2011). Understanding helping intention and its antecedents among instant messaging users. *Online Information Review*, 35(3), 386–400. DOI: 10.1108/14684521111151432

Madariaga, L., Nussbaum, M., Burq, I., Marañón, F., Salazar, D., Maldonado, L., Alarcón, C., & Naranjo, M. A. (2017). Online survey: A national study with school principals. *Computers in Human Behavior*, 74, 35–44. DOI: 10.1016/j.chb.2017.03.067

Mahmud, I., Ramayah, T., & Kurnia, S. (2017). To use or not to use: Modelling end user grumbling as user resistance in pre-implementation stage of enterprise resource planning system. *Information Systems*, 69, 164–179. DOI: 10.1016/j.is.2017.05.005

Malhotra, M. K., & Grover, V. (1998). An assessment of survey research in POM: From constructs to theory. *Journal of Operations Management*, 16(4), 407–425. DOI: 10.1016/S0272-6963(98)00021-7

McCaffrey, C. (2013). LibQUAL in Ireland: Performance assessment and service improvement in Irish university libraries. *Journal of Academic Librarianship*, 39(4), 347–350. DOI: 10.1016/j.acalib.2012.11.036

McCoy, M., & Hargie, O. (2007). Effects of personalization and envelope color on response rate, speed and quality among a business population. *Industrial Marketing Management*, 36(6), 799–809. DOI: 10.1016/j.indmarman.2006.02.009

Mokotjo, W., & Kalusopa, T. (2010). Evaluation of the agricultural information service (AIS) in Lesotho. *International Journal of Information Management*, 30(4), 350–356. DOI: 10.1016/j.ijinfomgt.2010.01.005
Mort, G. S., McColl-Kennedy, J. R., Kiel, G., & Soutar, G. N. (2004). Perceptions of marketing journals by senior academics in Australia and New Zealand. *Austral-asian Marketing Journal, 12*(2), 51–61. DOI: 10.1016/S1441-3582(04)70097-X

Morton, S. M., Bandara, D. K., Robinson, E. M., & Carr, P. E. A. (2012). In the 21st Century, what is an acceptable response rate? *Australian and New Zealand Journal of Public Health, 36*(2), 106–108. DOI: 10.1111/j.1753-6405.2012.00854.x
PMID: 22487341

Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? *Assessment & Evaluation in Higher Education, 33*(3), 301–314. DOI: 10.1080/02602930701293231

Overbye, M., Knudsen, M. L., & Pfister, G. (2013). To dope or not to dope: Elite athletes' perceptions of doping deterrents and incentives. *Performance Enhancement & Health, 2*(3), 119–134. DOI: 10.1016/j.peh.2013.07.001

Papadimitriou, E., Yannis, G., Bijleveld, F., & Cardoso, J. L. (2013). Exposure data and risk indicators for safety performance assessment in Europe. *Accident; Analysis and Prevention, 60*, 371–383. DOI: 10.1016/j.aap.2013.04.040 PMID: 23769621

Petrovčič, A., Petrič, G., & Manfreda, K. L. (2016). The effect of email invitation elements on response rate in a web survey within an online community. *Computers in Human Behavior, 56*, 320–329. DOI: 10.1016/j.chb.2015.11.025

Prasad, A., Green, P., & Heales, J. (2014). On governance structures for the cloud computing services and assessing their effectiveness. *International Journal of Ac-counting Information Systems, 15*(4), 335–356. DOI: 10.1016/j.accinf.2014.05.005

Prasad, A., Heales, J., & Green, P. (2010). A capabilities-based approach to obtaining a deeper understanding of information technology governance effectiveness: Evi-dence from IT steering committees. *International Journal of Accounting Information Systems, 11*(3), 214–232. DOI: 10.1016/j.accinf.2010.07.013

Proctor, M. D., & Marks, Y. (2013). A survey of exemplar teachers' perceptions, use, and access of computer-based games and technology for classroom instruction. *Computers & Education, 62*, 171–180. DOI: 10.1016/j.compedu.2012.10.022

Queiroz, M., Tallon, P. P., Coltman, T., Sharma, R., & Reynolds, P. (2020). Aligning the IT portfolio with business strategy: Evidence for complementarity of corporate and business unit alignment. *The Journal of Strategic Information Systems, 29*(3), 101623. DOI: 10.1016/j.jsis.2020.101623
Rindfuss, R. R., Choe, M. K., Tsuya, N. O., Bumpass, L. L., & Tamaki, E. (2015). Do low survey response rates bias results? Evidence from Japan. *Demographic Research*, 32, 797-828. DOI: 10.4054/DemRes.2015.32.26

Rothschild, J. M., Fang, E., Liu, V., Litvak, I., Yoon, C., & Bates, D. W. (2006). Use and perceived benefits of handheld computer-based clinical references. *Journal of the American Medical Informatics Association: JAMIA*, 13(6), 619-626. DOI: 10.1197/jamia.M2103 PMID: 16929041

Shadish, W., Cook, T. D., & Campbell, D. T. (2002). *Experimental and quasi-experimental designs for generalized causal inference*. Houghton Mifflin Company.

Sivo, S. A., Saunders, C., Chang, Q., & Jiang, J. J. (2006). How low should you go? Low response rates and the validity of inference in IS questionnaire research. *Journal of the Association for Information Systems*, 7(1), 17. DOI: 10.17705/1jais.00093

Sullivan, P., McBrayer, J. S., Miller, S., & Fallon, K. (2021). An Examination of the use of computer-based formative assessments. *Computers & Education*, 173, 104274. DOI: 10.1016/j.compedu.2021.104274

Syed-Abdul, S., Scholl, J., Lee, P., Jian, W.-S., Liou, D.-M., & Li, Y.-C. (2012). Study on the potential for delay tolerant networks by health workers in low resource settings. *Computer Methods and Programs in Biomedicine*, 107(3), 557-564. DOI: 10.1016/j.cmpb.2011.11.004 PMID: 22178071

Taherdoost, H. (2019). Electronic service quality measurement: Development of a survey instrument to measure the quality of e-service. *International Journal of Intelligent Engineering Informatics*, 7(6), 491-528. DOI: 10.1504/IJIEI.2019.104559

Thörn, C. (2010). Current state and potential of variability management practices in software-intensive SMEs: Results from a regional industrial survey. *Information and Software Technology*, 52(4), 411-421. DOI: 10.1016/j.infsof.2009.10.009

Türel, Y. K. (2011). An interactive whiteboard student survey: Development, validity and reliability. *Computers & Education*, 57(4), 2441-2450. DOI: 10.1016/j.compedu.2011.07.005

Turfus, S., Smith, J., Mansingh, A., Alexander-Lindo, R., & Roopchand-Martin, S. (2019). Supplementation practices, perceptions and knowledge about anti-doping among Jamaican high school athletes. *Performance Enhancement & Health*, 7(1-2), 100145. DOI: 10.1016/j.peh.2019.07.001
Vafaei-Zadeh, A., Ramayah, T., Hanifah, H., Kurnia, S., & Mahmud, I. (2020).
Supply chain information integration and its impact on the operational performance
of manufacturing firms in Malaysia. *Information & Management*, 57(8), 103386.
DOI: 10.1016/j.im.2020.103386

van de Oudeweetering, K., & Agirdag, O. (2018). Demographic data of MOOC learners: Can alternative survey deliveries improve current understandings? *Computers & Education*, 122, 169–178. DOI: 10.1016/j.compedu.2018.03.017

van Iwaarden, J., Van Der Wiele, T., Ball, L., & Millen, R. (2004). Perceptions about the quality of web sites: A survey amongst students at Northeastern University and Erasmus University. *Information & Management*, 41(8), 947–959. DOI: 10.1016/j.im.2003.10.002

Weber, C., Kamber, M., & Lentillon-Kaestner, V. (2016). Are doping substances imported into Switzerland mainly to increase athletic performance? *Performance Enhancement & Health*, 5(2), 66–76. DOI: 10.1016/j.peh.2016.06.001

Xie, J.-Q., Rost, D. H., Wang, F.-X., Wang, J.-L., & Monk, R. L. (2021). The association between excessive social media use and distraction: An eye movement tracking study. *Information & Management*, 58(2), 103415. DOI: 10.1016/j.im.2020.103415

Xu, J., & Quaddus, M. (2012). Examining a model of knowledge management systems adoption and diffusion: A Partial Least Square approach. *Knowledge-Based Systems*, 27, 18–28. DOI: 10.1016/j.knosys.2011.10.003

Xu, J., Wang, X., & Yan, L. (2021). The moderating effect of abusive supervision on information security policy compliance: Evidence from the hospitality industry. *Computers & Security*, 111, 102455. DOI: 10.1016/j.cose.2021.102455

Yang, C., & Chen, L.-C. (2007). Can organizational knowledge capabilities affect knowledge sharing behavior? *Journal of Information Science*, 33(1), 95–109. DOI: 10.1177/0165551506068135

Zhen, J., Xie, Z., & Dong, K. (2021). Impact of IT governance mechanisms on organizational agility and the role of top management support and IT ambidexterity. *International Journal of Accounting Information Systems*, 40, 100501. DOI: 10.1016/j.accinf.2021.100501
<figure><img src="image_4.png" /></figure>