This work shares unexpected findings obtained from the use of the Amazon Mechanical Turk platform as a source of participants for the study of technology adoption. Expressly,of the 564 participants from the United States , 126 ( 22.34\%) failed at least one of  three forms of attention check (logic, honesty, and time). We also examined whether characteristics such as gender, age, education, and income affected participant attention. Amongst all characteristics assessed, only prior experience with the technology being studied was found to be related to attentiveness. We conclude this work by reaffirming the need for multiple forms of attention checks to gauge participant attention . Furthermore, we propose that researchers adjust their budgets accordingly to account for the possibility of having to discard responses from participants determined not to be displaying adequate attention .
In this study, we investigate the attentiveness exhibited by participants sourced through Amazon Mechanical Turk (MTurk), thereby discovering a significant level of inattentiveness amongst the platform's top crowd workers (those classified as 'Master', with an 'Approval Rate' of 98\% or more, and a 'Number of HITS approved' value of 1,000 or more). A total of 564 individuals from the United States participated in our experiment. They were asked to read a vignette outlining one of four hypothetical technology products and then complete a related survey. Three forms of attention check (logic, honesty, and time) were used to assess attentiveness. Through this experiment we determined that a total of 126 ( 22.3\%) participants failed at least one of the three forms of attention check , with most (94) failing the honesty check - followed by the logic check (31), and the time check (27). Thus, we established that significant levels of inattentiveness exist even among the most elite MTurk workers. The study concludes by reaffirming the need for multiple forms of carefully crafted attention checks, irrespective of whether participant quality is presumed to be high according to MTurk criteria such as 'Master', 'Approval Rate', and 'Number of HITS approved' . Furthermore, we propose that researchers adjust their proposals to account for the effort and costs required to address participant inattentiveness .
R
This work shares unexpected findings obtained from the use of the
In this study, we investigate the attentiveness exhibited by participants sourced through
R
platform as a source of participants for the study of technology adoption. Expressly,of the
(MTurk), thereby discovering a significant level of inattentiveness amongst the platform's top crowd workers (those classified as 'Master', with an 'Approval Rate' of 98\% or more, and a 'Number of HITS approved' value of 1,000 or more). A total of
R
participants
individuals
R
,
participated in our experiment. They were asked to read a vignette outlining one of four hypothetical technology products and then complete a related survey. Three forms of attention check (logic, honesty, and time) were used to assess attentiveness. Through this experiment we determined that a total of
R
22.34\%)
22.3\%) participants
A
the
R
(logic, honesty, and time). We also examined whether characteristics such as gender, age, education, and income affected participant attention. Amongst all characteristics assessed, only prior experience with the technology being studied was found to be related to attentiveness. We conclude this work
, with most (94) failing the honesty check - followed by the logic check (31), and the time check (27). Thus, we established that significant levels of inattentiveness exist even among the most elite MTurk workers. The study concludes
R
attention checks to gauge participant attention
carefully crafted attention checks, irrespective of whether participant quality is presumed to be high according to MTurk criteria such as 'Master', 'Approval Rate', and 'Number of HITS approved'
R
budgets accordingly
proposals
R
possibility of having to discard responses from participants determined not to be displaying adequate attention
effort and costs required to address participant inattentiveness