  In this paper, we present our system design for conducting recurring daily
studies on Amazon Mechanical Turk. We implement this system to conduct a study
into touch dynamics, and present our experiences, challenges and lessons
learned from doing so. Study participants installed our application on their
Apple iOS phones and completed two tasks daily for 31 days. Each task involves
performing a series of scrolling or swiping gestures, from which behavioral
information such as movement speed or pressure is extracted.
  Taking place over a time period of 31 days, our study utilized a
self-contained app which workers used to complete daily tasks without requiring
extra HITs. This differs somewhat from the typical rapid completion of one-off
tasks on Amazon Mechanical Turk. This atypical use of the platform prompted us
to study aspects related to long-term user retention over the study period:
payment schedule (amount and structure over time), regular notifications,
payment satisfaction and overall satisfaction. We also investigate the specific
concern of reconciling informed consent with workers' desire to complete tasks
quickly.
  We find that using the Mechanical Turk platform in this way leads to data of
comparable quality to that of lab based studies, and that our study design
choices show a statistically significant effect in keeping workers engaged.
