# Serious Questions in Playful Questionnaires

Aneta Takhtamysheva, Jan Smeddinck

► To cite this version:

Aneta Takhtamysheva, Jan Smeddinck. Serious Questions in Playful Questionnaires. 11th International Conference on Entertainment Computing (ICEC), Sep 2012, Bremen, Germany. pp.449-452, 10.1007/978-3-642-33542-6_50. hal-01556164

HAL Id: hal-01556164

https://inria.hal.science/hal-01556164

Submitted on 4 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Serious Questions in Playful Questionnaires

Aneta Takhtamysheva and Jan Smeddinck

Digital Media Lab, TZI, University of Bremen, 28359 Bremen, Germany
{aneta, smeddinck}@tzi.de

**Abstract.** Conducting surveys is a time consuming and often expensive process. One of the main hurdles is motivating people to participate. The project presented herein proposes a playful approach, distributed via social network environments, where participants are intrinsically motivated to participate. A first evaluation showed an increased incentive to recommend friends to participate in the survey, as well as a preference of the playful approach to conducting surveys in general.

**Keywords:** Social networks, social gaming, social media, playful applications, opinion-mining, survey.

## 1 Introduction

Collecting feedback and opinions through playful applications is an approach that remains largely unexplored [1]. We argue that playful questionnaires, which attract users/gamers mostly through their playful aspect rather than through requests for a favor, can aid in circumventing some of the common challenges of human subject research, such as low response rates and survey fatigue [2]. In a wide range of research fields, opinion mining / surveys and quizzes [3] [4] [5] are tools are potential subjects to applying gamification for serious tasks which don't intend to bring any personal benefit to the users other than engaging experience. The finding that users play games not because they are useful, but mostly because they are fun [6] has already been applied to the games with a purpose [7] [8] [9] [10].

In order to evaluate the method of embedding serious questions in playful surveys and distributing them via social networks, we have created a sample playful application and conducted an explorative user experience study that involves having social network users interact with two versions of a survey: a playful one and a standard one.

## 2 Playful Questionnaire Design

The social network playful application called *Bake Your Personality* contains six serious questions about the video gaming preferences of the users. Following a pattern
that has proven successful in the development of social networks applications, the
playful questionnaire was developed relying on a universal theme that a wide
audience can connect with [7]; in this case the process of cooking. Questions are
asked using the metaphorical concept of a cooking process and of
choosing ingredients for a final product (here: a cake). Figure 1 shows the quest to
select 3 game genres that the player prefers to play most. Each question is presented
as an action the player may perform; possible answers are represented by items that
users interact with. At the end the “cake” is submitted for tasting, and user receives a
playful personality report based on the “taste” of the cake. The players are motivated
to be truthful with their answers in order to receive their correct personality report. In
the background, the decisions made by users are recorded.

<figure><img src="image_2.png" /><figcaption>**Figure 1.** Snapshot of the *Bake your Personality* application.</figcaption></figure>

The earnestness of the answers provided is checked by introducing a control question (a question that was asked before but appears again later on in a rephrased fashion).

In order to evaluate the validity and reception of the playful questionnaires
approach presented herein, a questionnaire with a generic visual style and with an
identical set of questions was developed. It contains single- and multiple-choice
questions that correspond to the same questions and answers embedded in the playful
application. After completing the standard questionnaire the users receive a playful
personality report that contains the same text as in the playful version.

The major differences between these two questionnaires thus are in visual style,
presence/absence of storytelling, and the amount of time necessary to complete the
questionnaire. The playful application requires around 3 minutes on average, while
the simple application takes less than 1 minute to finish.

3 Evaluation

The evaluation involved 20 students from 15 different study departments of the
University of Bremen. The participant pool consisted of 10 males and 10 females,
ranging in age from 20 to 35 years (M=25, SD=3.3 N=20). In order to counterbalance
the order in which subjects participated in the conditions the sequence was randomly
selected. Once participants completed both applications, they were asked to complete
a feedback form, which covered the following issues:

1. Whether the playful elements of the playful questionnaire distracted users from their major task.

2. How users perceived each application: as a survey, serious personality report, playful personality report, or a game.

3. Whether users would recommend any of the two applications to friends on a social network (or verbally if they have no social network account).

4. Which one of the two versions the users would prefer to use in the future?

1. The participants were asked whether the visual elements of the playful application notably distracted them from the major task of answering the “serious” questions. The results were compared to the responses given to the same question concerning the control application. A Wilcoxon Signed Rank test was applied to test the significance of the difference between the self-reported amount of influence of aesthetic and ludic elements on the answers provided by the users. It did not show any statistically significant differences between the degrees of “distraction” of these two applications: Z=1.26 (Mdn=4) p>.05 and showing a mild effect: r=.21.

2. The analysis of the responses received regarding the question “How do you perceive the application?” - given the answer options: *game*, *playful personality report*, *serious personality report*, *survey and undecided* - revealed significant differences in how participants perceived both applications (Z=-3.88, p<.00). The majority of players perceived the playful questionnaire as a *playful personality report*, closely followed by *game*. The other version was perceived mostly as a *survey*.

3. The question “Which of the applications would you recommend to a friend on a social network?” revealed significant differences in replies (Z=-2.67 p<.00), with the majority of participants being positive about the idea of spreading the playful questionnaire further to friends, while showing significantly less intention of doing so regarding the short questionnaire.

4. Concerning the question “Which application would you prefer to deal with”, most females expressed a preference of the playful questionnaire, while male players could be divided into two categories: those who would prefer the playful format and those who would distinctively prefer the short format.

Finally, when asked to estimate the degree of earnestness that they displayed while interacting with each application, the majority of the users believed that they considered their answers more seriously when filling out the short survey, rather than when interacting with the playful questionnaire.

While self-reported accounts of earnestness are bound to be biased and the observed difference can likely be explained by the playful mood of the playful questionnaire that inevitably sets a more relaxed approach to answering questions, it is important to further investigate the data provided by quality control questions, once the survey is run on a social network.
# 4 Discussion and Conclusion

The user experience evaluation of both applications suggests that the majority of the experiment participants prefer to deal with the longer playful applications rather than generic short surveys. The experiment participants showed distinctively more willingness to recommend the playful version to their friends. They also expressed the belief that the playful elements do not distract them from the main task of answering serious questions.

The approach of playful questionnaires offers intriguing benefits in terms of motivation, cost and demographic control. Yet, for the time being, further studies need to be executed in order to investigate the potential biases, to improve filtering techniques and, in the long run, to determine whether the approach can be generalized to the point where businesses and research institutes can target certain demographics by design, while the extend of bias can be quantified. At the same time, a playful and fun experience must be offered to potential users, customers and players if they are to be confronted with playful surveys more frequently.

# References

1. Schmidt, L.A. Crowdsourcing for Human Subjects Research. Proceedings of CrowdConf 2010 (2010)

2. For some consumers, surveys breed feedback fatigue, USA Today, http://www.usatoday.com/money/story/2012-01-07/consumer-feedback-fatigue/52432412/1, (2012), last viewed: 2012-07-05

3. Krause, M., Smeddinck, J., Takhtamysheva, A., Markov, V., Runge, N.: Playful Surveys: Easing Challenges of Human Subject Research with Online Crowds Challenges of Human Subject Research. In HComp’12 Proceedings of the AAAI Workshop on Human Computation (2012)

4. SlaveryFootPrint, http://slaveryfootprint.org, last viewed: 2012-07-05

5. Weichselbraun, A., Gindl, S., Scharl, A.: Using games with a purpose and bootstrapping to create domain-specific sentiment lexicons. In Proceedings of the 20th ACM international conference on Information and knowledge management (CIKM '11), ACM, New York, NY, USA, pp. 1053-1060 (2011)

6. Hsu, C.L., Lu, H.P.: Why do people play on-line games? An extended TAM with social influences and flow experience, Information and Management, 41 (7), pp. 853-868 (2004)

7. Von Ahn. L.: Human computation. In Proceedings of the 46th Annual Design Automation Conference (DAC '09). ACM, New York, NY, USA, pp. 418-419 (2009)

8. Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z.: Predicting protein structures with a multiplayer online game. J. Nature 466, 756-760 (2010)

9. Kawrykow, A., Roumanis, G., Kam, A., Kwak, D., Leung, C.: Phylo: A Citizen Science Approach for Improving Multiple Sequence Alignment. PLoS ONE 7/3 (2011)

10. Chrons, O., Sundell, S.: Digitalkoot: Making Old Archives Accessible Using Crowdsourcing. In: HCOMP '11 Proceedings of the ACM SIGKDD Workshop on Human Computation (2011)

11. Mahajan, A.: Zynga: Building Big Social Games. Game Developers Conference Presentation, Game Developers Conference (2009)