WORKSHOP REPORT

GamifIR 2016 – SIGIR 2016 Workshop on
Gamification for Information Retrieval
Michael Meder
Technische Universität Berlin
michael.meder@dai-labor.deFrank Hopfgartner
University of Glasgow
frank.hopfgartner@glasgow.ac.uk

Gabriella Kazai
Semion Ltdgabs@lumi.do

Udo Kruschwitz
University of Essex
udo@essex.ac.uk
Abstract

The third workshop on Gamification for Information Retrieval (GamifIR) took place on
the 21th of July 2016 in conjunction with SIGIR 2016 in Pisa, Italy. It was the first GamifIR
held in conjunction with the SIGIR, the first and second GamifIR workshops were both colocated with ECIR. The workshop program included one invited keynote presentation, sevenpaper presentations and a discussion session. The keynote presentation stated the necessityof proper theory for gamification design and resulting opportunities. The paper presentation
covered studies on diverse areas and approaches for the application of gamification.

1Introduction

Given the enormous response to the first [4] and second GamifIR [5] workshops that wereboth co-located with ECIR, GamifIR’16 was accepted to be organized in conjunction withthe SIGIR conference. Our main motivation as organizers of the workshop was to addressa broader research community with the range of emerging challenges and opportunities associated with Information Retrieval. The engagement and participation of users is often
crucial for IR challenges. Gamification is the approach to learn from often fun, creative andengaging games. In principle, it is about understanding games and applying game design
elements in a non-gaming environments. In the context of Information Retrieval (IR) it is
not hard to imagine that many tasks could benefit from gamification techniques. Besides
several manual annotation tasks of data sets for IR research, user participation is importantin order to gather implicit or even explicit feedback to feed the algorithms. Gamification,
however, comes with its own challenges and its adoption in IR is still in its infancy.

2Topics of Interest

In this workshop we wanted to discuss and learn from planned and already executed studies
and applications that use gamification to trigger users’ intrinsic motivation. Therefore we

ACM SIGIR Forum

47Vol. 50 No. 2 December 2016

called for the submission of position papers as well as novel research papers and demos
addressing problems related to gamification in IR. Topics of interest included but were notlimited to:• Gamification approaches in a variety of contexts, including document annotation andground-truth generation; interface design; information seeking; user modelling; knowledge sharing• Gamification in Crowdsourcing• Gamification to IR teaching• Gamification in recommender systems and apps
• User engagement and motivational factors of Gamification• Player types, contests, cooperative Gamification• Long-term engagement• Search challenges• Gamification design• Applied game principles, elements and mechanics

3Keynote and Paper Presentations

Continuing the tradition of inviting keynote speakers to GamifIR, we were very pleased towelcome Sebastian Deterding, who talked about Desperately Seeking Theory: Gamification,
Theory, and the Promise of a Data/AI-Driven New Science of Design [2]. Sebastian is oneof the leading experts in the field of gamification. In his talk, he discussed a number ofchallenges and opportunities for further research on gamification.
Seven full papers were selected by the programme committee from a total of 11 submissions. Each submitted paper was reviewed by three members of an international programmecommittee. All papers are published in the workshop proceedings [6] and were presented atthe workshop. For a very detailed summary on the keynote and paper presentations we recommend our workshop report at The Informer newsletter of the BCS Information Retrieval
Specialist Group1 . The list of papers includes:• Matthew Barr, Kay Munro and Frank Hopfgartner. Increasing Engagement with theLibrary via Gamification [1]• Laura Guillot, Quentin Bragard, Ross Smith, Dan Bean and Anthony Ventresque.
Towards a Gamified System to Improve Translation for Online Meetings [3]• Yuan Jin, Mark Carman and Lexing Xie. A Little Competition Never Hurt Anyone’sRelevance Assessments [7]• Ioannis Karatassis and Norbert Fuhr. Gamification for WebSAIL [8]• Andreas Leibetseder and Mathias Lux. Gamifying Fitness or Fitnessifying Games: aComparative Study [9]• Michael Meder, Till Plumbaum and Sahin Albayrak. Learning Gamification DesignAn Usability First Approach for the Enterprise Infoboard Experiment [10]1

http://irsg.bcs.org/informer/2016/08/gamification-workshop/

ACM SIGIR Forum48Vol. 50 No. 2 December 2016• Giorgio Maria Di Nunzio, Maria Maistro and Daniel Zilio. Gamification for Machine
Learning: The Classification Game [11]4Acknowledgments

We acknowledge the efforts of the programme committee, namely Raian Ali (BournemouthUniversity, UK), Omar Alonso (Microsoft, USA), Leif Azzopardi (University of Glasgow,UK), Jon Chamberlain (University of Essex, UK), Sebastian Deterding (University of York,UK), Carsten Eickhoff (ETH Zurich, CH), Christopher Harris (State University of New
York, USA), Hideo Joho (University of Tsukuba, JP), Till Plumbaum (Technische Universität Berlin, GER), Ashok Ranchod (University of Southampton, UK), Thomas Springer(Technische Universität Dresden, GER), Susanne Strahinger (Technische Universität Dresden, GER), Albert Weichselbraun (University of Applied Sciences Chur, CH), and Lincoln
Wood (University of Auckland, NZ).
We acknowledge financial support by the European Commission via the European Union
Seventh Framework Programme FP7/2007-2013, grant agreement number 610594 (CrowdRec).

References[1] M. Barr, K. Munro, and F. Hopfgartner. Increasing engagement with the library via
gamification. In Proceedings of the Third International Workshop on Gamification for
Information Retrieval co-located with 39th International ACM SIGIR Conference on
Research and Development in Information Retrieval (SIGIR 2016), Pisa, Italy, July 21,2016., pages 2–6, 2016.
[2] S. Deterding. Desperately seeking theory: Gamification, design, data, machine learning, and the promise of double loop learning systems. In Proceedings of the Third
International Workshop on Gamification for Information Retrieval co-located with 39th
International ACM SIGIR Conference on Research and Development in Information
Retrieval (SIGIR 2016), Pisa, Italy, July 21, 2016., page 1, 2016.
[3] L. Guillot, Q. Bragard, R. Smith, D. Bean, and A. Ventresque. Towards a gamified system to improve translation for online meetings. In Proceedings of the Third International
Workshop on Gamification for Information Retrieval co-located with 39th International
ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR2016), Pisa, Italy, July 21, 2016., pages 21–28, 2016.
[4] F. Hopfgartner, G. Kazai, U. Kruschwitz, and M. Meder, editors. Proceedings of the FirstInternational Workshop on Gamification for Information Retrieval, GamifIR@ECIR’14, Amsterdam, The Netherlands, April 13, 2014. ACM, 2014.
[5] F. Hopfgartner, G. Kazai, U. Kruschwitz, and M. Meder, editors. Proceedings of theSecond International Workshop on Gamification for Information Retrieval co-locatedwith the 37th European Conference on Information Retrieval (ECIR 2015), Vienna,Austria, March 29, 2015, volume 1345 of CEUR Workshop Proceedings. CEUR-WS.org,2015.

ACM SIGIR Forum49

Vol. 50 No. 2 December 2016

[6] F. Hopfgartner, G. Kazai, U. Kruschwitz, and M. Meder, editors. Proceedings of theThird International Workshop on Gamification for Information Retrieval co-located with39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2016), Pisa, Italy, July 21, 2016, volume 1642 of CEUR Workshop
Proceedings. CEUR-WS.org, 2016.
[7] Y. Jin, M. J. Carman, and L. Xie. A little competition never hurt anyone’s relevanceassessments. In Proceedings of the Third International Workshop on Gamification forInformation Retrieval co-located with 39th International ACM SIGIR Conference on
Research and Development in Information Retrieval (SIGIR 2016), Pisa, Italy, July 21,2016., pages 29–36, 2016.
[8] I. Karatassis and N. Fuhr. Gamification for websail. In Proceedings of the Third International Workshop on Gamification for Information Retrieval co-located with 39th
International ACM SIGIR Conference on Research and Development in Information
Retrieval (SIGIR 2016), Pisa, Italy, July 21, 2016., pages 15–20, 2016.
[9] A. Leibetseder and M. Lux. Gamifying fitness or fitnessifying games: a comparativestudy. In Proceedings of the Third International Workshop on Gamification for Information Retrieval co-located with 39th International ACM SIGIR Conference on Researchand Development in Information Retrieval (SIGIR 2016), Pisa, Italy, July 21, 2016.,
pages 37–44, 2016.
[10] M. Meder, T. Plumbaum, and S. Albayrak. Learning gamification design - an usability
first approach for the enterprise infoboard experiment. In Proceedings of the ThirdInternational Workshop on Gamification for Information Retrieval co-located with 39thInternational ACM SIGIR Conference on Research and Development in Information
Retrieval (SIGIR 2016), Pisa, Italy, July 21, 2016., pages 7–14, 2016.
[11] G. M. D. Nunzio, M. Maistro, and D. Zilio. Gamification for machine learning: Theclassification game. In Proceedings of the Third International Workshop on Gamification
for Information Retrieval co-located with 39th International ACM SIGIR Conference on
Research and Development in Information Retrieval (SIGIR 2016), Pisa, Italy, July 21,2016., pages 45–52, 2016.

ACM SIGIR Forum50Vol. 50 No. 2 December 2016