# "Do Not Touch the Paintings!" The Benefits of Interactivity on Learning and Future Visits in a Museum

Konstantinos Mikalef, Michail N. Giannakos, Konstantinos Chorianopoulos, Letizia Jaccheri

► To cite this version:

Konstantinos Mikalef, Michail N. Giannakos, Konstantinos Chorianopoulos, Letizia Jaccheri. "Do Not Touch the Paintings!" The Benefits of Interactivity on Learning and Future Visits in a Museum. 11th International Conference on Entertainment Computing (ICEC), Sep 2012, Bremen, Germany. pp.553-561, 10.1007/978-3-642-33542-6_73. hal-01556117

HAL Id: hal-01556117

https://inria.hal.science/hal-01556117

Submitted on 4 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# "Do not touch the paintings:" The benefits of interactivity on learning and future visits in a museum

Konstantinos Mikalef¹, Michail N. Giannakos², Konstantinos Chorianopoulos³,
Letizia Jaccheri²

¹Hellenic Open University, Patra 26335, Greece
kostas.mikalef@gmail.com

²Norwegian University of Science and Technology (NTNU), Trondheim 7491, Norway
{mgiannak, letizia}@idi.ntnu.no

³Ionian University, Corfu 49100, Greece
choko@ionio.gr

**Abstract.** Educators and museum curators have recognized the value of interactivity, but it remains unclear what is the right level of interactivity in informal learning settings, such as museums. In this study, we explore the effect of increasing levels of interactivity on learning performance and students' intention for future museum visits. We developed an educational mobile application based on QR codes and quiz software, in order to augment visual arts comprehension during a visit to an art gallery. In addition to the mobile-based version of the game, a paper-based version was also employed followed by a controlled experiment. A total of 60 lyceum students (between 15 and 16 years old) participated in a between-groups evaluation that compared the performance of three levels of interactivity (passive guided tour, paper-based, mobile-based), as well as the perceptions among the groups. The results indicate that the mobile-based student group had higher performance in the post-assessment when compared with the paper-based one. Notably, perceived interest for the game affects students' perceptions for a future museum visit. Further research should consider the effects of higher-fidelity types of mobile applications, such as 3D graphics, as well as augmented-reality games.

**Keywords:** Mobile learning, informal learning, performance, perceptions, mobile devices, museums.

## 1 INTRODUCTION

Mobile applications are increasingly used nowadays to assist many scenarios (e.g., museums, cultural heritage locations, outdoor activities). Educational medium effectiveness and students' perceptions for the medium highly affect the learning success. As the introduction of learning tools is often complex, students do not always use them as expected. For instance, students' perceptions regarding the importance and the interest for the medium are some of the most widespread barriers for the effective adoption (e.g. [5, 7]). In addition students' perceptions have an impact on what they have already learned and what they choose to do next [11]. Especially, for the mobile
devices, due to their portability and functionality, they can be used outdoors, in exhi-
bitions and in other non classroom settings, extending the depth and breadth of infor-
mal learning and the importance of students’ perceptions.

Research in educational games argues that well-designed games can meet some of
the needs of students and pursue them to enroll in learning procedures [13]. Evidence
of learning performance of educational games has been shown in several areas, such
as history, arts and cultural heritage [2, 3].

Portable devices have been successfully employed as learning tools in both formal and informal learning contexts. In the formal learning, handhelds have been used in teaching both within the classroom setting and out-classroom (i.e. [2]). In addition, the potential of handhelds to assist informal learning has been explored through different frameworks. For instance, handhelds have been used as museum guidebooks [6, 12] or as learning systems [8].

Researchers have often stressed the benefits of enhancing the functions of mu-
seums, parks and exhibitions by designing educational projects connected to learners’
informal learning, [8] and making non classroom environments more enjoyable and
motivating for learning [12]. In the context of museums, many researchers have em-
ployed new technologies and customized mobile devices for attracting and engaging
visitors. On that direction visitors’ beliefs is highly important [2]. For instance, one of
the most successful projects was conducted in the Exploratorium of San Francisco [6].
This study was one of the pioneers in the direction of combining mobile technology
and capturing experience in the real world.

Many studies have shown the potential of portable devices to increase learning opportunities. Informal learning with the real world experiences is some of the most remarkable advantages. Mobile devices’ portability and functionality make them suitable for out-classroom learning, for instance bird-watching, plant hunting and museum guiding are some of the most successful case studies [6, 8]. As such, it is subsequent that the use of mobile devices would bring many benefits and opportunities for informal learning in future. In particular, in this study we aim to provide insights of the following question: (How) Can interactive mobile technology be exploited to implement informal education?

As such, we designed an interactive mobile-based and a paper-based game and we
conducted a controlled experiment in order to explore the educational impact and
students' performance and perceptions affected by interactivity and mobile technolo-
gy.

2 METHODOLOGY

## 2.1 The Game Design

A quiz was designed to include several game elements, such as purpose, rules, score, and limitations (space-time). The main purpose of the quiz activity is to identify a series of paintings, at an Art Gallery (fig. 1), based on descriptions of visual elements. During our experiment, the quiz was played by teams of two players, who were given 25 minutes to answer 12 questions that lead them to certain paintings.
<figure><img src="image_2.png" /><figcaption>Fig. 1. The layout of the public art gallery and the 12 correct answers (paintings)</figcaption></figure>

Two free (one of them open-source) software tools were used for implementing the activity, a QR application and a quiz application. The only hardware requirement for running the software is a java enabled mobile phone. Camera feature was a requirement for the QR application<fnref n="1" />. The design of the questions was implemented with MyMLE<fnref n="2" /> application, which allows creating mobile learning content. Some of the basic advantages of this game is the low cost of the infrastructure and that there is no coding required in developing the quiz of questions.

There are two versions of the activity, a Mobile Based (MB) and a paper-based (PB) version. In MB version QR codes are placed next to paintings. Each team provided with two mobile phones, one where questions displayed and answers typed in, and a second one that was used to scan QR codes. When the players identify a painting as an answer to a question, they scan the QR code next to it and type a four digit number as the answer (Fig. 2). If the answer is correct they get a message indicating so. If the answer is wrong, they get a message indicating that they have one more chance to answer correctly. Hence in total each team was given up to two chances to identify a correct answer.

The PB version of the game was identical, except from the fact QR codes were replaced with numbered post-its and the questions were printed on paper. However the major difference in the gameplay between the two versions was that players do not receive any feedback regarding their answers, during the PB game.

<footnote n="1">¹ http://qrcode.sourceforge.jp</footnote>

<footnote n="2">² http://mle.sourceforge.net</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2. The QR codes were placed next to the paintings (left), the mobile application (right)</figcaption></figure>

## 2.2 Sampling

The research methodology included a survey composed by the four main measures and by score tests. The initial sample of the study was sixty 1st grade lyceum (upper general secondary school) students, aged between 15 and 16 years. All the students were attending an elective course in ICT and multimedia at a general lyceum at Corfu, Greece. Three groups were formed up with high similarity in their prior class performance. The three groups of twenty students each, corresponded to the three different classes of the aforementioned course. Group A consisted of 16 males and 4 females, Group B consisted of 17 males and 3 females and Group C consisted of 16 males, 4 females. It should be noted that according to the school's policy students were assigned to the classes of the elective course in a rather "random" manner. At the initial sample of 60 students, we had 5 drop-outs, one female student from Group A, two female students from Group B and two male students from Group C, leaving a total of 55 (19 for the first group, and 18 for the remaining two).

## 2.3 Procedures

The study was conducted over a three-week period from 10th to 28th of May 2010. During the first week formal classroom teaching was held. At the second week students visited the Art Gallery (Fig. 1) of the Municipality of Corfu and at the third week the posttest (questionnaire and assessment) took place. Upon completion of the tour students from Group A and Group B were briefed on the rules and played the game (mobile- and paper-based respectively). The students of Group A, were also provided with mobile phones with a sample quiz as well as sample QR codes in order to familiarize themselves with the software. The same devices were used during the game. Students of Group C where given instead 30 minutes extra time to spend at the gallery (extended tour), in order to identify visual elements on their own as they wished. For the final week of the study, the students from all three groups took the formal test (assessment) and answered the questionnaire.
## 2.4 Measures

In order to measure students' effectiveness, all groups completed the cognitive assessment at the end of the study. In addition, Groups A and B completed the survey with the four measures regarding students perceptions.

For the cognitive test a twelve-item assessment, consisting of multiple choice questions, was employed. Students were presented with a series of pictures and had to choose one out of the four descriptions that was more accurate. The description was referring to the same visual elements that were also employed during the game, which were combinations of all of the following: primary and secondary colors, warm and cold colors, geometric and organic shapes, depth and perspective. For instance, two questions were: "which painting has Religious Theme, Action Scene and Perspective in two spots?" (correct: The Beheading of the Precursor) and "which painting has Action Scene, Urban Landscape and Warm Colors?" (correct: Kapodistrias Assassination).

The second part included measures of the various questions identified in the literature from previous researches. These measures included Perceived Importance [9]; Perceived Comprehension [4]; and interest with the activity and the visit [1]. In particular the following constructs were employed: (1) How important do you believe that the activity is to your future visits (perceived importance of activity for future visit, PIAFV), (2) How well do you feel you understand the concepts taught (perceived comprehension, PC), (3) How interest did you find the visit (perceived Interest for the visit, PIV) and (4) How interest did you find the activity (Perceived Interest for the activity/game, PIG). For each statement, students have to mark their response on a five-point Likert scale in relation to four measures.

## 2.5 Data Analysis

Firstly, in order to examine the differences among mobile-based, paper-based game and extended tour guide on students' effectiveness, a post hoc test was conducted. A Games Howell criterion was used in order to examine separately for each group the influence of 1) mobile-based game, 2) paper-based game and 3) extended tour guide on students' effectiveness. In addition for identifying any differences in students' perceptions for perceived importance, perceived comprehension and interest with the activity and the visit a t-test was conducted among mobile-based and paper based game. Finally, for identifying any correlations among students' perceptions Spearman correlation coefficient was conducted.

# 3 RESEARCH FINDINGS

To examine the hypotheses regarding students' performance, the Post-hoc analyses using Games – Howell criterion were conducted to verify whether the different learning procedures (Group) are related to the different effectiveness of the students. The results of three groups, Group A ($M_1=9.00, SD_1=2.08$), Group B ($M_2=7.28, SD_2=1.67$) and Group C ($M_3=8.50, SD_3=1.97$), highlighted a significant difference only between Group A and Group B ($p<0.05$).
The results highlight that the enrolment with the mobile affects their performance. However, both (paper-mobile) games persuade the children in several ways. A team benefits from a strategy and game characteristics seem to motivate and excite them for the activity. In addition, based on the observations of the researcher students seemed to be more focused and immersed on the mobile application (fig. 3). Interestingly, mobile application did not push students to lose their interest for the museum as many paintings and its features are discussed from students by the end of the experiment and on the respective course on school. Another notable observation is that students from both groups spent approximately the same total time at the gallery (19 min on average). This may be derived from students' familiarity with mobile phones and the simple design of the game.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Students of the mobile-based game were immersed in questions displayed on their mobile phones</figcaption></figure>

Regarding students’ perceptions among mobile-based and paper-based game, a t-test was used for identifying the differences, with four independent variables (PIAFV, PC, PIG, PIV) and the one dependent variable (Group) were included. All variables demonstrate very high levels on both groups (fig. 4), but no significant difference is identified on students’ perceptions among the two groups.

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Students perceptions among mobile-based and paper-based game</figcaption></figure>

Spearman's correlation coefficient between the measures was used, which is about quantifying the strength of the relationship between the variables. Spearman's test suggests that some of the measures are related, in some cases relatively strong. More precisely, perceived interest for the game is strongly related with all measures. Perceived interest for the visit has correlation only with perceived interest for the game.
Moreover, perceived importance of activity for future visit and perceived comprehension have significant relation with all measures except perceived interest for the visit. Table 1 indicates the exact results of the test.

<table><caption>**Table 1.** Spearman correlations between measures</caption><thead><tr><th></th><th>PIAFV</th><th>PC</th><th>PIG</th><th>PIV</th></tr></thead><tbody><tr><th>PIAFV</th><td>1</td><td></td><td></td><td></td></tr><tr><th>PC</th><td><strong>.411</strong><sup>**</sup></td><td>1</td><td></td><td></td></tr><tr><th>PIG</th><td><strong>.342</strong><sup>**</sup></td><td><strong>.355</strong><sup>*</sup></td><td>1</td><td></td></tr><tr><th>PIV</th><td>.272</td><td>.213</td><td><strong>.560</strong><sup>**</sup></td><td>1</td></tr></tbody></table>

## 4 DISCUSSION AND CONCLUSIONS

The current study is one of the few so far, where a mobile application assists an informal educational process. Conducting a field study, of this kind with a sample of a 55 students, was not an easy task, especially when social interaction threats are concerned. The overall results prove that the mobile application had a positive effect, on students' performance at the final test and also does not affect their interest for visiting the gallery.

The results of the study allow us to argue that the enhancement of mobile device in a learning game context benefits students' performance in a highly significant way. This result may be explained from previous findings in the literature [8] stating that the usage of mobile devices motivates students and offers sufficient learning opportunities to improve learning procedures. From research we do not have sufficient evidence to suggest that the usage of a mobile application affects students' perceptions regarding the non-classroom learning environment. However, students' perceptions for the game interest positively affect their beliefs for the visit, for a future visit and their comprehension. Hence, we can argue that the enrolment of mobile devise can improve the effectiveness of game based learning and can compete the extended guiding effectiveness in a non classroom environment. In addition, students' interest with the game helps them to understand the benefits of the visit and provide a vehicle for future visits.

These findings have important implications for understanding how students regulate their learning and achievement. Previous studies have shown that students' perceptions of what they have already learned affect what they choose to do next [11]; thus, the differences that entity and incremental theorists exhibit in their perceived comprehension could potentially have long-term effects on their achievement, even if these differences are not currently reflected in their actual comprehension (as was the case in our experiment).

The findings of this study must be interpreted in light of some potential limitations. First, the generalizability of this study must be carefully made, as it was conducted in a single context with specific instructions. Secondly, the sampling of the study apart from students at one grade level and was not longitudinal; therefore the data could not reveal the continuation of the MB game behavior. Despite these limitations, the find-

<footnote>* Correlation is significant at the 0.05 level. ** Correlation is significant at the 0.01 level.</footnote>
ings generate valuable insights, which can be used as part of hypotheses for repre-
sentative follow-up studies in technological tools' educational effectiveness and expe-
rience.

Further studies may consider aiming to reveal several insights. First investigating
the effect of the game in students' performance under different context and for a long-
term period might help on the overall understanding. Also, another study involving
different educational levels and students with different levels of computer experience
might draw several important conclusions. Further research also needs to examine
students' intention to adopt Augmented Reality based learning.

Acknowledgements

The authors would like to thank all of the students, the schools' staff and the mu-
seum curator for their participation in the experiment. The research introduces in this
work is supported by the ERCIM "Alain Bensoussan" Fellowship Programme. This
programme is supported by the Marie Curie Co-funding of Regional, National and
International Programmes (COFUND) of the European Commission. This work is
also supported by project CULT (http://cult.di.ionio.gr). CULT (MC-ERG-2008-
230894) is a Marie Curie project of the European Commission (EC) under the 7th
Framework Program (FP7).

5 REFERENCES

1. Berger, C., & Carlson, E. 1988. Measuring computer literacy of teacher trainers. Journal of Educational Computing Research, 4 (9), pp. 289–303.

2. Cabrera J.S., Frutos H.M., Stoica A.G., Avouris N., Dimitriadis Y., Fiotakis G. & Liveri K.D. 2005 Mystery in the museum: collaborative learning activities using handheld devices. In Proceedings of the 7th international conference on Human computer interaction with mobile devices & services. MobileHCI '05. ACM, New York, NY, 315-318.

3. Costabile, M.F., De Angeli, A., Lanzilotti, R., Ardito, C., Buono, P., Pederson, T. 2008. Explore! Possibilities and Challenges of Mobile Learning. In Proceedings of CHI 2008. ACM Press, New York, NY.

4. Dunlosky, J., & Lipko, A. R. 2007. Metacomprehension: A brief history and how to improve its accuracy. Current Directions in Psychological Science, 16, 228–232.

5. Giannakos, M.N., & Vlamos, P. 2012. Using Webcasts in Education: Evaluation of its Efectiveness. British Journal of Educational Technology. doi:10.1111/j.1467-8535.2012.01309.x

6. Hsi H. 2003. A study of user experiences mediated by nomadic web content in a museum. Journal of Computer Assisted Learning 19, 308–319. doi: 10.1046/j.0266-4909.2003.jca_023.x

7. Hsu, C.-L., and Lin, J.C.-C. 2008. Acceptance of Blog Usage: The Roles of Technology Acceptance, Social Influence and Knowledge Sharing Motivation. Information and Management, 45, 1, 65-74. DOI= http://dx.doi.org/10.1016/j.im.2007.11.001
8. Huang Y., Lin Y. & Cheng S. 2010. Effectiveness of a mobile plant learning system in a science curriculum in Taiwanese elementary education. Computers & Education 54, 47–58. doi:10.1016/j.compedu.2009.07.006

9. Joshi, K. D., Kvasny, L., McPherson, S., Trauth, E. M., Kulturel-Konak, S. and Mahar, J. 2010. Choosing IT as a Career: Exploring the role of Self-Efficacy and Perceived Importance of IT Skills. ICIS 2010 Proceedings. Paper 154. http://aisel.aisnet.org/icis2010_submissions/154

10. King, D. L., Delfabbro, P. H., & Griffiths, M. D. (2010). Video game structural characteristics: a new psychological taxonomy. International Journal of Mental Health and Addiction, 8, 90–106.

11. Metcalfe, J., & Finn, B. (2008). Evidence that judgments of learning are causally related to study choice. Psychonomic Bulletin & Review, 15, 174–179.

12. Sung Y.T., Chang K.E., Hou H.T. & Chen P.F. (2010) Designing an electronic guidebook for learning engagement in a museum of history. Computers in Human Behavior 26, 74–83.

13. Tarumi, H., Yamada, K., Daikoku, T., Kusunoki, F., Inagaki, S., Takenaka, M., Hayashi, T., and Yano, M. 2008. KEI-time traveler: visiting a past world with mobile phones to enhance learning motivation. In Proceedings of the 7th international Conference on interaction Design and Children. IDC '08. ACM, New York, 161–164.