# RelicPad: A Hands-On, Mobile Approach to Collaborative Exploration of Virtual Museum Artifacts

Steven Neale, Winyu Chinthammit, Christopher Lueg, Paddy Nixon

► To cite this version:

Steven Neale, Winyu Chinthammit, Christopher Lueg, Paddy Nixon. RelicPad: A Hands-On, Mobile Approach to Collaborative Exploration of Virtual Museum Artifacts. 14th International Conference on Human-Computer Interaction (INTERACT), Sep 2013, Cape Town, South Africa. pp.86-103, 10.1007/978-3-642-40483-2_7. hal-01497484

HAL Id: hal-01497484

https://inria.hal.science/hal-01497484

Submitted on 28 Mar 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# RelicPad: A Hands-On, Mobile Approach to Collaborative Exploration of Virtual Museum Artifacts

Steven Neale¹, Winyu Chinthammit¹, Christopher Lueg², and Paddy Nixon¹

¹ HIT Lab AU, ² School of Computing and Information Systems, University of Tasmania, Australia

{Steven.Neale, Winyu.Chinthammit, Christopher.Lueg, Paddy.Nixon}@utas.edu.au

**Abstract.** In an ideal world, physical museum artefacts could be touched, handled, examined and passed between interested viewers by hand. Unfortunately, this is not always possible – artefacts may be too fragile to handle or pass around, or groups of people with mutual interests in objects may not be in the same location. This can be problematic when attempting to explain or make sense of the physical properties of artefacts.

To address these problems, we propose that direct manipulation of 3D content based on real-world interaction metaphors can help collaborators (both co and remotely located) to construct personal and mutual physical and spatial awareness of artefacts, while networked communication and collaboration allow for ideas and knowledge to be exchanged and shared.

We present our interpretations from two studies of *RelicPad*, a tablet-based application that allows users to manually manipulate museum artefacts and to 'point out' areas of interest to each other using 3D annotations, facilitating a mutual awareness of spatial properties and referencing during discussion.

**Keywords:** Museum artefacts; remote collaboration; tablet interfaces; 3D interaction techniques; virtual reality.

## 1 Introduction

Handling physical museum artefacts is acknowledged as being a powerful experience. As objects with meaning, artefacts help to create “strong personal connections” to pasts and people across time and culture, prompting thought, memory and understanding [1]. This makes it particularly useful to have artefacts at hand when discussing them or explaining them to other people. However, in situations where handling artefacts is not possible (objects being enclosed in display cases, too fragile or valuable to be handled, or located in a different part of the world), the vital spatial referencing and physical understanding that handling artefacts provides is lost.

The Queen Victoria Museum & Art Gallery (QVMAG) [2] in Launceston, Tasmania, who provided resources for our research, described to us an example scenario of
museum curators considering the purchase of an artefact from an overseas museum. With time, money, and effort at stake, it is hugely advantageous for curators to be able to understand the physical properties of artefacts before committing to a decision, even if they are not able to handle them in person. Other scenarios within the problem domain include the discussion of artefacts amongst online communities (digital museum visitors), between curators in different museums around the world, long distance lectures, and presentations.

This paper describes an initial design and evaluations of *RelicPad*, a tablet-based application that addresses this problem by facilitating physical, reality-based interaction with and collaborative discussion of virtual (3D representations of) museum artefacts. In collaboration with the QVMAG, a piece of nineteenth century scrimshaw (scrollwork, engravings, and carvings done in bone or ivory [3]) has been rendered in 3D for exploration and discussion using *RelicPad*. During a discussion, users:

* Manipulate the virtual museum artefact by using their hands to manually interact with the tablet.

* Leave 3D markers (referred to in this paper as ‘interest points’) on different parts of the virtual museum artefact.

These features allow users to manually manipulate a virtual museum artefact and to ‘point’ to specific areas of interest, despite not being able to see the actions and gestures of others. This allows users to construct their own awareness of an artefact’s physical and spatial properties and to communicate spatial references for others.

We first describe our motivation for the research, and present some related work. Next, we describe the application and provide an overview of its functionality. We then introduce and present the results of our two studies, and offer a discussion of our findings. Finally, we draw conclusions based on our interpretations of the discussion and finish by outlining our future work.

## 2 Motivation

While replicating the experience of physically handling an artefact is impossible, our research presents an alternative approach to making sense of artefacts through exploration and collaborative discussion, based on 3D representations of their physical properties. As well as the loss of the tactile and physical understanding and sensation that comes from handling objects individually, it is also problematic for collaborators to convey gestural clues about the spatial relationships between themselves and the objects around them without mutual access to objects [4], making it difficult for individuals in different locations to maintain focus on them.

Our research is aimed at providing a usable application for a number of different museum user groups, including researchers, curators, educators, and visitors (both physical and online). Each of these groups has their own sets of motivations for engaging with artefacts - overseeing and caring for collections, organising educational programs, public service and community outreach, authenticating, evaluating and categorizing artefacts, presenting information to the public, or receiving information
from the museum institution [5] [6] [7]. The unifying thread between users in these
groups is that they all have the desire to engage with artefacts, and that they all have
opinions, ideas, or knowledge to share, drawing on everything from culture and per-
sonal experience to specialized training or knowledge. We believe that this makes a
usable tool for exploration and collaborative discussion of virtual museum artefacts an
exciting prospect for users across the whole spectrum of museum user groups.

It has been suggested that task environments have an outer boundary of what is vis-
ible to the person(s) carrying out the task, known as the ‘horizon of observation’ [8].
Naturally, how an interactive system makes use of this boundary, particularly for
collaboration, is going to have “consequences for the process of acquiring
knowledge” [8]. However, there are many situations where the horizon of observation
around an artefact is too limited to be able to fully explore or understand it. Artefacts
themselves are often enclosed and cannot be seen or touched from all sides; if passed
around between large groups (of museum visitors, for example), only smaller sub-
groups of people will have good access to it at any given time; online visitors may
only have access to still images via museum websites; and in remote-collaborative
situations, even if one collaborator has access to the artefact, the rest of the collabora-
tors will be limited to what is described to them by voice, shown to them as still im-
ages, or in the case of videoconferencing held up in front of a camera.

*RelicPad* expands this ‘horizon of observation’ in ways not normally possible with existing technologies, supporting real-time exploration of a virtual museum artefact and collaboration with others, and allowing users to easily focus, communicate and discuss ideas and theories about artefacts in a 3D context. As a mobile, tablet-based application, *RelicPad* is also adaptable to numerous collaborative scenarios, from passing one or more tablets around a large group of museum visitors to remote collaboration by way of real-time networked interaction. Putting the task ‘in the hands’ using the mobile context makes the ‘horizon of observation’ or ‘window’ to the artefact applicable to different groups of people working across varied contexts of use.

3 Related Work

As outlined in the introduction, handling physical museum artefacts is a powerful
experience, linking the handler to individuals and periods of time that they would
otherwise have no tangible contact with [9]. This is largely down to the sensory quali-
ties of artefacts – they are “a 3D experience”, more tactile than a still image or a re-
cording, and can be turned over and viewed from all sides [9], providing handlers
with “significant insight and understanding of the physical and material aspects of
objects” [1]. While simultaneously replicating all of the sensory qualities associated
with handling artefacts is not a realistic aim, tablets provide a method of displaying a
high-quality 3D rendering of a museum artefact, and of using different interaction
techniques to view it from all sides.

3D scanning of museum artefacts is now “a practical reality” [10], providing a
wealth of 3D museum content for exploration and discussion. The challenge is inter-
acting with this content in a way that better represents the manual exploration that
users expect from traditional object handling. Tangible interactions using devices
such as tablets and touchscreens might better represent certain interaction concepts
than a mouse is able to [11]. But while there has been plenty of research into interac-
tion techniques for the manipulation of 3D virtual objects, their application to mobile
devices such as smartphones and tablets remains a scarcely explored area.

Most interaction techniques developed “for stationary computers” are “not appli-
cable” to such devices [12], which seems a missed opportunity considering that the
rapidly improved display capabilities of smartphones and tablets have made the deliv-
ery of rich, interactive 3D content very achievable. The University of Virginia Art
Museum’s (UVaM) *Interactive iPad Museum Catalog* [13] allows users to view a
number of pre-selected artifacts from the museum’s collection as high-quality 3D
visualizations, but interaction itself is limited to dragging the finger on a touchscreen
to rotate around a single axis, and scaling. Explorations into manipulating 3D content
on mobile devices have experimented with concepts such as ‘tilting’ devices using
either computer-vision techniques [12] or built-in sensors such as gyroscopes, com-
passes and accelerometers [14], which have proven to be a promising alternative to
more traditional 2D interaction or touch techniques for 3D content rotation tasks.

But regardless of whether touch or tilt techniques are being used, we believe that the techniques used to manipulate virtual representations of objects should be based “on the real world” [15]. When the user is already skilled at performing the actions that underpin the basic operation of the system, the “mental effort required” for that operation can be significantly reduced [15], and so a strong representational metaphor of object handling, requiring minimal thought and leaving users free to focus their attention on the physical nuances of the object (and collaborative discussion) rather than how to manipulate it, should be considered as part of the interaction technique.

As important as the manipulation of the virtual museum artefact are the interac-
tions for supporting and organizing discussion between collaborators, particularly
remotely. Physical objects can play an important role in collaboration. As well as
using them to “complete their own activities”, collaborators often use objects to “co-
ordinate [these activities] in real-time with the conduct of others” [16], and it has been
suggested that collaboration “relies upon [collaborators’] mundane abilities to develop
and sustain mutually compatible, even reciprocal, perspectives” of their environment
and the objects within it [16]. Naturally, interaction with a virtual museum artefact
needs to support similar processes of understanding and referencing between collabo-
rators, who may well be connected remotely and unable to see each other.

Common tools for collaboration and discussion (such as *Skype* or *Windows Live Messenger*) allow users to share and exchange files, view each others’ screens, send instant messages, and communicate in real-time using both voice and video. Existing research into remote collaboration systems has shown tags, metadata and annotations to be useful in mutually focusing attention for 2D data (e.g. text, images and video). In museum informatics, distributed systems for sharing information about artefacts and collections are a well-researched area in their own right, with various web [17] and multi-media [18] based approaches that encourage and facilitate the distribution of that museum data across as wide a museum community as possible.
Visual media in such systems are generally 2D, and interactive content (beyond basic text, images and video) is limited. 2D interactions such as clicking images and following hyperlinks reflect this and there are few examples of these principles being used in 3D contexts, leaving these technologies with a very fixed horizon of observation that can only be pushed so far. However, we now have such a wealth of available technologies for displaying and interacting with 3D content that interactive systems for remote collaboration could be making far better use of the visual channels available to them, particularly where the spatial referencing of 3D objects is concerned.

Annotations are a common feature of collaborative technologies, and increasing interactivity has seen annotations move from being solely about “managing data and metadata” to becoming “critical” resources in “supporting communicative practice” [19]. The *Vannotea* system used annotations as ‘metadata stores’ to enable “the collaborative indexing, browsing, annotation and discussion of [video] content between multiple groups at remote locations” [20], while the *Kinected Conference* sees annotations used to convey users’ whereabouts in 3D, using video depth, audio cues and face-tracking algorithms to assign dynamic and interactive context tags to remote collaborators in a videoconference [21].

As well as creative and interactive uses of annotations for spatial referencing in remote collaboration, there are also examples of previous attempts to expand the ‘horizon of observation’ around 3D virtual content. *Lighthouse*, a remote-collaborative system for troubleshooting printer problems, used synchronized 3D representations of printers visible to both the customer and the troubleshooter, allowing troubleshooters to manipulate a shared pointer to highlight problems with the printer for the customer [22]. However, even systems like *Lighthouse* still use a mouse and keyboard (2D interfaces) as the input methods for interactions with 3D content, and so there is room to explore how collaboration, discussion and annotation can be integrated with 3D object manipulation to expand the horizon of observation around virtual museum artefacts for exploration and discussion by multiple (remote) collaborators.

# 4 Description of RelicPad

Research suggests that the actions generated by physical manipulation of tangible interfaces help to “draw up previous knowledge” and “generate important motoric representations to support other forms of representation” [11]. Based on the idea that manually manipulating virtual artefacts and marking interest points supports the exploration and discussion of objects (just as physically handling an object helps to build context) and supported by our review of the problem area and related work, our approach was to develop an initial prototype for *RelicPad* (described in this section) and then to identify, explore and evaluate design issues through a series of user studies (sections 5 & 6).
## 4.1 Overview of RelicPad Features

As well as the application to user groups from a variety of museum contexts outlined in our motivations, the clear benefits of being able to physically manipulate the interface influenced our decision to use a mobile, tablet-based platform as the interface technology for our research. Tablets also have a unified input and output space; the interaction and the resulting action happen in the same place (usually the hands), and so the observation viewpoint is the same as when handling a physical object.

RelicPad aims to provide a digital alternative to two key physical interactions – physically moving an object around in the hands, and pointing at different areas of objects. Taking into account the underlying aim of enabling users to share information about these 3D artefacts, RelicPad can be broken down into three fundamental elements that underpin the application:

* Manipulation (rotation and scaling) of the virtual museum artefact in 3D.

* Real-time marking of interest points in 3D space.

* Interactive conversation history.

These three elements offer digital representations of physical object handling, pointing, and organizing topics of discussion, enabling users to explore and discuss virtual museum artefacts in collaborative scenarios. Users can manipulate (rotate and scale) the virtual artefact using their hands to build up an understanding of it in 3D, can mark interest points to show each other where (and in what context) something interests them, and can refer to a conversation history to remind themselves of how conclusions were made and revisit earlier topics of interest. An example of how these elements come together in the interface can be seen in Figure 1:

<figure><img src="image_2.png" /><figcaption>**Fig.1** RelicPad interface; 3D representation of artefact (A), interest point (B), rotation button or ‘clutch’ (C), and interactive conversation history (D).</figcaption></figure>

**Manipulation of the Virtual Museum Artefact in 3D.**
In order to expand the 'horizon of observation' around virtual museum artefacts in ways that current applications do not, it is important to ground these digital interactions in reality-based movements that are more akin to handling physical objects. It is largely the 3D experience of handling physical objects – turning them over, looking inside them, viewing them from all angles – that prompts thought and understanding, helping the handler to make sense of the object [1]. Finding an appropriate interaction metaphor that gives users freedom and control over the manipulation (rotation and scaling) of the virtual museum artefact is important.

Our research has looked at two interaction metaphors for manually manipulating 3D objects using a tablet as the interaction device - tilt and touch (Figure 2):

<figure><img src="image_3.png" /><figcaption>**Fig.2** Two interaction metaphors – tilt (left) and touch (right).</figcaption></figure>

The tilt metaphor works by using the orientation of the tablet, given by the built-in accelerometer and gyroscope, to calculate the orientation of the 3D content, and represents the idea of using the hands simultaneously to grasp a physical object and rotate, twist, and turn it around. Visually, it carries a strong representation of moving a physical object with both hands. In contrast, the 'touch' metaphor works by orientating the 3D content according to the position of the user's finger(s) on the device's screen, representing something more akin to holding a physical object (in this case the tablet) in one hand, and using the other to rotate, twist or turn the object *within* that hand. Although more visually abstract than the tilt, this is a more familiar and traditional tablet interaction technique.

Section 6 describes the second study undertaken as part of this research, a comparison between four different techniques for rotating the 3D artefact (two techniques for each of the two interaction metaphors described above) and three different techniques for scaling the 3D artefact, and outlines each of these techniques in more detail.

**Marking of Interest Points.**

With physical objects, people point at them to provide each other with a clear frame of reference and to clarify areas of interest. Pointing is a clear visual gesture that draws attention to something that is deemed to be interesting, and marking interest points with *RelicPad* supports discussion of virtual museum artefacts in the same way. Users attach interest points to virtual museum artefacts that other users can see
for themselves in 3D space, in much the same way as people use pointing to guide others to what they want them to see during discussions about physical objects.

Marking interest points using *RelicPad* is achieved by tapping the screen on the area of the virtual museum artefact where the interest point is to be placed. As well as being a very familiar gesture to users of mobiles, tablets and touchscreens, the 'tap' gesture also resembles pointing in the physical and visual sense, making use of a single extended finger. Tapping the virtual museum artefact brings up a box menu with a choice of three possible 'context' icons (see *Interactive Conversation History* subsection below) plus a fourth 'cancel' icon in case the user decides not to leave the point after all. Tapping one of these icons leaves an interest point with the selected context in the desired location.

Above the choice of context icons is a text-field, allowing users to assign a keyword(s) to the interest point if they wish. Interest points themselves are offset slightly to the left or the right of the screen (depending on which side of the middle of the screen the user decided to leave the interest point) and linked to the virtual museum artefact by a line, so that when initially left they are not obscuring it.

## Interactive Conversation History.

The interest points left during a discussion come together to form an interactive 'conversation history'. An important classification tool for this conversation history is the context that can be used to define an interest when it is being marked. There are three context choices available, representing aesthetics (something about the way the object looks that the user finds interesting), geometry (something about the shape of the object that the user finds interesting), or meaning (something that the user feels provides clues about the cultural significance or idea behind the object).

In addition to (optional) user provided keywords, the name of the user who left each interest point and the time at which it was left, the classification of context (identifiable by the look of the interest point) allows for a historical record of the activity in discussions, conversations and collaborations to be kept, organized according to the interest points users have created. For collaborative use, this provides "a persistent record of interaction and collaboration" that can be easily referred back to [23], enabling users to revisit earlier interest points and "remind [themselves] of the process by which they reached previous interim conclusions" [19].

*RelicPad*'s conversation history consists of a scrollable menu in the top-right hand corner of the screen that stores all of the interest points in a discussion, adding each new point at the top of the list. As well as the point's context icon, each interest point record in the conversation history also contains the keyword given (if any), the name of the user that left the point, and the time at which it was left. This interactive but relatively traditional two-dimensional list leaves the conversation history in the periphery of the user's attention until it is needed – users can focus on their exploration and discussion of the artefact and on the physical interactions used to operate other *RelicPad* features, but can refer back to earlier points of discussion when needed.
# 5 First Study - Communication and Collaboration Using

RelicPad

Our first study aimed to get a feel for *RelicPad* as a user experience, and give an early indication of how useful it could be in context. The idea was to find out if participants could, by connecting to each other remotely using *RelicPad*, discuss an artefact together, share ideas about it, and draw each other's attention to its various features, in order to arrive at a conclusion about what the artefact is.

## 5.1 Experiment

We invited 22 participants, not selected based on any specific criteria, to take part in 11 collaboration sessions (2 ‘remote’ collaborators discussing the virtual museum artefact with each other in each session). Rather than specifically recruit participants from any of the previously described museum user groups, we opted for participants with no prior knowledge of the artefact so that as opposed to relying on any pre-existing knowledge of what the artefact might be, they would use interest points and spatial referencing to collaboratively generate and share ideas and understanding. This scenario of ‘exploring from scratch’ is applicable to any of the museum user groups, even curators and researchers who regardless of knowledge and experience will still be presented with new and exciting objects to make sense of from time to time.

Participants were each given an iPad running *RelicPad* with our 3D model of the QVMAG’s scrimshaw piece loaded as the virtual museum artefact. Prior to the experiment, we identified (in collaboration with QVMAG curators) 7 key features of the scrimshaw piece that we expected participants might be able to identify together. During the sessions, participants were encouraged to talk to each other, share ideas, and mark interest points in order to form conclusions about what the artefact is, where it came from, and the story behind it. We anticipated that our subsequent data collection would shed some light on how they had used *RelicPad’s* features to do this.

Each experiment lasted for approximately 30 minutes – 10 minutes introduction and explanation, 10 minutes for the collaboration session, and 10 minutes to complete the subsequent questionnaire. Participants were seated opposite each other at adjacent desks, with a large screen obscuring their views of each other – they were able to hear each other as they would during a phone call or videoconference, but were unable to see each other’s movements, gestures, or interactions with the tablet. The investigator was seated to the side of the participants, with a clear view of both of them for making observations.

Data was recorded as a sequential analysis of activity and interaction between the participants, accompanied by questionnaires. The sequential analysis consisted of three elements - an audio recording of each discussion session, a log of interactions recorded by the iPads and stored in an online database (recording when an interest point was left, as well as the context and any keywords that were given to it), and handwritten observations made by the investigator of notable exchanges between the participants during each session. These three elements were time synchronized to
paint a picture of how and why participants came to different conclusions using *RelicPad*, and how *RelicPad* features were used in this process.

The questionnaires, given to each participant after the discussion was over, consisted of two sets of ten questions adapted from the System Usability Scale (SUS), a widely accepted, “simple, ten-item scale giving a global view of subjective assessments of usability” [24], the answers from which can be used to generate a single number representing “a composite measure of the overall usability of the system being studied” [24]. The first ten questions focused on the five basic factors of usability (effectiveness, efficiency, safety, utility and learnability) while the second ten questions focused on five core components of user engagement – identity, adaptivity, narrative, immersion and flow [25]. The remaining questions were a mixture of closed and open-ended questions seeking additional feedback on how users felt about different *RelicPad* features, what they felt they had learned from the collaboration, and their overall impressions of *RelicPad* as an experience.

During each session, participants were each given an iPad running a different version of the prototype. One participant was always using the tangible ‘tilting’ interaction method for rotating the virtual museum artefact, while the other participant was always using a more traditional 4-button ‘directional pad’ for rotating it. The tilt rotation was rate-controlled (the virtual museum artefact rotates faster according to how far the tablet is tilted), with a rotation button or ‘clutch’ held to initiate and released to cease rotation. The 4 directional buttons were used to rotate the artefact up or down around the x-axis, or left and right around the y-axis.

Neither participant knew that their collaborator was using a different technique to rotate the virtual museum artefact. The idea behind this was to evaluate whether or not there was any difference in how usable or engaging participants found *RelicPad* based on whether or not they used a physical, tangible rotation method, or a more traditional 2D interaction technique.

## 5.2 Results

Observations and audio recordings showed that during the 11 collaboration sessions, pairs of participants were able to arrive on average at 4.45 of the previously identified 7 conclusions about the scrimshaw. Certain things were particularly well noticed by participants, including faded lettering on the back of the scrimshaw (a focal point of the discussion in 10 of the 11 sessions) and the cracks in and discoloring of the scrimshaw (focal points of the discussion in 9 of the 11 sessions).

The combination of observations and activity logs gave an indication of how interest points are used during collaboration. On average, 12 interest points are left during each of the 11 sessions. Participants on average will specifically refer to 48% of these interest points, whether that be to tell their collaborator that they have just left (or are about to leave) the interest point, to tell them exactly what part of the artefact it is being attached to, or to give them directions to help them find out where it is. Participants also on average ask their collaborator for clarification or an explanation of 13% of interest points; asking where an interest point is, what it was supposed to be attached to, or whether they are looking at the correct one.
Questionnaire responses revealed a positive response to *RelicPad*. Although 8 out of the 22 participants unfortunately did not give a definitive answer to the question, 12 of the 14 participants who did respond described *RelicPad* as a positive experience. When asked to provide additional comments about the experience, 13 out of the 22 participants commented on the application being good for remote collaboration, conversation, and positional referencing, with 10 participants specifically referring to its suitability for interacting with artefacts in museum or educational contexts.

The first ten and second ten sets of questions were used to calculate single numbers for usability and user engagement, as described in the previous section. In both cases the mean scores were good – 70.6 for usability and 73.07 for engagement. Participants were asked to rate *RelicPad* interactions based on how frequently they felt they used them (1 being very infrequently and 5 being very often). Results showed that on average participants felt that they had marked interest points very frequently (4.05), often added keywords to them (3.09), and occasionally chosen to change the current interest point (2.77) (to refer back to something from earlier in the discussion).

Participants were asked to rate how positive or negative they felt (5 being very positive) about the three fundamental elements that, as previously mentioned, combine to create the overall *RelicPad* experience – rotation of the virtual museum artefact in 3D, real-time marking of interest points, and the interactive conversation history. On average, the marking of interest points was the most popular feature among the participants (3.95), while rotation of the virtual museum artefact (3.82) and the interactive conversation history (3.32) were also both well received.

# 6 Second Study – Techniques for Manipulating Virtual Museum Artefacts

Separating the results of the first study according to which rotation technique was used, we found that there was no significant statistical difference between the tilt and directional button techniques, and conclusions about which technique was more usable or affected the experience differently to the other could not be drawn. Two-tailed independent t-tests proved this to be the case for basic usability (t=-.981, df 15.6, P=.342), user engagement (t=-.612, df 20, P=.547), ease of rotation (t=-1.64, df 20, P=0.116) and overall impressions of rotation (t=-2.01, df 20, P=0.058).

However, ‘touch’ and ‘drag’ interaction techniques (well established interactions with tablet devices) and a direct (one-to-one) mapping of tilt to rotation (as opposed to rate-controlled) were both suggested as ways of improving the rotation technique. This was interpreted as an indication that while the rate-controlled tilt was not significantly worse than a more traditional 2D interaction, it was not significantly better, leaving room for further exploration of interaction techniques and metaphors that might better represent the physical exploration associated with object handling.

Our second study compared four rotation techniques for manipulation of a 3D virtual object – two making use of the ‘tilt’ metaphor and two making use of the ‘touch’ metaphor (both metaphors are outlined in section 4). The two ‘tilt’ techniques were a rate-controlled tilt (the angle at which the device is tilted defines the speed at which
the virtual object rotates) and a direct-mapping tilt (the virtual object's rotation follows the angle at which the device is tilted exactly). The two 'touch' rotation techniques were a 'virtual trackball' implementation (enclosing the virtual object in a sphere which is dragged with a single finger in order to rotate, described as a 'virtual sphere' in [26]) and a 'multi-touch' approach (dragging with a single finger to rotate the artefact on the x and y axes, and rotating two fingers clockwise or counterclockwise in order to rotate the artefact on the z axis).

Three techniques for scaling the virtual museum artefact were also compared – a ‘plus & minus buttons’ approach (holding one of two buttons to increase or decrease the scale of the artifact), a ‘slider bar’ approach (continuous dragging between the two end points on a slider bar widget to increase or decrease the scale), and a ‘multi-touch’ approach (increasing or decreasing the distance between two fingers, also known as ‘pinching’ and ‘spreading’, to increase or decrease the scale of the artefact).

## 6.1 Experiment

12 participants took part in a set of ‘object-matching’ trials, using the different rotation and scaling techniques to match a virtual museum artefact (the scrimshaw piece from the first study) with a semi-transparent target orientation of the same artefact. The rotation techniques were organized according to a balanced Latin-square design (to minimize biases caused by practice or fatigue), and for each technique there were 18 trials – 6 with each of the three scaling techniques. Of those 6 trials there were 2 simple, 2 medium and 2 complex rotation difficulties. Difficulty was defined as being whether matching the virtual artefacts required rotation on one (simple), two (medium), or all three axes. This makes a total of 72 trials per participant.

For each trial, participants’ speed (time taken to complete the trial), rotation error (difference between virtual and target artefacts in degrees) and scale error (difference in size of the virtual and target artefacts represented as vectors) were recorded. Participants were asked to think about each trial in terms of both speed and accuracy and move onto the next trial as soon as they were happy. However, to keep things moving and prevent the experiment from taking an unreasonable amount of time, participants were asked to move onto the next trial after around 90 seconds.

In between each rotation technique (every 18 trials) participants were asked to answer some questions about that technique, and after all of the trials had been completed to answer questions looking back on all of the rotation and scaling techniques together. These questions asked participants to rate different aspects of the various techniques numerically, or to provide a few short sentences on the techniques.

The design of the experiment was based on a number of similar object-matching experiments from past research into techniques and technologies for 3D object manipulation, from early explorations with desktop-based VR systems [26] [27] [28], to more recent approaches to manipulating 3D content with mobile devices [12] [14] and touch displays [29].
## 6.3 Results

Table 1 shows means and standard deviations (SD) for how quickly and how accurately participants were able to complete the trials, on average, using each of the four rotation techniques – Rate-Controlled (RC) Tilt, Tilt with Direct Mapping (DM), the Virtual Trackball (VT) touch technique, and Multi-Touch. Touch techniques performed better than tilt techniques, being both quicker and more accurate to use:

<table><caption>**Table 1.** Speed and accuracy of the four compared rotation techniques.</caption><thead><tr><th>Averages</th><th>Tilt (RC)</th><th>Tilt (DM)</th><th>Touch (VT)</th><th>Multi-Touch</th></tr></thead><tbody><tr><td>Time Taken<br>(seconds)</td><td>58.03<br>(SD 13.32)</td><td>51.75<br>(SD 14.59)</td><td>49.72<br>(SD 18.06)</td><td>47.99<br>(SD 10.84)</td></tr><tr><td>Rotation Error<br>(degrees)</td><td>12.22<br>(SD 9.92)</td><td>11.81<br>(SD 14.07)</td><td>6.38<br>(SD 3.91)</td><td>7.47<br>(SD 4.49)</td></tr></tbody></table>

Analysis of variance showed the effects of rotation technique on both the time taken to complete the trials (F = 3.966, df 3, 68, P = 0.011) and on the average rotation error (F = 4.278, df 3, 68, P = 0.008) to be statistically significant.

A comparison of how quickly and accurately trials could be completed, on average, using each of the scaling techniques is shown in table 2 (means and standard deviations (SD)). Here, the touch technique performed better than its 2D counterparts, proving to be both the quickest and the most accurate technique:

<table><caption>**Table 2.** Speed and accuracy of the three compared scaling techniques.</caption><thead><tr><th>Averages</th><th>+/- Buttons</th><th>Slider Bar</th><th>Multi-Touch</th></tr></thead><tbody><tr><td>Time Taken<br>(seconds)</td><td>56.08<br>(SD 9.73)</td><td>50.91<br>(SD 7.34)</td><td>48.62<br>(SD 10.39)</td></tr><tr><td>Scaling Error<br>(size difference)</td><td>7.03<br>(SD 3.93)</td><td>6.63<br>(SD 4.07)</td><td>4.65<br>(SD 1.70)</td></tr></tbody></table>

Analysis of variance shows the effects of scaling technique on the time taken to complete the trials to be of statistical significance (F = 3.857, df 2, 69, P = 0.026), but the effects of scaling technique on the average scaling error were minimal.

Comparing all combinations of rotation and scaling techniques together supports these results, with the combination of Multi-Touch rotation and Multi-Touch scaling resulting in the fastest average trial completion time at 41.61s (see Figure 3), the smallest average scaling error (3.93), and the second-smallest average rotation error (6.10 degrees). The worst combination was Tilt (Rate-Controlled) rotation with Plus & Minus Buttons for scaling, which gave the slowest average task completion time (64.65), largest average rotation error (19.69), and largest average scaling error (8.66). However, analysis of variance showed that apart from the Tilt (Rate-Controlled) rotation with Plus & Minus Buttons for scaling being by far the poorest combination of techniques, the differences between the rest of the combinations were minimal, and not of statistical significance.
<figure><img src="image_4.png" /><figcaption>**Fig. 3** Graph showing average time taken to complete trials (in seconds) for each rotation technique, separated by scaling technique.</figcaption></figure>

The questionnaires asked participants to rank the four rotation techniques in order from 1 (best) to 4 (worst) in relation to a number of different criteria, including: ease of rotation; perceived accuracy of rotation; perceived speed of trial completion; enjoyment; understanding of the movement of the artefact in 3D; perceived control over the artefact; and favourite technique. On average, Multi-Touch recorded the lowest (best) average (between 1.42 and 1.75) for all of the criteria, while Tilt (Rate-Controlled) recorded the highest (worst) average (between 2.92 and 3.33) for all of the criteria. Participants were also asked to rank the three rotation techniques in order from 1 (best) to 3 (worst) in relation to the same criteria – Multi-Touch scaling also recorded the lowest (best) average (between 1.42 and 1.92) for all of the criteria.

# 7 Discussion

Based on the results of the two studies, we have come to our own interpretations of how users have received our application, how applicable it is to the described problem domain, and which interaction metaphors give users the most control over the manipulation of virtual museum artefacts. We found that participants used interest points to share spatial references, that they could collaboratively construct solid ideas of what the artefact was, and that they felt positive about being able to rotate the virtual artefact in 3D, suggesting that *RelicPad* can facilitate exploration and collaborative discussion of virtual museum artefacts. We also found that touch interaction techniques leave users feeling in control of the manipulation of the virtual museum artefact.

In the first study, marking interest points proved to be the most popular and commonly used feature of *RelicPad*. Almost half of the interest points left were directly referred to by participants verbally, usually to explain what the interest point refers to as it is being left ("that there, it's a parasol"; "and you can see in this section here that
it's lighter”; “down below there, that marking”; “it's broken, on your left side – I'll
point it out for you”). This indicates that participants used the interest points for the
purpose we intended – ‘pointing out’ interesting areas or features of the artefact to
assist each other in maintaining a shared spatial reference of the topic of discussion.

Importantly, participants noticed and discussed most of the things that were identi-
fied prior to the experiment as providing the history, story and significance of the
scrimshaw piece. This indicates that participants not only enjoyed marking interest
points, but that they used them for the intended purpose and that this bore results in
terms of the collaboration. This is especially interesting given that to most participants
it was not obvious from the start that the artefact was scrimshaw; it was through dis-
cussion and theorizing with each other that they were able to come to conclusions
about what it was or wasn't. It did appear, however, that there was a tendency to no-
tice visual stimuli, however difficult to see, but not always to fully establish what they
represented. The lettering on the back of the scrimshaw was discussed at length in 10
out of the 11 sessions, but in only 3 of those 10 was it agreed that they could be the
initials of the creator (or the recipient) of the scrimshaw.

The first study compared the rate-controlled tilt technique for rotating the virtual
museum artefact against a directional button widget. With the initial design, the tilt
was seen as being a strong visual representation of moving a museum artefact with
the hands, but the difference in rotation method used seemed to have little impact on
the usability of the application or what participants were able to make of the experi-
ence of interacting with the virtual artefact. The second study looked to explore alter-
native interaction metaphors for manipulating the virtual artefact, and to see what
happened when scaling was brought into the picture as well as rotation.

Object matching trials comparing combinations of the different rotation and scaling
techniques showed that touch techniques, particularly ‘multi-touch’ techniques, per-
formed significantly better and were more popular than tilt techniques. Virtual objects
could be rotated and scaled more quickly using touch techniques, and also with more
accuracy. Participants also reported enjoying using them more, having a better under-
standing of how the virtual object moves in 3D using them, and crucially having more
control over the virtual museum artefact.

A number of factors may contribute to this – tablets have weight, and physically
rotating them for long periods of time was tiring for some participants. The issue of
viewpoint may also play a part here – with touch techniques the observation view-
point is fixed while the hands manipulate the virtual objects (more akin to kind of
viewing angles associated with physical object handling), while with the tilt it could
be thought of as being the opposite, with the hands moving the observation viewpoint
as opposed to the virtual object itself.

We interpret this, coupled with the ability to accurately ‘fine-tune’ the position of
the virtual artefact, as an indication that while the tilt metaphor carries a stronger vis-
ual representation of moving a physical object with the hands, the touch metaphor
gives the user more control. By leveraging interactions styles that are familiar to
many users, touch techniques and the control they provide leave users free to focus
their attention on the content and the collaboration itself, rather than the techniques
required to interact with the virtual artefact. As such, we believe that the touch meta-
phor provides a more reality-based representation of natural and relatively thoughtless interaction with the hands for exploring the spatial properties of (virtual) artefacts.

# 8 Conclusions

Our interpretation of results shows that using interest points to draw attention to notable features of a virtual museum artefact helps users to understand and clarify areas of particular interest, and that over the course of a collaborative discussion these interest points are used to drive conversations and exchange ideas and theories. Participants in our study appeared to relish the ability to use the touch-tapping interaction to ‘point’ at artefacts and to highlight areas of interest for each other.

Comparing different metaphors for the manipulation of virtual museum artefacts suggested that established tablet interaction techniques such as multi-touch are the most efficient and enjoyable for controlling the manipulation of virtual museum artifacts on a tablet, leveraging skills that users are already familiar with to comfortably control the 3D experience and freeing them to focus their attention on other aspects of the system, such as marking and discussing points of interest.

We interpret this as an indication that for interaction techniques to feel ‘reality-based’, familiarity and control (moving an object in the hands comes naturally, as do touch techniques when using tablets) are perhaps more important than a strong visual representation (tilting the tablet looks more like handling an object, but it isn’t the first thing people think to do with a tablet). This suggests an interesting compromise between how techniques look and how they feel when designing interactions.

As well as being well received as a positive experience by many participants, feedback showed that many of those participants saw the suitability of the application in museum contexts, specifically commenting that the application was good for remote collaboration and spatial referencing. *RelicPad* was able to expand each participant’s horizon of observation in relation to both the virtual museum artefact itself and also to the focal points of each other’s attention, and was used by participants to good effect. With the right interaction metaphors in place, it offers a promising solution to the scenarios presented in the introduction and motivation sections, and the example scenario provided by staff at the QVMAG.

## 8.1 Future Work

The experiments highlighted a number of interesting directions for the design of *RelicPad*. Many felt that the interest points themselves could have been more interactive – navigated to by touch, editable (renaming and deletion), more easily identifiable (either by keyword or by participant-based colour coding), and linked together to form a conversation thread. Combining the elements of interactivity from the conversation history into the interest points themselves may encourage collaborators to go beyond what they see of artefacts and to discuss further what those things might represent, giving spatial reference not only to areas of interest but to topics of interest as well.
In future experiments we also plan to evaluate whether the different museum user
groups (or combinations of these groups) outlined earlier in the paper have different
levels of success than others in coming to conclusions about artefacts whilst using the
system. We will also compare the outcomes of collaborations made using *RelicPad*
against those of face-to-face, co-located collaborations with the physical artefact to
hand, and of existing remote-collaborative chat and video-conferencing alternatives
such as *Skype*.

Acknowledgements

Thanks go to Andrew Johnson and Jon Addison from the QVMAG for their enthu-
siasm and for permission to use one of their artefacts during the experiment, and to
the Human-Computer Interaction research group at the University of Konstanz, Ger-
many for help and resources during the second study. Thanks also to Bruce Andrews
of the HIT Lab AU for his help with programming and development.

References

1. Boyes, A., Cousens, C.: In the Hand - The Effect of Museum Handling Sessions on Student Learning. In: Networks, Art Design Media Higher Education Academy Subject Centre, University of Brighton (2012)

2. Queen Victoria Museum & Art Gallery, http://www.qvmag.tas.gov.au/qvmag/

3. Wikipedia. Scrimshaw, http://en.wikipedia.org/wiki/Scrimshaw

4. Everitt, K.M., Klemmer, S.R., Lee, R., Landay, J.A.: Two Worlds Apart: Bridging the Gap Between Physical and Virtual Media for Distributed Design Collaboration. In: Conference on Human Factors in Computing Systems, pp.553-550. ACM Press, New York (2003)

5. Bureau of Labor Statistics, Curators, Museum Technicians, and Conservators, United States Department of Labor, http://www.bls.gov/ooh/Education-Training-and-Library/Curators-and-museum-technicians.htm, (2012)

6. National Museum of Australia, Research and Museology, National Museum of Australia, http://www.nma.gov.au/history/research.

7. Simon, N.: What's a Virtual Visitor Worth?, Museum 2.0, http://museumtwo.blogspot.de/2008/12/whats-virtual-visitor-worth.html (2008)

8. Hutchins, E.: Cognition in the Wild. MIT Press, Cambridge (1995)

9. Mastoris, S.: Learning Through Objects. Group for Education in Museums, http://www.gem.org.uk/res/advice/ball/res_lto.php

10. Smithsonian Institute, MCI Imaging Studio, Smithsonian Museum Conservation Institute, http://www.si.edu/MCIImagingStudio/3DScanning, (2012)

11. Manches, A., Price, S.: Designing Learning Representations Around Physical Manipulation: Hands and Objects. In: 10th International Conference on Interaction Design and Children, pp.81-89. ACM Press, New York (2011)

12. Henrysson, A., Marshall, J., Billinghurst, M.: (2007). Experiments in 3D Interaction for Mobile Phone AR. In: 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia, pp.187-194. ACM Press, New York (2007)
13. Put 3D Objects at your Visitors' Fingertips: UVaM on the iPad, Idea.org, http://www.ide.org/blog/2011/11/29/put-3d-objects-at-your-visitors-fingertips-uvam-on-the-ipad/ (2011)

14. Kratz, S., Rohs, M., Guse, D., Müller, M., Bailly, G., Nischt, M.: PalmSpace: Continuous Around-Device Gestures vs. Multitouch for 3D Rotation Tasks on Mobile Devices. In: International Working Conference on Advanced Visual Interfaces, pp.181-188, ACM Press, New York (2012)

15. Jacob, R.J.K., Girouard, A., Hirshfield, L.M., Horn, M.S., Shaer, O., Solovey, E.T., Zigelbaum, J.: Reality-Based Interaction: Unifying the New Generation of Interaction Styles. In: Conference on Human Factors in Computing Systems, pp.2465-2460, ACM Press, New York (2007)

16. Hindmarsh, J., Fraser, M., Heath, C., Benford, S. & Greenhalgh, C.: Object-Focused Interaction in Collaborative Virtual Environments. ACM Transactions on Computer-Human Interaction 7, pp.477-509 (2000)

17. Goodman, C., Schmitz, P., Sheppard, D., Clark, C.: Architecting CollectionSpace: A Web-Based Collections Management and Information System for 21st Century Museum Professionals. In: Museums and the Web, Archives & Museum Informatics, Toronto (2010)

18. Oomen, J., Baltussen, L.B., van Erp, M.: Sharing Cultural Heritage the Linked Open Data Way: Why You Should Sign Up. In: Museums and the Web, Archives & Museum Informatics, Toronto (2012)

19. Fraser, M., Hindmarsh, J., Best, K., Heath, C., Biegel, G., Greenhalgh, C., Reeves, S.: Remote Collaboration Over Video Data: Towards Real-Time E-Social Science. Computer Supported Cooperative Work 15(4), pp.257-279 (2006)

20. Schroeter, R., Hunter, J., Kosovic, D.: Vannotea – A Collaborative Video Indexing, Annotation and Discussion System for Broadband Networks. In: Knowledge Markup and Semantic Annotation Workshop, 2nd International Conference on Knowledge Capture, ACM Press, New York (2003)

21. DeVincenzi, A., Yao, L., Ishii, H., Raskar, R.: Kinected Conference: Augmenting Video Imaging with Calibrated Depth and Audio. In: Conference on Computer-Supported Cooperative Work, pp.621-624, ACM Press, New York (2011)

22. O'Neill, J., Castellani, S., Roulland, F., Hairon, N., Juliano, C., Dai, L.: From Ethnographic Study to Mixed Reality: A Remote Collaborative Troubleshooting System. In: Conference on Computer-Supported Collaborative Work, pp.225--234, ACM Press, New York (2011)

23. Stahl, G., Koschmann, T., Suthers, D.D.: Computer Supported Collaborative Learning. Cambridge University Press, Cambridge (2006)

24. Brooke, J.: SUS - A Quick and Dirty Usability Scale. Usability Evaluation in Industry, pp.189-194 (1996)

25. Benyon, D.: Designing Interactive Systems – A Comprehensive Guide to HCI and Interaction Design. Pearson, Harlow (2010)

26. Chen, M., Mountford, S. J., Sellen, A.: A Study in Interactive 3-D Rotation Using 2-D Control Devices. Computer Graphics 22(4), pp.121-129 (1988)

27. Hinckley, K., Tullio, J., Pausch, R., Proffitt, D., Kassell, N.: Usability Analysis of 3D Rotation Techniques. In: 10th Annual ACM Symposium on User Interface Software and Technology, pp.1-10, ACM Press (1997)

28. Poupyrev, I., Weghorst, S., Fels, S.: Non-Isomorphic 3D Rotational Techniques. In: Conference on Human Factors in Computing Systems, pp.540-547, ACM Press (2000)

29. Martinet, A., Casiez, G., Grisoni, L.: The Effect of DOF Separation in 3D Manipulation Tasks with Multi-Touch Displays. In: 17th ACM Symposium on Virtual Reality Software and Technology, pp.111-118, ACM Press (2010)