# Semiautomatic and User-Centered Orientation of Digital Artifacts on Multi-touch Tabletops

Lorenz Barnkow, Kai Von Luck

► To cite this version:

Lorenz Barnkow, Kai Von Luck. Semiautomatic and User-Centered Orientation of Digital Artifacts on Multi-touch Tabletops. 11th International Conference on Entertainment Computing (ICEC), Sep 2012, Bremen, Germany. pp.381-388, 10.1007/978-3-642-33542-6_34. http-01556129

HAL Id: hal-01556129

https://inria.hal.science/hal-01556129

Submitted on 4 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Semiautomatic and User-Centered Orientation of Digital Artifacts on Multi-Touch Tabletops

Lorenz Barnkow and Kai von Luck

Department of Computer Science, Hamburg University of Applied Sciences,
Berliner Tor 7, 20099 Hamburg, Germany
lorenz.barnkow@haw-hamburg.de, luck@informatik.haw-hamburg.de

**Abstract.** The orientation of objects on tables is of fundamental importance for the coordination, communication and proper understanding of content in group work. Similarly, the roles of orientation have to be taken into account when implementing software for multi-touch tabletops. This paper describes a combined approach to help with the orientation of artifacts, composed of both automatic and manual orientation methods. Using a custom test application, this study investigates the effects of automatic orientation of artifacts towards users.

**Keywords:** Multi-touch tabletops, Orientation of artifacts, territoriality, group work

## 1 Introduction

The growing availability of commercial and low-cost multi-touch hardware in the recent years has led to increased research interest in this area [1]. As a result, several new techniques have been presented to support individual and collaborative work on multi-touch tabletops. In addition to supporting collaborative work, interactive tabletops are also suitable as a base of digital conversions of classic board games [2] or as novel input devices for computer games (e. g. real-time strategy [3]). The spatial proximity of people at tables allows for direct eye contact, communication using speech and gestures as well as direct perception of what is happening on the table [4]. One of the key challenges to support groups on multi-touch tabletops is the correct orientation of digital artifacts to meet the different perspectives of the participants arranged around the table [5]. Numerous techniques for the technical implementation of the orientation of digital artifacts can be found in the literature of the HCI and CSCW fields. In this paper, a combination of automatic and manual orientation is presented, which reduces the manual effort while also providing a high degree of flexibility. In a preliminary evaluation both the facilitation of the work and the acceptance of this solution was examined quantitatively and qualitatively.

## 2 Related Work

While working at tables the people involved usually show a distinctive territorial behaviour, partitioning the table into three regions: personal, group and storage
territories [6]. The group territory refers to the shared table space in general, while the areas immediately in front of each person are called personal territory in which personal and private tasks are performed. Storage territories are mainly used to store artifacts for later use or to pass stacks of artifacts to other people [4]. The orientation of artifacts on the table serves three roles: comprehension, coordination and communication [5]. Comprehension refers, among other things, to the readability of text, which may decrease significantly in case of poor orientation [7]. Having artifacts face users supports coordination in such a way that personal territories and ownership become visible. Likewise, the active reorientation of artifacts towards one or more users usually initiates a communicative exchange.

Methods for the orientation of artifacts on multi-touch tabletops can be divided into three categories: manual, automatic and combined. Using means of manual reorientation each person is given the freedom to specify an arbitrary orientation for every artifact. These methods include explicit definition of angles, use of handles, physics-based approaches and multi-touch gestures [8]. All of them generally offer the same flexibility on multi-touch tabletops that would be expected from the real world, but they differ in terms of learnability and usability. On multi-touch tabletops, work can also be facilitated in such a way that the system automatically performs reorientation of the artifacts and thus relieves the users from this task. Such automatic orientation can be performed towards the spatially nearest table edge, based on established territories [9], towards the table edge of the spatially nearest user [10] or based on the position of the users [11]. However, entirely automatic orientation is rarely optimal [12] and negates the communicative role of orientation. Combined (or semiautomatic) methods complement the support of automatic systems with the flexibility of manual reorientation. One example is the use of vector fields and gestures to manipulate these vector fields [12]. However, since only regions rather than artifacts can be manually reoriented, the orientation is not preserved when artifacts are moved thus its coordinating role is compromised [5].

The presented automatic and combined methods pose difficulties regarding the flexibility, awareness as well as the communicative and coordinating roles of orientation. A facilitation of work due to automatic methods was elaborated on in the literature [12], yet it has not been proven or measured. On the contrary, one study suggests that a fully automatic orientation offers no measurable speed advantage over manual orientation and also restricts the freedom of users [11]. This paper presents a combined method of orientation, which was evaluated in a preliminary study to measure the facilitation of work and assess the usability based on the data collected from logs, questionnaires and interviews.

# 3 Evaluation of Semiautomatic Artifact Orientation

The semiautomatic orientation method examined in this paper combines automatic orientation towards the spatially nearest user with multi-touch gestures for manual rotation of digital artifacts. To reflect the previously described ter-
ritorial behavior in the system, a personal territory at the table edge is created for each user, similar to [13] (see Fig. 1). However, these personal territories are each associated with a different color and are manually created, moved and removed. By default, the digital artifacts (i. e. interactive widgets such as pictures or news articles) on the tabletop are automatically oriented towards a focal point relative to the spatially nearest personal territory, thus facing the appropriate user, and assume the color of that personal territory in order to strengthen the perception of ownership (see Fig. 1). At any time and for each artifact this automatic behavior may explicitly be turned on or off using a button on the artifact or be overridden by manually rotating an artifact using multi-touch gestures. Manually rotated artifacts do not change their orientation by themselves, unless the automatic orientation feature for this artifact is switched on again by the user. Also mapping the color of an artifact to the corresponding personal territory of its owner is only carried out with activated automatic orientation. The colors also indicate whether the automatic orientation is activated, thus make the current mode (see [14]) visible at all times for each interface item.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Screenshots of a personal space (left) and automatic orientation towards the user (right).</figcaption></figure>

**3.1 Participants**

Ten groups with three unpaid participants each (24 males, 6 females) were recruited. The subjects were between 18 and 45 years old and 24 of them were students. All used computers frequently and most of them (86%) used touchscreens frequently, though few (13%) had experience with large digital tabletops.

**3.2 Apparatus**

The experiments were performed on a custom built, rear-projected multi-touch tabletop with an interactive surface of about 200cm × 65cm and a screen resolution of 2560 × 800 pixels (see Fig. 2). The connected computer runs Windows 7 and the test application was built using C# and WPF. In addition to capturing interaction logs, all sessions were recorded using three cameras.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Experimental setting</figcaption></figure>

3.3 Task

To evaluate the semiautomatic orientation method, an appropriate task was pre-
pared, based on a realistic use case. Based on an interview the workflows in an
editorial office were studied, with particular focus on group work involving two
or more persons. These included firstly joint sifting through potential pictures
for news articles, and secondly an editorial meeting in which the status of the
upcoming issue was reviewed and discussed. The developed test application im-
plemented similar use cases loosely based on these workflows and provided tools
to browse for news articles and pictures on the internet, using Google News,
Flickr and Google Images. The search tools also offered a function to create
synchronized and connected copies of these widgets, so that, for example, search
results could be jointly browsed. Furthermore, a newspaper widget was made
available, which could be compiled into a virtual magazine, using drag-and-drop
gestures (see Fig. 3).

In this study, the participants worked in groups to create a newspaper. This
task included both loosely coupled phases during which the participants worked
on their own as well as tightly coupled phases with strong emphasis on commu-
nication and the exchange of artifacts. There were no explicit role assignments,
thus all participants had equal rights during the tightly coupled phases.

During the first part of the task every participant was instructed to search
for three news articles on two given topics (e. g. “sports” and “entertainment”)
and pass these articles on to their neighbors in counterclockwise direction. Upon
receiving articles from their neighbors each participant reviewed the articles and
selected one article for each subject, discarding the rest. The remaining articles
were presented and discussed to determine the sequence in which they should
be added to the newspaper. This initiated the second part of the task, during
which every participant searched one picture for each article of the newspaper.
These pictures were also presented to the group and discussed to jointly choose
the most appropriate picture for the corresponding article.
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Dragging a picture from the search tool into the newspaper widget (left) inserts it into the corresponding page (right).</figcaption></figure>

## 3.4 Procedure

Each group performed the aforementioned task in two configurations which differed in the given topics for the newspaper and whether the automatic orientation was available or not. Using questionnaires before the experiments, the demographic characteristics of the subjects and their computer skills were gathered. In addition, all groups received an introduction to the system prior to the experiments and then had 10-15 minutes to test it on their own and ask further questions. Due to the limited scope of the application and small number of tools and widgets available to the user, this initial trial phase could be kept relatively short. The combined amount of time spent working on both task configurations was between 40 and 100 minutes (*mean* = 69.5, *SD* = 21.7). With further questionnaires after the experiments, the usability and collaboration at the table was assessed. The experiments were concluded with an interview with each group.

# 4 Results

## 4.1 Effects of Automatic Orientation on the Spatial Use of the Tabletop

Figure 4 shows an example of the spatial use of the tabletop by a group in both test conditions based on the logged interactions. This plot illustrates that the use of automatic orientation has no apparent effect on the spatial use of the tabletop, in contrast to the available space, which visibly restricts the operating radius of the two users on the right side of the plot. This indicates a lack of table size, which was criticized in later interviews as well. The same usage pattern could be seen for all 10 groups across the experiments.

## 4.2 Facilitation of Work

Based on a statistical analysis of the interaction logs, the facilitation of work by the automatic orientation was measured. For this, the interactions were distinguished as to whether they involved a manual rotation gesture or not. For
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Use of the tabletop by region using automatic orientation (left) or manual rotation only (right).</figcaption></figure>

the task condition using the combined method for orientation (task condition 1) an average of 15.4% of all interactions also involved manual rotation. Within the task condition using only manual rotation (task condition 2), the share of rotational gestures was, on average, at 33.8%, thus more than twice as high. Figure 5 shows that this difference was visible across all groups and proved to be statistically significant ($F_{1,18} = 45.041; p < 0.00001$).

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Use of manual rotation during both task conditions</figcaption></figure>

## 4.3 User Experience

The postexperiment questionnaire consisted mainly of statements with a 6-point Likert scale (*fully agree* to *fully disagree*). Further open questions were omitted because of the subsequent interview. Overall, the subjects rated the operation of the test application to be easy (*mean* = 2.1; *SD* = 0.9). This was also reflected in the interviews, where the direct touch based interaction was described as an intuitive and fun experience. The assignment of colors to indicate ownership was both intuitive (*mean* = 1.2; *SD* = 0.5) and useful (*mean* = 1.8; *SD* = 1.1), which was reaffirmed in the interviews. The manual rotation of digital artifacts was easy to perform (*mean* = 1.9; *SD* = 1.0), while the automatic orientation was both useful (*mean* = 2.2; *SD* = 0.9) and satisfactory (*mean* = 2.2; *SD* = 0.9). The interviews revealed broad support for the automatic orientation, since it could be overridden at any time and thus offered both flexibility and facilitation of work. The automatic orientation was found particularly helpful while passing digital artifacts between the users. However, the continuous orientation towards
a focal point proved to be unpopular, because artifacts were slightly reoriented with every movement. Firstly, some users felt this was a loss of control and secondly they thought a perpendicular alignment (with respect to the table borders) might be more orderly and space-saving.

# 5 Conclusions and Further Work

Given the falling prices for commercial multi-touch hardware (e. g. Samsung SUR40) and its growing number of users, multi-touch tabletops present a great potential for a wide range of applications in very different areas, such as professional collaboration, learning together and entertainment software in both private and public spaces (e. g. in bars or hotel lobbies). This paper presents a combined method for orientation of digital artifacts on multi-touch tabletops and analyzes its effects on the spatial use of the tabletop, the facilitation of work and the user experience through a preliminary study. The analysis of interaction logs showed no discernible effect on the spatial use of the available table space, with and without automatic orientation. A statistical analysis revealed that the share of manual rotation gestures is doubled when no automatic orientation is available. The survey of test subjects by questionnaires and interviews indicated a high acceptance for the combined orientation method. The automatic orientation especially helped in the transfer of artifacts between the users, since they were able to concentrate on the actual task at hand. This might help to improve the usability and user experience of collaborative work on digital tabletops including joint browsing and organizing holiday pictures [15] or exchanging and sharing resources in digital board games [2].

Because of the hardware used it was not possible to unambiguously assign every touch contact to the right user. At this point, further investigations are needed because the facilitation of work could not be proven for each individual person. Moreover, the experimental tasks included both loosely and tightly coupled phases, meaning it is unclear whether the combined approach for orientation is appropriate in both situations.

# References

1. Schning, J.: Touching the future: the rise of multitouch interfaces. PerAdaMagazine (April 2010)

2. Wallace, J.R., Pape, J., Chang, Y.L.B., McClelland, P.J., Graham, T.N., Scott, S.D., Hancock, M.: Exploring automation in digital tabletop board game. In: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work Companion. CSCW ’12, New York, NY, USA, ACM (2012) 231–234

3. Chaboissier, J., Isenberg, T., Vernier, F.: Realtimechess: lessons from a participatory design process for a collaborative multi-touch, multi-user game. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. ITS ’11, New York, NY, USA, ACM (2011) 97–106
4. Pinelle, D., Gutwin, C., Subramanian, S.: Designing digital tables for highly integrated collaboration. Technical Report HCI-TR-06-02, Computer Science Department, University of Saskatchewan (2006)

5. Kruger, R., Carpendale, S., Scott, S.D., Greenberg, S.: How people use orientation on tables: comprehension, coordination and communication. In: Proceedings of the 2003 international ACM SIGGROUP conference on Supporting group work. GROUP '03, New York, NY, USA, ACM (2003) 369–378

6. Scott, S.D., Sheelagh, M., Carpendale, T., Inkpen, K.M.: Territoriality in collaborative tabletop workspaces. In: Proceedings of the 2004 ACM conference on Computer supported cooperative work. CSCW '04, New York, NY, USA, ACM (2004) 294–303

7. Wigdor, D., Balakrishnan, R.: Empirical investigation into the effect of orientation on text readability in tabletop displays. In: Proceedings of the ninth conference on European Conference on Computer Supported Cooperative Work, New York, NY, USA, Springer-Verlag New York, Inc. (2005) 205–224

8. Hancock, M., Vernier, F., Wigdor, D., Carpendale, S., Shen, C.: Rotation and translation mechanisms for tabletop interaction. In: Horizontal Interactive Human-Computer Systems, 2006. TableTop 2006. First IEEE International Workshop on. (jan. 2006) 8 pp.

9. Shen, C., Vernier, F.D., Forlines, C., Ringel, M.: Diamondspin: an extensible toolkit for around-the-table interaction. In: CHI '04: Proceedings of the SIGCHI conference on Human factors in computing systems, New York, NY, USA, ACM (2004) 167–174

10. Rekimoto, J., Saitoh, M.: Augmented surfaces: a spatially continuous work space for hybrid computing environments. In: Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit. CHI '99, New York, NY, USA, ACM (1999) 378–385

11. Schiavo, G., Jacucci, G., Ilmonen, T., Gamberini, L.: Evaluating an automatic rotation feature in collaborative tabletop workspaces. In: Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems. CHI EA '11, New York, NY, USA, ACM (2011) 1315–1320

12. Dragicevic, P., Shi, Y.: Visualizing and manipulating automatic document orientation methods using vector fields. In: ITS '09: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, New York, NY, USA, ACM (2009) 65–68

13. Klinkhammer, D., Nitsche, M., Specht, M., Reiterer, H.: Adaptive personal territories for co-located tabletop interaction in a museum setting. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. ITS '11, New York, NY, USA, ACM (2011) 107–110

14. Raskin, J.: The humane interface (book excerpt). Ubiquity 2000 (May) (May 2000)

15. Hilliges, O., Wagner, M., Terrenghi, L., Butz, A.: The living-room: browsing, organizing and presenting digital image collections in interactive environments. In: Intelligent Environments, 2007. IE 07. 3rd IET International Conference on. (sept. 2007) 552–559