# FriendGroupVR: Design Concepts Using Virtual Reality to Organize Social Network Friends

Frederic Raber, Christopher Schommer, Antonio Krüger

► To cite this version:

Frederic Raber, Christopher Schommer, Antonio Krüger. FriendGroupVR: Design Concepts Using Virtual Reality to Organize Social Network Friends. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.719-739, 10.1007/978-3-030-29387-1_42. hal-02553932

HAL Id: hal-02553932

https://inria.hal.science/hal-02553932

Submitted on 24 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# FriendGroupVR: Design concepts using virtual reality to organize social network friends

Frederic Raber<fnref n="1" />, Christopher Schommer<fnref n="2" />, and Antonio Krüger<fnref n="1" />

**Abstract.** Creating friend lists offers social network users the ability to select a fine-grained audience for their posts, thereby reducing the amount of unwanted disclosures. However, research has shown that the user burden involved in creating and managing friend lists leads to the fact that this functionality is rarely used, despite its advantages. In this paper, we propose two design concepts using virtual reality to allow the user to create and organize her friend lists. Whereas the first "pragmatic" concept is targeted towards usability and practicability using a metaphor similar to card sorting, the second "playful" concept has the goal to achieve a high user experience score by offering a VR game to sort and organize the friends. In a lab study, we compared the two concepts with the Facebook interface in terms of usability, user experience and error rate (like missing friends in a group or friends placed in the wrong group). We were able to show that both designs significantly outperform the Facebook interface in both usability and user experience. The playful interface is experienced as more interesting and stimulating than its pragmatic counterpart, at the cost of an increased error rate.

**Keywords:** First keyword · Second keyword · Another keyword.

## 1 Introduction

The perceived audience of a social network post consists only out of 27% of the actual audience [1]. This leads to a high amount of unwanted recipients of the posts information, which can be used for various attacks like stalking, identity theft, user manipulation (also known as "social engineering"), re-identification in other anonymized data sets or face re-identification [8]. The success rate for a stranger to be accepted as a facebook friend and thus be added to the post audience is surprisingly high: In a study from Gross & Acquisti, 75.000 out of 250.000 users accepted the friend invitation from an unknown person [8]. Social networks such as Facebook or Google+ allow their users to create custom friend lists and share content exclusively with these list. However, these tools are often not used [20]. Known causes for this behavior are the mental effort to group people [29] and usability problems, e.g. regarding the mechanics, general workflow or simply user-interface-related problems [15, 30, 16].

<footnote n="1">¹ DFKI Saarland Informatics Campus, Saarbrücken, Saarland, Germany
{frederic.raber, krueger}@dfki.de</footnote>

<footnote n="2">² Saarland University, Saarbrücken, Germany christopherschommer@gmx.de</footnote>
Compared to other sorting tasks, the task of social network friend grouping includes several special challenges to be solved: First, privacy preferences have shown to be highly individual in several domains like location sharing [22], mobile app permissions [24] or shopping scenarios [25] as well as the task of friend grouping [23]. Every user has her own different criteria to build groups and to categorize her friends into them, depending on her personal preferences, her personality and privacy attitude [23], and also her posting preferences, regarding post topics and intimacy of the shared information [23]. Second, there is no definite answer on the correct assignment of a friend. Some of the friends might fit into multiple groups; some might not fit in any group and will remain unassigned. This leads to the fact that for some friends, it is directly clear to which groups they should be assigned to, whereas the user needs a longer time to think about a correct assignment for other cases, as our study results will show.

Research tried to tackle this problem by creating new design concepts with an increased usability in order to reduce the mental effort to perform the friend sorting task. Some of the approaches use graph-based interfaces [7, 19], where groups are represented by vertices with the corresponding friends attached as their leaves; others rely on a conventional list-based design [18] improved by an auto-grouping algorithm based on community detection [2]. Nevertheless, the usage of virtual reality to enhance the usability of social network friend sorting on the one hand, and making the task more interesting and enjoyable by enhancing the user experience on the other hand, has not been discussed in research so far to the best of our knowledge. To be more precise, we try to solve the following research questions:

1. Can we enhance the usability and user experience of the social network friend sorting using a VR environment and metaphors?

2. Do users prefer a playful or a pragmatic approach for VR sorting?

3. How do the VR designs effect the errors made during friend sorting?

For this purpose, we created two different UI designs, one focused on further increasing the usability in a virtual reality environment by adapting traditional concepts such as card sorting (“pragmatic design”), and one that is geared towards making the sorting task as fun and enjoyable as possible by packaging the task as an interactive VR game (“playful condition”). In a study comparing these to a conventional sorting interface from the Facebook social network website, we found out that we could further increase the usability with the pragmatic design. The playful condition was perceived as highly motivating and achieved a significantly higher user experience score, at the cost of an increased error rate.

## 2 Related work

For the scope of our work, there are three research fields of interest that we want to discuss in this section: first, the usage of social network friend lists, strategies used and problems that arise within the current solution; second, user interface designs in other domains that enhance usability; and third, the questionnaires that are of importance for this research field and the later evaluation study.
## 2.1 Friend grouping in social networks

A comparative user study by Kelley et al. [16] in 2011 investigated different metaphors and user strategies that arise when users start grouping friends into friend lists on the Facebook social network. Their user study included three new interfaces that have been compared to the Facebook UI: a card sorting method using printed pictures of the friends; a grid tagging approach where all friend images are printed in a grid shape on one page, so that the user can use pens of different colors representing the friend groups the friend should be assigned to; and a file explorer where the friends are represented as files that are sorted into folders using the Windows explorer. Lastly, they used the current Facebook interface as a reference interface. In a user study, they found out that there are two user strategies: the “by friend” strategy, where the groups needed for this friend are created first, and then populated by all other friends that fit inside these groups before the user proceeds to the next friend; and the “by group” strategy where all needed groups are created first, and then populated by the friends one after another. As some of their design recommendations, they advised to always keep in mind that Facebook friend grouping is not a primary task, and secondly, that the grouping changes over time. In our work, we took up these design recommendations and created a user interface that motivates the user and that packages the uninteresting sorting task as a challenging VR game, so that the task is done more frequently, especially over time.

Other studies about usage frequencies and mental effort have shown that the usability and understanding the grouping interface is not a problem for social networks like Google+ [29, 14]. However, the mental effort required for finding a good assignment is very high, leading to the fact that users prefer to censor their posts rather than to use friend grouping to select the correct audience [29, 14]. According to a study by Javed et al. including 200 participants, more than 50% of all Facebook users have not created any personal friend lists at all. Another 15% and 10% have created only one or two friend lists, respectively [12]. Interestingly, automatic friend groups have a small amount of overlap between them, whereas user-created friend groups have little to none: Out of all members of a friend list, 90% are not present in any other friend list for user-created friend lists compared to about 58% for automatically created lists. Usually, users never add a friend to more than two friend lists; some are not added to any friend list. On average, each self-created friend list contains about 32 members. Based on the mentioned study results, we designed our UI designs so that being able to add friends to multiple lists should be easily possible, but is not a primary design goal of the UI.

## 2.2 Sorting in virtual reality and other domains

In general, the sorting task or *categorization task* described here involves several items that have to be assigned to one or multiple groups by the user, based on the individual features of each item. There are several abstraction levels of the items to be sorted that can be used for a sorting task, each with its own advantages
and drawbacks [26]. The most concrete form is *object sorting*, where the user is
given concrete objects to be sorted. Although this provides the most information
for the user, it can contain irrelevant and distracting features as well, that may
lead to a different evaluation and assignment of the object than intended. The
next abstraction level is *picture sorting*, where the object is represented solely by
an image of the object. Although this reduces the amount of sensory input, it is
possible to trim out unimportant features from the picture, reducing the amount
of irrelevant information for the user. Lastly, card sorting uses only written text
on a card describing the object. While this design is most restrictive and allows
limiting the information only to the sorting features that are of relevance, it can
only be used for items that are known to the user, and that he can imagine from
reading the description.

Apart from the level of detail, sorting techniques also differ in the group
creation policy used for sorting items into groups using a desktop interface.
Card sorting approaches [21] can be differentiated between “closed card sorting”,
where the categories are already pre-defined, and “open card sorting”, where the
categories are defined by the user during the sorting task. Whereas the former
leads to more comparable results between subjects, the latter gives the user more
degrees of freedom, which makes it more suitable for tasks with individual data,
like the task of friend grouping. Finally, the UI designs for card sorting also
differ in the design metaphor used to display and arrange the cards. There are
several commercial applications that use a *stacked card sorting metaphor*, where
the cards of the same category are arranged on top of each other. Examples are
CardZort<fnref n="3" /> or OpenSort<fnref n="4" />. Other designs like WebSort<fnref n="5" /> use a more *explorer-like*
*approach*, where the categories are displayed as a vertical list of terms that can
be opened like folders in the Windows explorer. Card sorting tools have been
proven to be highly efficient, but the preferred design greatly depends on the user
group [5]: although researchers preferred the explorer-like approach in WebSort,
most of the end-users liked the stack-based approach of OpenSort best. Studies
have shown that there are no significant differences in terms of performance when
comparing online and paper-based card sorting [4]. Sorting is also possible in VR,
as a study on “how similar looking products influence the overall performance
in a retail setting” has shown recently [11]: The participants were shown a VR
world, where they had to sort products arriving on a conveyor belt into two
different categories. Although the participants were generally successful in the
tasks, the error rate significantly increased when the similarity of the products
was increased.

For the task of social network friend sorting, the abstraction level of “card sorting” might not be sufficient, as users might not remember the actual person behind a username, since users often use fake names for their online profiles. Apart from that, we cannot rule out that two or more social network friends might have similar images, leading to an increased error rate. We therefore de-

<footnote n="3">³ https://cardzort.software.informer.com/</footnote>

<footnote n="4">⁴ https://sourceforge.net/projects/opensort/</footnote>

<footnote n="5">⁵ https://dirtarchitecture.wordpress.com/wesort/</footnote>
cided on an improved “card sorting+” design, which includes a profile picture and a textual description using the username as well. The task of social network friend sorting is highly individual, which led us to the decision to use *open card sorting*, using predefined groups for convenience with the possibility to add additional groups as needed. As most of the end users opted for the “stacked card metaphor” for arranging the cards, we based our interface designs on this metaphor as well.

## 2.3 Measuring user experience, usability and other aspects important for our work

The de-facto standard for measuring usability is the system usability scale (SUS) [3], which was introduced by John Brooke in 1986 to provide a way of “quick and dirty” quantification of a perceived usability. The SUS generates a score between 0 and 100, where a score above 68 is perceived as “good” usability. Although often used for measuring usability, the questionnaire captures only the pragmatic quality, e.g. the usability of an interface, without considering the hedonic aspects, e.g. the user experience regarding the experienced stimulation and fun when using the UI. The AttrakDiff questionnaire [10] contains questions capturing the pragmatic aspects, as well as the hedonic aspects like identity and stimulation of a user experience, and is geared especially towards comparing different interfaces to each other with regard to usability and user experience. The AttrakDiff generates three different scores that are of interest for our design experiment: The pragmatic quality (PQ) capturing the usability, similar to the SUS and two hedonic scores describing the stimulation perceived when using the interface (HQ-S) and how much subjects could identify with the UI (HQ-I). All scores range from -2.5 to 2.5, where a value above 1.0 or 2.0 is perceived as “good” or “excellent”, respectively. According to the goals of our experiment capturing usability as well as user experience in a comparative study, we decided for the AttrakDiff questionnaire in our case.

Capturing the perceived workload including factors like mental or physical demand and frustration is mostly done using the NASA TLX questionnaire [9]. The questionnaire has been developed over a three-year development cycle with more than 40 lab experiments and is now cited by more than 4400 studies, denoting the widespread influence of this questionnaire [6]. The questionnaire measures the *mental demand* (e.g. cognitive load), *physical demand*, *temporal demand* (e.g. perceived time pressure), the *perceived effort* needed to achieve the desired goal, as well as the perceived *frustration* and the subjects' own achieved *performance* estimated by the subject. Each single value ranges from 0 to 100 where 0 is the lowest (best) and 100 the highest (worst) workload. We included the questionnaire in our study especially to compare the mental and physical demand between the standard interface and the VR designs, and the frustration experienced with the different interfaces. Lastly, we added the motion sickness assessment questionnaire (MSAQ) to our experiment to ensure that motion sickness does not affect our UI designs in a negative way. Scores on
this measure range from 11.1 for no motion sickness to 100 for highest motion sickness effects.

To conclude, research has explored several different metaphors for sorting, whereas the metaphor of card sorting has been proven to be highly effective. Related work has shown that VR creates a higher immersion compared to traditional interfaces, resulting in a feeling of being “in the game” [13, 27]. VR applications can, if done well, lead to the most natural and most efficient interaction, far better than it can be achieved with traditional 2D or 3D applications [13, 27]. Research has also proven that using VR for e-commerce applications like VR shopping is perceived as significantly more useful (in terms of interaction techniques), immersive and interesting by users that its two-dimensional counterparts [17, 28]. These results leads us to the assumption that VR can also improve the usability and perceived fun of a friend grouping task, which we are eager to test in this paper. In our work, we build upon the idea of card sorting and transfer this metaphor into a VR design to enhance usability on the one hand, and the user experience on the other hand, to present the task of social network friend grouping as an interesting and enjoyable task that is still perceived to be easy to carry out with our designs.

## 3 FriendGroupVR Designs

We implemented *two* different VR design approaches to sort and organize social network friend lists, targeting different objectives: The first “pragmatic” approach is optimized towards *usability* in terms of efficiency and performance, whereas the second “playful” approach is focused on making the sorting task as enjoyable and interesting as possible. Each world was implemented in Unity using an HTC Vive VR Kit. The setup contained a 4m x 4m floor equipped with an HTC Vive Lighthouse setup that allows tracking the user’s movements inside the area (see Figure 1). Each user movement was reflected in the VR world as well. In order to track hand movements, each user was given two Vive controllers, one for each hand. Grabbing gestures were realized by usage of the trigger buttons of the controllers. For our lab study, we recorded the created friend lists and the contained friend lists locally instead of applying the changes to the user’s social network account.

A special problem of the friend sorting task is that the time needed for the assignment of a friend to one or multiple groups can be highly variable. For some of the friends, it is directly clear to which social circle(s) or which friend list(s) they belong, but for others it is less clear, so that the user might need some seconds to think before he can conduct the actual assignment task. Therefore, we put a special emphasis on the possibility to interrupt the sorting task between two friends, so that the user has the possibility to think about the best assignment options in advance.
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** VR setup in our lab using the HTC Vive. “L” denotes the positions of the lighthouse position trackers, “V” the initial position of the user wearing the HTC Vive</figcaption></figure>

**3.1 Pragmatic Design**

According to related work, the metaphor of card sorting is one of the most ef-
ficient methodologies [5], we therefore decided to transfer the open card sorting
metaphor into a VR world, leading us to an office metaphor as shown in Figure
2. The “cards”, i.e. the social network friends, are represented as picture frames
(“friend frames”) standing inside a bookshelf. Each friend frame consists of the
friend’s profile picture and forename on the front, and the fore- and surname on
the back, forming a combination of a card sorting and picture sorting metaphor
(“card sorting+”), as described in the related work section. Friends can be dis-
played with ascending tie strength (equivalent to the Facebook friend list order)
or sorted by fore- or surname. As space is limited, the shelf always contains only
nine friend frames at a time. To access the other frames, we placed two buttons
at the left and right edge of the shelf, allowing the user to access friends that
appear earlier or later in the sorted list, respectively.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Bookshelf in the pragmatic design, including friends represented by “friend frames” (left) and spawner to create new friend list boxes (right).</figcaption></figure>
According to the *stacked cards metaphor*, friend lists are represented as la-
beled boxes (“list box”) in which the user can drag & drop friend frames us-
ing a VR controller. As a starting point, the VR world contains the five most
frequently used friend lists according to related work [23], namely “family”,
“acquaintances”, “close friends”, “work” and “sport”, as a box. Boxes have no
physical weight in our VR world, and can therefore be placed in mid-air at any
desired location. As the task of arranging friend lists is highly individual, we
opted for an “open card sorting” design allowing users to create arbitrary addi-
tional friend lists. To create a new list box, we added the “box spawner” into the
environment (Figure 2): To create a list box, the user has to touch the red but-
ton with the VR controller, which opens a VR keyboard to enter the list name.
Pressing the enter button hides the keyboard and spawns the newly created list
box, as seen in the figure.

To manage the friend lists, a user typically starts with creating and arranging
the list boxes around the shelf. After that, the friend frames are traversed one
after another and placed inside one or multiple list boxes that should contain
the friend. As soon as a friend frame is placed inside a box, the frame is shrunk
to half of its size to save space. If a friend is placed inside the wrong box or if
the user decides to assign them to a different list box, he can always empty the
box on the floor or grab a picture inside the box and put it into another.

## 3.2 Playful design

In contrast to the former design, we concentrated on making the sorting task as
joyful and interesting as possible. We therefore decided to design the approach as
an interactive VR game that challenges the user, including gamification elements
like high score tables, upgrades and bonus items that should motivate the user
in conducting the task and competing with others. As stated in the beginning of
the section, the time needed for finding an optimal assignment is very different
from friend to friend. We therefore need a game design which can be interrupted
or delayed at certain points in time to allow the user to take her time for the
assignment decision. As related work has shown, most of the friends (about 90%)
are assigned only to one friend group; we therefore decided on a game with a
linear action line, where only one friend is part of the game at a time, with the
possibility to manually go back to a friend again if he or she has to be added to
multiple friend groups. We came up with the idea of a “can knockdown” game,
where the user can assign her friend to friend lists by shooting dispatched “friend
balls” to different can stacks representing the available friend lists. Using this
design, the user can always wait and think about the correct assignment, before
she starts the dispatch of the friend ball.

In a typical workflow, the user first creates the needed friend lists using a tool
similar to the box spawner in the pragmatic design. After this task is finished,
the friend lists are represented by can stacks (“list stacks”) at a distance of about
five meters in front of the user. The user then starts the assignment phase, where
a ball representing each social network friend is dispatched in the direction of
the user one after another. The user uses a bat to redirect the friend ball to
a can stack corresponding to the friend list the user has to be assigned to. As mentioned before, each friend ball is dispatched only once. If the user wants to add a friend to multiple groups, he has to press the “back” button on the control panel (see below) to display the last friend ball again and add her to another group. Depending on how many cans the user is able to hit with the friend ball, the user gains points to be added to his personal high score.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Can knockdown game in the playful design.</figcaption></figure>

A screenshot of the playful VR world from the user's initial position can be found in Figure 3: The shelf on the left side of the user (Figure 4 left) is used to create and arrange the friend lists, similar to the pragmatic design. In the shelf, friend lists are represented by a small board with the list name written on the front. Similar to the other design, the five most frequently used friend lists are already created in advance and placed at the bottom of the shelf. If the user wants to create a new friend list, he touches the button, which opens a keyboard to enter the friend list name, exactly like in the pragmatic design. To use a friend list in the can knockdown game, the user has to place a friend list board in one of containers in the shelf, which will display a can stack in the game at the respective location (e.g. if the board is placed at the container to the left of center, the corresponding can stack will also be shown to the left of center in the game). At the right hand side of the user is a control panel (Figure 4 top right) which allows the user to switch forward or backward between the social network friends, and a button to start and pause the game at any given time, for example if more time is needed to contemplate the correct friend list assignment. Using the control panel, the user can also go back to an earlier friend and dispatch her friend ball another time to assign him to another friend list.

To the front, the user is facing a panel (Figure 4 bottom right) which displays the name and profile picture of the next friend to be sorted, together with the
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Friend list management shelf (left), score and dispatch panel opposite the user, dispatching a friend ball (bottom right) and control panel to switch current friend and start/pause the game (top right) in the playful design.</figcaption></figure>

current score and the remaining time for the currently collected bonus item (see below). When the game is started by pressing the “start button”, the user has five seconds to think about the correct list stack that he wants to aim at. The five dots at the top of the panel represent the time in seconds that is remaining. When the last dot turns from grey to red, the friend ball is dispatched towards the user. If no can stack is hit, the same friend ball is again dispatched for another try. When the user does not want to assign the friend to any group, he can aim for the monster at the upper left of the VR world, which will then eat the friend ball, so that it is not dispatched again. If the user hits the wrong can stack, he can always undo the last assignment by hitting the “undo” buzzer directly in front of him. To further motivate the user, we integrated “bonus balls” into the game, which are dispatched in the direction of the user at randomized times. Collecting each bonus ball activates a special upgrade for a limited time, for example a score multiplier, or an increase of the friend ball size.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Different bats available to the user with ascending difficulty from left to right.</figcaption></figure>
The user has a choice of different bats (Figure 5) with different difficulties: The easiest bat is largest and catches the ball so that the user has the possibility to aim and shoot at the desired location by pressing a button. The second easiest bat has the same size, but directly reflects the ball without catching it first. The remaining two bats have the same behavior with a smaller size, making it more difficult to hit the ball. The more difficult a bat is, the more score is rewarded for each can hit. When all friend balls have been processed, the game stops and the user's high score is displayed on the high score table in the upper right of the VR world, along with the high scores of other users, and the friend lists are stored.

# 4 User study

We had the goal to find interaction designs using VR for the creation and maintenance of social network friend lists, that would be both more efficient and also more enjoyable than the current standard. In order to measure the differences from the Facebook UI, we conducted a lab study at our department, where the participants had to use both VR designs as well as a standard interface from the Facebook social network site using a desktop PC as a baseline. With each interface, the participants had to assign their 40 closest friends (according to the Facebook friend ordering) to friend groups. For each condition, we recorded usability and user experience scores using the AttrakDiff[10] questionnaire as well as an error rate (for example friends missing from a group, or friends assigned to the wrong group), as described below in more detail. To reduce training effects and to get users used to the VR environment, we implemented another vr "training" world which shows the user an overview of the 40 friends that have to be assigned in the experiment (Figure 6). To further reduce training effects, the order of conditions was permuted for each participant so that each sequence of conditions appears equally often during the study, leading to 3! = 6 different orders.

The procedure was the same for each participant but with a different order of conditions, as stated before. After signing a consent form and the privacy policy, the participant had to fill in a questionnaire about demographic data and previous experience using the Facebook friend grouping tool and virtual reality setups. She was then given a desktop screen to enter her Facebook login data. With the aid of the Selenium web browser automation toolkit<fnref n="6" />, a Python script then traversed the participant's friend list and extracted the friend names and profile pictures for later use during the study. After the process was finished, the participant was given instruction in the VR hardware, and had to put on the headset for the first time. We started the training level and gave her the time to get familiar with the VR world and the controllers, and to have a first look at the friends to be sorted and to contemplate the friend lists and the assignments to be made. When the participant stated she was ready, the training world was closed, and the main experiment phase started.

<footnote n="6">⁶ https://docs.seleniumhq.org/</footnote>
<figure><img src="image_7.png" /><figcaption>**Fig. 6.** Training world with 40 friends that the user is shown before the experiment starts.</figcaption></figure>

In the main phase, the three interface conditions were tested one after another in a different order, as stated above. For each condition, the participant was given an introduction to the interface with all of its interaction possibilities and some time to get familiar with it and to test each functionality once. When she stated she was ready, the world was reset, and the participant had to do the friend grouping with her 40 friends until she stated she was finished. Participants were told that they should do the task seriously, as wrong assignments would be recorded. In the following, the participant had to fill in several questionnaires about the current condition: the AttrakDiff questionnaire [10] measuring usability (PQ) and user experience (HQ-I, HQ-S), the NASA TLX capturing the mental and physical workload, an MSAQ questionnaire asking about motion sickness in the VR conditions as well as a custom questionnaire asking whether the interface was motivating or fun to use, and whether the participant thought it could be integrated into her daily life, on a five-point Likert scale. After a five-minute break to rest and recover, this procedure was repeated for the two other conditions. For each condition, we recorded the overall time spent on soritng. At the end of the study, the participant was asked which was their favorite interface, and had to traverse the friend lists created to check for errors made during the assignment. We recorded the following error measures:

- Person missing from a group (MISS)

- Person added despite not belonging to the group (TOOMUCH)

- Wrong group label (LABEL)

- Group should be split into multiple groups (SPLIT)

- Multiple groups should be merged to one group (MERGE)

# 5 Results

In total, we had 30 participants in the study, 18 female and 12 male. Participants were recruited at our university using postings and the university's social net-
work group. As a compensation, a $\epsilon$25 Amazon voucher was raffled off among all participants. The age ranged from 19 to 50 (mean=26.67, SD=5.474), representing a good portion of typical social network users $^7$. When asked about their experiences with virtual reality, 12 people had no experience (40%) and 5 almost no experience (16.7%). 25 people answered that they had never used Facebooks grouping interface (83.3%), while 5 people had used it (16.7%). On average, the main experiment was completed within 64 minutes.

The experiment results can be found in Table 1. Depending on whether an F-test showed a normal distribution of the data, we performed pairwise paired T-tests or Wilcoxon signed-rank tests to compare the pragmatic quality (PQ), also known as usability; the measures from the custom questionnaire asking about experienced fun (FUN) and motivation (MOTIV) and suitability for daily use (DAILY), the NASA-TLX workload values, times needed for sorting, the error rates and the hedonic scores HQ-I and HQ-S measuring the user experience between the three conditions.

<table><caption>**Table 1.** Results for the usability and user experience scores including pragmatic quality (PQ), hedonic quality regarding stimulation (HQ-S) and identification (HQ-I) and the custom questions asking about fun (FUN) and motivation (MOTIV) to do the task and suitability for everyday usage (DAILY), as well as the NASA TLX workload, time spent on sorting and the error rate.</caption><thead><tr><th>measure</th><th><i>M</i><sub>FB</sub></th><th><i>M</i><sub>playful</sub></th><th><i>M</i><sub>pragmatic</sub></th></tr></thead><tbody><tr><td>PQ</td><td>-0.15</td><td>0.61</td><td>1.99</td></tr><tr><td>HQ-S</td><td>-1.81</td><td>2.02</td><td>1.13</td></tr><tr><td>HQ-I</td><td>-0.54</td><td>0.96</td><td>1.21</td></tr><tr><td>FUN</td><td>1.57</td><td>4.77</td><td>4.57</td></tr><tr><td>MOTIV</td><td>1.65</td><td>4.23</td><td>4.33</td></tr><tr><td>DAILY</td><td>2.10</td><td>2.87</td><td>3.77</td></tr><tr><td>Workload</td><td>30.61</td><td>39.42</td><td>22.86</td></tr><tr><td>Errors</td><td>8.50</td><td>11.93</td><td>7.60</td></tr><tr><td>Time(s)</td><td>418</td><td>587</td><td>512</td></tr></tbody></table>

The usability (PQ) was highest for the pragmatic interface ($M = 1.99$) and significantly better than for the playful interface ($M = 0.61, T = 4.6, p < 0.001$) which is itself significantly more usable than the Facebook standard ($M = -0.15, T = 3.1, p = 0.004$). Regarding the user experience, the user could identify significantly better with the pragmatic interface ($M = 1.20$) than with the playful interface ($M = 0.096, T = 2.13, p = 0.042$) which was again better than Facebook ($M = -0.53, T = 8.38, p < 0.001$), but felt most stimulated by the playful interface ($M = 2.04$) followed by the pragmatic VR design ($M = 1.13, Z = -4.50, p < 0.001$) and distantly followed by the Facebook UI with a significantly lower score ($M = -1.81, T = 14.471, p < 0.001$). The FUN

<footnote>$^7$ https://www.statista.com/statistics/274829/age-distribution-of-active-social-media-users-worldwide-by-platform/</footnote>
was on average also highest using the playful design (M = 4.77) although we
could not prove the difference to be significant from the pragmatic interface
(M = 4.57, Z = 0.965, p = 0.334). The Facebook interface was again rated sig-
nificantly worse than the pragmatic interface (M = 1.57, T = 4.79, p < 0.001).
The most motivating interface is the pragmatic interface (M = 4.33) according
to the mean values, but is again not significantly better than the playful inter-
face (M = 4.23, Z = 1.62, p = 0.09). The Facebook UI is again significantly
worse than the playful UI (M = 1.65, Z = 4.75, p < 0.001). The same order
holds for the suitability to integrate the UI into everyday social network usage:
The pragmatic interface (M = 3.77) significantly outperforms the playful inter-
face (M = 2.87, Z = 3.24, p = 0.001) which is again significantly better than
the current standard on Facebook (M = 2.10, Z = 2.39, p = 0.017). The time
in seconds needed to perform the grouping task was lowest with the Facebook
interface (M = 418) and significantly higher with the pragmatic (M = 587, T =
2.722, p = 0.011) and playful VR designs (M = 588, T = 2.50, p = 0.018). A
visual analysis on the time distributions for the times needed to assign a single
friend showed that the times are very different for some of the users, supporting
our assumption that an interface is needed that allows users to pause the sorting
task, as the time needed for an assignment can differ. Figure 7 shows the time
distribution for three representative subjects of the study.

<figure><img src="image_8.png" /><figcaption>**Fig. 7.** Distribution of the time needed to assign a friend for three representative subjects.</figcaption></figure>

The motion sickness (MSAQ) scores (ranging from 11.1=best to 100=worst) were very low for both VR interfaces and did not differ significantly ($M_{pragmatic} = 16.02, M_{playful} = 16.37, Z = 0.991, P = 0.322$), attesting that motion sickness was not a noticeable problem in our UI designs. The pragmatic interface received on average the smallest error rate ($M = 7.60$). Nevertheless, the error rate using the baseline interface is not significantly higher ($M = 8.50, Z = 0.419, p = 0.675$). The playful interface led to the highest error rates, which are significantly higher than for the standard Facebook interface ($M = 11.933, Z = 2.204, p = 0.027$). The same holds for the workload, which is highest for the playful interface ($M = 39.42$) and significantly lower for the Facebook UI ($M = 30.61, T = 2.79, p = 0.0009$) and lowest for the pragmatic UI ($M = 22.85, T = 2.90, p = 0.007$). A detailed overview on the error rates and the different workload items can be found in Tables 2 and 3. We can clearly see that the main cause for the
higher workload in the playful design is that the VR game was perceived as challenging, as the mental demand (Z = 4.19, *p* < 0.001), temporal demand (T = 4.37, *p* < 0.001) as well as the effort (Z = 4.22, *p* < 0.001) is significantly higher compared to the pragmatic interface. The frustration was highest using the Facebook interface, supporting our claim that friend grouping is perceived as a very frustrating and uninteresting task. Using VR, the frustration is significantly smaller for both the pragmatic (T = 6.79, *p* < 0.001) as well as the playful design (T = 3.65, *p* = 0.001). The most favored interface was the pragmatic design (73.3%) followed by the playful design (23.33%). Only one participant claimed to like the standard Facebook interface best. We observed different behaviors regarding the choice of the used bat throughout the game: 17 participants used the “sticky” bat and 9 used the “reflective” most of the time (> 80% of the time), three switched between both. One used the baseball bat exclusively. However, we did not find any significant difference between these usage groups for any of our measures.

<table><caption>**Table 2.** Detailed results for the average number of errors per participants.</caption><thead><tr><th>measure</th><th><i>M</i><sub>FB</sub></th><th><i>M</i><sub>playful</sub></th><th><i>M</i><sub>pragmatic</sub></th></tr></thead><tbody><tr><td>MISS</td><td>2.5</td><td>4.6</td><td>1.43</td></tr><tr><td>TOOMUCH</td><td>0.57</td><td>4.6</td><td>0.57</td></tr><tr><td>LABEL</td><td>0.03</td><td>0.03</td><td>0.1</td></tr><tr><td>SPLIT</td><td>0.23</td><td>0.27</td><td>0.17</td></tr><tr><td>MERGE</td><td>0.07</td><td>0.03</td><td>0.03</td></tr></tbody></table>

<table><caption>**Table 3.** Detailed results of the NASA TLX.</caption><thead><tr><th>measure</th><th><i>M</i><sub>FB</sub></th><th><i>M</i><sub>playful</sub></th><th><i>M</i><sub>pragmatic</sub></th></tr></thead><tbody><tr><td>Mental demand</td><td>29</td><td>41.17</td><td>22.67</td></tr><tr><td>Physical demand</td><td>10.50</td><td>44.17</td><td>35.67</td></tr><tr><td>Temporal demand</td><td>39.50</td><td>47.83</td><td>25.67</td></tr><tr><td>Performance</td><td>30.33</td><td>34.83</td><td>20.17</td></tr><tr><td>Effort</td><td>26.33</td><td>42</td><td>21.50</td></tr><tr><td>Frustration</td><td>47.67</td><td>26.50</td><td>11.33</td></tr></tbody></table>

# 6 Discussion

## 6.1 Increased usability using vr

We presented two VR friend grouping interfaces, one geared towards maximizing the usability, and one towards maximizing the fun and motivation when sorting the friends. Comparing the usability scores of the interfaces, we can see that
both VR interfaces were perceived as significantly more useful than the Face-
book interface. The pragmatic design achieved the highest usability score of 1.99
which is very close to the theoretical maximum of 2.50 on a scale from -2.5 to
2.5, indicating that we achieved the goal of increasing the usability compared
to the current standard. Interestingly, the interaction time was lowest for the
Facebook interface, although it was rated with a significantly lower usability.
On the other hand, the error rate was higher for the Facebook interface, leading
to the assumption that the Facebook interface is fast to use on one hand, but is
complicated and leads to an increased error rate on the other hand, which leads
to a smaller perceived usability of this interface.

## 6.2 Challenging game design leads to increased error rates

The error rate is on average lowest for the pragmatic interface, although the dif-
ference to the standard interface is not significant. As stated earlier, we designed
the playful design to be challenging for the user, including bonuses, high scores,
and different levels of difficulty using different bats. This is also reflected in the
perceived workload according to the NASA-TLX, where especially the mental
and physical effort is higher compared to the other interfaces. Nevertheless, the
frustration is low compared to the Facebook interface, indicating that the stress
was perceived to be positive. However, the challenges may also lead to the in-
creased error rate, which is also highest for the playful interface. The game may
have been too challenging, or the design as a game may have led the participants
to take the task less seriously and pay less attention to a correct sorting; which
of these factors led to the increased error rate should be further investigated in
a follow-up study.

6.3 Significantly improved user experience, not only for the playful condition

The differences for the user experience scores are again larger than for the us-
ability scores when comparing the VR designs with the standard interface. The
playful design received a very high stimulus and FUN score, again indicating
that the game was perceived as challenging, stimulating and fun to use. But
the pragmatic design, which was not optimized towards user experience, also
achieved a high user experience score, which was significantly higher than for
the Facebook baseline. The pragmatic design was voted to be most motivating,
although not significantly more so than the playful interface. One reason why it
was rated to be more motivating on average might be the good combination of
an appealing and interesting user interface while still providing a high usability
without trying to challenge the user. Which factors led to the higher motivation
should therefore be investigated in the follow-up study.

6.4 Conclusion on the optimal design and favored interface

Taking all the aspects into account, the results indicate that VR designs are
perceived as more useful on the one hand, and as more fun and motivating on
the other, which gives them a clear advantage over the current mouse & keyboard
interface. However, such conventional interfaces have the advantage that every
computer is equipped with a mouse and keyboard; the audience that can use
the Facebook interface is therefore currently significantly larger than those who
own a VR kit at home to do the friend sorting with one of the two VR designs.
Nevertheless, VR interfaces will gain importance in the next few years, as the
number of VR users is increasing exponentially <fnref n="8" />. As stated in the introduction,
one of the major problems of friend sorting is that the task requires a high mental
demand, making it a task that is often avoided. A first approach is therefore to
shape the task as an interesting and challenging game, like our playful design.
However, as the results show, a playful design always has the drawback that it
can lead to the task being taken less seriously or being lost in the game without
paying attention to the actual task, leading to a decreased quality of the outcome
of the task. Therefore, according to our results, it seems that the optimal way,
and the way preferred by users, is a VR design which is targeted towards usability
and that could be enhanced with some small game elements, but without losing
the focus on the actual task too much. These results confirm the study findings
about card sorting, which was already shown to be very efficient using a desktop
interface [5], and which seems to be efficient for sorting within a VR world as
well. Regarding the differences in time needed for assigning a friend to a list,
our results indicate that this time indeed is very diverse, making it important
to design a user interface or a VR sorting game so that it can be paused at any
time, especially between the items to be sorted. Whether these assumptions can
be proven to be true remains for a follow-up experiment, where we will take
a closer look at the effects that led to the increased error rates in the playful
condition.

**6.5 Limitations & future work**

The aim of this paper was to have a first look at how virtual reality interfaces
can help in making the task of friend sorting more fun and interesting, leading
to possibly higher usage rates of friend lists in social networks. We created two
different designs, one trying to improve the usability of the sorting using VR
metaphors, and one geared towards making it a more fun and entertaining expe-
rience. Based on different criteria like the possibility to interrupt the sorting task
at any given time or by using gamification elements to enhance the user experi-
ence, we came up with two different designs for our user study. Notwithstanding,
plenty of other possible design ideas exist and might be suitable for this kind of
task. Nevertheless, we were able to show that, with our design ideas, the user
experience as well as the perceived usability could be improved. However, we
would like to elaborate on other designs in the future, especially game designs
that might be more prone to errors than our can knockdown game, although the
increased error rate might be an effect of the gamified design, which would hold
for other game types as well.

<footnote n="8">⁸ https://de.statista.com/statistik/daten/studie/426237/umfrage/prognose-zur-
anzahl-der-aktiven-virtual-reality-nutzer-weltweit/</footnote>
In the experiment, we used the current standard interface as a baseline to minimize side effects and to get a comparison of our designs to the current working standard. Although we were able to prove that both usability and user experience were higher using the VR design, we would like to elaborate more on the parts of the design that lead to this effect. We would especially like to discover which of the developed metaphors led to an increased rating; whether it was the representation of the friends as friend frames inside a shelf, the friend boxes or the interaction by inserting the frames inside the box. Also, the alone usage of virtual reality might already lead to some effect. In several follow-up studies we would like to find out more about which design elements have a positive effect in VR using A/B testing, and give concrete guidelines on which metaphors should be used, and which should be avoided. Finally, we would like to integrate our work into a social network, so that VR users can try the solution in their everyday social network usage. We are especially interested in acceptance and usage rates: whether the app is accepted after the first usage, or whether they fall back to using the Facebook interface after some time, and whether the usage of a VR app increases the frequency of social network friend sorting.

## 7 Conclusion

Neglecting privacy settings in online social networks can lead to serious harms, but privacy functionalities like friend lists are rarely used in social networks, because the mental effort for creating friend lists prior to their usage is too high, leading users to either censor their posts or to publish more information than they intended to. Related work focused on improving the usability of conventional desktop interfaces for friend sorting. In our paper, we took a first look at how friend sorting interfaces could look in virtual reality. We proposed an interface focused on usability by taking the idea of card sorting into vr, and a second interface having the goal to maximize the user experience by wrapping the sorting task in a challenging game. A comparative study with the Facebook sorting interface as a baseline has shown that both interfaces achieved their goal of improving the usability and user experience, although the error rate significantly increased within the playful design. However, which distinct factors led to the increased error rate, and which factors led to the increased user experience scores, should be further studied in future research.

## 8 Acknowledgements

This work was funded by the German Research Foundation (DFG) via the collaborative research center Methods and Tools for Understanding and Controlling Privacy (SFB 1223), project A7.
References

1. Bernstein, M.S., Bakshy, E., Burke, M., Karrer, B.: Quantifying the invisible audience in social networks. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 21–30. CHI ’13, ACM, New York, NY, USA (2013). https://doi.org/10.1145/2470654.2470658, http://doi.acm.org/10.1145/2470654.2470658

2. Blondel, V.D., Guillaume, J.L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. Journal of statistical mechanics: theory and experiment **2008**(10), P10008 (2008)

3. Brooke, J.: Sus: A quick and dirty usability scale. Usability Eval. Ind. **189** (11 1986)

4. Bussolon, S., Russi, B., Missier, F.D.: Online card sorting: As good as the paper version. In: Proceedings of the 13th European Conference on Cognitive Ergonomics: Trust and Control in Complex Socio-technical Systems. pp. 113–114. ECCE ’06, ACM, New York, NY, USA (2006). https://doi.org/10.1145/1274892.1274912, http://doi.acm.org/10.1145/1274892.1274912

5. Chaparro, B.S., Hinkle, V.D., Riley, S.K.: The usability of computerized card sorting: A comparison of three applications by researchers and end users. J. Usability Studies **4**(1), 31–48 (Nov 2008), http://dl.acm.org/citation.cfm?id=2835577.2835580

6. Colligan, L., Potts, H.W., Finn, C.T., Sinkin, R.A.: Cognitive workload changes for nurses transitioning from a legacy system with paper documentation to a commercial electronic health record. International Journal of Medical Informatics **84**(7), 469–476 (2015). https://doi.org/10.1016/j.ijmedinf.2015.03.003, http://www.sciencedirect.com/science/article/pii/S1386505615000635

7. De Wolf, R., Gao, B., Berendt, B., Pierson, J.: The promise of audience transparency. exploring users perceptions and behaviors towards visualizations of networked audiences on facebook. Telematics and Informatics **32**(4), 890–908 (2015)

8. Gross, R., Acquisti, A.: Information revelation and privacy in online social networks. In: Proceedings of the 2005 ACM Workshop on Privacy in the Electronic Society. pp. 71–80. WPES ’05, ACM, New York, NY, USA (2005). https://doi.org/10.1145/1102199.1102214, http://doi.acm.org/10.1145/1102199.1102214

9. Hart, S.G., Staveland, L.E.: Development of nasa-tlx (task load index): Results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, Advances in Psychology, vol. 52, pp. 139–183. North-Holland (1988). https://doi.org/10.1016/S0166-4115(08)62386-9, http://www.sciencedirect.com/science/article/pii/S0166411508623869

10. Hassenzahl, M., Burmester, M., Koller, F.: Attrakdiff: Ein fragebogen zur mes- sung wahrgenommener hedonischer und pragmatischer qualitaet. In: Szwillus, G., Ziegler, J. (eds.) Mensch & Computer 2003: Interaktion in Bewegung. pp. 187–196. B. G. Teubner, Stuttgart (2003)

11. Hubbell, B., Kaufman, A., Rivers, L., Schulte, K., Hagler, G., Clougherty, J., Cas- cio, W., Costa, D.: Understanding social and behavioral drivers and impacts of air quality sensor use. The Science of the total environment **621**, 886–894 (12 2017). https://doi.org/10.1016/j.scitotenv.2017.11.275

12. Javed, Y., Shehab, M.: How do facebookers use friendlists. In: Advances in Social Networks Analysis and Mining (ASONAM), 2012 IEEE/ACM International Conference on. pp. 343–347. IEEE (2012)
13. Jerald, J.: The VR Book: Human-Centered Design for Virtual Reality. Association for Computing Machinery and Morgan &amp;H38; Claypool, New York, NY, USA (2016)

14. Kairam, S., Brzozowski, M.J., Huffaker, D., Chi, E.H.: Talking in circles: Selective sharing in google+. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 12). pp. 1065–1074. New York, NY (2012), http://doi.acm.org/10.1145/2208516.2208552

15. Karr-Wisniewski, P., Wilson, D., Richter-Lipford, H.: A new social order: Mechanisms for social network site boundary regulation. In: Americas Conference on Information Systems, AMCIS (2011)

16. Kelley, P.G., Brewer, R., Mayer, Y., Cranor, L.F., Sadeh, N.: An investigation into facebook friend grouping. In: IFIP Conference on Human-Computer Interaction. pp. 216–233. Springer (2011)

17. Lee, K.C., Chung, N.: Empirical analysis of consumer reaction to the virtual reality shopping mall. Computers in Human Behavior **24**(1), 88104 (Jan 2008). https://doi.org/10.1016/j.chb.2007.01.018, https://doi.org/10.1016/j.chb.2007.01.018

18. Liu, Y., Mondal, M., Viswanath, B. Mondal, M., Gummadi, K.P., Mislove, A.: Simplifying Friendlist Management. In: Proceedings of the Twenty-First International World Wide Web Conference (WWW’12). Lyon, France (April 2012)

19. Mazzia, A., LeFevre, K., Adar, E.: The pviz comprehension tool for social network privacy settings. In: Proceedings of the Eighth Symposium on Usable Privacy and Security. pp. 13:1–13:12. SOUPS ’12, ACM, New York, NY, USA (2012). https://doi.org/10.1145/2335356.2335374, http://doi.acm.org/10.1145/2335356.2335374

20. Mondal, M., Liu, Y., Viswanath, B., Gummadi, K.P., Mislove, A.: Understanding and specifying social access control lists. In: Symposium on Usable Privacy and Security (SOUPS). vol. 11 (2014)

21. Nawaz, A.: A comparison of card-sorting analysis methods. In: APCHI ’12. Proceedings of the 10th Asia Pacific Conference on Computer-Human Interaction. vol. 2, pp. 583–592. Association for Computing Machinery, United States (2012)

22. Raber, F., Krüger, A.: Deriving privacy settings for location sharing: Are context factors always the best choice? In: 2018 IEEE Symposium on Privacy-Aware Computing (PAC). pp. 86–94 (Sep 2018). https://doi.org/10.1109/PAC.2018.00015

23. Raber, F., Kosmalla, F., Schneeberger, T., Krueger, A.: Fine-grained privacy setting prediction using a privacy attitude questionnaire and machine learning. In: Bernhaupt, R., G.Dalvi, Joshi, A., O’Neill, J., Winckler, M. (eds.) Human-Computer Interaction - INTERACT 2017. IFIP Conference on Human-Computer Interaction (INTERACT-17), 16th IFIP TC 13 International Conference, September 25-29, Mumbai, India. IFIP, Springer (2017)

24. Raber, F., Krueger, A.: Towards understanding the influence of personality on mobile app permission settings. In: Bernhaupt, R., Dalvi, G., Joshi, A., K. Balkrishan, D., O’Neill, J., Winckler, M. (eds.) Human-Computer Interaction – INTERACT 2017. pp. 62–82. Springer International Publishing, Cham (2017)

25. Raber, F., Ziemann, D., Krüger, A.: The retailio privacy wizard: Assisting users with privacy settings for intelligent retail stores. In: Weir, C., Mazurek, M. (eds.) EuroUSEC ’18 : 3rd European Workshop on Usable Security. EuroUSEC European Workshop on Usable Security (EuroUSEC-18), April 23-23, London, UCL, United Kingdom. Internet Society (2018)
26. Rugg, G., McGeorge, P.: The sorting techniques: a tutorial paper on card sorts, picture sorts and item sorts. Expert Systems 14(2), 80–93 (1997). https://doi.org/10.1111/1468-0394.00045, https://onlinelibrary.wiley.com/doi/abs/10.1111/1468-0394.00045

27. Slater, M.: Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical transactions of the Royal Society of London. Series B, Biological sciences 364, 3549–57 (12 2009). https://doi.org/10.1098/rstb.2009.0138

28. Speicher, M., Hell, P., Daiber, F., Simeone, A., Krüger, A.: A virtual reality shopping experience using the apartment metaphor. In: Proceedings of the 2018 International Conference on Advanced Visual Interfaces. pp. 17:1–17:9. AVI ’18, ACM, New York, NY, USA (2018). https://doi.org/10.1145/3206505.3206518, http://doi.acm.org/10.1145/3206505.3206518

29. Watson, J., Besmer, A., Lipford, H.R.: +Your Circles: Sharing Behavior on Google+. In: Proceedings of the Eighth Symposium on Usable Privacy and Security. pp. 12:1–12:9. SOUPS ’12, ACM, New York, NY, USA (2012). https://doi.org/10.1145/2335356.2335373, http://doi.acm.org/10.1145/2335356.2335373

30. Wisniewski, P., Lipford, H., Wilson, D.: Fighting for my space: Coping mechanisms for sns boundary regulation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 609–618. ACM (2012)