# VRSketch: Investigating 2D Sketching in Virtual Reality with Different Levels of Hand and Pen Transparency

Jonas Auda, Roman Heger, Uwe Gruenefeld, Stefan Schneegass

► To cite this version:

Jonas Auda, Roman Heger, Uwe Gruenefeld, Stefan Schneegass. VRSketch: Investigating 2D Sketching in Virtual Reality with Different Levels of Hand and Pen Transparency. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.195-211, 10.1007/978-3-030-85607-6_14. hal-04291227

HAL Id: hal-04291227

https://inria.hal.science/hal-04291227v1

Submitted on 17 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# VRSketch: Investigating 2D Sketching in Virtual Reality with Different Levels of Hand and Pen Transparency

Jonas Auda, Roman Heger, Uwe Gruenefeld, and Stefan Schneegass

University of Duisburg-Essen, Schützenbahn 70, 45127 Essen, Germany

<first_name>.<last_name>@uni-due.de

https://www.hci.wiwi.uni-due.de/

<figure><img src="image_3.png" /><figcaption>Fig. 1: The five transparency variations of hand and pen for sketching in VR: (a) both opaque $H_{100}P_{100}$, (b) both semitransparent $H_{50}P_{50}$, (c) hand invisible and pen opaque $H_0P_{100}$, (d) hand opaque and pen is replaced by a cursor $H_{100}P_0$, and (e) only the cursor with invisible pen and hand $H_0P_0$. Best seen in color.</figcaption></figure>

**Abstract.** Sketching is a vital step in design processes. While analog sketching on pen and paper is the defacto standard, Virtual Reality (VR) seems promising for improving the sketching experience. It provides myriads of new opportunities to express creative ideas. In contrast to reality, possible drawbacks of pen and paper drawing can be tackled by altering the virtual environment. In this work, we investigate how hand and pen transparency impacts users' 2D sketching abilities. We conducted a lab study ($N = 20$) investigating different combinations of hand and pen transparency. Our results show that a more transparent pen helps one sketch more quickly, while a transparent hand slows down. Further, we found that transparency improves sketching accuracy while drawing in the direction that is occupied by the user's hand.

**Keywords:** Virtual Reality · sketching · transparency · occlusion.

## 1 Introduction

Virtual Reality (VR) headsets have become increasingly popular for both consumers and professionals in recent years. While some use their headsets only for
entertainment purposes, VR looks promising for serious tasks such as 3D modeling [4], note taking [18], or exploring spreadsheets [7], among others. Bringing existing applications to VR is not restricted to implementing their original functionalities. For sketching, VR allows one to implement new ideas and features that are not feasible in the real world e.g. 3D modeling [11] or sketching in mid-air [4]. Moreover, VR enables users to be immersed in their favorite surroundings without any visual distractions as they would appear, for example, in an open office space. Further, VR allows the investigation of creative content in 3D space alone or together with others [6]. For example, an artist could get an impression of how a painting appears in a museum, gallery, or to viewers. Designers could quickly sketch a logo and add it to a product to get first impressions of their work [12] or feedback from customers. Engineers could sketch ideas, discuss implications of different design decisions in the context of technical drawings. Moreover, physics laws do not restrict the possibilities of such a sketching environment, enabling, for example, a transparent hand or pen which are not prone to occlusion. Nevertheless, while hand or pen transparency for sketching in VR sounds promising, to our knowledge, its effect on user performance has not been investigated in research thus far. Insight into the application of transparency to sketching utilities or the user in VR and its influence on the performance of the user could help VR designers and developers to improve future applications and experiences and enhance user performance by applying transparency to certain virtual objects.

Previous work has frequently explored hand transparency for integrating physical keyboards in VR, enabling occlusion-free typing [13,27]. Their study results look promising, suggesting that novice users benefit most from transparent hands [13]. For sketching in VR, different commercial solutions exist (e.g., Google Tilt Brush<fnref n="1" /> and Gravity Sketch<fnref n="2" />). Additionally, some researchers explored sketching experiences in VR [5,4]. However, all existing solutions focus on 3D sketching only, using VR controller-input (e.g., Gravity Sketch) or pen-input with different types of haptic feedback [5,4]. So far, little research explored 2D sketching in VR, which remains relevant, for example, for early design stages or user interface design. More importantly, no existing studies provide a systematic evaluation of users’ performance with controller/pen or hand transparency.

In this paper, we investigate the effect of different levels of hand and pen transparency on 2D sketching in Virtual Reality. To enable accurate sketching in VR, we developed a sketching system called *VRSketch* that allows tracking of a physical pen, the user's hand, and a sheet of paper to sketch on. The tracked items are integrated into the Virtual Reality experience in real-time, enabling fluid sketching. In a user study, we compare sketching performances for different hand and pen transparency levels for drawing on a 2D surface; a sheet of paper (c.f., Figure 1). Our results show that higher pen transparency allows users to sketch faster, while not losing accuracy. Moreover, while drawing participants

<footnote n="1">¹ Google Tilt Brush. https://www.tiltbrush.com, last retrieved June 9, 2021</footnote>

<footnote n="2">² Gravity Sketch. https://www gravitysketch.com, last retrieved June 9, 2021</footnote>
achieved a mean deviation of slightly above 0.1cm for each of the investigated techniques, indicating overall good performance for 2D sketching in VR.

## 1.1 Contribution

Our contribution is twofold: 1) we propose a system called *VRSketch* that allows sketching in Virtual Reality, and 2) conduct a comparative evaluation of five different levels of hand and pen transparency to understand the impact on users' 2D sketching performance.

# 2 Related Work

In the following, we review previous work exploring pen input for Augmented
and Virtual Reality, and hand/pen occlusion for different input modalities.

## 2.1 Pen Input for Augmented and Virtual Reality

As pens offer users a familiar form of input, they have been frequently investi-
gated for Augmented Reality (AR) and Virtual Reality (VR).

For AR, researchers explored how digital pen input can be used to anno-
tate analog paper documents, augmented via either projection [28,10,22] or by
using an Head-Mounted Display (HMD) [16]. Interestingly, annotations written
with the help of an AR pen are processable with Optical Character Recogni-
tion (OCR), and the resulting text can serve as input to interact with applica-
tions [16]. Beyond written text, pen input also allows direct ways of interacting
with AR applications, for example, to navigate menus [25]. Moreover, previous
works investigated pen input in AR for 3D modeling, empowering users to design
based on three-dimensional real-world objects [26].

For VR, researchers examined different interaction types with a digital pen in different scenarios. For example, for pointing and selecting interactions [17] in scenarios such as interacting with spreadsheets [7]. A interest of previous work is text input either by selecting letters on a virtual keyboard [3] or with the use of OCR [8]. Moreover, previous work studied sketching in VR using a pen as the input device. Here, an early approach is the Virtual Notepad by Poupyrev et al. [18]. The Virtual Notepad enables users to take notes and sketches in an Virtual Environment (VE), using a tracked tablet and pen. In later years, sketching with a pen in VR was primarily used for 3D sketching, often in the context of 3D modeling. In this context, either by expanding base sketches in the third dimension by lifting out single lines with pens [11] or by sketching lines mid-air [2,4,5]. The main focus of recent research on sketching mid-air is to create a believable haptic sensation for users. Results show that constraining the degrees of freedom by, for example, sketching on movable physical surfaces allows for higher accuracy [2,5] and can enhance interactions [4]. Further, virtual environments can provide other helpful features like gridlines that allow the user to draw 3D sketches by hand [19].
In sum, for sketching in VR, researchers focused mostly on 3D sketching, aiming for believable haptic sensations when drawing mid-air. Thus, typical 2D sketching experiences received little attention, while they remain relevant for many use-cases and allow for more straightforward to implement haptic feedback.

## 2.2 Hand and Pen Occlusion for Input

One problem when using pens for input is the obscuring of content or interface elements. When using a pen on a tablet, up to 47% of a 12" display can be hidden by hand, pen, and arm [24]. Besides hiding parts of interface, it can also result in a loss of precision and speed during input [14,1]. To avoid occlusion, interfaces can detect occlusion and display content in visible areas [24,30] or add offsets to controls [23]. However, while this improves precision for targeting tasks, it decreases the precision for tracing operations like sketching [15].

Another approach to compensate for occlusion while sketching is replacing the hardware pen tip with a semitransparent one rendered on the tablet [14]. A semitransparent pen tip leads to a 40% reduction in error rate and an improvement in drawing speed of up to 26% [14]. We adopt this promising concept to VR and take it further by applying the transparency to the pen and the hand.

# 3 Sketching in Virtual Reality

The goal of our work is to understand the influence of hand and pen transparency on a user's 2D sketching performance in VR. Inspired by the idea of the PhantomPen [14], we extended the concept to include both the user's hand and the used pen. We hypothesize that transparency can improve performance, empowering users to sketch more precisely and quickly than they otherwise could. Furthermore, we are interested in optimizing the experience and precision of sketching in VR. To investigate VR sketching, we implemented the *VRSketch* system that allows real-time tracking of a physical pen, the user's hand, a sheet of paper, and a table.

To systematically explore the design space, we first identified pen and hand as two involved entities that may be improved by transparent rendering. Then, we continued by differentiating three levels of transparency (similar to the work of Knierim et al. [13]) that are invisible (0% opacity), semi-transparent (50% opacity) and opaque (100% opacity) for the hand and pen each. Semitransparency in particular has the potential to help with spatial orientation by displaying information without occlusion of content [29]. The complete design space and the selected evaluation conditions are presented in Figure 2.

From the design space, we selected the following combinations of hand and pen transparency as conditions for our comparative study:

* **H₁₀₀P₁₀₀** is our baseline condition in which we render the user's hand and pen fully opaque, similar to a real-world environment (see Figure 1a).

* **H₅₀P₅₀** renders both hand and pen semi-transparent, providing spatial information and paper content (see Figure 1b).
*H₀P₁₀₀* shows the pen as fully opaque with no transparency, but it does not render the user's hand (see Figure 1c).

*H₁₀₀P₀* displays the user's hand as opaque with no transparency, while the pen is reduced to a small cursor point, representing the pen's tip (see Figure 1d).

*H₀P₀* removes all occlusion caused by hand and pen, rendering only the small cursor representing the tip of the pen (see Figure 1e).

# 4 Evaluation

To investigate 2D sketching in VR and the benefits of semi- and full-transparency for pen and drawing hand, we conducted a comparative user study with the selected conditions from the design space (see Figure 2). We opted for these conditions as they seemed promising to uncover the effects of transparency on sketching while keeping the experiment time within a reasonable limit. Especially the semi-transparency applied to the pen and hand seemed promising from the literature [13]. Future research might investigate the remaining conditions of the design space.

<table><caption>Fig. 2: The design space for hand and pen transparency and the five investigated conditions for 2D sketching in VR.</caption><thead><tr><th rowspan="2"></th><th rowspan="2">Opacity</th><th colspan="3">Hand</th></tr><tr><th>0%</th><th>50%</th><th>100%</th></tr></thead><tbody><tr><th rowspan="3">Pen</th><th>0%</th><td>H₀P₀</td><td></td><td>H₁₀₀P₀</td></tr><tr><th>50%</th><td></td><td>H₅₀P₅₀</td><td></td></tr><tr><th>100%</th><td>H₀P₁₀₀</td><td></td><td>H₁₀₀P₁₀₀</td></tr></tbody></table>

## 4.1 Study Design

To investigate different pen and hand transparency levels for sketching in VR, we conducted a within-subjects controlled laboratory user study in Virtual Reality with the Oculus Rift headset. Our independent variables were technique with five levels (*H*₁₀₀*P*₁₀₀ vs. *H*₅₀*P*₅₀ vs. *H*₀*P*₁₀₀ vs. *H*₁₀₀*P*₀ vs. *H*₀*P*₀, see Figure 1) and line type with two levels (*connected* vs. *unconnected*). Each technique was tested in a block consisting of four measured trials, with two trials evaluating *connected* lines and two trials evaluating *unconnected* lines. In each trial, participants had to draw a pattern consisting of 16 lines, drawing 64 lines for each block in total. To make the task more realistic, we varied the lines' orientation, introducing 16 different orientations (starting at 0-degree with 22.5-degree steps). Within each block, each line orientation was tested twice for each of both line types. We counterbalanced all blocks and the line types within each block using a Latin-square design to avoid learning effects. We used quantitative methods to evaluate
sketching performance, taking pattern completion time, sketching accuracy, and
the questionnaires as our dependent variables.

For this study, we asked: **(RQ) Which level of transparency for hand and pen results in the best sketching performance in Virtual Reality?** We posit the following hypotheses:

$H_1$ Semi-transparent rendering of the user's hand results in the shortest pattern completion times because it allows users to see the paper underneath while not losing spatial understanding.

$H_2$ We expect higher sketching accuracy for all conditions that render the pen semi-transparent or opaque compared to conditions in which it is fully transparent and replaced by a cursor because the cursor does not convey posture.

## 4.2 Apparatus

We implemented the *VRSketch* system to enable 2D sketching via pen in VR. We create an empty virtual room, centered around a sketching table, presented on the Oculus Rift headset. The scene was created using the Unity game engine 2018.2.20f1 and was running on a Windows PC with an Intel i7-7700K, 32GB RAM, and an Nvidia Geforce GTX 1080 Ti. We spatially synchronized VR and reality by tracking the real-world scene with an OptiTrack system and its Motive 2.2.0 motion capture software. The tracking apparatus involved seven OptiTrack Prime× 13W cameras near the sketching table to enable a high precision capturing of the sketching movements (see Figure 3a). Furthermore, four additional OptiTrack Prime× 13 cameras were placed at a greater distance for more general tracking. For the physical representations, we used a 3D printed pen and a DIN A4 sheet of paper, both shown in Figure 3a. The paper was glued to a thin sheet of acrylic glass for durability and flatness. Both had a unique configuration of retro-reflective markers to get tracked as rigid bodies by the OptiTrack System. Besides, the user's hand was tracked by wearing a thin glove with markers. Thus, we could render both the hand's general position and the grip motion when picking up the pen. We also tracked the table, the chair, and the VR headset to complete the spatial synchronization. After initial positioning, the head movement was tracked by the sensors of the HMD. The lines, sketched by the user, are determined and rendered by the Unity application via calculating the pen tip's contact points with the paper. For measuring the sketching precision, the calculated line points were logged with timestamps. We controlled the degree of transparency for hand and pen via adjusting the alpha channel of according texture in the Unity game engine.

## 4.3 Participants

We recruited 20 volunteer participants (7 female), aged between 19 and 60 years (M=33.3, SD=13.7). None suffered from color vision impairments. Participants with corrected-to-normal vision were requested to wear their contacts or glasses during the study. We asked participants to rate their sketching skills on a 7-point
<figure><img src="image_4.png" /><figcaption>Fig. 3: a) Hardware setup of the VRSketch system where hand, pen, paper, table, and chair are tracked via configurations of retro-reflective markers. Seven of the eleven OptiTrack cameras are close to the table for more precise tracking. b) Overview of the unconnected test patterns in the upper row and the connected ones in the lower row. The lines had to be drawn in the direction of the arrows.</figcaption></figure>

Likert-scale from 1 (cannot sketch at all) to 7 (can sketch on a professional level). Participants stated that they had limited sketching skills (Md=2.05, IQR=2.0). Furthermore, we asked participants for their experience with Virtual Reality. Five participants had never tried VR before, three used it once, and twelve participants said they use a VR headset regularly (at least once a month).

## 4.4 Procedure

At the beginning of the study, we informed participants about the procedure and asked them to sign a consent form. Afterward, we collected the participant's demographic data, sketching skills, and experience with VR. We then introduced the participant to the Oculus Rift and adjusted the headset for optimal fit and correct interpupillary distance. Then, we started the study. The study was conducted in five blocks for each participant, with one technique tested in each block. We counterbalanced all blocks using a Latin-square design. In each block, participants first took a seat at the sketching table, put on the tracked glove, and the HMD and picked up the tracked pen. Each block started with a warm-up pattern, which participants could try until they indicated that they were familiar with that block's respective technique. After the warm-up, participants continued with the measured trails. Participants had to trace lines in four test patterns for each block, two unconnected, and two connected ones (see Figure 3b). After one pattern was complete, the experimenter started the next pattern. After all four patterns were complete, the participants could take off the headset, pen, and glove, and fill out the questionnaires: UEQ-S [20], NASA Raw-TLX [9] and IPQ [21]. After completing all blocks, we conducted a final interview with the participants asking them about their impressions of sketching in VR and the individual techniques. Each participant took approximately 70 minutes to finish the experiment.
## 4.5 Data Preparation

In addition to the observations of users' impressions, sketching precision is used for the quantitative evaluation of the different techniques. We use the mean deviations of the drawn lines from the corresponding target lines of the patterns to measure precision. Four out of 400 (1%) recorded patterns were corrupted due to technical difficulties and replaced with the same participant's matching pattern of the same technique. We first corrected the lines' position and rotation according to the paper's position to calculate the mean deviations (see Figure 4).

<figure><img src="image_5.png" /><figcaption>Fig. 4: To calculate the mean deviation, the points of the sketched lines (a) are assigned to the lines of the target pattern (b). The assigned points are rotated around the center of the target line and the center is moved to the origin (c). The points are restricted to the area between the start and end of the target line, and the sketched line is resampled with 100 equidistant points (d).</figcaption></figure>

The line points were each assigned to a specific target line, as shown in Figure 4b and c. A point was always assigned if its minimum distance to the target line was less than 1 cm, whereby in the case of connected lines, the bisector between two lines served as the limit for the assignment. The lines were resampled at 100 equidistant points in line with previous work [2,26] (see Figure 4e). The mean deviation of a drawn line from its target line is then calculated as the arithmetic means of the Y-values' amounts at the measurement points.

## 4.6 Results

In the following, we present the results from our study analysis. We use mean (M), standard deviation (SD) to describe our data. We do not assume normal-distribution of our data, and thus, apply non-parametric tests. We ran Friedman tests and post-hoc Wilcoxon Signed-rank tests with Bonferroni correction to show significant differences.

**Pattern Completion Time** To understand how quickly participants were able to sketch with each technique, we looked at their pattern completion times. The times in ascending order are: $H_{100}P_0=41.88$ s (SD=16.25s), $H_0P_{100}=44.08$ s
<table><caption>Table 1: Significant comparisons of pattern completion times for the different techniques (with r: > 0.1 small, > 0.3 medium, and > 0.5 large effect).</caption><thead><tr><th>Comparison</th><th>W</th><th>Z</th><th>p</th><th>r</th></tr></thead><tbody><tr><td>H<sub>100</sub>P<sub>100</sub> vs. H<sub>50</sub>P<sub>50</sub></td><td>2291</td><td>3.22</td><td>0.011</td><td>0.25</td></tr><tr><td>H<sub>100</sub>P<sub>100</sub> vs. H<sub>100</sub>P<sub>0</sub></td><td>2654</td><td>4.96</td><td>&lt;0.001</td><td>0.39</td></tr><tr><td>H<sub>50</sub>P<sub>50</sub> vs. H<sub>100</sub>P<sub>0</sub></td><td>2318</td><td>3.35</td><td>0.007</td><td>0.26</td></tr><tr><td>H<sub>100</sub>P<sub>0</sub> vs. H<sub>0</sub>P<sub>0</sub></td><td>742</td><td>-4.21</td><td>&lt;0.001</td><td>0.33</td></tr></tbody></table>

(SD=15.37s), H<sub>50</sub>P<sub>50</sub>=45.86s (SD=19.01s), H<sub>0</sub>P<sub>0</sub>=48.32s (SD=23.13s), and H<sub>100</sub>-P<sub>100</sub>=50.17s (SD=21.66s). Figure 5 compares the pattern completion times. A Friedman test revealed a significant effect of technique on pattern completion time (χ<sup>2</sup>(4)=35.91, p<0.001, N=20). Post-hoc tests showed significant differences between some of the evaluated conditions (see Table 1). For the completion time, we conclude: H<sub>100</sub>P<sub>0</sub> < H<sub>50</sub>P<sub>50</sub> < H<sub>100</sub>P<sub>100</sub> and H<sub>100</sub>P<sub>0</sub> < H<sub>0</sub>P<sub>0</sub>. For H<sub>0</sub>P<sub>100</sub> we cannot make a statement.

<figure><img src="image_6.png" /><figcaption>Fig. 5: Boxplots of pattern completion times for the different techniques.</figcaption></figure>

**Sketching Accuracy** Throughout the study, participants drew exactly 6400 lines. To evaluate the sketching accuracy of each technique, we applied our data preparation step described in Section 4.5. The mean deviations of each line within each technique in ascending order are: H<sub>100</sub>P<sub>100</sub>=1.02mm (SD=0.55mm), H<sub>50</sub>P<sub>50</sub>=1.04mm (SD=0.55mm), H<sub>0</sub>P<sub>100</sub>=1.06mm (SD=0.55mm), H<sub>100</sub>P<sub>0</sub>=1.06mm (SD=0.6mm), and H<sub>0</sub>P<sub>0</sub>=1.08mm (SD=0.58mm). The mean deviations are compared in Figure 6. We applied a Friedman test, which revealed no significant differences between the techniques (χ<sup>2</sup>(4)=8.23, p=0.083, N=20).

**Sketching Accuracy for Different Sketching Directions** The area in the direction of sketching can be occluded, for example, by the virtual pen or the hand of the VR user. Hence, the sketching direction could influence sketching performance. To gain further insights about the effect of the transparency, we reviewed the influence of the sketching direction on the sketching accuracy by clustering the different line orientation into quadrants. The quadrants are Q1:
<figure><img src="image_7.png" /><figcaption>Fig. 6: Boxplot of the mean sketching deviations for the different techniques.</figcaption></figure>

upper right, Q2: upper left, Q3: lower left, and Q4 lower right. For example, if a line is drawn towards the upper left relative to its starting point, it belongs to Q2. The edge cases are clustered as follows: drawing upwards Q1, drawing to the left Q2, drawing downwards Q3, and drawing to the right Q4. The mean deviations for each technique and quadrant are shown in Table 2. To analyze the data of the different quadrants, we compared both the different techniques in each quadrant and the different quadrants of each technique.

<table><caption>Table 2: The mean sketching deviation (in mm) per technique and quadrant.</caption><thead><tr><th>Technique</th><th>Q1</th><th>Q2</th><th>Q3</th><th>Q4</th></tr></thead><tbody><tr><td>H<sub>100</sub>P<sub>100</sub></td><td>0.97 (SD 0.51)</td><td>0.96 (SD 0.46)</td><td>0.92 (SD 0.49)</td><td>1.06 (SD 0.54)</td></tr><tr><td>H<sub>0</sub>P<sub>100</sub></td><td>1.04 (SD 0.53)</td><td>1.03 (SD 0.55)</td><td>0.94 (SD 0.49)</td><td>1.09 (SD 0.57)</td></tr><tr><td>H<sub>100</sub>P<sub>0</sub></td><td>1.05 (SD 0.61)</td><td>1.08 (SD 0.51)</td><td>1.02 (SD 0.58)</td><td>1.09 (SD 0.65)</td></tr><tr><td>H<sub>0</sub>P<sub>0</sub></td><td>1.05 (SD 0.61)</td><td>1.13 (SD 0.58)</td><td>1.06 (SD 0.6)</td><td>1.11 (SD 0.56)</td></tr><tr><td>H<sub>50</sub>P<sub>50</sub></td><td>1.08 (SD 0.59)</td><td>1.15 (SD 0.61)</td><td>1.08 (SD 0.57)</td><td>1.14 (SD 0.61)</td></tr></tbody></table>

*Comparison of Techniques within Quadrants.* We performed Friedman tests for each quadrant. For Q1 ($\chi^2(4)=4.15$, p=0.386, N=20) and Q4 ($\chi^2(4)=3.88$, p=0.422, N=20), we observed no significant differences between the techniques. However, the Friedman tests for Q2 ($\chi^2(4)=24.09$, p=0, N=20) and Q3 ($\chi^2(4)=20.64$, p=0, N=20) revealed a significant effect of technique on the mean deviation. Post-hoc tests showed significant differences between some of the conditions (see Table 3). We conclude that $H_{100}P_{100}$ leads to significantly higher accuracy than $H_{50}P_{50}$, $H_0P_{100}$, and $H_0P_0$ in Q2 and that $H_{50}P_{50}$ and $H_0P_{100}$ lead to significantly higher accuracy than $H_{100}P_0$ and $H_0P_0$ in Q3.

*Comparison of Quadrants of each Technique.* For the comparison of mean deviations in the different quadrants for each technique the Friedman tests for the techniques $H_{100}P_0$ ($\chi^2(3)=3.61$, p=0.307, N=20) and $H_0P_0$ ($\chi^2(3)=6.3$, p=0.098, N=20) revealed no significant differences. For the techniques $H_{100}P_{100}$ ($\chi^2(3)=12.1$, p=0.007, N=20), $H_{50}P_{50}$ ($\chi^2(3)=19.19$, p=0, N=20), and $H_0P_{100}$ ($\chi^2(3)=14.79$, p=0.002, N=20) the Friedman tests revealed a significant effect
<table><caption>Table 3: Pairwise comparisons of mean deviations with significant results for the different techniques in the quadrants Q2 and Q3.</caption><thead><tr><th>Quadrant</th><th>Comparison</th><th>W</th><th>Z</th><th>p</th><th>r</th></tr></thead><tbody><tr><td>Q2</td><td>H<sub>100</sub>P<sub>100</sub> vs. H<sub>50</sub>P<sub>50</sub></td><td>20404</td><td>-3.19</td><td>0.014</td><td>0.13</td></tr><tr><td>Q2</td><td>H<sub>100</sub>P<sub>100</sub> vs. H<sub>0</sub>P<sub>100</sub></td><td>18443</td><td>-4.37</td><td>&lt;0.001</td><td>0.17</td></tr><tr><td>Q2</td><td>H<sub>100</sub>P<sub>100</sub> vs. H<sub>0</sub>P<sub>0</sub></td><td>19110</td><td>-3.97</td><td>0.001</td><td>0.16</td></tr><tr><td>Q3</td><td>H<sub>50</sub>P<sub>50</sub> vs. H<sub>100</sub>P<sub>0</sub></td><td>20066</td><td>-3.39</td><td>0.007</td><td>0.13</td></tr><tr><td>Q3</td><td>H<sub>50</sub>P<sub>50</sub> vs. H<sub>0</sub>P<sub>0</sub></td><td>19947</td><td>-3.46</td><td>0.005</td><td>0.14</td></tr><tr><td>Q3</td><td>H<sub>0</sub>P<sub>100</sub> vs. H<sub>100</sub>P<sub>0</sub></td><td>20438</td><td>-3.16</td><td>0.015</td><td>0.13</td></tr><tr><td>Q3</td><td>H<sub>0</sub>P<sub>100</sub> vs. H<sub>0</sub>P<sub>0</sub></td><td>20780</td><td>-2.96</td><td>0.03</td><td>0.12</td></tr></tbody></table>

of the quadrants on the mean deviation. Post-hoc tests showed significant differences between some of comparisons (see Table 4).

<table><caption>Table 4: Pairwise comparisons of mean deviations with significant results for the techniques $H_{100}P_{100}$, $H_{50}P_{50}$, and $H_{0}P_{100}$.</caption><thead><tr><th>Technique</th><th>Comparison</th><th>W</th><th>Z</th><th>p</th><th>r</th></tr></thead><tbody><tr><td>$H_{100}P_{100}$</td><td>Q1 vs. Q4</td><td>19307</td><td>-3.85</td><td>0.001</td><td>0.15</td></tr><tr><td>$H_{100}P_{100}$</td><td>Q2 vs. Q4</td><td>18994</td><td>-4.04</td><td>&lt;0.001</td><td>0.16</td></tr><tr><td>$H_{100}P_{100}$</td><td>Q3 vs. Q4</td><td>21033</td><td>-2.81</td><td>0.03</td><td>0.11</td></tr><tr><td>$H_{50}P_{50}$</td><td>Q1 vs. Q3</td><td>31759</td><td>3.67</td><td>0.001</td><td>0.15</td></tr><tr><td>$H_{50}P_{50}$</td><td>Q2 vs. Q3</td><td>32367</td><td>4.04</td><td>&lt;0.001</td><td>0.16</td></tr><tr><td>$H_{50}P_{50}$</td><td>Q3 vs. Q4</td><td>19011</td><td>-4.03</td><td>&lt;0.001</td><td>0.16</td></tr><tr><td>$H_{0}P_{100}$</td><td>Q2 vs. Q3</td><td>33294</td><td>4.6</td><td>&lt;0.001</td><td>0.18</td></tr><tr><td>$H_{0}P_{100}$</td><td>Q3 vs. Q4</td><td>19255</td><td>-3.88</td><td>0.001</td><td>0.15</td></tr></tbody></table>

Here we conclude that for technique $H_{100}P_{100}$ Q4 is significantly worse than for all other quadrants, that for $H_{50}P_{50}$ Q3 is significantly better than all other quadrants, and that for $H_{0}P_{100}$ Q3 is significantly better than Q2 and Q4.

**Questionnaires** Furthermore, we asked participants to fill out three different questionnaires (NASA Raw-TLX, User Experience Questionnaire, and iGroup Presence Questionnaire) after each technique. In the following, we report on the gathered results using median and interquartile range (IQR).

*NASA Raw-TLX.* To evaluate the workload of the different techniques, we analyzed the results of the NASA-TLX. The median scores in ascending order are: $H_{100}P_{0}$=18.75 (IQR=19.58), $H_{0}P_{0}$=19.17 (IQR=21.25), $H_{0}P_{100}$=20.42 (IQR=12.08), $H_{100}P_{100}$=20.83 (IQR=20.62), $H_{50}P_{50}$=22.92 (IQR=14.17). To compare the scores, we conducted a Friedman test that revealed no significant effect of technique on the NASA Raw-TLX score ($\chi^2(4)=5.83$, p=0.212, N=20).
*User Experience Questionnaire.* For insights on the user experience, we conducted the short version of the UEQ (see Table 5).

<table><caption>Table 5: Results of the UEQ-S for the different techniques.</caption><thead><tr><th>Technique</th><th>Pragmatic Quality<br>Median</th><th>IQR</th><th>Hedonic Quality<br>Median</th><th>IQR</th><th>Overall Quality<br>Median</th><th>IQR</th></tr></thead><tbody><tr><td>H<sub>100</sub>P<sub>100</sub></td><td>1.0</td><td>2.31</td><td>1.62</td><td>1.0</td><td>1.19</td><td>1.56</td></tr><tr><td>H<sub>50</sub>P<sub>50</sub></td><td>1.75</td><td>1.62</td><td>1.88</td><td>1.31</td><td>1.75</td><td>1.22</td></tr><tr><td>H<sub>0</sub>P<sub>100</sub></td><td>1.62</td><td>2.12</td><td>2.0</td><td>1.31</td><td>1.62</td><td>1.28</td></tr><tr><td>H<sub>100</sub>P<sub>0</sub></td><td>0.75</td><td>1.62</td><td>1.62</td><td>0.81</td><td>1.38</td><td>0.81</td></tr><tr><td>H<sub>0</sub>P<sub>0</sub></td><td>1.5</td><td>1.81</td><td>1.5</td><td>0.81</td><td>1.62</td><td>1.22</td></tr></tbody></table>

To compare the overall quality for the individual techniques, we conducted a Friedman test which revealed a significant effect. However, a post-hoc tests did not reveal any significant differences.

*iGroup Presence Questionnaire.* The results of the iGroup Presence Questionnaire (IPQ) are shown in Table 6. A Friedman test revealed a significant effect of technique on overall score. Post-hoc tests showed a significant difference between $H_{100}P_{100}$ and $H_0P_{100}$ (W=16, Z=-2.77, p=0.04, r=0.44), meaning that rendering hand and pen opaque results in lower presence, than rendering only the pen opaque and not the hand.

<table><caption>Table 6: Results of the IPQ for the different techniques.</caption><thead><tr><th>Technique</th><th>General Presence<br>Median</th><th>IQR</th><th>Spatial Presence<br>Median</th><th>IQR</th><th>Exp. Realism<br>Median</th><th>IQR</th><th>Involvement<br>Median</th><th>IQR</th><th>Overall Score<br>Median</th><th>IQR</th></tr></thead><tbody><tr><td>H<sub>100</sub>P<sub>100</sub></td><td>4.0</td><td>1.0</td><td>4.2</td><td>1.05</td><td>2.75</td><td>0.81</td><td>2.75</td><td>1.19</td><td>3.32</td><td>0.96</td></tr><tr><td>H<sub>50</sub>P<sub>50</sub></td><td>4.0</td><td>1.0</td><td>4.3</td><td>1.25</td><td>2.75</td><td>1.31</td><td>2.75</td><td>0.88</td><td>3.54</td><td>1.07</td></tr><tr><td>H<sub>0</sub>P<sub>100</sub></td><td>4.5</td><td>1.0</td><td>4.1</td><td>1.5</td><td>3.0</td><td>1.06</td><td>3.0</td><td>1.25</td><td>3.46</td><td>0.91</td></tr><tr><td>H<sub>100</sub>P<sub>0</sub></td><td>4.0</td><td>1.0</td><td>4.0</td><td>1.1</td><td>2.75</td><td>1.25</td><td>2.75</td><td>0.75</td><td>3.39</td><td>0.68</td></tr><tr><td>H<sub>0</sub>P<sub>0</sub></td><td>4.0</td><td>1.0</td><td>3.9</td><td>1.45</td><td>2.88</td><td>0.94</td><td>2.88</td><td>1.06</td><td>3.11</td><td>0.89</td></tr></tbody></table>

# 5 Discussion

In the following, we discuss the most important findings of our user study.

*Pattern Completion Time* In our results, we found that the more the opacity of the pen is reduced, the faster participants were able to sketch. This result is in line with similar findings in previous work. For example, Lee et al. found that rendering the pen tip transparent also increases the sketching speed [14]. In
contrast, reducing the opacity of the hand resulted in longer pattern completion
times. However, in *H*₁, we expected a semi-transparent rendering of the user's
hand would result in the shortest completion time. We could not verify this
in our study, and hence, cannot accept our hypothesis *H*₁. However, mixing
transparency and opacity, one on the hand and one on the pen, resulted in
shorter completion times compared to both elements being fully transparent or
fully opaque. This might indicate that providing both overview by transparent
elements and spatial information by visible elements together could indeed be
beneficial. In the future, further research could investigate more fine-grained
levels of transparency to uncover its definite influence on completion time.

*Sketching Accuracy* We found no significant influence of transparency on user's accuracy, neither for transparency of the hand nor for the pen. Therefore, we cannot accept our hypothesis *H*₂. While this result is in line with previous work (e.g., transparent hands for typing on physical keyboards [13]), we expected a higher sketching accuracy for semi-transparent rendering as it empowers users to see otherwise occluded sketch areas. Nonetheless, we think that we did not observe an effect because humans may have adapted to this constraint due to excessive practice (writing with a pen is one of the first skills we learn at school). Overall, the measurements with a maximum mean of 1.08mm for mean deviation show the high precision of VR sketching with *VRSketch*. In comparison, Arora et al. [2] found a mean deviation of 2.54mm (SD=1.87mm) for the data subset with the closest conditions of drawing straight, short lines on a horizontal writing surface using a VR-HMD. We downloaded the corresponding GitHub repository<fnref n="3" /> and applied our algorithm shown in Figure 4. The high precision of sketching with the *VRSketch* system confirms the positive effect of concrete writing surfaces and visual guidance aids, as shown by Arora et al. and Wacker et al. [2,26].

*Accuracy and Sketching Direction* For example, for *H*₁₀₀*P*₁₀₀, sketching in Q4 (downright / below hand and arm) was significantly worse than in all other directions, which shows the influence of occlusion as described by Vogel et al. [24]. In general, from our results, we learned that fully seeing the hand makes it easier to sketch away from arm and hand, while eradicating the pen makes it more challenging to sketch towards the down left quarter. Based on our findings, we suggest that it may be beneficial to adapt the transparency, dependent on the sketching direction dynamically, to reach an optimal accuracy.

*Perceived Workload* For the NASA Raw-TLX [9] questionnaires, we observed that not rendering the pen resulted in a lower workload. In contrast to previous work [13], we found that not rendering the hands did not lead to significantly higher workload. Quite the opposite, transparent and opaque hands and pen resulted in a higher workload. However, these results were not statistically significant.

<footnote n="3">³ https://github.com/rarora7777/VRSketchingStudyCHI17</footnote>
*User Experience and Presence* In the conducted UEQ-S questionnaires, we did not find any significant differences between the techniques. Nevertheless, our findings point in the direction that seeing the hand fully visible results in less pragmatic quality, while overall, the results indicate a good user experience. Seeing only the pen significantly increases presence compared to seeing the pen and virtual hand. This finding is very interesting and in line with some VR games<fnref n="4" /> that as soon as one grabs an object, do not render the users' hands anymore but instead only show the object that the user is holding.

*Limitations* Our work is limited by the rather complex setup that we used to implement our *VRSketch* system. It relies on several expensive OptiTrack sensors and enough space to set up the tracking system. Nonetheless, we argue that as VR advances tracking accuracy improves and in a few years, it may be possible to track physical objects in our surroundings to integrate them in the experience (as is demonstrated with integrated hand-tracking on the Oculus Quest).

# 6 Conclusion

In this work, we investigated five different levels of pen and hand transparency for sketching in VR. We proposed the *VRSketch* system that integrates users' hands and a pen into a virtual sketching environment. Our results show that drawing lines with our *VRSketch* system, on average, results in a mean deviation of slightly above 0.1cm. Moreover, we could show that not seeing the pen, allows users to draw more quickly while not losing accuracy. In the future, we want to experiment with dynamic transparency that adjusts pen and hand rendering based on the user's current sketching or writing direction.

# References

1. Annett, M., Bischof, W.F.: Hands, Hover, and Nibs: Understanding Stylus Accuracy on Tablets. In: Proceedings of the 41st Graphics Interface Conference. pp. 203–210. GI ’15, Canadian Information Processing Society, Toronto, Ont., Canada, Canada (2015), http://dl.acm.org/citation.cfm?id=2788890.2788926, event-place: Halifax, Nova Scotia, Canada

2. Arora, R., Kazi, R.H., Anderson, F., Grossman, T., Singh, K., Fitzmaurice, G.: Experimental Evaluation of Sketching on Surfaces in VR. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 5643–5654. CHI ’17, ACM, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3025474, http://doi.acm.org/10.1145/3025453.3025474, event-place: Denver, Colorado, USA

3. Bowman, D.A., Rhoton, C.J., Pinho, M.S.: Text input techniques for immersive virtual environments: An empirical comparison. In: Proceedings of the human factors and ergonomics society annual meeting. vol. 46, pp. 2154–2158. SAGE Publications Sage CA: Los Angeles, CA (2002). https://doi.org/10.3390/technologies7020031, https://doi.org/10.3390/technologies7020031

<footnote n="4">⁴ Job Simulator. https://store.steampowered.com/app/448280/Job_Simulator, last retrieved June 9, 2021</footnote>
4. Drey, T., Gugenheimer, J., Karlbauer, J., Milo, M., Rukzio, E.: Vrsketchin: Exploring the design space of pen and tablet interaction for 3d sketching in virtual reality. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. p. 1–14. CHI ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3313831.3376628, https://doi.org/10.1145/3313831.3376628

5. Elsayed, H., Barrera Machuca, M.D., Schaarschmidt, C., Marky, K., Müller, F., Riemann, J., Matviienko, A., Schmitz, M., Weigel, M., Mühlhäuser, M.: Vrsketchpen: Unconstrained haptic assistance for sketching in virtual 3d environments. In: 26th ACM Symposium on Virtual Reality Software and Technology. VRST ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3385956.3418953, https://doi.org/10.1145/3385956.3418953

6. Gerry, L.J.: Paint with me: Stimulating creativity and empathy while painting with a painter in virtual reality. IEEE Transactions on Visualization and Computer Graphics 23(4), 1418–1426 (2017). https://doi.org/10.1109/TVCG.2017.2657239

7. Gesslein, T., Biener, V., Gagel, P., Schneider, D., Kristensson, P.O., Ofek, E., Pahud, M., Grubert, J.: Pen-based Interaction with Spreadsheets in Mobile Virtual Reality. arXiv:2008.04543 [cs] (Aug 2020), http://arxiv.org/abs/2008.04543, arXiv: 2008.04543

8. González, G., Molina, J.P., García, A.S., Martínez, D., González, P.: Evaluation of text input techniques in immersive virtual environments. In: New Trends on Human-Computer Interaction, pp. 109–118. Springer (2009). https://doi.org/10.1007/978-1-84882-352-5_11, https://doi.org/10.1007/978-1-84882-352-5_11

9. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In: Advances in Psychology, vol. 52, pp. 139–183. Elsevier (1988). https://doi.org/10.1016/S0166-4115(08)62386-9, https://linkinghub.elsevier.com/retrieve/pii/S0166411508623869

10. Holman, D., Vertegaal, R., Altosaar, M., Troje, N., Johns, D.: Paper windows: interaction techniques for digital paper. In: Proceedings of the SIGCHI conference on Human factors in computing systems. pp. 591–599. ACM (2005). https://doi.org/10.1145/1054972.1055054, https://doi.org/10.1145/1054972.1055054

11. Jackson, B., Keefe, D.F.: Lift-Off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR. IEEE Transactions on Visualization and Computer Graphics 22(4), 1442–1451 (2016). https://doi.org/10.1109/TVCG.2016.2518099

12. Jetter, H.C., Rädle, R., Feuchtner, T., Anthes, C., Friedl, J., Klokmose, C.N.: "in vr, everything is possible!": Sketching and simulating spatially-aware interactive spaces in virtual reality. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. p. 1–16. CHI ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3313831.3376652, https://doi.org/10.1145/3313831.3376652

13. Knierim, P., Schwind, V., Feit, A.M., Nieuwenhuizen, F., Henze, N.: Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. p. 345. ACM (2018). https://doi.org/10.1145/3173574.3173919, https://doi.org/10.1145/3173574.3173919

14. Lee, D., Son, K., Lee, J.H., Bae, S.H.: PhantomPen: Virtualization of Pen Head for Digital Drawing Free from Pen Occlusion &#38; Visual Paral-
lax. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology. pp. 331–340. UIST ’12, ACM, New York, NY, USA (2012). https://doi.org/10.1145/2380116.2380159, http://doi.acm.org/10.1145/2380116.2380159, event-place: Cambridge, Massachusetts, USA

15. Lee, H., Ko, I., Tan, Y.R., Zhang, D., Li, C.: Usability Impact of Occlusion-free Techniques on Commonly-used Multitouch Actions. In: Proceedings of the 5th International ACM In-Cooperation HCI and UX Conference. pp. 96–105. CHIuXiD’19, ACM, New York, NY, USA (2019). https://doi.org/10.1145/3328243.3328256, http://doi.acm.org/10.1145/3328243.3328256, event-place: Jakarta, Surabaya, Bali, Indonesia

16. Li, Z., Annett, M., Hinckley, K., Singh, K., Wigdor, D.: HoloDoc: Enabling Mixed Reality Workspaces That Harness Physical and Digital Content. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. pp. 687:1–687:14. CHI ’19, ACM, New York, NY, USA (2019). https://doi.org/10.1145/3290605.3300917, http://doi.acm.org/10.1145/3290605.3300917, event-place: Glasgow, Scotland Uk

17. Pham, D.M., Stuerzlinger, W.: Is the Pen Mightier than the Controller? A Comparison of Input Devices for Selection in Virtual and Augmented Reality. In: 25th ACM Symposium on Virtual Reality Software and Technology. pp. 1–11. ACM, Parramatta NSW Australia (Nov 2019). https://doi.org/10.1145/3359996.3364264, https://dl.acm.org/doi/10.1145/3359996.3364264

18. Poupyrev, I., Tomokazu, N., Weghorst, S.: Virtual Notepad: handwriting in immersive VR. In: Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No. 98CB36180). pp. 126–132. IEEE (1998). https://doi.org/10.1109/VRAIS.1998.658467, https://doi.org/10.1109/VRAIS.1998.658467

19. Sandnes, F.E.: Panoramagrid: A graph paper tracing framework for sketching 360-degree immersed experiences. In: Proceedings of the International Working Conference on Advanced Visual Interfaces. p. 342–343. AVI ’16, Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2909132.2926058, https://doi.org/10.1145/2909132.2926058

20. Schrepp, M., Hinderks, A., Thomaschewski, J.: Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence 4(6), 103 (2017). https://doi.org/10.9781/ijimai.2017.09.001, http://www.ijimai.org/journal/node/1838

21. Schubert, T., Friedmann, F., Regenbrecht, H.: The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators and Virtual Environments 10(3), 266–281 (Jun 2001). https://doi.org/10.1162/105474601300343603, https://doi.org/10.1162/105474601300343603, publisher: MIT Press

22. Song, H., Guimbretiere, F., Grossman, T., Fitzmaurice, G.: MouseLight: Manual Interactions on Digital Paper Using a Pen and a Spatially-aware Mobile Projector. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 2451–2460. CHI ’10, ACM, New York, NY, USA (2010). https://doi.org/10.1145/1753326.1753697, http://doi.acm.org/10.1145/1753326.1753697, event-place: Atlanta, Georgia, USA

23. Vogel, D., Baudisch, P.: Shift: A technique for operating pen-based interfaces using touch. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 657–666. CHI ’07, Association for Computing Machin-
ery, New York, NY, USA (2007). https://doi.org/10.1145/1240624.1240727, https://doi.org/10.1145/1240624.1240727

24. Vogel, D., Cudmore, M., Casiez, G., Balakrishnan, R., Keliher, L.: Hand Occlusion with Tablet-sized Direct Pen Input. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 557–566. CHI ’09, ACM, New York, NY, USA (2009). https://doi.org/10.1145/1518701.1518787, http://doi.acm.org/10.1145/1518701.1518787, event-place: Boston, MA, USA

25. Wacker, P., Nowak, O., Voelker, S., Borchers, J.: Evaluating menu techniques for handheld ar with a smartphone & mid-air pen. In: 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. MobileHCI ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3379503.3403548, https://doi.org/10.1145/3379503.3403548

26. Wacker, P., Wagner, A., Voelker, S., Borchers, J.: Physical Guides: An Analysis of 3D Sketching Performance on Physical Objects in Augmented Reality. In: Proceedings of the Symposium on Spatial User Interaction. pp. 25–35. SUI ’18, ACM, New York, NY, USA (2018). https://doi.org/10.1145/3267782.3267788, http://doi.acm.org/10.1145/3267782.3267788, event-place: Berlin, Germany

27. Walker, J., Li, B., Vertanen, K., Kuhl, S.: Efficient typing on a visually occluded physical keyboard. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 5457–5461. ACM (2017). https://doi.org/10.1145/3025453.3025783, https://doi.org/10.1145/3025453.3025783

28. Wellner, P.D.: Interacting with paper on the DigitalDesk. Tech. rep., University of Cambridge, Computer Laboratory (1994). https://doi.org/10.1145/159544.159630, https://doi.org/10.1145/159544.159630

29. Zhai, S., Buxton, W., Milgram, P.: The Partial-occlusion Effect: Utilizing Semitransparency in 3D Human-computer Interaction. ACM Trans. Comput.-Hum. Interact. 3(3), 254–284 (1996). https://doi.org/10.1145/234526.234532, http://doi.acm.org/10.1145/234526.234532, place: New York, NY, USA Publisher: ACM

30. Zhang, Y., Pahud, M., Holz, C., Xia, H., Laput, G., McGuffin, M., Tu, X., Mittereder, A., Su, F., Buxton, W., Hinckley, K.: Sensing Posture-Aware Pen+Touch Interaction on Tablets. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. pp. 55:1–55:14. CHI ’19, ACM, New York, NY, USA (2019). https://doi.org/10.1145/3290605.3300285, http://doi.acm.org/10.1145/3290605.3300285, event-place: Glasgow, Scotland Uk