Original: FaceUI: Leveraging Front-Facing Camera Input to Access Mid-Air Spatial Interfaces on Smartphones. [Figure]
Revised: FaceUI: Leveraging Front-Facing Camera Input to Access Mid-Air Spatial Interfaces on Smartphones

Original: A BSTRACT

Original: We present FaceUI, a novel strategy to access mid-air face-centered spatial interfaces with off -the- shelf smartphones.
Revised: We present FaceUI, a novel strategy to access mid-air face-centered spatial interfaces with off-the-shelf smartphones.

Original: FaceUI uses the smartphone’s front-facing camera to track the phone’s mid-air posi- tion relative to the user’s face.
Revised: FaceUI uses the smartphone’s front-facing camera to track the phone’s mid-air posi- tion relative to the user’s face.

Original: This self-contained tracking mechanism opens up new opportunities to enable mid-air interactions on off-the-shelf smartphones.
Revised: This self-contained tracking mechanism opens up new opportunities to enable mid-air interactions on off-the-shelf smartphones.

Original: We demonstrate one possibility that leverages the empty mid-air space in front of the user to accommodate virtual windows which the user can browse by moving the phone in the space in front of their face.
Revised: We demonstrate one possibility that leverages the empty mid-air space in front of the user to accommodate virtual windows which the user can browse by moving the phone in the space in front of their face.

Original: We inform our implementation of FaceUI by first studying essential design factors, such as the comfortable face-to-phone distance range and appropriate viewing angles for browsing mid-air windows and visually accessing their content.
Revised: We inform our implementation of FaceUI by first studying essential design factors, such as the comfortable face-to-phone distance range and appropriate viewing angles for browsing mid-air windows and visually accessing their content.

Original: After that, we compare users’ performance with FaceUI to their performance when using a touch-based interface in an analytic task that requires browsing multiple windows.
Revised: After that, we compare users’ performance with FaceUI to their performance when using a touch-based interface in an analytic task that requires browsing multiple windows.

Original: We find that FaceUI offers better performance than the traditional touch based interface.
Revised: We find that FaceUI offers better performance than the traditional touch-based interface.

Original: We conclude with recommendations for the design and use of face-centered mid-air interfaces on smartphones.
Revised: We conclude with recommendations for the design and use of face-centered mid-air interfaces on smartphones.

Original: 1 I NTRODUCTION
Revised: 1 I NTRODUCTION

Original: When using touch-based input on a smartphone, people typically hold the phone more or less in front of the face.
Revised: When using touch-based input on a smartphone, people typically hold the phone more or less in front of the face.

Original: This posture allows for easy visual access to screen content.
Revised: This posture allows for easy visual access to screen content.

Original: Clearly, while interacting, having the phone in a stationary position seems preferable, and, accordingly, most smartphone interfaces assume a stable in-front- of-the-face posture.
Revised: Clearly, while interacting, having the phone in a stationary position seems preferable, and, accordingly, most smartphone interfaces assume a stable in-front- of-the-face posture.

Original: However, many people are also very skilled in sub-optimal situations when the phone is not still in front of their face, such as to text while walking.
Revised: However, many people are also very skilled in sub-optimal situations when the phone is not still in front of their face, such as to text while walking.

Original: In this paper, we explore how to design smartphone interfaces that require that the user deliberately moves the phone in the space in front of the face as part of the interaction.
Revised: In this paper, we explore how to design smartphone interfaces that require that the user deliberately moves the phone in the space in front of the face as part of the interaction.

Original: We use the high-resolution front-facing camera on a standard smartphone together with machine learning algorithms [10] to track the spatial location of the phone relative to the user’s face.
Revised: We use the high-resolution front-facing camera on a standard smartphone together with machine learning algorithms [10] to track the spatial location of the phone relative to the user’s face.

Original: This allows us to integrate the large empty space in front of the user into new spatial interactions and user interfaces.
Revised: This allows us to integrate the large empty space in front of the user into new spatial interactions and user interfaces.

Original: Prior research have explored ways to extend a smartphone’s input capabilities by shifting the interaction space into the empty in-air space surrounding users’ bodies or their mobile devices.
Revised: Prior research have explored ways to extend a smartphone’s input capabilities by shifting the interaction space into the empty in-air space surrounding users’ bodies or their mobile devices.

Original: For in- stance, Virtual Shelves [27] allows users to point their hand inside a hemisphere in front of their body to access a set of discrete virtual and invisible items, relying heavily on the users’ spatial recall.
Revised: For in- stance, Virtual Shelves [29] allows users to point their hand inside a hemisphere in front of their body to access a set of discrete virtual and invisible items, relying heavily on the users’ spatial recall.

Original: Similarly, the Imaginary Interface [11] is a mid-air interface in front of the user’s body that can be used for pointing and drawing activities.
Revised: Similarly, the Imaginary Interface [11] is a mid-air interface in front of the user’s body that can be used for pointing and drawing activities.

Original: In more recent work, Hasan et al.
Revised: In more recent work, Hasan et al.

Original: [19] present the AirPane system and demonstrate how the mid-air space surrounding a mobile device can be used for browsing information in an e-commerce application.
Revised: [20] present the AirPane system and demonstrate how the mid-air space surrounding a mobile device can be used for browsing information in an e-commerce application.

Original: These and most other prior projects that demonstrate approaches to leverage around-body or around-device interactions rely on external tracking systems, which is not practical in real-life usage situations.
Revised: These and most other prior projects that demonstrate approaches to leverage around-body or around-device interactions rely on external tracking systems, which is not practical in real-life usage situations.

Original: Furthermore, most earlier projects are also limited in that they either do not provide any visual representation of the in-air space (and its interaction objects) at all, or they provide very limited visual infor- mation that typically is decoupled from the actual location within the in-air space.
Revised: Furthermore, most earlier projects are also limited in that they either do not provide any visual representation of the in-air space (and its interaction objects) at all, or they provide very limited visual infor- mation that typically is decoupled from the actual location within the in-air space.

Original: In this paper, we present FaceUI, an approach that avoid these shortcomings.
Revised: We present FaceUI, an approach that avoids these shortcomings.

Original: FaceUI is a novel strategy that leverages mid-air space in front of the user.
Revised: FaceUI is a novel strategy that leverages mid-air space in front of the user.

Original: FaceUI uses a smartphone’s built-in front-facing camera to detect and track the phone’s position relative to the user’s face.
Revised: FaceUI uses a smartphone’s built-in front-facing camera to detect and track the phone’s position relative to the user’s face.

Original: This self-contained tracking approach allows visual access to the in-air space since the screen content is updated depending on the phone’s in-air location and the virtual content at that location.
Revised: This self-contained tracking approach allows visual access to the in-air space since the screen content is updated according to the phone’s in-air location and the virtual content at that location.

Original: The concept is visualized in Figure 1 where a user  navigates a FaceUI-based calendar.
Revised: Figure 1 shows a user who navigates a FaceUI-based calendar application.

Original: To our best knowledge [Table] phones have never been explored before.
Revised: To our best knowledge, ways to leverage face-centered in-air spaces to access virtual user interfaces (UIs) with off-the-shelf smartphones have never been explored before.

Original: With two user studies, we first investigate how the in-air space can be structured to accommodate virtual UIs used for information exploration on smartphones.
Revised: With two user studies, we first investigate how the in-air space can be structured to accommo- date virtual UIs used for information exploration on smartphones.

Original: We identify the comfortable phone-to-face distance range for accessing virtual UIs in the in-air space and suitable viewing angles for browsing and inspecting content that reside in the in-air space.
Revised: We identify the comfortable phone-to-face distance range for accessing virtual UIs in the in-air space and suitable viewing angles for browsing and inspecting content that reside in the in-air space.

Original: We use this knowledge to design FaceUI-based calendar application.
Revised: We use this knowledge to design FaceUI-based calendar application.

Original: In a third user study, we evaluate users’ performance in a calendar browsing task comparing our FaceUI-based calendar with a touch- based calendar interface.
Revised: In a third user study, we evaluate users’ performance in a calendar browsing task comparing our FaceUI-based calendar with a touch- based calendar interface.

Original: Our results show that the FaceUI-approach can offer considerable advantages compared to traditional touch- based interfaces.
Revised: Our results show that the FaceUI-approach can offer considerable advantages compared to traditional touch- based interfaces.

Original: We end our exploration with showcasing further
Revised: We end our exploration with showcasing further

Original: FaceUI-based applications.
Revised: FaceUI-based applications.

Original: Accordingly, our contributions include: 1) FaceUI, a novel face- centered spatial in-air interface-approach for off-the-shelf smartphones; 2) an exploration of suitable design parameters for FaceUI- based applications; 3) a performance comparison between a FaceUI application and standard touch interface in an analytic task; and 4) showcasing further promising FaceUI-enabled interactive applications that demonstrate the potential of face-centered smartphone interfaces.
Revised: Accordingly, our contributions include: 1) FaceUI, a novel face- centered spatial in-air interface-approach for off-the-shelf smartphones; 2) an exploration of suitable design parameters for FaceUI- based applications; 3) a performance comparison between a FaceUI application and standard touch interface in an analytic task; and 4) showcasing further promising FaceUI-enabled interactive applications that demonstrate the potential of face-centered smartphone interfaces.

Original: 2 B ACKGROUND AND R ELATED W ORK
Revised: 2 B ACKGROUND AND R ELATED W ORK

Original: We review prior work that has explored ways to design spatial inter- faces, interaction spaces, and interaction techniques.
Revised: We review prior work that has explored ways to design spatial inter- faces, interaction spaces, and interaction techniques.

Original: These earlier projects inspired the design of our face-centered spatial user inter- face, FaceUI.
Revised: These earlier projects inspired the design of our face-centered spatial user inter- face, FaceUI.

Original: The previous research closely aligned to components of FaceUI falls mainly under around-device interaction, on- and around-body interaction, and Face-Centered Input.
Revised: The previous research closely aligned to components of FaceUI falls mainly under around-device interaction, on- and around-body interaction, and Face-Centered Input.

Original: 2.1 Around-Device Interaction
Revised: 2.1 Around-Device Interaction

Original: There has been substantial prior research work exploring the use of mid-air space around mobile devices.
Revised: There has been substantial prior research work exploring the use of mid-air space around mobile devices.

Original: Researchers have demonstrated that the mid-air space can be used for novel interactions, such as for virtual content browsing and selection [14, 17, 19, 23, 37], map navigation [18,21], mode switching [21] and typing [30].
Revised: Researchers have demonstrated that the mid-air space can be used for novel interactions, such as for virtual content browsing and selection [14,17,18,20,25,39], map navigation [19,23], mode switching [23] and typing [32].

Original: For instance, AD-Binning [17] leveraged the empty 2D space around a smartphone to off-load and browse content into the space.
Revised: For instance, AD-Binning [17] leveraged the empty 2D space around a smartphone to off-load and browse content into the space.

Original: They further showed that the mid-air space could facilitate faster access to items than the standard touch input.
Revised: They further showed that the mid-air space could facilitate faster access to items than the standard touch input.

Original: In a similar work, Hasan et al.
Revised: In a similar work, Hasan et al.

Original: [19] showed that 3D in-air space around a device could be used for browsing m-commerce applications.
Revised: [20] showed that 3D in-air space around a device could be used for browsing m-commerce applications.

Original: Researchers also investigated ways to track users activities around the device with commercial tracking solutions (e.g., Vicon tracking [18,19]) or using different cameras or sensor-based solutions (e.g., depth camera [7,24] , distance sensor [6,23]).
Revised: Researchers also investigated ways to track users activities around the device with commercial tracking solutions (e.g., Vicon tracking [18–20]) or using different cameras or sensor-based solutions (e.g., depth camera [7,26] , distance sensor [6,25]).

Original: Though these solutions offer precise motion capture data, they require either environments, users or devices to be instrumented with sensors.
Revised: Though these solutions offer precise motion capture data, they require either environments, users or devices to be instrumented with sensors.

Original: This makes mobile devices less portable to be used in public spaces.
Revised: This makes mobile devices less portable to be used in public spaces.

Original: 2.2 On- and Around-Body Interaction
Revised: 2.2 On- and Around-Body Interaction

Original: Prior work investigated ways to use the on- and around-body space for designing novel interaction with devices [3,5,9,27].
Revised: Prior work investigated ways to use the on- and around-body space for designing novel interaction with devices [3,5,9,29].

Original: For instance, researchers [11–13] explored the use on body locations such as palm to access on-screen contents.
Revised: For instance, researchers [11–13] explored the use on body locations such as palm to access on-screen contents.

Original: Imaginary Phone [12] used user’s palm as the input surface for iPhone.
Revised: Imaginary Phone [12] used user’s palm as the input surface for iPhone.

Original: In a similar work, Gustafson et al.
Revised: In a similar work, Gustafson et al.

Original: [13] investigated palm-based imaginary interface for supporting visually impaired users.
Revised: [13] investigated palm-based imaginary interface for supporting visually impaired users.

Original: Imaginary Interfaces [11] allowed users to perform spatial interaction on empty palm and without visual feedback.
Revised: Imaginary Interfaces [11] allowed users to perform spatial interaction on empty palm and without visual feedback.

Original: In addition, palm has been used for trigger pre-defined functions [26], to perform 3D rotation [24], or use it as an input space for augmenting keyboards [34].
Revised: In addition, palm has been used for trigger pre-defined functions [28], to perform 3D rotation [26], or use it as an input space for augmenting keyboards [36].

Original: Similarly, researchers explored the skin as an interactive touch surface [16,39,40].
Revised: Similarly, researchers explored the skin as an interactive touch surface [16,41,42].

Original: They commonly used external depth cameras to detect and track hand and finger activities such as tapping and sliding on body parts.
Revised: They commonly used external depth cameras to detect and track hand and finger activities such as tapping and sliding on body parts.

Original: Researchers also investigated using the mid-air space around the body as a novel interaction space.
Revised: Researchers also investigated using the mid-air space around the body as a novel interaction space.

Original: For instance, Virtual Shelves [27] demonstrated that the mid-air space in front of users could be used to trigger shortcuts.
Revised: For instance, Virtual Shelves [29] demonstrated that the mid-air space in front of users could be used to trigger shortcuts.

Original: With a study, they showed that users could recall shortcuts by moving their phone into a 7 × 4 grid on a circular hemisphere in front of them.
Revised: With a study, they showed that users could recall shortcuts by moving their phone into a 7 × 4 grid on a circular hemisphere in front of them.

Original: Yee et al.
Revised: Yee et al.

Original: [38] designed a solution allowing users to move the mobile phone to different locations around the body and change the on-screen content based on the device’s location relative to the body.
Revised: [40] designed a solution allowing users to move the mobile phone to different locations around the body and change the on-screen content based on the device’s location relative to the body.

Original: Ens et al.
Revised: Ens et al.

Original: [9] designed Personal Cockpit leveraging the around-body space to display virtual windows in an head-worn displays.
Revised: [9] designed Personal Cockpit leveraging the around-body space to display virtual windows in an head-worn displays.

Original: In a similar work, Babic et al.
Revised: In a similar work, Babic et al.

Original: [5] explored Gesture Drawer, an one-handed interaction technique allowing users to define and interact with self-define imaginary interfaces while moving their hand to interact with the interfaces.
Revised: [5] explored Gesture Drawer, an one-handed interaction technique allowing users to define and interact with self-define imaginary interfaces while moving their hand to interact with the interfaces.

Original: Researchers have also investigated mid-air spatial interface specific to applications in mixed-reality [9,29,35], for games [33], workspace navigation [22].
Revised: Researchers have also investigated mid-air spatial interface specific to applications in mixed-reality [9,31,37], for games [35], workspace navigation [24].

Original: For instance, Lubos et al.
Revised: For instance, Lubos et al.

Original: [29] introduced kinespheres, an mixed-reality based body-centric spatial interface within arm’s reach.
Revised: [31] introduced kinespheres, an mixed-reality based body-centric spatial interface within arm’s reach.

Original: They received positive feedback from users on using their method compared to traditional head-centered interaction for mixed-reality.
Revised: They received positive feedback from users on using their method compared to traditional head-centered interaction for mixed-reality.

Original: Yan et al.
Revised: Yan et al.

Original: [35] explored an eyes-free target acquisition technique for mixed-reality by placing the targets in around-body space.
Revised: [37] explored an eyes-free target acquisition technique for mixed-reality by placing the targets in around-body space.

Original: Way Out [33] is a game scenario where players can navigate through an omni-directional panorama scene by moving the device around the body using the built-in motion sensors in smartphones.
Revised: Way Out [35] is a game scenario where players can navigate through an omni-directional panorama scene by moving the device around the body using the built-in motion sensors in smartphones.

Original: In a recent work, Kim et al.
Revised: In a recent work, Kim et al.

Original: [22] demonstrated image and map zoom-in and zoom-out using the vision-based interface OddEyeCam, that detects and tracks the location of the mobile phone with respect to the user’s body using external sensors such as wide-view RGB cameras and narrow-view depth cameras.
Revised: [24] demonstrated image and map zoom-in and zoom-out using the vision-based interface OddEyeCam, that detects and tracks the location of the mobile phone with respect to the user’s body using external sensors such as wide-view RGB cameras and narrow-view depth cameras.

Original: 2.3 Face-Centered Input
Revised: 2.3 Face-Centered Input

Original: Prior research investigated using head and face movements as an in- put to design new face-centered interactions on devices.
Revised: Prior research investigated using head and face movements as an in- put to design new face-centered interactions on devices.

Original: For instance,
Revised: For instance,

Original: Zhao et al.
Revised: Zhao et al.

Original: [41] used a combination of facial movements, device mo- tion and touch for designing face-centered interaction techniques on smartphones.
Revised: [43] used a combination of facial movements, device mo- tion and touch for designing face-centered interaction techniques on smartphones.

Original: Kumar et al.
Revised: Kumar et al.

Original: [25] leveraged eye gaze to scroll mobile phone contents.
Revised: [27] leveraged eye gaze to scroll mobile phone contents.

Original: Yang et al.
Revised: Yang et al.

Original: [36] used a face interpretation engine for enabling face-aware applications for smartphones using the phone’s front-facing camera and built-in motion sensors.
Revised: [38] used a face interpretation engine for enabling face-aware applications for smartphones using the phone’s front-facing camera and built-in motion sensors.

Original: Similarly, Babie et al.
Revised: Similarly, Babie et al.

Original: [4] designed Simo that used head movement as input for pointing on a distant large display.
Revised: [4] designed Simo that used head movement as input for pointing on a distant large display.

Original: Instead of using external cameras, they used the smartphone’s front-facing camera to detect face orientation.
Revised: Instead of using external cameras, they used the smartphone’s front-facing camera to detect face orientation.

Original: Rustagi et al.
Revised: Rustagi et al.

Original: [31] explored touchless typing using head gestures detected by the smartphone’s front-facing camera and used them to type on an on-screen QWERTY keyboard.
Revised: [33] explored touchless typing using head gestures detected by the smartphone’s front-facing camera and used them to type on an on-screen QWERTY keyboard.

Original: We also observed that the smartphone’s front-facing camera could be used to design new strategies such as for rotation of on-screen content on mobile devices by detecting direction changes of objects from the camera view [1,2,8].
Revised: We also observed that the smartphone’s front-facing camera could be used to design new strategies such as for rotation of on-screen content on mobile devices by detecting direction changes of objects from the camera view [1,2,8].

Original: The manifold opportunities of spatial mid-air interfaces, as demonstrated by earlier projects, inspire us to continue on this promising path.
Revised: The manifold opportunities of spatial mid-air interfaces, as demonstrated by earlier projects, inspire us to continue on this promising path.

Original: However, in opposite to most previous projects which use external sensors or cameras to identify interaction gestures, we are interested in using a self-contained tracking mecha- nism to detect in-air movements.
Revised: However, in opposite to most previous projects which use external sensors or cameras to identify interaction gestures, we are interested in using a self-contained tracking mecha- nism to detect in-air movements.

Original: Similar to a few face-tracking systems [4,31,41], our FaceUI-approach also uses the front-camera of a smartphone to detect changes in the relative positions of the user’s face and the phone.
Revised: Similar to a few face-tracking systems [4,33,43], our FaceUI-approach also uses the front-camera of a smartphone to detect changes in the relative positions of the user’s face and the phone.

Original: However, FaceUI differ from earlier systems in that it does not rely on any other sensors than the smartphone’s front-facing camera.
Revised: However, FaceUI differ from earlier systems in that it does not rely on any other sensors than the smartphone’s front-facing camera.

Original: Furthermore, we aim at interactions where the user keeps the head still while moving the phone.
Revised: Furthermore, we aim at interactions where the user keeps the head still while moving the phone.

Original: Earlier approaches [4,31,41] require the user to do the opposite, to move the head while holding the phone in a fixed position.
Revised: Earlier approaches [4,33,43] require the user to do the opposite, to move the head while holding the phone in a fixed position.

Original: In this way, we intend to create the sensation of a hemispherical interaction space that is anchored in front of the user’s face but moves along with the user (through the self-contained tracking).
Revised: In this way, we intend to create the sensation of a hemispherical interaction space that is anchored in front of the user’s face but moves along with the user (through the self-contained tracking).

Original: In this first exploration of such an interaction hemisphere, we focus on using virtual application windows that are located inside the hemisphere.
Revised: In this first exploration of such an interaction hemisphere, we focus on using virtual application windows that are located inside the hemisphere.

Original: When the user moves the smartphone to a location inside the hemisphere, the content of thevirtual windowthat resides atthat location is displayed on the smartphone’s screen.
Revised: When the user moves the smartphone to a location inside the hemisphere, the con- tent of the window at the smartphone’s screen.

Original: When the user re-positions the smartphone inside the hemisphere the screen displays the content of the virtual window that resides at the new position.
Revised: When the user re-positions the smartphone inside the hemisphere the screen displays the content of the virtual window that resides at the new position.

Original: Next we describe a few central aspects of the face-detection software and the setup we used in our user studies.
Revised: Next we describe a few central aspects of the face-detection software and the setup we used in our user studies.

Original: After that we present our three studies in turn and order.
Revised: After that we present our three studies in turn and order.

Original: 3 F ACE D ETECTION S OFTWARE AND S TUDY S ETUP
Revised: 3 F ACE D ETECTION S OFTWARE AND S TUDY S ETUP

Original: The self-contained tracking software facility we developed for our FaceUI-approach is based on the Face Detection API [10] in
Revised: The self-contained tracking software facility we developed for our FaceUI-approach is based on the Face Detection API [10] in

Original: Google’s ML Kit ( Machine learning for mobile developers ).
Revised: Google’s ML Kit ( Machine learning for mobile developers ).

Original: The
Revised: The

Original: API provides a comfortable and reliable way to track the position and orientation of a smartphone relative to the user’s face when the front-facing camera is used.
Revised: API provides a comfortable and reliable way to track the position and orientation of a smartphone relative to the user’s face when the front-facing camera is used.

Original: Among the available face-tracking related measures, our software relies on yaw data (the smartphone’s movements to the left or to the right relative to the detected face), pitch data (the smartphone’s up and down movements in the vertical direction relative to the detected face), and distance data (the cur- rent distance between the detected face and the camera lens of the front camera on the smartphone).
Revised: Among the available face-tracking related measures, our software relies on yaw data (the smartphone’s movements to the left or to the right relative to the detected face), pitch data (the smartphone’s up and down movements in the vertical direction relative to the detected face), and distance data (the cur- rent distance between the detected face and the camera lens of the front camera on the smartphone).

Original: Our software does not use any roll -related information.
Revised: Our software does not use any roll -related information.

Original: The Face Detection API delivers 0 ◦ for both yaw and pitch when the user holds the phone straight in front of the face.
Revised: The Face Detection API delivers 0 ◦ for both yaw and pitch when the user holds the phone straight in front of the face.

Original: Restrictions related to COVID-19 prevented us from meeting our study participants face-to-face.
Revised: Restrictions related to COVID-19 prevented us from meeting our study participants face-to-face.

Original: Instead, we conducted our studies remotely using teleconferencing software.
Revised: Instead, we conducted our studies remotely using teleconferencing software.

Original: Accordingly, our partic- ipants were required to have a laptop or a desktop computer with a stable Internet connection, a microphone, loudspeakers, and a webcam.
Revised: Accordingly, our partic- ipants were required to have a laptop or a desktop computer with a stable Internet connection, a microphone, loudspeakers, and a webcam.

Original: In the studies our participants used their own smartphone to run the study software.
Revised: In the studies our participants used their own smartphone to run the study software.

Original: The study software was designed for any phone running Android 4.2 to 11.
Revised: The study software was designed for any phone running Android 4.2 to 11.

Original: Our participants received the study software (i.e., the apk file) and all necessary instructions over email and we guided them through the installation process in the beginning of the study session.
Revised: Our participants received the study software (i.e., the apk file) and all necessary instructions over email and we guided them through the installation process in the beginning of the study session.

Original: The data logged during a study session was automatically transferred from the participant’s phone to a Cloud-Firestore data base when the participant had completed the last study tasks.
Revised: The data logged during a study session was automatically transferred from the participant’s phone to a Cloud-Firestore data base when the participant had completed the last study tasks.

Original: We ran all of our three studies remotely, where participants used the study apps on their smartphones in the wild as opposed to the controlled lab environment.
Revised: We ran all of our three studies remotely, where participants used the study apps on their smartphones in the wild as opposed to the controlled lab environment.

Original: All participants sat in front of their web- cam while completing the study tasks.
Revised: All participants sat in front of their web- cam while completing the study tasks.

Original: In each study, a study session lasted approximately 45 minutes, including instructions, practice trials, timed study trials, breaks, and completion of questionnaires.
Revised: In each study, a study session lasted approximately 45 minutes, including instructions, practice trials, timed study trials, breaks, and completion of questionnaires.

Original: As the study apps were designed for the Android platform, we only recruited participants who possessed an Android smartphone.
Revised: As the study apps were designed for the Android platform, we only recruited participants who possessed an Android smartphone.

Original: 4 S TUDY1: E XPLORING DIRECTION AND DISTANCE
Revised: 4 S TUDY

Revised: : E XPLORING D IRECTION AND D ISTANCE

Original: Prior research has reported arm fatigue and ‘heavy arm’-issues
Revised: Prior research has reported arm fatigue and ‘heavy arm’-issues

Original: re- lated to mid-air interactions [15] and that working with a bent arm in mid-air is more comfortable and less strenuous than working with a stretched arm [20].
Revised: re- lated to mid-air interactions [15] and that working with a bent arm in mid-air is more comfortable and less strenuous than working with a stretched arm [21].

Original: Since FaceUI involves mid-air hand movements arm fatigue is a potential problem.
Revised: Since FaceUI involves mid-air hand movements arm fatigue is a potential problem.

Original: Moreover, with FaceUI, the mid-air movements need to be constrained such that the user’s face is inside the front-camera’s field of view.
Revised: Moreover, with FaceUI, the mid-air movements need to be constrained such that the user’s face is inside the front-camera’s field of view.

Original: With FaceUI, we envision the mid-air interaction space as a semi- circular space in front of the user’s face.
Revised: With FaceUI, we envision the mid-air interaction space as a semi- circular space in front of the user’s face.

Original: Through a pilot test (with five participants) we found that the face tracking works best when the phone is between 5 and 80 centimetres away from the user’s face and the user moves the phone within a longitudinal range of 90 ◦ (from -45 ◦ to the left of the user’s face to 45 ◦ to the right of the user’s face) and a latitudinal range of 70 ◦ (from -35 ◦ below the user’s nose to 35 ◦ above the user’s nose).
Revised: Through a pilot test (with five participants) we found that the face tracking works best when the phone is between 5 and 80 centimetres away from the user’s face and the user moves the phone within a longitudinal range of 90 ◦ (from -45 ◦ to the left of the user’s face to 45 ◦ to the right of the user’s face) and a latitudinal range of 70 ◦ (from -35 ◦ below the user’s nose to 35 ◦ above the user’s nose).

Original: Whereas we know that movements inside this space are accurately tracked, we do not know how accurately, fast, and comfortably people can navigate around in this mid-air space.
Revised: Whereas we know that movements inside this space are accurately tracked, we do not know how accurately, fast, and comfortably people can navigate around in this mid-air space.

Original: Accordingly, we want to chart out the suitable dimensions and the granularity of the mid-air interaction space for
Revised: Accordingly, we want to chart out the suitable dimensions and the granularity of the mid-air interaction space for

Original: FaceUI in our first study.
Revised: FaceUI in our first study.

Original: 4.1 Study Design and Study Task
Revised: 4.1 Study Design and Study Task

Original: We oriented the study task and study design of our first study accord-
Revised: We oriented the study task and study design of our first study accord-

Original: ing to previous projects that have explored the dimensions and the granularity of the mid-air space in front of the user, e.g., the Virtual Shelves [27] and AD-Binning [17] projects.
Revised: ing to previous projects that have explored the dimensions and the granularity of the mid-air space in front of the user, e.g., the Virtual Shelves [29] and AD-Binning [17] projects.

Original: We used a simple item selection task where a trial consists of moving the smartphone to a specified position in mid-air to select the virtual item at that position.
Revised: We used a simple item selection task where a trial consists of moving the smartphone to a specified position in mid-air to select the virtual item at that position.

Original: We investigated horizontal movements and vertical movements when the phone is close or far from the user’s face.
Revised: We investigated horizontal movements and vertical movements when the phone is close or far from the user’s face.

Original: Figure 2 visualizes the study task and setup.
Revised: Figure 2 visualizes the study task and setup.

Original: We divided the mid-air space along the horizontal into seven equally wide onedimensional regions – or items –, each 12.85 ◦ ‘wide’ (Figure 2a).
Revised: We divided the mid-air space along the horizontal into seven equally wide onedimensional regions – or items –, each 12.85 ◦ ‘wide’ (Figure 2a).

Original: We used five one-dimensional regions – or items – in the vertical direction, each 12.85 ◦ ‘high’ (Figure 2b).
Revised: We used five one-dimensional regions – or items – in the vertical direction, each 12.85 ◦ ‘high’ (Figure 2b).

Original: From a user’s perspective, the size of these items in the air in front of the face depends on the distance between the phone and the face: the further away from the face, the larger the item becomes.
Revised: From a user’s perspective, the size of these items in the air in front of the face depends on the distance between the phone and the face: the further away from the face, the larger the item becomes.

Original: Accordingly, we decided to also test movements (horizontal and vertical) when performed close to the face and far away from the face.
Revised: Accordingly, we decided to also test movements (horizontal and vertical) when performed close to the face and far away from the face.

Original: 4.1.1 Study design
Revised: 4.1.1 Study design

Original: With this, we arrive at two independent factors for our study: 1) movement Direction : horizontal and vertical, and 2) Distance : close and far.
Revised: With this, we arrive at two independent factors for our study: 1) movement Direction : horizontal and vertical, and 2) Distance : close and far.

Original: Close represents the distance range within which partici- pants commonly and comfortably hold their phone when accessing on-screen content with touch.
Revised: Close represents the distance range within which partici- pants commonly and comfortably hold their phone when accessing on-screen content with touch.

Original: We regarded any distance beyond that range as far.
Revised: We regarded any distance beyond that range as far.

Original: However, where the comfortable range ends is likely to differ between participants (depending on arm length and pref- erence).
Revised: However, where the comfortable range ends is likely to differ between participants (depending on arm length and pref- erence).

Original: Therefore, it is critical to have a user-depended threshold value rather than using a common value for all participants.
Revised: Therefore, it is critical to have a user-depended threshold value rather than using a common value for all participants.

Original: We calibrated the individual value for each participant in the beginning of the study session.
Revised: We calibrated the individual value for each participant in the beginning of the study session.

Original: We asked the participant to provide us the phone-to-face distance where it started to feel awkward and less comfortable when moving the phone in front of the face.
Revised: We asked the participant to provide us the phone-to-face distance where it started to feel awkward and less comfortable when moving the phone in front of the face.

Original: Once the phone had reached these locations, the study app showed the distance between the face and the phone (in centimetres) on the screen.
Revised: Once the phone had reached these locations, the study app showed the distance between the face and the phone (in centimetres) on the screen.

Original: We asked the participant to  move the phone left, right, middle, up, and down and share the distance data.
Revised: We asked participants to keep their head static and move the phone left, right, middle, up, and down and share the distance data.

Original: We calculated the partici- pant’s upper value for the ‘close’ (i.e., comfortable) distance for the horizontal direction by averaging the left, right and middle values.
Revised: We calculated the participant’s upper value for the ‘close’ (i.e., comfortable) distance for the horizontal direction by averaging the left, right and middle values.

Original: For the vertical direction we averaged the up, down, and middle values.
Revised: For the vertical direction we averaged the up, down, and middle values.

Original: Weused a within-subjects study design.
Revised: We used a within-subjects study design.

Original: All participants performed four series of six blocks of trials, one series of blocks with each of the four Distance - Direction combinations: close-horizontal, close-vertical, far-horizontal, and far-vertical.
Revised: All participants performed four series of six blocks of trials, one series of blocks with each of the four Distance - Direction combinations: close-horizontal, close-vertical, far-horizontal, and far-vertical.

Original: Blocks in the hori- zontal direction consisted of six trials, one trial for each of the six target items (1, 2, 3, 5, 6, and 7, cf.
Revised: Blocks in the hori- zontal direction consisted of six trials, one trial for each of the six target items (1, 2, 3, 5, 6, and 7, cf.

Original: Figure 2a) in random order.
Revised: Figure 2a) in random order.

Original: Blocks in the vertical direction consisted of four trials, one trial for each of the four target items (1, 2, 4, and 5, cf.
Revised: Blocks in the vertical direction consisted of four trials, one trial for each of the four target items (1, 2, 4, and 5, cf.

Original: Figure 2b) in random order.
Revised: Figure 2b) in random order.

Original: Accordingly, each participant performed 120 trials: one block series of 36 horizontal trials at close distance + one block series of 24 vertical trials at close distance and one block series ofhorizontal trials at far distance + one block series of 24 vertical trials at far distance.
Revised: Accordingly, each participant performed 120 trials: one block series of 36 horizontal trials at close distance + one block series of 24 vertical trials at close distance and one block series ofhorizontal trials at far distance + one block series of 24 vertical trials at far distance.

Original: Half of the participants started with the two block series at close distance and then completed the two block series at far distance, the other half used the other order.
Revised: Half of the participants started with the two block series at close distance and then completed the two block series at far distance, the other half used the other order.

Original: The order of the two directions-series within a distance was random.
Revised: The order of the two directions-series within a distance was random.

Original: 4.1.2 Task procedure
Revised: 4.1.2 Task procedure

Original: To start a trial, the participant moved the phone to the middle
Revised: To start a trial, the participant moved the phone to the middle

Original: re- gion, straight in front of the face.
Revised: re- gion, straight in front of the face.

Original: In the horizontal direction, this corresponded to Item 4 in Figure 2a.
Revised: In the horizontal direction, this corresponded to Item 4 in Figure 2a.

Original: In the vertical direction Item 3 in Figure 2b was used as the start region.
Revised: In the vertical direction Item 3 in Figure 2b was used as the start region.

Original: Once the participant moved the phone inside the start region the screen turned green and displayed information for the upcoming trial, including the target prompt with the item number to select next, as shown in Figure 2c.
Revised: Once the participant moved the phone inside the start region the screen turned green and displayed information for the upcoming trial, including the target prompt with the item number to select next, as shown in Figure 2c.

Original: A trial started when the participant pressed down the thumb on the screen.
Revised: A trial started when the participant pressed down the thumb on the screen.

Original: If the phone was moved outside the start region before pressing down with the thumb, the screen turned red and showed instructions to move the phone into the start region.
Revised: If the phone was moved outside the start region before pressing down with the thumb, the screen turned red and showed instructions to move the phone into the start region.

Original: A thumb-press in the start region started timing for the trial.
Revised: A thumb-press in the start region started timing for the trial.

Original: A selection was registered and the trial time stopped when the thumb was released after having moved the phone into one of the items (or regions) outside the start region.
Revised: A selection was registered and the trial time stopped when the thumb was released after having moved the phone into one of the items (or regions) outside the start region.

Original: Speech output informed if the participant selected the correct item or not by playing “Correct selection” resp.
Revised: Speech output informed if the participant selected the correct item or not by playing “Correct selection” resp.

Original: “Wrong selection”.
Revised: “Wrong selection”.

Original: Erroneous trials were re-queued at a random position among the unfinished trials within the current block.
Revised: Erroneous trials were re-queued at a random position among the unfinished trials within the current block.

Original: During a running trial we relied on audio to inform participants about the current position of the phone.
Revised: During a running trial we relied on audio to inform participants about the current position of the phone.

Original: The app provided speech output when i) the phone entered a new item, by saying the number of the item, when ii) the participant moved the phone at the wrong distance, by playing “Move the phone further away” in far conditions or “Move the phone closer” in close conditions, and when iii) the face tracking software lost track of the face, by playing “Face out of camera view”.
Revised: The app provided speech output when i) the phone entered a new item, by saying the number of the item, when ii) the participant moved the phone at the wrong distance, by playing “Move the phone further away” in far conditions or “Move the phone closer” in close conditions, and when iii) the face tracking software lost track of the face, by playing “Face out of camera view”.

Original: Working with audio guidance was important: in our first study, we wanted to focus on the motoric aspects and movement properties that determine the dimensions of FaceUI’s interaction space.
Revised: Working with audio guidance was important: in our first study, we wanted to focus on the motoric aspects and movement properties that determine the dimensions of FaceUI’s interaction space.

Original: We wanted to exclude aspects that relate to how well a user can read screen content while moving the phone in mid-air space, such as the size of screen content and the viewing angle and distance.
Revised: We wanted to exclude aspects that relate to how well a user can read screen content while moving the phone in mid-air space, such as the size of screen content and the viewing angle and distance.

Original: We return to such visual issues in our second study.
Revised: We return to such visual issues in our second study.

Original: 4.2 Participants
Revised: 4.2 Participants

Original: We recruited twelve right-handed participants (mean age 27.08 years, s.d. 5.98, 6 male) via on-campus flyers and word-of-mouth.
Revised: We recruited twelve right-handed participants (mean age 27.08 years, s.d. 5.98, 6 male) via on-campus flyers and word-of-mouth.

Original: All participants were daily smartphone users.
Revised: All participants were daily smartphone users.

Original: 4.3 Results
Revised: 4.3 Results

Original: We first report on results regarding participants’ comfortable phone- to-face distance (calibrated in the beginning of a study session) that served as the basis for each participant’s individual threshold value that separated the close distance from the far distance.
Revised: We first report on results regarding participants’ comfortable phone- to-face distance (calibrated in the beginning of a study session) that served as the basis for each participant’s individual threshold value that separated the close distance from the far distance.

Original: After that we report on trial time, error rates, and subjective ratings.
Revised: After that we report on trial time, error rates, and subjective ratings.

Original: 4.3.1 Close/far threshold value
Revised: 4.3.1 Close/far threshold value

Original: Across all participants, the average face-to-phone distance where movements started to feel less comfortable was 39.31 cm (s.d. 7.61) for horizontal phone movements and 39.38 cm (s.d. 7.34) for vertical movements.
Revised: Across all participants, the average face-to-phone distance where movements started to feel less comfortable was 39.31 cm (s.d. 7.61) for horizontal phone movements and 39.38 cm (s.d. 7.34) for vertical movements.

Original: This critical threshold varied a lot between participants.
Revised: This critical threshold varied a lot between participants.

Original: In the horizontal movement direction it was between 30 and 61 cm and in the vertical direction between 28 and 61 cm (only one participant had values greater than 50 cm).
Revised: In the horizontal movement direction it was between 30 and 61 cm and in the vertical direction between 28 and 61 cm (only one participant had values greater than 50 cm).

Original: 4.3.2 Trial time
Revised: 4.3.2 Trial time

Original: The trial time analyses are based on error free trials only.
Revised: The trial time analyses are based on error free trials only.

Original: Figure 3a shows the mean trial time for each target position at both distances in the horizontal movement direction and Figure 3b shows the cor- responding results for the vertical direction.
Revised: Figure 3a shows the mean trial time for each target position at both distances in the horizontal movement direction and Figure 3b shows the corre- sponding results for the vertical direction.

Original: Mean trial times for close and far (across the two directions) were 2.93 sec and 2.99 sec, respectively.
Revised: Mean trial times for close and far (across the two directions) were 2.93s and 2.99s, respectively.

Original: The overall mean trial time (across the two distances) for the horizontal and vertical directions were 3.01 sec and 2.90 sec, respectively.
Revised: The overall mean trial time (across the two distances) for the horizontal and vertical directions were 3.01s and 2.90s, respectively.

Original: A 2 × 2 RM-ANOVA showed that there was no significant difference between the two distances ( F 1 , 11 = 0 .
Revised: A 2 × 2 RM-ANOVA showed that there was no significant difference between the two distances ( F 1 , 11 = 0 .

Original: 09, p = 0 .
Revised: 09, p = 0 .

Original: 76) or between the two directions ( F 1 , 11 = 0 .
Revised: 76) or between the two directions ( F 1 , 11 = 0 .

Original: 29, p = 0 . 59).
Revised: 29, p = 0 . 59).

Original: A one-way RM-ANOVA (independent factor block) indicated that participants became faster during the course of the study with significantly longer trial times in the first block of trials than in last two blocks ( F 5 , 55 = 5 . 85, p < 0 . 001).
Revised: A one-way RM-ANOVA (independent factor block) indicated that participants became faster during the course of the study with significantly longer trial times in the first block of trials than in last two blocks ( F 5 , 55 = 5 . 85, p < 0 . 001).

Original: The mean trial time decreased from 3.43 sec in Block 1 to 2.81 and 2.72 sec in Block 5 and 6, respectively.
Revised: The mean trial time decreased from 3.43s in Block 1 to 2.81s and 2.72s in Block 5 and 6, respectively.

Original: InFigure3 we also see a clear and expected pattern regarding the different target positions: given further phone-movement distances, selecting item at positions close to the start position (Position 4 for horizontal movements and Position 3 for vertical movements, cf.
Revised: In Figure 3 we also see a clear and expected pattern regarding the different target positions: given further phone-movement distances, selecting item at positions close to the start position (Position 4 for horizontal movements and Position 3 for vertical movements, cf.

Original: Figure 2) was quicker than selection items further away.
Revised: Figure 2) was quicker than selection items further away.

Original: This pattern we see for movements in both directions and at both the close and far distances.
Revised: This pattern we see for movements in both directions and at both the close and far distances.

Original: 4.3.3 Error rate
Revised: 4.3.3 Error rate

Original: Figure 3c shows the mean error rates for the four distance × direction combinations.
Revised: Figure 3c shows the mean error rates for the four distance × direction combinations.

Original: A Friedman test identified a significant difference among the combinations ( χ 2 ( 3 , N = 12 ) = 8 .
Revised: A Friedman test identified a significant difference among the combinations ( χ 2 ( 3 , N = 12 ) = 8 .

Original: 95 , p < 0 . 05)
Revised: 95 , p < 0 . 05)

Original: and posthoc Wilcoxon tests (Bonferroni adjusted α -level from 0.05 to 0.008) revealed that the close-vertical combination was significantly less error prone than close-horizontal combination and that there were no other pairwise differences.
Revised: and posthoc Wilcoxon tests (Bonferroni adjusted α -level from 0.05 to 0.008) revealed that the close-vertical combination was significantly less error prone than close-horizontal combination and that there were no other pairwise differences.

Original: 4.3.4 Subjective feedback
Revised: 4.3.4 Subjective feedback

Original: We asked participants to rate the two directions and the two distances according to their overall preference on a 5-point scale with 1 = bad , 3 = neutral , and 5 = good .
Revised: We asked participants to rate the two directions and the two distances according to their overall preference on a 5-point scale with 1 = bad , 3 = neutral , and 5 = good .

Original: We found an unsurprising and strong preference for the close distance with mean rating 4.52 compared to the far distance with mean rating 1.91.
Revised: We found an unsurprising and strong preference for the close distance with mean rating 4.52 compared to the far distance with mean rating 1.91.

Original: Participants were not that decided in their opinions regarding the two movement directions.
Revised: Participants were not that decided in their opinions regarding the two movement directions.

Original: They rated the horizontal movement direction only slighter better than the vertical direction, mean rating 4.23 vs. mean rating 3.1.
Revised: They rated the horizontal movement direction only slighter better than the vertical direction, mean rating 4.23 vs. mean rating 3.1.

Original: 4.4 Summary
Revised: 4.4 Summary

Original: Results from the subjective feedback indicate that participants had a slight preference for horizontal movements over vertical movements.
Revised: Results from the subjective feedback indicate that participants had a slight preference for horizontal movements over vertical movements.

Original: However, our analyses also revealed that there is no significant dif- ference between the movement directions in regard of trial time.
Revised: However, our analyses also revealed that there is no significant dif- ference between the movement directions in regard of trial time.

Original: But we see a clear, and unsurprising advantage for the close distance over the far distance.
Revised: But we see a clear, and unsurprising advantage for the close distance over the far distance.

Original: Accordingly, for our future FaceUI explorations, we learn that people are sensitive regarding the phone-to-face distance and that FaceUI-based applications should avoid requiring user to use large phone-to-face distances.
Revised: Accordingly, for our future FaceUI explorations, we learn that people are sensitive regarding the phone-to-face distance and that FaceUI-based applications should avoid requiring user to use large phone-to-face distances.

Original: Consequently, in our we continue utilizing regions along both the horizontal and vertical directions.
Revised: Consequently, in our we continue utilizing regions along both the horizontal and vertical directions.

Original: However, we observed increased trial time with items located in cer- tain vertical regions, e.g., Item 1, than others.
Revised: However, we observed increased trial time with items located in certain vertical regions, e.g., Item 1, than others.

Original: This warrants further investigation into factors such as visual angles that could influence users’ performance when reading screen content when holding the phone in such regions.
Revised: This warrants further investigation into factors such as visual angles that could influence users’ performance when reading screen content when holding the phone in such regions.

Original: 5 S TUDY 2: E XPLORING TARGET REGION AND TARGET ANGLE
Revised: 5 S TUDY 2: E XPLORING T ARGET R EGION AND A NGLE

Original: Application interfaces that are placed in FaceUI move in both hor- izontal and vertical regions with the user’s head movement along with the same regions.
Revised: Application interfaces that are placed in FaceUI move in both hor- izontal and vertical regions with the user’s head movement along with the same regions.

Original: Therefore, to read content that is located to the right or left on the FaceUI, a user needs to keep their head static and move their eyes to read the content.
Revised: Therefore, to read content that is located to the right or left on the FaceUI, a user needs to keep their head static and move their eyes to read the content.

Original: Prior research [32] showed that such eye movement could cause eyes fatigue, pain and tiredness.
Revised: Prior research [34] showed that such eye movement could cause eyes fatigue, pain and tiredness.

Original: Therefore, in this study, we explored suitable viewing angles where users can comfortably access on-screen items on smartphones.
Revised: Therefore, in this study, we explored suitable viewing angles where users can comfortably access on-screen items on smartphones.

Original: 5.1 Participants
Revised: 5.1 Participants

Original: We recruited fourteen right-handed participants (mean age 26.78 years, s.d. 6.07, 7 male) via on-campus flyers and word-of-mouth.
Revised: We recruited fourteen right-handed participants (mean age 26.78 years, s.d. 6.07, 7 male) via on-campus flyers and word-of-mouth.

Original: All participants were daily smartphone users.
Revised: All participants were daily smartphone users.

Original: None of the partici- pants had participated in Study 1.
Revised: None of the partici- pants had participated in Study 1.

Original: 5.2 Factors
Revised: 5.2 Factors

Original: We considered the following factors in this study.
Revised: We considered two factors, target region and target angle, described as follows.

Original: 5.2.1 Target Region
Revised: 5.2.1 Target region

Original: In this study, we considered placing items in two regions – vertical (up and down) and horizontal (left and right).
Revised: In this study, we considered placing items in two regions – vertical (up and down) and horizontal (left and right).

Original: Similar to the first study, we kept the middle location reserved as the starting point of a trial.
Revised: Similar to the first study, we kept the middle location reserved as the starting point of a trial.

Original: 5.2.2 Target Angle
Revised: 5.2.2 Target angle

Original: We decided to place a set of targets at angles both in horizontal and vertical regions.
Revised: We decided to place a set of targets at angles both in horizontal and vertical regions.

Original: With a pilot study, we choose to place items at ± 20 ◦ , ± 30 ◦ , ± 40 ◦ and ± 50 ◦ angles where positive and negative angles indicate items to the right and left regions, respectively.
Revised: With a pilot study, we choose to place items at ± 20 ◦ , ± 30 ◦ , ± 40 ◦ and ± 50 ◦ angles where positive and negative angles indicate items to the right and left regions, respectively.

Original: Results from our pilot study showed that participants were not able to see items that are located above +30 ◦ in the up.
Revised: Results from our pilot study showed that participants were not able to see items that are located above +30 ◦ in the up.

Original: Additionally, any items placed below -40 ◦ angles for the downward region were not accessible as the phone gets very close to the body.
Revised: Additionally, any items placed below -40 ◦ angles for the downward region were not accessible as the phone gets very close to the body.

Original: Therefore, we used +20 ◦ and +30 ◦ for the up region, and -20 ◦ , -30 ◦ and -40 ◦ for the down region.
Revised: Therefore, we used +20 ◦ and +30 ◦ for the up region, and -20 ◦ , -30 ◦ and -40 ◦ for the down region.

Original: 5.3 Procedure and Tasks
Revised: 5.3 Procedure and Tasks

Original: At the beginning of a trial, the participant was required to move the phone to the middle position (straight in front of the face).
Revised: At the beginning of a trial, the participant was required to move the phone to the middle position (straight in front of the face).

Original: As long as the phone was still and outside the middle position, the screen remained red and contained instructions to move the phone to the middle position.
Revised: As long as the phone was still and outside the middle position, the screen remained red and contained instructions to move the phone to the middle position.

Original: Once the phone was inside the middle position the screen turned green and displayed the target prompt for the next target (along with block and trial counts), as seen in Figure 4a.
Revised: Once the phone was inside the middle position the screen turned green and displayed the target prompt for the next target (along with block and trial counts), as seen in Figure 4a.

Original: Participants were asked to press on the screen with their thumb and move their phone at the target angle while keeping the thumb on the screen.
Revised: Participants were asked to press on the screen with their thumb and move their phone at the target angle while keeping the thumb on the screen.

Original: Tapping on the screen also started a timer.
Revised: Tapping on the screen also started a timer.

Original: The study application then removed the on-screen instructions, replaced with an empty black window, and kept it until participants moved the phone to the target angle.
Revised: The study application then removed the on-screen instructions, replaced with an empty black window, and kept it until participants moved the phone to the target angle.

Original: We used circle counting tasks in this study, where participants were required to count the number of circles presented in two windows located at a target angle.
Revised: We used circle counting tasks in this study, where participants were required to count the number of circles presented in two windows located at a target angle.

Original: For instance, if the target angle was +30 ◦ in the right region, we placed two more windows above and below the vertical plane defined by the user’s eye.
Revised: For instance, if the target angle was +30 ◦ in the right region, we placed two more windows above and below the vertical plane defined by the user’s eye.

Original: Participants could only see the windows once they reach to the instructed region and angle.
Revised: Participants could only see the windows once they reach to the instructed region and angle.

Revised: We asked participants to keep the head still and move only the phone during the study.

Original: They could now move their phone up and down (for horizontal region) or left and right (for vertical region) to access the windows while keeping the phone in the target angle (Figure 4b-c).
Revised: They could now move their phone up and down (for horizontal region) or left and right (for vertical region) to access the windows while keeping the phone in the target angle (Figure 4b-c).

Original: The windows contained a random number of non-overlapping black circles between 12 and 16.
Revised: The windows contained a random number of non-overlapping black circles between 12 and 16.

Original: Participants were required to count the total number of circles seen on both screens.
Revised: Participants were required to count the total number of circles seen in both windows.

Original: Once they believed counting allthe circles on both windows, they were asked to lift off their thumb from the touchscreen.
Revised: Once they believed having counted all circles in both windows, they were asked to lift off their thumb from the touchscreen.

Original: This action further popup a window containing multiple options for the summation results (Figure 4d).
Revised: This action further poped up a window containing multiple options for the summation results (Figure 4d).

Original: Once they selected the correct answer, the application stopped the trial time, provided voice feedback on whether they were corrector not and displayed the instruction for the next trial on the screen.
Revised: Once they selected the correct answer, the application stopped the trial time, provided voice feedback on whether the answer was correct and displayed the instruction for the next trial on the screen.

Original: If incorrect, the app stopped the trial time, provided audio feedback, and re-queued the trial at a random position among unfinished trials within a block of trials.
Revised: If incorrect, the app stopped the trial time, provided audio feedback, and re- queued the trial at a random position among unfinished trials within a block of trials.

Original: Participants were then required to move the phone back to the middle and continue trials until all the trials were finished.
Revised: Participants were then required to move the phone back to the middle and continue trials until all the trials were finished.

Original: Note that for either case, the app sent trial-related information (e.g., task completion time, correctness) to a database server.
Revised: Note that for either case, the app sent trial-related information (e.g., task completion time, correctness) to a database server.

Original: Each participant completed six blocks of trials with each of the Target Region (left, right, up and down) where one block contained one trial for each of the Target Angles .
Revised: Each participant completed six blocks of trials with each of the Target Region (left, right, up and down) where one block contained one trial for each of the Target Angles .

Original: Therefore, each participant performed 78 error-free trials (24 trials for left, 24 for right, 12 for up and 18 for down region).
Revised: Therefore, each participant performed 78 error-free trials (24 trials for left, 24 for right, 12 for up and 18 for down region).

Original: The presentation order of the target region were selected randomly between participants and the angles were presented in a random order.
Revised: The presentation order of the target region were selected randomly between participants and the angles were presented in a random order.

Original: Participants were provided with 2 blocks of practice trials.
Revised: Participants were provided with 2 blocks of practice trials.

Original: After completing all the trials, we collected the participants’ feedback on their preferences on Target Region and Target Angles .
Revised: After completing all the trials, we collected the participants’ feedback on their preferences on Target Region and Target Angles .

Original: This study required participants around 45 minutes to complete all the tasks.
Revised: This study required participants around 45 minutes to complete all the tasks.

Original: 5.4 Results
Revised: 5.4 Results

Original: Instead of analyzing the trial time, we were interested in the time that participants spent on counting the circles rather than moving the phone to the target position and taking time to answer questions.
Revised: Instead of analyzing the trial time, we were interested in the time that participants spent on counting the circles rather than moving the phone to the target position and taking time to answer questions.

Original: We called this time as Reading time .
Revised: We called this time as Reading time .

Original: 5.4.1 Reading time
Revised: 5.4.1 Reading time

Original: We used repeated measures ANOVA and post-hoc pairwise
Revised: We used repeated measures ANOVA and post-hoc pairwise

Original: compar- isons to analyze reading time.
Revised: compar- isons to analyze reading time.

Original: Results showed that Target Region had significant effects on reading time ( F 3 , 39 = 3 . 24, p < 0 . 05).
Revised: Results showed that Target Region had significant effects on reading time ( F 3 , 39 = 3 . 24, p < 0 . 05).

Original: Fig- ure 5a shows the mean reading time for all four regions: left (mean 10.40s) and right (mean 11.16s), down (mean 9.76s) and up (mean 13.84s).
Revised: Fig- ure 5a shows the mean reading time for all four regions: left (mean 10.40s) and right (mean 11.16s), down (mean 9.76s) and up (mean 13.84s).

Original: Post-hoc pairwise comparisons showed that Down was significantly faster than Up while accessing the items.
Revised: Post-hoc pairwise comparisons showed that Down was significantly faster than Up while accessing the items.

Original: No other pairwise difference was found.
Revised: No other pairwise difference was found.

Original: We also analyzed the reading time for each target angle on each target region.
Revised: We also analyzed the reading time for each target angle on each target region.

Original: Figure 5b shows the mean reading time for all target angle on each target region.
Revised: Figure 5b shows the mean reading time for all target angle on each target region.

Original: Target angles in Right showed significant effects on reading time ( F 3 , 39 = 6 . 71, p < 0 . 001).
Revised: Target angles in Right showed significant effects on reading time ( F 3 , 39 = 6 . 71, p < 0 . 001).

Original: Post-hoc pairwise comparisons between target angles showed that targets at 50 ◦ angle was significantly slower than target angles at 20 ◦ .
Revised: Post-hoc pairwise comparisons between target angles showed that targets at 50 ◦ angle was significantly slower than target angles at 20 ◦ .

Original: There were no other pairwise statistically significant differences.
Revised: There were no other pairwise statistically significant differences.

Original: We also observed that the target angles in Left target region had significant effects on reading time ( F 3 , 39 = 7 . 03, p < 0 . 001).
Revised: We also observed that the target angles in Left target region had significant effects on reading time ( F 3 , 39 = 7 . 03, p < 0 . 001).

Original: Similar to the right region, targets at 50 ◦ angles were significantly slower than targets at 20 ◦ .
Revised: Similar to the right region, targets at 50 ◦ angles were significantly slower than targets at 20 ◦ .

Original: There were no other statistically significant differences between the angles.
Revised: There were no other statistically significant differences between the angles.

Original: Like left and right regions, target angles in Up region showed significant effects on reading time ( F 3 , 39 = 23 .
Revised: Like left and right regions, target angles in Up region showed significant effects on reading time ( F 3 , 39 = 23 .

Original: 39, p < 0 .
Revised: 39, p < 0 .

Original: Targets at 30 ◦ angles were significantly slower than targets at angle 20 ◦ .
Revised: Targets at 30 ◦ angles were significantly slower than targets at angle 20 ◦ .

Original: Target angles in Down region also showed significant effects on trial time ( F 2 , 26 = 21 .
Revised: Target angles in Down region also showed significant effects on trial time ( F 2 , 26 = 21 .

Original: 12, p < 0 .
Revised: 12, p < 0 .

Original: Targets at 40 ◦ angle were significant slower than targets at 20 ◦ and 30 ◦ .
Revised: Targets at 40 ◦ angle were significant slower than targets at 20 ◦ and 30 ◦ .

Original: No other statistically significant differences were observed.
Revised: We did not find any other statistically significant differences.

Original: 5.4.2 Subjective feedback
Revised: 5.4.2 Subjective feedback

Original: Participants rated each target region using a 5-point Likert scale.
Revised: Participants rated each target region using a 5-point Likert scale.

Original: They preferred the right target region (mean rating 3.85) most, followed by down (mean rating 3.78) and left (mean rating 3.07).
Revised: They preferred the right target region (mean rating 3.85) most, followed by down (mean rating 3.78) and left (mean rating 3.07).

Original: Up target region was rated as the least preferred region to access items (mean rating 2.28).
Revised: Up target region was rated as the least preferred region to access items (mean rating 2.28).

Original: 5.5 Summary
Revised: 5.5 Summary

Original: The results shows thatvisually accessing  screen content while the phone is positioned in the upper region (up)is slower thanwhen the phone is positioned in any other region (the lower region, the left designing application
Revised: The results shows that accessing the screen contents visually is slower when the phone is positioned in the upper region (up) than in any other region (the lower region, the left and the right regions) of the in-air space.

Revised: For the target angles for each region, we see that participants’ performance degrades significantly at the highest angle in each region.

Revised: Accordingly, we suggest to avoid the extreme angles, 50 ◦ for both right and left, 30 ◦ for up, and 40 ◦ for down when designing a application that uses FaceUI.

Original: 6 S TUDY 3:P ERFORMANCE ANALYSIS OF F ACE UI WITH AN ANALYTIC TASK
Revised: 6 S TUDY 3: F ACE UI P ERFORMANCE IN AN A NALYTIC T ASK

Original: We conducted previous two studies exploring different design factors such as target angle and target region that could potentially influence users performance using FaceUI.
Revised: In our previous two studies we explored different design factors, such as target angle and target region, that could potentially influence users’ performance with FaceUI.

Original: In this study, we evaluate a practi- cal usage scenario using FaceUI where users are required to browse multiple windows for retrieving information.
Revised: In this study, we evaluate a practical usage scenario with FaceUI where the user is required to browse multiple windows to retrieve information.

Original: Consequently, we de- signed a calendar app and compared the performance of FaceUI with traditional touch interfaces.
Revised: Consequently, we designed a calendar app and compared users’ performance when using its FaceUI version and when using a its touch-based version.

Original: 6.1 Participants
Revised: 6.1 Participants

Original: We recruited twelve right-handed participants (mean age 25.5, s.d. 5.23, 6 male) via on-campus flyers and word-of-mouth.
Revised: We recruited twelve right-handed participants (mean age 25.5, s.d. 5.23, 6 male) via on-campus flyers and word-of-mouth.

Original: All participants were daily smartphone users.
Revised: All participants were daily smartphone users.

Original: None of the participants had participated in Study 1 or in Study 2.
Revised: None of the participants had participated in Study 1 or in Study 2.

Original: 6.2 Task, Procedure and Design
Revised: 6.2 Task, Procedure, and Design

Original: In this study we used an analytic task where users were required reviewing information on a calendar before reaching a decision.
Revised: The analytic task requires participants to review information in a calendar before reaching a decision.

Original: A trial starts with displaying the question along with a start button (e.g., ”Find the number of online meetings scheduled on July 7” ), as seen in Figure 6a.
Revised: A trial starts with displaying a query prompt along with a start button, such as “Find the number of online meetings scheduled on July 7” , as seen in Figure 6a.

Original: After reading the question, the user taps on the start button, whichalso starts the trial time and opens a new  window containing calendar dates for a month(e.g., July 2021 in our case) at the bottom of the screen (Figure 6b).
Revised: After reading the question, the user taps on the start button, which starts the trial time and opens a new window. The new window contains calendar dates for one month at the bottom of the screen.

Original: Once the user selects a date (e.g., July 7) with touch, the date gets highlighted with greencolor and a number of calendar events  on that specific date is displayed at the top.
Revised: Once the user taps on a date, the date is highlighted in green and a number of calendar events scheduled on that specific date is displayed at the top of the screen, as shown in Figure 6b.

Original: To find the number of calendar events that should be added for each date, webriefly surveyed students and faculty members and found that they commonly have 3-5 events (e.g., classes or meetings) per day, excluding weekends.
Revised: To determine a reasonable number of calendar events to add for each date, we surveyed stu- dents and faculty members and found that they commonly haveto 5 events (e.g., classes or meetings) per day, excluding weekends.

Original: Consequently, we added 3 to 5 events,  represented with an event title and time (e.g., ”Department Meeting 15:00-16:00” ), for each day except for Saturday and Sunday.
Revised: Consequently, we used 3 to 5 events, each represented with an event title and time (e.g., “Department Meeting 15:00-16:00” ), for each day except for Saturday and Sunday.

Original: Once the user taps on an event, the app opens a  new window(i.e., “detailed view”) containingdetailed information about that event (Figure 6c).
Revised: Once the participant taps on an event, the app opens a detail view in a new window containing information about that event, as shown in Figure 6c.

Original: The design of detailed view was inspired bythe Android’s generic calendar applications that contains event title, event time, persons hosting/attending in theevent (e.g., Host, Attendees), event type and mode (weekly meeting, online/in-person), reminder-related information (e.g., reminder type, reminder time).
Revised: The design of detail view is inspired by Android’s generic calendar application that contains event title, event time, names of people hosting or attending the event, the event type and mode (weekly meeting, online/in-person), reminder-related information (e.g., reminder type and time).

Original: After checking the detailed view, the user can  tap the “back” button or swipe left to return to the previous screen to view the other events on that day.
Revised: After checking the detail view, the participant can either tap the “back” button or swipe left to return to the previous screen to view the other events on that day.

Original: With FaceUI, a trial also starts withshowing the screen displaying a query prompt.
Revised: With FaceUI, a trial also starts with the screen displaying a query prompt and a start button.

Original: Once the user tapson thestart button, it opens a new window showing calendar dates at the bottom  (Figure 6d).
Revised: Once the user taps the button, trial time starts and a new window shows calendar dates at the bottom of the screen (Figure 6d).

Original: While designing FaceUI, we leveraged the empty mid-air space in front of the user to accommodate virtual windows for frequent browsing.
Revised: While designing FaceUI, we leveraged the empty mid-air space in front of the user to accommodate virtual windows for frequent browsing.

Original: Therefore, we used FaceUI in conjunction withtraditional touch input where touch is used to selectan item (e.g., a  datefrom the calendar) and FaceUI was used for browsing content (e.g., “detailed view”) by moving the phone in mid-air space.
Revised: Accordingly, we now use FaceUI in conjunction with touch input: touch is used to select a calendar date and FaceUI is used for browsing the detail views of the events taking place that date by moving the phone in mid-air space.

Original: Note that as we have maximum 5 events for a date, we decide to only place the detailed views between +40 ◦ and -40 ◦ in the horizontal direction  rather than placing them in a grid.
Revised: As we have at most five events for a date (according to our small survey), we place the corresponding detail views between +40 ◦ and -40 ◦ in the horizontal direction in mid-air rather than placing them in a grid.

Revised: We instructed the participant to keep the head still and to move the phone to browse the detail views in mid-air space.

Original: Once the user selects a date, FaceUI shows the details of an event (i.e.,  detailed view) on the top half of the screen and the user can browse other events on the date by moving the phone horizontally in mid-air (Figure 6d).
Revised: Once the participant selects a date (using touch), FaceUI shows the details of an event (i.e., its detailed view) on the top half of the screen. Now the participant can call in the details of the other events taking place that date by moving the phone horizontally in mid-air, as visualized in Figure 6d.

Original: Phone movements along vertical directions are ignored.
Revised: Phone movements in the vertical direction are ignored.

Original: After inspecting all the event details, the user tapson a check button (for both touch and FaceUI interfaces) to open a popup window containing multiple answers (Figure 6e). Once theuser selects an answer  and taps the select button, the  app provides audio feedback on the correctness of the answer.
Revised: When the participant believes having found the answer to the task question (after having inspected the events’ detail views), the participant submits the answer by calling in an answer window (Figure 6e) and then selecting one of the listed answer alternatives. The submission step is the same for both the touch interface and for FaceUI: the participant taps a check button (the green button in the middle of the screen in Figure 6b and d) to see the answer window and selects the answer using finger taps. The app provides audio feedback on the correctness of the answer.

Original: If theselected answer is correct, the trial time stops, and the  nexttrial is displayed on the screen.
Revised: If the answer is correct, the trial time stops, and the query prompt for the next is displayed.

Original: If incorrect, audio feedback for incorrect selection is provided,  and the user is required to redo the trial immediately.
Revised: If incorrect, audio feedback for incorrect answer is provided, trial time stops, the trial is marked as erroneous, and the participant is required to redo the same trial, from the beginning (i.e., with the query prompt along with the start button).

Original: The study used a 2 × 3 within-subjects design for factors Interface (FaceUI, Touch), and Number of events (3, 4, and 5).
Revised: The study used a 2 × 3 within-subjects design for factors Interface (FaceUI, Touch) and Number of events (3, 4, and 5).

Original: For each combination of factors, participants performed 10 repetitions result- ing in 60 error-free timed trials per participant.
Revised: Participants performed ten trials for each factor combination, resulting in 60 errorfree timed trials per participant.

Original: The order ofthe Interface  was counter-balanced across participants and Number of events orders were randomized within each Interface .
Revised: The order of Interface levels was counter-balanced across participants. The order of Number of events levels was randomized within each Interface level.

Original: Participants were provided with practice trials for each combination until they felt comfortable operating the two interfaces.
Revised: Participants were provided with practice trials for each combination until they felt comfortable operating the two interfaces.

Original: A study session lasted approximately 45 minutes.
Revised: A study session lasted approximately 45 minutes.

Original: 6.3 Results
Revised: 6.3 Results

Original: Measures : Our study tasks include three sub-tasks: (i) selecting a correct date from the calendar, (ii) browsing event details (i.e., detailed view) on the selected date  and (iii) selecting the correct answer.
Revised: Our study task includes three sub-tasks: (i) selecting the requested date from the calendar, (ii) browsing event details for the selected date (i.e., switching between the detail views), and (iii) selecting the answer from the answer window.

Original: Therefore, we recorded the following times: Browsing start time is the timewhen the participants read the question and click on the start button to  the time when they select the correct date from the calendar; Browsing time is the time  from when participants select the correct date to the time when they finish browsing detailed view and tap on the check button to open the interface containing four possible answers; and Selection time is the time from when they tap on the check button to  thetime when they select an answer and press the submit button.
Revised: Therefore, we recorded the following: start time is the time the participant needed from tapping the start button to tapping the requested date from the calendar; browsing time is the time needed to browse and inspect the detail views, i.e., from selecting the correct date to tapping the check button to call in the answer window; and selection time is the time from tapping the check button to tapping the select button after having selected one of the options in the answer window.

Original: In addition, we record trials where participants selected a wrong answer.
Revised: In addition, we recorded trials where participants submitted a wrong answer.

Original: 6.3.1 Error trials, outliers, and  trial time
Revised: 6.3.1 Error trials, outliers, and total trial time

Original: We marked a trial as an error trial if participants answered incor- rectly.

Original: We observed that participants selected wrong answers intrials (5.38%): 24 with  Touch Interface (3.15%) and 17 with FaceUI (2.23%).
Revised: We observed that participants submitted the wrong answers in 41 trials (5.38%): 24 with the Touch interface (3.15%) and 17 with FaceUI (2.23%).

Original: A Wilcoxon Signed-Rank test showed no difference between the two Interfaces .
Revised: A Wilcoxon Signed-Rank test showed no differ- ence between the two Interfaces .

Original: To analyze thetimes (e.g., browsing start time, browsing time or selection time), we first removed all erroneous trials and then removed eight outlier trials with a total trial time  outside of ± 3 SD.
Revised: To analyze the time results, we first removed all erroneous trials and then removed eight outlier trials with a total trial time (start time + browsing time + selection time) outside of ± 3 SD.

Original: Overall, we found that participants 13% faster with FaceUI than touch interfaces (with FaceUI 15.2s and touch 17.2s).
Revised: Overall, the total trial time was 11.6% faster with FaceUI than with Touch (FaceUI 15.2s and Touch 17.2s).

Original: We used Repeated Measures ANOVA and Bonferroni adjusted post-hoc pairwise comparisons to analyze the times.
Revised: We used Repeated Measures ANOVAs and Bonferroni adjusted post-hoc pairwise comparisons to analyze the different times measurements.

Original: 6.3.2 Browsing start time
Revised: 6.3.2 Start time

Original: Figure 7a shows thebrowsing start time for the two Interfaces and three Number of events .
Revised: Figure 7a shows the start time for the two Interfaces in the three Number of events conditions.

Original: There were no significant differences inbrowsing start time between the two Interfaces ( F 1 , 11 = 0 .
Revised: There was no significant difference in start time between Touch and FaceUI ( F 1 , 11 = 0 .

Original: 04, p = 0 . 84).
Revised: 04, p = 0 . 84).

Original: Participants with Touch interfaces were (2.31s, SE 0.31) slightly faster than FaceUI (2.35ms, SE 0.30).
Revised: The mean start time was 2.31s (SE 0.31) for Touch and 2.35s (SE 0.30) for FaceUI.

Original: Similarly, we did not find any significant differences inbrowsing start time between the  Number of events  ( F 2 , 22 = 0 .
Revised: There were also no significant differences in start time between the three Number of events conditions ( F 2 , 22 = 0 .

Original: 26, p = 0 . 78).
Revised: 26, p = 0 . 78).

Original: 2, 3 and 4 events were taking 2.28s (SE 0.30), 2.34s (SE 0.28) and 2.38s (SE 0.31), respectively.
Revised: Conditions with 3, 4, and 5 events took 2.28s (SE 0.30), 2.34s (SE 0.28), and 2.38s (SE 0.31), respectively.

Original: There was  no significant Interface × number of events effecton browsing start time ( F 2 , 22 = 0 .
Revised: There was also no significant interaction effect ( F 2 , 22 = 0 .

Original: 61, p = 0 .
Revised: 61, p = 0 .

Original: 6.3.3 Browsing time
Revised: 6.3.3 Browsing time

Original: Figure 7b displays the browsing time for the two Interfaces and three Number of events .
Revised: Figure 7b shows the browsing time for the two Interfaces in the three Number of events conditions.

Original: A RM-ANOVA showed that the browsing time was significantly different depending on Interfaces ( F 1 , 11 = 6 . 52, p < 0 .  and  the  Number of events ( F 2 , 22 = 39 .
Revised: The browsing time was significantly different between the two interfaces ( F 1 , 11 = 6 . 52, p < 0 . 05) and between the three levels of Number of events ( F 2 , 22 = 39 .

Original: 55, p < 0 .
Revised: 55, p < 0 .

Original: Bonferroni adjusted post-hoc comparisons showed that FaceUI with  9.4s (SE 0.56) was significantly faster than touch interfaces with  11.4s (SE 0.77).
Revised: FaceUI, with a mean Browsing time of 9.4s (SE 0.56), was significantly faster than touch interfaces, with a mean Browsing time of 11.4s (SE 0.77).

Revised: Across the three Number of events conditions, FaceUI (9.4s, SE 0.56) was 17.5% faster than Touch (11.4s, SE 0.77).

Original: For number of events , each pairwise comparison was significant  (all p’s ¡ 0.001) where the higher number events took longer time than the lower (3 events - 7.95s, SE 0.49, 4 events - 10.24s, SE 0.59 and 5 events - 13.09s, SE 0.82).
Revised: Bonferroni adjusted post-hoc pairwise comparisons between the three levels of Number of events showed significant differences (all p ’ s < 0 . 001): unsurprisingly, with more events to inspect the longer time it took (3 events, 7.95s, SE 0.49; 4 events, 10.24s, SE 0.59; and 5 events, 13.09s, SE 0.82).

Original: There was no significant Interface × number of events on browsing time ( F 2 , 22 = 1 . 70, p = 0 . 21).
Revised: The two factors did not interact ( F 2 , 22 = 1 . 70, p = 0 . 21).

Original: 6.3.4 Selection time
Revised: 6.3.4 Selection time

Original: Figure 7c shows the selection time.
Revised: Figure 7c shows the selection time for the two Interfaces in the three Number of events conditions.

Original: We observed no significant differences in selection time between the two Interfaces ( F 1 , 11 = 0 .
Revised: We observed no significant difference in selection time between Touch and FaceUI ( F 1 , 11 = 0 .

Original: 92, p = 0 . 
Revised: 92, p = 0 . 36).

Revised: The mean selection time was 1.59s (SE 0.06) for Touch and 1.64s (SE 0.04) for FaceUI.

Original: 36) or three  Number of events ( F 2 , 22 = 1 . 28, p = 0 . 30).
Revised: There were also no significant differences in selection time between the three levels of factor Number of events ( F 2 , 22 = 1 . 28, p = 0 . 30).

Original: 2, 3 and 4 events were taking 1.61s (SE 0.06), 1.59s (SE 0.05) and 1.66s (SE 0.04), respectively.
Revised: Conditions with 3, 4, and 5 events took 1.61s (SE 0.06), 1.59s (SE 0.05), and 1.66s (SE 0.04), respectively.

Original: There was no significant Interface × number of events effect on Selection time ( F 2 , 22 = 0 .
Revised: The two factors did not interact ( F 2 , 22 = 0 .

Original: 80, p = 0 .
Revised: 80, p = 0 .

Original: 6.3.5 Subjective Feedback
Revised: 6.3.5 Subjective feedback

Original: Participants had prior experience with touch interfaces and were very comfortable using smartphones.
Revised: Participants had prior experience with touch interfaces and were very comfortable using smartphones.

Original: Once participants mentioned: ”I have been using touch interfaces on smartphones for a long time, I feel comfortable with them” .
Revised: One participant mentioned: “I have been using touch interfaces on smartphones for a long time, I feel comfortable with them” .

Original: We also observed a bias for touch interfaces where we asked participants to rate the two directions according to their overall preference on a 5-point scale.
Revised: We also observed a bias for the Touch interface when we asked participants to rate the two interfaces according to their overall preference on a 5-point scale.

Original: Touch interfaces (mean rating 4.5, SD 0.5)was rated higher than FaceUI (mean rating 3.5, SD 1.1).
Revised: Touch was rated higher (mean rating 4.5, SD 0.5) than FaceUI (mean rating 3.5, SD 1.1).

Original: However, participants acknowledged that the concept of FaceUI was utterly new, and  they were not familiar with any similar concepts.
Revised: However, participants acknowledged that the concept of FaceUI was utterly new and interesting, and said they were not familiar with any similar concepts.

Original: Once participants commented: ”It’s a new method and I don’t have experience with it.
Revised: One participant commented: “It’s a new method and I don’t have experience with it.

Original: However, it seems a potential method for operating smartphones” .
Revised: However, it seems a potential method for operating smartphones” .

Original: We believe that once such interfaces  become available on commercial smartphones, people will feel comfortable using them for accessing on-screen content.
Revised: Along these lines, we believe that once spatial interfaces with front-facing camera input, such as our FaceUI-calendar, become available on commercial smartphones people will feel comfortable using them.

Original: 6.4 Summary
Revised: 6.4 Summary

Original: Our participants had more than 9 years of prior experience using the traditional touch interfaces where FaceUI was a newexperience for them.
Revised: All our participants had more than nine years of prior experience us- ing the traditional touch-based interfaces provided on smartphones, whereas FaceUI interactions were a new for them.

Original: Despite this,  results demonstrated that the FaceUI  offers faster access to information than traditional touch interfaces.
Revised: Despite this, our results demonstrate that the FaceUI approach offers faster access to information than the touch-based alternative.

Original: We found that the browsing starttime andthe selection time were comparable for both interfaces as participants used the same procedures (i.e., selecting dates or buttons with touches) with both interfaces.
Revised: Unsurprisingly, we found that both interfaces’ start and selection times were comparable; the corresponding parts required touch input (i.e., tapping on date entries and buttons) in both interfaces.

Original: How- ever, we observed that participants were significantly faster with browsing information with FaceUI (see figure 7).
Revised: Instead, the difference between the two interfaces concerns the way they provide access to different calendar events’ detail views and how the user can switch between these views.

Original: FaceUI enables quick retrieval of spatially located virtual UIs by moving the phone into large space whereas touch interfaces require users switching between windows with frequent tap and swipe, which is known to be costly [17].
Revised: FaceUI enables the user to quickly retrieve spatially located virtual windows by moving the phone inside a large space. In contrast, touch interfaces require the user to switch be- tween windows using frequent taps on small buttons or by swiping across the screen.

Revised: Our participants could browse the event infor- mation significantly faster using the FaceUI approach than with the touch interface (Figure 7b).

Original: From the results of the studies we believe that FaceUI can be a promising  supporting input technique to the conventional touch  input and this new interaction can serve the users with the best timing in browsing multiple windows.
Revised: Backed by our study results, we believe that FaceUI is a promising strategy that serves well as a supporting input technique in combination with conventional touch input. The study task required the participants to frequently alternate between touch input and spatial FaceUI-input during the study trials. Apparently, they could do so effortlessly.

Original: 7 D ESIGN G UIDELINES 
Revised: 7 D ESIGN G UIDELINES AND F ACE UI A PPLICATIONS

Original: We  summarize and present our key findings as design guidelines.
Revised: We first summarize and present our key findings as design guide- lines for FaceUI-enabled interfaces then we present three demo applications. 7.1 Design Guidelines

Original: Our investigation offers the following guidelines to designers for interfaces similar to FaceUI:
Revised: Our investigation offers the following guidelines to designers of FaceUI interactions and interfaces.

Original: 7.1 Distance
Revised: Movement distance .

Original: We found that participants preferred moving their hands within 40cm around the face.
Revised: We found that participants preferred mov- ing their hands within 40 cm from the face.

Original: Using this space for accessing virtual UIs will also help minimizing concerns related to arm fatigue.
Revised: Using this space for accessing mid-air regions will also help minimizing concerns re- lated to arm fatigue.

Original: Thus, we recommend designers placing UIs in the  mid-air space within 40 cm  from users’ faces.
Revised: Thus, we recommend designers to limit the use of mid-air space to maximum 40 cm away from the user’s face.

Revised: Within this range, both users with short and users with long arms can comfortably access the space.

Original: 7.2 Direction and Region
Revised: Movement direction and mid-air regions .

Original: Results indicate that participants preferred  the horizontal  over the vertical direction for moving the phone to access items.
Revised: Results indicate that participants preferred moving the phone in the horizontal direction over the vertical direction.

Original: Designers should emphasise placing items in this direction.
Revised: Accordingly, designers should prioritize horizontal movements and try to avoid interactions that require the user to perform extensive movements up and down in front of the face.

Original: In addition, partic- ipants reported difficulties accessing items in the up region. Thus, limited or no items should be placed in this region.
Revised: In addition, participants reported difficulties accessing areas far up “above” the nose, 20 ◦ above the nose or less works good.

Original: 7.3 Viewing Angle
Revised: Viewing angle .

Original: Items should be placed within a comfortable viewing angle as users performance significantly degrades once the targets are placed far from the  comfortable viewing angles. 
Revised: Designers do not only need to consider the ease with which users can physically move around and access areas in the mid-air space. Designers also need to consider how well the user can see the screen content when holding the phone in mid-air space. In general, positions further away from the mid-point in front of the user’s face provide less comfortable viewing angles. Our study results suggest that viewing angles between -40 ◦ (left of the nose) and +40 ◦

Revised: (right of the nose) in the horizontal direction allow for clear and unconstrained visual access to screen content.

Original: Caution is needed, especially when placing items in extreme angles which could create eyestrain. Results from our study suggest placing items between -40 ◦ left and +40 ◦ right in the horizontal direction, between +20 ◦ up and -30 ◦ down in the vertical direction.
Revised: In the vertical direction the range should be limited to +20 ◦ (above the nose) to -30 ◦ (below the nose). Extreme viewing angles warrant caution: they are only suitable for presenting screen content that can be grasped with a quick glimpse. Furthermore, extreme viewing angles can cause eyestrain.

Original: 7.4 Mid-Air Space for Browsing-Intensive Tasks
Revised: Mid-air space for browsing-intensive tasks .

Original: Study 3 results showed that mid-air space is more effective for browsing through UIs than the traditional touch interface.
Revised: Study 3 results showed that, compared to traditional touch-only interfaces, blending touch with mid-air interactions is an effective combination to quickly browse through and visually inspect different parts – or views – in an application.

Original: This is primarily due to the minimum switching costs(e.g., small device movements) involved with navigating between UIs with FaceUI.
Revised: This is primarily due to the minimum switching costs involved with navigating between the parts, e.g., small device movements in front of the face for FaceUI vs. taxing taps on navigation buttons in a touch interface.

Original: Therefore, we suggest that designers consider using  mid-air space to place UIs to perform any browsing-intensive tasks with FaceUIenabled interfaces.
Revised: Accordingly, we suggest that designers consider using the FaceUI approach to offload interactions into mid-air space particularly for lengthy browsing-intensive tasks that include frequent switching between different data views or windows. The more switching is required, the larger advantage can be expected from using the FaceUI approach.

Original: 8 F ACE UI ENABLED APPLICATIONS
Revised: 7.2 FaceUI-Enabled Applications

Original: We designed  the following three applications to demonstrate how face-centered spatial user interfaces can be used in a off-the-shelf smartphone.
Revised: We designed and implemented the following three applications to demonstrate how the FaceUI approach can be used on an off-theshelf smartphone.

Original: 8.1 Image Browser 
Revised: Image Browser .

Original: In Study 3, FaceUI implementation  did not consider a scenario where  items were located in a grid.
Revised: The FaceUI implementation in Study 3 did not consider scenarios where interface windows, views, or items are located in a grid in mid-air.

Original: However, manyFaceUI-enabled application could be benefited from such as an item arrangement style.
Revised: However, many application would likely benefit from such as an item arrangement style.

Original: Consequently, we developed an image browsing app that offloads a set of images into a5 × 3  grid on the mid-air space in front of users.
Revised: Consequently, we developed an image browsing app that offloads a set of images into a 3 × 5 grid in mid-air space in front of the user.

Original: Users can browse images by  moving the device in  horizontal and vertical directions.
Revised: The user can browse images by simply moving the device in the horizontal and vertical direction.

Original: While browsing the images, users can touch the screen to access further details about an image.
Revised: While browsing the images, the user can tap the screen to access further details about an image.

Original: Figure 8a-b demonstrates the app scenario where a user is browsing the images by moving the phone along horizontal and vertical directions.
Revised: Figure 8a-b demonstrates the app scenario.

Original: 8.2 Message History 
Revised: Message History .

Original: Scrolling through  items with touch interfaces is cumbersome and time-consuming, especially for one-handed interaction mode [28].
Revised: Scrolling through long lists of items with touch interfaces can be cumbersome and time-consuming, especially when using one-handed interaction mode [30].

Original: We developed an application leveraging face-centered UIs to scroll through messages in a messenger app with one hand.
Revised: We developed an application leveraging face-centered interactions to scroll through messages in a messenger app using one hand.

Original: In our implementation, the app allows users to scroll through messages by moving their phone vertically between +20 ◦ up and -30 ◦ down (Figure 8c).
Revised: In our implementation, the user can scroll through messages by moving the phone vertically between +20 ◦ up and -30 ◦ down (Figure 8c).

Original: Clutching can be used to browse extended message history by tapping on the screen.
Revised: Clutching works by pressing down on the screen and repositioning the device in mid- air space. When having released the press, the following device movements scroll the message list.

Original: 8.3 Map Navigation 
Revised: Map Navigation .

Original: We  designed  a FaceUI enabled map application where users move the phone in mid-air to zoom and pan the map.
Revised: We also designed and implemented a FaceUI- enabled map application where the user moves the phone in mid-air to navigate across the map and to zoom the map.

Original: Users phone movements to the left/right/up/down (relative to their face) were associated with  the map movement in the same direction (Figure 8d).
Revised: Phone movements going left, right, up, or down (relative to the user’s face) are associated with movements in the corresponding directions over the map which remains stationary under the phone, as visualized in Figure 8d.

Original: Map zoom in or zoom out was mapped with the distance from the face (smaller the distance, zoomed in the map).
Revised: The user can zoom in or zoom out by changing the distance between the face and the device. The user can also reposition the map under the screen. First the user grabs hold of the map by a pressing down on the screen and then moving the device, and so also the map. When the user releases the screen, the map is also released and anchored at its new position.

Original: 9 L IMITATION AND F UTURE WORK
Revised: 8 L IMITATION AND F UTURE W ORK

Revised: We have only started our exploration of the FaceUI approach and our current work has limitations.

Revised: Accordingly, we see a need for further investigations and also plenty opportunities for interesting future work.

Original:  FaceUI requires users to move the phone in mid-air, which may cause eye and arm fatigues for prolonged use.
Revised: For a start, FaceUI requires users to move the phone in mid-air to access screen contents, which may cause eye and arm fatigues for prolonged use.

Original: Further investigation is needed to explore solutions to minimize such fatigues.
Revised: Further investigation is needed to explore solutions to minimize such fatigues.

Revised: Suggestions and design guidelines for reducing arm fatigue provided by prior research [22] should be examined also for FaceUI scenarios.

Revised: Related to this, we asked our participants to keep the head still and to only move the phone in the studies but further studies could explore possible ways to leverage the combination of head and smartphone movements to access in-air regions in a way that does not causes any arm fatigue.

Original: In addition, using FaceUI in public spaces may trigger feelings such as embarrassment or discomfort due to hand movements which may attract by-passers’  attention.
Revised: Using FaceUI in public spaces may trigger feelings such as em- barrassment or discomfort due to hand movements which may attract by-passers’ undesired attention.

Original: Thus, it is also important to carry out studies to explore the social acceptance of FaceUI in public and private spaces.
Revised: Thus, it is also important to carry out studies to explore the social acceptance of performing FaceUIenabled interactions in public (and private) spaces.

Original: We asked participants to keep the head static and move the phone only during studies.

Original: Further studies can explore how to lever- age head and smartphone movements to access spatial UIs.

Original: While using FaceUI, on-screen information can be viewed by surrounding people,  triggering privacy concerns.
Revised: Moreover, when using FaceUI, on-screen information can be viewed by surrounding people, which may triggering privacy concerns.

Original: Future studies need to investigate users and by-passers privacy concerns and explore approaches such as software and hardware-based privacy filters to only keep the information visible to users.
Revised: Future studies need to investigate users’ and by-passers’ privacy concerns. Future studies could also explore strategies to implement software and hardware-based privacy filters that keep screen content only visible to the user.

Original: We only investigated the performance of FaceUI for accessing  windows located in the horizontal direction.
Revised: We only investigated the performance of FaceUI when accessing application windows located in the horizontal direction.

Original: Further investigations are needed to explore natural delimiters to switch between FaceUI and touch and the performance of FaceUIenabled applications, where windows are arranged in both horizontal and vertical directions. We foresee future research exploring ways to reduce switching between UIs for the traditional touch interfaces. Lastly, future work needs to examine the performance of FaceUI in different usage contexts such asstanding, sitting or walking in public and private spaces.
Revised: Further investigations are needed to explore natural delimiters to switch between FaceUI and touch and the performance of FaceUI-enabled applications, where windows are arranged in both horizontal and vertical directions, such as in our image browser (Section 7.2)

Original: 10 C ONCLUSION
Revised: 9 C ONCLUSION

Original: We have presented FaceUI, a novel  approach leveraging mid-air space to access face-centered spatial user interfaces.
Revised: We have presented FaceUI, a novel interaction approach for smart- phones which leverages mid-air space to access face-centered spatial user interfaces.

Revised: The FaceUI approach is basically a self-contained position tracking mechanism for off-the-shelf smartphones that uses the smartphone’s front-facing camera to track the phone’s position relative to the user’s face.

Original: Through two user studies, we first explored different factors that influence the design and performance of  FaceUI.
Revised: Through two user studies, we first explored different factors that influence the design and performance of applications relying on FaceUI.

Original: Based on the results, we designed a FaceUI-based calendar app and compared users’ performance using the calendar with a touch-based calendar interface.
Revised: Based on the results, we designed a FaceUI calendar app. We then compared users’ performance using this calendar with a touch-based calendar interface in analytical tasks where users searched for and compared different calendar events.

Original: Results showed that FaceUI is a promising approach to enable faster access to UIs than the traditional touch interfaces.
Revised: Results showed that FaceUI is a promising approach that enables fast access to parts of interfaces by off-loading these parts into mid-air space in front of the user where the user can easily switch between the parts by moving the phone to the corresponding in-air location, which displays the part on the screen.