# On the Effects of Progressive Reduction as Adaptation Strategy for a Camera-Based Cinematographic User Interface

Axel Hoesl, Mujo Alic, Andreas Butz

► To cite this version:

Axel Hoesl, Mujo Alic, Andreas Butz. On the Effects of Progressive Reduction as Adaptation Strategy for a Camera-Based Cinematographic User Interface. 16th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2017, Bombay, India. pp.513-522, 10.1007/978-3-319-67744-6_32 hal-01676178

HAL Id: hal-01676178

https://inria.hal.science/hal-01676178

Submitted on 5 Jan 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# On the Effects of Progressive Reduction as Adaptation Strategy for a Camera-based Cinematographic User Interface

Axel Hoesl, Mujo Alic, and Andreas Butz

LMU Munich, Munich, Germany
{axel.hoesl,andreas.butz}@ifi.lmu.de
mujo.alic@campus.lmu.de

**Abstract.** Camera-based user interfaces (UI) became increasingly relevant, especially on mobile devices, with location independent media production tools such as drones. The devices traditionally render the necessary user interface elements for remote control on top of the content they display. This often leads to occlusion and visual clutter. Progressive Reduction is a recently proposed UI adaption strategy that can help to minimize these issues while maintaining usability. It exploits learning and spatial memory effects in order to gradually reduce the visual footprint of UI elements. We conducted two user studies to investigate the effects of this approach. In the first study, we compared three design alternatives to obviate interference due to design (N=10). Based on the most promising design, we conducted a second user study (N=18) investigating the effects of two different reduction strategies (icons-first and background-first). We collected data on perceived control, workload and creativity support in addition to semi-structured interviews. Our results indicate that there was only a minor decrease in perceived control up to a certain amount of reduction. Beyond that, however, the negative effects on perceived control become unacceptable to users. This was observed for all applied reduction strategies.

**Keywords:**

Graphical user interface, minimalist user interface, progressive reduction, sense of control, cinematography, motion control, user-centered design.

## 1 Introduction

Mobile devices are commonly used for the casual creation and manipulation of photos and videos. Actually, they are also increasingly used (semi-)professionally, e.g., by pro-sumers, enthusiasts and even experts in film making. In cinematographic production, they often serve as remote control units and help to steer drones or other motorized motion control systems. Mobile devices are particularly attractive as they can combine a remote control unit with the equally necessary display unit (for steering or for reviewing results). They provide multi-touch interaction, general functional versatility, high connectivity, and often an easy set-up on location.
For enthusiast users, an unoccluded view on a transmitted video stream is important, as they create their results with high effort and great care to details. However, so far, the graphical user interface (UI) elements for motion control systems are often displayed as an overlay on top of the video streams, taking up screen space and hence partially occluding the view.

An established way of minimizing the number of visual elements displayed is *Progressive Disclosure (PD)* [15, 20] which follows the approach of disclosing or "hiding" features into a collapsed or off-screen menu. Items that are less frequently used are then only accessible via a menu. Only when the menu is opened or displayed, the hidden elements become visible again. This minimizes the need for displaying graphical UI elements. For continuous control elements and frequently used elements, however, disclosure might not be a suitable strategy to declutter the screen as they are particularly often needed. Especially when things go wrong, easy access is crucial. Additionally, the perceived affordances [17] of a user interface can diminish when too much is hidden from the user early on. A recently discussed alternative approach and/or extension to PD is *Progressive Reduction (PR)* [9]. PR follows the idea that interface elements that are frequently used become reduced in terms of their visual appearance. In detail, the characteristics of the used visual variables [4] (e.g. size, form or opacity) can be reduced. As this process happens gradually over time and not abruptly, it exploits the users ability to learn functions, abstract representations and locations of certain interface elements. The adaption of the UI is synchronized to a user's usage/learning curve for an application or could also be based on prior knowledge if tracked at system level.

## Contribution

In our work, we explored how the combined reduction of size, opacity, icon form and icon visibility of UI elements could help to declutter screens on mobile touch screen devices. We conducted two studies to determine the effects of Progressive Reduction as an adaptation strategy on a camera-based UI for a cinematographic motion control system. In the first study (N=10), we compared three design alternatives (*software joystick*, *extended software joystick* and *single knob*) to obviate interference due to the UI design. When using touch interaction, usually a diminished sense of precision and control can be observed. Hence, we used the most promising design (*extended software joystick*) to collect data on perceived *control* and *workload* in addition to *creativity support* and qualitative feedback to compare two reduction strategies (*icons-first* and *background-first*) in a second study (N=18). Our results indicated that it was generally possible to gradually reduce the appearance of UI elements without a major negative impact on the sense of control. However, there also was a certain limit to the amount of reduction that could be applied before the perceived *control* significantly decreased to an unacceptable level. This could be observed independently from the applied reduction strategy.
## 2 Related Work

Progressive Disclosure is a well-established concept in HCI literature as, for example, described by Nielsen [15] or Tidwell [20]. In contrast, *Progressive Reduction* is a rather recently proposed concept and so far was mainly discussed online [9]. Up to now, It has received less attention in the literature, which makes further studying of the subject matter necessary. However, dynamic adaption of UI elements has been investigated before. Most prominently in the domain of augmented reality. In particular, it was used to enhance text readability [8], filter information [13], adapt symbols in camera-based [3] or map-based user interfaces [16]. Adaptation is particulary interesting as it allows to design minimalist user interfaces by exploiting human spatial memory. Spatial memory was also successfully exploited before in UI design and (among other approaches) led to marking menus [14] and even imaginary interfaces [10]. In the case of PR however, only little is known about its side-effects and its limitations.

## 3 Design Alternatives and Implementation

We implemented three design alternatives (Figure 1) in a Unity 3D virtual environment on an off-the-shelf Android tablet. As a status quo design we used a *software joystick* as often found in the wild, e.g., for remote control of drones or in games (Figure 1a). Here, the left software joystick controls translation and the right software joystick rotation each in two dimensions (left-right, up-down). As the first alternative, we implemented an *extended software joystick* with a pie menu (Figure 1b). The pie menu lets users choose, which dimension(s) they want to map onto the software joystick depending on the use case. Additionally, the structure of a pie menu lends itself well to mental and physical learning (e.g., muscle memory) [14]. In consequence, once learned, not all options need to be displayed, which helps to minimize the number of visual elements necessary. As a second alternative, we used a UI that decoupled translational and rotational control to better suit the cinematographic context. In the field, the various axes are often controlled by different operators [12]. The first camera operator often delegates translational moves to a grip<fnref n="1" />. In our case, this could be delegated to an assistance system. In the User Interface, this was presented as a horizontally restricted joystick placed in the centre of the display (Figure 1c). This *single knob* could be dragged horizontally with the distance to the centre being mapped to speed of the camera motion. Once content with the settings, the users could "lock" them by swiping upward saving the current direction and speed. For all UIs the button size was at least 11x11mm as recommended [1, 2].

## 4 User Study Comparing Design Alternatives

To identify the most promising design alternative, we conducted a user study in a controlled laboratory environment.

<footnote n="1">¹ A technical operator responsible for supportive tools such as dollies or sliders etc.</footnote>
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** The software joystick (Figure 1a) and our design alternatives extended software joystick with fully expanded pie menu (Figure 1b) and the single knob (Figure 1c)</figcaption></figure>

**4.1 Study Tasks**

To provide a reasonably varying set of tasks, we asked the participants to execute a *product shot*, to follow a target moving along a horizontal *figure eight* (or infinity symbol) trajectory and to *roam freely*. The product shot is an often-used move, e.g., in advertising. It is rather easy to accomplish as the recorded object is static and only horizontal translation and rotation at low to medium speed is necessary. Following the trajectory of an eight on its side is more complex as the target is constantly in motion. Additionally, translation and rotation in two dimensions is necessary to properly follow it. This approach has already been taken in related work for the evaluation of a semi-automated camera crane [19]. The free roaming mimics a first exploration of possible camera angles and transitions. This task allows users to be more expressive than a task with a pre-defined goal, which is an important aspect in the design and evaluation of creativity support tools [18]. To keep participants interested, we spread multiple animated objects across the scene, thus encouraging exploration.

**4.2 Study Design**

We used a within-subject design with our design alternatives (3 levels) and study tasks (3 levels) described above as independent variables. To counteract learning effects, we provided each participant with a unique counter-balanced sequence of user interface and task combinations.

**4.3 Participants**

For the user study, we recruited 10 participants (3 female) with a median age of 26 years. Ages ranged from 20 to 37 years. All participants had normal or corrected-to-normal vision, were acquainted with touch devices and had no prior training in cinematographic camera control. One participant was left-handed. To be able to relate our inferences to the general population, we had no further requirements for the selection of the participants.
## 4.4 Procedure

We welcomed the participants, informed them about the procedure of the study and handed out a consent form. After having declared consent, we asked the participants to fill out a demographic questionnaire. This was followed by introducing the first design. We asked the participants to carry out the tasks in the given sequence while standing (to mimic an on-set usage context). After completing all tasks with one interface, we handed out a questionnaire asking the participants to rate it in terms of *control*, *workload* and *creativity support*. Then the next UI was introduced and the procedure repeated until all tasks were carried out with all interfaces. Before the debriefing, we conducted a semi-structured interview on their preference and the UIs usability.

## 4.5 Measurements

To collect quantified data on workload, we used the Task-Load Index (TLX) from NASA [11]. For the ratings on creativity support, we use a limited version of the Creativity Support Index (CSI) by Cherry and Latulipe [5]. We did not ask on all dimensions as some were, by design, not supported throughout all conditions, such as 'collaboration'. Thus, we only included questions on dimensions that were featured by our user interfaces, in particular exploration, motivation and enjoyment. For data on perceived control we handed out a modified version of the sense of control scale (SCS) provided by Dong et al. [6]. As the CSI is based on the TLX, they both use the same 20-point rating scale for each item. To be consistent with the prior items and to minimize confusion for the participants, we also used this report format in our version of the SCS.

## 4.6 Data Analysis and Results

For the data analysis, we used non-parametric tests (Friedman's ANOVA, Wilcoxon Signed-Rank) to test for statistical significance. A Bonferroni correction was used for the post-hoc tests to compensate for pairwise comparisons ($\alpha^* = .016$). Post-hoc tests were only conducted after a significant main effect was found.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Summary of the collected data of our study on design alternatives (Figure 2a) and progressive reduction strategies (Figure 2b)</figcaption></figure>
We found no significant difference for workload ($\chi^2(2)=2.0, p \le .368$) and creativity support ($\chi^2(2)=3.128, p \le .209$). However, for control we found a significant main effect ($\chi^2(2)=8.359, p \le .015$). Post-hoc pairwise comparison between the extended software joystick (Mdn=82.5) and the single knob (Mdn=45.0) indicated the extended software joystick could outperform the single knob ($Z \le 2.601, p \le .009, \eta^2 \le .677$). Comparing the extended software joystick to the traditional joystick (Mdn=67.5), we could not find a significant difference ($Z \le 2.310, p \le .021, \eta^2 \le .534$). Comparing the traditional joystick to the single also showed no significance ($Z \le 1.543, p \le .123, \eta^2 \le .238$).

In conclusion, the *extended software joystick* seemed most promising to us as it reached the highest absolute score in the control rating (Mdn=82.5), could outperform the single knob (p<.015) in terms of *control* (Figure 2a) and as it was also preferred when asked about in the interviews. We therefore used it to implement and evaluate two reduction strategies.

## 5 Reduction Strategies

Visual variables can be reduced in various ways. In our designs, we reduced the appearance of user interface element in terms of size, opacity, form and visibility.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Our reduction strategies *icons-first* (Figure 3a) and *background-first* (Figure 3b)</figcaption></figure>

To estimate the effects of the gradual reduction, we compared two strategies: *icons-first* (IF) and *background-first* (BF). To make the transition between the levels of reduction less abrupt and to provide easily identifiable icons even in small sizes, we introduced an intermediate reduction state for both strategies (Figure 3). In this intermediate state the icons transition from their original appearance (e.g., "cross with arrows") to a simplified version (e.g. "plus") when the overall opacity is reduced to a value below 75%. As the next step, they disappear in different ways depending on the strategy. With the *icons-first* strategy, icons disappear before the background disappears (opacity level below 35%, Figure 3a). In contrast, with *background-first* the background disappears before the icons (opacity level below 35%, Figure 3b). To counteract a diminished performance due to Fitts' Law [7] we kept the actual touch-sensitive areas of the UI elements to their original size of 11x11mm.
# 6 User Study Comparing Reduction Strategies

To estimate the effects of progressive reduction in general and of the *icons-first* and *background-first* strategies in particular, we conducted a second user study in a controlled laboratory environment.

## 6.1 Study Tasks

To isolate the effect of the reduction strategies in this study, we used a roaming task similar to the previous study. We wanted to ensure that the participants would use every function an equal amount of the time. Hence, we altered the task from a *free* roaming task to a *prompted* roaming task. Here, three boxes were displayed on top of the screen prompting which settings to choose and which camera motion to carry out. The left box would indicate the setting for the left joystick. The right box prompted the setting for the right joystick and the center box the camera motion to be executed. All three boxes changed prompts every 10 seconds. The sequence was chosen in a pseudo random fashion to avoid unreasonable combinations such as moving up with one joystick and down with the other simultaneously.

## 6.2 Study Design

To avoid interference due to a learning effect we chose a between-groups design. Each participant therefore only experienced one strategy.

## 6.3 Participants

We recruited a total of 18 participants (7 female) with ages ranging from 19 to 33 years and a median age of 22.5 years. All had normal or corrected-to-normal vision, were acquainted with touch devices and had no prior training in cinematographic camera control. Two participants were left-handed. Similar to the previous study, we had no further requirements regarding the sample.

## 6.4 Procedure

As in the previous study we first welcomed the participants, asked for consent and collected demographic data. Then the participants were subjected to a user interface with a given reduction strategy and the above-mentioned prompted roaming task. Again, the participants were asked to perform the tasks while standing. After the first 5 minutes the PR of the user interface began. The interface was reduced further every 5 minutes until the final reduction step (invisible or imaginary UI [10]) was reached. At the final stage the participants performed the prompted tasks again for 5 minutes. Having performed a study task at a given reduction level for 5 minutes, we asked the participants to rate their sense of *control*. Data on *workload* and *creativity support* was only collected at given points of interest (after 5, 25 and 35 minutes) to keep interruptions to a necessary minimum. In total 35 minutes were necessary to perform the tasks with all levels of reduction. Having completed the tasks, we debriefed the participants.
## 6.5 Measurements

To rate the perceived sense of *control*, *workload* and *creativity support*, we applied the adapted questionnaires from our previous study.

## 6.6 Data Analysis and Results

Similar to the preceding study, we used non-parametric tests (Friedman's ANOVA, Wilcoxon Signed-Rank) to test for statistical significance. Also, a Bonferroni correction was applied to the post-hoc tests in order to compensate for pairwise comparisons ($\alpha^* = .016$). Post-hoc tests were only conducted after a significant main effect was found.

Overall, we found no significant difference between the reduction strategies for *control* ($Mdn_{IF}=75.0, Mdn_{BF}=65.0, Z \le .00, p \le 1.000$), *workload* ($Mdn_{IF}=49.16, Mdn_{BF}=49.16, Z \le .036, p \le .971$) and *creativity support* ($Mdn_{IF}=66.0, Mdn_{BF}=55.0, Z \le .361, p \le .718$).

Regarding the data plot (Figure 2b) we conducted further significance tests comparing the data gathered after 5, 25 and 35 minutes for each dimension. For *workload* we found a significant main effect ($\chi^2(2)=20.111, p \le .001$). Post-hoc pairwise comparison between 35 and 5 minutes ($Z \le 3.376, p \le .001, \eta^2 \le .633$) as well as 35 and 25 minutes ($Z \le 3.201, p \le .001, \eta^2 \le .569$) indicated significant differences. No effect was found comparing 25 to 5 minutes ($Z \le 2.289, p \le .022$).

For *creativity support* we found a main effect ($\chi^2(2)=13.914, p \le .001$). Also, post-hoc pairwise comparison between 35 and 5 minutes ($Z \le 3.028, p \le .002, \eta^2 \le .572$) as well as 35 and 25 minutes ($Z \le 2.534, p \le .011, \eta^2 \le .357$) indicated significant differences. Also, no effect could be found comparing 25 to 5 minutes ($Z \le 2.226, p \le .026$).

For *control* we also found a significant main effect ($\chi^2(2)=30.629, p \le .001$). Post-hoc pairwise comparison showed a significant difference between all measurements with 35 and 5 minutes ($Z \le 3.741, p \le .001, \eta^2 \le .778$), 35 and 25 minutes ($Z \le 3.730, p \le .001, \eta^2 \le .773$) and 25 to 5 minutes ($Z \le 2.773, p \le .006, \eta^2 \le .427$).

## 7 Conclusion

We explored how the combined reduction of size, opacity, icon form and icon visibility of user interface elements affected the perceived *control*, *workload* and *creativity support* of a camera-based user interface. In a first study (N=10), we compared three design alternatives (*software-joystick*, *extended software-joystick* and *single knob*). We used the most promising design (*extended software-joystick*) to compare two reduction strategies (*icons-first* and *background-first*) in a second study (N=18). Our results indicated that it was possible to gradually reduce the appearance of UI elements without a major negative impact up to a certain amount of reduction, after which it became unacceptable. This was observed independently from the applied reduction strategy.
References

1. Metrics & Keylines - Layout - Material Design Guidelines. https://material.io/guidelines/layout/metrics-keylines.html\#metrics-keylines-touch-target-size (2017)

2. Targeting - UWP App Developer — Microsoft Docs. https://docs.microsoft.com/en-us/windows/uwp/input-and-devices/guidelines-for-targeting (2017)

3. Carmo, M.B., Afonso, A.P., Ferreira, A., Cláudio, A.P., Montez, E.: Symbol Adaptation Assessment in Outdoor Augmented Reality. In: 2014 International Conference on Computer Graphics Theory and Applications (GRAPP). pp. 1–10 (2014)

4. Carpendale, S.: Considering Visual Variables as a Basis for Information Visualization (Jan 2003)

5. Cherry, E., Latulipe, C.: Quantifying the Creativity Support of Digital Tools Through the Creativity Support Index. ACM Transactions on Computer-Human Interaction (TOCHI) 21(4), 21 (2014)

6. Dong, M.Y., Sandberg, K., Bibby, B.M., Pedersen, M.N., Overgaard, M.: The Development of a Sense of Control Scale. Frontiers in Psychology 6, 1733 (2015)

7. Fitts, P.M.: The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement. Journal of Experimental Psychology 47(6), 381 (1954)

8. Gabbard, J.L., Swan, J.E., Hix, D., Kim, S.J., Fitch, G.: Active Text Drawing Styles for Outdoor Augmented Reality: A User-Based Study and Design Implications. In: 2007 IEEE Virtual Reality Conference. pp. 35–42 (Mar 2007)

9. Grinshtein, A.: Progressive Reduction - LayerVault Blog. http://layervault.tumblr.com/post/42361566927/progressive-reduction (May 2013)

10. Gustafson, S., Holz, C., Baudisch, P.: Imaginary Phone: Learning Imaginary Interfaces by Transferring Spatial Memory from a Familiar Device. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. pp. 283–292. UIST ’11, ACM, Santa Barbara, California, USA (2011)

11. Hart, S.G.: NASA-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 904–908 (2006)

12. Hoesl, A., Wagner, J., Butz, A.: Delegation Impossible? Towards Novel Interfaces for Camera Motion. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. pp. 1729–1734. CHI EA ’15, ACM (Apr 2015)

13. Julier, S., Lanzagorta, M., Baillot, Y., Rosenblum, L., Feiner, S., Hollerer, T., Sestito, S.: Information Filtering for Mobile Augmented Reality. In: Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000). pp. 3–11 (2000)

14. Lepinski, G.J., Grossman, T., Fitzmaurice, G.: The Design and Evaluation of Multitouch Marking Menus. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 2233–2242. CHI ’10, ACM, Atlanta, Georgia, USA (2010)

15. Nielsen, J.: Progressive Disclosure. https://www.nngroup.com/articles/progressive-disclosure (Apr 2006)

16. Nivala, A.M., Sarjakoski, T.L.: User Aspects of Adaptive Visualization for Mobile Maps. Cartography and Geographic Information Science 34(4), 275–284 (2007)

17. Norman, D.A.: The Design of Everyday Things: Revised and Expanded Edition. Basic Books, New York, NY, USA, 2 edn. (2013)
10 Axel Hoesl, Mujo Alic, Andreas Butz

18. Shneiderman, B., Fischer, G., Czerwinski, M., Myers, B., Resnick, M.: NSF Workshop Report on Creativity Support Tools. Tech. rep., Washington, DC, USA (2005)

19. Stanciu, R., Oh, P.Y.: Feedforward-Output Tracking Regulation Control for Human-in-the-Loop Camera Systems. In: Proceedings of the 2005, American Control Conference, 2005. ACC'05, vol. 5, pp. 3676–3681. IEEE (2005)

20. Tidwell, J.: Designing Interfaces. O'Reilly and Associates, 2 edn. (Jan 2011)