<figure><img src="image_1.png" /></figure>

HAL
open science

# Flow Specification Patterns of End-User Programmers:
Lessons Learnt from a Health Mobile Application
Authoring Environment Design

Filipe Fernandes, Luís Duarte, Luís Carriço

► To cite this version:

Filipe Fernandes, Luís Duarte, Luís Carriço. Flow Specification Patterns of End-User Programmers: Lessons Learnt from a Health Mobile Application Authoring Environment Design. 14th International Conference on Human-Computer Interaction (INTERACT), Sep 2013, Cape Town, South Africa. pp.748-755, 10.1007/978-3-642-40480-1_52. hal-01501786

HAL Id: hal-01501786

https://inria.hal.science/hal-01501786

Submitted on 4 Apr 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Flow Specification Patterns of End-User Programmers:
lessons learnt from a health mobile application authoring
environment design

Filipe Fernandes, Luís Duarte and Luís Carriço

LaSIGE & Department of Informatics, University of Lisbon
Edificio C6, Campo-Grande, 1749-016 Lisboa, Portugal

{ffernandes, lduarte}@lasige.di.fc.ul.pt, lmc@di.fc.ul.pt

**Abstract.** This paper discusses a set of interaction patterns encountered during the development of an authoring tool for mobile therapeutic applications. Unlike static paper artefacts, mobile applications can be enriched via the inclusion of complex behaviors. Typical examples include the definition of simple sequential interaction among all screens or the involvement of basic rules and triggers. As part of an ongoing project in which we are designing an authoring environment for mobile applications in clinical interventions, we studied how clinicians with no programming background were able to intertwine different screens from an application according to different rules. We were especially interested in comparing the approaches adopted using a low-fidelity prototype and using a high-fidelity version of the authoring tool. Results show that, despite a few technology induced strategies, users tend to mimic their actions using the paper based prototype in the corresponding hi-fi version.

**Keywords:** Authoring Tool, Non-Expert Programming, Participatory Design.

## 1 Introduction

Clinical applications for mobile phones have been shown to improve a patient's state [5][7][14][16][18]. The increasing dissemination and power of smart-phones has further emphasized this possibility [4][13]. Success cases for pathologies and therapy procedures as diverse as autism [2], fear therapy [7], aphasia [13] or obsessive-compulsive disorder [10] are testaments on the benefits of technology.

However, a significant number of these applications fall short to success for longer intervention periods [19]. Several factors can account for this outcome, among which the inability to personalize and adapt content [17]. For instance, an application's presentation is typically the same for all users who download it. Yet, the expectations of a potential 8 year old user are quite different from those of a 45 year old patient [7]. Also, the evolution of the patient's health status often requires adjustments that appli-
cations are not ready to accompany. For example, monitoring thresholds vary, support messaging and data collection should be adapted to new clinical assessments [16][17].

The origin of this application stiffness builds on many factors. The complexity of the technology and of the application domain is certainly one of those reasons. In fact we believe that it is one of the most important factors: the dichotomy and complexity of knowledge involved. Information technology (IT) engineers and researchers understand technology and are able to handle its complexity. Clinicians on the other hand comprehend patients and the protocols they must put forward to provide them a better quality of life. Combining the two knowledge sources is no easy task. It gets worse because both knowledge domains evolve rapidly as well as the ultimate target, the patient wellbeing.

The usage of authoring tools is a possible solution. Past works prove valuable evidence in tackling similar situations [2]. These tools aim at joining two knowledge sources to a middle ground. They provide domain experts the mechanisms to customize and deeply adapt the applications, refining and embedding it with clinical decisions. For that, the tools must hide the technology complexity under a hopefully well-defined set of components, developed by IT staff. In the end they offer means for end-user programming [20]. Finding that adequate middle-ground can be complicated. It is not just about usability. It is also about programming and domain concepts, and ultimately the domain experts' perception of its combination. Available systems often recur to state charts or similar representations of the programmable elements [9]. Unfortunately and despite the existence of a few systems in this area, some design aspects were left unaddressed by researchers and IT experts alike. Our research focuses on one of these issues: the way information is organized as far as connections between programmable elements is concerned. We believe that sequential transitions between screens of a mobile application may be sufficient for some domains of intervention (e.g. mobile app prototyping [4][11][12]). Nevertheless, other domains possess a more critical nature which requires richer and more complex connections.

In this paper we present the design process of DETACH – DEsign Tool for smartphone Application Composition in the Health domain. The tool targets therapists and clinicians in general, and aims to provide them support to compose ubiquitous applications. For that, it offers a set of building blocks, as predefined, yet customizable, mobile app screens (e.g. a message panel, a mood selection panel). These can be assembled and customized to address the particular patients’ needs. As such, domain experts are the tool’s end-users and thus responsible to define the applications according to the adequate clinical procedures and evolution. Particularly relevant was the tool development process. We aimed at understanding what concepts and behaviors a domain expert formulates and expresses to grasp the programming endeavor itself. We were particularly focused in assessing how these end-users connect and define the control of flow between these building blocks, as this emerged as the most difficult task to assimilate. The assessment started in the early design stages and the understanding of users’ behavior in low-fidelity prototyping, and transitioned into how that behavior was translated into the high-fidelity (hi-fi), and its evaluation with end-users.
## 2 Sketching DETACH: Participatory Design of Low-Fidelity Prototypes

The DETACH's development process started with the creation of specific mobile applications, addressing cognitive behavior therapy procedures. Currently several of those applications are being used in clinical trials and for all of them therapists provide continuous screening and comments [5]. Several brainstorming sessions were conducted including a team of expert HCI researchers and the therapists specializing in different types of interventions and pathologies.

DETACH's development process was heavily rooted in participatory design [1][13] and thinking aloud sessions not only with therapists but clinicians in general (Fig. 1). We have conducted a substantial set of meetings to understand: a) their current requirements and expectations for a tool of this nature; b) how modern smart-phones can improve existing therapeutic practices without disrupting established procedures; c) interaction patterns employed by our stakeholders when organizing different elements of an application.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Application representation made by one of the participatory design users.</figcaption></figure>

Design sessions lasted for approximately 45 minutes per subject, with each subject being accompanied by an expert UI researcher. A total of 15 users (aged M = 41; SD = 7.3) had volunteered themselves to participate in these trials.

For each trial, participants were asked to design an application to support a child undergoing a fear therapy [7] treatment and containing 4 screens: screen 1 allowed the child to select his / her current emotional state from a set of pictorial representations (e.g. happy face, frowny face, etc.); screen 2 presented a question (with “yes” / “no” answers) asking whether the child was still feeling scared or not; screen 3 was presented if the child answered “no” in screen 2 and displayed a descriptive support text regarding typical procedures for fear therapy; screen 4 was a congratulatory animation which was displayed either after screen 3 or if the child responded “yes” in screen 2. Subjects were assisted as required if they were insecure about screen details and other technical details. They were given full freedom to organize the screens.

Each session started with the researcher explaining the purpose of the authoring tool and its main features. Clinicians were then briefed regarding the mobile application they would be creating during the session. Finally they were provided with all the necessary material to create the screens: pen, pencil, eraser, different colored post-its and white sheets of paper (Fig. 1). In order for us to study how clinicians would deal
with the provided material, some of the sessions had available additional template screen mockups and/or a guidance text of a specific application for them to represent.

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Examples of screen transition representations on low-fi prototypes.</figcaption></figure>

## 2.1 Results and Discussion

Upon completing the creation of the mobile application using the low-fi DETACH prototype, subjects were asked to explain us the rationale behind their strategy to organize and connect the screens. Some notable approaches emerged from this trial:

- Subjects position each screen at their will on the sheet. Transitions between each screen are represented by arrows, complemented with side notes either using plain text or a small post-it above the arrow line (Fig. 2– right).

- Some participants preferred to organize screens on a sequential fashion, touching each other to pinpoint the transitions. Special connections such as the one stemming from screen 2 to screen 3 were addressed by separating one of the screens slightly from the main group and annotating the rules associated with that transition (Fig. 2 – left).

- New screens emerged. Interestingly some of them addressing physiologic sensing. More importantly a new type of transition and condition was created by some (more active) clinicians, featuring an interruption of the normal flow. The example in the figure (Fig. 2– right, lower left corner) refers to: whenever heart beat raises above something show this screen.

# 3 Testing the High-Fidelity Prototype

Based on stakeholder feedback, we designed the first DETACH’s high-fidelity prototype (Fig. 3). The tool is a web application offering a workspace on which therapists can populate screens based on different templates and connect them according to a set of rules whose complexity may vary (e.g. from basic screen sequence to transitions based on previous patient answers). The top section displays the available screen templates (e.g. multiple choice answer, animation display, etc.). Therapists can drag a
template into the center canvas and configure each screen's particular elements in the rightmost panel.

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Resulting representation of one of the thinking aloud sessions for the initial application.</figcaption></figure>

Transitions between screens (represented by arrows) are defined by selecting a screen, hitting a button and finally stipulating the rules which trigger the transition (e.g. if the transition considers the selected element as the destination or its source, how is the transition triggered).

**3.1 High-Fidelity Prototype Trials**

The high-fidelity prototype participatory design sessions occurred roughly 4 months after the low-fi ones. This trial was performed by the same clinicians who had been present in the previous experimental period. To provide the most pleasant and relaxed experience to our subjects, testing occurred again in the clinicians' offices. The high-fidelity prototype trials were carried out in a laptop (HP Pavilion dv9890ep model).

Regarding screen organization on the prototype's canvas, subjects adhered to two main strategies: a) akin to the low-fi trial, users positioned the screens sequentially, typically reflecting creation order; b) users delineated imaginary columns upon which screens were positioned – if a screen generates two navigational branches (i.e. users can transition towards two different screens from the same origin) then all destination screens would fit inside the same virtual column in the canvas, with the branches collapsing into another screen if that was the case. A substantial number of participants did not show any particular spatial strategy here (as observed in **Fig. 3**).

**3.2 Results and Discussion**

We were able to identify a substantial number of strategies employed by our participants regarding the organization of the elements they were able to interact with. While there was a slight degree of variation in the presented approaches, we were able to cluster them into well-defined interaction patterns.
**Touch to Connect.** One of the approaches observed rooted itself in the low-fi version of the prototype. Clinicians (approximately 25%) often aggregated screens which had a sequential transition nature together in the canvas. A particular behavior was noted: subjects often organized the screen post-it in a way resembling a deck of cards, with each screen slightly touching each other it had a connection with. Transition rules were either described using side textual notes or by adding smaller post-its covering the screens involved. On the hi-fi prototype, these same users attempted to link screens using a different strategy: they dragged one screen towards another, "touching" it. The expectation was that a new transition was established between the "touched" screen and the dragged one. This strategy clearly shows a sequence oriented thinking towards building mobile applications. When confronted with the possibility of adding additional rules for these transitions (e.g. based on patient inserted content/answers) they argued that this still felt like the most natural way to interact with the elements in the canvas. They also reiterated the environment should leave the possibility of editing the transitions to add behaviors on top of the basic sequential connections (a feature already present, but not tested by these subjects).

**Origin-Destination Paradigm.** The most popular strategy adopted by users was inspired by the way they typically fill-in a postcard, an e-mail or a letter: they define the origin of the connection, the destination and then any related content with them. Approximately 55% of our subjects adopted this approach arguing "this is the way I naturally write" and "the way I did on the paper version". One may ask if using the same subjects and the previous experience with the low-fi prototype could influence this result: in part, we agree, but we must also note that a substantial number of clinicians did not follow the same connection strategies; also the timespan between both trials dissipates some of the "training" acquired in the first trial.

A minority of the clinicians (roughly 10%) approached this paradigm by completely switching the connection's order definition: they started by selecting the destination screen and then they picked the screens which would transit to the former. When asked to verbalize why they adopted this strategy, they argued "it made sense, considering a patient can reach the same screen from different branches", so "defining the destination first felt straightforward". Here, we must state such decision may have been influenced in part by the mobile application they were asked to create, since it featured a screen which could be reached from two different navigational branches. However, this behavior was only noted in the hi-fidelity prototype.

**Connection from Screen Elements.** The last adopted strategy pertains only to the hi-fidelity prototype participatory design sessions. When selecting and configuring each screen, some clinicians attempted to generate connections from the screen's components themselves (e.g. each answer, a button, etc.), justifying their behavior stating "the patient will transition to another screen if he/she presses this button". Even though no participant had previous programming experience, this is an approach reminiscent of existing Integrated Development Environments (IDE) such as Visual Studio or Eclipse and highly connected with event-based programming. In these tools, users may click a component, such as a button, to configure the application's behavior
when the button is pressed. It is interesting that despite the absence of experience, some participants actually prefer this strategy.

**Implications for DETACH.** The main findings from these design sessions pertain to the variety of approaches clinicians were able to adopt to accomplish the same goal. Nevertheless we must prioritize the strategies which gathered more followers while at the same time not neglecting the preferences manifested by some of our participants. As such, DETACH's primary approach towards the definition of screen connections will follow the origin-destination paradigm. All other strategies will be incorporated into the environment as user preferences. Ideally, the application should be configurable for each user, storing their preferences and preferred workflow strategies.

## 4 Conclusions & Future Work

Our research points that non-expert programmers embrace desktop and paper metaphors in a virtual environment. We observed that our subjects employed similar screen organization strategies in the authoring environment regardless of operating a low-fi or hi-fi prototype during the participatory design sessions. It is important to stress that DETACH's final design will reflect these findings, to alleviate the technology transition impact which our stakeholders will be subject to.

We are finishing the development of DETACH, currently focusing on the mobile application emulator and the XML specification that will be used in Android smartphones to recreate the therapists' designs. The final step of this research will encompass a set of clinical trials in which we will assess whether authored applications can foster patient commitment when compared to previous non-authored digital artifacts.

## References

1. Bergvall-Kåreborn, B. and Ståhlbrost, A. 2008. Participatory design: one step back or two steps forward? In Procs of PDC '08. Indiana University, Indianapolis, IN, USA, 102-111.

2. Boujarwah, F. A., et al. 2011. REACT: intelligent authoring of social skills instructional modules for adolescents with high-functioning Autism. SIGACCESS Access. Comput. 99 (January 2011), 13-23.

3. Boyd-Graber, J., et al, 2006. Participatory design with proxies: developing a desktop-PDA system to support people with aphasia. In Procs. of CHI '06, p. 151-160. ACM.

4. Carriço, L., et al, 2012. Therapy: location-aware assessment and tasks. In Procs of AH '12, Article 2, 5 pages. ACM Press.

5. Carriço, L., et al, 2008. A Mixed-Fidelity Prototyping Tool for Mobile Devices. In Procs of AVI '08. ACM, New York, NY, USA, 225-232.

6. Das, A. K., 2002. Computers in Psychiatry: A Review of Past Programs and an Analysis of Historical Trends. In Psychiatry Quarterly, No.79 (4).

7. de Sá, M. and Carriço, L., 2012. Fear therapy for children: a mobile approach. In Procs of EICS '12, pp. 237-246. ACM.

8. Grasso, M. A., 2004. Clinical Applications of Hand Held Computing. 17th IEEE Symposium on Computer Based Medical Systems, pp. 141-146.
9. Hartmann, B., et al. d.tools: Visually Prototyping Physical UIs through Statecharts. In Ext. Abstracts of ACM Symposium on User Interface Software and Technology (UIST 2005), 2005.

10. Herman, S. and Koran, L. (1998). In vivo measurement of obsessive-compulsive disorder symptoms using palmtop computers. *Computers in Human Behaviour* 14(3).

11. Klemmer, S. R., et al, SUEDE: A Wizard of Oz Prototyping Tool for Speech User Interfaces. In Proc. of UIST’00, San Diego, California, USA, pp. 1-10, ACM

12. Landay, J., 1996. SILK: Sketching Interfaces Like Krazy. In Procs. of CHI’96, pp.398-399, ACM.

13. Matthews, M., et al, 2011. In the mood: engaging teenagers in psychotherapy using mobile phones. In procs. of CHI’11, p.2947-56. ACM.

14. Mattila, E., et al. 2008. Mobile Diary for Wellness Management - Results on Usage and Usability in Two User Studies. IEEE Transactions on Information Technology in Biomedicine, 12(4): p. 501-512. IEEE.

15. Moffatt, K., et al., 2004. The Participatory Design of a Sound and Image Enhanced Daily Planner for People with Aphasia. In Procs of CHI '04. ACM.

16. Morris, M., et al, 2010. Mobile Therapy: Case Study Evaluations of a Cell Phone Application for Emotional Self-Awareness. Journal of Internet Medical Research, 12(2).

17. Offer, D., et al, 1991. "To whom do adolescents turn for help? Differences between disturbed and non-disturbed adolescents." Journal of the American Academy of Child and Adolescent Psychiatry, 30(4) p. pp. 623-630.

18. Przeworski, A. and Newman, M. G., 2004. Palmtop computer-assisted group therapy for social phobia. Journal of Clinical Psychology, 60(2):179-188.

19. Robinson, S., et al. 2006. Aftercare intervention through text messaging in the treatment of bulimia nervosa - Feasibility pilot. *Intl Journal of Eating Disorders, 39*(8): p. 633-638.

20. Lieberman, H., et al. (eds), 2006. End-User Development, Springer Verlag, ISBN-10 1-4020-4220-5.

21. Timms, P. et al, 2007. "Cognitive Behavioral Therapy", In leaflet for the Royal College of Psychiatrists' Public Education Editorial Board, March, 2007.