# Evaluation of Informative Content of Health Data
Submitted Through a Mobile Serious Game

Konrad Peters, Stephanie Bührer, Marisa Silbernagl, Fares Kayali, Helmut Hlavacs, Anita Lawitschka

<figure><img src="image_1.png" /></figure>

## To cite this version:

Konrad Peters, Stephanie Bührer, Marisa Silbernagl, Fares Kayali, Helmut Hlavacs, et al. Evaluation of Informative Content of Health Data Submitted Through a Mobile Serious Game. 1st Joint International Conference on Entertainment Computing and Serious Games (ICEC-JCSG), Nov 2019, Arequipa, Peru. pp.366-376, 10.1007/978-3-030-34644-7_30. hal-03652014

HAL Id: hal-03652014

https://inria.hal.science/hal-03652014

Submitted on 26 Apr 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Evaluation of informative content of health data submitted through a mobile serious game

Konrad Peters¹, Stephanie Bührer², Marisa Silbernagl¹, Fares Kayali³, Helmut Hlavacs¹, and Anita Lawitschka²

¹ University of Vienna, Research Group Entertainment Computing
https://ec.cs.univie.ac.at/
{konrad.peters, marisa.silbernagl, helmut.hlavacs}@univie.ac.at

² St. Anna Children's Hospital, Vienna
{stephanie.buehrer, anita.lawitschka}@stanna.at

³ Vienna University of Technology
fares@igw.tuwien.ac.at

**Abstract.** In the presented study, the informative content of health data of a handwritten health diary was compared with health data submitted through a serious game. Physicians are able to derive a better and more complete health status of patients with higher informative content in health reports. The serious game was implemented in the project Interacct, in which patients submit health data through a mobile app on a daily basis to receive in-game rewards. The main hypothesis is, that the informative content of health data, which was submitted through a serious game, is higher than the informative content of hand written health diaries. Statistical results confirm this hypothesis significantly. This is especially important for young and adolescent cancer patients, where the results were even more conclusive than in the control group of students. Considering positive side effects of the serious game (such as high engagement, compliance and pleasure reporting health data through a serious game), the proposition of the authors is to conduct further studies and experiments regarding this topic.

**Keywords:** serious games, serious games for health, health reports, informative content, cancer outpatient treatment, m-health

## 1 Introduction

There are various ways of remotely recording health-related data, facilitated by the development of new technologies [30]. However, the usage of these technological applications are not always backed by scientific evaluations and thus the safety of users can be at risk [20]. This is especially true if the users are members of a vulnerable group, such as chronically ill cancer patients. Further, it is hard to quantify the benefits of new technologies, especially when compared to analog methods of gathering data (e.g. through handwritten health diaries).

The project Interacct has the aim to foster communication between young leukaemia patients and their treating physicians using a mobile serious game. The
project is conducted by the St. Anna Childrens Hospital Vienna<fnref n="4" />, in cooperation
with the University of Vienna (Research Group Entertainment Computing)<fnref n="5" /> the
University of Applied Arts Vienna<fnref n="6" /> and T-Systems Austria<fnref n="7" />. Interactc includes
a serious game for patients with a health reporting function, as well as a web-
interface for physicians to evaluate and observe the submitted data.

The target group of project Interactc consists of children and adolescents who have been treated by an allogeneic hematopoietic stem cell transplantation (HSCT). This method is used to cure diseases like red-cell disorder or leukemia. After treatment, patients struggle with a variety of symptoms, risks, and sequelae [3][26]. Although many symptoms occur independently of the cancer type and the treatment method [6][31], no screening tool exists that can be used to query symptoms in a standardized way [8].

At the St. Anna Children's Hospital, health monitoring of HSCT patients in childhood and adolescence is currently based on a paper diary. In this paper diary, the young patients document health-related changes on a daily basis. Once they visit the clinic, they hand over the diary to their aftercare physician. This procedure is accompanied by two major disadvantages. First, the adherence of patients regarding paper diaries is low [19][27]. Second, the diaries are often filled later and not regularly, which is partly caused by the low adherence [16][25]. These disadvantages may be prevented by real-time electronic symptom checking, for example via smartphone app [7][18]. If information about symptoms is transmitted daily and directly to the physician, negative changes in health can be detected quickly without visiting the hospital. Therefore, patients can receive an immediate response after symptoms have been reported and can seek medical care or initiate coping strategies [14]. Due to the timestamps of electronic data, it is furthermore easy to arrange and evaluate data according to the date of entries [25]. Another advantage of mobile applications is that children and adolescents feel more comfortable with the electronic exchange of sensitive information [2][15][1].

Although a lot of health-related software is available, a review published in 2015 revealed only four pilot-tested smartphone applications for young cancer patients [28]. Often medical professionals are not included during the development of health-related software [14], but these four apps are theoretically based and evaluated scientifically. Currently, no study exists that has developed as well as evaluated such an app and compared its content directly with the traditional paper method. It is also unclear, whether a meaningful health profile can be conveyed via electronic data transmission without seeing the patient personally, due to the rare involvement of physicians in the development of those apps.

Therefore, this study aimed to develop and test a serious game for HSCT
patients that provides information about the health condition of treated chil-
dren and adolescents. Physicians evaluated the information content delivered

<footnote n="4">⁴ https://www.stanna.at/, accessed 21.06.2019</footnote>

<footnote n="5">⁵ https://ec.cs.univie.ac.at, accessed 21.06.2019</footnote>

<footnote n="6">⁶ https://www.dieangewandte.at, accessed 21.06.2019</footnote>

<footnote n="7">⁷ https://www.t-systems.com/de/at, accessed. 21.06.2019</footnote>
through the serious game and compared it with the information obtained from
the paper diary. The information content of the serious game was expected to
be significantly higher than the one derived from the diary.

1.1 Related Work

Hochstenbach et al. tested the feasibility of a mobile and web-based self-management tool for outpatients with cancer pain. Patients (n=11) and nurses (n=3) used the tool for pain monitoring, medication monitoring, and educational sessions. Results show, that patients and nurses were positive about using the tool. The authors conclude, that the system demonstrates feasibility in everyday practice [11]. A similar system is *PainBuddy*, which is discussed by Fortier et al.: young cancer patients keep track of their pain during cancer treatment using tablet computers. The provided app uses an animated avatar and gamification components, as well as remote symptom monitoring. A pilot study has shown, that patients (n=12) were highly satisfied [10]. In *PainSquad*, adolescent cancer patients keep track of their pain during therapy through an iPhone app. The app was found to be appealing to the patients, which resulted in high compliance rates [24]. A more specific approach was investigated by Rodgers et al. by implementing *EAT*!, which should assist adolescents with self-management of eating-related issues during HSCT recovery. While patients (n=16) initially embraced the app, the use decreased extensively over time. Authors suggest further development and studies [22]. Baggott et al. implemented an electronic diary for adolescent cancer patients. In a 3-week trial, patients (n=10) showed high adherence (≥ 90%) [2]. Wesley and Fizur reviewed several studies dealing with cancer treatment with app support. They conclude positively but state the demand for more empirical data and further research effort [29].

Outside of the field of cancer patient aftercare, Charlier et al. conducted a
meta-analysis on the effect of serious games for improving knowledge and self-
management in young people with chronic conditions. The authors conclude,
that using serious games can significantly improve self-management as well as
knowledge transfer [4]. Fishbein et al. give suggestions for the design and de-
velopment of mobile applications to promote oral drug adherence and symptom
management during chemotherapy [9]. Klaassen et al. have evaluated a coaching
and gamification platform for the self-management of young diabetes patients.
They give concise suggestions for the development of coaching and gamification
platforms in medical practice [13]. Price et al. developed *Painpad*, a tangible
device to self-log pain, and have shown increases in both frequency and compli-
ance with pain logging [21]. They further note that data self-reported this way
is more faithful than data reported to nurses.

2 Methods

2.1 Study Design & Statistical Methods

Patients and students between ages 7-18 participated in this study to compare
the Interact serious game app with a paper health diary. All participants were
asked to use the diary as well as the serious game for 5 days (Monday-Friday)
and to document given body function parameters (e.g.: tiredness, appetite, pain,
etc.) as conscientiously as possible. Only participants who used both, paper diary
and app, for at least one day were included. Due to this criterion, 10 students
were excluded prior to the analyzes and the final sample consisted of 15 patients
and 27 students. Due to organizational reasons, a completely balanced design
(crossover between starting with the diary or the app) could not be realized and
the diary was always used before the app.

The information content of the diary entries and the app data was checked by
two independent, blinded physicians. The physicians used a scale ranging from
1 to 5 points to evaluate the information. One point reflects low information,
whereas 5 points indicate a high information content. The interrater-reliability
was evaluated by calculating the intraclass correlation coefficient (ICC), which
was based on a two-way mixed model and the absolute agreement. Subsequently,
a one-way repeated-measures ANOVA was used to check whether the information
content of the app differs from the diary. Also, the between-subject-factor *group*
was included to examine any differences between patients and students. T-tests
for associated samples were conducted post-hoc to calculate contrasts between
the groups.

After participation, the children were asked to answer a usability question-
naire regarding general user satisfaction as well as the suitability of the app
design. User satisfaction was measured on a 5-point Likert scale, where 1 rep-
resented low values (not at all) and 5 high values (extraordinary). To evaluate
the app design, participants could award 3 points for each app-function, with 3
points being the most positive rating. The median and the interquartile range
(IQR) were calculated for each item. Additionally, participants had the oppor-
tunity to write down feedback to reveal additional ideas for the improvement of
the app.

## 2.2 Technical Components

The technical components used in the study were a *Unity 3D*<fnref n="8" /> app for the pa-
tients and an *ASP.net*<fnref n="9" /> web-interface for the physicians. The persistence layer
was implemented through the *ASP.net*-backend, accessing a *Microsoft SQL*
database, running in a secure *Windows Server 2016* environment. The connec-
tion between clients and server was established using *Secure Socket Layer - SSL*.
Basic authentication and cross-site request forgery (CSRF) tokens were used to
ensure data integrity and security.

## 2.3 Interact Client

The Interact client is a native smartphone application for Android and iOS,
developed in Unity 3D. It was used as the main tool for the presented study.

<footnote n="8">⁸ https://unity3d.com/, accessed 21.06.2019</footnote>

<footnote n="9">⁹ https://dotnet.microsoft.com/apps/aspnet/, accessed 21.06.2019</footnote>
The main components of the client app are remote medical data entry as well as the game content itself. Completing the medical data entry survey, users were rewarded with virtual in-game currency, used to progress in the main game. Registration of new users was only possible through the administrative team of the project.

The following subchapters will describe the main components of project Interact. A detailed project description, as well as design considerations, are covered by Kayali et. al in [12].

**Game Design** The core game idea of Interact is to collect * avatars* and complete procedurally generated levels. The avatar explores an island and needs to fight off hostile NPC (*non-player characters*) monsters to complete a level. Fights with the NPC monsters are easier for the player, once the avatar has reached a certain level or skill set. To upgrade an avatar, the player can spend *science points*, which are earned through the completion of health reports. At the end of each island, a boss monster is encountered and has a chance of dropping an *egg shell*, which eventually allowed the player to hatch a new avatar. The levels come in different graphical settings and with different NPC monsters to provide variety.

**Health Reports / Remote Data Entry** Within the Interact serious game, users can to report a set of health parameters. The parameters mostly follow a 0-3 scale and were adapted to match LOINC<fnref n="10" /> codes, where possible. The parameters are listed and explained in Table 1. The health report data could be submitted several times per day by each user and was accumulated by the backend for each full day. For completing the report for the first time each day, the user was rewarded with *science points*, a virtual in-game currency, which could be used to extend the capabilities of the user’s avatar. Fig. 1 shows the health report UI with all health parameter categories. Fig. 2 shows the input for fluid intake amount and if the patient felt any pain during fluid intake.

## 3 Results

A high degree of interrater-reliability was found between the physicians. The average measure ICC was .867 (95% confidence interval [CI] from .733 to .932, F(41,41) = 8.530, p ≤ .001) for the paper diary and .912 (95% CI from .794 to .958, F(41,41) = 14.001, p ≤ .001) for the app. Over the entire sample, the physicians gave an average of 3.77 (± .91) points for the information content of the diary and 4.14 (± 1.14) points for the app. The repeated measures ANOVA revealed a significant difference between information content of the diary compared to the app (F(1, 40) = 5.571, p = .023, η = .12). This means that the app

<footnote n="10">¹⁰ https://loinc.org/, accessed 21.06.2019</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 1: Interactt serious game client with the health report categories</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 2: Interactt serious game client showing input for fluid intake amount and fluid intake pain</figcaption></figure>
<table><caption>Table 1: Health parameters and their meaning, used by the Interact serious game / health report tool</caption><thead><tr><td>Health Parameter</td><td>Meaning</td></tr></thead><tbody><tr><td>Fluid intake amount</td><td>Amount of fluid intake</td></tr><tr><td>Fluid intake pain</td><td>Pain during drinking</td></tr><tr><td>Food intake amount</td><td>Amount of food intake</td></tr><tr><td>Appetite</td><td>Appetite before food intake</td></tr><tr><td>Stool consistency</td><td>Consistency of stool (soft/normal/hard)</td></tr><tr><td>Stool frequency</td><td>Frequency of defecation</td></tr><tr><td>Stool pain</td><td>Pain during defecation</td></tr><tr><td>Playtime duration</td><td>Amount of playing in minutes</td></tr><tr><td>Walking duration</td><td>Amount of walking in minutes</td></tr><tr><td>Physical exercise duration</td><td>Amount of physical exercise in minutes</td></tr><tr><td>Urine frequency</td><td>Frequency of urination</td></tr><tr><td>Nausea</td><td>Level of nausea</td></tr><tr><td>Vomiting</td><td>Amount of vomiting</td></tr><tr><td>Anxiety</td><td>Level of anxiety</td></tr><tr><td>Rage</td><td>Level of rage</td></tr><tr><td>Fear</td><td>Level of fear</td></tr><tr><td>Tiredness</td><td>Level of tiredness</td></tr><tr><td>Mouth pain</td><td>Level of mouth pain</td></tr><tr><td>Body temperature</td><td>Body temperature in °C</td></tr><tr><td>Localized skin pain</td><td>Level of skin pain + localization</td></tr></tbody></table>
provided significantly more information than the diary. The between subject factor group was not significant (F(1, 40) = .522, p = .474) and also no significant interaction effect was found (F(1, 40) = 1.807, p = .186).

Although there were no group-differences shown in the ANOVA, post-hoc t-tests revealed that the significant main effect was primarily driven by the patients (Fig. 3). Patients received an average of 3.73 (± .63) points for the paper diary and 4.43 (± 1.07) points for the app. This difference was significant (t(1,14) = -2.941, p = .011), while there was no significant difference in information content for students (t(1,26) = -.772, p = .447). The information content of their diaries was rated with 3.78 (± 1.05) points and the app with 3.98 (1.17) points. The information content of the students diary and app was very similar to the information given by the patients in the diary. These results indicate that only the patients, who were probably the more intrinsically motivated group, documented the body function parameters conscientiously. It is quite possible that the students revealed rather superficial information.

This assumption is supported by considering the number of days participants used the diary and the app. Patients used the paper diary for an average of 5.67 (± 1.23) days, which was more than the required 5 days. The app was used by this group for 4.67 (± 2.53) days, but the difference to the diary was not significant (t(1,14) = 1.479, p = .161). The students used the diary as long as patients (5.89 ± 1.31 days), but there was a significant difference in the use of the app (t(1,26) = 6.138, p ≤ .001), which they used only 3.41 (± 1.69) days. As mentioned in the introduction, it is easy to fill the diary afterward, which is impossible with the app. Therefore, the probability is quite high that especially the students did not keep the diary up to date.

Furthermore, user satisfaction, as well as the evaluation of the app-design, were positive (Table 2). There was no evidence that the diary was preferred to the app.

<table><caption>Table 2: User satisfaction and evaluation of the app design</caption><thead><tr><th>User satisfaction</th><th>Median score (IQR)<br/>[1 = not at all; 5 = extraordinary]</th></tr></thead><tbody><tr><td>I was bored</td><td>3.00 (IQR: 3.00)</td></tr><tr><td>I was impressed</td><td>3.00 (IQR: 3.00)</td></tr><tr><td>I felt frustrated</td><td>1.00 (IQR: 1.00)</td></tr><tr><td>I found it tiring</td><td>2.00 (IQR: 2.00)</td></tr><tr><td>I was irritated</td><td>1.00 (IQR: 1.00)</td></tr><tr><td>I felt skillful</td><td>3.00 (IQR: 3.00)</td></tr><tr><td>I was satisfied</td><td>4.00 (IQR: 4.00)</td></tr><tr><td>I felt challenged</td><td>2.00 (IQR: 2.00)</td></tr><tr><td>I had to put a lot of effort into playing</td><td>1.00 (IQR: 1.00)</td></tr><tr><td>I felt good</td><td>4.00 (IQR: 4.00)</td></tr></tbody></table>
<figure><img src="image_5.png" /><figcaption>Fig. 3: Main result: awarded points for paper diary and app compared between patients and students (error bars: 95% CI)</figcaption></figure>
# 4 Discussion

## 4.1 Main findings

High usability of an app is not sufficient to motivate patients to enter data in the long term [23]. In addition to usability, illness experience, information technology infrastructure, emotional activation, degree of burden caused by the app and relevance of symptom monitoring are important factors for patient motivation [5]. Especially children and adolescents who are not interested in health-related symptom tracking and changes, such as the students participated in this study, are quickly demotivated if they must enter a large amount of data every day [17]. The fewer data users are asked to enter, the greater is the level of adherence in general [18].

# 5 Conclusion

In the presented study, the informative content of health data of a handwritten health diary was compared with health data submitted through a serious game. The serious game was implemented in the project Interactc, in which patients can submit health data through a mobile app on a daily basis to receive in-game rewards. Patients and students submitted 2 weeks of consecutive health reports, first with the Interactc serious game, eventually with the paper diary. Two independent, blinded physicians reviewed the health reports and rated the informative content.

The main hypothesis is, that the informative content of health data, which was submitted through a serious game, is higher than the informative content of handwritten health diaries. The statistical results confirm this hypothesis significantly. This is especially important for young and adolescent cancer patients, where the results were even more conclusive than in the control group of students.

Considering the positive side effects of the serious game (such as high engagement, compliance, and pleasure reporting health data through a serious game), the authors propose to conduct further studies and experiments regarding this topic.

# References

1. Aiello, E.J., Taplin, S., Reid, R., Hobbs, M., Seger, D., Kamel, H., Ballard-Barbash, R.: In a randomized controlled trial, patients preferred electronic data collection of breast cancer risk-factor information in a mammography setting. Journal of Clinical Epidemiology 59, 77–81 (2006)

2. Baggott, C., Gibson, F., Coll, B., Kletter, R., Zeltzer, P., Miaskowski, C.: Initial evaluation of an electronic symptom diary for adolescents with cancer. Journal of Medical Internet Research (2012). https://doi.org/10.2196/resprot.2175
3. Bhatia, S., Davies, S.M., Baker, S., Pulsipher, M.A., Hansen, J.A.: NCI, NHLBI first international consensus conference on late effects after pediatric hematopoietic cell transplantation: Etiology and pathogenesis of late effects after HCT performed in childhood methodologic challenges. Biology of Blood and Marrow Transplantation **17**, 1428–1435 (2011)

4. Charlier, N., Zupancic, N., Fieuws, S., Denhaerynck, K., Zaman, B., Moons, P.: Serious games for improving knowledge and self-management in young people with chronic conditions: A systematic review and meta-analysis. Journal of the American Medical Informatics Association (2016). https://doi.org/10.1093/jamia/ocv100

5. Cohen, D.J., Keller, S.R., Hayes, G.R., Dorr, D.A., Ash, J.S., Sittig, D.F.: Developing a model for understanding patient collection of observations of daily living: A qualitative meta-synthesis of the Project HealthDesign Program. Personal and Ubiquitous Computing **19**, 91–102 (2015)

6. Collins, J.J., Byrnes, M.E., Dunkel, I.J., Lapin, J., Nadel, T., Thaler, H.T., Portenoy, R.K.: The measurement of symptoms in children with cancer. Journal of Pain and Symptom Management **19**, 363–377 (2000)

7. Dale, O., Hagen, K.B.: Despite technical problems personal digital assistants outperform pen and paper when collecting patient diary data. Journal of Clinical Epidemiology **60**, 8–17 (2007)

8. Dupuis, L.L., Ethier, M.C., Tomlinson, D., Hesser, T., Sung, L.: A systematic review of symptom assessment scales in children with cancer. BioMed Central Cancer **12**(430) (2012)

9. Fishbein, J.N., Nisotel, L.E., MacDonald, J.J., Amoyal Pensak, N., Jacobs, J.M., Flanagan, C., Jethwani, K., Greer, J.A.: Mobile Application to Promote Adherence to Oral Chemotherapy and Symptom Management: A Protocol for Design and Development. JMIR Research Protocols **6**(4), e62 (4 2017). https://doi.org/10.2196/resprot.6198, http://www.researchprotocols.org/2017/4/e62/

10. Fortier M.A., Chung, W.W., Martinez, A., Gago-Masague, S., Sender, L.: Pain buddy: A novel use of m-health in the management of children's cancer pain. Computers in Biology and Medicine **76** (2016). https://doi.org/10.1016/j.compbiomed.2016.07.012

11. Hochstenbach, L.M., Zwakhalen, S.M., Courtens, A.M., van Kleef, M., de Witte, L.P.: Feasibility of a mobile and web-based intervention to support self-management in outpatients with cancer pain. European Journal of Oncology Nursing **23**, 97–105 (2016). https://doi.org/10.1016/j.ejon.2016.03.009, http://dx.doi.org/10.1016/j.ejon.2016.03.009

12. Kayali, F., Silbernagl, M., Peters, K., Mateus-Berr, R., Reithofer, A., Martinek, D., Lawitschka, A., Hlavacs, H.: Design considerations for a serious game for children after hematopoietic stem cell transplantation. Entertainment Computing **15**, 57–73 (2016)

13. Klaassen, R., Bul, K.C., Op Den Akker, R., Van Der Burg, G.J., Kato, P.M., Di Bitonto, P.: Design and evaluation of a pervasive coaching and gamification platform for young diabetes patients. Sensors (Switzerland) **18**(2), 1–27 (2018). https://doi.org/10.3390/s18020402

14. Lalloo, C., Jibb, L.A., Rivera, J., Agarwal, A., Stinson, J.N.: Theres a pain app for that: Review of patient-targeted smartphone applications for pain management. Clinical Journal of Pain **31**, 557–563 (2015)
15. Lane, S.J., Heddle, N.M., Arnold, E., Walker, I.: A review of randomized controlled trials comparing the effectiveness of hand held computers with paper methods for data collection. In: BMC Medical Informatics and Decision Making, vol. 6, pp. 1–10. BioMed Central (2006)

16. Lauritsen, K., DeglInnocenti, A., Hendl, L., Praest, J., Lytje, M.F., Clemmensen-Rotne, K., Wiklund, I.: Symptom recording in a randomised clinical trial: Paper diaries vs. electronic or telephone data capture. Controlled Clinical Trials **25**, 585–597 (2004)

17. Marceau, L.D., Link, C., Jamison, R.N., Carolan, S.: Electronic diaries as a toll to improve pain management: Is there any evidence? *Pain Medicine* **8**, 101–109 (2007)

18. Morren, M., van Dulmen, S., Ouwerkerk, J., Bensing, J.: Compliance with momentary pain measurement using electronic diaris: A systematic review. *European Journal of Pain* **13**, 354–365 (2008)

19. Palermo, T.M., Valenzuela, D., Stork, P.: A randomized trial of electronic versus paper pain diaries in children: Impact on compliance, accuracy, and acceptability. *Pain* **107**, 213–219 (2004)

20. Pandey, A., Hasan, S., Dubey, D., Sarangi, S.: Smartphone apps as a source of cancer information: Changing trends in health information-seeking behavior. *Journal of Cancer Education* **28**, 138–142 (2013)

21. Price, B.A., Kelly, R., Mehta, V., McCormick, C., Ahmed, H., Pearce, O.: Feel My Pain: Design and Evaluation of Painpad, a Tangible Device for Supporting Inpatient Self-Logging of Pain. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. p. 169:1169:13. CHI ’18, ACM, New York, NY, USA (2018). https://doi.org/10.1145/3173574.3173743, http://doi.acm.org/10.1145/3173574.3173743

22. Rodgers, C.C., Krance, R., Street, R.L.J., Hockenberry, M.J.: Feasibility of a symptom management intervention for adolescents recovering from a hematopoietic stem cell transplant. *Cancer nursing* (2013). https://doi.org/10.1097/NCC.0b013e31829629b5

23. Scott, A.R., Alore, E.A., Naik, A.D., Berger, D.H., Suliburk, J.: Mixed-methods analysis of factors impacting use of a postoperative mHealth app. *JMIR mHealth and uHealth*, 2–5 (2017)

24. Stinson, J.N., Jibb, L.A., Nguyen, C., Nathan, P.C., Maloney, A.M., Dupuis, L.L., Gerstle, J.T., Alman, B., Hopyan, S., Strahlendorf, C., Portwine, C., Johnston, D.L., Orr, M.: Development and testing of a multidimensional iphone pain assessment application for adolescents with cancer. *Journal of Medical Internet Research* (2013). https://doi.org/10.2196/jmir.2350

25. Stone, A.A., Shiffman, S., Schwartz, J.E., Broderick, J.E., Hufford, M.R.: Patient compliance with paper and electronic diaries. *Controlled Clinical Trials* **24**, 182–199 (2003)

26. Tamari, R., Castro-Malaspina, H.: Allogeneic haematopoietic stem cell transplantation for primary myelofibrosis and myelofibrosis evolved from other myeloproliferative neoplasms. In: *Current Opinion in Hematology*, vol. 22, pp. 184–190. LWW (2015)

27. Walker, I., Sigouin, C., Sek, J., Almonte, T., Carruthers, J., Chan, A., Heddle, N.: Comparing hand-held computers and paper diaries for haemophilia home therapy: A randomized trial. *Heamophilia* **10**, 698–704 (2004)

28. Wesley, K.M., Fizur, P.J.: A review of mobile applications to help adolescent and young adult cancer patients. *Adolescent Health, Medicine and Therapeutics* **6**, 141–148 (2015)
29. Wesley, K., Fizur, P.: A review of mobile applications to help adolescent and young adult cancer patients. Adolescent Health, Medicine and Therapeutics p. 141 (2015). https://doi.org/10.2147/AHMT.S69209

30. Wilcox, A.B., Gallagher, K.D., Boden-Albala, B., Bakken, S.R.: Research data collection methods: From paper to tablet computers. Medical Care **50**, 68–73 (2012)

31. Wolfe, J., Orellana, L., Ullrich, C., Cook, E.F., Kang, T.I., Rosenberg, A., Dussel, V.: Symptoms and distress in children with advanced cancer: Prospective patient-reported outcomes from the PrediQUEST study. Journal of Clinical Oncology: Official Journal of the American Society of Clinical Oncology **33**, 1928–1935 (2015)