Palli-MONITOR: A Phase II Mixed-Methods Study to Implement and Test the Electronic Version of the Integrated Palliative Care Outcome Scale (eIPOS) in an Specialist Palliative Home Care
Phase: NA
Conditions: Advanced Disease
Sponsor: Ludwig-Maximilians - University of Munich

1
Bolzani A, et al. BMJ Open 2021;11:e042266. doi:10.1136/bmjopen-2020-042266
Open access
Monitoring of Palliative Care Symptoms 
and Concerns in Specialized Palliative 
Home Care Using an Electronic Version 
of the Integrated Palliative care 
Outcome Scale (Palli-­MONITOR): 
protocol for a mixed-­methods study
Anna Bolzani  ‍ ‍ ,1 Christina Ramsenthaler,2,3 Farina Hodiamont,1 
Isabel Sophie Burner-­Fritsch,1 Claudia Bausewein1
To cite: Bolzani A, 
Ramsenthaler C, Hodiamont F, 
et al.  Monitoring of Palliative 
Care Symptoms and Concerns 
in Specialized Palliative Home 
Care Using an Electronic Version 
of the Integrated Palliative 
care Outcome Scale (Palli-­
MONITOR): protocol for a mixed-­
methods study. BMJ Open 
2021;11:e042266. doi:10.1136/
bmjopen-2020-042266
►
►Prepublication history for 
this paper is available online. 
To view these files, please visit 
the journal online (http://​dx.​doi.​
org/​10.​1136/​bmjopen-​2020-​
042266).
Received 30 June 2020
Accepted 28 April 2021
For numbered affiliations see 
end of article.
Correspondence to
Dr Anna Bolzani;  
​Anna.​Bolzani@​med.​uni-​
muenchen.​de
Protocol
© Author(s) (or their 
employer(s)) 2021. Re-­use 
permitted under CC BY-­NC. No 
commercial re-­use. See rights 
and permissions. Published by 
BMJ.
ABSTRACT
Introduction  Over the last decades, patient-­reported 
outcome (PRO) measures have been developed to better 
understand the patient’s perspective and enable patient-­
centred care. In palliative care, the Integrated Palliative 
care Outcome Scale (IPOS) is recommended as a PRO 
tool. Its implementation in specialised palliative home care 
(SPHC) would benefit from an electronic version validated 
for the setting.
Following the Medical Research Council (MRC) guidance, 
the study Palli-­MONITOR is developing (phase 1) and 
testing the feasibility (phase 2) of implementing the 
electronic version of IPOS (eIPOS) in the SPHC setting to 
inform a cluster-­randomised phase 3 trial.
Methods and analysis  Palli-­MONITOR is a multicentre, 
sequential mixed-­methods, two-­phase development and 
feasibility study. The study consists of four substudies. In phase 
1 (MRC development phase), qualitative patient interviews 
and focus groups with SPHC professionals are used to identify 
barriers and facilitators of eIPOS (substudy I). Substudy II tests 
the equivalence of eIPOS and IPOS in a crossover randomised 
controlled trial. Phase 2 (MRC feasibility/piloting phase) includes 
a quasi-­experimental study with two control groups (substudy 
III), and qualitative interviews as well as focus groups to explore 
the feasibility and acceptability of the developed intervention 
(substudy IV).
Qualitative data will be analysed with thematic analysis 
following the framework approach. Quantitative analysis 
uses a two-­way intraclass correlation coefficients model for 
the equivalence testing. Quantitative analysis of the quasi-­
experimental study will focus on the primary outcomes, 
recruitment rates and completeness of eIPOS. Secondary 
outcomes will include intraindividual change in palliative 
symptoms and concerns, quality of life and symptom burden.
Ethics and dissemination  Approval of the ethics 
committee of the Ludwig Maximilian University Munich 
was received for all study parts. Results and experiences 
will be presented at congresses and in written form. 
Additionally, participating SPHC teams will receive 
summarised results.
Trial registration number  NCT03879668.
INTRODUCTION
In Germany, patients with severe and life-­
threatening conditions are entitled under 
law to receive specialised palliative home 
care (SPHC) since 2007.1 Currently, over 
300 SPHC teams are established in Germany, 
providing multiprofessional palliative care at 
home for patients with advanced life-­limiting 
diseases and complex symptoms.2–4 Besides 
professional care focusing on symptom 
control, this also includes psychological and 
spiritual support as well as coordination 
between other healthcare professionals, for 
instance, general physicians.2 3
Over the last decades, patient-­reported 
outcome (PRO) measures have been devel-
oped for a better understanding of patient 
Strengths and limitations of this study
►
►This exploratory study systematically develops and 
tests an electronic version of the Integrated Palliative 
care Outcome Scale, a valid and reliable paper-­and-­
pencil patient-­reported outcome in palliative care, 
for the specialist palliative home care setting.
►
►This protocol follows the Medical Research Council 
guidance on the development and evaluation of 
complex interventions which will increase the valid-
ity of results.
►
►Early involvement of stakeholders such as palliative 
care patients and professionals in the development 
phase will ensure rich data from multiple perspec-
tives to inform a feasible intervention for this spe-
cialised setting.
►
►Potential sampling bias due to the convenience sam-
ple based on staff identification of eligible patients 
will be limited by training staff and co-­designing the 
intervention with staff from the participating sites.
Protected by copyright.
 on April 19, 2023 at Universitaetsbibliothek der LMU Muenchen.
http://bmjopen.bmj.com/
BMJ Open: first published as 10.1136/bmjopen-2020-042266 on 1 June 2021. Downloaded from 

2
Bolzani A, et al. BMJ Open 2021;11:e042266. doi:10.1136/bmjopen-2020-042266
Open access
needs. They are also used as quality standards for the 
care received.5 However, implementation in health-
care outside of clinical research is still lacking even 
though feedback in a timely fashion improves patients’ 
outcomes.6 Correct interpretation of outcomes, ques-
tioning the impact of PROs in healthcare, and limited 
time for added PRO collection within a busy clinical 
care setting are mentioned as barriers for wider use.7–9 
The costs associated with the implementation of the 
routine use of PROs for patient monitoring are espe-
cially considered as a barrier. However, these costs can 
be reduced by up to 75% compared with using paper 
formats.10 11 The provision and recording of electronic 
data reduce costs of printing and sending the instru-
ments as well as staff time needed for documentation 
and administration.12
Implementing PROs digitally may facilitate the assess-
ment of patient information and its use in clinical prac-
tice, thus improving processes of care. For a successful 
implementation, electronic PROs (ePROs) need to agree 
with the users’ needs. Poor user-­friendliness was a major 
obstacle for the use of previous PRO systems for patients 
at the end of life and for patients with sensory or cogni-
tive limitations in old age, thus preventing high and long-­
term participation and compliance with the self-­reported 
assessment.13
Examining the equivalence of ePROs to paper-­based 
versions is still focused mainly on curative popula-
tions or cancer populations not at the end of life.11–17 
Testing the concurrent validity and agreement has 
yielded moderate to high intraclass correlation coef-
ficients (ICCs) for most patients, but with the notable 
exception of older adults over 55 years.18 19 One recent 
study established ICCs for test–retest reliability of an 
ePRO versus its paper-­based version for measuring 
adverse events in patients with curative cancer.11 How 
and whether ePROs show a comparable performance 
in terms of validity and reliability in a severely ill, 
older and frail population such as the SPHC popula-
tion remains to be determined.
In the palliative care setting, the Integrated Palliative 
care Outcome Scale (IPOS) is recommended as a PRO 
tool.16 IPOS measures palliative care symptoms and 
concerns of patients with advanced cancer or non-­cancer 
disease and is available for patients and carers.14 16 17 The 
questionnaire consists of 17 items measuring the impact 
of physical symptoms, emotional and spiritual issues for 
the patient and family, and information and practical 
concerns. The questionnaire has been tested in several 
inpatient and home care settings as a standardised tool 
to measure current and changing palliative needs of 
patients.15 20
In the home care setting, implementation of PRO 
measures such as IPOS, which are only available in paper 
format, is challenging. Paper versions cannot easily 
be transported between team members and patients, 
resulting in delays in clinical feedback to identified prob-
lems. This is detrimental to the frequent monitoring 
of symptoms necessary for timely and patient-­centred 
care for this vulnerable patient group. A way to monitor 
patients’ perspectives in this setting is the use of ePRO 
measures. Different tools have been developed in the last 
years and tested in clinical practice.21 However, the poten-
tial of PRO monitoring in real time in a palliative care 
setting is yet to be determined.
The project ‘Monitoring of Palliative Care Needs in 
Specialized Palliative Home Care Using an Electronic 
Version of the Integrated Palliative care Outcome Scale’ 
(Palli-­MONITOR) is developing and testing the imple-
mentation of a new electronic version of the previously 
validated paper-­based IPOS (eIPOS). The eIPOS aims 
to support professional healthcare through standardised 
collection of patients’ perspectives about palliative 
symptoms and concerns in real time. Palli-­MONITOR 
is designed following the updated Medical Research 
Council (MRC) guidelines for the development and eval-
uation of complex interventions,22 focusing on the devel-
opment and the feasibility phase.
The overall aim of Palli-­MONITOR is to inform a subse-
quent cluster-­randomised phase 3 trial testing the effects 
of the developed intervention.
The primary aim of phase 1 (development) is the develop-
ment of an acceptable and validated eIPOS.
Objectives are:
1.	 To explore barriers and facilitators of an ePRO as a 
standardised tool in SPHC.
2.	 To test the equivalence of the electronic and paper 
IPOS (patient version).
The primary aim of phase 2 (feasibility/piloting) is testing 
the feasibility of the planned implementation process.
Objectives are:
1.	 To evaluate the feasibility of eIPOS use in terms of par-
ticipant recruitment, data completion, usage rate and 
technical practicability of the planned intervention.
2.	 To analyse the presumed change in care processes be-
fore and after the implementation of eIPOS.
3.	 To explore the acceptability of the ePRO measure for 
patients and SPHC professionals.
METHODS AND ANALYSIS
Palli-­MONITOR is a sequential mixed-­methods, two-­
phase development and feasibility study (figure 1). Each 
of the two project phases consists of two substudies. The 
results from substudy I and II will inform an acceptable 
and validated electronic version of eIPOS which will 
be used in the feasibility study in phase 2 consisting of 
substudies III and IV. We follow the MRC guidance and 
use substudies I and II to develop the intervention (MRC 
phase 1) and substudies III and IV to test the feasibility of 
the intervention (MRC phase 2).22
The project is led by a research team in the Depart-
ment of Palliative Medicine, University Hospital of 
the Ludwig Maximilian University (LMU) Munich, 
supported by two information technology companies. 
Five SPHC teams participate in Palli-­MONITOR based 
Protected by copyright.
 on April 19, 2023 at Universitaetsbibliothek der LMU Muenchen.
http://bmjopen.bmj.com/
BMJ Open: first published as 10.1136/bmjopen-2020-042266 on 1 June 2021. Downloaded from 

3
Bolzani A, et al. BMJ Open 2021;11:e042266. doi:10.1136/bmjopen-2020-042266
Open access
on the geographical area: two teams provide care in 
a mostly urban setting, two teams in a rural area and 
one team in a mixed region. The project is conducted 
between 2018 and 2021/2022. Substudies III and IV 
will continue until 2022.
Development of the intervention (MRC phase 1)
Phase 1 consists of two substudies: for the development 
of eIPOS, exploring the implementation of eIPOS with 
stakeholders (substudy I); and testing the equivalence 
of the developed eIPOS and IPOS (substudy II). Both 
substudies aim to identify and explore in depth the 
barriers and facilitators of routinely using an ePRO in a 
severely ill population and best ways of incorporating this 
information into everyday practice. Within this develop-
ment phase, processes and outcomes of implementing 
eIPOS will be modelled according to implementation 
science regarding PROs.23–25
Substudy I: qualitative interviews and focus groups
Semistructured interviews with patients and focus groups 
with SPHC professionals will be conducted. The inter-
view guides will be developed with experienced meth-
odological experts and patients and consist of several 
topics about the acceptance of electronic monitoring in 
the SPHC setting (see table 1). The topic guides will be 
piloted with two members of a patient and public involve-
ment (PPI) group.
Patient interviews will be sampled using a sampling 
matrix, covering age, gender, residential area (urban 
or rural) and primary disease to represent potentially 
different views in the broad variety of patients in SPHC. 
For recruiting, professionals of the participating SPHC 
teams will ask eligible patients if they would like to partic-
ipate in the study.
Patients will be interviewed in their home (or another 
location chosen by the participant) by one researcher 
preferably alone to avoid any response bias. If preferred, 
the participant can choose his or her caregiver to be 
part of the interview. The interview will be conducted 
in person and will last around 45–60 min. One focus 
group will be conducted with the urban teams in the 
rooms of the Department of Palliative Medicine and 
one with the rural teams in the rooms of one partici-
pating SPHC team due to large distances between the 
research team and participating teams and limited 
resources. Focus groups will last about 90–120 min.
Interviews and focus groups will be digitally recorded 
and transcribed verbatim using anonymisation by a 
professional transcription office.
Inclusion criteria
Patients
Currently cared for by one of the participating SPHC 
teams (including access to emergency services) regardless 
of the underlying disease; ≥18 years; having the capacity 
to give written consent; not too distressed or ill to partici-
pate in the study (assessed by the clinical team); and suffi-
ciently fluent in German.
Figure 1  Study design of the project Palli-­MONITOR. eIPOS, electronic IPOS; IPOS, Integrated Palliative care Outcome Scale.
Table 1  Topic guides for interviews and focus groups
Qualitative interviews with patients
Focus groups with professionals
Electronic 
monitoring in 
SPHC
►
►General effects of filling in eIPOS (for example, 
reflecting of own current situation)
►
►Frequency of eIPOS use (for example, daily, 2–3 
times a week, etc)
►
►Support in the identification of current unknown 
symptoms
►
►Ways of implementing results of eIPOS in current 
SPHC
Barriers and 
facilitators
►
►Technical challenges
►
►Motoric issues due to underlying disease
►
►Design of eIPOS
►
►Presentation of patient-­reported data
►
►Identification of eligible patients
►
►Technical challenges
eIPOS, electronic Integrated Palliative care Outcome Scale; SPHC, specialised palliative home care.
Protected by copyright.
 on April 19, 2023 at Universitaetsbibliothek der LMU Muenchen.
http://bmjopen.bmj.com/
BMJ Open: first published as 10.1136/bmjopen-2020-042266 on 1 June 2021. Downloaded from 

4
Bolzani A, et al. BMJ Open 2021;11:e042266. doi:10.1136/bmjopen-2020-042266
Open access
Professionals
Participants have to be  ≥18 years, work in one of the 
participating SPHC teams and be sufficiently fluent in 
German.
Recruitment
Patients
The participating SPHC teams will identify eligible 
patients for the interviews. If a patient fulfils the inclu-
sion criteria and has agreed to share their contact details, 
the SPHC teams will inform the research team. One 
researcher will then explain the study. If the patient is 
interested in participating, the researcher will make an 
appointment for introducing the electronic system to the 
patient. Before the appointment, the patient will need 
to give written consent to participate in the study. After 
the introduction, the patient will test the eIPOS for up to 
2 weeks before being interviewed by the same researcher.
Professionals
Professional members of the participating SPHC teams 
will be asked to participate in focus groups, consisting of 
at least one medical and one nursing professional each. 
Eligible participants will be identified by the research 
team and/or the contact person(s) of each SPHC team. 
Written consent will be obtained before participation.
Sample size
Interviews will be conducted until thematic saturation is 
reached.26 27 Following the sampling criteria aiming at 
maximal variation, saturation is expected to be achieved 
after about 20 patient interviews. Two focus groups, with 
the recommended number of six to nine professionals of 
the SPHC teams, will be held.
Data analysis
Data will be analysed with qualitative thematic analysis 
following the framework approach established by Ritchie 
and Spencer.28 29 Two researchers of the project team 
will independently code the transcripts of five interviews. 
Preliminary codes will be developed both inductively and 
deductively based on existing implementation models for 
PROs in healthcare.23–25 Afterwards, the researchers will 
exchange and reflect their ideas and interpretations of 
the transcripts. Based on these discussions, one researcher 
will develop a comprehensive coding framework and 
arrange identified categories around the major themes.30 
The focus is the understanding of barriers and facilitators 
for the use of electronic monitoring in an SPHC setting. 
Findings shall inform necessary requirements of patients 
and professional carers to develop a feasible implemen-
tation process. A multidisciplinary research team will 
accompany this process in several meetings to bring up 
alternative interpretations, give feedback and gain possi-
bilities of reflection and to avoid bias.31 The final coding 
framework will then be applied to the whole data set. 
MAXQDA will be used to support data management.32
Data from this substudy will inform phase 2 (feasibility/
piloting of the intervention) of this project by helping 
to identify optimal procedures for implementing a self-­
report ePRO into routine care. Co-­designing the eIPOS 
intervention with staff members from SPHC services will 
ensure that eIPOS can help detect the needs of patients, 
thereby informing care processes and symptom manage-
ment in routine care.
Substudy II: equivalence testing
A randomised crossover trial will be conducted to deter-
mine the equivalence of eIPOS and IPOS. Participants 
will complete the two versions of the IPOS in randomised 
order with a 30-­minute waiting period between the ques-
tionnaires. The period should be long enough to not 
complete the second version by memory, but at the same 
time short enough for symptoms to not change between 
the two measurement points. The equivalence testing of 
the electronic version follows the ISPOR guidelines for 
testing measurement equivalence between electronic and 
paper-­based PRO measures.33
To complete the electronic version, participants need 
to open a website (​www.​umfrageonline.​com/​s/​pallimon-
itor) and enter an anonymised ID that will be provided 
by the researcher. After that, they will complete the ques-
tionnaire. The overall appointment lasts about 45–60 min 
(5–10 min for each version and 30-­minute waiting period).
Inclusion criteria
The inclusion criteria for participants are: currently cared 
for by one of the participating SPHC teams (including 
access to emergency services) or in a different pallia-
tive setting (palliative care unit or generalist palliative 
home care) regardless of the underlying disease; ≥18 
years; having the capacity to give written consent; not too 
distressed or ill to participate in the study (as judged by 
the clinical team); and sufficiently fluent in German.
Recruitment
Eligible patients will be identified by the participating 
care teams. If a patient fulfils the criteria and has agreed 
to share contact details, the research team will be 
informed. One researcher will then explain the objec-
tive, study procedure and requirements in detail to the 
patient. If the patient is interested, the researcher will 
make an appointment. Patients on the palliative care unit 
who fulfil the inclusion criteria are asked by the respon-
sible physician if they would like to take part in this study. 
Patients will need to give written consent to participate in 
the study.
Demographic and clinical data collection
Data collection will take place in the participant’s home 
(or another location chosen by the participant) or in the 
palliative care unit in the hospital. Preferably, the partici-
pant will be alone during the time of testing to avoid any 
response bias. Participants will be given as much infor-
mation as needed and the research team will explain the 
procedure.
In addition to the two IPOS questionnaires, the 
following sociodemographic and clinical data will be 
Protected by copyright.
 on April 19, 2023 at Universitaetsbibliothek der LMU Muenchen.
http://bmjopen.bmj.com/
BMJ Open: first published as 10.1136/bmjopen-2020-042266 on 1 June 2021. Downloaded from 

5
Bolzani A, et al. BMJ Open 2021;11:e042266. doi:10.1136/bmjopen-2020-042266
Open access
collected: age, gender, nationality (German or non-­
German), main diagnosis according to International 
Classification of Diseases, 10th Revision groups (cancer/
chronic heart disease/lung disease/others), frequency 
of using electronic devices, type of electronic device 
(personal computer, notebook, tablet, smartphone, etc), 
and operating system (Windows, Apple, Android, etc). 
Furthermore, the research team will measure the time (in 
minutes) that participants need to complete each IPOS 
version. After completing the electronic and the paper 
IPOS, the researcher will ask patients which version they 
prefer.
Sample size calculation
With a power of 80%, a target ICC of 0.9 and a signif-
icance level of α=0.05, the calculated sample size is 47. 
Considering possible drop-­outs, it is planned to include 
50 participants.
Data analysis
A two-­way ICC model with fixed effects will be calcu-
lated.34 The Wilcoxon test will be used for comparing 
IPOS values of both versions and completion time. 
Equivalence will be established for individual IPOS 
items, for IPOS subscales (physical symptoms subscale, 
emotional concerns, communication and quality of care 
subscale)17 and the total IPOS score. Bland-­Altman plots 
will be created to present the equivalence and differ-
ences between both versions visually for IPOS subscale 
values and the IPOS total score. To judge the amount of 
sampling bias, the sociodemographic details of partici-
pants will be compared with population profiles of partic-
ipating sites via descriptive statistics.
Data from substudy II will inform the development of 
a valid and reliable eIPOS that can be implemented into 
routine clinical care to be tested in MRC phase 2 of this 
study.
Feasibility/piloting (MRC phase 2)
MRC phase 2 of this project focuses on testing procedures 
of the intervention (piloting), estimating recruitment/
retention rates and determining the acceptability of the 
intervention (feasibility). Phase 2 consists of two parts 
(figure 1 for details of the integration within the overall 
study design): the intervention and implementation 
procedures developed in phase 1 will be tested (substudy 
III). Furthermore, the acceptability of the intervention 
will be determined using a qualitative approach (substudy 
IV).
Substudy III: quasi-experimental study to pilot and test the 
feasibility of the eIPOS intervention
Substudy III aims to introduce the eIPOS into the routine 
care processes of each participating SPHC service. In addi-
tion to the intervention group receiving eIPOS, there will 
be two control groups: (a) a prospective control group 
with patients not receiving the eIPOS, and (b) a retro-
spective (historical) control group consisting of medical 
record/chart reviews for patients in the services prior to 
the introduction of eIPOS. The design of this substudy is 
summarised in figure 2. All patients cared for by one of 
the participating SPHC teams are eligible for this study. 
The intervention consists of patients completing eIPOS 
for a period of 2 weeks, with corresponding care processes 
(score review by clinical staff, identifying areas of need 
and targeting interventions for symptom management 
accordingly). Participants in the control group, who chose 
not to use the eIPOS or were ineligible, will not complete 
any self-­report PROs. Care processes for this group will 
consist of usual care provided by the SPHC service. The 
retrospective control groups will consist of patients who 
were cared for by SPHC teams during the last 6 months 
before the implementation of eIPOS.
Figure 2  Substudy III—data collection and inclusion criteria of the quasi-­experimental study. *Assessed by the clinical team. 
eIPOS, electronic Integrated Palliative care Outcome Scale; ESAS, Edmonton Symptom Assessment Scale; QoL, Quality of Life; 
SPHC, specialised palliative home care.
Protected by copyright.
 on April 19, 2023 at Universitaetsbibliothek der LMU Muenchen.
http://bmjopen.bmj.com/
BMJ Open: first published as 10.1136/bmjopen-2020-042266 on 1 June 2021. Downloaded from 

6
Bolzani A, et al. BMJ Open 2021;11:e042266. doi:10.1136/bmjopen-2020-042266
Open access
Intervention group
At baseline, a study nurse will introduce the patient to 
eIPOS. After that, the participant will complete eIPOS 
every 3 days. There will be no obligation to answer each 
item of the eIPOS. Standardised procedures (for example: 
frequency of measurement, time point of introduction) 
will be finalised after phase 1. The information from self-­
reported eIPOS will be used in routine care processes of 
the service.
Prospective control
All patients who are cared for by one of the participating 
SPHC teams, but do not use the eIPOS, will serve as a 
control group. No additional interventions are planned 
for this group. However, care processes for this prospec-
tive control group might change because of staff being 
introduced to training around outcome measurement.
Retrospective control
A retrospective chart review will be conducted to provide 
information on care processes in the services before the 
introduction of a self-­report PRO measure.
Inclusion criteria
The inclusion criteria are similar to substudy I and 
summarised in table 2.
Recruitment
Intervention group: patients will be recruited from each 
participating team with the support from a study nurse. A 
study nurse will inform eligible patients in detail about the 
aims, study procedure and requirements. If the patient is 
interested, the study nurse will make an appointment for 
the introduction of the electronic system and perform 
the baseline assessment. The patient will need to give 
written consent before participation.
Outcome measures
Recruitment as well as drop-­out rates, technical feasi-
bility (completeness of each eIPOS) and the influence of 
eIPOS on provided care will be measured as the primary 
outcomes of interest.
Secondary outcomes will include change in palliative 
concerns, quality of life and symptom burden measured 
with the eIPOS, the EORTC QLQ-­C15-­PAL (measuring 
health-­related quality of life of palliative patients with 
cancer)35 and the Edmonton Symptom Assessment Scale 
(assessing symptoms of palliative care patients)36 at base-
line and after 2 weeks.
Data extracted from medical records for both the inter-
vention and control groups will include the number and 
severity of palliative symptoms and concerns and the steps 
taken by professionals for reducing the symptoms. Addi-
tionally, the number of unplanned hospital admissions 
will be collected.
Sample size calculation
As sample size calculation in feasibility studies should 
not be based on the primary outcomes of interest,37 the 
intraindividual comparison of quality of life between 
baseline and 2-­week assessment was used as the basis for 
the estimation. An assumed small effect of d=0.20 (with a 
minimal clinical difference of 4–9 for the EORTC QLQ 
scale), power=80% and α=0.05 for a Wilcoxon test with 
paired data lead to a sample size of 164. Based on an 
assumed drop-­out rate of 30%, it is aimed to include 213 
participants.
For a better understanding of the feasibility of the 
sample size, and the possible success of the recruit-
ment plan, the total number of patients cared for by the 
participating SPHC teams (~1.275 patients/year) will 
be extracted as well. The control groups will include all 
patients cared for in the time period explained above.
Data analysis
Quantitative analysis will include descriptive analysis of 
the user rate of eIPOS, recruitment and drop-­out rate 
as well as analysis of the technical feasibility based on 
completeness of eIPOS and number of missing values. 
Intraindividual changes in the secondary outcomes 
before and after the implementation of eIPOS will be 
explored using non-­parametric tests (either Wilcoxon or 
Friedman). Number of symptoms and amount of provided 
care before and after the implementation of eIPOS will 
be compared using McNemar tests and Cochran tests for 
changes in proportions.
Table 2  Inclusion criteria for substudy III
Intervention group
Prospective control group
Historical control
►
►Currently cared for by one of the participating 
SPHC teams (including access to emergency 
services) regardless of the underlying disease
►
►≥18 years
►
►Having the capacity to give written consent
►
►Not too distressed or ill to participate in the 
study (assessed by the clinical team)
►
►Sufficiently fluent in German
►
►Currently cared for by 
one of the participating 
SPHC teams (including 
access to emergency 
services) regardless of the 
underlying disease
►
►≥18 years
►
►Not using eIPOS
►
►Inclusion/exclusion criteria will be 
finalised after analysing data from phase 
1
►
►Currently planned: inclusion of all 
patients cared for by the participating 
SPHC teams (including access to 
emergency services) during the last 
6 months before the implementation of 
eIPOS
eIPOS, electronic Integrated Palliative care Outcome Scale; SPHC, specialised palliative home care.
Protected by copyright.
 on April 19, 2023 at Universitaetsbibliothek der LMU Muenchen.
http://bmjopen.bmj.com/
BMJ Open: first published as 10.1136/bmjopen-2020-042266 on 1 June 2021. Downloaded from 

7
Bolzani A, et al. BMJ Open 2021;11:e042266. doi:10.1136/bmjopen-2020-042266
Open access
Substudy IV: acceptability of eIPOS (feasibility)
The methods and procedures will be identical with 
substudy I. In short, semistructured interviews with 
patients and focus groups with SPHC professionals will be 
conducted, focusing on the experience with eIPOS and 
the influence of it on the received/provided care.
Inclusion criteria
Participants who used the eIPOS in substudy III are eligible 
for the interviews. Apart from that, inclusion criteria are 
identical with the inclusion criteria of substudy I. Addi-
tional inclusion and exclusion criteria may be defined 
after analysing data of substudy I.
Recruitment
Recruiting participants for this part will be identical with 
the approach in substudy I.
Sample size
The same approach for the number of included partici-
pants will be followed as in substudy I.
Data analysis
The analysis will be identical with substudy I, following 
the framework approach established by Ritchie and 
Spencer.28 29 The analysis will focus on the experiences 
and opinions of patients and professionals on the imple-
mentation of the eIPOS and the influence on the received 
care.
Data management
Contact information of patients will only be shared with 
the research team if patients agreed to do so.
Qualitative data (I, IV): an external company will tran-
scribe the interviews and anonymise them in the process. 
This company has to comply with data protection and confi-
dentiality regulations. Audio recordings will be deleted once 
the transcript is provided. Additional data (for example, age 
and gender) will be collected anonymously and be linked 
to the transcript with a random study ID.
Quantitative data (II, III): data from medical records 
will be extracted anonymously. Original data with personal 
information will only be visible to study nurses who are 
part of the SPHC teams. Researchers will not have any 
access to personal patient information. Quantitative data 
will be saved for 10 years on a secured server at the LMU 
Munich.
Adverse events
Even though adverse events are judged to be unlikely 
for this intervention, all study nurses and research 
team members will follow a distress protocol and report 
any adverse events associated with the intervention. 
Researchers and study nurses are asked to note any 
reasons for drop-­outs and withdrawals during the course 
of this project.
Patient and public involvement
PPI in research is of particular importance, especially in 
palliative care with a vulnerable group of patients.38 In 
Palli-­MONITOR, a PPI group will be involved in all stages 
of the study to take the perspective of stakeholders and 
lay persons into account.39 Adults with advanced diseases, 
relatives of palliative patients, voluntary hospice workers 
and professional carers will be part of this group.
The group members will be involved in the develop-
ment of documents, such as participant information or 
interview guides. Furthermore, progress of the study, 
unexpected challenges and aspects of research ethics 
from the perspective of those potentially affected by the 
research will be discussed.
Strengths and limitations
In this study, the research team will work with a variety of 
stakeholders to consider different requirements, perspec-
tives and opinions in this very challenging research 
setting. A mixed-­methods approach was chosen to 
explore different barriers and facilitators of the planned 
intervention.
Recruitment is a natural challenge in the palliative 
care setting. Even more so, patients eligible for SPHC 
suffer from life-­limiting diseases with a complex symptom 
burden. Developing a rapid process for the identification 
and inclusion of eligible patients with the participating 
SPHC teams as well as the fact that all study appointments 
will take place in the patients’ home should increase the 
recruitment rates for all study parts. A limitation of the 
study is its convenience sample based on staff identifica-
tion of eligible patients in participating study sites. There-
fore, gatekeeping and sampling of only comparatively 
fitter patients cannot be ruled out completely. We hope 
to mitigate this sampling bias by training staff and co-­de-
signing the intervention with staff from the participating 
sites. We will compare the sociodemographic details of 
recruited patients with population data from the services 
to quantify this possible bias.
The use in clinical practice will depend on the compli-
ance with the newly developed intervention.40 Other 
researchers, for instance, Lang et al41 explored non-­
compliance with a telemedical intervention for multi-
morbid patients over 65 years and discovered that missing 
additional benefits of the intervention, missing content 
variety and no interest in telemedical care were the main 
reasons for drop-­outs and/or not using the intervention 
at all. Studies with patients suffering from heart failure or 
chronic obstructive pulmonary disease testing telemed-
ical interventions have a drop-­out rate of around 20% and 
almost one-­third of the patients refuse to participate.42
Therefore, the involvement of patients and profes-
sionals is crucial in the development and implementa-
tion of eIPOS. In phase 1, barriers and facilitators of the 
planned intervention will be explored. The results will 
inform the development of the implementation process. 
There will also be training sessions on site for each SPHC 
team between phase 1 and 2 to present results, and to 
discuss positive as well as negative comments received in 
phase 1.
Protected by copyright.
 on April 19, 2023 at Universitaetsbibliothek der LMU Muenchen.
http://bmjopen.bmj.com/
BMJ Open: first published as 10.1136/bmjopen-2020-042266 on 1 June 2021. Downloaded from 

8
Bolzani A, et al. BMJ Open 2021;11:e042266. doi:10.1136/bmjopen-2020-042266
Open access
Results of this project will influence the preparation 
and conduct of a planned randomised phase 3 study to 
determine the effect of the developed intervention.
ETHICS AND DISSEMINATION
Approval of the research ethics committee of the LMU 
Munich for all study parts was received (18-815; 18-871; 
19-512; 19-586; 19-585).
Any changes of this protocol that will be made during 
the conduct of this project will be discussed in further 
publications. Besides presenting results and experiences 
at congresses and in written form, each participating 
SPHC team will receive a summary of the results.
Due to different healthcare structures and the concept 
of the SPHC related to the German context, some of the 
results may be published in German journals.
Author affiliations
1Department of Palliative Medicine, University Hospital, LMU Munich, Munich, 
Germany
2Department of Palliative Medicine, Albert Ludwig University of Freiburg, Freiburg im 
Breisgau, Germany
3Cicely Saunders Institute, Department of Palliative Care, Policy and Rehabilitation, 
King’s College London, London, UK
Contributors  CB is the chief investigator and responsible party for the design of 
the study. AB is the coordinating investigator and responsible for the conduct of 
this study. CB, FH, CR and AB conceptualised the study. AB drafted the protocol 
manuscript. ISB-­F supported the development of the manuscript. All authors 
provided critical feedback to the manuscript, read and approved the final draft.
Funding  This work was supported by Federal Joint Committee German Innovation 
Fund (Innovationsausschuss des Gemeinsamen Bundesausschuss), grant number 
01VSF17014.
Competing interests  None declared.
Patient and public involvement  Patients and/or the public were involved in the 
design, or conduct, or reporting, or dissemination plans of this research. Refer to 
the Methods section for further details.
Patient consent for publication  Not required.
Provenance and peer review  Not commissioned; externally peer reviewed.
Open access  This is an open access article distributed in accordance with the 
Creative Commons Attribution Non Commercial (CC BY-­NC 4.0) license, which 
permits others to distribute, remix, adapt, build upon this work non-­commercially, 
and license their derivative works on different terms, provided the original work is 
properly cited, appropriate credit is given, any changes made indicated, and the use 
is non-­commercial. See: http://​creativecommons.​org/​licenses/​by-​nc/​4.​0/.
ORCID iD
Anna Bolzani http://​orcid.​org/​0000-​0002-​3049-​9805
REFERENCES
	 1	 Gemeinsamer Bundesausschuss (G-­BA). Richtlinie des 
Gemeinsamen Bundesausschusses Zur Verordnung von 
spezialisierter ambulanter Palliativversorgung (spezialisierte 
ambulante Palliativversorgungs-­Richtlinie/SAPV-­RL), 2007.
	 2	 Alt-­Epping B, Nauck F. Spezialisierte Ambulante Palliativversorgung 
(SAPV). Bundesgesundheitsblatt - Gesundheitsforschung - 
Gesundheitsschutz 2015;58:430–5.
	 3	 Groh G, Vyhnalek B, Feddersen B, et al. Effectiveness of a 
specialized outpatient palliative care service as experienced by 
patients and caregivers. J Palliat Med 2013;16:848–56.
	 4	 Kassenärztliche Bundesvereinigung. Spezialisierte Ambulante 
Palliativversorgung, 2017. Available: http://​gesundheitsdaten.​kbv.​de/​
cms/​html/​16744.​php
	 5	 Schneider W, Eschenbruch N, Thoms U, et al. Wirksamkeit 
und Qualitätssicherung in Der SAPV-­Praxis—Eine explorative 
Begleitstudie. Ergebnisbericht, 2011.
	 6	 Currow DC, Allingham S, Yates P, et al. Improving national 
hospice/palliative care service symptom outcomes systematically 
through point-­of-­care data collection, structured feedback and 
benchmarking. Support Care Cancer 2015;23:307–15.
	 7	 Dunckley M, Aspinal F, Addington-­Hall JM, et al. A research study to 
identify facilitators and barriers to outcome measure implementation. 
Int J Palliat Nurs 2005;11:218–25.
	 8	 Noonan VK, Lyddiatt A, Ware P, et al. Montreal Accord on Patient-­
Reported Outcomes (PROs) use series - paper 3: patient-­reported 
outcomes can facilitate shared decision-­making and guide self-­
management. J Clin Epidemiol 2017;89:125–35.
	 9	 Radionova N, Becker G, Mayer-­Steinacker R, et al. The views of 
physicians and nurses on the potentials of an electronic assessment 
system for recognizing the needs of patients in palliative care. BMC 
Palliat Care 2020;19:1–9.
	10	 Basch E, Iasonos A, Barz A, et al. Long-­term toxicity monitoring 
via electronic patient-­reported outcomes in patients receiving 
chemotherapy. J Clin Oncol 2007;25:5374–80.
	11	 Bennett AV, Jensen RE, Basch E. Electronic patient-­reported 
outcome systems in oncology clinical practice. CA Cancer J Clin 
2012;62:337–47.
	12	 Kotronoulas G, Kearney N, Maguire R, et al. What is the value of 
the routine use of patient-­reported outcome measures toward 
improvement of patient outcomes, processes of care, and health 
service outcomes in cancer care? A systematic review of controlled 
trials. J Clin Oncol 2014;32:1480–501.
	13	 Velikova G, Booth L, Smith AB, et al. Measuring quality of life in 
routine oncology practice improves communication and patient well-­
being: a randomized controlled trial. J Clin Oncol 2004;22:714–24.
	14	 Bausewein C, Daveson BA, Currow DC, et al. EAPC White 
Paper on outcome measurement in palliative care: Improving 
practice, attaining outcomes and delivering quality services–
Recommendations from the European Association for Palliative 
Care (EAPC) Task Force on Outcome Measurement. Palliat Med 
2016;30:6–22.
	15	 Murtagh FE, Ramsenthaler C, Firth A, et al. A brief, patient- and 
proxy-­reported outcome measure in advanced illness: validity, 
reliability and responsiveness of the integrated palliative care 
outcome scale (IPOS). Palliat Med 2019;33:1045–57.
	16	 Onkologie Leitlinienprogramm. Palliativmedizin für Patienten MIT 
einer nicht heilbaren Krebserkrankung, Langversion 1.1, 2015.
	17	 Schildmann EK, Groeneveld EI, Denzel J, et al. Discovering the 
hidden benefits of cognitive interviewing in two languages: the first 
phase of a validation study of the integrated palliative care outcome 
scale. Palliat Med 2016;30:599–610.
	18	 Marcano Belisario JS, Jamsek J, Huckvale K, et al. Comparison of 
self-­administered survey questionnaire responses collected using 
mobile apps versus other methods. Cochrane Database Syst Rev 
2015;7:Mr000042.
	19	 Muehlhausen W, Doll H, Quadri N, et al. Equivalence of electronic 
and paper administration of patient-­reported outcome measures: a 
systematic review and meta-­analysis of studies conducted between 
2007 and 2013. Health Qual Life Outcomes 2015;13:167.
	20	 MySupport. MySupport: Implementierung und evaluation einer 
routinemäßigen Erfassung von patient-­centered outcome measures 
(PCOM) in onkologischen und palliativen Versorgungskontexten, 
2019. Available: https://www.​drks.​de/​drks_​web/​navigate.​do?​
navigationId=​trial.​HTML&​TRIAL_​ID=​DRKS00016681
	21	 Castelnuovo G, Mauri G, Simpson S, et al. New technologies for the 
management and rehabilitation of chronic diseases and conditions. 
Biomed Res Int 2015;2015:180436.
	22	 et alCraig P, Dieppe P, MacIntyre S. Developing and evaluating 
complex interventions: new guidance 2019. Available: https://​mrc.​
ukri.​org/​documents/​pdf/​complex-​interventions-​guidance/ [Accessed 
14 Apr 2020].
	23	 Antunes B, Harding R, Higginson IJ, et al. Implementing patient-­
reported outcome measures in palliative care clinical practice: 
a systematic review of facilitators and barriers. Palliat Med 
2014;28:158–75.
	24	 Greenhalgh J. The applications of PROs in clinical practice: what are 
they, do they work, and why? Qual Life Res 2009;18:115–23.
	25	 Greenhalgh T, Abimbola S. The NASSS framework - a synthesis of 
multiple theories of technology implementation. Stud Health Technol 
Inform 2019;263:193–204.
	26	 Przyborski A, Wohlrab-­Sahr M. Qualitative Sozialforschung. ein 
Arbeitsbuch: Oldenburg Wissenschaftsverlag, 2014.
	27	 Morse JM. Sampling in grounded theory. The SAGE handbook of 
grounded theory, 2010: 229–44.
Protected by copyright.
 on April 19, 2023 at Universitaetsbibliothek der LMU Muenchen.
http://bmjopen.bmj.com/
BMJ Open: first published as 10.1136/bmjopen-2020-042266 on 1 June 2021. Downloaded from 

9
Bolzani A, et al. BMJ Open 2021;11:e042266. doi:10.1136/bmjopen-2020-042266
Open access
	28	 Ritchie J, Spencer L. Qualitative data analysis for applied policy 
research. analyzing qualitative data. Routledge, 2002: 173–94.
	29	 Schreier M. Qualitative content analysis in practice. Sage 
publications, 2012.
	30	 Rädiker S, Kuckartz U. Analyse qualitativer Daten MIT MAXQDA, 
2019.
	31	 Hopf C. Schriften zu Methodologie und Methoden qualitativer 
Sozialforschung: Herausgegeben von Wulf Hopf und Udo Kuckartz. 
Springer-­Verlag, 2016.
	32	 Kuckartz U, Rädiker S. Datenaufbereitung und Datenbereinigung 
in Der qualitativen Sozialforschung. Handbuch Methoden Der 
empirischen Sozialforschung. Springer, 2019: 441–56.
	33	 Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on 
evidence needed to support measurement equivalence between 
electronic and paper-­based patient-­reported outcome (PRO) 
measures: ISPOR ePRO good research practices task force report. 
Value Health 2009;12:419–29.
	34	 Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic 
and paper-­and-­pencil administration of patient-­reported outcome 
measures: a meta-­analytic review. Value Health 2008;11:322–33.
	35	 Groenvold M, Petersen MA, Aaronson NK, et al. The development 
of the EORTC QLQ-­C15-­PAL: a shortened questionnaire for cancer 
patients in palliative care. Eur J Cancer 2006;42:55–64.
	36	 Bruera E, Kuehn N, Miller MJ, et al. The Edmonton symptom 
assessment system (ESAs): a simple method for the assessment of 
palliative care patients. J Palliat Care 1991;7:6–9.
	37	 Avery KNL, Williamson PR, Gamble C, et al. Informing efficient 
randomised controlled trials: exploration of challenges in 
developing progression criteria for internal pilot studies. BMJ Open 
2017;7:e013537.
	38	 Daveson BA, de Wolf-­Linder S, Witt J, et al. Results of a transparent 
expert consultation on patient and public involvement in palliative 
care research. Palliat Med 2015;29:939–49.
	39	 Wright M, Kilian H, Block M, et al. Partizipative Qualitätsentwicklung: 
Zielgruppen in alle Phasen Der Projektgestaltung einbeziehen. Das 
Gesundheitswesen 2015;77:S141–2.
	40	 Dockweiler C. Akzeptanz Der Telemedizin. eHealth in Deutschland. 
Springer, 2016: 257–71.
	41	 Lang C, Scheibe M, Voigt K, et al. Motive für die Nichtakzeptanz und 
Nichtnutzung einer Telemonitoring-­Anwendung Im häuslichen Umfeld 
durch multimorbide Patienten über 65 Jahre. Zeitschrift für Evidenz, 
Fortbildung und Qualität im Gesundheitswesen 2019;141:76–88.
	42	 Gorst SL, Armitage CJ, Brownsell S, et al. Home telehealth uptake 
and continued use among heart failure and chronic obstructive 
pulmonary disease patients: a systematic review. Ann Behav Med 
2014;48:323–36.
Protected by copyright.
 on April 19, 2023 at Universitaetsbibliothek der LMU Muenchen.
http://bmjopen.bmj.com/
BMJ Open: first published as 10.1136/bmjopen-2020-042266 on 1 June 2021. Downloaded from