# Health-related quality of life in cancer patients treated with immune checkpoint inhibitors: A systematic review on reporting of methods in randomized controlled trials

Stephane Faury, Jérôme Foucaud

► To cite this version:

Stephane Faury, Jérôme Foucaud. Health-related quality of life in cancer patients treated with immune checkpoint inhibitors: A systematic review on reporting of methods in randomized controlled trials. PLoS ONE, 2020, 15 (1), pp.e0227344. 10.1371/journal.pone.0227344. hal-03120097

HAL Id: hal-03120097

https://hal.science/hal-03120097

Submitted on 25 Jan 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
RESEARCH ARTICLE

# Health-related quality of life in cancer patients treated with immune checkpoint inhibitors: A systematic review on reporting of methods in randomized controlled trials

Stéphane Faury¹, Jérôme Foucaud²*

1 Laboratory Handicap, Activity, Cognition, Health, EA, University of Bordeaux, Bordeaux, France, 2 INCa, Institut National du Cancer, Boulogne Billancourt, France

* jfoucaud@institutcancer.fr

<figure><img src="image_2.png" /></figure>

## Abstract

### Objective

Immune checkpoint inhibitors (ICIs) have recently shown tremendous promise in the treatment of diverse cancers. The available data suggests that ICIs are well tolerated in terms of health-related quality of life (HRQOL) compared to other anticancer therapies. However, it appears that instruments currently used to evaluate HRQOL in this context may fail to capture important symptomatology unique to ICIs. This systematic review was designed to assess the adequacy of methods used to report HRQOL in cancer patients treated with ICIs and to identify the quality of life scales used.

### Method

A systematic review was performed (systematic registration number: PROSPERO: CRD42019121427). A search of the PubMed, PsycINFO, PsycARTICLES, Psychology and Behavioral Sciences collection, and SocINDEX databases was carried out for publications in English and in French. Relevant databases were searched from the earliest records through to March 2019. Publications were selected if they reported on HRQOL in patients with cancer treated by ICIs. Risk of bias was scored using the Cochrane Collaboration bias assessment tool.

### Results

Our search identified 144 publications between 2012 and 2019, of which 15 RCTs met the inclusion criteria. The results suggest that even though the overall reporting of HRQOL was deemed to be of good quality, the data available was marred by methodological aspects such as the lack of HRQOL research hypotheses and the lack of questionnaires validated for cancer patients treated with immunotherapy.

<figure><img src="image_3.png" /></figure>

OPEN ACCESS

**Citation:** Faury S, Foucaud J (2020) Health-related quality of life in cancer patients treated with immune checkpoint inhibitors: A systematic review on reporting of methods in randomized controlled trials. PLoS ONE 15(1): e0227344. https://doi.org/10.1371/journal.pone.0227344

**Editor:** Ali Montazeri, Iranian Institute for Health Sciences Research, ISLAMIC REPUBLIC OF IRAN

**Received:** July 13, 2019

**Accepted:** December 15, 2019

**Published:** January 24, 2020

**Copyright:** © 2020 Faury, Foucaud. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

**Data Availability Statement:** All relevant data are within the manuscript and its Supporting Information files.

**Funding:** Financial support for this research was provided by INCa (SF) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

**Competing interests:** The authors have declared that no competing interests exist.
**Conclusion**

This study provides a comprehensive analysis of the current state of the art and identifies
gaps in knowledge on HRQOL analysis with respect to ICIs. It also suggests avenues for fur-
ther research.

**Introduction**

Cancer immunotherapy has revolutionised the treatment of cancer and represents a new
option for clinicians [1–3]. Immune Checkpoint Inhibitors (ICIs) have garnered attention as
one of the most promising types of immunotherapy [4,5]. The U.S. Food and Drug Adminis-
tration (FDA) approved seven ICIs for clinical use: Ipilimumab, Pembrolizumab, Nivolumab,
Atezolizumab, Durvalumab, Avelumab and Cemiplimab [6].

Checkpoint inhibition is associated with a unique spectrum of side effects known as immune-related adverse events (irAEs), defined as toxicities caused by non-specific activation of the immune system [7, 8, 9]. Some irAEs can be serious and even life-threatening, and thus substantially affect patients' daily lives [10]. Abdel-Rahman et al. (2018) performed a systematic review to explore time to HRQOL deterioration among cancer patients receiving PD-(L)1 inhibitors compared to those receiving traditional cytotoxic therapy. The results showed that PD-(L)1 inhibitors were associated with a consistent prolongation of the time to symptomatic deterioration [11]. Recently, Hall et al. (2019), in their systematic review, suggested that ICIs are well tolerated in terms of HRQOL compared to other anticancer therapies. However, for these authors, currently used instruments may fail to capture important symptomatology unique to ICIs [12]. For example, in the CheckMate141, trial of Nivolumab versus therapy of investigator's choice in recurrent or metastatic head and neck, the QLQ-H&N35 was applied to evaluate HRQOL. But skin toxicities (e.g. o, L er). QOL e prominent role as important end-points in cancer RCTs.atuc review to i...ndart therapy arm (t not be covered.contsuchrash and pruritus), the most frequent irAEs with Nivolumab [8,13], are not assessed with this HRQOL measure [13]. Results of Hall et al. (2019) should therefore be interpreted with caution.

To date, HRQOL refers to a multidimensional concept including the domains of physical,
emotional and social functioning [14]. Assessing HRQOL in cancer patients is necessary to
evaluate the full impact of the cancer experience [15] and to improve understanding of how
different therapeutic interventions influence patients' outcomes [16]. HRQOL is increasingly
recognized as an essential end point in cancer clinical trials [17]. HRQOL outcomes from
RCTs increasingly inform cost-effectiveness analyses used by policymakers to decide on the
allocation of resources [18,19]. Therefore, it is imperative that findings from RCTs are robust.

HRQOL evaluations are made using standardised and validated self-assessment methods
[20]. The most important methodological issue to consider in evaluating HRQOL endpoints
in an oncology clinical trial is the selection of appropriate outcome measurements [21]. There
is also evidence that the potentially invaluable insights that HRQOL data provide on the treat-
ment and care of patients may not be adequately reported [19,22–24].

To our knowledge, the reporting of quality-of-life methods in trials of cancer patients treated with ICIs has not been systematically assessed. The following systematic review aims to evaluate the quality of HRQOL methodological assessment used to evaluate the effects of ICIs on HRQOL in randomised controlled trials (RCTs) to determine how improvements can be made, and to explore the value added by these measures to clinical decision-making in a trial
setting. We focused on RCTs they represent the gold standard for evaluating new medical treatments, developing decision-making policies, and planning new treatment approaches [25,26].

## Method

In accordance with French regulations, this study was exempted from IRB approval. The methods discussed in this review were previously published in the PROSPERO database [27], under number CRD42019121427. (http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42019121427).

## Search strategy

We used standard procedures: Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [28,29], in accordance with the principles outlined in the Cochrane Handbook for Systematic Reviews of Interventions [30]. A systematic literature search was performed from 1st March 2019; we searched for all RCTs published up to 28th February 2019, irrespective of their start or completion date. We searched PubMed, PsycINFO, PsycARTICLES, Psychology and Behavioral Sciences collection, SocINDEX, limiting the search to RCTs of adults (≥ 18 years at diagnosis) published in English or French, and confined to original human studies published in peer-reviewed journals. Companion papers that focused only on HRQOL were reviewed with the original publication.

We identified relevant references using the following terms: “quality of life”, “cancer”, “neoplasms” and individual drug names. The reference lists of key papers were checked to find relevant references for inclusion. Note that, when we started our research (March 2019), the Cemiplimab had been recently approved by the FDA (September 2018). It was not specifically examined in our search strategy due to the lack of data on HRQOL available in March 2019 for this ICIs (clinicaltrials.gov).

The full search algorithm, based on Hall et al. [12], used to identify potential studies in PubMed is included in S1 Search algorithm and was adapted for the other databases.

## Selection criteria

The patient population assessed included patients with any cancer randomised to treatment with an approved ICI. Predefined exclusion criteria were: RCTs that assessed treatment that did not include ICIs, RCTs on patients with other illnesses, phase I studies, and RCTs with fewer than ten patients per group. As a single domain (e.g. fatigue) is not considered as HRQOL, trials assessing only one aspect of HRQOL were excluded. Companion papers focusing on a subgroup of the total RCT sample were excluded. Unpublished reports, conference abstracts, and dissertations were excluded due to the lack of peer-review oversight. These are often subsequently published in peer-reviewed journals, risking duplication.

## Data extraction

An extensive search of electronic databases was conducted by one person (SF). Literature search results were uploaded to Zotero (www.zotero.org) which facilitates bibliographic source management. Following the removal of duplicates, two reviewers (SF and JF) independently screened titles and abstracts of all references identified by the search according to eligibility criteria. Full articles were obtained for all titles that either appeared to meet the inclusion criteria or where uncertainty as to the eligibility criteria existed. A final selection pertaining to the
suitability of the full-text papers to be analysed was verified by two reviewers (SF and JF). A
third reviewer (CC) was available as a mediator in the event of disagreement.

Data analysis

The criteria used to evaluate quality of reporting on HRQOL were based on those proposed by Efficace et al. [25]. On the basis of good practice in reporting HRQOL [24, 31–34], Efficace et al. [25] extracted 11 basic and essential issues that a given trial should report to reach methodologically sound outcomes. The checklist items were devised to have a dichotomous answer: these can be scored as ‘yes’ (giving a score of 1) or ‘no’ (giving a score of 0), with higher scores indicating the robustness of the outcomes. The items included in the checklist are self-explanatory and a brief description is also provided in S1 Table.

Determining study quality

Risk of bias was assessed by two reviewers using the Cochrane Collaboration bias assessment tool [35]. Responses in each domain (random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data and selective outcome reporting) were assessed as having a ‘low’, ‘unclear’ or ‘high’ risk of bias.

Results

Key characteristics of identified trials

The literature search yielded 144 records. After removal of duplicates, the titles and abstracts of 138 records were screened; 109 records were identified as clearly non-relevant and consequently excluded: full-text articles were thus obtained for 29 papers. Fourteen of these publications were excluded as not meeting the selection criteria (e.g. lack of HRQOL endpoint, protocol summary, RCTs, specific subgroup of RCTs). This gave a total of 15 publications to be included in the systematic review (Fig 1). Publications covered a seven-year period (2012–2019). The key demographic characteristic results from the 15 RCTs included in this systematic review are summarised in Table 1.

Nine of the 15 RCTs were published in high-impact journals (> 10). For 13 of the 15 identi-
fied RCTs, the HRQOL data were published in a companion paper. Results of HRQOL data
are not presented in the main papers of these 13 RCTs [36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46,
47, 48]. Eight of the thirteen HRQOL companion papers were published in the year after publi-
cation of the main clinical results, four within two years, and one within three years. All trials
were industry-sponsored and/or affiliated with the pharmaceutical industry through one or
more investigators, as noted in the financial sponsorship statements of the articles. All trials
were multicentre trials and performed in more than one country. In total, 9332 patients were
enrolled in the 15 RCT trials. Trial size ranged from 272 to 951 patients. The ICIs studied in
these RCTs were Ipilimumab (33%), Pembrolizumab (27%), Atezolizumab (7%) and Nivolu-
mab (47%). No relevant RCTs with HRQOL assessment focused on Durvalumab or Avelumab.
The non-checkpoint inhibitor controls used in some of the studies included conventional
treatment, chemotherapy, sunitinib, placebo, or everolimus, and gp100. Eight RCTs assessed
ICI treatments for melanoma, three for lung cancer, one for head and neck squamous cell car-
cinoma, one for urothelial cancer, and two for advanced renal cell carcinoma. All trials had
overall survival or progression-free survival as the primary endpoint; all reported significant
differences in one of these endpoints between treatment arms. HRQOL was a secondary
endpoint in nine trials [49,50,51,52,53,54,55,56,57], an exploratory endpoint in five RCTs
[58,59,60,61,62] and for one trial it was unclear if the HRQOL endpoints were secondary or
<figure><img src="image_4.png" /><figcaption>**Fig 1.** Selection strategy-study inclusion and exclusion flowchart. HRQOL = health-related quality of life. RCTs = randomized controlled trials. ICIs = immune checkpoint inhibitors. * = some RCTs included more than one ICI.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0227344.g001

exploratory [63]. Seven trials discussed HRQOL in their main publication [50,59,60,52,61, 56,57].

## Quality of HRQOL measurements

The conceptual measurement and methodology for evaluating HRQOL outcomes from the 15 RCTs included in this systematic review are summarised in Table 2 and S2 Table. HRQOL was
<table><caption>Table 1. Key characteristics of 15 RCTs between 2012 and 2019.</caption><thead><tr><th rowspan="2"></th><th colspan="2">Number of RCTs</th></tr><tr><th>n</th><th>%</th></tr></thead><tbody><tr><td>Total number of reviewed RCTs</td><td>15</td><td>100</td></tr><tr><td>Number with HRQOL as primary endpoint</td><td>0</td><td>0</td></tr><tr><td>Number with HRQOL as secondary endpoint</td><td>9</td><td>60</td></tr><tr><td>Number with HRQOL as exploratory endpoint</td><td>5</td><td>33</td></tr><tr><td>Number with additional HRQOL publication</td><td>13</td><td>87</td></tr><tr><td>Number discussing HRQOL in the main publication</td><td>7</td><td>47</td></tr><tr><td>Multi-country locations</td><td>15</td><td>100</td></tr><tr><td>Industry funded</td><td>15</td><td>100</td></tr><tr><td>Type of ICI*</td><td></td><td></td></tr><tr><td>Atezolizumab</td><td>1</td><td>7</td></tr><tr><td>Avelumab</td><td>0</td><td>0</td></tr><tr><td>Durvalumab</td><td>0</td><td>0</td></tr><tr><td>Ipilimumab</td><td>5</td><td>33</td></tr><tr><td>Ipilimumab + Nivolumab</td><td>2</td><td>13</td></tr><tr><td>Nivolumab</td><td>7</td><td>47</td></tr><tr><td>Pembrolizumab</td><td>4</td><td>27</td></tr><tr><td>Non-checkpoint inhibitor controls</td><td>12</td><td>80</td></tr><tr><td>Chemotherapy</td><td>8</td><td>53</td></tr><tr><td>Placebo</td><td>1</td><td>7</td></tr><tr><td>Everolimus</td><td>1</td><td>7</td></tr><tr><td>GP100</td><td>1</td><td>7</td></tr><tr><td>Sunitinib</td><td>1</td><td>7</td></tr><tr><td>Type of cancer</td><td></td><td></td></tr><tr><td>Melanoma</td><td>8</td><td>53</td></tr><tr><td>Non-Small Cell Lung Cancer</td><td>3</td><td>20</td></tr><tr><td>Squamous Cell Carcinoma of the Head and Neck</td><td>1</td><td>7</td></tr><tr><td>Advanced renal cell carcinoma</td><td>2</td><td>13</td></tr><tr><td>Urothelial cancer</td><td>1</td><td>7</td></tr><tr><td>Sample size</td><td></td><td></td></tr><tr><td>272 to 500</td><td>5</td><td>33</td></tr><tr><td>501 to 800</td><td>5</td><td>33</td></tr><tr><td>801 to 951</td><td>5</td><td>33</td></tr><tr><td>Peer-reviewed clinical journals</td><td></td><td></td></tr><tr><td>N Engl J Med (IF = 72.406)</td><td>1</td><td>7</td></tr><tr><td>Lancet Oncol (IF = 33.900)</td><td>5</td><td>33</td></tr><tr><td>J Clin Oncol (IF = 26.303)</td><td>2</td><td>13</td></tr><tr><td>Ann Oncol (IF = 11.855)</td><td>1</td><td>7</td></tr><tr><td>J Thorac Oncol (IF = 6.595)</td><td>1</td><td>7</td></tr><tr><td>Eur J Cancer (IF = 6.029)</td><td>3</td><td>20</td></tr><tr><td>Clin Lung Cancer (IF = 3.66)</td><td>1</td><td>7</td></tr><tr><td>Health Qual Life Outcomes (IF = 2.143)</td><td>1</td><td>7</td></tr></tbody></table>

RCTs = randomised controlled trials. HRQOL = health-related quality of life. ICI = immune checkpoint inhibitors.

* = Some RCTs used more than one ICI.

https://doi.org/10.1371/journal.pone.0227344.t001
<table><caption>**Table 2.** Level of reporting according to the minimum standard checklist for evaluating HRQOL outcomes in cancer clinical trials.</caption><thead><tr><th rowspan="2">HRQOL Issue</th><th colspan="2">Reports</th></tr><tr><th>No.*</th><th>%</th></tr></thead><tbody><tr><td>Conceptual</td><td></td><td></td></tr><tr><td>A priori hypothesis stated</td><td>5/10</td><td>50</td></tr><tr><td>Rationale for instrument reported</td><td>13/15</td><td>87</td></tr><tr><td>Measurement</td><td></td><td></td></tr><tr><td>Psychometric properties reported</td><td>14/15</td><td>93</td></tr><tr><td>Cultural validity verified</td><td>4/15</td><td>27</td></tr><tr><td>Adequacy of domains covered</td><td>0/15</td><td>0</td></tr><tr><td>Methodology</td><td></td><td></td></tr><tr><td>Instrument administration reported</td><td>8/15</td><td>53</td></tr><tr><td>Baseline compliance reported</td><td>14/15</td><td>93</td></tr><tr><td>Timing of assessment documented</td><td>15/15</td><td>100</td></tr><tr><td>Missing data documented</td><td>15/15</td><td>100</td></tr><tr><td>Interpretation</td><td></td><td></td></tr><tr><td>Clinical significance addressed</td><td>13/15</td><td>87</td></tr><tr><td>Presentation of results in general</td><td>13/15</td><td>87</td></tr></tbody></table>

HRQOL: Health-Related Quality Of Life.

* Number of articles reporting item/number of articles to which item is applicable.

https://doi.org/10.1371/journal.pone.0227344.t002

measured with common cancer HRQOL assessment tools in all of the studies. The European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire (EORTC QLQ-C30, used in twelve of the 15 RCTs), and the European Quality of Life 5 Dimension (EQ-5D, used in eleven of the 15 trials) were the most frequently used. In two of the twelve studies, the EORTC QLQ-C30 was supplemented with the EORTC Quality of Life 13-item Lung Cancer-specific Questionnaire (EORTC QLQ-LC13), and, in another, with the EORTC module for Head & Neck cancer (EORTC QLQ-H&N35). Additional HRQOL instruments used included the Functional Assessment of Cancer Therapy (FKSI-DRS or FΚSI-19, used in two trials), the Functional Assessment of Cancer Therapy-General (FACT-G, used in one trial), and the Lung Cancer Symptom Scale (LCSS, used in one trial). Twelve trials used more than one HRQOL questionnaire.

Five of the 15 RCTs did not specify the HRQOL research hypothesis due to the HRQOL endpoint being an exploratory objective [58,59,60,61,62]. For Efficace et al. [25], to satisfy the "a priori hypothesis stated" criterion, the study needed to predefine HRQOL end point and/or state expected changes because of the specific treatment. Only five trials of the 15 RCTs satisfied this criterion but none of them included an *a priori* hypothesis for the expected HRQOL outcomes [49,50,51,54,55].

Two of the 15 studies failed to report their reasoning for using a specific HRQOL measure and none gave a reason for choosing a particular HRQOL instrument. For 14 trials, the validity and reliability ("psychometric properties reported" criterion) of the instruments used was reported by referencing the appropriate validation studies; one trial provided no reference to validity or reliability [62]. All but one of the questionnaires (EQ-5D) were validated for the specific cancer population ("Cultural validity verified" criterion). The EQ-5D is a non-cancer-specific measure of generic health for clinical and economic appraisal. None of the questionnaires used in these 15 trials (EORTC questionnaires, EQ-5D, LCSS, FΚSI, FACT-G, etc.) were validated among cancer patients treated with ICIs ("adequacy of domains covered"
criterion). Checkpoint inhibition is associated with a unique spectrum of side effects that are not assessed with the HRQOL measure used. The HRQOL domains covered by the questionnaires were inadequate for several of the trials identified. Details describing how the HRQOL assessment was done were often not reported. Some details were noted for only eight of the trials (“instrument administration reported” criterion). All the RCTs reported the timing of HRQOL assessment and documented missing data.

We observed that presentation of the HRQOL results was adequate for all but two of the trials. For these two trial presentations of HRQOL, the results were defined as limited (main publication of RCTs), with incomplete HRQOL details and a discussion of the HRQOL outcomes in terms of clinical significance not reported. HRQOL data was judged as high quality if at least eight of the 11 criteria were satisfied; furthermore, three of the eight or more criteria satisfied needed to cover the three high-priority concerns identified by the experts (“baseline compliance reported,” “psychometric properties reported,” and “missing data documented”). If one or two of the 11 items on the checklist were assessed as not applicable, the cutoff was set at seven or six criteria respectively (although the three mandatory criteria still had to be met). Only for ten trials of the 15 RCTs HRQOL data was judged as high quality [49, 58, 50, 59, 51, 60, 52, 54, 55, 63]. The others trials were considered to have some possible reported methodological limitations because of studies addressed (1) at least eight (or seven if one of the 11 items on the checklist were assessed as not applicable) issues but did not take into account the mandatory items [52, 61, 62]; (2) fewer than eight (or seven) issues [56, 57].

## Risk of bias

Fig 2 summarises the risk of bias for all of the RCTs evaluated. All the RCTs used block randomisation and/or stratified randomisation to generate the random sequence and were assessed as having a ‘low’ risk of bias for random sequence generations. For ten RCTs, participants were randomly assigned by an interactive voice response system. These trials were assessed as having a ‘low’ risk of bias for allocation concealment. The five other studies were assessed as having an ‘unclear’ risk of bias for this item. Ten RCTs were open-label (trial in which both the researchers/investigators and participants were not blinded to treatment allocation). This procedure could affect patients’ perceptions of HRQOL. For these ten open-label RCTs, the blinding of outcome assessment and blinding of participants and personnel were assessed as having a ‘high’ risk of bias. The five other RCTs were double-blind trials and were assessed as having a ‘low’ risk of bias for these two items. Four RCTs were assessed as ‘low’ risk of bias and ten as having an ‘unclear’ risk of bias for attrition criteria. For twelve RCTs, pre-specified HRQOL outcomes were reported and were assessed as having a ‘low’ risk of bias for reporting bias. The key reason for the ‘unclear’ risk of bias assessment was the lack of method details.

## Discussion

Introduced in the past ten years, immunotherapy is starting to revolutionize the treatment of cancer. ICIs are increasingly used to treat a variety of cancers, but little is known about HRQOL. Hall et al. (2019) [10] performed a systematic review to examine HRQOL among cancer patients receiving ICIs as compared to other anticancer therapies: they did not analyse investigate the HRQOL assessment methods used in RCTs on cancer patients treated with the ICIs and it is the purpose of our work.

Fifteen RCTs meeting the inclusion criteria were identified in the literature. None of the identified RCTs studied Durvalumab or Avelumab HRQOL, probably as a result of delay in publishing data. Our analysis showed that all included RCTs were international, multicenter and supported by commercial sponsors. The primary endpoint of all of the RCTs was efficacy
<table><thead><tr><th></th><th>Random sequence generation*</th><th>Allocation concealment†</th><th>Blinding of participants and personnel§</th><th>Blinding of outcome assessment ¶</th><th>Incomplete outcome data#</th><th>Selective reporting ¥</th></tr></thead><tbody><tr><td>Bordoni 2018</td><td>+</td><td>+</td><td>-</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Rittmeyer 2017</td><td>+</td><td>+</td><td>-</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Brahmer 2017</td><td>+</td><td>+</td><td>-</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Reck 2016</td><td>+</td><td>+</td><td>-</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Cella 2016</td><td>+</td><td>+</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Motzer 2015</td><td>+</td><td>+</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Cella 2019</td><td>+</td><td>+</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Motzer 2018</td><td>+</td><td>+</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Coens 2017</td><td>+</td><td>+</td><td>+</td><td>+</td><td>?</td><td>+</td></tr><tr><td>Eggermont 2015</td><td>+</td><td>+</td><td>+</td><td>+</td><td>?</td><td>+</td></tr><tr><td>Harrington 2017</td><td>+</td><td>+</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Ferris 2016</td><td>+</td><td>+</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Larkin 2018</td><td>+</td><td>+</td><td>-</td><td>-</td><td>-</td><td>?</td></tr><tr><td>Weber 2015</td><td>+</td><td>+</td><td>-</td><td>-</td><td>-</td><td>?</td></tr><tr><td>Long 2016</td><td>+</td><td>+</td><td>+</td><td>+</td><td>?</td><td>+</td></tr><tr><td>Robert 2015</td><td>+</td><td>+</td><td>+</td><td>+</td><td>?</td><td>+</td></tr><tr><td>Petrella 2017</td><td>+</td><td>?</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Robert 2015</td><td>+</td><td>?</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Reck 2018</td><td>+</td><td>?</td><td>-</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Brahmer 2015</td><td>+</td><td>?</td><td>-</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Revicki 2012</td><td>+</td><td>?</td><td>+</td><td>+</td><td>?</td><td>?</td></tr><tr><td>Hodi 2010</td><td>+</td><td>?</td><td>+</td><td>+</td><td>?</td><td>?</td></tr><tr><td>Schadendorf 2016</td><td>+</td><td>+</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Ribas 2015</td><td>+</td><td>+</td><td>-</td><td>-</td><td>?</td><td>+</td></tr><tr><td>Schadendorf 2017</td><td>+</td><td>+</td><td>+</td><td>+</td><td>?</td><td>+</td></tr><tr><td>Larkin 2015</td><td>+</td><td>+</td><td>+</td><td>+</td><td>?</td><td>+</td></tr><tr><td>Vaughn 2018</td><td>+</td><td>?</td><td>-</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Bellmunt 2017</td><td>+</td><td>?</td><td>-</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Weber 2017</td><td>+</td><td>?</td><td>+</td><td>+</td><td>?</td><td>?</td></tr><tr><td>Same</td><td>+</td><td>?</td><td>+</td><td>+</td><td>?</td><td>?</td></tr></tbody></table>

**Fig 2. Risk of bias.** + = low risk of bias. - = high risk of bias. ? = unclear risk of bias. * = bias due to inadequate generation of a randomised sequence. † = bias due to inadequate concealment of allocations before assignment. § = bias due to knowledge of the allocated interventions by participants and personnel during study. ¶ = detection bias due to knowledge of the allocated interventions by outcome assessment. # = due to amount, nature or handling of incomplete outcome data. ¥ = bias due to selective outcome reporting.

https://doi.org/10.1371/journal.pone.0227344.g002
and safety, using progression-free survival or overall survival. HRQOL was identified as a sec-
ondary, or even exploratory, endpoint. Overall, a recent systematic review [64] about quality
of patient-reported outcome (PRO) reporting across 557 cancer RCTs showed that PRO were
secondary endpoints in 421 RCTs (76%). About HRQOL endpoint, RCTs on cancer patients
treated with ICIs are in line with RCTs across cancer patients. For most of the RCTs, the
HRQOL data were published in a companion paper in a peer-reviewed journal. Most of the
companion papers were published within two years after publication of the main clinical
results. The number of additional HRQOL papers is significant and indicative of interest not
only for the efficacy of new drugs, but also for the patients' experience.

Previous studies [11,12] showed that patients receiving ICIs experience similar-to-improve
HRQOL compared to patients receiving other treatments for advanced cancer (such as chemo-
therapy, placebo, everolimus or gp100). Based on our findings, these results must be inter-
preted with caution in view of a number of potential weaknesses associated with the RCTs
included.

First, as there are no international guidelines concerning the statistical analysis of HRQOL data in cancer RCTs [65], a clear ‘*a priori* hypothesis’ is needed to develop an analysis plan [20]. In Efficace et al. (2015) systematic review [64], about PRO reporting across 557 cancer RCTs, the three items most frequently omitted from reports were (1) the statement of a PRO hypothesis, (2) the description of statistical approaches for dealing with missing PRO data, and (3) the documentation of methods for PRO data collection, which were reported in only 17%, 20%, and 24% of studies, respectively. In line with the results of Efficace et al. (2015), the ‘stated *a priori* hypothesis’ is the most important HRQOL assessment method that needs to be implemented in RCTs about patients treated with ICIs. Indeed, none of the trials included an *a priori* hypothesis for expected HRQOL outcomes. As already mentioned [10,20,22,25], defining and reporting a hypothesis is an essential requirement for an appropriate study design. The choice of HRQOL questionnaire and statistical analysis in RCTs is dependent on the objective. A clear objective and specific hypotheses improve the analysis and credibility of the results [10]. The lack of hypotheses in many RCTs might partly be due to the exploratory nature of the analysis of HRQOL. Failure to pre-specify hypotheses might generate spurious HRQOL results, potentially emphasising findings that are statistically, but not clinically, relevant [25]. Consequently, we should be careful to avoid interpreting the HRQOL results of previous studies [11,12] as if based on a confirmatory analysis.

Second, as the choice of instrument affects reporting, analysis and interpretation of the trial
findings deserve adequate justification [20]. In theory, both generic and disease-specific ques-
tionnaires may be used for a given condition. In practice, it is very important to choose the
most suitable questionnaire to explore the domains relevant to the treatment(s) [66]. In our
systematic review, while the characteristics of the HRQOL instrument used were presented for
all trials but one, none of them justified their choice of a particular HRQOL instrument. As the
questionnaires used in the RCTs included are considered by researchers to be the standard for
use in cancer RCTs, they are often reported without a justification for their particular selection.
The lack of a stated *a priori* hypothesis plays a part in the choice of the instrument as justifica-
tion for the use of a specific method should depend mainly on the hypothesis being examined
[20]. All but one of the questionnaires (EQ-5D) were validated for the specific cancer popula-
tion. Nevertheless, none of these, or their clinically meaningful cut-off, were validated for can-
cer patients treated with immunotherapy. This is of the utmost importance as ICIs have a
different toxicity profile to chemotherapy [20]. For example, skin toxicities (e.g. o, L er). QOL
e prominent role as important endpoints in cancer RCTs.atuc review to i...ndart therapy arm
(t not be covered.contsuchrash and pruritus) are the most frequent irAEs with anti-PD1
[8,13], but skin problems are not assessed with QLQ-H&N35. Consequently, HRQOL might
be affected by ICIs in domains that are not covered by standard instruments, whereas side effects of chemotherapy (for control groups of RCTs) are covered by these standard instru- ments. Moreover, for the QLQ-C30, for example, although mean score changes of 10 points are widely viewed as clinically significant [67], minimally important differences as low as 4 points have been reported in other cancer trials [68,69]. When interpreting the trial results, it should be kept in mind that important HRQOL issues might not be covered [70].

Furthermore, quality of life may be defined as an individual's self-perceived satisfaction at any stage of life [71]. Since the disease and cure are associated with symptoms and side effects, the patient learns to adapt to them [72]. This may change the patient's internal standard of assessment. Such changes indicate the phenomenon of 'response shift' [73,74]. Thus, measure- ment of HRQOL changes may be biased [75-77]. None of the RCTs measured this bias, although different methods can be used to measure the phenomenon (e.g., Then-test method, Ideal Scale approach, Schedule for the Evaluation of Individual Quality of life).

Moreover, human behaviour is influenced by what we know or believe. The common sense model of self-regulation of health and illness (CMS) proposed by Leventhal (1980) [78] postu- lates that individual perceptions of health threats, and the ensuing emotional response, guide peoples' coping responses. CSM classifies illness representations into distinct dimensions: identity, cause, timeline, consequences, personal control, treatment control and illness coher- ence. In their systematic review and meta-analysis, Richardson et al. demonstrated that lower identity scores (referring to the description of the health threat and its symptoms) were associ- ated with better quality of life [79]. In research, there is a risk of expectations influencing find- ings, most obviously when there is some subjectivity in the assessment, as in HRQOL, leading to biased results. To eliminate such bias, patients and assessors need to be blinded to treatment allocation [80]. In our systematic review, ten RCTs were open-label, and patients' perception of their quality of life could have been influenced by their knowledge of the treatment received. Most RCTs used centralised treatment allocation (interactive voice response system) and reported that the method adopted to stratify patients can help prevent potential treatment imbalances. However, instrument administration was rarely reported (in accordance with the results of Efficace et al. (2015) systematic review [64] about RCTs across cancer patients), mak- ing it hard to determine whether open-label trials could have influenced the responses to the questionnaires.

Finally, mains weaknesses associated with HRQOL in RCTs about patients treated with ICIs are the same as those described in cancer RCTs (the statement of a PRO hypothesis, the documentation of methods for PRO data collection, etc.). However, in immunotherapy RCTs, the “adequacy of domains covered” criterion is no satisfy because none of the questionnaire used in these RCTs covered, at least, the main HRQOL dimensions relevant according to the specific research question, namely irAEs. There is a need to develop standard methods for evaluating HRQOL for patients treated with ICIs. But HRQOL and irAEs are different accord- ing to cancer type and/or ICIs administered. We propose that HRQOL about patients treated with ICIs is assessed with the common cancer HRQOL assessment tools (like QLQ-C30, FACT-G), a specific module according to the type of cancer (as QLQ-LC13, QLQ-H&N35, LCSS) and a specific module for evaluate irAEs according to ICIs administered. At present, in the absence of HRQOL tool for patients treated with ICIs and for studying HRQOL more rigorously:

1. we propose that HRQOL, in RCTs about patients treated with ICIs, was a primary endpoint not secondary or exploratory. Indeed, HRQOL endpoints are increasingly being used in cancer RCTs but often in secondary endpoint [64,81]. However, regardless of cancer type,
quality of reporting was typically higher in RCTs where HRQOL were primary endpoints
[81];

2. As already mentioned, in our research, none of RCTs included an *a priori* hypotheses for the expected HRQOL outcomes whereas a clear hypothesis is an essential requirement for an appropriate study design, the choice of the instrument, etc. [20, 9, 22, 25]. So, in future RCTs about patients treated with ICIs a hypothesis must be included;

3. In our research, HRQOL data was judged as high quality in ten RCTS. Because methodo-logical rigor is essential to the conduct and reporting of RCTs [64], some recommendations have been established with the aim to facilitate adherence with key methodological aspects and to increase the transparency and consistency of PRO reporting in RCTs (for example, the Consolidated Standards of Reporting Trials (CONSORT) group published the CON-SORT PRO extension in 2013; the Minimum Standard Checklist for evaluating HRQOL Outcomes in Cancer Clinical Trials by Efficace et al. [25]). In cancer RCTs, the overall level of reporting according to the new CONSORT PRO extension was poor. However, adher-ence to the CONSORT PRO extension was higher in RCTs in which PROs were included as primary endpoints versus RCTs with PROs as secondary outcomes [65]. All cancer RCTs, including RCTs with patients treated with ICIs, must implement of the CONSRT PRO extension for example;

4. Patient-generated outcome measures have been developed in an effort to capture the indi-vidualistic nature of HRQOL. These measures differ from traditional HRQOL instruments in that they allow patients to individually define HRQOL domains or weights [82]. Differ-ent tools are available: Schedule for the Evaluation of Individual Quality of Life (SEIQoL), Repertory Grid, and Asthma Quality of Life Questionnaire (AQLQ) or Patient-Generated Index (PGI). For Patel et al., (2003), patient-generated outcome measures appear to be use-ful primarily in complementing traditional HRQOL measures, guiding individual patient treatment decisions, and assisting the design of new measures [82]. For example, the PGI is designed to both ask and document HRQOL concerns [83]. The PGI consists of three stages in which patients: i) self-identify the most important areas or activities of their lives affected by their condition; ii) score the degree to which each area is affected; and iii) allocate points among the items listed to represent the amount in which they would like each area improved [84]. A recent study provides evidence that the PGI would be a good measure for cancer patients and clinicians to use together [83]. Tang et al. (2014) systematic review aims to identify current literature in which PGI has been used as a tool to assess quality of life in cancer patients [84]. An overarching theme observed in these studies highlighted the con-cerns mentioned by patients that were not targeted or detected by standardized quality of life measures [84]. In HRQOL RCTs about patients treated with ICIs, patient-generated outcomes measures could be used to identify areas of cancer which are not evaluated by common cancer tools and that require attention and monitor changing needs

Study limitations

Our systematic review has several limitations. We focused only on RCTs published in French and English, but as the most important RCTs tend to be published in English journals, we believe that we have included the most significant trials. Another limitation is that we based our work on the information that the clinical community could access and only with the litera-ture that reported publishing HRQOL. Unpublished RCTs and RCTs including HRQOL end-point by design, but that had not published their results, protocols, or statistical analysis plans,
were excluded. Our review was therefore subject to publication bias. Furthermore, though we used subjective judgments for several assessment criteria, each criterion was independently assessed by two reviewers under the supervision of a third researcher. We therefore believe that our results are reproducible and consistent. Lastly, since checkpoint inhibition is associated with a unique spectrum of side effects that are not assessed with the HRQOL measure used in the RCTs identified, the HRQOL domains covered by these questionnaires were not adequate for all the trials. As such, it is difficult to explore the value added by these measures to clinical decision-making in a trial setting.

## Conclusion

To the best of our knowledge, we conducted one of the first systematic reviews aimed at investigating the HRQOL assessment methods used in RCTs for cancer patients treated with ICIs. HRQOL quantifies how a patient feels or functions, provides an economic basis for decision-making and contributes to clinical decision-making. As such, HRQOL is playing a more prominent role as an important endpoint in cancer clinical trials. The quality of the measurements and interpretation of results is crucial. Previous findings suggest that ICIs maintain HRQOL compared with standard treatments. However, these results must be interpreted with caution in view of a number of potential weaknesses associated with the RCTs included (such as open-label trials, under-reporting of HRQOL research hypotheses, instruments not validated for patients treated with immunotherapy) and insufficient guidance, as well as the lack of internationally recognised standard methods for analysing and reporting HRQOL. Hence, there is a strong need to develop standard methods for evaluating HRQOL for patients treated with ICIs. A better methodology would lead to a more efficient understanding of HRQOL outcomes.

## Supporting information

### S1 Prisma checklist.

(DOC)

### S1 Search algorithm.

(DOCX)

### S1 Table. Minimum Standard Checklist for evaluating HRQOL Outcomes in Cancer Clinical Trials.
* If a study explicitly states an exploratory HRQOL evaluation. ¶ If the HRQOL measure is validated in the same population as the one of the trial.

(DOCX)

### S2 Table. Assessment methods for HRQOL.
HRQOL = health-related quality of life. EORTC QLQ-C30 = European Organisation for Research and Treatment of Cancer Core (EORTC) Quality of Life Questionnaire. EQ-5D = European Quality of Life 5 Dimension. EORTC QLQ-LC13 = EORTC Quality of Life 13-item Lung Cancer-specific Questionnaire. EORTC QLQ-H&N35 = EORTC module for Head & Neck cancer. LCSS = Lung Cancer Symptom Scale. FACT-G = Functional Assessment of Cancer Therapy-General. N/A = Not Available.

(DOCX)

## Acknowledgments

We would like to acknowledge Norbert Ifrah, President of the French National Cancer Institute and Christine Chomienne (CC) director of the research of the French National Cancer Institute, for their advices and for reviewing the draft.
## Author Contributions

**Conceptualization:** Stéphane Faury, Jérôme Foucaud.

**Data curation:** Stéphane Faury, Jérôme Foucaud.

**Formal analysis:** Stéphane Faury, Jérôme Foucaud.

**Funding acquisition:** Stéphane Faury.

**Investigation:** Stéphane Faury, Jérôme Foucaud.

**Methodology:** Stéphane Faury, Jérôme Foucaud.

**Resources:** Stéphane Faury.

**Validation:** Stéphane Faury, Jérôme Foucaud.

**Writing – original draft:** Stéphane Faury.

**Writing – review & editing:** Stéphane Faury, Jérôme Foucaud.

## References

1. Gálffy G, Puskás R. Role of pembrolizumab in the treatment of non-small cell lung cancer (NSCLC). Magy Onkol. 2017; 61(1): 107–110. PMID: 28273195

2. Emens LA, Ascierto PA, Darcy PK, Demaria S, Eggermont AMM, Redmond WL, et al. Cancer immunotherapy: opportunities and challenges in the rapidly evolving clinical landscape. Eur J Cancer 2017; 81: 116–129. https://doi.org/10.1016/j.ejca.2017.01.035 PMID: 28623775

3. Morrissey KM, Yuraszeck TM, Li CC, Zhang Y, Kasichayanula S. Immunotherapy and novel combinations in Oncology: current Landscape; challenges and opportunities. Clin Transl Sci. 2016; 9(2): 89–104. https://doi.org/10.1111/cts.12391 PMID: 26924066

4. Dine J, Gordon R, Shames Y, Kasler MK, Barton-Burke M. Immune Checkpoint Inhibitors: An innovation on Immunotherapy for the Treatment and Management of Patients with Cancer. Asia Pac J Oncol Nurs. 2017; 4(2): 127–135. https://doi.org/10.4103/apjon.apjon_4_17 PMID: 28503645

5. Topalian SL, Hodi FS, Brahmer JR, Gettinger SN, Smith DC, McDermott DF, et al. Safety, activity, and immune correlates of anti-PD-1 antibody in cancer. N Engl J Med. 2012; 366(26): 2443–2454. https://doi.org/10.1056/NEJMoa1200690 PMID: 22658127

6. Administration, U.S.F.a.D. Drugs@FDA: FDA approved drug products. 2018 february 28th; Available from: https://www.accessdata.fda.gov/scripts/cder/daf/.

7. Puzanov I, Diab A, Abdallah K, Bingham CO, Brogdon C, Dadu R, et al. Managing toxicities associated with immune checkpoint inhibitors: consensus recommendations from the Society for Immunotherapy of Cancer (SITC) Toxicity Management Working Group. J Immunither Cancer 2017; 5(1): 95. https://doi.org/10.1186/s40425-017-0300-z PMID: 29162153

8. Voudouri D, Nikolalaou V, Laschos K, Charpidou A, Soupos N, Triantafyllopoulou, et al. Anti-PD1/PDL1 induced psoriasis. Curr Probl Cancer 2017; 41(6): 407–412. https://doi.org/10.1016/j.currproblcancer.2017.10.003 PMID: 29096940

9. Brahmer J, Lacchetti C, Schneider BJ, Atkins MB, Brassil KJ, Caterino JM, et al. Management of Immune-Related Adverse Events in patients treated with immune checkpoint inhibitor therapy: American Society of Clinical Oncology Clinical Practice Guideline. J Clin Oncol. 2018; 36(17): 1714–1768. https://doi.org/10.1200/JCO.2017.77.6385 PMID: 29442540

10. Ghislain I, Zikos E, Coens C, Quinten C, Balta V, Tryfonidis K, et al. Health-related quality of life in locally advanced and metastatic breast cancer: methodological and clinical issues in randomised controlled trials. Lancet Oncol. 2016; 17(7): 294–304.

11. Abdel-Rahman O, Oweira H, Giryes A. Health-related quality of life in cancer patients treated with PD-(L)1 inhibitors: A systematic review. Expert Rev Anticancer 2018; 18(12): 1231–1239.

12. Hall ET, Singhal S, Dickerson J, Gabster B, Wong HN, Aslakson RA et al. Patient-reported outcomes for cancer patients receiving checkpoint inhibitors: Opportunities for palliative care- A systematic review. JPSM 2019. https://doi.org/10.1016/j.jpainsymman.2019.03.015

13. Ferris RL, Blumenschein G, Fayette J, Guigay J, Colevas AD, Licitra L, et al. Nivolumab for recurrent squamous-Cell carcinoma of the head and neck. N Engl J Med. 2016; 375: 1856–1867. https://doi.org/10.1056/NEJMoa1602252 PMID: 27718784
14. Downing A, Morris EJ, Richards M, Corner J, Wright P, Sebag-Montefiore D, et al. Health-related quality of life after colorectal cancer in England: A patient-reported outcomes study of individuals 12 to 36 months after diagnosis. J Clin Oncol. 2015; 33(6): 616–624. https://doi.org/10.1200/JCO.2014.56.6539 PMID: 25559806

15. Ramsey SD, Andersen MR, Etzioni R, Moinpour C, Peacock S, Potosky A, et al. Quality of life in survivors of colorectal carcinoma. Cancer 2000; 88(6): 1294–1303. PMID: 10717609

16. Kopp I, Bauhofer A, Koller M. Understanding quality of life in patients with colorectal cancer: Comparison of data from a randomised controlled trial, a population based cohort study and the norm reference population. Inflamm Res. 2004; 53 (Suppl 2): 130–135.

17. Bottomley A, Flechtner H, Efficace F, Vanvoorden V, Coens C, Therasse P, et al. Health related quality of life outcomes in cancer clinical trials. Eur J Cancer 2005; 41(12): 1697–1709. https://doi.org/10.1016/j.ejca.2005.05.007 PMID: 16043345

18. Au HJ, Ringash J, Brundage M, Palmer M, Richardson H, Meyer RM, et al. Added value of health-related quality of life measurement in cancer clinical trials: the experience of the NCIC CTG. Expert Rev Pharmacoecon Outcomes Res. 2010; 10(2): 119–128. https://doi.org/10.1586/erp.10.15 PMID: 20384559

19. Fielding S, Ogbuagu A, Sivasubramaniam S, MacLennan G, Ramsay CR. Reporting and dealing with missing quality of life data in RCTs: has the picture changed in the last decade? Qual Life Res.2016; 25 (12): 2977–2983. https://doi.org/10.1007/s11136-016-1411-6 PMID: 27650288

20. Zikos E, Ghislain I, Coens C, Ediebah DE, Sloan E, Quinten C, et al. Health-related quality of life in small-cell lung cancer: a systematic review on reporting of methods and clinical issues in randomised controlled trials. Lancet Oncol. 2014; 15(2): 78–89.

21. Jacobsen PB, Davis K, Cella D. Assessing quality of life in research and clinical practice. Oncology 2002; 16 (9 Suppl 10): 133–139.

22. Bottomley A, Therasse P. Quality of life in patients undergoing systemic therapy for advanced breast cancer. Lancet Oncol. 2002; 3(10): 620–628. https://doi.org/10.1016/s1470-2045(02)00876-8 PMID: 12372724

23. Sanders C, Egger M, Donovan J, Tallon D, Frankel S. Reporting on quality of life in randomised controlled trials: Bibliographic study. BMJ 1998; 317(7167):1191–1194. https://doi.org/10.1136/bmj.317.7167.1191 PMID: 9794853

24. Lee CW, Chi KN. The standard of reporting of health-related quality of life in clinical cancer trials. J Clin Epidemiol. 2000; 53(5):451–458. https://doi.org/10.1016/s0895-4356(99)00221-8 PMID: 10812316

25. Efficace F, Bottomley A, Osaba D, Gotay C, Flechtner H, D'haese S, et al. Beyond the development of Health-Related Quality-of-life (HRQOL) measures: A checklist for Evaluating HRQOL Outcomes in Cancer Clinical Trials-Does HRQOL Evaluation in Prostate Cancer Research Inform Clinical Decision Making? J Clin Oncol. 2003; 21(18): 3502–3511. https://doi.org/10.1200/JCO.2003.12.121 PMID: 12972527

26. Moher D, Jones A, Lepage L, CONSORT Group. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. Jama 2001; 285:1992–1995. https://doi.org/10.1001/jama.285.15.1992 PMID: 11308436

27. http://www.crd.york.ac.uk/PROSPERO/

28. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015; 350. https://doi.org/10.1136/bmj.g7647 PMID: 25555855

29. Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4:1. https://doi.org/10.1186/2046-4053-4-1 PMID: 25554246

30. Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions version 5.1.0. Cochrane Collab, 2011. Julian PT Higgins and Sally Green.

31. Staquet M, Berzon R, Osoba D, Machin D. Guidelines for reporting results of quality of life assessments in clinical trials. Qual Life Res. 1996; 5(5): 496–502. https://doi.org/10.1007/bf00540022 PMID: 8973129

32. Chassany O, Sagnier P, Marquis P, Fullerton S, Aaronson N, ERIQA. Patient-reported outcomes: The example of health related quality of life—A European guidance document for the improved integration of health related quality of life assessment in the drug regulatory process. Drug Inf J. 2002; 36(1): 209–238.

33. de Haes J, Curran D, Young T, Bottomley A, Flechtner H, Aaronson N, et al. Quality of life evaluation in oncological clinical trials: The EORTC model—The EORTC Quality of Life Study Group. Eur J Cancer 2000; 36(7): 821–825. https://doi.org/10.1016/s0959-8049(00)00007-1 PMID: 10785585
34. Sprangers MA, Moinpour CM, Moynihan TJ, Patrick DL, Revicki DA, Clinical Significance Consensus Meeting Group. Assessing meaningful change in quality of life over time: A users' guide for clinicians. Mayo Clin Proc. 2002; 77(6): 561–571. https://doi.org/10.4065/77.6.561 PMID: 12059127

35. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011; 343. https://doi.org/10.1136/bmj.d5928 PMID: 22008217

36. Rittmeyer A, Barlesi F, Waterkamp D, Park K, Ciardiello F, von Pawel J, et al. Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): a phase 3, pen-label, multicentre randomised controlled trial. Lancet 2017; 389(10066): 255–65. https://doi.org/10.1016/S0140-6736(16)32517-X PMID: 27979383

37. Reck M, Rodríguez-Abreu D, Robinson AG, Hui R, Csöszi T, Fülöp A et al. Pembrolizumab versus chemotherapy for PD-L1-Positive Non-Small-Cell lung cancer. N Engl J Med. 2016; 375: 1823–1833. https://doi.org/10.1056/NEJMoa1606774 PMID: 27718847

38. Motzer RJ, Escudier B, McDermott DF, George S, Hammers HJ, Srinivas S, et al. Nivolumab versus everolimus in advanced renal-cell carcinoma. N Engl J Med. 2015; 373: 1803–1813. https://doi.org/10.1056/NEJMoa1510665 PMID: 26406148

39. Motzer RJ, Tannir NM, McDermott DF, Arén Frontera O, Melichar B, Choueiri TK et al. Nivolumab plus ipilimumab versus sunitinib in advanced renal-cell carcinoma. N Engl J Med. 2018; 378(14): 1277–1290. https://doi.org/10.1056/NEJMoa1712126 PMID: 29562145

40. Eggermont A, Chiarion-Sileni V, Grob Jean-Jacques, Dummer R, Wolchok JD, Schmidt H et al. Adjuvant ipilimumab versus placebo after complete resection of high-risk stage III melanoma (EORTC 18071): a randomised, double-blind, phase 3 trial. Lancer Oncol. 2015; 16: 522–530.

41. Weber JS, D'Angelo SP, Minor D, Hodi FS, Gutzmer R, Neyns B et al. Nivolumab versus chemotherapy in patients with advanced melanoma who progressed after anti-CTLA-4 treatment (CheckMate 037): a randomised, controlled, open-label, phase 3 trial. Lancet Oncol. 2015; 16: 375–384. https://doi.org/10.1016/S1470-2045(15)70076-8 PMID: 25795410

42. Robert C, Long G, Brady B, Dutriaux C, Maio M, Mortier L et al. Nivolumab in previously untreated melanoma without BRAF mutation. N Engl J Med. 2015; 372: 320–330. https://doi.org/10.1056/NEJMoa1412082 PMID: 25399552

43. Robert C, Schachter J, Long G, Arance A, Grob JJ, Mortier L et al. Pembrolizumab versus Ipilimumab in advanced melanoma. N Engl J Med. 2015; 372: 2521–2532. https://doi.org/10.1056/NEJMoa1503093 PMID: 25891173

44. Brahmer J, Reckamp K, Baas P, Crinò L, Eberhardt EE, Poddubskaya E, et al. Nivolumab versus docetaxel in advanced squamous-cell non-small-cell lung cancer. N Engl J Med. 2015; 373: 123–135. https://doi.org/10.1056/NEJMoa1504627 PMID: 26028407

45. Hodi FS, O'Day S, McDermott DF, Weber RW, Sosman JA, Haanen JB et al. Improved survival with ipilimumab in patients with metastatic melanoma. N Engl J Med. 2010; 363(8): 711–723. https://doi.org/10.1056/NEJMoa1003466 PMID: 20525992

46. Ribas A, Puzonov I, Dummer R, Schadendorf D, Hamid O, Robert C, et al. Pembrolizumab versus investigator-choice chemotherapy for ipilimumab-refractory melanoma (KEYNOTE-002): a randomised, controlled, phase 2 trial. Lancet Oncol. 2015; 16(8): 908–918. https://doi.org/10.1016/S1470-2045(15)00083-2 PMID: 26115796

47. Larkin J, Chiarion-Sileni V, Gonzalez R, Grob JJ, Cowey CL, Lao CD et al. Combined nivolumab and ipilimumab or monotherapy in untreated melanoma. N Engl J Med. 2015; 373: 23–34. https://doi.org/10.1056/NEJMoa1504030 PMID: 26027431

48. Bellmunt J, de Wit R, Vaughn DJ, Fradet Y, Lee JL, Fong L, et al. Pembrolizumab as second-line therapy for advanced urothelial carcinoma. N Engl J Med. 2017; 376(11): 1015–1026. https://doi.org/10.1056/NEJMoa1613683 PMID: 28212060

49. Bordini R, Ciardiello F, von Pawel J, Cortinovis D, Karagiannis T, Ballinger M, et al. Patient-reported outcomes in OAK: A phase III study of atezolizumab versus docetaxel in advanced non-small-cell lung cancer. Clin Lung Cancer 2018; 19(5): 441–449. https://doi.org/10.1016/j.cllc.2018.05.011 PMID: 30017645

50. Cella D, Grünwald V, Nathan P, Doan J, Dastani H, Taylor F, et al. Quality of life in patients with advanced renal cell carcinoma given nivolumab versus everolimus in CheckMate 025: a randomised, open-label, phase 3 trial. Lancet Oncol. 2016; 17(7): 994–1003. https://doi.org/10.1016/S1470-2045(16)30125-5 PMID: 27283863

51. Coens C, Suciu S, Chiarion-Sileni V, Grob JJ, Dummer R, Wolchok JD, et al. Health-related quality of life with adjuvant ipilimumab versus placebo after complete resection of high-risk stage III melanoma (EORTC 18071): secondary outcomes of a multinational, randomised, double-blind, phase 3 trial. Lancet Oncol. 2017; 18(3): 393–403. https://doi.org/10.1016/S1470-2045(17)30015-3 PMID: 28162999
52. Larkin J, Minor D, D'Angelo S, Neyns B, Smylie M, Miller WH, et al. Overall survival in patients with advanced melanoma who received nivolumab versus investigator's choice chemotherapy in Check-Mate 037: a randomized, controlled, open-label phase III Trial. J Clin Oncol. 2018; 36(4): 383-393. https://doi.org/10.1200/JCO.2016.71.8023 PMID: 28671856

53. Long GV, Atkinson V, Ascierto PA, Robert C, Hassel JC, Rutkowski P, et al. Effect of nivolumab on health-related quality of life in patients with treatment-naïve advanced melanoma: results from the phase III CheckMate 066 study. Ann Oncol. 2016; 27(10): 1940-1946. https://doi.org/10.1093/annonc/mdw265 PMID: 27405322

54. Petrella T.M, Robert C, Richtig E, Miller WH, Masucci GV, Walpole E, et al. Patient-reported outcomes in KEYNOTE-006, a randomised study of pembrolizumab versus ipilimumab in patients with advanced melanoma. Eur JCancer 2017; 86: 115-124.

55. Reck M, Taylor F, Penrod J, DeRosa M, Morrissey L, Dastani H, et al. Impact of Nivolumab Versus Docetaxel on Health-Related Quality of Life and Symptoms in Patients With Advanced Squamous Non-Small Cell Lung Cancer: Results From the CheckMate 017 Study. J Thorac Oncol. 2018; 13(2): 194-204. https://doi.org/10.1016/j.jtho.2017.10.029 PMID: 29129758

56. Schadendorf D, Larkin J, Wolchok J, Hodi FS, Chiarion-Sileni V, Gonzalez R, et al. Health-related quality of life results from the phase III CheckMate 067 study. Eur J cancer 2017; 82: 80-91. https://doi.org/10.1016/j.ejca.2017.05.031 PMID: 28651159

57. Weber J, Mandala M, Del Vecchio M, Gogas HJ, Arance AM, Cowey CL, et al. Adjuvant Nivolumab versus ipilimumab in resected stage III or IV melanoma. N Engl J Med. 2017; 377: 1824-1835. https://doi.org/10.1056/NEJMoa1709030 PMID: 28891423

58. Brahmer JR, Rodriguez-Abreu D, Robinson AG, Hui R, Csösi T, Fülöp A, et al. Health-related quality-of-life results for pembrolizumab versus chemotherapy in advanced, PD-L1-positive NSCLC (KEY-NOTE-024): a multicentre, international, randomised, open-label phase 3 trial. Lancet Oncol. 2017; 18(12): 1600-1609. https://doi.org/10.1016/S1470-2045(17)30690-3 PMID: 29129441

59. Cella D, Grünwald V, Escudier B, Hammers HJ, George S, Nathan P, et al. Patient-reported outcomes of patients with advanced renal cell carcinoma treated with nivolumab plus ipilimumab versus sunitinib (CheckMate 214): a randomised, phase 3 trial. Lancet Oncol. 2019; 20: 297-310. https://doi.org/10.1016/S1470-2045(18)30778-2 PMID: 30658932

60. Harrington K, Ferris R, Blumenschein G, Colevas AD, Fayette J, Licitra L, et al. Nivolumab versus standard, single-agent therapy of investigator's choice in recurrent or metastatic squamous cell carcinoma of the head and neck (CheckMate 141): health-related quality-of-life results from a randomised phase 3 trial. Lancet Oncol. 2017; 18(8): 1104-1115. https://doi.org/10.1016/S1470-2045(17)30421-7 PMID: 28651929

61. Schadendorf D, Dummer R, Hauschild A, Robert C, Hamid O, Daud A, et al. Health-related quality of life in the randomised KEYNOTE-002 study of pembrolizumab versus chemotherapy in patients with ipilimumab-refractory melanoma. Eur J cancer 2016; 67: 46-54. https://doi.org/10.1016/j.ejca.2016.07.018 PMID: 27596353

62. Vaughn DJ, Bellmunt J, Fradet Y, Lee JL, Fong L, Vogelzang NJ, et al. Health-related quality of life analysis from keynote-45: A phase III study of pembrolizumab versus chemotherapy for previously treated advanced urothelial cancer. J Clin Oncol 2018; 16(1): 1579-1587.

63. Revicki DA, van den Eertwegh AJM, Lorigan P, Lebbe C, Linette G, Ottensmeier CH, et al. Health related quality of life outcomes for unresectable stage III or IV melanoma patients receiving ipilimumab treatment. Health Qual Life outcomes 2012; 10: 66. https://doi.org/10.1186/1477-7525-10-66 PMID: 22694829

64. Efficace F, Fayers P, Pusic A, Cemal Y, Yanagawa J, Jacobs M, et al. Quality of patient-reported outcome reporting across cancer randomized controlled trials according to the CONSORT patient-reported outcome extension: A pooled analysis of 557 trials. Cancer. 2015; 121(18):3335-3342. https://doi.org/10.1002/cncr.29489 PMID: 26079197

65. Hamel JF, Saulnier P, Pe M, Zikos E, Musoro J, Coens C, et al. A systematic review of the quality of statistical methods employed for analysing quality of life data in cancer randomised controlled trials. Eur J Cancer 2017; 83: 166-176. https://doi.org/10.1016/j.ejca.2017.06.025 PMID: 28738257

66. European Medicines Agency. Committee for Medicinal Products for Human Use (CHMP). Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products https://www.ispor.org/workpaper/emea-hrql-guidance.pdf (accessed mars 8, 2018).

67. Osoba D, Rodrigues G, Myles J, Zee B, Pater J. Interpreting the significance of changes in health-related quality-of-life scores. J Clin Oncol. 1998; 16: 139-144. https://doi.org/10.1200/JCO.1998.16.1.139 PMID: 9440735
68. Maringwa JT, Quinten C, King M, Ringash J, Osoba D, Coens C, et al. Minimal important differences for interpreting health related quality of life scores from the EORTC QLQC30 in lung cancer patients participating in randomized controlled trials. Support Care Cancer 2011; 19(11): 1753–1760. https://doi.org/10.1007/s00520-010-1016-5 PMID: 20886240

69. Schadendorf D, Amonkar MM, Milhem M, Grotzinger K, Demidov LV, Rutkowski P, et al. Functional and symptom impact of trametinib versus chemotherapy in BRAF V600E advanced or metastatic melanoma: Quality-of-life analyses of the METRIC study. Ann Oncol. 2014; 25: 700–706. https://doi.org/10.1093/annonc/mdt580 PMID: 24504441

70. Singer S. Quality of life after Nivolumab treatment for head and neck cancer. Lancet Oncol. 2017; 18(8): 993–994. https://doi.org/10.1016/S1470-2045(17)30463-1 PMID: 28651926

71. Campbell A, Converse PE, Rodgers WL. The quality of American life: Perceptions, evaluations, and satisfactions: RSF. 1976.

72. Hosseini B, Nedjat S, Zendehdel K, Majdzadeh R, Nourmohammadi A, Montazer A. Response shift in quality of life assessment among cancer patients: A study from Iran. Med J Islam Repub Iran 2017; 31:120. https://doi.org/10.14196/mjiri.31.120 PMID: 29951421

73. Sprangers M, Schwartz C. The challenge of response shift for quality- of-life-based clinicaloncology research. Ann Oncol. 1999; 10(7): 747–749. https://doi.org/10.1023/a:1008305523548 PMID: 10470418

74. Sprangers MA, Schwartz CE. Integrating response shift into health- related quality of life research: a theoretical model. Soc. Sci. Med. 1999; 48(11): 1507–1515. https://doi.org/10.1016/s0277-9536(99)00045-3 PMID: 10400253

75. Breetvelt I, Van Dam F. Underreporting by cancer patients: the case of response-shift. Soc. Sci. Med. 1991; 32(9): 981–987. https://doi.org/10.1016/0277-9536(91)90156-7 PMID: 2047902

76. Andrykowski MA, Hunt JW. Positive psychosocial adjustment in potential bone marrow transplant recipients: Cancer as a psychosocial transition. Psychooncology. 1993; 2(4): 261–276.

77. Kagawa-Singer M. Redefining health: living with cancer. Soc. Sci. Med. 1993; 37(3): 295–304. https://doi.org/10.1016/0277-9536(93)90261-2 PMID: 8356479

78. Leventhal H, Meyer D, Nerenz D. The common sense representation of illness danger. In Rachman S, ed. Contributions to Medical Psychology. New York: pergamon press 1980; 7–30.

79. Richardson E, Schüz N, Sanderson K, Scott JL, Schüz B. Illness representations, coping and illness outcomes in people with cancer: a systematic review and meta-analysis. Psychoncology 2016; 26(6): 724–737.

80. Day SJ, Altman DG. Blinding in clinical trials and other studies. BMJ 2000; 321: 504. https://doi.org/10.1136/bmj.321.7259.504 PMID: 10948038

81. Sparano F, Aaronson NK, Sprangers MAG, Fayers P, Pusic A, Kieffer JM et al. Does the quality of patient-reported outcomes (PROs) assessment in randomized controlled trials (RCTs) differ across cancer types and over time? A pooled analysis of 610 RCTs published between 2004 and 2018. J. Clin. Oncol. 2019; 37(15). https://doi.org/10.1200/JCO.2019.37.15_suppl.e18218

82. Patel KK, Veenstra DL, Patrick DL. A review of selected patient-generated outcome measures and their application in clinical trials. Value Health. 2003; 6(5): 595–603. https://doi.org/10.1046/j.1524-4733.2003.65236.x PMID: 14627066

83. Aburub AS, Gagnon B, Rodriguez AM, Mayo NE. Using a personalized measure (Patient Generated Index (PGI)) to identify what matters to people with cancer. Support Care Cancer. 2016; 24(1):437–445. https://doi.org/10.1007/s00520-015-2821-7 PMID: 26099901

84. Tang JA, Taemin OH, Scheer JK, Parsa AT. The Current Trend of Administering a Patient-Generated Index in the Oncological Setting: A Systematic Review. Oncol Rev. 2014; 8(1): 245. https://doi.org/10.4081/oncol.2014.245 PMID: 25992233