International development of four EORTC disease-specific quality of life questionnaires for patients with Hodgkin lymphoma, high- and low-grade non-Hodgkin lymphoma and chronic lymphocytic leukaemia

This paper describes the international, cross-cultural development of four disease-specific EORTC QoL questionnaires, to supplement the EORTC QLQ-C30, for patients with Hodgkin lymphoma (HL), high- or low-grade non-Hodgkin lymphoma (HG/LG-NHL), and CLL.Questionnaire development was conducted according to guidelines from the EORTC Quality of Life Group. Phase I comprised generation of QoL issues relevant to patients. Phase II included operationalization and assessment of item relevance. In phase III, items were pretested in a cross-cultural sample.In Phase I, 75 issues were identified through focus groups and systematic literature searches. Interviews with 80 health-care professionals and 245 patients resulted in a provisional module of 38 items (phase II) representing items relevant for all or at least one of the four malignancies. In Phase III, this was tested in 337 patients from five European countries and resulted in a questionnaire with 27 items for HL (EORTC QLQ-HL27), 29 items for HG-NHL (EORTC QLQ-NHL-HG29), 20 items for LG-NHL (EORTC QLQ-NHL-LG20) and 17 items for CLL (EORTC QLQ-CLL17).This study provides four new EORTC modules for use in clinical research and routine practice in conjunction with the EORTC QLQ-C30 for assessing QoL in patients with lymphoma and CLL.

section_content: Treatment of patients with lymphoproliferative disorders, i. e. Hodgkin lymphoma (HL), high-grade (HG: aggressive) and low-grade (LG: indolent) non-Hodgkin lymphoma (NHL) and chronic lymphocytic leukaemia (CLL), has witnessed dramatic changes in the last two decades, leading to more prolonged, intensive treatments with improved survival rates and/or remission duration [1] [2] [3] [4]. To date, more than 80% of patients diagnosed with HL are expected to be disease free at 5 years or more after diagnosis [4] [5] [6]. The overall 5-year relative survival rate for patients with NHL (2006-2012) is 63-82%, [1, 4, 6] depending on the NHL type, stage of disease at diagnosis, treatment and age of the patient. 

Recently, a number of targeted therapies have been made available for NHL and CLL patients. For the latter group of patients, for example, these include monoclonal antibodies that have shown to be effective or are currently investigated in combination with other drugs. Results from the Swedish population-based Lymphoma Registry Study showed a significant survival improvement in patients diagnosed with Follicular Lymphoma between 2000 and 2010. This improvement correlated with the increasing use of first-line rituximab over time and with regional differences in firstline rituximab use [2]. The Swedish observation is in line with dramatic improvements of relative survival in the last decade for NHL patients as recently reported from the SEER database [4]. 

Despite the changing landscape of treatment of lymphoproliferative disorders and in contrast to the large number of quality of life (QoL) studies in patients with solid tumours, relatively few studies have reported QoL in patients with haematological malignancies [7]. Also, the American Society of Hematology (ASH) has voiced concern about the lack of data in this area, advocating urgent efforts to raise the standards of QoL research [8]. Likewise, international recommendations for various hematologic diseases are also now paying greater attention to QoL assessment [8] [9] [10] [11] [12]. Studies investigating effects and complications after haematological cancer treatment have identified problems in several domains, including eye, oral, endocrine, neurosensory and cardiopulmonary impairments [13] [14] [15] [16] [17] [18] [19] [20]. The FACT-Lymphoma questionnaire has been developed more than a decade ago to assess QoL in the broad group of all subtypes of lymphoma cancer patients. From a pool of 69 items, following expert relevancy ratings (n = 17), patient input (n = 75) and item correlations, a lymphoma subscale of 15 items was constructed [21]. This questionnaire covers some, but not all issues that are sometimes only relevant to subtypes of lymphoma patients (e. g. HL, HG or LG-NHL). 

A number of trials conducted in patients with hematologic malignancies have typically used cancer-specific or general measures such as the EORTC QLQ-C30 [22] or the SF-36 [23] [24] [25]. This possibly limits a full appraisal of outcome differences between arms. It is envisaged that use of haematological cancer-specific questionnaires would increase sensitivity to detect functional limitations and symptoms in future trials of patients with lymphoproliferative disorders. While new emerging therapies for these patients are enlarging therapeutic options, we need standardized and validated tools to measure the impact of these on QoL and symptom burden. 

The main objective of this international study was to develop questionnaires (to be used in conjunction with the EORTC QLQ-C30) to more comprehensively assess QoL of patients with HL, HG/LG-NHL or CLL. As differences in age at disease onset, treatment and survival outcomes among these cancers may influence QoL, a secondary objective was to evaluate whether it would be possible to develop only one questionnaire, covering all relevant issues for these patients, or whether their QoL issues are sufficiently different to warrant the development of separate questionnaires. 

These guidelines consist of four phases: (1) generation of QoL issues through literature searches, focus groups and interviews with patients and health-care professionals (HCPs); (2) operationalization and assessment of item relevance, issues derived from phase 1 were operationalized into items according to the response format and time frame of the EORTC QLQ-C30; (3) pretesting the questionnaire module among a new and larger sample of lymphoma and CLL patients from five countries to identify problems regarding wording and comprehensiveness; (4) large-scale international field testing. This paper presents the phase 1-3 results and phase 4 will be carried out in a future study. The main steps of the whole development process are summarized in Fig. 1. 

Eligibility criteria were adult patients (> 18 years) with a diagnosis of HL, HG/LG-NHL or CLL either with current or past treatment for primary or relapsed disease, who were able to understand and speak the local language. Patients with psychiatric disorders or major cognitive dysfunctions were excluded. Ethical approval from each participating centre was obtained and all patients provided written informed consent. Extensive literature searches through PUBMED and PsychINFO were conducted to identify all relevant HRQoL issues in the last 15 years, to include issues of patients treated longer ago but also to include symptoms as a result of newer therapies. In addition, two focus groups among Dutch CLL/NHL (N = 9) and HL (N = 7) patients were composed in order to discuss relevant HRQoL issues in general, with respect to their phase of disease. We also added issues that were included in the earlier developed EORTC QLQ-CLL16, but that were not yet in our list. The list of issues that was gathered was evaluated in semistructured interviews with patients and health-care professionals (HCPs). Patients and HCPs ratings of each item were collected for the following criteria (1) relevance, which was rated on a four-point likert scale ranging from 'not at all relevant' (1 point) to 'very much relevant' (4 points) and it refers to the frequency with which a problem or symptom occurs and the trouble it may cause. (2) priority for inclusion (yes/no), which was rated for each item to identify those items that affect patients' HRQoL most and should definitely be included in the final questionnaire. (3) breadth of coverage was investigated by asking patients and HCPs to suggest any relevant issues, which were not included in the item list and should therefore be added.
section_name: Introduction

section_name: Phase I

section_name: Phase II

section_name: Phase III

section_content: In Phase 2, the issues derived from Phase 1 were operationalized into items according to the response format and time frame of the EORTC QLQ-C30. Rather than developing new items, if available, items that matched content in the modules were selected from the EORTC Quality of Life Group Item Library.
section_name: Phase 2

section_content: The pretesting of the module was tested among a new and larger sample of lymphoma and CLL patients from five countries to identify problems regarding wording and comprehensiveness. In this Phase, first patients completed the general EORTC QLQ-C30 and the provisional lymphoma/ CLL-specific module. Then they took part in a semi-structured interview where they were asked if any items were annoying, confusing, upsetting or intrusive, and if there were irrelevant or missing issues.
section_name: Phase 3

section_content: Results from phase 1 and 3 interviews were analysed using descriptive statistics according to the EORTC guidelines and data were analysed per tumour group. Analyses were performed with IBM SPSS statistics version 19. 

Phase 1 issues were retained if patients reported a mean score of ≥ 1. 75 (range 1-4) for the relevance question and > 15% of patients considered the item a priority, and HCPs reported a mean score of ≥ 2. 00 for the relevance question and > 25% considered it a priority. For patients, two points were assigned for every criterion that was met and for HCPs, one point was assigned. This was done to give more importance to the results of patients. Subsequently, issues that had a score of 4, 5 or 6 points for at least one of the four tumour groups were selected for inclusion in the provisional phase 3 item questionnaire. 

In phase 3, the following criteria were used for item selection: (1) patients reported a mean score ≥ 1. 6 (range 1-4);

(2) prevalence ratio (number of patients scoring 2: 'a little', 3: 'quite a bit' or 4: 'very much' divided by the total number who completed the item) ≥ 30%; (3) range of scores > 2 points; (4) responses in categories 3 and 4 ≥ 15%; (5) at least 95% response to item; (6) no more than 3% of patients expressed significant concerns of a particular item (e. g. item is upsetting, confusing); ( 7 ) consistency across languages and cultures (mean and prevalence). Items were retained in the list if (1) four or five criteria were met from the first five criteria; (2) no concerns were expressed by > 3% of patients on an item; (3) consistency across countries was observed [26]. A hypothesized scale structure was developed based on content. Descriptive statistics and preliminary psychometric testing of the hypothesized scales included evaluation of internal consistency (Cronbach's α) and clinical validity (known-group comparisons). We hypothesized that patients on treatment would report worse symptom burden, fatigue, physical function, emotional impact and worries compared to patients off treatment. Based on previous findings, we furthermore hypothesized that older patients would report lower scores on emotional impact and worries compared to younger patients. Full psychometric testing requires larger patient numbers and will be performed in phase 4 of the module development.
section_name: Data analyses and criteria for item selection

section_name: Results

section_content: Extensive literature searches through PUBMED and PsychINFO (January 2011) resulted in thirteen studies [13] [14] [15] [16] [17] [18] [19] [20] [27] [28] [29] [30] [31] performed between 1998 and 2011 that formed the basis of the provisional list as no new issues came out of further searches of other databases. Also, previous work that had been conducted by EORTC QoL group members on the development of the EORTC QLQ-CLL16 was included and updated in the module development process.
section_name: Phase 1: generation of issues

section_content: Focus group interviews were held and any issues identified through the focus groups but not came out of literature search were added to the provisional list. This extensive process yielded an initial list of 75 potential relevant issues. 

This list with 75 issues was used as a basis to conduct semi-structured interviews with 245 patients, of whom 75 had a diagnosis of HL, 66 of high-grade (aggressive) non-Hodgkin lymphoma (HG-NHL), 41 of low-grade (indolent) non-Hodgkin lymphoma (LG-NHL) and 63 had a diagnosis of Chronic Lymphocytic Leukaemia (CLL). Patients were recruited in four different countries; 100 from Italy, 48 from the UK, 83 from the Netherlands, and 14 from Taiwan. Sociodemographic and clinical characteristics of patients participating in phase 1 are shown in Table 1. 

The list of 75 issues was also used as a basis to conduct semi-structured interviews with HCPs (haematologists, radiation oncologists, medical oncologists, nurses and psychologists), all of whom were experienced in working with Hl, NHL or CLL patients. Thirty HCPs completed the list for HL, 29 HCPs for NHL and 21 HCPs completed the list for CLL. HCPs were recruited from the same countries as patients. 

Of the 75 issues, 36 met all inclusion criteria relating to relevance, priority and breadth of coverage for at least one of the four tumour groups and these items were included in the phase III questionnaire. Thirty-seven issues did not meet the inclusion criteria in any of the tumour groups and were deleted. Furthermore, one issue was added about 'worries on treatment causing future health problems' to assess health problems in general as patients reported many symptoms. 2. Since large differences were observed in the retention of items for the four tumour groups as shown in Table 3, results are presented per disease type. Some items were not relevant or even upsetting to certain subgroups, while very relevant to others. In order to create relevant and at the same time not upsetting questionnaires, we decided to continue the development of four disease-specific questionnaires.
section_name: 3

section_content: Of the 38 items in the provisional module, 27 items met all inclusion criteria and were retained in the final module. Eleven items were deleted, because ten items (i. e. 34, 35, 37, 38, 40, [42] [43] [44] [45] 47) did not meet all inclusion criteria and item 57 was rated as too upsetting by 4. 5% of patients. Four items (39, 46, 54, 59 ) needed small English formatting changes and were reworded. This resulted in a list of 27 items for phase IV interviews among patients with HL (EORTC QLQ-HL27; Table 3 ).
section_name: Hodgkin lymphoma

section_content: Of the 38 items, 29 met all inclusion criteria and were retained in the final module. Nine items were deleted, because 8 items (i. e. 37-39, 41-43, 53, 55) did not meet all inclusion criteria and item 57 was rated as too upsetting by 3. 4% of patients. Three items (46, 54, 59 ) needed small English formatting changes and were reworded. This resulted in a list of 29 items for phase IV interviews among patients with high-grade NHL (EORTC QLQ-NHL-HG29; Table 3 ).
section_name: High-grade (HG: aggressive) non-Hodgkin lymphoma

section_content: Of the 38 items, 20 met all inclusion criteria and were retained in the module. Eighteen items were deleted, because 16 items (i. e. 33, 34, 37-45, 47, 51, 55, 65, 68) did not meet all inclusion criteria, item 57 was rated as too upsetting by 1. 5% of patients and 3% reported additional comments. Item 61 was deleted since low-grade NHL is already a chronic disease and this question is therefore not applicable to these patients. Furthermore, 4. 5% of patients reported a comment about the word 'chronic' in this question. Three items (46, 54, 59 ) needed small English formatting changes and were reworded. This resulted in a list of 20 items for phase IV interviews among patients with low-grade NHL (EORTC QLQ-NHL-LG20; Table 3 ).
section_name: Low-grade (LG: indolent) non-Hodgkin lymphoma

section_content: Of the 38 items, 17 met all inclusion criteria and were retained in the final module. Twenty-one items were deleted, because 19 items (i. e. 34, 36-39, 41, 43-45, 47, 51-56, 58, 65, 68) did not meet all inclusion criteria, item 57 was rated as too upsetting by 4. 7% of patients and item 61 was deleted since CLL is already a chronic disease and 8% of patients reported a comment about the word 'chronic' in this question. Two items (46, 59) were reworded. This resulted in a list of 17 items for phase IV interviews among patients with CLL (EORTC QLQ-CLL17; Table 3 ).
section_name: Chronic lymphocytic leukaemia

section_content: Based on content and clinical relevance, three to five multiitem scales were proposed for the different questionnaires: Symptom burden due to disease and/or treatment, Neuropathy (as a distinct scale only in NHL-HG29), Physical condition/Fatigue, Emotional impacts (not in CLL17) and Worries/fears about health and functioning. Internal consistency ranged from 0. 67 (only the symptom burden scale for NHL-LG20 was below 0. 70) to 0. 92 for the proposed scales indicating moderate to good internal consistency (Table 4 ). 

Comparisons of the mean scale scores between patients on and off treatment showed differences in all scales in the expected direction, with the strongest impact of treatment on symptom burden, neuropathy and physical condition among patients with LG/HG-NHL. Comparison between older and younger patients (dichotomized according to median age per subgroup) showed less emotional impact and worries about cancer and its treatment among the elderly in all subgroups. 15 ) 20 (23) Symptom burden, physical condition and fatigue were more negatively impacted among the older HL and LG-NHL. In contrast, older HG-NHL and CLL patients experienced fewer symptoms and less impact on physical functioning than younger patients. The phase 3 development process report was peerreviewed and formally approved by the EORTC QLG module development committee (MDC), therefore the development can be taken forward to phase 4.
section_name: Proposed scale structure

section_content: Four EORTC questionnaires to more comprehensively assess QoL in patients with HL (EORTC QLQ-HL27), NHL-HG (QLQ-NHL-HG29), NHL-LG (QLQ-NHL-LG20) and CLL (QLQ-CLL17) have been developed on an international basis. Large differences were observed in the mean and prevalence of items for the four tumour groups, where some items were relevant to certain subgroups, while at the same time upsetting to other subgroups. We therefore decided that four distinct questionnaires would result in better content validity and usability for future studies. After completion of phase 4 full psychometric testing, these questionnaires are to be used in conjunction with the EORTC QLQ-C30 core questionnaire and are expected to raise standards of outcome measurements in patients with lymphoproliferative disorders in future trials. 

The EORTC approach that includes extensive literature review, and interviews with both patients and health-care providers from several different countries ensures content validity with cross-cultural relevance. This is particularly important for use in multicenter international studies. 

Preliminary psychometric testing confirms the hypothesized multi-item scales, although full psychometric evaluation will be done in phase 4. Internal consistency was good (> 0. 70) for all scales of all questionnaires, except the symptom burden scale of the NHL-LG20 questionnaire of showing a moderate internal consistency (0. 67). Known-group comparisons of scores between patients on and off treatment and younger and older patients confirmed clinically relevant differences in outcomes. Our observation that older participants reported less emotional impact and fewer worries or fears about their health has been reported previously [32]. 

Importantly, full psychometric testing requires larger patient numbers and will be performed in a forthcoming prospective international field testing. Nevertheless, the phase 3 modules already receive a lot of attention as in recent years the treatment of patients with lymphoproliferative disorders has changed dramatically while at the same time international recommendations are advocating more research into the QoL of these patients [8] [9] [10] [11] [12]. The only other lymphoma questionnaire that currently exists is the Fact-Lym, that includes 15 lymphoma-specific items [21]. However, certain issues that we identified as relevant (e. g. neuropathic symptoms) are not included in the Fact-Lym that was developed in 2005. 

In the past, treatment options for patients with lymphoma were limited to radiation and cytotoxic chemotherapies. In recent years, significant advances have been achieved in the understanding of the pathogenesis of lymphomas, leading to the emergence of a large number of new therapeutic agents targeting the signalling pathways, surface antigens of microenvironment. Rituximab was the first monoclonal antibody approved by the U. S. Food and Drug Administration (FDA) in 1997 and has consistently been shown to significantly improve progression-free and overall survival [33] [34] [35] [36]. 

In the absence of a NHL-specific questionnaire, previous studies investigating the impact of (R-)CHOP or (R-)CVP on QoL of follicular lymphoma or diffuse large B-cell lymphoma have often used generic and non-validated questionnaires and added extra items to cover expected symptoms, like tingling in hands/feet [37, 38]. More recent randomized phase III trials have greatly contributed to a better understanding of the impact of therapy in rituximab-refractory NHL patients [39]. 

With regard to HL, a recent systematic review [40] has shown a considerable number of studies with QoL as endpoint published over the last few years, with some half of the 65 studies identified, published after 2005. However, the majority of studies used non-HL disease-specific questionnaires. Also, there was a lack of studies documenting QoL outcomes of patients during active treatment as the majority were on patients off treatment. 

Furthermore, in recent years, many novel targeted therapies for CLL have been introduced, majorly transforming CLL treatment, although watch and wait remains the standard approach for patients not meeting treatment criteria 1 3 [41]. Only CLL patients with active or symptomatic disease or with advanced Binet or Rai stages require therapy [42]. For physically fit patients, chemoimmunotherapy with fludarabine, cyclophosphamide and rituximab is currently the standard therapy [43]. For unfit patients, treatment with an anti-CD20 antibody plus milder chemotherapy (chlorambucil) may be applied [42]. Several ongoing and planned phase III trials will determine, whether the novel targeted therapies will further improve outcomes for patients with CLL. For example, Ghia and colleagues have recently shown that in patients with relapsed CLL, treatment with rituximab plus idelalisib provided better QoL outcomes and superior symptom control than rituximab plus placebo [44]. 

It seems likely that a combination of therapies will increasingly be used for long-term disease control in specific subgroups of patients. While novel agents have demonstrated improvements in disease-free and overall survival, they are also associated with unique toxicities (e. g. neutropenia, leukocytopenia) [43] that have not been previously observed with conventional therapies, requiring careful monitoring and management strategies [45]. In the absence of disease-specific QoL measures, previous landmark trials like the CLL8 evaluated QoL using a generic cancer HRQOL measure such as the EORTC QLQ-C30 [46]. This tool may have possibly limited the full appraisal of treatment outcome differences between arms. Although EORTC QoL group members had previously developed the CLL-16 questionnaire, important developments in CLL treatment urged us to develop a new, up-to-date module that includes symptoms associated with new therapies. With the development of the EORTC-CLL17, we now have a more precise, reliable and responsive measure for the evaluation of the impact of CLL and therapies to HRQOL changes. 

As new lymphoma and CLL therapeutics are rapidly evolving, each class of drugs with its own toxicity profile, our newly developed questionnaires may need regular updates to ensure adequate assessment of side effects of treatment. The development of four different modules for the four patient groups increases flexibility when it comes to updating the modules in the (near) future. But, as the process of module updating may still be quite time consuming, the EORTC Quality of Life Group has recently adopted a more flexible approach in the inclusion of items to assess patientreported outcomes after cancer and its treatment [47]. This approach employs a combination of the EORTC QLQ-C30 questionnaire, a condition-specific module, such as in this paper described and if needed, additional items from the EORTC item library. The EORTC maintains an Item Library of 600 items (http://groups. eortc. be/qol/item-bank) in many languages derived from its internationally validated modules, which is available to all interested researchers. Thus, if certain side effects, associated with new drugs, are not covered in the existing lymphoma modules, researchers are encouraged to use the item library to add items to their questionnaire. This flexible approach of combining standardized patient-reported questionnaires with validated items from the EORTC item libraries ensures up-to-date assessment of not only the specific side effects of novel therapies, but also their impact on the common functional health problems reported by patients. 

In conclusion, four new EORTC questionnaires for patients with lymphoproliferative disorders have been crossculturally developed according to highest quality international standards. The proposed scale structure will be tested more rigorously and further validated in the forthcoming international field test to confirm psychometric properties.
section_name: Discussion

section_content: Acknowledgements LP, SO and FE designed the study, SO, AB, CK, MS, SP, LD, CC, KC, SM, GC, WC and FE collected the data. LP and SO analysed the data, while AB, CK, MS, SP, LD, CC, KC, SM, GC, WC and FE interpreted findings. LP, SO wrote the paper, AB, CK, MS, SP, LD, CC, KC, SM, GC, WC and FE revised it critically. All authors approved the submitted version.

section_name: Funding

section_content: The project was allocated an EORTC Quality of Life Group (QLG) grant for module development, Phase I-III. This study was funded by EORTC Quality of Life Group.
section_name: Funding

section_name: Compliance with ethical standards

section_name: Informed consent Informed consent was obtained from all individual participants included in the study.

section_content: Conflict of interest Fabio Efficace has received grants from Lundbeck, TEVA and Amgen, and has consultancy activities for Bristol-Myers Squibb, Seattle Genetics, TEVA and Incyte. The rest of the authors declare that they have no conflict of interest. 

Ethical approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

section_content: Open Access This article is distributed under the terms of the Creative Commons Attribution 4. 0 International License (http://creativecommons. org/licenses/by/4. 0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
section_name: Informed consent Informed consent was obtained from all individual participants included in the study.