# Development of a holistic communication score (HoCoS) in patients treated for oral or oropharyngeal cancer: Preliminary validation

Mathieu Balaguer, Julien Pinquier, Jérôme Farinas, Virginie Woisard

► To cite this version:

Mathieu Balaguer, Julien Pinquier, Jérôme Farinas, Virginie Woisard. Development of a holistic communication score (HoCoS) in patients treated for oral or oropharyngeal cancer: Preliminary validation. International Journal of Language and Communication Disorders, 2023, 58 (1), pp.39-51. 10.1111/1460-6984.12766 . hal-03765649v2

HAL Id: hal-03765649

https://hal.science/hal-03765649v2

Submitted on 19 Jan 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

# Development of a holistic communication score (HoCoS) in patients treated for oral or oropharyngeal cancer: Preliminary validation

Mathieu Balaguer<fnref n="1" />,<fnref n="2" /> ID

Julien Pinquier<fnref n="1" />

Jérôme Farinas<fnref n="1" />

Virginie Woisard<fnref n="2" />,<fnref n="3" /> ID

## Correspondence

Mathieu Balaguer, IRIT Institut de Recherche en Informatique de Toulouse, SAMOVA Team, 118 Route de Narbonne, F-31062 Toulouse Cedex 9, France.

Email: mathieu.balaguer@irit.fr

## Funding information

This study was funded by the Hospitals of Toulouse; and by the French National Research Agency [RUGBI project, grant ANR-18-CE45-0008].

## Abstract

**Background:** In head and neck cancer, many tools exist to measure speech impairment, but few evaluate the impact on communication abilities. Some self-administered questionnaires are available to assess general activity limitations including communication. Others are not validated in oncology. These different tools result in scores that does not provide an accurate measure of the communication limitations perceived by the patients.

**Aim:** To develop a holistic score measuring the functional impact of speech disorders on communication in patients treated for oral or oropharyngeal cancer, in two steps: its construction and its validation.

**Methods & Procedures:** Patients treated for oral/oropharyngeal cancer filled six self-questionnaires: two about communicative dynamics (ECVB and DIP), two assessing speech function (PHI and CHI) and two relating to quality of life (EORTC QLQ-C30 and EORTC QLQ-H&N35). A total of 174 items were initially collected. A dimensionality reduction methodology was then applied. Face validity analysis led to eliminate non-relevant items by surveying a panel of nine experts from communication-related disciplines (linguistics, medicine, speech pathology, computer science). Construct validity analysis led to eliminate redundant and insufficiently variable items. Finally, the holistic communication score was elaborated by principal component factor and validated using cross-validation and latent profile analysis.

**Outcomes & Results:** A total of 25 patients filled the questionnaires (median age = 67 years, EIQ = 12; 15 men, 10 women; oral cavity = 14, oropharynx = 10, two locations = 1). After face validity analysis, 44 items were retained ($κ > 0.80$). Four additional items were excluded because of a very high correlation ($r > 0.90$) with other items presenting a better dispersion. A total of 40 items were finally included in the factor analysis. A post-analysis score prediction was performed (mean = 100; SD = 10). A total of 24 items are finally retained for the construction of the holistic communication score (HoCoS): 19 items from

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

© 2022 The Authors. *International Journal of Language & Communication Disorders* published by John Wiley & Sons Ltd on behalf of Royal College of Speech and Language Therapists.

<footnote n="1">¹IRIT, Université de Toulouse, CNRS, Toulouse INP, UT3, Toulouse, France</footnote>

<footnote n="2">²Hôpital Larrey, Hôpitaux de Toulouse, Toulouse, France</footnote>

<footnote n="3">³Laboratoire de Neuro-Pscho-Linguistique LNPL, Université Toulouse II, Toulouse, France</footnote>
<figure><img src="image_3.png" /></figure>

questionnaires assessing communicative dynamics (13 from the ECVB and six from the DIP), four items from a perceived speech impairment questionnaire (PHI) and one from a quality-of-life questionnaire (EORTC QLQ-H&N35). The reliability is good (five-fold cross-validation: r_s = 0.91) and the complementary latent profile analysis shows a good validity of the HoCoS, clustering subjects by level of communication performance.

**Conclusions & Implications:** A global score allowing a measure of the impact of the speech disorder on communication was developed. It fills the lack of this type of score in head and neck oncology and allows the better understanding of the functional and psychosocial consequences of the pathology in the patients' follow-up.

### KEYWORDS

assessment, communication, oncology, speech

### What this paper adds

*What is already known on the subject*

* Because of their anatomical location, head and neck cancer degrades the speech abilities. Few tools currently allow the assessment of the impact of the speech disorder on communication abilities. In ENT oncology, self-administered questionnaires are available to assess activity limitations and participation restrictions (International Classification of Functioning (ICF)—WHO). Other tools from the field of neurology allow an evaluation of communication dynamics. But these different tools, constructed by items, give global additive or averaged scores. This implies an identical weighting of each item, resulting in global scores that are not very representative of the communication limitations really perceived by the patients.

*What this paper adds to existing knowledge*

* A new global holistic score allowing a measurement of the impact of speech impairment on communication after treatment of oral or oropharyngeal cancer has been developed. The methodology of its construction allows a better reflection of the symptomatological, pragmatic and psychosocial elements leading to a degradation of communication abilities.

*What are the potential or actual clinical implications of this work?*

* The developed HoCoS score fills the gap in the absence of this type of tool in head and neck oncology. It may allow a better understanding of the factors involved in the functional and psychosocial limitations of these patients, and better customize their follow-up.

## INTRODUCTION

Cancer of the oral cavity and oropharynx is very common, with a high incidence: they represent more than

475,000 new cases worldwide in 2020.¹ At the same time, mortality from lip-mouth-pharynx cancer is decreasing. Thus, the increase in life expectancy following cancer means that patients are now living longer with the
<figure><img src="image_4.png" /></figure>

sequelae of cancer and treatment (Borggreven et al., 2007).

In this context, the functional and psychosocial repercussions after oncological treatment must be considered, alongside the analytical and dynamic deficits.

For this purpose, various conceptual frameworks, based on bio-psychosocial models, have been developed. The International Classification of Functioning, Disability and Health (ICF) developed by the World Health Organization (WHO) (2001) suggests looking beyond the impairment to the functional (activity limitations) and psychosocial consequences (participation restrictions) of pathologies. In this classification, personal and environmental factors can impact functional and psychosocial levels. These models provide a better understanding of the impact of therapeutic procedures in head and neck cancer on patients' quality of life (Borggreven et al., 2007).

Complementary conceptual models have been described to specify the different levels involved in the functional and psychosocial dynamics, and to establish the causal relationships that may exist between them (Wilson & Cleary, 1995): biological and physiological factors, symptomatic status, functional status, general health perceptions and overall quality of life (Murphy et al., 2007).

Due to their location, oral and oropharyngeal cancer impact the speech abilities (Balaguer et al., 2019, 2020; Mlynarek et al., 2008), and are a common complaint of these patients. This symptomatology influences functional status, altering the patients' communication abilities (Eadie et al., 2018). Yet, while many perceptual and automatic tools currently exist in head and neck oncology to measure speech impairment (Middag et al., 2009; Woisard et al., 2021) few assess the functional impact on communication abilities (Bolt et al., 2016; Meyer et al., 2004).

Some questionnaires assess activity limitations and participation restrictions, such as the phonation handicap index (PHI) (Balaguer et al., 2020) or the 'phonation' or 'psychosocial' domains of the carcinologic handicap index (CHI) (Balaguer et al., 2021). Other items related to communication, social contact or speech are present in both modules of the European Organization for Research and Treatment of Cancer (EORTC) quality of life (QoL) questionnaires (Aaronson et al., 1993; Bjordal et al., 1999).

Other questionnaires assess communication function but are not validated in head and neck oncology, such as the ECVB (Échelle de Communication Verbale de Bordeaux) (Mazaux et al., 2006) and the dysarthria impact profile (DIP) (Letanneux et al., 2013; Walshe et al., 2009). Some others target the assessment of communication abilities but present a very short format limiting the comprehensiveness of the assessment of the communication

situation, for example, the Communicative Participation Item Bank (CPIB) (Baylor et al., 2009).

Moreover, the scoring of communication impairment is a crucial matter in the development of these tools. Indeed, these questionnaires result in scores per item, grouped in global scores. Because of their construction, by addition or average, these scores and subscores relating to a functional dimension make the hypothesis that each item carries the same weight in the construction of the final communication score. However, this strong assumption is difficult to support in a clinical setting because of the specific perceptions of each patient of their communication capacity. Moreover, all these questionnaires target different aspects of communication, but none of the tools allows to obtain a global, holistic score, representative of the impact on communication of speech disorders in head and neck oncology.

The objective of the present study is to develop a holistic communication score measuring the functional impact of speech disorders on communication in patients treated for oral or oropharyngeal cancer. The development of this score includes two sub-objectives: its construction and its validation.

# METHODS

## Design

This is a cross-sectional observational study.

The study protocol was approved by the Committee for the Protection of Persons (CPP: Ouest IV, 19 February 2020, reference 11/20_3) within the framework of the ANR RUGBI project.²

## Participants

Patients coming for consultation or hospitalization in an ear, nose and throat (ENT) service were recruited by the medical staff.

Inclusion criteria were: being of legal age (at least 18 years old) and having been treated for cancer of the oral cavity or oropharynx (surgical treatment and/or radiotherapy and/or chemotherapy) for at least 6 months (stable disorders). Patients with any other associated chronic disease were excluded.

All subjects who could be included during the inclusion period (October 2019–December 2020) were asked to participate in this study. The inclusion period corresponds to the inclusion period of the quality-of-life work package of the main study (RUGBI project).
<figure><img src="image_5.png" /></figure>

# Selection process of questionnaires

## Communication-related questionnaires

Two questionnaires relating to communicative dynamics were retained in their entirety because of their comprehensiveness construction and their conceptual proximity to our objective of measuring the alteration of communicative abilities: the ECVB (Mazaux et al., 2006) and the DIP (Letanneux et al., 2013; Walshe et al., 2009). These questionnaires are only validated in an adult population with neurological pathologies. Nevertheless, they were retained in the constitution of the corpus of this study because they allow an ecological measurement (i.e., close to the real situations of real daily life) of communication, while taking into account the psycho-affective dimension.

The ECVB (Mazaux et al., 2006) includes 34 items divided into seven dimensions corresponding to daily communication situations: expression of intentions (three items), conversation (seven items), telephone (seven items), shopping (four items), social relations (five items), reading (four items) and writing (four items). Initially validated with stroke patients, the dimensions assessed concern all aspects of communication that may be impaired, whether oral or written. Each item is rated on the principle of a Likert scale with four levels ('never', 'rarely', 'often' and 'very often', from 0 to 3 for questions with a positive polarity, and from 3 to 0 for negative ones). The lower the scores, the greater is the discomfort. In this study, this questionnaire was filled by the patient himself, as it is usually done in the patients reported outcomes (PRO) questionnaires (Doward & McKenna, 2004).

The DIP, initially validated in English (Walshe et al., 2009), has been translated and validated in French (Letanneux et al., 2013). Intended for subjects with Parkinson's disease, it includes 48 items in four dimensions: 'the effect of dysarthria on me as a person' (12 items), 'accepting my dysarthria' (10 items), 'how I feel others react to my speech' (14 items), and 'how dysarthria affects my communication with others' (12 items). These 48 items are also presented in the form of a five-level Likert scale (strongly disagree, agree, not sure, disagree, strongly disagree: the direction of the scoring depending on the polarity of the questions). For this study, we added an item to section 'how I feel others react to my speech' to clarify an abstruse French formulation. The DIP therefore includes here 49 items.

## Questionnaires assessing speech function

Some questionnaires used in the routine care include items relating to the functional or psychosocial consequences of

the speech disorder in head and neck cancer: the phonation handicap index (PHI) (Balaguer et al., 2020) and the Carcinologic handicap index (CHI) (Balaguer et al., 2021).

Focusing on speech self-assessment, the PHI was retained in its entirety: five items in the physical signs (PHI-F) domain, five items in the functional impact (PHI-C) domain, five items of the psychosocial repercussions domain (PHI-E), and three complementary questions (What degree of severity do you give to your speech difficulties?; How difficult is it for you to produce understandable speech?; and How much does your speech impairment affect your daily life?).

Allowing a measure of patients' needs after cancer treatment, the CHI also includes dimensions and items related to speech: four items in the dedicated phonation domain, and four items of the psychosocial domain.

## Quality of life (QoL) questionnaires

Finally, other questionnaires are designed to measure psychosocial consequences in terms of impact on QoL. The EORTC reference questionnaires were retained for further analyses: 30 items of the EORTC QLQ-C30 (Aaronson et al., 1993) and 35 of the EORTC QLQ-H&N35 (Bjordal et al., 1999) measuring global and speech-related quality of life.

## Construction of the score

Characterization of the variable type for the holistic communication score (HoCoS)

The determination of the type of the targeted variable is essential because it will condition the statistical methodology used in dimensionality reduction (Carreira-Perpinán, 1997; Chadeau-Hyam et al., 2013; Cunningham, 2014). This methodology is required due to the format of the data, where a large number of items (n = 174) was retained.

The holistic communication score (HoCoS) was considered as a latent variable (Borsboom, 2008): it cannot be directly observed or measured, and it requires several manifest variables as indicators that are observable and measurable. In our study, the latent HoCoS influences the values of the measured manifest variables, that is, the items from the different questionnaires. Although the responses to these variables are constructed in the form of Likert scales (i.e., ordinal categorical), they were treated in this study according to a naive approach, that is, as quantitative variables. A Shapiro-Wilk test shows that less than a quarter of these manifest variables do not have a normal distribution, which supports the choice of this approach.
<figure><img src="image_6.png" /></figure>

The HoCoS was thus elaborated as a quantitative latent variable.

## Face validity

A panel of nine experts from different communication-related disciplines was surveyed to define the criteria for inclusion of items in the HoCoS. All the experts (two computer scientists, two speech and language pathologists and PhD students, two speech therapists, one phoniatrician, two researchers in linguistics) were involved for at least 5 years in several research projects related to pathological speech analysis. Moreover, all the speech therapists currently worked in ENT departments.

This selection of relevant items was performed in two steps. First, the experts were surveyed to get a consensus definition of the communication abilities in the context of oral or oropharyngeal cancer. According to the consensus definition, the items from the questionnaires that did not comply with this consensus definition were removed. The experts then participated to an individual selection of the items.

Items were finally retained if they met one of the following two criteria:

* The I-CVI (item-level content validity index) (Lynn, 1986; Polit et al., 2007) was >0.777, which corresponds to an agreement of seven out of nine experts to keep the item.

* The Kappa of agreement was ≥0.81 (Landis & Koch, 1977): ‘almost perfect’ agreement.

## Construct validity

A statistical selection of items respecting face validity was carried out on the criteria of non-redundancy and sufficient variability.

The criterion of non-redundancy allows an analysis if the scores of the items are statistically not associated with each other. An analysis of the inter-item correlation matrix using Spearman coefficients (non-parametric) was used because of the small sample size (n < 30). A threshold of 0.90 was chosen: only one of the items correlated with each other at ≥0.90 could be retained for further analysis.

In this case, to obtain sufficiently variable items, and thus allowing a more specific measure of inter-individual variability, only the item with the highest coefficient of variation (if the distribution of this item is gaussian, tested by the Shapiro-Wilk test) or the highest dispersion index (if the distribution is not gaussian) was retained.

## Elaboration of the holistic communication score (HoCoS)

Once relevant, non-redundant and sufficiently variable items were selected, the HoCoS was elaborated, using principal component factor (PCF) analysis (Roscoe et al., 1982). PCF analysis is commonly used in data reduction (Acock, 2018) because it attempts to explain as much as possible the variance of a set of items by a single dimension, in other words when a set of items all measure the same concept (communalities set to one, no uniqueness).

This statistical technique therefore suits the objective of this study, where a single quantitative latent holistic score (the HoCoS) is sought among the set of manifest variables (corresponding to the selected items).

Thus, a prediction of the values in PCF analysis for factor 1, corresponding to the latent variable HoCoS, was made. This prediction is derived from a regression analysis on the set of new variables created by estimation of the first factor.

Thus, per subject, the score is predicted by the sum of the standardized values of each item weighted by the regression coefficient corresponding to factor 1.

## Validation of the HoCoS

The validation of the score was done in two steps. First, a five-fold cross-validation of the predicted score was led to verify the reliability of the score. A latent profile analysis (Cai, 2012) leading to the construction of a qualitative HoCoS (‘HoCoS-Qual’) was then compared with the (quantitative) HoCoS.

This type of analysis, based on generalized structural equation modelling (GSEM) models (Coma et al., 2013), allows one to determine which individuals are most likely to belong to a group (corresponding to a category of the latent variable) according to information carried by other variables. Two-, three- and four-class models were computed, and the model with the best Akaike information criterion (AIC) and Bayesian information criterion (BIC) criteria—i.e., the lowest parameters—was selected (Cameron & Trivedi, 2005).

Finally, the class of each subject was predicted. For each subject, the categorical latent variable HoCoS-Qual thus takes one of the values corresponding to one of the classes.

## Statistical analyses

The statistical analyses were carried out using Stata 16.1 software (StataCorp. 2019. Stata Statistical Software: Release 16. College Station, TX, USA: StataCorp LLC).
<figure><img src="image_7.png" /></figure>

<table><caption>TABLE 1 Characteristics of the subjects</caption><thead><tr><th></th><th>Total (%)</th></tr></thead><tbody><tr><td>TNM Classification: T (tumour size)</td><td>25</td></tr><tr><td>0</td><td>0</td><td>0.0%</td></tr><tr><td>1</td><td>2</td><td>10.5%</td></tr><tr><td>2</td><td>6</td><td>31.6%</td></tr><tr><td>3</td><td>3</td><td>15.8%</td></tr><tr><td>4</td><td>8</td><td>42.1%</td></tr><tr><td>Missing data</td><td>6</td></tr><tr><td>Treatments</td><td>25</td></tr><tr><td>Surgery alone</td><td>1</td><td>4%</td></tr><tr><td>Radiotherapy alone</td><td>1</td><td>4%</td></tr><tr><td>Surgery + radiotherapy</td><td>8</td><td>32%</td></tr><tr><td>Surgery + radiotherapy + chemotherapy</td><td>13</td><td>52%</td></tr><tr><td>Radiotherapy + chemotherapy</td><td>2</td><td>8%</td></tr><tr><td>Speech and language therapy</td><td>25</td></tr><tr><td>Ongoing</td><td>12</td><td>55%</td></tr><tr><td>Once a week</td><td>3</td></tr><tr><td>At least twice a week</td><td>3</td></tr><tr><td>No information</td><td>6</td></tr><tr><td>None</td><td>10</td><td>45%</td></tr><tr><td>Missing data</td><td>3</td></tr></tbody></table>

In all analyses, a level of significance at 5% was chosen.

The normality of distribution of quantitative variables was tested using a Shapiro-Wilk test.

Correlation analyses were performed using the following thresholds (Mukaka, 2012): >0.9 (very high correlation), 0.7-0.9 (high correlation), 0.5-0.7 (moderate correlation), 0.3-0.5 (low correlation), and <0.3 (negligible correlation).

# RESULTS

## Participants

A total of 25 patients filled the questionnaires (median age = 67 years, IQR = 12; 15 males and 10 females; oral cavity = 14, oropharynx = 10, two locations = 1).

A total of 88% of the subjects were treated surgically, 96% by radiotherapy.

The mean time after treatment was 87.2 months (SD = 121.8; median = 40; interquartile range = 123).

Details of TNM classification and treatment strategies are given in Table 1.

## Construction of the HoCoS

A set of 174 items were initially collected. The list of the 174 items and reasons for exclusion (if applicable) are given in the additional *supporting information*.

The construction process is represented in Figure 1 and will be developed below.

### Face validity

As a first step, the panel identified the following consensus criteria for retaining items:

* Items related to oral communication: exclusion of items related to written communication unless they allow for compensation or an increase in oral communication.

* Items relating to expression: exclusion of only comprehension-related items.

* Items relating to interaction between speaker and interlocutor, even if implicit.

This first step led to exclude 83 items: those relating to reading, writing, understanding conversation, items relating to fatigue, pain, specific speech symptoms such as speed of speech, etc.

A total of 91 items (52.3%) were thus retained for further analysis.

In a second step, an online questionnaire using the LimeSurvey tool was submitted to the nine experts. They were asked to indicate which of the remaining 91 items should be retained in the elaboration of the holistic communication score. The experts had to tick the boxes corresponding to all the items to be kept.

A total of 44 items (48.4%) met one of the two previously defined conditions (I-CVI > 0.777 or $κ ≥ 0.81$) and were retained for further analysis (Figure 2).

### Construct validity

Four items were excluded because of a too high correlation with other items (considered as redundant items). In that case, only the item with the bigger variation coefficient or dispersion index was retained (Table 2).

A total of 40 items were finally retained for the construction of the holistic communication score.

### Elaboration of the holistic communication score (HoCoS)

The PCF analysis has two conditions of application: first, the first main factor must explain a 'substantial'
<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /><figcaption>FIGURE 1 Construction process of the holistic communication score</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>FIGURE 2 I-CVI scores and Kappa of agreement, with cut-off (black line)</figcaption></figure>

part of the total variance for all the items; and second, most of the items must have a load of ≥0.4 on this factor.

Both conditions are met. The eigenvalue of factor 1 (i.e., the proportion of total variance attributable to factor 1) is 17.51 compared with 4.57 for factor 2 (Figure 3). Moreover, the proportion of variance explained solely by factor 1 is 0.44 (0.11 for factor 2). Finally, 34 out of 40 items (85%) have a loading of ≥0.4 on this factor.

Since PCF analysis is applicable, a prediction of values for factor 1, corresponding to the latent variable HoCoS, was performed.

A total of 16 items have a coefficient equal to zero, neutralizing the value taken by the item, and thus indicating that they will not be considered in the overall calculation of the HoCoS. Therefore, 24 items out of 40 (60%) have non-zero coefficients and will be retained for the HoCoS calculation.
<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /><figcaption>FIGURE 3 Scree plot of the eigenvalues of the factors following the PCF analysis</figcaption></figure>

To facilitate readability and interpretation, the predicted score, initially centred on zero and having a standard deviation (SD) of 1, was centred on 100 with a SD of 10. This transformed score constitutes the HoCoS.

The HoCoS is therefore calculated as follows:

$$HoCoS = \left( \frac{\left[ \sum \left( \frac{X_{item} - m_{item}}{s_{item}} \times \beta_{item} \right) \right] - m_{initial\_score}}{s_{initial\_score}} \times 10 \right) + 100 \\ = \left( \frac{\left[ \sum \left( \frac{X_{item} - m_{item}}{s_{item}} \times \beta_{item} \right) \right] - 0.0353408}{0.999787} \times 10 \right) + 100$$

<table><caption>TABLE 2 Non-redundancy analysis</caption><thead><tr><th>Correlation between the two items of the pair</th><th>Item code</th><th>Item</th><th>Variation coefficient<sup>a</sup> or dispersion index<sup>b</sup></th></tr></thead><tbody><tr><td>0.95</td><td>ecvb11</td><td>Do you have difficulty calling your family?</td><td>0.67<sup>a</sup></td></tr><tr><td></td><td>ecvb12</td><td>Do you have difficulty phoning your friends?</td><td>0.71<sup>a</sup></td></tr><tr><td>0.92</td><td>chiph3</td><td>Do you speak less with your family, friends, neighbours?</td><td>0.77<sup>a</sup></td></tr><tr><td></td><td>phic2</td><td>My speech difficulties limit my personal and social life</td><td>0.84<sup>a</sup></td></tr><tr><td>0.91</td><td>ecvb8</td><td>And with someone you don't know very well (the letter carrier or a cab driver for example), are you embarrassed to have a conversation on simple subjects? (The weather; what you did the day before; the flowers in your garden ...)?</td><td>0.87<sup>b</sup></td></tr><tr><td></td><td>ecvb10</td><td>Do you find it difficult to speak when you are with people you don't know well (at a dinner party, an outing, an evening out ...)?</td><td>0.53<sup>a</sup></td></tr><tr><td>0.90</td><td>chiph2</td><td>Do people have difficulty understanding you?</td><td>0.61<sup>a</sup></td></tr><tr><td></td><td>phic4</td><td>I am asked to repeat myself because of my difficulty to speak</td><td>0.69<sup>a</sup></td></tr></tbody></table>

Note: Retained items are shown in bold.

where $X_{item}$ represents the raw item value (score obtained on the item by the subject); $m_{item}$ is the mean of the item obtained in the study sample, $s_{item}$ is the SD of the item obtained in the study sample, $\beta_{item}$ represents the regression coefficient of the item; $m_{initial\_score}$ is the mean of the initially predicted score; and $s_{initial\_score}$ is the SD of the initially predicted score.

The HoCoS are shown in Figure 4.

The complete formula is as follows:

$$HoCoS = ((((ecvb1 - 2.36)/0.8103497*0.03264) + ((ecvb4 - 2.08)/0.9539392*0.01566) + ((ecvb5 - 1.8)/0.9128709*0.18626) + ((ecvb6 - 1.4932)/0.8338601*0.10411) + ((ecvb7 - 0.34)/0.8524999*0.01695) + ((ecvb9 - 1.8)/0.9574271*0.00532) + ((ecvb12 - 1.68)/1.215182*0.01413) + ((ecvb13 - 1.56)/1.356466*0.10219) + ((ecvb16 - 1.72)/1.137248*0.12197) + ((ecvb17 - 2.3268)/0.8902*0.06856) + ((ecvb19 - 1.4932)/1.054509*0.01007) + ((ecvb25 - 1.82)/1.081376*0.15096) + ((ecvb26 - 2.28)/0.9363048*0.08137) + ((dipd2 - 2.88)/1.563117*0.04681)$$
<table><caption>TABLE 3 Akaike information criterion (AIC) and Bayesian information criterion (BIC) values of GSEM models (latent profile analysis—LPA)</caption><thead><tr><th>Model</th><th>AIC</th><th>BIC</th></tr></thead><tbody><tr><td>Latent variable with two classes</td><td>2811.97</td><td>2959.45</td></tr><tr><td><strong>Latent variable with three classes</strong></td><td><strong>2729.59</strong></td><td><strong>2927.04</strong></td></tr><tr><td>Latent variable with four classes</td><td>2747.96</td><td>2995.39</td></tr></tbody></table>

Note: The retained model is shown in bold.

$$ \begin{aligned} & + ((dipd3 - 2.2)/0.8660254 * 0.1402) \\ & + ((dipd5 - 3.36)/1.113553 * -0.03138) \\ & + ((dipd6 - 3.48)/1.262273 * 0.14321) \\ & + ((dipd7 - 2.64)/1.254326 * 0.05331) \\ & + ((dipd9 - 2.84)/1.344123 * -0.12441) \\ & + ((hn53 - 2.2)/1 * -0.071) \\ & + ((phif4 - 2.32)/1.519868 * -0.00858) \\ & + ((phic1 - 0.99)/1.251 * -0.24677) \\ & + ((phic3 - 1.76)/1.422439 * -0.08623) \\ & + ((phic4 - 1.68)/1.180395 * -0.07575)) \\ & - 0.0353408)/0.999787 * 10) + 100 \end{aligned} $$

## Validation of the HoCoS

### Five-fold cross-validation

A strong correlation of 0.91 was found between the HoCoS and the values predicted by the five-fold cross-validation (i.e., training on 20 observations and prediction on the other five observations, repeated five times in this case) (Figure 5).

### Complementary validation by latent profile analysis

The same 40 items meeting the face and construct validi- ties were retained as manifest variables of the qualitative latent score (HoCoS-Qual).

The model selection parameters AIC and BIC were calculated for models with two, three and four classes (Table 3). The model resulting in a three-class latent vari-

able was thus retained because it has the lowest AIC and BIC criteria.

The class to which each subject belongs was then pre- dicted. For each subject, the categorical latent variable HoCoS-Qual thus takes one of the three values correspond- ing to one of the three classes (1, 2 or 3).

A comparison of the values of the two latent vari- ables, quantitative (reference HoCoS score) and categorical (HoCoS-Qual), shows that class 1 of the HoCoS-Qual score corresponds to the subjects with the lowest HoCoS scores, class 2 corresponds to the subjects with the highest HoCoS scores and finally class 3 to the subjects with intermediate HoCoS scores (Table 4).

The latent profile analysis leads to a qualitative vari- able corresponding to a level of impact on communication closely related to the HoCoS score. This analysis also con- firms that the HoCoS does correspond to a level of impact on communication, thus validating the construction of this holistic indicator.

## DISCUSSION

### Psychometrics

A new index measuring communication impairment in patients treated for oral or oropharyngeal cancer was developed in this study.

Despite its construction from limited sample (n = 25), the HoCoS shows good performances in validity and relia- bility. On the one hand, the reliability in cross-validation is high (r<sub>S</sub> = 0.91). On the other hand, it is also a valid score, which measures the level of impact on communication, which was confirmed by the construction of a qualitative index by a latent profile analysis.

## Limitations

However, to better ensure generalizability of the results, this study needs to be completed.

First, increasing the sample size would allow a better statistical power and thus a better generalization of the results. Thus, this study concerns the first validation step of this innovative tool. This preliminary study will have to be followed by a validation of the HoCoS on a larger sample size. To date, there is no consensus on the number of sub- jects required for score validation. The inclusion of a new sample of about 100 subjects could thus be considered to ensure greater statistical robustness.

Then this score should be evaluated on a new sam- ple of patients for external validation. The analysis of the performance of the HoCoS on a new sample of patients
<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /><figcaption>FIGURE 4 Holistic communication scores (HoCoS) by subject</figcaption></figure>

<table><caption>TABLE 4 Comparison of latent classes predicted by latent profile analysis (LPA) analysis and the HoCoS score (subjects are ranked by increasing order of the HoCoS)</caption><thead><tr><th>Subject</th><th>Predicted class</th><th>HoCoS</th><th>Subject</th><th>Predicted class</th><th>HoCoS</th><th>Subject</th><th>Predicted class</th><th>HoCoS</th></tr></thead><tbody><tr><td>CMH07</td><td>1</td><td>77.81</td><td>CMV27</td><td>3</td><td>95.95</td><td>CFS10</td><td>2</td><td>104.97</td></tr><tr><td>CFP02</td><td></td><td>87.39</td><td>CMS19</td><td></td><td>96.12</td><td>PFP03</td><td></td><td>106.29</td></tr><tr><td>CMH25</td><td></td><td>87.96</td><td>CFA14</td><td></td><td>97.39</td><td>PMD23</td><td></td><td>106.78</td></tr><tr><td>CMB11</td><td></td><td>88.97</td><td>CMH08</td><td></td><td>97.88</td><td>CML21</td><td></td><td>109.96</td></tr><tr><td>CMP15</td><td></td><td>89.32</td><td>PMB01</td><td></td><td>99.26</td><td>PMS22</td><td></td><td>109.99</td></tr><tr><td>PMD06</td><td></td><td>90.55</td><td>CFW16</td><td></td><td>100.91</td><td>PMG09</td><td></td><td>111.41</td></tr><tr><td>CFR24</td><td></td><td>91.50</td><td>PME20</td><td></td><td>103.32</td><td>CFN26</td><td></td><td>111.77</td></tr><tr><td>CFC12</td><td></td><td>94.29</td><td></td><td></td><td></td><td>PMB18</td><td></td><td>112.62</td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td>PFG13</td><td></td><td>113.16</td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td>PFB04</td><td></td><td>114.44</td></tr></tbody></table>

would allow to ensure its reliability, and thus again its generalizability.

Finally, the temporal reliability of this score remains to be analysed. This point is closely related to the temporal reliability performance of the questionnaires from which the items used to calculate the HoCoS are taken. However, the non-preservation of all the items of the initial questionnaires modifies the global structure of these questionnaires, and the temporal reliability remains to be verified on specific items presented in a different order. The 24 items retained for the construction of the HoCoS

will thus have to be presented in two stages (D0 and D7) to a new sample of patients.

## Perspectives

### Speaker and interlocutor in communication situation

The holistic communication score is elaborated solely from items from self-reported questionnaires. The measure
<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /><figcaption>FIGURE 5 Scatterplot representing HoCoS and score predicted by cross-validation, with regression line</figcaption></figure>

of communication impairment is therefore only self-reported.

However, when measuring communication abilities, the speaker is a communication partner in the same way as the interlocutor. The latter is particularly important because communication is only effective if the message is not only correctly transmitted but also correctly received.

The inclusion in the HoCoS of an external measure of communication abilities by a listener would allow to consider other dimensions linked more globally to the impact of the disorder on the comprehensibility of speech (by a listener) (Pommée et al., 2021) and by ripple effect on communication and quality of life. Different tools could be used for this purpose, such as an evaluation by the listener using a visual analogue scale, or communicative dynamics questionnaires for example. The analysis of the results of a score combining internal and external measures according to the tools used could provide new insights into the dynamics of communication in a social context.

## Communication environment according to the bio-psychosocial models

Bio-psychosocial models such as Wilson's (Wilson & Cleary, 1995) represent the links between symptomatic (speech disorders) and functional (communication) status. According to these models, factors related to the characteristics of the individual or the environment can influence both the speech disorder and the communication abilities.

Taking into account these factors, such as the cognitive and anxiety-depressive state (Böhm et al., 2016; Eadie et al., 2018) the constitution of social circles around the patient (Danker et al., 2010) or the patient's self-perception

of the speech impairment (Bolt et al., 2016) would allow a better understanding of the functional dynamics in patients treated for oral cavity or oropharyngeal cancer.

More globally, the association of the holistic communication score with indicators related to the individual and his environment and a measure of the speech disorder could also allow a more effective prediction of the psychosocial impact of speech disorders in these patients, and their quality of life.

## Clinical implications

The HoCoS is an index that is voluntarily holistic in its construction, taking into account symptomatologic (e.g., item phif4 ‘I use a great deal of effort to speak’), interactional (e.g., item phic3 ‘I have trouble communicating with unfamiliar people’), pragmatic (e.g., ecvb25 ‘At the restaurant/coffee shop, do you find it difficult to place your order yourself?’), but also psycho-affective (e.g., dipd9 ‘I feel comfortable speaking in most situations both at home and outside’) dimensions. Strategies for compensating for communication difficulties are also considered (e.g., dipd3 ‘I try other ways of getting my message across when people don’t understand me’).

In our sample, 55% of the subjects included had speech therapy. However, speech therapy follow-up can modify patients' perception of their own communication abilities (Jacobi et al., 2010). The HoCoS would have a relevant clinical applicability in the rehabilitation process of head and neck cancer patients as one of the indicators to be considered by speech therapists, by providing a valid and reliable measure of progress in follow-up.

This indicator thus fills a gap in tools for measuring the functional consequences of oral cavity and oropharyngeal cancer treatment on speech and communication in daily care. The use of the HoCoS in clinical care would allow to better target the daily problems met by the patients in their communication with peers, and thus to better adapt the therapeutic strategies to their reported needs.

It thus seems interesting to let the patient in head and neck oncology to fill the HoCoS, that is, the 24 items from the ECVB, DIP, PHI and EORTC QLQ-H&N35 questionnaires. This score structured in few items allows to quickly target the impact of the speech disorder on communication. This measurement could be systematized in consultation or evaluation and would allow the therapeutic strategy to be adjusted as closely as possible to the patient's needs.

Finally, to consider the patients' quality of life, beyond the functional dimension evaluated by our new score, the HoCoS could thus be associated with specific quality of life
<figure><img src="image_17.png" /></figure>

questionnaires (Raquel et al., 2020) in a bio-psychosocial follow-up approach.

## CONCLUSIONS

A global score allowing a measurement of the impact of speech impairment on communication after treatment of oral or oropharyngeal cancer has been developed. The methodology of its construction allows a better reflection of the symptomatological, pragmatic and psychosocial elements leading to a degradation of communication abilities. The temporal reliability of this score and its external validity remains to be explored. Nevertheless, it fills the gap in the absence of this type of tool in head and neck oncology, and may allow a better understanding of the factors involved in the functional and psychosocial limitations of these patients.

## ORCID

Mathieu Balaguer https://orcid.org/0000-0003-1311-4501

Virginia Woisard https://orcid.org/0000-0003-3895-2827

## NOTES

## ACKNOWLEDGEMENTS

The authors acknowledge Benoît Lepage for his help in methodological considerations.

## CONFLICTS OF INTEREST

All authors declare that they have no conflict of interest.

## DATA AVAILABILITY STATEMENT

Data and the database are available from the corresponding author upon request.

## REFERENCES

Aaronson, N.K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N.J. et al. (1993) The European organization for research and treatment of cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. *JNCI Journal of the National Cancer Institute*, 85(5), 365–376. Available from: http://jnci.oxfordjournals.org/content/85/5/365.short

Acock, A.C. (2018) *A gentle introduction to stata*, 6th edition, Tex: StataCorp LP, p. 570.

Balaguer, M., Boisguérin, A., Galtier, A., Gaillard, N., Puech, M. & Woisard, V. (2019) Assessment of impairment of intelligibility and of speech signal after oral cavity and oropharynx cancer. *European Annals of Otorhinolaryngology, Head and Neck Dis-

eases*, 136(5), 355–359. Available from: https://doi.org/10.1016/j.anorl.2019.05.012

Balaguer, M., Champenois, M., Farinas, J., Pinquier, J. & Woisard, V. (2021) The (head and neck) carcinologic handicap index: validation of a modular type questionnaire and its ability to prioritise patients’ needs. *European Archives of Oto-Rhino-Laryngology*, 278(4), 1159–1169. Available from: http://link.springer.com/10.1007/s00405-020-06201-6

Balaguer, M., Farinas, J., Fichaux-Bourin, P., Puech, M., Pinquier, J. & Woisard, V. (2020) Validation of the French versions of the speech handicap index and the phonation handicap index in patients treated for cancer of the oral cavity or oropharynx. *Folia Phoniatrica et Logopäedica*, 72(6), 464–477. Available from: https://www.karger.com/Article/FullText/503448

Balaguer, M., Pommée, T., Farinas, J., Pinquier, J., Woisard, V. & Speyer, R. (2020) Effects of oral and oropharyngeal cancer on speech intelligibility using acoustic analysis: systematic review. *Head & Neck*, 42(1), 111–130. Available from: https://doi.org/10.1002/hed.25949

Baylor, C.R., Yorkston, K.M., Eadie, T.L., Miller, R.M. & Amtmann, D. (2009) Developing the communicative participation item bank: Rasch analysis results from a spasmodic dysphonia sample. *Journal of Speech, Language, and Hearing Research*, 52(5), 1302–1320. Available from: https://doi.org/10.1044/1092-4388/282009/07-0275/29

Bjordal, K., Hammerlid, E., Ahlner-Elmqvist, M., de Graeff, A., Boysen, M., Evensen, J.F. et al. (1999) Quality of life in head and neck cancer patients: validation of the European organization for research and treatment of cancer quality of life questionnaire-H&N35. *Journal of Clinical Oncology*, 17(3), 1008–1008. Available from: https://doi.org/10.1200/JCO.1999.17.3.1008

Böhm, N., Knipfer, C., Maier, A., Bocklet, T., Rohde, M., Neukam, F. et al. (2016) Sprechqualität und psychische Beeinträchtigung nach der Therapie von Mundhöhlentumoren. *Laryngo-Rhino-Otologie*, 95(09), 610–619. Available from: https://doi.org/10.1055/s-0042-102256

Bolt, S., Eadie, T., Yorkston, K., Baylor, C. & Amtmann, D. (2016) Variables associated with communicative participation after head and neck cancer. *JAMA Otolaryngology – Head & Neck*, 142(12), 1145–1151. Available from: https://doi.org/10.1001/jamaoto.2016.1198

Borggreven, P.A., Verdonck-De Leeuw, I.M., Muller, M.J., Heiligers, M., De Bree, R., Aaronson, N.K. et al. (2007) Quality of life and functional status in patients with cancer of the oral cavity and oropharynx: pretreatment values of a prospective study. *European Archives of Oto-Rhino-Laryngology*, 264(6), 651–657.

Borsboom, D. (2008) Latent variable theory. *Measurement: Interdisciplinary Research and Perspectives*, 6(1–2), 25–53. Available from: https://doi.org/10.1080/15366360802035497

Cai, L. (2012) Latent variable modeling. *Shanghai Archives of Psychiatry*, 24(2), 118–120.

Cameron, C. & Trivedi, P. (2005) *Microeconometrics: methods and applications*. Cambridge: Cambridge, p. 1056.

Carreira-Perpinán, M. (1997) *A review of dimension reduction techniques*. Sheffield: Department of Computer Science, The University of Sheffield Technical Report CS-96-09, pp. 1–69. Available from: http://www.pca.narod.ru/DimensionReductionBrifReview.pdf

Chadeau-Hyam, M., Campanella, G., Jombart, T., Bottolo, L., Portengen, L., Vineis, P. et al. (2013) Deciphering the complex-

<footnote n="1">¹Source: Globocan—Global Cancer Observatory 2020 World Health Organization, https://gco.iarc.fr/today/data/factsheets/cancers/3-Oropharynx-fact-sheet.pdf.</footnote>

<footnote n="2">²See https://www.irit.fr/rugbi/.</footnote>
<figure><img src="image_18.png" /></figure>

methodological overview of statistical models to derive OMICS-
based biomarkers. *Environmental and Molecular Mutagenes*,
54(7), 542–557. Available from: https://doi.org/10.1002/em.
20575

Coma, E., Ferran, M., Méndez, L., Iglesias, B., Fina, F., & Medina,
M. (2013) Creation of a synthetic indicator of quality of care as a
clinical management standard in primary care. *Springerplus*, 2(1),
51. Available from: https://doi.org/10.1186/2193-1801-2-51

Cunningham, P. (2014) Dimension reduction. *Machine learning techniques for multimedia*. Berlin, Heidelberg: Springer Berlin Heidelberg, p. 91–112. Available from: https://doi.org/10.1007/978-3-540-75171-7_4

Danker, H., Wollbrück, D., Singer, S., Fuchs, M., Brähler, E. & Meyer, A. (2010) Social withdrawal after laryngectomy. *European Archives of Oto-Rhino-Laryngology*, 267(4), 593–600. Available from: https://doi.org/10.1007/s00405-009-1087-4

Doward, L.C. & McKenna, S.P. (2004) Defining patient-reported outcomes. *Value Heal*, 7(Supplement 1), S4–S8. Available from: https://doi.org/10.1111/j.1524-4733.2004.7s102.x

Eadie, T., Faust, L., Bolt, S., Kapsner-Smith, M., Pompon, R.H., Baylor, C. et al. (2018) Role of psychosocial factors on communicative participation among survivors of head and neck cancer. *Otolaryngology - Head & Neck Surgery*, 159(2), 266–273. Available from: https://doi.org/10.1177/0194599818765718

Jacobi, I., van der Molen, L., Huiskens, H., van Rossum, M.A. & Hilgers, F.J.M. (2010) Voice and speech outcomes of chemoradiation for advanced head and neck cancer: a systematic review. *European Archives of Oto-Rhino-Laryngology*, 267(10), 1495–1505. Available from: https://doi.org/10.1007/s00405-010-1316-x

Landis, J.R. & Koch, G.G. (1977) The measurement of observer agreement for categorical data. *Biometrics*, 33(1), 159.

Letanneux, A., Walshe, M., Viallet, F. & Pinto, S. (2013) The dysarthria impact profile: a preliminary French experience with Parkinson's disease. *Parkinsons Disease*, 2013, 1–6. Available from: http://www.hindawi.com/journals/pd/2013/403680/

Lynn, M.R. (1986) Determination and quantification of content validity. *Nursing Research*, 35, 382–386. Available from: http://ijoh.tums.ac.ir/index.php/ijoh/article/view/26

Mazaux, J.-M., Daviet, J.-C., Darrigrand, B., Stuit, A., Muller, F., Dutheil, S. et al. (2006) Difficultés de communication des personnes aphasiques. *Évaluation des Troubles Neuropsychologiques en vie Quotidienne*, 73–82. Springer, Paris. http://ijoh.tums.ac.ir/index.php/ijoh/article/view/26

Meyer, T.K., Kuhn, J.C., Campbell, B.H., Marbella, A.M., Myers, K.B. & Layde, P.M. (2004) Speech intelligibility and quality of life in head and neck cancer survivors. *Laryngoscope*, 114(11 1), 1977–1981.

Middag, C., Martens, J.P., Van Nuffelen, G. & De Bodt, M. (2009) Automated intelligibility assessment of pathological speech using phonological features. *EURASIP Journal on Advances in Signal Processing*, 2009(5), Springer Science and Business Media LLC. https://doi.org/10.1155/2009/629030

Mlynarek, A., Rieger, J., Harris, J., O'Connell, D., Al-Qahtani, K., Ansari, K. et al. (2008) Methods of functional outcomes assessment following treatment of oral and oropharyngeal cancer: review of the literature. *Journal of Otolaryngology-Head and Neck Surgery*, 37(1), 2–10.

Mukaka, M.M. (2012) Statistics corner: a guide to appropriate use of correlation coefficient in medical research. *Malawi Medical Journal*, 24(3), 69–71.

Murphy, B.A., Ridner, S., Wells, N. Dietrich, M. (2007) Quality of life research in head and neck cancer: a review of the current state of the science. *Critical Reviews in Oncology/Hematology*, 62(3), 251–267.

Polit, D.F., Beck, C.T. & Owen, S.V. (2007) Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. *The Research in Nursing & Health*, 30(4), 459–467. Available from: https://doi.org/10.1002/nur.20199

Pommée, T., Balaguer, M., Maucclair, J., Pinquier, J. & Woisard, V. (2021) Intelligibility and comprehensibility: a Delphi consensus study. *The International Journal of Language & Communication Disorders*, 57(1), 21–41. Available from: https://doi.org/10.1111/1460-6984.12672

Raquel, A.C.S., Buzaneli, E.P., Silveira, H.S.L., Simões-Zenari, M., Kulcsar, M.A.V., Kowalski, L.P. et al. (2020) Quality of life among total laryngectomized patients undergoing speech rehabilitation: correlation between several instruments. *Clinics* (*Sao Paulo*), 75, e2035.

Roscoe, B.A., Hopke, P.K., Dattner, S.L. & Jenks, J.M. (1982) The use of principal component factor analysis to interpret particulate compositional data sets. *Journal of the Air Pollution Control Association*, 32(6), 637–642.

Walshe, M., Peach, R.K. & Miller, N. (2009) Dysarthria impact profile: development of a scale to measure psychosocial effects. *The International Journal of Language & Communication Disorders*, 44(5), 693–715. Available from: https://doi.org/10.1080/13682820802317536

Wilson, I.B. & Cleary, P.D. (1995) Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. *JAMA The Journal of the American Medical Association*, 273(1), 59–65. Available from: https://doi.org/10.1001/jama.273.1.59

Woisard, V., Balaguer, M., Fredouille, C., Farinas, J., Ghio, A., Lalain, M. et al. (2021) Construction of an automatic score for the evaluation of speech disorders among patients treated for a cancer of the oral cavity or the oropharynx: the Carcinologic Speech Severity Index. *Head & Neck*, 44(1), 71–88. Available from: https://doi.org/10.1002/hed.26903

World Health Organization (WHO). (2001) *International classification of functioning, disability and health: ICF*. WHO. Available from: https://apps.who.int/iris/handle/10665/42407

SUPPORTING INFORMATION

Additional supporting information can be found online
in the Supporting Information section at the end of this
article.

How to cite this article: Balaguer, M., Pinquier, J., Farinas, J. & Woisard, V. (2023) Development of a holistic communication score (HoCoS) in patients treated for oral or oropharyngeal cancer: Preliminary validation. *International Journal of Language & Communication Disorders*, 58, 39–51.
https://doi.org/10.1111/1460-6984.12766