# Disgust assessment: Factorial structure and psychometric properties of the French version of the Disgust Propension and Sensibility Scale Revised-12

Caroline Novara, Julie Boiché, Cindy Lebrun, Alexandra Macgregor, Yohan Mateo, Stephane Raffard

► To cite this version:

Caroline Novara, Julie Boiché, Cindy Lebrun, Alexandra Macgregor, Yohan Mateo, et al. Disgust assessment: Factorial structure and psychometric properties of the French version of the Disgust Propension and Sensibility Scale Revised-12. PLoS ONE, 2019, 14 (1), pp.e0210639. 10.1371/journal.pone.0210639. hal-02572460

HAL Id: hal-02572460

https://hal.science/hal-02572460

Submitted on 25 May 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
RESEARCH ARTICLE

# Disgust assessment: Factorial structure and psychometric properties of the French version of the Disgust Propension and Sensibility Scale Revised-12

Caroline Novara<sup>1,2,*</sup>, Julie Boiché<sup>1*</sup>, Cindy Lebrun<sup>1*</sup>, Alexandra Macgregor<sup>3‡</sup>, Yohan Mateo<sup>2‡</sup>, Stéphane Raffard<sup>1,3&</sup>

1 Univ Paul Valéry Montpellier 3, Univ. Montpellier, EPSYLON EA, Montpellier, France, 2 Groupe Ramsay Gds, Clinique RECH, Montpellier, France, 3 Service Universitaire de Psychiatrie Adulte, CHU Montpellier, Montpellier, France

© These authors contributed equally to this work.

‡ These authors also contributed equally to this work.

* caroline.novara@etu.univ-montp3.fr

<figure><img src="image_2.png" /></figure>

## Abstract

The present study examined the internal and external validity of the French version of the 12-item Disgust Propensity and Sensitivity Scale-Revised (DPSS-12) in a nonclinical sample from the general population. Two hundred and eighty-two participants completed the DPSSf-12 questionnaire as well as the Anxiety Sensitivity Index (ASI), Anxiety Trait (STAI B), Obsessional Belief Questionnaire 44 items (OBQ 44), Obsessive Compulsive Inventory-Revised (OCI-R) and Positive and Negative Affect Schedule (PANAS). Confirmatory Factor Analysis supported a 2-factor structure after two sensitivity items were removed. The 10-item scale showed good internal consistency, construct validity and test-retest reliability. These adequate psychometric properties make the DPSSf-10 appropriate for use by researchers and practitioners.

<figure><img src="image_3.png" /></figure>

OPEN ACCESS

**Citation:** Novara C, Boiché J, Lebrun C, Macgregor A, Mateo Y, Raffard S (2019) Disgust assessment: Factorial structure and psychometric properties of the French version of the Disgust Propension and Sensibility Scale Revised-12. PLoS ONE 14(1): e0210639. https://doi.org/10.1371/journal.pone.0210639

**Editor:** Francesca Chiesi, University of Florence, ITALY

**Received:** April 28, 2018

**Accepted:** December 28, 2018

**Published:** January 28, 2019

**Copyright:** © 2019 Novara et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

**Data Availability Statement:** All relevant data are within the paper and its Supporting Information files.

**Funding:** CIFRE device (2016/1119) signed with the psychiatric clinic RECH belonging to RAMSAY group, financed by the French Ministry of research and industry implemented by the French national association for research and technology.

**Competing interests:** The authors have declared that no competing interests exist.

## Introduction

Disgust is recognized as a universal emotion [1, 2], with distinct developmental features, behavioral, physiological dimensions and cognitive biases [3,4,5,6,7,8]. The theoretical model put forward by Rozin, Haidt, and McCauley [9] represents the main reference which has inspired literature on disgust over the last decades. From an evolutionary perspective, there is a broad consensus that disgust plays a key role in motivating behavior that reduces exposure to pathogens, and this concept has developed as a mediator of a dynamic adaptive system, a “behavioral immune system”, motivating disease avoidance [10, 11, 12, 13].

From a clinical perspective, disgust has been shown to be involved in the development and maintenance of several mental disorders including spider phobia [14, 15], contamination-based Obsessive-Compulsive Disorder (OCD; [16, 17]), Blood-Injury-Injection (BII) phobia [18], hypochondriasis [19], Post Traumatic Stress Disorder [20], sexual dysfunctions disorders [21] and eating disorders [22].
Despite the fact that disgust represents a public health concern [11], very few studies have examined disgust in certain countries, such as France. For example, it seems that emotional reactions linked to disgust played a major role regarding the consumption of beef during The Mad Cow disease crisis in France [23]. One of the reasons why French researchers and clinical practitioners have paid little attention to disgust is the lack of a formal measurement tool to assess individual's tendencies to disgust reactions. Such a tool would enable examining the role of disgust in some disgust-relevant psychopathological conditions, thus enabling to verify the applicability of scientific knowledge concerning the role of disgust in psychopathology as it was demonstrated in other cultural spheres. The construction of a standardized measure instrument for the French general population is thus necessary, because it will not only enable investigations of the specific characteristics of disgust in France, but also provide elements upon shared cross-cultural components of disgust.

It is now well admitted that disgust can be distinguished between Disgust Sensitivity (DS, i.e., the extent to which an individual is embarrassed to feel disgust) and Disgust Propensity (DP, i.e., trait disgust or the tendency to experience disgust frequently and intensely) [24, 25]. The Disgust Propensity and Sensitivity Scale (DPSS) has been specifically developed in English to assess both DS and DP and has been validated in various languages such as Japanese [26], Dutch [25], or Italian [27].

The original DPSS scale consisted of 32 items used to measure DS and DP with 16 items per factor. The scale demonstrated good psychometric properties, including a good internal consistency for both the total scale and its subscales [28]. Later factor analyses, however, conducted to the proposition of two abbreviated versions. First, Van overveld, De Jong, Peters, Cavanagh and Davey [25] proposed a 16-item scale (DPSS-R) with two subscales of 8 items for DS and DP factors. The item selection process was not only data driven, but also lead by theoretical considerations. Olatunji, Cisler, Deacon, Conolly, and Lohr [29] conducted exploratory analyses and found that 4 items loaded on a different factor, compared to the results obtained by Van overveld et al. [25]. Fergus and Valentiner [24] reported similar results, and suggested that a 12-item version, without these 4 items, guaranteed better psychometric properties. Taken together, the evidence provided by previous psychometric properties suggest that DPSS-12 is the most valid measure to assess sensitivity and propensity to disgust among available versions [30].

Recently, Goetz, Couple, and Lee [31] suggested that heterogeneous items included in the DPSS-12 question the adequacy of the scale. Indeed, although the DPSS-12 did reveal a DS and a DP factor, two items did not load on them. The authors pointed out that the DPSS-10 produces a better consistency than a one-dimension version, or a two-or three-factor version with the 12 items. Altogether, recent work on the DPSS-12 supported the presence of 2 distinct factors, DS and DP, for which good reliability was showed. However, the loading of all items within this, 12-item version appears susceptible to cultural specificities. Indeed, although basic emotions are expected to be similar among all individuals, certain emotional components are likely to be influenced by the environment and the individual's experiences [32]. It has been suggested that culture may influence the type of stimulus triggers, display codes, associated affects, and behavioral consequences [33]. Although general themes of disgust triggers seem to be consistent across cultures, some variations appear in the subjective experience of disgust emotion as well as in specific triggers [34, 35]. The peculiarity of cultural specificities in regard of the emotion of disgust has furthermore been advanced as a valid argument by Fergus and Valentiner [24] who assumed that the factorial differences observed in Dutch and American populations could partly be explained by cultural specificities. This underlines the importance of examining this issue specifically in the French population.

Significant associations with phobias related to disgust, and especially with contamination concerns and other relevant psychopathological traits (sensitivity to anxiety), have been
enlighten and allowed then to study the unity of these constructs under this framework. However, the heterogeneity of the psychometric data obtained suggests a need for clarification. Validating a French version of the DPSS-12 would bring new information on the validity of the analyzed constructs, their applicability and their generalization to a French general population. In order to assess the external validity of the scale, we chose to focus on obsessive-compulsive symptomatology in its behavioral and cognitive dimensions, as well as on anxiety. The cognitive models of OCD have been suggested to provide a useful starting point for the examination of potential thought processes in pathological disgust responses. These models posit that the vast majority of the population has intrusive, undesirable thoughts that are similar to the content of obsessions experienced by people with OCD [36]. It is therefore not the content of thoughts but the interpretation of the personal meaning of thoughts, or their responsibility for the perceived consequences causing distress and repetition, that characterize OCD [37]. Beliefs about intrusive obsessive thoughts have therefore been suggested as a natural parallel to interpretations of disgust reactions [38].

In previous studies, it has been suggested that high disgust propensity would play a specific role in OCD [29]. High DP was found to be significantly higher in OCD linked to contamination (C-OCD) than in other subtypes of OCD [39]. However, many studies that have investigated this issue only evaluated DP and did not include a measure of DS in their assessments [40, 41]. Recent research has suggested that DS may also be associated with C-OCD symptomatology and is, in addition, associated with a general inability to regulate one's emotions [42]. DP has been found to be uniquely associated with OCD when compared with general anxiety disorder, indicating that DP may be more specific to OCD than to generalized anxiety [43]. Moreover, a strong DS was correlated with a high sensitivity to anxiety, present in anxiety disorders, suggesting that the DS could be more general in anxiety disorder and less specific to C-OCD.

The objective of this study was to examine the psychometric properties of the French version of the DPSS-12. First, we examined factorial structure using Confirmatory Factor Analysis. More precisely, we tested the fit of a 2-factor model (i.e., DS and DP) containing 6 items each [24] in a large sample of French participants from the general population.

Next, we evaluated external validity and more precisely we hypothesized that DP would be more strongly associated with behavioral dimensions of OCD symptomatology as measured by OCI-R than DS, and that DS would be more strongly associated with cognitive dimensions of OCD symptomatology as assessed by OBQ-44 and cognitive sensitivity related to anxiety (ASI) than DP. Although previous studies suggest a strong association of C-OCD with anxiety [44], our assumption is that the relationship between washing compulsion and disgust will be maintained while controlling for anxiety.

Last, in order to examine the temporal reliability of the scale, we assessed disgust twice with a 2-month interval in the participants, using the French DPSS-12. Considering that disgust is conceived as a relatively stable individual disposition, we assumed that the scores would be stable over this short time period.

# Material and method

## Participants

Two hundred and eighty-four adults voluntarily completed self-reported questionnaires containing the 12 items of the DPSSR as the main measure of interest. All participants provided written informed consent, prior to the experiment. The research has been submitted to the members of the GCS Ramsay Générale de Santé for Education and Research Scientific Orientation Committee and has received a favorable opinion on 18/10/18. The committee testifies
that the study appears to be in accordance with the Scientifics principles generally accepted
and to the ethical standards of research. IRB's number which was assigned is: COS-RGDS-
2018-10-002-Avis IRB-NOVARA-C. We relied both on a secured online research platform set
up by the Epsylon laboratory and paper questionnaires filled in small groups to recruit partici-
pants from the general population (187 females, 72 males, 25 did not provide gender,
Mage = 31.39; SDage = 13.38 ranging from 19 to 77; 23,1% of whom had a level of education
lower than bachelor's degree and 62% of whom were undergraduate students).

**Measures and procedure.** In order to adapt the scale in French, a standard procedure of translation and back-translation was carried out. At first, two expert psychologists independently translated the scale from English to French. The different translations were compared to obtain a first consensual translation of the tool. A bilingual expert then translated the French version of the questionnaire into English. Finally, the back-translation was submitted and validated by Dr Van Overveld. In order to examine the stability of the scale over time, the DPSSf was administered twice two months apart. In addition, participants completed the following scales:

**Anxiety sensitivity Index Revised (ASI-R)** [45], was used to measure individual differ-
ences in sensitivity to anxiety. A 2-factor structure was observed for the French version of this
scale [46]: "Fear of the consequences related to the physical sensations of Anxiety"(CP)
includes 19 items, and "Fear of Social and Cognitive Consequences of Anxiety"(CSC) includes
17 items rated on a 5-point likert scale (from “very little” to “very much”). In this study, the
internal consistency of both subscales (CP subscale, Cronbach’s α = .87; CSC subscale, Cron-
bach’s α = .93) was adequate.

**State-Trait anxiety inventory (STAI-B)** [47] is a widely used inventory for assessing indi-
vidual differences in trait anxiety, containing 20 items rated on a 4-point Likert scale (from 1
"Not at all" to 4 "Extremely"). The internal consistency was satisfactory in the current sample
(Cronbach’s α = .89).

**Obsessional Belief Questionnaire 44 (OBQ-44)** [48] The OBQ-44 is a self-reported
44-item questionnaire used to assess the presence and severity of obsessive beliefs associated
with OCD rated on a 7-point likert scale (from “strongly disagree” to “strongly agree”). Three
subscales assess the beliefs associated with the over-importance of the need to control one’s
thoughts (control of thought), the beliefs about an increased sense of responsibility to prevent
danger or perceived threats (responsibility), and the beliefs about the need for perfectionism
and intolerance to uncertainty (perfectionism). In this sample, the internal consistency was
satisfactory (respectively, Cronbach’s α = .89, .89 and .88) for all subscales.

**Obsessive-compulsive Inventory-Revised (OCI-R)** [49] is an 18-item self-reported scale
used to measure the severity of obsessive symptoms, through six 3-item subscales: washing,
checking, obsessive thoughts, neutralization, hoarding and symmetry, rated on a 5-point likert
scale (from “not at all” to “extremely”). The internal consistency in this study ranged from .59
to .82 (M = .73)

**Positive affects and negative affects Schedule (PANAS)** [50] is a self-reported scale con-
taining 20 items measuring affective states. This scale contains two subscales assessing respec-
tively positive affects and negative affects. The PANAS items are scored on a 5-point Likert
scale (ranging from 1 "Not at all or very little" to 5 "Extremely"). The internal consistency of
the two subscales was satisfactory (PANAS-positive affect, Cronbach’s α = .82; PANAS-nega-
tive affect, Cronbach’s α = .87)

**Data analysis**

First, a series of preliminary analysis were run. We examined missing data and checked the
presence of univariate and multivariate outliers. The normality of the distribution was verified
through the distribution kurtosis and skewness for each item. Potential collinearity was tested
through Pearson correlations. Next, using R software [51], a Confirmatory Factorial Analysis
(CFA) for continuous data using a maximum likelihood estimation method with robust
(Huber-White) standard errors and a scaled test statistic that is (asymptotically) equal to the
Yuan-Bentler test statistic was applied to examine if a 2-factor model showed adequate fit with
the observed data. We estimated the model of interest through the commonly used indices, fol-
lowing Kline [52]: the Akaike information criterion (AIC), the comparative fit index (CFI),
the Tucker-Lewis index (TLI), the root mean square error approximation (RMSEA) RMSEA
value was supplemented with a confidence interval (CI). In presence of good fit, the suggested
lower limit of the RMSEA's CI falls at or below .05 and upper limit should not exceed .10.

There is no definitive cut-off point for these indices although recommended thresholds are: for AIC (the model with the weaker score show the best parsimony principle), CFI > .90, TLI > .90, RMSEA < .05. We further submitted the different models to the ratio Chi square on free-dom degree (χ²/df < 3.00).

Second, to test external validity, using SPSS, we examined the correlations between the
DPSSf factors and other measures of anxiety, and obsessive-compulsive symptomatology
through Pearson's correlations. We then tested the significance of differences between correla-
tion coefficients to shed light on the utility of the factorial solution proposed as presenting rele-
vant dimensions subscales of DPSSf through William's test. As the behavior commonly
associated with C-OCD is compulsive washing, we used regressions to measure the predictive
value of DS and DP on washing dimension of the OCI-R, controlling for the scores obtained
regarding anxiety sensitivity (ASI-R CP, ASI-R CSC), trait anxiety (STAI B), and cognitive dis-
tortions relevant for OCD (OBQ 44: perfectionism, responsibility and control of thoughts).

Last, in order to examine the temporal reliability of the scale we examined Intra-Class Coef-
ficients, considering values >.75 as satisfactory [53].

Results

Preliminary analyses

For the preliminary analysis of the data we used recommended criteria from Tabachnik and Fidell [54]. Two participants were excluded because they had more than 10% missing data, Multiple imputations, using the SPSS Impute Missing Data Values module, were operated for 10 participants who had less than 10% missing data. Examination of standardized scores revealed the presence of 14 simple outliers (standardized score ± 3.29). Their score were replaced by the corresponding extreme, within a range of normal value for the identified items. No response pattern indicated the presence of multiple outliers were (distance Mahalanobis significant at p < .001). The main analyses were launched on a final sample of 282 participants. Skewness values were ranged from -.707 to 1.798, and kurtosis values were comprised between -1.015 to 2.192, indicating a deviation from normality. Therefore, a Confirmatory Factor analysis using a maximum likelihood robust method was considered appropriate. Because all correlations were below .85, it was concluded that there was no multicollinearity in the dataset.

Confirmatory factor analysis

A two-factor model was tested (Model 1), including the estimation of the 12 target loadings, 2 factors variance, correlations between the 2 factors, as well as uniqueness values for all 12 items. For identification purpose, the loading between the first indicator of each latent construct and its target factor were fixed at 1.0. This model displayed only partly convincing fit Robust χ²(53) = 105, χ² / df = 1.98; AIC = 8334.3, Robust CFI = .91, Robust TLI = .89, Robust RMSEA = .06, 90% Robust CI RMSEA = [.04-.06]. An examination of the factor loadings
enabled to identify one item that did not show salient loading on the expected factor (see Table 1). Even if only item 12 was below this threshold in the 2\*6 factor model, when running a 11-item model, it appeared that (1) some indicators of fit were not entirely satisfactory (significant robust $\chi^2$) and (2) the factor loading of item 11 did not reach the cut-off anymore.

A two factor model (Model 2) including 6 items on the first factor (DP) and 4 items on the second (DS) was thus tested, leading to satisfactory fit indices: Robust $\chi^2 = 46.67$, df = 34, $\chi^2 / df = 1.40$; AIC = 8812.72, Robust CFI = .97, Robust TLI = .96, Robust RMSEA = .03, Robust CI RSMSEA = [.00-.04]. All 10 items demonstrated distinctive and salient loadings ranging from .46 to .72, with an average of .60 onto one of the two factors (see Table 1). The interfactor correlation between DS and DP was significant ($r = .39$). Cronbach's alpha values for the two subscales ($\alpha_{DP} = .77$; $\alpha_{DS} = .63$) and for the total score ($\alpha = .76$) were acceptable.

## External validity

The correlations between disgust scores and measures of anxiety (ISA, STAI) and obsessional-compulsive symptomatology (OCI-R, OBQ 44) are shown in Table 2. The total DPSSf-10

<table><caption>Table 1. Confirmatory factor analysis and item statistics of the disgust propensity and sensitivity scale revised-12 (DPSS-12) (N = 282).</caption><thead><tr><th>Items description</th><th>M1</th><th>M2</th></tr></thead><tbody><tr><td><strong>Disgust Propensity</strong></td><td></td><td></td></tr><tr><td>1. J'évite les choses dégoutantes.<br><em>I avoid disgusting things.</em></td><td>.58</td><td>.58</td></tr><tr><td>4. Je ressens de la répulsion.<br><em>I feel repulsed.</em></td><td>.67</td><td>.67</td></tr><tr><td>5. Les choses dégoutantes me retournent l'estomac.<br><em>Disgusting things make my stomach turn.</em></td><td>.64</td><td>.64</td></tr><tr><td>6. Je grimace quand je ressens du dégoût.<br><em>I screw up my face in disgust.</em></td><td>.53</td><td>.53</td></tr><tr><td>8. Je ressens du dégoût<br><em>I experience disgust.</em></td><td>.70</td><td>.70</td></tr><tr><td>10. Il y a des choses que je trouve dégoutantes.<br><em>I find something disgusting.</em></td><td>.53</td><td>.53</td></tr><tr><td><strong>Disgust Sensibility</strong></td><td></td><td></td></tr><tr><td>2. Quand je me sens dégouté, j'ai peur de m'évanouir.<br><em>When I feel disgusted, I worry that I might pass out.</em></td><td>.40</td><td>.46</td></tr><tr><td>3. J'ai peur quand j'ai la nausée.<br><em>It scares me when I feel nauseous.</em></td><td>.63</td><td>.68</td></tr><tr><td>7. Quand je m'aperçois que j'ai la nausée, j'ai peur de vomir.<br><em>When I notice that I feel nauseous, I worry about vomiting</em></td><td>.65</td><td>.72</td></tr><tr><td>9. Cela m'effraie de me sentir faiblir<br><em>It scares me when I feel faint.</em></td><td>.50</td><td>.46</td></tr><tr><td>11. Je me sens gêné(e) lorsque je me sens dégouté(e)<br><em>It embarrasses me when I feel disgusted.</em></td><td>.34</td><td>—</td></tr><tr><td>12. Je pense que ressentir du dégoût est mauvais pour moi<br><em>I think feeling disgust is bad for me.</em></td><td>.25</td><td>—</td></tr><tr><td><strong>α Propension</strong></td><td>.77</td><td>.77</td></tr><tr><td><strong>α Sensibility</strong></td><td>.57</td><td>.63</td></tr></tbody></table>

M1 = Model 1, DPSS-12, Fergus & Valentiner, 2009; M2 = Model 2, DPSSf-10

https://doi.org/10.1371/journal.pone.0210639.t001
<table><caption>Table 2. DPSSf-10 correlations with convergent and discriminant measures.</caption><thead><tr><th></th><th>SCALES</th><th>DP<sup>a</sup></th><th>DS<sup>a</sup></th><th>TOTAL SCORE<sup>a</sup></th><th>t value<sup>b</sup></th></tr></thead><tbody><tr><td>STAI B</td><td></td><td>.26**</td><td>.30**</td><td>.34**</td><td>-0.64</td></tr><tr><td>ASI</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>CS</td><td>.34**</td><td>.50**</td><td>.49**</td><td>-2.80**</td></tr><tr><td></td><td>CPC</td><td>.29**</td><td>.46**</td><td>.43**</td><td>-2.89**</td></tr><tr><td>OBQ</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>Responsability</td><td>.08</td><td>.22**</td><td>.17**</td><td>-2.17*</td></tr><tr><td></td><td>Control of thoughts</td><td>.20**</td><td>.07</td><td>.14*</td><td>2</td></tr><tr><td></td><td>Perfectionnism</td><td>.16**</td><td>.21*</td><td>.24**</td><td>-0.77</td></tr><tr><td>OCI-R</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>Checking</td><td>.10</td><td>.05</td><td>.09</td><td>0.76</td></tr><tr><td></td><td>Washing</td><td>.25**</td><td>.16*</td><td>.25**</td><td>1.40</td></tr><tr><td></td><td>Thought</td><td>.18**</td><td>.23**</td><td>.24**</td><td>-0.78</td></tr><tr><td></td><td>Neutralization</td><td>.04</td><td>.11</td><td>.09</td><td>-1.06</td></tr><tr><td></td><td>Symetry</td><td>.23**</td><td>.15*</td><td>.23**</td><td>1.24</td></tr><tr><td></td><td>Hoarding</td><td>.03</td><td>.11</td><td>.08</td><td>-1.22</td></tr><tr><td>PANAS</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>Positive Affects</td><td>-.14*</td><td>-.04</td><td>-.11</td><td>-1.53</td></tr><tr><td></td><td>Negative Affects</td><td>.18**</td><td>.29**</td><td>.27**</td><td>-1.88*</td></tr></tbody></table>

STAI B = State trait anxiety inventory; ASI CS = inventory sensitivity to anxiety fear of the social consequences; ASI CPC = inventory sensitivity to anxiety fear of the cognitive and physical consequences; OBQ = obsessive beliefs questionnaire; OCI = obsessive compulsive inventory; PANAS = positive and negative affects schedule; DP = Disgust Propension; DS = Disgust sensibility. (N = 282)

https://doi.org/10.1371/journal.pone.0210639.t002

score was significantly related to anxiety levels, behavioral symptoms and cognitive distortions of OCD. If the DP factor was strongly correlated with washing and symmetry dimensions of the OCI-R, the DS factor was strongly linked with the anxiety measures and the cognitive distortions of OCD.

A regression analysis was performed, entering simultaneously as predictor variables DS and DP scores, as well as negative affects, cognitive distortions of OCD (perfectionism, responsibility and control of thoughts), and anxiety trait scores. Together, the seven predictor variables explained a significant proportion of the variance in washing ($r^2$ = .20, F(8,256) = 7.93, p < .001) assessed with the OCI. After controlling for the other variables, the amount of unique variance of DP as a predictor for OCI washing score is statistically significant ($\beta$ = .19, t = 3.34, p = 0.001), in a model that also significantly include perfectionism and intolerance to uncertainty.

## Temporal reliability

The ICC computed between the DPSS total scores ($r$ = .77 CI [.61 - .86]; $p$ < .001) and the two subscales scores (DP, $r$ = .74 CI [.62 - .82]; $p$ < .001; DS, $r$ = .76 CI [.65 - .84]; $p$ < .001) were significant and high, suggesting adequate temporal stability.

## Discussion

The aim of this study was to examine the psychometric properties of the French version of the DPSS-12 in a sample of participants from the general population. After the removal of two

<footnote>a Pearson's correlation</footnote>

<footnote>b William's test for correlation comparison</footnote>

<footnote>** p < .001</footnote>

<footnote>*p < .05</footnote>
items, two latent constructs emerged from our analyses: (a) disgust propensity (DP), (b) dis-
gust sensitivity (DS). This 10-item factorial solution produced by Confirmatory Factor Analy-
sis provides a correctly fitted model across multiple indexes when compared to the 12-item
model of the English version.

Factor structure

Contrary to previous factor analyses offering a 12-item scale [24], items 11 and 12 appeared problematic. However, as previously observed by Goetz et al. [31], if items 11 et 12 do not function in the way they were originally intended by Fergus and Valentiner [24], a plausible explanation may be that they are less conceptually related to their purported factors compared to other items, and might be related to another disgust related dimension.

The DP dimension (items 1, 4, 5, 6, 8, 10) was stable as it has been previously shown in the different validations of this scale [24, 31, 30], and was correctly associated with the tendency to have overactivated reactions of disgust toward various stimuli. The DS dimension (items 2, 3, 7, 9) corresponds, in the French validation of the scale, to items that are specifically sensitive to autonomic activation, such as fear or nausea, linked to feelings of disgust (« *It scares me when I feel nauseous.*»).

Reliability and external validity

We examined the reliability of the sub-dimensions of the DPSSf-10 to estimate Cronbach's alpha coefficient as parameter of internal consistency, and intra-class correlation coefficient as the parameter of test-retest reliability. The results are satisfactory and represent a substantial improvement on the original DPSS-12. To test the convergent and discriminant validity of the DPSSf-10, correlations and regression analyzes were performed and showed that the two concepts, although correlated, have specific associations with distinct constructs. As hypothesized, and consistently with previous work [29], DP was more strongly associated with the behavioral dimension of OCD symptomatology. Also, the propensity factor was correlated with washing compulsion even when anxiety, cognitive distortions and negative affects were controlled. DS was more strongly associated with the cognitive dimensions of OCD symptomatology (responsibility and perfectionism), with anxiety trait, and with both physical concern and social and cognitive concerns of anxiety sensitivity, suggesting that DP may be more specific to C-OCD, whereas DS may be more generally associated with anxiety disorders. Finally, the DPSSf-10 showed no association with positive affects.

These results provide valid support for the construct validity of the DPSSf-10 and the con-
ceptual independence of its subscales. The consistency in findings across studies that the DP
and DS scales are uniquely related to different constructs firmly supports the notion that DP
and DS possess differential predictive value. Overall, these findings suggest that the DP and DS
scales adequately assess disgust per se and do not appear redundant with fear-based constructs
as assessed by anxiety trait or anxiety sensitivity scales. The use of this scale therefore opens up
the possibility for researchers to examine the similarities and differences in the spectrum of
anxiety disorders with particular attention to the behavioral characteristics and somatic disgust
sensitivity.

Limitations

First, as mentioned above, it has been suggested that the heterogeneity of the results in charac-
terizing the experience of disgust could be attributed to cross-cultural differences [55, 56].
Thus, the proposed DPSSf-10 in this article might reflect a specific cultural trait in the treat-
ment, recognition, conceptualization and verbalization of disgust. In fact, its validity is limited
to the French general population. However, the DPSSf-10 may be useful for future research in clinical populations or other groups of healthy subjects to assess the invariance of scale measures. Furthermore, our sample was relatively young and included a large majority of females. A study involving a more representative sample of the French population, including more males and elderly people, would further help understand the experiences of disgust sensitivity and disgust propensity in these populations. Finally, the presence of common method variance (i.e., self-report) may have inflated the correlations among the study variables.

**Perspectives.** Berle, Phillips [57], Olatunji [58, 59] suggested that a complete emotional assessment must take into account cognitive (threat interpretation), behavioral (safety behavior) and physiological (nervous parasympathetic system activation patterns) dimensions to adequately explore the disgust phenomenon. Yet, the French validation of the scale excluded two items representative of cognitive sensibility (items 11 and 12). In the previous validation of DPSS-12 [31] the original sensitivity factor was decomposed into two sub-dimensions including somatic sensitivity and ruminative / self-disgust, comprising these two items, describing more specific cognitive sensibility. This factorial solution, although theoretically justified, seems nevertheless to be statistically fragile. A refinement of the scale in line with these recommendations, by inserting items of a nature to reflect the cognitive assessments that individuals make of an experience of disgust, could thus be relevant.

**Conclusion**

Disgust is a complex emotion and plays an important role in the development and maintenance of various psychopathological conditions and specifically C-OCD. The development of adequate tools providing a dimensional assessment of the forms of disgust regardless of their context of appearance is of importance not only to increase the volume of research on this subject but also to improve the management of this emotion that patients as well as practitioners are not used to identify. Provide a standardized index of the magnitude of the emotional experience regardless of its onset context, taking into account its cognitive, behavioral, and physiological dimensions, is an important effort that research must carry out. Evaluation tools should continue to be optimized to provide valuable data to guide and adjust clinical interventions.

**Supporting information**

S1 Data. Complete raw data can be found on COMPLETE DATA DPSSf. (XLSX)

**Author Contributions**

**Conceptualization:** Caroline Novara, Alexandra Macgregor.

**Data curation:** Caroline Novara.

**Formal analysis:** Julie Boiché, Cindy Lebrun.

**Funding acquisition:** Caroline Novara.

**Methodology:** Julie Boiché.

**Project administration:** Yohan Mateo, Stéphane Raffard.

**Supervision:** Stéphane Raffard.

**Validation:** Stéphane Raffard.

**Visualization:** Stéphane Raffard.
**Writing – original draft:** Caroline Novara.

**Writing – review & editing:** Caroline Novara, Julie Boiché, Cindy Lebrun, Alexandra Macgor, Yohan Mateo, Stéphane Raffard.

## References

1. Darwin C, Ekman P. The expression of the emotions in man and animals, 3rd ed. New York, NY: Oxford University Press; 1872.

2. Ekman P. Universal facial expressions in emotion. Studia Psychologica. 1973; 15(2):140–7.

3. Chapman HA, Kim DA, Susskind JM, Anderson AK. In bad taste: Evidence for the oral origins of moral disgust. Science. 2009; 323(5918):1222–6. https://doi.org/10.1126/science.1165565 PMID: 19251631

4. Ekman P, Levenson RW, Friesen WV. Autonomic nervous system activity distinguishes among emotions. Science. 1983; 221(4616):1208–10. PMID: 6612338

5. Olatunji BO, Cisler J, McKay D, Phillips ML. Is disgust associated with psychopathology? Emerging research in the anxiety disorders. Psychiatry Research. 2010; 175(1–2):1–10. https://doi.org/10.1016/j.psychres.2009.04.007 PMID: 19969378

6. Rozin P., Haidt J., &McCaulley CR Disgust. In: Lewis JM, Haviland-Jones LF, Barrett, editors. Handbook of emotions (2nd ed). New-York, NY, US: Guilford Press; 2008. pp.757–776

7. Rozin P, Lowery L, Ebert R. Varieties of disgust faces and the structure of disgust. Journal of Personality and Social Psychology. 1994; 66(5):870–81. PMID: 8014832

8. Stark R, Walter B, Schienle A, Vaitl D. Psychophysiological Correlates of Disgust and Disgust Sensitivity. Journal of Psychophysiology. 2005; 19(1):50–60.

9. Rozin P., Haidt J., &McCaulley CR Disgust. In: Lewis JM, Haviland-Jones LF, Barrett, editors. Handbook of emotions. New-York, NY, US: Guilford Press; 2000. pp. 637–653

10. Curtis V, Aunger R, Rabie T. Evidence that disgust evolved to protect from risk of disease. Proceedings Biological sciences. 2004; 271 Suppl 4(Suppl 4):S131–3.

11. Curtis V, Barra Md, Aunger R. Disgust as an adaptive system for disease avoidance behaviour. 2011; 366(1563):389–401. https://doi.org/10.1098/rstb.2010.0117 PMID: 21199843

12. Oaten M, Stevenson RJ, Case TI. Disgust as a disease-avoidance mechanism. Psychological Bulletin. 2009; 135(2):303–21. https://doi.org/10.1037/a0014823 PMID: 19254082

13. Tybur JM, Lieberman D, Griskevicius V. Microbes, mating, and morality: Individual differences in three functional domains of disgust. Journal of Personality and Social Psychology. 2009; 97(1):103–22. https://doi.org/10.1037/a0015474 PMID: 19586243

14. Davey GCL, Marzillier S. Disgust and animal phobias. Disgust and its disorders: Theory, assessment, and treatment implications. Washington, DC: American Psychological Association; 2009. p. 169–90.

15. Olatunji BO. Evaluative learning and emotional responding to fearful and disgusting stimuli in spider phobia. Journal of Anxiety Disorders. 2006; 20(7):858–76. https://doi.org/10.1016/j.janxdis.2006.01.005 PMID: 16504462

16. McKay D, Moretz MW. The intersection of disgust and contamination fear. Disgust and its disorders: Theory, assessment, and treatment implications. Washington, DC: American Psychological Association; 2009. p. 211–27.

17. Olatunji BO, Williams NL, Lohr JM, Sawchuk CN. The structure of disgust: Domain specificity in relation to contamination ideation and excessive washing. Behaviour Research and Therapy. 2005; 43(8):1069–86. https://doi.org/10.1016/j.brat.2004.08.002 PMID: 15967177

18. Page AC, Tan BJ. Disgust and blood-injury-injection phobia. Disgust and its disorders: Theory, assessment, and treatment implications. Washington, DC: American Psychological Association; 2009. p. 191–209.

19. Davey GCL, Bond N. Using controlled comparisons in disgust psychopathology research: The case of disgust, hypochondriasis and health anxiety. Journal of Behavior Therapy and Experimental Psychiatry. 2006; 37(1):4–15. https://doi.org/10.1016/j.jbtep.2005.09.001 PMID: 16253205

20. Engelhard IM, Olatunji BO, de Jong PJ. Disgust and the development of posttraumatic stress among soldiers deployed to afghanistan. Journal of Anxiety Disorders. 2011; 25(1):58–63. https://doi.org/10.1016/j.janxdis.2010.08.003 PMID: 20800428

21. De Jong PJ, Peters ML. Sex and the sexual dysfunctions: The role of disgust and contamination sensitivity. Disgust and its disorders: Theory, assessment, and treatment implications. Washington, DC: American Psychological Association; 2009. p. 253–70.
22. Troop N, Baker A. Food, body, and soul: The role of disgust in eating disorders. Disgust and its disorders: Theory, assessment, and treatment implications. Washington, DC: American Psychological Association; 2009. p. 229–51.

23. Sinaceur M, Heath C, Cole S. Emotional and Deliberative Reactions to a Public Crisis: Mad Cow Disease in France. Psychological Science. 2005; 16(3):247–54. https://doi.org/10.1111/j.0956-7976.2005.00811.x PMID: 15733207

24. Fergus TA, Valentiner DP. The Disgust Propensity and Sensitivity Scale-Revised: An examination of a reduced-item version. Journal of Anxiety Disorders. 2009; 23(5):703–10. https://doi.org/10.1016/j.janxdis.2009.02.009 PMID: 19278821

25. Van Overveld WJM, de Jong PJ, Peters ML, Cavanagh K, Davey GCL. Disgust propensity and disgust sensitivity: Separate constructs that are differentially related to specific fears. Personality and Individual Differences. 2006 Nov; 41(7):1241–52.

26. Iwasa K, Tanaka T, Yamada Y. Factor Structure, Reliability, and Validity of the Japanese Version of the Disgust Propensity and Sensitivity Scale-Revised. PLOS ONE. 2016; 11(10):e0164630. https://doi.org/10.1371/journal.pone.0164630 PMID: 27732659

27. Martoni RM, Rancoita PMV, Di Serio C, Brombin C. Validating the Italian version of the Disgust and Propensity Scale-Revised. Frontiers in Psychology. 2017; 8.

28. Cavanagh K, and Davey GCL. The development of a measure of individual differences in disgust. Paper Presented to the British Psychological Society, Winchester. 2000

29. Olatunji BO, Cisler JM, Deacon BJ, Connolly K, Lohr JM. The Disgust Propensity and Sensitivity Scale-Revised: Psychometric properties and specificity in relation to anxiety disorder symptoms. Journal of Anxiety Disorders. 2007; 21(7):918–30. https://doi.org/10.1016/j.janxdis.2006.12.005 PMID: 17236747

30. Viar-Paxton M, Olatunji BO, Measurement of Disgust Proneness. In: Meiselman HL, editor. Emotion Measurement: Woodhead Publishing; 2016. p. 513–35.

31. Goetz AR, Cougle JR, Lee H-J. Revisiting the factor structure of the 12-item Disgust Propensity and Sensitivity Scale—Revised: Evidence for a third component. Personality and Individual Differences. 2013; 55(5):579–84.

32. Abe JAA, Izard CE. The developmental functions of emotions: An analysis in terms of differential emotions theory. Cognition and Emotion. 1999; 13(5):523–49.

33. Rosenberg EL, Ekman P. Coherence between expressive and experiential systems in emotion. Cognition and Emotion. 1994 May; 8(3):201–29.

34. Haidt J, Rozin P, McCauley C, Imada S. Body, psyche, and culture: The relationship between disgust and morality. Psychology and Developing Societies. 1997; 9(1):107–31.

35. Tsai JL, Chentsova-Dutton Y, Freire-Bebeau L, Przymus DE. Emotional expression and physiology in European Americans and Hmong Americans. Emotion. 2002; 2(4):380–97. PMID: 12899371

36. Abramowitz JS, Fabricant LE, Taylor S, Deacon BJ, McKay D, Storch EA. The relevance of analogue studies for understanding obsessions and compulsions. Clinical Psychology Review. 2014; 34(3):206–17. https://doi.org/10.1016/j.cpr.2014.01.004 PMID: 24561743

37. Rachman S., Shafran R. Cognitive and behavioral features of obsessive-compulsive disorder. In: Swinson RP, Antony M, Rachman S, Richter MA, Editors. Obsessive-compulsive disorder: Theory, research, and treatment. New York, NY, US: Guilford Press; 1998. pp. 51–78

38. Teachman BA. Pathological Disgust: In the Thoughts, Not the Eye, of the Beholder. Anxiety, Stress & Coping: An International Journal. 2006 Dec; 19(4):335–51.

39. Woody SR, Tolin DF. The relationship between disgust sensitivity and avoidant behavior: Studies of clinical and nonclinical samples. Journal of Anxiety Disorders. 2002; 16(5):543–59. PMID: 12396211

40. Cisler JM, Reardon JM, Williams NL, Lohr JM. Anxiety sensitivity and disgust sensitivity interact to predict contamination fears. Personality and Individual Differences. 2007; 42(6):935–46.

41. Rozin P, Taylor C, Ross L, Bennett G, Hejmadi A. General and specific abilities to recognise negative emotions, especially disgust, as portrayed in the face and the body. Cognition and Emotion. 2005; 19 (3):397–412. https://doi.org/10.1080/02699930441000166 PMID: 22686649

42. Olatunji BO, Lohr JM, Smits JAJ, Sawchuk CN, Patten K. Evaluative conditioning of fear and disgust in blood-injection-injury phobia: Specificity and impact of individual differences in disgust sensitivity. Journal of Anxiety Disorders. 2009; 23(2):153–9. https://doi.org/10.1016/j.janxdis.2008.06.002 PMID: 18650059

43. Olatunji BO, Tart CD, Ciesielski BG, McGrath PB, Smits JAJ. Specificity of disgust vulnerability in the distinction and treatment of OCD. Journal of Psychiatric Research. 2011; 45(9):1236–42. https://doi.org/10.1016/j.jpsychires.2011.01.018 PMID: 21353249
44. Armstrong T, Zald DH, Olatunji BO. Attentional control in OCD and GAD: Specificity and associations with core cognitive symptoms. Behaviour Research and Therapy. 2011; 49(11):756-62. https://doi.org/10.1016/j.brat.2011.08.003 PMID: 21871608

45. Reiss S, Peterson RA, Gursky DM, McNally RJ. Anxiety sensitivity, anxiety frequency and the predictions of fearfulness. Behaviour Research and Therapy. 1986; 24(1):1-8. PMID: 3947307

46. Bouvard M, Ayxères-Vighetto A, Dupont H, Aupetit J, Portalier S, Arrindell W. Index de la sensibilité à l'anxiété révisé (anxiety sensibility index revised): Validation préliminaire de la version française. L'Encéphale: Revue de psychiatrie clinique biologique et thérapeutique. 2003; 29(2):157-64.

47. Spielberger CD, Gorsuch RL, & Lushene RE. STAI. Manual for the State-Trait Anxiety Inventory (Self-evaluation Questionnaire). Palo Alto California: Consulting Psychologists Press. 1983

48. Steketee G. Psychometric validation of the obsessive belief questionnaire and interpretation of intrusions inventory—part 2: Factor analyses and testing of a brief version. Behaviour Research and Therapy. 2005; 43(11):1527-42. PMID: 16299894

49. Foa EB, Huppert JD, Leiberg S, Langner R, Kichic R, Hajcak G, et al. The Obsessive-Compulsive Inventory: Development and validation of a short version. Psychological Assessment. 2002; 14(4):485-96. PMID: 12501574

50. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology. 1988; 54(6):1063-70. PMID: 3397865

51. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2012. ISBN 3-900051-07-0, URL http://www.R-project.org/

52. Kline TJB. Psychological testing: A practical approach to design and evaluation. Thousand Oaks, CA: Sage Publications, Inc; 2005.

53. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment. 1994; 6(4):284-90.

54. Tabachnick BG, Fidell LS. Using Multivariate Statistics (5th ed.). New York: Allyn and Bacon. 2007

55. Olatunji BO, Lohr JM, Sawchuk CN, Tolin DF. Multimodal assessment of disgust in contamination-related obsessive-compulsive disorder. Behaviour Research and Therapy. 2007; 45(2):263-76. https://doi.org/10.1016/j.brat.2006.03.004 PMID: 16697976

56. Soto JA, Lee EA, Roberts N. Convergence in feeling, divergence in physiology: How culture influences the consequences of disgust suppression and amplification among European Americans and Asian Americans. Psychophysiology. 2016 Jan 1; 53(1):41-51. https://doi.org/10.1111/psyp.12579 PMID: 26681616

57. Berle D, Phillips ES. Disgust and Obsessive-Compulsive Disorder: An Update. Psychiatry: Interpersonal and Biological Processes. 2006; 69(3):228-38.

58. Elwood LS, Olatunji BO. A cross-cultural perspective on disgust. Disgust and its disorders: Theory, assessment, and treatment implications. Washington, DC: American Psychological Association; 2009. p. 99-122.

59. Olatunji BO, Ebesutani C, Kim EH. Does the measure matter? On the association between disgust proneness and OCD symptoms. Journal of Anxiety Disorders. 2016; 44:63-72. https://doi.org/10.1016/j.janxdis.2016.10.010 PMID: 27776255