<figure><img src="image_1.png" /></figure>

Check for updates

OPEN ACCESS

EDITED BY
Yann Hode,
Association psychoeducation Profamille,
France

REVIEWED BY
Pasquale Pezzella,
University of Campania Luigi Vanvitelli, Italy
Teresa Sanchez-Gutierrez,
University of Cordoba, Spain

*CORRESPONDENCE

Romain Rey

<figure><img src="image_2.png" /></figure>

romain.rey@ch-le-vinatier.fr

RECEIVED 22 January 2024

ACCEPTED 19 April 2024

PUBLISHED 09 May 2024

CITATION

Lespine L-F, de Martène B, Zeltner B, Chenu B, Berbey CD and Rey R (2024) Leo program, a short multi-family skill-based psychoeducational program for caregivers of relatives living with a severe mental disorder: a retrospective pilot study. *Front. Psychiatry* 15:1374540.
doi: 10.3389/fpsyt.2024.1374540

COPYRIGHT

© 2024 Lespine, de Martène, Zeltner, Chenu, Berbey and Rey. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Leo program, a short multi-family skill-based psychoeducational program for caregivers of relatives living with a severe mental disorder: a retrospective pilot study

Louis-Ferdinand Lespine<fnref n="1" />,², Bénédicte de Martène<fnref n="1" />,
Blandine Zeltner³, Bénédicte Chenu⁴,
Céline Dubien Berbey<fnref n="1" /> and Romain Rey<fnref n="1" />,⁵,⁶*

**Background:** Caring for a relative with a severe mental disorder (SMD) is associated with high levels of burden and poor physical and mental health. There is a dire need for family psychoeducational programs that can be provided as early as possible. This manuscript describes the pilot testing of “Leo” a motivational-based psychoeducational program for caregivers of individuals with a SMD. The Leo program aims to provide caregivers with skills to best support their relative and to adopt self-care behaviors.

**Methods:** We retrospectively analyzed medical records of caregivers who enrolled in a short, multi-family, skill-based psychoeducational program, consisting of eight 3-hour sessions over 8 weeks. Outcomes of interest included: i) adherence to the program, ii) satisfaction and perceived usefulness, and iii) pre-post changes in self-reported levels of depression (CES-D), burden (ZBI), and skills (10 Likert-scaled items). A network analysis was used to investigate the relationships between pre-post changes in self-evaluated skills and pre-post changes in burden and depression levels.

**Results:** Over the 91 enrolled participants, 87 (95.6%) completed the program attending at least 5/8 sessions, 80.5% attending all sessions. Seventy-six caregivers fulfilled the questionnaires at baseline and after the program, and were included in the analysis. Although there was no evidence for significant change in self-reported depression levels (Cohen's d=0.19, p=0.210), burden scores and all evaluated skills were improved post-intervention, with medium to strong effect size (Cohen's ds from 0.47 to 0.87; p<0.001). Network output indicated that increased self-evaluated competence in 5 skills were associated

<footnote n="1">¹Center for Caregivers in Psychiatry of Lyon, Le Vinatier Hospital, Bron, France, ²Division for Clinical Research and Innovation, Le Vinatier Hospital, Bron, France, ³School of Medicine, UFR Simone Veil, Université Versailles Saint-Quentin-en-Yvelines (Paris Saclay University), Paris, France, ⁴Collectif Schizophrénies, Paris, France, ⁵Fondation FondaMental, Créteil, France, ⁶University Lyon 1, CNRS, INSERM, Lyon Neuroscience Research Center U1028 UMR5292, PSYR2, Bron, France</footnote>
with a global improvement in caregivers' burden and/or depression scores. Post-intervention, 89.7% of caregivers were "very satisfied" and 82.1% found the program "extremely useful".

**Conclusion:** This pilot retrospective study shows high levels of satisfaction, perceived usefulness, and adherence to "Leo", a short, multi-family, skill-based psychoeducational program with promising results in improving caregivers' burden, self-evaluated competence in coping with caregiving demands and in self-care behaviors. This study provides preliminary insights into the mechanisms by which family psychoeducation might alleviate burden of care. A larger-scaled, controlled, randomized study with follow-up assessments is warranted to determine how burden, depression, and skills, as well as their inter-relationships, evolve over time.

KEYWORDS

caregivers, family psychoeducation, skills, severe mental disorder, burden

# 1 Introduction

Support for individuals living with a severe mental disorder (SMD) is often provided by family caregivers. Since family caregivers fulfill multiple roles in the care of their relative, they are key actors for promoting their recovery (1). However, caring for a relative with a SMD often results in high levels of *burden*, which entails the physical, psychological, social and financial consequences borne by family caregivers (2, 3). As compared to the general population, family caregivers are more likely to experience social isolation, low quality of life, and poor physical and mental health (2–5). In turn, this increased distress and burden impede caregivers’ ability to provide help that their relative may need to achieve recovery goals.

In this light, various interventions have been created to support caregivers, empower them and prevent burden development. Among such resources, *family psychoeducation* (FPE) is an evidence-based practice (6) and an important part of optimal treatment for people with psychotic, bipolar or depressive disorders (7–9). FPE aims to provide caregivers with knowledge and skills to better support their relative with a SMD as well as to maintain their own well-being. FPE is one of the most effective psychosocial interventions for relapse and re-admission prevention in schizophrenia (10) with effect sizes equating to those obtained by antipsychotic medication (1). FPE is also associated with improved adherence to medication (11), better social functioning and increased employment (1, 11). Although less studied, positive

outcomes on family have been reported including reduction in burden and psychological distress, and improvement in knowledge, coping and family functioning (12–15).

Although meta-analyses and international guidelines advocate that FPE should be provided early and systematically (9, 10, 16–20), it remains insufficiently available. For instance, among the 4.5 million French caregivers in psychiatry, only 3% have benefited from FPE, provided on average 10 years after the disease onset (21, 22). In France, FPE is mainly provided through long-lasting, diagnostic specific programs. Although highly efficient, such programs remain scarce and many caregivers in early settings are reluctant to attend them. Therefore, further development of brief FPE and research into its benefits is warranted (23, 24).

In a participatory approach with family associations (i.e., peer-led organizations supporting caregivers of individuals living with a SMD), we recently co-designed and implemented in our psychiatry department a short, multi-family, multi-diagnostic, FPE program called "Leo". This program is designed to provide caregivers with day-to-day skills based on two complementary motivational approaches: Motivational Interviewing (MI) and the Zurich Resource Model (ZRM). MI is a well-established strategy for facilitating behavior (25). It includes a set of communication tools designed to decrease defensiveness and rigidity (26). Although originally developed for health professionals, non-professionals have been trained to MI with positive effects on risky health behaviors (27, 28). Training caregivers to MI is considered a promising approach to improve their relationships with a relative living with recent onset schizophrenia (29) or early psychosis (24). The ZRM, a self-management training concept, was incorporated to help caregivers durably adopt self-care behaviors. The ZRM relies on the development of motto-goals to activate a person's resources towards complex situations while activating intrinsic motivation (30, 31). The ZRM has shown positive effects on stress and affect regulation abilities in various target groups such as subjects

<footnote>**Abbreviations:** CES-D, Center of Epidemiological Studies – Depression; FPE, Family Psychoeducation; MI, Motivational Interviewing; RCT, Randomized Controlled Trial; SMD, Severe Mental Disorder; ZBI, Zarit Burden Interview; ZRM, Zurich Resource Model.</footnote>
with eating disorders or asthma and individuals at risk for burnout syndrome (32–35).

The aim of this retrospective pilot investigation, based on a single-group pre-post quasi-experimental design, was to assess the satisfaction, adherence rate and potential effectiveness of the Leo program on self-reported levels of burden, depression, and skills, among caregivers of a relative living with a SMD.

# 2 Materials and methods

## 2.1 Participants

From March 2021 to April 2023, a total of 91 caregivers participated in the Leo program which was performed 10 times. The participants of this study met the following criteria: i) relatives of individuals living with a severe mental disorder (first episode of psychosis, schizophrenia or schizoaffective disorder, bipolar disorder, major depressive disorder, anxiety disorder, borderline personality disorder, according to the Diagnostic and Statistical Manual (DSM-5), based on the consensus rating by the psychiatrists in charge). Noteworthy, some caregivers didn't know the precise diagnosis of their relative (when the onset of the disorder was recent or when the diagnosis had changed over time or because their relative had not shared it with them). In such a situation, caregivers could still participate in the program when they reported that their relative was receiving/had received substantial psychiatric care, ii) relatives of individuals who were inpatients or outpatients, iii) aged 18 years or older. There were no exclusion criteria.

## 2.2 The family psychoeducation program

The Leo program was co-designed with members of family associations in a participatory research approach. It consists of eight 3-hours-sessions, over the course of 8 to 10 weeks, in groups of 8 to 10 participants (caregivers). The first session is led by a psychiatrist and consists in a theoretical course on medical knowledge about SMD. Sessions 2 to 5 provide competences to better support one's relative. During these sessions, caregivers practice motivational interview techniques: asking open-ended questions, making affirmations, using reflections, summarizing, giving advice in order to facilitate behavioral changes. Caregivers are also trained to express dissatisfaction, to set limits or optimize a request. During the last 3 sessions (sessions 6 to 8), caregivers are trained to the self-management ZRM technique: they progressively build their own motto-goal to sustainably adopt a new attitude toward difficult situations or to approach a personal goal. The program is led by two professionals (typically a psychologist and a psychiatric nurse) present during the entire program. Three months after the eighth session, caregivers are invited to attend an optional session which allows them to re-train the competences previously taught in the form of solving concrete situations. To promote competence acquisition, a significant proportion of time is devoted to active training. Indeed, sessions 2 to 8 are organized as follows: 1 hour of theoretical input and 2 hours of practical training. During the

program, caregivers are regularly divided in groups of 4, allowing simulation-based training techniques such as peer role-play and simulated patients and fostering peer support and empowerment.

## 2.3 Measures and procedure

Caregivers who registered in the psychoeducational program completed self-reported questionnaires evaluating their levels of depression, burden, and perceived competences, before the first (pre-questionnaire) and immediately after the last (post-questionnaire) session. Socio-demographics (age, gender), relationships to the relative, and relative's diagnosis were collected before the first session. Measures of satisfaction and perceived usefulness (introduced after the program started, in late 2022) were evaluated immediately after the last session.

### 2.3.1 Depression

Depression symptoms were assessed using the 20-item Center of Epidemiological Studies - Depression (CES-D) scale (36, 37), a self-report measure where participants indicate how often over the past week various statements such as 'I felt lonely' or 'I felt sad' applied to them. Each item is rated on a 4-point Likert scale that ranges from 0 ('rarely' or 'none of the time') to 3 ('most of the time'), total scores ranging from 0 to 60. Cronbach's α of the CES-D in the present study was 0.92.

### 2.3.2 Burden

Caregiving burden was assessed using the 22-item version of the Zarit Burden Interview (ZBI) (38, 39), a self-report measure where individuals indicate how often various statements such as 'you feel that your relative asks for more help than he/she needs' or 'you feel embarrassed about your relative's behavior' applied to them. Each item is rated on a 5-point Likert scale that ranges from 0 ('never') to 4 ('nearly always'), total scores ranging from 0 to 88. Cronbach's α of the ZBI in the present study was 0.90.

<table><caption>TABLE 1 Competences self-evaluated at pre- and post-intervention.</caption><thead><tr><th></th><th>On the scale of 0 ('none') to 10 ('complete'), how would you rate your competence...</th></tr></thead><tbody><tr><td>C1</td><td>To communicate with your relative</td></tr><tr><td>C2</td><td>To set limits</td></tr><tr><td>C3</td><td>To make a request to the care team</td></tr><tr><td>C4</td><td>To make a request to your relative</td></tr><tr><td>C5</td><td>To spot the warning signs of a crisis</td></tr><tr><td>C6</td><td>To communicate with your relative in the presence of delusions or hallucinations</td></tr><tr><td>C7</td><td>To identify your signs of exhaustion</td></tr><tr><td>C8</td><td>To locate your resources</td></tr><tr><td>C9</td><td>To identify your needs</td></tr><tr><td>C10</td><td>To sustainably adopt new behaviors</td></tr></tbody></table>
### 2.3.3 Competences/skills

The 10 main competences provided during the program (listed in Table 1) were divided into two subgroups: namely, to better *support one's relative* (competences 1 to 6) and to adopt self-care *behaviors* (competences 7 to 10). Self-estimation of the 10 competences was assessed, each competence being rated from 0 ('none') to 10 ('complete').

### 2.3.4 Satisfaction and perceived usefulness

Participants were asked to rate their level of satisfaction on a 4-point Likert scale as ‘dissatisfied’, ‘neither satisfied nor satisfied’, ‘satisfied’, or ‘very satisfied’. Similarly, perceived usefulness of the program was rated on a 5-point Likert scale as ‘not useful’, ‘slightly useful’, ‘moderately useful’, ‘very useful’, or ‘extremely useful’.

## 2.4 Ethical consideration

The protocol was approved by Le Vinatier hospital review board prior to data extraction (CEREVI/2024/015). All caregivers gave consent

for the use of their health data in the present study, which was approved by the French Data Protection Commission (No. 17372720).

## 2.5 Statistical analyses

Data collected from the caregivers who attended the 10 Leo programs organized between March 2021 to April 2023 were collapsed for the present analysis. Item-level missing data for the CES-D, ZBI, and competences/skills, when occurred, ranged from 1.3% (n=1) to 10.4% (n=8), and were replaced by the median derived from same-scale values of the corresponding participant (Supplementary Tables 1, 2). Pre-post changes in total scores of the CES-D and the ZBI, and each competences/skills were evaluated using two-tailed paired *t*-tests with Bonferroni correction, effect size being calculated using Cohen's *d*. Because deviation from normality, when occurred (for competences/skills), was minimal, *t*-tests were preferred over less-powered non-parametric alternative. The McNemar test with the mid-p-value approach (40) was used to determine the significance of pre-post differences in proportions of

<figure><img src="image_3.png" /><figcaption>FIGURE 1
Flowchart of participants.</figcaption></figure>
caregivers who had a CES-D score ≥ 16 (41), and a ZBI score ≥ 41 (moderate-to-severe burden). Exploratory network analysis was used in order to get insight into the relationships between i) pre-post changes in competences and ii) pre-post changes in ZBI (network A) or CES-D (network B) scores. Network analysis includes graphical representations of the relationships between variables represented by 'nodes' connected by 'edges' (42). Networks were estimated using the *estimateNetwork* function in the R package *bootnet* (43) with the "EBICglasso" method computing a Gaussian graphical model with the graphical LASSO (44) and extended Bayesian information criterion EBIC; (45) for model selection. In Gaussian graphical models, the parameters (i.e., edges) represent the association among two variables after conditioning on all other variables in the network (i.e., partial correlations). The LASSO ('least absolute shrinkage and selection operator' (46)) is a regularization technique allowing parameters to be zero, resulting in a sparse network. The strength of the penalty of the LASSO is controlled by a parameter λ, selected using the EBIC which itself has a tuning parameter γ (we used the default value 0.5).

# 3 Results

## 3.1 Adherence, characteristics of the sample and satisfaction/usefulness

Flowchart of participants is shown in Figure 1. Over the 91 participants, 87 (95.6%) completed the program, attending at least

<table><caption>TABLE 2 Sample characteristics (n = 76).</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>Age, mean (SD)</td><td>58.5 (10.6)</td></tr><tr><td>Gender, n (%)</td><td></td></tr><tr><td>Women</td><td>58 (76.3)</td></tr><tr><td>Men</td><td>18 (23.7)</td></tr><tr><td>Relationship to the relative, n (%)</td><td></td></tr><tr><td>Parent/Step-parent</td><td>55 (72.4)</td></tr><tr><td>Partner</td><td>9 (11.8)</td></tr><tr><td>Sibling</td><td>8 (10.5)</td></tr><tr><td>Child</td><td>2 (2.6)</td></tr><tr><td>Other</td><td>2 (2.6)</td></tr><tr><td>Relatives' diagnosis, n (%)</td><td></td></tr><tr><td>Schizophrenic or schizoaffective disorder</td><td>30 (39.5)</td></tr><tr><td>Bipolar disorder</td><td>18 (23.7)</td></tr><tr><td>First episode of psychosis</td><td>8 (10.5)</td></tr><tr><td>Borderline personality disorder</td><td>7 (9.2)</td></tr><tr><td>Major depressive disorder</td><td>2 (2.6)</td></tr><tr><td>Anxiety disorder</td><td>2 (2.6)</td></tr><tr><td>No diagnosis/No knowledge of diagnosis</td><td>9 (11.8)</td></tr></tbody></table>

5/8 sessions, 98.9% attended 6/8 sessions, 95.4% attended 7/8 sessions, and 80.5% attended all sessions. Although some caregivers spontaneously reported impediment or unavailability, reasons for absence were not systematically collected. Notably, when a participant missed a session, he was systematically offered the opportunity to catch up on the content through a supplementary short individual session. When asked about reasons for quitting the program, participants (n=4) reported either i) feeling mentally exhausted (so much that they felt unable to continue the program) or ii) unready for a group experience (it was emotionally difficult to be confronted with others). Among the remaining 87 caregivers, 76 fulfilled the pre- and post-intervention questionnaires and were included in the analysis. More precisely, 2 (2.2%) participants did not return the pre-intervention questionnaires, and 9 further participants (9.9%) did not return the post-intervention questionnaires (or were not exploitable).

The characteristics of the 76 subjects included in the analysis are presented in Table 2. The mean age of caregivers was 58.5 years (SD=10.6 years). Most caregivers were women (76.3%) and were caring for their child (72.4%). Relatives' psychiatric diagnosis included schizophrenic or schizoaffective disorder (39.5%), bipolar disorder (23.7%), a first episode of psychosis (10.5%), borderline personality disorder (9.2%), major depression disorder (2.6%), and anxiety disorder (2.6%). The diagnosis was unknown or unspecified for 11.8% of caregivers.

Finally, most caregivers — for whom the satisfaction and usefulness questionnaires were available (n= 39; 51.3%) — were "very satisfied" (89.7%) and found the program "extremely useful" (82.1%). None of the participants were either "dissatisfied" or rated the program as "not useful" (Figure 2).

## 3.2 Pre-post changes in depression, burden, and competences

Results are reported in Table 3 and Supplementary Table 3. Although CES-D scores slightly decreased from baseline to post-intervention, there was no evidence for a statistical significance (*p*=0.103, *d*=0.19). Forty caregivers (52.6%) had a CES-D score ≥16 at baseline, 35 (46.1%) after the intervention (McNemar test, *p*=0.210). Likewise, there was no evidence (McNemar test at *p*=0.057) for differential proportions of subjects with a ZBI score ≥41 (moderate-to-severe burden) between baseline (n=45; 59.2%) and post-intervention (n=38; 50.0%). However, ZBI scores showed significant improvement from baseline to post-intervention, with moderate effect size (*p*<0.001, *d*=0.52). Competences/skills moderately or strongly improved post-intervention as compared to baseline, Cohen's *ds* ranging from 0.47 to 0.87 (all *p*-values <0.001).

## 3.3 Network output

Estimated networks are presented in Figure 3. The corresponding partial correlations matrices are reported in Supplementary Table 4. Pre-post changes in competences/skills were inter-related. For instance, better skills at "identifying one's
<figure><img src="image_4.png" /><figcaption>FIGURE 2
(A) Level of satisfaction and (B) perceived usefulness of the program, assessed immediately after the program (last session).</figcaption></figure>

own signs of exhaustion" (C7), "one's resources" (C8) and "one's needs" (C9) were closely related. Improvement in "spotting the warning signs of a crisis" (C5) was primarily associated with better skill at "making a request to one's relative" (C4), the latter being also associated with improvement in "communicating with them" (C1). Better skill at "communicating with one's relative in the presence of hallucinations or delusions" (C6) was linked to improvement in

"setting limits" (C2) and "making a request to the care team" (C3). Better competence at "adopting new behaviors" (C10) was primarily associated with improvement at "identifying one's needs" (C9) and "communicating with one's relative" (C1). Global improvement in self-reported burden and depression (i.e., decrease in total score) were both primarily associated with better skill at "communicating with one's relative" (C1). Improvement in

<table><caption>TABLE 3 Pre-post change in participants' self-evaluated depression, burden, and competences (n=76).</caption><thead><tr><th></th><th>Baseline M (SD)</th><th>Post-intervention M (SD)</th><th>t-test<sub>(75)</sub> (p-value)</th><th>Cohen's d</th></tr></thead><tbody><tr><td><strong>Depression</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>CES-D</td><td>18.1 (10.2)</td><td>16.8 (10.0)</td><td>1.651 (0.103)</td><td>0.19</td></tr><tr><td><strong>Burden</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>ZBI</td><td>41.6 (14.2)</td><td>37.2 (15.2)</td><td>4.564 (&lt;0.001)</td><td>0.52</td></tr><tr><td><strong>Competences (ranged by effect size)</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>C10</td><td>5.0 (2.1)</td><td>6.9 (1.3)</td><td>-7.569 (&lt;0.001)</td><td>0.87</td></tr><tr><td>C6</td><td>4.0 (2.4)</td><td>6.1 (2.0)</td><td>-7.331 (&lt;0.001)</td><td>0.84</td></tr><tr><td>C9</td><td>5.5 (1.9)</td><td>7.1 (1.2)</td><td>-6.690 (&lt;0.001)</td><td>0.77</td></tr><tr><td>C5</td><td>5.7 (2.3)</td><td>7.1 (1.5)</td><td>-6.479 (&lt;0.001)</td><td>0.74</td></tr><tr><td>C4</td><td>5.5 (2.1)</td><td>6.8 (1.6)</td><td>-6.121 (&lt;0.001)</td><td>0.70</td></tr><tr><td>C1</td><td>5.7 (1.8)</td><td>6.8 (1.3)</td><td>-6.090 (&lt;0.001)</td><td>0.70</td></tr><tr><td>C8</td><td>5.5 (2.1)</td><td>7.1 (1.4)</td><td>-5.496 (&lt;0.001)</td><td>0.63</td></tr><tr><td>C2</td><td>5.1 (2.2)</td><td>6.4 (1.6)</td><td>-5.142 (&lt;0.001)</td><td>0.59</td></tr><tr><td>C7</td><td>5.8 (2.0)</td><td>7.2 (1.5)</td><td>-4.576 (&lt;0.001)</td><td>0.53</td></tr><tr><td>C3</td><td>6.0 (2.7)</td><td>7.2 (1.9)</td><td>-4.094 (&lt;0.001)</td><td>0.47</td></tr></tbody></table>

Except for CES-D, t-tests resulted in a p-value lower than the Bonferroni-corrected alpha (0.05/12 = 0.0042).

C1: To communicate with one's relative. C2: To set limits. C3: To make a request to the care team. C4: To make a request to one's relative. C5: To spot the warning signs of a crisis. C6: To communicate with one's relative in the presence of delusions or hallucinations. C7: To identify one's signs of exhaustion. C8: To locate one's resources. C9: To identify one's needs. C10: To sustainably adopt new behaviors.
A

<figure><img src="image_5.png" /></figure>

B

<figure><img src="image_6.png" /></figure>

**Skills subgroup 1: to better support one's relative**

* C1: To communicate with one's relative

* C2: To set limits

* C3: To make a request to the care team

* C4: To make a request to one's relative

* C5: To spot the warning signs of a crisis

* C6: To communicate with one's relative in the presence of delusions or hallucinations

**Skills subgroup 2: to adopt self-care behaviors**

* C7: To identify one's signs of exhaustion

* C8: To locate one's resources

* C9: To identify one's needs

* C10: To sustainably adopt new behaviors

<caption>FIGURE 3</caption>

(A) Network displaying the relationships between pre-post change in competences and pre-post change in burden. (B) Network displaying the relationships between pre-post change in competences and pre-post change in depression. Nodes represent pre-post change in self-evaluated depression, burden, and competences. Blue and red edges represent positive and negative partial correlations between nodes, respectively. The thickness of the line indicates the strength of the association (i.e., the edge weight).

burden scores was further associated with better competence at "communicating with one's relative in the presence of delusions or hallucinations" (C6), and at "identifying one's signs of exhaustion" (C7). Improvement in depression scores was further associated with better competence at "setting limits" (C2), and "spotting the warning signs of a crisis" (C5).

# 4 Discussion

In the present pilot study, we retrospectively analyzed data from caregivers of individuals living with a SMD who enrolled in a short, multi-family, skill-based FPE program. Participants showed a high level of satisfaction and adherence to the program. Caregivers' self-evaluated skills were all significantly improved post-intervention. Following the FPE program, we report preliminary evidence of a significant overall improvement in caregivers' burden whereas we found no evidence for a significant change on self-reported depression.

Participants' satisfaction with the Leo program was high. Indeed, 89.7% of the participants who completed the post-intervention satisfaction survey were "very satisfied", and 82.1% found the program "extremely useful". Of note, none of the participants were either "dissatisfied" or rated the program as "not useful". Furthermore, we report a high adherence rate to the program, since 95.6% of the participants achieved the Leo program, attending at least 5 sessions, most of them (80.5%) attending all sessions. Altogether, findings from this pilot study support the feasibility of delivering a short, multi-family, skill-based FPE program to caregivers of subjects living with various SMD. This is possibly due to our prioritization of feasibility in the design of the intervention. Indeed, the program is accessible to all caregivers caring for a relative living with a SMD, without any constraint of caring duration or diagnosis (even if the diagnosis is not yet known). Furthermore, the high level of commitment probably

reflects our responsiveness to caregivers requesting a brief intervention that does not require the participation of the relative living with a SMD. Altogether, such characteristics allow all interested caregivers to benefit from the Leo program without any delay or constraint, thus preserving a high level of motivation.

In the present study, caregivers' self-evaluated competences were all significantly improved post-intervention as compared to baseline with medium to strong effect sizes. Importantly, caregivers reported increased sense of competence in both subgroups of skills (i.e., to better support one's relative and to adopt self-care behaviors). This is consistent with the previous report that a short, skill-building FPE program increased self-efficacy and self-care behaviors of caregivers of individuals with dementia (47). Our findings are thus preliminary evidence supporting the efficiency of a short, skill-based FPE program to improve competence of caregivers in psychiatry. This is consistent with a previous RCT reporting the beneficial effect of another short, skill-based FPE program on caregiver's self-efficacy (24). In contrast, a previous RCT by Kulhara et al. evaluating a long FPE program reported no significant improvement in caregiver-coping immediately after the program (48). Different factors may explain the discrepant results. First, the two studies explored different outcomes since caregivers' competence and caregivers' coping strategies are different constructs (49). Second, the two FPE programs differed regarding key characteristics: especially the number of sessions devoted to skills training (2 sessions in Kulhara et al.'s program vs. 7 sessions in the Leo program) or the education techniques used during the FPE. Indeed, skill training is a more difficult educational objective as compared to knowledge improvement. Therefore, simulation-based training techniques such as peer role-play and simulated patients were incorporated in the Leo program to improve skills learning. In simulation-based education, the educational objective of role-play training is to rehearse situations to improve learner's abilities to face with similar situations in daily life (50). Since simulation-based
training is considered an efficient teaching technique to promote skills learning (51), it is possible that devoting significant time to simulation during the Leo program helped to improve caregivers' sense of competence.

As expected, the caregivers included in the present study reported high levels of depressive symptoms and burden at baseline. Indeed, 52.6% of them were possibly depressed (CES-D score ≥ 16) while 59.2% exhibited a moderate-to-severe burden (ZBI score ≥ 41). Notably, the percentage of caregivers with a CES-D score ≥ 16 was higher than the ones previously reported in caregivers of subjects with bipolar disorder or schizophrenia (respectively, 22-to-33% and 42%) (52–55). In contrast, the rate of caregivers with a ZBI ≥ 41 was consistent with those (ranging from 30 to 68%) previously observed in caregivers of patients with SMD (56–59). Altogether these results confirm that caregivers in psychiatry are a vulnerable population characterized by a high risk of depression as compared to the general population (2, 3) and therefore require dedicated interventions such as FPE.

In the present study, we found no evidence for a significant difference in self-reported depression score immediately after the Leo program. Remarkably, a recent meta-analysis reported no significant effect of FPE combined with skills training on patient's relapse at 6 months whereas, a significant reduction was observed at 12 months and further (17). This suggests that a delay might be necessary to observe the beneficial effects of this type of skill-based FPE. Indeed, skills are often more complex to implement in daily-life as compared to knowledge. Caregivers may implement them gradually after the FPE which could explain why beneficial effects are observed only after a delay. In psychiatry, although previous studies of short, knowledge based FPE programs have reported a significant reduction of caregivers' depression immediately after the intervention (60) or at 6 months (61), unfortunately, we are not aware of any previous studies investigating the effect of a short, skill-based FPE program on caregivers' depression. Further studies investigating the impact of skill-based FPE on caregivers' depression are needed and should explore caregivers' outcomes at various time points after FPE.

In contrast, a significant decrease of caregivers' burden was observed after the intervention. Indeed, ZBI scores showed significant improvement from baseline to post-intervention, with a medium effect size. This is in accordance with previous pilot studies reporting a significant decrease in family burden immediately following a FPE program (60, 62). Similarly, two randomized controlled trials of FPE programs reported that skill-based FPE was associated with lower levels of family burden albeit at 6 and 12 months following the intervention (63, 64). Notably, the differential proportions of subjects with a ZBI score ≥ 41 (moderate-to-severe burden) between baseline (59.2%) and post-intervention (50.0%) was marginally significant (p=0.057). Family burden is a much broader and subjective concept as compared to depression. It involves not only physical and psychological impact but also sense of losing control (65). For caregivers, increased caregiving-related skills enhance self-competence in challenging difficulties (66). In this light, it is possible that the decrease of caregivers' burden observed following the Leo program was related to the significantly improved sense of competence reported by caregivers after the intervention.

Improved caregiving skills are associated with better health outcomes for caregivers of subjects living with a SMD (67). Such findings have informed the development of psychoeducational interventions to support caregivers. However, the existing models of FPE greatly differ from one another. Although some criteria regarding the “key ingredients” underlying FPE efficiency have emerged (e.g., the type and number of participants, format, frequency of sessions, duration, location, teaching techniques), they are not precise enough when it comes to specify the content of the FPE sessions (coping, communication, problem solving) (23). There is thus a dire need to identify more precisely which type of content underlies the effectiveness of FPE and should therefore be prioritized in FPE programs (68).

In this regard, increased self-evaluated competence in 3 skills were associated with an improvement in caregivers' burden: namely, *to communicate with one's relative, to communicate with one's relative in the presence of delusions or hallucinations* and *to identify one's signs of exhaustion*. Similarly, increased self-evaluated competence in 3 skills were associated with an improvement in self-reported depression: *to communicate with one's relative, to set limits* and *to spot warning signs of a crisis*. Therefore, feeling more skilled in communicating with one's relative was associated with an improvement in both self-reported burden and depression. Noteworthy, among the 10 skills provided during the Leo program, *to communicate with one's relative* showed the strongest association with the improvement of both caregivers' outcomes. More research is needed to confirm if these skills should be considered as “key ingredients” and, as such, should be systematically incorporated in FPE skill-based programs and prioritized among other contents (e.g., more time should be devoted to them). Identifying such “key ingredients” is of utmost importance to achieve well standardized psychoeducational programs with clear definitions of the content of interventions (69).

Noteworthy, competences from to the two subgroups (subgroup 1, *to better support one's relative* and subgroup 2, *to adopt self-care behaviors*) were linked to an improvement in caregivers' burden, while only competences from subgroup 1 were associated with an improvement in self-reported depression. Furthermore, numerous interrelationships between competences belonging to each of the two subgroups were observed in each network, supporting a potential synergistic effect. Especially, *to sustainably adopt new behaviors* (C10, subgroup 2) exhibited the greatest number of interrelationships with competences from another subgroup in both networks. Altogether, these preliminary findings suggest that both types of competences should be considered as “key ingredients” for FPE programs. Moreover, providing both type of competences within a single FPE program could be associated with an increased effectiveness on caregivers’ burden. Eventually, it should be underlined that out of the 10 competences incorporated in the Leo program after a careful selection process involving members of family associations, 5 were linked to an improvement in caregivers’ outcomes. This strongly supports the strategic relevance of co-designing the content of FPE programs in a participatory research approach involving the recipients of such programs (70, 71).
In regard to its pilot nature the present study has several limitations. Firstly, the sample size was small and there was no control group. Moreover, the single-center design of the present study may affect the representativeness of the included sample. Secondly, the post-intervention evaluation was realized immediately after the FPE program while existing literature has shown that improvement in the main outcomes is observed at 6 to 12 months (17). Thus, the long-term consequences of this short FPE program remain unclear. Thirdly, the satisfaction questionnaires were introduced after the program started and were available for only a subset of the participants. Our results can thus be subjected to sampling bias. Eventually, the quantitative measures of skills used in the pre- and post-intervention questionnaires were specifically developed for the evaluation of the FPE program; they were thus not validated. Indeed, validated questionnaires assessing skills of caregivers are lacking. In this regard, when evaluating FPE programs incorporating skills training, other authors have also relied on non-evaluated instruments (47) or questionnaires evaluating coping strategies (48). None of these latter questionnaires were precise enough to specifically explore the skills targeted by the Leo program, therefore we developed self-administered questionnaires with consideration of previous research on skills of caregivers in psychiatry (72). Furthermore, to improve content validity, the questions were designed to explore the psychoeducational objectives of the Leo program. Such questionnaires are routinely used in FPE program research (47), although their accuracy has been questioned as compared to objective measures (73). Notably, a good agreement between self-administered evaluation and objective measures of performance was reported, supporting self-administered questionnaires as valid tools to assess performance on specific learning objectives (74).

In light of these limits, a larger, multi-center, randomized controlled study should be undertaken to confirm the present preliminary results and to explore caregivers' and relatives' outcomes in the medium and long-term. Eventually, health economic outcomes should be explored, since the efficiency of psychoeducation is central in making it an attractive intervention for managers and policy makers. As a well-standardized FPE program with clear definitions of the content of the interventions, Leo is a promising intervention for researchers planning to develop evidence-based programs for caregivers.

## 5 Conclusion

In addition to showing high levels of participant satisfaction and adherence to the FPE program Leo, this pilot study provides preliminary evidence that caregivers significantly improved burden as well as self-evaluated competence in supporting their relative and in self-care behaviors immediately after the intervention. In light of its brief format and large accessibility (no constraint as to the caring duration or diagnosis of the relative), the Leo program is expected to be easily disseminated within the mental health system. For the field of FPE, the present results have several implications. They support the feasibility and efficiency in routine

care of a short, multi-family, multi-diagnostic, skill based FPE program for caregivers of subjects living with a SMD. The present research also provides insights into the critical contents through which FPE might alleviate burden of care. Eventually, these results strongly support the relevance of engaging caregivers in the coproduction of FPE programs. More well-standardized FPE programs with clear definitions of the content of the interventions should be developed. Additional randomized controlled studies are needed to confirm the present results and to explore caregivers', relatives' and health economic outcomes in the medium and long-term.

## Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

## Ethics statement

The studies involving humans were approved by the Comité d'Éthique Recherche du Vinatier (CERECVI)/Centre Hospitalier Le Vinatier. The studies were conducted in accordance with the local legislation and institutional requirements. The ethics committee/ institutional review board waived the requirement of written informed consent for participation from the participants or the participants' legal guardians/next of kin because as a retrospective study processing data drawn from personal medical files, the present research was granted MR-004 validation, which corresponds to the Commission Nationale Informatique et Libertés (CNIL) validation. In accordance with the French law for MR-004 standard, all subjects were informed regarding the research project and had the possibility of not consenting to the reuse of their health data for research.

## Author contributions

L-FL: Data curation, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing. BM: Investigation, Project administration, Supervision, Writing – original draft, Writing – review & editing. BZ: Investigation, Writing – original draft, Writing – review & editing. BC: Investigation, Writing – original draft, Writing – review & editing. CB: Data curation, Writing – original draft, Writing – review & editing. RR: Conceptualization, Investigation, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing.

## Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
## Acknowledgments

The authors would like to thank all those who participated in the Leo program and also those involved in developing it— especially the Team of the Center for Caregivers in Psychiatry of Lyon, the family associations Collectif Schizophrénies and Unafam (Union Nationale des Amis et Familles de personnes Malades et/ou handicapées psychiques), and the association Take a BREF.

## Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

## References

1. McFarlane WR. Family interventions for schizophrenia and the psychoses: A review. *Family Process*. (2016) 55:460–82. doi: 10.1111/famp.12235

2. Gupta S, Isherwood G, Jones K, Van Impe K. Assessing health status in informal schizophrenia caregivers compared with health status in non-caregivers and caregivers of other conditions. *BMC Psychiatry*. (2015) 15:162. doi: 10.1186/s12888-015-0547-1

3. Mittendorfer-Rutz E, Rahman S, Tanskanen A, Majak M, Mehtälä J, Hoti F, et al. Burden for parents of patients with schizophrenia—A nationwide comparative study of parents of offspring with rheumatoid arthritis, multiple sclerosis, epilepsy, and healthy controls. *Schizophr Bulletin*. (2019) 45:794–803. doi: 10.1093/schbul/sby130

4. Hayes L, Hawthorne G, Farhall J, O'Hanlon B, Harvey C. Quality of life and social isolation among caregivers of adults with schizophrenia: policy and outcomes. *Community Ment Health J*. (2015) 51:591–7. doi: 10.1007/s10597-015-9848-6

5. Steele A, Maruyama N, Galynker I. Psychiatric symptoms in caregivers of patients with bipolar disorder: A review. *J Affect Disord*. (2010) 121:10–21. doi: 10.1016/j.jad.2009.04.020

6. Lucksted A, McFarlane W, Downing D, Dixon L. Recent developments in family psychoeducation as an evidence-based practice. *J Marital Family Ther*. (2012) 38:101–21. doi: 10.1111/j.1752-0606.2011.00256.x

7. *Psychosis and schizophrenia in adults: prevention and management*. London: National Institute for Health and Care Excellence (NICE) (2014). Available at: http://www.ncbi.nlm.nih.gov/books/NBK555203/.

8. Reinares M, Bonnin CM, Hidalgo-Mazzei D, Sánchez-Moreno J, Colom F, Vieta E. The role of family interventions in bipolar disorder: A systematic review. *Clin Psychol Rev*. (2016) 43:47–57. doi: 10.1016/j.cpr.2015.11.010

9. Katsuki F, Watanabe N, Yamada A, Hasegawa T. Effectiveness of family psychoeducation for major depressive disorder: systematic review and meta-analysis. *BJPsych Open*. (2022) 8:e148. doi: 10.1192/bjo.2022.543

10. Bighelli I, Rodolico A, Garcia-Mieres H, Pitschel-Walz G, Hansen WP, Schneider-Thoma J, et al. Psychosocial and psychological interventions for relapse prevention in schizophrenia: a systematic review and network meta-analysis. *Lancet Psychiatry*. (2021) 8:969–80. doi: 10.1016/S2215-0366(21)00243-1

11. Pharoah F, Mari JJ, Rathbone J, Wong W. Family intervention for schizophrenia. *Cochrane Database Systematic Rev*. (2010) 12:CD000088. doi: 10.1002/14651858.CD000088.pub3/abstract

12. Cuijpers P. The effects of family interventions on relatives' burden: A meta-analysis. *J Ment Health*. (1999) 8:275–85. doi: 10.1080/09638239917436

13. Lobban F, Postlethwaite A, Glentworth D, Pinfold V, Wainwright L, Dunn G, et al. A systematic review of randomised controlled trials of interventions reporting outcomes for relatives of people with psychosis. *Clin Psychol Rev*. (2013) 33:372–82. doi: 10.1016/j.cpr.2012.12.004

14. Sin J, Norman I. Psychoeducational interventions for family members of people with schizophrenia: A mixed-method systematic review. *J Clin Psychiatry*. (2013) 74:16055. doi: 10.4088/JCP.12r08308

15. Sin J, Gillard S, Spain D, Cornelius V, Chen T, Henderson C. Effectiveness of psychoeducational interventions for family carers of people with psychosis: A systematic review and meta-analysis. *Clin Psychol Review*. (2017) 56:13–24. doi: 10.1016/j.cpr.2017.05.002

16. Galletly C, Castle D, Dark F, Humberstone V, Jablensky A, Killackey E, et al. Royal Australian and New Zealand College of Psychiatrists clinical practice guidelines

## Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

## Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2024.1374540/full#supplementary-material

for the management of schizophrenia and related disorders. *Aust N Z J Psychiatry*. (2016) 50:410–72. doi: 10.1177/0004867416641195

17. Rodolico A, Bighelli I, Avanzato C, Concerto C, Cutrufelli P, Mineo L, et al. Family interventions for relapse prevention in schizophrenia: a systematic review and network meta-analysis. *Lancet Psychiatry*. (2022) 9:211–21. doi: 10.1016/S2215-0366(21)00437-5

18. Miklowitz DJ, Efthimiou O, Furukawa TA, Scott J, McLaren R, Geddes JR, et al. Adjunctive psychotherapy for bipolar disorder: A systematic review and component network meta-analysis. *JAMA Psychiatry*. (2021) 78:141–50. doi: 10.1001/jamapsychiatry.2020.2993

19. Yatham LN, Kennedy SH, Parikh SV, Schaffer A, Bond DJ, Frey BN, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) and International Society for Bipolar Disorders (ISBD) 2018 guidelines for the management of patients with bipolar disorder. *Bipolar Disord*. (2018) 20:97–170. doi: 10.1111/bdi.12609

20. Kendall T, Morriss R, Mayo-Wilson E, Marcus E. Guideline Development Group of the National Institute for Health and Care Excellence. Assessment and management of bipolar disorder: summary of updated NICE guidance. *BMJ*. (2014) 349:g5673. doi: 10.1136/bmj.g5673

21. Petitjean F, MC B, Hodé Y, Tramier V. *Psychoéducation dans la schizophrènie*. Paris, France: EMC Psychiatrie. Elsevier Masson (2014) p. 1–11.

22. Gautier A, Dru A. Conseil Economique Social et Environnemental. Improve psychiatric care organization [Améliorer le parcours de soin en psychiatrie]. *Les éditions Des Journaux Officiels*. (2021).

23. Harvey C. Family psychoeducation for people living with schizophrenia and their families. *BJPsych advances*. (2018) 24:9–19. doi: 10.1192/bja.2017.4

24. Kline ER, Thibeau H, Sanders AS, English K, Davis BJ, Fenley AR, et al. Motivational interviewing for loved ones in early psychosis: development and pilot feasibility trial of a brief psychoeducational intervention for caregivers. *Front Psychiatry*. (2021) 12:659568. doi: 10.3389/fpsyt.2021.659568

25. Rubak S, Sandbaek A, Lauritzen T, Christensen B. Motivational interviewing: a systematic review and meta-analysis. *Br J Gen Pract*. (2005) 55:305–12.

26. Miller WR, Rollnick S. *Motivational Interviewing Preparing People for Change. 3rd ed.* New York: New York Guildford Press. - References - Scientific Research Publishing (2013). Available at: https://www.scirp.org/(S(i43dyn45teexjx455qlt3d2q))/reference/ReferencesPapers.aspx?ReferenceID=1445954.

27. Sorsdahl K, Myers B, Ward CL, Matzopoulos R, Mtukushe B, Nicol A, et al. Adapting a blended motivational interviewing and problem-solving intervention to address risky substance use amongst South Africans. *Psychother Res*. (2015) 25:435–44. doi: 10.1080/10503307.2014.897770

28. Schoenthaler AM, Lancaster KJ, Chaplin W, Butler M, Forsyth J, Ogedegbe G. Cluster randomized clinical trial of FAITH (Faith-based approaches in the treatment of hypertension) in blacks. *Circ Cardiovasc Qual Outcomes*. (2018) 11:e004691. doi: 10.1161/CIRCOUTCOMES.118.004691

29. Smeerdijk M, Keet R, Dekker N, van RB, Krikke M, Koeter M, et al. Motivational interviewing and interaction skills training for parents to change cannabis use in young adults with recent-onset schizophrenia: a randomized controlled trial. *psychol Med*. (2012) 42:1627–36. doi: 10.1017/S0033291711002832

30. Weber J, Storch M. Motivation and Goal Setting with Motto-Goals in Coaching. In: Greif S, Möller H, Scholl W, Passmore J, Müller F, editors. *International Handbook*
of Evidence-Based Coaching: Theory, Research and Practice. Springer International Publishing, Cham (2022). p. 637–47 doi: 10.1007/978-3-030-81938-5_52

31. Storch M, Krause F. Selbstmanagement – Ressourcenorientiert: Theoretische Grundlagen und Trainingsmanual für die Arbeit mit dem Zürcher Ressourcen Modell [Self-Management - Resource-Oriented: Theoretical Basics and Training Manual for Working with the Zurich Resource Model]. Bern: Hogrefe (2017).

32. Storch M, Gaab J, Küttel Y, Stüssi AC, Fend H. Psychoneuroendocrine effects of resource-activating stress management training. *Health Psychol.* (2007) 26:456–63. doi: 10.1037/0278-6133.26.4.456

33. Storch M, Keller F, Weber J, Spindler A, Milos G. Psychoeducation in affect regulation for patients with eating disorders: a randomized controlled feasibility study. *Am J Psychother.* (2011) 65:81–93. doi: 10.1176/appi.psychotherapy.2011.65.1.81

34. Storch M, Olbrich D. Das GUSI-programm als beispiel für gesundheitspädagogik in präventionsleistungen der deutschen rentenversicherung [The GUSI program as an example of health education in prevention services of the German Pension Insurance]. In: Knörzer W, Rupp R, editors. Gesundheit ist nicht alles – was ist sie dann?: Gesundheitspädagogische Antworten, 1st Edn. Schneider Hohengehren, Baltmannsweiler (2011). p. 111–26.

35. Steurer-Stey C, Storch M, Benz S, Hobi B, Steffen-Bürgi B, Steurer J, et al. Motivational training improves self-efficacy but not short-term adherence with asthma self-management: a randomized controlled trial. *Prim Health Care Res Dev.* (2015) 16:32–41. doi: 10.1017/S1463423613000480

36. Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. *Appl psychol Measurement.* (1977) 1:385–401. doi: 10.1177/014662167700100306

37. Morin AJS, Moullec G, Maïano C, Layet L, Just JL, Ninot G. Psychometric properties of the Center for Epidemiologic Studies Depression Scale (CES-D) in French clinical and nonclinical adults. *Rev d'Epidemiologie Santé Publique.* (2011) 59:327–40. doi: 10.1016/j.respe.2011.03.061

38. Hébert R, Bravo G, Girouard D. Fidélité de la traduction française de trois instruments d'évaluation des aidants naturels de malades déments. *Can J Aging/La Rev Can du vieillissement.* (1993) 12:324–37. doi: 10.1017/S0714980800013726

39. Zarit S, Orr NK, Zarit JM. The Hidden Victims of Alzheimer's Disease: Families Under Stress. New York: NYU Press (1985). p. 231.

40. Fagerland MW, Lydersen S, Laake P. The McNemar test for binary matched-pairs data: mid-p and asymptotic are better than exact conditional. *BMC Med Res Methodol.* (2013) 13:91. doi: 10.1186/1471-2288-13-91

41. Weissman MM, Sholomskas D, Pottenger M, Prusoff BA, Locke BZ. Assessing depressive symptoms in five psychiatric populations: a validation study. *Am J Epidemiol.* (1977) 106:203–14. doi: 10.1093/oxfordjournals.aje.a112455

42. Borsboom D, Deserno MK, Rhemtulla M, Epskamp S, Fried EI, McNally RJ, et al. Network analysis of multivariate data in psychological science. *Nat Rev Methods Primers.* (2021) 1:58. doi: 10.1038/s43586-021-00055-w

43. Epskamp S. Bootstrap Methods for Various Network Estimation Routines [R package bootnet version 1.4.3]. Comprehensive R Archive Network (CRAN (2020). Available at: https://CRAN.R-project.org/package=bootnet.

44. Friedman J, Hastie T, Tibshirani R. Sparse inverse covariance estimation with the graphical lasso. *Biostatistics.* (2008) 9:432–41. doi: 10.1093/biostatistics/kxm045

45. Chen J, Chen Z. Extended Bayesian information criteria for model selection with large model spaces. *Biometrika.* (2008) 95:759–71. doi: 10.1093/biomet/asn034

46. Tibshirani R. Regression shrinkage and selection via the lasso. *J R Stat Society: Ser B (Methodological).* (1996) 58:267–88. doi: 10.1111/j.2517-6161.1996.tb02080.x

47. Rosney DM, Noe M F, J Horvath P. Powerful tools for caregivers, a group psychoeducational skill-building intervention for family caregivers. *J Caring Sci.* (2017) 6:187–98. doi: 10.15171/jcs.2017.019

48. Kulhara P, Chakrabarti S, Avasthi A, Sharma A, Sharma S. Psychoeducational intervention for caregivers of Indian patients with schizophrenia: a randomised-controlled trial. *Acta Psychiatr Scand.* (2009) 119:472–83. doi: 10.1111/j.1600-0447.2008.01304.x

49. Liem GAD. Perceived competence and coping strategies. *Educ Psychol.* (2022) 42:275–7. doi: 10.1080/01443410.2022.2065093

50. Nestel D, Kelly M, Jolly B, Watson M. Healthcare Simulation Education: Evidence, Theory and Practice. Wiley-Blackwell. Hoboken, New Jersey: John Wiley and Sons Ltd (2017).

51. Gelis A, Cervello S, Rey R, Llorca G, Lambert P, Franck N, et al. Peer role-play for training communication skills in medical students: A systematic review. *Simul Health.* (2020) 15:106–11. doi: 10.1097/SIH.0000000000000412

52. Perlick DA, Hohenstein JM, Clarkin JF, Kaczynski R, Rosenheck RA. Use of mental health and primary care services by caregivers of patients with bipolar disorder: a preliminary study. *Bipolar Disord.* (2005) 7:126–35. doi: 10.1111/j.1399-5618.2004.00172.x

53. Perlick DA, Rosenheck RA, Miklowitz DJ, Chessick C, Wolff N, Kaczynski R, et al. Prevalence and correlates of burden among caregivers of patients with bipolar

disorder enrolled in the Systematic Treatment Enhancement Program for Bipolar Disorder. *Bipolar Disord.* (2007) 9:262–73. doi: 10.1111/j.1399-5618.2007.00365.x

54. Bassirnia A, Briggs J, Kopeykina I, Mednick A, Yaseen Z, Galynker I. Relationship between personality traits and perceived internalized stigma in bipolar patients and their treatment partners. *Psychiatry Res.* (2015) 230:436–40. doi: 10.1016/j.psychres.2015.09.033

55. Gonçalves-Pereira M, González-Fraile E, Santos-Zorrozá B, Martin-Carrasco M, Fernández-Catalina P, Domínguez-Panchón AI, et al. Assessment of the consequences of caregiving in psychosis: a psychometric comparison of the Zarit Burden Interview (ZBI) and the Involvement Evaluation Questionnaire (IEQ). *Health Qual Life Outcomes.* (2017) 15:63. doi: 10.1186/s12955-017-0626-8

56. Rahmani F, Roshangar F, Fholizadeh L, Asghari E. Caregiver burden and the associated factors in the family caregivers of patients with schizophrenia. *Nurs Open.* (2022) 9:1995–2002. doi: 10.1002/nop2.1205

57. Shamsaei F, Cheraghi F, Bashirian S. Burden on family caregivers caring for patients with schizophrenia. *Iran J Psychiatry.* (2015) 10:239–45.

58. Siddiqui S, Khalid J. Determining the caregivers' burden in caregivers of patients with mental illness. *Pak J Med Sci.* (2019) 35:1329–33. doi: 10.12669/pjms.35.5.720

59. Udoh EE, Omorere DE, Sunday O, Osasu OS, Amoo BA. Psychological distress and burden of care among family caregivers of patients with mental illness in a neuropsychiatric outpatient clinic in Nigeria. *PloS One.* (2021) 16:e0250309. doi: 10.1371/journal.pone.0250309

60. Delibaş DH, Erdoğan E. Effects of a psychoeducation program on disease burden, depression, and anxiety levels in relatives of psychotic patients in a community mental health center. *Perspect Psychiatr Care.* (2022) 58:940–5. doi: 10.1111/ppc.12880

61. Tessier A, Roger K, Gregoire A, Desnavailles P, Misdrahi D. Family psychoeducation to improve outcome in caregivers and patients with schizophrenia: a randomized clinical trial. *Front Psychiatry.* (2023) 14:1171661. doi: 10.3389/fpsyt.2023.1171661

62. Reinares M, Vieta E, Colom F, Martínez-Arán A, Torrent C, Comes M, et al. Impact of a psychoeducational family intervention on caregivers of stabilized bipolar patients. *Psychother Psychosom.* (2004) 73:312–9. doi: 10.1159/000078848

63. Xiong W, Phillips MR, Hu X, Wang R, Dai Q, Kleinman J, et al. Family-based intervention for schizophrenic patients in China. A randomised controlled trial. *Br J Psychiatry.* (1994) 165:239–47. doi: 10.1192/bjp.165.2.239

64. Veltro F, Magliano L, Morosini P, Fasulo E, Pedicini G, Cascavilla I, et al. [Randomised controlled trial of a behavioural family intervention: 1 year and 11-years follow-up]. *Epidemiol Pschiatr Soc.* (2006) 15:44–51. doi: 10.1017/S1121189X00002025

65. Gumus F, Dikec G, Ergun G. Relations among internalized stigmatization, depressive symptom frequency and family loading in first-degree caregivers of the patients treated in the psychiatry clinic of a state hospital. *Arch Psychiatr Nurs.* (2017) 31:522–7. doi: 10.1016/j.apnu.2017.05.003

66. Graham C, Ballard C, Sham P. Carers' knowledge of dementia, their coping strategies and morbidity. *Int J Geriat Psychiatry.* (1997) 12:931–6. doi: 10.1002/(ISSN) 1099-1166

67. Zhou Z, Wang Y, Feng P, Li T, Tebes JK, Luan R, et al. Associations of caregiving knowledge and skills with caregiver burden, psychological well-being, and coping styles among primary family caregivers of people living with schizophrenia in China. *Front Psychiatry.* (2021) 12:631420. doi: 10.3389/fpsyt.2021.631420

68. Dillinger RL, Kersun JM. Caring for caregivers: Understanding and meeting their needs in coping with first episode psychosis. *Early Interv Psychiatry.* (2020) 14:528–34. doi: 10.1111/eip.12870

69. Xia J, Merinder LB, Belgamwar MR. Psychoeducation for schizophrenia. *Cochrane Database Syst Rev.* (2011) 6:CD002831. doi: 10.1002/CD028315

70. Van den Steene H, Van West D, Glazemakers I. [Potential of participatory action research for clients, professionals and researchers in mental healthcare]. *Tijdschr Psychiatr.* (2019) 61:343–51.

71. Graffigna G, Gheduzzi E, Morelli N, Barello S, Corbo M, Ginex V, et al. Place4Carers: a multi-method participatory study to co-design, piloting, and transferring a novel psycho-social service for engaging family caregivers in remote rural settings. *BMC Health Serv Res.* (2021) 21:591. doi: 10.1186/s12913-021-06563-5

72. Scazufca M, Kuipers E. Coping strategies in relatives of people with schizophrenia before and after psychiatric admission. *Br J Psychiatry.* (1999) 174:154–8. doi: 10.1192/bjp.174.2.154

73. Stewart J, O'Halloran C, Barton JR, Singleton SJ, Harrigan P, Spencer J. Clarifying the concepts of confidence and competence to produce appropriate self-evaluation measurement scales. *Med Educ.* (2000) 34:903–9. doi: 10.1046/j.1365-2923.2000.00728.x

74. Raupach T, Münscher C, Beissbarth T, Burckhardt G, Pukrop T. Towards outcome-based programme evaluation: using student comparative self-assessments to determine teaching effectiveness. *Med Teach.* (2011) 33:e446–453. doi: 10.3109/0142159X.2011.586751