Esketamine nasal spray for treatment-resistant depression

Evidence-based recommendations on esketamine (Spravato) for treatment-resistant depression in adults.

# Recommendations
This guidance only includes recommendations for treatment-resistant depression.
Esketamine nasal spray for treating major depressive disorder is being evaluated in NICE's technology appraisal guidance on esketamine for treating major depressive disorder in adults at imminent risk for suicide.
Esketamine nasal spray with a selective serotonin reuptake inhibitor (SSRI) or a serotonin-norepinephrine reuptake inhibitor (SNRI) is not recommended, within its marketing authorisation, for treatment-resistant depression that has not responded to at least 2 different antidepressants in the current moderate to severe depressive episode in adults.
This recommendation is not intended to affect treatment with esketamine that was started in the NHS before this guidance was published. People having treatment outside this recommendation may continue without change to the funding arrangements in place for them before this guidance was published, until they and their NHS clinician consider it appropriate to stop.
Why the committee made these recommendations
The company positioned esketamine nasal spray for people who have had at least 3 antidepressants before, with or without another treatment like lithium or an antipsychotic medicine. This is narrower than the marketing authorisation, and also how clinical experts advised esketamine would likely be used in NHS practice.
The clinical evidence at this positioning is uncertain because it only considers a small number of people from the full clinical trial population. But it suggests that for people who have had at least 3 antidepressants with or without another treatment, esketamine with an SSRI or SNRI is likely more effective than placebo with an SSRI or SNRI. Because the trials were short the long-term benefits of esketamine are uncertain.
Also, the trial evidence excluded people with characteristics of depression like psychosis or recent suicidal ideation with intent. This limits how well the evidence applies to the NHS, because people having treatment for depression in the NHS may have psychosis or recent suicidal ideation with intent.
The clinical uncertainty means the economic modelling is also uncertain, including:
how treatment-resistant depression was modelled
how long people would take esketamine for
the costs of using esketamine in the NHS.
The limitations in the clinical evidence and economic model mean it is not possible to determine a reliable cost-effectiveness estimate. Esketamine is unlikely to be an acceptable use of NHS resources, so it is not recommended. Further research is recommended to address some of the uncertainties.# Information about esketamine

# Marketing authorisation indication
Esketamine nasal spray (Spravato, Janssen) with a selective serotonin reuptake inhibitor (SSRI) or serotonin-norepinephrine reuptake inhibitor (SNRI), is indicated 'for adults with treatment-resistant major depressive disorder who have not responded to at least 2 different treatments with antidepressants in the current moderate to severe depressive episode'. The scope of this appraisal is only for this indication.

# Dosage in the marketing authorisation
The dosage schedule for esketamine is available in the summary of product characteristics for esketamine.

# Price
The device is single use and delivers 28 mg of esketamine in 2 sprays, one 14 mg spray per nostril. The list prices are as follows (excluding VAT; BNF online, accessed April 2022):
163 for a 28 mg dose (one 28 mg device)
326 for a 56 mg dose (two 28 mg devices)
489 for an 84 mg dose (three 28 mg devices).The company had a commercial arrangement, which would have applied if the technology had been recommended.# Committee discussion
The appraisal committee considered evidence submitted by Janssen, reviews of these submissions by the evidence review group (ERG), NICE's technical report, and responses from stakeholders from 2 consultation documents. See the committee papers for full details of the evidence.

# The condition and current treatment

## Treatment-resistant depression has a negative effect on people
The patient experts explained that treatment-resistant depression has a substantial burden on all aspects of life, with a range of symptoms. The patient experts emphasised that people with treatment-resistant depression often have feelings of hopelessness, fear and despair. This can affect the person's family and carers. The clinical expert noted that the lives of children of people with treatment-resistant depression are also affected. The committee concluded that the condition has a negative effect on people, their families and carers.

## There is an unmet need for effective treatment options
The patient experts explained that people with treatment-resistant depression often feel hopeless because treatments are ineffective. The clinical expert noted that people will try different courses of treatments to alleviate symptoms. The patient experts highlighted that, when multiple courses of treatment do not work, the feelings of hopelessness get worse. They added that this was an inherent aspect of the 'treatment-resistant' nature of the condition. A patient expert who had recovered from treatment-resistant depression emphasised the importance of independence and return of character upon remission. The clinical expert noted that a large proportion of people keep taking antidepressants that are not working, sometimes for a year or more. The committee concluded that the effectiveness of current treatments for treatment-resistant depression is limited and that there is an unmet need for new treatment options.

# Treatment pathway and comparator

## Current clinical practice includes several different types of treatment
The company submission defined treatment-resistant depression as 'people with major depressive disorder who fail to respond to 2 different oral antidepressants'. It included the recommended treatment pathway for this population from NICE's guideline on depression. Based on the guideline, NICE's esketamine appraisal scope and the company submission, the treatment options for people with treatment-resistant depression include:
-ral treatments such as sertraline, citalopram, fluoxetine, venlafaxine, vortioxetine, mirtazapine, amitriptyline and monoamine oxidase inhibitors
augmentation therapy (when an antidepressant is used with a non-antidepressant), for example, an antidepressant with lithium or an antidepressant with an antipsychotic treatment
combination therapy (an antidepressant with another antidepressant)
electroconvulsive therapy (ECT).NICE's guideline on depression also includes cognitive behavioural therapy (CBT) and other psychological therapies as options combined with the above treatments. However, the company noted that the treatment pathway in clinical practice is different to the guideline. The clinical expert explained that the treatment pathway for treatment-resistant depression can vary between services across the UK. They explained that, in general, most people with treatment-resistant depression have 3 to 4 standard oral antidepressant treatments from their GP. Only a small proportion (company estimate of 9.6%) are referred to a psychiatrist. Then, the first treatment choice is normally to optimise the dose of oral antidepressant or switch to a new oral treatment. Then 1 or 2 trials of augmentation therapy, with an antipsychotic drug or lithium combination therapy, would be considered before ECT. The committee acknowledged that the summary of product characteristics (SPC) states that esketamine nasal spray (from now, esketamine) must be prescribed by a psychiatrist. The clinical expert noted that people who have been referred to a psychiatrist are likely to be at risk of suicide or have symptoms that have not responded to any treatments for an extended period. The committee concluded that NICE's guideline on depression may not represent clinical practice and multiple further lines of treatment are considered for treatment-resistant depression.

## Esketamine is likely to be used later in the treatment pathway because it has a higher treatment burden than oral antidepressants
The clinical expert explained that esketamine has a higher treatment burden than oral antidepressants. A person having esketamine would have to attend hospital or a suitable community health centre site twice a week and then weekly or every 2 weeks for some time, for approximately 2 hours or more each visit. Treatment-resistant depression is characterised by a lack of energy and motivation so this may not suit all people. Travel to and from the hospital may be difficult because it is not possible to drive after taking esketamine. So, carer support may be needed. For these reasons, the clinical expert considered that esketamine would be used later in the treatment pathway than it was in the clinical evidence, for depression that is more severe and more treatment resistant. This would be after 1 or 2 augmentation therapies. The committee noted that in clinical practice having 1 or 2 augmentation therapies would happen over several years. After the second consultation, consultees confirmed that the appropriate positioning for esketamine was after 1 to 2 augmentation therapies. The committee concluded that the treatment burden, combined with the administration and monitoring concerns (see section 3.18), would mean esketamine is used later in the treatment pathway.

## The 3 or more treatments and 3 or more treatments and augmentation subgroups are appropriate
Treatment options for treatment-resistant depression depend on a person's treatment history, response to treatments and their preferences. Initially the company provided evidence for people whose depression had not responded to at least 2 treatments. But at the fourth committee meeting, the company provided evidence for 2 subgroups:
depression that had not responded to at least 3 treatments in the current episode (from now, 3 or more treatments subgroup)
depression that had not responded to 3 or more treatments and augmentation therapy in the current episode (from now, 3 or more treatments and augmentation subgroup).The company explained that the use of esketamine in these subgroups fulfils an unmet need for an option for people whose depression has not responded to several treatment options and where there is a high burden of illness. The committee recalled that the patient expert expressed feelings of hopelessness when depression does not respond to multiple courses of treatment. The committee considered that the burden of illness does increase at later lines of treatment so the 3 or more treatments and augmentation subgroup is the most appropriate positioning for esketamine. The committee concluded that it was appropriate to consider both subgroups based on the increased uncertainty of the evidence for later lines of therapy.

## Placebo with oral antidepressants may not be the most relevant comparator for the treatment subgroups
The company submission included oral antidepressants as comparators, stating that these were the most common treatments for treatment-resistant depression and that they were the comparators used in the trials. The trials compared esketamine plus a newly started oral antidepressant with placebo plus a newly started oral antidepressant. The clinical experts highlighted that it does not reflect clinical practice to start a new oral antidepressant at the same time as esketamine. The committee noted that different treatments are used at different times and that esketamine may be used later in the treatment pathway (see section 3.3 and section 3.4). The clinical expert noted that esketamine may be used as a preferable alternative to ECT. Consultees commented that ECT would most often be offered to people who are more acutely unwell and whose depression may have psychotic characteristics. But esketamine would be used with caution in these situations as described in the SPC. The company provided a network meta-analysis of esketamine compared with all comparators for the acute phase of treatment (the time frame that measures initial response to a treatment). However, the company and ERG noted there was substantial heterogeneity in the study design, inclusion criteria and time of outcome measurement, because of a lack of available evidence, which made the results unreliable. The committee concluded that the results comparing esketamine with some of the relevant comparators listed in the scope, such as combination or augmentation therapy and ECT, were highly uncertain. So, it considered only the results from the trials. At the fourth committee meeting, the company did not update the comparators despite changing the proposed position of esketamine in the treatment pathway. The ERG noted that an augmentation therapy could be the appropriate comparator for the 3 or more treatments subgroup, because an augmentation therapy was included in the company's next line of treatment. The committee also noted that oral antidepressants would not be the only comparators in clinical practice. The committee concluded that the appropriate comparators for esketamine were highly uncertain but likely included augmentation therapy. It further concluded that it was unable to assess the relative effectiveness of esketamine compared with augmentation therapy based on the comparative evidence available.

## The effect of using psychological therapy with drug treatments is an unresolvable uncertainty with the evidence available
The patient expert explained that psychological therapy can help with developing coping strategies and alleviate cognitive symptoms. An expert from NICE's guideline on depression noted that psychological therapies were not included as comparators or with combination treatments in the company's submission. The clinical expert explained that CBT is used with drug treatment to treat depression. But not all people with depression can effectively engage with CBT because of the severity of their physical and cognitive symptoms. A patient expert suggested that treatment with esketamine may improve symptoms for enough time for people to engage with CBT. But the clinical expert added that, because of the potential dissociative effects of esketamine treatment, someone would not be able to have psychological therapy immediately after having esketamine. The company clarified that people taking esketamine cannot have psychological therapy at the same time as having esketamine at their clinic visits, but could have therapy on a different day. At consultation, some consultees commented that there is limited evidence for the efficacy of psychological therapies in treating treatment-resistant depression and that including them was not considered for other pharmacological interventions. But in the fourth committee meeting, a patient expert added that they found intensive psychological therapies more useful than pharmacological treatments. However, the provision of intensive psychological therapies is inconsistent in the NHS. The patient expert noted that introducing more pharmacological treatments should not reduce other, already inconsistently provided, resources such as psychological therapies. The committee concluded that psychological therapies are an adjunctive therapy and a relevant part of the treatment pathway. But it noted that their effect would likely be variable depending on the treatment population and severity of depressive symptoms. It considered the estimation of clinical effectiveness of combining psychological therapies with esketamine treatment to be an unresolvable uncertainty with the evidence available.

# Clinical effectiveness

## The results from the flexible dose of esketamine in TRANSFORM-2 are the main source of randomised evidence
The company's key clinical effectiveness evidence came from 2 randomised phase 3 trials, TRANSFORM2 and SUSTAIN1. The company also provided supporting evidence from esketamine trials with different doses and populations (TRANSFORM1 and TRANSFORM3) and from 2 long-term safety studies (SUSTAIN2 and SUSTAIN3). The key trials were in adults aged 18 to 64 with treatment-resistant depression and compared:
a flexible dose of esketamine plus oral antidepressant with
placebo plus oral antidepressant.TRANSFORM2 provided randomised evidence for the acute phase of treatment for the 4week induction phase of the study, measuring symptom response and remission rates. SUSTAIN1 provided longer-term evidence about the continuation and maintenance of esketamine treatment. This was for people whose symptoms responded or people whose symptoms went into remission and were randomised to stop treatment. It measured symptom relapse rates. People could take part in SUSTAIN1 as new participants or they could transfer from TRANSFORM1 or TRANSFORM2 if their depression was in stable remission or stable response. Evidence for acute treatment of depression in people aged 65 and over came from TRANSFORM3, although this also included a lower starting dose of 28 mg, the same as in the SPC. The committee noted that the dose in TRANSFORM1 was a fixed regimen dose. The committee considered all of the clinical evidence, and noted that TRANSFORM1 and TRANSFORM3 did not show statistically significant improvements in outcomes for esketamine plus oral antidepressant compared with placebo plus oral antidepressant.

## MADRS is used to measure depression severity and treatment effect
The Montgomery-Asberg Depression Rating Scale (MADRS) measures severity of depression. It is scored between 0 and 60, 0 meaning no depressive symptoms. Primary outcomes of response and remission in TRANSFORM2 and relapse rates in SUSTAIN1 were measured using MADRS. Moderate to severe depression was defined in TRANSFORM 2 as a MADRS score of 28 or more. The mean baseline MADRS score was around 37. Symptom response was defined as a reduction in score of 50% or more from baseline. The clinical expert explained that this is a standard criterion for response. Remission was defined as a MADRS score of 12 or less with minimal or no symptoms. The clinical expert considered that remission is normally measured by a MADRS score of 10 or less (as in NICE's technology appraisal guidance on vortioxetine) but that this would not substantially affect the results. Relapse was defined as a MADRS score of 22 or more for 2 consecutive assessments or other clinically relevant event such as hospitalisation for depression. Recovery was defined as symptoms remaining in remission for about 9 months and recurrence was defined as depression relapsing after recovery. Stable response and remission were also used to define entry criteria for SUSTAIN1. These were the same definitions as above, but remission criteria had to be met for 3 out of the 4 weeks before randomisation and response criteria had to be met for the last 2 weeks before randomisation. The clinical expert noted that MADRS is non-linear, meaning that a change in score at the lower end of the scale does not mean the same, in terms of clinical importance, as a change in score at the higher end of the scale. The committee noted that remission and relapse are fixed to MADRS, but response measurement depends on the score at baseline, which complicates interpretation. The committee also noted that the MADRS score used for relapse in SUSTAIN1 (22 or more) was not the same as the MADRS score for moderate to severe depression used in the selection criteria for TRANSFORM2 (28 or more). This affected the health state utility values and transitions in the economic model (see section 3.26 and section 3.21). The committee took this into account in its decision making.

## Trial data suggests that esketamine is likely more effective than the comparator in the 3 or more treatments subgroup but the evidence is uncertain
The company initially positioned esketamine for people whose depression had not responded to at least 2 different antidepressants. For this population the adjusted mean reduction in MADRS score from baseline was 19.8 for esketamine and 15.8 for placebo plus oral antidepressant in TRANSFORM2. At the second committee meeting, the committee considered that it was difficult to distinguish the placebo response from the true treatment effect in the trial (see section 3.13). Also, the trial had a short 4week duration (see section 3.14). The committee concluded that it was unclear how effective esketamine was in the entire population. In response to the second consultation, the company provided this same trial data divided into 2 specific subgroups based on number of previous treatments. The company considered this analysis confidential and so it cannot be reported here. The company considered that the relative treatment effect of esketamine is greater in the 3 or more treatments subgroup than the full population. The ERG considered that the increased treatment effect for esketamine in the 3 or more treatments subgroup could be because of a lessened response in the placebo plus oral antidepressant arm. The committee noted that NICE's guide to the methods of technology appraisal 2013 (section 5.10.1) states a 'subgroup should be clearly defined and should preferably be identified on the basis of an expectation of differential clinical or cost effectiveness because of known, biologically plausible mechanisms'. The clinical expert considered that the difference in subgroups was plausible and would be expected, because a newly started oral antidepressant would likely be less effective in the 3 or more treatments subgroup than in the entire population. The committee considered that the evidence of benefit for esketamine in the full treatment population was driven by the change in MADRS score from baseline in the 3 or more treatments subgroup. However, it noted the following uncertainties with the subgroup analysis:
The relatively small size of the 3 or more treatments subgroup and overall low patient numbers (the exact patient numbers are confidential and cannot be reported here).
People in the 3 or more treatments subgroup in the trial might have different characteristics of depression than those expected to take esketamine in clinical practice (see section 3.16).In response to consultation, the company pooled and weighted clinical data from TRANSFORM2 and TRANSFORM3 (from now referred to as pooled TRANSFORM studies) for the 3 or more treatments subgroup. The NHS England clinical adviser noted that TRANSFORM3 was a smaller study than TRANSFORM2 with a 4week treatment phase. The committee considered there was still substantial uncertainty associated with the clinical evidence for esketamine but it is likely more effective than the comparator in the 3 or more treatments subgroup.

## The adjusted trial evidence for esketamine's benefit in the 3 or more treatments and augmentation subgroup is highly uncertain
The company used the relative treatment effect between the 3 or more treatments subgroup and the 3 or more treatments and augmentation subgroup in SUSTAIN2, a long-term safety study. The proportional treatment effect was applied to the esketamine arm from the 3 or more treatments subgroup from the pooled TRANSFORM studies, generating an estimate of effectiveness for the esketamine arm in the 3 or more treatments and augmentation subgroup. The company maintained the efficacy from the pooled TRANSFORM studies for the oral antidepressant plus placebo arm. The ERG noted the effect of augmentation was unclear from the evidence provided and the difference in treatment effect between the esketamine arm and oral antidepressant plus placebo arm was uncertain. The ERG was also unclear why the company used the relative treatment effect from SUSTAIN2 instead of estimates from the TRANSFORM studies. The committee recalled the uncertainty about the comparators for both treatment subgroups. It noted that if the comparator for the 3 or more treatments and augmentation subgroup was not an oral antidepressant the effectiveness of the comparator could be underestimated. The committee concluded that the adjustment to the trial evidence to establish the benefit of esketamine in the 3 or more treatments and augmentation subgroup was highly uncertain.

## There is supportive evidence from 2 non-randomised studies in Europe
In its response to consultation, the company provided data from 2 real-world studies to support the evidence for esketamine's treatment efficacy. One was a retrospective, observational study of 160 people (157 people were included in the analysis) taking esketamine who had treatment-resistant depression and who had an average of 2 suicide attempts during their life. The other study was a compassionate use study in Spain of 32 people whose depression had not responded to 2 or more antidepressants, 1 augmentation therapy and a non-pharmacological therapy. The company acknowledged there were a low number of people in both studies. The observational study showed a decrease in MADRS scores from baseline over 6 months. However, this was a single-arm observational study so there were no estimations of the effects of a comparator treatment. The ERG was unclear how the studies would overcome concerns regarding the generalisability of esketamine to the NHS. The committee acknowledged the effort of the company to identify additional supporting evidence but noted the low numbers of people in the studies and limitations of the observational evidence.

# Limitations in the clinical evidence

## It is not appropriate to adjust the efficacy estimates of the comparator arm in the trials
For the full population, the company considered that the efficacy estimates (response and remission) for the placebo plus oral antidepressant arm of the TRANSFORM2 trial were high compared with other studies considering depression. The company suggested a post-hoc adjustment of the TRANSFORM2 data to account for some of these differences. The committee considered the reasons for the high placebo response rate:
In the trial, people visited the clinic more than in clinical practice. In the 4 weeks, people who had esketamine had 8 clinic visits. People who had the placebo nasal spray also had 8 clinic visits to preserve blinding. However, the company estimated that in clinical practice people taking oral antidepressants would only have 2 visits with healthcare professionals over a 4week period. The clinical expert highlighted that increased clinical contact could increase the effect of treatment. The committee considered that the additional clinical contact involved in administering esketamine included support from mental health nurses and establishing relationships. A patient expert noted this was an important part of treatment (see section 3.35). The committee noted that planned and structured clinical contact improves outcomes and that in NHS practice oral antidepressant treatment is ideally combined with other psychological therapies, which would also be structured. The expert from NICE's guideline on depression considered that the efficacy estimates in the placebo plus oral antidepressant arm seemed higher than expected. The ERG considered it inappropriate to apply adjustment to the placebo plus oral antidepressant arm because it is impossible to be confident about the placebo effect associated with esketamine in clinical practice. The committee concluded that the trial design may have increased clinical contact but there was no evidence this would cause the placebo response. It also concluded that any adjustment to account for clinical contact was not appropriate because of the risk of bias.
In the trial, people may have had a high expectation of esketamine because it has a novel treatment mechanism. But the committee considered that blinding could be an issue in the trials. This is because for people having placebo plus oral antidepressant, the absence of psychoactive effects and other effects expected with esketamine could lead to consequent negative expectations and a lower response to treatment. So the true treatment effect is unclear if the blinding was not preserved, and adjustment of a placebo effect could lead to unknown biases.
In the trial, people's symptoms also responded to the new oral antidepressant given alongside placebo. The committee noted that in clinical practice, oral antidepressants would not be newly started at the same time as esketamine, because it is not clinical practice to try 2 new therapies at the same time. So any response from trying the new oral antidepressant is difficult to separate from the treatment effect of esketamine. The committee noted that for the 3 or more treatments subgroup, a reduced response to the new oral antidepressant was likely to explain the lessened response of the comparator arm (see section 3.10). So any adjustment of placebo response could also account for this effect with unknown bias.The committee concluded that the randomised design of the trial helps to mitigate for the placebo response. So it was not appropriate to adjust the efficacy estimates of the placebo plus oral antidepressant arm in the trials. Any adjustment would not explore all potential sources of difference between treatment arms so could introduce bias.

## The response and remission evidence from the TRANSFORM studies should be considered with caution when used in the economic model
For the full population with treatment-resistant depression, TRANSFORM2 measured a statistically significant difference in MADRS score after 28 days between esketamine plus newly started oral antidepressant compared with placebo plus newly started oral antidepressant. The committee noted a separation of treatment effect for the full trial population after 2 days (or 1 treatment), which remained for the duration of the 4 weeks. The NHS England clinical adviser added that the treatment benefit of esketamine in TRANSFORM3 was not statistically significant compared with placebo plus oral antidepressant. The NHS clinical adviser also stated that no rapid effect of esketamine was observed, with a separation of treatment effect compared with the placebo plus oral antidepressant arm after 22 days. The committee noted that NICE's guideline on depression recommended an initial assessment at 2 to 4 weeks to assess response to oral antidepressant, but further regular assessments and dose optimisation would be considered after this point. The committee considered that the evidence suggested there was likely to be changes in MADRS score as part of initial response to treatment. So, it considered that 4 weeks was not an appropriate endpoint on which to base longer-term extrapolations of response and remission. Also, a consultee commented that splitting data into 2 groups, response or remission compared with no response or remission, can lead to an overestimation of differences between arms. The committee acknowledged that this could have inflated the differences between arms which would increase the uncertainty of response and remission rates. For example, the committee noted that response to esketamine was higher for the 3 treatments or more subgroup, despite a lower change from baseline in MADRS score. The committee concluded that although the response and remission evidence from TRANSFORM2 showed a statistically significant difference between esketamine and placebo plus oral antidepressant, the data should be considered with caution when used to generate transition probabilities in the economic model because of the duration of the trial (see section 3.19).

## The withdrawal design of SUSTAIN-1 could introduce bias
SUSTAIN1 measured withdrawal of esketamine for a randomised population of people whose depression was in stable response or stable remission. The ERG commented that there was potential for selection bias using these criteria. This is because if esketamine is tolerated, people who have the drug for 16 weeks and do not stop (induction and optimisation phases) stay in the trial by design. This means the people selected to stay in the trial are less likely to be affected by the treatment burden and do not have adverse events that make them stop treatment. So this does not represent the full population in the acute phase of treatment (from TRANSFORM trials) and may underestimate relapse rates for those taking esketamine. After the optimisation phase, some people were randomised to stop having esketamine and instead had placebo. Everyone continued taking the same oral antidepressant they had at the start of the trial. A consultee commented that there is potential for functional unblinding with this design because people randomised to placebo may notice the absence of psychoactive effects. The consequent negative expectations could affect the results. In response to the second consultation, the company provided analysis that showed censoring people who had dissociative symptoms and relapsed did not significantly affect the results. Because this analysis focused on dissociative effects only, the committee was not persuaded that this necessarily showed the blinding was effective. The committee concluded that the withdrawal design of SUSTAIN1 meant that if any unblinding had occurred, it would have biased the results in favour of esketamine. It noted that the withdrawal trial design was mandated by the regulator, but the faster onset of action may differ from oral antidepressants.

## The evidence from the trials is limited in its generalisability to the NHS
The company assumed that data from TRANSFORM2 and SUSTAIN1 were generalisable to NHS clinical practice, but no people were recruited in the UK. Also, TRANSFORM2 and SUSTAIN1 excluded people:
with moderate to severe substance or alcohol dependence according to the Diagnostic and Statistical Manual of Mental Disorders fifth edition (DSM5) criteria
with some psychiatric comorbidities
with depression that had not responded to at least 7 treatments with ECT in the current major depressive episode
who had vagus nerve stimulation or deep brain stimulation
who had suicidal ideation with intent in the previous 6 months or suicidal behaviour in the previous 12 months. The ERG noted that those excluded from TRANSFORM2 and SUSTAIN1 could represent a substantial proportion of people with treatment-resistant depression. It considered that excluding these people limited the generalisability of the trials to the NHS. The expert from NICE's guideline on depression agreed and noted that people with treatment-resistant depression are likely to have an increased risk of suicide. A clinical expert also noted that suicidal ideation is often an integral part of the disease. The committee noted that many people referred to a psychiatrist (a requirement of the SPC) in NHS clinical practice would be at higher risk of suicide. The clinical experts acknowledged the limitations of the other exclusion criteria but explained that these are standard for trials in this population. Comments received at consultation confirmed that uncertainty introduced by excluding these people is common in trials in this disease area. The committee was aware of the comments in the European public assessment report (EPAR) about the precautions that need to be taken if people with psychiatric comorbidities take esketamine. The committee also noted that the population in the TRANSFORM and SUSTAIN trials may not be in line with the population expected to have esketamine in clinical practice (see section 3.4) and that people with more severe symptoms may be more likely to be excluded using these criteria. The committee considered that the other exclusion criteria could inhibit the generalisability of the trial results. The committee concluded that excluding people with recent (defined in TRANSFORM2 and SUSTAIN1 as within 6 months to the start of the screening or prospective observational phase) suicidal ideation with intent or suicidal behaviour limited the generalisability of the trials to the NHS for people with treatment-resistant depression. It considered this was likely to be more of an issue when considering populations whose depression is more treatment resistant, which may correlate with increased potential for psychiatric comorbidities and suicidal ideation.

## Some of the clinical uncertainties are common in clinical trials in mental health
The clinical expert considered that there was a mismatch between the evidence from the clinical trials and how esketamine would be used in clinical practice. They noted that it was difficult to collect randomised evidence in this disease area. The committee considered that some of these clinical uncertainties are common in clinical trials in mental health.
The committee acknowledged that the treatment pathway is not clearly defined by line of therapy and instead is based on patient history and patient preference (see section 3.3). But it accepted the treatment line approach for the practical purposes of the economic modelling. The committee acknowledged it may be practically challenging to overcome this uncertainty so concluded this was currently an unresolvable uncertainty and did not consider it further.
The committee also recalled the uncertainty of the benefit of esketamine in a later treatment line (see section 3.11). The committee noted that a trial done in the population that the treatment is being positioned in (3 or more treatments subgroup with or without augmentation) would help to overcome this uncertainty. It was aware of an ongoing phase 3 clinical trial (ESCAPETRD), which it considered would be in a population more aligned with who would likely have esketamine in the NHS. So the committee concluded that this was a resolvable uncertainty.
The committee accepted that clinical trials in depression have a mandated regulatory minimum length. It noted this when assessing esketamine's clinical-effectiveness evidence. But it noted that the trials are short and may not reflect clinical practice (see section 3.14 and section 3.4), which generated large uncertainty in the cost-effectiveness modelling. The committee considered that doing a longer trial than the mandated regulatory minimum length required could reduce this uncertainty. The committee highlighted that the lack of long-term evidence was a key uncertainty and noted an assumed long-term benefit was a key driver of the cost-effectiveness estimates. The committee concluded that the company could have done longer trials to meet the requirements of health technology assessment and so this was a resolvable uncertainty.
The committee recalled that the study designs result in relatively larger placebo response than in other disease areas but preferred not to adjust the efficacy estimates of the comparator arm for the purposes of the economic modelling (see section 3.13). To overcome this uncertainty, the committee considered that a trial with a third arm without the placebo nasal spray may help with considering how people are affected by being in a trial (for example an improvement in MADRS score because of more frequent contact with trial investigators than would happen with clinicians in clinical practice). The committee noted that this would not necessarily overcome all the issues relating to the placebo response but would contribute to a better understanding of the relative effect of treatment observed in the trial. The committee concluded that this was a partially resolvable uncertainty.
The committee stated that psychological therapy is likely an important part of treatment but provision and efficacy would vary substantially in a multinational clinical trial setting (see section 3.7). The committee noted that the uncertainty could have been overcome by doing an additional trial that included psychological therapy. But it acknowledged that this may be practically challenging because of the need for a longer follow-up and differences in the types of psychological therapies available. Also introducing another treatment into a clinical trial may make it difficult to measure the effect of esketamine. The committee noted that the uncertainty was partially resolvable and accepted the evidence without considering the effect of psychological therapies.
The committee noted that excluding some people from the trial limits the generalisability of effectiveness results to later lines of therapy and NHS clinical practice (see section 3.16). It recalled that this may help with understanding relative effect so is common practice in clinical trials. But the committee recalled that the generalisability of the data led to substantial uncertainty in the cost-effectiveness estimates. The committee considered that the uncertainty could have been reduced if the trial was done in a population that was a closer match to the people most likely to have esketamine in NHS clinical practice but acknowledged the practical challenges of doing this. This is because it may lead to a more heterogenous population so it may be more difficult to measure the effect of esketamine. But the committee noted that some of the uncertainty could have been resolved if the company included people with recent suicidal ideation with intent. The committee considered this would likely not have increased the heterogeneity of the population with treatment-resistant depression. The committee concluded that this was a partially resolvable uncertainty.The committee recognised the difficulty of designing, recruiting and interpreting results from clinical trials in this disease area, and that the evidence requirements of health technology assessment may be different than the licensing requirements captured through regulatory endpoints (see section 4). The committee agreed that these clinical uncertainties were important to the economic modelling and took this into account when considering the degree of certainty of the cost-effectiveness results. The committee also acknowledged that there is ongoing research in this disease area could potentially address some of the uncertainties.

# Safety

## Additional monitoring and supervision is required with esketamine in line with the SPC
The European Medicines Agency identified some risks of esketamine use in the SPC. These included drug dependence, transient dissociative states and perception disorders, disturbances in consciousness, and increased blood pressure. A register for administering and monitoring esketamine to prevent dependence and misuse has been set up with the Medicines and Healthcare products Regulatory Agency (MHRA). The NHS commissioning expert explained that, because esketamine is a schedule 2 drug, it is subject to the full controlled drug requirements relating to prescriptions and storage. The committee acknowledged that the monitoring period would likely mitigate the other risks identified in the risk management plan and the committee did not need to consider these further. The committee considered comments received in consultation regarding suicides in people who stopped esketamine in a population who had no recent suicidal ideation or behaviour. The clinical expert explained that the increase in suicidal ideation could have happened despite people having esketamine, rather than because of it, given the nature of depression. The committee noted the SPC states that the effectiveness of esketamine in preventing suicide or in reducing suicidal ideation or behaviour has not been demonstrated. The SPC notes that general clinical experience shows that the risk of suicide may increase in the early stages of recovery. The committee considered that the MHRA are responsible for assessing safety concerns. It considered that the precautions regarding risk of suicide and supervision and monitoring in the SPC should be taken into account when prescribing esketamine, particularly during early treatment and after dose changes. The committee concluded that it was not a safety committee and could not make recommendations about safety.

# Economic model

## The company's economic model uses uncertain clinical inputs so its results should be interpreted with caution
The company economic model had 5 health states: major depressive episode (MDE), response, remission, recovery and death. The transitions between each health state were determined by the relapse, remission and response rates in the TRANSFORM studies and SUSTAIN2 (see section 3.10 and section 3.11) and values in the literature, for example, the STAR*D trial (a large-scale clinical trial for people with depression). All people start in the MDE state. In each of the arms and subgroups the first treatment is followed by 3 more potential subsequent treatments after non-response or relapse, and then a non-specified mixture of treatments (best supportive care). The committee considered that the key drivers of the economic model results in both treatment subgroups included the following:
The pooled 4week initial response to treatment from the acute efficacy trials has the most influence on the modelled differences between treatment arms (see section 3.20) and is the only randomised comparative data that addresses the decision problem.
The relapse rate for the esketamine treatment arm uses data from SUSTAIN1, which likely underestimates relapse rates because of selection bias from including only people with stable response and stable remission (see section 3.15 and section 3.21).
The relapse rate for the placebo plus oral antidepressant arm uses data from the STAR*D trial which has generalisability issues and bias from using a different trial design and population (see section 3.21).
The effect of the 3 lines of subsequent treatments does not reflect how subsequent treatments are used in clinical practice (see section 3.22).
The efficacy of the non-specified mixture of treatments after the subsequent treatments has a substantial effect on the modelled long-term outcomes, resulting in a large amount of time spent in the MDE health state, which is a key driver of the costs (see section 3.22 and section 3.32).The committee concluded that the clinical inputs informing the model are highly uncertain and that any modelled results should be interpreted with caution.

## The modelled difference between treatment arms is driven by response and remission in the acute phase of treatment
The first 4-week cycle of the economic model represents the transition from the MDE health state to response or remission, informed mostly by TRANSFORM2 with some information from TRANSFORM3. The committee noted that the initial response rate was uncertain because of the short time frame and because it included a placebo response that may not be seen in clinical practice. The committee noted the importance of this initial response or remission rate because later transitions to different health states are more gradual, determined by relapse rates and subsequent treatments. So, the committee noted a key driver of the difference between arms was the initial response. So, accurate response and remission rates in the acute phase of treatment are needed to give robust results.

## The relapse rate data comes from different sources, which leads to uncertainty and potential generalisability issues
For the relapse rates in the economic model, the esketamine arm used transitions between health states from the SUSTAIN1 trial. The comparator arm used relapse data from the STAR*D trial. The relapse and loss of response rates for SUSTAIN1 are based on a MADRS score of 22 or more for 2 consecutive assessments. The committee noted that this was not equivalent to the inclusion criteria in the trials, which was a MADRS score of 28 or more. The committee considered it unclear if some transitions categorised as relapses are rather fluctuations in severity of depression, consistent with a chronic disease model (see section 3.23) and whether the transition would mean a change of treatment in clinical practice. The STAR*D trial used a Quick Inventory of Depressive SymptomatologySelf-Report score of 11 or more to measure relapse. The committee considered that the different definitions of relapse criteria contribute uncertainty to the comparison. The committee also had concerns about the generalisability of the trial design and population of STAR*D to NHS practice. The committee concluded that using different sources of data for relapse leads to potential generalisability issues and bias in the economic model.

## The effect of subsequent treatments does not match what would be seen in clinical practice
The ERG noted that the response and remission rates of subsequent treatments used in the company's original base case could not be validated and were likely to be underestimated. The ERG proposed a scenario that applied a proportional reduction in each line of therapy based on data from the STAR*D trial. In response to consultation, the company provided this scenario in its revised base case. The response and remission rates were calculated on a 4weekly basis to be implemented per cycle in the model. This meant people moved between treatments quickly if their symptoms did not respond within 4 weeks. The committee considered that moving through treatments would not happen that quickly in clinical practice. It also recalled the uncertainty with using the STAR*D trial data and its generalisability to UK practice in the expected population (see section 3.21). Also, the proportional reduction was applied to the loss of response and relapse rates, which resulted in a relapse rate of 99% every 4 weeks for the non-specified treatment mix that represents best supportive care. The ERG considered this to be implausible so provided a scenario with a cap on relapse and loss of response as described in the original submission. After the third meeting, the company base case included further changes to this modelling assumption. The committee considered that despite the increased efficacy of subsequent treatments, the best supportive care transitions still had the greatest effect on long-term outcomes, which were highly uncertain. This mostly affected the costs because it meant a large amount of time was spent in the MDE health state. The committee considered there was minimal evidence for all transitions in the best supportive care state, and there was considerable uncertainty about how the course of the disease was modelled. It concluded that neither the company's revised base case nor the ERG's treatment cap would accurately model what happens to patients in clinical practice. Instead, the company and the ERG estimated the proportion of people in the MDE health state at later stages of the model, for which there was no available evidence.

## The disease course of treatment-resistant depression is uncertain and further research is needed
In the second appraisal consultation document, the committee considered that the economic model likely overestimated the number of people in the MDE health state in both treatment arms and did not reflect the course of the disease or its episodic nature. In response to consultation, the company provided a targeted literature review that it considered supported the modelled output. The model output suggested that, for a person with treatment-resistant depression, 66% of a person's life is spent in the MDE health state. The targeted review showed generally low longer-term remission rates. However, the committee noted heterogeneity in the definition of remission and response, trial design and inclusion criteria of the trials included in the review. In particular, it considered that the trials in the review may have included people with depression that was much more treatment resistant and severe than the modelled population. The company also provided evidence from a UK cohort, which showed the mean duration of an episode of treatment-resistant depression was 6.1 years. The ERG provided another data source which was a large retrospective cohort study of insurance databases in the US. This study attempted to characterise the treatment journey for someone with treatment-resistant depression at an episodic level based on length of treatment on oral antidepressants. The results showed the mean length of a first episode of treatment-resistant depression was 1.56 years, and the mean length of remission for those who had a second episode was 0.90 years. The clinical expert considered this data would be of limited use because it used time on treatment data and it would not necessarily fit to an episodic model. The ERG considered that a treatment-resistant episode was inconsistently defined and would likely be measuring different outcomes. It noted that the outcomes of interest for modelling the MDE health state were the severity of depression for the full time horizon (see section 3.24) and the costs accrued while in the state (see section 3.31). The committee also considered consultee comments that stated that improvements in depression are generally maintained at the end of acute treatment, and on average symptoms improve further. It also heard from consultees who commented that depression can be highly episodic, and that treatment can be successful when people adhere to it. The clinical expert estimated that currently 20% to 30% of people with treatment-resistant depression have chronic longer-term disease that has not responded to any treatment. For these people, severity would likely fluctuate, and this would not fit the episodic disease model well. After the company repositioned esketamine as a treatment used at a later line, the committee considered a chronic longer-term disease model may be more appropriate to capture the profile of this group. The committee noted substantial uncertainty with all the longer-term data for treatment-resistant depression and agreed with the ERG that the available evidence is likely measuring heterogeneous outcomes in heterogeneous populations. The committee understood the difficulties of modelling a heterogeneous population with differing disease models. But it concluded that the analysis did not appropriately capture either the chronic or episodic nature of the condition. It also concluded that the literature for longer-term outcomes for treatment-resistant depression is poor, so the outcomes are highly uncertain. The committee recommended further research to understand the course of the disease (see section 4).

## The long-term evidence of esketamine is too uncertain to justify a substantial modelled benefit over a 20-year time horizon
The company originally modelled a 5year time horizon to reflect that treatment-resistant depression is an episodic condition. The ERG noted that differences in the modelled costs and quality-adjusted life years (QALYs) between treatments continued for 20 years, so it preferred a 20year time horizon. The clinical experts considered a longer time horizon was appropriate because depression is a chronic condition for some people. The expert from NICE's guideline on depression agreed that a longer time horizon was needed to account for the duration of the condition and the need for any subsequent treatments. After the company repositioned esketamine as a treatment used at a later line, the committee considered the 20year time horizon was likely appropriate. But it noted that there was insufficient data to populate this model for the full time horizon because of the uncertainty about the inputs into the model and esketamine's long-term outcomes (see section 3.23). It noted that the model was sensitive to assumptions about the length of the time horizon because esketamine costs were modelled in the short term but benefits accrued over the full time horizon. The committee also recalled that the disease course of treatment-resistant depression is uncertain and capturing the fluctuating nature of the condition and treatment is difficult. The committee explored this uncertainty through sensitivity analysis. It concluded that the long-term evidence for esketamine is too uncertain to justify substantial modelled benefit over the full time horizon.

## It is not appropriate to include an effect of esketamine on mortality in the model
In its economic model, the company assumed there were 2 risks for dying: all-cause mortality risk (specific to age and gender) and an excess annual mortality for treatment-resistant depression associated with suicide. The company initially modelled a reduction in treatment-resistant depression (which is associated with excess mortality). This indirectly decreased the risk of excess mortality with esketamine. The committee considered it plausible that esketamine could affect mortality. However, with the other structural uncertainties and no evidence of longer-term benefit of esketamine, the committee considered this was speculative. It also noted that the SPC states: 'The long-term efficacy of Spravato to prevent suicide has not been established'. Because of issues with generalisability, excluding people with recent suicidal ideation with intent and the lack of data, the committee concluded it could not accept a reduced suicide, or mortality, risk.

# Utility values

## The difference in utility values between health states is likely overestimated
The company measured utility in the TRANSFORM2 and SUSTAIN1 trials as EQ5D5L measurements and mapped these to EQ5D3L utility values as in the NICE reference case. These utility values were applied to the modelled health states. The committee noted that the utility value for MDE of 0.417 was measured from the baseline utility scores in TRANSFORM2 at a mean MADRS score of 37. However, the transition from relapse or remission to the MDE state needed a MADRS score of 22 or more for 2 consecutive assessments. The committee recalled that response criteria were not fixed to absolute MADRS values. This made interpreting the utility values difficult because they could have come from people with MADRS scores of between 13 and a maximum value above the threshold for relapse of 22 or more. In response to consultation, the company provided a scenario that provided MDE state utility values that represented people with moderate depression to include a lower relapse rate. The ERG noted this did not address the problem of the estimate of relapse based on the threshold with a lower MADRS score. The committee noted that symptom severity could fluctuate, and that this would not be consistent with a fixed state with large utility transitions. The clinical expert noted that a MADRS score of 37 represents very severe depression as would be expected during an acute period of depressive symptoms. The committee considered that in clinical practice, people will likely have less severe MADRS scores on average for prolonged periods of time. It recalled the difficulty of defining and characterising a treatment episode. The clinical expert added that people tend to have symptoms at least half of the time, but these symptoms are not always severe enough to reach diagnostic criteria. The committee was also concerned that the utility values within each health state could be highly heterogeneous. It concluded that the MDE health state utility would likely represent the baseline utility values of patients with a MADRS score of 37, but would likely underestimate quality of life over the full time horizon.

## It is appropriate to consider applying a carer disutility in the model and to consider the effect without it
The company submission included a disutility value applied to the model for the effect of treatment-resistant depression on carers and families. This was done by applying a disutility to the MDE health state. This was the difference in utility between carers of people with symptomatic treatment-resistant depression and carers of people with treatment-resistant depression that was in remission. The ERG noted that this implied that carers of all people in the MDE health state would have a utility value associated with being in remission. The ERG argued that a methodologically better way to estimate disutility for a specific state is to subtract the utility of that state from the utility for full health. The ERG applied a lower value to the disutility by using this method to calculate the utility values. The committee acknowledged that treatment-resistant depression affects carers and families and considered the ERG scenario to be the most appropriate. But it considered that there was uncertainty about how appropriate including a carer disutility was. This was because of the lack of data on the direct effect treatment-resistant depression had on carers. The committee noted the lack of evidence on any direct benefit to carers after treatment with esketamine. It also noted the potential for an increased treatment burden for carers as well as people with depression. The committee considered that carer utility is only applied in the MDE health state, which is likely to be overpopulated in the economic model. It noted that carer disutility was not considered in NICE's technology appraisal guidance on vortioxetine. The committee concluded that it was appropriate to consider scenarios with both the ERG carer disutility scenario and no carer disutility because the effect was uncertain.

## The disutility of adverse events should have been considered in the modelling
The committee recalled that esketamine is associated with some potentially serious adverse events. SUSTAIN2 reported 6.9% of people had serious adverse events including depression, suicidal ideation, suicide attempt, anxiety and gastroenteritis. The most common treatment-emergent adverse events (any event not present before the start of treatment) included dizziness and dissociation. The company did not consider the disutility or cost of these adverse events in the model because most adverse events were resolved on the day (75% of adverse events were resolved on the day in SUSTAIN2) and so it considered these were transient. The patient expert described unwanted effects they experienced while having esketamine. The committee considered that a large proportion of patients may experience these effects, and it was likely a major consideration for the treatment experience. It considered these events could be experienced up to once a week which, combined with fear of these adverse events, could cumulatively contribute to a substantial disutility associated with the treatment that was not captured in the model. The committee concluded that adverse events of treatment had not been fully explored, but would contribute additional uncertainty to the modelled treatment benefit.

# Stopping treatment

## There is limited evidence on the effect of stopping esketamine for reasons other than lack of efficacy
The company assumed that some people would stop taking esketamine for reasons other than lack of efficacy, in line with the criteria in the SPC and additional guidance on stopping treatment. In response to consultation, the company provided scenarios for stopping treatment and scenarios that explored a utility decrement after stopping treatment. The stopping rates were based on research questionnaires from clinicians. Stopping treatment was assumed to stop costs for esketamine incurring but have no effect on QALYs. The company modelled that 60% of people whose depression was in stable remission would immediately stop treatment after 2 years. The ERG also provided a scenario that assumed no immediate treatment stopping and instead modelled a continued exponential reduction based on extrapolation of the trial data. This was because no evidence was submitted that showed the effect of stopping on long-term symptoms or quality of life. The ERG and clinical experts also highlighted that there was no data to accurately determine stopping rates in clinical practice. The ERG noted that no data was collected for people who stopped treatment for reasons other than lack of efficacy after recovery, and the reasons why they stopped were not explored. The committee noted that the SUSTAIN1 trial is designed to answer the clinical question of whether stopping treatment affects relapse rates. It showed that there is a significant effect of stopping treatment. However, the committee noted this was measured at 16 weeks of treatment rather than after recovery. The committee considered it likely that people would stop esketamine for various reasons over a 2year period. This could include recovery, adverse events or because of the high treatment burden associated with its use. However, it considered that the research data informing the company revised base case may not be generalisable to NHS practice. This was because it classified people into risk levels and applied these to the full population in SUSTAIN1. With the company's revised positioning, this could include people whose depression would be resistant to more lines of therapy than in SUSTAIN1, likely have more comorbidities and may include more people who would have prolonged or repeated use of esketamine. The committee also noted that the data for the utility decrement came from the SUSTAIN2 study and had limited use in the model because of the high proportion in the MDE state, so did not explore this scenario further. The committee concluded that more data for stopping treatment for reasons other than lack of efficacy was needed to justify modelling the additional stopping guidance provided by the company.

## Stopping treatment in clinical practice would be based on people's individual circumstances and may include prolonged or repeated treatment
The clinical expert explained that stopping treatment is variable in clinical practice. They would expect that the decision to stop treatment would be made after a discussion of the person's individual circumstances. They also considered that this could involve treatment pauses to assess how a person feels without esketamine. The clinical expert noted that the best indicator for what treatment would work would be what the person's depression had responded to previously. Also, people who consider esketamine to be effective may want to carry on taking it. A patient expert suggested that if treatment with esketamine worked for someone then they would consider having the treatment again when symptoms returned. They also noted that people would be concerned and worried about relapse if they stopped treatment. Another patient expert explained that their esketamine treatment was in the process of being tapered off slowly with careful monitoring of response. The committee recognised that people would be fully involved in the decisions about continuing treatment, and that decisions about how long treatment lasts and reasons for stopping it vary based on individual circumstances. The committee also considered that it is possible more people's symptoms would respond to treatment, but not all of them would be considered to be in remission if their depression were more treatment resistant or severe at baseline. These people would not stop treatment immediately at 2 years using the company's stopping criteria. The committee also noted the uncertain course of the disease. It considered that a fluctuation in symptom severity might not mean that esketamine was stopped, as it was in the trials. It concluded that in clinical practice stopping treatment may not be guided by the company criteria and could include ongoing repeated or prolonged treatment based on symptom severity. This would particularly be the case for the expected population in NHS clinical practice and the 3 or more treatments subgroup.

# Resource use

## The cost of a course of esketamine treatment may be underestimated
The company confirmed that the dose of esketamine used in the model was an average from the trial evidence. The committee was concerned that it had not been presented with a dose response curve or a clear analysis of how the flexible dosing strategy was implemented. It also considered that it was unclear if people develop a tolerance to esketamine and need increased doses (up to the maximum dose) to achieve the same therapeutic effect. This would be particularly important for people who have treatment for a long time. The committee noted that frequency of dose during maintenance was dependent on meeting remission criteria (for a weekly dose, MADRS score of more than 12, and for a 2weekly dose, a MADRS score of 12 or less). The committee considered that a change in what is considered remission, for example a MADRS score of 10 or less (as in NICE's technology appraisal guidance on vortioxetine), could affect the costs of treatment. Also, in line with a chronic disease model, some people who met the relapse criteria may have been considered to have had a fluctuation in symptom severity, rather than a relapse. So, they may have had prolonged or repeated treatment. Also, issues with generalisability of the trial evidence and esketamine's changed positioning in the treatment pathway could increase the number of people whose depression was considered to have only responded, compared with people with depression considered to be in remission. The committee noted that the new observational data presented after the third committee meeting likely used higher or more frequent dosing for most people than was modelled. Although the committee considered that this was not used in the economic model, it might represent the expected use in clinical practice. The committee concluded that the model may underestimate the cost of a course of esketamine treatment for the intended treatment population. It also noted that a course of treatment may not easily be defined in the context of a chronic condition with repeated or prolonged treatment.

## Healthcare resource use costs are highly uncertain and contribute to the economic model's uncertainty
The company modelled healthcare resource use by health state as defined in the economic model. The committee noted the importance of the MDE health state because of the amount of time people spent in it. The company measured resource use for each health state using a retrospective chart review of patients in UK clinical practice. This asked 30 psychiatrists and 9 GPs to provide resource use for the last 10 people with treatment-resistant depression they had seen. These were converted into costs by health state in the company study. These costs included primary care visits, secondary care visits, psychological-based interventions, occupational therapy, hospitalisations and crisis resolution home teams. The clinical expert considered that, in clinical practice, the distribution of costs in treatment-resistant depression were heavily skewed. This means most healthcare resource is used by a proportionally small number of people. The committee understood this was mostly through a small number of people needing hospitalisation. The committee noted that there was potential for selection bias because people that are seen more frequently are more likely to be included in the study. The committee was concerned that some of the costs seemed implausibly high. For example, crisis resolution and home treatment teams contributed 33% of all costs in the MDE health state, which was nearly equivalent to the total costs of hospitalisation. The ERG provided an alternative healthcare resource use scenario using a large database of general practice records used in Byford et al. (2011). This was used in NICE's technology appraisal guidance on vortioxetine and NICE's guideline on depression to provide cost information. The ERG considered that only the population with severe depression should be used to represent people in the MDE health state. However, it noted that the definitions used in the Byford study for severity included many types of depression, including depression with psychosis. Only people who had at least 2 treatments and whose symptoms had not remitted within a year were included in the Byford study costs for the MDE health state. This did not match the company's definition of treatment-resistant depression. Also, the Byford study linked primary care records with secondary care referrals and hospitalisations. The company considered that the study did not fully capture hospitalisation costs or community interventions, which have increased since the study was done. The committee agreed that some of the costs of secondary care or hospitalisation could have been missed in the Byford dataset because of the study design. However, the committee noted the substantial difference between the 2 studies, with the costs associated with the MDE health state in Byford being around 8% of what was reported in the company's retrospective chart review. After the third committee meeting, the company proposed 2 new treatment subgroups. It modelled a split of 75% of costs from its own costing study and 25% from the Byford study for the 3 or more treatments subgroup, and 100% of costs from its own costing study for the 3 or more treatments and augmentation subgroup. It also presented supporting evidence from a secondary care mental health setting using Clinical Record Interactive Search data from the South London and Maudsley NHS Foundation Trust. The company considered it showed increasing healthcare resource use with each line of therapy. The ERG noted that, similar to the company cost study, the costs were largely driven by hospitalisations (in cost of bed days). There was also evidence of strongly skewed data, with high costs on average, but with most people not requiring any hospitalisation. The ERG also noted that the relationship between increasing healthcare resource use and line of therapy was not clearly defined and the difference between 3 prior oral antidepressants and 4 prior oral antidepressants was minimal. The committee noted that all of the evidence considered about resource use was characterised by strongly skewed data, which introduced substantial uncertainty to estimates of non-pharmacological healthcare resource costs within the model. These accounted for almost all of the total costs over the full time horizon in the company base case and were a key driver of the cost-effectiveness results. The committee considered that the generalisability of the treatment costing study to NHS clinical practice was crucial to understanding whether esketamine would reduce hospitalisations and other healthcare resource use. It questioned whether the same people who are hospitalised would have esketamine, because there are precautions for its use for people with certain psychiatric comorbidities. The committee noted that cost savings are a key driver of the cost-effectiveness estimates. The committee also noted that cost savings are driven by reducing costs in a small group of people (those who would be hospitalised). But it considered there was no evidence that esketamine would be beneficial in the group of people for whom hospital costs are largest. While the evidence showed there was an improvement in MADRS scores in the subgroups it was not possible to predict the benefits for subgroups within these subgroups. The committee also heard from clinical experts that for some people who are hospitalised, other options such as ECT are more likely to be considered than esketamine. The committee concluded that the modelled benefit of esketamine was not robust given the uncertainty in the evidence. It recommended further research to fully understand the costs associated with treatment-resistant depression and hospitalisations (see section 4).

## Significant investment would be needed to use esketamine in the NHS
The company assumed people in the 3 or more treatments and augmentation subgroup would have treatment in secondary care. It also assumed that the implementation of esketamine would be done using existing infrastructure. So, the company did not include costs for implementation in this subgroup. The ERG noted it was unclear how a change in infrastructure would not be needed for the 3 or more treatments and augmentation subgroup. For the 3 or more treatments subgroup the company proposal to convert ECT suites to esketamine treatment clinics was used to inform costs in the economic analysis. The company included administration costs for esketamine in the economic model, but did not consider any other costs, considering these to be minimal with the conversion of ECT suites. It also said it would provide staff training to administer and monitor esketamine, needed to manage adverse effects, at no additional cost. The NHS commissioning experts noted several costs for adopting esketamine that were not included in the analysis:
costs of conversion of ECT suites or sourcing other appropriate treatment settings
costs of medical equipment to monitor and manage any post-dose medical complications
staff training to manage post-dose complications, including potential costs of recruitment if there are not enough staff currently available in practice
costs associated with the controlled nature of the drug, including storage, transportation, disposal and adequate staffing and governance training
costs associated with transporting people to have esketamine in hospital.The NHS commissioning expert also noted that resources would be needed for each new person having treatment with esketamine. The committee noted that the costs of implementation would depend on the expected population in clinical use and the expected treatment setting. The committee noted that NICE's guide to the methods of technology appraisal 2013 (section 5.5.8) states that if introduction of the technology needs changes in infrastructure, costs or savings should be included in the analysis. The committee noted that NICE's guide to the methods of technology appraisal 2013 (section 6.2.14) states that the 'committee will want to be increasingly certain of the cost effectiveness of a technology as the impact of the adoption of the technology on NHS resources increases'. The committee concluded that there would need to be significant investment to use esketamine in the NHS using the company's implementation proposal, which was not fully captured in the analysis. It considered the costs using the company's proposal would underestimate the true cost of implementing esketamine clinics in clinical practice.

## The company's implementation proposal using converted ECT suites as treatment clinics may not be feasible
The company proposed a plan to convert ECT suites to esketamine treatment clinics as part of esketamine's implementation proposal. The NHS commissioning expert explained the current 5year plan for mental healthcare implementation in England. The plan focuses on integrated primary care and community care for people with serious mental illnesses such as treatment-resistant depression. The committee considered it was unclear what treatment setting esketamine would be used in, because:
a psychiatric referral is needed
it is proposed to be delivered in hospitals in converted ECT suites
integrating secondary mental health care with primary and community care is currently challenging, for example, there are long waiting lists.The company considered that esketamine would not take long to become part of NHS practice, quoting market research that showed 82% of mental health trusts have some plans for how they would use esketamine. However, the NHS commissioning experts considered the plan to be impractical because negotiating use of ECT suites may be complex for some trusts and not possible for others. They considered that esketamine's use could not be limited to trusts that have an ECT suite that can be easily converted. The committee was aware of a potential equality issue (see section 3.40), and considered that esketamine's use in a community mental health specialist clinic would enable easier access to treatment. The clinical expert noted that some trusts have large geographical areas and access would not be available for everyone. The NHS commissioning experts also advised that the structure and delivery of services would need to be changed to accommodate esketamine. So, a longer timeframe than NICE mandates for NHS England to comply with the recommendations would be needed to establish esketamine in clinical practice. They explained that significant investment would be needed for esketamine to become part of NHS clinical practice, beyond the costs proposed as part of the company's implementation plan. They noted that esketamine would have to displace other mental health treatments because of its cost. However, this was dependent on the proposal to convert ECT suites. The committee considered the balance between new treatment options and maintaining the ability to offer ECT for people who need it. The committee concluded that, based on NHS England's commissioning experts' feedback, the company's proposal to implement esketamine clinics in ECT suites may not be feasible.

## It is not possible to accurately estimate nursing and monitoring costs without certainty about esketamine's treatment setting
The committee recalled the precautions regarding risk of suicide and close supervision and monitoring in the SPC (see section 3.18). The company proposed administration and nursing costs for esketamine based on its proposal to run esketamine clinics in converted ECT suites. This assumed a ratio of 2 nurses to 6 patients when esketamine is administered, and 1 nurse to 6 patients during monitoring after treatment. The ERG preferred to model a 1 to 1 ratio throughout administration and monitoring because it considered this to be the most plausible in clinical practice. The clinical expert suggested that a ratio of 1 to 1 or 1 to 2 may be necessary when a service first starts administering esketamine, but that the ratio may increase to 1 nurse to a group of patients once the service becomes experienced and established. However, there could be logistical challenges in scheduling administration and monitoring with many people at one time. The patient expert said that building a relationship with a healthcare professional was an important part of treatment and recovery. At the second appraisal committee meeting, the committee concluded that a 1 to 2 ratio of nurses to patients could be appropriate. In response to the second consultation the company estimated a 1 to 2 ratio of nurses to patients in the 3 or more treatments subgroup and a 1 to 1 ratio of nurses to patients in the 3 or more treatments and augmentation subgroup. But the committee noted that its previous conclusion would only apply if using the company's proposed implementation plan to run esketamine clinics in ECT suites. If implemented differently, the administration and monitoring costs would change and it would not be possible to estimate ratios of nurses to patients. So the committee could not conclude on what would be the most plausible costs of nursing and monitoring.

# Cost-effectiveness estimates

## The cost-effectiveness estimates for esketamine in both treatment subgroups are highly uncertain
The company's cost-effectiveness estimate included a patient access scheme discount, the results of which cannot be presented because of confidentiality. The company's revised base case after the fourth committee meeting gave an incremental cost-effectiveness ratio (ICER) range below 20,000 per QALY gained for esketamine plus oral antidepressant compared with oral antidepressant alone in both treatment subgroups. The ERG's ICER range was generally higher than the company's estimates. The committee considered that the company and ERG's estimates were subject to substantial uncertainties, including:
being based on clinical evidence that does not represent the expected use of esketamine in NHS clinical practice (see section 3.4)
how appropriate the comparators are (see section 3.6)
the clinical inputs informing the economic model (see sections 3.13 to 3.17)
the limited generalisability of the trial evidence (see section 3.16)
a cap on relapse and loss of response (see section 3.22)
the uncertainty of long-term outcomes for depression (see section 3.24)
a preference for no excess effect of esketamine on mortality (see section 3.25)
a preference for a range considering no carer disutility and sensitivity analysis with the ERG's method of applying carer disutility (see section 3.27)
uncertainty about when people would stop esketamine (see section 3.29 to 3.30)
the costs of esketamine and potential for repeat or prolonged treatment (see section 3.31)
the healthcare resource use in treatment-resistant depression (see section 3.32)
the substantial costs of adopting esketamine in clinical practice that have not been included in the model, which would bias the results in favour of esketamine (see section 3.33 and section 3.34).NICE's guide to the methods of technology appraisal 2013 notes that judgements about the acceptability of a technology as an effective use of NHS resources will take into account the degree of certainty around the ICERs. The committee will be more cautious about recommending a technology if it is less certain about the ICERs presented. Because of the substantial resolvable uncertainty in all components of the economic modelling and ICER calculation, the committee considered that the benefits and costs of treatment with esketamine were highly uncertain.

# Other factors

## It is not methodologically appropriate to consider the societal burden of depression in this appraisal
The company considered that treatment-resistant depression has a substantial societal burden, mostly because of time off work. The committee noted that NICE's guide to the methods of technology appraisal 2013 (section 5.1.7) states 'the perspective on outcomes should be all direct health effects, whether for patients or other people'. And section 5.1.9 states 'the reference-case perspective on costs is that of the NHS and personal social services'. The committee recalled that considering benefits incurred outside of the NHS and personal social services was not detailed in the remit from the Department of Health and Social Care and the final scope. The committee also noted that productivity costs are not included in the reference case. If a non-reference case has been agreed with the Department of Health and Social Care, productivity costs are not included in this either. The committee concluded it was not appropriate to consider the societal burden of treatment-resistant depression.

## Mental health services need further investment
The company considered that there has historically been significant underinvestment in mental health services. It noted that this context could be considered in the decision for esketamine, because more treatment options and investment in the area could contribute to the parity of esteem between physical and mental health (as required by the Health and Social Care Act 2012). The clinical and commissioning experts agreed that mental health services are stretched, contributing to a shortage in secondary care, meaning many people with treatment-resistant depression are not able to access services. The committee understood the NHS has a responsibility to deliver parity of esteem for physical and mental health, and considered the uncertainties around current mental health service provision in its decision. It acknowledged the funding issues in mental health and the limited new treatment options. The committee also noted that improved access to psychological therapies could benefit people with depression, particularly as patient experts noted regional disparities with access to treatment. However, it recalled that NICE's remit from the Department of Health and Social Care for this technology appraisal was to appraise the clinical and cost effectiveness of esketamine within its licensed indication. The committee concluded that equity of access could not be addressed as part of this appraisal.

## Esketamine is innovative because it has a novel biological mechanism
The company considers esketamine to be innovative because it represents a step change in the treatment of treatment-resistant depression. The company noted esketamine has a novel biological mechanism of action in a disease area that has not had a new mechanism for 30 years. Also, esketamine is sprayed in the nose which means it works rapidly and is non-invasive compared with ECT. The committee considered that the biological mechanism of esketamine could be innovative, but it was uncertain if it would be a step change in treatment because of the uncertainty of the clinical evidence. The committee concluded that it had not been presented with robust evidence of additional benefits not captured in the QALY calculations.

## Equalities
The company, patient organisation and the ERG highlighted that, because esketamine nasal spray needs to be administered and monitored at a clinic, geographical access may be an equalities consideration. The committee considered that symptoms of depression include lack of energy and motivation, so it may be difficult for people to travel a long way to attend esketamine clinics. It considered that administering esketamine in a community setting would be necessary to ensure equity of access to treatment and that conversion of ECT suites would be insufficient to address these equity concerns. Also, the patient expert raised that people with physical health conditions may need additional support when accessing treatment, and the patient organisation noted that some people may have difficulties self-administering treatment or attending a clinic. But because the committee's recommendation does not restrict access to treatment for some people over others, the committee agreed these were not potential equalities issues. The NHS commissioning expert raised concerns about equity of access for people in the criminal justice system. The committee considered that the recommendations do not prevent access to esketamine in the criminal justice system over any other setting. It understood that there were likely to be existing processes in place for managing controlled substances in the criminal justice system, which would not prevent access to esketamine were it recommended. The patient organisation raised that there may be cultural or religious objections to treatment with esketamine. The committee was aware that these objections may also apply for other existing treatments for depression. However, it agreed that this equality issue could not be addressed in a recommendation. The clinical expert noted people who are underserved are more likely, in the clinical expert's experience, to have severe depression. The committee recognised that depression can have a substantial and long-term adverse effect on a person's ability to do normal day-to-day activities. So people with depression may be covered under the disability provision of the Equality Act (2010). The committee was mindful that its role is to appraise treatments and recommend those that are a clinically and cost-effective use of NHS resources. The committee carefully considered the uncertainties common in clinical trials in mental health and recognised the difficulties of collecting clinical data in the population of people with treatment-resistant depression (see section 3.17). It noted that no specific adjustments to the considerations of evidence had been proposed. So it considered all the available evidence including a wide range of views from patient and clinical experts alongside clinical trial data. The committee concluded that it was still unable to recommend esketamine for routine use (see section 3.41).

# Conclusion

## Esketamine is not recommended
The committee considered the burden that treatment-resistant depression has on people, the unmet need for effective treatment options and the innovative nature of how esketamine is administered. The committee acknowledged that obtaining reliable clinical evidence for technologies for depression can be challenging. It noted specifically the challenges of defining the treatment pathway and including psychological therapies, understanding and accounting for placebo effects, and the generalisability of clinical trials in treatments for depression to routine clinical practice. It also noted that large inequities remain in treatments for mental health conditions compared with other disease areas and considered this in its decision making. However, the costs and benefits of esketamine were very uncertain. The committee noted NICE's guide to the methods of technology appraisal 2013 (section 6.3.3) states that 'the committee will be more cautious about recommending a technology when they are less certain about the ICERs presented' and took this into account in its decision making. The committee recalled the clinical uncertainties, summarised in section 3.17, and noted these increased the uncertainty in the clinical inputs in the economic model and that some of these were partially or fully resolvable with further evidence. The committee highlighted that the company could have done more to reduce the uncertainties, including doing longer trials in a population who are more likely to have esketamine in clinical practice. The committee noted that the ICER range was likely to be an underestimate because of the uncertainty in the long-term benefits, stopping treatment, healthcare resource use and implementation costs. On balance, after taking these factors into account, the committee considered that esketamine was not a cost-effective use of NHS resources for the full marketing authorisation. The committee also noted that the ICER range in both subgroups considered could be below or within the range NICE normally considers a cost-effective use of NHS resources. However, not all the issues outlined above were included in these ICER calculations either. These ICERs are highly uncertain and the range would likely underestimate the true cost effectiveness of esketamine. So esketamine could not be considered a cost-effective use of NHS resources for the 3 or more treatments subgroup and 3 or more treatments and augmentation subgroup. So, esketamine is not recommended for use in the NHS for treating treatment-resistant depression.# Recommendations for research
Further research is recommended to help address remaining clinical and cost-effectiveness uncertainties for esketamine, summarised in section 3.36.
Further research is recommended in the wider clinical area of depression on:
how clinical data from regulatory trials in depression could appropriately be used in health technology assessment and decision modelling
the natural history and long-term course of treatment-resistant depression and health-related quality of life in the long-term
characterising the healthcare resource use of people with depression, including exploring which people use services like hospitals and crisis resolution home teams.