QAngio XA 3D QFR and CAAS vFFR imaging software for assessing coronary stenosis during invasive coronary angiography

Evidence-based recommendations on QAngio XA 3D QFR and CAAS vFFR imaging software for assessing coronary stenosis during invasive coronary angiography.

# Recommendations
There is not enough evidence to recommend using QAngio XA 3D quantitative flow ratio (QAngio QFR) and CAAS vessel fractional flow reserve (CAAS vFFR) during invasive coronary angiography to assess coronary stenosis in stable angina. QAngio QFR's diagnostic accuracy is considered acceptable for assessing coronary stenosis during invasive coronary angiography, but its clinical effectiveness is uncertain. CAAS vFFR's diagnostic accuracy and clinical effectiveness is uncertain. Further research is recommended in both diagnostic-only catheter labs and interventional catheter labs.
Further research is recommended (see section 5) on:
people's experiences of QAngio QFR and CAAS vFFR compared with the reference standard of FFR or instantaneous wavefree ratio (iFR)
test failure rates of QAngio QFR and CAAS vFFR in clinical practice and how these affect whether revascularisation is done
the clinical benefit of using QAngio QFR and CAAS vFFR
the diagnostic accuracy of CAAS vFFR.
Why the committee made these recommendations
FFR or iFR can be used with invasive coronary angiography to assess coronary stenosis. However, they can have unpleasant side effects and increase the risk of adverse events, such as damage to the artery.
CAAS vFFR and QAngio QFR use Xray images taken during an invasive coronary angiography to construct a 3D image of the artery. This image is used to estimate the effect of coronary stenosis on blood flow through the artery without the side effects and risk of adverse events of FFR or iFR.
Published evidence shows that the diagnostic accuracy of QAngio QFR is similar to FFR, but the diagnostic accuracy of CAAS vFFR is very uncertain. Whether QAngio QFR or CAAS vFFR affect clinical outcomes and improve quality of life is also uncertain. Also, in clinical practice the quality of the images varies depending on if they are done in a diagnosticonly centre or one that offers interventional procedures. Poor image quality might mean the tests fail.
The costeffectiveness estimates for CAAS vFFR and QAngio QFR are uncertain but suggest that they are more cost effective than invasive coronary angiography alone. The estimates suggest that, compared with FFR and iFR, CAAS vFFR is less cost effective and QAngio QFR is slightly cheaper but less clinically effective.
There are multiple tests in use that assess coronary stenosis and it is not clear what clinical benefits QAngio QFR and CAAS vFFR offer over these. Therefore, QAngio QFR and CAAS vFFR are not recommended for use in the NHS, and further research is recommended.# The diagnostic tests

# Clinical need and practice
Angina is chest pain caused by insufficient blood supply to the heart (myocardial ischaemia). Stable angina is brought on by physical activity or emotional stress and goes away with rest. It is the key symptom of coronary artery disease, which is one of the main causes of morbidity and mortality in economically developed countries.
Options for managing stable angina include lifestyle advice, drug treatment and revascularisation using percutaneous (stent placement during percutaneous coronary intervention) or surgical techniques (such as coronary artery bypass surgery). Choosing the appropriate management option relies on correctly detecting and characterising coronary stenosis. Therefore, the diagnostic pathway for stable angina:
confirms a diagnosis of stable angina
defines the severity of coronary stenosis, which provides prognostic information and identifies people who are likely to benefit from myocardial revascularisation, in addition to optimal medical therapy.
The NICE guideline on assessment and diagnosis of chest pain of recent onset recommends diagnostic testing for people in whom stable angina cannot be excluded by clinical assessment alone. It recommends offering 64slice (or above) CT coronary angiography as the first-line diagnostic test when:
clinical assessment indicates typical or atypical angina or
clinical assessment indicates non-anginal chest pain but 12-lead resting ECG has been done and indicates ST-T changes or Q waves.
For people in whom 64-slice (or above) CT coronary angiography has shown coronary artery disease of uncertain functional significance, or is non-diagnostic, the guideline recommends offering non-invasive functional imaging for myocardial ischaemia. This could be:
myocardial perfusion scintigraphy with single-photon emission CT (MPS with SPECT) or
stress echocardiography or
first-pass contrast-enhanced magnetic resonance (MR) perfusion or
MR imaging for stress-induced wall motion abnormalities.
If the results of non-invasive functional imaging are inconclusive, invasive coronary angiography is recommended. Invasive coronary angiography shows whether the arteries are blocked or narrowed, and the degree of stenosis. It is usually used as a third-line investigation for stable angina or during the initial stages of percutaneous coronary intervention. However, it is difficult to differentiate between functionally significant and non-significant (not substantially affecting blood supply) coronary stenosis using visual assessment of invasive coronary angiograms.
If it is necessary to more accurately understand the functional significance of a stenosis, fractional flow reserve (FFR) or instantaneous wave-free ratio (iFR) measurements can be done during invasive coronary angiography. These invasive techniques use a pressure wire with or without a vasodilator drug, such as adenosine, and can only be done in interventional catheter laboratories.
QAngio XA 3D quantitative flow ratio (QAngio QFR) and CAAS vessel FFR (CAAS vFFR) are analytical software that can be used during invasive coronary angiography to assess the functional significance of coronary stenosis. By avoiding unnecessary invasive measurement of FFR or iFR, these technologies could help avoid the risks associated with passing the pressure wire to the coronary arteries, and with adenosine infusion.

# The interventions
Both tests included in the assessment are CE marked and available to the NHS.

## CAAS vFFR
The CAAS vFFR software (Pie Medical Imaging) works by building a 3D reconstruction of a coronary artery as well as assessing the pressure drop across the stenosis and calculating a vFFR value. Therefore, it gives both anatomical and functional assessments of the stenosis. It uses 2 standard X-ray angiograms, and is compatible with most X-ray systems (that is, it is vendor independent). The company claims that the total analysis time is about 2 minutes per coronary artery. Thresholds for interpretation of vFFR are not provided in the instructions for use document.

## QAngio XA 3D QFR
The QAngio software (Medis Medical Imaging) uses X-ray angiographic images taken during invasive coronary angiography. Two images are needed, which have to be taken with at least 25 degrees difference in viewing angle and with a frame speed of at least 12.5 frames per second. High image quality is crucial for appropriate results. The QAngio software creates a 3D anatomical model of a coronary artery from these 2 images, and then estimates QFR from the 3D vessel anatomy and flow velocity. The company claims that the total analysis time is about 4 to 5 minutes per coronary artery. The analysis time may decrease with routine use of the software. The QFR represents an assessment of the pressure drop over the artery, with a value of 1 representing a normally functioning artery with no pressure drop. A 20% or more drop in blood pressure (QFR value of 0.80 or less) is usually considered a significant obstruction, where revascularisation should be considered.
The QAngio software offers 2 different flow models to calculate QFR:
fixed-flow QFR (fQFR), using fixed-flow velocity and
contrast QFR (cQFR), using contrast frame count in an angiogram without hyperaemia.Fixed-flow QFR is faster to compute, but may be less accurate than contrast QFR.

# The comparator
The comparator is clinical decision making based on the visual interpretation of the images from invasive coronary angiography, alongside clinical judgement. The reference standard for assessing diagnostic accuracy is FFR or iFR.# Evidence
The diagnostics advisory committee considered evidence on QAngio XA 3D quantitative flow ratio (QAngio QFR) and CAAS vessel fractional flow reserve (CAAS vFFR) for assessing coronary stenosis during invasive coronary angiography from several sources. Full details of all the evidence are in the committee papers.

# Clinical effectiveness
The external assessment group (EAG) identified 41 unique studies that met the selection criteria for inclusion in the review. Of the included studies, 39 evaluated QAngio QFR, 3 evaluated CAAS vFFR and only 1 study directly compared QAngio QFR with CAAS vFFR. There were 2 studies that did not report diagnostic accuracy data but included other eligible outcomes. Seventeen of the studies were conference abstracts only, 15 of which were included in the diagnostic accuracy review.
Fifteen of the studies were done in multiple centres. Most studies were done in Asia, including 33 with sites in Japan, 5 in China, 4 in South Korea and 1 site in Singapore. A total of 22 studies had sites in Europe, 3 of which were in the UK. Two of the studies had sites in the US and 2 separate single studies had sites in Brazil and Australia.
Of the 22 QAngio QFR studies, 11 were at low risk of bias. The main source of bias was related to patient selection. The EAG also noted concerns that a high number of studies had been done retrospectively (offline use of QAngio QFR) rather than as part of invasive coronary angiography and before FFR.
Of the CAAS vFFR studies, all did CAAS vFFR analyses retrospectively (offline), and 2 were done at a single centre. Only the ILUMIEN I study had a full text manuscript. This study was considered at high risk of selection bias because of the large percentage of lesions excluded.

## Diagnostic test accuracy
Of the 4 studies reporting the diagnostic accuracy of CAAS vFFR only 1 (ILUMIEN I) reported a 2 x 2 table of diagnostic accuracy, and only 1 presented a BlandAltman plot (FAST; Masdjedi et al. 2019) from which data were extracted to calculate diagnostic accuracy. Two of the studies were conference abstracts and only reported sensitivity and specificity without confidence intervals (Jin et al. 2019 and FAST EXTEND). One of these studies used an acquisition speed of 7.5 frames per second rather than the 12.5 frames per second recommended in the instructions for use (Jin et al. 2019). There was notable heterogeneity across this small number of studies. The FAST EXTEND study was used in the base-case cost-effectiveness analysis. The ILUMIEN I and Jin et al. (2019) studies were not included in the base-case cost-effectiveness analysis. Instead, they were included in separate scenario analyses to test the sensitivity of the cost-effectiveness results.
The EAG noted that the meta-analyses of the CAAS vFFR studies should be interpreted with caution because imputation of data (replacing missing data with substituted values) was needed. This was for 2 studies on the prevalence of FFR results below and above the cut-off for revascularisation decisions (0.80 or less), and because of the high heterogeneity across studies. The results of these bivariate meta-analyses are summarised in table 1.
Analysis
Sensitivity
% confidence intervals
Specificity
% confidence intervals
Using FAST
(Masdjedi et al. 2019)
to 83.22
to 88.89
Using FAST EXTEND
to 95.11
to 86.95
Only 1 study, reported as a conference abstract, directly compared CAAS vFFR with QAngio QFR. It concluded that diagnostic performance of CAAS vFFR was poorer than for QAngio QFR, with area under the curves of 0.719 (95% confidence interval  0.621 to 0.804) for CAAS vFFR and 0.886 (95% CI 0.807 to 0.940) for contrast QFR (cQFR).
The EAG did a meta-analysis of the included studies, focusing on the diagnostic accuracy of QAngio QFR to detect lesions or vessels needing intervention (defined as having an FFR of 0.80 or less). Two approaches were used. The primary analysis consisted of a meta-analysis of reported diagnostic accuracy data. The secondary analysis used a data extraction approach in which FFR and QAngio QFR values from published plots were extracted and used to calculate diagnostic accuracy. This second approach allowed for a wider range of analyses.
The EAG identified 26 studies with sufficient diagnostic accuracy data to be included in the primary meta-analysis. Both univariate and bivariate meta-analyses of sensitivity and specificity were done and compared. These were divided into 3 modes of QAngio QFR: fixed-flow QFR (fQFR), contrast QFR (cQFR) and studies in which the type of QAngio QFR was not specified. Most studies included in the primary analysis used FFR as the reference standard, using a cut-off of 0.80, although 1 study used instantaneous wavefree ratio (iFR) as the reference standard. The EAG noted that there was no conclusive evidence of a significant difference between cQFR and fQFR.
In the univariate meta-analysis for the random-effect analysis, QAngio QFR at a cut-off of 0.80 had good diagnostic accuracy to predict FFR (also at a cut-off of 0.80). cQFR had a sensitivity of 85% (95% CI 78% to 90%) and specificity of 91% (95% CI 85% to 95%); fQFR had a sensitivity of 82% (95% CI 68% to 91%) and specificity of 89% (95% CI 77% to 95%). Studies that did not specify the mode of QAngio QFR had a sensitivity of 84% (95% CI 78% to 89%) and specificity of 89% (95% CI 87% to 91%).
Summary positive predictive values were 77% (95% CI 69% to 83%) for fQFR, 85% (95% CI 80% to 89%) for cQFR and 80% (95% CI 76% to 84%) for non-specified QAngio QFR (see figure 27 in the appendix of the diagnostics assessment report). Summary negative predictive values were 92% (95% CI 89% to 94%) for fQFR, 91% (95% CI 85% to 94%) for cQFR and 91% (95% CI 87% to 93%) for non-specified QAngio QFR.
The results of the bivariate meta-analysis were almost identical to the univariate analyses, with no conclusive evidence of a significant difference between fQFR and cQFR. The results of this analysis are summarised in table 2.
Mode
Sensitivity
% confidence intervals
Specificity
% confidence intervals
cQFR
to 89.48
to 95.24
fQFR
to 90.66
to 95.38
Non-specified QFR
to 88.68
to 90.61
cQFR or
non-specified QFR
to 87.85
to 92.45
Abbreviations: QFR, quantitative flow ratio; cQFR, contrast QFR; fQFR, fixed-flow QFR.
The mean difference between QAngio QFR and FFR was almost exactly zero for all 3 modes of QAngio QFR testing. For fQFR the mean difference was 0 (95% CI -0.05 to 0.06), for cQFR the mean difference was -0.01 (95% CI -0.06 to 0.04) and for non-specified QAngio QFR the mean difference was 0.01 (95% CI -0.03 to 0.05). FFR and QAngio QFR were highly correlated in all studies, with correlation coefficients of 0.78 (95% CI 0.72 to 0.82) for fQFR, 0.78 (95% CI 0.70 to 0.85) for cQFR and 0.79 (95% CI 0.73 to 0.83) for non-specified QAngio QFR.
The secondary analysis allowed for a wider range of analyses, such as considering different QAngio QFR and FFR cut-offs, and the effect of using a grey zone, in which people with intermediate QAngio QFR values go on to have confirmatory FFR.
A bivariate meta-analysis of diagnostic accuracy using data extracted from figures gave summary estimates for sensitivity and specificity of 84.6% (95% CI 80.7% to 87.8%) and 87.2% (95% CI 83.4% to 90.3%), respectively. This was similar to the results from the primary analysis when cQFR and non-specified QFR were combined.
QFR, as measured by QAngio, was highly correlated with FFR (r=0.80). In 50% of people, QFR and FFR differed by no more than 0.04. In 95% of people, values differed by no more than 0.14.
In the grey-zone analysis:
If QAngio QFR is more than 0.84: continue without stenting or bypass and defer FFR (test negative).
If QAngio QFR is 0.78 or less: proceed directly to stenting or bypass without FFR (test positive).
If QAngio QFR is between 0.78 and 0.84: do an FFR and proceed based on that result (at 0.80 cut-off).
This strategy increased diagnostic accuracy compared with using QAngio QFR alone. The sensitivity was 93.1% (95% CI 90.1% to 94.9%) and the specificity was 92.1% (95% CI 88.3% to 94.5%). A total of 20.1% of people were in the grey zone and would have confirmatory FFR. However, only 30.4% of people with QAngio QFR results in the grey zone had results that differed from their FFR.
The EAG identified 5 studies included in the meta-analysis that also reported 2 x 2 table data on the diagnostic accuracy of using 2D or 3D invasive coronary angiography alone. These studies used 50% diameter stenosis as the cut-off and FFR of 0.80 or less as the reference standard. Given the small number of studies, and because 2D and 3D invasive coronary angiography may have very different performance, no bivariate meta-analysis of these data was done. However, the results of the individual studies showed that the diagnostic accuracy of invasive coronary angiography was inferior to QAngio QFR.
To inform the economic analysis, the EAG did an additional pragmatic search for studies that compared 2D invasive coronary angiography with FFR assessment. Data extracted from these studies showed that compared with QAngio QFR, the correlation of 2D invasive coronary angiography with FFR was much weaker (correlation coefficient -0.432). A bivariate meta-analysis of these extracted data produced summary sensitivity and specificity estimates of 62.6% (95% CI 51.5% to 72.5%) and 61.6% (95% CI 53.1% to 69.4%), respectively.

## Other intermediate outcomes
The most reported (15 studies) causes of exclusion were issues with image acquisition and quality (for example, lack of at least 2 projections with a 25 degree angle in between, or poor image quality). The second most reported reason for exclusion was anatomical features of arteries (for example, excessive overlapping or foreshortening, ostial lesions, severe tortuosity).
Exclusion rates for QAngio QFR were higher overall in retrospective studies (median 28%, range 6% to 92%) compared with prospective studies (median 17%, range 7% to 52%). This may be partly explained by the fact that invasive coronary angiography images in retrospective studies were less likely to have been collected following manufacturer instructions.
There were only 2 retrospective CAAS vFFR studies that reported exclusion rates, and these were both high at 63% and 65%. In both studies most exclusions were because of angiographic image processing issues such as lack of suitable projections or poor image quality (rather than directly because of CAAS vFFR).
There were 8 studies that reported outcomes data on reproducibility of QAngio QFR readings between 2 different analysts (inter-observer variability). QAngio QFR was found to have a moderate to high level of inter-observer reliability. In 2 studies, CAAS vFFR was also found to have a high level of inter-observer reliability.
There were 8 retrospective studies that reported outcomes data on intra-observer reproducibility of QAngio QFR readings. The time gap between initial and repeated measurements was reported in 4 studies and ranged from 3 days to 2 weeks. Most studies reported a high level of intra-observer reliability for QAngio QFR. One study evaluated both QAngio QFR and CAAS vFFR and found high levels of repeatability and no statistically significant changes between repeated tests.
There were 6 studies of QAngio QFR that reported the time needed to complete QFR analysis. Time to QFR data acquisition ranged from an average of 2 minutes and 7 seconds to 10 minutes (standard deviation 3 minutes). One study of 268 patients reported that time to image acquisition significantly decreased with the number of invasive coronary angiographies analysed, from 5 minutes and 59 seconds to 2 minutes and 7 seconds between the first and last 50 patients.
There were 3 cohort studies that reported mortality or major clinical outcomes in eligible patients with QAngio QFR measurements. All found that a clinically significant QAngio QFR predicted a higher incidence of long-term major cardiovascular adverse events. No data were reported for CAAS vFFR.
Five studies included in the diagnostic accuracy review retrospectively derived a grey-zone strategy based on their diagnostic accuracy results to model a potential reduction in adenosine and FFR use. These results are summarised in table 3.
Study
Grey zone
Diagnostic accuracy of grey-zone strategy (QFR compared with FFR)
Percentage of adenosine or FFR procedures avoided
FAVOR II Europe-Japan Westra (2018)
to 0.86
Sensitivity and specificity more than 95%
Kanno (2019) (A) (conference abstract)
to 0.84
Positive predictive value and negative predictive value more than 90%
Mejia-Renteria (2019)
to 0.84
More than 95% agreement
Smit (2019)
to 0.86
Sensitivity: 95%, specificity: 92.5%
WIFI II
to 0.87
Sensitivity and specificity more than 90%
WIFI II
to 0.90
Sensitivity and specificity more than 95%
Abbreviations: FFR, fractional flow reserve; QFR, quantitative flow ratio.
Because of the lack of published data on QAngio QFR's clinical effectiveness, the EAG did a simulation study to investigate its possible effect on coronary outcomes compared with FFR.
The sample population was taken from data extracted from published BlandAltman figures. Only cQFR or non-specified QAngio QFR data were used, for 3,193 people, each with an FFR measurement and its associated QAngio QFR measurement. To predict coronary outcomes, the results of the recent IRIS-FFR registry report were used. This represented 5,846 people who either had revascularisation (stent or bypass surgery) or continued with current management without surgery based on their measured FFR result. The IRISFFR study used major adverse cardiovascular events as its primary outcome.
Three strategies for deciding whether to revascularise were investigated:
FFR only: do FFR for all and revascularise if FFR is 0.80 or less.
QAngio QFR only: do QAngio QFR for all and revascularise if QAngio QFR is 0.80 or less, without measuring FFR.
Grey zone: do QAngio QFR for all and:
revascularise if QAngio QFR is 0.78 or less
defer if QAngio QFR is more than 0.84
if QAngio QFR is between 0.78 and 0.84, do FFR and revascularise if FFR is 0.80 or less.
If using the FFR only strategy 40.2% of people would have revascularisation. Using the QAngio QFR only strategy 42.0% would have revascularisation, and using the grey-zone strategy 43.2% would have revascularisation. Using QAngio QFR therefore moderately increased the revascularisation rate, and using it with a grey zone increased it further.
These simulations suggest that using FFR may prevent slightly more major adverse cardiovascular events, at around 1 event per 1,000 people, but the overlap in simulated distributions means it is highly uncertain whether the difference is genuine. By contrast, the simulation suggests that QAngio QFR increases the number of revascularisations done, without substantially improving the number of major adverse cardiovascular events prevented. Overall these simulations suggested that there was little conclusive clinical difference between using QAngio QFR and FFR to make revascularisation decisions.

# Cost effectiveness

## Systematic review of cost-effectiveness evidence
The EAG did a search to identify studies investigating the cost effectiveness of using QAngio QFR and CAAS vFFR imaging software to assess the functional significance of coronary stenosis during invasive coronary angiography. No studies were found so a review of published cost-effectiveness studies evaluating invasive coronary angiography (alone or with FFR) in managing coronary artery disease was done. The EAG identified 21 relevant studies and of these, 2 models (Walker et al. 2011 and Genders et al. 2015) were good examples of alternative ways to evaluate diagnostic strategies in patients with suspected stable angina.
For the economic analysis, the following 5 diagnostic strategies were considered:
invasive coronary angiography alone (strategy 1)
invasive coronary angiography followed by confirmatory FFR or instantaneous wavefree ratio (iFR; reference standard, strategy 2)
invasive coronary angiography with QAngio QFR (strategy 3)
invasive coronary angiography with QAngio QFR, followed by confirmatory FFR or iFR if QFR is inconclusive (strategy 4)
invasive coronary angiography with CAAS vFFR (strategy 5).

## Economic model
The EAG developed a de novo economic model. It was designed to estimate the cost effectiveness of using QAngio QFR and CAAS vFFR during invasive coronary angiography to assess the functional significance of coronary stenosis in people with stable angina whose angiograms showed intermediate stenosis. The model had 2 parts, a diagnostic model and a prognostic model. The diagnostic model was used to link the diagnostic accuracy of QAngio QFR and CAAS vFFR to short-term costs and consequences relating to decisions about revascularisation. The prognostic model took the diagnostic outcomes and modelled the risk of longer-term events, such as myocardial infarction, sudden cardiac death and the need for urgent or unplanned revascularisation.
The population consisted of people with stable coronary artery disease whose invasive coronary angiograms showed intermediate stenosis. The age and sex distribution of the population was derived from the IRIS-FFR registry (mean age of 64 years and 72% men).
The prevalence of functionally significant stenosis in the population was based on studies that reported values of FFR and cQFR or non-specified QFR. It was assumed that the population in these QAngio QFR studies reflected the UK population. This suggested a prior likelihood of functionally significant stenosis of 40.2%, based on the proportion of people in the studies who had an FFR measurement of 0.80 or less.
The proportion of positive or negative test results when using the QAngio QFR, CAAS vFFR or invasive coronary angiography (strategies 3, 5 and 1) was based on the estimated accuracy of the 3 tests. The diagnostic accuracy estimates for these 3 tests are shown in table 4.
Test
Strategy
Analysis
Sensitivity
Specificity
Source
QAngio QFR
Base case
Bivariate meta-analysis for combined cQFR and non-specified QFR mode
QAngio QFR
Scenario
Bivariate meta-analysis for cQFR mode
QAngio QFR
Scenario
Bivariate meta-analysis for fQFR mode
CAAS vFFR
Base case
FAST EXTEND (2019)
CAAS vFFR
Scenario
ILUMIEN I (2019)
CAAS vFFR
Scenario
Jin et al. (2019)
ICA
Base case
Bivariate meta-analysis of 6 studies
ICA
Scenario
Danad et al. (2017) per vessel analysis
Abbreviations: ICA, invasive coronary angiography; QFR, quantitative flow ratio; cQFR, contrast QFR; fQFR, fixed-flow QFR; vFFR, vessel fractional flow reserve.
The diagnostic accuracy of QAngio QFR in strategy 4 was based on the joint distribution of QFR and FFR measurements in the extracted individual-level patient data. The probabilities of QAngio QFR test results being positive (QFR less than 0.78), negative (QFR more than 0.84) or inconclusive (QFR of 0.78 to 0.84) are shown in table 5.
QAngio QFR test result
Probability
Functionally significant stenosis (FFR 0.80 or less)
Non-significant stenosis (FFR 0.80 or more)
Positive
QFR less than 0.78
Inconclusive (grey zone)
QFR 0.78 or more to 0.84 or less
Negative
QFR more than 0.84
Abbreviations: FFR, fractional flow reserve; QFR, quantitative flow ratio.
The rates of FFR and iFR procedural complications applied in the base-case analysis are summarised in table 6.
Serious procedural complication
Rate
Source
Coronary dissection
IRIS-FFR registry
Venous occlusion
IRIS-FFR registry
Ventricular arrhythmia
IRIS-FFR registry
Conduction disturbance needing treatment
IRIS-FFR registry
Bronchospasm
IRIS-FFR registry
Thrombus formation
IRIS-FFR registry
Death
Fearon et al. (2003)
The rate of procedural deaths associated with revascularisation was sourced from UK audit data, which gives a 0.99% death risk for non-emergency coronary artery bypass graft and 0.17% for percutaneous coronary intervention. The mortality rate associated with revascularisation was estimated as a weighted average of the mortality rates for percutaneous coronary intervention and coronary artery bypass graft. This was relative to the proportion of percutaneous coronary interventions and coronary artery bypass graft procedures. In the base case, 87% of revascularisation procedures were assumed to be percutaneous coronary intervention, and 13% were assumed to be coronary artery bypass graft.
The reported 1year and long-term (up to 3 years) cumulative incidence of major adverse cardiovascular events in the IRISFFR registry for deferred lesions was used in the model to estimate the baseline risk of major adverse cardiovascular events for the first year and subsequent years. The baseline risk of major adverse cardiovascular events used in the model for people in the group with the highest FFR values (0.91 or more) was 0.64% in the first year and 0.32% per year in subsequent years. The hazard ratios were 1.06 (95% CI 0.99 to 1.13), 1.09 (95% CI 1.05 to 1.14), 1.07 (95% CI 1.06 to 1.09) per 0.01 decrease in FFR for cardiac death, myocardial infarction, and unplanned or urgent revascularisation, respectively.
The treatment effect of revascularisation on major adverse cardiovascular events in people with stable coronary artery disease is highly uncertain. The ISCHEMIA trial, a randomised, parallel, open-label clinical trial comparing revascularisation with optimal medical therapy, did not find evidence that revascularisation reduced the risk of major adverse cardiovascular events. Therefore, in the base-case analysis, the diagnostic tests did not benefit major adverse cardiovascular events outcomes. Scenario analyses were done to explore the effect of this assumption.
By identifying the appropriateness for revascularisation, the tests can have health benefits through greater symptom relief and, therefore, higher health-related quality of life (HRQoL). Because the base-case analysis assumed that there was no treatment effect of revascularisation on major adverse cardiovascular events, the improvement in symptom relief was the only benefit. The HRQoL effects of revascularisation were based on the FAME trials. Both were randomised, parallel, open-label clinical trials. FAME I compared invasive coronary angiography with FFR for guiding percutaneous coronary interventions in patients with multivessel coronary artery disease. FAME II compared clinical outcomes, safety and cost effectiveness of FFR-guided percutaneous coronary intervention with optimal medical treatment alone in patients with stable coronary artery disease. These trials showed that HRQoL improved significantly from baseline after percutaneous coronary intervention.
In the diagnostic model a one-off procedural disutility was applied for people having invasive FFR or iFR and for those who had revascularisation. In the prognostic model, a one-off utility decrement was also applied for people who had a non-fatal myocardial infarction or needed an unplanned revascularisation. A separate utility decrement was applied to the post-myocardial infarction health state, to reflect a decrease in HRQoL for those with a history of myocardial infarction.
The base-case analysis made an assumption that the quality-adjusted life year (QALY) loss applied for FFR or iFR was representative of both types of pressure wire procedures. The QALY loss estimates associated with each procedure in the diagnostic model are summarised in table 7.
Procedure
Mean QALY loss (95% confidence interval)
Source
ICA
Assumed to cancel across strategies
FFR/iFR
(0.0051 to 0.0062)
Assumed the same as for PCI (in the absence of any other source)
PCI
(0.0051 to 0.0062)
Bagust et al. (2006)
CABG
(0.031 to 0.035)
Bagust et al. (2006)
Abbreviations: ICA, invasive coronary angiography; FFR, fractional flow reserve; iFR, instantaneous wave-free ratio; PCI, percutaneous coronary intervention; CABG, coronary artery bypass graft; QALY, quality-adjusted life year.
The base-case cost of QAngio QFR with a throughput of 200 people per year was 430.61 per person tested. This was based on the purchase of vouchers for 100 people, which covered the cost of the software licence and the training and certification of up to 4 QAngio QFR users, in addition to a staff cost per person tested of 7.76. An update to the QAngio QFR price structure was submitted during consultation. Using the base-case throughput of 200 people per year, the new voucher price reduced the cost to 362.94 per person tested. An alternative annual licence option reduced this further to 223.50 per person tested. The base-case cost of CAAS vFFR with a throughput of 200 people per year was 172.18 per person tested. This included staff training and annual maintenance and was based on the purchase of a perpetual licence, which allows analysis of as many people as needed per year. The model did not consider a cost for invasive coronary angiography because all people who entered the diagnostic model had this test.
The unit cost for FFR and iFR was estimated as the difference between the activity weighted average of the healthcare resource group codes for complex and standard cardiac catheterisation (436.80).
The following assumptions were applied in the base-case analysis:
A diagnostic threshold of 0.80 was used to define functionally significant stenosis for QAngio QFR and FFR.
A grey-zone boundary of 0.78 to 0.84 for QAngio QFR was used as suggested by the manufacturer of QAngio QFR.
The baseline risk of major adverse cardiovascular events in the absence of revascularisation depends on disease severity as measured by FFR, while the distribution of FFR values differs by diagnostic strategy.
There is no treatment effect of revascularisation on risk of major adverse cardiovascular events, based on the findings of the ISCHEMIA trial.
Costs of QAngio QFR and CAAS vFFR were based on an average annual throughput of 200 people.
The base case assumed all diagnostic procedures took place in an interventional setting. The diagnostic-only setting was considered in scenario analyses.
HRQoL benefits of revascularisation and optimal medical therapy observed at 1 year for the true positive and false negative health states applied for a lifetime duration.
Procedural disutility associated with FFR was equivalent to that of percutaneous coronary intervention.
The deterministic and probabilistic cost-effectiveness results for the base-case analysis, expressed in terms of net health benefit at a maximum acceptable incremental cost-effectiveness ratio (ICER) of 20,000 per QALY gained, are shown in tables 8 and 9, respectively. The incremental net health benefit was calculated for each strategy compared with invasive coronary angiography alone. The results were consistent for both the deterministic and probabilistic analysis.
Strategy
Identification
Total QALYs
Total costs
NHB
INHB
NHB rank
ICA alone
ICA with FFR
ICA with QAngio QFR
ICA with QAngio QFR and confirmatory FFR (grey zone)
ICA with CAAS vFFR
NHB and INHB are measured at a maximum acceptable ICER of 20,000 per QALY gained. Incremental NHB is relative to ICA alone. Abbreviations: ICA, invasive coronary angiography; FFR, fractional flow reserve; QFR, quantitative flow ratio; vFFR, vessel FFR; QALY, quality-adjusted life year; NHB, net health benefit; INHB, incremental NHB.
Strategy
Identification
Total QALYs
Total costs
NHB
INHB
NHB rank
Probability cost effective at 20,000 per QALY gained
ICA alone
ICA with FFR
ICA with QAngio QFR
ICA with QAngio QFR and confirmatory FFR (grey zone)
ICA with CAAS vFFR
NHB and INHB are measured at a maximum acceptable ICER of 20,000 per QALY gained. Incremental NHB is relative to ICA alone. Abbreviations: ICA, invasive coronary angiography; FFR, fractional flow reserve; QFR, quantitative flow ratio; vFFR, vessel FFR; QALY, quality-adjusted life year; NHB, net health benefit; INHB, incremental NHB.
Strategy 2 (invasive coronary angiography with FFR) had the highest net health benefit and the highest probability of being cost effective, although the differences between all the strategies were small. Strategy 1 (invasive coronary angiography alone) was the cheapest and had the lowest QALY gain, while strategy 5 (invasive coronary angiography with vFFR) was the most expensive and had the highest QALY gain.
Results from the scenario analyses showed that the base-case results were generally robust when alterations were made to the sources of data used in the model and when different assumptions were made. However, sometimes these alterations resulted in significant changes to the net health benefit rankings of the different strategies.
In the base case, the diagnostic accuracy estimates for vFFR were based on the FAST EXTEND study (sensitivity 97.0% and specificity 74.0%), the largest study of vFFR (330 patients). Using accuracy estimates from ILUMIEN I reduced the cost effectiveness of vFFR, but estimates from Jin et al. (2019) increased it. This resulted in vFFR being the second most cost-effective strategy. This highlighted the substantial uncertainty surrounding the cost effectiveness of vFFR in strategy 5.
When QAngio QFR was considered to have the same diagnostic accuracy as FFR (that is, 100% sensitivity and specificity), the total QALYs and costs for strategy 3 increased by 0.017 QALYs and 6 per person from the base-case scenario. In this scenario strategy 3 became cost effective with the highest net health benefit, largely because of greater total QALYs gained for strategy 3 compared with strategy 2. This difference was mainly because of the procedural disutility associated with FFR or iFR.
When the procedural disutility of FFR was more than that used in the base case, the net health benefit of strategies 2 and 4 were affected most. The total QALYs for both strategies were reduced, resulting in strategy 2 becoming the second least cost effective and strategy 3 the most cost effective. An FFR disutility of 0.014 QALYs resulted in an equal net health benefit for strategies 2 and 3. This procedural disutility was 2.5 times greater than that associated with percutaneous coronary intervention, but less than half the disutility associated with coronary artery bypass graft.
In terms of how duration of HRQoL affected cost effectiveness, the benefits need to last for at least 7 years to offset the disutility associated with FFR or iFR in the base case for strategy 2 to remain more cost effective than strategy 3.
The benefits of revascularisation, in terms of improved HRQoL, suggested that the sensitivity of test results was a more important driver of cost effectiveness than specificity. This was because true positive test results translated into higher QALY gains than mismanagement of false negative test results.
In a diagnostic-only setting, the large additional costs of repeating diagnostic catheterisation at a subsequent appointment in an interventional centre for strategies involving measuring FFR or iFR (strategies 2 and 4) meant that strategies without this testing component were more cost effective. Strategy 3 (QAngio QFR alone) became the strategy with the highest net benefit, followed by strategy 5 (CAAS vFFR alone).# Committee discussion

# Clinical need

## FFR and iFR are not frequently used so QAngio QFR and CAAS vFFR may help with decision making during invasive coronary angiography
Clinical experts explained that in general, physiological testing using fractional flow reserve (FFR) and instantaneous wave-free ratio (iFR) is available but not frequently used in the UK. People typically have an invasive coronary angiography after a previous functional test (see section 2.4). Sometimes decisions about revascularisation are based on the images from the invasive coronary angiography, results of the previous tests and patient history. If the revascularisation decision is still uncertain after invasive coronary angiography, people may be referred for FFR or iFR. Using QAngio XA 3D quantitative flow ratio (QAngio QFR) and CAAS vessel FFR (CAAS vFFR) during invasive coronary angiography may provide more information to help with decision making. It could also mean that in some cases clinical decisions could be made without needing FFR.

## Less invasive tests may benefit patients and carers by reducing anxiety, unpleasant side effects and risk of complications
A patient expert explained the potential benefits of testing using QAngio QFR or CAAS vFFR. These included reduced anxiety, discomfort and distress than more invasive testing, which may be needed if a definitive treatment decision cannot be made during the initial invasive coronary angiography. Using an invasive test like FFR with a pressure wire means using an adenosine infusion. The committee noted that around 30% of people may experience chest pain and shortness of breath from this. These side effects usually pass quickly but can be distressing. Around 3% of people may experience discomfort from the pressure wire itself and there is a small risk of rupture of the blood vessel. By avoiding adenosine infusion and a pressure wire, QAngio QFR and CAAS vFFR could reduce unpleasant side effects and risk of complications.

# Clinical effectiveness

## The diagnostic accuracy evidence for CAAS vFFR is highly uncertain
The committee noted that there were only 3 studies using CAAS vFFR that matched the inclusion criteria for review in the diagnostics assessment report. These included 500 patients. The external assessment group (EAG) explained that there was notable heterogeneity across this small number of studies and that the meta-analyses of the CAAS vFFR studies should be interpreted with caution. Where reported, there was a high exclusion rate because of angiographic image processing issues. In 2 of the studies, the technology was not used in the way it was intended (ILUMIEN I and Jin et al. 2019). The committee concluded that the diagnostic accuracy of CAAS vFFR was highly uncertain and recommended further research (see research recommendation 5.4).

## The diagnostic accuracy of QAngio QFR appears to be similar to FFR
The committee noted that 39 studies using QAngio QFR matched the inclusion criteria for the review in the diagnostics assessment report. These included 5,440 patients. These studies showed that QAngio QFR had good diagnostic accuracy to predict the FFR result. The clinical experts explained that there was good agreement between QFR and FFR values particularly at the extremes of measurement. While there was some disagreement between QFR and FFR results within the grey zone, (in the range of 0.78 to 0.84), the clinical experts noted that there is likely to be a limited clinical effect of not identifying someone with an FFR of between 0.76 and 0.80, that is, a false negative result. A more significant effect could occur for people with an FFR result of less than 0.76 who have a negative result on QAngio QFR (0.80 or higher). The EAG noted that modelling suggested around 3% of people with an FFR result of less than 0.76 would be misdiagnosed if using QFR for functional imaging. Clinical experts also explained that with FFR values close to the 0.80 cut-off, it is unclear whether there is any added benefit of revascularisation compared to optimal medical therapy. The committee concluded that there was good agreement between QAngio QFR and FFR values. Although there was some uncertainty around the grey zone, this was not a particular concern.

## Technical failure rates in diagnostic-only centres may be higher because of lower quality angiography images
In the UK, invasive coronary angiography is usually done in diagnostic-only catheter laboratories or in interventional catheter laboratories that can also do percutaneous coronary intervention in the same procedure. Clinical experts explained that the quality of angiography images from diagnostic-only centres was generally lower than those from interventional centres. This is because in the diagnostic centre, invasive coronary angiography is done so the information can be used to guide decisions about what further testing and treatment might be needed. In the interventional centre, invasive coronary angiography is often done to help plan percutaneous coronary intervention. Clinical experts noted that QAngio QFR and CAAS vFFR need high-quality angiography images so in diagnostic centres the tests may have a high technical failure rate. The committee concluded that because all the data considered were from interventional centres, it was not certain what the technical failure rate would be in diagnostic-only centres.

## It is unclear how clinical history and symptoms affect clinical decisions based on QAngio QFR or CAAS vFFR results
The NICE guideline on assessment and diagnosis of chest pain of recent onset recommends invasive coronary angiography as a third-line test. People who have invasive coronary angiography should have already had a previous assessment such as 64-slice coronary angiography and non-invasive functional imaging tests, but this may vary between centres. Some people may also have HeartFlow FFRCT which is recommended in the NICE medical technologies guidance on HeartFlow FFRCT for estimating FFR from coronary CT angiography. The clinical experts explained that these previous assessments can rule out the need for interventional treatment. Therefore, it is likely that people who do go on to have invasive coronary angiography have more severe disease than the people in the diagnostic accuracy studies. While the previous functional assessments may be used to guide further testing decisions such as whether to do an FFR, the QAngio QFR and CAAS vFFR results would be used to guide high-level treatment decisions with substantial consequences. Therefore, clinicians need to be confident that making a decision based on the results of these tests would lead to improved outcomes for patients. The committee commented that the diagnostic accuracy studies did not incorporate clinical history and the effect that symptoms had on decision making based on the QAngio QFR result. Therefore, it is unclear how this additional information combined with a QAngio QFR or CAAS vFFR result affects clinical decision making about revascularisation.

## QAngio QFR may slightly increase revascularisation rates compared with FFR but this is uncertain
The EAG did a simulation study analysis to investigate the possible effect of using QAngio QFR compared with invasive coronary angiography and FFR on coronary outcomes such as revascularisation rates and major adverse cardiovascular events. QAngio QFR (with or without a grey zone) led to slightly more revascularisations compared with FFR (40.2% revascularisations using FFR compared with 42.0% for QAngio QFR and 43.2% using the grey-zone strategy). Both methods prevented broadly the same number of major adverse cardiovascular events (FFR may prevent more major adverse cardiovascular events but only for 1 in 1,000 people). However, the committee noted that the simulation study made numerous assumptions, so its results were uncertain.

## Clinical outcome data from large endpoint studies for QAngio QFR and CAAS vFFR are needed
The clinical experts noted the lack of prospective outcome data when a QAngio QFR or CAAS vFFR-based approach was used to guide revascularisation decisions after invasive coronary angiography. The clinical experts explained that there was a need for clinical outcome data from large endpoint studies comparing these imaging software with FFR or invasive coronary angiographyguided treatment. There are already multiple tests in the care pathway, and it was unclear how QAngio QFR or CAAS vFFR could offer additional clinical benefit. There are currently 2 ongoing clinical trials of QAngio QFR. The FAVOR III Europe-Japan study will compare QFR with standard FFR-guided percutaneous coronary intervention, and the FAVOR III China study will compare QFR with angiographyalone guided percutaneous coronary intervention. These trials will be completed in December 2023 and February 2023, respectively. There is 1 ongoing trial of CAAS vFFR. The LIPSIA STRATEGY trial will compare vFFR with FFR for the assessment of intermediate coronary stenosis and is due to be completed in November 2026. The committee concluded that data from trials like these are essential to be confident that revascularisation decisions based on QAngio QFR or CAAS vFFR results would improve patient outcomes (see research recommendation 5.3).

# Cost effectiveness

## The disutility associated with FFR or iFR used in the model may not be appropriate
In the model the procedural disutility for FFR was assumed to be the same as for percutaneous coronary intervention. This was because no data were available on the disutility of FFR. The clinical experts explained that this assumption may not accurately reflect the actual side effects or people's experiences of the procedure. The EAG also looked at different scenario analyses where the disutility of FFR was increased. The committee commented that a disutility equivalent to percutaneous coronary intervention was likely too high. This affected the cost effectiveness of FFR more than might be expected in clinical practice. It noted further that there were important differences between FFR and iFR that were considered the same in the model. Because iFR does not need a hyperaemic agent such as adenosine, it avoids the associated unpleasant side effects. This disutility was a key driver of the cost-effectiveness results, but because of a lack of evidence it was uncertain what disutility should be used. The committee recommended further research into the disutility associated with FFR (see research recommendation 5.1).

## Test costs may not be accurate because test failure rates are not adequately captured in the model
Test failure rates were high in the studies, especially the retrospective ones, because the invasive coronary angiography images were not good enough to run QAngio QFR and CAAS vFFR. The committee noted that in clinical practice some images may not be of a sufficient quality for the software programs to produce a result. Therefore, the cost per test may have been underestimated because test failure rates were not factored into the model. Only people who had a QAngio QFR or CAAS vFFR result were included. The EAG did a scenario analysis in which patient throughput was varied, which affected the cost per test. However, the committee noted that this may not have explored a wide enough range to sufficiently capture this effect. It suggested that failure rates in routine clinical practice would be reduced over time as the operator gained experience in using the system. However, the clinical experts explained that this may be dependent on the setting, with sub-optimal angiography images more likely in diagnostic-only centres (see section 4.5). The committee concluded that further research on QAngio QFR and CAAS vFFR failure rates in clinical practice would be beneficial (see research recommendation 5.2).

## QAngio QFR and CAAS vFFR are more cost effective than invasive coronary angiography alone, but the results are uncertain
In the base case QAngio QFR was within the range NICE considers cost effective compared with invasive coronary angiography alone in both the deterministic and probabilistic analyses. However, the clinical experts commented that the strategy of invasive coronary angiography alone was not representative of clinical practice, where the results of previous tests and people's preferences would also influence a treatment decision (see section 4.6). The committee noted that similar results were seen for CAAS vFFR but concluded that there was greater uncertainty in this result because of the lack of diagnostic accuracy evidence.

## More data are needed because the clinical utility and cost effectiveness of QAngio QFR and CAAS vFFR are uncertain
Compared with the reference standard of FFR or iFR, QAngio QFR (with and without a grey zone) and CAAS vFFR were less cost effective (generated less quality-adjusted life years  but were slightly cheaper) in the base-case analysis. However, the committee noted that a review of the accuracy of FFR or iFR was not done by the EAG and they were assumed in the model to be 100% accurate. It also noted that the difference between the new technologies and the reference standard was small at 0.007 QALYs or 140 per person for QAngio QFR and 0.011 QALYs or 220 per person for CAAS vFFR. Following an update to the price structure of QAngio QFR by the company during consultation, QAngio QFR using an annual licence became slightly cheaper but remained less clinically effective than the reference standard of FFR or iFR. However, given the small difference in costs and outcomes, the committee reiterated the need for clinical outcome data from studies that directly compare QAngio QFR and CAAS vFFR with FFR or iFR (see research recommendation 5.3). These data would give clinicians confidence in their decision making based on the results of the tests. The committee concluded that given the uncertainty in clinical utility the cost-effectiveness results were also uncertain.

## The potential role of QAngio QFR and CAAS vFFR in a diagnostic-only setting is unclear
In a scenario analysis in which the tests were done in a diagnostic-only setting, QAngio QFR and CAAS vFFR became the most cost effective options in the fully incremental analyses. This was because of the additional cost of onward referral for the reference standard tests. However, clinical experts explained that fewer people are having invasive coronary angiography because it is recommended as a third-line test in the NICE guideline on assessment and diagnosis of chest pain of recent onset (see section 4.6). The appropriate use of CT coronary angiography and functional testing has resulted in a fall in the number of people having invasive coronary angiography in diagnostic-only centres. Having an angiographic procedure in an interventional centre means that invasive coronary angiography, FFR or iFR and percutaneous coronary intervention can be done in a single visit, if appropriate. This reduces the need for multiple hospital visits, which has the potential to reduce people's anxiety. A clinical expert explained that according to 2017 to 2018 data from the National Institute of Cardiovascular Outcomes Research, around 35,000 invasive coronary angiography procedures were done in diagnostic-only settings, compared with around 205,000 in interventional centres. The committee concluded that the future role of QAngio QFR and CAAS vFFR in a diagnostic-only setting is unclear because diagnostic-only catheter laboratories are likely to decline in number.

## QAngio QFR and CAAS vFFR are not recommended for routine use
The committee noted that QAngio QFR and CAAS vFFR were more cost effective than invasive coronary angiography alone. QAngio QFR using the proposed annual licence was slightly cheaper but less clinically effective than FFR or iFR. However, the incremental difference in terms of costs and QALYs between the tests was small. For CAAS vFFR, the committee recalled that the diagnostic accuracy data was highly uncertain so it could not be recommended for routine use. The committee noted further that there were no clinical outcome studies for QAngio QFR or CAAS vFFR, which meant that the EAG had to make assumptions about treatment decisions and clinical outcomes, which led to uncertainty in the results. The clinical experts commented that these tests may be used to guide high-level clinical decisions about treatment, so clinicians need to be confident when making decisions based on the tests' results. The committee considered that clinical utility is uncertain and more data are needed. There were concerns around the results of the simulation study that showed that QAngio QFR could lead to an increase in revascularisations (see section 4.7). The committee also recalled the trend in clinical practice of moving away from diagnostic-only settings to interventional centres (see section 4.13) and considered that there was too much uncertainty to consider QAngio QFR and CAAS vFFR in this scenario. The committee concluded that because of the uncertainty in diagnostic accuracy and clinical evidence, CAAS vFFR was not recommended. Despite having good diagnostic accuracy evidence, QAngio QFR should not be recommended for use until further data showing that it improves patient outcomes are available.# Recommendations for further research
A patient experience study is recommended to better understand the general effect on patients of having an invasive coronary angiography and the range and severity of side effects and complications from QAngio XA 3D quantitative flow ratio (QAngio QFR), CAAS vessel fractional flow reserve (CAAS vFFR), FFR and instantaneous wave-free ratio.
Further research is recommended on test failure rates of QAngio QFR and CAAS vFFR and how these affect clinical decision making for revascularisation in clinical practice.
Outcome studies are needed to understand the clinical benefit of using QAngio QFR and CAAS vFFR (see section 4.8). These include rates of major adverse cardiovascular events, mortality and EQ-5D data to assess the effect on quality of life.
More diagnostic accuracy studies are needed for CAAS vFFR against an appropriate reference standard.