Calcium scoring using 64-slice MDCT , dual source CT and EBT : a comparative phantom study Purpose Assessment of calcium scoring -LRB- Ca-scoring -RRB- on a 64-slice multi-detector computed tomography -LRB- MDCT -RRB- scanner , a dual-source computed tomography -LRB- DSCT -RRB- scanner and an electron beam tomography -LRB- EBT -RRB- scanner with a moving cardiac phantom as a function of heart rate , slice thickness and calcium density . Methods and materials Three artificial arteries with inserted calcifications of different sizes and densities were scanned at rest -LRB- 0 beats per minute -RRB- and at 50 -- 110 beats per minute -LRB- bpm -RRB- with an interval of 10 bpm using 64-slice MDCT , DSCT and EBT . Images were reconstructed with a slice thickness of 0.6 and 3.0 mm . Agatston score , volume score and equivalent mass score were determined for each artery . A cardiac motion susceptibility -LRB- CMS -RRB- index was introduced to assess the susceptibility of Ca-scoring to heart rate . In addition , a difference -LRB- Δ -RRB- index was introduced to assess the difference of absolute Ca-scoring on MDCT and DSCT with EBT . Results Ca-score is relatively constant up to 60 bpm and starts to decrease or increase above 70 bpm , depending on scoring method , calcification density and slice thickness . EBT showed the least susceptibility to cardiac motion with the smallest average CMS-index -LRB- 2.5 -RRB- . The average CMS-index of 64-slice MDCT -LRB- 9.0 -RRB- is approximately 2.5 times the average CMS-index of DSCT -LRB- 3.6 -RRB- . The use of a smaller slice thickness decreases the CMS-index for both CT-modalities . The Δ-index for DSCT at 0.6 mm -LRB- 53.2 -RRB- is approximately 30 % lower than the Δ-index for 64-slice MDCT at 0.6 mm -LRB- 72.0 -RRB- . The Δ-indexes at 3.0 mm are approximately equal for both modalities -LRB- 96.9 and 102.0 for 64-slice MDCT and DSCT respectively -RRB- . Conclusion Ca-scoring is influenced by heart rate , slice thickness and modality used . Ca-scoring on DSCT is approximately 50 % less susceptible to cardiac motion as 64-slice MDCT . DSCT offers a better approximation of absolute calcium score on EBT than 64-slice MDCT when using a smaller slice thickness . A smaller slice thickness reduces the susceptibility to cardiac motion and reduces the difference between CT-data and EBT-data . The best approximation of EBT on CT is found for DSCT with a slice thickness of 0.6 mm . Introduction The presence of calcium in coronary arteries is known to be a strong indicator for coronary artery disease -LRB- CAD -RRB- -LSB- 1 -RSB- . It has been shown that quantification of coronary calcium enables the assessment of cardiac event risk stratification -LSB- 1 -RSB- . In 1990 , Agatston et al. described a method which determines the amount of coronary calcium from tomographic images -LSB- 2 -RSB- . This method , known as the Agatston score -LRB- AS -RRB- , depends on the area and the maximum CT density of the calcification detected by electron beam tomography -LRB- EBT -RRB- . Since then , EBT is generally accepted as the gold standard for determining the amount of coronary calcium . Alternative scoring methods have been proposed , such as volume scoring -LRB- VS -RRB- , depending on the volume of the calcification , and equivalent mass -LRB- EM -RRB- scoring , which depends on the volume and the average density of the calcification -LSB- 3 -- 5 -RSB- . Calcium scoring -LRB- Ca-scoring -RRB- on EBT is known to be less susceptible to cardiac motion compared to other CT-modalities , because of its relatively high temporal resolution . However , since the appearance of multi-detector computed tomography -LRB- MDCT -RRB- , scanners of this type are also widely used for Ca-scoring as an alternative to EBT . Although the temporal resolution of MDCT is lower than EBT , the spatial resolution is much higher -LRB- 0.4 vs. 1.0 mm -RRB- , enabling the detection of smaller lesions . Whereas Ca-scoring on EBT can only be used in sequential scanning mode , MDCT facilitates Ca-scoring in sequential and spiral mode . Spiral mode scanning has shown to decrease the variability of Ca-scoring when compared to sequential mode scanning -LSB- 6 -RSB- . With the development of dual source computed tomography -LRB- DSCT -RRB- in 2006 , CT is finally approaching the temporal resolution of EBT combined with a high spatial resolution -LSB- 7 -RSB- . In order to use Ca-scoring as a useful diagnostic test , it must be demonstrated as accurate , clinically relevant and reproducible . Monitoring of coronary atherosclerosis by repeated scans is advocated by Callister et al. to test the response to lipid-lowering pharmacologic therapy -LSB- 8 -RSB- and Budoff et al. -LSB- 9 -RSB- showed that statin therapy induced a 61 % reduction in coronary calcium progression rate . Therefore a highly reproducible scan-method independent of in-vivo conditions to test the accuracy of Ca-scoring is desirable . In this study a cardiac phantom was used to investigate the influence of cardiac motion on the absolute Ca-score for different kinds of scanners . To our knowledge no previous study has systematically investigated the influence of the heart rate on the absolute Ca-score using EBT , 64-slice MDCT and DSCT . The purpose of this study was therefore to assess Ca-scoring on 64-slice MDCT and DSCT versus EBT on a moving cardiac phantom as a function of heart rate , slice thickness and calcification density using 3 different Ca-scoring methods . Methods and materials Cardiac phantom A moving cardiac phantom -LRB- QRM , Möhrendorf , Germany -RRB- was used to simulate the movement of the coronary arteries -LRB- Fig. 1 , left -RRB- -LSB- 7 , 10 -RSB- . The phantom consists of a robot arm which performs a pre-programmed motion -LRB- Fig. 2 -RRB- . The robot arm moves in a water container inside a thorax phantom -LRB- QRM , Möhrendorf , Germany -RRB- -LSB- 11 -RSB- . Different inserts can be attached to the robot arm . The motion curves used in this study were based on velocity curves for the LAD given in literature in order to simulate the human coronary motion as realistically as possible -LSB- 12 -RSB- . Three different artificial arteries were investigated which were custom built by QRM . The artificial arteries were made of hydroxyapatite -LRB- HA -RRB- with a diameter of 4 mm and a length of 55 mm -LRB- Fig. 1 -RRB- . Each artery contained three artificial calcifications with a length of 10 mm , a spacing of 5 mm and a thickness of 0.5 , 1.0 and 2.0 mm , respectively . The density of the calcifications was different in each artery , one with high density calcifications -LRB- HDC -RRB- , one with medium density calcifications -LRB- MDC -RRB- and one with low density calcifications -LRB- LDC -RRB- . The concentration and density of the calcifications in the three artificial atereries is given in Table 1 . The artificial artery had a density of 50 Houndsfield Units -LRB- HU -RRB- , simulating human blood . Fig. 1Left : the cardiac phantom . Right : schematic figure of the artificial artery , the dimensions are given in millimetersFig . 2Motion curve of the phantom at 70 bpm . The curve is defined by the time-deflection points T1 -- T8 and the reconstruction intervals of the DSCT and 64-slice MDCT are indicated by the grey areas . Other heart rates are obtained by a time scaling of the data points . For higher heart rates -LRB- > 90 bpm -RRB- the data point T5 was omitted to reflect the relative larger diminishing of the diastolic phaseTable 1The three artificial coronary arteries high , medium and low density calcification -LRB- HDC , MDC and LDC -RRB- with the properties of the inserted calcifications as specified by the manufacturerArtificial arteryConcentration -LRB- mgHA/cm3 -RRB- Density -LRB- g/cm3 -RRB- HDC7961 .58 MDC4011 .30 LDC1971 .16 Data acquisition The phantom was positioned at an angle of 45 degrees relative to the center axis of the scanner . Every scan was repeated five times with a small random translational -LRB- approximately 2 mm -RRB- and small random rotational repositioning -LRB- approximately 2 degrees -RRB- of the phantom after each scan . The ECG signal from the phantom was recorded during scanning to enable synchronization with the scanner . The scan parameters on the 64-slice MDCT -LRB- Somatom Sensation 64 , Siemens , Forchheim , Germany -RRB- were : tube voltage 120 kV , tube current 250 mAs effective , collimation 64 × 0.6 mm and rotation time 330 ms. A DSCT -LRB- Somatom Definition , Siemens , Forchheim , Germany -RRB- was used with similar scan parameters : tube voltage 120 kV , tube current 100 mAs/rot -LRB- equivalent to the tube current of 64-slice MDCT -RRB- , collimation of 64 × 0.6 mm and rotation time 330 ms. A spiral scanning mode was used on both scanners for a better reproducibility . A standard hospital calcium scoring protocol was used on the EBT-scanner -LRB- e − Speed , GE Imatron , San Francisco , USA -RRB- . This protocol uses a sequential mode with a tube voltage of 130 kV , a tube current of 44 mAs , a collimation of 3.0 mm and a scan speed of 50 ms. A standard calcium scoring kernel -LRB- B35f -RRB- was used for reconstruction of the CT-data . Images were retrospectively reconstructed with a slice thickness of 0.6 mm -LRB- increment 0.4 mm -RRB- and 3.0 mm -LRB- increment 3.0 mm -RRB- for both CT scanners . The phases with minimal motion were selected from the motion curves of the coronary arteries -LRB- Fig. 2 -RRB- and used for reconstruction of the raw data -LRB- Table 2 -RRB- . The data from the EBT-scanner were reconstructed with a slice thickness of 3.0 mm -LRB- increment 3.0 mm -RRB- at 40 % of the RR-interval with a standard calcium kernel according to the standard calcium scoring protocol used in our hospital . Table 2Phases used for reconstruction of the images in percentage of the beat time at different heart rates used in beats per minute -LRB- bpm -RRB- Heart rates -LRB- bpm -RRB- 506070809010011064-Slice MDCT-phase -LRB- % -RRB- 76746058565351DSCT-phase -LRB- % -RRB- 83827069696766 Ca-scoring was performed on the reconstructed image sets with commercially available software -LRB- Syngo CaScore , Siemens , Forchheim , Germany -RRB- . Three different scoring methods were used : Agatston scoring , volume scoring and equivalent mass scoring . A standard scoring threshold of 130 HU was used during the procedure . Detailed descriptions of these scoring methods can be found extensively elsewhere -LSB- 4 , 11 , 13 -- 15 -RSB- . The three calcifications of the arteries could not be detected individually at higher heart rates -LRB- at heart rates larger than 60 bpm for 64-slice MDCT and larger than 90 bpm for DSCT -RRB- combined with thin slices in some of the scans . Therefore the Ca-score of the total artery was used instead of the Ca-scores of the individual calcifications . Data analysis Two root mean square measures were used to analyze the scoring results . The first measure quantifies the susceptibility of the calcium score to cardiac motion . The second measure quantifies the deviation of the calcium score from the reference value . We defined a cardiac motion susceptibility -LRB- CMS -RRB- index in order to assess the susceptibility to cardiac motion of the Ca-scoring methods : in which x0 is the Ca-scoring result at 0 bpm , xi is the scoring result at heart rate i and N is the total number of heart rates used . In the equation for the CMS-index a factor 1/x0 is introduced to make the index independent of the absolute score which enables comparison of Ca-scores obtained at different slice thicknesses and with different scoring methods as a function of cardiac motion . A small CMS-index is equivalent to a low susceptibility of Ca-scores to cardiac motion . A second measure was introduced to compare the calcium score results of the two CT scanners to the results of the EBT scanner . The deviation of the calcium score on CT versus the reference value on EBT is defined using a Δ-index : in which yi is the EBT-score at heart rate i , zi is the CT-score at heart rate i and yav is the average EBT-score over all heart rates . The normalization factor yav was inserted to make the Δ-index independent of the absolute score and to enable comparison of Δ-indexes obtained with different Ca-scoring methods and slice thicknesses . A low Δ-index is equivalent to a small difference between Ca-scores on CT and EBT . The delta-index , as defined in Eq . 2 , is used to quantify the difference in Agatston and volume scores on CT and EBT . For these scoring methods EBT provides the reference value . For the equivalent mass score , however , the reference value is given by the physical mass . The use of a phantom enables the possibility of calculating the true amount of calcium . Therefore the equivalent mass scoring results have been compared to the true values instead of the EBT-values , thus yi is the true value and zi is the CT/EBT-score at heart rate i. Noise levels were measured using a standard Region of Interest -LRB- ROI -RRB- technique . The ROI was placed in a section of a slice containing only water . The standard deviation of the measured HU-values within the selected ROI was considered to be a measure for the noise level . All measurements are considered to be normally distributed . Mean and standard deviation -LRB- sd -RRB- are given for each measurement . Results The Ca-scoring results of the different arteries obtained with 64-slice MDCT , DSCT and EBT are shown in Fig. 3 as a function of slice thickness and heart rate using the three different scoring methods . The scoring results are relatively constant at low heart rates -LRB- 50 -- 60 bpm -RRB- . At heart rates higher than 60 bpm , however , the scores deviate from the values at lower heart rates and an increase or decrease of scoring results is observed depending on modality , slice thickness , calcification density and scoring method . Fig. 3Calcium scores as a function of heart rate in beats per minute using 64-slice MDCT at 0.6 mm -LRB- dotted line with circles -RRB- , 64-slice MDCT at 3.0 mm -LRB- solid line with triangles -RRB- , DSCT at 0.6 mm -LRB- dotted line with circles -RRB- , DSCT at 3.0 mm -LRB- solid line with triangles -RRB- , EBT -LRB- solid line with squares -RRB- . Agatston score -LRB- AS -RRB- , volume score -LRB- VS -RRB- and equivalent mass -LRB- EM -RRB- score from top to bottom ; artificial arteries high density calcification -LRB- HDC -RRB- , medium density calcification -LRB- MDC -RRB- and low density calcification -LRB- LDC -RRB- from left to right . The thick dotted black lines in the figures in the bottom row represent the physical amount of calcium The results show a general underestimation of the Ca-score for Ca-scoring obtained at 3.0 mm slice thickness when comparing CT-data and EBT at all heart rates except for the Agatston and volume score of the high density calcifications at 70 and 80 bpm . In general , the Ca-scores obtained with 0.6 mm slice thickness on CT are overestimated compared to the EBT-data or are similar to the EBT-data at all heart rates . The scores obtained with EBT -LRB- squares -RRB- increased at heart rates above 90 bpm for the artery containing the high density calcifications -LRB- Fig. 3 , left column -RRB- , whereas the artery containing the medium density calcifications remained relatively constant throughout the whole range of heart rates -LRB- Fig. 3 , middle column -RRB- . The artery containing the low density calcifications showed decreased scoring results at higher heart rates -LRB- Fig. 3 , right column -RRB- . The 64-slice MDCT with a slice thickness of 3.0 mm -LRB- solid lines with triangles -RRB- showed increased Ca-scores for the Agatston score at 70 -- 90 bpm and for the volumes score at all heart rates for the high density calcification , whereas the equivalent mass score showed a slight decrease . The medium and low density calcification also showed a decrease in scoring results at higher heart rates . The 64-slice MDCT with a slice thickness of 0.6 mm -LRB- dotted lines with circles -RRB- showed highly increased Ca-scores above 70 bpm for the high density calcification for all scoring methods . This is also seen for the medium density calcification for the volume score , whereas the equivelnt mass and Agatston score showed a peak in Ca-scores at 80 bpm . The low density calcification showed diminished results at higher heart rates for all scoring methods . The Ca-scores of the medium and low density calcification obtained with DSCT with a slice thickness of 3.0 mm -LRB- solid lines with triangles -RRB- were decreased at elevated heart rates . The results of the high density calcification were relatively constant over the whole range of heart rates . Finally DSCT at 0.6 mm -LRB- dotted lines with circles -RRB- showed increased results for Agatston and volume score of the high density calcification . The Agatston score of the medium density calcification showed a small decrease and relatively constant results were observed for the equivalent mass score of the high density calcification and volume and equivalent mass score of the medium density calcification . Diminishing results with increasing heart rate were observed for all methods for the low density calcification . The influence of cardiac motion on the Ca-score -LRB- CMS-index -RRB- using the different scoring methods is calculated using Eq . 1 and is summarized in Fig. 4a -- c for all scanners and slice thicknesses . Looking at the results of the Agatston score , the average CMS-index for EBT was approximately similar to the CMS-index of DSCT at 0.6 mm , which was for its part approximately 60 % smaller than the CMS-index of 64-slice MDCT at 0.6 mm . The CMS-index of DSCT at 3.0 mm was approximately 50 % higher than the CMS-index at 0.6 mm . The CMS-index of 64-slice MDCT at 3.0 mm was approximately twice as large as the index of DSCT at 3.0 mm -LRB- Fig. 4a -RRB- . The results of the susceptibility to cardiac motion using volume and equivalent mass score were similar to the results obtained with the Agatston score , except for the relatively small CMS-index for 64-slice MDCT at 0.6 mm using for equivalent mass score . The absolute CMS-indexes using equivalent mass were approximately 10 % lower compared to the other two methods . The CMS-indexes averaged over scoring method , slice thickness and calcification density were 2.5 for EBT , 3.6 for DSCT and 9.0 for 64-slice MDCT . Fig. 4Cardiac motion susceptibility-index -LRB- see text -RRB- determined with Agatston score -LRB- AS -RRB- -LRB- a -RRB- , volume score -LRB- VS -RRB- -LRB- b -RRB- and equivalent mass -LRB- EM -RRB- -LRB- c -RRB- score for the high , medium , low density lesions and the average using EBT -LRB- with slice thickness of 3.0 mm -RRB- , 64-slice MDCT -LRB- with slice thickness of 3.0 and 0.6 mm -RRB- and DSCT -LRB- with slice thickness of 3.0 and 0.6 mm -RRB- . A small CMS-index represents a low susceptibility to cardiac motion . The standard deviations of the CSM-index are indicated by error bars . 64S = 64-slice MDCT ; DS = Dual Source CT The difference between the scoring results using Agatston and volume score of 64-slice MDCT and DSCT compared to EBT are calculated using Eq . 2 and are shown in Fig. 5a -- b. For Agatston score -LRB- Fig. 5a -RRB- , the best Δ-index was observed for DSCT with a slice thickness of 0.6 mm -LRB- 35.9 ± 10.0 averaged over all densities -RRB- . A Δ-index approximately twice as large was observed for 64-slice MDCT at 0.6 mm -LRB- 65.7 ± 9.0 averaged over all densities -RRB- . Both CT-modalities at 3.0 mm had a Δ-index approximately two times the Δ-index of DSCT at 0.6 mm -LRB- 91.0 ± 10.1 and 88.4 ± 9.1 for DSCT and 64-slice MDCT respectively averaged over all densities -RRB- . Comparable results were observed for the volume score measurement -LRB- Fig. 5b -RRB- , although the Δ-indexes for the measurements at 0.6 mm were higher with the highest Δ-index for 64-slice MDCT at 0.6 mm . Fig. 5Δ-index -LRB- see text -RRB- determined with Agatston score -LRB- AS -RRB- -LRB- a -RRB- , volume score -LRB- VS -RRB- -LRB- b -RRB- and equivalent mass -LRB- EM -RRB- -LRB- c -RRB- score for the high , medium , low density lesions and the average using 64-slice MDCT and DSCT , both with slice thicknesses of 3.0 and 0.6 mm . For Agatston and volume score , EBT has been used as a reference value -LRB- a and b , respectively -RRB- , whereas for equivalent mass score the physical mass has been used as a reference value -LRB- c -RRB- . The equivalent mass measurement includes the EBT as well . A small Δ-index represents a good correspondence with the EBT results . 64S = 64-slice MDCT ; DS = Dual Source CT A Δ-index was calculated for all scanners comparing the equivalent mass results to the theoretical true values . The results are shown in Fig. 5c . The smallest Δ-index was observed for 64-slice MDCT -LRB- 55.9 ± 6.8 -RRB- followed by higher indexes for DSCT -LRB- 68.3 ± 8.3 -RRB- and EBT -LRB- 71.3 ± 7.9 -RRB- both with a slice thickness of 0.6 mm , however the indexes of 64-slice MDCT and DSCT and the indexes of EBT and DSCT are within each margins of error shown by the error bars . Both CT-modalities at 3.0 mm showed Δ-indexes approximately twice as large compared to the results at 0.6 mm -LRB- 140.1 ± 7.8 and 131.1 ± 8.5 for DSCT and 64-slice MDCT respectively averaged over all densities -RRB- . The Δ-indexes were 53.2 for DSCT and 72.0 for 64-slice MDCT both with a slice thickness of 0.6 mm averaged over the scoring methods and densities . The Δ-indexes at 3.0 mm were 102.0 for DSCT and 96.9 for 64-slice MDCT averaged over the scoring methods and densities . Noise levels were as follows : 64-slice MDCT showed 36.1 ± 2.9 HU and 13.2 ± 1.2 HU for 0.6 and 3.0 mm slice thickness , respectively . DSCT showed 43.0 ± 1.6 HU and 16.1 ± 1.0 HU for 0.6 and 3.0 mm slice thickness , respectively . EBT with a slice thickness of 3.0 mm showed a noise level of 20.5 ± 0.8 HU . The noise did not vary at different heart rates . Discussion An assessment was made of Ca-scoring on 64-slice multi-detector computed tomography and dual-source computed tomography versus electron beam tomography on a moving cardiac phantom as a function of heart rate , slice thickness and calcification density using 3 different Ca-scoring methods . From the results it can be concluded that the Agatston , volume and equivalent mass scores depend on heart rate , slice thickness and the CT-system used . Furthermore DSCT is approximately 50 % less susceptible to cardiac motion as 64-slice MDCT in Ca-scoring . It has been shown in previous studies that the amount of calcium in coronary arteries is generally underestimated in MDCT with respect to the gold standard EBT . Stanford et al. showed an underestimation of coronary calcium with 4-slice MDCT compared to EBT -LSB- 16 -RSB- and the same effect was reported by Horiguchi et al. using 16-slice MDCT -LSB- 14 , 17 -RSB- . Our results showed underestimation as well , but only for 3.0 mm slice thickness , whereas 0.6 mm showed an overestimation at all heart rates . Surprisingly the Agatston scores of the medium density calcification at rest using 3.0 mm slices are different for the 64-MDCT and DSCT , while similar scores are expected -LRB- approximately 165 for 64-slice MDCT and 135 for DSCT -RRB- . The same effect is observed for heart rates of 50 and 60 bpm . A possible explanation for this phenomenon lies within the scoring algorithm of the Agatston score . For each calcification the maximum HU value within the calcification is obtained . Based on this maximum value the area of the calcification is multiplied by a weighting factor . For a maximum of more than 400 HU this factor is 4 , for a maximum between 300 HU and 400 HU this factor is 3 -LSB- 2 -RSB- . The medium density calcification has a CT density of 400 HU . The difference in scoring results can be explained by a small difference in HU between the two scanners . Where the maximum CT density within the medium density calcification could be over 400 HU using the 64-slice MDCT , the maximum CT might have been below 400 using the DSCT . If this explanation is applied , the score obtained using 64-slice MDCT is more similar to the score obtained using DSCT -LRB- 165 * 3/4 = 124 -RRB- . Although the difference in HU is very small , the weighting factor of the Agatston algorithm can cause a large difference in scoring result . At heart rates above 70 bpm Agatston , volume and equivalent mass score differ from the results at rest and at low heart rates . This difference depends on the density of the calcification as can be seen from Fig. 3 : calcifications with a high density show elevated scoring results , whereas low density calcifications are associated with diminished scoring results . A decrease of Agatston and equivalent mass score on increased heart rates using a calcification of 400 HU has also been reported by Ulzheimer et al. using a 4-slice MDCT in accordance with our results -LSB- 11 -RSB- . We considered the influence of image blurring on the Ca-score as a function of the calcification density in Fig. 6 for a possible explanation for this effect . Two calcifications of identical size are shown by black lines , one with a high density -LRB- X -RRB- and one with a low density -LRB- Y -RRB- . The corresponding apparent images at a relatively low and high heart rate are given by the solid grey and dotted grey line , respectively . In addition , the default Ca-scoring threshold of 130 HU is shown by the dotted black line . At the level of the threshold the apparent width at high heart rates is larger than the apparent width at low heart rates for the high density object . The reverse effect is observed for the low density object ; at high heart rates the apparent size is reduced compared to the apparent size at low heart rates . From this analysis it can be concluded that at high heart rates the apparent volume of high density objects is increased and the apparent volume of low density objects is decreased . With this model we can explain the increase of calcium score on increasing heart rate for high density calcifications , and a decrease of calcium score on increasing heart rate for low density calcifications , as observed in Fig. 3 . Decreasing scoring results with increasing movement have previously been reported on 4-slice CT -LSB- 15 -RSB- . Fig. 6Theoretical estimated CT profiles for two objects -LRB- black -RRB- with high -LRB- X -RRB- and low density -LRB- Y -RRB- exhibiting a relatively low -LRB- solid grey -RRB- and high -LRB- dotted grey -RRB- movement . The dotted black line represents the standard Ca-scoring threshold of 130 HU The susceptibility of calcium score on heart rate has been assessed by the CMS-index using the 3 different scoring methods available . The results show that the CMS-index of EBT is the lowest for all methods . Therefore it can be concluded that EBT is the least susceptible to cardiac motion . The CMS-index of DSCT is approximately half the CMS-index of 64-slice MDCT , showing a reduction of 50 % of the influence of cardiac motion on Ca-scoring on DSCT with respect to 64-slice MDCT . These results can be explained with the improved temporal resolution of DSCT compared to 64-slice MDCT -LRB- 83 vs. 165 ms -RRB- . A reduction of the slice thickness also results in a lower CMS-index . Therefore we conclude that the use of a small slice thickness reduces the susceptibility to cardiac motion for both 64-slice MDCT and DSCT . The difference between CT-data and EBT-data has been assessed by the Δ-index using the Agatston and volume score , the equivalent mass results have been compared to the physical amount of calcium . The results show the lowest Δ-index for DSCT with a slice thickness of 0.6 mm for Agatston and volume score . The CT modalities at 0.6 mm and EBT showed similar Δ-indexes for the approximation to the physical mass . A reduction of the Δ-index was observed comparing the two CT-modalities at 0.6 mm and 3.0 mm . The best resemblance between EBT and CT was observed for DSCT with a slice thickness of 0.6 mm . The use of a smaller slice thickness has some disadvantages although it was beneficial to the scoring results in this phantom study . The noise measurements showed increased noise levels for the 0.6 mm slices compared to 3.0 mm slices . It is expected that for patient scanning the noise levels at 0.6 mm are too high to guarantee a reliable outcome of the Ca-scoring . To overcome these increased noise levels the tube current can be increased . However , this increases the patient dose as well . Although dose-reduction techniques have been investigated leading to dose-reductions up to 57 % -LSB- 18 -- 21 -RSB- , a good balance between patient dose and accuracy of calcium scoring needs to be found . Limitations The EBT-data acquisition of this study was performed with a standard hospital protocol using a tube voltage of 130 kV , whereas CT scanning was performed with a tube voltage of 120 kV . Although higher energies tend to show less density , Nelson et al. reported very small differences between EBT at 130 kV and CT at 120 kV -LSB- 22 -RSB- . Therefore we expect that the influence of the difference in tube voltage can be neglected . The pre-programmed movement of the calcified coronary arteries was 1-dimensional in contrast with the in vivo situation where the motion of human coronary arteries is 3-dimensional and the direction and orientation of the human coronary arteries can vary . In our study the movement of the calcified coronary arteries was in the -LRB- x , z -RRB- plane with a 45o angle relative to the z-direction of the scanner . We expect that movement more perpendicular to the z-direction of the scanner will cause more blurring in the -LRB- x , y -RRB- plane and reduce blurring in the z-direction . In addition , we expect that movement more parallel to the z-direction of the scanner will be more subject to partial volume effects when using thick slices . Thin slices will be less subject to the PVE due to the isotropic resolution of 0.6 mm . The motion of the robot arm was programmed according to patient data -LSB- 11 -RSB- and therefore we expect that our analysis shows a good correspondence with a clinical situation , but a clinical validation is advocated . The coronary artery we used for our simulation , the LAD , exhibits lesser motion than the LCX and especially the RCA , which exhibits very large motion swings especially in systole . In our study we , however , wanted to show the influence of motion on the coronary calcium score independent of a specific major coronary artery . We therefore have used motion curves with velocities similar to the LAD to simulate the motion , because if a dependency of calcium score on coronary motion could be proven for the lowest velocity of the LAD , we expect an even stronger dependence for the higher velocities of the LCX and RCA . In our study we have shown that for higher heart rates the under - or overestimation of the calcium score increases as a function of calcification density , independently of the absolute velocity of the artery , but depending on the relative heart rate difference from 0 bpm . Because this motion dependent effect is pronounced visible for the relative low velocity of the LAD , we expect that the results can also be applied to the vaster moving other major arteries . Conclusion The results of Ca-scoring are influenced by heart rate , slice thickness and modality used . DSCT is approximately 50 % less susceptible to cardiac motion than 64-slice MDCT using a robot phantom . Susceptibility is further reduced with a smaller slice thickness . DSCT gives a better approximation of the absolute calcium score on EBT than results obtained with 64-slice MDCT when using a smaller slice thickness -LRB- 0.6 mm -RRB- . The two modalities show similar results when using larger slice thicknesses -LRB- 3.0 mm -RRB- . In general , the use of a smaller slice thickness further reduces the difference between CT-data and EBT-data . The best approximation to the physical amount of calcium was found using a small slice thickness , where 64-slice MDCT and DSCT show similar results . The best approximation of Ca-scoring on EBT is observed for DSCT with a slice thickness of 0.6 mm .

O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


SAPS 3 -- From evaluation of the patient to evaluation of the intensive care unit . Part 1 : Objectives , methods and cohort description Objective Risk adjustment systems now in use were developed more than a decade ago and lack prognostic performance . Objective of the SAPS 3 study was to collect data about risk factors and outcomes in a heterogeneous cohort of intensive care unit -LRB- ICU -RRB- patients , in order to develop a new , improved model for risk adjustment . Introduction Following the publication in the early 1980s of the Acute Physiology and Chronic Health Evaluation Score -LRB- APACHE -LSB- 1 -RSB- -RRB- , Simplified Acute Physiology Score -LRB- SAPS -LSB- 2 -RSB- -RRB- , and-some years later -- APACHE II -LSB- 3 -RSB- systems , outcome prediction became an important topic among European intensivists . Ten years later , a new generation of these instruments was published : APACHE III -LSB- 4 -RSB- , SAPS II -LSB- 5 -RSB- , and Mortality Probability Model -LRB- MPM -RRB- II -LSB- 6 -RSB- . All of these newer systems were developed by using sophisticated statistical techniques in large multinational databases , and were found to perform better than their predecessors -LSB- 7 , 8 -RSB- . The availability of such sophisticated methods for risk adjustment facilitated outcome research in critically ill patients , which became increasingly important over time . Risk adjustment systems now have a fixed place in critical care research for various purposes . At the patient level , the reporting of severity of illness and the use of risk-adjusted mortality rates to draw inferences from their results are a prerequisite for any study to be published . At the intensive care unit -LRB- ICU -RRB- level , observed-to-expected mortality ratios -LRB- or the use of direct standardisation techniques based on severity scores -RRB- have become standard for assessing the impact of ICU-related factors on outcome , such as the effects of organisation and management -LSB- 9 , 10 -RSB- . However , a series of studies assessing the performance of risk adjustment systems unveiled a lack of prognostic performance of these systems : In most cases , lack of calibration was evident over several subgroups of patients , often accompanied by an underestimation of mortality in low-risk patients and an overestimation in high-risk patients . This pattern was observed for all published outcome prediction models in several countries -LSB- 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 -RSB- and seemed to be worsening over time -LSB- 19 -RSB- . For this reason , several researchers tried to improve the prognostic performance of various systems through recalibration , using one of two possible approaches . A level 1 customization requires calculation of a new equation for the prediction of hospital mortality -LRB- without changing the weights of the constituent variables -RRB- . A level 2 customization involves a reweighting of each variable contained in the model . Although recalibration was able to improve prognostic accuracy in some cases -LSB- 13 , 14 -RSB- , it generally did not solve the various problems inherent in the models . These problems can be classified as either user - , patient - , or model-dependent . User-dependent problems include differences in the definitions and application criteria -LSB- 20 , 21 -RSB- . Patient-dependent problems are mainly shifts in the baseline characteristics of the populations over time -LSB- 22 -RSB- : age distribution , distribution of illnesses , and the development of new treatments , all of which affect prognosis . Model-dependent problems have many different causes , such as the lack of important prognostic variables -LRB- e.g. , diagnostic information -LSB- 4 , 23 -RSB- -RRB- or the presence , location and aetiology of infection -LSB- 24 , 25 , 26 -RSB- . Confounding variables and statistically wrong assumptions -LSB- 9 , 27 -RSB- also distort performance results . If recalibration is not sufficient to improve the performance of the prognostic model , the only alternative is to develop a new model that takes into account the results of studies done since the original model was developed . This means incorporating missing variables that have been shown to affect outcome , minimizing problems with the application of the model , and reducing the possibility of other confounders . The objective of the SAPS 3 project was to cope with the above-stated problems by developing a new model for improved risk adjustment in critically ill patients . Another important goal was to make the new model available free of charge for use in the scientific community . In the SAPS 3 study -LRB- which took place at the end of 2002 -RRB- , data about risk factors and outcomes in an international multicentric cohort of critically ill patients were prospectively collected so that a high-quality database would be available for further analysis of the associations between risks and outcomes in our patients . Materials and methods Project Organization The SAPS 3 project was conducted by the SAPS 3 Outcomes Research Group . The project was endorsed by the European Society for Intensive Care Medicine -LRB- ESICM , http://www.esicm.org -RRB- and conducted in cooperation with the Section on Health Services Research and Outcome of the ESICM . The SAPS 3 Outcomes Research Group consists of a project coordinator and a steering group . The steering group was responsible for the scientific conduct and consistency of the project . An additional advisory board integrated further scientists with special expertise who were asked for comments on the scientific content and for help in conducting the project . The complete board lists can be found in Appendix D of the Electronic Supplementary Material -LRB- ESM -RRB- . During the data collection phase , a coordination and communications centre -LRB- CCC -RRB- was installed . The CCC was responsible for the management and control of the project . This included the administration of all project tasks and implementation of actions and activities as necessary ; communication between project partners -LRB- e.g. , centres , researchers and institutions -RRB- through sampling and distribution of necessary information ; and pooling and administration of the data provided by project participants . In addition , the CCC provided almost around-the-clock service to answer urgent questions and resolve problems during the phase of data collection . In each country , a country coordinator was responsible for operational management and direct communication with the participating ICUs in that country , including giving specific help when necessary . The country coordinator was responsible for ensuring completion of the various tasks required of the participating ICUs . The list of country coordinators can be found in Appendix E of the ESM . At the ICU level , an ICU coordinator was responsible for local activities , such as obtaining approval from the local ethics or data-protection committees where applicable . In addition , the ICU coordinator was responsible for supervising the daily data collection , problem management , controlling the completeness of the data , data quality control , training medical and nonmedical staff for data collection , management of the data , and transmission of the data to the CCC or country coordinator . The list of ICU coordinators can be found in Appendix F of the ESM . Data collection Patient data were recorded by using either online data collection software -LRB- provided by iMDsoft , Tel Aviv , Israel -RRB- or the SAPS 3 stand-alone database system -LRB- provided by the CCC -RRB- . The latter software used a Microsoft Access database -LRB- Microsoft Corporation , Redmond , WA , USA -RRB- for data storage and needed no Internet connection for data entry . Both systems maintained a variety of plausibility controls to ensure the quality of the recorded data . Each variable was precisely defined before the start of data collection -LRB- see Appendix C of the ESM -RRB- . Detailed definitions of the variables were available to participants in both paper and electronic form . To facilitate plausibility checking , each variable was assigned a probability range , encompassing the range of probable values for that variable . In addition , a range of possible values -LRB- storage range -RRB- for that variable was defined -LRB- e.g. , for FiO2 , no values < 21 % or > 100 % could be accepted -RRB- . Thus , formal plausibility controls in the software systems were used wherever possible and ensured the maximum of data quality checking during data collection . Participants who could not use one of the two software options were allowed to record the data on paper forms and submit them to the CCC -LRB- n = 26 ICUs -RRB- . Patient data were then entered into the SAPS 3 stand-alone software system and thus checked for plausibility . In cases of uncertainty , ICU coordinators were contacted for clarification . In addition , each ICU received a questionnaire with detailed questions about ICU structures and about resources available in other areas of the hospital . Data were collected at ICU admission , on days 1 , 2 and 3 , and on the last day of the ICU stay . Data from the day of admission -LRB- aside from sociodemographic data such as age and sex -RRB- were categorized into different levels : -LRB- i -RRB- data about the condition of the patient before ICU admission , such as chronic conditions and medical diseases ; -LRB- ii -RRB- data about the patient 's condition at ICU admission , such as the reason for admission , infection at admission , and surgical status ; and -LRB- iii -RRB- data about the patient 's physiologic derangement at ICU admission . These data were collected within an hour before or after ICU admission . On the following days of the ICU stay , further information was collected : severity of illness , as measured by the SAPS II -LSB- 5 -RSB- ; number and severity of organ dysfunction , as measured by the Sequential Organ Failure Assessment -LRB- SOFA -RRB- -LSB- 28 -RSB- ; length of ICU and hospital stay ; and outcome data , including vital status at ICU and hospital discharge . All patients were subjected to mandatory follow-up until hospital discharge , but not longer than 90 days after ICU admission . Patients still remaining in the hospital at 90 days were at that time classified as being `` still in the hospital '' . To record diagnoses , a three-level system was used . -LRB- i -RRB- An acute medical disease was recorded for all patients , independent of surgical status , i.e. , the acute -LRB- or acute on chronic -RRB- disease that best explained the ICU admission . If the reason for ICU admission was infectious disease , then this was recorded . -LRB- ii -RRB- Surgical status at admission and the anatomic site of surgery were recorded for all patients undergoing surgery during the hospital stay before ICU admission . -LRB- iii -RRB- A concrete reason for admission had to be selected . At least one reason for admission was required , but several selections were possible -LRB- one within each organ system -RRB- . If no other reason was present , at least `` basic and observational care '' had to be selected . All participants received detailed documentation of patient - and ICU-based data items as well as a detailed description of the data collection process . Moreover , specific forms to check the completeness of the patient-based documentation were provided . Additionally , a training session for ICU coordinators was organised at the 15th Annual Congress of the ESICM in Barcelona , Spain , before the start of data collection . Throughout the project , the project website -LRB- http://www.saps3.org -RRB- provided all necessary information . In addition , the CCC was available to answer questions by email , fax and phone . Data were to be collected from all consecutively admitted patients between 14 October and 15 December 2002 . ICUs with a high number of beds -LRB- and thus also admissions -RRB- could stop patient enrolment after contributing 100 patients . Database Data were collected and pooled by the CCC . The final database file was then imported into the SAS system , Version 8e -LRB- SAS Institute Inc. , Cary , USA -RRB- . Data cleaning was accomplished through the application of a variety of additional plausibility controls and cross-checking of information between redundant data fields . A total of 22,791 admissions were recorded in the 309 participating ICUs during the study period . For patients who were admitted more than once -LRB- n = 1,455 -RRB- , only the first admission was included , giving 21,336 admitted patients . Patients who were < 16 years of age -LRB- n = 628 -RRB- , those without ICU admission or discharge data -LRB- n = 1,074 -RRB- , and those with records that lacked an entry in the field `` ICU outcome '' -LRB- n = 57 -RRB- were excluded . The Basic SAPS 3 Cohort thus comprises 19,577 patients from 307 ICUs . For the development of a predictive model for hospital mortality as outcome , patients with a missing entry in the field of `` vital status at hospital discharge '' -LRB- n = 2,540 -RRB- or an entry of `` still in the hospital '' at the end of the follow-up period -LRB- n = 253 -RRB- were further excluded . The SAPS 3 Hospital Outcome Cohort thus comprises 16,784 patients from 303 ICUs . Because the study was an observational study and no additional interventions were performed , the need for informed consent was waived by the institutional review board . Each ICU , however , was made responsible for obtaining local permissions as necessary . Data quality Recorded data were evaluated for completeness of the documentation and reliability . Interrater quality control was performed through rescoring of the data from day 0 -LRB- the day of ICU admission -RRB- for three randomly selected patients in each ICU . From the rescored data , kappa coefficients and intra-class correlation coefficients were calculated , as appropriate . Availability of the variables necessary to calculate the SAPS II was used as an indicator for the completeness of the data . Statistical analysis Statistical analysis was performed using the SAS system , version 8e -LRB- SAS Institute Inc. , Cary , NC , USA -RRB- . A P value of < 0.05 was considered significant . Unless otherwise specified , results are expressed as median and interquartile ranges -LRB- quartile -RRB- . Observed-to-expected -LRB- O/E -RRB- mortality ratios were calculated by dividing the number of observed deaths per group by the number of expected deaths per group -LRB- as predicted by the SAPS II -RRB- . To test for statistical significance , we calculated 95 % confidence intervals -LRB- CI -RRB- according to the method described by Hosmer and Lemeshow -LSB- 29 -RSB- . The Hosmer-Lemeshow goodness-of-fit Ĥ-statistic and Ĉ-statistic -LSB- 30 -RSB- were used to evaluate the calibration of the SAPS II . Discrimination was tested by measuring the area under the receiver operating characteristic -LRB- aROC -RRB- curve , as described by Hanley and McNeil -LSB- 31 -RSB- . Reliability of data collection was analysed using K-statistics or intra-class correlation coefficients , as appropriate . Statistical methods used for the development of the predictive model are described in Part 2 of this report . Results Data quality Four hundred eighty-three rescored patients could be identified and linked to their original counterparts -LRB- 2.5 % of admitted patients -RRB- . Data quality was found to be excellent , with the majority of coefficients being > 0.85 . Only two of the more than 50 tested variables had coefficients < 0.80 -LRB- body weight , 0.79 ; positive end-expiratory pressure , 0.72 -RRB- , and only one was < 0.70 -LRB- leukocytes -LSB- maximum -RSB- , 0.57 -RRB- . For a detailed list of coefficients see Table E1 in the ESM . Data completeness was also found to be satisfactory , with 1 -LSB- 0 -- 3 -RSB- SAPS II parameter missing per patient . Description of ICUs The Basic SAPS 3 cohort includes 307 ICUs from 35 countries . On average each ICU contributed 50 -LRB- 27 -- 78 -RRB- patients to the cohort . To assess heterogeneity of results between different geographic regions , seven regions were defined : Australasia , Central and South America , Central and Western Europe , Eastern Europe , North America , Northern Europe , and Southern Europe and Mediterranean countries . The allocation of countries to these regions can be seen from Table E10 of the ESM . Seventy percent of the participating ICUs identified themselves as mixed medical-surgical -LRB- Table E2 , ESM -RRB- . Roughly half of the ICUs -LRB- 46 % -RRB- were located in university-affiliated or teaching hospitals . Eighty-four percent of ICUs -LRB- n = 258 -RRB- reported having a full-time medical director , and 272 -LRB- 88.6 % -RRB- reported having a full-time nursing director . On weekdays , 76.6 % of ICUs reported having an intensivist available on the ICU 24 hours per day , whereas 6.2 % had an intensivist available in the hospital . In 12.1 % of ICUs , the intensivist was at home , on-call , during the daytime . During weekends , this proportion did not change much -LRB- 74.3 % , 5.5 % , and 15.0 % on the ICU , in the hospital , and on-call , respectively -RRB- . None of the participating ICUs reported having no intensivists available during night or weekend shifts . Description of patients The Basic SAPS 3 Cohort comprises 19,577 patients admitted to participating ICUs during the study period . More than 70 % of patients were admitted from the same hospital as the ICU , with operating rooms , emergency departments and normal wards contributing most of the patients -LRB- Table 1 -RRB- . Almost two thirds of the admissions were classified as unplanned . The mean age of patients was 60.0 ± 17.7 years -LRB- Fig. 1 -RRB- , and 39.2 % were female . Comorbidities were recorded in 65 % of admitted patients , with arterial hypertension , chronic obstructive pulmonary disease , and chronic heart failure being the most frequent -LRB- Table E3 , ESM -RRB- . Table 1 ICU admission data for the two cohorts -LRB- Basic cohort : SAPS 3 basic cohort ; HO cohort : SAPS 3 Hospital Outcome Cohort ; n : number of patients -RRB- Basic cohortHO cohortn % n % Number of patients19 ,57716,784100.0 Gender Female7 ,67839.26,61039.4 Male11 ,88160.710,16160.5 Missing180 .1130.1 Age , years -LRB- median , quartiles -RRB- 6349-746449-74Origin Home2 ,81014.42,34314.0 Same hospital13 ,92671.112,06371.9 Chronic care facility740 .4640.4 Public place5192 .74322.6 Other hospital2 ,12510.91,79110.7 Other800 .4590.4 Missing430.2320.2Intra-hospital location before ICU admission Emergency room5 ,41927.74,63027.6 Intermediate care unit/High dependency unit5622 .94752.8 Operating room7 ,53738.56,44938.4 Other5522 .84132.5 Other ICU6983 .66113.6 Recovery room4822 .54002.4 Ward3 ,41117.43,03618.1 Missing9164 .77704.6 ICU admission status Planned admission6 ,75034.55,59833.4 Unplanned admission12 ,33863.010,80164.4 Missing4892 .53852.3 Acute Infection at ICU admission No infection15 ,25477.912,96877.3 Clinically improbable/colonization3421 .72981.8 Clinically probable/documented2 ,76114.12,42214.4 Microbiologically documented12066 .21,0836.5 Missing130 .1130.1 Surgical status No surgical procedure8 ,43743.17,30543.5 Scheduled surgery6 ,80034.75,70034.0 Emergency surgery3 ,32117.02,93017.5 Missing1 ,0195.28495.1 Fig. 1 Age distribution and associated mortality . The figure shows the age distribution of the Basic SAPS 3 Cohort -LRB- n = 19,577 -RRB- and the corresponding ICU mortality rates for each stratum . Columns : Number of patients as percentages of the whole cohort ; squares : ICU mortality rates for the corresponding stratum Cardiovascular , respiratory and neurologic diseases were the most frequent organ-specific reasons for admission -LRB- Table E4 , ESM -RRB- . The acute medical diseases necessitating ICU admission included a broad spectrum of diagnoses -LRB- Table E5 , ESM -RRB- . Approximately one half of the patients underwent surgery before ICU admission , with abdominal , cardiac and vascular surgery being the most frequent procedures -LRB- Table E6 , ESM -RRB- . Regarding discharge details -LRB- Table 2 -RRB- , it is notable that a high percentage of patients were discharged unplanned -LRB- 8.15 % -RRB- , i.e. , without at least a 12-hour planning window . 15.2 % of patients from the SAPS 3 Basic cohort died within the ICU . As can be seen from Table 3 , patient cohorts differed significantly between regions . Both , ICU and hospital mortality rates exhibited a broad spectrum between ICUs : hospital mortality was on average 28 % -LRB- 17 -- 42 % -RRB- in the SAPS 3 Hospital outcome cohort.Table 2 ICU discharge and outcome data for the two cohorts -LRB- Basic cohort : SAPS 3 basic cohort ; HO cohort : SAPS 3 Hospital Outcome Cohort ; n : number of patients ; ICU LOS : ICU length of stay ; IMCU/HDU : intermediate care unit/high dependency unit ; Q1 , Q3 : lower and upper interquartile range , respectively -RRB- Basic cohortHO cohortn % n % Number of patients19 ,577100.016,784100.0 ICU LOS , days -LRB- median , quartiles -RRB- 21 -- 621 -- 6ICU discharge -- destination Home4382 .23612.2 Same hospital14 ,94676.312,47774.3 Other hospital1 ,0295.38525.1 Missing3 ,16416.23,09418.4 Intrahospital discharge Emergency room580 .3500.3 IMCU/HDU2 ,22211.41,87311.2 Other3031 .52571.5 Other ICU5833 .04792.9 Recovery room3061 .62181.3 Ward12 ,25062.610,29161.3 Missing3 ,85519.73,61621.5 ICU discharge -- status Planned discharge14 ,87276.012,26273.1 Unplanned discharge1 ,5958.11,4678.7 Missing3 ,11015.93,05518.2 Risk adjustment SAPS II score -LRB- median , Q1 -- Q3 -RRB- 3020 -- 423121 -- 43 SOFA score -LRB- median , Q1 -- Q3 -RRB- 96 -- 1196 -- 11Outcome ICU mortality -LRB- % -RRB- 15.217.7 Table 3 ICU admission and discharge data for the seven defined geographic regions -LRB- SAPS 3 Basic Cohort ; n = 19,577 -RRB- AustralasiaCentral & South AmericaEastern EuropeCentral and Western EuropeNorthern EuropeSouthern Europe and Mediterranean countriesNorth AmericaNumber of patients2 ,235 2,540 1,084 4,712 355 7,854 797 Females , % 38.0 44.6 42.2 40.1 42.5 36.9 38.0 Age , years -LRB- median , quartiles -RRB- 5945 -- 713146 -- 746248 -- 726552 -- 756651 -- 766459 -- 746449 -- 75SAPS II score -LRB- median , Q1-Q3 -RRB- 2819 -- 403020 -- 422718 -- 432921 -- 403525 -- 463222 -- 442920 -- 39SOFA score -LRB- median , Q1-Q3 -RRB- 86 -- 1086 -- 1197 -- 1185 -- 1197 -- 1197 -- 1196 -- 11ICU mortality , % 12.7 17.4 16.9 10.8 20.6 18.1 8.5 Performance of the SAPS II The performance of the original SAPS II model -LSB- 5 -RSB- -LRB- using data from the first 24 hours -RRB- was tested in the SAPS 3 Hospital Outcome Cohort -LRB- n = 16,784 -RRB- . Discrimination was good with an aROC of 0.83 -LRB- 95 % CI : 0.824 -- 0.838 -RRB- . SAPS II showed underestimation of hospital mortality : The O/E ratio of the overall cohort was 1.08 -LRB- 1.06 -- 1.10 -RRB- . O/E ratios significantly differed between regions : from 0.86 -LRB- 0.81 -- 0.91 -RRB- for Central and Western Europe to 1.32 -LRB- 1.25 -- 1.38 -RRB- for Central and South America , with four out of the seven defined regions exhibited O/E ratios significantly different from 1 -LRB- Table E7 , ESM -RRB- . Calibration , as assessed by the Hosmer-Lemeshow Ĥ + Ĉ statistics , was poor for the overall cohort : Ĥ 227.21 , Ĉ 184.70 ; both p < 0.0001 ; This lack of calibration was present for all tested subgroups except for the region of North America -LRB- see Table E7 , ESM -RRB- . Discussion To the best of our knowledge , the SAPS 3 study is the largest prospective epidemiologic multicentre , multinational study conducted in health services and outcomes research in intensive care medicine to date . The project was first intended to focus on Europe because it was believed such a strategy would produce a more homogeneous cohort of patients , which would in turn provide a more stable reference line for further comparisons . This idea was discussed during several investigator meetings and finally abandoned -- first , because interest from outside Europe was enormous : 39 % of ICUs that registered for the project were located outside Europe . The SAPS 3 board members thus agreed that such a high level of interest should not be ignored . Second , some investigators questioned whether a concentration on European ICUs would be successful in reducing heterogeneity anyway . Provision of intensive care in Europe is extremely variable , with enormous differences in severity of illness , provision of treatments and mortality from north to south and from west to east -LSB- 32 , 33 -RSB- . For these reasons ICUs from regions outside Europe were invited to participate . Our results prove that we were right in our assumptions : First , one can easily see that the four European regions -LRB- as defined in our study -RRB- are hardly comparable : severity of illness as measured by the SAPS II varied from 27 to 35 points , and ICU mortality ranged from 10.8 to 20.6 % -- almost a doubling of mortality figures -LRB- Table 3 -RRB- . Second , almost a third of the patient cohort -LRB- 28.5 % -RRB- was contributed from regions outside Europe . Although the decision to accept ICUs worldwide probably increased the heterogeneity of our sample , it also allowed the SAPS 3 database to better reflect important differences in patients ' and health care systems ' baseline characteristics that are known to affect outcome . These include , for example , different genetic makeups , different styles of living or a heterogeneous distribution of major diseases within different regions , as well as issues such as access to the health care system in general and to intensive care in particular , or differences in availability and use of major diagnostic and therapeutic measures within the ICUs -LSB- 32 , 34 -RSB- . Although the integration of ICUs outside Europe and the U.S. surely increased it 's representativeness , it must be acknowledged , that the extent to which the SAPS 3 database reflects case-mix on ICUs worldwide can not be determined yet . It should additionally be noted that allocation of countries to regions does not always follow geographic borders -LRB- Table 3 ; see also Table E10 in the ESM -RRB- . Partitioning of the sample was done to adjust for some of the above-stated differences between different populations and to develop a system that uses several different reference lines to compare ICUs on a similar level . Thus , patients are not necessarily representative of their respective regions . To minimize possible seasonal influences , we chose late fall in the Northern Hemisphere for data collection . Thus , participants in both late fall/winter -LRB- Northern Hemisphere -RRB- and spring/summer -LRB- Southern Hemisphere -RRB- are represented in our cohort . A recent study -LSB- 35 -RSB- showed , moreover , that differences in seasonal mortality rates , at least in a sample of ICUs in the United Kingdom , were related to variations in case mix rather than to a specific impact of season on outcome . Performance of the SAPS II was , not surprisingly , found to be similar to that in previous studies : acceptable discrimination but lack of calibration . Possible reasons for this have already been alluded to in the Introduction . In contrast to previous studies , however , we found an underestimation of hospital mortality , which contradicts the rationale that the shifting in calibration is due only to the development of new and possibly better therapies and thus to better ICU performance -LSB- 19 -RSB- . Analyzing the various geographic regions provides evidence that the underestimation of hospital mortality by the SAPS II might be partially attributable to the composition of the cohort : SAPS 3 is the first large international study on severity of illness systems to include patients from all continents . South America , for example , where provision of intensive care is much more limited than it is in Europe or North America , contributed extensively to the patient cohort . High O/E ratios have already been reported for this continent -LSB- 36 -RSB- and are probably linked to the limited availability of resources . Data quality was one of our major concerns . Completeness of the documentation was found to be satisfactory : The amount of missing data is in fact smaller than reported from previous cohort studies on severity of illness systems -LSB- 11 , 12 , 16 -RSB- . With respect to reliability , intraclass-correlation coefficients and kappa coefficients were generally similar to or even better than those found in previous studies , showing a high degree of interrater agreement -LRB- see Table E1 in ESM -RRB- -LSB- 37 , 38 -RSB- . We did , however , experience problems with the cohort of rescored patients : First , we had to exclude all rescored patients for whom the original counterpart was also excluded due to the application of any of the exclusion criteria . Second , in some cases the original patient identification was either missing or documented in such a way that a unique allocation was not possible . Both of these exclusions reduced the number of rescored patients available for analysis . Two strategies to build up a cohort are available : first , to recruit only patients who meet well-documented inclusion criteria -LRB- such as documented vital status at hospital discharge -RRB- or , second , to document all patients and then exclude patients based on a predefined set of exclusion criteria . For the SAPS 3 study we chose the second option -- to form two different cohorts -- because we needed to provide a basic cohort for all further analyses of the SAPS 3 database . Since some studies will focus on different outcomes -LRB- e.g. , ICU outcome rather than hospital outcome -RRB- , we decided to use missing ICU outcome -LRB- and not hospital outcome -RRB- as an exclusion criterion for the basic cohort . A possible limitation of the SAPS 3 database is that vital status at hospital discharge was not available for all admitted patients . Despite several efforts from the CCC and sufficient time to allow for a close follow-up , we did not succeed to receive all hospital outcomes documented . Recording of hospital outcome -LRB- or later outcomes -RRB- still poses major problems for ICUs in European and non-European hospitals , either because of technical problems or possibly because of data security algorithms in the hospitals . Exclusion of these patients did , however , not affect major criteria , such as geographic representation , ICU admission or discharge data , co-morbidities , or the distribution of the reasons for admission -LRB- Tables 1 and 2 -RRB- . We conclude that the SAPS 3 database is within the above discussed limits of high quality and reflects the heterogeneity of current intensive care provision . As such , it provides an excellent basis for the development of a new risk adjustment system . Electronic Supplementary Material -LRB- PDF 794 KB -RRB-

O O O O O O O O O O O O B I I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O

intensive care unit
severity of illness
icu mortality
hospital mortality
risk adjustment