ESRI Research Note No.33

Measuring Health Care Output
Shigeru Sugihara, Koichi Kawabuchi,
Yasuko Ikemoto, and Ikumi Imamura

July 2017

Economic and Social Research InstituteCabinet Office
Tokyo, JapanThe views expressed in “ESRI Research Note” are those of the authors and not those of
the Economic and Social Research Institute, the Cabinet Office, or the Government of Japan.
(Contact us: https://form.cao.go.jp/esri/en_opinion-0002.html)

ESRI Research Note No.33"Measuring Health Care Output"

Measuring Health Care Output1

Shigeru Sugihara 2
Koichi Kawabuchi3
Yasuko Ikemoto 4
Ikumi Imamura 5

July 2017

This paper was originally presented at the ESRI seminor held on 30 March, 2011. Achange of positions for one of the authors just after the meeting resulted in the delay in
its revision.
2 Social and Economic Research Institute, Cabinet Office.
3 Tokyo Medical and Dental University.
4 Social and Economic Research Institute, Cabinet Office.
5 Northern Territories Affairs Administration, Cabinet Office.
1

1

ESRI Research Note No.33"Measuring Health Care Output"

Abstract
This paper explores the possibilities of measuring health care output adjusting for
quality of care. We explore methods to incorporate quality of health care. First of theseadjusts for quality using crude mortality and complication rates. Adjustment of crude
rates can be misleading, however, because risk factors change from year to year andchance variation may dominate yearly fluctuations in mortality and complication rates.
Hence, second set of indexes incorporates risk-adjusted mortality and complication rates.
In controlling for chance variations, we compared three estimation methods which may
differ in their ability to control variability of estimates: maximum likelihood, thehierarchical model and autoregressive (AR) restrictions on random effects. Further, toexamine how much information is required for reasonable risk adjustment, we
compared three risk adjustment models with differing degrees of detailed risk factors.
Overall conclusion of the paper is that we do need adjustment of quality of care in theconstruction of output index of health care. Risk adjustment is also of crucial
importance although the choice of method of estimation to control for chance variation
may be less crucial. As for the risk factors, the more detailed, the better.

2

ESRI Research Note No.33"Measuring Health Care Output"

1. Introduction
Measurement of output and deflators in non-market service sectors such as health careis problematic because in these sectors prices are not determined by competitivemarkets. Often, governments try to correct market mechanisms in these areas throughregulation, public insurance, and other measures. Even when governments do notintervene, prices determined in the market do not reflect true consumer preferences
thanks to asymmetric information and externalities. Without market prices, proper
deflators to use in the calculation of output are hard to obtain.
Recently direct measures of output of non-market service sectors are actively
studied and advocated. The Atkinson Review (2005) recommends to measure output
directly by counting the number of units for whom services are provided instead of
measuring output by aggregating costs of producing the services. In addition, theAtkinson Review (2005) encourages that output is adjusted for the change in quality ofservices. Eurostat (2001) also recommends direct measurement and quality adjustment.
In the United Kingdom, the Office for National Statistics calculates and publishes directand quality-adjusted output indexes for public sector activities. (Office for National
Statistics, 2007, 2008, 2015.) These are based on work done by U.K. researchers(Dawson, et al., 2005). German researchers are investigating how to utilize the DRG
system in the direct measurement of health care output (Pierdzioch, 2008). In the UnitedStates, pioneering research has been conducted during the 1990s, particularly at the
National Bureau of Economic Research, measuring the quality of health care andadjusting for quality change in the calculation of deflators (Cutler and Berndt,eds. 2001).
The Bureau of Economic Analysis is now developing a Health Care Satellite Accountbased on treatments of diseases (Dun, et al., 2015).
A major trend in the practice of National Accounts especially in EU countries is, as ismentioned above, activity-based output index. Activity-based output index measures
health care activities such as the number of patients treated or operations performed, andso on. Simple activity-based measure is not appropriate because it is assumed that themore activities are appended, the better. Furthermore, activity-based measure is notunlike input.
A refinement may be to use the DRG system to account for quality of care. This is valid
as long as the classification of the DRG system corresponds to quality of care deliveredby hospitals. Quality adjustment by the DRG system is not adequate, however. Because
the classification typically depends on operations and major procedures, thanks to the
choice of treatment on the part of hospitals, not all the patients in the same DRG have3

ESRI Research Note No.33"Measuring Health Care Output"

the same severity and, conversely, patients with the same severity do not necessarily
belong to the same DRG. Further, output index based on the numbers of patients of
DRG classifications could have adverse effects. For example, simply increasing thenumber of patients by way of “three-minute consultation” will raise the measured
output although the quality of care could decline.
Hence, search for more direct methods to adjust for the quality of care are
warranted.
In sum, there are two ways to adjust for quality of care. One is activity-based output
index, which tries to control quality of care through disaggregation, where
disaggregation of health care activities into homogenous activities is assumed toguarantee uniform quality of care within each activity. The other is quality-adjusted
output index, which tries to control quality by utilizing explicit measures of quality ofcare. The goal of these two indexes is the same: adjustment for the quality of care.
There is a trade-off between the robustness and refinement of adjustment. Quality
adjustment through disaggregation is robust because it does not depend on specificmodels, while its resultant adjustment is not so complete because the criteria ofclassification is relatively crude. Quality-adjusted indexe can utilize a rich set of risk
factors which affect patients’ outcomes so that its quality adjustment can be refined,
while it is not so robust because of its dependency on models used to adjust for risk
factors.
This paper first gives examples of construction of activity-based output index covering
all diseases. Next, we try to construct output indexes which reflect quality of care. Dueto data limitations, we restrict our construction to health care of hospitalized AMI
patients. Qualities we consider are mortality and complications. In adjusting for
mortality and complications, we first use crude rates, then, we adjust for risk factors ofpatients.
As for adjustment using crude mortality and complication rates, we compile two
indexes. The first one adjusts only for mortality by simply putting utility at zero if apatient dies during hospitalization. In this case, all the patients who are discharged alive
are assumed to have the same health utility as the healthy people.
Next, we will try to attach utilities to the case where a patient is discharged alive.
We do not have data on the quality of life when a patient survives. We have data oncomplications, however. We follow Timbie, et al. (2009:Composite Measures paper) inassigning health utilities to individual complications and, then, infer utilities according

4

ESRI Research Note No.33"Measuring Health Care Output"

as they have the specified complications.
Since risk factors change from year to year, proper risk adjustment is essential. Without
controlling for such changes in underlying risk factors, output indexes adjusted for
quality of care can be misleading with too much or too little adjustment.
There is no lack of controversies concerning risk adjustment methods. This paper
specifically deals with two questions. One is whether it is possible to properly adjust forrisk factors despite wild random variation of outcomes in the context of small sample
size. The other is how detailed the risk adjustment model should be in view of costly
information gathering.
As to the first question, we investigate how statistical methods can cope withchance variation of outcomes by comparing methods differing in their ability to control
chance variation.
We consider three estimation methods of a risk adjustment model: the MLE(maximum likelihood estimates), the hierarchical model and AR regression. The second
method, the hierarchical model, is especially suitable for controlling chance variation byshrinking individual estimates toward overall mean. The third method imposes
restrictions on the rate that quality of care can vary.
As to the second question posed above, we estimated several risk models withvarious degrees of detailed information. If they imply different mortalities andcomplications after adjustment, modeling risk factors is of crucial importance.
In sum, we measure health output by adjusting, first, for quality of care and, second, forrisk factors. The process and ingredients are summarized as follows.

Quantity+

Quality adjustment+

Risk adjustment

Mortality

MLE

QOL (Complications)

Hierarchical model
AR regression

The paper is structured as follows. The second section explains the general frameworkfor output indexes which adjust for quality of care. The third and fourth sections
incorporate quality of care into the output indexes, of which the former calculates output
indexes adjusted for crude mortality and complications while the latter adjusts risk

5

ESRI Research Note No.33"Measuring Health Care Output"

factors in measuring the quality of care. The fifth section concludes.

2. A Digression on the Concept of the Quality of Health Care
Output of health care is the improvement of health caused by medical interventions.
Therefore, output index should include not only the number of patients but also theimprovement of health status attributable to health care. This is what quality-adjusted
output index is intended to do.
The concept of the quality of health care can be explained as follows. This exposition is
inspired by the discussion in Jacobs, et al. (2006) although much modified. Let theoriginal health status of a patient at time t be htO =0.5. Suppose that when she undergoes
a treatment, her health status will be htT =0.7 and that when she does not undergo atreatment, her health status will be htC =0.4. The true quality of care is htT − htC =0.3
while we can observe only htT − htO =0.2 because we usually do not know the natural
history of disease htC .

The Concept of Quality of Health Care Services
Health status

htT
htO
htC

Beforeintervention

Afterintervention

6

ESRI Research Note No.33"Measuring Health Care Output"

Now suppose that in the next year we have htO+1 =0.5, htT+1 =0.8 and htC+1 =0.4. Namely,
the original health status and the natural history are the same as time t. Then, the qualityof health care at time t+1 is htT+1 − htC+1 =0.4 while we observe only htT+1 − htO+1 =0.3.
However, since htO+1 = htO and htC+1 = htC , we can calculate the change in the true
quality of health care as ( htT+1 − htO+1 ) − ( htT − htO ) = htT+1 − htT .
It is not necessarily the case that the original statuses, htO and htO+1 , are equal.
Hence, we have to adjust the original statuses in order to compare outcomes, htT and

htT+1 . Risk adjustment just does this. Somewhat formally, we can model the health statusof the treated patient as a function of the original status: htT = f t ( htO ) andhtT+1 = f t +1 ( htO+1 ) . Adjusted health statuses with a common original status, h O , are
htT = f t ( h O ) and htT+1 = f t +1 ( h O ) . Then, the difference between these two outcomes isthe change in quality of health care.

3. Activity-Based Output Indexes
In this section we calculate activity-based indexes with two kinds of units of measure,the ICD and the DPC. These measures cover all diseases. Data used in the constructionof activity-based output indexes are taken from public statistics, the Patient Survey andthe Analysis and Evaluation of the Effects of the Introduction of the DPC System,
which present aggregate data, while the construction in the fifth section of the quality
adjusted output index utilizes data collected by authors which consist of data on
individual patients. This is because we are not able to classify each patient contained in
the latter data set into DPC categories, hence, calculation based on the DPC
classification is impossible in the latter data set.
By disaggregating unit of measurement, a homogenous classification of activities
may obtain and, hence, quality of output could be accounted for by such detailed

7

ESRI Research Note No.33"Measuring Health Care Output"classification. The assumption is that activities directed to a specific classification are of
the same quality and that activities directed to different activities are of different quality.
Its success depends on, of course, how the classification is done. We will return to thispoint later.
These indexes are cost-weighted. The quantity of output is weighted by cost assumingthat cost is proportional to the marginal social value of output (Castelli, et al., 2007).

∑ x ⋅c,=x⋅c∑i ,t +1

I

x
t +1

it

i

it

it

i

where xit is the quantity of i-th output at time t and cit is unit (average) cost of i-th
output at time t.
(1) ICD-based unit of measure
Here, activity is defined relative to the ICD diseases. Health care activities devoted to a
specific disease are compounded into an activity. It is assumed that quality of healthcare is the same within a specific disease and different among different diseases. Surely,
this assumption is hard to justify. As a reference, however, we calculate output
according to this definition. This index is for inpatient services.
Diseases disaggregated to the block level diseases (three digits classification with
one alphabet and two numbers) in the ICD system. The data source is the Patient Survey
conducted every three years. A finer classification of diseases is available, but wecannot find corresponding costs form the source sited below.
The index is cost-weighted. Cost is the average charge for each ICD disease which istaken from the Survey of Medical Care Activities in Public Health Insurance.
We were able to compile a consistent classification going back from 2008 to 1984.
The classification before 1984 is so different that we cannot connect it to the current
classification. Further, the 1984 Survey of Medical Care Activities did not include
patients covered by the National Health Insurance System. Therefore, we constructed
output index from 1987 onward.
The result is shown in Figure 1. Output of inpatient services increased sharply from1987 to 1990. After that, inpatient output followed a declining trend with especially
rapid decrease during the 2000s.

8ESRI Research Note No.33"Measuring Health Care Output"(2) DPC-based unit of measure
The Diagnosis Procedure Combination (DPC) system was introduced in 2003 as aprospective payment system for acute care of patients treated by the Specific Function
Hospitals. Thereafter, the DPC system has been expanded to include other eligible
hospitals. As of July 2010, the DPC system covers 1,391 hospitals and around 460,000
beds, which account for 50.4% of total beds.
The classification of patients starts with the diagnosis which absorbed resources themost among their diagnoses. Patients are further classified by whether specified
operations are performed or not. Then, the final classification is reached according as
whether the patient has comorbidities or not.
The DPC system is intended for use in a Prospective Payment System. But it retains
characteristics of fee-for-service. For example, payments are per diem, not for the whole
hospitalization episode, and the system does not apply to operations and some othercostly procedures. Therefore, it provides incentives to reduce LOS as well as incentives
to increase operations.
To calculate output index based on the DPC classification the number of patients ofeach DPC is aggregated with the average length of stay of each DPC as weight. In
theory, we should use billing rates as weights. But, since some DPC categories are
reimbursed on the fee-for-service basis, billing rates are not available for thesecategories in the published data. Therefore, we used length of stay as a proxy for cost. It
is well documented that the correlation between length of stay and cost is high.
The DPC classification is revised every two years. It is not possible to re-classify
patients retrospectively without patient-level data. Every year’s publication of statistics
provides data for two years, the current and the previous years, from which we
calculated output indexes for two years. Then, we linked these indexes at theoverlapping year.
Since the number of hospitals covered by the DPC system is being expanded overtime, we cannot simply aggregate the output of hospitals in each year. We restrict thecalculation to hospitals which started their participation in 2003, 2004 and 2005. We canobtain data on these hospitals from 2005 through 2009. These hospitals are only asubset of all hospitals and, clearly, early participants in the DPC system have different
characteristics from other hospitals. They are large and high-technology-oriented
hospitals, in general.
Therefore, the DPC-based output index we constructed is not representative for theentire health care system. However, by comparing the DPC-based index to the raw

9ESRI Research Note No.33"Measuring Health Care Output"

number of patients, we can obtain some insight into the extent to which quality
adjustment using DPCs as unit of measurement has impact on output index.
Figure 2 shows the growth rates of the output index based on the DPC classification
together with the raw number of patients. Overall trend in the output index is similar to
the number of patients, but there is a noticeable difference between the two. The
difference translates into around 1 % difference in the growth rates in 2006 and 2007.
The DPC classification is meant to be used as a payment system. It is intended to group
together diseases with homogenous costs, not disease with homogenous health care
activities.
Hence, the classification is heavily dependent on operations performed. This isreasonable from the point of view of the original intention of grouping diseases withhomogenous cost. Further, it could be that a specific treatment represents a specificquality of care as in the case of high-technology treatments.
Quality adjustment by the DPC system is not adequate, however.
First, outcomes such as mortality which are expected from specified operations orprocedures differ among hospitals and doctors. The same operations and procedures do
not always represent the same quality in different hospitals.
Second, the DPC classification is based on the medical procedures selected byhealth care providers. Inclusion of choice variables into classification criteria may resultin problems as statistics. Increase in inappropriate but costly use of medical procedures,such as PCIs for stable coronary patients or low back pain surgeries, will increase theoutput of health care!
Third, the level of quality of each DPC category is fixed at the beginning. As
medical technologies progress, better outcomes are expected to obtain. The
measurement based on the DPC system cannot take into account such improvements ofoutcomes over time.
Fourth, while the DPC system is intended to create homogeneous categories withrespect to cost, cost does not necessarily reflect relative levels of quality of DPCs.
Costly operations and procedures do not always result in superior outcomes. It is only
after we validate the outcomes of individual operations and procedures that we canproperly calibrate relative value of each DPC.
Fifth, introduction of output index based on the DPC system could have adverse
effects. For example, “three-minute consultation”, for which the Japanese health care

10

ESRI Research Note No.33"Measuring Health Care Output"

system is notorious, will increase measured output by treating more patients within agiven time. “The sooner, the sicker” phenomenon, observed in the United States when
the DRG system was introduced, is also a cause for concern. The DPC system imposesa very strong incentive to shorten the length of stay. If shorter length of stay is dictatedby economic incentives, there is no guarantee that we have higher quality of care, but
again the measured output will rise. Last example to mention is selection bias caused by
the gap between the classification and payment systems. Since operations are notincluded in the DPC payment system, hospitals may be eager to increase the number ofoperations. Since the DPC classification is dependent on operations, increased operation
will result in increase in health care output.

4. General Framework of Quality Adjusted Output Index
We follow Dawson, et al. (2005) and Castelli, et al. (2007) in the construction of quality
adjusted output index. Output consists of three components: unit of measurement,
quality of care and valuation of the quality. Schematically, we can write:＝

Output×

Unit of measurement

Quality of care×

Valuation

In terms of equation, the above scheme is expressed as follows.

∑ x ∑q π,=∑ x ∑q π
i , t +1

I

x
t +1

i

ij , t +1

jt

j

it

i

ijt

jt

j

where xit is the quantity of i-th output at time t, qijt is the quantity of j-th attribute ofi-th output at time t and π it is the value of j-th attribute of i-th output at time t.
In this paper, we use data on hospitalized AMI patients, hence, i = AMI. In thiscase, the above formula simplifies to

I

x
t +1

=

x AMI ,t +1 ∑ q AMI , j ,t +1π jt
jx AMI ,t ∑ q AMI , j ,tπ jtj11ESRI Research Note No.33"Measuring Health Care Output"

Unit of measurement is episode of inpatient care. Although cooperation between
hospital care and primary care is important factor determining total quality of care, nodata are available to evaluate the care process as a whole.
The attributes we will consider include: death, survival with complications andsurvival without complications. Health utilities we assign are: π =0 for death, π =0.9
for survival without complications and π =0.7 for survival with complications. These
numbers will be explained later.
Note that we assigned health utilities to attributes (death and complications).
Health utilities are not, strictly speaking, the value of the quality of life in terms ofmoney. In view of the difficulty in evaluating the value of life, however, we avoid
assigning monetary valuation of quality of care.
In this section, we construct two indexes adjusting for mortality and complication rates.
The first one adjusts only for mortality and the second incorporates quality of life ofsurvivors by adjusting for complications.
First, we adjust only for mortality by simply putting utility at zero if a patient dies
during hospitalization. In this case, all the patients who survived are assumed to havethe same utility as the healthy people, namely π =1.
Next, we will try to attach utilities to the case where a patient is discharged alive.
In the case of output as discharge alive, all the survivors are counted as 1. However,survived patients have different quality of life. Output measure should incorporate thisdifference, although difficult.
We do not have data on the quality of life when a patient survives. We have data oncomplications, however. One important factor that affects quality of life iscomplications during hospitalization. We follow Timbie, et al. (2009) in assigning
utilities to individual complications and, then, infer utilities according as patients havethe specified complications.
Since risk factors change from year to year, proper risk adjustment is needed. Risk
adjustment is done by estimating a logistic regression model to measure the influence ofrisk factors on mortality.
We compare three methods of estimation of the logistic regression model. The firstis maximum likelihood, which is a standard estimation method in statistics. It is pointed
out, however, that when estimating random effects as we will do in this paper, theirestimates tend to take on extreme values because of small sample variability. Therefore,

12

ESRI Research Note No.33"Measuring Health Care Output"

some shrinkage is desired by “borrowing” information from other years and shrinking
their estimates toward the overall mean. The second approach does just this.
The second estimation method adopts the hierarchical approach to achieveshrinkage. This approach assumes that random effects are exchangeable so that theycome from a probability distribution with common mean and variance.
The difference between the first and the second methods is succinctly explained asfollows (Spiegelhalter, et al., 2004). Suppose that random variables ct ’s come from anormal distribution ct ~ N (0,σ 2 ) . MLE corresponds to the case where σ 2 = ∞ while
the hierarchical model corresponds to the case where 0 < σ 2 < ∞ .
The third method imposes autoregressive restrictions on random effects. In thismethod, a year effects is related to the previous year in the spirit of autoregressivemodels. Future variability is constrained by the realization of previous year’s random
effect and this year’s random errors.
We re-transform the linear predictor in the logistic regression back to the probability
scale for individuals. Then, we average across all patients within each year to obtain thepredicted outcome. To adjust for case mix differences across years, we follow Timbie, etal. (2009:Cost-Effectiveness paper) who adopted indirect standardization. We estimate
counterfactual outcomes for each year assuming underlying quality levels of the entire
population while conditioning on each year’s case mix. We take the difference between
this expected outcome and the predicted outcome to yield an excess mortality for eachyear.
Concrete steps of indirect adjustment are the following. Patient mix (distribution ofrisk factors) is fixed at actual mix in each year for both predicted and expected
outcomes. We compare mortality rates of the following two cases for each year.
Outcome 1 uses realized quality of care with the relationship between risk factors andoutcome being actual one for each year. Outcome 2 uses average quality of care with the
hypothetical relationship between risk factors and outcome being estimated by
supposing that each year’s quality of care is the same as the total year. Then, excess
mortality is calculated as the difference between two outcomes.

13

ESRI Research Note No.33"Measuring Health Care Output"

Figure 2

Indirect Standardization

Quality of carein a specific year

Outcome 1β̂ t
Excess mortality

Risk factors
in each year

βt

Outcome 2

Average quality of care

5. Adjusting for the Quality of Health Care I: Crude Mortality and Complication(1) Data and Basic statistics
We take as unit of measurement episode of hospitalization of AMI patient. Figure 3
depicts long-term trend of hospitalization of AMI patients in the Patient Survey. The
number of AMI patients continued to decline after it peaked in 1990 at a little above 9
thousands. In 2008, the number of hospitalization is a little below 5 thousands, nearly
half the level in 1990.
Data were collected on AMI patients in 9 hospitals with a record of hospitalization atsome period of time from April 1, 2004 to March 31, 2007 These hospitals had agreed
to cooperate in the research for the consecutive years upon approval of in-hospital
ethical committee.
We created structured questionnaires for data collection. Questionnaire I asked for
detailed clinical information on the patient as well as information on the treatment thepatient received. Claim data and physicians profile were collected by Questionnaire II.
Questionnaire III collected overall information on AMI treatment at the hospital, such as
the annual total number of CABG conducted. A part-time lecturer with physician’s
license in Thoracic-Cardiovascular Surgery Section of Tokyo Medical and Dental
University stayed throughout the research to fill Questionnaire I from patient medical
records including nursing records and discharge summary at each hospital.

14

ESRI Research Note No.33"Measuring Health Care Output"

Questionnaire II and III were filled by hospital staffs who were approved of the accessto claim data at each hospital.
Sample is restricted to ST-Elevation AMI in the following analyses. This choice isintended to secure homogeneity in the sample as is epitomized by the separate
compilation of ACC/AHA guidelines for the management of patients with ST-Elevation
myocardial infarction from those for the management of patients with
Non-ST-Elevation myocardial infarction.
A data set of patients hospitalized in nine hospitals was constructed. Only thepatients hospitalized in hospitals with more than ten STEMI patients in every year are
retained in the analysis. Observations are 2631 in total, of which 598 are in 2004, 612 in
2005, 672 in 2006 and 749 in 2007.
Table 1 shows basic statistics of patients for all hospitals. The upper panel contains
outcome variables: mortality and complications. The average mortality rate is 10.6%
with an upward trend from 2004 to 2007. The average complications rate is 18.4% alsowith an upward trend. A vast majority of complications is repeat revascularization.
Around 5% of survived patients experience complications such as cardiovasculardisorder, renal failure and new infarction. The lower panel of Table 1 exhibits basic
statistics of risk factors. The Average age is 68.9 years old and a little less than a third
patients is female. About a half of patients are in the Killip class 1, a quarter in the class
2 and a little less than 15% in classes 3 and further 14% in the class 4. Occlusion of theleft main trunk, left bundle branch block and ventricular fibrillation account for around
4 to 6% of patients, respectively. More than a half of patients are with hypertension anda little less than 40% and a little more than a third are with hyperlipidemia and diabetes
mellitus. 8% of patients suffer from heart failure and 10% from renal failure. The shareof patients with cancer is 8%.
Table 2 exhibits characteristics of sample hospitals. Three out of nine hospitals aredesignated as tertiary critical care hospitals and all except one hospitals are designatedas teaching hospitals. The average number of beds is 434. Hospitals in the sample arelarge in general, but the size varies. One hospital holds nearly 1000 beds while two
hospitals have less than 200 beds. The average number of AMI patients is 86, but thevariation is large. Two hospitals admitted more than 150 AMI patients while two
hospitals admitted only around 20. The average number of PCI performed is 297, which
is a large number in the Japanese standard. Again, there is a great variation amonghospitals. A hospital performed more than 700 PCI while two hospitals performed only
a little more than 100 PCI.

15

ESRI Research Note No.33"Measuring Health Care Output"

Figure 4 shows mortality and complication rates during the sample period.
In-hospital mortality increased from 9.2% in 2004 to 12.2% in 2006, then, slightly
dropped to 10.8% in 2007. Complication rate is on the upward trend during this period.
The complications we use in this paper will be listed below.
(2) Adjusting for crude mortality and complication rates
Two measures are calculated which accounts for quality change. One incorporates onlychanges in crude mortality. The other adjusts not only for mortality but also forcomplication rate.
In the case of the first output index which adjusts only for mortality, we put the value of
utility π =0 if a patient dies during hospitalization and π =1 if a patient is discharged
alive. In this case, no distinction is made between being discharged alive but withcomplication and being discharged alive without complication.
In the case of the second output index which adjusts not only for mortality but also forcomplications, the assumed values of utility are as follows (Table 3). We put π =0 if apatient dies during hospitalization. When she survives, we put π =0.9 withoutcomplication and π =0.7 with complication. The value of survival withoutcomplications is taken from Weintraub, et al. (2008). The value of survival withcomplications is calculated as follows.
In the adjustment of output for quality of care, complications we use include
myocardial infarction, stroke, cardiovascular diseases, renal failure, repeat
PCI/emergency CABG and cardiac arrest or shock within 48 hours. Timbie, et al.
(2009:Composite Measures paper) provide utility estimates for stroke, renal failure andrepeat PCI/emergency CABG. We were unable to find good estimates for the quality oflife (QOL) for cardiovascular diseases and cardiac arrest or shock within 48 hours,
however. We then proceed in two steps. First, we calculate weighted average of utilities
of complications using only stroke, renal failure and repeat PCI/emergency CABG.
Weights are the number of patients of each complication in the sample while utility
estimates are those of Timbie, et al. (2009:Composite Measures paper). The resultis π =0.77. Second, we adjust the estimate downward a bit to π =0.7 considering that theremaining complications, cardiovascular diseases and cardiac arrest or shock within 48
hours, appear to be serious ones. The downward adjustment is rather arbitrary, it should
be admitted. Caution must be exerted in interpreting the results. In the future, we will
improve on the utility estimates of complications to obtain more accurate quality

16

ESRI Research Note No.33"Measuring Health Care Output"

adjustment.
Figure 5 shows growth rates of output indexes adjusted for crude mortality andcomplications. The impact of adjustment is substantial, especially in 2006. In 2006,
output declined by nearly 4 % without adjustment for mortality while after adjusting for
crude mortality, it declined more than 6 %. The difference between growth of output
with and without complication adjustment is small. To conclude that adjustment ofcomplications is not essential is premature, though. As is noted above, utility estimatesof complications are far from perfect. We should investigate further to answer thequestion whether complication adjustment is required or not.
It could be very important to adjust for mortality, at least. But this conclusion canbe premature again because change in mortality reflects not only underlying change inthe quality of care, but also change in risk factors and chance variation of mortality. The
next section will deal with this problem.

6. Adjusting for the Quality of Health Care II: Risk Adjustment(1) Methods
We follow the method taken by Timbie, et al. (2008:Cost-Effectiveness paper). We
created a measure of disease severity, severity index, for each patient. A logistic
regression was used to model the effect of demographic and clinical risk factors of
in-hospital mortality. Risk factors are selected by checking statistical significance andsigns of estimated coefficients. Risk factors include age, female, Killip classes 2, 3 and4, occlusion of the left main trunk, left bundle branch block (LBBB), ventricularfibrillation, hypertension, hyperlipidemia, diabetes mellitus, heart failure, history of
myocardial Infarction, history of PCI, history of CABG, cancer, bleeding tendency, renal
failure, cerebrovascular diseases, aneurysm and Chronic Obstructive Pulmonary Disease(COPD). The adopted risk factors are not far from those proposed in Krumholz, et al.
(2006: Administrative Claims Model paper) on which Timbie, et al. (2008) base theirconstruction of severity variable. Estimation result is shown in Appendix Table A1.
Severity index is estimated as a linear predictor using the coefficients from theestimated logistic regression:Pseverityit = ∑ βˆ p ⋅ xitp ,
p =1

17

ESRI Research Note No.33"Measuring Health Care Output"

where xitp denotes p-th covariate of i-th patient at time t. Age is centered at the sample
mean.

(i)
MLE
To adjust for risk factors, we estimated logistic regression models with outcomes as
dependent variables. The outcome variable, yit , takes the value one if a patient i intime t dies and zero if she survives. In this subsection, we estimated this model by the
maximum likelihood method.
logit [ p( yit = 1 | xit )] = α t + β t ⋅ xit
where xit is severity index. Then we re-transform the linear predictor into the original
probability scale:p ( yit = 1 | xit ) =exp(α t + β t ⋅ xit )≡ Λ(α t + β t ⋅ xit )1 + exp(α t + β t ⋅ xit )The resulting estimates are used to calculate excess mortality by way of indirect
standardization. As is explained above, indirect standardization compares mortality ratesof the following two cases for each year: Outcome 1 which uses realized quality of careand Outcome 2 which uses average quality of care.
Outcome 1 utilizes actual relationship between risk factors and mortality for eachyear so that parameters are estimated using the sample of each year separately.
Parameters, α t and β t , depend on time t.

yit = 1[α t + β t ⋅ xit + uit > 0]
Once we obtain estimates, α̂ t and β̂ t , we re-transform the linear predictor into theoriginal probability scale:pˆ ( yit = 1 | xit ) = Λ (αˆ t + βˆt ⋅ xit )18

ESRI Research Note No.33"Measuring Health Care Output"

Then, we average individual probabilities of death for each year: t = 2004, 2005, 2006
and 2007.
nt

1
Dˆ t = ∑ Λ (αˆ t + βˆt ⋅ xit )nt i =1
Then survival rate is Eˆ t = 1 − Dˆ t .

Outcome 2 sets up a hypothetical relationship between risk factors and mortality foreach year by supposing that each year’s quality of care is the same as the average year.
Parameters are estimated using the sample from all years so that parameters, α and β ,do not depend on t: common parameters for all years.

yit = 1[α + β ⋅ xit + uit > 0]
With the estimates, α and β , we re-transform the linear predictor into the original
probability scale:p ( yit = 1 | xit ) = Λ (α + β ⋅ xit )Again, we average individual probabilities of death for each year: t = 2004, 2005, 2006
and 2007.
1 nt
Dt = ∑ Λ (α + β ⋅ xit )nt i =1
Then survival rate is E t = 1 − Dt .

Excess survival rate is the difference between Outcome 1 and Outcome 2, Êt − Et .

19

ESRI Research Note No.33"Measuring Health Care Output"(ii)
Hierarchical Model
Logistic regression model is estimated with random intercept, α t , for each year andrandom coefficient, β t , for each year.

yit = 1[α t + β t ⋅ xit + uit > 0]
Prior specifications are as follows. Two random coefficients are assumed to follow
bivariate normal with a mean vector, µ , and a precision matrix Σ −1 : ct ~ N ( µ , Σ −1 )α with ct ≡  t  . The random effect, ct , for each year comes from the same normal
 βt 

distribution so that shrinkage toward the overall mean, µ , is expected.

µ is assumed to follow a normal distribution with mean 0 and variance 100:
µ ~ N (0,100 ) . The choice of the variance of 100 is intended to represent a diffuse prior.
Gelman and Hill (2007) give a thoughtful discussion on the appropriateness of thisvalue in the context of the logistic models or log-transformed regressors. They argue
that in logistic and logarithmic regressions, typical changes in outcomes are on the scaleof 0.1 or 1, but not 10 or 100, so that one would not expect to see coefficients muchhigher than 10 in absolute values as long as the regressors are also on a reasonable scale.
Their choice of the value of variance is 1002 (standard deviation of 100), which states,
roughly, that we expect the coefficient to be in the range (-100, 100). Our choice 102
implies that the expected range is (-10, 10). We believe that this range is wide enough sothat the prior distribution is providing little information in the inference. In fact, mean
estimates of µ obtained below are (-3.77, 1.03), which are well in the range (-10, 10).
The precision matrix is assumed to follow Wishart distribution with scale matrix
and 2 degrees of freedom: Σ −1 ~ Wishart (Ω,2) . The choice of the 2 degrees ofΩ
freedom is intended to represent diffuse prior. Ω is, in turn, specified as I 2 .

The model was estimated with Markov chain Monte Carlo methods using WinBUGS
software. To check the convergence, three parallel chains were run to calculate theGelman-Rubin statistic. A burn-in of 10,000 iterations for each chain was allowed forthe model to converge. Additional 20,000 samples for each chain were drawn from thejoint posterior distribution for the estimation of all model parameters.

20

ESRI Research Note No.33"Measuring Health Care Output"(iii)
AR Restrictions
Logistic regression model is estimated with random intercept, α t , for each year andrandom coefficient, β t , for each year. AR restrictions are imposed on the movement ofrandom effects over time.

yit = 1[α t + β t ⋅ xit + uit > 0]

α t = γ αt ⋅ α t −1 + vt
β t = γ βt ⋅ β t −1 + wt

To calculate the effect on mortality of random effect for each year, we re-transform thelinear predictor into the original probability scale in the same way as MLE.
The model was estimated with Markov chain Monte Carlo methods using WinBUGS
software. The number of chains, check of convergence, burn-in and samples for
estimation are the same as the hierarchical model.
Prior specifications are also similar. Random intercept for each year, α t , isassumed to follow a normal distribution with mean µαt and variance, σ α2t ≡

1

σ α2t

:

α t ~ N ( µαt ,σ α2t ) . µαt is assumed to follow a normal distribution with mean 0 andvariance 100: µαt ~ N (0,100 ) .
A uniform prior on the standard deviation, σ αt , is adopted: σ αt ~ Uniform(0,100 )The coefficient, γ αt , on AR relations between α t ’s and other parameters are assumed
to follow normal distributions with mean zero and variance 100: γ αt ~ N (0,100 ) , etc.
Similar priors are specified for β t and γ βt .

(2) Results
Estimation results for the cases of MLE, hierarchical priors and AR restrictions areshown in Appendix Tables A2, A3 and A4 for mortality and Table A6, A7 and A8 forcomplications. Pooled estimations are in Tables A5 for mortality and A9 forcomplication.
Figure 6 contrasts crude mortality rates together with risk-adjusted mortality rates.

21

ESRI Research Note No.33"Measuring Health Care Output"

Crude and adjusted rates differ more than one percentage point in 2004 and 2006.
Hence, risk adjustment exerts significant influences on the mortality. The difference
among different risk-adjustment methods is small. One possibility is that the sample
includes several hundreds of patients for each year so that chance variations are wellcontrolled by even the MLE.
Figure 7 contrasts crude complication rates with risk-adjusted complication rates.
The complication rates estimated with MLEs are not greatly different from the crude
rates. In the cases of hierarchical prior and AR restriction, crude and adjusted rates
differ significantly. The differences between the case of MLE and the cases ofhierarchical prior and AR restriction are relatively large. Recognizing that our
adjustment for complications is rudimentary, we may suspect that the hierarchical model
may be required in the case of noisy data.
Figures 8, 9 and 10 show growth rates of output indexes with risk adjustment by MLE,
hierarchical priors and AR restrictions. Overall behavior is the same as the case ofadjustment by crude rates shown in Figure 6. However, the magnitude of adjustment ismuch smaller than the latter case.
To see this more clearly, Figures 11 and 12 superimpose growth rates by various
adjustment methods. In 2006, for example, the magnitude of adjustment with risk
adjustment is around half the magnitude of the case by crude rates.
This indicates that quality adjustment by crude rates is too much. Change in themortality and complication rates include not only true change in the quality of care but
also change in risk factors. Therefore, not all the changes in the output index adjustingfor quality by crude rates does not represent changes in quality-adjusted output.
(3) How much is enough? – Comparing Various Severity Indexes
A remaining question is: do we need detailed risk adjustment or is it sufficient to adjustfor only basic demographic factors? To examine this question, we created two additional
severity indexes which involve different degrees of risk adjustment. First one, denoted
as severity 2, includes only age and female. The second one, denoted as severity 3
includes, in addition to age and female, Killip classes, ventricular fibrillation and renal
failure.
Excess mortalities are calculated using these additional severity indexes. Here,
excess mortality means deviation from the four-year average. The results are shown in
Figure 13. The mortality adjusted with severity 2 is not very different from the crude
mortality. The mortality adjusted with severity 3 is half way between the crude mortality

22

ESRI Research Note No.33"Measuring Health Care Output"

and the mortality with baseline severity index.
Therefore, the extent of risk adjustment exerts significant impact on the estimatesof adjusted mortality, and hence on the output index.

7. Conclusion
In this paper we investigated the question: how to measure output of health care. We
explored methods to incorporate quality of health care.
First of these adjusts for crude mortality and complication rates. Adjustment of
crude rates can be misleading, however, because risk factors change from year to year
and chance variation may dominate yearly fluctuations in mortality and complication
rates.
Hence, second set of indexes incorporates risk-adjusted mortality and complication
rates. In controlling for chance variations, we compared three estimation methods:
maximum likelihood, the hierarchical model and autoregressive (AR) restrictions on
random effects.
Further, to examine how much information is required for reasonable risk
adjustment, we compared three risk adjustment models with differing degrees ofdetailed risk factors.
Overall conclusion of the paper is that we do need adjustment of quality of care in theconstruction of output index of health care. Mortality adjustment has much larger
impact than complication adjustment. However, our adjustment of complications is far
from perfect. It is very important to improve on adjustment of complications in the
future research.
Risk adjustment is also of crucial importance although the choice of method of
estimation to control for chance variation may be less crucial. This is especially true inthe case of mortality adjustment. One possibility is that the sample includes several
hundreds of patients for each year so that chance variations are well controlled by eventhe MLE. In the case of complication adjustment, however, the differences between the
results of MLE and the hierarchical model are relatively large. Recognizing that our
adjustment for complications is rudimentary, we may suspect that the hierarchical model
may be effective in the case of noisy data with measurement error.
As for the risk factors included in risk adjustment, the more detailed, the better.
In this paper, we restrict our attention to the quality of care of AMI patients. This is only

23

ESRI Research Note No.33"Measuring Health Care Output"

because of data limitation. Methods in this paper (or improved ones) to incorporate
quality of care in output index can be applied to health care in general.
The fundamental barrier to the measurement of health care output is data limitation.
It is imperative to enrich our data environment by routinely collecting detailed data on
risk factors and outcomes, especially quality of life.
It would be best to directly measure health utility by way of establishedinstruments such as EQ-D5 or SF-36, but such direct measurement may be impractical.
Stewart, et al. (2005) proposed to relate data on symptoms and impairments to healthutility in order to monitor population health. If we collect data on symptoms andimpairments of patients, we can infer health utility of individual patients by assigning
health utilities to complications.

24

ESRI Research Note No.33"Measuring Health Care Output"

References
Abraham, Katharine G., and Christopher Mackie, eds. (2005) Beyond the Market: Designing
Nonmarket Accounts for the United States. National Academies Panel on Conceptual, Measurement
and Other Statistical Issues in Developing Cost-of-Liv ing Indexes. Washington, DC: The National
Academies Press for the National Research Council.

Aizcorbe, Ana, Bonnie Retus and Shelly Smith. (2007) Toward a Health Care Satellite Account.
Survey of Current Business, May 2007.

Atkinson, Tony (2005) Atkinson Review: Final Report. Measurement of Government Output andProductivity for the National Accounts.

Aiguilar, Omar, and Mike West. (1999) Analysis of Hospital Quality Monitors Using HierarchicalTime Series Models. In C. Gatsonis, et al., eds., Case Studies in Bayesian Statistics, vol.9,
pp.287-302.

Berndt, Ernst R., Susan H. Busch and Richard G. Frank. (2001) Treatment Price Indexes for Acute
Phase Major Depression. In David Cutler and Ernst Berndt, eds. Medical Care Output andProductivity, University of Chicago Press, pp.463-505.

Berndt, Ernst, David Cutler, Richard Frank, Zvi Griliches, Joseph Newhouse and Jack Triplett.
(2001) Price Indexes for Medical Care Goods and Service: An Overview of Measurement Issues. In
David Cutler and Ernst Berndt, eds. Medical Care Output and Productivity, University of Chicago
Press, pp.141-198.

Birkmeyer JD, Siewers AE, Finlayson EVA, Stukel TA, Lucas FL, Batista I, Welch HG, Wennberg
DE Birkmeyer JD, Siewers AE, Finlayson EVA, Stukel TA, Lucas FL, Batista I, Welch HG,
Wennberg DE. (2002) Hospital Volu me and Surgical Mortality in the United States. New England
Journal of Medicine 346:1128-1137.

Birkmeyer JD, Stukel TA, Siewers AE, Goodney PP, Wennberg DE, Lucas FL. (2003) Surgeon
Vo lume and Operative Mortality in the United States. New England Journal of Medicine
349:2117-2127.

Bradley EH, Herrin J, Elbel B, McNamara RL, Magid DJ, Nallamothu BK, Wang Y, Normand SLT,25

ESRI Research Note No.33"Measuring Health Care Output"

Spertus JA, Krumholz HM. Hospital quality for acute myocardial infarction: correlat ion amongprocess measures and relationship with short-term mortality. Journal of the American Medical
Association 2006; 296:72–78.

Bronskill, Susan, Sharon-Lise Normand, Mary Landrum and Robert Rosenheck. (2002) Longitudinal
Profiles of Health Care Providers. Statistics in Medicine 21: 1067-1088.

Burgess, James, Cindy Christiansen, Sarah Michalak and Carl Morris. (2003) Medical profiling:
improving standards and risk adjustments using hierarchical models. Journal of Health Economics,
19(3): 291-309.

Carey, K., Burgess, J.: On measuring the hospital cost/quality trade-off. Health Economics 8,509–520 (1999)

Castelli, Adriana, Diane Dawson, Hugh Gravelle and Andrew Street. (2007) Improving the
Measurement of Health System Output Growth. Health Economics 16: 1091-1107.

Castelli, Adriana, Mauro Laudicella and Andrew Street. (2008) Measuring NHS Output Growth
CHE Research Paper 43, Centre for Health Economics, University of York.

Castelli A, Laudicella M, Street A, Ward P. Getting out what we put in: productivity of the English
NHS. Health Economics, Policy and Law; Forthcoming.

Clement, J.P., Vald manis, V.G., Bazzoli, G.J., Zhao, M., Chukmaitov, A.: Is more better? An
analysis of hospital outcomes and efficiency with a DEA model of output congestion. Health CareManagement Science 11, 67–77 (2008)

Cutler, David and Ernst Berndt, eds. (2001) Medical Care Output and Productivity, University of
Chicago Press.

Cutler, David and Mark McClellan. (2001) Is Technological Change in Medicine Worth IT? Health
Affairs 20(5): 11-29.

Cutler, David, Mark McClellan, Joseph Newhouse and Dahlia Remler. (1998) Are Medical Prices
Declin ing? Evidence from Heart Attack Treatment. Quarterly Journal of Economics 113(4):991-1024.
26

ESRI Research Note No.33"Measuring Health Care Output"

Cutler, David, Mark McClellan, and Joseph P. Newhouse. (1999) The Costs and Benefits ofIntensive Treatment for Cardiovascular Disease. In Jack Trip lett, ed., Measuring the Prices ofMedical Treatments, Washington, D.C.: The Brookings Institution, 34-71.

Cutler, David M., and Elizabeth Richardson. (1999) Your Money and Your Life: The Value ofHealth and What Affects It. In Alan M. Garber ed., Frontiers in Health Policy Research, Vol. 2,
Cambridge, MA: MIT Press, pp. 99-132.

Cutler, David, Allison Rosen and Sandeep Vijan. (2006) The Value of Medical Spending in the U.S.,
1960-2000. New England Journal of Medicine 355(9): 920-927.

Daniels MJ, Gatsonis C. (1999) Hierarchical generalized linear models in the analysis of variationsin health care utilization. Journal of the American Statistical Association 94(445):29–42.

Daniels MJ, Normand SLT. (2006) Longitudinal profiling of health care units based on continuousand discrete patient outcomes. Biostatistics 7(1):1–15.

Dawson D, Gravelle H, O'Mahony M, Street A, Weale M, Castelli A, Jacobs R, Kind P, Loveridge P,
Martin S, Stevens P and Stokes L. Developing new approaches to measuring NHS outputs andproductivity. Centre for Health Economics, University of York; CHE Research Paper 6; 2005.

Draper, David, and Mark Gittoes. (2004) Statistical analysis of performance indicators in UK higher
education. Journal of the Royal Statististical Society series A, 167, Part3, pp. 449–474.

Duan, N. (1983) Smearing estimates. Journal of the American Statistical Association 78: 605-610.

Dun, Abe, Lindsey Rittmueller and Bryn Whitemire. (2015) Introducing the New BEA Health Care
Satellite Account. Survey of Current Business, January 2015, pp.1-21.

Eurostat (2001) Handbook of Price and Volume Measures in National Accounts.

Evans, David, Ajay Tandon, Christopher Murray and Jeremy Lauer. (2000) The Comparative
Efficiency of National Health Systems in Producing Health: An Analysis of 191 Countries. GPE
Discussion Paper, No.29, World Health Organization.

27

ESRI Research Note No.33"Measuring Health Care Output"

Fisher, Elliott S., David E. Wennberg, Thérèse A. Stukel, Daniel J. Gottlieb, F. L. Lucas, and Étoile L.
Pinder. (2003) The Implications of Regional Variat ions in Medicare Spending. Part 1: The Content,
Quality, and Accessibility of Care. Part 1, Annals of Internal Medicine, Feb 2003; 138: 273 - 287.
Part 2, Ann Intern Med, Feb 2003; 138: 288 - 298.

Ford, Earl S., Umed A. Ajani, Janet B. Croft, et al. (2007) Explaining the Decrease in U.S. Deaths
from Coronary Disease, 1980–2000. New England Journal of Medicine 356: 2388-2399.

Fukui, Tadashi and Yasushi Iwamoto. (2004) Medical Spending and the Health Outcome of theJapanese Population. A paper for ESRI International Joint Research Project.
Garber, Alan, and Jonathan Skinner. (2008) Is American Health Care Uniquely Inefficient? Journalof Economic Perspectives 22(4): 27-50.
Gelman, Andrew, and Jennifer Hill. (2007) Data Analysis Using Regression andMultilevel/Hierarchical Models. Cambridge University Press.
Go ldstein, Harvey, and David Spiegelhalter. (1996) League Tables and Their Limitations: Statistical
Issues in Co mparisons of Institutional Performance. Journal of the Royal Statistical Society Series A
159: 385-443.

Granger CB, Steg PG, Peterson E, et al. (2005) Medication performance measures and mortality
following acute coronary syndromes. American Journal of Medicine 118:858-865.

Greene, William. (2005) Reconsidering Heterogeneity in Panel Data estimators of the Stochastic
Frontier Model. Journal of Econometrics 126: 269-303.

Hvenegaard A, Nielsen Arendt J, Street A, Gyrd-Hansen D. (2010) Exp loring the relationshipbetween costs and quality: Does the joint evaluation of costs and quality alter the ranking of Danish
hospital departments? European Journal of Health Economics

Hannan EL, O’Donnell JF, Kilburn H Jr, Bernard HR, Yazici A. (1989) Investigation of therelationship between volume and mortality for surgical procedures performed in New York Statehospitals. Journal of the American Medical Association 262:503-510.

Halm, Ethan, Clare Lee and Mark Chassin. (2000) How Is Volume Related to Quality in HealthCare? A Systematic Review of the Research Literature. In Interpreting the Volume-Outcome
28

ESRI Research Note No.33"Measuring Health Care Output"

Relationship in the Context of Health Care Quality. Maria Hewitt for the Committee on the Qualityof Health Care in America and the National Cancer Policy Board. Wachington DC: Institute ofMedicine, National Academy Press.

Hollingworth, Bruce. (2003) Non-Parametric and Parametric Applications Measuring Efficiency in
Health Care. Health Care Management Science 6: 203–218.

Hollingsworth, B.: The measurement of efficiency and productivity of health care delivery. Health
Economics 17, 1107–1128 (2008)

Jacobs R, Smith PC, Street A. Measuring Efficiency in Health Care: Analytic Techniques and Health
Policy. Cambridge University Press, 2006.

Kawabuchi, Koichi, and Shigeru Sugihara. (2006) Vo lume-Outcome Relationship in Japan: the Caseof Percutaneous Transluminal Coronary Angioplasty (PTCA) Volu me on Mortality of Acute
Myocardial Infarction (AMI) Patients. In David Wise and Naohiro Yashiro, eds., Health Care Issues
in the United States and Japan (National Bureau of Economic Research Conference Report),
University of Chicago Press, 2006, pp.113-145.

Kru mholz HM, Wang Y, Mattera JA, et al. (2006) An administrative claims model suitable for
profiling hospital performance based upon 30-day mortality rates among patients with an acute
myocardial infarction. Circulation 113:1683-1692.

Landrum MB, Bronskill SE, Normand S-LT. (2000) Analytic methods for constructing
cross-sectional profiles of health care providers. Health Services and Outcomes Research
Methodology 1(1):23–47.

Landrum, Mary, Sharon-Lise Normand and Robert Rosenheck. (2003) Selection of Related
Multivariate Means: Monitoring Psychiatric Care in the Department of Veterans Affairs. Journal of
the American Statistical Association 98: 7-16.

Luciano, Mariagrazia. (2006) Measurement of non-market output in education. Paper prepared forthe joint OECD/ONS/Government of Norway workshop ”Measurement of non-market output ineducation and health” London, Brunei Gallery, October 3 – 5, 2006.

Marshall, Clare, and David Spiegelhalter. (2001) Institutional Performance. In Alastair Leyland and29

ESRI Research Note No.33"Measuring Health Care Output"

Harvey Goldstein, eds., Multilevel Modelling of Health Statistics. John Wiley & Sons, pp.127-142.

Mark B. McClellan. (1997) Hospital Reimbursement Incentives: An Empirical Analysis. Journal of
Economics and Management Strategy, Vol. 6 no. 1, page(s) 91-128.

McClellan, Mark, Barbara J. McNeil, and Joseph P. Newhouse. (1994) Does More Intensive
Treatment of Acute Myocardial Infarction Reduce Mortality? Journal of the American Medical
Association, 272(11): 859-866.

McClellan, Mark and Douglas Staiger. (1999) The Quality of Health Care Providers. NBER Working
Paper No.7327.

McClellan, Mark and Douglas Staiger. (2000) Comparing the Quality of Health Care Providers. In
Alan Garber (ed.) Frontiers in Health Policy Research, Volu me 3. 2000, The MIT Press, Cambridge
MA, pp. 113-136.

McKay, N.L., Deily, M.E. (2008) Cost inefficiency and hospital health outcomes. Health Economics
17, 833–848.

Morey, R.C., Fine, D.J., Loree, S.W., Retzlaff-Roberts, D.L., Tsubakitani, S. (1992) The trade-off
between hospital cost and quality of care. An exploratory empirical analysis. Medical Care 30,
677–698.

Murphy, Kevin M. and Robert H. Topel (2006) “The Value of Health and Longevity,” Journal of
Political Economy, Vol. 114, pp. 871–904.

Murray, Christophe, and David Evans, eds. (2003) Health Systems Performance Assessment:
Debates, Methods and Empiricism. World Health Organization.

National Research Council. (2008) Strategies for a BEA Satellite Health Care Account. Summary ofa Workshop, National Academies Press.

Nordhaus, William D. 2003. “The Health of Nations: The Contribution of Improved Health to
Living Standards.” In Measuring the Gains from Medical Research: An Economic Approach, editedby Kevin M. Murphy and Robert H. Topel. Chicago: Univ. Chicago Press.

30

ESRI Research Note No.33"Measuring Health Care Output"

Normand SLT, Glickman ME, Gatsonis CA. (1997) Statistical methods for profiling providers of
medical care: issues and applications. Journal of the American Statistical Association
92(439):803–814.

OECD (2000) A System of Health Accounts.

O’Hagan A, Stevens JW. (2001) A framework for cost-effectiveness analysis from clin ical trial data.
Health Economics 10:303–315.

Orosz, Eva. (2005) The OECD System of Health Accounts and the US National Health Accounts:
Improving Connections through Shared Experiences. Paper Prepared for the conference on“Adapting National Health Expenditure Accounting to a Changing Health Care environment”.
Centers for Medicare and Medicaid Services, 21-22 April 2005.

Orosz, Eva, and David Morgan. (2004) SHA-Based Bational Health Accounts in Thirteen OECDCountries: A Comparative Analysis. OECD Health Working Papers No.16.

Peterson ED, Rose MT, Mulgund J, et al. (2006) Association between hospital process performanceand outcomes among patients with acute coronary syndrome. JAMA 295:1912-1920.

Rosen, Allison, and David Cutler. (2007) Measuring Medical Care Productivity: A Proposal for U.S.
National Health Accounts. Survey of Current Business, June 2007.

Skinner, Jonathan, Elliott Fisher and John Wennberg. (2005) The Efficiency of Medicare. In David
Wise, ed., Analyses in the Economics of Aging, pp. 129 – 160.

Skinner, Jonathan, and Douglas Staiger. (2009) Technology Diffusion and Productivity Growth in
Health Care. NBER Working Paper No. 14865.

Skinner, Jonathan , Douglas Staiger and Elliott Fisher. (2006) Is Technological Change in Medicine
Always Worth It? The Case of Acute Myocardial Infarction. Health Affairs, 25(2): w34-w47.

Smith, Peter, and Andrew Street. (2006) Analysis of Secondary School Efficiency: Final Report.
Research Report No.788, Department of Education and Skill, UK.

Smith, Peter. (2006) Public Services: Are Composite Measures a Robust Reflection of Performance
31

ESRI Research Note No.33"Measuring Health Care Output"

in the Public Sector, CHE Research Paper 16, Centre for Health Economics, University of York.

Spertus, John A., Martha J. Radford, Nathan R. Every, et al. (2003) Challenges and Opportunities in
Quantifying the Quality of Care for Acute Myocardial Infarction. Circulation, vol.107,
pp.1681-1691.

Spiegelhalter, David, Abrams, Keith R., Myles, Jonathan P. (2004) Bayesian Approaches to Clinical
Trials and Health Care. John Wiley & Sons.

Spiegelhalter DJ, Tho mas A, Best NG, Gilks WR. (1996) BUGS: Bayesian Inference Using Gibbs
Sampling.

Staiger, D. O., J. B. Dimick, O. Baser, Z. Fan, and J. D. Birkmeyer. (2009) Empirically derived
composite measures of surgical performance. Medical Care 47(2):226-233.

Street A, Hakkinen U. Health system productivity and efficiency. In: Smith PC, Mossialos E,
Leatherman S, Papanicolas IN, editors. Performance measurement for health system improvement:
experiences, challenges and prospects: World Health Organization; 2009.

Street A, Scheller-Kreinsen D, Geissler A, Busse R (2010): Determinants of hospital costs andperformance variation: Methods, models and variables for the EuroDRG pro ject. Working Papers in
Health Policy and Management Vol. 3 May 2010, Berlin: Universitätsverlag der TU Berlin.

Street A, Dawson D (2002) Costing Hospital Activity: the Experience with Healthcare ResourcesGroups in England. European Journal of Health Economics 3: 3-9.

Stewart, Susan, Rebecca Woodward, Allison Rosen and David Cutler. (2007) A Proposal for
Monitoring Symptoms, Impairments and Health Ratings. NBER Working Paper No.11358.

Justin Timbie, Joseph Newhouse, Meredith Rosenthal, Sharon-Lise Normand. (2008) A
Cost-Effectiveness Framework for Profiling the Value of Hospital Care. Medical Decision Making
28: 419-434.

Justin W. Timbie, David M. Shahian, Joseph P. Newhouse, Meredith B. Rosenthal and Sharon-Lise T.
Normand. (2009) Co mposite measures for hospital quality using quality-adjusted life years. Statistics
in Medicine 28:1238–1254
32

ESRI Research Note No.33"Measuring Health Care Output"

Justin W. Timb ie and Sharon-Lise T. Normand. (2008) A comparison of methods for combining
quality and efficiency performance measures: Profiling the value of hospital carefollowing acute myocardial infarction. Statistics in Medicine 27:1351–1370

Triplett, Jack. (2000) What’s Different about Health? In David Cutler and Ernst Berndt, eds. Medical
Care Output and Productivity, University of Chicago Press, pp.15-94.

UK Department of Health. (2005) Healthcare Output and Productivity: Accounting for Quality
Change.

UK Office for National Statistics. (2007) The ONS Productivity Handbook: A Statistical Overview
and Guide.

UK Office for National Statistics. (2008) Sources and Methods for Public Service Productivity:
Health.

UK Office for National Statistics. (2015) Public Service Productivity Estimates: Healthcare 2013.

Viscusi W, Aldy J. (2003) The value of a statistical life: a critical review of market estimates 504
throughout the world. Journal of Risk and Uncertainty, vol. 27(1), pages 5-76.

Weintraub, William, William Boden, Zugini Zhang, et al. (2008) Cost-Effectiveness of PercutaneousCoronary Intervention in Optimally Treated Stable Coronary Patients. Circulation CardiovascularQuality and Outcomes 1: 12-20.

WHO (2000) Health Systems: Improving Performance.

33

Table 1 Basic Statistics
Total
Mortality
Complication rate
Gappei
New infarction
Cardiovascular disorder
Renal failure
StrokeRepeat revascularization
Repeat PCI
Emergency CABG
Change within 48 hours
Cardiac arrest
Shock

0.106
0.184
0.046
0.006
0.033
0.010
0.001
0.183
0.164
0.022
0.002
0.000
0.002

2004

2005

2006

2007

0.092
0.177
0.042
0.007
0.026
0.011
0.004
0.176
0.153
0.029
0.002
0.000
0.002

0.098
0.168
0.024
0.005
0.013
0.007
0.000
0.169
0.142
0.032
0.005
0.000
0.005

0.122
0.190
0.053
0.007
0.044
0.005
0.000
0.192
0.181
0.010
0.000
0.000
0.000

0.108
0.198
0.061
0.004
0.045
0.015
0.001
0.192
0.177
0.016
0.000
0.000
0.000

2006
69.0
0.275
0.510
0.211
0.153
0.125
0.058
0.061
0.064
0.487
0.360
0.351
0.063
0.112
0.095
0.022
0.098
0.015
0.116
0.125
0.030
0.018

2007
69.0
0.320
0.462
0.248
0.183
0.107
0.052
0.075
0.048
0.525
0.405
0.314
0.075
0.093
0.081
0.013
0.087
0.015
0.085
0.140
0.023
0.015

Note: Figures for complications are based on survivors only.
Risk fctors
Total
Age
68.9
Female0.296
Killip1
0.482
Killip2
0.237
Killip3
0.147
Killip4
0.135
Left main trunk occluded
0.051
LBBB
0.067
Ventricular fibrillation
0.044
Hypertension
0.539
Hyperlipidemia
0.375
Diabetes mellitus
0.348Heart failure
0.078
History of myocardial infarction0.108
History of PCI
0.095
History of CABG
0.015
Cancer0.076
Bleeding
0.019
Renal failure
0.102
Cerebrovascular diseases
0.123
Aneurysm
0.025
COPD
0.021

2004
68.9
0.298
0.527
0.199
0.124
0.151
0.042
0.057
0.030
0.587
0.375
0.370
0.100
0.100
0.107
0.007
0.060
0.020
0.119
0.097
0.027
0.023

2005
68.8
0.289
0.430
0.288
0.119
0.163
0.052
0.072
0.031
0.565
0.355
0.364
0.078
0.127
0.101
0.018
0.056
0.029
0.090
0.124
0.023
0.028

Table 2 Characteristics of Hospitals

hospital ID

The level of
The status ofdesignatedclinical resident
Emergency Care training hospital1)

The number ofhospital beds

Total hospitalpatients

AMI patients

The number of
PCI(Average/year) (Average/year) (Average/year) (Average/year)(2006-2009)(2006-2009)(2006-2009)(2006-2009)
956
304,183
164
483

1

◎

○

2

○

○

524

89,224

69

103

3

○

○

322

7,839

19

109

4

○

○

530

1,601

81

299

5

○

○

202

72,410

186

712

6

◎

○

592

187,739

89

253

7

◎

○

469

159,961

93

185

8

○

-

151

27,275

22

163

9

○

○

165

3,198

50

367

Average of 9 hospitals

434

94,826862971) ◎: Tertiary Emergency Care, ○ : Secondary Emergency Care
"Tertiary Emergency Care"-provide patients with high-acuity conditions who require admission to the intensive care or emergency surgery.
"Secondary Emergency Care"-provide patients with moderate-acuity conditions who require admission to a general inpatient bed.

Table 3 Assumed Utilities of Complications

Utility

ReferenceBaseline

0.9

Weintraub, et al.

Stroke0.52

Timbie, et al.

Renal failure

0.63

Timbie, et al.

Repeat PCI/Emergency CABG

0.78

Timbie, et al.

Figure 1 Output Index of Inpatient Services: Patient Survey
104

102

100

98

96

94

92

90
1987

1990

1993

1996

1999

2002

2005

2008

Figure 2 Growth of DPC-Based Output Index and the Number of Patients
7

6

5

4
Output Index
Number of Patinets
3

2

1

0
2006

2007

2008

2009

Figure 3 Hospitalized AMI Patients(Thousand)
10

9

8

7

6

5

4

3

2

1

0
1984

1987

1990

1993

1996

1999

2002

2005

2008

Figure 4 Mortality and Complication Rates
0.21
0.14
0.19
0.12
0.17
0.1

0.15

0.08
0.13

Mortality (Right Scale)Complication (Left Scale)

0.06
0.11

0.04

0.09

0.02

0.07

0

0.05
2004

2005

2006

2007

Figure 5 Output Index Growth:
Adjustment by Crude Mortality & Complication Rate
0
2005

2006

2007

-1

-2

-3
Without adjustment
Adjusted for survival
Full adjustment
-4

-5

-6

-7

Figure 6 Crude and Adjusted Mortality
0.13

0.12

0.11

0.1
Crude mortality
MLE

0.09

Hierarchical prior
AR
0.08

0.07

0.06

0.05
2004

2005

2006

2007

Figure 7 Crude and Adjusted Complication Rate
0.2

0.195

0.19

0.185

0.18
Crude complication rate
MLE

0.175

Hierarchical prior
AR

0.17

0.165

0.16

0.155

0.15
2004

2005

2006

2007

Figure 8 Output Index Growth: Adjustment by MLE
0
2005

2006

2007

-1

-2

-3
Without adjustment
Adjusted for survival
Full adjustment
-4

-5

-6

-7

Figure 9 Output Index Growth: Adjustment by Hierarchical Priors
0
2005

2006

2007

-1

-2

-3
Without adjustment
Adjusted for survival
Full adjustment
-4

-5

-6

-7

Figure 10 Output Index Growth: Adjustment by AR
0
2005

2006

2007

-1

-2

-3
Without adjustment
Adjusted for survival
Full adjustment
-4

-5

-6

-7

Figure 11 Output Index Growth: Adjusted for Survival
0
2005

2006

2007

-1

-2

-3

Without adjustment
Crude survival rate
MLE
Hierarchical prior

-4

-5

-6

-7

AR

Figure 12 Output Index Growth: Full Adjustment
0
2005

2006

2007

-1

-2

-3

Without adjustment
Crude survival rate
MLE
Hierarchical prior

-4

-5

-6

-7

AR

Figure 13 Excess Mortalities by Risk Adjustment
0.02

0.015

0.01

0.005
Crude excess mortality
Severity0

Severity2
Severity3-0.005-0.01-0.015

-0.02

2004

2005

2006

2007

Appendix Table A1 Creating Severity Index
Logistic regression
Log likelihood = -551.31298
death
Age
FemaleKillip2
Killip3
Killip4
Left main trunk occluded
LBBB
Ventricular fibrillation
Hypertension
Hyperlipidemia
Diabetes mellitus
Heart failure
History of myocardial infarction
History of PCI
History of CABG
CancerBleeding
Renal failure
Cerebrovascular diseases
Aneurysm
COPD
Constant

Number of obs =Pseudo R2=
Coefficient
0.051
0.475
1.663
2.993
3.668
0.230
0.300
0.965
-0.409-1.012
0.149
0.040
0.232
-0.349
0.901
0.728
0.564
0.478
0.014
0.574
0.435
-4.8092631
0.3789Standard Error t statistics
0.008
0.166
0.369
0.351
0.355
0.278
0.225
0.251
0.162
0.232
0.168
0.230
0.262
0.338
0.494
0.226
0.405
0.195
0.192
0.372
0.407
0.349

6.5
2.86
4.5
8.53
10.33
0.83
1.34
3.85
-2.53
-4.36
0.89
0.17
0.88
-1.03
1.82
3.23
1.39
2.45
0.07
1.54
1.07
-13.8

p-value
0
0.004
0
0
0
0.408
0.182
0
0.011
0
0.374
0.861
0.377
0.302
0.068
0.001
0.164
0.014
0.941
0.123
0.285
0

Table A2 Mortality - MLE
2004

Logistic regression
Log likelihood = -104.47098
death
severity
_cons

2005

Logistic regression
Log likelihood = -114.54049
death
severity
_cons

2006

Coefficient
1.153-4.141

Logistic regression
Log likelihood = -146.82795
severity
_cons

2007

Coefficient
1.179-4.169Coefficient
1.014-3.614

Logistic regression
Log likelihood = -181.08996
death
severity_cons

Coefficient
0.804-3.169Pooled Logistic regression
Log likelihood = -551.31298
death
severity_cons

Coefficient
1.000-3.663

Number of obs =Pseudo R2=Standard Error
0.143
0.413

p-value
0.000
0.000

Number of obs =Pseudo R2=Standard Error
0.132
0.385

749
0.2943p-value
0.000
0.000

Number of obs =Pseudo R2=Standard Error
0.054
0.157

672
0.4110p-value
0.000
0.000

Number of obs =Pseudo R2=Standard Error
0.084
0.234

612
0.4165p-value
0.000
0.000

Number of obs =Pseudo R2=Standard Error
0.103
0.309

598
0.4311p-value
0.000
0.000

2631
0.3789

Table A3 Mortality - Hierarchical Priors
Constant
Severity
2005
Constant
Severity
2006
Constant
Severity
2007
Constant
Severity
Overall mean of constantOverall mean of coefficient
Correlation coefficient of constant and coefficient
Variance of constantCorrelation of constant and coefficient
Variance of coefficientmean-4.0781.149-4.065
1.129-3.6731.032-3.2790.838-3.7671.030-0.1530.844-0.1270.534

Number of observations

2631

2004

sd
0.353
0.125
0.339
0.119
0.287
0.097
0.233
0.084
0.486
0.366
0.443
1.445
0.741
0.946

2.50%
-4.8280.919-4.7800.909-4.2660.850-3.7510.678-4.7530.302-0.8710.133-1.3100.098

97.50%-3.4391.411-3.4501.377-3.1391.230-2.8421.007-2.820
1.750
0.735
3.672
0.680
2.221

Table A4 Mortality - AR Restriction
Constant

Severity

 t
 t

t

 t

 t

 t

2004
2005
2006
2007
2004
2005
2006
2007
2005
2006
2007
2005
2006
2007
2004
2005
2006
2007
2004
2005
2006
2007
2004
2005
2006
2007
2004
2005
2006
2007

mean
-4.202-4.172-3.659-3.196
1.189
1.162
1.028
0.812
0.707
0.446
0.483
0.381
0.278
0.312
-1.500-2.948-1.850-1.550
0.580
0.450
0.320
0.253
26.96046.88041.40039.15020.64041.84031.31028.580

sd
0.413
0.392
0.314
0.239
0.143
0.135
0.104
0.086
5.529
6.531
6.950
7.847
8.126
8.295
8.012
23.17027.24022.240
7.251
9.347
9.484
6.753
25.28028.96027.95027.81022.85029.19026.84026.490

Number of observations
2004
2005
2006
2007

598
612
672
749

Total

2631

2.50%
-5.068-5.010-4.308-3.691
0.928
0.916
0.833
0.651
-11.220-13.640-14.420-15.780-16.690-17.060-16.910-51.230-58.630-47.710-15.200-18.790-19.530-13.810
0.684
1.862
1.567
1.486
0.324
1.188
0.853
0.625

97.50%-3.453-3.465-3.082-2.752
1.486
1.443
1.242
0.987
12.28014.11014.80016.27016.82017.24015.89046.57056.72046.27015.72019.48019.68014.11090.45096.96095.61095.28085.76096.42093.12092.330Table A5 Mortality - Pooled
Constant
Severity
Overall mean of constantOverall mean of coefficient
Correlation coefficient of constant and coefficient
Variance of constantCorrelation of constant and coefficient
Variance of coefficientmean-3.6781.005-3.4530.944-0.011139.800-4.51837.300

Number of observations

2631

sd
0.159
0.055
2.420
2.390
0.674
19080.000
3783.000
1742.000

2.50%-3.9970.900-8.033-4.406-0.9880.194-37.380
0.190

97.50%-3.379
1.114
2.181
5.928
0.986
124.100
33.130120.800

Table A6 Complication - MLE
2004 Logistic regression
Log likelihood = -286.57969
complication
severity_cons

Coefficient
0.150-1.4582005 Logistic regression
Log likelihood = -286.8045
complication
severity_cons

Coefficient
0.122-1.5382006 Logistic regression
Log likelihood = -340.90053
complication
severity_cons

Coefficient
0.032-1.3552007 Logistic regression
Log likelihood = -383.16597
complication
severity_cons
PooledCoefficient
0.016-1.336

Logistic regression
Log likelihood = -1301.6275
complication
severity_cons

Coefficient
0.070-1.410

Number of obs =Pseudo R2=Standard Error
0.049
0.106

p-value
0.002
0.000

Number of obs =Pseudo R2=Standard Error
0.050
0.108

749
0.0002p-value
0.705
0.000

Number of obs =Pseudo R2=Standard Error
0.022
0.049

672
0.0009p-value
0.437
0.000

Number of obs =Pseudo R2=Standard Error
0.041
0.090

612
0.0102p-value
0.015
0.000

Number of obs =Pseudo R2=Standard Error
0.042
0.096

598
0.0162p-value
0.002
0.000

2631
0.0037

Table A7 Complication - Hierarchical Priors
Constant
Severity
2005
Constant
Severity
2006
Constant
Severity
2007
Constant
Severity
Overall mean of constantOverall mean of coefficient
Correlation coefficient of constant and coefficient
Variance of constantCorrelation of constant and coefficient
Variance of coefficientmean-1.5040.151-1.5850.088-1.486-0.057-1.420-0.027-1.4970.032-0.0090.518-0.0090.506

Number of observations

2353

2004

sd
0.111
0.059
0.112
0.060
0.108
0.053
0.098
0.049
0.361
0.351
0.446
0.891
0.579
0.942

2.50%
-1.7240.035-1.809-0.030-1.702-0.162-1.614-0.124-2.213-0.669-0.8160.094-0.7660.093

97.50%-1.2880.267-1.3690.205-1.2790.045-1.231
0.068-0.782
0.718
0.806
2.135
0.715
2.065

Table A8 Complication - AR Restriction
Constant

Severity

 t
 t

t

 t

 t

 t

2004
2005
2006
2007
2004
2005
2006
2007
2005
2006
2007
2005
2006
2007
2004
2005
2006
2007
2004
2005
2006
2007
2004
2005
2006
2007
2004
2005
2006
2007

mean
-1.500-1.590-1.487-1.4170.151
0.081-0.058-0.027
0.458
0.384
0.399
0.030
-0.1020.170-0.663-0.683-0.603-0.5670.124
0.009-0.011-0.00822.10042.37033.16033.08016.54036.13017.79015.590

sd
0.114
0.115
0.110
0.099
0.059
0.060
0.054
0.049
7.533
7.740
7.995
9.529
9.162
9.146
7.490
11.31012.32011.370
6.593
1.533
0.908
0.514
23.50029.23027.04027.08021.30029.80024.08023.280

Number of observations
2004
2005
2006
2007

543
552
590
668

Total

2353

2.50%
-1.727-1.816-1.706-1.6160.035-0.033-0.165-0.123-15.270-16.030-16.280-18.850-18.490-18.680-16.150-23.570-25.890-23.910-14.330-3.169-2.004-1.135
0.402
1.266
0.948
0.972
0.077
0.289
0.079
0.040

97.50%-1.281-1.369-1.276-1.224
0.268
0.201
0.046
0.068
15.77016.31016.71018.67018.37018.45015.62022.97025.46023.24014.720
3.182
1.967
1.131
87.17096.46093.71093.58081.43095.63086.86085.670Table A9 Complication - Pooled
Constant
Severity
Overall mean of constantOverall mean of coefficient
Correlation coefficient of constant and coefficient
Variance of constantCorrelation of constant and coefficient
Variance of coefficient
Number of observations

mean-1.4820.026-1.3940.014-0.007118.800-2.27936.820

sd
0.054
0.027
2.392
2.375
0.674
15980.000
3144.000
1761.000
2353

2.50%
-1.589-0.027-6.228-5.180-0.9870.193-35.980
0.191

97.50%-1.378
0.080
3.850
5.040
0.986
120.700
33.210120.400