the data file of recommendations by pbac was created by abstracting data from the pbac web site ( http://www.health.gov.au/internet/main/publishing.nsf/content/public-summary-documents-by-product ) . data were taken from recommendations made from july 2005 through november 2009 . in the data if a product had more than one submission for the same indication , more than one record was created under the same unique identification number . however , if a product had multiple submissions that included a different indication , a new unique identification number was created for that product and indication . the data file included pbac 's recommendation ( to recommend a listing with or without restrictions , not to recommend a listing , or to defer ) , the incremental cost - effectiveness ratio ( cost per quality - adjusted life - year [ qaly ] ) , and the highest value of the financial impact presented in the summary document for each product . in addition , other variables were abstracted that have been shown to be associated with pbac 's reimbursement recommendations in previous studies ( 12;13 ) ( supplementary table 1 , which can be viewed online at www.journals.cambridge.org/thc2013079 ) . the four categories of outcomes abstracted were combined to create a binary variable with categories recommended ( yes , restricted yes ) , and not recommended ( no , deferred ) . the predictors of the pbac recommendation that were tested in the analyses were either abstracted or derived variables that were classified as either economic variables or clinical variables as seen in previous studies ( 1214 ) . economic variables included the following : ( i ) population size , created from reported patient population size treated per year : low ( < 10,000 ) , medium ( 10,00050,000 ) , and high ( > 50,000 ) . ( ii ) estimated financial impact per annum , using the upper bound value presented and estimated for the impact on only the drug budget : a$0 , > a$0 to < a$10 million , a$10 million to a$30 million , and > a$30 million . a categorical variable was used for the analysis because the pbac summaries presented financial impact as an upper bound or range rather than a continuous variable for most of the submissions . ( iii ) cost per qaly , using the upper bound base case analyses : > a$0 to a$45,000 , > a$45,000 to a$75,000 , and > a$75,000 as used in the chim and colleagues study ( 12 ) of the impact of cost - effectiveness on reimbursement decisions . a fourth category , no cost - effectiveness analysis presented , was assigned to those submissions that used a cost - minimization approach for the economic evaluation . clinical variables included the following : ( i ) active comparator ( yes or no ) that indicated whether an active comparator was used as the comparison group in at least one of the pivotal studies ; ( ii ) manufacturer claim for the clinical benefits of the new product : noninferior or equivalent or superior ; ( iii ) comparative clinical evidence available from randomized clinical trials only ( rct ) or from rct data plus a meta - analysis or indirect comparison analysis ( rct plus meta - analysis or indirect comparison analysis ) ; ( iv ) disease category ( oncology or other ) , as a proxy measure of likelihood of reduced life expectancy and the  dread  factor associated with the disease ( 12 ) ; and ( v ) surrogate end point ( yes or no ) , derived from a review of the end points in the submission . the unit of analysis for all analyses was the unique drug and indication submission after july 2005 . only the first observed submissions of the unique drug and indication combination within our database were included in the univariate and initial multivariable logistic analyses because subsequent resubmissions were correlated with the first observed submission . a test result was declared statistically significant if p value was < .05 and marginally statistically significant if p value was > .05 but  .1 . first , a univariate analysis was performed to explore the association between the pbac recommendation and the variables described previously . next , a multivariable logistic regression was performed to evaluate the relationship between the probability of a positive recommendation and the categorical financial impact , while adjusting for other factors . the variables included in the logistic model were those that had an association with the recommendation with a p value .30 in the univariate analysis ( 15 ) . a discrete time - to - event analysis was performed , including all extracted data : both the first observed submission data and all resubmission data to determine the relationship between multiple submissions and pbac recommendations while accounting for the correlations between repeated submissions and to determine the impact of the omission of the resubmissions on our estimates for financial impact . we performed the analysis using the logistic model as described in allison ( 16 ) . total submission count was determined by counting the number of times the same drug plus indication was submitted . only nine records had a total count of four or more , and these records were omitted from the analysis . the variable time since previous submission was included because the resubmissions happened at irregular intervals . a submission or resubmission for a drug plus indication that happened once or more than once but that was not recommended for reimbursement also , submissions could be left censored if data for the first observed submission indicated that previous submissions had occurred before july 2005 . finally , a decision tree analysis was performed using the recursive partitioning algorithm in jmp analysis software ( sas , cary , nc ) . recursive partitioning is a nonparametric classification technique that splits into subsets , called nodes , observations with similar response values for predictor variables ( 17 ) . in each node , the predictor variable with the strongest association to the outcome variable is chosen for splitting the node . for a categorical outcome , the recursive partitioning algorithm in jmp analysis software uses log10 ( p value ) , also called logworth , where the p value is an adjusted p value given by a likelihood ratio test . to avoid overfitting , only partitions that had a logworth value 1.12 , which corresponded to log10 ( 0.05 ) , because this is a nonparametric method , it has an advantage over logistic regression by not assuming any functional form for the association between predictors and outcome . furthermore , recursive partitioning has the advantage of detecting possible complex interactions between predictors that may not have been detected by the logistic regression , and because it is easy to visualize and interpret , it is suitable for a decision - making process ( 18 ) . moreover , the relative importance of the predictors can be inferred by the order in which they partition the data ( i.e. , the earlier the predictor is used by the partition algorithm , the more important it is ) . the logistic regression and decision tree analyses were also performed using only those submissions with a reported cost per qaly to assess the importance of financial impact in this subset of the total submissions . a total of 260 submissions , representing 214 unique drug plus indication combinations and 46 resubmissions during the data abstraction time period , were extracted from the pbac web site . most , or 170 of the 214 ( 79.5 percent ) unique drug plus indication combinations , were not submitted before july 2005 ; 27 ( 12.6 percent ) were submitted once before july 2005 ; and 17 ( 7.9 percent ) were submitted multiple times before july 2005 . of the 260 submissions , 106 ( 40.8 percent ) were recommended for reimbursement , 47 ( 18.1 percent ) were partially recommended for reimbursement ( i.e. , recommended with restrictions ) , 100 ( 38.4 percent ) were not recommended for reimbursement , and 7 ( 2.7 percent ) were deferred . therefore , the binary variable pbac recommendation for reimbursement had 153 submissions ( 58.9 percent ) that were recommended and 107 submissions ( 41.1 percent ) that were not recommended . five variables  financial impact , cost per qaly , manufacturer 's claim , active comparator , and disease category  had a statistically significant association with the pbac recommendation . with the exception of the financial impact category > a$30 million , the percentage of submissions that were recommended decreased as the financial impact and the cost per qaly increased . in addition , 74.3 percent of the submissions that did not report a cost per qaly were recommended as compared with 40.0 percent for those submissions with a reported cost per qaly . the percentage of recommendations was higher for entries that claimed noninferiority or equivalence , that had used an active control as the comparator in at least one pivotal clinical trial , and that were not in the oncology category . the other included variables ( population size , comparative clinical evidence , and surrogate end points ) did not have a statistically significant association with the pbac recommendation . based on the univariate analyses , six possible predictors of pbac recommendations with a univariate p value .30 table 1.univariate association between pbac recommendations and predictorsvariableno . in each variable categorypercentage in each variable category recommended for reimbursement by pbacp value for difference between categoriesfinancial impact ( million a$)>302245.5%<.000110 to 304235.7%>0 to < 1010154.5%03992.3%cost per quality - adjusted life - year ( thousand a$)>751915.8%<.0001>45 to 752733.3%>0 to 455950.9%none10974.3%population sizehigh4047.5%.3671medium5259.6%low12259.8%manufacturer claimsuperior or advantages10843.5%<.0001noninferior or equivalent10671.7%comparative clinical evidencerandomized controlled trial9953.5%.2792rct + meta - analysis or indirect comparison analysis11560.9%active comparatorno5232.7%<.0001yes16265.4%disease categoryoncology3740.5%.0219other17761.0%surrogate end pointyes15058.0%.5846no5453.7% percentages were calculated out of the available data for the respective variable category . p value was calculated using pearson 's chi - square test for difference between the variable categories.pbac , pharmaceutical benefits advisory committee . univariate association between pbac recommendations and predictors   percentages were calculated out of the available data for the respective variable category . p value was calculated using pearson 's chi - square test for difference between the variable categories . figure 1 and supplementary table 2 , which can be viewed online at www.journals.cambridge.org/thc2013079 , present the results of the multivariable logistic analyses . only the effect of financial impact ( p = .0242 ) , cost per qaly ( p = .0235 ) , and active comparator ( p = .0365 ) were statistically significant . after adjusting for the other factors in the model , the only statistically significant odds ratios for financial impact were for comparing either category a$10 million to a$30 million ( 0.12 ; 95 percent confidence interval [ ci ] : 0.030.51 ) or > a$0 to < a$10 million ( 0.16 ; 95 percent ci : 0.040.60 ) with the category a$0 . the odds ratio for the financial impact category > a$30 million compared with the category a$0 was not significant ( 0.25 ; 95 percent ci : 0.051.34 ) . for cost per qaly , the only statistically significant odds ratios were for comparing the category > a$75,000 with either the category > a$0 to a$45,000 ( 0.11 ; 95 percent ci : 0.020.55 ) or to the category none ( 0.06 ; 95 percent ci : 0.010.40 ) . the odds of recommending a drug submission that used an active comparator were 2.49 ( 95 percent ci : 1.065.85 ) times the odds of recommending a drug submission that used placebo as the primary comparator . figure 1 presents a plot of selected odds ratios obtained from the first logistic model . financial impact was not statistically significant ( p = .1801 ) when the same model was run using only submissions that reported a cost per qaly ( n = 103 ) . cost per qaly was the only statistically significant factor ( p = .0158 ) in this model . figure 1.multivariable logistic regression results ( n = 204 ) : odds ratios with 95% ci plots . ci , confidence interval ; qaly , quality - adjusted life - year ; rct , randomized controlled trial . multivariable logistic regression results ( n = 204 ) : odds ratios with 95% ci plots . ci , confidence interval ; qaly , quality - adjusted life - year ; rct , randomized controlled trial . figure 2 and supplementary table 2 present the results of the analysis of discrete - time survival data , using multivariable logistic analysis . the statistically significant effects were total submission count ( p = .0029 ) , financial impact ( p = .0021 ) , cost per qaly ( p = .0135 ) , and an active comparator ( p = .0229 ) . the odds ratio estimates for financial impact , cost per qaly , and an active comparator were very similar to the corresponding odds ratios obtained in the first model . after adjusting for other factors in the model , the odds of recommending a drug that was submitted a third or second time were 9.62 ( 95 percent ci : 2.4737.42 ) or 3.66 ( 95 percent ci : 1.359.96 ) times the odds of recommending a drug submitted the first time , respectively . the odds of recommendation were 0.83 lower for a 1-trimester increase in time since the previous submission . figure 2 presents the odds ratio plots for the second logistic model . when the same model was run only using submissions ( n = 126 ) with a reported cost per qaly , financial impact was still statistically significant ( p = .0223 ) . figure 2.multivariable logistic regression results for discrete time - to - event data ( n = 238 ) : odds ratios with 95% ci plots . ci , confidence interval ; qaly , quality - adjusted life - year ; rct , randomized controlled trial . multivariable logistic regression results for discrete time - to - event data ( n = 238 ) : odds ratios with 95% ci plots . ci , confidence interval ; qaly , quality - adjusted life - year ; rct , randomized controlled trial . in addition to the variables used in the logistic analyses , population size and surrogate end point were also included . categorical financial impact was the most important factor to make the first partition ( logworth = 7.18 ) by grouping the three financial impact categories , > a$0 to < a$10 million , a$10 million to a$30 million , and > a$30 million , into a single category and comparing it with a$0 . cost per qaly , active comparator , and disease category were the next predictors selected for recursive partitioning . for cost per qaly , the categories none and > a$0 to a$45,000 were combined by the model program and compared with > a$45,000 to a$75,000 and > a$75,000 . the results of the recursive partitioning model indicated that the chance of being recommended for reimbursement for drug submissions with a financial impact a$0 was 91.4 percent , and the chance for drug submissions with a financial impact > a$0 and a cost per qaly either not estimated or a$0 to a$45,000 was 57.5 percent as compared with 24.4 percent for submissions with a financial impact > a$0 and cost per qaly > a$45,000 . lower chances for reimbursement recommendation were estimated for drug submissions with a financial impact > a$0 , a cost per qaly either not calculated or > a$0 to a$45,000 , and without an active comparator ( 29.6 percent ) . when a new tree was constructed considering only submissions with a reported cost per qaly , the cost per qaly was the only important factor in the partitioning analysis . the results of the analyses presented in this study indicate that the estimated financial impact of a drug on the australian drug budget is a predictor of the pbac reimbursement recommendation , even when controlling for the cost - effectiveness ratio and other confounding variables . in the descriptive analysis , there was a gradient in probability of reimbursement , with the highest probability for drugs that were estimated to be cost - saving and the lowest probability for drugs that were estimated to increase annual costs between a$10 million and a$30 million . however , probability of recommendation was higher for those submissions with a financial impact of > a$30 million compared with those with a financial impact of a$10 to a$30 million . the logistic analyses demonstrated that this pattern was similar even when controlling for the cost - effectiveness ratio and other confounding variables and the number of submissions and even when only including those submissions that presented a cost per qaly estimate . a review of the submissions with an estimated financial impact > a$30 million ( 22 submissions ) found that products in this category that were recommended for reimbursement either had cost per qaly estimates in the lower end of the a$0 to a$45,000 range , or very favorable clinical benefits , or indications where there were no alternative treatments that might explain this seemingly anomalous result . the impact of multiple submissions on the probability of recommendation for reimbursement was significant with an odds ratio of 9.62 for a third and 3.66 for a second observed submission compared with the first observed submission . supplementary table 3 , which can be viewed online at www.journals.cambridge.org/thc2013079 , presents a summary of the changes in the categorical values among the 29 multiple submissions . a review of these changes indicated that , although reductions in price were likely key factors in obtaining a positive reimbursement recommendation in many cases , changes in the clinical data submitted were also influential in obtaining a positive recommendation . finally , the recursive partitioning decision analysis supported the importance of a positive financial impact for the reimbursement decision with the full database because the variable with the greatest discriminative power for reimbursement recommendations was shown to be a positive financial impact of any magnitude , followed by the cost per qaly . however , the results for the subset of submissions that report a cost per qaly , all of which had a positive financial impact , indicated that the cost per qaly variable had the greatest discriminatory power . a threshold value of a$10 million was used for the financial impact analysis because full approval by the cabinet of the federal government was needed for drugs when their annual financial impact was expected to exceed a$10 million in any 12-month period within the first 4 full years of product listing . having mandated that a multiyear financial impact analysis be included in submissions has allowed the pbs to put in place price - volume contracts for those drugs that exceed the a$10 million per year limit ( 19 ) , although the specific agreements are confidential . the results of our analyses are supported by and extend those from three previous analyses of pbac data ( 1214 ) . all three studies included financial impact on the healthcare sector as an explanatory variable . in the supplemental appendix of clement et al . ( 13 ) , in a descriptive analysis of submissions not needing a cost - per - qaly estimate , the probability of reimbursement in australia was much higher ( 81 percent ) than for those needing a cost - per - qaly estimate ( 44 percent ) ( 13 ) . in the harris et al . ( 14 ) study , for a sample of the decisions that also included an estimate of the cost - per - qaly gained , harris and colleagues ( 14 ) estimated that for each increase of a$5 million in financial impact above the mean value , the probability of a pbac recommendation for reimbursement would decrease by 0.03 . however , by excluding any submissions that estimated financial cost - savings and many that estimated a budget impact < a$10 million , they may have underestimated the importance of the financial impact for reimbursement decisions , especially for drugs with multiple submissions . finally , chim and colleagues ( 12 ) compared the impact on reimbursement of two financial impact categories : < a$10 million and a$10 million . in a logistic analysis , they estimated a statistically significant odds ratio of reimbursement of 0.46 for submissions with a financial impact of a$10 million compared with those with a financial impact of < a$10 million . the strengths of our analysis included the use of multiple analytic techniques that all provided similar results . the use of the survival analysis allowed us to include data from multiple submissions as well as right- and left - censored submissions , appropriately accounting for the correlation among repeat submissions of the same drug / indication pair . the use of the recursive partition decision analysis estimation technique is important because the results provided a simple stepped process for predicting whether a submission will be recommended for reimbursement in australia . the major limitation is that , in many cases , the financial impact on the drug budget or the cost per qaly was provided in the public summary document only as an inequality or range . thus , it was not possible to enter financial impact into the model as a continuous variable . instead , we created four financial impact categories . also , the sample size was relatively small ( n = 260 ) , limiting our ability to include a high number of explanatory variables or more than four categories of financial impact . there was also the possibility of multicollinearity among the independent variables used in the analysis . to test for this , we estimated  tolerance scores for each independent variable and found that all variable scores were above 0.40 , values not considered to be of concern for multicollinearity ( 16 ) . supplementary table s4 , which can be viewed online at www.journals.cambridge.org/thc2013079 , presents the regression results . the choice of oncology as a proxy for disease severity was also a source of uncertainty because other diseases may also be associated with very limited life expectancy and  dread . the implications of the findings in this study are that , in australia , the financial impact on the drug budget is an important determinant of whether a new drug is recommended for reimbursement with the strongest positive impact for products that have a zero or negative financial impact . our findings are consistent with the conclusions of the recent published analyses on the possibility that financial impact plays a role in the hta agency reimbursement recommendations , although not specifically listed as a criterion ( 68 ) . in addition , the explicit instructions provided in the pbac submission guidelines for performing financial impact analysis indicate the potential importance of these results for the pbac review . although financial impact is a specific part of the decision making for reimbursement in australia , this is not necessarily the case in other countries . whether financial impact influences reimbursement decisions in these countries is an empirical question that can be resolved only by performing similar analyses in these jurisdictions . supplementary table 1 : www.journals.cambridge.org/thc2013079 supplementary table 2 : www.journals.cambridge.org/thc2013079 supplementary table 3 : www.journals.cambridge.org/thc2013079 supplementary table 4 : www.journals.cambridge.org/thc2013079 josephine mauskopf , phd , vice president health economics ( jmauskopf@rti.org ) , costel chirila , phd , senior statistician , biometrics , catherine masaquel , mph , associate director in the market access and outcomes strategy , rti health solutions , po box 12194 , research triangle park , nc , 27709 kristina s. boye , phd senior director global health outcomes , diabetes , lee bowman , phd senior director , global health outcomes oncology , julie birt , pharmd senior research scientist , global health outcomes , health technology assessment  center of expertise , david grainger , bs global public policy director , lilly and company , indianapolis , indiana , united states josephine mauskopf , catherine masaquel , costel chirila report payment to their institution by a contract from eli lilly and company for leading the research reported in the manuscript .
objectives : the aim of this study was to estimate the relationship between the financial impact of a new drug and the recommendation for reimbursement by the australian pharmaceutical benefits advisory committee ( pbac).methods : data in the pbac summary database were abstracted for decisions made between july 2005 and november 2009 . financial impact  the upper bound of the values presented in the pbac summary database  was categorized as a$0 , > a$0 up to a$10 million , a$10 million up to a$30 million , and > a$30 million per year . 
 descriptive , logistic , survival , and recursive partitioning decision analyses were used to estimate the relationship between the financial impact of a new drug indication and the recommendation for reimbursement . 
 multivariable analyses controlled for other clinical and economic variables , including cost per quality - adjusted life - year gained.results : financial impact was a significant predictor of the recommendation for reimbursement . in the logistic analysis , 
 the odds ratios of reimbursement for drug submissions with financial impacts a$10 million to a$30 million or > a$0 to < a$10 million compared with a$0 were 0.12 ( 95 percent confidence interval [ ci ] : 0.030.51 ) and 0.16 ( 95 percent ci : 0.040.60 ) , respectively . in the recursive partition decision analysis , 
 the first split of the data was for submissions with a positive financial impact compared with those with a zero or negative financial impact.conclusions : in australia , financial impact on the drug budget is an important determinant of whether a new drug is recommended for reimbursement when cost - effectiveness estimates and other clinical and economic variables are controlled .