OBJECTIVE There are no formal prognostic models predicting adverse outcomes (excessive length of stay or mortality) in hospitalized patients with diabetes. In this study, we aimed to develop a prediction model that will help identify patients with diabetes who are most likely to have an adverse event during their hospital stay.
RESEARCH DESIGN AND METHODS Analysis was based on 25,118 admissions with diabetes to University Hospital Birmingham, Birmingham, U.K., over 4 years (2007–2010). Adverse events are defined as either excessive length of stay or inpatient mortality. Key predictors were variables that are often available in the first 72 h of admission and included demographic characteristics, clinical pathological test results, and use of insulin. Models were constructed using logistic regression, discrimination and calibration was assessed, and internal validation was carried out.
RESULTS The model performed well with an area under the curve (AUC) of 0.802 with only a mild reduction being noted in the internal validation (AUC 0.798). At a cutoff value of 25% probability of having an adverse outcome the sensitivity was 76%, specificity was 70%, and the positive predictive value was 49%. If it is used for a case-finding approach limiting to noncritical care settings, then at the same cutoff value, two-thirds (sensitivity 69%) of the admissions with adverse outcomes could potentially be identified.
CONCLUSIONS Once externally validated, we suggest that our model will be a useful tool for identifying diabetic patients who are at risk for poor outcomes when admitted to hospital.
People with diabetes are twice as likely to be admitted to hospital and have longer lengths of stay, higher frequency of complications, and higher mortality rates compared with the general population (1–5). Clinical guidelines and research findings have suggested that models of health delivery based on specialist diabetes teams providing enhanced care, dietetic and foot care services for high-risk inpatients, and better care pathways can reduce adverse outcomes (6–10). However, as 15–20% of hospitalized patients have diabetes, it has become increasingly difficult to identify who within this patient group will end up with poor clinical outcomes. We therefore hypothesized that an active case-finding approach using clinical information systems can detect diabetes patients who subsequently go on to have adverse outcomes. These patients could then be targeted for early appropriate management.
To be of practical use, tools predicting poor clinical outcomes in diabetes patients should be available on hospital information systems at or around the time of admission. Several patient characteristics, clinical pathology results, and comorbidities, some of which are available in clinical information system, have been identified as useful markers of prognosis in secondary care for both patients with and without diabetes (11–17). However, there is no formal prognostic model available to stratify diabetic patients for poor clinical outcomes. We have developed a prediction model that will help identify, at or early in an admission, patients with diabetes who are most likely to have an adverse event during their stay. Adverse events are defined as either excessive length of stay or inpatient mortality.
The setting is University Hospital Birmingham, a tertiary hospital providing secondary care services to an ethnically diverse population in the West Midlands, U.K. Data were derived from the Patient Administration System (PAS) and the Patient Information and Communication System (PICS) and included inpatient admissions from 2007 to 2010. The PAS database records demographic details, admission and discharge time, discharge diagnostic codes, and death. The PICS database contains patients’ clinical pathology test results, electronic observations, and prescriptions. The PAS database was linked to the PICS database using individual patient identification numbers and admission time, and patients with both records were included in the analysis (18). Patients were identified as having diabetes using both discharge diagnostic codes and prescription data as published previously (18).
Initially, all admissions (both with and without diabetes) were categorized into 260 groups of clinical conditions based on primary diagnosis from the discharge diagnostic code. The 260 groups are defined as per the Agency for Healthcare Research and Quality clinical classification system and were recently adopted by the National Health Service Information Centre (19). The median length of stay was derived for each clinical condition for nondiabetic patients. The excess length of stay for each admission with diabetes was defined as the difference between the actual length of stay and the median length of stay for nondiabetic patients with the same group of clinical conditions. An excessive length of stay is defined as an excess length of stay >75th centile of all diabetic admissions. This cutoff was selected because 25% of admissions with diabetes accounted for 85% of excess length of stay in patients with diabetes. The cutoff point also corresponds to 6 days more than what would be expected for any given group of clinical conditions in a nondiabetic patient.
Our methodology avoids the need to know the condition with which the patient is admitted; this is important because the diagnosis (and hence group of clinical conditions) may not be clear at the time of admission.
Three models are presented: a pragmatic model, a test model, and an ideal model. The pragmatic model, is intended to be used to predict adverse outcomes early in admission. It uses clinical pathological test results instead of a measure of comorbidity because diagnoses may not be available at or around the time of admission. The test model replaces the clinical pathological test results from the pragmatic model with a measure of comorbidity (modified Charlson comorbidity score) to determine whether these clinical pathological test results are a good alternative for measuring case-mix. The ideal model includes all variables available at discharge and so includes clinical pathological test results, modified Charlson comorbidity score, and a measure of deprivation.
Prognostic factors included in the pragmatic model are those that can be reasonably expected to be available within the first 72 h of admission. Data that are readily available include age, sex, ethnicity, and admission type (emergency or elective). The presence of foot disease is also included in the pragmatic model because recommendations suggest that it should be recorded in the first 24 h of admission (8). We assume that patients prescribed insulin during their stay had this initiated within 72 h of admission. Similarly admissions to an intensive care unit (ICU) setting are mainly within the first 72 h (75% at 72 h, 68% at 48 h, and 52% at 24 h in our database). Therefore, both of these variables are included in the pragmatic model. Clinical pathological test results that predict adverse outcomes in hospital settings are also included. These include albumin (20,21), estimated glomerular filtration rate (GFR) (22), serum sodium (13,14), serum potassium (23), C-reactive protein (CRP) (15), and hemoglobin (16). White cell counts are also a valid predictor (24), although we decided to use neutrophil counts instead, as this is a better predictor of infection and/or sepsis (25).
Although blood glucose concentrations predict poor clinical outcome (17,26–29), we were not able to include admission glucose concentrations in the pragmatic model as 74% of glucose measurements were missing. Information on comorbidities may not be available to extract from hospital information systems at the time of admission, and therefore, comorbidities are not included in the pragmatic model but are included in both test and ideal models. Deprivation levels are difficult to compute at or around the time of admission, and also definitions differ across countries and were therefore excluded from the pragmatic model but included as part of the ideal model.
Foot diseases were defined based on National Institute for Health and Care Excellence (NICE) guidelines using ICD-10 codes and OPCS4 codes (a U.K. classification system for interventions and procedures) that were available as a discharge diagnostic code in the database (8). These included foot ulcers, peripheral vascular disease, peripheral neuropathy, arthropathy, and operative procedures such as wound debridement and amputation. Codes for foot ulcers were similar to a national report recently published in the U.K. (30), except we excluded the codes indicating decubitus ulcers. These codes were excluded as pressure ulcers could be either in the sacrum, buttock, or heel (31,32), are less specific, and are less likely to occur early in an admission. Furthermore, patients with decubitus ulcers are known to have poor clinical outcomes (33), and, as a result, estimated effect size for the foot disease will be biased if these codes were included. Modified Charlson comorbidity score (27) was calculated by excluding the scores linked to diabetes from the Charlson comorbidity score (11). Deprivation quintiles were defined using disaggregated income component of the Indices of Multiple Deprivation score (34).
The first value of the clinical pathology tests for each admission was included in the analysis. Most were available (94%) within the first 72 h of admission. Cutoff points to categorize the clinical pathological results were based on 1) normal ranges, 2) definitions of severity for a given marker [i.e., GFR reflecting stages of renal disease as per guidelines (35)], 3) adequate number of groups to illustrate any dose-response relationship, and 4) sufficient numbers in each group to observe any meaningful results.
The demographic characteristics, clinical pathology test results, and morbidity characteristics of the patients with and without the adverse outcome are summarized using means (SD) or medians (interquartile range) as appropriate for continuous data and using proportions for categorical data.
Odds ratios (ORs) (95% CIs) were estimated using logistic regression models. To account for the clustering effect created by the same patient being admitted more than once, we used population-averaged methods (36).
In the multivariable model, prognostic factors were selected based on their clinical significance as identified from the literature and were therefore preserved in the model irrespective of their statistical significance (37,38). This meant we did not carry out any stepwise procedures. Neither did we use any interaction terms, as these add complexity and are difficult to use in clinical practice.
Data were complete for all variables other than for clinical pathology test results. For these variables, to allow for the missing data, multiple imputations (10 imputations) were carried out using the Multiple Imputations by Chained Equations with predicted mean matching. The chained equations allow imputation of both categorical and continuous variables, while predicted mean matching restrict imputations to the appropriate range (by matching the predicted value to the closest value in the dataset).
Model performance is assessed by its ability to distinguish between those with and without the outcome of interest (discrimination) and by looking at the agreement between the frequency/proportion of observed and predicted outcomes (calibration). In addition, the sensitivity, specificity, positive predictive values, negative predictive values, and likelihood ratios were determined. To assess discrimination, a receiver operating characteristic curve was constructed and area under the curve (AUC) (Harrell’s C-statistics) was calculated. C-statistics were compared between the three models (pragmatic model, test model, and ideal model) using nonparametric tests (39). Calibration was assessed by plotting predicted probabilities of outcome, by decile groups, against observed probabilities of outcome (in same decile group), and by overlaying a smoothed calibration curves (using lowess algorithm) to judge against a linear line (37).
In a sensitivity analysis, models were internally validated using bootstrapping. This process repeatedly samples, with replacement, samples of datasets from the original dataset and reestimates model parameters in each resampled dataset. Parameter estimates are then averaged over all of the resampled datasets. We found that results from the internally validated model and the primary model differed very little, and so, we report only the results from the primary model. This process will ensure that we identify any evidence of overfitting to the original dataset (that is producing estimates that are a good fit to the developmental data but perform poorly in prediction) and accommodate this by reporting shrunken regression coefficients (38). To implement this for each of the 10 imputed datasets, we applied a bootstrap procedure (2,000 resamples) to obtain 10 sets of shrunken regression coefficients (with SEs) and C-statistics (with SEs), which we then combined using Rubin’s rule (40).
Coefficients, model performance, and sensitivity analysis are reported for the pragmatic model only. For the other two models (test model and ideal model), the AUC is reported. All data analyses were carried out using Stata 12 software (Stata Statistical Software, release 12; StataCorp LP, College Station, TX).
Out of the 171,067 admissions (2007–2010) included in the analysis, 25,118 (14.7%) had diabetes. Out of these 25,118 admissions, 6,281 (25%) were categorized as having an excessive length of stay and 1,286 (5.1%) died. Excluding overlaps, this meant 6,928 (28%) had an adverse outcome. Among these 25,118 admissions, 10,596 (42%) admissions had an expected length of stay less than or equal to the median length of stay for the same presenting clinical condition of nondiabetic patients. The remaining patients (N = 14,522; 58%) contributed to 146,680 excess days with a median of 4.7 days and a mean of 10.1 days. The 6,281 admissions identified as contributing markedly to excess length of stay accounted for 85% (124,803 days) of the excess days.
Admission characteristics and clinical pathology test results of those with and without adverse events are given in Tables 1 and 2. Coefficients and the ORs of the prognostic factors used in the pragmatic model are given in Table 3. In the pragmatic model, there was an incremental rise in the odds of adverse outcome with increasing age. The oldest age group (≥85 years) had an OR of 5.64 (95% CI 4.66–6.81) in comparison with the youngest (16–44 years). Females had slightly higher odds of having an adverse event (1.08 [1.00–1.16]), while those of Asian ethnic minority (largely South Asian ethnicity in this region of the U.K.) had a significant lower chance of having an adverse event (0.86 [0.78–0.94]). Those admitted as an emergency (2.94 [2.69–3.21]), on insulin therapy (1.89 [1.76–2.03]), or with foot disease (2.46 [2.16–2.80]) were at higher odds of having an adverse event. Receiving treatment in an ICU setting had the highest OR (10.79 [9.52–12.22]).
Severe hyponatremia (<125 mmol/L) had higher odds (1.71 [1.26–2.32]) of having an adverse outcome than mild to moderate (125–134 mmol/L) hyponatremia (1.17 [1.06–1.28]). Similarly, the greater the severity of hypernatremia, the larger the effect size was (145–154 mmol/L, 1.31 [1.12–1.54]; and >155 mmol/L, 4.05 [1.84–8.91]). Such a dose-response relationship favoring an adverse outcome was also noted with lowering hemoglobin concentrations (anemia) and hypoalbuminemia and rising CRP concentrations and neutrophil count. Hypokalemia (1.79 [1.34–2.39]) but not hyperkalemia (1.00 [0.80–1.26]) was associated with the adverse outcome. A GFR <30 mL/min/1.73 m2 had an OR of 1.31 (95% CI 1.15–1.48) in comparison with normal GFR (≥90 mL/min/1.73 m2).
The pragmatic model had an AUC of 0.802 (95% CI 0.795–0.808), performing significantly (P < 0.001) better than the test model (AUC 0.784 [95% CI 0.777–0.790]), suggesting that the clinical pathology results replaced comorbidities as a measure of case-mix well. The ideal model performed better than the pragmatic model (0.810 [0.804–0.816]; P < 0.001), but the difference between the pragmatic model and ideal model was minimal (AUC 0.802 vs. 0.810) (Fig. 1).
Receiver operating characteristic (ROC) curves for model comparison and assessment of discrimination.
The sensitivity and specificity total was maximal at a cutoff of >25% predicted chance of an adverse event. In this study, the sensitivity was 76%, specificity was 70%, and the positive predictive value was 49% (Supplementary Fig. 1). However, in reality, patients in ICU will not be part of the active case-finding approach, and if one were to see only those in the noncritical care setting, the approach will have a sensitivity of 69%, specificity of 72%, and a positive predictive value of 43%. In contrast, the so-called false-positive patients in noncritical care whom will be identified in an active case-finding approach have characteristics such as high comorbidity index (41% had modified Charlson comorbidity score of ≥2), insulin use (66%), foot disease (10%), and age >75 years (51%). Therefore, false positives are not necessarily admissions that will not benefit from enhanced models of care.
The lowess calibration plot was in close proximity to a line drawn at 45°, suggesting the calibration of the model was good; i.e., the predicted probabilities were similar to that of the observed probabilities (Supplementary Fig. 2).
The sensitivity analysis revealed that there was little evidence of overfitting. This was demonstrated both by minimum differences between regression coefficients (Table 3) and also by little difference between C-statistics. For example, the internal validated bootstrapped sample had an AUC of 0.798 (95% CI 0.792–0.805), only a marginal difference to that observed in the pragmatic model (0.802 [0.795–0.808]).
We show that predictor variables, routinely available at the time of admission, facilitate the identification of diabetic patients at risk for an excessive length of stay or death. At a cutoff value of 25% probability of having an adverse outcome, the sensitivity was 76%, specificity was 70%, and the positive predictive value was 49%. The pragmatic model performed well with an AUC of 0.802 with only a small reduction being noted in the internal validation (AUC 0.798 [95% CI 0.792–0.804]).
The association of the clinical pathology test results with length of stay or death in our analysis of diabetic patients corresponds with those previously identified for unselected patients. These include hypernatremia (13,14), anemia (16), hypokalemia (23), and raised CRP concentrations (15). Our data validates these findings in people admitted to hospital with diabetes. Furthermore, some of these criteria may be more relevant in the context of diabetes: sodium and potassium imbalance can challenge the management of diabetic ketosis; anemia and reduced renal function may indicate advanced diabetic nephropathy; and an elevated CRP and neutrophilia may indicate compromised sepsis/wound healing in diabetes.
We believe our model will have important clinical utility. Timely identification of high-risk patients provides the opportunity for early intervention and improvement in clinical outcome. In our hospital, we plan to incorporate the model as part of decision support within the electronic medical records. On identification of high-risk patients, an automated consultation request would be generated to the inpatient diabetes team. They can then be reviewed early in their admission. We believe this will be beneficial, as there is clear evidence that specialist nurse review can reduce length of stay (6,7). Other potential uses include redesigning care models and pathways for patients with diabetes admitted to hospital. This is particularly important with planning for seasonal fluctuations in hospital admission rates.
There are limitations to our approach. First, we have assumed diabetes was present at the time of admission and that all admissions with diabetes can be identified through information systems. The latter may not be possible in all hospitals with electronic health information systems. However, we have previously shown this is achievable using either mandatory entry of the diabetes status on admission or in a comparable way using electronic prescription data alone (18). Second, blood results may not be available at the time of diagnosis, and these therefore will be categorized as normal values when using the prediction model. Although this may compromise the validity of the model, additional sensitivity analysis undertaken by replacing the missing categories with normal categories and applying the coefficients obtained from the multiple imputation model to the dataset suggested better performance than shown (AUC 0.816, sensitivity 73%, specificity 75%, and positive predictive value 53%) (Supplementary Fig. 3). We were not able to differentiate between type 1 and type 2 diabetes, as the recorded ICD-10 codes are not reliable discriminators. The strength of our study is that the model is developed in a hospital with a diverse population from a large dataset with effect sizes that had narrow CIs and rigorous methodological quality. Furthermore, our definition of excessive length of stay is novel and mitigates the need to know the presenting condition. Similarly, our model does not rely on knowing the comorbidities, replacing this instead with routinely performed blood tests.
We conclude that our active case finding model is a useful tool to identify patients with diabetes who may be at risk for poor outcomes such as increased length of stay and death. Further studies should aim to 1) externally validate the model, 2) assess the practicality of using the model, and 3) demonstrate if the active case-finding approach either on its own or in combination with additional clinical indicators of poor outcomes (such as hypoglycemia, hyperglycemia, and insulin infusions identified through electronic records), followed by appropriate care (e.g., review by the diabetes specialist nurse) will positively impact on reducing adverse outcomes for patients with diabetes.
This study was funded by the National Institute for Health Research through the Collaborations for Leadership in Applied Health Research and Care for Birmingham and Black Country programme.
K.N. was responsible for conception and design, analysis and interpretation of data, drafting the article, and revising it critically for important intellectual content. K.H. was responsible for analysis and interpretation of data and revising the article critically for important intellectual content. P.N. and T.M. were responsible for interpretation of data and revising the article critically for important intellectual content. J.J.C. was responsible for conception and design, interpretation of data, and revising the article critically for important intellectual content. All authors gave approval for the final version to be submitted. K.N. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
The views expressed in this publication are not necessarily those of the National Institute for Health Research, the Department of Health, NHS Partner Trusts, University of Birmingham, or the Collaborations for Leadership in Applied Health Research and Care for Birmingham and Black Country Theme 9 Management/Steering Group.
Yorkshire and Humber Public Health Observatory. Mortality among inpatients with diabetes [article online], 2012. Available from http://www.yhpho.org.uk/default.aspx?RID=139172. Accessed 1 February 2013
. The relation between hyperglycemia and outcomes in 2,471 patients admitted to the hospital with community-acquired pneumonia. Diabetes Care 2005;28:810–815pmid:15793178
National Institute for Health and Care Excellence. Inpatient management of diabetic foot problems [article online], 2013. Available from http://www.nice.org.uk/nicemedia/live/13416/53556/53556.pdf. Accessed 7 September 2012
. An evaluation of the association between systemic inflammation—as measured by C-reactive protein—and hospital resource use. Curr Med Res Opin 2007;23:2785–2792pmid:17908371
. The relationship between admission hemoglobin level and outcome after hip fracture. J Orthop Trauma 2002;16:39–44pmid:11782632
. Hypoglycaemia is associated with increased length of stay and mortality in people with diabetes who are hospitalized. Diabet Med 2012;29:e445–e448pmid:22937877
. Hyperglycaemia is associated with poor outcomes in patients admitted to hospital with acute exacerbations of chronic obstructive pulmonary disease. Thorax 2006;61:284–289pmid:16449265
Kerr M. Foot care for people with diabetes: the economic case for change [article online], 2013. Available from www.diabetes.nhs.uk/document.php?o=3372. Accessed 21 August 2012
Noble M, McLennan D, Wilkinson K. The English Indices of Deprivation 2007, Department of Communities and Local Government, London [article online], 2008. Available from http://webarchive.nationalarchives.gov.uk/+/http:/www.communities.gov.uk/communities/neighbourhoodrenewal/deprivation/deprivation07/. Accessed 12 April 2013
National Institute for Health and Care Excellence. Chronic kidney disease [article online], 2013. Available from http://www.nice.org.uk/nicemedia/live/12069/42116/42116.pdf. Accessed 1 February 2013
You are going to email the following A Prediction Model for Adverse Outcome in Hospitalized Patients With Diabetes
Description: Design and analyze operational solutions for projects such as transportation optimization, network modeling, process and methods analysis, cost containment, capacity enhancement, routing and shipment optimization, and information management.
Description: Design objects, facilities, and environments to optimize human well-being and overall system performance, applying theory, principles, and data regarding the relationship between humans and respective technology. Investigate and analyze characteristics of human behavior and performance as it relates to the use of technology.
Looking for talented iOS Developers who are motivated to join an international team and making an impact on the world through great tech products (and why not, curious to enjoy the lifestyle of South East Asia).
In addition to having excellent technical skills, you have a keen interest in the latest technologies and have a feel for user experience. You’ll be responsible for building exciting applications and implementing flows and mock ups as provided by UX and Design teams. Apart from having the ability to write fast and clean code with great attention to detail, you also come up with solutions and suggestions for improvement as you will be working closely with Product Development and Marketing teams.
We are expanding our Mobile Development team, so we are looking for iOS Developers who can craft awesome native iOS Apps.
At Mindvalley we are developing an ecosystem of apps to dominate the personal growth market and reach millions of users. We develop modules and libraries which are shared by multiple apps.
Being one of our developers, you'll have an opportunity to learn and grow your technical as well as personal skills. You will be an active part of creating our products, so check out the AppStore for the latest releases.
You'll be responsible for building exciting applications and implementing flows and mock-ups provided by UX and Design teams. Apart from having the ability to write performant and clean code with great attention to detail, you also contribute with solutions and suggestions for improvement as you work closely with Product Development and Marketing teams.
You are more than just a developer, someone who understands that a great app must do great things yet is simple and intuitive. You are pixel-perfect, knowing that a pixel to the right is not merely a pixel to the right. You understand that an organized code with the well-applied design patterns will make a difference. You are fascinated about technology and eager to learn everything you can, applying the coolest and most amazing techniques there are. You add a "magic touch" that will dazzle your users.
Have you have worked on some iOS app? Of course you have! Include the AppStore links in your resume!
Ideally you are excited about Mindvalley’s mission. If you have a special interest in personal development, education, meditation, health & fitness or related topics tell us about it in your application!
Rush Haywards Heath opened its doors on 5th September 2015. Franchisee Ruth talks to us about her decision to franchise and why she has been overwhelmed with the support and encouragement in her franchising process with Rush.
I have a background in hairdressing, when I left school I started as an assistant and then worked my way up to a salon manager over 6 years. I did have a break from hairdressing when I went travelling and lived in Sydney for 4 years where I took a different career path working as a Resource and MI Analyst. I came back to the UK and moved to Haywards Heath last year.
I have always wanted to run my own business and tried to open my own salon earlier in my career but had to pull out as I didn’t have enough support to carry me through. I chose to franchise with Rush because I wanted to be part of a company that prides themselves on exceptional customer service with a proven business model and a strong support system which is very important for me. As soon as I met the chairman, I knew I was in very safe hands and any fears and doubts I had disappeared. Since I’ve started the franchising process with Rush I have been overwhelmed with the continual support and encouragement from everyone I’ve worked with.
The head office team at Rush is really hands on and it is so refreshing to work in an organisation where everyone shares the same passion and does everything to support and encourage the growth of their team members.”
Ruth with her family, Rush Co-founders Andy Phouli and Stel Andrew, Vice-Chairman Peter Phouli and Deputy Mayor Jim Knight
The process was very slick, everything was well thought out and pre-planned. Rush has many years of experience in franchising and they definitely know what they are doing; even the minutest of details have been included. The Operations Team were incredibly organised and are very experienced in launching a new salon smoothly. The same can be said for the training; Rush have all the bases covered and tailor the training to suit the individual through their training academies whether they are a stylist, manager or front of house, or even for people completely new to the hair and beauty industry.
I recently relocated to Haywards Heath and felt it would be a fantastic location for a Rush salon. Haywards Heath is a commuter town a short distance between the Sussex Downs and Brighton. It has lots of lovely restaurants, bars and high street shops so believe that because Rush is a big high street brand it will be very popular in Haywards Heath.
I wanted to be part of a company that prides themselves on exceptional customer service with a proven business model and a strong support system which is very important for me.”
My team are committed to providing the very best service to ensure every client has a fantastic experience at Rush Haywards Heath. They are all experienced professionals, trained through Rush’s Academies who have been chosen for their excellent skills and knowledge.
Do your research, meet people on the ground and ask lots of questions, but most importantly make sure that the brand is a good fit for you. Listen to your heart and your head, but most of all go with your gut feeling!