Validating a computational model of patient illness: the simcare patient model

The contributions of this thesis can be organized into three related parts, 1) a two-part validation study of a computational model of patient illness, 2) a conceptual model to be used as the basis for generating explanations for model behavior, and 3) a novel form of operational validation using an N=1 experimental approach to measure meaningful variation in individual patients over time. The validation is presented to satisfy the interests of two overlapping research communities – those interested in the content of the model: the healthcare research community; those interested in computational modeling and validation techniques: the computer science community. The validation study is divided into a conceptual validation and an operational validation. The conceptual validation establishes the set of relevant theories identified in the natural system to be represented in the model. These theories enable the explanations of the model to be generalized and learned from, and they define the intent and contexts for relevant uses of the model. The operational validation performs two types of simulation studies that characterize the outputs of the model under two different real-world contexts. The first set of experiments compares the simulation of populations of individuals under treatment protocols to the outcomes of three well-known clinical trials in the diabetes community. This distinguishes the model as being able to simulate controlled trials to the extent that a population of individuals can be generated and treatment protocols defined. In the second set of simulation experiments, a series of N=1 trials are conducted using retrospective, outpatient clinical care data to demonstrate that SimCare accurately represents meaningful variation over time in individuals being treated for diabetes in a clinical (i.e., less controlled) setting over time. In this setting, meaningful variation is defined as the non-random, clinically relevant variation in outcomes that can emerge over time given a specific course of treatment and an initial patient state. For example, if a physician were to treat two simulated patients via model software, and each patient had identical, observable initial states and received same treatments, the physician would not expect the two patients to exhibit identical responses to the treatments. This variation that exists in the real world of clinical care and causes an individual patient to exhibit meaningful differences in outcomes over time is an intentional part of the SimCare model and requires its own validation study. This distinguishes the model as being one of individual patients (rather than population-based) representing common, primary care encounters (rather than pre-screened patients under controlled protocols). The results of the conceptual validation show that the SimCare model is clinically transparent and capable of generating explanations related to treatment outcomes. The operational validation shows that the SimCare model is able to capture meaningful and typical variation both in individual patients over time and across sets of patients in controlled cohorts. (Abstract shortened by UMI.)