Working with a gold-plated standard
Authors: Terry Therneau
Journal: Hepatology (2010)

Liver fibrosis is a final common pathway of chronic liver disease. The degree of fibrosis is a strong predictor of patient outcomes and thus plays a central role in patient management decisions. The gold standard for assessment of fibrosis is a thorough pathological assessment of the organ, and rightfully so. Multiple hallmarks of the progressive process have been cataloged over the years; these signs are visible to a trained pathologist and have been validated for their association with important clinical endpoints. The problem in practice is the word "thorough." Liver disease is often focal, particularly in the earlier stages where there is the most room for meaningful clinical intervention. Accurate disease assessment requires samples of sufficient size from multiple regions of the liver, which is realistic only with an explanted organ. Actual liver biopsies from clinical practice are far less comprehensive. Markov models of progression that I have worked on in primary biliary cirrhosis, primary sclerosing cholangitis, and hepatocellular carcinoma suggest a misclassification rate of 20%-30% or greater, estimates that are confirmed by multiple other studies using more direct assessments. This, along with the invasiveness of biopsy, means there is a huge appetite for alternate procedures. As we assess these new methods, there are a few important principles that we should keep in mind. First, although pathological characterization of the liver is the gold standard, the pathological report from a single small-needle biopsy is not. It should perhaps be called a "gold-plated" standard. Although the logical first assessment of a new method is to compare it to biopsy on a set of patients, the results of that assessment give only a partial indication of utility. An analogy would be if we wanted to know the genetics on some subject "X." You have a sample "Y" with 50% correlation (for example, a sibling of subject X) and I supply a sample "Z" with 50% correlation to Y. Is my assessment of X better or worse than yours? The answer can range all the way from perfection (an identical twin of X) to being only half as good (grandchild of X). A similar situation arises with two new candidates for assessment of fibrosis: the better candidate may actually have a worse association with biopsy. A perfect candidate would have at least 20%-30% error in predicting biopsy results. Even the conventional nomenclature inherited from a gold standard can constrain results: do not confuse a categorization of a process with the process itself. The degradation process in chronic liver disease is a continuous one; the pathological description of it is a small set (five to seven) of discrete categories. For a naturally continuous measurement, such as liver stiffness, most analyses start with a Procrustean step of forcing the measurement into a set of discrete boxes. One consequence of this is a built-in nonreproducibility: if the F2 versus F3 threshold is set at 12.5 kPa, then a subject whose first measurement is 12.48 kPa will almost assuredly have class variation in multiple measurements. It also forces a compression of the data. If there were multiple regions measured, the discordance between them as well as the average may be an important clue as to the liver state, and similarly when a score is the result of multiple laboratory measurements. The most important point to remember is fitness for purpose. Fibrosis or other staging scores are not an end in themselves in clinical practice. Rather, they are valued for their proven relation to important patient outcomes. The important step for a new method is the same: not "does it predict the biopsy result," but "does it predict the patient result." Long-term follow-up of patients after extracting the new measure should be our target, along with serious thought about what that measure or measures should be. That is the gold standard for whether a new method is an important addition to the practice.