Clinical investigation: A guide to its evaluation

Criteria for accurately determining the validity of clinical investigations are described with special regard to the physician oriented toward clinical practice rather than research. Chance is the one factor the investigator can neither eliminate nor neutralize but it should be carefully managed so as to arrive at an accurate estimate of its limits. The range of distribution of events due solely to chance is statistically defined as "variation due to randon sampling". In a truly random sample there is a predictable probability for any given distribution but the probability is accurate only to the degree that the sample is truly random. Nonresponse such as missing histories or study dropouts detract from the randomness of the sample. Bias or distortion of samples is most effectively eliminated when 1) all patients are selected from a single identifiable universe; 2) selection and assignment to treated and untreated groups are truly random; 3) randomization remains undisturbed by program alterations. Clinical investigation is distinguished from clinical survey by the fact that the former activity follows design rather than having already taken place. A sound investigative report will begin by clearly describing the criteria established for subject eligibility and the methods used in selection and categorizing subjects. Choice of a control group should be truly random as well as defined by the same universe as the experimental group and control data is most accurate when gathered in the same manner as experimental data. Evaluation of therapeutic activity through retrospective controls is considered an inherently fallacious research method. Use of a patient as his own control can be advantageous but primarily when the treated condition is farily stable and the effect of treatment is temporary and reversible. The placebo effect (i.e. the patients experience what they expect or desire) can be controlled by the double blind technique in which preparations are code labeled and neither investigator nor subject know which is which. When a series of observations has been made and an acceptable confidence level has been determined mathematical operations determine the confidence limits. If an infinite number of sets of observations are made randomly from a given universe the confidence limits based on the observed frequency of the anticipated event in the set would bracket the "true" frequency in that per cent of the sets chosen as the confidence level. The most commonly used confidence level for clinical investigations is 95%. The reader should familiarize himself with the mathematical background of the statistics he reads about or seek the cooperation of a biostatistician. Any set of measurements will be characterized by two features: the value around which the observations cluster and the degree of scatter in their distribution. Standard deviation is a measure of the scatter of values about their mean and a most useful indice of variability. The coefficient of variation which permits comparison of frequency distributions employing different units is found by reducing the standard deviation to a percentage of the mean. With small samples (less than 20 observations) calculation of standard deviation may be quite inaccurate and should be interpreted with caution.