  Linear combinations of multinomial probabilities, such as those resulting
from contingency tables, are of use when evaluating classification system
performance. While large sample inference methods for these combinations exist,
small sample methods exist only for regions on the multinomial parameter space
instead of the linear combinations. However, in medical classification problems
it is common to have small samples necessitating a small sample confidence
interval on linear combinations of multinomial probabilities. Therefore, in
this paper we derive an exact confidence interval, through the use of fiducial
inference, for linear combinations of multinomial probabilities. Simulation
demonstrates the presented interval's adherence to exact coverage.
Additionally, an adjustment to the exact interval is provided, giving shorter
lengths while still achieving better coverage than large sample methods.
Computational efficiencies in estimation of the exact interval are achieved
through the application of a fast Fourier transform and combining a numerical
solver and stochastic optimizer to find solutions. The exact confidence
interval presented in this paper allows for comparisons between diagnostic
methods previously unavailable, demonstrated through an example of diagnosing
chronic allograph nephropathy in post kidney transplant patients.
