  In most commonly used ranking systems, some level of underlying transitivity
is assumed. If transitivity exists in a system then information about pairwise
comparisons can be translated to other linked pairs. For example, if typically
A beats B and B beats C, this could inform us about the expected outcome
between A and C. We show that in the seminal Bradley-Terry model knowing the
probabilities of A beating B and B beating C completely defines the probability
of A beating C, with these probabilities determined by individual skill levels
of A, B and C. Users of this model tend not to investigate the validity of this
transitive assumption, nor that some skill levels may not be statistically
significantly different from each other; the latter leading to false
conclusions about rankings. We provide a novel extension to the Bradley-Terry
model, which accounts for both of these features: the intransitive
relationships between pairs of objects are dealt with through interaction terms
that are specific to each pair; and by partitioning the $n$ skills into
$A+1\leq n$ distinct clusters, any differences in the objects' skills become
significant, given appropriate $A$. With $n$ competitors there are $n(n-1)/2$
interactions, so even in multiple round robin competitions this gives too many
parameters to efficiently estimate. Therefore we separately cluster the
$n(n-1)/2$ values of intransitivity into $K$ clusters, giving $(A,K)$
estimatable values respectively, typically with $A+K<n$. Using a Bayesian
hierarchical model, $(A,K)$ are treated as unknown, and inference is conducted
via a reversible jump Markov chain Monte Carlo (RJMCMC) algorithm. The model is
shown to have an improved fit out of sample in both simulated data and when
applied to American League baseball data.
