  In this paper I describe and reduce to practice an objective protocol for
evaluating the cognitive capabilities of a non-human system against human
cognition in a laboratory environment. This is important because the existence
of a non-human system with cognitive capabilities comparable to those of humans
might make once-philosophical questions of safety and ethics immediate and
urgent. Past attempts to devise evaluation methods, such as the Turing Test and
many others, have not met this need; most of them either emphasize a single
aspect of human cognition or a single theory of intelligence, fail to capture
the human capacity for generality and novelty, or require success in the
physical world. The protocol is broadly Bayesian, in that its primary output is
a confidence statistic in relation to a claim. Further, it provides insight
into the areas where and to what extent a particular system falls short of
human cognition, which can help to drive further progress or precautions.
