In practice, most sites reported on contrasts that reflected their particular interests or areas of research. Word error will be the primary metric. Upon request, NIST will make scoring software available in a timely manner to participants. An existing scoring approach was adapted to the current evaluation, as the necessary tools and conventions had already been worked out. Not mentioned in the specification was the use of another mechanism, adjudication, a process by which sites were allowed to challenge NIST's scoring on particular utterances.
Adjudication catches transcription errors as well as ambiguities in the transcription conventions.
In contrast to previous evaluations, little adjudication took place for Hub 4. Reportage Separate scores will be computed for each part of the evaluation set and will always be clearly separated in all accounts of the evaluation. NIST will tabulate for each part both a global score and a breakdown according to categories deemed of interest to the community. Categories of interest will be specified as we acquire experience with the domain. NIST will chose a initial list of such, based on its own experience with the domain and in consultation with the H4 committee.