# Measurement of Head-Related Transfer Functions: A Review
## 7 Discussion
### System Evaluation  
As described in previous sections, there are a variety of HRTF measurement systems, and the measurement results may vary when using different setups or methods. It is therefore important to evaluate the quality of measured HRTFs. In general, the measured HRTFs can be assessed through objective or subjective methods.  
If no reliable reference measurement result is available, the headphone-based subjective evaluation, e.g., localization test [124; 148], distance evaluation [79], is an optimal method to evaluate measured HRTFs (headphones should be carefully equalized). For the objective evaluation, SNRs of measured HRTFs ([PERSON] et al. [194] defined various SNR types) can serve as a metric. In addition, the change of binaural cues across measurement points can be calculated and compared with the theory [120].  
If highly reliable reference HRTF data (measured or calculated) are available, the HRTF evaluation can be considered as detecting the similarity between two HRTF datasets (measured and reference HRTFs). Various objective metrics are available, e.g., differences in HRTF spectra [161; 195], differences in binaural cues [84], multi-dimensional scaling analysis [196], principal component analysis (PCA) weights of HRTF magnitude spectra [197], correlation of HRTFs [198], cross-validation of temporal and spectral structures [174], binaural auditory model [199; 200], etc. In the case of the subjective evaluation, since the reference HRTFs are available, discriminative tests such as ABX test [167], three-alternative forced-choice (3-AFC) test [161] can be applied to determine whether there is an audible difference between two HRTF pairs. In addition, a more detailed listening test with respect to various perceptual attributes may be considered [201].  
The evaluation of different measurement systems can not be accomplished in one institute/laboratory because of difficulties in constructing various measurement setups as shown in Figures 8 and 9. In the literature, several comparison studies have been performed based on one hardware setup, e.g., [PERSON] et al. [194] compared HRTF measurements with MLS signals, exponential sweeps, and adaptive filtering methods based on a single-loudspeaker setup (see S\({}_{\text{E}}\) in Figure 9), [PERSON] [84] compared measurements with MESM and adaptive filtering approaches based on a multi-loudspeaker setup (see M\({}_{\text{B}}\) in Figure 8), etc. [PERSON] and [PERSON] [202] initiated an international round-robin study, \"Club Fritz\", to compare HRTFs measured or simulated from different institutes, and the Neumann KU-100 dummy head is used as an unique artificial head for the comparison study. That study aims to investigate the repeatability of HRTFs measured from different systems and to establish a reference for data quality. [PERSON] et al. [203] objectively evaluated 12 HRTF datasets and observed variations in magnitude spectra of up to 12.5 dB below 6 kHz and up to 23 dB for high frequencies, and in ITDs of up to 235 \(\upmu\)s. Recently, [PERSON] et al. [204] tested the localization performance of 12 HRTF datasets in the mid-sagittal plane based on auditory models, and illustrated that four datasets have comparable performances. These comparison results underline the complexity of measuring HRTFs and show noticeable differences between the HRTF measurement results from different laboratories. It may therefore be necessary to establish a reference/standard for HRTF quality [203].