## 2 Objective Evaluation Test
### Panning Test  
Criterion: The direction of the virtual sound source in the stereo-to-multichannel upmix should correspond to the direction of the virtual sound source in the initial stereo configuration. This is accompanied with the result that the spatial representation of sound events is preserved true to original. Figure 1: Schematic procedure of the evaluation tests. The panning test (see Figure 2) is conducted in two versions. The evaluation score \(score_{PT}\) results from the evaluation scores of the time- and frequency-independent panning test \(score_{PT1}\) and the time- and frequency-dependent panning test \(score_{PT2}\), weighted with \(g_{PT1}\) and \(g_{PT2}\), given by  
\[score_{PT}=\frac{g_{PT1}\cdot score_{PT1}+g_{PT2}\cdot score_{PT2}}{g_{PT1}+g_{ PT2}} \tag{2}\]  
Initially, at an interval of \(1^{\circ}\) virtual test sound sources \(\varphi_{i}\) are defined with angles from \(-30^{\circ}\) to \(30^{\circ}\) according to the reference loudspeaker arrangement [10]. With the tangent law as the modified stereophonic law of sines, the two panning coefficients \(a_{L,i}\) and \(a_{R,i}\) can be calculated from \(\varphi_{i}\)[11]. A signal, weighted with the left panning coefficient \(a_{L,i}\), represents the left part of a stereo signal \(x_{L,i}\). A signal, weighted with the right panning coefficient \(a_{R,i}\), represents the right part of a stereo signal \(x_{R,i}\). For every angle \(\varphi_{i}\) a stereo test signal is generated as input signal for the tested upmix. This is done by multiplying the resulting panning coefficients \(a_{L,i}\) and \(a_{R,i}\) with white Gaussian noise \(x_{wgn}\) according to  
\[ \begin{split} x_{L,i}(n)&=a_{L,i}\cdot x_{wgn}(n)\\ x_{R,i}(n)&=a_{R,i}\cdot x_{wgn}(n)\end{split}  \tag{3}\]  
With the output signals of the upmix algorithm for the three front channels \(x_{FL,i}\) (front left), \(x_{C,i}\) (center) and \(x_{FR,i}\) (front right), two panning coefficients \(\hat{a}_{L,i}\) and \(\hat{a}_{R,i}\) are determined, and from these panning coefficients the direction of the virtual sound source \(\hat{\varphi}_{i}\) is calculated. Note that two-to-five upmix algorithms are tested, but only the three front channels are used for the panning test. The difference \(\Delta\varphi_{i}=\varphi_{i}-\hat{\varphi}_{i}\), which is the deviation of the angle of the virtual sound source of the upmix from the defined test signal, serves as the basis for evaluation. The score of the time- and frequency-independent panning test \(score_{PT1}\) is calculated from the mean of the normalized absolute deviation of all test cases with \(\varphi=(\varphi_{1},\dots,\varphi_{m})\in W(1\leq i\leq m)\) and \(W=\{n\in\mathbb{Z}|-30\leq n\leq 30\}\) given by  
\[score_{PT1}=1-\frac{1}{\max(|\varphi|)}\cdot\frac{1}{m}\sum_{i=1}^{m}|\Delta \varphi_{i}| \tag{4}\]  
The division by \(\max(|\varphi|)\) is needed to normalize the mean of the deviations so that the evaluation score assumes values ranging from 0 to 1. In the case of the time- and frequency-dependent panning test, the angle of the virtual test sound source is randomly generated in the range of \(-30^{\circ}\) to \(30^{\circ}\). This is done at any time \(n\) and frequency \(k\) in a time-frequency representation with the help of a short-time Fourier transform (STFT). So, the ability of the upmix to respond to fast changes of the virtual sound source should be tested. The score  
Figure 2: Block diagram: Time- and frequency-independent panning test. is calculated from the mean of the normalized absolute deviation across all \(N\) times and \(K\) frequencies with \(\varphi_{max}\) as the maximum absolute value of all angles given by  
\[score_{PT2}=1-\frac{1}{\varphi_{max}}\cdot\frac{1}{N}\frac{1}{K}\sum_{n=1}^{N} \sum_{k=1}^{K}\left|\Delta\varphi(n,k)\right| \tag{5}\]  
The evaluation allows the comparison of the angle of the virtual sound source of the stereo input signal with the angle of the virtual sound source of the multichannel output signal (see Figure 3).