## 3 Proposed Frequency-Dependent Amplitude Panning for Stereophonic Image Enhancement
### Azimuth-Frequency Analysis Using Time Delay  
A stereo signal recorded using a stereo omni-directional microphone array, \(x_{L}(n)\) and \(x_{R}(n)\), can be represented as a delayed and attenuated version of the desired signal, \(s(n)\), such as [21]  
\[\left[ \begin{array}{c}x_{L}(n)\\ x_{R}(n)\end{array} \right]=\left[ \begin{array}{c}a_{L}s(n)\\ a_{R}s(n-\tau)\end{array} \right]+\left[ \begin{array}{c}v_{L}(n)\\ v_{R}(n)\end{array} \right], \tag{3}\]  
where \(v_{L}(n)\) and \(v_{R}(n)\) are ambient noise recorded by the left and right microphones, respectively. In addition, \(a_{L}\) and \(a_{R}\) are the respective attenuation factors, and \(\tau\) is the relative time delay measured between the left and right microphones. Note here that Equation (3) is designed using the far-field model [18; 19], because the spacing between the stereo omni-directional microphones is small. Moreover, we can assume \(a_{L}=a_{R}\approx~{}1\)[22]. Applying an \(N\)-point STFT to Equation (3) provides the following relationship:  
\[\mathbf{X}=\mathbf{d}S(k)+\mathbf{V}, \tag{4}\]  
where \(\mathbf{X}^{T}=\left[ \begin{array}{cc}X_{L}(k)&X_{R}(k)\end{array} \right]\) and \(\mathbf{V}^{T}=\left[ \begin{array}{cc}V_{L}(k)&V_{R}(k)\end{array} \right].\) In addition, \(S(k)\) is the \(k\)-th spectral component of \(s(n)\), and \(\mathbf{d}\) is a steering vector of  
\[\mathbf{d}^{T}=\left[ \begin{array}{cc}1&\exp\left(-j\frac{2\pi k\tau}{N} \right)\end{array} \right], \tag{5}\]  
where \(\tau\) can be determined by the speed of sound \(c\), the spacing between the microphones \(l\), and the direction of the source \(\theta\), as \(\tau=(f_{s}/c)l\text{sin}\theta\), where \(f_{s}\) is the sampling rate. Thus, we have the following equation:  
\[\mathbf{d}^{T}=\left[ \begin{array}{cc}1&\exp\left(-j\frac{2\pi k}{N}\frac{f _{s}}{c}l\text{sin}\theta\right)\end{array} \right]. \tag{6}\]  
If \(\theta\) is known, we can separate \(S(k)\) and \(\mathbf{d}\) from Equation (4). Then, we can modify \(\mathbf{d}\) by replacing \(\theta\) with another value to improve the obtained stereophonic images. This is because the listener cannot feel the actual direction of \(S(k)\) when two stereo microphones are placed very close together. In practice, it is difficult to separate the direction \(\mathbf{d}\) and source \(S(k)\), and it is even more difficult to do so under ambient noise conditions and/or with multiple sound sources [23]. Therefore, instead of separating the sound source and its steering vector in this paper, we apply a panning law to the recorded signal \(\mathbf{X}\), with the estimated direction. To estimate the source direction, we consider the time delay \(\tau\) in Equation (3) using the stereo signal \(x_{L}(n)\) and \(x_{R}(n)\), where we have assumed that \(v_{L}(n)\) and \(v_{R}(n)\) are negligible under high signal-to-noise ratio (SNR) conditions. In other words, the time delay is estimated as \(\hat{\tau}=\text{argmin}_{\tau}\left|x_{L}(n)-x_{R}(n-\tau)\right|\). We can then extend this concept in the frequency domain, as:  
\[\hat{\tau}(k)=\underset{\tau}{\text{argmin}}\left|X_{L}(k)-e^{-j\frac{2\pi}{N} k\tau}X_{R}(k)\right|. \tag{7}\]  
In this paper, \(\tau\) in Equation (7) can be considered as a function of the direction \(\theta\). Therefore, the right-hand side of Equation (7), which is a function of the frequency \(k\), and the direction \(\theta\), is referred to as an A-F plane and defined as [18]. \[AF(k,\theta)=\left|X_{L}(k)-e^{-j\frac{2\pi}{N}k\tau(\theta)}X_{R}(k)\right|. \tag{8}\]We can estimate the direction \(\hat{\theta}(k)\) so that \(AF(k,\theta)\) is minimized at the \(k\)-th frequency bin. However, when \(AF(k,\theta)\) is used for estimating \(\hat{\theta}(k)\), many local minima exist. To mitigate this problem, a smoothing window is applied to \(AF(k,\theta)\) prior to estimating \(\hat{\theta}(k)\), such that:  
\[AF_{s}(k,\theta)=\frac{1}{B(k)+1}\sum_{m=k-B(k)/2}^{k+B(k)/2}AF(m,\theta), \tag{9}\]  
where \(B(k)\) corresponds to a critical bandwidth of the auditory filter [2].