## 4 Evaluation
### Subjective Evaluation  
Subjective evaluation consists of two major listening tests. For all cases (the overdetermined case, determined case, and underdetermined case), the perceptual quality of speech sources generated in section IV-A-1 corresponds to the case where \(K=\{2,3,4,5,6\}\). The separation is \(\{30^{\circ},\,40^{\circ},\,50^{\circ},\,60^{\circ}\}\), and the source radius is \(1\) m. Note that each separated speech source is evaluated separately by using headphones for playback. A MUSHRA [36] listening test which contains 16 listeners is employed to measure the subjective perceptual quality with four conditions, namely, Ref, Pro-BSS, Pro-SCS, SCS, and Anchor. Condition Ref refers to the original speech sources in each test, which are also served as the hidden references of this MUSHRA test. Condition SCS, Pro-SCS, and Pro-BSS are the same as in the objective evaluation. Condition Anchor is the unprocessed (W channel of the B-format recording) mixed signal. In total, 16 listeners participated in the test. Figure 9: PESQ results (**a**) Anechoic Room; (**b**) Room1; (**c**) Room2. Error bars represent 95% confidence intervals. For each source number and separation, we calculated the average of all tested speeches and results are shown in Figures 13-16. It can be observed that our proposed method achieves significantly higher scores compared to the fixed threshold BSS approach, which uses the same number of microphones as our proposed method. In addition, the PSM scores decreases as the source number increases from 2 to 6, and rises as the separation between the two adjacent sources gets larger. For the cases \(K=2,3\), the scores are always about 0.8, which reaches a nearly excellent quality. For the undermined case, the MUSHRA scores are about 0.7 when the source number is four, while for the cases \(K=5,6\), the quality of the extracted speech is below 0.6 but still over 0.4. This means that the extracted speech is not quite euphonious but can still represent clear and understandable speech. To compare the extracted speech quality with the reference method in objective evaluation further, a MUSHRA test was also employed to measure the subjective quality of the separated speech. Six middle sources from each test group were selected for the listening test. Similarly, the unprocessed (W channel of the B-format recording) mixed signal was used as the anchor and the original speech was used as the hidden reference. Note that each separated speech source is evaluated separately by using loudspeakers for playback in the Anechoic Room, Room 1, and Room 2. Average MUSHRA scores are presented in Figure 17.  
Figure 10: PESQ results. Error bars represent 95% confidence intervals, (**a–d**) represent the results for separations \(\{30^{\circ},\ 40^{\circ},\ 50^{\circ},\ 60^{\circ}\}\), \(SNR=20\) dB, \(R760=0.5\) s.  
It can be seen that a significant improvement in the separation quality is achieved by applying the proposed scheme. The MUSHRA score for the proposed method is of nearly 'excellent' quality, the second best score is about 'good'. It should be noted that we just use one B-format microphone, while C-BSS adopts a pair. The majority of listeners indicated that their choice for the closest match to the reference was based on files which contained the minimal amount of crosstalk and musical distortion. For other conditions, listeners reported that while the target speech is significantly separated from the mixture, there is audible crosstalk from other talkers with higher musical distortion. Figure 11: SDR results. Error bars represent 95% confidence intervals, (**a–d**) represent the results for separations \(\{30^{\circ},\,40^{\circ},\,50^{\circ},\,60^{\circ}\}\), \(SNR=20\) dB, \(RT60=0.5\) s.  
Figure 12: SIR results. Error bars represent 95% confidence intervals, (**a–d**) represent the results for separations \(\{30^{\circ},40^{\circ},50^{\circ},60^{\circ}\}\), \(SNR=20\) dB, \(RT60=0.5\) s.  
Figure 13: MUSHRA test results of \(separation=30^{\circ}\). Error bars with 95% confidence intervals.