  Stereophonic audio, especially binaural audio, plays an essential role in
immersive viewing environments. Recent research has explored generating
visually guided stereophonic audios supervised by multi-channel audio
collections. However, due to the requirement of professional recording devices,
existing datasets are limited in scale and variety, which impedes the
generalization of supervised methods in real-world scenarios. In this work, we
propose PseudoBinaural, an effective pipeline that is free of binaural
recordings. The key insight is to carefully build pseudo visual-stereo pairs
with mono data for training. Specifically, we leverage spherical harmonic
decomposition and head-related impulse response (HRIR) to identify the
relationship between spatial locations and received binaural audios. Then in
the visual modality, corresponding visual cues of the mono data are manually
placed at sound source positions to form the pairs. Compared to
fully-supervised paradigms, our binaural-recording-free pipeline shows great
stability in cross-dataset evaluation and achieves comparable performance under
subjective preference. Moreover, combined with binaural recordings, our method
is able to further boost the performance of binaural audio generation under
supervised settings.
