# Multi-Level Feature Dynamic Fusion Neural Radiance Fields for Audio-Driven Talking Head Generation
## 1 Introduction  
Audio-driven talking head generation is an elaborate procedure that involves realistic human-like faces synchronizing with the audio. The generation of audio-visual content [1] has a wide range of applications, such as film production, online education, and virtual customer care. Due to rapid advancements in deep learning and computational graphics, researchers have dedicated themselves to creating talking head videos that accurately match the source identity using input audio sequences [2, 3, 4]. Recently, researchers have incorporated neural radiation fields (NeRF) [5] into their work due to their exceptional ability to generate 3D content. A NeRF-based approach [6, 7] uses multi-layer perceptron (MLP) to transform audio and image features into dynamic neural networks, creating specific talking heads. These methods have made significant advancements in picture quality and in the preservation of personality identification. However, the lengthy training time and slow inference speed of earlier approaches limited their widespread use. Recent studies [8; 9; 10] in the field of neural representation have demonstrated that substituting a portion of the MLP with sparse feature grids leads to substantial speed improvements compared to the vanilla NeRF approach. Instant-NGP [8] uses hash-encoded voxel grids to model static scenes, allowing efficient and high-quality rendering in compact models. RAD-NeRF [11] initially used the technique to generate talking heads driven by audio and obtained the best picture quality and rendering speed performance. Hash conflicts impose limitations on the technology's rendering quality and model convergence time when applied to modeling 3D dynamic heads. To address the issue, ER NeRF [12] incorporates a tri-plane hash representation into this job. This approach greatly improves the accuracy of facial motions, while also ensuring efficient model training and rendering. However, previous approaches [11; 12] that use single-level hash representations for image sampling may not capture more detailed features associated with facial motion, thus limiting the impact on image rendering. Developing a concise and expressive representation approach is critical to better depict the dynamic heads of specific features. Although ER-NeRF [12] works well with tri-plane hash representation, when modeling 3D dynamic heads from monocular 2D images, single-level hash grids sample all image features equally, lowering the talking head's rendering quality. To overcome this problem, we propose a unique multi-level tri-plane hash representation based on NeRF. We initially project the 3D spatial coordinates into three orthogonal 2D planes and then utilize three sets of tri-pane hash with varying limitation resolutions to capture different degrees of detail and global data. Then, we design a dynamic feature fusion network to remove duplicate features at different levels, thereby building accuracy and rich image feature representations. Meanwhile, the method allows the network to better capture the mapping between audio and face features, precisely reconstruct head anatomy, and develop highly detailed and lifelike head motions. To better capture the relationship of the cross-modal mapping between audio and image features, SD-NeRF [13] has designed a spatially adaptive cross-attention mechanism. Inspired by multi-head attention and tri-plane hashing, we proposed an efficient audio-visual fusion module, which uses position encoding to associate audio features with spatial information. The module adopted a novel and efficient way to learn the mapping between the image features of different plane regions and the audio, as shown in Figure 1. Therefore, the method captures more information from the dynamic area of the head to produce accurate and natural facial movements. At the same time, the static part remains stable and unaffected by the speech signal. This article's contributions are as follows:  
1.