# MPFM-VC: A Voice Conversion Algorithm Based on Multi-Dimensional Perception Flow Matching
## 4 Methods
### Multi-Dimensional Perception Network  
Existing flow-matching models typically adopt a U-Net-based Transformer architecture to predict the vector field [17; 41]. In this work, we propose a novel multi-dimensional feature perception network, as illustrated in Figure 4, which is designed to improve the model's adaptability to varying acoustic conditions. Prior to being fed into the proposed feature perception blocks, all inputs undergo an encoding process as follows: First, a Transformer [17]-based sinusoidal positional encoding layer is introduced to generate time embeddings \(E_{t}\). Second, speaker identity features \(I_{spk}\) are projected onto a latent space via a linear layer to produce speaker embeddings \(E_{spk}\). For the complex content representation \(I_{content}\), a Transformer encoder is employed to obtain content embedding \(E_{content}\). Simultaneously, pitch sequences \(I_{pitch}\) are processed through a Transformer embedding layer to yield pitch embeddings \(E_{pitch}\). Additionally, at each time step \(t\) in the flow-matching process, the intermediate sample \(z_{t}\) is computed by using linear interpolation: \(z_{t}=z_{0}\cdot t+z_{1}\cdot(1-t)\). Finally, the above multimodal embeddings are concatenated along the feature dimension and are used as inputs to the feature perception block. The multi-dimensional perception network proposed in this study adopts a modular architecture composed of multiple feature-awareness blocks. While all blocks share the same structural design, they utilize distinct condition matrices to capture diverse contextual information. Each block integrates conditional information through two complementary mechanisms:  
(1) **Scale Shift**, defined in Equation (4), modulates the feature distribution by applying learnable scaling and bias transformation. (2) **Gated Shift**, formulated in Equation (5), employs a gated unit to dynamically regulate the flow of information through adaptive feature reweighting. The computations of these mechanisms are given by  
\[\text{ScaleShift}(x)=x\cdot\gamma+\beta \tag{4}\]  
\[\text{GateShift}(x)=x(1+\alpha)+\alpha \tag{5}\]  
where \(\alpha\), \(\gamma\), and \(\beta\) are learnable parameters of the network, and \(x\) denotes the input matrix. Figure 3: The overall framework of MPFM-VC. Notably, the proposed network, which serves as the acoustic model, does not directly predict the mel spectrogram; instead, it predicts the vector field used in the ODE formulation of flow matching. Specifically, the embedding of conditional information is carried out through a hierarchical processing mechanism. First, the time encoding is transformed by a one-dimensional convolutional layer to produce two learnable parameters, \(\gamma_{t}\) and \(\beta_{t}\), which are used for time-specific feature modulation. Next, the condition matrix is passed through a multi-layer perceptron (MLP) and mapped to six adaptive parameters \((\alpha_{1},\gamma_{1},\beta_{1},\alpha_{2},\gamma_{2},\) and \(\beta_{2})\), enabling conditional feature encoding. The network then executes the following steps sequentially: (1) the application of layer normalization to the input feature matrix followed by conditional feature modulation; (2) cross-dimensional feature interaction based on a multi-head self-attention mechanism; (3) the application of a gated control mechanism to regulate the flow of information. To ensure training stability, a residual connection is introduced at the end of the block. This not only preserves the original feature information but also effectively mitigates issues related to gradient instability. In the proposed multi-dimensional perception network, the condition matrix is used to embed auxiliary information, including energy (\(I_{\textit{energy}}\)), prosody (\(I_{\textit{prosody}}\)), pitch (\(I_{\textit{pitch}}\)), and speaker voiceprint (\(I_{\textit{spk}}\)). These conditioning features are incorporated into different feature-awareness blocks, each responsible for modeling a specific type of information.