## 3 Method Architecture
### Progressive Contrast Constraint  
After obtaining the feature vectors of the interactive fusion of pinyin and semantics, and the feature vectors of the interactive fusion of part of speech and semantics, in order to effectively constrain the model to achieve feature alignment for these three types of heterogeneous features, this paper proposes a progressive contrastive learning constraint. Its learning framework is shown in Figure 3.  
This paper proposes a two-stage contrastive learning framework to address the heterogeneity problem between the main features and auxiliary features (including auxiliary feature 1--pinyin features; auxiliary feature 2--POS features) in text processing, aiming to effectively integrate these features and improve the learning efficiency and stability of the model.  
The phase 1 comparison is the interactive feature fusion. In this phase, this paper first focuses on how to promote effective interaction between features while maintaining feature discriminability. By designing a multi-head attention mechanism, the main feature (the most discriminative semantic feature) and the auxiliary feature generate interactive feature A and interactive feature B, respectively. Specifically, this mechanism promotes the interaction between the pinyin feature and the semantic feature to form the pinyin-semantic interactive fusion feature; it also promotes the interaction between the POS feature and  
Figure 3: The framework of progressive contrastive learning constraint.  
the semantic feature to form the POS-semantic interactive fusion feature. As described in the summary in Section 3.5, this process enhances the information expression within each feature and lays the foundation for subsequent deep integration. The phase 2 comparison is the contrast constraint optimization. The interactive fusion features generated in the first phase are further optimized through the contrast constraint mechanism. Specifically, the pinyin-semantic interactive fusion feature (fusion feature 1) and the POS-semantic interactive fusion feature (fusion feature 2) are placed in the same contrast learning framework to enhance the model's ability to recognize the differences and commonalities of heterogeneous features. This comparison not only consolidates the mutual understanding and complementation between features but also promotes the model's comprehensive grasp of complex contextual information, ensuring excellent generalization performance in a diverse feature space.  
This progressive contrastive learning strategy improves the stability of the model when processing heterogeneous features by gradually deepening the interaction and fusion of features in stages while overcoming the training oscillation phenomenon caused by feature heterogeneity when directly minimizing the cosine distance. In addition, by gradually introducing contrast constraints, the model is guided to learn a heterogeneous feature fusion mode that is more in line with the text context, thereby demonstrating the ability to distinguish and generalize in the feature space and improving the overall performance of the model. This method provides new ideas and practical paths for processing the fusion of heterogeneous features.