###### Abstract
With these advancements, speech recognition systems have seen substantial improvements in accuracy, robustness, and adaptability to dynamic environments, particularly in handling multiple languages, dialects, and complex background conditions. However, despite the significant progress made in speech recognition technology, several challenges and pain points remain. One notable issue is the lack of robustness. Speech recognition systems experience a significant drop in accuracy when faced with noisy environments and varying background noise [5]. This limitation hinders the performance of speech recognition technology in complex and diverse real-world applications. Another challenge lies in the shortcomings of current decoding algorithms. These algorithms have limitations in handling the complexity and diversity of natural language, which affects the accuracy of speech recognition systems and constrains their response speed and efficiency in real-time applications. This is particularly evident when dealing with continuous speech and complex commands, where the efficiency and accuracy of existing algorithms need improvement [6]. To address these issues, this paper proposes the following innovative methods: First, during the model training phase, to alleviate overfitting and local optima, we introduce an innovative network architecture and propose a loss-weighted model fusion approach. This method combines multiple model structures by adjusting their weights during the convergence phase, enhancing the model's generalization ability and robustness. Second, to address the balance between accuracy and efficiency in decoding algorithms, we design a dynamic adaptive decoding algorithm. This algorithm integrates CTC and attention mechanisms using nonlinear functions, enabling more precise capture of the complex relationships and interactions in speech data. Furthermore, to improve decoding efficiency, we introduce a dynamically adjusted penalty mechanism that limits the model's reliance on a single decoder, thereby enhancing the robustness of the decoding process and accelerating the decoding speed. The structure of this paper is divided into six sections, arranged as follows: Section 2 presents the related work, reviewing the development history of speech recognition models, discussing challenges in model performance, decoding algorithms, and the scarcity of datasets, and outlining the solutions adopted in this paper. Section 3 introduces the methodology, detailing the design concepts behind model fusion and the innovations in the decoding algorithm. Section 4 describes the experimental setup, including the model parameter configuration, dataset sources, and performance evaluation metrics. Section 5 presents the experimental results, exploring the effects of model fusion and decoding algorithm optimization, conducting ablation experiments, and comparing the performance of the proposed method with mainstream network models. Section 6 concludes the paper based on the experimental results. Section 7 discusses future work, providing a critical examination of the limitations inherent in the proposed methods and suggesting strategies to address these challenges, while identifying avenues for further exploration.