## 1 Introduction  
Physics-Informed Neural Networks (PINNs) are a relatively new data-driven solver of partial differential equations (PDEs) [1, 2, 3]. The neural networks' capability to approximate complex functions is their basis for solving partial differential equations. While the idea of using neural networks to estimate PDE solutions dates back to the 1990s, it initially garnered limited attention for various reasons. With the rapid advancements in deep neural network technology, the exponential growth in computing power, and the thriving open deep learning community, PINNs have recently garnered substantial interest and acclaim. PINNs possess several notable advantages that make them a competitive method compared to mature, traditional numerical approaches for PDEs. PINN, as a meshless method, directly embeds mathematical equations into the network structure. The dual reliance on observational data and mathematical models equips PINNs to handle noisy observational data. Moreover, a PINN offers a consistent framework for forward and inverse problems through optimization algorithms [3]. By simply extending the neural network with additional output channels, PINNs can be employed to solve inverse problems. In inverse design, PINNs can impose PDEs as rigorous constraints, enhancing their utility. While neural networks grapple with the curse of dimensionality as problems become more complex, PINNs strive to resolve PDEs and their inversion challenges in domains characterized by intricate geometries and high dimensions, where numerical simulations are notably challenging. PINNs have produced compelling results on various problems in computational science and engineering, such as computational fluid dynamics [4], acoustics [5], solid mechanics [6], elastodynamics [7; 8; 9], and geo-physics [10; 11; 12]. However, training PINNs to achieve fast convergence and accuracy is a persistent challenge. This challenge is intricately linked to the highly complex and non-convexity of the loss function, which makes a PINN hard to train [13; 14]. Furthermore, training PINNs also suffers from spectral bias. The neural networks prioritize learning low-frequency patterns over high-frequency details [15; 16]. When the problem contains high-frequency features, the PINN models often fail to converge to the desired solution due to this phenomenon [17; 18; 19]. PINNs' inherent ability to encapsulate domain knowledge and exploit neural network architectures has made them particularly attractive for simulating complex physical systems. However, as the applications of PINNs extend to problems characterized by high-frequency oscillations and intricate multi-scale phenomena, they face significant hurdles. These challenges often manifest in numerical instability, slow convergence, and increased computational demands, making it imperative to develop strategies that enhance the robustness and efficiency of PINNs in such scenarios. The primary objective of this research is to elucidate the prevailing challenges encountered in PINNs when applied to high-frequency and multi-scale problems. To mitigate these challenges, we investigate the utility of transfer learning. By incorporating transfer learning into the PINN framework, we aim to harness the benefits of pre-trained models and transferable knowledge, potentially enhancing the convergence and accuracy of PINNs for high-frequency and multi-scale applications. Moreover, the choice of optimizer plays a crucial role in training neural networks, including PINNs. Different optimization algorithms possess distinct characteristics and may perform differently regarding convergence speed and solution quality. In this study, we empirically evaluate a range of optimizers to determine their effectiveness in training the foundational model of the PINN. Through a comparative analysis, we seek to identify the optimizer that best suits the specific requirements and challenges of PINNs in the context of high-frequency and multi-scale problems.