## 1 Introduction  
Stock price prediction is essential in the financial domain, shaping investment strategies, risk management, and market efficiency [1]. Stock price fluctuations in financial markets are influenced by multiple factors, including economic indicators, market sentiment, and policy changes [2, 3], resulting in highly nonlinear and complex characteristics of financial time series data. Accurate prediction of stock price trends remains a critical challenge in finance. Over the years, numerous traditional machine learning techniques, including support vector machines (SVM) [4], decision trees [5], and k-nearest neighbors (KNN) [6], have been employed to address this challenge. While these methodologies have yielded valuable insights and proven effective in certain scenarios, they are encumbered by significant limitations in the context of complex financial markets. Specifically, they frequently fail to adequately capture the intricacies of financial markets [7; 8]. The deficiencies of these models, especially in their capacity to manage nonlinear relationships and long-term dependencies, have prompted researchers to investigate more sophisticated methodologies. In addition to these machine learning approaches, non-AI-based methods such as state modeling have also been applied to financial forecasting tasks. For instance, [PERSON] et al. [9] proposed a state model with binary-temporal representation to effectively model and predict crude oil prices, highlighting the continued relevance of traditional modeling paradigms in specific financial contexts. The advent of deep learning has precipitated the emergence of more sophisticated models, which have been shown to effectively capture nonlinear features in financial data. It is noteworthy that long short-term memory (LSTM) networks [10], convolutional neural networks (CNNs) [11], and recurrent neural networks (RNNs) [12] have been shown to be particularly effective for time series forecasting. LSTM networks have become a subject of particular interest due to their capacity to model temporal dependencies and manage sequential data effectively [13; 14; 15]. Prior research has shown that LSTM models effectively capture short-term market trends, leading to significant improvements in stock price prediction accuracy [16; 17; 18]. Nevertheless, despite their effectiveness, LSTM models are confronted with certain challenges. Firstly, the necessity of manual hyperparameter tuning is a key issue. This may not fully exploit the model's potential and can result in suboptimal performance. Furthermore, LSTM networks are susceptible to gradient explosion issues during the training process, which results in substantial limitations with regard to modelling long-term dependencies [10; 19]. Such constraints have the potential to result in a reduction in the accuracy of predictions. To address these challenges, swarm intelligence optimization algorithms have been integrated into the hyperparameter tuning process for LSTM models. These algorithms mimic collective behaviors in nature, combining local and global search strategies to identify optimal solutions in high-dimensional search spaces [20]. By leveraging swarm intelligence to optimize LSTM hyperparameters, model performance can be significantly enhanced, reducing the time and complexity of manual tuning while improving prediction accuracy [21]. Additionally, the introduction of Transformer models offers an effective solution for capturing long-term dependencies. The self-attention mechanism in Transformers calculates relationships between different positions in the input sequence, enabling the model to flexibly focus on critical parts of the sequence [22]. This effectively mitigates the gradient explosion issues encountered during LSTM training. Combining swarm intelligence for optimizing LSTM hyperparameters with the self-attention mechanism provides a more accurate and efficient solution for tackling complex time series forecasting problems.