## 2 Related Work
### Attention Mechanism  
The attention mechanism has become a crucial component of deep learning models, particularly for sequence-based tasks. The core idea of attention is to assign different weights to each element in the input sequence, thereby allowing the model to focus on more important elements. This approach helps the model handle long-range dependencies more effectively, and prioritizes the learning of important information. This concept was first introduced by [PERSON] et al. (2015) in the context of machine translation, where it significantly improved translation quality [28]. Following this, various modifications and enhancements were proposed, including the work by [PERSON] et al. (2015), who introduced effective approaches for further improving machine translation performance using attention-based methods [29].  
Among these advancements, self-attention and multi-head attention, which play critical roles in transformer models, have garnered significant attention. The self-attention mechanism allows the model to consider the relationships between all the elements in the input sequence simultaneously, enabling it to generate more refined and contextually relevant representations. [PERSON] et al. (2018) applied these mechanisms to neural machine translation systems, leading to substantial improvements in translation tasks and for a wide range of natural language processing (NLP) tasks [30]. The impacts of these developments extend beyond NLP. [PERSON] et al. (2017) proposed the use of self-attention to structure sentence embeddings, and demonstrated that this approach effectively captures complex relationships within textual data [31]. This capability is particularly relevant to time-series data such as bus arrival time prediction, for which understanding the intricate relationships between different data points is crucial. Attention mechanisms are particularly advantageous for handling complex time series data such as transportation information. [PERSON] et al. (2022) developed a bus arrival time-prediction model using attention mechanisms, achieving a higher accuracy than traditional models [32]. Their research showed that attention mechanisms can effectively capture important patterns in bus-operation data and accurately model the relationships between bus stops. This highlights the importance of attention mechanisms in the learning and prediction of complex patterns. The application of attention mechanisms is not limited to NLP or time-series data, but extends to various domains. [PERSON] et al. (2018) introduced a method for performing language-understanding tasks using a directional self-attention network, without relying on recurrent or convolutional neural networks [33]. This study demonstrated the broad applicability of attention mechanisms across different fields. Similarly, [PERSON] et al. (2021) proposed an efficient attention mechanism called the performer, which reduces the computational complexity of traditional self-attention while maintaining the performance [34]. This evolution of attention mechanisms demonstrates how they continue to effectively address computational challenges.  
These studies provide both a theoretical foundation and practical validation of attention mechanisms, supporting their application in improving bus arrival time-prediction models. The advanced processing capabilities of attention mechanisms render them suitable for handling complex data patterns, particularly in real-time prediction systems. This study aims to leverage the strengths of attention mechanisms to further enhance the accuracy of bus arrival time predictions.