## 4 The Proposed TransTLA Method
### Attention Mechanism Layer  
In the task of household appliance sales forecasting, it is crucial to recognize that not all extracted features possess equal importance, and existing features may not be sufficient to capture critical and subtle information. Thus, optimizing feature representation is essential. To address this, we incorporate the attention mechanism to enhance the model's capacity to capture pivotal information within sales data, thereby improving the predictive performance. The attention mechanism is inspired by the human visual attention system, which automatically focuses on the most critical parts when processing a large amount of information [24]. The core of the attention mechanism lies in computing a weighted sum based on the relevance of each element in the input sequence to a central query. Define an attention pooling function \(f\), as shown in Equation (11). It accepts a query vector \(\mathbf{q}\in\mathbb{R}^{q}\) and a set of \(m\) \"key-value\" pairs, where each key is \(\mathbf{k}_{i}\in\mathbb{R}^{k}\) and each value is \(\mathbf{v}_{i}\in\mathbb{R}^{v}\). \[f(\mathbf{q},(\mathbf{k}_{1},\mathbf{v}_{1}),\dots,(\mathbf{k}_{m},\mathbf{v} _{m}))=\sum\limits_{i=1}^{m}\alpha(\mathbf{q},\mathbf{k}_{i})\mathbf{v}_{i} \in\mathbb{R}^{v} \tag{11}\]  
The attention weight \(\alpha(\mathbf{q},\mathbf{k}_{i})\) is a scalar derived from the attention scoring function \(a\) and normalized through the softmax function to ensure the sum of all weights equals 1, as shown in Equation (12). The function \(a\) maps the query vector \(\mathbf{q}\) and key vector \(\mathbf{k}_{i}\) to a scalar. \[\alpha(\mathbf{q},\mathbf{k}_{i})=\frac{\exp(a(\mathbf{q},\mathbf{k}_{i}))}{ \sum\limits_{j=1}^{m}\exp(a(\mathbf{q},\mathbf{k}_{j}))} \tag{12}\]  
In the proposed TransTLA, we utilize the scaled dot-product attention mechanism to compute attention weights, enabling the model to more precisely predict home appliance sales while maintaining computational efficiency. This mechanism calculates the square root of the dimension of the key vectors, which is  
\[a(\mathbf{q},\mathbf{k}_{i})=\frac{\mathbf{q}\mathbf{k}_{i}^{\top}}{\sqrt{d_ {k}}}. \tag{13}\]  
Compared to the traditional dot-product attention, this scaling operation stabilizes the gradients and mitigates numerical stability issues associated with high-dimensional key vectors, thereby enhancing the robustness of the attention mechanism and its efficacy in processing inputs of diverse dimensions. However, attention mechanisms can be computationally expensive, particularly with very long sequences. The specific implementation of the attention mechanism is illustrated in Figure 5. Specifically, the hidden state of the final time step from the LSTM network serves as the query, while the entire sequence of hidden states output by the LSTM network is fed into the attention mechanism layer as the key and value. This design enables the model to assess the significance of each time step in the sequence while considering long-term dependencies. By focusing on the critical components of the LSTM output, the model can more accurately capture both long-term patterns and short-term dynamics within the sales sequence data, which is essential for precise sales forecasting. The scaled dot-product attention mechanism involves a key tensor \(\mathbf{K}\) derived from a nonlinear transformation using the tanh activation function. This key tensor interacts with the query tensor \(\mathbf{Q}\) to compute attention scores that reflect the similarity between each time step and the last hidden state. The softmax function is then applied to these scores to yield the attention weights \(\alpha\). These weights are used to perform a weighted sum with the value tensor \(\mathbf{V}\), producing a context vector that aggregates the features of each time step according to their importance, as depicted in Equation (14). \[\mathbf{c}=\sum_{i=1}^{t}\alpha_{i}\cdot\mathbf{V}_{i} \tag{14}\]  
Instead of directly utilizing the weighted context vector, we concatenate it with the last hidden state.