## 1 Introduction  
The shipping industry is a significant source of global greenhouse gases, contributing 2.89% of global carbon emissions [1]. The IMO adopted the 2023 IMO Strategy on Reduction of GHG Emissions from Ships, which sets explicit phased targets for reducing both the total amount and intensity of ship carbon emissions, aiming for near-zero emissions by around 2050. The IMO has introduced a series of short-term, medium-term, and long-term measures to promote emission reductions. A key short-term measure is reducing the carbon emission intensity of ships [2]. From 1 January 2023, ships with a gross tonnage exceeding 5000 engaged in international navigation must comply with the Operational Carbon Intensity Index (CII) rating, which increases by 2% annually until 2026 (BIMCO, 2022). Ships that fail to meet the required ratings will need to make necessary adjustments, which could impact their operations, leasing, and sales [3]. Accurately forecasting ship carbon intensity and its rating is essential for effectively managing and controlling ship CII ratings [4]. The key to predicting a ship's carbon intensity lies in forecasting its fuel consumption. By converting fuel consumption into carbon emissions and considering the sailing distance and capacity of the ship, the carbon emission intensity can be calculated. Ship carbon intensity can be measured on various time scales, including hourly, daily, and annually, with annual carbon intensity being the critical data for determining the ship's CII rating. The fuel consumption of ships is not only closely related to the operating cost of the ship, but it also has a significant connection with the ship's carbon emissions. Hence, the research on ship fuel consumption prediction has received wide attention [5; 6; 7; 8]. With the increasing availability of data, more researchers are using machine learning techniques to predict ship fuel consumption. [PERSON] et al. (2018) employed the Least Absolute Shrinkage and Selection Operator (LASSO) regression algorithm, which proved particularly effective. The results demonstrated that the LASSO regression model had a superior prediction performance when compared to neural networks and support vector regression [9]. [PERSON] et al. (2018) investigated the Artificial Neural Network (ANN) model and found that it a had higher prediction accuracy for main engine fuel consumption than polynomial regression and support vector machines [10]. [PERSON] et al. (2022) examined the ridge regression model with four different datasets, including Automatic Identification System (AIS) data, Measurement, Reporting and Verification (MRV) data, and MRV-normalized data. They found that the model based on MRV reports achieved the optimal result [11]. [PERSON] et al. (2022) compared the results of various prediction models under different combinations of data source such as nautical logs, meteorological data, and AIS data [12; 13; 14]. [PERSON] et al. (2020) also investigated several fuel consumption prediction models, for example, the kernel ridge regression, Bayesian ridge regression, and Adaboost. They found that the ridge regression model had greater accuracy [15]. However, there are not too many studies that focuses on applying deep learning methods to ship fuel consumption prediction. In 2017, Google put forward the Transformer model based on the self-attention mechanism, which enhanced the accuracy of 11 Natural Language Processing (NLP) tasks [16]. Since then, the self-attention mechanism has been introduced in many fields and scenarios, achieving remarkable research results. For instance, [PERSON] et al. (2022) proposed a prediction model based on Long Short-Term Memory (LSTM) and the spatial attention mechanism, which effectively enhanced the accuracy of power consumption prediction in the raw cement material grinding system [17]. [PERSON] et al.