## 3 BAT-Transformer: Bus Arrival Time with Transformer Encoder
### Stage 1: Data Preprocessing  
Figure 3 shows the data-collection and -preprocessing pipeline used in this study. The bus's current location, arrival information, and route information were collected through the OPEN API provided by the Seoul Metropolitan Government's public data portal. The data-collection period spanned from 29 April to 20 May, with information gathered at intervals of 2-4 min. This study focused on Gangnam-gu, a region characterized by high  
Figure 2: BAT-Transformer architecture. population mobility, involving 55 buses, 35 trunk buses, and 20 branch buses. The collected data consisted of 19 attributes: RecordedAtTime, DirectionRef, PublishedLineName, OriginName, OriginLat, OriginLong, DestName, DestLat, DestLong, VehicleRef, VehicleLocation.Lat, VehicleLocationLong, NextStopPointName, NextStopTime, DistanceFromStop, FullSectDist, ArrivalProximityTime, and RoadType. The data collected through each API comprised approximately 4 million instances. These were consolidated into a single dataset and filtered to derive meaningful observations. Note that only data reflecting bus movements between stops were used. To achieve this, data samples with the value \"at-stop\" in the ArrivalProximityText field were removed. Additionally, data samples marked as \"1 stop before\" in the NextStopStation field were excluded, and a new variable, Bus_stop, was created by removing samples with the value \"1 stop before\" in the ArrivalProximityText field. This resulted in a total of 1.5 million data records. From this dataset, various input features, such as bus routes, stops, time-related variables, and distance-related variables, were extracted. Figure 4 shows the structure of the input and output variables used in the model. The input variables were broadly categorized into numerical, time, and binary types, whereas the output variable was NextStopTime, which the model was tasked with predicting. After feature extraction, we performed a feature selection process to identify features that would have a significantly impact on the performance of the model. The feature selection process focused on removing unnecessary features while retaining the most important ones to reduce model complexity, mitigate multicollinearity, and increase prediction accuracy. The bus routes, which are categorical features, were initially represented as integer values to differentiate between each route. However, this approach was not only inefficient, but the numeric values could lead to unintended relationships between routes. To address this, we converted the bus routes to binary features using one-hot encoding. Consequently, 232 additional binary features were generated, aiding the model in learning the differences  
Figure 4: Input and output data features. Figure 3: Stage 1: data preprocessing. between routes. For example, suppose you have three bus routes (Route A, Route B, and Route C), which were originally represented as integers 1, 2, and 3. After applying one-hot encoding, each route was represented as a binary vector: Route A is represented as [1, 0, 0], Route B as [0, 1, 0], and Route C as [0, 0, 1]. This allowed each route to be uniquely identified and removed the order relationship between routes. As a result, the model was better able to learn and distinguish the characteristics of each route (e.g., different transportation conditions, number of passengers, average travel time, etc.). Vehicle_Type, which represents the order of stops, plays an important role in understanding transportation flow and passenger demand. This variable was kept because it represents the location of the bus and the level of service at each stop, and was processed using an integer encoding. Features such as time-related variables RecordAtTime, ScheduledArrivalTime, Rush_hour, and Day_type were retained because they are essential to understanding transportation patterns at each time of day.