## 3 Methodology
### Data Preprocessing  
In developing our fare prediction model, we initiated the process by rigorously cleaning an already high-quality and meticulously detailed ticket transactions dataset to ensure data integrity and accuracy. This dataset, characterized by its comprehensive features and adherence to industry standards, provided a solid foundation for our analysis. The initial stage involved the systematic elimination of duplicate entries and the correction of misaligned or incorrect values, with a specific focus on the _Booking Class_, which varies from \(A\) to \(Z\). We identified records that, although identical in _Travel Date_, _Distance_, _Class of Service_, _Seg Orig_., and _Seg Dest_., diverged solely in _Booking Class_. These duplications, though infrequent, required careful evaluation to determine whether they represented true redundancies or necessary variations in fare classes reflecting different pricing strategies and service levels. Upon review, we found that while the duplicates could be removed, certain variations in _Booking Class_ were critical to retain, as they provided insights into fare adjustments and reflected different service levels and associated costs. This approach helped us maintain essential data variations that are important for accurate fare predictions. By using _Booking Class_ as a reference for identifying duplications and verifying their legitimacy, we reduced unnecessary data redundancy, enhancing the dataset's robustness. Subsequently, we addressed anomalies in the _Booking Class_ feature, where entries were mistakenly replaced by _Class of Service_. To correct these, we engaged domain experts and employed mode imputation, replacing incorrect entries with the most frequently occurring valid booking class within that fare range. The next phase of our data preparation focused on strategically managing missing data to maintain uniformity and completeness across the dataset. For continuous variables such as _Distance_ and _Actual Fare_, we corrected missing values using median values. This choice was guided by the skewed nature of these variables, where the median provides a more robust measure. Specifically, in cases of missing _Distance_ data, we identified similar flight segments by cross-checking our dataset's _Seg Orig_. and _Seg Dest_. to find records with the same route characteristics. This allowed us to accurately impute missing distances based on observed values from identical or similar routes. For categorical variables, we applied mode imputation, filling in missing entries with the most frequently occurring value within similar segments. Following the feature engineering phase, we engaged in data transformation and normalization to meet the requirements of our machine learning algorithms. We implemented two key data transformation techniques: data encoding and data normalization. Categorical features such as _Senson_, _Booking Class_, _Class of Service_, and _Segment Origin_ were transformed using one-hot encoding. For continuous variables like _Distance_ and _Actual Fare_, we applied min-max normalization, scaling the values to a range between 0 and 1. This scaling is achieved using the following formula:  
\[xt=\frac{x-min(X)}{max(X)-min(X)} \tag{1}\]  
where \(min(X)\) and \(max(X)\) are the minimum value and maximum value of the feature \(X\) in the dataset, respectively. After data preprocessing, we obtained a cleaned dataset consisting of 840,158 airline fare transaction records. We divided this dataset into three parts: the training set, the validation set, and the test set, with allocations of 80%, 10%, and 10%, respectively. A random selection method was applied to ensure unbiased distribution among these sets. Table 1 below shows the distribution of records across each set after the division. Table 2 displays further a few sample features employed in the prediction model.