Havayolu yolculuk deneyimini iyileştirmek için makine öğrenmesi yöntemleriyle uçuş gecikmesi tahmini
Machine learning techniques for enhancing airline passenger experience through flight delay prediction
Yazar: ESMA ERGÜN
Danışman: DR. ÖĞR. ÜYESİ SÜHA TUNA
İstanbul Teknik Üniversitesi / Bilişim Enstitüsü / Bilişim Uygulamaları Ana Bilim Dalı / Bilgi ve Haberleşme Mühendisliği Bilim Dalı
Derece: Yüksek Lisans, 2024
Konu: Bilgisayar Mühendisliği Bilimleri-Bilgisayar ve Kontrol = Computer Engineering and Computer Science and Control

Özet:
Havacılık endüstrisi, milyonlarca insanın seyahat etmesine olanak sağlayan önemli bir sektördür. Havaalanları, hava trafik kontrolü, hava koşulları, uçak bakımı, personel düzenlemeleri gibi faktörler, uçuşların planlandığı gibi gerçekleşmesini etkileyebilir.
Uçuş gecikmeleri ve iptalleri, havayolu şirketlerini maliyet artışı, operasyonel sıkıntılar ve müşteri memnuniyetsizliği ile karşı karşıya bırakabilir. Yolcular ise zaman kaybı, alternatif ulaşım arayışı ve planlarının aksaması gibi sorunlarla karşılaşabilirler.
Bu çalışmada, Amerika Birleşik Devletleri Ulaştırma Bakanlığı tarafından paylaşılan uçuş verileri kullanılarak uçuş aksaklıklarının çeşitli makine öğrenmesi teknikleri ile tahmin edilmesi hedeflenmektedir. Bu tahminlerin yolculara sunulması ile doğru uçuş zamanı ve havayolu seçimi konusunda yol gösterici olacaktır.
Bu çalışma, yolcular için bir uygulama geliştirilmesine olanak sağlarken, havayolu ve havaalanlarının operasyonel süreçlerini optimize etmelerine de yardımcı olabilir. Ayrıca, aksaklıkları öngörüp önleyerek havayolu şirketlerinin tazminat ödemelerinden tasarruf etmelerine de katkıda bulunabilir.
 Bu çalışmada uçuş ve hava durumu verileri birleştirilerek kullanılmıştır. Uçuş verisi Amerika Birleşik Devletleri Ulaştırma Bakanlığı'nın halka açık olarak paylaşılan "Airline On Time Performance" isimli veri tabanından, hava durumu verisi de Ulusal Okyanus ve Atmosfer İdaresi Servisi(NOAA)'nden elde edilmiştir. Amerika Birleşik Devletleri'nin en yoğun havaalanlarından biri olan John F. Kennedy Uluslararası Havalimanı (New York, JFK) seçilmiş ve bu havalimanından kalkan uçuşlar incelenmiştir.
Veri önişleme adımında, elde edilen veriler modellerin eğitiminde kullanılmak üzere hazırlanmıştır. Bu hazırlık sürecinde korelasyonların incelenmesi, veri tamamlama, ayrıştırma, öznitelik seçimi, öznitelik birleştirme, örnek seçimi gibi birçok önişleme tekniği kullanıldı.
Veri kümesinde pozitif örneklerin sayısı negatif örneklerin sayısından çok daha fazla olduğu için veri dengeleme amacıyla SMOTE ve aşağı örnekleme(Undersampling) yöntemleri uygulandı.
Makine öğrenmesi, sınıflandırma ve regresyon yöntemlerinden L-GBM, SVM, MLP, Naives Bayes, Lojistik Regresyon, KNN yöntemleri kullanılmıştır. Bu modellerde parametre optimizasyonunu sağlamak üzere Grid Search CV metodu çapraz doğrulama yöntemi ile uygulandı.
Değerlendirme metrikleri olarak sınıflandırma tekniklerinde kullanılan Doğruluk, Kesinlik, Duyarlılık ve F1-Skoru üzerinde duruldu. Modellerin birbirleriyle karşılaştırılması için ROC Eğrileri ve ROC-AUC kriteri kullanıldı.
Tüm modeller ve örnekleme metotları arasında en iyi AUC değerini L-GBM'in orijinal dengesiz veri kümesinde verdiği görüldü. F1-Skor açısından incelendiğinde de en yüksek skorun 0,567 ile SVM'de olduğu ve 0,003 farkla 0,564 değeri ile de L-GBM'in onu takip ettiği görüldü.
Çalışma zamanları açısından incelendiğinde ise L-GBM'in 0.3 saniye değeri ile diğer modellerden çok daha hızlı olduğu görüldü.
Bu çalışmada, çeşitli sınıflandırma teknikleri kullanıldı ve bu tekniklerin performansı incelendi. L-GBM Modeli, hesaplama hızı açısından en etkili bulundu, SVM ise genel doğruluk açısından en başarılı yöntem olarak belirlendi. Sonuçların kalitesini artırmak amacıyla, Grid Search ve 8 katlı Çapraz Doğrulama gibi yöntemlerle parametre optimizasyonu gerçekleştirildi. Ayrıca, SMOTE ve aşağı örnekleme gibi teknikler arasında yapılan karşılaştırmada, aşağı örnekleme yönteminin bu veri kümesinde daha iyi sonuçlar verdiği tespit edildi.
Geliştirilen algoritma aracılığıyla modeli rota bilgileri ve zaman aralığı ile besleyerek, belirli bir rota için kullanıcıya en uygun uçuş tarihlerini ve havayolu seçeneklerini öneren bir yapı oluşturuldu.

Abstract:
The aviation industry is a significant sector that enables millions of people to travel. Factors such as airports, air traffic control, weather, aircraft maintenance, and staffing arrangements can affect the smooth operation of flights. Flight delays and cancellations can lead to increased costs for airlines, operational challenges, and customer dissatisfaction, while passengers may encounter issues such as time loss, seeking alternative transportation, and disruption of their plans.
In this study, the aim is to predict flight irregularities using various classification techniques. Providing these predictions to passengers can assist in making informed decisions about the right flight times and airline choices. Furthermore, this study aims to facilitate the development of an application for passengers while helping airlines and airports optimize their operational processes. Additionally, by anticipating and preventing irregularities, it can contribute to cost savings for airlines by reducing compensation payments.
Flight and weather data were integrated in this study. Flight data was obtained from the publicly available Airline Ontime Performance dataset shared by the United States Department of Transportation, while weather data was sourced from the National Oceanic and Atmospheric Administration (NOAA). The John F. Kennedy International Airport (New York, JFK), one of the busiest airports in the United States, was selected, and flights departing from this airport were analyzed.
In the data preprocessing stage, the acquired data was prepared for training models by employing various preprocessing techniques such as correlation analysis, data completion, parsing, feature selection, feature merging, and sample selection. Weather data and flight data were combined to create a unified dataset. The target variable used in the models is Irregularity, which combines the features Cancelled (Flight cancellation status) and DepDel15 (Flight delayed by more than 15 minutes). This variable indicates the presence of a flight delay exceeding 15 minutes or a cancellation.
To examine the correlation between numerical features, a Pearson Correlation Matrix was plotted. Since the Irregularity feature was chosen as the predicted value, the distributions of some other features with respect to Irregularity were examined, and the data was segmented based on these distributions. Instead of a total of 98 features, 14 features were created, significantly reducing the number of features. Considering that an increase in the number of features leads to increased model complexity and decreased performance rates, this segmentation was expected to be beneficial. However, it was found that this segmentation was not beneficial and thus not used in the proposed model.
Furthermore, categorical features were converted to binary variables (1-0) for faster processing by the models. Additionally, numerical features were transformed to have values between 0 and 1 using the Min-Max Normalization method. This method prevented the diminishment of the effect of small values in models that evaluate different numerical features simultaneously.
Features that do not provide information for classification were identified and removed from the dataset, such as airline numerical code, airline name, and city name. Similarly, attributes such as Source and Report Type in the weather data were not used in the models. These decisions were made based on judgment.
Missing weather feature values were filled using an interpolation method that calculates the average of the nearest upper and lower values. The Holiday feature (1-0) was derived from the Flight Date feature using the Python Holidays library and added to the dataset.
In the weather data, in addition to hourly records, there were daily summary records. Since the hourly weather data was used, the daily summaries with the Report Type SOM or SOD were filtered out from the dataset.
Subsequently, two techniques, SMOTE and Undersampling, were used to address the imbalance in the number of delayed and non-delayed samples in the dataset. These methods respectively involve synthetic data augmentation and reducing the number of majority class samples. The dataset was then divided into validation, training, and test sets after data balancing. Parameter ranges were determined for each model, and their optimal values within these ranges were obtained using the Grid Search CV method. This process was carried out on the validation dataset. New models were then created with the optimal parameters and trained on the training set. Subsequently, measurement methods suitable for the problem and models were determined, and evaluations were conducted on the test dataset based on the measurement criteria. The most successful model was identified based on the evaluation results and designated as the proposed method.
Since the aim of the data study is to determine whether flights will experience irregularities, classification techniques were employed. Additionally, regression methods were utilized, where examples with probabilities above a certain threshold were assigned to the positive (delayed) class. The techniques used in this study include L-GBM, a decision tree-based application, Logistic Regression, a regression method, Gaussian Naive Bayes from the Bayes classifier family, K Nearest Neighbors, Multi-Layer Perceptron, and Support Vector Machines.
In this study, all models except for the Naive Bayes method are parametric models. Therefore, it is necessary to determine the most suitable parameters for the data. Various methods exist for determining the most suitable parameters. The Grid Search Cross Validation (CV) method in the Python scikit-learn library was used to determine the best parameters. With this method, a list of parameters to be optimized for a model and the possible values for these parameters are given to the Grid Search CV method, so the algorithm creates a model for each parameter value within the given range. These models are then evaluated based on measurement criteria, and the parameter set that produces the best measurement result is obtained as the output. The F1-Score was used as the measurement criterion in this study, as both balanced and imbalanced datasets were worked on.
The dataset was divided into three segments: 20% validation, 20% test, and 60% training. The validation set was used only for parameter optimization, while the test set was used solely to test the trained model, meaning the model was trained and tested with data it had not encountered before.
The values provided by the Grid Search CV method were used in the training of the models. The method was applied to the original imbalanced dataset, as well as datasets applied SMOTE and Undersampling, leading to changes in the most suitable parameter values.
The evaluation metrics focused on in the classification techniques were Accuracy, Precision, Recall, and F1-Score. For comparing the models, Receiver Operating Characteristic (ROC) curves and the Area Under the ROC Curve (ROC-AUC) criterion were used. 
It is observed that the L-GBM model gives the best AUC value among all models and sampling methods in the original imbalanced dataset. This is an expected outcome due to the fact that L-GBM is a decision tree-based algorithm, and decision tree-based algorithms are known to be insensitive to imbalanced datasets.
For SVM, polynomial SVM, and MLP models, the AUC values in the undersampled dataset are higher than those in the original dataset or the dataset with SMOTE applied. This indicates that these models are sensitive to data imbalance.
When examined in terms of F1-Score, the highest score is 0.567 for SVM, with Light-GBM following closely with a score of 0.564, a difference of 0.003.
looking at the processing times, it is observed that the fast performing model is L-GBM, with a processing time of 0.3 seconds. Although SVM and polynomial SVM give results close to or even better than L-GBM in some cases, it is understood that their processing times are much longer than L-GBM. In this study, SVM took approximately 43 minutes to yield results, while polynomial SVM took 32 minutes.
Additionally, in the comparison of techniques such as SMOTE and Undersampling, it was determined that Undersampling yielded better results in this dataset, and thus, this method was used in the proposed model. When assessed in terms of processing times, L-GBM significantly outperformed SVM. Various classification techniques were employed in this study, and their performances were evaluated. The L-GBM model was found to be the most effective in terms of computational speed, while SVM was identified as the most successful method in terms of overall accuracy. To enhance the quality of the results, parameter optimization was carried out using methods such as Grid Search and 8-fold Cross-Validation. 
Through the developed algorithm, a structure was created that feeds the model with route information and time intervals, and recommends the most suitable flight dates and airline options for a specific route.
In future work, additional features that can be obtained from airlines (such as personnel, planning, passenger and baggage counts, and ground handling times) are planned to be added to the dataset to obtain a more comprehensive dataset. This way, more effective models can be developed using a dataset that contains more information about the issue. Additionally, in this study, it was observed that the Multilayer Perceptron Neural Network Model yields close results to the L-GBM Model in some cases. Therefore, considering the use of appropriate Deep Learning methods, it is anticipated that the results can be further improved.