Combining Data Science and Machine Learning with the Aviation Industry: A Personal Journey through a Capstone Project (Part II)
By: Christopher Kuzemka
Tags: Data, Machine Learning, Aviation, Engineering, Python

As a result of these initial scores, Figure 7 and Figure 8 above have been created to analyze the residuals within our model for training and testing. Each figure tells us that our model was appropriately chosen as the residuals do not show any clear patterns and also decently reveal an even spread of points above and below the horizontal red line. The horizontal red line represents the idealized regression of our model while the scattered points represent the difference in value of each consecutive prediction and label. Discovering a pattern in the residuals with our own eyes would indicate that we can use a better model to describe the system — logically, if you recognize a pattern in your errors, then that means you should recognize that there is a way to reduce that error by incorporating such pattern into your model. If all the residuals in both graphs showed uneven spread — where a significant amount of points exist in one threshold separated by the horizontal line over the other, or where residual show a pattern of slowly spreading out further and further from the horizontal line — it means your model could still be tweaked to account for evening out the residuals.

Conclusions, Study Summary, and Thanks…

It’s safe to start with stating that the study showed signs of a moderate success. From a variety of constraints — such as limited air travel in the new regime change, problems integrating API data together, unbalanced data, time constraints, data access restraints, and more — we were able to come out with a “proof-of-concept” study to show how a machine learning model can be used to predict the minimum cost threshold of airline tickets in this new regime. On the contrary, it is still appropriate to mention that the limitations of our model can make this attempt pointless in a business perspective due to how niche the study had become, where some corners had to be cut, and how practical/reliable the models will be. However, almost all models will face general limitations due to niche barriers, which leaves us hopeful to say that our model is just like any other models used to predict the same label (it can only get better).

Through this study, a lot was covered, and it may be difficult now to fully gather together what was taken from both of the articles written here. To summarize, we as data scientists were approached by a business to answer the question, “What is the minimum cost threshold an airliner can charge their passengers per flight and how can we make a model that discovers this?” We were given a month time to have a working proof of concept. We achieved these goals, to some degree. Through the process, we learned about regime change and how the Covid-19 pandemic has played an instrumental role in our data collection phase. We learned about the limitations of using different APIs and what sort of cleaning must be done for data. We discussed how we can overcome data domain issues and create data out of our collection methods by using unique ways to increase data and add diversity to it. We learned about bootstrapping to expand our data and machine learning methods such as Linear regression, cross validation, KNN regression, and Decision Tree regression. We learned about ensemble machine learning such as Bagging regression and Random Forest and simultaneously discussed the data imbalance occurring from the trends we uncovered. We ultimately created a model with cross validated mean absolute error of $173 — all done without heavy hyperparameter tuning.

To me, this capstone project served as a very personal journey to prove myself in the world of data. I wanted to use what I knew about the aerospace industry and combine it with data analytics to ultimately showcase a project worth investing more time and resources into. With a more stable regime, more access to data, and more time, this project has the capability to scale to help a real-world business. Writing this series has been a joy as I wanted to really express what I know to the world and also help someone learn from the study. Machine learning is a popular field to talk about, but it is often hard to find real-world practical implementations models. Many times, machine learning is left back with research as its uses for studies become too personalized around a specific problem. Ultimately I want the people, who took their precious time from their day to read this article, to come out of this series learning something new and thinking on ways to better analyze data and to apply machine learning to societal problems. I greatly appreciate your time and give my sincerest thanks. Do not stop teaching yourself and do push on to bigger and better things while you still can in this world; cherish what you learn as you go on. Cheers!

BONUS: Learning How Thermodynamics can be Used to Predict Pricing

My background is in engineering and I did as best as I could to reflect that here, but I know there is more which could have been done to better integrate engineering concepts into the study. In truth, this capstone project was initially inspired by a project I had done in my undergraduate studies with Hofstra University. One of my final design projects was meant for my Thermal Engineering course, which is the field that studies thermodynamic principles on machines. In this senior design project, I analyzed a trans-Atlantic flight I flew on earlier that year and collected data regarding that flight’s speed and altitude from FlightAware.com. At the time, it was much easier to get this data ported into an Excel spreadsheet, without the need for scraping — and this was before I knew how to use Python, how to web scrape, or how to play with APIs. From this data alone, along with some idealized assumptions, I was able to successfully perform a comprehensive idealized jet propulsion engine analysis on the plane’s engines and determine many important properties of the air traveling through the turbofans.

I wanted to include this section as a bonus for one to ponder on when thinking about this study, as it is the “lost link” still wished to be integrated into the project, but could not have been implemented due to the FlightXML API constraints in the study and the lack of flight data due to Covid-19. Instead, I will give a walkthrough description of the project performed back in my undergraduate course in 2018, as an example to show what extra data could have been generated to potentially help better our model.

On October 17th, 2018 I flew on flight BA115 (British Airways flight 115) to travel from London Heathrow Airport to JFK International Airport. The goal of such previous project was to perform and analysis on the thermodynamic cycle states of the engines. FlightAware.com was used primarily to access accurate speed and altitude measurements of the individual flight, as the plane logged such information over time to nearby radio checkpoints along the route. From FlightAware.com, it was also discovered that the plane flown was a B747–400, which is a quad-jet aircraft and is a variant of the B747 series — notoriously considered the most notable aircraft design in all of history of human flight.

Above are two images of the October 17th flight flown in 2018. Unfortunately, it is not possible to access the graphic above from FlightAware.com anymore as basic access users only have access to 3 months of data history. As a result, take the referenced source for the above graphics with appropriate consideration that it may not be reflecting the same exact path described in the study, but is similar enough for our intended purposes.

From FlightAware.com, I was able to retrieve 660 data points of speed, altitude, direction, time, and more of flight BA115. From this data, it was determined that the flight time lasted eight hours and fifteen minutes (29,700 seconds equivalent) and the total distance flown was 5,922.386 kilometers. The weather of that particular day showed clear skies throughout the entirety of the flight, eliminating the need for massive environmental constraints.

Ideal Jet Propulsion Cycle referenced from (Cengal Y.A. & Boles M.A., 2002. Thermodynamics: An Engineering Approach. McGraw-Hill Companies Inc. New York, NY. 483–487)

In analyzing thermodynamic machines, graphics such as the one above are used to understand the different property states occurring through the machine’s life cycle. The one above showcases the relationship between entropy and temperature within the jet engine’s idealized lifecycle. Entropy (commonly defined as the variable “S” in theoretical thermodynamics…unlike our graphic denoting a lowercase “s” for “specific entropy” on the horizontal axis) is a measure of “how much energy is not available to do work in a system”. It is often closely associated with chaos or a measure of how a system becomes more disordered over time. The units of entropy are measured in Joules/Kelvin. Engineers, must understand what entropy is, but will more often use specific entropy in their analysis where its units are measured as kiloJoules/(kilograms*Kelvin) or more easily defined as entropy per mass. Specific entropy is often used to analyze a certain form of mass existing within a system, hence the inclusion kilogram units in the denominator. Specific entropy can change dependent on different states within a system — often for water, empirical steam tables are referenced to analyze the specific entropy of water to provide a better understanding of a system. For our case, we will be analyzing air, where such tables are not readily needed.

Reverting back to our diagram above, the T-s diagram (Temperature vs entropy diagram) indicates how energy of air within the plane’s engine is analyzed during its lifecycle with pressure held constant across some processes. In entropy states one through three, pressure is increased through isentropic compression. Here, air travels first from the engine’s inlet, through the diffuser and then through the compressor. In process three to four, the air then travels through a burner/combustion chamber from the compressor. The air itself is heated up at constant pressure, resulting in higher entropic value, which simultaneously increases the heat transfer per unit mass (denoted as the variable “q”). The air currently is prone to a higher energy release. In process four through six, the air undergoes isentropic expansion where pressure is released in the journey from the combustion chamber, through the turbine, and through the nozzle exit. Such pressure release ultimately thrusts the aircraft forward with immense force, constantly.

Ideal Jet Propulsion Cycle referenced from (Cengal Y.A. & Boles M.A., 2002. Thermodynamics: An Engineering Approach. McGraw-Hill Companies Inc. New York, NY. 483–487)

Throughout the study, several different assumptions and underlying initial conditions need to be addressed for its analysis. First, it was important to treat air as an ideal gas, meaning that air properties will behave normal to standard temperatures and pressures. From Appendix A.1 in the fourth edition of Engineering Thermodynamics by M. David Burghardt and by James A. Harbach, the specific heat of air constant pressure used in the study was 1.005 kiloJoules/(kilograms*Kelvin) and the Boltzmann constant used was 1.4. Both of such numbers will be important for our mathematical formulas used in learning about engine properties. Specific heat is a measure of how much energy must be added to a property to raise its temperature. Specific heat can change depending on surrounding factors, which is why it is important to denote what is held constant in such factors when relying on the ideal gas law (which is why we assumed air to be idealistic in our analysis). Specific heat can be affected by changing volumetric and pressure values which is why we only analyze such measures for ideal purposes by holding one of the independent variables constant. In our case, our engine will undergo a constant pressure process, which is why we will use the specific heat number for constant pressure for air as is the deterministic nature of the study. The Boltzmann constant is a number measuring the proportion of a property’s energy to thermodynamic temperature. The operating conditions of the engine were assumed to be steady state, which means that system state variables were to remain constant.

Self-made whiteboard image.

The plane was theoretically assumed to be a stationary object where recorded speed of the aircraft was factored as the air free-stream velocity (denoted as “v­­” subscript “∞”) to equal/represent the velocity of the aircraft (denoted as “v” subscript “aircraft”). Such assumptions allows us to state that the free-stream velocity can act as each jet engine intake. Furthermore, the kinetic and potential energy in the system were negligible except at the inlet and exit conditions, plus the atmospheric temperature, pressure, and air density were to be averaged values between zero and 15,000 meters altitude. From the UK Civil Aviation Authority Engine Type Certification Data Sheet №1048, it was uncovered that four Rolls Royce RB211–524H jet engines are typically used on a B747–400. For this study, only one engine will be analyzed and such findings from the individual engine analysis will be assumed across all four jets mounted on the airplane. A uniform diameter is assumed for the jet shaft with a length of 2.19 meters. The turbine work was assumed to be equal to the compressor work. The velocity of the air leaving the diffuser was assumed to be equal to zero meters per second. The thrust generated from bypassing air was to be neglected. The combustion chamber temperature was assumed to be 2,273 degrees Kelvin. The compression ratio, the ratio of total volume to clearance volume in a piston, was found to be 32.9:1.

Generalized Analysis: Where Thermodynamics could have Merged with Machine Learning

With all such assumptions and initial conditions factored in, the below follows the mathematical analysis performed to analyze the ideal jet propulsion cycle. All of such analysis is stemming off the concepts learned from my undergraduate studies and following the teachings of Dr. Burghardt himself, who wrote the heavily used textbook mentioned earlier as Engineering Thermodynamics. Through such analysis, we will primarily observe engine states by remarking a measure known as enthalpy, or enthalpy per unit mass for our system. Enthalpy is a measure of energy transfer between a system and its surroundings. Throughout our analysis, understanding this enthalpy will help us uncover all of the property states needed to tell us more about what is going on with our engine.

This is where the magic occurs, in my opinion. In a rather elegant way, we scientists have the ability to know what is going on at every stage of our engine by simply measuring how much energy is moving within the engine — and we don’t even have to physically be on the airplane to do this. It was my intention to use such analysis to perform very comprehensive EDA for our initial machine learning study and include an analysis for the different engines present for different planes found out in our study — ultimately building a highly accurate tool able to give a better cost estimate of the journey analyzed. From such known data, we could find a way to factor important features such as measured average air temperature/pressure states, measured engine efficiency, measured thrust output, and measured propulsive thrust power. Expanding further on this could allow us to analyze air-fuel ratios, changing fuel mass over time on a flight, and maybe help us know how passenger payload can play a role in savings.

To start before observing some MATLAB generated plots on these dependent variables, we will first go over the generalized analysis taken.

Self-created LaTeX.