An NLP -Focused Pilot Training Agent for Safe and Efficient Aviation Communication (2024)

Abstract: Aviation communication significantly influences the success of flight operations, ensuring safety of lives and efficient air transportation. In day-to-day flight operations, air traffic controllers (ATCos) would timely communicate instructions to pilots using specific phraseology for aircraft manipulation . However, pilots, originating from diverse backgrounds and understanding of English language, have struggled with conforming to strict phraseology for readback and communication in the live operation, this problem had not been effectively addressed over the past decades. Traditionally, aviation communication training involved expensive setups and resources, often relying on human-in-the-loop (HIL) air traffic simulations that demand allocating a specific environment, domain experts for participation, and substantial amount of annotated data for simulation. Therefore, we would like to propose an NLP-oriented training agent and address these challenges. Our approach involves leveraging only natural language capabilities and fine-tuning on communication data to generate instructions based on input scenarios (keywords). Given the absence of prior references for this business problem, we investigated the feasibility of our proposed solution by 1) generating all instructions at once and 2) generating one instruction while incorporating conversational history in each input. Our findings affirm the feasibility of this approach, highlighting the effectiveness of fine-tuning pre-trained models and large language models in advancing aviation communication training.

## introduction efficient and accurate communication is crucial in air traffic management @xcite . during real flying scenarios, air traffic controllers (atcos) engage in timely communication with numerous aircraft in designated airspace, including aviation instructions, aeronautical announcements, traffic advisories, aerodrome announcements, and weather updates. unfortunately, miscommunications frequently occur, attributed to pilot readback errors @xcite and phraseology issues @xcite . this long-standing problem, highlighted in an analysis of nextgen 2013 found pilot mishearing (28%) and no pilot readback (20%) as predominant factors, constituting 74% of human errors based on 382 miscommunication messages @xcite . a 2023 study showed that 92% of aviation respondents believed language training was essential @xcite . therefore, providing training in pilots' phraseology and domain-specific language is crucial to addressing communication challenges for safety and operational efficiency. on the other hand, the aviation industry has been actively seeking solutions amid the rapid advancement and widespread adoption of artificial intelligence technologies @xcite . ai-driven tools, such as natural language processing (nlp), are seamlessly integrated into every facet of modern aviation @xcite . noteworthy implementations include human-in-the-loop training involving flight simulations and communications @xcite , aviation ontology construction @xcite , readback error detection @xcite , and phraseology training @xcite . despite these efforts, none have addressed the requirements for enhancing pilots' communication. training that incorporated human actors often incurred substantial costs for setup @xcite , significant resources @xcite , and necessitated labor-intensive annotated data for simulation @xcite . for instance, an mitre report @xcite revealed that simulating an air traffic scenario for communication training in a small class required the involvement of four domain experts, four ai assistants, and one traffic simulation system, all within a meticulously configured environment integrating ai, network, and interfaces. to address the above challenges, we propose an nlp-focused training agent that exclusively utilizes aviation communication data. this agent offers language training to pilots, aiming to enhance their proficiency in domain-specific phraseology while minimizing resource utilization and costs. essentially, pilots can effortlessly create a tailored aviation communication environment, generating a list of aviation instructions with just a few clicks on a single machine. subsequently, they can systematically perform readbacks of each instruction through an application interface, transmitting natural language data to the backend for processing. throughout the training loop, pilots receive prompt feedback on their readbacks and are prompted for repetition in the event of a readback error. upon successfully executing readbacks for all instructions in a training session, pilots have the flexibility to opt for another training session with a different set of instructions using the same setup. the core of this method revolves around aviation instruction generation, for which we leveraged transformerbased pre-trained language models, including gpt-2 @xcite , bart(seq2seq model) @xcite , and llama2 @xcite . our contributions include: the proposed method is dedicated to generating contextually-aware aviation instructions using only communication data. unlike traditional approaches, it eliminates the need for a massive amount of annotated input and expensive flight simulation setups, providing promising task outcomes at a fraction of the cost. ## training agent workflow according to the international civil aviation communications organization (icao), aviation communication phraseology is precisely defined with specific vocabulary and content (icao, 2020). while aviation instructions draw from a relatively limited set of words, their meanings within this context can significantly diverge from general language, especially concerning terminology (icao, 2020). for example, consider a message, fakeair one two niner direct to november echo papa whiskey whiskey, which signifies that it is an air traffic control (atc) communication directed at a pilot flying the aircraft. it consists of domain-specific language for alphabets, numbers, the placement of call-sign, and aviation instruction for aircraft manipulation and readback. in a real-world scenario involving communication between a controller and an aircraft, a set of aviation instructions would be issued sequentially. if pilots provided correct readbacks, or if atcos rectified readback errors through clarification, communication proceeded smoothly. however, instances of miscommunication could occur during live flying operations. this becomes a focal point for us, aiming to minimize such occurrences through cost-effective simulations of these interactions with pilots. these interactions include user input processing, context-aware instruction generation, pilot response/readback detection, and a reproducible workflow for generating content with various instruction scenarios. consequently, we come up with a training agent with an integrated solution leveraging existing nlp techniques, as illustrated in figure 1 . at the application level, pilots initiate a request for a flight communication scenario, specifying the communication channel and desired instruction length for practice. the request is transmitted to the instruction graph module, which generates a sequence of instruction keywords based on transitional probabilities specific to the chosen communication channel. subsequently, the instruction scenario is forwarded to the instruction generation module to produce instruction content. the module then returns either a list of instructions, offloading the instruction issuance to the application layer, or transmits one instruction at a time along with conversational history. pilots provide readbacks of individual instructions at the application layer and transcribe into natural language data. each readback is processed by the readback detection module to raise a signal if a readback error is found. in case errors are detected, pilots are prompted for a new readback; otherwise, the issuance of instructions continues. once all instructions are successfully read back and verified, pilots can initiate the process for further practice. ## methodology the proposed training agent aims to generate aviation instructions that are both meaningful and contextually aware. to achieve this goal, we devise a specific input format termed instruction scenario, which comprises a collection of distinct keywords representing various types of instructions and communication channels. these instruction scenarios are retrieved and stored in instruction graphs, serving as inputs for generation at the application layer. subsequently, the instruction generation process utilizes the instruction scenario as input and generates words auto-regressively, where each token is predicted based on the previously generated tokens. for this task, we employ transformer-based language models and fine-tuned them with domainspecific data. instruction scenario and graph. we establish two state spaces: one for instruction keywords, denoted as @xmath0, kw 2 , ..., kw n }, and another for communication channels, represented by @xmath1, c 2 , ..., c m }. to calculate the transitional probabilities among different types of instructions, the transitional probability p i,j from one instruction kw i to another instruction kw j is calculated by dividing the count of transitions from kw i to kw j by the total count of transitions from kw i to all possible instructions on a communication channel c. thus, each instruction scenario is defined as a combination of a communication channel keyword and a sequence of instruction keywords (kw ) that have occurred on that channel (c). subsequently, we organize the keywords and their corresponding transitional probabilities into matrices @xmath2, p 2 , ...p m }, which are of size (n + 2, m) to accommodate an initial state and an end state to kw . equation 1 elucidates that the summation of each row's transitional probabilities for kw always equals 1, where i and j signifies rows and columns respectively, and p i,j denotes the transitional probability from kw i to kw j . consequently, we derive directed graphs for kw , de-noted as -→ @xmath3, g 2 , ...g m }, incorporating the transitional matrices p . the graph can be symbolized as -→ @xmath4, e, w ), where @xmath5, kw @xmath6,j |p i,j , w i,@xmath7. instruction generation. we employ the transformer neural network architecture @xcite for the generation, including both decoderonly and seq2seq architectures to explore and compare methods. our goal is to generate instructions for a given instruction scenario by predicting the next token in the sequence based on the context of the preceding tokens. consequently, each training instance is designed with a set of domain-specific vocabulary v , an instruction scenario @xmath8, x 2 , ..., x m } where x @xmath9, and a sequence of instructions @xmath10, y 2 , ..., y n } where y i ⊆ v . in summary, equation 2 elucidates how the model factorizes the probability of vocabulary across instructions via the chain rule @xcite and subsequently generates the current instruction y i given q and all previous instructions y 1:i-1 . to systematically generate consecutive and correlated instructions given an instruction scenario, we design two data templates, as illustrated in figure 2. these templates compose the input context differently, aiding in the coherent generation of instructional content. to generate instructions one at a time, we append previously issued instructions as dynamic context to the instruction scenario. it involves feeding the model the instructions already generated, allowing it to build upon its understanding of the scenario as it progresses. as demonstrated in the second example of figure 2 , this approach facilitates the sequential generation of each instruction with additional context. finetuning. similar to predicting the next sequence of tokens based on previous tokens as input @xcite , our objective is to minimize the language modeling loss through fine-tuning the domain-specific dataset e. this fine-tuning process is devised to minimize the negative loglikelihood l with parameters θ, as illustrated in equation 3 . we adopt lora @xcite for parameter fine-tuning within finite gpu resources. lora preserves the llm parameters while introducing trainable rank-decomposition matrices into each layer. in equation 4, a and b represent the decomposed matrices, where b has dimensions b ⊆ r d×r and a has dimensions a ⊆ r r×k , with the number of rank denoted by r (where 1 ≤ @xmath11, and n refers to the model's dimension of its dense layer. the objective is to customize an llm for a specific task by updating its parameters using trainable matrices with dimensions n × r + r × n, without altering the original llm parameters, which are of size n × n. readback error detection. this component takes in a pair of instruction texts from different speakers to indicate whether an error is detected in the pilot's readback when compared against the air traffic controller's (atco) instruction. the definition of an error may vary depending on the implementation, such as the outcome of basic string matching, the semantic distance between two strings, or the matching of words only within the essential semantic attributes of a specific type of instruction, akin to the approach by @xcite . for the sake of simplicity in this experiment, we adopt the string-matching approach. ## experiments the objective of this experiment is to objective the potential of generating correlated aviation instructions exclusively with the proposed instruction scenario and natural language input. for instruction generation, we finetune pre-trained language models using domain-specific communication data, including gpt-2 @xcite , bart-base @xcite , and llama2-7b @xcite . due to lack of aviation metadata, we assume ## conclusion accurate and clear communication plays a pivotal role in ensuring aviation safety and operational efficiency. nevertheless, pilots with diverse backgrounds frequently encounter difficulties in adhering to strict phraseology requirements, potentially hindering operational effectiveness. to tackle this challenge, we present an nlp-focused training agent that leverages natural language features and existing communication data to generate personalized instructions tailored to specific input scenarios. our experimental results demonstrate the significant efficacy of the proposed method. consequently, this approach eliminates the need for costly human-in-the-loop simulations and extensive annotated data entries, paving the way for a cost-effective and accessible future in aviation training.