Semantics-aware prompting for translating NO tices To A ir M en (2025)

Abstract: A NOTAM or NOtice To AirMen is a crucial notice for different aviation stakeholders, particularly flight crews. It delivers essential notifications about abnormal conditions of Aviation System components such as changes to facilities, hazards, service, procedure that are not known far enough in advance to be publicized through other means. NOTAM messages are short, contain acronyms, and look cryptic in most of the cases. Writing and understanding these messages put heavy cognitive load on its end users. In this work, we take up the task of translating NOTAMs into English natural language using LLMs. Since NOTAMs do not adhere to English grammar rules and have their own decoding rules, large language models (LLMs) cannot translate them without effective prompting. In this paper, we develop a framework to come up with effective prompts to achieve the translations. Our approach uses context-aware semantic prompting techniques, paired with domain-specific rules, to improve the accuracy and clarity of translations. The framework is evaluated using comprehensive experiments (6 LLMs of varying sizes, and with 5 different prompting setups for each) and eight evaluation metrics measuring different aspects of the translation. The results demonstrate that our methodology can produce clear translations that accurately convey the information contained in NOTAMs.

## introduction notams are essential communications within the aviation domain. details of flight path and pilot briefings are conveyed through notams. they are constructed in specialized language that adheres to specific domain rules and templates. this unique format, characterized by a variety of abbreviations and short forms, necessitates a dedicated effort to decode and interpret notams accurately. having a translated version of the notam reduces the strain on the user to decode it from scratch and makes it easier for the subject matter experts (sme) to confirm its accuracy. as per reported statistics, around 35,000 notams are used for communication in the global air transport system every day. in this work, we propose context-aware semantic prompting approach for translating notams. we employ k-shot prompting, where the examples incorporated into the prompt are carefully selected using the proposed algorithm that account for both the syntactic and semantic characteristics of the input notam. we reached out to notam users, including pilots and subject matter experts (smes) to get the gold reference translations which we used to evaluate the quality of our translations. the impact of our proposed method is as follows: ## related works due to the safety-critical and proprietary nature of the task of translating notams combined with the limited accessibility to datasets, there has not been significant research towards automated translation of notams. in this section, we focus on existing works on notams. existing research on notam translation is limited to a few specific types, and are not exhaustive. in @xcite , the authors consider summarizing taxiway and runway closure type notams. the work commented that chatgpt is the best online tool for notam translation. @xcite focuses on information extraction from no-tams. it considers translating notams as one of the tasks. however, for translation, they convert notam to a specific format called airlang format and not to english language. they trained an encoder-decoder model using 20,000 pairs of manually annotated data, concentrating on specific entities present in taxiway closure notams. the approach requires a significant amount of training data as they train the model from scratch. on the other hand, llms can work well with zero-shot prompting, enabling us to get the translations without any specific training. this is useful as access to domain experts is difficult in this specific scenario. although not for translation, processing no-tam data for various other tasks have been considered in a few recent works. the work in @xcite , also discussed above, considered named entity recognition and criticality prediction of notams. in (dani and desarkar, 2024), the authors perform segmentation of notams to help in easier understanding of the complex and cryptic messages. @xcite propose methods for classification, filtering and sorting of notams. although there is an increasing interest in automated processing of notam data, to the best of our knowledge, there is not existing published work that focuses on translation of notams to their corresponding english natural language versions. ## motivation notams were initiated in 1947. in the early days, bandwidth for communication channels might have been a concern, due to which the number of characters in the notams were kept low purposefully. since then, notams have been an integral part of air traffic communications. using messages in such a format even in recent times has been mostly due to legacy issues among other reasons. the federal aviation administration (faa) mandates clear and concise communication to ensure safety and efficiency in the aviation sector. the use of standardized abbreviations and codes were opined to (a) reduce the risk of misinterpretation and (b) ensure that critical information is conveyed accurately. it also ensures that notams are understood universally, regardless of language barriers. this is crucial for international aviation operations where the personnel from different parts of the world speak different languages and may have varied proficiency in a single medium of communication such as english. having said that, while introduced with clarity in communication in mind, over the time notams have evolved to have a convoluted abbreviation system, lack of standardization, and ever expanding volume that pilots find difficult to keep up with @xcite . in response to these challenges, faa has initiated efforts to deliver notams in plain language. however, approximately 30 percent of notams still lack digitization and translation, particularly those pertaining to airspace, leaving many notams available only in their traditional complex format. new pilots often need to consult manuals repeatedly to interpret notams correctly, while experienced personnel must stay updated on any new rules or abbreviations introduced in the notam journal. moreover, the considerable volume of notams included in a single briefing-ranging from 80 to 120 notams-can lead to significant information overload, where critical details may be overlooked due to the abbreviated nature of the messages. a complete shift from notams to messages in natural language requires a mega policy change at the federal aviation administration level, which is quite difficult considering the scale of round-theclock global flight operations covering multiple countries, airports and impacting a huge global population. due to these factors, translation of no-tams into plain english language is an important task, which we take up in this paper. due to the ## dataset in this work, we consider nine of the most common notam categories foot_0 . these nine types of no-tams are of high significance and include taxiway (twy), runway (rwy), airspace, apron, aerodrome/airport (ad/ap), obstacle (obst), navigation (nav), and communication (com) @xcite . each of these notams has a unique composition. we approached subject matter experts (smes) of notams and obtained translations for 270 no-tams, with 30 translations from each considered category foot_1 . this is our test dataset used for inference. additionally, we use 27 notams translation pairs (3 for each category) that are used for our one-shot prompting methodologies with namely random, max length and max overlap approaches (described in detail later in section 5). there is a unique one-shot example for each category and each method, giving us the total as 9 × 3 = 27 one-shot examples. we treat each category of notam separately since every category has its own specific nature of attributes. inside each category, the notams can be of varied nature depending on the specific information it covers. this fine-grained information is captured by the clustering. we use 50 notams to implement the semantic prompting methodol-ogy. as each notam category has a different structure, the number of clusters was allowed to be different across categories. the exact number of clusters were determined using silhouette scores. the number of clusters for the different notam categories were as follows: 3 for twy, 9 for svc, 2 for obst, 4 for nav, 10 for com, 2 for apron, 8 for airspace, 9 for ad/ap, and 3 for rwy. different statistics for each notam category considered in the work are presented in ## method since notams adhere to strict rules for word usage and do not allow morphological variations, we use the word-level notam tokenizer developed by @xcite . the vocabulary consists of 29,999 tokens, which include words and measurements like nautical miles, feet, feet above ground level,coordinates, and asr numbers necessary to notify users. we also use the pretrained model based on bert-tiny m pn from @xcite . this model has ∼4m trainable parameters, 128 hidden units, 2 layers, 2 attention heads, intermediate size is 512. it was trained on 3,93,858 notams. we describe our proposed methodology below 3 . ## analysis from the experiments we find, using the longest notam or max-overlap notam takes longer time and processing power because it has to process more tokens in the case of longest prompt every time, irrespective of the length of the notam to be translated. the max-overlap prompt methodology also experiences the same issue (refer to we observed that the semantic prompting shows better accuracy compared to other methodologies evaluated (refer to fig. 2 ). the results of llama3 with 8b parameters using the semantic prompting methodology are comparable to gpt-4o using zeroshot prompting. precision, recall, similarity, me-teor, and ter scores are better in llama3 with semantic prompting compared to gpt-4o with zeroshot prompting. additionally, the metric scores for gpt-4o improved with semantic prompting. we also compared the performance of the semantic prompting methodology across different models. our findings indicate that llama3 with semantic prompting produces results comparable to the largest gpt-4o model, with accuracy at an acceptable level (refer to fig. 3 ). ## next steps this work can be integrated with the electronics flight bag (efb) where most of the briefing today occurs. the next part of the project will look closely at the integration of the proposed system with devices. it will also look into the aspect of translation and summarization of multiple no-tams coming together as a single input. notams could be premise or cloud-based services. however, raw !sva 12/007 sva ad ap all sfc wip sn removal 2112090022-2112100022 human translation savoonga airport (sva) with notam 12/007 aerodrome all surfaces work in progress (wip) for snow removal from december 09, 2021, 12.22 am to december 10, 2021, 12.22 am. zero shot work on snow removal at savoonga airport is underway from december 9th, 2021, 12:22 am to december 10th, 2021, 12:22 am. missing: all surface work one-randomshot snow removal work is in progress at savoonga airport (sva) from december 9, 2021, 00:22 utc to december 10, 2021, 00:22 utc. missing: all surface work one-maxoverlapshot snow removal work is in progress at savoonga airport (sva) from december 9, 2021, 00:22 utc to december 10, 2021, 00:22 utc. missing: all surface work one-maxlengthshot at savoonga airport, all surface work including snow removal is complete as of december 9, 2021, at 12:22 am. one-semanticshot the work to remove snow on all surfaces at savoonga airport (sva) has started and is making progress as of december 9, 2021, at 12:22 am, with the expected completion by december 10, 2021, at 12:22 am. ## conclusion in this work, we focus on different prompting techniques for notam translation. the semantic prompting methodology leverages the context of the query notam to select appropriate prompts for translation, making it suitable for resourceconstrained environments like edge devices and cockpits. we evaluate the performance of different prompting techniques, ranging from small-sized models to openai's largest model. we demonstrate that the semantic prompting methodology is more efficient and suitable for resource-constrained environments.