Patent Description:
Published aeronautical charts, such as, for example, Instrument Approach Procedure (IAP) charts, Standard Terminal Arrival (STAR) charts or Terminal Arrival Area (TAA) charts, Standard Instrument Departure (SID) routes, Departure Procedures (DP), terminal procedures, approach plates, and the like, depict and describe the procedures for operating aircraft at or in the vicinity of various airports, runways, or other landing and/or departure locations. These charts graphically illustrate and describe the specific procedure information and instructions (e.g., minimum descent altitudes, minimum runway visual range, final course or heading, relevant radio frequencies, missed approach procedures) to be followed or otherwise utilized by a pilot for executing a particular aircraft procedure. These charts are typically provided by a governmental or regulatory organization, such as, for example, the Federal Aviation Administration in the United States.

Many airports have numerous different published procedures associated with a particular runway, and the particular procedures flown during flight are often assigned by air traffic control (ATC) during flight. Thus, aircraft operators lack any knowledge or insight regarding what procedures will be flown prior to flight. Additionally, in practice, the procedure "as flown" by the aircraft will likely deviate from the exact specifications provided in the published procedure, thereby complicating retrospective analysis. The lack of knowledge regarding what procedures were flown limits the aircraft operator's availability to analyze the costs or other potential impacts attributable to different procedures. Accordingly, it is desirable to facilitate retrospective analysis of flown procedures to facilitate more efficient or otherwise improved operations. Furthermore, other desirable features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and this background. Prior art cited during prosecution includes <CIT>, which relates to an aircraft data analysis program and method that allows users to perform custom measurements on flight data collected from various sources, enabling post-flight analysis where users can define their own measurement parameters for analyzing flight data.

Aspects and preferred embodiments of the invention are defined in the appended claims. Disclosed herein are methods and systems for analyzing flight records to assign procedures to the flight records.

A first aspect of the invention relates to a method for analyzing a flight record and determining performance metrics associated with an aircraft procedure according to appended Claim <NUM>. Preferred embodiments are defined in the appended dependent claims.

Another aspect of the invention relates to a system according to appended Claim <NUM>.

Embodiments of the subject matter will hereinafter be described in conjunction with the following drawing figures, wherein like numerals denote like elements, and:.

Embodiments of the subject matter described herein generally relate to systems and methods for retrospectively analyzing travel data and mapping the travel data to a predefined procedure. In this regard, although the subject matter is described herein primarily in an aviation context involving flight records where flight data representing an aircraft procedure as it was flown is utilized to identify that procedure by mapping the flight data back to the constituent legs or segments of that procedure, it should be understood that the subject matter may be similarly utilized in other applications, not encompassed by the claims invention, involving execution of a predefined travel procedure in the context of travel records or vehicle management systems associated with another type of vehicle (e.g., automobiles, marine vessels, trains).

As described in greater detail below, exemplary embodiments described herein map the trajectory defined by data captured during flight to a particular procedure and thereby identify that procedure as having been flown by the aircraft. Based on the particular airport, runway, and potentially other constraints, a set of potential aircraft procedures may be initially identified. Thereafter, the flown trajectory is compared to the constituent legs or segments of each procedure of the set of potential procedures to determine the degree to which the flown trajectory adheres to the defined trajectory associated with that respective procedure. In this regard, for each procedure being analyzed, one or more adherence metrics may be calculated or otherwise determined to quantify the relative degree to which the flown trajectory aligns with the defined trajectory for that respective procedure. After analyzing all of the potential procedures, the potential procedures may be sorted, ranked, or otherwise ordered according to their respective adherence metric(s). In one or more exemplary embodiments, the potential procedure having the highest rank or priority is selected or otherwise identified as the procedure that the flown trajectory corresponds to, and then assigns or otherwise associates the flight record or data set for that flight with the identified procedure.

After identifying the procedure that was flown, the flight data representing that identified procedure as it was flown by the aircraft is analyzed to determine one or more performance metrics associated with the identified procedure (e.g., an estimated fuel required for executing the segment, an estimated time required for executing the segment, an estimated cost index value for executing the segment, and/or the like). Additionally, the flight data may be utilized to quantify or qualify operational norms for executing the procedure, identify anomalous deviations from the procedure, identify potential risks associated with executing the procedure, or otherwise assess the procedure for purposes of safety, training, efficiency, and/or the like based on the as-flown trajectory. In this regard, aggregated flight data from different flight records that individually represent different trajectories mapped to the same procedure may be analyzed or combined to determine a nominal trajectory or probabilistic trajectory for the procedure, and based thereon, calculate or otherwise nominal or probabilistic estimates of the costs or performance associated with that procedure.

<FIG> depicts an exemplary embodiment of a flight analysis system <NUM> for an aircraft <NUM>. The illustrated system <NUM> includes computing system <NUM> that receives flight data from one or more aircraft <NUM> and creates or otherwise maintains, in a data storage element <NUM>, flight records <NUM> that store the flight data corresponding to individual flights by an aircraft <NUM>. The computing system <NUM> also obtains procedure information from a procedure database <NUM> or similar data storage, and for each flight record <NUM>, the computing system <NUM> maps the trajectory represented by a subset of the flight data to an aircraft procedure that is then assigned or otherwise associated with that flight record <NUM> for further analysis.

In various embodiments, the computing system <NUM> may be located at a ground operations center or other facility located on the ground that is equipped to track, analyze, and otherwise monitor operations of one or more aircraft <NUM>. In an exemplary embodiment, the computing system <NUM> includes a processing system <NUM> and a data storage element <NUM>. The processing system <NUM> generally represents the hardware, circuitry, processing logic, and/or other components configured to map flight data to individual procedures and perform additional processes, tasks and/or functions to support operation of the flight analysis system <NUM>, as described in greater detail below. Depending on the embodiment, the processing system <NUM> may be implemented or realized with a general purpose processor, a controller, a microprocessor, a microcontroller, a content addressable memory, a digital signal processor, an application specific integrated circuit, a field programmable gate array, any suitable programmable logic device, discrete gate or transistor logic, processing core, discrete hardware components, or any combination thereof, designed to perform the functions described herein. The data storage element <NUM> generally represents any sort of memory or other computer-readable medium (e.g., RAM memory, ROM memory, flash memory, registers, a hard disk, or another suitable non-transitory short- or long-term storage media), which is capable of storing computer-executable programming instructions or other data for execution that, when read and executed by the processing system <NUM>, cause the processing system <NUM> to execute and perform one or more of the processes tasks, operations, and/or functions described herein.

In one or more embodiments, the computing system <NUM> includes one or more physical interfaces, such as ports, card readers, disk readers, and/or the like that are communicatively coupled to the processing system <NUM> that allow the processing system <NUM> to access, retrieve, or otherwise obtain the flight data captured by an aircraft <NUM> via a corresponding physical medium (e.g., a memory card, a flash drive, a Universal Serial Bus (USB) cable, or the like). That said, in other embodiments, the computing system <NUM> may include a network interface or other communications interface or system communicatively coupled to the processing system <NUM> that allows the processing system <NUM> to access, retrieve, or otherwise obtain the flight data captured by an aircraft <NUM> via a communications network, such as the Internet, a satellite network, a cellular network, or the like, that supports unicast or point-to-point communications to/from the aircraft <NUM>. In some embodiments, the computing system <NUM> may include hardware and/or other components configured to support data link communications to/from the aircraft <NUM> using a data link infrastructure and/or a data link service provider.

The data storage element <NUM>, alternatively referred to herein as the flight record database, generally represents any sort of memory or other computer-readable medium capable of storing or maintaining flight data in different flight record entries <NUM>. In exemplary embodiments, each flight record entry <NUM> corresponds to an individual flight flown by an aircraft <NUM> from a departure airport to a destination airport. The flight record <NUM> may maintain an association between the departure airport, the departure runway utilized at the departure airport, the destination airport, the destination runway (or arrival runway) utilized at the destination airport, and aircraft position data samples (alternatively referred to herein as aircraft position reports) that were captured or otherwise obtained during flight while en route between the departure and destination airports. In this regard, aircraft position reports may be created by a system onboard the aircraft <NUM> periodically recording or capturing the current latitude, current longitude, current altitude, the current heading, the current speed, and other flight data characterizing the status of the aircraft <NUM>. In exemplary embodiments, the flight record entry <NUM> is also capable of maintaining an association with particular aircraft procedures flown by the aircraft <NUM> while en route between the departure and destination airports, as described in greater detail below.

As described in greater detail below, in an exemplary embodiment, the computing system <NUM> includes or otherwise accesses data storage element <NUM>, alternatively referred to herein as the procedure database, which contains aircraft procedure information for a plurality of airports and maintains the association between the aircraft procedure information for different aircraft procedures and their corresponding airports and/or runways. As used herein, aircraft procedure information should be understood as a set of operating parameters or instructions associated with a particular aircraft action (e.g., approach, departure, arrival, climbing, and the like) that may be undertaken by the aircraft <NUM> at or in the vicinity of a particular airport. In an exemplary embodiment, the procedure information for a particular aircraft action (or aircraft procedure) includes various possible criteria for various categories or types of procedure information (e.g., the name or identification of the procedure, possible radio frequencies for the procedure, possible minima for the procedure, auxiliary instructions or notes on the procedure, and the like) along with information that defines the trajectory associated with a respective procedure (e.g., navigational reference points, navigational segments, procedure turns, and the like) or otherwise provides the operating parameters or instructions for executing the aircraft action associated with the respective procedure. For example, the procedure information for an approach procedure for an airport may include categories for the name or identification of the approach, the possible radio frequencies for the approach, the possible minima for the approach, instructions or notes on the missed approach procedure, and the like, along with information characterizing the navigational segments (or legs) that make up the approach course or otherwise defining the operating parameters or instructions for operating the aircraft in accordance with the respective procedure.

As used herein, an airport should be understood as referring to a location suitable for landing (or arrival) and/or takeoff (or departure) of an aircraft, such as, for example, airports, runways, landing strips, and other suitable landing and/or departure locations, and an aircraft action should be understood as referring to an approach (or landing), an arrival (or descent), a departure (or takeoff), an ascent (or climb), taxiing, or another aircraft action associated with an airport and having associated aircraft procedure information. For example, each airport may have one or more predefined aircraft procedures associated therewith (e.g., approach procedures, departure procedures, arrival routes, departure routes, and the like), wherein the aircraft procedure information for each aircraft procedure at each respective airport may be maintained by the procedure database <NUM>. The aircraft procedure information may be provided by or otherwise obtained from a governmental or regulatory organization, such as, for example, the Federal Aviation Administration in the United States. The aircraft procedure information may include instrument approach procedures, standard terminal arrival routes, instrument departure procedures, standard instrument departure routes, obstacle departure procedures, or the like, traditionally displayed on a published charts, such as Instrument Approach Procedure (IAP) charts, Standard Terminal Arrival (STAR) charts or Terminal Arrival Area (TAA) charts, Standard Instrument Departure (SID) routes, Departure Procedures (DP), terminal procedures, approach plates, and the like.

As described in greater detail below, the aircraft procedure information for a given procedure defines a number of procedural legs that constitute the trajectory for the procedure. In exemplary embodiments, each procedural leg is defined by a leg path and a leg termination point (or endpoint). In this regard, the leg path may be defined by a heading, course, arc, or other directional qualifier, while the leg termination point may be defined by a specific navigational reference point (or waypoint) or the aircraft satisfying a particular condition (e.g., a certain altitude). The combined sequence of procedural legs defines the trajectory associated with the procedure.

Still referring to <FIG>, in the illustrated embodiment, the aircraft <NUM> includes, without limitation, a display device <NUM>, a user input device <NUM>, a processing system <NUM>, a display system <NUM>, a communications system <NUM>, a navigation system <NUM>, a flight management system (FMS) <NUM>, and one or more avionics systems <NUM>. The display device <NUM> is an electronic display capable of graphically displaying flight information or other data associated with operation of the aircraft <NUM> under control of the display system <NUM> and/or processing system <NUM>. The user input device <NUM> is coupled to the processing system <NUM>, and the user input device <NUM> and the processing system <NUM> are cooperatively configured to allow a user (e.g., a pilot, co-pilot, or crew member) to interact with the display device <NUM> and/or other elements onboard the aircraft <NUM>.

The display system <NUM> generally represents the hardware, firmware, and/or other components configured to control the display and/or rendering of one or more displays pertaining to operation of the aircraft <NUM> and/or systems <NUM>, <NUM>, <NUM>, <NUM> on the display device <NUM> (e.g., synthetic vision displays, navigational maps, and the like). In this regard, the display system <NUM> may access or include one or more databases suitably configured to support operations of the display system <NUM>, such as, for example, a terrain database, an obstacle database, a navigational database, a geopolitical database, a terminal airspace database, a special use airspace database, or other information for rendering and/or displaying content on the display device <NUM>.

The navigation system <NUM> generally represents the onboard components configured to obtain or provide real-time navigational data and/or information regarding operation of the aircraft <NUM>. The navigation system <NUM> may be realized as a global positioning system (GPS), inertial reference system (IRS), or a radio-based navigation system (e.g., VHF omnidirectional radio range (VOR) or long-range aid to navigation (LORAN)), and may include one or more navigational radios or other sensors suitably configured to support operation of the navigation system <NUM>, as will be appreciated in the art. The navigation system <NUM> is capable of obtaining and/or determining the instantaneous position of the aircraft <NUM>, that is, the current (or instantaneous) location of the aircraft <NUM> (e.g., the current latitude and longitude) and the current (or instantaneous) altitude (or above ground level) for the aircraft <NUM>. The navigation system <NUM> is also capable of obtaining or otherwise determining the heading of the aircraft <NUM> (i.e., the direction the aircraft is traveling in relative to some reference).

In the illustrated embodiment, the aircraft <NUM> also includes a communications system <NUM> configured to support communications to and/or from the aircraft <NUM>. In this regard, the communications system <NUM> may include a data link system or another suitable radio communication system that supports communications between the aircraft <NUM> and the computing system <NUM>.

The FMS <NUM> is coupled to the navigation system <NUM>, the communications system <NUM>, and one or more additional avionics systems <NUM> to support navigation, flight planning, and other aircraft control functions in a conventional manner, as well as to provide real-time data and/or information regarding the operational status of the aircraft <NUM>. Practical embodiments of the aircraft <NUM> will include numerous avionics systems <NUM> for providing various functions and features, as will be appreciated in the art. For example, practical embodiments of the aircraft <NUM> will likely include one or more of the following avionics systems suitably configured to support operation of the aircraft <NUM>: an air traffic management system, a traffic avoidance system, an autopilot system, an autothrust system, a flight control system, hydraulics systems, pneumatics systems, environmental systems, electrical systems, engine systems, trim systems, lighting systems, crew alerting systems, electronic checklist systems, an electronic flight bag and/or another suitable avionics system. Additionally, the aircraft <NUM> may include one or more flight data recorders (FDRs), quick access recorders (QARs), and/or the like configured to capture or otherwise obtain flight data in real-time.

The processing system <NUM> generally represents the hardware, circuitry, processing logic, and/or other components configured to support storage, maintenance, and/or provisioning of flight data captured or otherwise obtained by one of more onboard systems <NUM>, <NUM>, <NUM> for purposes of the procedure mapping processes and related task, functions and/or operations, as described in greater detail below. However, it should be noted that in other embodiments, any features and/or functionality of processing system <NUM> described herein can be implemented by or otherwise integrated with the features and/or functionality provided by the FMS <NUM> or another onboard component or system (e.g., by an FDR, QAR, or other onboard data recording system). In other words, some embodiments may integrate the processing system <NUM> with another system or component onboard the aircraft <NUM>, that is, the processing system <NUM> described herein may be a component of the FMS <NUM> or other avionics <NUM>. Additionally, it should be understood that <FIG> is a simplified representation of the flight analysis system <NUM> for purposes of explanation and ease of description, and <FIG> is not intended to limit the application or scope of the subject matter described herein in any way.

As described in greater detail below in the context of <FIG>, in exemplary embodiments, the computing system <NUM> analyzes flight records <NUM> for completed flights to automatically identify which procedure(s) in the procedure database <NUM> were flown during the respective flights and correspondingly update the flight records <NUM> to maintain the association between the flight record <NUM> and the identified procedures flown during that flight. In this regard, a flight record <NUM> may include periodically recorded or reported aircraft position data that may be utilized to derive an aircraft trajectory. For example, the current latitude, current longitude, current altitude, the current heading, the current speed, and other flight data may be periodically recorded or captured by the processing system <NUM> or another onboard component (e.g., a FDR, QAR, or the like) at regular intervals (e.g., on a per second basis, every <NUM> seconds, every <NUM> seconds, etc.) and timestamped or otherwise maintained in a time ordered sequence, thereby allowing the aircraft speed and trajectory to be estimated, interpolated, or otherwise determined.

Based on the combination of the airport and runway associated with the flight, a set of potential procedures associated with that combination may be identified in the procedure database <NUM>. The flown aircraft trajectory is then compared against each of the potential procedures to identify or otherwise determine which procedure most closely corresponds to or aligns with the flown aircraft trajectory. The computing system <NUM> updates the flight record <NUM> to assign the potential procedure that the flown aircraft trajectory most closely adheres to as the procedure that was flown by the aircraft during that flight. For any given flight record <NUM>, the computing system <NUM> may map the flown aircraft trajectory represented by its associated flight data to multiple different procedures, including, for example, a departure procedure associated with the combination of departure runway and departure airport associated with the flight record <NUM>, an arrival route associated with the combination of destination runway and destination airport associated with the flight record <NUM>, an approach procedure associated with the combination of destination runway and destination airport associated with the flight record <NUM>, and the like.

Referring now to <FIG>, the flight analysis system <NUM> is configured to support a procedure mapping process <NUM> to identify and assign aircraft procedures to flight records based on the flight data associated with the respective flight records. In this regard, the procedure mapping process <NUM> may be performed in response to receiving a new flight record or with respect to a previously-stored flight record that lacks an existing association with one or more procedures. For example, the computing system <NUM> may perform the procedure mapping process <NUM> on each new flight record received from an aircraft <NUM> prior to storing the flight record in the flight records database <NUM>, or alternatively, the computing system <NUM> may analyze the flight record database <NUM> to identify flight records that lack association to one or more different types of aircraft procedures. The various tasks performed in connection with the illustrated process <NUM> may be implemented using hardware, firmware, software executed by processing circuitry, or any combination thereof. For illustrative purposes, the following description may refer to elements mentioned above in connection with <FIG>. In practice, portions of the procedure mapping process <NUM> may be performed by different elements of the system <NUM>, however, exemplary embodiments may be described herein as primarily being performed by the computing system <NUM> and/or the processing system <NUM>. It should be appreciated that the procedure mapping process <NUM> may include any number of additional or alternative tasks, the tasks need not be performed in the illustrated order and/or the tasks may be performed concurrently, and/or the procedure mapping process <NUM> may be incorporated into a more comprehensive procedure or process having additional functionality not described in detail herein. Moreover, one or more of the tasks shown and described in the context of <FIG> could be omitted from a practical embodiment of the procedure mapping process <NUM> as long as the intended overall functionality remains intact.

The procedure mapping process <NUM> begins by identifying or otherwise determining the combination of runway and airport associated with the flight record under analysis and then identifying or otherwise determining a subset of potential procedures for analysis based on that runway and airport combination (tasks <NUM>, <NUM>). In this regard, the procedure mapping process <NUM> identifies a subset of potential aircraft procedures that represents the potential procedures that the aircraft <NUM> could have flown to/from the identified runway at the identified airport. For example, based on the runway and airport associated with a flight record <NUM>, the computing system <NUM> and/or processing system <NUM> may query the procedure database <NUM> to obtain a listing of the aircraft procedures associated with that runway at that particular airport. In some embodiments, the subset of potential aircraft procedures may be further reduced based on whether the airport is the departure or arrival location for the flight. For example, in the case of a departure airport, the computing system <NUM> and/or processing system <NUM> may query the procedure database <NUM> for only the departure procedures associated with the identified runway at that departure airport. Similarly, for an arrival airport, the computing system <NUM> and/or processing system <NUM> may query the procedure database <NUM> for only the arrival route and/or approach procedures associated with the identified runway at that arrival airport.

The procedure mapping process <NUM> receives or otherwise obtains at least the portion of the flight data from the flight record that corresponds to the aircraft's operation in the vicinity of the airport associated with the subset of potential procedures to be analyzed, and then evaluates or otherwise compares the flight data to the procedure information for each of the potential procedures to determine how well the flight data adheres to or aligns with the respective procedure (tasks <NUM>, <NUM>). In this regard, for each potential procedure of the subset, the procedure mapping process <NUM> attempts to map the actual trajectory flown by the aircraft <NUM> to the trajectory defined by the procedure information for the respective procedure, as described in greater detail below in the context of <FIG>. In exemplary embodiments, for each potential procedure, the procedure mapping process <NUM> calculates or otherwise determines one or more metrics that quantifies the degree to which the flown aircraft trajectory adheres to or otherwise realizes the procedure's trajectory, such as, for example, the number of legs of the procedure trajectory that were flown, the percentage of the procedure trajectory that was flown, and/or the like.

Still referring to <FIG>, in exemplary embodiments, after analyzing the flown aircraft trajectory with respect to each potential procedure of the identified subset, the procedure mapping process <NUM> ranks, prioritizes, or otherwise orders the potential procedures based on the degree of adherence between the flown aircraft trajectory and the procedure-defined trajectory (task <NUM>). In this regard, when multiple different adherence metrics are determined, an aggregate adherence score may be calculated or otherwise determined for each procedure based on the adherence metric(s) calculated for the flown aircraft trajectory with respect to the procedure's trajectory, for example, using a weighted average of the adherence metrics or some other function of the different adherence metrics. Each potential procedure may be ranked or ordered with respect to other potential procedures based on its associated adherence score relative to the adherence scores associated with those other potential procedures to obtain an ordered listing of the potential procedures according to adherence with the flown aircraft trajectory.

The illustrated procedure mapping process <NUM> continues by assigning a potential procedure to the flight record based on its adherence ranking relative to the other potential procedures (task <NUM>). For example, in one or more embodiments, the procedure mapping process <NUM> identifies the potential procedure having the highest adherence score or metric as the procedure most likely to have been attempted to be flown by the aircraft and assigns the highest ranked procedure to the flight record <NUM>. In this regard, the computing system <NUM> and/or the processing system <NUM> updates the entry for the flight record <NUM> to include the name or other indicia of the identified procedure to be assigned to the flight record <NUM> and thereby maintain an association between the flight record <NUM>, its associated flight data, and the identified procedure.

In one or more exemplary embodiments, the procedure mapping process <NUM> is performed to identify and assign departure procedures, approach procedures, arrival routes, and the like to each flight record <NUM> maintained in the flight records database <NUM>. Additionally, in response to new flight records uploaded to the computing system <NUM> and/or the flight records database <NUM>, the procedure mapping process <NUM> may be performed to assign aircraft procedures to the newly received flight records.

<FIG> depicts an exemplary aircraft procedure <NUM> suitable for use with the procedure mapping process <NUM> to analyze the aircraft procedure <NUM> with respect to a flown aircraft trajectory <NUM> depicted in <FIG>. Based on the relationship between the flown aircraft trajectory <NUM> and the trajectory defined by the aircraft procedure <NUM>, the procedure mapping process <NUM> calculates or otherwise determines one or more adherence metric(s) associated with the aircraft procedure <NUM> for that aircraft trajectory <NUM> to be utilized in determining whether to assign the aircraft procedure <NUM> to the aircraft trajectory <NUM>, as described in greater detail below.

Referring to <FIG>, the procedure information associated with the illustrated aircraft procedure <NUM> defines eight procedural legs <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> from an initial navigational reference point <NUM> (or waypoint) to a final waypoint <NUM> via intermediate waypoints <NUM>, <NUM>, <NUM>. The procedure information defines both conditional legs <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> and an unconditional leg <NUM> for traversing the waypoints <NUM>, <NUM>, <NUM>, <NUM>. In this regard, an unconditional leg <NUM> has a substantially fixed length between endpoints defined by waypoints <NUM>, <NUM> while conditional legs <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are defined by manner in which they are flown or executed according to the parameters or instructions associated with the procedure <NUM> such that the exact trajectory and length of any individual conditional leg <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> may vary depending on the aircraft performance satisfying a particular termination condition (e.g., a particular altitude level). For example, the first conditional procedural leg <NUM> may be realized as a course from a fix (e.g., waypoint <NUM>) to an altitude, such that the termination point <NUM> of the first conditional procedural leg <NUM> occurs when the aircraft reaches that prescribed altitude while flying along the prescribed course or heading after traversing the initial waypoint <NUM>. It should be noted that both conditional or unconditional legs may have any number of constraints associated therewith that define the respective leg, which could be applicable at either or both endpoints of a respective leg or while the aircraft is en route between endpoints of the leg.

<FIG> depicts an exemplary embodiment of a trajectory analysis process <NUM> suitable for use in a flight analysis system <NUM> in connection with the procedure mapping process <NUM> (e.g., task <NUM>) to determine the degree to which a flown trajectory, such as trajectory <NUM>, adheres to a reference trajectory defined by an aircraft procedure of interest (alternatively referred to herein as a procedure-defined trajectory or procedure trajectory), such as aircraft procedure <NUM>. It should be appreciated that the trajectory analysis process <NUM> may include any number of additional or alternative tasks, the tasks need not be performed in the illustrated order and/or the tasks may be performed concurrently, and/or the trajectory analysis process <NUM> may be incorporated into a more comprehensive procedure or process having additional functionality not described in detail herein. Moreover, one or more of the tasks shown and described in the context of <FIG> could be omitted from a practical embodiment of the trajectory analysis process <NUM> as long as the intended overall functionality remains intact.

Referring to <FIG> with continued reference to <FIG>, in exemplary embodiments, the trajectory analysis process <NUM> begins by defining recovery fixes along the procedure trajectory based on the relationship between the flown trajectory and the procedure trajectory (task <NUM>). The computing system <NUM> and/or the processing system <NUM> obtains or otherwise identifies, based on the procedure information from the procedure database <NUM>, the geographic location of the waypoints <NUM>, <NUM>, <NUM>, <NUM>, <NUM> involved in the aircraft procedure <NUM> being analyzed. The computing system <NUM> and/or the processing system <NUM> also obtains the flight data from the flight record <NUM> that defines the flown trajectory <NUM> and then analyzes the flown trajectory <NUM> with respect to the location of the waypoints <NUM>, <NUM>, <NUM>, <NUM>, <NUM> to designate or otherwise identify one or more of the waypoints as a recovery fix when the flown trajectory <NUM> comes within at least a threshold distance (d) of the respective waypoint(s). For example, in the illustrated embodiment of <FIG>, the first, second, fourth and fifth waypoints <NUM>, <NUM>, <NUM>, <NUM> are defined as recovery fixes based on the flight data from the aircraft position reports indicating that at least a portion of the flown trajectory <NUM> is within a threshold distance of each of the respective waypoints <NUM>, <NUM>, <NUM>, <NUM>. Conversely, the third waypoint <NUM> is not designated as a recovery fix or is otherwise excluded from the group of recovery fixes because the flown trajectory <NUM> does not come within the threshold distance of the third waypoint <NUM>.

After defining recovery fixes, the trajectory analysis process <NUM> sequentially analyzes the flown trajectory <NUM> with respect to the procedure trajectory by selecting or otherwise identifying an initial procedure leg for analysis and determining whether the flown trajectory maps or otherwise corresponds to that procedure leg (tasks <NUM>, <NUM>). In this regard, the computing system <NUM> and/or the processing system <NUM> obtains or otherwise identifies, based on the instructions, conditions, or other procedure information from the procedure database <NUM>, the initial procedural leg <NUM> from the initial waypoint <NUM> (or initial recovery fix) for the procedure <NUM>. The computing system <NUM> and/or the processing system <NUM> then analyzes or otherwise compares the flight data for the portion of the flown trajectory <NUM> starting at or near the initial waypoint <NUM> to identify or otherwise determine whether the flown trajectory <NUM> adheres to or otherwise satisfies the instructions, conditions, or other criteria that define the initial procedural leg <NUM>.

For example, if the initial procedural leg <NUM> is realized as a fix-to-altitude leg having instructions that provide a defined heading or course from the waypoint <NUM> until reaching an altitude criterion defining the leg termination point <NUM>, the computing system <NUM> and/or the processing system <NUM> starts from the aircraft position report closest to the initial waypoint <NUM> and then analyzes flight data from successive aircraft position reports to determine whether the flown trajectory <NUM> is substantially aligned with the course or heading associated with the initial procedural leg <NUM> for the duration of time or travel distance required to reach the altitude criterion that defines the leg termination point <NUM>. In this regard, for each successive aircraft position report from the initial waypoint <NUM>, the computing system <NUM> and/or the processing system <NUM> may verify the current latitude and longitude coordinates of the aircraft <NUM> at the time of the aircraft position report is within a threshold distance of the lateral trajectory for the initial procedural leg <NUM>, that the current heading or course of the aircraft <NUM> at the time of the aircraft position report is within a threshold of the heading or course associated with the initial procedural leg <NUM>, that the current altitude of the aircraft <NUM> at the time of the aircraft position report satisfies one or more altitude criteria associated with the initial procedural leg <NUM> (e.g., below a maximum altitude associated with the waypoint <NUM> and/or the termination point <NUM>, above an altitude defining the termination point <NUM>), and the like.

When the flown trajectory is maintained within a threshold (e.g., a threshold distance, a threshold course, or the like) difference of the procedure-defined trajectory for the leg under analysis, the trajectory analysis process <NUM> designates or otherwise identifies that procedure leg as having been flown or otherwise realized by the flown trajectory <NUM> during the flight under analysis (task <NUM>). For example, in the illustrated embodiment, the initial procedural leg <NUM> is determined to have been flown by the aircraft based on the portion of the flown trajectory <NUM> following the initial waypoint <NUM> being maintained within a threshold difference of the procedure-defined trajectory for the initial procedural leg <NUM>. After determining a procedure leg was flown, the trajectory analysis process <NUM> continues by selecting the next procedural leg in the procedure sequence for analysis until reaching the end of the procedure (tasks <NUM>, <NUM>, <NUM>, <NUM>). For example, similar to the initial procedural leg <NUM>, the second procedural leg <NUM> may be identified or otherwise designated as having been flown by the aircraft based on the flight data from the aircraft position reports following the termination point <NUM> for the initial procedural leg <NUM> indicating the flown trajectory <NUM> after executing the initial procedural leg <NUM> substantially aligns with or otherwise adheres to the instructions, parameters, or other criteria defining the second procedural leg <NUM> by being maintained within a threshold of the procedure-defined trajectory until the aircraft reaches or otherwise satisfies the condition(s) that define the termination point <NUM> for the second procedural leg <NUM>. Likewise, the third procedural leg <NUM> may be identified or otherwise designated as having been flown by the aircraft based on the flight data from the aircraft position reports between the second leg termination point <NUM> and the second waypoint <NUM> indicating the flown trajectory <NUM> aligns with or otherwise adheres to the procedure information for the third procedural leg <NUM>.

When the flown trajectory does not map to the procedure-defined trajectory or otherwise deviates from the procedure-defined trajectory by more than the adherence threshold, the trajectory analysis process <NUM> designates or otherwise identifies the procedural leg as having not been flown and advances to the next recovery fix to resume analysis of the procedure (tasks <NUM>, <NUM>). In this regard, the current procedural leg of interest and any intervening procedural legs between the current procedural leg and the next recovery fix along the procedure-defined trajectory are designated as having not been flown due to the flown trajectory deviating from the procedure-defined trajectory. The next recovery fix is then utilized to resume analysis of the flown trajectory from that recovery fix onward. For example, as illustrated in <FIG>, the flown trajectory <NUM> deviates from the trajectory defined by the instructions, criterion, or other procedure information for the fourth procedural leg <NUM> by more than the adherence threshold prior to reaching the third waypoint <NUM>, resulting in the fourth procedural leg <NUM> being designated as not flown by the aircraft. Additionally, intervening procedural legs <NUM>, <NUM> along the procedure-defined trajectory between the next recovery fix following the fourth procedural leg <NUM> (fourth waypoint <NUM>) and the fourth procedural leg <NUM> are also designated as having not been flown or realized by the aircraft.

Thereafter, the trajectory analysis process <NUM> selects the seventh procedural leg <NUM> emanating from the recovery fix waypoint <NUM> for resuming analysis (task <NUM>). The loop defined by tasks <NUM>, <NUM>, <NUM>, and <NUM> repeats until reaching the final waypoint <NUM> of the procedure <NUM> to identify or otherwise designate the seventh and eight procedural legs <NUM>, <NUM> as having been flown based on the deviations between the flown trajectory <NUM> and the respective procedural legs <NUM>, <NUM> being maintained within the applicable adherence thresholds.

In some embodiments, the trajectory analysis process <NUM> may be configured to identify or otherwise designate some procedural legs as partially flown when the flown trajectory <NUM> only maps to a portion of a respective procedural leg. For example, in the embodiment shown in <FIG>, the fourth procedural leg <NUM> may be designated as partially flown based on the flown trajectory <NUM> adhering to the instructions, criteria, or other procedure information for flying the fourth procedural leg <NUM> after traversing the second waypoint <NUM> before deviating beyond the threshold at some later point prior to reaching the third waypoint <NUM>.

Additionally, it should be noted that in some embodiments, the trajectory analysis process <NUM> may also analyze the procedure in reverse from the final waypoint or recovery fixes rather than designating intervening procedural legs as having not been flown or realized based solely on analyzing the procedure in the sequential order as flown. This allows the trajectory analysis process <NUM> to identify additional legs that were completely and/or partially flown or otherwise improve the mapping of the flown trajectory to the procedure, thereby improving the accuracy or reliability of the mapping between the trajectory and the procedure, which, in turn improves the accuracy or reliability of the subsequently calculated adherence metrics and related analysis. For example, the trajectory analysis process <NUM> may advance to the recovery fix waypoint <NUM> and resume analysis backwards to determine whether the preceding procedural leg <NUM> was still flown (e.g., by analyzing the trajectory to see if the aircraft arrived at the recovery fix waypoint <NUM> via the prescribed course for the preceding procedural leg <NUM>). In this regard, if the aircraft had flown the fourth procedural leg <NUM> and then deviated from the fifth procedural leg <NUM> before executing the sixth procedural leg <NUM>, analyzing the flown trajectory backwards from the recovery fix waypoint <NUM> would allow for the trajectory analysis process <NUM> to identify the sixth procedural leg <NUM> as having been completely or partially flown even though the preceding procedural leg <NUM> between recovery fixes <NUM>, <NUM> was not flown.

After analyzing the flown trajectory with respect to the entire procedure, the illustrated embodiment of the trajectory analysis process <NUM> calculates or otherwise determines one or more adherence metrics for the procedure based on the number or amount of procedural legs of the procedure that were mapped to the flown trajectory (task <NUM>). For example, the computing system <NUM> and/or the processing system <NUM> may determine the total number of procedural legs that the flown trajectory <NUM> mapped to (e.g., <NUM>), the percentage of procedural legs that the flown trajectory <NUM> was mapped to (e.g., <NUM>%), the number or percentage of unconditional legs flown (e.g., <NUM>), the number or percentage of conditional legs flown (e.g., <NUM> or <NUM>%), and/or the like. Additionally, the computing system <NUM> and/or the processing system <NUM> may calculate or otherwise determine the amount of travel distance or travel time during which the flown trajectory <NUM> was in adherence with the procedure-defined trajectory, the corresponding percentage of the travel distance or travel time in adherence with the procedure-defined trajectory (e.g., based on the total distance or travel time between end points <NUM>, <NUM>), and/or other metrics that quantify or otherwise characterize the degree of adherence between the flown trajectory <NUM> and the procedure <NUM>. In this regard, the adherence metrics quantify how close the flown trajectory <NUM> came to executing or otherwise realizing the particular procedure <NUM> of interest as defined by its associated instructions, parameters, criteria or other procedure information.

In some embodiments, the adherence metrics may be utilized to score the overall adherence to the procedure of interest or otherwise determine a probability or likelihood that the aircraft intended or attempted to fly the procedure of interest. For example, a product or weighted combination of the number or percentage of procedural legs flown and the percentage of travel time flown in adherence with a procedure may be utilized to assign a probability of having been flown to a given procedure, such that a procedure having a relatively higher percentage of procedural legs flown over a greater percentage of travel time is assigned a higher probability than a procedure having a relatively lower percentage of procedural legs flown and/or a relatively lower percentage of travel time flown in adherence.

Referring again to <FIG>, in one or more exemplary embodiments, the trajectory analysis process <NUM> is performed with respect to each potential procedure to determine how well the flight data for the flight record of interest adheres to each potential procedure for a given runway and airport combination (e.g., task <NUM>). As described above, the adherence metric(s) determined for each of the potential procedures may be utilized to rank, score, prioritize, or otherwise assess the potential procedures to identify the potential procedure that the flown trajectory best adheres to or the potential procedure that was most likely to have been the procedure that the aircraft was attempting to execute or realize. Moreover, some embodiments may assign multiple potential procedures to a flight record <NUM> according to their relative probabilities or adherences. For example, different procedures may be assigned to a flight record <NUM> but assigned different weightings, probabilities, or the like to enable more robust probabilistic or statistical analysis of the different procedures. For example, to assess the relative cost, efficiency, safety, or the like for an individual procedure, the flight records database <NUM> may be queried to identify all of the flight records <NUM> that are assigned or otherwise associated with that procedure. The corresponding flight data for those flight records <NUM> may be combined using the relative weightings or probabilities characterizing the degree of adherence with that procedure to probabilistically construct a nominal "as-flown" trajectory for that procedure based on the historical flight records <NUM>, with the nominal trajectory then being analyzed to estimate performance (e.g., fuel required, travel time required, and/or the like), safety, or other operational impacts or effects associated with the particular procedure.

By mapping or otherwise correlating historical flight records and their associated data to the procedures that were flown during those flights, an improved analysis of the procedures may be performed based on the flight data representing the procedures as they are actually likely to be executed or flown, rather than relying solely on the static procedure information that represents the procedure as defined on paper. Thus, more accurate or realistic impacts on performance (e.g., fuel, travel time, and/or the like) associated with a procedure may be determined, while also allowing for improved analysis of the safety, traffic impacts, or other impacts of flying a given procedure based on how the procedure is likely to be actually flown, which, in turn may be utilized to identify preferred procedures for a particular runway assigned to the aircraft en route to/from an airport (e.g., the approach procedure having the highest adherence rates, the highest overall usage rate, etc.). For example, if the flight records indicate a particular procedure flown significantly more often than any other potential procedure for a particular runway at a particular airport, that procedure may be identified as the preferred procedure for that runway, which, in turn, may be utilized for training purposes to improve safety (e.g., by increasing use of that procedure in flight simulators, increased briefing of that procedure, etc.).

For the sake of brevity, conventional techniques related to aircraft procedures, flight planning, data recorders, avionics systems, and other functional aspects of the systems (and the individual operating components of the systems) may not be described in detail herein. It should be noted that many alternative or additional functional relationships or physical connections may be present in an embodiment of the subject matter.

The subject matter may be described herein in terms of functional and/or logical block components, and with reference to symbolic representations of operations, processing tasks, and functions that may be performed by various computing components or devices. It should be appreciated that the various block components shown in the figures may be realized by any number of hardware components configured to perform the specified functions. Furthermore, embodiments of the subject matter described herein can be stored on, encoded on, or otherwise embodied by any suitable non-transitory computer-readable medium as computer-executable instructions or data stored thereon that, when executed (e.g., by a processing system), facilitate the processes described above.

The foregoing description refers to elements or nodes or features being "coupled" together. As used herein, unless expressly stated otherwise, "coupled" means that one element/node/feature is directly or indirectly joined to (or directly or indirectly communicates with) another element/node/feature, and not necessarily mechanically. Thus, although the drawings may depict one exemplary arrangement of elements directly connected to one another, additional intervening elements, devices, features, or components may be present in an embodiment of the depicted subject matter. In addition, certain terminology may also be used herein for the purpose of reference only, and thus are not intended to be limiting.

Claim 1:
A computer-implemented method of analyzing a flight record (<NUM>) for a flight completed by an aircraft (<NUM>)and determining performance metrics of an aircraft procedure, the method comprising:
identifying (<NUM>) a runway and an airport associated with the flight record;
identifying (<NUM>) a set of predefined aircraft procedures for association with the flight record (<NUM>) based on the identified runway and the airport;
obtaining (<NUM>) flight data from the flight record (<NUM>);
identifying a probable procedure (<NUM>) travelled by the aircraft (<NUM>) from among the set of procedures based on a mapping of an aircraft trajectory (<NUM>) for the flight completed by the aircraft (<NUM>) to the probable procedure (<NUM>), the probable procedure (<NUM>) defined by a set of legs,
wherein the identifying of the probable procedure (<NUM>) comprises:
determining (<NUM>), for each procedure of the set of procedures, a respective adherence metric value based on a relationship between a respective trajectory defined by the respective procedure (<NUM>) and position data associated with the flight record (<NUM>);
ranking (<NUM>) the set of procedures according to the respective adherence metric values;
selecting the probable procedure (<NUM>) from among the set of procedures based on the respective adherence metric value associated with the probable procedure relative to other procedures of the set of procedures, wherein selecting the probable procedure (<NUM>) comprises identifying a highest ranked procedure in the ranked set of procedures;
updating (<NUM>) the flight record (<NUM>) to maintain an association with the probable procedure; and
analyzing updated flight data of the updated flight record to determine one or more performance metrics associated with the identified probable procedure.