Altimetry System Error sensitivity to meteorological models
Creators: name: Binder, Matthew
Publisher: name: No Publisher Supplied
Year: 2022

description: Altimetry system error (ASE) measures an aircraft's ability to maintain altitude. To calculate ASE, the geometric altitude of the aircraft's pressure altitude, measured by sensitive altimeters, is compared to its true geometric altitude. ASE is an important component in measuring the Target Level of Safety (TLS) of Reduced Vertical Separation Minimum (RVSM) airspace, which allows aircraft to fly with a vertical separation of 1,000 ft. The International Civil Aerospace Organization (ICAO) established an acceptable operational envelope for aircraft ASE in RVSM airspace to have a mean within 80 ft in magnitude and a mean plus or minus three standard deviations within 245 ft in magnitude. This study collected the appropriate altitude data from a total of 6,471 flight hours from messages transmitted by 56 aircraft from three unique aircraft groups via ADS-B using Flightradar24, an online global flight tracking service. The pressure altitude from this data set was converted into geometric altitude using three meteorological models: the Global Forecast System (GFS), the High Resolution Rapid Refresh (HRRR), and the ECMWF Reanalysis (ERA5). The corresponding ASE for each model is denoted as ASE<sub>GFS</sub>, ASE<sub>HRRR</sub>, and ASE<sub>ERA5</sub>, respectively. ASE<sub>HRRR</sub> was only calculated over limited subset of the overall data set, comprising of 75.7 hours from two aircraft with nominal and aberrant altimetry behavior, due to computational limitations.Over the limited subset for the nominal aircraft, ASE<sub>HRRR</sub> and ASE<sub>ERA5</sub> were calculated to be within the operational envelope defined by ICAO, with ASE<sub>HRRR</sub> having a smaller magnitude than ASE<sub>ERA5</sub>. However, ASE<sub>HRRR</sub> and ASE<sub>ERA5</sub> had the same standard deviation and the coefficient of quartile dispersion was twice as large for ASE<sub>HRRR</sub> compared to ASE<sub>ERA5</sub>. ASE<sub>GFS</sub> had a mean outside of the defined operational envelope and a larger standard deviation. Time histories for the aberrant aircraft revealed bimodal behavior in the altimetry system that was reflected in all three ASE calculations and similar statistics that were outside the defeined operational envelope. The same analysis was applied over the entire data set for ASE<sub>GFS</sub> and ASE<sub>ERA5</sub>. ASE<sub>ERA5</sub> had a mean and standard deviation smaller in magnitude than ASE<sub>GFS</sub> for all aircraft that was statistically significant. ASE<sub>GFS</sub> did not appear to have the horizontal or vertical resolution required to accurately calculate ASE. Calibration curves, which estimate the magnitude by which an altimetry system is offset from a true value of zero, revealed that the majority of aircraft ASE follows a normal distribution, and the calibration magnitude estimated by ASE<sub>ERA5</sub> was smaller in magnitude than the ASE<sub>GFS</sub>. The average ASE for each unique flight level was correlated to its pressure altitude, where it was observed that a larger vertical resolution in GFS and ERA5 corresponded with a more negative ASE. Calibration curves were calculated for ASE<sub>GFS</sub> and ASE<sub>ERA5</sub>, showing that the distribution of ASE<sub>ERA5</sub> was more normal than ASE<sub>GFS</sub>. A statistically significant linear correlation with elevation was calculated, with both ASE<sub>GFS</sub> and ASE<sub>ERA5</sub> increasing by 50 feet from 2,000 feet in local elevation to 12,000 feet in local elevation. Additionally, the time of the day was not found to correlate with any increases or biases in ASE.