Railroad Safety: Quiet Zone Analyses and Inspections Could Be Improved
Released: 2017-10-31T00:00:00

Letter

Background

The United States railroad system consists of a vast network of operations that includes more than 780 railroads operating across 220,000 miles of track—including about 212,000 grade crossings. Both freight and passenger railroads operate across the system. The freight railroad industry is dominated by the seven largest railroads, referred to as class I railroads, whereas passenger rail service includes Amtrak and 29 commuter railroads.

FRA is responsible for providing regulatory oversight of the safety of both freight and passenger railroads. To accomplish this oversight, FRA issues and enforces numerous safety regulations, including requirements governing track, signal and train control systems, grade crossing warning systems, and railroad-operating practices. FRA monitors railroads’ compliance with federal safety regulations through routine and special emphasis inspections on railroads’ systems. FRA’s inspectors generally specialize in one of five areas. These inspection areas are called disciplines and include: (1) operating practices, (2) track, (3) hazardous materials, (4) signal and train control, and (5) motive power and equipment.

FRA also has specific responsibilities related to the safety of grade crossings, including issuing regulations regarding the use of train horns at grade crossings. FRA issued regulations in August 2006, after FRA’s analysis illustrated the dangers of whistle bans. Federal regulations require that train horns be sounded in advance of all public grade crossings. However, the regulations also provide an opportunity for public authorities to reduce the effects of noise associated with the train horn by establishing quiet zones. While railroads are directed to cease the routine sounding of the train horn at-grade crossings within quiet zones, the final rule states that train horns may still be sounded in emergency situations and to comply with other federal regulations and railroad operating rules.

As of June 2017, there were 570 new quiet zones located across 42 states (see fig. 1).

Train engineers are generally required to sound the horns at least 15 seconds, and no more than 20 seconds, in advance of all grade crossings. Train horns must be sounded in a standardized pattern of 2 long, 1 short and 1 long blast, with the volume ranging from 96 decibels to 110 decibels.

Each pedestrian crossing or private crossing to an active commercial or industrial site must be reviewed by a diagnostic team and equipped or treated in accordance with its recommendations. The public authority must invite the state agency responsible for grade crossings’ safety and all affected railroads to participate in the diagnostic review. FRA is not required to participate in diagnostic reviews. The Notice of Intent provides railroads and state agencies with an opportunity to provide comments and recommendations on the quiet zone. A complete and accurate U.S.

Department of Transportation Grade Crossing Inventory Form must be on file with FRA for all crossings within the quiet zone to reflect the current conditions at each crossing. A Notice of Quiet Zone Establishment must be issued to FRA, applicable railroads, and relevant state agencies indicating a quiet zone is being established at least 21 days prior to the establishment date.

Throughout the process public authorities may work with a number of stakeholders who have roles and responsibilities related to grade crossings. These include:

FRA: In addition to issuing rules and regulations governing train horns and quiet zones, FRA has staff—in headquarters and in FRA’s eight regional offices—that review public authority applications for use of ASMs, issue guidance on implementing federal regulations, answer questions from the public, and provide technical assistance related to the establishment of quiet zones. For example, FRA’s 19 regional GCMs serve as subject matter experts on the train horn regulations and respond to questions from public authorities, while FRA program officials approve ASMs and conduct required annual reviews of quiet zones established relative to the Nationwide Significant Risk Threshold to ensure they equal or fall below this risk index.

Railroads: Railroads work with public authorities to: (1) identify appropriate safety measures at grade crossings; (2) participate in diagnostic review meetings when the quiet zone includes public, private, or pedestrian grade crossings; (3) receive and comment on public authority’s quiet zone notifications (e.g., the Notice of Intent and Notice of Quiet Zone Establishment); (4) install safety measures on railroad property; and (5) direct train crews not to sound horns in established quiet zones.

State departments of transportation and rail regulatory agencies: These agencies receive and comment on Notices of Intent, public authority applications, and Notices of Quiet Zone Establishment; review, and in some cases approve grade crossing modifications; and participate in diagnostic reviews.

Private industry consultants: In some cases, public authorities hire consultants to provide subject matter expertise on establishing quiet zones. Consultants may perform such tasks as determining the feasibility of a quiet zone; arranging diagnostic reviews; assessing quiet zone risks; and identifying appropriate safety measures.

According to FRA officials, federal funding is available to reduce the risks of accidents at grade crossings, but funding specific to quiet zones is limited and no dedicated source exists. The primary source of federal funding to improve grade crossings’ safety is the Federal Highway Administration’s (FHWA) Railway-Highway Crossings (Section 130) Program, which received a set-aside of $230 million for fiscal year 2017 from amounts authorized for the Highway Safety Improvement Program. While the funds are not specific to quiet zones, Section 130 funds may be used to upgrade crossing infrastructure, an upgrade that may result in a public authority’s being more easily able to establish a quiet zone. However, according to FRA program officials, the program is competitive and funding must be used for safety projects. They said projects are selected on a safety priority basis, and quiet zones are generally considered a quality of life issue, not a safety improvement. Hence, it is unlikely that many public authorities will obtain these funds to establish quiet zones. Further, the officials said that while other federal funding is available for which grade crossing improvements may be an eligible expense, none is dedicated to quiet zones.

According to FRA officials, limited federal funding is available because quiet zones are not a national issue. They produce highly localized quality-of-life benefits and little or no improvement in the level of safety at grade crossings, but rather the safety measures are installed to compensate for silencing the sound of a train horn at grade crossings. As a result, public authorities seeking to establish quiet zones generally fund the installation of SSMs and ASMs. Given limited funding, public authorities determine whether the benefits of establishing a quiet zone outweigh the costs to establish them.

Benefits of Quiet Zones Have Not Been Quantified, and Costs Depend on Many Factors

While Benefits of Quiet Zones Have Not Been Quantified, Selected Stakeholders Highlighted Benefits for Communities

Benefits derived from establishing quiet zones and reducing noise from the train horn have not been quantified in research we reviewed or by the public authorities (i.e., communities) that we interviewed. Specifically, our review of literature did not identify any studies that had quantified the benefits resulting from public authorities establishing quiet zones at grade crossings where the horn was previously sounded. Further, FRA has not quantified benefits associated with quiet zones, but did note in its RFIA that quiet zones would likely result in localized quality-of-life benefits from silencing of the horn at locations where it had previously been sounded. Finally, none of the public authorities we interviewed have conducted any analysis that has quantified benefits associated with quiet zones or were aware of any studies that quantified these benefits.

While the benefits of quiet zones have not been quantified, the majority of stakeholders whom we interviewed stated that quiet zones do provide benefits for communities. The most commonly cited benefit (35 of 40 stakeholders) was the reduction in noise due to the absence of routine sounding of the train horn. Stakeholders told us this noise reduction led to improvements in quality of life from, for example, the ability to sleep better at night, as well as a reduction in residents’ noise complaints. To a lesser extent, stakeholders also cited economic development and safety as benefits for communities. Almost half of the stakeholders (19 of 40) we interviewed told us that areas with new quiet zones saw an increase in economic development from such things as new businesses or residential developments. Similarly, almost half of the stakeholders (17 of 40) said that quiet zones increased safety along rail lines, given the addition of new safety measures at the grade crossings.

While the benefits associated with quiet zones have not been measured, more generally, researchers have analyzed the effect of transportation noise on property values and health to understand the effects.

Property values: Our review identified two studies that analyzed the effect of freight train noise on property values in selected communities and found mixed results. In one study, the authors looked at the effect of a freight rail line on home prices and concluded that, while for smaller homes results suggest a negative and statistically significant effect on sale prices, results for medium and larger units were mixed. In the second study, the author examined the effect of a railroad’s decision to ignore whistle bans and found that proximity to rail lines and crossings had a negative and statistically significant effect on residential property values in some communities, with the effect varying depending on distance to the rail line. The author concluded that the crossing effects were largely temporary, because over time, buyers less sensitive to noise would likely move into the area, reducing or eliminating any long-term effect of the railroad’s decision. However, both of these studies have limitations, are based on data almost two decades old, and the results might not be representative of the economic effects associated with quiet zones.

Health effects: In 2002, FRA summarized available academic literature on the undesirable effects of noise—primarily focusing on transportation noise associated with aircraft, highways, and railroads. According to the research, transportation noise can cause undesirable psychological health effects, such as annoyance, and physiological health effects, such as hearing impairments and sleep disturbance on individuals.

Costs to Establish Quiet Zones Depend on Many Factors

Total costs to establish quiet zones depend on many factors and vary widely. Prior to issuing regulations, in the RFIA, FRA identified the types of costs associated with establishing quiet zones that can be incurred by public authorities, states, railroads, and FRA. These factors included such things as upgrading signals at grade crossings; purchasing, installing, and maintaining safety measures like flashing lights and gates; developing, reviewing, and evaluating quiet zones; and designing public education and awareness efforts. The actual cost that public authorities incur to establish quite zones will vary and depend on these and other factors. Both FRA program officials and FRA guidance has stated that, in general, the factors that affect the costs include such things as the number of grade crossings in a quiet zone, the geography of the area in which the quiet zone is established, and the types of safety measures a public authority decides to install. For example, some grade crossings may require upgrades to constant-warning-time devices or installation of complex and costly SSMs (e.g., four-quadrant gates), whereas other grade crossings may require fewer upgrades or less complex safety measures (e.g., traffic channelization devices). In 2013, FRA published guidance for public authorities in which it estimated that the capital costs public authorities may incur to establish quiet zones may range from about $30,000 to more than $1 million per grade crossing, depending on the types of safety improvements and existing infrastructure at grade crossings. The RFIA stated that, because grade crossings may differ significantly, public authorities must analyze the characteristics of each and the safety measures needed to accurately estimate costs to establish quiet zones.

Public authorities we interviewed confirmed that the costs to establish quiet zones do vary and depend on many factors. All 13 public authorities we interviewed often said that in establishing quiet zones they incurred costs for identifying safety measures for grade crossings, purchasing and installing these safety measures, and maintaining quiet zones, among other things. According to the public authorities we spoke with in our review, the cost to establish quiet zones ranged from about $14,000 to several million dollars. However, this range also reflects different levels of quiet zone activity; for example, one public authority established a quiet zone at a single grade crossing, while another established a quiet zone that encompassed 60 grade crossings.

In addition, railroads, states, and FRA may incur costs as part of establishing quiet zones. For example, officials from seven of the eight railroads we interviewed stated that they incur costs for such things as (1) participating in diagnostic reviews, (2) commenting on Notice of Intents and Notice of Quiet Zone Establishments; and (3) notifying and training crews not to sound horns in quiet zones. States may also incur costs. Two states included in our review—California and Colorado—have public utility commissions that told us they are required to review and approve any modifications to grade crossings in their states, including those associated with quiet zones. Finally, FRA incurs costs related to quiet zones. This cost includes reviewing quiet zone applications, participating in diagnostic reviews when invited, and the time GCMs or other FRA staff spends providing technical assistance to public authorities and others on establishing quiet zones.

While public authorities are generally responsible for paying the costs to establish quiet zones, about half of the public authorities we interviewed (10 of 13) said they obtained funding from outside sources to help pay for the zones, for example:

Federal funds: Six of the public authorities we interviewed reported receiving federal funds to help establish their quiet zones. In particular, one public authority that we interviewed reported receiving a $3.3 million Transportation Investment Generating Economic Recovery grant to establish a quiet zone. Alternatively, public authorities in the remaining five communities were eligible for grade crossing safety improvement efforts that were designated by the state through FHWA or other programs.

State or railroad funds: For three of the public authorities we interviewed, quiet zones were established in conjunction with larger state department of transportation highway or railroad projects and these entities paid a portion of the costs.

Grade crossing incentive funds: Four of the public authorities we interviewed received grade-crossing incentive funds from railroads or state departments of transportation to close grade crossings that were part of a quiet zone.

Private funds: In two communities, private investors provided financial assistance to public authorities for a quiet zone. For example, a private developer paid for a quiet zone in order to facilitate the building of residential developments.

Selected Public Authorities and Other Stakeholders Reported Several Challenges in Establishing Quiet Zones and Suggested Potential Improvements

Public authorities and other stakeholders that we spoke with reported several types of challenges with establishing quiet zones. These stakeholders noted three primary challenges, which included the cost to establish quiet zones, obtaining stakeholder cooperation, and the process to establish quiet zones. As aforementioned, public authorities generally incur costs to establish quiet zones, so cost plays a major role in a public authority’s decision of whether to pursue a quiet zone or not. The most commonly cited challenge was cost (29 of 40 stakeholders). In some cases, officials whom we interviewed reported that costs were the main reason that public authorities delayed or discontinued the process to establish a quiet zone.

In addition to cost, stakeholders cited two other primary challenges to establishing quiet zones—obtaining cooperation among quiet zone participants and the process for establishing quiet zones—and suggested a variety of improvements related to bolstering the process.

Cooperation among quiet zone participants (18 of 40): Although stakeholders we spoke with cited a number of cooperation issues, including difficulties in getting private grade crossing owners to participate and lack of state cooperation, over half (10 of 18) cited cooperation between public authorities and railroads as a challenge. Such cooperation is important since both must typically work together to establish quiet zones. However, there are natural tensions between public authorities and railroads with respect to establishing quiet zones. As discussed earlier, stakeholders we spoke with supported quiet zones believing they not only maintain safety, but improve quality of life. On the other hand, all eight railroads told us that the train horn is the most effective safety measure.

The process for establishing quiet zones (16 of 40): In general, the stakeholders we spoke with cited a variety of process related challenges, including that the train horn regulations are difficult to understand, FRA waivers are difficult to obtain, and that the quiet zone process could be better explained by FRA. In particular, over half of the stakeholders whom said that process was a challenge (10 of 16) explained that the quiet zone process was either difficult to understand or navigate or that the requirements to establish a quiet zone were confusing. For example, one public authority told us that rules for establishing a quiet zone can be difficult to interpret and that this difficulty could impact public authorities’ establishment of quiet zones. Four of the 16 stakeholders also told us the process was time consuming and, in some instances, can take years to complete. FRA program officials said the turnaround time for FRA reviews depends on the quality of materials provided. They said it generally takes 90 to 120 days for FRA to complete its review, but it can take longer if there is missing information or other problems with a public authority’s application, as is often the case.

Stakeholders we spoke to suggested three types of process-related improvements: administrative changes to improve the efficiency of the process, improvements to FRA’s role in the quiet zone process, and improvements to FRA guidance that public authorities use to establish quiet zones.

Administrative improvements: Twenty-five of the 40 stakeholders that we interviewed identified one or more types of administrative process improvements to improve the efficiency of the process for establishing quiet zones or better facilitate their establishment. These suggested improvements included:

Making the quiet zone process more user-friendly (11 of 40 stakeholders that offered suggestions related to the quiet zone process): Stakeholders we interviewed identified various improvements that could streamline some administrative requirements of the quiet zone process. These steps include standardizing or automating the quiet zone process, developing sample Notices of Intent or Notices of Quiet Zone Establishment that public authorities could use to input information, and making quiet zone materials available electronically. For example, GCMs in one FRA region told us that by standardizing the paperwork all regions would receive the same documents, a step that would make review easier. In addition, these officials said public authorities often forget to include key information in the Notice of Intent and with a standard form this may not occur.

Requiring diagnostic reviews for all quiet zones (7 of 40): As discussed earlier, when there are private grade crossings that allow public access to active commercial or industrial sites or pedestrian grade crossings in a quiet zone, a diagnostic review is required. The regulations require public authorities to provide state agencies and affected railroads, among others, the opportunity to participate in diagnostic reviews. According to FRA program officials, FRA is not required to participate in diagnostic reviews. Diagnostic reviews evaluate conditions at proposed quiet-zone crossings and a diagnostic review team makes recommendations about measures that are needed to protect safety at these crossings. Seven stakeholders we interviewed suggested that diagnostic reviews should be required for all quiet zones, not just instances when there are private or pedestrian crossings. For example, one GCM told us conducting a review for all grade crossings provides a better idea of what safety measures are needed and is a prudent action to protect public safety.

FRA’s Role in the Process: About half of the stakeholders we spoke with (21 of 40) suggested improvements related to FRA and its role in the quiet zone process: Increase FRA oversight and inspection of quiet zones (11 of 40): In general, these stakeholders believe FRA should be more involved with inspections and oversight of quiet zones, particularly between when a Notice of Quiet Zone Establishment is issued and when a quiet zone is established. Most of the railroad stakeholders we spoke with (6 of 8) believe there is a need for increased FRA involvement with quiet zones’ oversight. Among the railroad concerns were that without additional FRA oversight, quiet zones may not achieve compliance with the train horn regulations, and that public authorities may not actually install the safety measures identified in the Notice of Quiet Zone Establishment. A GCM in one FRA region told us that officials discovered noncompliant safety measures and missing signs after quiet zones had been established in this region, and that safety measures that were supposed to be installed were not. We discuss quiet zone oversight issues later in this report.

Expedite FRA’s review of quiet zone applications (10 of 40): As discussed earlier, FRA plays a role in the quiet zone process, in part, by reviewing quiet zone applications when ASMs are used. The 10 stakeholders felt that FRA should expedite its review process. For example, a GCM in one FRA region suggested FRA shorten the review time by developing a list of frequently used ASMs and their safety effectiveness ratings and posting them online, a process that would save FRA time when reviewing ASMs.

Guidance about the process: Finally, stakeholders we spoke with suggested guidance on the quiet zone process could be improved (17 of 40).

In particular, 13 of the 17 stakeholders whom offered suggestions about guidance said that FRA’s quiet zone guidance should be clearer or that training about the quiet zone process is needed. As previously discussed, some stakeholders told us the quiet zone process is difficult to understand or navigate, or that FRA could better explain the process. In particular, two public authorities suggested some form of step-by-step guide is needed to better describe the process, and GCMs in three FRA regions also suggested classes or other types of education were needed to better help public authorities understand the quiet zone process. According to FRA program officials, FRA’s quiet zone guidance consists of its user guide and a document on how to create a quiet zone. The train horn regulations also specifies how public authorities are to establish quiet zones and includes steps to follow under the public authority designation or public authority application processes.

Moving forward, FRA is in the process of conducting a retrospective regulatory review and deciding what, if any, changes may be needed. In March 2016, FRA issued a Notice of Safety Inquiry, which, according to FRA, is a retrospective review of the train horn regulations. The Notice of Safety Inquiry solicited comments about many aspects of the train horn regulations, including whether FRA can decrease the barriers public authorities encounter when establishing a quiet zone. Among other things, the inquiry seeks comments about whether there should be an online process for submitting notices and other required quiet zone paperwork, whether diagnostic reviews should be required for all quiet zones, and if the regulations should be amended to include common ASMs in the list of approved SSMs. The Inquiry is also looking at other aspects of the quiet zone process and guidance. As of July 2017, FRA was still in the process of reviewing comments received in response to the notice. FRA program officials did not indicate what, if any, changes may result from this inquiry, but said any changes that are made would be handled through a rulemaking. However, FRA program officials noted that a rulemaking would not be necessary for the agency to provide public authorities with additional tools to aid in the development of a quiet zone, such as guidance.

FRA Has Conducted Analyses of Safety in Quiet Zones and Is Formalizing Quiet Zone Inspections, but Limitations Exist

FRA’s Analyses Generally Indicate That Grade Crossings in Quiet Zones Are As Safe As The Same Grade Crossings When the Train Horn Was Sounded, but Methodology Has Limitations

One way FRA evaluates the effectiveness of its train horn regulations is through conducting analyses of data on the safety of grade crossings in quiet zones. Those analyses show that grade crossings in quiet zones are generally as safe as the same grade crossings when the train horn was sounded. Specifically, FRA conducted analyses in 2011 and 2013 to assess whether there was a statistically significant difference in the number of accidents before and after implementation of quiet zones. The results showed that there was generally no statistically significant difference in the number of accidents that occurred before and after quiet zones were established. To conduct the analyses, FRA grouped quiet zones by the number of years of available data since establishment of the quiet zone, using an equal number of months before and after establishment. FRA’s analyses in 2011 and 2013 included 359 and 203 quiet zones, respectively.

While FRA’s analyses of quiet zones generally showed that grade crossings in quiet zones were as safe as the same grade crossings when the train horn was sounded, in 2013 FRA identified one exception that FRA program officials reported resolving in a subsequent analysis. Specifically, while FRA’s 2011 analysis did not show any differences in safety after establishment of the quiet zones, in 2013 FRA concluded that for quiet zones established from May 2010 through April 2011, there was a statistically significant increase in the number of accidents that occurred after the establishment of the quiet zones. Specifically, FRA found that accidents doubled from 11 accidents before establishment of the quiet zones to 22 accidents following the establishment of the quiet zone. After that finding, FRA program officials conducted a preliminary analysis for 2017 and reported that the results did not show a statistically significant increase in accidents for any period of quiet zones, including those established from 2010 through 2011. In addition to looking at quiet zones by establishment year, FRA’s 2013 analysis also grouped quiet zones by how they were established, such as with safety measures at all crossings or against FRA’s risk indexes. Results from this analysis did not show an increase in accidents by any establishment method analyzed. As a result, FRA program officials told us that they believe the result in 2013 for quiet zones established from 2010 through 2011 was likely an anomaly and that those quiet zones are as safe as other crossings.

Before-and-after analysis is a methodologically acceptable practice, but the reliability of the results decrease over time because unlike other types of analyses, they do not control for factors that may change over time. In particular, FRA’s analyses assume that the number of accidents experienced before the quiet zone is established is a good estimate of the number of accidents that would be expected in the future had the quiet zone not been established. However, FRA’s before-and-after analyses have limitations because, unlike other methodologies, they do not take into account changes to characteristics of grade crossings over time. For example, a multivariate method can control for changes to characteristics at grade crossings that may impact safety. These characteristics can include changes to train or vehicle traffic, train or vehicle speeds, time of day when train activity occurs, number of highway lanes, the number of tracks in use, or other changes to surrounding roads or infrastructure at a crossing. For example, if train or vehicle traffic increased over time, it is possible that the number of incidents would increase, while the risk of an accident would stay the same. Specifically, closing a grade crossing near a quiet zone or increases in traffic from new businesses around a quiet zone could increase traffic after the establishment of a quiet zone; however, these changes would not be factored into FRA’s current methodology for conducting safety analyses. This inherent limitation is exacerbated over time, because the assumption that there would be no changes to relevant characteristics of the grade crossings is less likely to be the case as more time passes.

FRA also conducts annual reviews of selected quiet zones to ensure their safety, and FRA program officials told us that this review further validates its before-and-after analyses. As mentioned previously, FRA conducts annual reviews of quiet zones established against the Nationwide Significant Risk Threshold because the measure is variable and subject to change over time. According to FRA program officials, about 11 percent of all quiet zones are established against the Nationwide Significant Risk Threshold and are thus included in this annual review. To ensure that established quiet zones fall at or below the Nationwide Significant Risk Threshold, FRA is required to recalculate this measure on an annual basis and notify a public authority if the Quiet Zone Risk Index no longer falls at or below the threshold. By doing so, FRA program officials told us that they are further validating that the grade crossings in quiet zones are as safe as other grade crossings. While this annual review may provide FRA with additional support that grade crossings in quiet zones are as safe as others, it does not address the underlying limitations of a before-and-after analysis.

While the reliability of a before-and-after analysis may decrease over time, FRA has no plans to revise its methodology. In fact, as mentioned previously, FRA program officials told us that preliminary results for their 2017 safety study mirror results from 2011, showing that there was no statistically significant difference in accidents before and after the establishment of quiet zones. According to FRA program officials, the agency is not required to conduct this analysis, but moving forward, program officials plan to conduct the same analysis on a biennial basis to internally validate that grade crossings in quiet zones are as safe as others.

By continuing to rely on the current methodology, FRA’s future analyses may continue to provide the agency with information that does not account for changes in characteristics of grade crossings over time. The Standards for Internal Control in the Federal Government states that management should use quality information to make informed decisions. This requirement can be satisfied by, for example, obtaining relevant data from reliable sources, obtaining that information on a timely basis, and processing that data into quality information that accurately represents what it purports to represent. Furthermore, a previous FRA study that the agency relied on in developing the final rule has reported that changes in grade crossings’ characteristics can affect the results of analyses used to predict accidents at grade crossings. As a result, FRA’s Rail-Highway Crossing Resource Allocation Procedures recommended that analyses used to predict accidents at grade crossings only include accident data for the most recent 5 years because older accident history information may be misleading due to changes that occur in grade crossings’ characteristics over time. While FRA’s recommendation was not developed to analyze the safety of grade crossings in quiet zones, the agency’s recommendation that accident data older than 5 years may be misleading because of changes that occur to grade crossings’ characteristics over time is relevant to those analyses. Nevertheless, FRA program officials told us that they have no plans to revise the methodology because it effectively compares the safety of grade crossings in quiet zones to other grade crossings. By continuing to use the same methodology, the agency may be missing an opportunity to fully understand the safety of grade crossings in quiet zones.

FRA Has Taken Steps to Formalize Quiet Zone Inspections, but Lacks Formal Guidance

In addition to conducting studies, FRA also oversees quiet zones by inspecting them to ensure their safety and compliance with train horn regulations. According to FRA program officials, FRA is not required to inspect quiet zones; rather, public authorities, in conjunction with the railroads, are responsible for maintaining quiet zones and ensuring compliance with train horn regulations. Until recently, FRA has utilized its GCMs to, among other things, informally inspect quiet zones and work with public authorities to resolve issues affecting the safety of quiet zones—issues such as foliage covering signage, maintenance issues with safety devices, or outdated pavement markings. In fact, GCMs in all eight regions told us that they informally inspect quiet zones. According to FRA program officials, the agency has recently identified the need for “more eyes on the ground” to more systematically address maintenance issues within quiet zones and to ensure compliance with train horn regulations. As a result, FRA is transitioning its informal inspection program for quiet zones to a more formal inspection process.

As of August 2017, FRA had not terminated any quiet zones because of violations or fined any entities for quiet zone violations.

August 2017 that they planned to hire 24 new Inspectors. As of August 2017, FRA had also developed the Inspector training curriculum, and began training three Inspectors. FRA program officials expressed uncertainty over when the remaining 21 Inspectors will be hired because of uncertainty regarding FRA’s hiring and training priorities, among other things. In September 2017, FRA program officials told us that one of the newly hired Inspectors had completed the training and had begun inspecting quiet zones.

While FRA has started conducting formal quiet-zone inspections, we found that FRA has not developed guidance on how the inspections should be conducted, including guidance on how frequently these inspections should be conducted and what should be examined. As a result, such guidance is not included as part of the training curriculum developed for Inspectors. According to FRA program officials, this guidance has not been developed because program officials are still finalizing the inspection program. Although no guidance has been developed, FRA program officials told us that they are considering inspecting all new quiet zones between when the public authority submits a Notice of Quiet Zone Establishment and when the quiet zone is established. Additionally, FRA program officials told us that existing quiet zones would be inspected based on mission requirements, risk, and the availability of resources, but ideally every 3 years. With respect to how the quiet zones are to be inspected, FRA program officials said they plan to develop guidance for Inspectors that is akin to the other FRA safety disciplines. FRA program officials told us that they are working toward establishing an Audit Division, which would be responsible for developing this guidance. However, as of August 2017, FRA program officials had not provided a timeline for when this division or guidance would be completed.

The absence of guidance on inspections is inconsistent with internal control standards. Specifically, the Standards for Internal Control in the Federal Government states that management should implement control activities through its policies that document each unit’s responsibility, or further delineates day-to-day procedures. These procedures may also include the timing of when a control activity occurs and state that management should communicate these policies to its staff. Without this type of guidance, FRA cannot have reasonable assurance that inspections are being conducted consistently across FRA’s eight regions and as FRA intends.

Conclusions

Grade crossing collisions are one of the leading causes of fatalities in the railroad industry, and ensuring safety in these areas, including those within quiet zones, is a vital part of FRA’s mission. While public authorities are primarily responsible for safety in quiet zones, FRA can help ensure that grade crossings in quiet zones are as safe as others. However, the methodology FRA uses to assess the safety of quiet zones has limitations because it does not account for changes to grade crossings’ characteristics over time. By continuing to rely on this methodology, FRA may be missing an opportunity to ensure that established quiet zones are providing the same level of safety as when train horns were sounded.

In addition to its safety studies, FRA is also taking steps to formalize its process for conducting physical inspections of quiet zones. While FRA has started hiring and training a few Inspectors, it lacks guidance on how and when quiet zone inspections are to be performed. Without this guidance, FRA cannot ensure that quiet zones will be inspected consistently across FRA’s eight regions.

Recommendations for Executive Action

We are making the following two recommendations to FRA: The Administrator of FRA should revise the methodology for the analysis of safety in quiet zones to take into account relevant changes over time— including changes in train and automotive traffic, or in the physical characteristics of the grade crossing. (Recommendation 1)

The Administrator of FRA should develop guidance for Inspectors on the nature and frequency of quiet zone inspections. (Recommendation 2)

Agency Comments and Our Evaluation

We provided a draft of this report to the Department of Transportation for review and comment. The department provided a written response (see app. II), as well as technical comments that we incorporated as appropriate. The department concurred with the second recommendation regarding developing guidance for quiet zone inspectors and partially concurred with the first recommendation regarding revising the methodology for analyzing the safety of quiet zones. The department said it would consider our recommendation to revise its methodology as it explores options for updating its methodology.

We are encouraged that FRA is willing to consider revising its methodology for analyzing the safety of grade crossings in quiet zones. However, we continue to believe that our recommendation is valid and that to fully understand quiet zone safety FRA needs to revise its methodology to account for relevant characteristics of quiet—zone grade crossings. As we state in the report, the reliability of FRA’s current methodology will likely decrease over time because it does not control for relevant changes to grade crossings in quiet zones including changes to vehicle or train traffic or speeds. These and other factors are critical determinants of grade-crossing safety. Further, developing a methodology that incorporates characteristics that affect safety at grade crossings in quiet zones may also provide FRA insight into the safety of grade crossings more generally. Since grade-crossing accidents are a major source of fatalities and, according to the department, are expected to increase as train- and highway-traffic increases, it will become increasingly important to have reliable information about grade-crossing safety, both in quiet zones and across grade crossings more generally.

We will send copies of this report to appropriate congressional committees, the Secretary of Transportation, and the Administrator of the Federal Railroad Administration. In addition, we will make copies available to others upon request, and the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-2834 or flemings@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

The Fixing America’s Surface Transportation Act included provisions for GAO to review the effectiveness of the Federal Railroad Administration’s (FRA) final rule governing the use of train horns at highway-rail grade crossings. The objectives of this report were to determine: (1) what is known about the benefits and costs of quiet zones, (2) what challenges, if any, public authorities and others encounter in establishing quiet zones, and (3) how, if at all, FRA is evaluating the effectiveness of federal train horn regulations. The scope of this report was limited to new quiet zones—that is, quiet zones that were established since FRA published the final rule in August 2006. Federal regulations govern the use of train horns at public-highway-rail-grade crossings (grade crossings) and provide public authorities—typically a city, town, or county—with the opportunity to create quiet zones where train horns are not sounded. We focused on new quiet zones to better understand the benefits, costs, challenges, and safety impacts associated with the regulations.

To obtain information about quiet zones, we reviewed FRA’s data on quiet zones established from 2005 through 2017. To assess the reliability of these data, we examined FRA’s reports, analyzed the data to identify any outliers, and interviewed FRA officials about how the data were collected and used. We determined that the data were sufficiently reliable for our purposes.

For each of our objectives, we reviewed pertinent law and FRA regulations and documents; interviewed FRA program officials in headquarters; and conducted in-depth interviews with a nongeneralizable sample of 40 stakeholders. This sample included stakeholders from 8 freight railroads, 5 private industry consulting firms with experience helping public authorities establish quiet zones, 6 state agencies, 13 public authorities within these six states, and FRA Grade Crossing Managers (GCMs) in each of FRA’s 8 regions. The railroads selected included all seven class I railroads, plus the Florida East Coast Railway. The latter was selected due to its previous experience with whistle bans, and it was located in a state where we conducted interviews. The private industry consultants were selected based on several factors, including (1) experience with assisting public authorities in establishing quiet zones, (2) recommendations from FRA and other stakeholders we interviewed, and (3) geographic dispersion.

We selected six states as part of a nongeneralizable sample for interviews. These states included California, Colorado, Florida, Illinois, Maryland, and Texas. The states were selected based on a variety of factors, including the number of new quiet zones and the number of grade crossings in new quiet zones. Five of the six states accounted for about 48 percent of new quiet zones (California, Colorado, Florida, Illinois, and Texas). We also conducted interviews in Maryland before we conducted other interviews to test our interview protocol. Maryland was selected for this purpose to, among other things, minimize resources. Within these states, we conducted interviews with 13 judgmentally selected public authorities (see table 1). The public authorities were also selected based on factors such as the number of new quiet zones and recommendations from FRA and other stakeholders we interviewed.

For all our objectives we also conducted a literature review of pertinent studies in scholarly/peer-reviewed journals, conference papers, non-profit or think tank publications, and trade publications or industry articles to identify research on quiet zones. We restricted our review to results published between January 1, 1996, and October 17, 2016, and our search yielded 99 results. Of these 99 results, we reviewed each abstract or full article if available, to determine whether it was relevant to any of our objectives. Our analysis identified 10 results pertaining to safety, 11 results related to benefits and costs, and 1 result related to challenges. With respect to the articles related to costs and benefits, we also looked at citations within the studies we reviewed to identity whether any of these were relevant to our objective on costs and benefits of quiet zones. Using this approach we identified one additional study. Each abstract was reviewed by two analysts to determine whether it seemed relevant. Where disagreement existed with respect to whether the abstract was relevant, we included the abstract in our request for the complete study. We then developed criteria/requirements for each objective and reviewed each study against our criteria/requirements. Namely, we were only interested in studies that quantified the benefits or costs of quiet zones or that used data or analytics to measure safety at grade crossings in quiet zones or compared safety at-grade crossings in quiet zones to grade crossings where train horns sound. Further, each study was reviewed by an analyst and a statistician or economist to determine its relevance.

With respect to our objective on the effectiveness of the train horn regulations, we determined that none of the studies met our underlying criteria. Specifically, none of the studies measured the safety at grade crossings in quiet zones or compared the results to grade crossings where the train horn sounded. Conversely, with respect to our objective on the costs and benefits of quiet zones, we determined that six studies were relevant. To assess the reliability and methodological soundness of the studies we determined were relevant, we compared the studies with general guidelines based on standards for assessing research and analysis from the literature, past GAO reports on evaluating research programs, and our internal expertise in research design. These guidelines include, for example, examining a study based on: (1) the extent to which it was well designed and the methodology supports the objectives; (2) whether the assumptions were reasonable and explicitly stated; (3) whether the study used the best available data; and (4) whether the conclusions and recommendations were balanced and supported by data analysis.

To determine what is known about the benefits and costs of quiet zones, we reviewed the literature search discussed above and analyzed any studies obtained using the methodology described above. We also reviewed FRA’s Regulatory Evaluation and Regulatory Flexibility Assessment for Use of Locomotive Horns at Highway-Rail Grade Crossings Final Rule (RFIA). The RFIA was issued before the final rule and analyzed the potential economic effects of requiring the train horn to be sounded at all public grade crossings, of eliminating whistle bans, and of providing conditions under which the train horn can be silenced at- grade crossings. To review the RFIA, we compared it to selected principles from Office of Management and Budget’s (OMB) guidance for developing regulatory analyses. These principles included whether the analysis considered alternatives; whether the analysis estimated the incremental effect of the rule compared to a business-as-usual baseline; and whether the analysis analyzed uncertainty. In evaluating the RFIA, an analyst and economist independently reviewed the analyses and subsequently came to consensus about each element’s adherence to OBM guidance. We also reviewed FRA’s September 2013 user guide for quiet zones. This guide provides a high-level overview of the quiet zone process, including an estimated cost range to establish quiet zones. We discussed the cost range with FRA, including the source of the information and its reliability. Since FRA program officials told us it was an “order of magnitude” estimate and not meant to represent actual costs to establish quiet zones, we did not determine the reliability of the information. As a result, the cost range information is used for illustrative purposes only, and we included a disclaimer about its reliability. Finally, we interviewed FRA GCMs in all eight of FRA’s regional offices, states, public authorities, railroads, and private industry consultants about the benefits and costs of establishing quiet zones. Some of these stakeholders provided information about costs to establish quiet zones, but this was anecdotal, and we did not attempt to verify its completeness or accuracy.

To determine the challenges encountered by public authorities and other stakeholders in establishing quiet zones and improvements stakeholders suggested to the quiet zone process, we interviewed FRA GCMs, states, public authorities, railroads, and private industry consultants. We asked these stakeholders to identify the primary challenges in establishing quiet zones and for suggested improvements to the quiet zone process. We then analyzed the information obtained to identify common themes of challenges or suggested improvements. Based on this analysis, an initial list of categories for each challenge and improvement was then developed along with their definitions. The definitions identified specific types of comments to be included in each challenge or improvement category. After reviewing the initial list for overlaps and duplication, as well as to keep the list manageable, a final consolidated list was developed that consisted of eight types of challenges and seven types of improvements (see table 2). Using this list, an analyst then reviewed each interview and judgmentally assigned the information into one of the categories. A second analyst then independently reviewed these assignments using the consolidated list of categories and definitions. Any differences were then reconciled by the two analysts.

To further enhance our understanding of quiet zone challenges and improvements, we reviewed guidance issued by FRA about quiet zones and the train horn rule. This included FRA’s How to Create a Quiet Zone document (posted to the FRA website in September 2012) and FRA’s user guide about quiet zones published in September 2013. Additionally, we reviewed FRA’s regulations governing train horns and quiet zones. We also interviewed FRA program officials about the quiet zone process, application processing, various aspects of the train horn rule, and obtained information from FRA about quiet zone guidance.

To determine how FRA is evaluating the effectiveness of the federal train horn regulations, we reviewed FRA’s analysis of the safety of quiet zones at highway-rail-grade crossings completed in 2011 and 2013, which compared the safety of grade crossings in quiet zones to the safety of grade crossings where the train horn is sounded. We also discussed with FRA program officials the methodologies used to prepare these studies, and concerns with the data, conclusions, and plans to conduct future analyses. To assess the reliability and methodological soundness of the studies, we used the same approach as above. Both analyses were reviewed by a statistician and economist to corroborate the review. In addition to developing criteria for reviewing the analyses, we also reviewed guidance by FRA and others regarding analyzing incident data at grade crossings and about the limitations of a paired t-test—FRA’s methodology for comparing the grade crossings.

To assess the extent to which FRA’s methodology generally reflects internal control principles, we reviewed it against practices for presenting accurate information and communicating with internal and external stakeholders outlined in the Standards for Internal Control in the Federal Government. We also conducted data reliability assessments with respect to the underlying data FRA used in its analyses. FRA’s analyses used data that originated from two distinct FRA databases: ccmMercury (CCM) and the Safety Data Analysis website. CCM is a correspondence management system which includes all data on quiet zones—such as the establishment date and grade crossings included, among others. This information is contained in the Notice of Quiet Zone Establishment that the public authority establishing the quiet zone is required to provide to FRA. Alternatively, the Safety Data Analysis website contains two datasets: the Grade Crossing Inventory System (GCIS) and the Railroad Accident/Incident Reporting System (RAIRS). The GCIS contains information on every crossing in the nation and was used to identify the characteristics of the individual crossings within the quiet zone, whereas the RAIRS contains details about each crossing collision incident that has occurred. To assess the reliability of the data used in our review, we examined FRA reports, reviewed prior GAO data reliability material, and interviewed FRA stakeholders about how the data were collected, stored, and used. We determined that the data were sufficiently reliable for the purposes of our objectives.

Finally, to understand how FRA conducts oversight of quiet zones, we interviewed FRA program officials about oversight of quiet zones, guidance to staff and public authorities, and any planned changes for how the agency conducts oversight of quiet zones. We also interviewed GCMs in each of FRA’s eight regions to understand how they carry out oversight of quiet zones and to learn about the extent to which differences exist across regions. We also reviewed prior GAO reports that summarized FRA’s oversight approach to the rail industry, including its more traditional inspection disciplines. We also asked stakeholders included in our sample of FRA GCMs, states, public authorities, railroads, and private industry consultants about the challenges of establishing quiet zones and potential improvements to the quiet zone process. We then assessed FRA’s oversight approach using the Standards for Internal Control in the Federal Government.

We conducted this performance audit from July 2016 to October 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Transportation

Appendix III: List of Organizations Contacted by GAO

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Susan Zimmerman (Assistant Director), Krister Friday, Sarah Gilliland, Timothy Guinane, Richard Jorgenson, SaraAnn Moessbauer, Malika Rice, Amy Rosewarne, Melissa Swearingen, Larry Thomas, and Crystal Wesco made significant contributions to this report.