Air Pollution: Limited New Data on Inspection and Maintenance Program's Effectiveness
Published: 1996-03-21T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO provided information on the 50-percent penalty, also called a discount, that states suffer under the Environmental Protection Agency's (EPA) Clean Air Act enforcement for operating vehicle emissions inspection and maintenance (I&M) test-and-repair networks instead of test networks.
section_title: What GAO Found
paragraphs: GAO found that: (1) EPA issued the 50-percent discount rule after it found that test-and-repair networks were less effective in controlling motor vehicle emissions; (2) since issuing the rule, EPA has not conducted any studies to obtain new data on the effectiveness of test-and-repair networks; (3) 14 of the 23 states required to implement enhanced I&M programs have not conducted studies on the 50-percent discount, mostly because they implemented, or intended to implement, test-only programs; (4) the other 9 states did not conduct studies to assess the relative effectiveness of the two kinds of I&M networks; (5) California studied relative effectiveness and concluded that there was little difference between test-and-repair networks and test-only networks; (6) Georgia is conducting a study, and the preliminary findings lead Georgia to believe that it should suffer only a 35- to 45-percent discount for its hybrid I&M program; and (7) Congress enacted legislation to eliminate automatic discounts based on the type of testing network and allow states 18 months to study their I&M programs' effectiveness. GAO also found that EPA has allowed the states great flexibility in implementing enhanced I&M programs by: (1) providing I&M system models; (2) establishing credits for test-only networks, for states that require that only certified mechanics perform repairs, and for less costly testing systems; (3) relaxing performance standards for states that can still meet their overall targets for reducing emissions.

section_title: Letter
paragraphs: To reduce and control air pollution, the Clean Air Act requires the Environmental Protection Agency (EPA) to establish national air quality standards and prescribes deadlines for states and localities to attain these standards. The act requires states to develop strategies for attaining or maintaining the standards and to set forth their strategies in documents known as state implementation plans. These plans explain in detail how, and by how much, states or localities will reduce or control emissions from both mobile and stationary sources. EPA reviews and approves these plans by calculating the estimated emission reductions that the planned actions should achieve and comparing these estimates with the overall reductions needed.
Because motor vehicles contribute substantially to air pollution, EPA, in November 1992, issued a rule requiring 83 of the more seriously polluted areas in 23 states to implement more stringent programs—called enhanced inspection and maintenance (I&M) programs—for testing and reducing vehicles’ emissions. This rule established credits for the reductions in emissions anticipated from implementing the various elements of an I&M program. It also allowed states to vary these elements, including the type of testing network to be used in periodically testing vehicles’ emissions, provided the overall reductions anticipated from implementing a state’s I&M program would be at least as great as the reductions achievable from implementing EPA’s model, or benchmark, program.
Testing networks, which consist of authorized facilities throughout an area, may be test-only (inspectors are prohibited from making any repairs), test-and-repair (inspectors/mechanics are allowed to make repairs), or hybrid (using both types of facilities). However, EPA’s data, collected before November 1992, indicated that test-and-repair networks were less effective in controlling emissions than test-only networks. Consequently, EPA’s rule provided that the number of credits assigned to test-and-repair networks would be decreased by 50 percent, while test-only networks would receive full credit. Some states and other interested parties questioned the adequacy of EPA’s supporting data for this 50-percent discount and maintained that it amounted, in practice, to an inflexible requirement for using test-only networks. In light of the concerns raised by the 50-percent discount, you asked us to provide you with (1) the results of any audits, surveys, or studies performed since November 1992 that have a bearing on this discount and (2) the status of EPA’s efforts to provide states with more flexibility in designing I&M programs.
subsections: section_title: Background
paragraphs: According to EPA, motor vehicles produce much of the pollution that forms ozone (nitrogen oxides and hydrocarbons), and they are likewise a major source of carbon monoxide in urban areas. About 30 percent of the nitrogen oxides, about 50 percent of the hydrocarbons, and about 90 percent of the carbon monoxide emitted annually in major urban areas come from motor vehicles, according to EPA’s estimates. Although EPA points out that today’s new cars are up to 90 percent cleaner than their 1970 counterparts, the number of vehicle-miles traveled has more than doubled since 1970 and is still increasing, effectively offsetting much of the gain from cleaner vehicles. In addition, EPA and others have found that malfunctioning and poorly maintained vehicles produce excess emissions. According to EPA, such vehicles can emit from 2 to 17 times as much pollution as they were designed to emit. To control emissions from mobile sources, the Congress amended the Clean Air Act in 1990 to require states, as part of their strategy for reaching attainment, to implement enhanced I&M programs in areas classified as serious or worse ozone nonattainment areas and in certain areas with carbon monoxide problems.
To implement the act, EPA promulgated an enhanced I&M rule on November 5, 1992, which specified a performance standard, or minimum emission reduction requirement, for pollutants in each area required to implement an enhanced I&M program. EPA’s November 1992 rule also established a model program whose use, EPA believes, will enable states to meet the performance standards for areas under their jurisdiction. The model program assumes that states will implement test-only programs and that they will require annual emission testing for all 1968 and newer vehicles and more extensive testing for 1986 and newer vehicles using high-tech, computer-controlled emission analyzers, combined with other tests of vehicles’ emission control systems. Use of the model program, according to EPA, would reduce hydrocarbons by 31.9 percent, carbon monoxide by 35.4 percent, and nitrogen oxides by 13.4 percent by 2000. The actual performance standard for each area varies on the basis of local factors, such as the age of the local vehicle fleet, the relative proportions of light-duty trucks and passenger vehicles, and the average speed and trip length for vehicles in the area.
EPA’s 1992 rule allows a state to vary the model program’s design elements as long as the state’s I&M program reduces vehicles’ emissions by as much or more than is needed to meet the minimum performance standard for carbon monoxide, hydrocarbons, and nitrogen oxides. Besides the type of testing network, which is one of the key variables that EPA considers in assessing an I&M program’s effectiveness, a state may vary other design elements, including the frequency of inspections, types of automobiles (by model and year) to be inspected, types and weights of light-duty trucks to be inspected, type of testing equipment, number and types of vehicle emission control systems to be inspected, stringency of the tailpipe test, number and percent of vehicles that may receive a waiver, availability of multiple waivers, amounts that owners must spend toward repairing their vehicle’s emission control system before a waiver may be granted, and other factors. Some factors that states do not choose also affect an I&M program’s effectiveness in reducing emissions, such as the composition and age of the area’s vehicles, miles traveled, speeds traveled, lengths of the trips taken, types of fuels used, climate, and temperature. Thus, the type of testing network is only one of many variables that can influence an I&M program’s effectiveness.
To establish emission reduction credits for the type of testing network (test-only, test-and-repair, or hybrid) used in an enhanced I&M program, EPA obtained supporting data from three primary sources: audits,tampering surveys, and special studies. These data, combined with more than 15 years’ experience with I&M programs, led EPA, in its November 1992 rule, to require a 50-percent reduction in the number of emission credits assigned to tailpipe and selected other tests performed under test-and-repair programs. This 50-percent discount applied by default if a state did not have operating data from its existing test-and-repair program showing that its program had achieved a higher rate of effectiveness. EPA officials pointed out that although the number of credits for a test-and-repair network is discounted by 50 percent, the number of credits for an entire program is generally discounted by only 35 to 45 percent, depending upon the program’s other design elements and the local area’s parameters.
section_title: EPA Has Few New Data Addressing the 50-Percent Discount
paragraphs: Since 1992, EPA has not conducted any audits or tampering surveys to obtain new data supporting its position on the 50-percent discount. However, EPA is participating with academia, industry, and the state of Georgia in an ongoing study that addresses the discount. This study, which is supported by a 5-year, $5 million research grant to the Georgia Institute of Technology, calls for, among other things, evaluating the effectiveness of Atlanta’s I&M program.
According to EPA officials, additional data were not required because the agency adequately supported its 1992 decision and a May 1994 court ruling upheld this decision. The court’s opinion noted that “ample evidence in the record supports the EPA’s imposition of the 50 percent penalty.” Nevertheless, EPA officials told us that they plan to do more audits and tampering surveys in the future, once states have more experience with operating enhanced I&M programs. They also pointed out that, beginning 2 years after implementing an enhanced I&M program, a state must conduct an ongoing evaluation of at least one-tenth of 1 percent of the vehicles subject to annual inspection in order to quantify the program’s emission reduction benefits.
section_title: States and Others Have Few New Data Addressing the 50-Percent Discount
paragraphs: Officials from 14 of the 23 states required to implement enhanced I&M programs told us that their states have not conducted audits or tampering surveys to address the 50-percent discount, primarily because their states either already have implemented, or at one point had planned to implement, a test-only I&M program. Therefore, the states believed that data from audits or tampering surveys were not needed or were not a high priority in relation to their I&M program’s other needs.
Officials from the other nine states required to implement enhanced I&M programs told us that while their states have conducted some audits and/or tampering surveys since 1992, these efforts were not designed to assess the relative effectiveness of different types of I&M networks and therefore could not be used, according to the officials, to compare test-only to test-and-repair programs. For example, New Jersey, which has both test-only and test-and-repair facilities, audits each of the 35 test-only stations twice each year but uses a different approach to target test-and-repair stations for audits. Given a far larger number of test-and-repair stations and limited resources, New Jersey generally targets these stations for audits on the basis of customers’ complaints, low failure rates, or other signs of problems. The state’s goal when doing these audits is to improve compliance by targeting the worst-offending stations—not to assess the relative effectiveness of different network types. Officials from the other eight states also indicated that their audit data cannot be used to empirically assess the effectiveness of the different network types.
State officials said that there were several reasons why their states had not developed data since 1992 to address the 50-percent discount. They cited the 1994 court case, EPA’s 1995 commitment to provide states with greater flexibility in designing I&M programs, and uncertainty about how to demonstrate the effectiveness of different I&M network types while making major changes in their programs.
EPA officials also said that there were several reasons why states had not developed data to address the 50-percent discount. They said some states have intended to use test-only facilities all along and therefore did not have a reason to develop information on alternative networks. Other states, according to EPA, did not believe that their I&M programs were more than 50 percent as effective in reducing emissions as test-only programs; therefore, they did not see the need to collect data to justify higher credits. Appendix II summarizes the states’ audits, tampering surveys, and other data collection activities since November 1992.
subsections: section_title: States’ Special Studies
paragraphs: Although none of the 23 states have developed audit or tampering data to address the 50-percent discount, 2 states (California and Georgia) have conducted special studies since the November 1992 rule was issued. The California Senate Transportation Committee commissioned a study by the RAND Corporation, which, in October 1994, found no empirical evidence requiring the separation of the test and repair functions. In February 1995, another California study, performed for the California I/M Review Committee, found little difference in the effectiveness of a test-only program and a test-and-repair program in reducing emissions. EPA has taken issue with these findings and stated in March 1995 that it does not consider the studies cited in support of such conclusions to be based on sound methodology. These California studies are discussed in appendix II.
Georgia is also conducting a special study to address the 50-percent discount as part of an overall assessment of the Atlanta I&M program’s effectiveness. As of December 1995, Georgia Tech had completed a 2-year assessment of the Atlanta hybrid network’s effectiveness and was incorporating peer review comments into the final paper. On the basis of the data collected thus far, the principal researcher believes that the Atlanta program should receive only a 35-to 45- percent discount—not a 50-percent discount. EPA officials said that they have some concerns about the study’s use of remote sensing devices to demonstrate the program’s effectiveness in reducing emissions. EPA said it will work with Georgia Tech to ensure that the data are suitable for making such determinations. The study is expected to be issued early in 1996, according to the principal researcher. This study is also discussed in appendix II.
section_title: Others’ Efforts
paragraphs: In addition to contacting EPA and state officials, we contacted the principal investigators for other interested and affected parties identified to us as having started or completed work since 1992 that might have a bearing on the 50-percent discount. Our discussions with these investigators revealed that, of the eight studies identified, five provided some indication of the overall effectiveness of one or more I&M programs but did not directly assess the effectiveness of the I&M programs’ testing networks. The other three studies, according to their principal investigators, not only provided an indication of an I&M program’s effectiveness but also directly assessed the I&M network’s effectiveness in reducing emissions. However, no clear consensus on the relative effectiveness of test-and-repair and test-only programs emerged from these studies. For example, in contrast to the Georgia study discussed earlier, which indicated that Atlanta’s I&M hybrid program was more effective in reducing emissions than EPA’s 50-percent discount would indicate, a California study found little difference in the effectiveness of a test-only program and a test-and-repair program and observed that both were less effective than EPA’s model program. These studies’ findings are summarized in appendix II.
section_title: EPA Has Provided More Flexibility to States Implementing Enhanced I&M Programs
paragraphs: EPA has recognized that uncertainty about the effectiveness of different types of I&M testing networks has delayed the implementation of enhanced I&M programs in some states. Accordingly, the agency has taken several actions to give states more flexibility in designing and implementing enhanced I&M programs that the states believe will meet their individual needs. For example, in February 1995 EPA established emission reduction credits for different types of hybrid I&M networks. In essence, EPA provided states with examples of approvable I&M programs by establishing alternative I&M network designs that, if properly implemented, would be as effective in reducing emissions as EPA’s test-only model program and would enable states to meet their performance standards. These alternative I&M networks included (1) an age-based hybrid, in which older vehicles are sent to test-only facilities, while newer vehicles may go to test-and-repair facilities; (2) a retest hybrid, in which all vehicles are initially tested at test-only facilities and vehicles that require retesting are sent, after being repaired, to test-and-repair facilities if they have failed only once and to test-only facilities if they have failed more than once (repair verification); and (3) a targeted hybrid, in which information on the emission performance of vehicles (by make, model, and engine type) is used to target poorly performing types of vehicles for test-only inspections while other types of vehicles are sent to test-and-repair facilities.
Also in February 1995, EPA established credits for the states that require repairs to be performed by trained, certified mechanics, and for less costly testing systems. States can earn extra credits if repairs are performed by trained, certified mechanics because such repairs are generally more effective and longer lasting. Furthermore, now that EPA provides credits for less costly and less complex testing equipment, more test-and-repair facilities can participate in enhanced testing programs. Additionally, EPA has been working with states and other stakeholders to establish emission reduction credits for the use of remote sensing devices to measure vehicles’ emissions in actual traffic conditions. Once identified, poorly performing vehicles can be called in for repairs before they are due to be reinspected. By giving credits for using remote sensing devices to identify vehicles needing out-of-cycle repairs, EPA will be able to help the states with alternative networks meet their performance standards. EPA officials believed that such credits would be finalized by the spring of 1996.
Additionally, in September 1995 EPA revised its 1992 rule to allow states to meet a new, less stringent performance standard as long as the states could still meet their overall targets for reducing emissions. This change gave more states the flexibility to offset the 50-percent discount for test-and-repair networks with additional reductions in emissions from stationary sources. According to EPA, in making such a decision, a state must consider the impact that the decision could have on stationary sources. For example, stationary sources may have to replace or add more pollution control equipment, reconfigure their manufacturing processes, or change the raw materials they use—actions that can be more expensive than obtaining equivalent reductions from mobile sources. In October 1995, EPA also proposed greater flexibility for certain areas within a 13-state region of the northeastern United States where the movement of ozone pollution is a problem. This proposal, if approved, would allow qualified areas to meet an even lower performance standard, thereby helping these areas meet the commitments in their state’s plan while choosing to implement test-and-repair I&M networks.
EPA’s actions, when taken together, provided states with significantly more flexibility than was previously available. However, the agency continued to discount the number of credits for test-and-repair programs by 50 percent, and many expressed concern that states did not have sufficient flexibility to design I&M programs that best suited their needs. The Congress responded to these concerns in late November 1995 by passing a law that eliminated EPA’s automatic 50-percent discount and gave states 18 months to collect new data supporting their particular I&M program’s effectiveness in reducing vehicles’ emissions. On December 12, 1995, EPA issued guidance to its regions suggesting innovative program features that states may use to improve the effectiveness of their test-and-repair programs. Subsequently, officials from seven states told us they would likely collect new data supporting the effectiveness of their test-and-repair programs, and officials from several other states were considering this new option. The current I&M plans for each state appear in appendix I.
EPA officials told us they will work with these states to ensure that adequate test data are collected during this 18-month evaluation period. They pointed out that the agency plans to meet with all affected states in March 1996 to help these states develop appropriate methodologies for assessing the effectiveness of their interim I&M programs in reducing emissions. They said that these states’ enhanced I&M programs will be assessed using criteria similar to those specified in the evaluation component of the 1992 enhanced I&M rule. According to EPA, this approach should provide consistent, reliable data for use in quantifying each state’s emission reduction benefits.
section_title: Agency Comments
paragraphs: We provided copies of a draft of this report to EPA for its review and comment and obtained comments from officials in the Office of Mobile Sources, including the Director of that office. These officials said that, overall, the report is accurate and fair in its presentation. They also suggested clarifying changes, which we incorporated into the report. For example, they suggested that we refer to the reduction in emission credits for test-and-repair networks as the 50-percent discount, a term used in a 1995 statute, rather than as the 50-percent penalty, a term used earlier by a federal court. We made this change throughout the report. They further suggested that we point out early in the report that recent legislation has eliminated any automatic discounts of test-and-repair networks. This information already appeared near the beginning of our report; however, we added a citation identifying the relevant legislation and noted that EPA had provided guidance to its regions. Finally, the officials suggested that we include an I&M contractor’s observation supporting the effectiveness of the test-only sites in a state that is not required to implement an enhanced I&M program. Although we included this observation in appendix I, we did not incorporate it into the body of the report because it does not fall within the scope of our review.
We conducted our review from July 1995 through February 1996 in accordance with generally accepted government auditing standards. A detailed discussion of our scope and methodology appears in appendix III.
As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 15 days after the date of this letter. At that time, we will send copies to the Administrator of EPA and other interested parties. We will also make copies available to others upon request.
Please call me at (202) 512-6111 if you or your staff have any questions. Major contributors to this report are listed in appendix IV.
section_title: States’ Audits, Tampering Surveys, and Studies Addressing the 50-Percent Discount Since November 1992
paragraphs: The number of credits that EPA assigns for the design elements in a state’s inspection and maintenance program is pivotal to EPA’s approval of the state’s implementation plan. Additionally, EPA may withhold highway funds and impose other sanctions against a state that fails to develop an adequate plan by the prescribed deadline. Thus, a state seeks to obtain as many credits as possible from each of the design elements in its inspection program, including the type of testing network.
States may use three primary sources of data—audits, tampering surveys, and special studies—to establish the relative effectiveness of different types of enhanced inspection and maintenance (I&M) networks, according to EPA. We found that only 2 (California and Georgia) of the 23 states that are required to implement an enhanced I&M program have developed data since November 1992, when EPA issued its enhanced I&M rule decreasing the number of credits assigned to test-and-repair networks by 50 percent.According to state air program officials, this 50-percent discount has not been more widely addressed for several reasons, which are discussed in this appendix. In addition, the appendix provides further details on the states’ audits, tampering surveys, and studies conducted since November 1992, as well as on the type of I&M network planned for each state as of December 1995.
subsections: section_title: Audits
paragraphs: According to EPA, states generally use two types of audits (overt and covert) to evaluate their I&M program’s effectiveness. In an overt audit, a state inspector may observe vehicles being tested, review a facility’s files and records of past tests, analyze data, check equipment calibrations, or perform other such activities. In a covert audit, undercover vehicles are purposely set to fail either tailpipe emission tests or tampering checks. Officials from most of the states required to have an enhanced I&M program said that while they have done overt and/or covert audits, these efforts were not designed to evaluate the appropriateness of EPA’s 50-percent discount. In some instances, state officials told us that they generally targeted problem stations, often on the basis of consumers’ complaints, for overt and covert audits in order to minimize the number of improper inspections. For example, New Jersey, which has both test-only and test-and-repair facilities, audits each of the 35 test-only stations twice each year while using a different approach to monitor test-and-repair stations. Because there are many more test-and-repair stations in New Jersey and resources are limited, the state generally targets test-and-repair stations for a covert audit on the basis of customers’ complaints, low failure rates, and other indicators of poor performance or noncompliance. The state’s goal when auditing these stations is to improve compliance by targeting the worst offenders—not to assess the relative effectiveness of different network types.
section_title: Tampering Surveys
paragraphs: Tampering surveys involve pulling motorists over to the roadside to inspect their vehicle’s emission control system to see if this system has been altered or removed. If properly implemented, tampering surveys can serve as a basis for comparing tampering rates among different I&M programs. However, such surveys are costly and generally require the aid of local or state police. Although EPA conducted such surveys on over 62,000 vehicles in 40 states from 1978 to 1990, since November 1992 most states, according to EPA, have not independently conducted roadside tampering surveys to compare the relative effectiveness of test-and-repair and test-only networks; none of the 23 states required to implement enhanced I&M programs have conducted such surveys since November 1992. Although the motor vehicle departments in some states, such as New Jersey, have conducted thousands of roadside pullovers to check for safety problems and, in some cases, evidence of tampering, air quality officials in these states told us these data have not been used to assess the effectiveness of different network types. Some pointed out that, in the past, they generally had not seen the need for roadside tampering surveys because vehicles were checked for tampering during their inspection at a testing facility. Additionally, EPA and others have pointed out that tampering is less of a concern with newer-technology vehicles, and many see remote sensing as a less expensive way to identify problem vehicles in actual driving conditions.
section_title: Special Studies
paragraphs: Of the 23 states required to implement enhanced I&M programs, only California and Georgia have conducted special studies to address the 50-percent discount. Both of these studies are discussed in appendix II. EPA officials also pointed out that the agency recently assisted Virginia in analyzing previously collected data from its test-and-repair program. While not a special study of the 50-percent discount, this analysis compared the effectiveness of Virginia’s test-and-repair network to that of Minnesota’s test-only network to determine how much credit Virginia’s program should receive during the 18-month period allotted for collecting new data under the new statute. According to EPA, the analysis indicated that Virginia’s program is effective enough to receive full credit during the 18-month interim period. EPA pointed out that the agency plans to meet with Virginia and other affected states in March 1996 to help these states develop appropriate methodologies for assessing the effectiveness of their interim I&M programs in reducing emissions. We did not identify any other studies to collect quantitative data on the 50-percent discount that had been completed since November 1992 by the 23 states required to implement enhanced I&M programs.
section_title: States’ Changing Needs for Data
paragraphs: As states have changed the plans for their I&M programs, their needs for data have also changed. For example, as table I.1 shows, in August 1994, 19 of the 23 states were planning to implement test-only I&M networks, which would have received full credit when EPA reviewed the states’ implementation plans. Thus, these states did not see the need to collect data to address the 50-percent discount. Most of the officials from these 19 states indicated that their plan to employ a test-only network was a primary reason they had not collected data to address the 50-percent discount. However, as time passed and EPA provided states with more flexibility in designing programs that they believed would best suit their needs, more and more states moved away from test-only networks. As of December 1995—just weeks after the Congress enacted legislation eliminating EPA’s 50-percent discount—only nine states were planning to implement test-only programs. Air quality officials from seven states said they were now planning to collect data on their I&M network’s effectiveness, and officials from several other states were considering this option.
However, data relevant to the 50-percent discount have not been developed for other reasons. For example, some state officials told us they did not oppose EPA’s 50-percent discount of test-and-repair programs because, in their opinion, it was appropriate. Elsewhere, however, the reasons this provision has not been more widely addressed since 1992 vary by state and by time period. For example, in October 1993 some states indicated that they had neither the time nor the resources to develop the data to meet EPA’s demonstration requirements for a test-and-repair I&M program. EPA officials also noted that prior to May 1994, some states were awaiting the outcome of the court case challenging the 50-percent discount, and some wanted to see the outcome of EPA’s initiatives for giving greater flexibility to states before moving forward with an enhanced I&M program. Also, officials from several states pointed to uncertainty about how to demonstrate the effectiveness of different I&M network types while making major changes in their programs.
section_title: Synopsis of Studies Having a Bearing on the 50-Percent Discount Since November 1992
paragraphs: We identified seven completed studies and one ongoing study that the principal investigators believe have a bearing on the 50-percent discount. Three of the studies address the discount and the other five, while not directly addressing it, provide an indication of an I&M program’s effectiveness. We reviewed each study’s methodology and results and discussed them with the principal researchers, who said that their studies’ methodologies did not isolate their findings to the relative effectiveness of different I&M network types and thus were of limited usefulness in quantitatively addressing the 50-percent discount. A synopsis of each study, listed chronologically, is presented below. the same rate for the first 2 years after the I&M program began as they had for the 5 prior years. According to the study, ambient carbon monoxide levels had declined steadily by 6 percent per year since 1987, and no measurable change occurred in that pattern after annual vehicle inspections began in July 1991. Thus, the monitoring data failed to demonstrate a systematic reduction in ambient carbon monoxide levels following the I&M program’s implementation. The study credited most of the reduction in emissions to improvements in tailpipe emission standards and new car technology. Researchers concluded from direct measurement that only 1.3 percent of the improvement in air quality could be credited to the vehicle inspection system, with a margin for error of plus or minus 1.4 percent. The principal researcher said this study was not designed to evaluate the relative effectiveness of test-and-repair and test-only I&M networks and therefore did not directly address the 50-percent discount. However, the study did question whether this test-only I&M program had achieved statistically significant reductions.
subsections: section_title: Restructuring Smog Check: a Policy Synthesis
paragraphs: The RAND Corporation prepared this October 1994 study for the California Senate Transportation Committee. The study had multiple objectives, including critically evaluating the analytic, scientific, and empirical bases for EPA’s 1992 enhanced I&M rule. The study concluded that official evaluations of the I&M program are too unreliable and uncertain for use in policy-making, but other more reliable data originally gathered for other purposes imply that key aspects of the program are approaching failure. RAND’s investigation and synthesis provided the researchers with little confidence in the ability of their regulatory agency’s existing methodologies to realistically and reliably assess and forecast performance, compare alternatives, and provide a rational guide to policy. Accordingly, RAND suggested that an independent comprehensive inquiry into the state of emission and evaluation protocols is urgently needed.
Furthermore, the study concluded that existing national data, limited as they are, suggest little difference in the measures of effectiveness between centralized (test-only) and decentralized (test-and-repair) I&M programs. Thus, they concluded there was no empirical basis for requiring that the test and repair functions be separated.
section_title: Audit Results: Aircare I/M Program
paragraphs: This December 1994 study was prepared by the Radian Corporation and de la Torre Klausmeier Consulting, Incorporated, for the British Columbia Ministry of Environment, Lands and Parks, and the British Columbia Ministry of Transportation and Highways. The purpose of this study was to (1) estimate the reductions in emissions that are occurring as a result of the program, (2) determine the effectiveness of the current emission testing procedures in identifying and ensuring the repair of high-emitting vehicles, (3) generate statistics from the program to determine whether it is meeting its design specifications, and (4) identify needed program enhancements. The principal researcher said that the study was not designed to evaluate the 50-percent discount provision; however, the study indicated that the Vancouver, British Columbia, test-only network, which was implemented in September 1992, is superior to a test-and-repair I&M program. The study found that the failure rate for vehicles in British Columbia dropped from 14 percent to 11 percent from 1993 to 1994 and concluded that the program had a lasting impact on reducing emissions in the province. A majority of the vehicles that failed in the first year passed in the second year, indicating that repairs generally were not just temporary adjustments. Ninety-nine percent of the vehicles were tested correctly, although in about 1 percent of the inspections, the contractor used a less stringent standard for nitrogen oxide, resulting in a few vehicles (less than 1 percent) being improperly passed.
section_title: An Analysis of EPA’s 50-Percent Discount for Decentralized I/M Programs
paragraphs: This February 1995 study was prepared by the California Inspection and Maintenance Review Committee. Its purpose was to evaluate the scientific basis for EPA’s 50-percent discount by evaluating EPA’s audits and tampering surveys of I&M programs, as well as other researchers’ studies of on-the-road vehicle emissions. The study challenges the relevance of EPA’s 50-percent discount for test-and-repair networks as compared with test-only networks.
The study concluded that direct measurements of the variables that I&M is supposed to reduce—emission system tampering and tailpipe emission levels—show little difference between centralized (test-only) and decentralized (test-and-repair) I&M programs. The study concluded that whether an I&M program is centralized or decentralized has not been an important factor in determining the program’s effectiveness. Among other things, the study also concluded that (1) on-the-road and ambient measurements of vehicles’ emissions indicate that both centralized and decentralized programs have performed poorly, (2) EPA’s tampering surveys show little difference in tampering rates between centralized and decentralized programs, (3) data collected and analyzed by EPA included errors in favor of centralized programs, (4) EPA’s audits included structural biases against decentralized programs, (5) EPA did not collect audit data that could be used to assess reductions in emissions, and (6) EPA did not present a methodology for converting the audit data into a quantifiable discount.
section_title: Emission Reduction Benefits Attributable to the Minnesota Vehicle Inspection Program
paragraphs: This March 1995 study was conducted for the American Lung Association of Minnesota by Sherman Engineering, Incorporated. The purpose of the study was to document reductions in emissions achieved from the Minnesota Vehicle Inspection Program. The principal researcher said this study was not designed to evaluate the appropriateness of the 50-percent discount.
The study’s findings are based upon an analysis of a set of inspection data collected from July 1991 through June 1993. Tailpipe test data were used, since they represented the only available set of data that constituted a direct measure of the emission reduction capabilities of the I&M program. The principal researcher said that the results of the study may be used in weighing the environmental impact of any action to reduce the number of vehicles to be tested in Minnesota. The study concluded that tailpipe test data from the Minnesota I&M program demonstrate that significant reductions in mobile source emissions have been achieved and that these reductions are comparable to and may exceed the benefits predicted by approved federal modeling techniques.
section_title: Evaluation of the California Pilot I&M Program
paragraphs: This March 1995 study was prepared for the California Bureau of Automotive Repair by the Radian Corporation. The objectives of the study were to (1) determine the emission reduction effectiveness of alternative loaded mode tests, such as the acceleration simulation mode tests compared to the IM-240 test; (2) demonstrate how well high-emitting vehicles within a designated geographical area can be identified using remote sensing equipment; and (3) demonstrate the effectiveness of using a high-emitting vehicle profile (based on factors such as the model year, engine family defect history, tampering probability, number of times a vehicle was sold, and remote sensing data) to identify vehicles with the highest probability of failing an emission test.
This study was broken down into two major parts: the El Monte Pilot and the Sacramento Pilot. Data to evaluate the repair effectiveness of alternative loaded mode tests were generated in the El Monte portion of the California Pilot program. The El Monte Pilot was performed by the California Air Resources Board. It tested vehicles using three different types of testing equipment. The Sacramento Pilot was conducted to demonstrate the effectiveness of remote sensing devices in identifying problem vehicles and also to evaluate techniques for profiling vehicles’ emissions. The study used the 1994 data obtained from the El Monte and Sacramento pilot studies to conclude that the planned California hybrid program can be equivalent to EPA’s performance standard for vehicles’ exhaust emissions if 18 percent of the vehicles targeted in enhanced I&M areas are sent to test-only stations for annual emission testing. According to the principal researcher, this study was not designed to address the 50-percent discount.
section_title: Emission Testing Policies for the Commonwealth of Pennsylvania
paragraphs: This June 1995 study was a project by 33 students from the Department of Engineering and Public Policy and the Department of Social and Decision Sciences at Carnegie Mellon University. The purpose of the study was to evaluate the automotive testing policies of the Commonwealth of Pennsylvania. The evaluation focused on automotive emission inspection and maintenance technologies; emissions of carbon monoxide, volatile organic compounds, and nitrogen oxides; and concentrations of ozone resulting from automotive emission control programs involving both (1) centralized emission inspection only and (2) decentralized inspection and maintenance. An economic analysis of these alternatives was also done. The study did not address the 50-percent discount.
The study used data from two other studies as well as ozone concentrations measured in Allegheny County, Pennsylvania, as factors in two computer models. One computation showed that ozone would be reduced only by 1.7 percent if IM-240 testing were introduced. Thus, the study concluded, among other things, that (1) IM-240 testing in Pennsylvania would lead to negligible reductions in ozone emissions, (2) centralized emission testing is significantly more costly than decentralized testing, and (3) achieving emission reductions from mobile sources appears to cost more than achieving similar reductions from stationary sources.
section_title: Measuring I/M Program Effectiveness Using Remote Sensing Data
paragraphs: According to the principal researcher, this ongoing study, subtitled Results of the Continuous Atlanta Fleet Evaluation, is in final draft form and should be completed early in 1996. The data are being developed by the Georgia Institute of Technology in association with 11 other organizations, including the Georgia Department of Natural Resources and EPA’s Office of Research and Development. While the study addresses the 50-percent discount, its primary purpose is to evaluate the effectiveness of the Atlanta, Georgia, I&M program. The study compares the results of remote sensing device measurements from nine Georgia counties that do not have I&M requirements with the results from four counties that do. It also compares the ratio of the percentage reductions in hydrocarbons and carbon monoxide actually achieved with the ratio of emission reductions postulated from the EPA model for a program of the same type.
The study found that the four counties with an I&M program have lower emission averages than the nine counties without an I&M program. It concluded that the reasons for the higher levels of emissions in the nine counties are (1) a higher proportion of trucks compared with the other four counties and (2) the absence of an I&M testing program. The principal researcher said their research provides evidence that, as a whole, test-and-repair programs are probably less effective than test-only programs. However, there is sufficient variability in both network types to change the effectiveness of any particular program. He estimated that test-only programs vary from 50 to 90 percent in effectiveness (when compared with the EPA mobile model) and that test-and-repair programs vary from 40 to 80 percent in effectiveness. Thus, he believes that the results of the research are relevant to the 50-percent discount because the results provide evidence that the Georgia I&M program’s effectiveness ranges from 55 to 65 percent.
section_title: Objectives, Scope, and Methodology
paragraphs: The Chairman, Subcommittee on Oversight and Investigations, House Committee on Commerce, asked us to (1) provide the results of any audits, surveys, or studies performed since EPA issued its November 1992 enhanced I&M rule that have a bearing on the rule’s provision decreasing the number of credits for test-and-repair programs by 50 percent as compared with test-only programs and (2) determine the status of EPA’s efforts to provide states with more flexibility in designing I&M programs that best suit their needs.
To identify audits, tampering surveys, or studies that might have a bearing on the 50-percent discount, we reviewed five electronic databases; the rulemaking docket (Public Docket Number A-95-08) for EPA’s September 18, 1995, revision to the enhanced I&M rule; and the hearing transcript for selected hearings on I&M issues. We also attended two symposiums where I&M issues were discussed. Additionally, we interviewed officials and obtained documents from EPA’s Office of Air and Radiation in Washington, D.C.; Office of Mobile Sources in Ann Arbor, Michigan; and Office of Research and Development in Research Triangle Park, North Carolina. We also obtained information from knowledgeable air quality officials in the 23 states required to implement enhanced I&M programs under the 1990 act.
To ensure complete identification of the studies conducted by other interested and affected parties since November 5, 1992, we also interviewed people who had addressed the 50-percent discount in documents they had sent for inclusion in EPA’s docket, as well as individuals identified to us by EPA or state officials; by representatives of academia, industry, or environmental groups; or through our own efforts discussed above. After developing a list of potential studies, we obtained and reviewed the studies and discussed their impact on the 50-percent discount with the authors or principal investigators. Also, with the EPA and state officials identified above, we discussed EPA’s efforts to provide states with more flexibility in designing I&M programs.
We conducted our review from July 1995 through February 1996 in accordance with generally accepted government auditing standards.
section_title: Major Contributors to This Report
subsections: section_title: Resources, Community, and Economic Development Division, Washington, D.C.
section_title: Norfolk Regional Office
paragraphs: James R. Beusse, Evaluator-in-Charge Philip L. Bartholomew, Staff Evaluator Hamilton C. Greene, Jr., Staff Evaluator DeAndrea M. Leach, Staff Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (301) 258-4066, or TDD (301) 413-0006.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.